Programming 8820 from scratch! Your chance to influence it.

[Symbol]

Hello all,

I have been lurking on this board for a while. I have also been experimenting with a number of things "remote control".
Recently, I purchased a URC-8820. Today I "burned my bridges".
I erased the processor memory in it and put in a small test program (which just blinks the visible LED).
So, either I successfully program this remote from scratch, or it will be a wasted piece of hardware.
I have the schematic for this remote (thanks to this board) and will be attempting to program it all the way up from scratch.
I have had experience with 8-bit Freescale processors in the recent past, and have many years of Software Development experience.
I have a few ideas on how to do this, and probably will spend the next few weeks doing some basic development of the peripherals and the software to control them. (reading buttons, sending IR code, running the IR as a receiver for learning, basic timing facilities, etc.)
I have all the development tools I need to make this project a reality.
I cannot promise I will ever finish it, though. I do have an interesting full-time job and can only work on this as time allows.
I intend to make all of the source code available on this board (or wherever it is appropriate instead)
Right now, though, I am soliciting input from people on this board (especially the advanced/experienced users) on what they would like to see in the code, how they would like the remote to "run" and some ideas on how to structure the data formats, and how they would like the general infrastructure to be built.
I expect to "publish" a proposal (here) on how the design will work before I start full development.

I have already done a lot of preliminary work, since I obtained another Freescale evaluation kit (for free) a couple of months ago.
So far I have modified/programmed this evaluation board to allow me to capture IR signals (basic learning) from IR transmitters running close to 38Khz modulation frequency. This ("close to 38Khz") is because the sensor hw has built-in 38Khz demodulation. I have also programmed it to emit IR signals. So far it is limited to 38.4 Khz modulated signals, but that is easily programmable.
I have had it emit Sony codes and the Panasonic codes that my DVD recorder uses.
I have also wired an 8Mbit SPI eeprom chip to it, and have it successfully accessing that chip.
I have additionally wired up a JP1 (not JP1.x) interface to it (it has hardware for IIC communications) and intend to allow it to use that to read and write JP1 remotes (I have also a 6131n that I have modded with the 2K chip and the header)
This will hopefully be accomplished in the near future.
I have learned that the *.ir file that IRxxx.exe makes and reads appears to be a complete binary copy of the contents of the EEPROM in the JP1 remote.
I intend to provide for import/export of this file from my Windows machine to my "IR experimenting" system. I also expect to have that small system programmed to update/read the EEPROM from my URC-6131n directly through the JP1 connector.
This means I will have a slightly different and possibly unique method of programming my 6131n, but it still should be interoperable with all the JP1 stuff done here.

Anyway, all comments about the usefulness of this project and attempts to steer its direction are appreciated.

I also have a different project I intend to work on (not JP1 at all). We don't have to say much about this, but if anybody is interested, we can maybe talk about it on another thread.

I will be attempting to create my own programmable Remote from scratch. Note that this will not likely be very cost effective, but is being done for the fun of the project. I expect it to be very programmable and likely include lots of memory and probably "learning" facilities as well. If anybody is interested in this, start another thread and e-mail me about it.

I am not sure what proportion of my hobby time will be split between these projects. It will probably depend somewhat on the level of interest shown by others.

Anyway, let me have those ideas!

Symbol

[Symbol]

I forgot to include that in the original post.

It is (modified to prevent 'bots picking it up)

steve underscore munnings at sympatico dot (short for canadian domain)

[johnsfine]

I have quite a lot of ideas for that project. Enough that I really wish I could find time to do it myself. Probably more than you want to hear.

For a previous discussion of this, I wrote up a lot of detail on the low level of IR generation. I strongly feel that the correct design is to build a strong foundation of basic IR generation so that individual protocols are less work.

A major focus was containing all the interactions between processing time and IR time to that foundation layer. Assuming a higher layer doesn't waste absurd amounts of time computing, it generally shouldn't have to worry about delivering data too late to the lower layer and never needs to worry about delivering data too early to the lower layer.

I forget where I put all that text. But if you want that much advice, ask and I'll dig it up.

Switching to a different point of view: Have you looked at my IRP notation? That is a very concise, generic way of specifying an IR protocol. I use it primarily for documenting IR protocols, but it is designed to also be used to generate IR. A simple program can read and understand IRP notation and use it to generate the pulse stream of an IR signal. That could be used in one of two ways in a universal remote design:

1) Put the IRP understanding engine inside the universal remote (obviously much harder programming task than keeping that engine in a PC program). Then store all the protocols you want in the remote in IRP notation. That is much more concise than the relatively concise way UEI stores protocols. So if you wanted hundreds of protocols in a universal remote the overhead of the engine would be more than paid for by the savings in individual protocols. So you could fit much more in the same flash memory.

2) Invent some simpler (for a program) less concise generic form and write a translator on the PC from IRP notation to that form. That form might just be a parse tree of the IRP string. Keep all the IRP notation for hundreds of protocols on the PC and load just the ones you want to use into each remote. The big savings by using IRP this way is in programming time. The IRP parser and generic support on the remote side would be harder to program than a few protocols. But long before you reach a practical (for a "universal" remote) number of protocols, this generic approach would take far less time than coding individual executors.

I think method 1 is more elegant. It also allows for a design where the end user can program an individual remote without help from a PC. But I don't really like the idea of programming an individual remote without help from a PC and method 2 generally makes more sense.

Switching topics again: Have you looked at the device selection methods in some of the newer extenders that were inspired by my extenders but improved upon what I did? (Sorry I forget who came up with that improvement. I ought to give credit here).

I think that is a very powerful concept for a universal remote:

A) You have a collection of what in JP1 are "device modes", each of which potentially defines one signal for each key (viewing shifted key as a different key).

B) The keys are divided up into sets (for example Vol+, Vol- and Mute are a set; Play, Pause, FF, and related keys are a set; etc.)

C) Device selection is always done by macros. Within the macro you use two kinds of select commands. One selects a specific device mode. Another assigns the currently selected device mode to a specific key set. While any device mode is "selected" in that sense the key set assignments are remembered but not immediately used; All keys come from the selected device mode. At the end of the outermost macro or any time in a macro that a dev_cancel command is given, the single device selection is dropped and all keys go through the key sets to select device mode.

I'll describe part of my own unusual use to explain the value. Hopefully you can see how the same feature might be used by someone with more current AV devices:

I have one cable tuner, two tv's and several VCRs connected to each TV. I used to use an IR relay system to control the cable tuner and vcrs that are in the first room from the second room, pretend I still do that.

1) Press a button for a TV select macro. Most of the buttons now control that TV. Maybe use some of those buttons.

2) Press a button for a VCR select macro. Most of the buttons now control that VCR. But the Vol and PIP button sets are still set to whichever TV I selected in step 1. Maybe use some of those buttons.

3) Press the button to select the cable tuner. Now the Channel and Menu buttons control the cable tuner. But the transport buttons still control whatever VCR was selected in step 2 and the vol and pip buttons control whatever TV was selected in step 1.

4) Watch a broadcast show. At each commercial, turn on the PIP to show that commercial and start the tape playing (to a different show taped earlier). At the end of the commercial stop the tape and turn off the pip.

In step 4 there could be one of two TVs with one of five different VCRs. But I never needed to define combinatorial modes for all of that nor remember unique keys to select such modes and I never need to press any device keys while in step 4. The device select macros simply omit specific key sets in order to let the remote remember the last time each such key set was selected.

Unlike a JP1 extender, I think a scratch design should let the user define the key sets when he sets up the remote, rather than having the key sets hard wired into the design. There are a bunch of other unnecessary complications left over from the design built into the JP1 remotes that a scratch design should skip. But the basic concepts are very powerful.

[Symbol]

Hi John,
I have seen and enjoyed quite a bit of your work in the last few months.
Thanks for all of it.
I also feel strongly about the appropriate synchronization of data flow between layers of software. I have quite a bit of experience with real-time software development and intend to use that kind of design philosophy quite heavily.
I am willing to take in all the advice I can get at this point. My experience in the past is that time spent to make a good design is handsomely rewarded down the road. So, please send me all that text "advice".

I just had the thought that since I have the ability to completely re-program this remote at will that I might start by developing a rudimentary "diganostic" remote as a proof of concept project. Based on the work I have done with my evaluation kit already, I think I could crank out the following in a couple of weeks: A remote that has 2 protocol sets, the Sony-12 protocol, limited to sub-device 00 (usually does most of the day to day operations) and a variation of the Panasonic code. One would use one of the device buttons (say TV for Sony, and DVD for Panasonic). These protocols are being selected at this time, because those are for devices I have, and as such my research has focussed on them up to this point. At that point, one would simply key in the OBC code (3 digits) on the numeric buttons and then press "enter" to send the code. This is not very practical as a day to day remote, but might be a valuable tool for somebody researching what codes ones devices respond to, and what they do. <<Caution if you plan to do this --- some devices have service and diagnostic activities tied to normally unused codes and could royally mess up the device>> I guess I don't have to tell you this John, it is for the benefit of others also reading this thread.

I have looked at your IRP notation. But lacking a guide to its meaning, and not willing to spend a lot of time deciphering it, I do not yet understand it. If there were a reasonably clear guide to the notation, I am sure that it might be a very good way to implement IR code generation internally on the device. If the notation is clear, then I see no real problem (other than code space, maybe) with implementing the understanding engine within the remote. If anything I might want a more compressed form of the notation (not necessarily human-readable) within the remote itself.

I have decided not to focus (yet) on the mechanics of how to implement devices. I want to hear from interested parties first. The method you describe is actually pretty straight forward to implement. Another paradigm is the "activity-based" approach. I am not sure how well that works in a remote where all the keys are already labelled for one. If one had free reign in labelling the keys, then it might be more feasible.

[vickyg2003]

John

John S. Fine wrote

For a previous discussion of this, I wrote up a lot of detail on the low level of IR generation. I strongly feel that the correct design is to build a strong foundation of basic IR generation so that individual protocols are less work.

Could you point me to that reading?

Have you looked at my IRP notation?

Could you point me to that reading as well?

I really haven't had any interest in the IR signals up to this point. I've studiously avoided learning about that aspect of the remotes. I guess now that I'm interested in extenders and such I probably should read more.

Learning about extenders makes me want to learn more.

Have you looked at my IRP notation?

Vicky

[johnsfine]

Are you using interrupts to time the IR generation?

I haven't worked through enough of the details yet (of how timers and interrupts interact in this chip) to be sure it is practical. But if practical, I think it is much better than using wait loops.

Without digging up that old detailed description (yet), the core idea is a circular buffer holding the next few durations (of alternating on and off durations). An interrupt at the end of each interval would reverse the state of the actual signal and program the next interval from the circular buffer. The main line code would wait until the circular buffer isn't full then compute and store the next interval.

[Symbol]

Are you using interrupts to time the IR generation?

I haven't worked through enough of the details yet (of how timers and interrupts interact in this chip) to be sure it is practical. But if practical, I think it is much better than using wait loops.

Without digging up that old detailed description (yet), the core idea is a circular buffer holding the next few durations (of alternating on and off durations). An interrupt at the end of each interval would reverse the state of the actual signal and program the next interval from the circular buffer. The main line code would wait until the circular buffer isn't full then compute and store the next interval.

Yes, I am using interrupts to time the IR generation. Is there any other really efficient way to do it?

This general idea of the circular buffer is how I am doing it. I call it the "pulse queue". When an interval expires (and sets off the interrupt), I simply load the registers (from previously set variables), then (after the new pulse is started) pick up the next entry from the queue to put into place for the next interrupt. Keeping latency low is important to getting accurate and repeatable signal generation.
Actually, "pulse queue" is a misnomer. It is really only half a pulse for each entry (a new level and associated duration). The durations do not even have to be alternating. If one needs a long interval (longer than the timer maximum period), one can put multiple entries of the same level into the queue.

Because the IR signal is modulated, one can set up the timer circuits to continuously generate the carrier frequency. Then one only uses the interrupts to time how long that carrier signal is active or turned off.

[johnsfine]

Yes, I am using interrupts to time the IR generation. Is there any other really efficient way to do it?

There is no other good way that I know of. But so far as I understand, UEI does not use interrupts that way. They do something that exposes difficult timing issues all the way up to the individual protocol executor.

When an interval expires (and sets off the interrupt), I simply load the registers (from previously set variables), then (after the new pulse is started) pick up the next entry from the queue to put into place for the next interrupt. Keeping latency low is important to getting accurate and repeatable signal generation.

Assuming latency is moderately low, it is more important to keep it consistent than to keep it very low. The maximum length path from the interrupt to the action (starting the next interval and setting the level) should be nearly identical to the minimum length path. If all edges are delayed by the same amount, that amount doesn't matter.

But it sounds like you have that aspect well under control.

Actually, "pulse queue" is a misnomer. It is really only half a pulse for each entry (a new level and associated duration). The durations do not even have to be alternating. If one needs a long interval (longer than the timer maximum period), one can put multiple entries of the same level into the queue.

Great. I was hoping to do that for very long Off intervals. I don't think any On inervals should be that long.

At a practical executor level many protocols include sequential segments of the same type (two On in a row or two Off in a row). I was hoping to NOT use the lowest level of non alternating (that you just described) for THAT purpose, because I don't trust it to be free of phase and timing glitches (for any very long Off interval there is no phase and small timing errors don't matter). Instead I hoped to have non-alternation support at the insert end of the queue as well. If you try to insert non alternating values and they aren't too large they would be automatically summed by the insertion code.

Again contrasting to the UEI designs, phase glitches are the likely result if a JP1 protocol executor simplistically generates two On intervals in a row. The black magic required to avoid phase glitches is an area only half understood by the best JP1 experts. The actual device often ignores phase glitches, but not always. Other remotes learning from a JP1 usually get confused by phase glitches. So I think a good foundation layer of the design should cover that issue such that the higher level code doesn't need to worry about it. That doesn't necessarily require summing values at the insertion end of the queue. But doing so at least makes coverage of the issue more obvious to a casual review.

Because the IR signal is modulated, one can set up the timer circuits to continuously generate the carrier frequency.

That part is very clear in the chip documentation and I think anyone programming a remote would do it that way.

Then one only uses the interrupts to time how long that carrier signal is active or turned off.

That's the part that was unclear from the documentation. How you consistently restart the timer for the next interrupt at the end of each interrupt and how much (if anything) you need to adjust the values to compensate for the latency of interrupting and reprogramming the timer.

I'll be interested to see your code for that.

[Symbol]

Assuming latency is moderately low, it is more important to keep it consistent than to keep it very low. The maximum length path from the interrupt to the action (starting the next interval and setting the level) should be nearly identical to the minimum length path. If all edges are delayed by the same amount, that amount doesn't matter.

Indeed. That is true, and I think I have it under control.

That's the part that was unclear from the documentation. How you consistently restart the timer for the next interrupt at the end of each interrupt and how much (if anything) you need to adjust the values to compensate for the latency of interrupting and reprogramming the timer.

You don't actually restart the timer. You only change the compare register (the point in time that it will interrupt the next time). Since the hardware timer just keeps running, as long as your latency is low and consistent, you will get very accurate timing. And if the carrier is already on, you would simply fail to turn it off (in the case of two consecutive active levels). That takes care of any phase glitches.

[johnsfine]

I have decided not to focus (yet) on the mechanics of how to implement devices. I want to hear from interested parties first.

We may need to split this thread somehow into operational aspects vs. technical details, so all my verbose posts on the technical details don't scare away comments from others on the operational issues.

It sounds like you want to hear from the other extender writers and users on device selection macros and other end-user-programmable as they would have been done in extenders if we weren't lacking space an constrained by what's in the ROM. Hopefully we can get some of those ideas posted.

The method you describe is actually pretty straight forward to implement.

If it wasn't, we couldn't have fit it in an extender. But in addition to being easy to implement, it gives the user great flexibility without either the tedious detail or complex paradigm required by other methods of providing the same flexibility.

Another paradigm is the "activity-based" approach.

To answer that, I want to distinguish between two end users: The end user (1) who does the "end-user programming" of the remote vs. the end user (2) who actually uses it. Even when they are the same person this distinction is important.

For user 2, an activity-based paradigm may be exactly right. Whether it is right depends on how his A/V equipment is interconnected and used. We shouldn't commit to any particular position on that.

The device selection method I described is perfectly suited to an activity-based paradigm for user 2. You simply include more of the key group assignment commands in the activity selection macros and make less use of the ability to remember a previous group assignment.

I've played with the PC side software for some remote that tries to give user 1 an activity-based paradigm. Maybe I'm missing something, but I just don't get it. The whole concept seems to be rooted in the idea that the software designers knew more about how my A/V equipment is interconnected and used when they wrote that software than I know when I try to use that software. Maybe their user 1s are that ignorant, but that still doesn't mean the software designers are that good at predicting their customers A/V usage. So far as I can tell you just end up fighting the software to deal with the things it gets wrong.

So activity-based for user-2, certainly whenever user-1 configures activity-based macros with his non activity based programming paradigm.

All that is of course just my view. I expect some other experts here have better insight into what real user-1's will understand.

Could you point me to that reading?

I hope to find it.

Learning about extenders makes me want to learn

Meanwhile, do you want to comment on the operational ideas?

[Symbol]

We may need to split this thread somehow into operational aspects vs. technical details, so all my verbose posts on the technical details don't scare away comments from others on the operational issues.

It sounds like you want to hear from the other extender writers and users on device selection macros and other end-user-programmable as they would have been done in extenders if we weren't lacking space an constrained by what's in the ROM. Hopefully we can get some of those ideas posted.

Yes, please feel free to split the thread. Several times if necessary. Just let me know what threads get started from this one. I think if you do it, as site manager, it will get done better, since you are more experienced in what topics should go where.

Yes, I want to hear from anybody who wants to chime in about how the remote should act in the user's hands, and how they would like to see things done under the covers. I could just go ahead, but usually products get better designs if many people give their input.

I understand your comments about the different users. I would like to hear from any of the "virtual users" either the end-user, the programmer-user, or even the "I could design this better" user.

what protocol does the current remote use when talking to JP1.2 code (on the PC) over pins 4 and 6? I would like to implement the same communication protocol in my finished code.

Also, if anyone knows it: What does the circuit around pin 4 on the processor (PTB3) do?
It makes zero sense to me as a hardware designer.
If the pin was capable of ADC reading, it might have some use as a battery low power sensor, but that pin does not have ADC capability (as far as I can tell by looking at the datasheet).
The only Analog capability this chip has is on pins 30 and 31 and that is only a comparison function. (Still enough for low battery sensing, but these pins are not hooked up to anything according to the schematic)
Now if PTB3 and PTD5 were somehow confused with each other by the person responsible for creating the schematic.....

BTW. Large thanks to whoever figured out the 8820 schematic. They have saved me a long and arduous task.

[jsevinsk]

After using a Harmony remote, I have taken the activity-based approach when I programmed my 6131. The device keys are activity keys; when I press the DVD button, the remote knows that it has to turn on the DVD player and also the TV and receiver. It then pauses until the devices are ready, then sends the commands to switch inputs.

When I then press the PVR button, it turns on the PVR and switches the receiver and TV to the right inputs. It leaves the DVD player on (for now), but I could have programmed it to turn it off at this time.

The remote uses the ToadTog bits to remember which devices are on, and a long key press of the power button turns all those devices off.

There's always a chance that something will go wrong, and either the TV or the receiver won't come on, or will be set to the wrong inputs. Pressing the device key a second time will re-send the input selection commands. A long key press of a device key will just toggle the power for that device and not kick of that activity. This is most useful when I walk into the living room and see that somebody has left the DVD/VCR/receiver/etc on by accident.

One thing I need to add is Harmony's "leave this device on" concept. The DVD player in the living room is hooked up to a modulator so that you can watch DVDs on the other TVs in the house. When I'm done watching the DVD in the living room, I don't always want the "all off" button to turn off the DVD player because someone else in another room might be watching it.

So far, this approach has been working well for me, and I now find that the 6131 is easier to use than the Harmony remote.

One thing that's not handled very well are delays. Each device needs a delay between when you turn it on and when it can receive a command. The TV is the worst -- it needs 9 seconds. When I press the DVD button to turn everything on, the remote could be sending other commands to the DVD player or receiver during those 9 seconds, because they are ready much faster. It would be nice if the remote could handle this better -- maybe separate command queues for each device that get worked on in parallel? What would be really cool is if I could pick up the remote and press "PVR 1 2 3 ENTER" all at once and have it turn on the TV, receiver, and the TiVo, wait a while, switch the TV and receiver to the right inputs, and change the TiVo to channel 123. Yeah, that would be cool.

[johnsfine]

When I looked at that in the schematic, I just assumed it was a pull up for an unused input. But I didn't research to see if that was true.

Some input pins of some chips cannot be safely left floating (when unused), because the gates just beyond the input would oscilate and waste power. The spec sheet for the chip ought to tell you which pins have that issue. I haven't checked whether it does. But two jobs and many years ago, I was involved in projects where we used chips for which the spec sheet was wrong about that and discovered places where pull ups on unused inputs saved power despite the spec sheet indicating otherwise.

[Symbol]

Well, that could be why that is there, but...

a) That pin is programmable as an output (in this circuit, make it a high output for power savings), or an input with an internal pull-up (which would waste a little more power) - according to the data sheet.
b) Freescale (aka Motorola) have been one of the best in the business in having complete and correct datasheets. And for putting out corrections and errata when indicated.
c) There are quite a few other unconnected pins that would default to inputs on reset.
d) The datasheet specifically (in the Parallell Port section) states that unconnected GPIO pins should be initialized to outputs for this very reason.

So, if the PTB3 circuit is there for that reason, somebody at UEI got the other pins right, but for some reason left this pin the "other way"

This might be a mystery never solved...
It should cause no real problem, however.

[johnsfine]

That's what I deserve/get for answering off the top of my head with no research.

After slightly more research I have a totally different answer. I'm not sure of this one either, but I saw some supporting evidence.

A single firmware version was coded for multiple models, some of which have in internal "modem" for .wav upgrades and some don't. The firmware configures that pin as an input and uses it in the .wav upgrade process. The versions without the "modem" need a pull up resistor there either to keep the input from oscillating or to keep the software from seeing random input when it tries a modem upgrade.