JP1 Remotes

Just a bunch of small questions.
I saw some kind of "device address" tables. Am I right to assume that device/protocol upgrades do not have to immediately follow each other and can be spread around memory with possible holes in between? (unlike macro/keymoves)
Is the same organization supported by extenders or they do something different?
It seems that immediately after upload these addresses are stored at standard locations in the header, and extender's code sits as a device upgrade. After extender got invoked for the first time it modifies those offsets but leaves them at the same place. Is this so?
If yes, then is there exact offset in the extender code where the _future_ addresses for keymove zone, device table etc. are located? If so, are they different in different extenders?
Is there some addresses (except for header's) that are hardcoded in mcu and thus can not be moved?
Are the answers for above questions the same for all remotes known so far?
Sorry for a long list and appreciate your input.

I can only speak for the S3C80 models and Mark has told me that my ideas based on those models are not valid for some of the other models. However, I think an improvement that is enabled only for S3C80 models would be better than waiting to find a more general solution that we'll probably never have time for.

The MCU hard codes the start and end of the KeyMove/Macro area and strictly requires all KeyMoves/Macros to lie contiguously between those.

The MCU hard codes the start and end of the Learned signals area and strictly requires all learned signals to lie contiguously between those.

The MCU hard codes the location of the root object of the upgrade area but does not in any way constrain the location, sequence or contiguity of any other objects in the upgrade area.

Some extenders (with significant difficulty) override the MCU rules for KeyMoves/Macros to use a different contiguous range. It is not a simple override, so having more overrides is probably not practical within a 2K eeprom (would take enough code that it would destroy any possible benefits of more flexible use of what eeprom space remains).

Most upgrade objects have lengths that cannot be determined simply by reading the object. The remote has no need to know the length of any upgrade object, so this length problem imposes no additional constraint on object locations from the viewpoint of the remote.

But we need IR to be able to read and understand eeprom images as well as write them. So IR must be able to deduce the length of upgrade objects.

Currently IR writes upgrade objects contiguously in a strict sequence. So that the length of all but the physically last object can be computed by knowing which object is physically next and using its start address to compute the length of the current object. A specific object whose length IS readable was chosen to always go last.

My proposal is to invent a totally new way to arrange the objects such that their lengths can be deduced. With an extender we can create the maximum practical KeyMove/Macro area (not need any flexibility to move the boundary). With or without an extender we could use any unused parts of all other sections for upgrades.

1) Fill the used part of all other sections first before starting to compute upgrade object locations.
2) The address of the root object of the upgrade area is fixed.
3) Compute the size and location of each "area" available for other upgrade, including the rest of the original upgrade area (after the root) and including the free remainder of each other section (with current RDF rules, the only sections that could have free remainders are Keymove and Learned signals).
4) Assign the set of upgrades to the pool of areas by some fitting algorythm. A crude first fit method would be trivial and would be a giant step forward from current status, but some better fit algorythm might be justified. Note this step just assigns an area number to each upgrade object, not an exact address.
5) Within each area consume space from the end first, working backward toward the beginning.
5a) Init an L pointer for each area as one byte beyond the end of the area.
5b) Assign objects in logical sequence, but each to the area decided for it in step 4.
5c) As each object is assigned set its start address to the area's L pointer minus the object's length, then set the L pointer to that start address.

When you read back objects (in logical sequence) you have the address of each object and must deduce its length.
6a) Init an L pointer for each area as one byte beyond the end of the area.
6b) From the address of an object you can easily look up what area it is in.
6c) The length of the object is the L pointer of that area minus the start of the object.
6d) Update that L pointer to equal that start address.

We would also need a simple check to cover the case that the upgrade area is laid out the old way. That would best be done by a look ahead from the second logical object to the third (but it could be safely done on any other objects as well). If the third object lies between the second object and the value of the L pointer (at the time the second object is read) then the old layout is being used.

Eh.. hmmm... I was asking some simple questions and you put the weight of an intellect on my shoulders

Ok, what I'm attempting to write is a framework for eeprom image access and management. The basic concept is that any such image can be split into a set of non-intersecting binary chunks of sertain types (e.g. header, macro, device etc.) If it is possible to formalize the placement rules for these types (e.g. type "macro" must follow other macro, keymove or beginning of macro area) then it should be possible to write a platform-independent memory optimizer. (under "platform" I mean mcu and/or the make of the remote)

Then, the framework would consist of:
- generic memory map components;
- image loader; parses the image and creates a memory map, something like simplified DOM model.
- manager; moves objects around, optimizes memory distribution, allows adding/removing objects. Applications, such as RM, would be able to simply say "add this object" and manager will do the rest.
- optional viewer/editor; this can be made pluggable into any application as additional pane in the UI.

As you can see, only image loader is platform-dependent piece in this picture. If fact, it is the only part related to the remotes, everything else can be used for other equipment e.g. cell phones (is there such thing as JP1 phone? ) Right now I'm considering two possibilities: generic loader that uses RDF file vs. loading drivers. Basically, my questions above were all related to this choice. Unfortunately, I haven't found any complete documents on the memory organization (suppose, there isn't any). So, I have to deduce it form the pieces of source code available at yahoo.

As for your proposal on the memory distribution the fun part is that it produces almost the same result as my "type placement rules", being much simplier to implement at the same time. It's just that I'm afraid it might not be able to cope with all future models of remotes.

Thanks a lot!

wwwoholic wrote: Ok, what I'm attempting to write is a framework for eeprom image access and management. The basic concept is that any such image can be split into a set of non-intersecting binary chunks of sertain types (e.g. header, macro, device etc.) If it is possible to formalize the placement rules for these types (e.g. type "macro" must follow other macro, keymove or beginning of macro area) then it should be possible to write a platform-independent memory optimizer. (under "platform" I mean mcu and/or the make of the remote)

I think you may be targetting too high a level of generality.

wwwoholic wrote: Then, the framework would consist of:
- generic memory map components;
- image loader; parses the image and creates a memory map, something like simplified DOM model.
- manager; moves objects around, optimizes memory distribution, allows adding/removing objects. Applications, such as RM, would be able to simply say "add this object" and manager will do the rest.
- optional viewer/editor; this can be made pluggable into any application as additional pane in the UI.

I don't really follow the above, and I'm sure I'd have some more specific disagreement if I did follow it. I think you're implying an incremental view of operations, that would be both harder and less useful than the way extinstall does this (and the way IR sort of does it):

We have specific collections of objects that should exist in the program quite independent from their existence in the eeprom image. Most of the manipulations by the program should apply to those collections of objects. The interface to the eeprom image data should have the ability to replace the entire set of objects in the eeprom image with a copy of those contents of those collections and the ability to replace the entire collections of objects with a copy of objects extracted from the image.

The user needs to be told if the current collection of objects doesn't fit, which would need a test method if the store method is too slow to repeat on every GUI step. But the fact that it doesn't fit shouldn't invalidate the collections of objects. The user should have flexibility in resolving the problem, but can't upload until it's resolved (without some serious requested enhancements to IR the user also can't save until it's resolved).

There is no benefit to an Add/Remove view of objects in eeprom memory. Add/Remove should occur in flexible collections within the program.

johnsfine wrote:I think you may be targetting too high a level of generality.

I hear this about twice a day at my work. But you should see a look on the face of a customer, who is asking for major changes in the product and hears response "no problem, one week... including QA".

Seriously, so far I have 4 placement rules: movable, fixed, stickup, stickdown. The last two can be accompanied by the list of allowed neighbours. Doesn't look too complicated to me. BTW application of "stickdown" rule will result in exactly your algorithm.

I think you're implying an incremental view of operations, that would be both harder and less useful than the way extinstall does this
<...> Add/Remove should occur in flexible collections within the program.

Agreed, but:

The user needs to be told if the current collection of objects doesn't fit

Shouldn't it be done at the moment new object (e.g. keymove) is added and not when "save" button pressed?

which would need a test method if the store method is too slow to repeat on every GUI step.

I don't see how tossing around a dozen objects in memory can be too slow. Even a simple permutation will immediately exclude a lot of variants just because many of the objects have same size. Furthermore, until a certain amount is reached it will not even begin, because objects will fit in on the first try.

Here is much more important argument against object-oriented api: it requires multiple changes all around GUI.

So, which one you'd rather have? Both?

Also, is there anything [I didn't ask about because had no clue] in the extender's code that has to be updated before writing final image into eeprom? In the same way as device table is updated with final addresses.

And last: If macro base is fixed and we've reached next hardcoded address, and the length of the data over there is less than 15 bytes, can we jump over it by creating artificial macro which will never be invoked?

wwwoholic wrote:

The user needs to be told if the current collection of objects doesn't fit

Shouldn't it be done at the moment new object (e.g. keymove) is added and not when "save" button pressed?

Absolutely.
But (1) after it fails to fit the GUI should not reject the addition of the new object. Rather the GUI should display the problem message and the user should be able to continue working and later the user might delete something else that makes it fit.
But (2) if this is in the Java version, Java is SLOW! C++ is fast. In C++ I'd never consider writing a "test" routine to do a subset of the work of this "store all" routine. I'd just use the "store all" routine and look at it's success/failure result.

wwwoholic wrote: I don't see how tossing around a dozen objects in memory can be too slow.

There are usually more than a dozen KeyMoves. A test routine need only look at the total, not touch them individually. I expect even Java is fast enough that it doesn't matter, but I wasn't SURE, so I allowed for the possibility that the test might be a performance advantage.

wwwoholic wrote: Here is much more important argument against object-oriented api: it requires multiple changes all around GUI.

Sorry, no clue what you're saying.

wwwoholic wrote: Also, is there anything [I didn't ask about because had no clue] in the extender's code that has to be updated before writing final image into eeprom? In the same way as device table is updated with final addresses.

Before I can answer that, you need my view on kinds of memory:
1) A bunch of fixed locations, such as the setup code slots, the settings variables, the CRC, etc.. That the RDF file explicitly tells what data goes in which locations.
2) A section for KeyMoves and macros.
3) A section for Learned signals.
4) A section for Upgrades.
5) Areas that must not be touched by IR because they were too much effort to put in the RDF.
6) Disconnected free memory.

Anything not explicitly covered by the RDF is either (5) or (6). Right now IR can't tell the difference. In most 2K eeproms there is no significant amount of (6) anyway. It would be more elegant to add something to the RDF to distinguish (5) from (6).

Now I can answer: Most of the extender is of type (5). Scattered through the extender are a few type (1) things, that are specified in the RDF in a way that IR has no reason to see them as any different from any other type (1) things. Any RDF could be written that made the entire extender code be type (1) rather than type (5), which would fix certain operational problems with extenders, but then a tiny edit to an extender would cause a massive change to the RDF. Until that massive change could be automated it would be an unreasonable burden on extender writing.

wwwoholic wrote: And last: If macro base is fixed and we've reached next hardcoded addres, and the length of the data over there is less than 15 bytes, can we jump over it by creating artifical macro which will never be invoked?

The end address for the (keymove) macro area is in all current cases quite solid. There is no way to jump over that end point and use anything beyond it.

For an extender that has learning disabled, you might want to redesign things so the keymove area takes more away from the upgrade area. In that case you would have a problem supplying the single fixed byte of 0 that must go (for safety) at the start of the original learned signal area. Your idea could fix that.

If the image store/load routines detected and managed the case of a type (1) item landing within the keymove/macro area, they could create a dummy object (keymove or macro depending on boundary details) to encapsulate that type (1) item and hide it from the extender.

That would allow an extender design that steals the maximum memory from everything else for Keymove, which combines with memory manager automatically reclaiming unused keymove memory for upgrades, resulting in nearly complete memory flexibility (for extenders that kill learned signal support).

However, almost no one ever wants that much memory for keymoves and learned signals, so it really isn't worth the bother.

johnsfine wrote:Java is SLOW! C++ is fast.

Off topic, but still... it depends! I saw results of thoroughly performed tests that say Java outperforms C++ in server applications, thanks to built-in memory management and optimized multithreading. Heck, our own server stays like a rock under 10,000 users logging in within 1 min. GUI on the other hand is definitely slow.

johnsfine wrote:

wwwoholic wrote: Here is much more important argument against object-oriented api: it requires multiple changes all around GUI.

Sorry, no clue what you're saying.

Only that if existing application has to make calls to eeprom memory manager upon every user action, then these invocations should be inserted into the code at many places, on different windows and dialogs. Too much code change. And the result is the same as when you call it once upon "save" button.

Ah... I got it. when I said "object-oriented api" I meant "set of functions that work with memory on per-object basis" as opposed to group operations. Wrong wording.

The end address for the (keymove) macro area is in all current cases quite solid. There is no way to jump over that end point and use anything beyond it.

I hoped that because "end" is defined by 00h you can stretch it to the end of physical memory.