IrScrutinizer: capturing, generating, analyzing, import, exp

Post by **mathdon** » Sun May 03, 2020 5:53 am

Barf wrote:Glitches: are you talking about the IrWidget or the Arduino?

Both. I have given up trying to find a reproducible example as it seems to be random, but I have not given up on finding what the glitches are doing to make a signal undecodable. I will do some more experiments and report later.

I am slightly uncomfortable with your huge gaps within a sequence (60ms, 100ms, 400ms). Is this one sequence with a lunch break, or is it several sequences? Yes, my firmware has a problem with it, but I am pretty sure that it goes for many other types of hardware too. (Not all of them have setable ending timeout...)

There seem to be two separate questions here: what is the correct IRP for the Dyson signals, and can the capture hardware capture the Dyson signals accurately. I don't think the hardware question should influence what is put in IrpProtocols.xml. That should be a representation of what a Dyson remote sends, which is what I have tried to do. As I explained earlier, the 100ms and 400ms gaps are real, they occur in the signal generated by the briefest press of the button. IrpTransmogrifier can decode these signals if they are input in a .ict file from IRScope, it can generate these signals as raw time sequences and executors can be written to transmit them. I have written such an executor for MAXQ processors for use in my URC6440. Any simplified form of IRP would have to miss out the ditto frames, as these are sent after two complete frames separated by the gap in question, not after a single frame. As regards hardware that cannot capture the complete signal, that is the purpose of having Dyson_relaxed.

On the capture question, it seems to me that IrScrutinizer has all the data it needs to handle the large gaps. I understand that the Timestamp field is a Java Date, which can be read as a time in milliseconds. So if the capture process treats two frames as distinct signals if they are separated by 100ms or more, the true gap between them can be calculated from their timestamps. If the ending silence parameter means what I expect, that a signal continues until a gap of this size is encountered, then the calculated gap can be compared with the ending silence value to see if these should be joined into a single signal. This, of course, presumes that both frames have been captured without glitches, but that is a different issue.

Barf · Post by **Barf** » Sun May 03, 2020 5:55 am

Graham, unless you "have no time to sharpen your axe because you have too many trees to fell", I strongly recommend that you get to know a good diff program.

I have no idea if it solves your problem, but I have made the parameter ignoreLeadingGarbage (that you know for RemoteMaster) user settable (default false) Options -> Ignore leading garbage on decode. Please try it.

Barf · Post by **Barf** » Sun May 03, 2020 6:24 am

mathdon wrote:
Barf wrote:Glitches: are you talking about the IrWidget or the Arduino?
Both. I have given up trying to find a reproducible example as it seems to be random, but I have not given up on finding what the glitches are doing to make a signal undecodable. I will do some more experiments and report later.

For Arduino, please turn on Options -> Verbose. That makes the communication to be echoed to the console, and can then be analyzed.

On the capture question, it seems to me that IrScrutinizer has all the data it needs to handle the large gaps. I understand that the Timestamp field is a Java Date, which can be read as a time in milliseconds. So if the capture process treats two frames as distinct signals if they are separated by 100ms or more, the true gap between them can be calculated from their timestamps. If the ending silence parameter means what I expect, that a signal continues until a gap of this size is encountered, then the calculated gap can be compared with the ending silence value to see if these should be joined into a single signal. This, of course, presumes that both frames have been captured without glitches, but that is a different issue.

Better to use endingTimeout.

Post by **mathdon** » Sun May 03, 2020 12:31 pm

I've just done some more testing and now have not found any glitches with the Arduino. I certainly did before, as when I started with IrScrutinizer I intended to use only the Arduino, and only tried the Widget because of those glitches. In current testing with the Widget with ignore leading garbage turned on, I have got decodes whose timing display shows garbage at the beginning that consists of the correct signal but starting part way through. So with the Widget it definitely can miss the start of the signal.

Post by **mathdon** » Mon May 04, 2020 8:45 am

I have just done some testing with the Widget, with parameters Start capture=6000, Max capture length=2000, Ending silence=300. Repeat finder and Signal cleaner turned off. Using "Scrutinize signal" and the (single-shot) Capture button. Several times in a row I got "No signal received" almost immediately after pressing Capture, before having any opportunity to send a signal. I had set Start capture to 6000 to make quite sure that this was not the capture slot timing out. Other oddities have included "Internal error: IrSequence had odd length=97" before I had sent a signal, and strangest of all, a decode "RC6-6-20: {D=0,F=2,S=12}, beg=48, end=95, reps=1 {UNDECODED. length=48}" appearing before I had actually sent a signal.

One that I suspect is particularly revealing was one Capture displaying

Code: Select all

Freq=36400Hz[+2637,-873][][]

after a signal was sent, followed by the following on the next press of Capture BEFORE any signal was sent:

Code: Select all

Freq=36071Hz[+360,-462,+444,-434,+444,-889,+444,-890,+887,-444,+444,-434,+444,-462,+444,-434,+444,-462,+444,-434,+444,-444,+444,-444,+887,
-434,+444,-889,+444,-434,+444,-417,+444,-444,+444,-434,+444,-462,+444,-434,+887,-878,+887,-123734,+2661,-889,+444,-434,+444,-462,+444,-889,+444,
-878,+887,-434,+444,-462,+444,-434,+444,-444,+444,-417,+444,-434,+444,-434,+444,-462,+887,-462,+444,-889,+444,-434,+444,-434,+444,-434,+444,-434,
+444,-417,+444,-444,+887,-890,+887,-123761,+2661,-890,+444,-434,+444,-462,+444,-862,+444,-862,+887,-417,+444,-444,+444,-434,+444,-462,+444,-434,
+444,-462,+444,-434,+444,-444,+887,-462,+444,-890,+444,-434,+444,-462,+444,-434,+444,-444,+444,-417,+444,-434,+887,-889,+887,-1417][][]

If I concatenate the first sequence with the part of the second before the first leadout, it decodes as "RC6-6-20: {D=0,F=5,S=12}, beg=0, end=45, reps=1", which is the same as the decode of the sequence after that first leadout until the second one. The lead-out of an RC6-6-20 signal in its IRP is 100ms, reasonably consistent with the 123ms occurring here. This appears to be one signal with the first frame split into two, showing as [+2637,-873] on the first press of Capture with the rest of that frame being ignored initial garbage for the rest of the signal showing on the next press of Capture without any new signal being sent.

I hope this is sufficient evidence to persuade you that there is a bug involved here. I cannot see how this behaviour can be the result of extraneous light or any such thing. I hope it may also be sufficient detail for you to identify what is going on.

Post by **mathdon** » Mon May 04, 2020 11:12 am

On further consideration, I don't think I ever had capture problems with the Arduino device. My initial report on Capture problems was before I received the Arduino. After I received it, I started working with the Dyson signals and the trouble I was having with the Arduino with them was a result of the Arduino not being able to capture the leadout of > 65ms. I was not aware of that limitation at the time and put it down to similar glitches to the Widget, but that was not the case. I have done further testing with the Arduino with the various Capture facilities and have had no glitches, so my apologies for maligning your device. The Widget glitches are definitely real, however.

Barf · Post by **Barf** » Mon May 04, 2020 1:34 pm

mathdon wrote:... In current testing with the Widget with ignore leading garbage turned on, I have got decodes whose timing display shows garbage at the beginning that consists of the correct signal but starting part way through. So with the Widget it definitely can miss the start of the signal.

So at least "ignore leading garbage" works as intended!

Several times in a row I got "No signal received" almost immediately after pressing Capture, before having any opportunity to send a signal. I had set Start capture to 6000 to make quite sure that this was not the capture slot timing out. Other oddities have included "Internal error: IrSequence had odd length=97" before I had sent a signal, and strangest of all...

So I tried again on my Linux desktop: rock solid, very reliable captures. Switched to my Windows laptop: then the "fun" started. Could reproduce your problem, only seldomly a usable capture. Tried to update the drivers (FTDI), both from Microsoft and from FTDI. Tried to fumble with some of the driver parameters. Still 75% garbage. Sigh... (Tried with same laptop but Fedora Linux: works perfectly.) So there is definitely a deeper problem. I have planned to replace the ancient RXTX-system for a LONG time, so it may be better to migrate to a better system, and see if it fixes the problem with Windows. (I am pretty confident that I did at least elementary testing on Windows when IrScrutinizer was first released, and that it worked better then.) The IrWidget uses the serial port in a pretty unorthodox fashion...Issue.

On further consideration, I don't think I ever had capture problems with the Arduino device.

happy to hear that.

a result of the Arduino not being able to capture the leadout of > 65ms.

I "defend" myself by saying that this is truly a bona fide "limitation, not a bug". It comes from using 16 bit integers for storing the durations in micro seconds (2^16 -1 = 65535). Still, I plan to fix it.

Barf · Post by **Barf** » Mon May 11, 2020 11:18 am

Following, here requested fixes have been fixed, and are available in the CI build:

* The internal hyperlinks (also with embedded double-quotes) in the enclosed IrScrutinizer.html now works also with MS Edge (and with IE).
* In the distribution, the "confusing" *.md5, *.sha1, *.sha512 files have been replaced by the files checksums.md5, checksums.sha1, checksums.sha515.
* The non-working accelerators in the Scrutinizer Remote tables (Ctrl-Up, etc) have been replaced by working key bindings. They are listed in the help test (at the end).
* The "T" combo box on "Generate" has been replaced by a normal text field.

Post by **mathdon** » Tue May 12, 2020 4:48 am

I haven't tried the new fixes yet, but thank you for them. This is a comment on the use of GlobalCache iTach Flex as a capturing device for Dyson signals, with the IrpProtocols.xml entries that I offered in an earlier post. In contrast to the Arduino device, the iTach Flex can handle the 100ms leadout between the two full frames of the Dyson signal, and so decodes all except the Power signal (with its 400ms leadout between them) as Dyson rather than Dyson_relaxed. This includes signals with ditto frames, with the number of dittos correctly reported. The Power signal gets reported as two Dyson_relaxed frames rather than as Dyson2.

I have continued to try, without success, to get IrScrutinizer to treat the two frames of the Power signal, with their 400ms separation, as a single signal. As I have reported before, it seems to me that setting "Ending silence" to more than 400ms should achieve this. That does not do it. You replied "Better to use endingTimeout", but I can find no such parameter in IrScrutinizer and you have not said what "Ending silence" does if it is not the minimum duration of silence needed to be regarded as the end of the signal. Could you clarify, please?

Post by **mathdon** » Tue May 12, 2020 5:28 am

Just to report that the fixes in the latest CI build all work as intended.

Barf · Post by **Barf** » Tue May 12, 2020 8:56 am

"Ending Silence" is the same as endingTimeout. I do not think that there are many hardware solutions that will let you capture IR signals with 400ms intra gap (but IrWidget and IrToy (has 1.x seconds soldered in) might do). The GlobalCache does not have a user settable parameter like endingTimeout. Also not quite implemented yet.

Possibly we should consider creating a "macro recorder" as a new project, cf the "Glow with the show" discussion.

Post by **mathdon** » Tue May 12, 2020 10:37 am

I don't know about the "Glow with the show" discussion, but IRScope essentially records macros in its normal use. Each signal in the macro is decoded and displayed separately but as part of a single waveform, the start and end values distinguishing the separate signals, each of which is highlighted in red when you select that signal in the display panel. So the gap duration between the signals is preserved and forms the lead-out value for the preceding signal.

IrScrutinizer ALMOST does this on its "Scrutinize remote" panel. I haven't tried scrutinizing a macro, but the Dyson "Power" signal appears as two separate entries of Dyson_relaxed. If it were possible to read the Java timestamp in microseconds, the duration between the two frames could be determined.

Post by **mathdon** » Wed May 13, 2020 6:13 am

Miscellaneous reports on IrScrutinizer, FYI.

(a) On my 64-bit Win 8.1 machine:

IR Widget works, subject to its usual issues.
Arduino device needed manual install of driver, but then works (only capture tested).
Global Cache iTach Flex needed manual entry of IP address, but then works (only capture tested).

(b) Ubuntu 16.04 32-bit with OpenJDK 1.8.0_252 in VirtualBox on Win 8.1 machine:

Global Cache iTach Flex needed manual entry of IP address, but then works (only capture tested).
Neither Widget nor Arduino device will connect, as no serial ports are found, even after refresh. They were tried separately, so that in each case there should be exactly one serial port. Command dmesg shows that this is ttyUSB0 in each case.

(c) Windows XP SP3 32-bit with Java 1.8.0_05 in VirtualBox on Win 8.1 machine:

IR Widget works, subject to its usual issues.
Arduino device driver installed automatically and it then works (only capture tested).
Global Cache iTach Flex needed manual entry of IP address, but then works (only capture tested).

Java 8 is not actually supported for Windows XP but I found these excellent instructions on how to install it. IMHO it is rare to find instructions on the web that work perfectly, with every step behaving exactly as described, but these meet that criterion.

The only issue these tests have shown up is the failure to find the serial ports on 32-bit Ubuntu 16.04.

Barf · Post by **Barf** » Wed May 13, 2020 7:45 am

Glow with the show was discussed here, the idea that someone had, was to record a complete "Glow-with-the-Show"-show as an IR sequence ("macro recorder").

I have been planning to write an macro recorder for several years. This

If it were possible to read the Java timestamp in microseconds, the duration between the two frames could be determined.

is nothing but an idea of how to write such an animal.

But probably it is easier to allow the Arduino firmware to accept larger endingTimeouts.

mathdon wrote: IR Widget works, subject to its usual issues.

Will fix in the context of this.

Arduino device needed manual install of driver, but then works (only capture tested)

OK. IIRC Win10 will find the driver itself.

Global Cache iTach Flex needed manual entry of IP address,

A firewall/portfilter is blocking the beacon, or UDP port 9131.

(b) Ubuntu 16.04 32-bit with OpenJDK 1.8.0_252 in VirtualBox on Win 8.1 machine:
Global Cache iTach Flex needed manual entry of IP address, but then works (only capture tested).
Neither Widget nor Arduino device will connect, as no serial ports are found, even after refresh. They were tried separately, so that in each case there should be exactly one serial port. Command dmesg shows that this is ttyUSB0 in each case.

Accessing hardware, like serial ports, on a virtual machine is, well, "hairy". No reason to expect that it should be working. You might try as root, to see if it is a permission problem.

Thanx for the report on XP.

Post by **The Robman** » Wed May 13, 2020 8:34 am

Barf wrote:Glow with the show was discussed here

I actually bought one of those "Glow with the Show" hats and spent a lot of time writing a custom executor for it, then nobody used it. I do recall that the hat was very particular about the timings, you had to get them exactly right in order for it to work, there wasn't the usual tolerance for "close enough" signals.