Arduino sketch to generate raw code for use with IR Scope

[Barf]

I have done some testing. I have tested with an Arduino Uno (atmega328) with a QSE159 (inverting). The limited memory is really a problem (says bufferSize=801), so cannot capture a complete IR signal (e.g. NEC1). (I just ordered a Mega 2560 though). Moreover, it works only every even invocation, (see this log): It appears as on the odd invocations, the gaps/pulses are out of phase, see the "compact dump".

I made some smaller changes in the inclosed cpp file, but these are not to be considered as finished.

On the whole, I am impressed by the precision and the reproducibility of the measurements!

What do you say about a version making the "aggregation" during the capture, for saving memory? I simply do not find it technologically sound to require (e.g.) a Mega2560 for capturing IR signals.

[MikeT]

I have tested with an Arduino Uno (atmega328) with a QSE159 (inverting). The limited memory is really a problem (says bufferSize=801), so cannot capture a complete IR signal

I am working on a version which just uses the rising edges of the signal which will save 50% RAM. This version cannot calculate the duty cycle but I think this is normally not really important. More than 50% duty cycle make no sense; everything less than 50% saves some power and less than 20% will probably become unreliable so it will normally be between 30% and 50%?
Depending on the edge steepness of the analog IR-signal the duty cycle cannot be detected very accurate.

Moreover, it works only every even invocation
I made some smaller changes in the inclosed cpp file, but these are not to be considered as finished.

Thanks, I will have a look at it.

What do you say about a version making the "aggregation" during the capture, for saving memory? I simply do not find it technologically sound to require (e.g.) a Mega2560 for capturing IR signals.

The current code needs around 2.8 µs for each edge. When only the rising edges are used this saves 50% time and RAM so it should be possible to do the aggregation during the capture.

The ATmega32U4 on the Leonardo has some more RAM than the ATmega328P and the USB interface should be much faster than the UART interface. In addition to the optimization this could become a pretty good IR-Widget.

Michael

[Barf]

What do you say about a version making the "aggregation" during the capture, for saving memory?

What I had in mind was something like this (in pseudo-code):

Code: Select all

// positiveEdge and negativeEdge are wrt the IR. If using inverting sensors,
// like QSE157/159, this is assumed to have been taken into account.

assert(BUFFSIZE % 2 == 0);
time_t data[BUFFERSIZE];
unsigned int dataIndex;

void doCapture(time_t beginTimeout, time_t endTimeout, time_t aggregateTheshold) throws BeginTimeoutException, TooLongException {
    dataIndex = 0;

    // Find the first rising edge, with a timeout.
    setTimeoutTimer(beginTimeout);
    try {
	waitForPositiveEdge();
    } catch (TimeoutException) {
	throw new BeginTimeoutException("No signal received");
    }
    unsetTimeoutTimer();
    time_t gobble = 0;
    boolean virgin = true;

    // This loop normally terminates when a silence of endTimeout has been detected.
    // If memory fills, an exception is thrown.
    while (true) {
	// Check that it is not too long, if so, take action
	if (dataIndex >= BUFFERSIZE - 2) {
	    cleanUp();
	    throw new TooLongException("Signal too long for memory, truncating");
	}

	// handle positive edge
	// the first edge should not be stored, as it just marks the start and carries no information
	if (!virgin) {
	    if (timeSinceLastEdge() > aggregateTheshold) {
		// should store...
		// first aggregated pulse...
		data[dataIndex++] = gobble;
		gobble = 0;
		// ... then current gap
		data[dataIndex++] = timeSinceLastEdge(), 
	    } else {
		// short value, just aggregate it
		gobble += timeSinceLastEdge();
	    }
	} else {
	    virgin = false;
	}
	 
	// handle negative edge
	waitForNegativeEdge(); // No timeout, pulses will always end :-)
	gobble += timeSinceLastEdge();

	setTimeoutTimer(endTimeout);
	try {
	    waitForPositiveEdge();
	} catch (TimeoutException) {
	    // this is the successful way to exit the loop, when a "long" silence has been detected.
	    break;
	}
	unsetTimeoutTimer();	
    }
}

It has the definite disadvantage that the user is required to enter an a priori aggregateThreshold, but I currently see no way around it. Whatdoyousay?

To the error log in my previous message: Here I am repeatedly "shooting" the IR signal RC5,D=5,F=0 to the thing. The first time it succeeds (temporarily ignoring the truncation), the second time it fails (probably messing up pulses and gaps), the third time it works (after three captures the log ends)., the forth if fails, etc. (So my statement on "even invocations" of course means when starting counting by 0...)

PS. My Mega2560 just arrived.

[MikeT]

It has the definite disadvantage that the user is required to enter an a priori aggregateThreshold, but I currently see no way around it. Whatdoyousay?

The aggregateThreshold directly depends on the carrier frequency, e.g.

Code: Select all

aggregateThreshold = (1 / F_carrier) * 1.3

Some 30 percent extra to compensate for inaccurate signals.

When the carrier frequency is between 20 kHz and 60 kHz and a pause is normally longer than one period, it would be ok to use a constant time of

Code: Select all

aggregateThreshold = 1 / 20000 * 1.3 = 65µs

This value could be used as start value and adapted during the capture. The next capture would use the already adapted carrier frequency / aggregateThreshold.

To the error log in my previous message: Here I am repeatedly "shooting" the IR signal RC5,D=5,F=0 to the thing.

I've found and hopefully fixed the problem: ICES was not explicitly cleared in TCCR when using a non-inverting diode.

PS. My Mega2560 just arrived.

You can find the new version of the source code here
It additionally supports the ATmega2560 as well as the new mode which only uses the positive edge which doubles the number of pulses which can be captured (switched by the #define CAPTURE_RISING_EDGE_ONLY).

The ICT output format has changed a bit:

• * the pause values got a second number which is the duration divided by the period
  * I had to move "sample_count" below the output of the values because it will not be known before the last value has been captured

/edit: in the meanwhile I implemented the aggregation during the capture (#define USE_AGG_CAPTURE 1). At the moment it doesn't take the duty cycle or pulsewidth into account so the values are a bit different to the other capture strategies (each aggregate value by the unknown pulse width). The output only contains the length of a aggregated pulse and not the number of pulses of an aggregated value, but this could probably be added after restructuring the code and is normally not needed.

Which of the three capture modes is useful? Should all three be supported and should there be a command to switch between the modes?

Michael

[MikeT]

You can find the new version of the source code here

I've uploaded a new version under the same URL which supports using the timer prescaler which means the maximum pulse/gap which can be captured has increased from 64 ms to 1024 ms. This makes it possible to capture also macros or repetitions of a signal.

The precision of the output is the same as before because the version without prescaler used 62,5ns as internal capture unit and the one with prescaler (factor 8) uses 500ns. Both are below the printed resolution of 1µs.

Michael

[MikeT]

You can find the new version of the source code here

Fixed an accuracy bug when converting from CPU cycles to ns.

Michael

[MikeT]

You can find the new version of the source code here

Fixed next bug: the last pulse length was not printed out.
/edit: and large gap values were missing the upper bits, so pauses between signals were printed too small

Michael

[Barf]

Thanx a lot for the latest versions. It looks really promising. You fixed two problems (missing last pulse, chopping signals too much (e.g. a NEC1 signal with repeats was chopped, every repeat turned into a separate sequence)) before I had the time to report it.

It appears as (most?) captures end with a pulse, not with a gap. Possibly this is somewhat of a philosophical question: LIRC always end with a pulse, in the JP1/IRP world, the signals always end with a gap of well-defined length. Of course, when capturing, there is no way to reproduce this well-defined length, unless the signal is immediately followed by another signal, which we do not want to assume, and introduces the new problem of splitting the two. The solution I prefer and I have implemented in IrScrutinizer is to have the user specify an "endTimeout" , and consider the capture ended when a silence of this length has been detected (cf. the pseudo code in my last message). Any chance?

I do not think you have to discard the duty cycle computation when not storing the negative edges: Just add up all the "small" gaps (one variable!) and compare it to the sum of all the pulses.

I see the "aggregateThreshold" as an a-priori parameter the user supplies, as a price for not having to store all edges. For practical uses, say up to 60kHz, the value is probably rather uncritical, so that a default values should serve well in almost all cases. There are alternatives, either a learning phase (I think LIRC does something like this), or some adaptation, but it has its disadvantages.

Which of the three capture modes is useful? Should all three be supported and should there be a command to switch between the modes?

From a practical (users) point of view, it should "just work", and produce frequency, "aggregated timing", optionally duty cycle. I also thing it is reasonable to require a "production" solution to run on a 2k SRAM Arduino (like Uno, Leonardo, nano,...). Which is to say the "aggregating" mode. When that does not suffice, I do not think it is reasonable to search a universal solution.

The ICT output format has changed a bit:

My suggestion: remove it. IMHO, software for measuring devices need only produce data in one format; it is the task of the host computer to convert it.

// TODO: automatically determine if the sensor is inverting:
// wait for a long period without level changes (e.g. 1ms) and use the value as the low value

(comment in code) Good idea. You might want to peek into lirc_serial.c, init_port().

Code: Select all

#include <arduino.h>

It is "Arduino.h" (captalized). Relying to the file system to forgive is non-portable.

Code: Select all

#if USE_AGG_CAPTURE
#undef USE_PULSE_AGGREGATION
#define USE_PULSE_AGGREGATION 0
#undef CAPTURE_RISING_EDGE_ONLY
#define CAPTURE_RISING_EDGE_ONLY 1
#endif

I consider it a bad idea just to quietly "fix" contradictory input data without telling the user. Better just to barf, and have the user fix it. Suggestion:

Code: Select all

#if USE_AGG_CAPTURE & USE_PULSE_AGGREGATION
#error these options are exclusive
#endif

End of setup():

Code: Select all

  for(uint16_t i = 0; i < captureCount; i++)
    captureData[i] = 0;

Problem 1: captureCount is here uninitialized (you mean bufSize?). Problem 2: If initialization is needed, it is needed for every capture.

The sensor I am presently using QSE159 has open collector, i.e. requiring a pull-up resistor. Do you know if there are internal pull-up resistors, per SW controllable, on some pins, like in the Raspberry Pi? Would that be feasible?

[MikeT]

You fixed two problems before I had the time to report it.

I'm sure there are further bugs and missing optimizations
One thing I have to check is the time needed for a loop which means what is the highest modulation frequency which can be handled.
What is the highest frequency that is needed? Is 56kHz enough?

It appears as (most?) captures end with a pulse, not with a gap.
The solution I prefer and I have implemented in IrScrutinizer is to have the user specify an "endTimeout" , and consider the capture ended when a silence of this length has been detected

This is already the way it works, but currently the timeout is fixed:

Code: Select all

if(ovlCnt >= (_BV(RANGE_EXTENSION_BITS) - 1))

This is the maximum value that can be stored as capture value.
Actually this check is no longer exactly right, because the real check must be if the 32 bit aggVal can be converted to a 16 bit value using the range extension which can now also happen for a aggregated pulse. In real world signals this should never happen, the correct comparison would cost me lots of additional cpu cycles...

I don't understand why you want to make this time shorter. When there are additional pulses, they are captured. When there is nothing it will take some hundred milliseconds longer until the capture data is dumped.
Why make it configurable?

I do not think you have to discard the duty cycle computation when not storing the negative edges: Just add up all the "small" gaps (one variable!) and compare it to the sum of all the pulses.

The problem is the cpu time. I think I don't have enough time to handle the negative edge at all. For the aggregation I had to switch from 16 bit to 32 bit arithmetic in the main capture loop which is very expensive so I think it won't be possible to calculate the duty cycle.

I see the "aggregateThreshold" as an a-priori parameter the user supplies, as a price for not having to store all edges. For practical uses, say up to 60kHz, the value is probably rather uncritical, so that a default values should serve well in almost all cases. There are alternatives, either a learning phase (I think LIRC does something like this), or some adaptation, but it has its disadvantages.

It is already adaptive. At the moment I start with a rather long value: 2 periods at 20 kHz which would mean 6 periods at 60 kHz. After 4, 8, 16, 32, 64 and 256 periods I adjust the aggCount and period value and store it also for the frequency calculation. This is very costly therefore currently I only do it during the first pulse. This is ok when the first pulse is a preamble with a length of at least 64 periods but it might not be ok for protocols without a preamble. Again, time is very critical so even the comparison of aggCount with multiple values might take too long so I have to use less comparisons.

From a practical (users) point of view, it should "just work", and produce frequency, "aggregated timing", optionally duty cycle.

Currently the code is not really readable because of the many #if and #define. I will remove the CAPTURE_RISING_EDGE_ONLY define and the code which depends on it which should improve the readability somewhat.

I consider it a bad idea just to quietly "fix" contradictory input data without telling the user. Better just to barf, and have the user fix it

The whole #defines have to be reworked, I hope I can remove most of them.

End of setup():

Code: Select all

  for(uint16_t i = 0; i < captureCount; i++)
    captureData[i] = 0;

The reason for this loop is not obvious. Its not to initialize the data to a known value, but to overwrite the array memory with a value which would crash the cpu when the array collides with the stack and to do this at a early stage. Otherwise you would only get a crash when capturing very long signals. This is useful when identifying how much RAM can be used for the array.

The sensor I am presently using QSE159 has open collector, i.e. requiring a pull-up resistor. Do you know if there are internal pull-up resistors, per SW controllable

Sure, no problem. The following line in setupCapture (after the cbi) should do the job:

Code: Select all

sbi(CAT2(PORT, CAP_PORT), CAP_PIN); // enable the internal 10k pull-up resistor

Michael

[Barf]

What is the highest frequency that is needed? Is 56kHz enough?

I would be very happy if someone like the Rob, Kevin, or Dave (3FG) could give their opinion here. In meantime: IrpProtocols.ini lists protocol up to 58kHz. As far as I am aware of, the sensors mentioned in this thread hardly go much higher either. Above that, there is the 455 kHz B&O stuff, that, at least in this forum, hardly is considered. Possibly why we do not master it?

I think we fare pretty well if limit our selves up to 58kHz, at least for now.

I don't understand why you want to make this time shorter. When there are additional pulses, they are captured. When there is nothing it will take some hundred milliseconds longer until the capture data is dumped.
Why make it configurable?

I have in IrScrutinizer three user parameters (Options -> Timeouts in the GUI), beginTimeout (how long to wait before the signal starts), max capture length (max length to capture), and endTimeout (how long silence to require until the signal is considered as finished). I think this is a very flexible and powerful interface. Using sensible it is not intrusive to the novice user. Not all capturing devices must implement all of these parameters, but that is the goal.

Still, the captures do not end with a gap, and that is posing problems, see my previous post.

The reason for this loop is not obvious. Its not to initialize the data to a known value, but to overwrite the array memory with a value which would crash the cpu when the array collides with the stack and to do this at a early stage. Otherwise you would only get a crash when capturing very long signals. This is useful when identifying how much RAM can be used for the array.

Ok, then please a comment... It still appears to me that you mean bufSize, not captureCount which is not initialized here.

Code: Select all

sbi(CAT2(PORT, CAP_PORT), CAP_PIN); // enable the internal 10k pull-up resistor

Thanx. Have not tried it yet though.

[3FG]

I think 60KHz is high enough. Bang and Olafsen does of course run at 455KHz, and JP1 remotes can send these signals (PID 00EB, TV 0565, 0620, and Audio 0799). JP1 remotes can learn the signals in a sense, but detect the frequency as 0. It is possible to buy IR detectors which respond to at least a few MHz, but I don't really see much value in designing a circuit to support such detectors. I have seen no other consumer IR that runs above 60KHz.

As I understand it, detecting a single edge precludes capturing non-modulated IR signals, like Revox, Barco, Archer, PID 0000, and 0004. That's one of the deficiencies of the Widget approach. MiikeT's initial posts were interesting to me because it included detecting both edges,

Another issue with Widgets is the inability to accurately determine frequency for signal with short on bursts. As a practical matter, a true 56KHz signal may be decoded as 50KHz. I imagine the approach taken here will provide good frequency accuracy.

On a different tack, what do you guys think about the possibility to use a Leonardo or similar board with full speed USB capability (12Mbits/sec) to implement a Widget approach but using, say, 5uSec counting intervals? For single edges, that would be 1.6Mbits/sec into a computer. That way, the SRAM is only needed for buffering the USB communication.

[MikeT]

I think 60KHz is high enough.

This means I have 266 cycles for my code in the main loop
I don't know where I'm now because I don't find my logic analyzer for the measurement.

Bang and Olafsen does of course run at 455KHz

The complex aggregation, value compression and frequency calculation cannot be done in 35 mcu cycles, that's too fast for this kind of capture.

As I understand it, detecting a single edge precludes capturing non-modulated IR signals, like Revox, Barco, Archer, PID 0000, and 0004. That's one of the deficiencies of the Widget approach. MiikeT's initial posts were interesting to me because it included detecting both edges,

How long are these signals, that means how many hi=>lo / lo => edges do these contain? Are 780 edges enough?

Another issue with Widgets is the inability to accurately determine frequency for signal with short on bursts. As a practical matter, a true 56KHz signal may be decoded as 50KHz. I imagine the approach taken here will provide good frequency accuracy.

I reduced the accuracy somewhat by enabling the prescaler, so each single measurement has an error of 1µs. For 40 kHz this gives an accuracy of around 4% for a single pulse. When adding up multiple pulse-width, it should be better than 1%.
Currently I do the frequency measurement only for the first burst.

On a different tack, what do you guys think about the possibility to use a Leonardo or similar board with full speed USB capability (12Mbits/sec) to implement a Widget approach but using, say, 5uSec counting intervals? For single edges, that would be 1.6Mbits/sec into a computer. That way, the SRAM is only needed for buffering the USB communication.

This might be possible. The Saleae logic analyzer works in a similar way. Let's first finish a gadget which supports the two modes (raw and aggregated).

Michael

[3FG]

The non-modulated IR protocols mostly have such short on durations that modulation is not needed or in some cases not even practical. Probably the most challenging for a Widget is Barco:

Code: Select all

{0k,10}<1,-5|1,-15>(1,-25, D:5,F:6, 1,-25,1,120m)+

So this is 10uSec of on time, followed by either 50 or 150uSec of off. If one knows in advance that the signal is Barco, then the Widget output is sufficient to recover the bits. But an algorithm which purports to measure on and off durations wiht no a priori infomation can't handle this. Of course, we don't expect the Widget to handle signals with information content well past the Nyquist limit.

There are other non-modulated UR protocols with 500uSec on durations, but I don't recall the names or detailed characteristics. I doubt any equipment designed in the last decade or so would use this approach.

[MikeT]

Probably the most challenging for a Widget is Barco:
10uSec of on time

The original Arduino Sketch version without any pulse aggregation is able to capture any edges with at least 3µs distance. Also the number of edges (up to around 780) should not be a problem for this protocol.

Is Barco PID 0x002A?

Michael

[3FG]

Yes, Barco is PID 002A.