Proper way to deal with any errors in a buffered uart - Page 1
Forum section: electronics / general

author: cv007
author_rank: Frequent Contributor
timestamp: October 20, 2019, 06:47:26 pm
content: What is a proper way to deal with uart errors when using a buffer? I am refining my avr0/1 usart code, and have an optional tx and/or rx buffer along with a printf (via vsnprintf). When a buffer is not specified for tx, it just blocks on a 'normal' putch instead of going through a buffer. When a buffer is not specified for rx it returns a signed 16bit value where it either indicates an error/no data available/valid char.Now when a buffer is specified for tx, there is no problem as the tx isr just pumps out data as long as there is any data in the buffer, and using putch may block on the buffer if its full but no data is lost. When an rx buffer is specified, what is to be done with any errors? Also, what to do when buffer full?some options (I think)-if parity/framing error, just don't add it to buffer (probably useless anyway), inc a error counter if wantedwhen rx buffer full- drop incoming data (losing latest data), inc an overflow counter for infowhen rx buffer full- remove byte from buffer then add new byte (only keeping latest data), inc an overflow counter for infoI doubt in practice errors show up much, so is probably just as well to drop bad data and set flags for information and let something further up the line deal with it if wanted. I am inclined to drop new data when the buffer is full, as the existing data may still have some value, but will probably be of no value if one starts overwriting data that is about to be used by whatever is removing/using it.Better ideas? What is 'normal' for mcu's?my current code in progress-https://github.com/cv007/ATTiny416XplainedNano/blob/master/UsartNew.hpp(line 500 area is get/put code and isr functions)Uart<Usart0_Alt, 64, 0> uart0; //Usart0, alternate pins, 64 bytes tx buffer, no rx bufferuart0.baud( 19200, 3333333 ); //set baud, provide clock (if none provided, gets actual from Clkctrl)uart0.txOn(); //enable tx, pins setup as neededuart0.printf("%06x: Hello %s\r\n", Sigrow::deviceId(), "World");//since tx buffer in effect, tx irq will be in useOne more question-in the putch function, I insert a byte into the buffer, which returns false if the buffer was full-while( not (m_txbuf << v) );so block until a byte can be inserted, but the insert into the buffer is interrupt protected-https://github.com/cv007/ATTiny416XplainedNano/blob/master/Buffer.hppso interrupt lock is being used in a loop-does this matter at all? I'm thinking it doesn't matter as long there is opportunity for an irq to fire somewhere in that loop, so am only delaying any irq by whatever time irq's may be off in that loop. Is that correct or is there something else to consider?Thanks.

author: SiliconWizard
author_rank: Super Contributor
timestamp: October 20, 2019, 09:45:07 pm
content: It's up to you to decide how to handle incoming data in case of errors, but the important thing is not to ignore errors (which could lead to incorrect interpretation of data).One thing that can be a bit tricky, but equally important, is to make your receiving routine robust to errors: it should not only reject errors (and notify other parts of your code), but also be able to fully recover from errors: communication errors should not get your state machines stuck in any way. Bad implementations can lead to just that.

author: cv007
author_rank: Frequent Contributor
timestamp: October 21, 2019, 07:08:55 am
content: In any case, the user of the data has to deal with errors seen or unseen or data missing for any reason (its not the usart's or buffer's problem). In the case of non-buffer use, my getch returns -1 for no data available (so is non-blocking), and -1|error bits for any errors (>=0 is good data). So using the non-buffer version, simply checking for >=0 is all that is needed for the data user and its up to it to decide what to do when it gets a value <0. The hardware in this case (avr0/1) does not need anything done as all the error bits are associated with only the current data byte.Maybe in the buffer use case, I just have the error bits as flags set in the rx isr, then in getch I return the data + the error flags. If there are no errors, getch returns data >=0, if any error bits set the return value now is -1|error flags and the user must clear the error flags, or simply ignore and use the data only (read as a byte instead of 16bits). Any corrupt data that sets the parity/framing errors is unusable, so is not added to buffer but flag is set. Now the buffer version is the same as the non-buffer version, except for the need to clear the error flags (put in the flush function, or in a separate function).Another variation for buffer version described above- set the error flag(s) in the rx isr, the getch returns the flags with the data same as non-buffer version, but also clears the flags in the getch function. Then there is no additional action required from the user- if -1, no data available, if <0 an error occurred 'somewhere' in the buffer (most likely the error is 'later' in time than where the user is currently at), and can decide what to do. The only difference is the buffer version now has an advanced warning of problems (most likely, unless buffer not really being utilized at all). Instead of a 'this byte is bad' for the non-buffer version, the buffer version becomes 'heads up, missing data ahead, you have been warned'.

author: Jeroen3
author_rank: Super Contributor
timestamp: October 21, 2019, 10:00:15 am
content: Quote from: cv007 on October 20, 2019, 06:47:26 pmThat depends what the communication is used between.Party or Framing error can be used to abort early.Parity in UART isn't very usable for m2m (machine to machine), since that is often packet based.Usually it's up to the layer above to deal with the errors. It reads a block of data, checks the error register. And on an error, discard it and request re-transmit. It doesn't need to know which byte was bad. It irrecoverable anyway. It just needs to know that in the block some byte is wrong.Can the layer above detect errors itself? Eg: by checksum?Can the layer above deal with missing data? Eg: shorter, or more often exploited, longer frames?Can the layer above recover from errors? Eg: by ECC/FEC.

author: cv007
author_rank: Frequent Contributor
timestamp: October 21, 2019, 10:31:44 am
content: >Can the layer above detect errors itself?I'm not thinking about the layers above right now, they have to just assume the worst, and its not a problem the uart/buffer has to deal with. Was just wondering what is common/best for handling any errors at the lowest levels.Looking at various 'popular' mcu code- mbed os, zephyr, and so on, it appears there is no standard way of dealing with uart errors. I have a hard time reading some of this kind of code (and to some degree a little amazed it works), but they all do something different. If I read mbed os correctly, and I'm not sure the api agrees with the hardware code I chose to look at, but it looks like they just 'flush' the whole buffer when any error is seen. Zephyr seems to require a specific request to get its error status. Others do various things- some just don't care about errors at all.It seems there is no standard method, so I guess doing whatever makes sense is the way to go. I think for now I will just continue as I have already done- let polling and interrupt/buffer both act mostly the same way, and let the layer above decide when any error is seen- if it wants to flush, it can flush, if it can deal with missing data in its own protocol it can just ignore errors.

author: Jeroen3
author_rank: Super Contributor
timestamp: October 21, 2019, 11:22:56 am
content: Quote from: cv007 on October 21, 2019, 10:31:44 amEither you fix them. Or you pass them on.There are three options: ignore, mark or discard.If you're writing a driver, create these three options and let the application decide.Linux uses flags to inform the application of faults and errors, the application should check these.There is no universal way to deal with errors if you are not able to correct them.

author: westfw
author_rank: Super Contributor
timestamp: October 24, 2019, 04:18:41 am
content: Meh. IMNSHO, I think you should just ignore the per-byte errors you get from most UARTs (Framing error, overrun error, parity error.) They're essentially useless to higher levels of any error-detection protocol, and most of the higher-level protocols will have some additional way of checking errors that probably can't make use of per-byte error info anyway. For example: XMODEM implementations want to receive a block of 128 characters, after which they'll do a checksum or CRC. Remove a character from the stream because it had a parity error, and most implementations will need to time out that block instead of detecting the error early. It's not like you can abort the block in the middle, anyway, if you did detect an error. The whole "asynchronous" nature of UART communications isn't very friendly to doing anything intelligent.Exceptions I've seen/used:"We" had a UART implementation that would interpret the combination of "framing error" plus "char == 0" as a BREAK condition, which was stored out-of-band in the UART structure.Optiboot, the Arduino UART bootloader, normally resets the WDT in the UART receive function. If it detects a framing error, it will NOT reset the WDT. This means that streams of wrong-speed data (presumably aimed at an application, rather than at the bootloader) will cause the bootloader to end "sooner" than it might otherwise. (OTOH, the bootloader UART isn't buffered.)

author: cv007
author_rank: Frequent Contributor
timestamp: October 24, 2019, 11:23:33 am
content: Since with just a little more effort, I also have scanf/fscanf hooked to my uart class (directly, or indirectly via the f versions). Since this particular version of vfscanf can accept a device error from the 'get' function provided to FILE (my getch function), returning an error (from my getch) when an error is seen will at least prevent scanf from hiding any problems.If I'm looking for a number, and scanf returns >= 0, at least you can be reasonably sure the data is not corrupted. If I ignore errors at the uart level, vfscanf will never see the missing data and just fill in what it sees, returning what appears to be a valid value. With errors handled- 123 -> 13 = -1, or ignore errors- 123 -> 13 = 1 (1 value converted, but is wrong). You are still left with the question of what to do, and using a buffer makes it a little more difficult, but at least you have the information and can either act on it or ignore. (I have not used scanf much, but seems to be a good example).I thought maybe this was a settled question, but seems not to be.I'll just continue what I was doing-no buffer-getch() returns -1 if no data, >= 0 if data valid, -1|error bits|data if any errors (still get whatever was in rxdata register)buffer-getch() returns -1 if no data in buffer, >=0 if valid data, -1|error bits if any data dropped (data with framing/parity is not added to buffer)the isr just sets error bits, the getch() will clear them, and there will be no info where the missing data would have beenIf the layer above uart can do whatever it wants. The uart will have done its job as good as it can.