Serial protocol for filling an array from a PC: is there a standard protocol? - Page 1
Forum section: electronics / general

author: cdwijs
author_rank: Regular Contributor
timestamp: September 09, 2014, 12:32:42 pm
content: Hi All,I'm working on a project where I need a serial protocol. The idea is that the uC has a small fifo buffer, and the PC has to keep the buffer filled. The uC program takes one value out of the fifo buffer at a time, about 100 times per second.I have made a small protocol that works like this:1) PC sends a message: start2) uC sends a message: request first datablock3) PC sends first datablock4) uC checks if the buffer is empty enough to request the next datablock5) PC sends the next datablock.6) PC sends the stop command7) uC stops.The uC also sends data back:a) if there's more than one block of data to send, send the data, and wait until there's again more than one block to send.While this protocol works, I wonder if there's a standard protocol for this. I figure if I follow a standard, somebody has thought a lot more than me about corner cases. For instance when serial packets are damaged. Also, it could be that other people have already made PC program that use the protocol. Then My uC would automatically be compatible.Does anybody knows about a serial protocol that does the above?Kind regards,Cedric

author: apelly
author_rank: Unknown
timestamp: September 09, 2014, 12:46:05 pm
content: Sounds like you just invented handshaking...You can choose between hardware, software or both.https://en.wikipedia.org/wiki/Handshakinghttps://en.wikipedia.org/wiki/Serial_port#Flow_controlhttps://stackoverflow.com/questions/7578812/serial-port-handshake-what-the-difference-between-hardware-and-none-handshaking

author: eurofox
author_rank: Unknown
timestamp: September 09, 2014, 01:51:52 pm
content: You forgot something very importantYou should check the integrity of the message by adding a checksum that should be checked on reception with reply of an ACK or NACK and resend, a resend counter in case of problem and error procedures.

author: nctnico
author_rank: Super Contributor
timestamp: September 09, 2014, 01:53:10 pm
content: Look for XOn/Xoff handshaking. That is standard.

author: cdwijs
author_rank: Regular Contributor
timestamp: September 09, 2014, 02:55:04 pm
content: Quote from: eurofox on September 09, 2014, 01:51:52 pmI would like to use CRC-16. As far as I know this has the best chance of catching transmission errors, but I think this costs a lot of processing power. Alternative I am thinking about 2 1-byte checksums, one with a XOR of all the previous bytes, and one with an addition of all the bytes. This is very cheap in terms of processing power, but how good is this in catching transmission errors?Kind regards,Cedric

author: Rufus
author_rank: Super Contributor
timestamp: September 09, 2014, 03:20:28 pm
content: Quote from: eurofox on September 09, 2014, 01:51:52 pmDepends on the application. A short RS232 link in a benign environment won't see a bit error in months. I once implemented a robust master to multi-slave protocol running over RS485. A PC application and USB to RS485 converter was used to test and configure the nodes. I didn't bother implementing any error recovery in the PC application because in days of operation flat out at 115k baud I didn't see one error that needed recovering.In passing and anything using 'NACK' isn't robust. A validated message can be acknowledged, you don't know what anything else is so how or when do you know to not acknowledge it?

author: eurofox
author_rank: Unknown
timestamp: September 09, 2014, 04:06:20 pm
content: Quote from: Rufus on September 09, 2014, 03:20:28 pmI wrote communication protocol since the 80's, by using a serial line you have absolutely no warranty that the data that arrive on the other side is correct, an "industrial grade" system always use an integrity check by adding a byte or 2 bytes checksum.Even in the most secure wiring and shielding, if someone simply disconnect the cableon the other side no problem it will run with the data in the buffer

author: FrankBuss
author_rank: Unknown
timestamp: September 09, 2014, 04:16:45 pm
content: Quote from: cdwijs on September 09, 2014, 02:55:04 pmIt could miss problems like when the same bit is more than once wrong. I would use CRC8. It is faster, too, just a table lookup:http://www.maximintegrated.com/en/app-notes/index.mvp/id/27

author: cdwijs
author_rank: Regular Contributor
timestamp: September 09, 2014, 06:10:53 pm
content: Quote from: FrankBuss on September 09, 2014, 04:16:45 pmThank you for proving XOR-ADD can miss problems. I'll check the CRC8 technique, it looks promising.Kind regards,Cedric

author: westfw
author_rank: Super Contributor
timestamp: September 09, 2014, 10:30:25 pm
content: QuoteThere are LOTS of standard protocols that do this sort of thing. How about LAPB over Async HDLC?You should be able to find lots of existing code, because this is essentially the protocol used by amateur packet radio (which also adds an X.25 layer), and Async HDLC is the link layer for PPP over dialup. You can even find (I think) serial controller chips that include hardware support for AHDLC (DMA, CRC generation and checking, byte stuffing, etc.)

author: T3sl4co1l
author_rank: Super Contributor
timestamp: September 09, 2014, 10:52:17 pm
content: Quote from: Rufus on September 09, 2014, 03:20:28 pmYou feelin' cozy enough with that to run it through EMC susceptibility?Tim

author: Rufus
author_rank: Super Contributor
timestamp: September 09, 2014, 11:37:08 pm
content: Quote from: FrankBuss on September 09, 2014, 04:16:45 pmAnd 8 bit CRCs can miss problems when different bits are wrong. Random garbage will produce a valid 8 bit CRC 1 in 256 times the same as some simpler 8 bit checksum. There is nothing magic or strong about CRCs they are just a bit better at detecting short bursts of errors and have value if that is the kind of error you are most likely to get.Quote from: T3sl4co1l on September 09, 2014, 10:52:17 pmI didn't need to. The protocol is robust. I didn't bother implementing any error recovery in a PC application talking to one node over a short cable used in a benign production test environment. An error would lock up the link and they would have to manually reconnect so it wouldn't be a disaster but it just doesn't happen. It is in production today and has been for more than 5 years. Like I said it depends on the application.

author: ejeffrey
author_rank: Super Contributor
timestamp: September 10, 2014, 05:53:57 am
content: A checksum is important if a garbled message could do damage, or if you have some way to request retransmission of data. What is more important is to make your protocol self synchronizing. If you are transmitting 16 bit words and don't use any framing characters you will have the problem that if you drop a byte or gain an extra (say when initially connecting) you will be forever misinterpreting your data -- until you reset everything. The normal way to do this is to have a reserved beginning/end of message character, such as a newline in most text protocols. When the decoder sees the special character it resets to a known state. The downside is that if you are transmitting binary data you need to escape the end-of-message.

author: Jeroen3
author_rank: Super Contributor
timestamp: September 10, 2014, 06:47:32 am
content: Have a look at the first 32 characters of acsii. They are designed to do this.http://www.ascii-code.com/rts/cts or xon/xoff flow control is designed to not overflow receive buffers. As in "flow" control. I would recommend to enable this because an ATMega wouldn't be fast enough to process a 200 byte array reasonable baud rate* due to the missing hardware fifo.A reasonable baud rate is the one where sending data from the atmega has the right interval of tx ready interrupts.On problem with this... Your data must be ascii instead of binary, which reduces throughput.Solution, hex code your data, so 0xDEADBEEF will be sent as stx"deadbeaf"etx.

author: nctnico
author_rank: Super Contributor
timestamp: September 10, 2014, 03:19:09 pm
content: Another idea: how about the X-modem protocol? With some modification it can be used to transfer a continuous stream of data.