Rules for standard UART format - Page 1
Forum section: electronics / general

author: Kittu20
author_rank: Regular Contributor
timestamp: December 29, 2022, 10:15:06 am
content: What are the rules for standard UART ?Suppose we have two different size of microcontroller's one is 32 bits long and other is 8 bit long. We want to transfer one byte from 32 MCU to 8 bit MCU using UART over RS232 interface.UART is configured for 9600,N,8,1Can 32 bit MCU tranfer 1byte to 8 bit MCU using UART?

author: Siwastaja
author_rank: Super Contributor
timestamp: December 29, 2022, 10:23:51 am
content: There are absolutely no standards or rules.Both sides are yours? You decice.Communicating with existing system? Read the manual.1 byte is of course obvious: there are no different options with byte orders, headers or whatever. Just send the byte.

author: SiliconWizard
author_rank: Super Contributor
timestamp: December 29, 2022, 07:22:07 pm
content: The most common standard for "UART" communication is RS-232, in which a number of parameters are specified.But UART is a generic term, and UART is not RS-232. Unless you absolutely have to be compliant with RS-232, you can do whatever you want. You won't get a fine if you don't comply with RS-232. That's a guarantee!Obviously if you have control over the two sides, you can use whatever speed, voltage level and parity you want, and there is absolutely no problem with UART at a few Mbps these days as long as you have a short cable/PCB trace or use differential drivers such as RS485/CAN/etc (in which case up to 10 Mbps over 1 km is possible. And it's still 'UART' communication.)

author: Siwastaja
author_rank: Super Contributor
timestamp: December 30, 2022, 07:05:25 am
content: Well, reading the opening post carefully again, it contains all the relevant information, basically a full specification of the system.The answer to OP's question is simply: yes, of course, why not?

author: nctnico
author_rank: Super Contributor
timestamp: December 30, 2022, 07:12:41 am
content: Quote from: Kittu20 on December 29, 2022, 10:15:06 amUARTs work on a per character basis. The easiest ways to talk between systems is using a text based protocol. For example: HTTP that is used for websites is text based. Text doesn't depend on byte order or anything system specific.

author: Siwastaja
author_rank: Super Contributor
timestamp: December 30, 2022, 10:33:50 am
content: Quote from: nctnico on December 30, 2022, 07:12:41 amWell, OP did specify "one byte".Generally speaking, interfaces often provide hardware delimitation, i.e., packetization or framing (however you call it), for certain maximum message length. If the payload happens to fit within this limitation, implementation will be extremely trivial; user will get complete "packets" out of any (sanely designed) peripheral.For UART, this limitation is typically 8 bits, which is so little that usually one needs to add another layer of framing, in software. But not always; maybe 8-bit messages are all the OP needs.If the OP needs 8 bits of information, then not using text, but instead sending a single byte is far easier, as one does not need to think about framing at all. Sending 8 bits over text would require a protocol specification (of framing) plus implementation and verification of this protocol. It can be as simple as converting back and forth using itoa()/atoi() and using semicolon (';') to delimit messages, but it's still some work.However, if the actual payload is limited to approx 6 bits of information, then this single byte can be "text", i.e., limited to printable ASCII characters, which indeed makes machine-human interaction easier. I use such simple protocols a lot, especially in throw-away tests/debugging, e.g. a simple keypress from '0' to '9' sets a power level. No need to write any parser. But of course, you are limited to realistically maybe 4-5 bits of information, picking convenient characters.

author: Simon
author_rank: Unknown
timestamp: January 04, 2023, 10:11:39 am
content: You may find that the buffer size on your 32 bit micro controller is still 8 or 9 bits, I am working with SAMC and the most bits you can do is 9 in one send. I kind of wish 32 were possible but 8/9 bits is the standard we picked.

author: Psi
author_rank: Super Contributor
timestamp: January 04, 2023, 11:11:47 am
content: I usually just create a fixed length framed packet with a 3 char start code. Like STR. (different 3 byte start codes can signify different packet lengths with different data)Then I have some data in groups of 2 separated by an underscore. (Usually the data is raw binary data, not ascii encoded decimals/hex or anything. but being in groups of 2 it's easy to encode as hex if that makes more sense)And then I end with 1 xor checksum byte of the data .So like STR_34_1A_FWhere STR says start of frame, 341A is 4 bytes of data and F is xor checksum of data.The underscore before every 2 bytes is simply a dirty hack to ensure STR can never happen in the data.Is it a good packet structure, noIs it efficient, noShould you copy it, probably not.But it's simple and easy to work with as bytes arrive in the RX interrupt. You can detect packet start, copy it into array and set flag if checksum correct so action can be taken in main() based on the data.

author: Siwastaja
author_rank: Super Contributor
timestamp: January 04, 2023, 12:29:54 pm
content: Quote from: Psi on January 04, 2023, 11:11:47 amIt's very typical, and there is nothing really wrong with it. It is easy to understand and implement, and well scalable because unsupported messages can be ignored (thanks to length field - I assume length isactuallypart of your start header - if not, make it so!). It is also very efficient because lack of features like bit stuffing, escaping etc. allows zero-copy and/or DMA implementations - payload data is usable as is after checksum verification.Can you make something more robust against large amounts of corruption or data loss? Something that can be guaranteed to correctly synchronized in any weird data pattern? Of course. But are UART links normally that unreliable, or does unrecoverable wrong synchronization happen in real world with the scheme you describe? I'd say no. It's fine; I do that all the time and never had to replace it with something else due to any kind of problems.

author: Doctorandus_P
author_rank: Super Contributor
timestamp: January 04, 2023, 02:05:50 pm
content: Quote from: Kittu20 on December 29, 2022, 10:15:06 amIt's standard to set the parameters in the order they are sent, so that would be: 9600,8,N,1 And that means 9600 baud 8 Databits, No parity, 1 Stop bit. and that is the same, regardless of whether you are using a tiny 4-bit microcontrolloller, or an 64bit monster PC. RS232 uses inverted signal levels at +/- 10V (or thereabouts) while the "UART" uses signal levels which are the same as whatever process is used to etch the silicon.Timing is quite important. You can have around 2% to 3.5% timing error. The arduino boards tend to use a 16MHz crystal that is not compatible with standard uart baudrates, and combined with the limited baudrate generator of most AVR controllers. As a result most of the timing budget is already used up because of this mismatch. If you're unlucky and the "other side" is also off by a percent or so you get bit errors.

author: Kalvin
author_rank: Super Contributor
timestamp: January 05, 2023, 10:08:04 am
content: Modern UART peripherals have typically an option for using 9-bit data for easier framing and/or automatic address recognition. The 9th bit will be set at at the start of a new frame, otherwise the 9th bit will remain cleared.If one cannot or want to use 9th bit frames, there are two tricks for creating simple framing:- Modern UART peripherals are able to generate and detect a break condition.- It is possible to introduce a deliberate parity error as a framing marker.If those options are not feasible, one can use a simple a SLIP frame format:https://en.wikipedia.org/wiki/Serial_Line_Internet_ProtocolThere are more efficient, but more complex implementations for creating this kind of soft framings.Another simple solution for transmitting arbitrary binary data is to transmit the data bytes using ASCII hex bytes, ie. each 8-bit data byte will be transmitted as two bytes. Framing is very simple as one can use ASCII characters for frame markers. This will reduce the bandwidth to half.

author: Psi
author_rank: Super Contributor
timestamp: January 05, 2023, 10:14:05 am
content: Yes, the 9th bit is very useful if you have the option for it.

author: Psi
author_rank: Super Contributor
timestamp: January 05, 2023, 10:16:03 am
content: Quote from: Siwastaja on January 04, 2023, 12:29:54 pmI like having the 3 char header to identify a very wide range of packet structures , lengths and contents.I mean it kind does signify the length, just not defined as a number.

author: voltsandjolts
author_rank: Unknown
timestamp: January 05, 2023, 10:43:48 am
content: Quote from: Kalvin on January 05, 2023, 10:08:04 amPerhaps you're alluding toCOBS, which is a little trickier than SLIP and needs more RAM, but it can be much more efficient with its low overhead.But I think the OP is really looking for simplicity, if they are even here anymore.

author: Simon
author_rank: Unknown
timestamp: January 05, 2023, 10:56:37 am
content: The 9th bit is generally optional, I've not come across a micro that forces you to use it. On an 8 bit system being forced to do 9 bits would be silly as it doubles the buffer writes for an 8 bit bus (OK maybe they are 16 bit busses) but all these things harp back to 40+ years ago so will have all sorts of options that are for backward compatibility or have just always been there.

author: voltsandjolts
author_rank: Unknown
timestamp: January 05, 2023, 11:00:29 am
content: Quote from: Simon on January 05, 2023, 10:56:37 amNah, the 9th bit isn't a data bit.It's just a flag which normally means "this is an address/command byte" and is used on multi-node master driven uart busses, likethis devicefor example.

author: Simon
author_rank: Unknown
timestamp: January 05, 2023, 11:14:54 am
content: Yes so back in the day, 8 bits was standard, the 9th bit was added later but you can always choose. Given how low level the hardware is you get to do what you need to do for any situation, but there will always be the option to choose 8 or 9 bits.

author: Siwastaja
author_rank: Super Contributor
timestamp: January 05, 2023, 11:26:10 am
content: Quote from: Psi on January 05, 2023, 10:16:03 amI understand. The benefit ofexplicitlygiving a number is, you can extend the protocol and make the old nodes ignore messages they do not recognize (unknown message id), by skippinglenbytes. If you need to look up the length based on the message id, then you need to know all the messages (their lengths) in advance.Without knowing the length, you always go out of sync, and getting back in sync would mean losing some messages.

author: Siwastaja
author_rank: Super Contributor
timestamp: January 05, 2023, 11:31:31 am
content: Quote from: voltsandjolts on January 05, 2023, 11:00:29 amThe 9th bit would be excellent way to introduce robust packet framing; for example, just set 9th bit as '1' at the first header byte of message. Or toggle the bit between messages.Sadly, 9-bit UART is a rarity! Not all MCUs, UARTs on PCs or USB-UART adapters have support for it.I remember doing a protocol based on the same idea, using the 8th bit for framing, and limiting to 7-bit data. But that 7-bit data was total PITA so I quickly got rid of it.

author: voltsandjolts
author_rank: Unknown
timestamp: January 05, 2023, 11:39:48 am
content: 8051 and a lot of PICs (all?) have the 9th bit option, and option to only get an interrupt if the 9th bit is set. That's ideal for multi-node bus, slaves don't waste time processing every packet/byte to see if its addressed to them. That scheme is used in the PIC firmware example I linked above. But I hate using 8051 and PICs now, only when forced to

author: Psi
author_rank: Super Contributor
timestamp: January 05, 2023, 01:04:45 pm
content: Quote from: Siwastaja on January 05, 2023, 11:26:10 amShould never go out of sync, any occurrence of a known 3byte start block zeros the array index and begins packet read.Is the advantage of having that underscore in the packet. The 3byte start block can only ever exist at the start

author: AndyC_772
author_rank: Super Contributor
timestamp: January 05, 2023, 01:12:48 pm
content: One 'gotcha' to note - some UART peripherals (or at least, their documentation) consider the 9th bit to be the parity bit. If you configure them to 8 bit mode with parity, you only actually get 7 data bits, because the parity bitisthe 8th bit.If you want 8 data bits, plus parity, then you have to use "9 bit" mode, and if you want 9 data bits plus parity, then you're out of luck.

author: Psi
author_rank: Super Contributor
timestamp: January 05, 2023, 01:23:47 pm
content: Pretty sure AVR's are fine with 9bit and parity.Infact i think AVRs have a multi processor mode for linking many MCUs on one UART bus and filter out packets in hardware if they are not addressed to the MCU in question. And i think that feature uses the 9th bit.

author: Doctorandus_P
author_rank: Super Contributor
timestamp: January 05, 2023, 02:34:58 pm
content: I've designed my own "home automation" protocol built around the 9th bit for recognizing the first (address) byte of a packet.Both AVR and STM32 microcontrollers have built in hardware especially designed for this.For example you can have it just ignore all traffic until it received a byte with that 9th bit active, and this reduces overhead significantly on an RS485 network with a lot of traffic for different nodes.After an uC recognizes start of a packet, it can change it's state, decode (part of) the header and then decide what to do with the rest.A disadvantage is that it reduces the available bandwidth by about 10% which is quite significant. More modern protocols such as CAN and USB do not use start and stop bits as the old UART does, but use bit stuffing instead. But both protocols have limited application. I decided not to use CAN because of it's very short 8 byte payload and weird adress space.The old Modbus was quite well thought out for it's time. Packets must be sent as a continuous stream, and a silence on the serial stream indicates the end of a packet (and a re-sync event for the next packet). This is more efficient than using the 9th bit for packets longer then about 20 bytes. (assume 20 bits worth of idle time) STM32 has hardware support for this too. You can tell the uart peripheral to ignore data until after this "idle frame" is detected.Using a pause between packets like modbus does, also makes it compatible with any uart on the market, because you only need 8N1 for the datastream.

author: PCB.Wiz
author_rank: Super Contributor
timestamp: January 05, 2023, 11:24:49 pm
content: Quote from: Kittu20 on December 29, 2022, 10:15:06 amYes, it would be very rare to find any MCU UART that cannot send 8 bits.They usually have more options.Quote from: Kittu20 on December 29, 2022, 10:15:06 amThe common 'rule' is a leading start bit and a trailing stop bit.That's a historic decision that unfortunately makes auto-baud more difficult than it could have been, but way, way back when that was done, MHz clocked MCU's were never dreamed of - 100+ years ago, things were using electromechanical technologyhttps://en.wikipedia.org/wiki/Asynchronous_serial_communicationQuote from: Kittu20 on December 29, 2022, 10:15:06 amA single byte is easy, simply read the manuals.Multiple bytes get more complex, if you want to recover from a break or disturbance. You need to know where in the packet you are.A very simple coding scheme for small packets is to nibble encode, where some portion of each 8 bits is used to identify that data's location, or a start of frame.a 4/4 bit split, allows 16 nibbles to be sent, each one easily identified, so 8 bytes can be managed over 16 characters sent. Unpack/repack code is very simple.If you need > 8 byte payloads, one nibble value can mean 'repeat and increment', and code is slightly more complex.It's a little wasteful in bandwidth, but you can use the predictable nibble as noise immunity improvement.Some hardware supports 9th bit and BREAK send/detect, but that is less universal.

author: Siwastaja
author_rank: Super Contributor
timestamp: January 06, 2023, 07:57:54 am
content: Quote from: AndyC_772 on January 05, 2023, 01:12:48 pmThen it isnota "9th bit", it's called parity bit. If some datasheet calls parity-capable UART with max 8-bit data "9-bit", then they are bullshitting. Never seen that, though.8O1/8E1 and 9N1 are conceptually different even if they have same number of symbols on the line.

author: AndyC_772
author_rank: Super Contributor
timestamp: January 06, 2023, 10:43:56 am
content: https://www.st.com/resource/en/reference_manual/rm0090-stm32f405415-stm32f407417-stm32f427437-and-stm32f429439-advanced-armbased-32bit-mcus-stmicroelectronics.pdfPage 969, section 30.3.1. The last bit transmitted on the wire is labelled a "possible parity bit".

author: Siwastaja
author_rank: Super Contributor
timestamp: January 06, 2023, 02:50:29 pm
content: Quote from: AndyC_772 on January 06, 2023, 10:43:56 amYes, but that part does support 9-bit data (and they do not call parity bit "data") so what's the problem?It does not support 9 bit data plus parity, though, but that would be quite rare anyway.

author: pardo-bsso
author_rank: Frequent Contributor
timestamp: January 10, 2023, 10:41:02 am
content: Quote from: Siwastaja on January 05, 2023, 11:26:10 amWe switched to using Google's Protobuf.Besides having a single spec for all the messages it takes care of generating c/c++ and python wrappers to do all the parsing.This may be overkill for the OP question but has saved us from a lot of time otherwise spent debugging and coordinating among teams.

author: Doctorandus_P
author_rank: Super Contributor
timestamp: January 10, 2023, 10:56:18 am
content: Quote from: Siwastaja on January 06, 2023, 07:57:54 amIt's a bit of a "meh". You may be right "in theory", but it is the practice that matters.From what I remember of stm32, they do support 9 bit, or parity, but not at the same time.When parity is "odd" or "even", it has a clear function (does anyone even use parity?) Sometimes a parity of "mark" or "space" is used, and then the distinction becomes minimal.And when it comes to microcontrollers. when interfacing with the outside world, only the signals on the pins matter. It does not matter whether the datasheet claims whether it is a "9th bit" or a "parity bit". You are free to abuse uC features to generate the signals you need for your application. The biggest problem here is that if you rely more on obscure features of your microcontroller, your code becomes less portable.

author: MIS42N
author_rank: Frequent Contributor
timestamp: January 10, 2023, 12:31:00 pm
content: Quote from: Kittu20 on December 29, 2022, 10:15:06 amA 'standard' UART will signal at a particular voltage, and 9600 is the rate per second at which bits are transmitted. The A in UART is Asynchronous - a transmission of a byte (configuring for 8 data bits) starts at an arbitrary time after the completion of the transmission of the previous byte. UART output is usually high when idle, and bits are 1 = high, 0 = low voltage. In the case of sending bytes, every byte is sent with a start bit which is a 0, and a stop bit which is 1. Each bit is signalled for 1/9600th of a second (about 104us) except the stop bit which is a minimum of 104us but morphs into the idle state if there is a delay before the next byte. Bits are sent most significant first. So it requires 10 bit times to send 8 data bits.For UART to UART communication, the only compatibility issue is the voltages. Connecting the UART of a 5V device to the UART of a 3.3V device is asking for trouble (it does work sometimes).The OP specified RS232. To get RS232 signals from a UART it is necessary to use a TTL to RS232 converter with both UARTs. RS232 signals are -ve for idle or 1 and +ve for a 0 (opposite polarity to UART) and a value between -3 and +3V should be ignored (although many receivers will take 0V as idle or 1). The voltages for RS232 are defined by the standard so any RS232 transmitter should be compatible with any RS232 receiver.As others have pointed out, it is unnecessary to use RS232 signalling unless required by some device. UART to UART just requires a level shifter if the two devices run at different voltages.So yes, just about any UART can send a byte to another UART.

author: Siwastaja
author_rank: Super Contributor
timestamp: January 10, 2023, 02:34:31 pm
content: Quote from: Doctorandus_P on January 10, 2023, 10:56:18 amI also covered practice by saying, never seen that in real life. But by all means, if you know a MCU or standalone UART which explicitly calls "9-bit" in documentation yet is unable to send or receive 9-bit data, let us know.STM32 surely can, but as usual, both 9-bit data and parity features are not available simultaneously. This is normal; anyone who has been bitten a few times knows that any combination of two or more front page features might not be available at the same time, and will recheck if you can have both features X and Y simultaneously.

author: Nominal Animal
author_rank: Super Contributor
timestamp: January 11, 2023, 03:34:08 pm
content: I must admit I am surprised to see it so vehemently disagreed as to what is "standard" for an UART transceiver. I guess I have been pampered with the support I've had with the microcontrollers I've used!NXP i.MX RT1062 (as used on Teensy 4.x) supports 7 to 10 bit data (the transmitter and receiver have 9 to 13 bit shift registers), and internally supports both idle and break "characters" (break length being 9 to 15 bit times, depending on the relevant settings), and even hardware flow control (RTS/CTS).Even Atmel (now Microchip) ATmega32U4, an 8-bit AVR with native USB support, supports 5 to 9 bit data, all with no or even or odd parity, and with one or two stop bits (the datasheet explicitly mentions all 30 combinations are supported). The other 32-bit NXP chips I often use (specifically on Teensy LC and 3.x) have similar support.Quote from: Kalvin on January 05, 2023, 10:08:04 amAs mentioned,COBS or Consistent Overhead Byte Stuffingis one.One could say that using the BREAK condition –– which is basically the same size as a normal serial data frame but one that is separate from all valid data frames –– would be even simpler, and just as efficient as COBS for framing. Unfortunately, not all UART/USART implementations support sending or receiving BREAK conditions/characters in a sensible manner.I personally prefer a variant of SLIP with 1/Nmaximum overhead (depending on implementation complexity) and minimal buffering (four bytes for 33% maximum overhead). The major difference to SLIP and COBS is that my variant allows multiple frame byte values that will not occur in the data. (There are also multiple byte values that always begin an escape sequence.)I developed the variant for an use case where I wanted to multiplex multiple logical byte streams onto a single byte stream with minimal latency (thus minimal buffering and very short uninterruptible sequences) and minimal work for both multiplexing and demultiplexing.Using an extra bit to differentiatestructureanddatabytes is an interesting approach. It means data can be sent as-is, without any need for escaping. However, the extra bit does mean something like 8% to 10% constant overhead (because each UART data frame will typically have at least a START bit and one or two STOP bits in addition to the data bits), plus the frame structuring bytes on top. Of course, if one has just two byte streams to multiplex, then the extra bit (or bits!) could be used to indicate the stream the data bits are part of.The two-stream-multiplexing by an extra bit reminds me of the SPI bus variant many OLED and small TFT display module driver chips (ILI9341, ST7735S, ST7789, etc.) use for (serial) communications with a microcontroller: they have a separate Data/Command signal indicating whether the current/next byte is a new command, or data related to the previous command.

author: Doctorandus_P
author_rank: Super Contributor
timestamp: January 12, 2023, 01:13:18 am
content: Quote from: Siwastaja on January 10, 2023, 02:34:31 pmThis reminds me of programming in assembly (which was lukicly long ago.Some uC's use a "trick" for clearing a gegister, by XOR-ing a register with itself.Other uC's had a specially defined opcode, documented an all for clearing a register. So I was curious and looked at the machinecode, and they were the same. But it still was useful, as the opcode to clear a register made a clear statement to the intention of the programmer.