RPi + custom micro-controller board + python - Page 1
Forum section: electronics / general

author: Simon
author_rank: Unknown
timestamp: October 12, 2025, 05:52:20 pm
content: I'm trying to devise hardware and software for a HMI, yes, for real this timeSo given how the overall system may develop I want to use a single board computer that communicates with a custom board with a microcontroller that talks over some sort of simple serial connection UART or SPI seem to be the options. And I'm looking at python as I'm now satisfied that it's the fastest way to get things off the ground.From what I see SPI could be faster than UART. I'm not sure where to start either way. On a microcontroller I would deal with serial comms in terms of strings of bytes and knowing exactly where to find them. With a serial port in python I'm not sure how I identify my start and end of data.

author: radiolistener
author_rank: Super Contributor
timestamp: October 12, 2025, 06:13:08 pm
content: You don’t need to manually detect the start or end of data when using a standard UART interface — this is handled by the hardware. Each byte is framed by a start bit and one or more stop bits, so the receiver automatically reconstructs complete bytes. At the output, you receive a fully formed byte stream, ready for processing.Typically, commands are organized as packets with a standard structure that includes fields such as packet length and a checksum. This way, you simply read the incoming byte stream as packets in FSM: if a packet validates correctly, you process it, if there’s an error or a timeout, you reset the receive buffer and start over. This process repeats continuously in a loop.

author: Simon
author_rank: Unknown
timestamp: October 12, 2025, 06:28:16 pm
content: Yea I'm not worried about the bits no more than I have to worry on a microcontroller. It's when I get data from the port it will be a string of bytes that needs to represent the message from start to end.I see that the serial library has methods to sit and wait for data up to a certain amount of characters so in theory I can get one packet at a time.

author: voltsandjolts
author_rank: Unknown
timestamp: October 12, 2025, 07:16:20 pm
content: You could connect a display directly to the RPi, so why the mcu?

author: Simon
author_rank: Unknown
timestamp: October 12, 2025, 07:27:39 pm
content: Because the display will be connected to the RPi. The MCU acts as an interface, any analogue channels, encoders, buttons and the CAN bus. Importantly the MCU is real time, so I have the best of both worlds.

author: voltsandjolts
author_rank: Unknown
timestamp: October 12, 2025, 07:43:14 pm
content: Ah, yes, got it.If you do go the uart route for comms between the mcu and RPi, you might consider using COBS for framing.https://www.eevblog.com/forum/microcontrollers/implementing-uart-data-packets-with-consistent-overhead-byte-stuffing-(cobs)/

author: abeyer
author_rank: Frequent Contributor
timestamp: October 12, 2025, 08:24:25 pm
content: Quote from: Simon on October 12, 2025, 06:28:16 pmThat works for simple stuff, and may be enough. Another built-in option if you use a plain text(ish) format is to usereadlinewith an appropriate EOL character to read line-wise, which can allow variable length packets without extra work on your part. You'll likely want/need to set and handle timeouts for robustness, too.As you get into more complexity it can be very beneficial to decouple the IO from the logic... At the simplest this could just be a ring buffer where your serial IO code just drops bytes at the tail as it receives them, while the protocol handler logic watches the head of the buffer for complete packets that it can process. A step up in complexity (but also, imho, a very nice decomposition model) is to define your protocol as a "pure" definition of the communications rules in a state machine that defines packet structure and any higher level statefulness. The protocol just consumes bytes as they arrive and produces some sort of "domain events" as the state machine dictates. Then your business logic gets built as a set of "handlers" that can respond to those events as needed.

author: rteodor
author_rank: Frequent Contributor
timestamp: October 12, 2025, 08:38:55 pm
content: HDLC (High Level Data Link Control) framing. Its simple, can be implemented in a few lines of code.Classic async HDLC use 0x7E to mark start and end of a frame or separation between frames. If 0x7E or 0x7D appear in the data, then data is "stuffed" a.k.a prefixed with 0x7D and the byte is sent with byte 5 inverted. That is all.But it does not have to be exactly that: I use accolades for start and end (0x7B and 0x7D) because its easy to spot a frame during debug. For stuff char I use 0x7E and no inverting of bits.

author: Simon
author_rank: Unknown
timestamp: October 15, 2025, 06:05:33 am
content: Most of what I transmit is numbers, leading zero's are going to be very likely so converting them to something else will be a pain. As I transmit numbers there is no combination that is off limits. Literally the only way to avoid that is to use ASCII only, not impossible I suppose but it would more than double the bandwidth.As I am coming from CAN bus in the system (CAN open) maybe I am overthinking this and should just use a CAN bus shield or adapter. The reason I was looking to make my own adapter was that I don't want to tie myself to a particular SBC or CAN bus adapter and be forever sorting out drivers. SBC's all seem to have inbuilt serial ports making it a good universal boundary between things that will be there on any SBC.

author: Simon
author_rank: Unknown
timestamp: October 18, 2025 03:24:47 pm
content: Well I got a CAN bus hat to work fairly easily. I thought it would be harder. It seems that dtoverlay is everyone's friend. I came across many discussions about the hardware pin drivers that seem to have to be updated with every new board and there is not one for the Pi5 as it's more than some address changes as the IO is now on a dedicated IO chip rather than the CPU.If I understand correctly once I setup the dtoverlay parameters I can use any board I like without messing about with drivers.So I guess next job is to try and make something graphical that will send and receive messages putting rxed messages on screen and with some sort of input to trigger a txed message. I suspect it's time to lean about concurrency and threading.CAN bus will have the advantage that I won't need to actually transmit things like the message ID and time stamp over the serial bus, they are just there. I am assuming that CAN bus is easier on the system than UART as the controller handles the buffering and verification of the data integrity. 8 bytes of payload is generally enough for me as that is how I have to roll anyway with the rest of the system.

author: rteodor
author_rank: Frequent Contributor
timestamp: «Reply #10 on:Todayat 10:55:14 am »
content: The CAN signals concept is like having thousands of virtual wires each one dedicated for transmitting a single "signal".Serializing data over a CAN frame works too, there is a standard now in cars (UDS I think) for transmitting data that is longer than 8 bytes. That is for diagnosis or RDS text strings between ECU's for example.

author: Simon
author_rank: Unknown
timestamp: «Reply #11 on:Todayat 02:24:40 pm »
content: Well I tend to use the CAN open system as I work with motor drivers and they tend to use it. Having created the required code to run a CAN open system off a microcontroller I find it sufficient to deal with handing data around all my devices as they either sit on the same network or are selectively passing on data that may have been reprocessed that was originally gained over a CAN bus running CAN open so it's all the same. CAN open also has means of transmitting data that is longer than 8 bytes but I have not needed to resort to that.What I will have fun with next is working out how to run program sections in different threads as from what I can see receiving CAN data is a blocking operation where the thread just halts waiting for a message. I don't want to hold up the entire program like that.