Firmware bench test validation approaches and tools - Page 1
Forum section: electronics / general

author: bonzer
author_rank: Regular Contributor
timestamp: June 18, 2025, 08:10:12 pm
content: Hello guys. I do firmware design in the field of industrial furnaces. I was wondering what is the typical approach when it comes to perform firmware tests while dealing with a simulated environment like a "test bench" when the real furnace is not used.The most popular one I heard about is connecting resistors, potentiometers, switches to simulate the sensors and then real valves and fans but not connected to the gas or air supply. This is most of the time a "static" environment where you can't simulate anything you want. Designing good hardware simulators could be a big deal and then you would still have to get your hands on the set up to manually "simulate" the behavior (for instance, turning the knot of a potentiometer that simulates an air pressure sensor).A programmable approach where the PC simulates real world events while connected to the microcontroller of the board is more interesting and I know many folks use LabVIEW but I barely know it thus I used Python for some simple things where I wrote some input variables, read some outputs and used it while the PC was running Python connected through the UART-RS-232 and FTDI to USB converter. The problem with the UART though is that you are limited to the baud rate, and it could sometimes be useful to read data at a high speed (under 1ms for instance). Taking advantage of the debug probe like the ST-LINK with STM32 and the debug mode could be interesting to be able to read data faster. I know Tiva environment probes in CCS with TI microcontrollers also allow to do real time RAM read-writing without breakpoints which is cool in my opinion. I was wondering whether this is something I could do by creating a kind of bridge between Python and the debug mode, out of the IDE world just relying on the GNU/GDB debugger commands and the debug probe. That would allow me to do much faster logs and probably easier read/write operations to the variables without passing through the UART channel. After surfing the net it looks like there are pyOCD or jlink Python libraries and so forth, that would allow to do that. I was wondering, has anyone ever tried to use them and are there any known approaches out there that could fit well for this kind of purposes even completely different from the debug path?

author: Doctorandus_P
author_rank: Super Contributor
timestamp: June 19, 2025, 12:46:55 am
content: You can use one of the many single board (Linux) computers as the heart of your simulator. You also mentioned STM32. Many of those processors have on-board USB @ 12Mbps. There are also quite a lot of USB <--> serial chips that support baudrates up to 3Mbps. (= 1/4 of "full speed" USB).But why would you need faster data acquisition for an oven? How much is it going to heat in a milli second? If you want to simulate some complicated mechanical interlocking system with switches (I think I counted 6 microswitches on a microwave oven, just to check that the door is really closed), then such tasks are better handled by an external microcontroller instead of a PC. PC's (and SBC's) are generally not real time systems, they can be doing some unrelated tasks and be unresponsive for a few milli seconds.If you want to simulate mechanical things, a simple and quick way is to add a systsem based on GRBL. GRBL can control up to 8 stepper motors. You could even use a stepper motor to control a real poteniometer if that is beneficial for some reason.I am a quite DIY / hacking sort of person. Others may use a standard PLC for tasks like this.

author: bonzer
author_rank: Regular Contributor
timestamp: June 19, 2025, 06:57:25 am
content: Thanks for the response. Actually the timing requirement is more related to logging the software or microcontroller processes rather than the mechanical data. I know that it seems crazy because one would expect a "slow" software as a consequence of the slow mechanics involved but it's not always the case as there is the safety compliance involved which requires some additional checks (like microcontroller self-tests or flame monitoring) that could introduce fast processes. And then when it happens to deal with some bugs like unexpected interrupts, from what I've experienced - the more data you log the better tracing you have. Anyway, it would be a "nice to have", not something really mandatory.In regards to Linux computer, it seems interesting. I'm gonna read more about it.

author: Fire Doger
author_rank: Regular Contributor
timestamp: June 19, 2025, 01:27:55 pm
content: Depends on application.For some applications it's possible to have a scaled down model, fully or partially, with single or multiple jigs.For example you could have a box with same temperature step curve (just 20 times less volume)You could have a miniature oven with 20 micro switches as safety chain switches, etc..Having a software to simulate a process based on input can also have bugs and it's also time consuming to develop and maintain it...In a case I know in elevator industry, it was cheaper and simpler to build a tower than develop an inhouse simulator...

author: AVI-crak
author_rank: Regular Contributor
timestamp: June 19, 2025, 03:07:36 pm
content: Heat up a temperature sensor analog through a copper strip in very good thermal insulation.This is not a 100% physical emulator, but it can emulate thermal inertia.

author: NorthGuy
author_rank: Super Contributor
timestamp: June 19, 2025, 03:21:40 pm
content: I would create a board where an MCU produces all necessary stimulus (signals, voltages) etc. These stimulus would be fed to the DUT. The MCU would also collect control signals from the DUT. This would emulate a furnace. The MCU could also log control signals continuously, as well as various failure cases. I would also create a PC app to analyze the logs.I would build a number of boards with this setup (DUT + Control MCU + PC link). I would write a number of tests programs for MCU to simulate test cases which represent various scenarios. You can then run a number of such boards simultaneously, then extract the logs and analyze the results. You can use these for development, stress tests, regression tests and whatnot.

author: Siwastaja
author_rank: Super Contributor
timestamp: June 19, 2025, 06:07:12 pm
content: My partial suggestion to the problem is to write the firmware "application logic" in portable C (e.g., by avoiding architecture-specific special constructs when they are unnecessary, and using stdint.h stuff), and add thin layers of "PC simulation" code within the firmware, replacing the peripheral accesses. Can be separate modules compiled for PC target, or can be just #ifdef within the project, or combination thereof.For example, I have used POSIX timers to trigger execution of handlers that normally trigger on timer interrupts. This allows adjusting speed of execution, too.Of course running the "PC firmware" won't replace testing with physical hardware, but there are great benefits: it is extremely easy to add a lot more logging, like you can just fopen() a file and write a gigabyte of CSV data without having to think about data links, or open a socket and control the thing through telnet/web browser/whatever level of complexity you are comfortable with. Speeding up the time allows complex tests to run quickly, and you can simulate with large datasets. You can add extra asserts() all around of your codebase, without worrying these extra checks wasting program memory or execution cycles on real hardware. This way you also add visibility and error checkinginside the DUT, not just observing the outputs.But eventually, you willalsowant the things mentioned by most others in this thread, because the pc-compiled firmware would miss all of the peripheral stuff and replace them with some simplified simulation models.

author: nctnico
author_rank: Super Contributor
timestamp: June 19, 2025, 10:03:15 pm
content: Another option is to have a compile time setting which compiles the firmware in simulation mode. It will simulate the (hardware) system it controls. Through a command line interface you can turn conditions/ failures on and off.