How to properly structure code with multiple states and inputs - Page 1
Forum section: electronics / general

author: craigc952
author_rank: Newbie
timestamp: January 14, 2024, 12:47:42 pm
content: I'm making a personal project of an alarm clock using a Raspberry Pi Pico and the C/C++ SDK with 4 buttons connected to the Pico to use for setting the time and alarm etc. The software I have now works, but it seems very messy, for example my main loop is a few hundred lines long as I need to check each input and respond to it differently depending on the current state (ie, whether I'm updating the time or alarm). Additionally I just have a global class instance for each button to store whether the button has been pressed and how long for.Is there a better way to structure the code? The code I have seems to go against every "rule" I read about avoiding global variables and long functions, but I'm not sure how I would go about improving that.This is just for a personal project, so ultimately it doesn't really matter, but I want to learn the proper way to write embedded code as I'm an electronics student and I'm interested in working with embedded hard/software so want to get a good understanding. Thanks

author: tggzzz
author_rank: Super Contributor
timestamp: January 14, 2024, 01:13:56 pm
content: Quote from: craigc952 on January 14, 2024, 12:47:42 pmWelcome to the forum. it is good to see someone observing and thinking about improvements.I note in passing that beginners think about "how thing will work", whereas experienced professionals think about "how things can fail and what to do about failures". Good FSM design and implementation isveryvaluable for the latterThere are several "design patterns" for this, each with their own advantages and disadvantages.One I've used for simple FSMs is to have a 2-dimensional table where one dimension is the current state and the other is the input event. Each table entry is a pointer to a function that is executed when an new event occurs in the current state. The function is responsible for executing the actions and specifying the next state. Both "current state" and "events" are implemented as integers used as indexes into the table.That has the advantages that each event+state action has to be explicitly specified, so it is easy to see that all combintions have been covered. The disadvantage is that it cannot directly express a hierarchy of states, thus complicating some designs.Another design pattern is more suitable for complex and evolving FSMs, and is described athttps://www.gofpattern.com/behavioral/patterns/state-pattern.phpand related design patterns. Each state is a class, each event is a method, the body of the method is the actions required. All classes implement the methods (i.e. events) that must be dealt with, and other superclasses implement methods for events that should not occur in normal operation or which are applicable to many states (e.g. a "PSU fault" event in any state should cause the "clean shutdown" actions and a transition to a "fault state" where all events are ignored). "Current state" is implemented as a singleton instance of the relevant class.That technique has the advantage of making it trivial to encode a terse record of each event that occurred (and when), plus the state transition trajectory. Those can beveryvaluable during commissioning and operation, since they aid understanding where your code needs improving and/or demonstrating to other companies that their code needs improving. The latter is a good defence against lawyers

author: craigc952
author_rank: Newbie
timestamp: January 14, 2024, 03:14:14 pm
content: Quote from: tggzzz on January 14, 2024, 01:13:56 pmThank you very much for the detailed reply! This seems like exactly what I need and I'll have a good look into it

author: nctnico
author_rank: Super Contributor
timestamp: January 14, 2024, 04:39:05 pm
content: When dealing with state machines also keep in mind that it can be helpfull to create sub-state machines that performs a certain task. This de-clutters the primary state machine.

author: tggzzz
author_rank: Super Contributor
timestamp: January 14, 2024, 05:06:41 pm
content: Quote from: nctnico on January 14, 2024, 04:39:05 pmThere can be two variants of that.One is to have a single FSM, e.g. an FSM for a door might be where the "in operation" state has substates "open" and "closed", and the "faulty" state has substates "technician called" and "dead".The other is to split a single FSM into multiple independent FSMs that send messages/events to each other. An example of that might be a telephone billing system where one FSM "understands" the network events to decide what the phone is doing (e.g. on hook, dialling, connected, ended) and another "understands" the pot of gold (in credit, credit low, out of credit).Sooner or later the OP will come across UML StateCharts. I like many aspects of them, but the semantics of pre and post event actions can be a pain in the backside to implement. Choose whatever is useful.

author: Nominal Animal
author_rank: Super Contributor
timestamp: January 15, 2024, 06:48:48 am
content: I like to use a state machine based on the user interface.For example, consider a minimal 24-hour alarm clock with HH:MM display, and perhaps an extra LED to indicate alarm is enabled, some kind of a noisemaker, and three buttons.We have four possible events – alarm triggered, and three button press or autorepeat events. We have six different user interface states:Normal– display current time.Alarm event triggers a switch toAlarmstate, and starts the alarm blaring.Button 1 event togglesalarm_enabled.Button 2 event switches toTime/Hourstate.Button 3 event switches toAlarm/Hourstate.Alarm– display current time, with the alarm blaring.Alarm event does not trigger anything. (It is already blaring.)Button 1 event stops the alarm blaring, clearsalarm_enabled, and switches toNormalstate.Button 2 event stops the alarm blaring, and switches toNormalstate.Button 3 event stops the alarm blaring, and switches toNormalstate.Time/Hour– modify current time (hour)Alarm event does not trigger anything. (Alarm is disabled while modifying time.)Button 1 event switches toTime/Minutestate.Button 2 event increments current time by one hour.Button 3 event decrements current time by one hour.Time/Minute– modify current time (minute)Alarm event does not trigger anything. (Alarm is disabled while modifying time.)Button 1 event switches toNormalstate.Button 2 event increments current time by one minute.Button 3 event decrements current time by one minute.Alarm/Hour– modify alarm time (hour)Alarm event does not trigger anything. (Alarm is disabled while modifying alarm time.)Button 1 event switches toAlarm/Minutestate.Button 2 event increments alarm time by one hour.Button 3 event decrements alarm time by one hour.Alarm/Minute– modify alarm time (minute)Alarm event does not trigger anything. (Alarm is disabled while modifying alarm time.)Button 1 event switches toNormalstate.Button 2 event increments alarm time by one minute.Button 3 event decrements alarm time by one minute.I find it very useful to visualize each such user interface state separately; perhaps sketch it out on paper, or draw in Inkscape or Gimp (layers are useful here) or whatever program I find most suitable. If the user experience is important, one can create a mock-up or simulation in a browser using static HTML + CSS + SVG + JavaScript: those open in any browser, does not require an internet connection. Very good for creating expensive custom gadgets for a client, and pretty quick to cobble together in JS when you get used to it.State machines can be documented in various forms, from spreadsheets, to tables, and to graphs. In particular, if you haveKevent triggers that can cause a transition to another state, andNstates, you can describe the target states in aN-row,K-column table, with additional columns describing other details for the user interface state on that row. For example, you could use another set ofKcolumns to describe side effects for each event, for example if an event changes a some internal value.The code implementation details vary a lot – immutable structures or arrays in ROM/Flash, or aswitch () { }statement using a numeric state identifier are common, sometimes the source code even generated from simpler text descriptions using scripts! – but organizing the main loop along the user interface states is the key here I'm pushing.Chosen input button handling approach affects the design a lot, though; especially if you support modifier buttons (like Shift key on your keyboard). One option is to have an event for each useful button combination. If you haveNbuttons, there areN×(N-1)/2 two-button combinations; orN×(N+1)/2 one- or two-button combinations. Another option is to have the modifier button press event switch to a modified state, release event switch back to the corresponding un-modified state, so that the other keys can correspond to different events in the two states. (It tends to be a question of what kind and how many button event types you need/want to implement.)

author: Kalvin
author_rank: Super Contributor
timestamp: January 15, 2024, 07:58:46 am
content: In addition to the valuable insights already provided in previous replies, I highly recommend exploring Hierarchical State Machines (HSMs), especially when dealing with complex projects. HSMs offer a robust framework for managing intricate state machines, significantly simplifying your design process.Key advantages of HSMs include:1. Nested States: This feature allows for better organization within the state machine. By nesting states, you can effectively manage complexity, reducing both state explosion and code duplication.2. Optional Entry/Exit Actions: HSMs provide the flexibility to define actions when entering or exiting a state, which can be crucial for certain applications.3. Streamlined State Transitions: The model ensures proper handling of entry and exit actions during state transitions, enhancing reliability and predictability.For a deeper understanding of HSMs, I strongly suggest looking into the work of Miro Samek. His websitehttps://www.state-machine.com/is a treasure trove of information on this topic.Additionally, Miro Samek's book, "Practical UML Statecharts in C/C++, 2nd Ed: Event-Driven Programming for Embedded Systems," is an invaluable resource. You can download it here:https://www.state-machine.com/psicc2While initially, the implementation of HSMs in C/C++ might seem complex or even daunting, the effort pays off significantly in the long run.Even if you're not planning to implement HSMs in your current project, understanding them is beneficial. They offer a compelling alternative to traditional flat FSMs (Finite State Machines) and can broaden your perspective on managing state in embedded systems.

author: craigc952
author_rank: Newbie
timestamp: January 15, 2024, 09:22:25 am
content: Thanks everyone for taking the time to reply. Lots of great information here for me to dig into and try out. Cheers!

author: 5U4GB
author_rank: Super Contributor
timestamp: January 15, 2024, 11:07:31 am
content: Quote from: tggzzz on January 14, 2024, 01:13:56 pm+1 for everything in the original message, but adding a pointer to additional documentation: For the first 15-20 odd years of Windows programming the main part of every program was a giant event loop (the message loop in Windows terminology), so an awful lot has been written by some very sharp people about ways of dealing with that sort of thing. It's been too long for me to remember any references on event loop design offhand and a lot of it will predate the web, but if you have access to a source of old books/magazines like MSDN Journal...

author: Nominal Animal
author_rank: Super Contributor
timestamp: January 15, 2024, 07:29:07 pm
content: Event-driven programming(link to Wikipedia) is a good place to start looking.The same event-based paradigm is still used everywhere for user interfaces, including web pages (JavaScript DOM Events).I often mixstate machinesand event-driven programming in my microcontroller projects. I like to use an interrupt to scan the button states, and 'post' the events thus generated to a queue of some sort. The main state machine loop then does not need to run at regular intervals, as it can handle multiple events in a batch, then go to sleep waiting for a new event.One very common issue with event-driven programming and user interfaces is that programmers often try to do too much in the event handler itself. It leads to that user interface becoming unresponsive until that event handler completes. The proper approach is to do as little as is necessary in the event handler itself, and then 'post' or 'queue' the work to be done when there is idle time. In web pages, this is easiest to do by usingwindow.setTimeout()JavaScript method (on the browserwindowobject).On a microcontroller, this means that it is better to have a lightweight interrupt (10 Hz to 1000 Hz) scanning the buttons, and based on the states and changes in states, provide a 'queue' of button events the main state machine look can examine.

author: tggzzz
author_rank: Super Contributor
timestamp: January 15, 2024, 09:06:05 pm
content: Quote from: Nominal Animal on January 15, 2024, 07:29:07 pmThat is good advice. It leads to scalable systems that can be understood, maintained and extended.I'll add only two levels of priority: normal operation and emergency shutdown.Of course any FSM implementation will fail if deadlock or livestock occurs. Ensuring that can't happen is non-trivial.

author: nctnico
author_rank: Super Contributor
timestamp: January 15, 2024, 09:26:44 pm
content: Quote from: Nominal Animal on January 15, 2024, 07:29:07 pmNot really. A queue means overhead and using an interrupt for something non-critical as scanning buttons is adding more complexity then necessary. Avoid needing to carry information between process boundaries with all the necessary process synchronisation issues and code overhead. A better solution is to scan the buttons from a main loop based on a tick counter.

author: SiliconWizard
author_rank: Super Contributor
timestamp: January 15, 2024, 11:01:28 pm
content: No, I have a better solution! All of the above is bad.

author: Nominal Animal
author_rank: Super Contributor
timestamp: January 15, 2024, 11:03:37 pm
content: As the RP2040 is a dual-core 32-bit ARM Cortex M0+ with ample memory for small event queues, we can use 32-bit unsigned integers to hold the event descriptors in a reasonably sized queue. Obviously not just keypresses, but all sorts of events; even X-Y coordinate pairs for touch events on touchscreens.To protect against concurrent access, the queues need a spin lock (that work across cores and interrupts). Since no queue function will access another queue, a single spin lock across all queues will suffice.spin_lock_t *queue_lock;It must be initialized at runtime by allocating one of the very few available spinlocks viaqueue_lock = spin_lock_init(spin_lock_claim_unused(true));The Pico SDK implementation not only blocks the other core, but also temporarily disables interrupts, so the push and pop functions can be used on the same queue on either core and from interrupt handlers.To get some polymorphicity from plain C99 or later code (no templates), we can declare one or more queues as just uint32_t arrays with an extra element. The length must be between 4 and 2048, inclusive, and preferably even (or you'll waste four bytes). For example,volatile uint32_t example_queue;Each such queue is initialized at run time viaqueue_init(example_queue, sizeof example_queue);which sets the first word to reflect an empty queue of that particular size.Bits 31..22 of the first word ism; the size of the queue is 2m+1.Bits 21..11 contain the index of the oldest element in the queue.Bits 10..0 contain the number of elements in the queue.Code:static inline void queue_init(volatile uint32_t *const q, uint32_t bytes) {// TODO: assert that bytes is between 12 and 8192, inclusive. If not a multiple of eight, some bytes are wasted.q = ((bytes - 8) & 0x1ff8) << 19;}/* Push a (nonzero) event to a queue.* Returns event if successful, zero to indicate queue is full.*/static inline uint32_t queue_push(volatile uint32_t *const q, uint32_t ev) {uint32_t saved = spin_lock_blocking(queue_lock);const uint32_t key = q;const uint32_t have = key & 0x7FF;const uint32_t base = (key >> 11) & 0x7FF;const uint32_t size = (key >> 21) | 0x001; // Always oddif (have >= size) {// Queue is fullev = 0;} else {uint32_t i = base + have;if (i >= size)i -= size;q[i + 1] = ev;q = key + 1; // In q: have++;}spin_unlock(queue_lock, saved);return ev;}/* Pop a (nonzero) event from the queue.* Returns event if successful, zero to indicate the queue was empty.*/static inline uint32_t queue_pop(volatile uint32_t *const q) {uint32_t saved = spin_lock_blocking(queue_lock);const uint32_t key = q;const uint32_t have = key & 0x7FF;uint32_t base = (key >> 11) & 0x7FF;const uint32_t size = (key >> 21) | 0x001; // Always odduint32_t ev;if (have <= 0) {// Queue is emptyev = 0;} else {ev = q[++base];if (base >= size) {// In q: have--, base = base - size, with base incremented by oneq = key + 0x7FF - (size << 11);} else {// In q: have--, with base incremented by oneq = key + 0x7FF;}}spin_unlock(queue_lock, saved);return ev;}Above, the buffer is circular. The math applied to the first word is "tricky" (in the bad, not easily maintained sense), because instead of repacking it, we can apply the operations directly to it using a single addition or an addition and a subtraction. The simple idea is that ifval= ((size& 0x7FE) << 21) | ((base& 0x7FF) << 11) | (have& 0x7FF);incrementingbaseand then repackingvalis the same as adding(1<<11)=0x800toval. Similarly, incrementinghaveis the same as incrementingval.sizenever changes.Note thatsizealways being odd means we don't need to store its least significant bit, which allows us to store three 11-bit values in a single 32-bit value. This corresponds to a queue structure whose size is a multiple of eight bytes. The maximum queue length is then 2047 events.On raw metal on single-core ARM Cortex-M,queue_lockis not needed, anduint32_t saved = spin_lock_blocking(queue_lock);is replaced by something likeuint32_t saved = __get_PRIMASK(); if (saved) __disable_irq();, andspin_unlock(queue_lock, saved);is replaced with something like__set_PRIMASK(saved);orif (saved) __enable_irq();. The exact details vary a bit between M0, M4, M7, etc.On AVRs,queue_lockis not needed, anduint32_t saved = spin_lock_blocking(queue_lock);is replaced withunsigned char saved = SREG; cli();andspin_unlock(queue_lock, saved);is replaced withSREG = saved;.In general, on single-core microcontrollerssaved=spin_lock_blocking()needs to disable interrupts that may use queue functions, returning the previous interrupt enable/disable state, whichspin_unlock(saved)then needs to restore.Quote from: nctnico on January 15, 2024, 09:26:44 pmI like my buttons reliable and precise, with software debouncing and accelerating autorepeat. I hate devices that occasionally miss keypresses, because you are "too fast". Varying autorepeat rate also feels very clunky and amateurish to me.If the buttons are directly connected (not a matrix), the periodic scanning interrupt can be normally disabled, and instead only a state change interrupt active that re-enables the periodic scanning interrupt on the first detected state transition. (On many Cortex-M implementations, there is only one hardware input pin interrupt per GPIO bank, so this is quick and easy to do.) The periodic scanning interrupt also counts the number of iterations with no events and no keys being pressed, and when that exceeds some configured limit, disables itself and enables the state change interrupt. This saves resources.When the input events include combined button presses and/or doublepresses, it greatly simplifies the input handling, because the state machine only notices verified events, and can run at varying intervals, without having to considerelapsed timebetween iterations. Instead of counting milliseconds, I count scanning loop iterations, and can tune the scan rate to something suitable.As a particular example, consider a capacitive touchscreen on a small 240×320 display (2.8",3.2", or3.5"size). You can use an event queue with 32-bit entries to record the touch tracks (id:pressure:X:Y), and even support multitouch (one id per concurrent touch). The rate at which you poll the touchscreen determines how many waypoints you get for gestures and such. If you poll at specific regular intervals, the waypoints are regularly spaced in time, and delta between consecutive points is directly proportional to velocity. Otherwise, your waypoints are basically randomly polled along the track the user dragged.The code is not hard to craft nor use; it is really only the RAM used by such queues that can be a limiting factor.Consider my example code earlier in this message: the state machine simply callsuint32_t ev = queue_pop(input_events);whenever it wants to see if there are pending input events that would affect the state machine. The correspondingqueue_push(input_events, new_ev)calls can occur in interrupt service routines, or even on another core (or in an interrupt service routine on another core) on Raspberry Pi Pico (aka RP2040).If your gadget provides a serial port or USB serial for commands (so it can be remotely operated instead of using the physical buttons), your serial command parser can generate the corresponding input events too, without added complexity.My point here is to show that such event queues can be used on microcontrollers, even AVRs; and that they can be used in various ways with finite state machines. Whether they fit to any specific use case, depends obviously on the use case, with the memory they need being their most limiting factor.I don'talwaysuse them in my UI projects, either: after all, they're just another tool one can use when appropriate.Quote from: SiliconWizard on January 15, 2024, 11:01:28 pm:'(

author: tggzzz
author_rank: Super Contributor
timestamp: January 15, 2024, 11:09:25 pm
content: Quote from: nctnico on January 15, 2024, 09:26:44 pm"Overhead" (whatever that means) is normally unimportant nowadays.Arguably more complex when considering a very simple FSM. But that has to be weighed against the simplicity of there being only a single flow-of-control design pattern and implementation.A half-way house would be to have several FSMs, one for the buttons plus a separate FSM for the main execution processing.Or you could just do it properly, and dedicate one processor to the buttons, and one or more processors to the main execution processing. Go xC+xCORE; you know it makes sense

author: nctnico
author_rank: Super Contributor
timestamp: January 15, 2024, 11:24:30 pm
content: Quote from: tggzzz on January 15, 2024, 11:09:25 pmWhat you see with a lot of programmers is that during their career they go through roughly 3 stages:1) Do everything in a single loop2) Discover the existence of processes so start to put everything into seperate processes only to find out this makes software A) unreliable, B) difficult to debug and C) difficult to maintain3) Go back to implementing related tasks into single 'super loop' processes and only use seperate processes after very careful deliberation

author: tggzzz
author_rank: Super Contributor
timestamp: January 15, 2024, 11:28:46 pm
content: Quote from: nctnico on January 15, 2024, 11:24:30 pmYup, but hopefully the code in (3) is better architected and implemented than in (1)A good start are Hoare's CSP concepts, as implemented in xC and partially implemented in Rust, Go, etc.

author: 5U4GB
author_rank: Super Contributor
timestamp: January 16, 2024, 05:59:08 am
content: Quote from: Nominal Animal on January 15, 2024, 11:03:37 pmI think for Windows memorial purposes you need to sprinkle at least a few HANDLE_MSG()'s throughout the code though.More seriously though, you could at least look at Windows message maps because they do a lot of the messy stuff for you, the handling is done via macros so you could just retarget them for your own purposes. Not saying they're the perfect solution, but they are aready-made one that's evolved over time to handle this sort of thing.

author: Psi
author_rank: Super Contributor
timestamp: January 16, 2024, 06:09:53 am
content: While a FSM with function points is super useful and is sometimes the best option, it also can make things harder to read.A big switch statement but where each case is just a single function call to somewhere else can sometimes be better.It really depends on the situation

author: tggzzz
author_rank: Super Contributor
timestamp: January 16, 2024, 10:01:46 am
content: Quote from: Psi on January 16, 2024, 06:09:53 amFor FSMs that areguaranteedto remain tiny, I agree a switch statement is sufficient. But when they evolve to be less than tiny, the readability becomes questionable - and modifiability becomes problematic.I've seen an FSM that was extended over the years and it had mutated to the extent that if-then-else statements were nested10 deep. Each individual mutation was sensible and justified, based on evolving customer requirements. Yes, that was heroic and incompetent in retrospect, but it was a classic case of boiling a frog by raising the temperature one degree at time.All techniques have their brick walls, but IMHO the switch statement technique hits the brick wall earlier than most.Example os a "tiny" FSM: debouncing a switch before then sending a up/down event to a "real" FSM.

author: nctnico
author_rank: Super Contributor
timestamp: January 16, 2024, 02:45:16 pm
content: Quote from: tggzzz on January 16, 2024, 10:01:46 amIt also depends on the tooling you are using to write code. Typically an IDE won't be able to show call hiearchies through lookup tables. So an FSM implemented using a lookup table is harder to deal with compared to using switch / case statements. Even with thourough documentation, many programmers are not keen to work on a project which uses an FSM implemented using lookup tables. Been there, done that.So a lot is to say for using switch / case statements (or equivalent as not all language support the switch / case construct). For example: have one function with a switch / case per state could be an implementation which is more friendly to maintain.

author: tggzzz
author_rank: Super Contributor
timestamp: January 16, 2024, 04:13:08 pm
content: Quote from: nctnico on January 16, 2024, 02:45:16 pm"Naked" function pointers will "disrupt" most analysis tools; FSMs aren't unique in that respect.QuoteI refused to even look at that 10-deep if-then-else mess, except to make sure the implementation was a grotesque as people said. It was. Quite astounding.IDEs are a crap way to document and/or understand FSMs, full stop. The best way is a combination of diagram showing the states and interconnecting events, plus text for the details of any actions. While it is possible to auto-generate code from such diagrams, it isn't easy and requires discipline.The real issue is that before long someone will patch thebinariesauto-generated code. Either it won't be back-annotated or the modifications won't be easily expressible in the pretty pictures.QuoteWhile I agree the table technique is limited to small/medium FSMs, in my experience switch/if statements are worse in the medium and long terms.The table technique has the virtue of being simple, explicit, and has an obvious direct correspondance with the FSM design spec. That satisfies my meta-requirement that it should be easy to determine exactly which line of code needs to be changed, and that changing it won't accidentally change other parts of the FSM.But overall it is a case that an engineer should be aware that there are multiple techniques, and to be able to choose the appropriate one for the current job. "Horses for courses", "many ways to skin a cat", etc.Oh, it is also necessary that an engineer recognises that they are in the process of building an FSM. Too many softies think "FSM==parsing in a compiler", and just throw a lot of spaghetti if-then-else statements in the implementation.

author: Nominal Animal
author_rank: Super Contributor
timestamp: January 16, 2024, 05:16:18 pm
content: Quote from: nctnico on January 15, 2024, 11:24:30 pmI've spent quite a lot of time in systems programming (libraries, services, backends), and in high-performance computing, specializing in parallel and distributed computing. Instead of step 2 and step 3, step 2 was an explosion of new opportunities, techniques, and approaches for me; and going back to step 1 or step 3 feels extremely constricting. Even on single-core 8-bit microcontrollers, I find the way interrupts behaves analogous to pre-emptive threads on a single-core, and I enjoy applying the associated techniques.So, all I can say is that I donotsee that. Sure, I do know and meet a lot of programmers who are not able or willing to spend the time and effort to really understand parallel and distributed computing, message-passing techniques, and all the related problems and the various ways they can be approached, but when I help others learn, I do steer them resolutely towards the path I took and found useful, powerful, and rewarding.Quote from: tggzzz on January 16, 2024, 04:13:08 pmYep. This is why I prefer the inverse method, easily done when the object file format is ELF: use a macro for declaring the state structure, and move all state structures to their own section. This means they will be consecutive in memory, like an array; and easy to pluck out from the final linked firmware image. Parse using few lines of Python (struct.unpack()as itsformatmakes it easy), describing each entry inGraphvizDOT language for visualization.With a bit more work, you can use another section for structures that describe each state in human terms, and contain a pointer to the referred to state structure; maybe another that describes each handler (with a pointer to the function). These sections are dropped from the firmware image, but you have a second link target that does contain these sections and the state structure section (and code and data sections, so you get the function pointers). You can then use a simple Python script to extract the state structures and their descriptions, and generate the documentation and state graphs from the actual data structures used on the target.If you use a gdb-derived debugger,pretty-printing structures and graphs and nested structuresonly requires you to write a Python helper gdb can use. This means that although many intensely dislike Python, it is a good tool for build helpers and generators for those using gcc-based toolchains.

author: tggzzz
author_rank: Super Contributor
timestamp: January 16, 2024, 05:44:33 pm
content: Quote from: Nominal Animal on January 16, 2024, 05:16:18 pmI've done that successfully, on a hard realtime Z80 system many decades ago. Two of the reasons it worked was because I was the only designer and implementer, and I could structure it into a cooperating RTOS based around a yield().IMHO it doesn't scale well, either with number of developers, or with long-duration interrupt processing, or with modern multicore MCUs. Yes, itcanbe made to work - but I don't feel I am in control, nor that I can prove to others that it will work, nor that I can see how well it is/isn't working.FSMs, messages and event queues are a superb mechanism for decoupling relatively independent processing at the design stage. During implementation and commissioning they provide a neat conceptual point for logging/measuring what the system is doing and how efficiently it is working.They are a great way to be able to say "this log shows exactly what my system did and what your system didn't do; your problem, you fix it"QuoteI presume such home-grown tool rely on specific implementation artefacts being visible in the binaries. That's regrettably fragile, being reliant on informal design patterns in the code and specific compiler optimisations.As for Python, semantically significant whitespace is too close to brainfuck for my taste. But it gets quick and dirty hacks done, so "fake it till you make it"/"move fast andbreak thingspick up the pieces" youngsters like it.

author: nctnico
author_rank: Super Contributor
timestamp: January 16, 2024, 07:26:06 pm
content: Quote from: tggzzz on January 16, 2024, 04:13:08 pmThe problem is that a lot of people don't care about documentation (or it is just too much work to look things up). So the better the code is suited for analysis through the tools a typical IDE provides, the easier / quicker a piece of software is to maintain.

author: tggzzz
author_rank: Super Contributor
timestamp: January 16, 2024, 08:03:38 pm
content: Quote from: nctnico on January 16, 2024, 07:26:06 pmOnly up to a point. That point being a low-level view of how it is put together.You cannot reconstruct the source code from optimised binaries. Similarly you cannot recover the intent of the code (particularly the design patterns, the preconditions and postconditions, the architecture, the constraints) from the code. The most you can hope to recover is what it does, not what it ought to be doing.We pretty much agree, but I have doubts that many programmers care about design. After all, if unit tests pass then it works. Doesn't it?

author: SiliconWizard
author_rank: Super Contributor
timestamp: January 16, 2024, 10:18:26 pm
content: You mean, if it compiles then it works. Right?

author: nctnico
author_rank: Super Contributor
timestamp: January 16, 2024, 10:33:23 pm
content: Quote from: SiliconWizard on January 16, 2024, 10:18:26 pmYes. The real work begins after a piece of code compiles.

author: tggzzz
author_rank: Super Contributor
timestamp: January 16, 2024, 11:49:24 pm
content: Quote from: nctnico on January 16, 2024, 10:33:23 pmIn a decent language the IDE accurately signals problems as you type, before you get anywhere near compiling.When I was a boy compilation errors were avoided by carefully examining the printout before submitting the tape. Turnaround was hours at best, a week at worst

author: tggzzz
author_rank: Super Contributor
timestamp: January 16, 2024, 11:55:05 pm
content: Quote from: SiliconWizard on January 16, 2024, 10:18:26 pmModern practice requires the unit tests have to pass.Nobody looks at the quality of the unit tests.Nobody asks if the tests could even theoretically ensure correctness, e.g. ACID transaction properties, recovery when a high availability cluster is violently partitioned and subsequently rejoined (split brain problem? Wossat?)Bah humbug.

author: Nominal Animal
author_rank: Super Contributor
timestamp: January 17, 2024, 01:15:06 am
content: Quote from: tggzzz on January 16, 2024, 05:44:33 pmOh no, I didn't mean RTOS complexity levels: just that the interactions and problems that may occur when an interrupt and the main loop access the same variables, are very similar to the issues when two threads need to access the same variables. On a single core, the two problems are identical.Quote from: tggzzz on January 16, 2024, 05:44:33 pmYes.Analogously, if we look at say implementation of a network protocol stack, and theOSI model, the layering helps us understand the system, but an implementation does not need to follow the layer boundaries.Quote from: tggzzz on January 16, 2024, 05:44:33 pmNo, that'd be as bad as generating the code or data structures!The binary parser (Python script) is target ABI dependent, and need to be kept in sync with the structure type definition, but otherwise it just looks at the ELF object code.If you look at the ELF object files, the relocation records include all referenced symbols. This means you can extract what functions and variables a function or data structure refers to by name. Generate a graph of these, apply disjoint-set to flatten the dependencies –– a function may call another function that returns the new state ––, and you the set of states reachable from each state via the functions named in its function pointers (and functions they may call).If each state is described by a structure, and its transitions/actions/event handlers named in function pointers, you can get the entireactualstate graph including all possible transitions, this way.If you do not want to dig into the ELF object files –– which is kinda funny avoidance, because you're already using one such that creates the uploadable/flashable/burnable firmware image! –– you can settle for documenting the states and functions at declaration time in the source code.Yes, this does require that the declarations (including strings and states reachable) are maintained by the developers: we're not talking about how toforcemaintainers to do their job, only about how we can make it easier for those who are willing to do the job in the first place.In C, the structure variables for each state do need to be declared before they are used, so that event handlers can check for specific states and optionally return new states to transition to. So, instead of just a block ofconst struct state_structstate_var_name;, you use a block ofDECLARE_STATE(state_var_name,description-string);so that the state variable names can be referenced in the event handler functions.Before we can define the state structures thus declared, we also need to define or at least declare all event handler functions, so the state structures can be populated with function pointers to them. Instead of just a block ofrettypehandler_name(argspec)), you use a block ofDECLARE_HANDLER(func_name,description-string);orDECLARE_HANDLER(func_name,description-string,state_var_name...);listing the possible states the declared/defined handler can transition to. (The list can be expanded to either as a list of addresses (& applied to each), to a list of strings, or to a string containing the comma-separated list ofstate_var_names in parentheses, or"()"if none specified.) These macros do emit the structure and function declaration itself, but also astatic const structto a dedicated section that is discarded when linking the firmware. That is, those strings will never be embedded in the final firmware, and do not count against its RAM/ROM/Flash use. They only bloat the object files during building a bit. They can be dumped in binary form usingobjcopy --dump-sectionsection-name=filenameobject-file-name, which suffices for string-only descriptions. For symbolic references to other variables, additional work is needed.I do use custom ELF sections a lot, both in embedded and systems development, and still find new uses for them all the time.For example, if you wanted to add important notes related to a function or file, instead of a comment, you could use a macro,FUNCTION_NOTE(.author = "Nominal Animal", .when = "2024-01-17 23:59:59 EET",.text = "Interrupts must be disabled when entering this function!");BUGFIX_NOTE(.cve = "CVE-xxxx", .author = "Nominal Animal", .applied = "2024-01-17 23:59:59 EET",.text = "Fixed by clearing 'foo' before returning from the function.");which expands__FILE__,__LINE__,__FUNCTION__, populating a suitable structure in a dedicated section discarded from the final firmware image, or to nothing during normal build. If you use a documentation generator (say, Sphinx/reStructuredText or Markdown), it could pull these in from the most recently compiled object files automagically, or usegcc -Eon the source files with the two macros defined to produce an easily-sed'd strings with delimiters, and pull the contents from the raw source files. (That is, apply only the preprocessor step, with the macros producing special delimiters that do not occur in normal source code, making the contents of these macros easy to extract using regular expressions from the preprocessor output.)I am not saying these are the ways things should be done. I am saying I have found some of these, in varying combinations, useful; and that when one knows about them before starting a new project, some of them can be really useful in making that project easier to maintain in the long term –– depending of course on the project details.

author: 5U4GB
author_rank: Super Contributor
timestamp: January 17, 2024, 04:05:40 am
content: Quote from: tggzzz on January 16, 2024, 11:49:24 pmSigh, the youth of today. When I were a lad... actually before I was born, but at the school I went to, you punched your program and added the cards to the mailbag for when it went out to the computer, possibly an IBM 1620, and a few weeks later got back the results. You spent a loooong time desk-checking your code before submitting it to this multi-week turnaround.Tell that to the youth of today...

author: 5U4GB
author_rank: Super Contributor
timestamp: January 17, 2024, 04:07:51 am
content: Quote from: tggzzz on January 16, 2024, 11:55:05 pmIf you're really unlucky you've got test vectors published by a government body that are wrong.However, for some reason everyone's unit tests pass all of the tests that require incorrect results.

author: Nominal Animal
author_rank: Super Contributor
timestamp: January 17, 2024, 05:51:23 am
content: Theory and musing is all fine and dandy, but real world is king. Time for a practical example!I outlined a simple state machine description for an alarm clock with three buttonsin reply #5.If we combine action events (like incrementing the current time by one hour) and transition events (switching to a new state), we can describe each state with just four function pointers that return a new state or NULL (alarm being triggered right now, plus one for each button being pressed or autorepeated), plus a function pointer for updating the display state (since different things are displayed in different states). Something like this (edited the first few lines to fix the compiler warnings and errors):Code:#include <stddef.h>struct state {// The parameter to update_display() is zero after a transition, otherwise the last return value from update_display(). For blinking.unsigned char (*const update_display)(const unsigned char);// Events return NULL or the new state to transition to.const struct state * (*const alarm_event)(void);const struct state * (*const button_1_event)(void);const struct state * (*const button_2_event)(void);const struct state * (*const button_3_event)(void);};// Declare all possible states.const struct state am_disarmed;const struct state am_armed;const struct state am_alarming;const struct state am_snoozing;const struct state am_armed_setting_hours;const struct state am_armed_setting_minutes;const struct state am_snoozing_setting_hours;const struct state am_snoozing_setting_minutes;const struct state am_disarmed_setting_hours;const struct state am_disarmed_setting_minutes;const struct state am_armed_setting_alarm_hours;const struct state am_armed_setting_alarm_minutes;const struct state am_snoozing_setting_alarm_hours;const struct state am_snoozing_setting_alarm_minutes;const struct state am_disarmed_setting_alarm_hours;const struct state am_disarmed_setting_alarm_minutes;// Define event handlers.const struct state *do_reset(void) { /* Initialize time from RTC, disable alarm set LED */ return &am_disarmed; }const struct state *do_disarm(void) { /* Disable alarm set LED */ return &am_disarmed; }const struct state *do_arm(void) { /* Enable alarm set LED */ return &am_armed; }const struct state *do_start_alarm(void) { /* Start sound */ return &am_alarming; }const struct state *do_stop_alarm(void) { /* Stop sound */ return &am_armed; }const struct state *do_snooze_alarm(void) { /* Stop sound */ return &am_snoozing; }const struct state *do_disarm_alarm(void) { /* Stop sound */ return &am_disarmed; }const struct state *do_set_hours_armed(void) { return &am_armed_setting_hours; }const struct state *do_set_minutes_armed(void) { return &am_armed_setting_minutes; }const struct state *do_update_rtc_armed(void) { /* Save time to RTC */ return &am_armed; }const struct state *do_set_hours_snoozing(void) { return &am_snoozing_setting_hours; }const struct state *do_set_minutes_snoozing(void) { return &am_snoozing_setting_minutes; }const struct state *do_update_rtc_snoozing(void) { /* Save time to RTC */ return &am_snoozing; }const struct state *do_set_hours_disarmed(void) { return &am_disarmed_setting_hours; }const struct state *do_set_minutes_disarmed(void) { return &am_disarmed_setting_minutes; }const struct state *do_update_rtc_disarmed(void) { /* Save time to RTC */ return &am_disarmed; }const struct state *do_set_alarm_hours_armed(void) { return &am_armed_setting_alarm_hours; }const struct state *do_set_alarm_minutes_armed(void) { return &am_armed_setting_alarm_minutes; }const struct state *do_update_alarm_armed(void) { /* Save alarm time */ return &am_armed; }const struct state *do_set_alarm_hours_snoozing(void) { return &am_snoozing_setting_alarm_hours; }const struct state *do_set_alarm_minutes_snoozing(void) { return &am_snoozing_setting_alarm_minutes; }const struct state *do_update_alarm_snoozing(void) { /* Save alarm time */ return &am_snoozing; }const struct state *do_set_alarm_hours_disarmed(void) { return &am_disarmed_setting_alarm_hours; }const struct state *do_set_alarm_minutes_disarmed(void) { return &am_disarmed_setting_alarm_minutes; }const struct state *do_update_alarm_disarmed(void) { /* Save alarm time */ return &am_disarmed; }const struct state *do_inc_hours(void) { /* Time +1h */ return NULL; }const struct state *do_dec_hours(void) { /* Time -1h */ return NULL; }const struct state *do_inc_minutes(void) { /* Time +1m */ return NULL; }const struct state *do_dec_minutes(void) { /* Time -1m */ return NULL; }const struct state *do_inc_alarm_hours(void) { /* Alarm +1h */ return NULL; }const struct state *do_dec_alarm_hours(void) { /* Alarm -1h */ return NULL; }const struct state *do_inc_alarm_minutes(void) { /* Alarm +1m */ return NULL; }const struct state *do_dec_alarm_minutes(void) { /* Alarm -1m */ return NULL; }// Define display functions; actual display code omitted. ticks is for blinking; zeroed at each transition.unsigned char display_time_disarmed(unsigned char ticks) { return ticks + 1; }unsigned char display_time_blink_hours_disarmed(unsigned char ticks) { return ticks + 1; }unsigned char display_time_blink_minutes_disarmed(unsigned char ticks) { return ticks + 1; }unsigned char display_time_armed(unsigned char ticks) { return ticks + 1; }unsigned char display_time_blink_hours_armed(unsigned char ticks) { return ticks + 1; }unsigned char display_time_blink_minutes_armed(unsigned char ticks) { return ticks + 1; }unsigned char display_time_snoozing(unsigned char ticks) { return ticks + 1; }unsigned char display_time_blink_hours_snoozing(unsigned char ticks) { return ticks + 1; }unsigned char display_time_blink_minutes_snoozing(unsigned char ticks) { return ticks + 1; }unsigned char display_time_alarming(unsigned char ticks) { return ticks + 1; }unsigned char display_alarm_blink_hours_disarmed(unsigned char ticks) { return ticks + 1; }unsigned char display_alarm_blink_minutes_disarmed(unsigned char ticks) { return ticks + 1; }unsigned char display_alarm_blink_hours_armed(unsigned char ticks) { return ticks + 1; }unsigned char display_alarm_blink_minutes_armed(unsigned char ticks) { return ticks + 1; }unsigned char display_alarm_blink_hours_snoozing(unsigned char ticks) { return ticks + 1; }unsigned char display_alarm_blink_minutes_snoozing(unsigned char ticks) { return ticks + 1; }// Define state structuresconst struct state am_disarmed = {.update_display = display_time_disarmed,.alarm_event = NULL,.button_1_event = do_arm,.button_2_event = do_set_hours_disarmed,.button_3_event = do_set_alarm_hours_disarmed,};const struct state am_armed = {.update_display = display_time_snoozing,.alarm_event = do_start_alarm,.button_1_event = do_disarm,.button_2_event = do_set_hours_armed,.button_3_event = do_set_alarm_hours_armed,};const struct state am_alarming = {.update_display = display_time_alarming,.alarm_event = NULL,.button_1_event = do_disarm_alarm,.button_2_event = do_snooze_alarm,.button_3_event = do_stop_alarm,};const struct state am_snoozing = {.update_display = display_time_snoozing,.alarm_event = do_start_alarm,.button_1_event = do_disarm,.button_2_event = do_set_hours_snoozing,.button_3_event = do_set_alarm_hours_snoozing,};const struct state am_armed_setting_hours = {.update_display = display_time_blink_hours_armed,.alarm_event = NULL,.button_1_event = do_set_minutes_armed,.button_2_event = do_inc_hours,.button_3_event = do_dec_hours,};const struct state am_armed_setting_minutes = {.update_display = display_time_blink_minutes_armed,.alarm_event = NULL,.button_1_event = do_update_rtc_armed,.button_2_event = do_inc_minutes,.button_3_event = do_dec_minutes,};const struct state am_disarmed_setting_hours = {.update_display = display_time_blink_hours_disarmed,.alarm_event = NULL,.button_1_event = do_set_minutes_disarmed,.button_2_event = do_inc_hours,.button_3_event = do_dec_hours,};const struct state am_disarmed_setting_minutes = {.update_display = display_time_blink_minutes_disarmed,.alarm_event = NULL,.button_1_event = do_update_rtc_disarmed,.button_2_event = do_inc_minutes,.button_3_event = do_dec_minutes,};const struct state am_snoozing_setting_hours = {.update_display = display_time_blink_hours_snoozing,.alarm_event = NULL,.button_1_event = do_set_minutes_snoozing,.button_2_event = do_inc_hours,.button_3_event = do_dec_hours,};const struct state am_snoozing_setting_minutes = {.update_display = display_time_blink_minutes_snoozing,.alarm_event = NULL,.button_1_event = do_update_rtc_snoozing,.button_2_event = do_inc_minutes,.button_3_event = do_dec_minutes,};const struct state am_armed_setting_alarm_hours = {.update_display = display_alarm_blink_hours_armed,.alarm_event = NULL,.button_1_event = do_set_alarm_minutes_armed,.button_2_event = do_inc_alarm_hours,.button_3_event = do_dec_alarm_hours,};const struct state am_armed_setting_alarm_minutes = {.update_display = display_alarm_blink_minutes_armed,.alarm_event = NULL,.button_1_event = do_update_alarm_armed,.button_2_event = do_inc_alarm_minutes,.button_3_event = do_dec_alarm_minutes,};const struct state am_disarmed_setting_alarm_hours = {.update_display = display_alarm_blink_hours_disarmed,.alarm_event = NULL,.button_1_event = do_set_alarm_minutes_disarmed,.button_2_event = do_inc_alarm_hours,.button_3_event = do_dec_alarm_hours,};const struct state am_disarmed_setting_alarm_minutes = {.update_display = display_alarm_blink_minutes_disarmed,.alarm_event = NULL,.button_1_event = do_update_alarm_disarmed,.button_2_event = do_inc_alarm_minutes,.button_3_event = do_dec_alarm_minutes,};const struct state am_snoozing_setting_alarm_hours = {.update_display = display_alarm_blink_hours_snoozing,.alarm_event = NULL,.button_1_event = do_set_alarm_minutes_snoozing,.button_2_event = do_inc_alarm_hours,.button_3_event = do_dec_alarm_hours,};const struct state am_snoozing_setting_alarm_minutes = {.update_display = display_alarm_blink_minutes_snoozing,.alarm_event = NULL,.button_1_event = do_update_alarm_snoozing,.button_2_event = do_inc_alarm_minutes,.button_3_event = do_dec_alarm_minutes,};I replaced some details with comments, and the display functions obviously do nothing – their names should describe their intent – but the state machine is complete.How wouldyouverify the states make sense, and each state is reachable?The code is well-formed enough you can extract the data using regular expressions from the source, but this is extremely fragile for general use.The function names were chosen to be descriptive enough to give an intuitive picture of what is happening, but generally the purpose tends to drift so one cannot expect such informative function names –– or even such clear patterns of function names! –– in real life.For what it is worth, attached is an example of the kind of state graph you can get from Graphviz. No tweaking by hand (other than requantizing the PNG image to 128 colors and recompressing it); DOT was generated by a Python script, and PNG viadot -Tpng.

author: uliano
author_rank: Regular Contributor
timestamp: January 17, 2024, 08:39:40 am
content: Quote from: Nominal Animal on January 17, 2024, 05:51:23 amcan you share also that?

author: tggzzz
author_rank: Super Contributor
timestamp: January 17, 2024, 10:07:16 am
content: Quote from: 5U4GB on January 17, 2024, 04:07:51 amOr the marketing equivalent... Implement a better, new X using modern technology, but it must behaveexactlylike the existing X.Or the financial compliance one... Don't change anything, since we have to be able to use it to repeat historic calculations. There are good business reasons why IBM COBOL programs from the 60s are still running on "IBM360s"Or the safety audit one. That's why PDP11s running nuke power stations will continue to be supported until 2038 (IIRC).

author: tggzzz
author_rank: Super Contributor
timestamp: January 17, 2024, 10:21:03 am
content: Quote from: Nominal Animal on January 17, 2024, 01:15:06 amAgreed.QuotePreciselyQuoteRoutinely looking at any compiler output is heroic in my book. Exception: checking the optimised binary is actually does preserve the properties you hoped the compiler flags would preserve.Documentation techniques are good, provided youngsters follow them, business idiots don't scream "get it out the door now", and religious zealots don't remove all comments on the basis that comments are misleading and code is all that counts (seen that; went ballistic).Decent languages allow custom annotations denoting state/event/preconditions/etc, that can easily be extracted for post-processing. Reduces problems in the previous paragraph.QuoteAgreed. There are many ways to skin cats.

author: Psi
author_rank: Super Contributor
timestamp: January 17, 2024, 10:58:35 am
content: Quote from: nctnico on January 14, 2024, 04:39:05 pmYeah, that is often overlooked because It feels wrong to do it, but it often makes things so much simpler and easier to understand.