Disabling Compiler Optimization vs Using volatile - Page 1
Forum section: electronics / general

author: EVblog1
author_rank: Contributor
timestamp: June 24, 2025, 03:42:57 pm
content: I want to set up a timer interrupt on the 8051, using keil ,I’m trying to understand the proper use of the volatile keyword in embedded C. From what I’ve read, if we declare a variable as volatile, we’re basically telling to the compiler that its value might change unexpectedly for example, due to an interrupt or hardware so the compiler should not optimize it by caching the value in a register. Instead, it should always read it from the actual memory location.Is that the main purpose of using volatile? if we already know that compiler optimizations can cause issues in such cases, then why don’t we just turn off optimization entirely during compilation? Wouldn’t that avoid the problem?

author: eutectique
author_rank: Frequent Contributor
timestamp: June 24, 2025, 04:03:02 pm
content: Quote from: EVblog1 on June 24, 2025, 03:42:57 pmYesQuote from: EVblog1 on June 24, 2025, 03:42:57 pmWhat issues? In what cases?Optimisation has nothing to do with volatile. If you declare a variable volatile, and have two consecutive reads (for example), the compiler must generate code to execute two consecutive reads. Otherwise, it is broken.Quote from: EVblog1 on June 24, 2025, 03:42:57 pmWhat problem?

author: hans
author_rank: Super Contributor
timestamp: June 24, 2025, 04:08:55 pm
content: Your understanding of volatile sounds correct.Do mind that race conditions can still occur even when a variable is volatile, as there are some instructions between reading from the volatile memory location and then processing that value. It depends what you're doing if this is going to be a problem.If you turn off optimization in the compiler, your program may run very slow or become very large. On my ARM projects the difference between -O0 of -O2 in GCC can literally shrink a C++ project from >30kB down to 5kB FLASH with an appropriate runtime speed up as well.Its generally an indicator of poor code if it changes behavior when the optimizer is turned on (or off). The job of a good compiler is to avoid having to manually write optimized assembly to achieve the goals your program needs to achieve.Note that again for race conditions, having slow code enlarges the time window (probability) at which these can occur, so its not the case that optimized or unoptimized code will contain less bugs. They contain the same amount of bugs, they simply show in different ways.

author: gf
author_rank: Super Contributor
timestamp: June 24, 2025, 05:39:58 pm
content: Quote from: EVblog1 on June 24, 2025, 03:42:57 pmThe guarantees and semantics of the C standard are independent of optimization. The standard does not offer additional guarantees or special semantics for non-optimized code.

author: Siwastaja
author_rank: Super Contributor
timestamp: June 24, 2025, 05:48:50 pm
content: Quote from: gf on June 24, 2025, 05:39:58 pmThis, this and this. The usual explanation that volatile somehow "bypasses optimization" is trying to be helpful, but really isn't. It's an unnecessarily deep compiler-internal detail, which is better left not mentioned, than oversimplified.C language is, since late 1980's, a high-level abstract programming language defined upon the terms of observable behavior of the C abstract machine. The whole concept of high-level languages is that programmer writes a formal description (computer program in high-level language) about whatresultthe program is supposed to produce, and the compiler turns it into machine code, trying to do as good job as possible. Doing "good job" can be also calledoptimizing, but it really isn't some separate process.In other words, even if you try to disable all optimizations by modifying compiler parameters, it is still possible that broken code is still broken. Fix: write correct code. That means, using volatile qualifier if it is possible that variable will be modified outside of the usual code flow, or if you need to generate load and store instructions for every access, like you might do with peripheral registers.Besides, optimizations are very necessary. Performance and size are both pretty important. A large number of real-world MCU projects will just become too big that they do not fit in flash memory, or too slow they can't satisfy the timing needs, if you try to compile them with something like -O0 (on GCC).So by disabling optimizations, you can't guarantee broken code starts to work reliably (it might by luck), but on the other hand, you sacrifice a lot of performance. This is why it is usually a good idea to optimize either at -Os or -O2 (or even -O3).Some people find that changing optimization settings is a good way to fight bugs, but I disagree. It is too coarse to even find bugs. And if lower optimization setting successfully hides a bug, then what? You still need to find it - develop good bughunting strategies instead, you will need them.And indeed, volatile only guarantees memory accesses will be generated instead of "caching" (a very loose term) within the compiler. If you need atomicity, then that's a completely separate subject.

author: westfw
author_rank: Super Contributor
timestamp: June 24, 2025, 09:32:22 pm
content: QuoteWhy would thateverbe a preferred solution? "volatile" is correct and more efficient overall.Turning off optimization is likely to be particularly costly on the less-C-friendly architectures like PIC16 and 8051.Also, "no optimization" is likely to be ambiguous and compiler dependent, and I don't think you can count on it resulting in the desired behavior for shared ("volatile") variables. gcc's "-O0" spits out really awful code and will indeed always load variables from "memory." I don't know what Keil does...

author: brucehoult
author_rank: Super Contributor
timestamp: June 25, 2025, 01:41:29 am
content: Quote from: Siwastaja on June 24, 2025, 05:48:50 pmI usually find very little additional benefit in size or speed from anything more than-Oaka-O1on my own code.I don't like the general code size expansion of-O3so if I'm tempted by that I'll either isolate it to specific functions, or hand-code those functions.Never, ever, use-O0(aka no-Oat all). You might as well code in Python. Well, ok ... JavaScript.

author: tggzzz
author_rank: Super Contributor
timestamp: June 25, 2025, 09:08:16 am
content: Prediction: the OP will never reply.Why? They have made 4 posts, all thread starters.

author: Siwastaja
author_rank: Super Contributor
timestamp: June 25, 2025, 10:33:14 am
content: Quote from: tggzzz on June 25, 2025, 09:08:16 amNormal Indian AI training bot. To reply, or not to reply, that's an interesting question. I get a feeling feeding incorrect data to it on purpose would be funny; then again, good replies serve all the others on this forum.I try to avoid answering to very specific questions of these bots, but for something of universal value to others, I do reply.

author: EVblog1
author_rank: Contributor
timestamp: June 25, 2025, 12:55:28 pm
content: Quote from: tggzzz on June 25, 2025, 09:08:16 amYour prediction is wrong I’m right here.Just because I started a two threads doesn’t mean I won’t follow up. I genuinely appreciate the help and I’m actively learning. Also, please try to keep the discussion on-topic it really helps those of us who are here to learnI don’t want to write code without understanding the basics. I’ve seen many people just copy-paste code without really knowing what it does and honestly, I don’t like that approach. I’ve decided to read the datasheet carefully, write the code myself. That’s how I believe real learning happens.Right now, I’m working on setting up a 1ms timer interrupt using the AT89S52 microcontroller. I’m using the Keil compiler along with an Atmel programmer. I’ve been going through the datasheet and studying all three timers to figure out which one would be the best fit but that’s a separate topic.The reason I’m asking about the volatile keyword is because it’s often used when sharing variables between the main program and the ISR.back to topic when we say that the value of a variable can change at any time from outside the program, what exactly does that mean?Does it mean that the program itself won’t change the value, but external factors like an interrupt or hardware peripheral might update it?

author: tggzzz
author_rank: Super Contributor
timestamp: June 25, 2025, 02:05:20 pm
content: Quote from: EVblog1 on June 25, 2025, 12:55:28 pmFourthreads at the time of writing.Your single followup wasafterit was noted you hadn't followed up to any of your four threads.BTW,"volatile" is a frequently misunderstood qualifier; it is not simply and easily understood in a few words.BTW C is full of surprising "implementation defined" and "undefined" behaviour that catches out even experienced users.As an example of that realise that people had to beforcedto realise that in C/C++ "Threads cannot be implemented as a library"https://web.archive.org/web/20230821125225/https://www.hpl.hp.com/techreports/2004/HPL-2004-209.pdfThe resolution of that took a decade(!) and required the introduction of the C memory model (the necessity for that C had studiously tried to ignore for >30 years!).So, if you want to have a multithreaded system (e.g. several threads and/or interrupts) you need to either understand the memory model, or use a threading library, or go outside the C specification.Perhaps that will prompt you to realise why alternatives to C are becoming more popular.

author: eutectique
author_rank: Frequent Contributor
timestamp: June 25, 2025, 03:00:47 pm
content: Quote from: EVblog1 on June 25, 2025, 12:55:28 pmYes. Consider PSW register in 8051 realm. Parity flag indicates even/odd number of ones in the accumulator -- which may or may not change on every command that involves accumulator, completely independently of your program's flow.

author: Siwastaja
author_rank: Super Contributor
timestamp: June 25, 2025, 03:26:20 pm
content: Quote from: EVblog1 on June 25, 2025, 12:55:28 pmYes, external factors like another thread (in multi-threaded programming) writing to that variable, DMA controller, another CPU core, hardware peripheral writing to it.Typical example would be memory-mapped peripheral register. Like GPIO input register, which autonomously changes its value all the time based on the voltage of the pin. You most likely want to read the current value every time you access it in C code. Therefore, these input registers are qualified volatile in the header files supplied by the manufacturer.But also: code where the CPU jumps to autonomously, like interrupt handler (common on microcontrollers, or if writing operating system kernel), or signal handler (if writing userland programs on operating systems; function that is automagically called when user hits CTRL+C in Unix would be typical example). Code flow that isn't visible to the compiler.If you e.g. call a function between accesses to a global variable, compiler will figure out if the variable might be modified in the function, and if unable to do so, assumes it might be modified - no problem, compilers are very good at this and compiler bugs are extremely rare. But if interrupt controller moves the execution to a function, compiler cannot have any idea when that might happen or might not. Therefore, for variables set in interrupt handlers, you may want to qualify them as volatile if you want the effect to be visible to rest of the program. You can of course leave the volatile qualifier out at your own risk if you know what you are doing, or inspect the compiler output yourself.

author: HwAoRrDk
author_rank: Super Contributor
timestamp: June 25, 2025, 04:12:29 pm
content: It should probably be mentioned as part of this discussion of volatile variables that making a variablevolatiledoes not also solve the problem ofatomicity.Atomicity can be a concern when working with 8-bit microcontrollers, as you're frequently dealing with variables bigger than 8 bits - e.g. 16-bit, 32-bit integers, etc.The problem is probably best illustrated with an example:Say you have a global 32-bit integer variablefoo. That variable is set within an interrupt service routine, and read by and used in some calculation by other code in the main loop. You have it declared as volatile, so the compiler won't make any assumptions about its value, and when reading it in the main loop will always read from the relevant memory location every time. But! Your CPU is only 8 bits, so it takes at least 4 instructions to read each byte of the variable's value into registers and work on it (assuming it even has that many registers and doesn't have to expend even more instructions playing musical chairs). That means the CPU is not reading the variable's valueatomically(i.e. all in one go), and there is potential for the IRQ to interrupt half-way through reading the variable's value and change it.That potential sequence of events could go like:1. Value of foo = 0xAABBCCDD2. CPU loads byte 0 (0xAA) into a register3. CPU loads byte 1 (0xBB) into a register**** IRQ, jump to ISR ****4. ISR sets foo = 0xCAFEDEAD**** ISR returns ****5. CPU loads byte 2 (0xDE) into a register6. CPU loads byte 3 (0xAD) into a register7. CPU sees value of the variable as 0xAABBDEAD,not0xAABBCCDDor0xCAFEDEAD!The solution is, when reading the variable any time outside an ISR, to globally disable interrupts (or, at least, the relevant individual IRQ), take a copy of the variable, then re-enable interrupts, and work on the copy.When you're working with 32-bit MCUs, this class of problem tends to go away, as you're infrequently working with variables larger than the CPU's native word size (32 bits), which it can typically work with atomically.

author: Tation
author_rank: Regular Contributor
timestamp: June 25, 2025, 05:59:44 pm
content: Quote from: HwAoRrDk on June 25, 2025, 04:12:29 pmWith 32 bit (or any other size) processor you still have a problem if you want to read a variable, modify it and write it back. If such read-modify-write sequence is not performed atomically, any ISR, thread or core can interrupt the sequence, access the variable and make a mess.volatile will not solve this, neither playing with compiler CLI options. Disabling interrupts can help preventing ISR or other threads from breaking havoc, but will not protect from other cores accessing the variable. For this you need hardware support to ensure that the RMW operation executes atomically, or to know that somebody attempted to access or accessed the variable during it (ARM's LDREX/STREX instructions, LR/SC and AMO instruction in RISC-V,...).Look for LL/SC, if wanting more.

author: tggzzz
author_rank: Super Contributor
timestamp: June 25, 2025, 06:38:59 pm
content: Quote from: Tation on June 25, 2025, 05:59:44 pmThat is indeed one of the problems pointed out in Hans Boehm's paper "Threads cannot be implemented as a library".When I started using C in the early 80s, the two (and only two) books on C were explicit that multithreading was outside the scope of the language. That seemed to be wished away by practitioners of all degrees of competence. They relied on code libraries that were compiler and machine specific, andcould not be expressed in C.10 years later (in 1994/6) Java addressed the issue explicitly, and hence became the language of choice for new heavily multithreaded applications. Just as well since multicore processors were becoming commonplace.For another 10 years C had its head in the sand, being more concerned with whether "casting away constness" was obligatory or forbidden. Finally, in 2004 Boehm felt it necessary to "remind" people about that multithreading hole. Boehm also pointed out what was necessary: a memory model.Eventually, afteryet another10 years had passed, C finally gained a memory model - 20 years after Java had nailed the issue.And C people claim they are the most knowledgeable wizards around!

author: westfw
author_rank: Super Contributor
timestamp: June 26, 2025, 05:12:08 am
content: QuoteSo what?Why is it such a big deal to require a small core of machine-specific assembly language to implement some feature?Back in the old days, we took it for granted that there were things that could not be done within the limits of the "preferred" language. Sometimes your fortran program had to have code written in assembly language, or your BASIC program had to POKE some hex code into memory and then CALL it. It was no big deal...

author: Siwastaja
author_rank: Super Contributor
timestamp: June 26, 2025, 06:50:40 am
content: tggzzz's posts are perfect example of the discrepancy between theory, and the real world.In real world, no one is using Java for "new heavily multi-threaded applications" anymore (if they ever did) and Java is notorious for its poor performance. In the past at university, for half a year, I had to try to maintain a server project written in Java which did absolutely trivial amount of processing (like hundreds of bytes per second from a handful of connections!) and was not only leaking memory (like Java applications so often do), but alsoleaking threads, and we had a supervisor process which killed and restarted the whole thing when thread count exceeded 200. The whole thing was written in Best Java Practices (problemFactoryBeanFactory etc.), and was completely unmaintainable.Of course, the whole thing would have been obvious in single-threaded C application using select() or something similar, and fewerlinesof code total, than thefile countof the Java project.Most people are very happy to see Java disappear. Like communism, it had many really good ideas, but did not work in practice, for whatever reasons that would be interesting to analyze but outside the scope of this discussion, because, like many wise men say, practice trumps theory any day. Meanwhile, C might be inelegant and "threads cannot be implemented as a library and constness should not be cast away" or whatever, but in real world C is used A LOT for performance critical multithreaded applications, and threads are implemented as libraries all the time and people seem to deal with this just fine, except those who do not actually write these programs but instead only talk about them on internet forums.

author: brucehoult
author_rank: Super Contributor
timestamp: June 26, 2025, 09:15:28 am
content: Quote from: westfw on June 26, 2025, 05:12:08 amThe big deal is getting people to accept that a few instructions of asm are needed, not the difficulty of writing them.It's on the same level as "a C program can't set up its own stack". And, in fact, you hit exactly the same thing as part of thread creation too -- you can *almost* do a cooperative threading library in portable C, using setjmp/longjmp, but not quite. You need to know where in the jmp_buf the stack pointer is stored, and overwrite it. The PC too. "Volatile" handles the rest.Preemptive threads needs a bit more, including a way to do critical sections, but mostly that they have to save and restore ALL registers and some other state, not just the ones preserved by function call/return.Also, when I did this on Windows I had to write my own entire setjmp & longjmp because the standard one had a whole lot of extra SEH bollocks that also needed patching (or ignoring) for new threads.

author: shapirus
author_rank: Super Contributor
timestamp: June 26, 2025, 09:24:22 am
content: This is interesting.I had a case where I had to disable compiler optimizations for parts of the code using gcc-specific inline instructions to prevent the compiler from optimizing out the "write-then-read" sequence (or double reads, I don't remember it clearly now) when accessing memory directly in a GPIO pin bitbanging scenario.Apparently the canonical way of solving it would be to use the volatile modifier. Live and learn. I will need to check it if I ever touch that code again.

author: tggzzz
author_rank: Super Contributor
timestamp: June 26, 2025, 09:30:15 am
content: Quote from: westfw on June 26, 2025, 05:12:08 amAll that is very true, and is exactly what we did.Only after 30 years of sweeping the problems under the carpet did the C standards finally evolve to the point at which that was no longer necessary.C is a very complicated language, with many nooks and crannies. Hence of necessity beginners are taught (often badly!) about small parts of C. They then begin to expand from that small core onto "advanced" topics like volatile etc - and generate code which seems to work but has many subtle unrepeatable problems.If you have a good way of quickly forcing beginners to realise the inadequacy of the bits of C they have been taught, I'd like to know. Shockingly IMNSHO Hans Boehm had to forceexperts into understanding that. nominal and/or self-proclaimed experts on the various committees

author: tggzzz
author_rank: Super Contributor
timestamp: June 26, 2025, 09:38:56 am
content: Quote from: Siwastaja on June 26, 2025, 06:50:40 amYou are joking, aren't you?!You can only make that assertion by ignoring telecom systems and enterprise systems.QuoteA university project written by students that wasn't up to the job? Who would have thought it.Fortunately every C/C++ university project works perfectly and is not based on anti-patterns. Perish the thought.QuoteI have written many such applications over the decades. Of particular relevance is that one of my soft realtime Java telecoms applications blew the equivalent C/C++ application out of the water in terms of performance.

author: tggzzz
author_rank: Super Contributor
timestamp: June 26, 2025, 09:41:06 am
content: Quote from: brucehoult on June 26, 2025, 09:15:28 amPreciselyThere is too much of the attitude that "C is both necessary and sufficient". Neither is true.

author: shapirus
author_rank: Super Contributor
timestamp: June 26, 2025, 10:07:19 am
content: Quote from: tggzzz on June 26, 2025, 09:30:15 amNo. It is as simple as it gets (besides assembler and machine code). It allows you to tell the computer what you want it to do in very simple terms, and it does precisely that. It allows the programmer to understand what he is doing, but it also requires the programmer to understand what he is doing. C is almost perfect, but only for those who can express their thoughts and goals clearly and coherently. Not all people can do it, so other languages have become a necessity.edit: crossed out a logically incorrect conclusion.

author: SteveThackery
author_rank: Super Contributor
timestamp: June 26, 2025, 10:24:05 am
content: Isn't FreeRTOS written in C? Does that count as a library?

author: tggzzz
author_rank: Super Contributor
timestamp: June 26, 2025, 10:39:28 am
content: Quote from: shapirus on June 26, 2025, 10:07:19 amI don't think you do know as much as you think you know.Here's one document illustrating one aspect of the inconsistent "definitions" in C.N.B. MacLaren really is/was an expert: his job was to help highly intelligent people get the maximum performance from their C code. Hence he knows where the skeletons are buried, since he kept on uncovering them.N.B. where this was presented and discussed: the standards committee, who can be expected to know more than average or good practitioners.N.B. this is a far more fundamental question than mere modifiershttps://www.open-std.org/jtc1/sc22/wg14/9350What is an Object in C Terms?This is asking the question "What is an object when it is at home?" Weneed to start with an explanation of why a precise answer to thequestion is so important....The vast majority of commercial implementors and important users are notmembers of SC22/WG14 or even any national body affiliated with it, andwork solely from the standard. As they should, because ISO rules (andall normal commercial practice) specify that the standard is theofficial document, and records of its development should be used for atmost clarification....I can witness that this area has caused major headaches ever since C90was developed, and that many  implementors are very seriously unhappyabout C99.They simply do not know what most of it implies and, as isgenerally agreed, the subtleties in this area are critical for bothoptimisation and the development of robust applications.Most of theones that I have contacts with had severe problems deciding exactlywhat optimisations were allowed in C90, and often went through severaliterations before and industry consensus emerged.<many examples omitted>...Nick Maclaren,University of Cambridge Computing Service,New Museums Site, Pembroke Street, Cambridge CB2 3QG, England.Perhaps things were completely unambiguously clarified in later standards, butgiven the starting pointI strongly doubt it.

author: eutectique
author_rank: Frequent Contributor
timestamp: June 26, 2025, 11:32:58 am
content: Quote from: SteveThackery on June 26, 2025, 10:24:05 amYes, with the kernel functions implemented as__asm volatile()or#pragma inline_asmpatches.https://github.com/FreeRTOS/FreeRTOS-Kernel/tree/main/portable

author: tggzzz
author_rank: Super Contributor
timestamp: June 26, 2025, 12:29:24 pm
content: Quote from: eutectique on June 26, 2025, 11:32:58 amSurely that means itisn'twritten in C.

author: brucehoult
author_rank: Super Contributor
timestamp: June 26, 2025, 12:36:43 pm
content: Quote from: tggzzz on June 26, 2025, 12:29:24 pmOnly 99%.

author: tggzzz
author_rank: Super Contributor
timestamp: June 26, 2025, 01:20:26 pm
content: Quote from: brucehoult on June 26, 2025, 12:36:43 pmPreciselyYou could achieve that with many other languages.

author: ejeffrey
author_rank: Super Contributor
timestamp: June 26, 2025, 02:30:57 pm
content: Quote from: westfw on June 26, 2025, 05:12:08 amThat is the opposite of the point.The point of the article is that you can't just write a few assembly routines to create threads and implement locks, at least not if you want it to be guaranteed correct. You need the compiler to be thread aware when generating any code that might be run in a multi threaded environment. And you need your language or platform to specify what operations are allowed.Just implementing threads as an assembly library almost always works but it had some uncommon but severe bugs in the 90s that were the result of compilers not being thread aware. By the early 2000s compilers had mostly fixed these bugs, implicitly defining a memory model but it took to 2011 for the standard to catch up.And the benefit of a proper memory model in the language was that we could go beyond locks and use atomic operations and lock free algorithms. Because while the library approach mostly works for locks it really falls down for lock free algorithms.

author: hans
author_rank: Super Contributor
timestamp: June 26, 2025, 02:45:00 pm
content: Quote from: shapirus on June 26, 2025, 10:07:19 amThats exactly the point why its so complicated. Easy to learn, hard to master. There are so many traps to fall into (implicit type casts, undefined behaviour, etc.). Things like pragmas or __attribute__((naked, used)) is typically a clear indicator we're doing something that is not part of the vanilla C spec. These definitions are different per compiler, and their implementation may differ per architecture too. A library such as FreeRTOS has some bookkeeping functions that are mostly C, but are sprinkled with in-line assembler constructs (such as critical sections), and the context switch routine is (almost?) always pure assembler.C code is dragged into situations where it was not "designed" for. However, stretching C up this thin is a necessary evil in the embedded field to achieve the things we want to (e.g. work directly with hardware). On a MCU, there is literally not much of a choice to serve an IRQ: its either served in time with correct behaviour, or the system may just as well not even exist.If that then means certain specifications must be met (such as making functions reentrant), then we must resolve race conditions, inform the compiler that a variable may have writers that are out of scope, etc. And as often is the case with the 'lean abstraction' approach in C, it means that many programmers must reinvent their own wheels. I don't mean this as a critique like "C programmers are stubborn, use C++/Rust/etc. its so much fancier", its a necessary evil to have low overhead to run code on smallest of microcontrollers.Languages like C# or Java have much easier to write code because they can incur the high overhead penalty on desktop computers. These languages knew their added value was in this aspect, and thus first focused for decades on a complete language specification and memory model, and are incrementally trying to improve performance (e.g. with a better JIT). Its funny because C# has the volatile keyword as well, but when I searched for it on StackExchange, I came across a typical desktop software engineer answer as: "don't use it, you must be doing something crazy -- use ConcurrentQueue instead"Only now their language specification has become mature, are they finally focusing on overhead and performance. C# has received major speeds up to their LINQ library for example, where up to 50x speed ups for some functions is possible.

author: shapirus
author_rank: Super Contributor
timestamp: June 26, 2025, 02:53:57 pm
content: Quote from: hans on June 26, 2025, 02:45:00 pmCorrect. That is why I remarked about knowing what you're doing. As a tool, C is still simple, but situations you can end up in while solving problems with it can be complicated indeed.Of course it's not a universal tool. Yes you can implement anything in C, but sometimes there are better tools that can save time and/or prevent human errors (typically at the expense of something, of course).

author: Siwastaja
author_rank: Super Contributor
timestamp: June 26, 2025, 03:15:00 pm
content: This discussion always runs in circles.> "Systems programming in C is ugly!"< "OK, suggest something else"> "Blah blah blah" (not capable of suggesting anything)or> Suggests a language completely incapable of any systems programming and only for that reason, lacking those ugly constructsor> Suggest an even more complex language with 10 new drawbacks for each 1 fixed thing.C is used because it does the job, is nottoougly for them who actually use it, and due to failure for anyone to really suggest something better without new sacrifices. It appears that with any new alternative (e.g. Rust), comes some disaster (Pythonesque library version dependency hell for example)

author: Nominal Animal
author_rank: Super Contributor
timestamp: June 26, 2025, 04:05:03 pm
content: I habitually compile my own C code with-Wall -O2, sometimes using-Wall -Wextra -O2. I never, ever need to "disable optimizations".I sometimes do useunsafeoptimizations that let the compiler generate faster code but no longer follow the C standards exactly, but the-O2(or-Osor-Og) options never enable such by themselves.Quote from: EVblog1 on June 24, 2025, 03:42:57 pmAs explained above,volatiletells the compiler it cannot infer the value of thevolatileobject, and therefore must do the actual access.Another similar issue wrt. optimization is the precision and range of numerical expressions. If you want to calculatex * y, but limit it to the range and precision ofunsigned char, all you need to do is a cast:(unsigned char)(x * y). You do not need a temporary variable at all.The third issue, common on 8-bit architectures, is non-atomic accesses. As detailed above by explained above by HwAoRrDk, it occurs because the object is read from or written to in smaller parts, and the interrupt can occur mid-way during the write. The solution is to temporarily disable interrupts, do the access, and then re-enable interrupts. Typical code implements that assaving the interrupt state, then restoring it, so that if you already have interrupts disabled, you don't enable them too early. On 8051, you should have a function that saves the Interrupt Enable register, then disables interrupts globally, and another that restores interrupts.I don't use Keil, so I don't know what they actually are for OP. If we assume they are, for simplicity,unsigned char save_interrupts_disable(void); // Returns current Interrupt Enable register, disabling interrupts globallyvoid restore_interrupts(unsigned char); // Set Interrupt Enable register to a previously saved valuethen it would be best to useaccessor functionsto manipulate the variables (larger than 8 bits) used in both the interrupt and other code:static volatileTYPEshared_variable;static inlineTYPEget_shared_variable(void) {unsigned char state;TYPEvalue;state = save_interrupts_disable();value =shared_variable;restore_interrupts(state);return value;}static inline void set_shared_variable(constTYPEvalue) {unsigned char state;state = save_interrupts_disable();shared_variable= value;restore_interrupts(state);}static inlineTYPEreplace_shared_variable(constTYPEnewvalue) {unsigned char state;TYPEoldvalue;state = save_interrupts_disable();oldvalue =shared_variable;shared_variable= newvalue;restore_interrupts(state);return oldvalue;}The variable is declaredvolatileexactly because it can be changed from an interrupt context at any time, so the compiler must not infer its value at any point, and must do explicit accesses to it.Thesave_interrupts_disable()function is used to stop interrupts from occurring during the access, so that even if the variable is copied piecewise, it will not be interrupted, and will thus beeffectively atomic.Therestore_interrupts()restores the interrupt enable register, so that these functions can be used from both normal code, as well as from within an interrupt handler.You could open-code the interrupt blocking for the access duration, but that is error-prone.Finally, thestatic inlinedesignation tells the compiler that this function will only be used in the current compilation unit (.c source file), and that the compiler can inline the code (copying it) instead of generating a function and calls to that function. Because the accessor functions are only available in the current compilation unit, it makes sense to limit the shared variable to the current compilation unit, therefore the shared variable is declaredstatic.C11 and later did introduceatomic typesvia#include <stdatomic.h>, but I do not know if Keil supports those, implements them internally via implicit interrupt disable/restore like above, or does something else.

author: tggzzz
author_rank: Super Contributor
timestamp: June 26, 2025, 04:32:57 pm
content: Quote from: hans on June 26, 2025, 02:45:00 pmIn the late 80s and early 90s C came to a fork in the road: to be a high-level language suitable for general purpose applications or to be a low-level language close to the silicon. Either choice was possible and valid, and a good language could have been created from either objective.Unfortunately the C fraternity had a limited background (to exaggerate, operating systems on PDP11) and were blind to experience and knowledge gained in other domains. Consequently they naively presumed C was The Answer, and decided they didn't need to decide. The language thus mutated into something that was unnecessarily poor for both for both high and low level systems. In trying to be all things to all people, it ended up as being substandard in all domains.QuoteFor modern applications on modern computers, C compilers have to bepessimisingcompilers. That's a consequence of its roots in single-thread single-core systems, a perfectly reasonable assumption half a century ago. In order to try to get around those inhibitions, all sorts of subtle nasal daemons have had to be introduced, essentially of the form "I know the compiler can't determine X, but since I and the people who use and will maintain my code are all wizards like me, you can trust us all when we say we haven't violated X".As for performance, it isn't so clear cut, as was demonstrated by HP's Dynamo compiler project, and is visible in the various "profile guided optimisations" added to some compilers. Java has such profile guided optimisations built in; they are called "HotSpot".QuotePrecisely.Exactly the same is true for Java.QuoteC# (like C) can only base optimisations on what can be proved from the source code. Java's HotSpot, OTOH, optimises on what the code is actually doing at run time, and hence can make many more optimisations. I wonder whether the performance optimisations you indicate are derived from HotSpot-style profile guided optimisations.

author: tggzzz
author_rank: Super Contributor
timestamp: June 26, 2025, 04:40:30 pm
content: Quote from: Siwastaja on June 26, 2025, 03:15:00 pmC is like COBOL. It continues to be used because teachers (think they are) familiar with it and compilers are ubiquitous - not because it is inherently the best language.Rust isn't perfect by any means, but it is a much better foundation for computing on modern multicore machines.C will continue to underlay many systems, just as COBOL continues to be the basis of many systems. But the future belongs to other languages. Time to stop mangling undergrads minds with C's IB, UB etc, and let them concentrate their minds on more interesting and important problems.

author: Siwastaja
author_rank: Super Contributor
timestamp: June 26, 2025, 06:22:49 pm
content: Quote from: tggzzz on June 26, 2025, 04:40:30 pmThis is a ridiculously detached-from-reality argument. Maybe in 1980's this was true. C was nearly totally abandoned by the academy already in late 1990's, and actually one of the biggest problems of C is that it isnottaught properly anywhere, so most academic literature about it and code examples are seriously outdated.In 2005, we were taught C++; many other universities used Java as their primary teaching language. Today I'm sure Python is very popular also in teaching.QuoteDid you ever realize that the fact that "modern replacements" do not have IB and UB in the standards is because either there is no standardat all(Rust), or standard is so poorly thought out that such concerns are just blatantly ignored? Compilers just do whatever they want; even Java was very well infamous for "write once, debug everywhere". In C at least you can blame yourself for doing IB/UB.IB and UB are not nice things and ideal, perfect language would have none, but at least they are formally documented. Along with the hated preprocessor, having a decent standard is one of the technically strongest points of C.

author: tggzzz
author_rank: Super Contributor
timestamp: June 26, 2025, 06:49:06 pm
content: Quote from: Siwastaja on June 26, 2025, 06:22:49 pmCome to think of it, I suspect you are right. I haven't had contact with university output since then.Nonetheless, the rest of that sentence stands and is supported by that observation, viz: C is like COBOL. It continues to be used becauseteachers (think they are) familiar with it andcompilers are ubiquitous - not because it is inherently the best language.I ought to remind people of the old aphorism "science advances one funeral at a time", and apply it to computer "science".QuoteRust is indeed imperfect, and IMHO it is yet to be proven that it will succeed in the long term.Nonetheless, I bet that it will succeed in the long term, because it has significant advantages in current and future computing environments, and there is significant momentum behind it and improvements are being made.If I had my time again, I would learn Rust in preference to C.QuoteThey are also subtle and pervasive that "defining them" amounts to little more than painting "keep out" on a door in a poorly maintained industrial estate.C has an indecent standard; standards should be sufficiently good that they can be comprehended by mere mortals. Everything I've seen (e.g. see ref above) indicates that experts have great difficulty understanding what they mean, and even difficulty understanding that they don't have a deep understanding.

author: Nominal Animal
author_rank: Super Contributor
timestamp: June 26, 2025, 08:21:21 pm
content: Quote from: Tation on June 25, 2025, 05:59:44 pm8051 architecture does not have that; it is much closer to compare-exchange than load-linked store-conditional.For single bytes in internal data memory (0x00..0xFF) loads and stores (MOV), increments (INC), decrements (DEC), and exchange/swap (XCH) between data memory byte and the accumulator are atomic.For external memory (0x0100..0xFFFF), only single byte loads and stores are atomic.

author: SiliconWizard
author_rank: Super Contributor
timestamp: June 27, 2025, 02:49:27 am
content: On the simplest, single-core architectures, the usual way of implementing atomic operations when there's lack of specific instructions is just to disable interrupts / instructions that must execute "atomically" / enable interrupts (if they were enabled before), as the only way of interrupting instruction flow is precisely from interrupts. Sounds barbarian, but that works and that's pretty much how atomics are usually implemented, for instance, on the Cortex-M0.Fun fact: this approach is not necessarily less efficient than using LL/SC approaches (when available), sometimes it's just the opposite. It all depends, but something to be aware of.

author: brucehoult
author_rank: Super Contributor
timestamp: June 27, 2025, 05:49:11 am
content: Quote from: SiliconWizard on June 27, 2025, 02:49:27 amIt's not really comparable.LL/SC is specifically for multiple cores and in fact will not provide exclusion on a single core unless the task-switching code takes steps to invalidate any reservation that is currently held by doing an SC to a scratch location.On the other hand, if you have two 6502s or z80s or something on the same memory bus then doing CLI on one will not prevent the other CPU from interfering in an atomic operation. You'd need some other external mechanism perhaps detecting RMW instuctions (which both CPUs have e.g. INC/DEC) or a mempry-mapped latch that each CPU could write to to pause the other CPU's, or pause if it tries to write, or write to a specific address.

author: cv007
author_rank: Frequent Contributor
timestamp: June 27, 2025, 06:55:59 am
content: QuoteWorks fine for others also, whether an M33 or an M0 or an avr. I would guess most here are not touching multiple core mcu's, and those that are most likely are using standard methods provided to them to deal with those type of problems.The following are a couple simple examples of what that looks like using C++, and becomes quite easy to use. The interrupt status is saved and disabled at the point of object creation, and is restored at the end of the scope which could be a function, or braces within a function. I your function is on the long side, you can restrict the time interrupts are off by using curly braces around just the code needing protection, or if the function is short just protect the whole function.This also points out where 32bit in a way gets harder-you no longer have a single interrupt level, so 'outside the ISR' thinking no longer applies. If you want to be able to adjust interrupt priorities at any time throughout a project you will not be able to assume you know when interrupt protection is needed so will have to assume it is always needed. Maybe with 4 run levels you could keep track of things up to a point, but move to 16 and you are not going to want to figure it out every time you make an adjustment to priorities.The systick example has a u64 for systemMS_ which updates every second. Now, if some higher priority isr wants to get a time value (via another function in the Systick class) its irq could have fired between the u64 write and the time function could be using an incorrect systemMS_ value. In this case its only a once in ~49 day possible problem but no matter- you simply protect things that need protection and are not making probability calculations with a cutoff point.A different kind of thing exists in the second example for an stm32. Even though writes to gpio registers are 32 bit (or 8,16) you still have shared registers. If you want to set a pin to analog for example, or turn on/off a pullup, etc., and do not want to worry about where in your code this is done (various run levels/priorities) you will want to protect those shared registers otherwise you have the possibility some higher priority code 'losing' the changes it made to the lower priorty code which was interrupted in the middle of also changing a pin on the same port,In both cases what needed protection was protected, and you have eliminated potential bugs whatever the probabilities were.Code:class Systick {...static inline u64 systemMS_; //no overflow possible in anyone's lifetime//get 0-999ms from counterstatic voidisr () //irq rate 1/sec{InterruptLock lock;systemMS_ += 1000; //is a u64, so irq protect the write}static u64now (){ /* return systemMS_ + computed 0-999ms from counter */ }...};class GpioPin {...autoanalog (){CPU::InterruptLock lock;deinit(); //default state, which is analogreturn *this;}...};Note- the Systick now() function is dealing with a running systick counter so interrupt protection will not help, but is easy enough to check the counter before and after the systemMS_ read to see if a rollover occurred. One can also check if systick irq is pending first (could be called with irq's disabled) and manually call the isr first if so.

author: peter-h
author_rank: Super Contributor
timestamp: June 27, 2025, 09:06:32 am
content: FreeRTOS (I use it) is written in C but has a few bits of asm where stuff is not possible in C e.g. direct stack manipulation.I wrote a simple RTOS in the 1980s and did the same thing there.We have some long threads on optimisation. I use -Og because one can still debug quite well and the result is not significantly worse than -O3. And developing in say -Og and releasing with -O3 is bad risk management because you lose all the regression testing that was naturally done during development.

author: Siwastaja
author_rank: Super Contributor
timestamp: June 27, 2025, 09:09:38 am
content: Quote from: tggzzz on June 26, 2025, 06:49:06 pmWe hear you: C is horrible and no-go, because the standard has a few special edge cases which areundefined, meaning that compiler writers can do whatever they want with a few select constructs, and programmers should avoid those.In Rust, on the other hand,every construct is undefined or implementation-defined behavior, meaning that compiler writers can do andwill dowhatever they want, changing with every version, and programmers should, by the same definition, avoid the whole language, but for you that only means it's imperfect, but something you could consider learning.I enjoy double standards.

author: Siwastaja
author_rank: Super Contributor
timestamp: June 27, 2025, 09:21:19 am
content: Quote from: peter-h on June 27, 2025, 09:06:32 amAbsolutely. You should be optimally delivering the same binary that was used in testing; regardless if it's automated or manual testing.Which is also why I dislike the test-with-debugger idea. As the name suggest, debuggers are designed for... debugging. Finding bugs that are otherwise very hard to locate.I strongly suggest building in self-verification, self-testing (asserts or similar) and logging features in the firmware itself. Those work the same way both in "development", and then later on the field. After you used the same concepts, same tooling during development, and problems still sometimes arise on the field, it is then super easy to go into your mental "developing mode" - extract logs, maybe add more logging - and it's an advantage if you are not limited to the physical debugger interface like SWD/JTAG, but can do that even remotely.There is some size overhead from checking and logging, but so there is size overhead from -Og. But code that checks and logs offers way more, than just -Og. Of course, you can have both.

author: cv007
author_rank: Frequent Contributor
timestamp: June 27, 2025, 10:45:56 am
content: I start out at -Os, stay at -Os, and never move out of -Os. If there is ever a need to understand generated asm code that optimization makes difficult to read, I will make the function temporarily noinline so will be able read it easier and understand if it is doing what I think it should be doing. Using a single optimization from start to finish means you are dealing with a compiler that does not change the way it thinks so you get a consistency you don't get by moving around in optimization. When reading generated asm for example, you are looking at a compiler with one personality instead of three or four.Simple example -Os vs -Og-https://godbolt.org/z/G4bq6WfGvWhether that is a great example or not, the -Og version will take a long time to get through, and as soon as you went to -Os the whole loop basically disappeared. Which points out at minimum you are introducing changes in timing. When using -Os from the start, you will notice when optimization has unintended results (optimized away) at the point this code is fresh in your mind, and not a month later when you decide you need -Os and have already accumulated a dozen such problems.

author: Nominal Animal
author_rank: Super Contributor
timestamp: June 27, 2025, 11:43:11 am
content: Quote from: SiliconWizard on June 27, 2025, 02:49:27 amYou mean, like the example I showed for OP inreply #34?

author: brucehoult
author_rank: Super Contributor
timestamp: June 27, 2025, 12:05:34 pm
content: Quote from: cv007 on June 27, 2025, 10:45:56 amWhy unintended? You wrote code that does nothing except set a to 0. The optimised code has precisely the same effect, but faster. That's what you WANT.If you actually wanted a to be stepped through all 4 billion values then you'd slap a "volatile" on it.To be honest with a non-extern variable I'm surprised it didn't just delete the useless variable entirely.

author: Siwastaja
author_rank: Super Contributor
timestamp: June 27, 2025, 12:35:52 pm
content: Quote from: cv007 on June 27, 2025, 10:45:56 amThis is a sane advice in my experience for at least AVR and ARM Cortex-M<anything> targets. Sometimes I test attributing some specific speed-critical functions to -O2 or even -O3, but probably half of the time I get back to -Os even with them when I measure that the time difference is negligible, and size difference significant. Sometimes -O3 gives worse performance but almost always a very significant size increase.Most important is to stay away from -O0, and avoid the thinking pattern that adjusting optimization level is useful for any other purposes (e.g. debugging) than finetuning size - speed relationships when performance matters, and sometimes it does.

author: cv007
author_rank: Frequent Contributor
timestamp: June 27, 2025, 12:50:46 pm
content: QuoteIt was an example to make a point that you can go from intended results to unintended results with a simple optimization change. If you had wanted a loop to kill time of course volatile is the answer, but if you had something similar in place which was not correct, but was acting perfectly fine while you sat on -Og for a month, as soon as you go the some other optimization all your code that acted fine may change to something you didn't expect. If you had been on -Os from the start, that incorrect code would never have produced the results you wanted and you would be forced to deal with it at the time of code creation.QuoteSeems odd it didn't disappear, and if you remove the static the code doesn't change either. Sort of my point- for this simple example, even though pointless code, ends up producing about 3 different versions with the various optimizations. Why not deal with just a single version, always.

author: brucehoult
author_rank: Super Contributor
timestamp: June 27, 2025, 03:39:32 pm
content: Quote from: cv007 on June 27, 2025, 12:50:46 pmThat's where you're going wrong. That is not a valid use of a high level language. If you want to write assembly language then write assembly language, don't abuse C.If you want a delay then use a timer. Or write in assembly language if your CPU gives execution time guarantees.Quote"acting" perfectly fine doesn't mean it's correct, only that you got lucky. Today, a particular optimisation level on a particular compiler might seem to work. But a different compiler, a different machine, even a new release of the same compiler — any of of those can expose your buggy code for what it is.

author: SiliconWizard
author_rank: Super Contributor
timestamp: June 27, 2025, 03:55:21 pm
content: Quote from: Nominal Animal on June 27, 2025, 11:43:11 amYes, pretty much.

author: tggzzz
author_rank: Super Contributor
timestamp: June 27, 2025, 07:09:40 pm
content: Quote from: Siwastaja on June 27, 2025, 09:09:38 amThe problem is that the edge cases and "surprising" cases are commonly encountered and commonly trip people up. And even if they don't trip people up today, the next compiler version may have "better" optimisation. Solution: fix on a compiler version, and don't change it during the projectmaintenance/enhancementperiod.Other languages manage to avoid that.

author: tggzzz
author_rank: Super Contributor
timestamp: June 27, 2025, 07:12:09 pm
content: Quote from: peter-h on June 27, 2025, 09:06:32 amThe compiler flags should be regarded as part of the source code. Ditto compiler version.

author: westfw
author_rank: Super Contributor
timestamp: June 27, 2025, 10:22:28 pm
content: QuoteYes, that sounds good.How do we feel about "Link Time Optimization" ("-flto" for gcc; something else for the "pro" versions of MCC, I think.)It's sure nice in its results on program size, and in simplifying the amount of thought you have to apply "up front" in factoring your libraries/etc. But it can REALLY obscure the object code!

author: brucehoult
author_rank: Super Contributor
timestamp: June 27, 2025, 11:58:54 pm
content: Quote from: tggzzz on June 27, 2025, 07:12:09 pmDitto OS version, libraries depended on, ...

author: tggzzz
author_rank: Super Contributor
timestamp: June 28, 2025, 12:02:23 am
content: Quote from: brucehoult on June 27, 2025, 11:58:54 pmLibrariesalmostgoes without saying.I didn't include OS version since I was thinking of cross-compiling for an MCU. Probably confusing two independent current threads; if my daughter was part of this conversation she would ask me what day of the week it is and who the PM is

author: cv007
author_rank: Frequent Contributor
timestamp: June 28, 2025, 12:24:55 am
content: QuoteYou seem to be agreeing with me. Creating incorrect code is not an unusual thing, nor is it that unusual to get what appears to be correct results from incorrect code (and could be because of optimization setting). I'm advocating using a single optimization, particularly -Os, so you get incorrect results from incorrect code, sooner. Or more accurately you get consistent results, whatever your code correctness may be or what optimization you choose.The example was only to point out extremely different results from different optimizations with just a few lines of code, not about how to produce delays. I'll try another example and I know that I'm doing it wrong, that is part of the point, I also know a delay in main does nothing useful or that this code would typically be different-https://godbolt.org/z/nW6P661x1in this case if you were using -Og you would get correct results with incorrect code, and when you move from -Og to anything else it will break (in different ways- -O1 will block forever, and -O2/Os will simply optimize completely away). The code IS incorrect, and if you had started at -Os, or in this case anything other than -Og, you would have realized your mistake right away. The point, again, is not the specific details of the example or at what optimization setting the code starts to break, but that optimizations are producing radically different results. Typically -Os gets the compiler to 'critique' your code the most, of course not directly via error or warning messages, but by actual results. Just as I want -Wall and -Werror, I want the compiler to use its optimizations to reveal any code that cannot stand up to those optimizations. Right now, not a month later.Now, these simple/small examples may not be exact representations of real code, so instead take a project that is of some size and you now have a lot of code so there are much more opportunities to make a mistake that only a higher optimization may reveal. No one is going to switch to a higher optimization just as the mcu goes out the door no matter how confident one may be in the code, so why not pick a final optimization setting when a project starts and stick to it.QuoteProbably depends on how little flash you have or how close you are to filling it with little to no options for an upgraded mcu. It seems to be less of a problem in recent years finding mcu's with enough flash to not get overly concerned about filling them up and I would guess at least in the 32bit world most users are not pushing the limits of flash storage except for maybe those that are in the small range such as 4/8K.I have an stm32 project up that is 6.8K (-Os), added flto and it compiles to about 6.7K. Probably not a good representation as there are only 2 cpp source files and all the rest are headers (cpu, mcu, nvic, gpio, print, i2c, dma, systick, uart, and so on). I guess its already mostly all-in-one.

author: brucehoult
author_rank: Super Contributor
timestamp: June 28, 2025, 12:38:31 am
content: Quote from: tggzzz on June 28, 2025, 12:02:23 amAny given compiler generally needs a specific range of OS versions and std libs to even run.

author: Siwastaja
author_rank: Super Contributor
timestamp: June 28, 2025, 05:32:11 am
content: Quote from: tggzzz on June 27, 2025, 07:09:40 pmIt's nice to have strong faith...

author: hans
author_rank: Super Contributor
timestamp: June 28, 2025, 07:10:09 am
content: I agree with cv007 statement about finding things that fall over now instead of later.You can keep optimization levels the same during development, but that only works when compiler versions are also kept the same. There a ton of tunables behind optimization levels which can be changed at any moment. If you're using the system compiler which gets updated, then that's already not the case.For my hobby projects, I've a Jenkins CI environment that checks if all unit tests still pass, and then runs almost all said test cases through a special emulator environment I developed. Its based on RISC-V (either rv32i or rv32im) and basically runs each test case in isolation to monitor for regressions in speed and code size as well. I run these cases against an array of optimizations I'm most interested in: -O0, g, s, 2 and 3. The graphs I post has left, runtime in cycles, and right, image size in bytes.I personally use -O2 the most. The reason is a balanced optimization in cases where -Os can get really slow. In my experience I often don't need the extra FLASH and I'd rather not incur massive runtime penalties.Now most code I write is trying to be as insensitive as possible to optimization level (not looping over uninitialized variables etc.). This is the majority of my code base: boring statistics of Os being faster/smaller than Og, and O2 often beating Os speed by a tiny bit (and O3 not doing much at all).However, some code is much more prone to poor/good optimization than others. In particular in C++ with virtual methods, or in C with some function pointers, the compiler will not perform devirtualization on level Os whereas it will on O2 and O3. For my GUI/display chipset stack, this results in quite a performance hit for framebuffer manipulations.Another case is in my CircularQueue object which can take any object type and create a circular storage for it. The copying in/out of the object type is done via the assignment operator. In -Os, this will typically be fine, but then when I test a 17-byte object I get a bytewise memcpy that is very very slow. Even -Og is fine with this, so -Os shows a massive regression in performance.Now these graphs are microbenchmarks so not best evaluated for a whole application, but you can do the same tests (manually) for different optimization levels and reach simillar conclusions.

author: brucehoult
author_rank: Super Contributor
timestamp: June 28, 2025, 07:59:03 am
content: Quote from: hans on June 28, 2025, 07:10:09 amNice setup.You missed my default, -O aka -O1. I've never included -Og in my own comparisons, but I think -O does do some size/speed optimisations that -Og won't. But I don't have anything against -O2 and take your point about C++ devirtualisation.I've sometimes found, especially on RISC-V, that the -Os heuristics aren't all that well tuned -- especially because gcc is NOT aware of the C extension -- that you can surprisingly often end up with fewer instructions but larger code than -O2. But you won't see that effect with rv32i or rv32im.I'm not aware of any commercial microcontrollers that don't have the C extension (even the 10c CH32V003 is rv32ec), so I guess you're using an FPGA soft core? It gives more code size reduction than -Os with less (or zero) speed cost.

author: cv007
author_rank: Frequent Contributor
timestamp: June 28, 2025, 10:54:20 am
content: QuoteIs this simply the result of using packed structures?This example shows when packed is used then memcpy is called, and would assume it will resort to byte copy since the destination address will/could be unaligned (I think BIGBLOCKSIZE is 16 bytes so byte copy would not be due to LEN). Without a packed structure, in this case with a M0+ it results in the use of ldmia/stmia-https://godbolt.org/z/5neP85Tj7edit- there is also a define in newlib memcpy.c which, when set at lib creation, will select the byte copy mode only. So if the compiler wants to call memcpy it will be using byte mode when using -Os. If the struct is packed then it appears memcpy is always called unless the struct is only a single member (or if the single member is an array, and only a single array element), so would be using byte copy if in -Os because it is calling memcpy. If not packed, then the compiler is happy (in the case of this example anyway) to generate its own copy instructions up to a struct that is 48 bytes in size (padded) via ldmia/stmia.So if a 17 byte object is being used, and memcpy is then being called, then maybe packed structs are in use (but then would not explain <17 byte being 'fine' in -Os). This example is not your code, probably not your mcu, or your compiler, your library, and so on. It does show there are a lot of variables in play- such as the use of packed structs, or which version of memcpy you end up getting and at what point does the compiler call memcpy instead of doing its own thing, etc. At least in the particular example above it would be a good idea to skip the use of packed structs simply for performance (and probably a good idea in general). I don't use packed structs, but is probably an option in the ide (if used) that could easily be enabled without realizing it (default option, or copy of a project which had it enabled, or turned on for some good reason).edit again- and another variable in play- which specs you are using and what library then gets used. In the case of using nano specs you only get the byte copy version of memcpy, so if you wanted something better I guess you have to understand what you are getting and why, and figure something out.

author: peter-h
author_rank: Super Contributor
timestamp: July 03, 2025, 05:43:11 pm
content: I use -Og exclusively, except for some bits of hardware-tied code which has been extensively scope tested and on which I use -O0 (as an attrib on the specific function only). Obviously I also stay with the same GCC version.The example above of an ISR decrementing a timer not declared volatile would, IIRC, fail with -Og too.As I said before developing with say -Og and then releasing with -O3 ispoor risk management, because the necessary regression testing is probably impossible on any nontrivial product.There is another very similar thread on optimisation.

author: cv007
author_rank: Frequent Contributor
timestamp: July 03, 2025, 09:59:04 pm
content: Here is another simple example-https://godbolt.org/z/TzshG55PEThis will work with -Og, but anything else will optimize away the use of ledRate in main. Of course there is a solution to the mistake, but if using -Og all along it will not show up and the led rate changes as you desired. Change optimization and your led blink rate no longer changes like it did for the last 2 months.This also points out there are two sides of volatile- and the forgotten side in this case is in main, where the compiler sees no need to deal with ledRate as it appears to do nothing so optimizes away from -O1 and on. The compiler cannot see any use of the callback function, in relation to main, so concludes there is no need for this var. The callback function does not write to ledRate, so one can mistakenly think there is no need for volatile as this var is NOT changing anywhere other than in main. Isr's are typically the producer of values, non-isr the consumer of them, but in this case that is reversed.

author: peter-h
author_rank: Super Contributor
timestamp: July 04, 2025, 07:31:01 am
content: I get the arguments here, and fully agree, but changing from -Og to -O3 is just like rebuilding your half megabyte FLASH image from a GCC compiler to say an IAR compiler and magically expect it to work if it throws up no error messagesIt just magically might work but it is a really dumb and risky thing to do.Of course the purists will say I am talking BS, but they all write perfect codeMuch more importantly they are working for some company and get paid by the hour, and sleep at night, whereas if I send out a product with a bug I will get so much s**t dumped onmethat I will wish I never did whatever it was.Compiler writing has been going sideways for years, with stuff like identification of loop structures and replacing them with memcpy(). The exec time saving is probably zero.

author: brucehoult
author_rank: Super Contributor
timestamp: July 04, 2025, 08:43:03 am
content: Quote from: peter-h on July 04, 2025, 07:31:01 amHow would you feel about a compiler replacing...Code:while (sz-- > 0) *dst++ = *src++;... with ...Code:beqz a2,1f0: vsetvli a3, a2, e8,m8vle8 v0,(a1)vse8 v0,(a0)add a0,a0,a3add a1,a1,a3sub a2,a2,a3bnez a2,0b1:... instead of ...Code:beqz a2,1f0: lb a4,(a1)sb a4,(a0)add a0,a0,1add a1,a1,1sub a2,a2,1bnez a2,0b1:The first is exactly the entirety of the fastest memcpy implementation on any RISC-V CPU implementing the V extension, copying 256 bytes per loop on the most common SBCs at the moment.I think you'll agree this is not a hugely difficult or risky transformation for a compiler to do, assuming "restrict" qualifier on the src and dst pointers.Thevsetvli a3, a2setsa3equal to eithera2or to the vector register size in bytes isa2is greater than the register size. It also makes thevle8andvse8limit themselves to loading or storing that number of bytes, there is no possibility if overwriting adjacent existing content, or going over a protection boundary and causing a trap (if the original byte by byte code won't).

author: hans
author_rank: Super Contributor
timestamp: July 04, 2025, 09:14:44 am
content: Quote from: brucehoult on June 28, 2025, 07:59:03 amThanks.Yes, I did spot the accidental omission of O1 too. I will add it later. Running a single sweep of all tests may take a few minutes, but for a (home)server thats otherwise idling all day its no big deal.I'm also at a point where I may need to include more instruction extensions, like floating point and compressed instructions. I'll have to see how I can plot it in such a way the data is still useful.QuoteI've seen this behaviour toowithoutthe C extension with RISC-V GCC 13.X as well. The result I showed is from an older compiler version. Not sure why this is exactly though.QuoteIts an emulator I wrote myself in Python which I did as a preparation for becoming familiar with writing a FPGA soft-core. Unfortunately, the soft-core is still a TODO, but the project's utility was quite good so far.The emulator supports rv32im, recently added a subset of atomic (RTOS work), and also some coprocessor extensions (machine interrupts)I've some basic peripherals like timers, plus a special emulator peripheral which can interact with the emulator environment.For example, instrumentation time start/stop, enabling/disabling a full instruction trace for hotpoints in my code, etc.The emulator has a few debugging features such as simply crashing instruction stream outright on a failed instruction (or infinite loops), which then gives a triggered PC location + C line location (taken from .elf debug symbols). The instruction trace also contains this info, as well as manipulated register values. Tracking down which instruction or line of code caused a null pointer dereference becomes a lot easier.There is also a "delayed" soft interrupt. Basically you write a delay count to a peripheral register, and it will count down to zero (at CPU clock rate) and fire an interrupt. This way I can sweep a range of cycle counts through function calls that are supposed to be reentrant. Again, super useful for low-level work.The only thing I'm missing sometimes is GDB stepping through code (sifting through 1GiB traces is not the first thing I would do).. but unit testing higher level code on x86 helps a ton.Another wishlist thing: more speed. Python is nice in quickly developing tools like this, but it only runs binaries at 50-100kHz. I wrote a basic C++ version that ran at 50MHz, but unfortunately not all these instruction tracing and benchmarking features are included yet.Quote from: cv007 on June 28, 2025, 10:54:20 amI don't think an attribute packed in my code.. but it appears very much like so.Perhaps GCC decided that a memcpy call is smaller than 5 loads + 5 stores, and that is what it prioritized.

author: peter-h
author_rank: Super Contributor
timestamp: July 04, 2025, 02:57:40 pm
content: QuoteThe real Q is whether it will ever make any measurable difference to the actual product.In general, if you are having to copy many bytes from A to B then your architecture is probably wrong.Sometimes one might do it out of laziness. For example I wrote a GPS data processing module which gets streamed NMEA data from a NEO-M9N via SPI, one byte at a time. I had to scan the data for specific strings as it arrived. There are lots of cunning ways to do string searches, including some very cunning recursive (and possibly dangerous) ways, but I am not clever enough and I just shifted the received data buffer left 1 byte (using memmov or memcpy) and searched the first n bytes of the buffer for various strings. On the next byte arriving, I loaded it into the end of the buffer, having shifted the content left 1 byte. It was primitive but obviously works fine. That buffer only needs to be long enough for the longest search pattern, plus whatever following data you might want to pick out.

author: brucehoult
author_rank: Super Contributor
timestamp: July 04, 2025, 03:11:29 pm
content: Quote from: peter-h on July 04, 2025, 02:57:40 pmFor large copies you get very little advantage because even the simplest byte by byte loop saturates the memory bandwidth. Where by "large" I mean bigger than your caches -- many KB or even MB.I did some measurements of the vector version vs GNU libc memcpy on the Allwinner D1 SoC (THead C906 core) back in early 2021. The same core is used today in e.g. the CV1800B in the $5 Milk-V Duo.For copies between 8 bytes and 128 bytes the vector version is at least 3x faster.I don't consider that size range to be so unreasonably large as to be indicative of bad architecture.https://hoult.org/d1_memcpy.txt

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 04, 2025, 03:22:53 pm
content: Quote from: cv007 on July 03, 2025, 09:59:04 pmYes. There are really 3 (that I can think of) main cases for volatile:1. A variable is modified in a function that the compiler doesn't see being called. That's typically the case for ISRs / thread functions, which are not explicitely called by any other function in the code.2. A variable is modified in a function, and used in another, both being explicitely called, but that lie respectively in 2 different compilation units. That's one of the limitations of the "separate compilation" model of C. A C compiler doesn't know what happens outside of the compilation unit it's currently compiling. If the compiler was aware of the whole codebase, thise case would not require a volatile qualifier.3. All accesses of a given variable must be honored - that's typically the case not for "normal" variables, but those that point to a specific address in memory (such as a device "register"). For these, all accesses may have a side-effect (for instance, reading a register may also reset its content), so all of them should be honored verbatim.Many people seem to wish case 3. was default, but they just don't realize how inefficient the generated code would be. The difference in execution time can be very large.To me, case 2. is really the one that could be alleviated with "smarter" compilers and a slightly different compilation model. (But that's just a wish: not how C is defined.)Case 1. is trickier, but could also be alleviated IMO with smarter compilers.Now even with smart compilers, some code could be in a library with no source code or code in a different language and so, determining what is being done with a given variable ("object") in memory is not always possible. So, the simplest approach was just to add this "volatile" keyword and make the programmer responsible. That's the whole idea of C.One technical issue with this with most existing compilers is that the optimization phase happens *after* any kind of static analysis of the code. In other words, it's too late - optimizers do not emit any warning or report (which I think is a shame).It can still bite even experienced programmers occasionally. Maybe compilers could emit more useful reports so that programmers could spot potential issues of that kind. Meanwhile, don't hesitate to use static analyzers on your code, which can spot a lot that compilers will just blindly compile.

author: ejeffrey
author_rank: Super Contributor
timestamp: July 04, 2025, 05:30:03 pm
content: Quote from: peter-h on July 04, 2025, 02:57:40 pmThat's just almost categorically wrong.What's far more common is that people overestimate the value of zero copy architectures. It's super common for people to spend a lot of effort developing a zero copy system, have a lot of bugs to squash, then find out it makes system performance worse.And yes, every global profile trace I have seen shows a significant fraction of all CPU cycles are spent running memcpy. Even small improvements provide real benefits.

author: gf
author_rank: Super Contributor
timestamp: July 04, 2025, 05:46:26 pm
content: Quote from: SiliconWizard on July 04, 2025, 03:22:53 pmIf the functions are called from the same thread, you do not need to care whether the functions reside in different compilation units. If the compiler cannot see what a called function does internally, it must assume the worst-case and renounce some load/store optimizations for the caller. There is no need to use volatile to ensure consistent visibility of the modification in both functions.

author: ejeffrey
author_rank: Super Contributor
timestamp: July 04, 2025, 05:56:46 pm
content: Quote from: SiliconWizard on July 04, 2025, 03:22:53 pmI'm not sure what you mean here. Access from separate compilations units doesn't require volatile qualifiers unless some other assumption is also violated (such as your #1).

author: Siwastaja
author_rank: Super Contributor
timestamp: July 04, 2025, 06:26:21 pm
content: Quote from: SiliconWizard on July 04, 2025, 03:22:53 pmNooOoOoo, do not use unnecessary volatile qualifier for this. This is perfectly normal C and works by design.Which is kind of an interesting point in itself - it forces compiler to make assumptions that global state could be modified in function calls, making some optimizations impossible. LTO could be able to solve that, by seeing what is exactly modified and where.But volatile is definitely not required and misusing it causes even further unnecessary performance penalty.

author: hans
author_rank: Super Contributor
timestamp: July 04, 2025, 06:35:31 pm
content: Quote from: ejeffrey on July 04, 2025, 05:30:03 pmIn addition, many algorithms simply make a memory and time tradeoff.The example peter-h gave is basically trading more memory usage (extra RAM for a shift register, redunant copy of data) or more processor cycles (strcmp for e.g. a circular buffer, which can't simply ptr++ to move forward).There are many more examples of this in common data structures, look-up tables, database indices, path finding, sorting algorithms, graphics (e.g. mipmaps) etc. etc.With MCUs gaining more (high-speed) RAM, it only makes sense that some tradeoffs will be made differently over time.

author: Nominal Animal
author_rank: Super Contributor
timestamp: July 04, 2025, 06:58:38 pm
content: A related case is when modifying a volatile function from interrupt context atomically, disabling and enabling interrupts:Code:oldstate = disable_interrupts();volatile_var++;restore_interrupts(oldstate);Here, the implementation ofdisable_interrupts()andrestore_interrupts()must include a compiler barrier (like__COMPILER_BARRIER(), orasm volatile ("":::"memory");in GCC and Clang), so that the compiler does not move the (volatile) accesses across the points where interrupts are enabled/disabled.You see, if the interrupt enable/disable functions are "pure", not interacting with thevolatilevariable accesses as far as the compiler can tell, and especially if they do not involve any kind of memory accesses, the compiler may reorder them with respect to thevolatileaccesses, moving thevolatileaccess outside the interrupt-protected region.The compiler barrier, as its name suggests, generates no code, and only tells the compiler to not move any reads or writes (any memory accesses) across that barrier.If you have inner functions that may use circuitous ways to access the variables in ways the compiler may not see, adding a compiler barrier before and after such a function call means you don't need to mark the variables accessed in both the outer and inner functions asvolatile. Technically, a function callshouldbe a compiler barrier, so explicitly adding the barriers before and after such calls should not do any harm. I've only seen this happen when I have weak-linkage do-nothing inner functions defined in the same scope, replaced/interposed with different functions at link time. As the weak-linked functions are globally visible, the compiler cannot omit the function calls, but since they are in the same scope, it may assume that because the functions do nothing, the call can be reordered wrt. variable accesses, unless explicit compiler barriers are used.Correct qualification of variables and the use of compiler barrier is notcomplicatedper se; it is just so deeply technical that it always takes some thinking to get it right. It's like the proverbial triangular hole, when us human developers are used to round or square holes instead. Having a good intuitive or explicit model of how different parts of the code interact and what the code is trying to achieve –– and having comments describing the intent of the code and the understanding of the developer,notjust describing what the code does –– is absolutely crucial for this. I personally wish I had learned to write such comments when I learned to write proper code, because I still find it hard to write such useful comments; they take at least the same amount of time as the code itself for me!

author: tggzzz
author_rank: Super Contributor
timestamp: July 04, 2025, 07:13:25 pm
content: Quote from: hans on July 04, 2025, 06:35:31 pmIt goes further than that...“Almost all programming can be viewed as an exercise in caching.” - Terje Mathisen

author: brucehoult
author_rank: Super Contributor
timestamp: July 04, 2025, 07:16:00 pm
content: Quote from: Nominal Animal on July 04, 2025, 06:58:38 pmThat doesn't sound right to me. And if right then it's awful!I don't see how a compiler can know that the implementation it sees may be replaced, and at the same time make assumptions about what the runtime function does based on the internals of the implementation it can see -- it should base decisions only on the interface (prototype) of the function.

author: tooki
author_rank: Super Contributor
timestamp: July 04, 2025, 07:34:42 pm
content: Quote from: tggzzz on June 26, 2025, 06:49:06 pmCOBOL and C will never go away, and it has nothing to do with higher education. What will keep them around forever isexisting code bases.COBOL was designed for and thus used for business applications, which in practice meant financial institutions (insurance companies and banks, mostly). And while financial institutions are certainly happy to migrate their front-end software and middleware to whatever the latest programming language trend may be, they are notoriously (and understandably!!) EXTRAORDINARILY cautious when it comes to modifying the core bookkeeping software that actually runs the numbers. They aren’t going to take on the risk of rewriting their known-good COBOL software in C/C#/Java/Python/whatever comes next, because whatever advantages itmightbring simply are not worth the risk of fucking something up. Financial institutions pay IBM the really big money for mainframes that can still run OS/360 so that the core bookkeeping software runs just as it always has. And they train new programmers in COBOL so they can modify the existing COBOL software (after the whole stack of approvals and reviews has been collected) rather than do a rewrite in C or whatever.Similarly, there is just tons and tons of C software out there, and the risk of rewriting it in another language just isn’t worth it.

author: tggzzz
author_rank: Super Contributor
timestamp: July 04, 2025, 08:31:09 pm
content: Quote from: tooki on July 04, 2025, 07:34:42 pmWe agree completely. I'll add that two other reasons for not replacing existing COBOL code is lack of engineers and the requirement to duplicate functionality exactly, including "errors" and where there is no design specification.But that wasn't the point. COBOL is not the future for all those reasons plus we have learned to create better languages in the 65 years since COBOL was created. And exactly the same is true for the 45 year old language, C.

author: Nominal Animal
author_rank: Super Contributor
timestamp: July 05, 2025, 12:47:09 am
content: Quote from: brucehoult on July 04, 2025, 07:16:00 pmHere, the "may" refers to practical reality, and not that it is allowed by the C standard.If a compiler assumes the variable access/caching can be reordered wrt. to such a function call, I do consider it an optimization bug.I did check a few cases at godbolt.org, and could not reproduce the bad behaviour with GCC, Clang, or ICC on x86-64, rv32gc, or rv64gc. They always do the proper variable accesses before the function call, sometimes leaving the values in registers doing any arithmetic with the registers after the function call, which is perfectly okay.So, I cannot be certain if I recall correctly and have seen it happen, or just recall incorrectly.Also, while GCC, Clang, and ICC do, technically the C compiler doesn't know that the runtime functions can be replaced because that (and symbolweakness) is outside the scope of the C standard, and is just a "linker trick".(One technically can modify the symbol linkage directly in the ELF object file, for example viaobjcopy --globalize-symbol=symbolname, but that is dangerous because the compiler can make assumptions about local visibility objects (static) it cannot for global visibility objects, and will optimize code differently.)The practical effect of what I described is that if one uses GCC/Clang/ICC extended inline assembly to implement the disable/restore interrupt functions, ie.Code:static inline UINT disable_interrupts(void) {UINT oldmask;asm volatile ( "..ARCH-SPECIFIC-ASSEMBLY..": "..REGISTERSPEC.." (oldmask): /* no inputs */: "memory" );return oldmask;}static inline void restore_interrupts(UINT mask) {asm volatile ( "..ARCH-SPECIFIC-ASSEMBLY..": /* no outputs */: "..REGISTERSPEC.." (mask): "memory" );}the"memory"extended asm clobber is essential, as it turns the inline assembly statement (often just one or two instructions) into a compiler barrier also.Leave it out at your own peril. With it included, the behaviour is what one would expect, and generates no unnecessary or extra code, and does not harm desired optimizations at all.With other C compilers, something like a__COMPILER_BARRIER()macro (specific to that compiler!) must be added into the function bodies before and after accessing the interrupt mask, to ensure correct ordering with respect to all other operations.One discovers these kinds of details not via debugging per se, but by carefully observing the actual generated assembly code, and comparing that to the developer requirements and expectations. Fortunately,readingassembly is much easier than writing fast/efficient assembly, so I do consider it a necessary skill for any low-level/embedded C developer.

author: gf
author_rank: Super Contributor
timestamp: July 05, 2025, 09:20:54 am
content: Quote from: Nominal Animal on July 05, 2025, 12:47:09 amI'm not sure it's essential for your example. A function call implies two sequence points, andvolatilememory access is not re-ordered across sequence points.QuoteSo I think your exampleCode:oldstate = disable_interrupts();volatile_var++;restore_interrupts(oldstate);should be sequenced correctly even without explicit barrier, provied that disable_interrupts() an restore_interrupts() arefunctions(not macros!), and volatile_var is declared volatile.QuoteMy understanding ist that a function call is not a compiler barrier, but de facto, calling a non-inlined external function has to act like a compiler barrier with regard to all C objects which are visible by the caller and which may be visible by the called function (including all C objects whose address might have "escaped" indirectly to the called function). In doubt, if it cannot prove the opposite, the compiler must assume that an object is visible by the callee (and in many cases it won't be able to disprove visibility).If a function call is inlined then the callee becomes part of the caller's control flow graph and the call is no longer an implied barrier with regard to any non-volatile object. The inlined function call still adds two sequence points to the control flow graph, though.I don't know if common compilers really "look into" non-inlined global functions defined in the same compilation unit as the caller, in order to aid optimization of the caller. At least for weak functions, they must not. Naturally, weak functions also don't qualify for inlining.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 05, 2025, 06:59:53 pm
content: Quote from: Siwastaja on July 04, 2025, 06:26:21 pmWell, that's right. The cases I had in mind for 2. eventually circle back to my point 1.As an illustration, one can play with this "minimal" example:https://godbolt.org/z/6Tozej3h4Uncomment the call to Bar() in Foo() and see the difference.Of course, this minimal example makes it rather obvious. There are lots of real-life cases where things are not so obvious.

author: peter-h
author_rank: Super Contributor
timestamp: July 05, 2025, 07:21:24 pm
content: Surely the time when the over-use of volatile will cause a performance hit is when accessing peripherals (accessing which generates some large number of wait states) but all those I/O defitions are volatile anyway.

author: Nominal Animal
author_rank: Super Contributor
timestamp: July 05, 2025, 10:34:24 pm
content: Quote from: gf on July 05, 2025, 09:20:54 amYou're right, and I was wrong.I don't know what I was thinking there, really. I should've actually verified first that I recall the details right!(I did some more godbolt tests, and I cannot replicate any of the scenarios of volatile + compiler barrier I was thinking of.)Teaches me to check before I blab from memory! Apologies if my assertions scared anyone.

author: westfw
author_rank: Super Contributor
timestamp: July 06, 2025, 12:37:06 am
content: QuoteAccessing memory is slower than accessing registers.Accessing peripherals is not always slower than accessing memory (for instance, CM0+ may have single-cycle IOBUS, while RAM isn't necessarily single-cycle.)Consider this "not unusual" clock tick ISR:Code:volatile uint32_t seconds;volatile uint32_t ticks;void RTC_handler() {ticks += 1;if (ticks >= TICKSPERSECOND) {seconds += 1;ticks -= TICKSPERSECOND;}}How many times, worst case, does it load or store "ticks" (and for no good reason.)

author: brucehoult
author_rank: Super Contributor
timestamp: July 06, 2025, 01:02:46 am
content: Quote from: westfw on July 06, 2025, 12:37:06 amThis is an interesting case and there are two efficient solutions:1) load ticks into a local variable at the start of the ISR and write it back at the end. This is what I'd just automatically do myself for any function that repeatedly uses a global variable or array or struct member. I know compilers can CSE it, but I like to be explicit.2) it is completely valid under the C standard to declaretickswithoutvolatilein the module with the interrupt handler, allowing the compiler to load and store it only once in the ISR, but withvolatilein the header file used by the main program to ensure a loop reading it repeatedly will see when it changes. The two declarations are compatible, thevolatiledoes not affect linkage, they are the same variable. All thatvolatileaffects is local code generation. This is a less intrusive change, but is probably too subtle for most maintainers of the code, and it has the unfortunate effect that you can't#includethe header in the implementation module (at least without other preprocessor trickery) as the compiler will complain about the redefinition.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 06, 2025, 06:30:02 am
content: Quote from: brucehoult on July 06, 2025, 01:02:46 amOpinions about adding volatile qualifier by type casting?It often happens that I could not care less when the variable is exactly written within the ISR, as long as it is eventually written in the ISR before it returns. Which is of course what would be guaranteed to happen, since the ISR would be a non-static-qualified function, writing to a non-static qualified variable.Then it should work to only apply the volatile qualifier to any loop which polls the variable. Right?I admit I have never tested that.

author: westfw
author_rank: Super Contributor
timestamp: July 06, 2025, 09:41:43 am
content: QuoteI didn't know that! It'd make me very nervous, though, worrying about the isr-handling declaration getting out-of-sync with other declarations...

author: cv007
author_rank: Frequent Contributor
timestamp: July 06, 2025, 10:16:19 am
content: QuoteAssuming this is an mcu with multiple irq levels such as a cortex-m (which _handler suggests), and assuming this example is complete and not just an example to point out volatile use, I would be more worried about a user of them having no guarantee they belong to the same second. The need for interrupt protection does not go away when inside an isr, at least for mcu's with multiple irq levels.edit- this would also be an example where allowing anyone to access these vars is not a great idea as the user of them would also need to make sure they have a pair that belongs to the same second. Interrupt protecting the write in the isr takes care of the writing, but the reading of the two vars also needs special care either by also using interrupt protection or by reading seconds, ticks, then reading seconds again to see if it changed (no change means seconds and ticks belong to the same second). So, instead of making a user deal with that, you would provide a function for all to use which takes care of those details and returns the 'time' (struct), also making the vars static so not accessible to others.

author: brucehoult
author_rank: Super Contributor
timestamp: July 06, 2025, 11:56:29 am
content: Quote from: cv007 on July 06, 2025, 10:16:19 amI would consider it very poor system design to have the handler for a high priority interrupt depending on internal consistency of values maintained by a low priority interrupt.If you're ok with having that higher priority interrupt wait while stuff inside a SEI/CLI pair (or whatever they are on your CPU) happens then you should also be ok with having have the ticks handler run at a higher priority in the first place -- especially on machines such as Cortex-M that chain interrupts so that the register save/restore happens only once, not between each handler.

author: cv007
author_rank: Frequent Contributor
timestamp: July 06, 2025, 12:44:37 pm
content: QuoteI guess if you want to keep track of 16 priority levels, and have them fixed so they never can be adjusted, then I guess you can figure it all out ahead of time. As soon as you start making adjustments to priorities you will have to revisit all those previous decisions you made, and by then you will have long forgotten about the details that were inside of the isr. So I think it would be a good design to not make assumptions about priority levels.If my isr was set to the highest priority, for today anyway, it would be much simpler to just assume it can be interrupted and do the protection for something that may need protection whether in the current priority it is needed or not. That protection, if not needed, will not harm anyone and the day I start adjusting priorities I will not have to revisit the decision about these priority dependent things. I can easily live with thedozen3 or so extra cpu cycles when the interrupt protection was not really needed such as already being in the highest priority, or disabling irq's forthose~12 cpu cycles when there were no other higher priority irq's that were actually interested in my vars. For something like 'time' in the example, it would be safe to assume anyone will be wanting to use the time var(s).

author: brucehoult
author_rank: Super Contributor
timestamp: July 06, 2025, 02:21:56 pm
content: Quote from: cv007 on July 06, 2025, 12:44:37 pmMy bad.If you don't have a carefully figured out reason to make things different priorities, and know exactly which ones should be higher and which lower, with all the mental work that involves, then just make them all the same priority and then ISRs will never interrupt each other at all, but automatically always wait until the previous one is finished, since on any sensible ISA entering an interrupt at a given priority will automatically prevent any ISRs for the same or lower priority from firing until either the ISR returns, or else explicitly reenables interrupts at that level.

author: tggzzz
author_rank: Super Contributor
timestamp: July 06, 2025, 03:24:03 pm
content: Quote from: cv007 on July 06, 2025, 12:44:37 pmMy attitude is there should be only three priorities: normal processing, (short) ISR, and panic shutdown.If absolutely necessary, have some software decide what fragment to process next - but do that at the normal priority.One alternative is for software to jiggle the priorities randomly, to attempt to break dead/livelocks. Apparently WindowsNT does that; I hope it wouldn't be beneficial in an embedded system!

author: ejeffrey
author_rank: Super Contributor
timestamp: July 06, 2025, 03:42:48 pm
content: Quote from: Siwastaja on July 06, 2025, 06:30:02 amI think that should work in practice, I'm not sure if it's technically defined. However, keep in mind that volatile access isn't a compiler barrier for non-volatile access. So even without casting, if the isr only writes a single volatile variable the compiler is free to reorder anything it wants.

author: ejeffrey
author_rank: Super Contributor
timestamp: July 06, 2025, 04:08:22 pm
content: Quote from: cv007 on July 06, 2025, 12:44:37 pmThat doesn't make any sense to me. If you some day decide that a specific ISR needs a higher priority, presumably that means you explicitly don't want it blocked by a low priority interrupt. Disabling interrupts in the low priority interrupt largely defeats the purpose of having those priorities. Which is fine: people seem to often over think interrupt priorities when they don't really need them. But then just set everything to the same priority.Unless your system is *extremely* complicated, evaluating which variables are read and which are potentially modified at eqch nesting level should be pretty easy. You should just repeat that each time you propose changing priorities to ensure you don't have a high priority interrupt depending on a low priority one. If you paint yourself into a corner and absolutely must do that, it's probably more appropriate to block a single interrupt, rather than globally disable them.

author: tggzzz
author_rank: Super Contributor
timestamp: July 06, 2025, 05:50:28 pm
content: Quote from: ejeffrey on July 06, 2025, 03:42:48 pmThat kind of subtlety is missed by too many people, even nominally experienced people."volatile" is to C as "glory" is to Humpty Dumpty.https://libquotes.com/lewis-carroll/quote/lbu9j1c

author: cv007
author_rank: Frequent Contributor
timestamp: July 06, 2025, 06:34:07 pm
content: QuoteThat sounds good in theory, and if there is some formal process you are required to go through to make proposals/changes, then I guess it would be taken care of.I see it more along these lines- a 'time' isr that is updating your system time is created which of course has an isr which updates var(s), something like the systick timer. You give it the highest priority since its small, and was your only reason for using the high priority since calling the time function which makes use of the vars will account for any missed irq (as timer keeps running no matter what). So, you think about it and conclude since the highest priority is being used there is no need to protect updating the vars inside the isr. All good.Then, you start thinking about the priorities available, or in the case of an M0 think about the lack of them comparatively speaking, and start thinking that there are more important irqs that should get serviced before others, and eventually after you have forgotten about the lack of protection in that systick isr it gets demoted to a lower priority because it will be perfectly fine in any priority. Sure, you may catch it at the time priorities are changed, but maybe not. The problems that would show up if missed are not going to be obvious, or typically very frequent so will most likely conclude the changes were good as nothing broke. A week or a month later...I'm not sure what the downside is in allowing for different priorities to be used, without the need to change or look over the code each time. Different priorities were already accounted for when it was created, and the cost was minimal in terms of code and cpu cycles. Even if you wanted to use a single priority level to make your life simpler, you may still reach a point where some of them are taking longer than you would like so will want to give one or more a chance to run while this happens. Since your code already does not depend on priorities, you simply change them as you see fit. Any problems you have by doing so will not be because you have not taken into account any needed interrupt protection, because its already in place.

author: ejeffrey
author_rank: Super Contributor
timestamp: July 06, 2025, 07:00:19 pm
content: Quote from: cv007 on July 06, 2025, 06:34:07 pmBut why are you using different priorities at all?Without a concrete reason why interrupt A has to have higher priority than interrupt B, the only sane thing to do is have them be the same priority. Once you have a concrete reason, it's extremely unlikely that it is OK for the lower priority interrupt to block the higher priority one.To be clear, by priority I mean the ability for a high priority interrupt to interrupt a lower priority. There is also the ability to set the order in which multiple pending interrupts are serviced, but that doesn't affect any of this.QuoteThe point is, that isn't what you are doing. In the scenario you have described, reprogramming the interrupt controller to change the "priority" is just a placebo. The priority is set by your use of disabling interrupts.QuoteThe process is: "I am changing the priority of interrupt foo. What global variables does it access, and what other interrupt handlers also access those" If you aren't doing that then you absolutely shouldn't not be messing with it.

author: JPortici
author_rank: Super Contributor
timestamp: July 06, 2025, 07:22:35 pm
content: Quote from: tggzzz on July 06, 2025, 03:24:03 pmgood for you!In my target application there is the highest priority to some serial bus i have to bitbang, then that i have to deocde, then some DMA'd peripheral that i have to process BEFORE another DMA'd event comes in or the cycle breaks, then the other DMA'd peripherals, then the timer interrupt, then the COMMs' interrupt, then the "task" interrupts (which are not short at all, they just need to run over the main thread which does stuff that i don't care if it's delayed by tens of ms of other processing) and then the main thread.I get by with 8 levels of interruptsQuote from: ejeffrey on July 06, 2025, 07:00:19 pmExactly. in my case i have a very good reason to keep interrupts at different priorities, very short and frequent events that must be processed before anything else. And i never have to change the priorities after that.

author: tggzzz
author_rank: Super Contributor
timestamp: July 06, 2025, 07:39:36 pm
content: Quote from: JPortici on July 06, 2025, 07:22:35 pmHow on earth do you guaranteeby designthat you meet all those requirements? Or do you just cross your fingers and hope a rare timing miss doesn't cause a product failure that in turn causes an employment failure?What youreallywant and need is 8 small simple cores, one for each of those tasks. No interrupts, no cache, just guaranteed performance.Failing that, alittlehardware support would remove the tightest timing constraints.Interrupts and priorities are a bodge that was useful when processorswerelimited and expensive. Since neither is true now, you can dispense with priorities and interrupts.

author: cv007
author_rank: Frequent Contributor
timestamp: July 07, 2025, 12:42:38 am
content: QuoteMaybe I'm not using different priorities today, maybe I will later, and maybe there will be a good reason, or maybe the reason will not be good. I simply protect something needing protection so whether priorities get changed or not, it needs no additional thought later. I can apply this same 'rule' consistently to all isr's with anything that needs protection. Maybe there are only a few of these around, or maybe there are quite a few, it doesn't matter as they all get the same treatment.If protection is not needed due to it being the highest priority or all irq's are the same priority, the cost of the unneeded protection is simply an extra 3 instructions. If a higher priority irq is blocked for the time protection is active, and it had no need of the var being protected, then that irq was delayed for a dozen cpu cycles unnecessarily. If a higher priority irq is blocked for the time protection is active, and it DID have a need to use the protected var then that protection was necessary and useful. A minimal cost to never have to deal with it again. The code, such as for a systick timer, can move over to any number of cortex mcu's and it will never be subject to the decisions being made about irq priorities. There will be no notes in the code that will specify it needs to be highest priority, because it simply doesn't matter. Instead of checking your code for these problems you may have left for yourself if/when changing priorities, you can instead just focus on whether your priority change was useful or not.example- are these interrupt locks necessary? maybe not in someIsr() and maybe not in time() as I have not said what priority someIsr() is using nor did I specify from where time() may be called. But why pre-determine at what priority time() can be safely called and who will be calling it, and also why pre-determine what irq priority someIsr() will be in relation to all the others? The protection is simple and of such little cost, it seems to me it will be worth doing so you never have to revisit this particular potential problem ever again. You figured out a var could need interrupt protection for both the write and read and took care of it. It will always be correct from day 1, and a future me cannot screw it up later.static volatile u64 ms;void someIsr(){InterruptLock lock;ms++;}u64 time(){InterruptLock lock;return ms;}These class interrupt protections can also be put in a scope, which you would do in a larger function to keep the protection only where its needed. The class constructor saves the irq state at the point of object definition, the destructor restores irq state at the end of scope.Another example would be something like stm32 gpio registers. Some of them are shared among pins on the same port. If you want to turn on a pin pullup, or reconfigure a pin, or set it to analog, or whatever else that will modify these shared registers- do you want to pre-determine in what run level you can safely do those things? do you restrict all code that touches these registers to the same run level? do you want to restrict all pin config to a single point in time? why not simply interrupt protect any modifications to these registers so one no longer has to worry about whether some isr is changing a pin while some lower run level was also changing a pin on the same port. You no longer have a restriction about where you can reconfigure a pin- it can be done anywhere and you no longer have a potential problem. Whether you setup all pins in a single place once, or let any code setup or modify a pin, it will always work correctly and there will be no possibility one of these register modifications will be interrupted where another run level may be wanting to modify the same register. I think this is also worth doing simply because it eliminates a potential problem that never has to dealt with or thought about ever again, also with very little cost.I have looked around to see who also interrupt protects these gpio registers, and found that NuttX does, not sure about Zephyr as I cannot read what they are doing, and I doubt any ST HAL code is doing so. If NuttX is doing the same, at least is suggests my thinking is correct.I'm not interrupt protecting everything in sight, nor am I declaring everything volatile. The protection is done when it is needed or may be needed as previously stated, the volatile is done when needed, after some thought and after checking the compiler output (and in -Os so no false assurances). I think I am acting sanely, but if not at least whatever I'm doing is working out well for me.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 07, 2025, 02:42:25 pm
content: Quote from: tggzzz on July 06, 2025, 07:39:36 pmJust like airplanes and space rockets are designed with extensive practical testing, and not mathematical proof alone, since human kind is still incapable of fully relying on latter, yes, we measure the performance, add safety margins, and understand the order of magnitude of potential edge cases and whether they are relevant or not.We rarely rely on whole concepts from single manufacturers, with single point of failure, process-wise, no matter how good they sound on paper, and no matter how good marketing shills they employ. We understand the risk of vendor lock-in.This works brilliantly well in the real world, especially with microcontrollers with no caches and no TLBs, and just a few clock cycles of timing jitter, as we have repeatedly discussed.For really critical work, worst case timingscanbe formally analyzed. Register transfer level simulation models are available from ARM, for example.Besides, I have sad news for you. How the XMOS silicon is synthesized from HDL is based on timing estimates based on decades of practical research and testing, just like design of any other IC. There isno guarantee, in your terms, that the XMOS won't just internally enter metastability. The quality is justtested into the product. Your nice "guarantees" all ride on top of this kind of handwaving on which the real world is fully based on.

author: JPortici
author_rank: Super Contributor
timestamp: July 07, 2025, 03:44:17 pm
content: Quote from: tggzzz on July 06, 2025, 07:39:36 pmNo, i want a widely available architecture, a good conforming c compiler and well unerstood peripheralsplease stop the propagandaI'd probably need 32-40 cores XMOS to do everything (counting by the peripherals / function blocks), or keep using a dsPIC and still have margin to add new functionality.Guess what

author: John Coloccia
author_rank: Super Contributor
timestamp: July 09, 2025, 02:17:23 pm
content: Quote from: JPortici on July 07, 2025, 03:44:17 pmThere are some really strange ideas about C and processors kicking around.As to the original question, just turn all of your optimizations off, and leave them off. It's very standard practice to do no optimizations unless you run into a specific need for it. I've yet to run into such a need on a real embedded project. That doesn't mean you don't need to appropriately still use "volatile" where required. They're unrelated to each other. Having optimizations turned on just makes it more likely that sloppy volatile usage will cause problems, but having them off doesn't make it unescessary.

author: JPortici
author_rank: Super Contributor
timestamp: July 09, 2025, 02:53:23 pm
content: Quote from: John Coloccia on July 09, 2025, 02:17:23 pmIt's a bad practice, though.I get that writing good C can be hard, and i remember how "bad" was my writing 10 years ago. I also remember how iffy were compilers for embedded targets 10 years ago, when your advice made sense. From time to time i still find the occasional compiler bug, but more often than not it was me that was doing something wrong.Optimizers these days are crazy good, provided you write correct code (use the appropriate types. Don't use unsigned char if you mean uint8_t, don't use unsigned int if you mean intptr_t or size_t, the compiler can perform better when you convey the correct meaning. Don't inline assembly, use assembly modules and write functions that conform to the ABI and call them from C.)the gains in code size and execution speed you can get from -O0 to -O1 can be massive, depending on the architecture of course

author: Siwastaja
author_rank: Super Contributor
timestamp: July 09, 2025, 03:35:15 pm
content: Quote from: John Coloccia on July 09, 2025, 02:17:23 pmI think this has happened onevery real projectI have done in last 10 years. -O0 produces extremely crappy code. Size alone is a problem; I don't like to pay premium by getting the overly large flash just because I would like to compile at -O0, which has no upsides at all. I routinely end up spending at least half of the flash.There really is no real reason for -O0 to even exist. The world would probably be a better place without it, because then people could not give bad advice like this.Really, just use -Os or -O2 or -O1 or -Og (if you prefer to use debuggers) by default. Do not let your friends compile with -O0.

author: John Coloccia
author_rank: Super Contributor
timestamp: July 09, 2025, 04:06:57 pm
content: Quote from: JPortici on July 09, 2025, 02:53:23 pmWhat are you talking about? Who said to turn them off to fix bad code?Quote from: Siwastaja on July 09, 2025, 03:35:15 pmMaybe you're programming toasters and need to save every last penny. Across time and space the general advice has always been to compile without optimizations unless you need them. It makes debugging and object code coverage more difficult, and it exposes you to more compiler bugs.Compile however you want. LOL....whatever.

author: brucehoult
author_rank: Super Contributor
timestamp: July 09, 2025, 04:19:17 pm
content: Quote from: John Coloccia on July 09, 2025, 04:06:57 pmMaybe last century, or in the 1970s or something.To a 99.9999% approximation, gcc and llvm don't have optimisation bugs. They are used by millions of people to compile billions of lines of code, with optimisation turned on.If you think you've found a compiler bug then1) you're wrong, it's a bug in your code2) if somehow you really have ... which is EXTREMELY unusual ... then report it and it will get fixed.

author: hans
author_rank: Super Contributor
timestamp: July 09, 2025, 05:04:56 pm
content: No or low optimization level produces crap assembly output. There are plenty of applications that simply wont work if your code isn't fast.Take for example anything battery powered. Every processor cycle consumes energy. If you want to extend battery life, then make the code complete as fast as it can.Another would be embedded graphics. Large microcontrollers can render GUIs on (half a) mexapixel displays. I have a graphics project that takes 3-4M cycles (between O2 and O3) to perform multiple bitmap copying, colour keying and alpha blending. Thats already on the limit for 60FPS on a >500MHz ARM Cortex-M7. If I turn optimization level off it drops to <10Hz, which is complete rubbish since the GUI requires smooth animations.USB, audio, circular DMA streams processing.. all have either hard real-time requirements or the performance can be a banner spec of the device.You can always buy a bigger battery, a faster/larger microcontroller, but this not about penny pinching, this is about the extent it becomes impractical. Or your competitor is doing this work, so then you're making an inferior product because of these reasons.At some point there needs to be a transition from "praying it works" to engineering with scrutiny. I understand Siwastaja's point about testing quality into a design, on the other hand, I think it doesn't beat the hard work that goes into learning how your tools work (like correct code). In software there is no guarantee that newer compiler versions wont 'break userland' by having always-on optimizations. Similar to how we're losing some features already in newer C(++) versions, such as register keyword, removing &= or |= operators on volatile types, etc.In my 10+ years of using GCC, I've encountered a single bug in GCC. It didn't even produce bad assembly, GCC simply segfaulted entirely. I reported the bug and it got fixed. Coincidentally, the bug was later reported about half a dozen times since ST's IDE used the older GCC version at a later release which didn't include the fix yet. The bug was triggered by ST's driver code for their new DMA controller in U5 series, where it was doing some bitwise arithmetic on pointer variables.Imo practically this goes against some advice given here: e.g. don't update compiler when it works (it wont introduce new bugs but it will keep all the existing ones), trust that 3rd party code will work since it has already been written (it may do something exotic that works in IAR but not on Keil/GCC/etc.), or [tinfoil-hat] that you need an optimization level in the first place to have bugs be introduced into the assembly output.

author: ejeffrey
author_rank: Super Contributor
timestamp: July 09, 2025, 05:07:15 pm
content: Quote from: Siwastaja on July 09, 2025, 03:35:15 pmYeah. The main purpose of -O0 back in the day was to _compile as fast as possible_. That was important when debugging X11 on a 486 with gcc 2.6, but it's completely irrelevant for cross compiling for a microcontroller target on a modern desktop. The secondary purpose was to make it look better in a debugger. Some old compilers couldn't even generate debug symbols with optimizers on, and even those that did were often confusing. Compilers and debuggers have gotten a lot better and -Og now exists, so -O0 is now pretty much obsolete.Quote from: John ColocciaNo, it hasn't. That has _never_ been general advice, for as long as optimizing compilers have existed. The "compile without optimization because it's potentially risky" mentality only seriously exists in a small corner of embedded development where people have been burned by misuse of volatile and don't really understand why. It doesn't really exist in the broader world of software development. Also, outside of embedded development, if you are using C or C++ you care about performance. If you didn't care about performance you would use a different language.In fact, while I think this is less true today, it used to be that you really wanted to always compile with optimization on because a lot of very important warnings were only generated based on analysis done by the optimizer. So if you didn't use optimizations, you were giving up a powerful tool to find bugs before they happened.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 09, 2025, 05:50:47 pm
content: Quote from: ejeffrey on July 09, 2025, 05:07:15 pmYes. This, this and this! Being too young myself, I justbarelyremember this (from newsgroups discussions etc.) from around the turn of the century. Optimization settings existed because compiling C code was slow. Heck, back then one of the main lines of criticism of C was that the syntactical design choices make itslow to compile- Pascal was suggested as something that would "compile faster".This issue has just totally disappeared - nowadays C is fast to compile, even on the highest optimization settings.And because it does not matter anymore, many others have completely forgotten all of this discussion - or some are too young (say, started programming in 2000's) so that they never saw it. The logical outcome of how human psychology works out is they "figure out" new explanations why optimization level settings like -O0 must exist.And sure enough, there are valid reasons - choosing e.g. between -Os and -O2 and -O3 comes tofine-tuning, and fine-tuning sometimes matters. E.g., you might want to compile some timing-critical module for speed, and all the rest for size.But it is worth noting that the "original" idea of choosing between -O0, -O1 and -O2 was fast, slower and even slowercompilation time. Now, since even -O3 compiles fast enough, there really is no reason for -O0. At least I have not figured out any, and some presented on these forums are fallacies.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 09, 2025, 06:00:31 pm
content: Quote from: ejeffrey on July 09, 2025, 05:07:15 pmI think this is still very relevant today. GCC has gotten way better in static analysis, detecting UB, and warning about it. It is easy to see that some of these warnings simply go away with -O0: often such that it "seemingly works", hiding both the bug, and the warning about the compiler statically finding the bug during compile - the best thing that can happen!Another aspect: if higher optimization level causes a difficult bug to surface (e.g., lack of volatile qualifier or lack of atomicity), that's agood thing- even if it takes a lot of manual debugging work to find the bug. The last thing you should do is to turn off the optimizer, hide the bug, and hope it's not there. I mean, what you are going to do? Manually inspect assembly to verify that turning off the optimization indeeddidremove the bug, then make a comment about it and warn yourself/others from ever updating the compiler? For this to be possible, you would need to know where the bug is. In which case, you could just fix it, e.g. by adding the volatile qualifier - which makes your code both correct, and as efficient as possible (by enabling optimizations). And if you do not know where the bug is, the last thing you want is to adjust compiler setting so that it seems to disappear. It is just too likely it's still there.TLDR: -Os -Wall -Werror. Let the compiler be your friend. They are really well designed tools.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 09, 2025, 06:18:48 pm
content: Quote from: hans on July 09, 2025, 05:04:56 pmOf course. My comment was tongue-in-cheek reference to tggzzz's recurring comments that quality cannot be tested into a product, which is easy to agree with. It's just that in actual real world, in every real project (when looked as a whole, not just a small subset), some quality always ends up being tested into the product; handwaving will be applied, practical measurements will be used instead of mathematical proof.Nothing indeed beats the hard work of understanding the theory, formulate the whole system as equations, and prove that the system performs to specifications. But human kind is far from capable of applying such pattern to everything; this still is possible only in very limited scope.tggzzz prefers to use a CPU where instruction timing can be proven to accuracy of single cycle; yet he must accept that it runs on a silicon where propagation delays are hand-waved with decades of practical experience and formulae with ad-hoc coefficients, and manufactured on dies with process variations and measurement and binning of the resulting chips, all with ad hoc safety margins. And he lives in a building where all structural dimensioning is based on centuries long practical experience of what typical materials can carry and when they collapse, with generous safety margins added.The fact that measuring ISR timing on a single-core microcontroller CPU with an oscilloscope, understanding that the subtleties like flash prefetch or branch prediction won't be realistically makinga bigdifference at all, then applying a safety margin, does not differ at all from all other engineering work applied to all other fields - safety critical or not.And quite funnily, really the biggest difference between general purpose and safety critical engineeringis in testing.But yeah. Of course I agree with tggzzz and you: the #1 priority is to try to understand, model, calculate and prove. It is just not always possible - sometimes notat all, sometimes theoretically possible but prohibitively expensive.

author: JPortici
author_rank: Super Contributor
timestamp: July 10, 2025, 07:16:59 am
content: Quote from: John Coloccia on July 09, 2025, 04:06:57 pmThe premise of this whole thread? Also, it's what i picked up from your post, which is what i've read online from very old posts, or mediocre programmers (not assuming you are the latter, but the former: a person with long time experience, that is maybe fixed in the old way of doing things.)

author: JPortici
author_rank: Super Contributor
timestamp: July 10, 2025, 07:19:36 am
content: Quote from: brucehoult on July 09, 2025, 04:19:17 pm3) you are compiling for an unusual targetXC16 doesn't have the wave of programmers you have for X86 / ARM, the change you find one are higher

author: brucehoult
author_rank: Super Contributor
timestamp: July 10, 2025, 07:28:54 am
content: Quote from: JPortici on July 10, 2025, 07:19:36 amSure, a boutique compiler for a weird ISA, probably maintained by a team of 2 or 3 people, is more likely to have optimisation bugs.AVR has a GCC port, so shouldn't have any more optimisation bugs than any other GCC target. Some madman has recently done a 6502 back-end for LLVM, so ditto there.But .. never having seen or used it .. I thought XC16 was based on GCC also?In which case you might expect the odd code generation bug, but not language conformance or optimisation.

author: JPortici
author_rank: Super Contributor
timestamp: July 10, 2025, 07:51:33 am
content: Quote from: brucehoult on July 10, 2025, 07:28:54 amYes, bugs that happened on code generation side, but that came up with optimizations enabled. One of my favourite is when it forgot all the addressing modes and kept calculating all addresses by hand instead of just using indirect load/store, which shouldn't have done in -O0 anyway (and it took me a long time to make the zombie at first level support understand that) or another in which a particular pattern created bad opcodes (an updated device added bitfield insertion/extraction instructions, i somehow managet to write a structure type that also used bitfields which broke the instruction, but only when optimizing because in -O0 it wouldn't use bitfield instructions, a slight change in the structure would make the problem go away)all usually patched within daysI'm trying to remember if i ever caught a bug in language conformance. Probably, but only once - and i'm pretty sure that was in XC8 which at the time was not GCC nor LLVM so it doesn't apply

author: newbrain
author_rank: Super Contributor
timestamp: July 10, 2025, 12:07:45 pm
content: Quote from: brucehoult, snipped by me on July 09, 2025, 04:19:17 pmAgree on all counts.As for 2) they may take take their good time though!https://github.com/llvm/llvm-project/issues/54014Written on clang 13, still there in 20...Not the highest priority, granted. I'll bump it, maybe.As always I start from the assumption that a PEBKAC, so I had a hard time understanding it was a compiler bug.Records in an I²C E²PROM were randomly not being saved or reappearing after being marked as deleted, the compiler was at the bottom of the suspects list.This is one of the cases when a debugger was useful - I could clearly see the write in the C code happening, with the assembly doing nothing!

author: brucehoult
author_rank: Super Contributor
timestamp: July 10, 2025, 12:44:06 pm
content: Quote from: newbrain on July 10, 2025, 12:07:45 pmOhh nasty! Reproduced on RISC-V, as you'd expect. It seems to be a Clang bug not LLVM optimiser or back end.Even worse, maybe, in Clang adding the following function...Code:int g(void) {return block->counter;}... it simply returns a constant 255 regardless of the optimisation level.The store to the memory location inf()is omitted at higher than -O0.GCC always does the right thing, and every optimisation level, withg()loading the value from memory.Ifg()does not exist then GCC omits the store to memory inf()at O1 or higher, and indeed omits the variableblockentirely, but this is ok because it returns the new value, 252, and no one else can see the modified value.The bug in Clang with optimisation on is not that it omits the store, but that it returns the initial value not the modified value.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 10, 2025, 01:51:11 pm
content: Quote from: John Coloccia on July 09, 2025, 04:06:57 pmI realized what John Coloccia's misunderstanding here probably is; indeed I have never heard this "general advice" anywhere, but instead, I have heard many times that "premature optimization is the root of evil", and similar remarks. Which is generally useful idea; correct to an extent. Maybe John Coloccia misunderstood this advice and thought it means compiler optimization settings, but that's really not the point. We are advised against prematuremanual(micro-)optimization because it easily leads to harder-to-understand, harder-to-maintain code, and also because knowing what the actual effects are is surprisingly difficult.Instead, really the general advice is to try to write as high-level code as possible, designed for understanding, maintainability, correctness and error handling, and thenlet the compiler figure out how to micro-optimize it.Don't take that thinking too far, though - this principle only applies tomanual micro-optimization, and micro means small localized adjustments. Choosing efficient data structures and algorithms is sometimes of paramount importance and compilers can't do anything about it if you chose a O(n^2) sorting algorithm which takes 100x too long, or chose a memory representation of data which takes 2x the available RAM.Also worth noting that if one enjoys manual micro-optimizations and gets them right, who are we to say it should not be done. But compiling at -O0 and then manually micro-optimizing instead would be probably worst use of one's time ever.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 10, 2025, 02:31:18 pm
content: Quote from: Siwastaja on July 09, 2025, 06:18:48 pmAnd even so: formal proofs are great when doable, but they can still only prove that a system is correct according to its specifications.Thing is, most of us know that in practice, a very significant proportion of bugs/issues are not due to a bad implementation (that's essentially a "skill" issue, and sure, some tools and methods can make it easier), but to incorrect or incomplete specifications, and the only way we can find this out is to test the system in its nominal environment, as it will be used (this step is usually called "validation"). Some studies show that as much as 80% of defects in a complex product come from defects in its specifications rather than in the implementation. No language or hardware is going to help with that.So there's definitely a matter of perspective here. From the perspective of an engineer in charge of *implementation*, tools are more of a concern. Whatever can help them output correct implementations is welcome (and sometimes seen as a necessity). From the perspective of people in charge of the whole system, that's a different story. If you design a rocket with provably 0 software or hardware defect and it crashes on the day it's launched - because some specification on what to do when some combination of sensors reading was lacking - then it's still game over. That would mean hundreds of millions or billions lost, possibly severed reputation and in worst cases, deaths. But sure - the team in charge of the implementation will not be held responsible.

author: tggzzz
author_rank: Super Contributor
timestamp: July 10, 2025, 02:49:24 pm
content: Quote from: SiliconWizard on July 10, 2025, 02:31:18 pmFormal proofs were very fashionable in the 80s, but almost always they ran into a showstopper: there were significant system components about which formal reasoning wasn't possible because it was too complicated.I remember discussing possibilities with a research group working on FSMs. They were proud of their progress. We showed them the FSMs for the computer network we were implementing (FDDI, shudder!), and asked whether they could prove freedom from deadlock/livelock. They took one look and threw their hands up in horror!Testing will always be necessary, but using good tools (of any type including brainpower) should enable whole swathes of problems to disappear. Certainly measure and hope you've spotted the worst case is highly suboptimal.

author: John Coloccia
author_rank: Super Contributor
timestamp: July 10, 2025, 03:53:08 pm
content: Quote from: Siwastaja on July 10, 2025, 01:51:11 pmWell, yeah, you should definitely not pre-optimize what you're writing as high-level code for sure. If you really need the speed, the compiler will almost certainly optimize the code better than you can. And trying to get clever at the high-level can often result in WORSE generated code. Compiler designers have gotten quite good at figuring out what you intend to do, and hand optimization often just confuses things.It's more about understanding the software you're writing, understanding where the pain points are, if any, and keeping things as simple as possible. If you can get away with no optimizations, there's absolutely no reason to do anything else. Also, as optimizations go, the vast majority of the time that I've needed something optimized it was because the ALGORITHM sucked. Poor memory organization, misunderstanding (or lack of understanding) how caches work, pipelining etc. All the compiler optimizations in the world aren't going to help with that, and that's most often where I see it go badly and where I get the huge gains, if I need them. The compiler optimization is a band-aid in those cases, but the real problem is a terrible algorithm.Here's a great example....sitting inside an outer loop:Code:outer loop {//do a bunch of stuff.../** This looks funny. Please leave it alone unless* you know what you're doing and can make it faster.* This is optimized for the processor.*/if (data[tmpy * FRAME_WIDTH + tmpx] > 253) {if (data[tmpy * FRAME_WIDTH + tmpx] == 255) {//do a bunch more stuff}break;}}Why not just check for ==255? I will tell you that it runs VERY slowly on this particular processor if you do that. I know more about the problem domain than the compiler does, and it's way more important to keep the instruction pipeline from flushing than it is to save a few instructions. With optimizations ON, in fact, it's practically impossible to arrange to get the code I want to make the processor reliably do what I want.So at the end of the day, I recommend you understand your pain points before doing any optimization at all, and see what it does. If you then find that you want to optimize for size and/or speed because it's useful for your particular project, go for it. But if it's not necessary, then it can ONLY make things worse. If I have a tight, real-time requirement, and I'm hitting it with margin, hitting it with more margin is just not normally helpful.I'm more or less constantly telling people "stop being clever...just go simple, and see if it works." And they feel like it's wasteful...and I'm like, "Would you rather it just sit around twiddling it's thumbs for the next 500us?"

author: brucehoult
author_rank: Super Contributor
timestamp: July 10, 2025, 04:12:59 pm
content: Quote from: John Coloccia on July 10, 2025, 03:53:08 pmWeird.QuoteBut surely slow comparison to 255 is EXACTLY what should be in the compiler's knowledge? It's certainly not problem domain.And why not?Code:uint8_t val = data[tmpy * FRAME_WIDTH + tmpx];if (val > 253 && val == 255) {...}Mind you, any decent compiler is going to delete the 253 test anyway.WHY is the 255 test slow?

author: John Coloccia
author_rank: Super Contributor
timestamp: July 10, 2025, 05:46:36 pm
content: Quote from: brucehoult on July 10, 2025, 04:12:59 pmThe code itself isn't slow, but the predictive branching in this case always predicts wrong and forces a pipeline flush if you code it sanely. This is sort of an extreme case, but mostly the optimizations I've found that make the biggest differences are related to memory, pipelines and things like that.As an example, I have many other cases where I've ripped out very code and memory efficient algorithms, and replaced them with brute force searches through a large array. Extremely inefficient and "wasteful" on the surface, but in practice it absolutely screams because I can keep my instruction pipeline from breaking, eliminate instruction cache misses, and minimize memory cache misses. Basically, giving the processor every chance to use every cycle for something meaningful. Especially on more complicated processors, the vast majority of problems I/O bound. It's basically sitting around doing nothing if you've been bad about organizing your data and algorithm to keep it fed.But I will agree that optimizing compilers are mostly excellent these days.

author: gf
author_rank: Super Contributor
timestamp: July 10, 2025, 06:00:13 pm
content: Quote from: John Coloccia on July 10, 2025, 05:46:36 pmDid you try to add likely/unlikely/__builtin_expect hints to the code?Or does your compiler not support that?

author: peter-h
author_rank: Super Contributor
timestamp: July 10, 2025, 06:26:50 pm
content: QuoteThat I am certain is true, but it is tangential to the debate.-O0 does produce truly wasteful code. It will usually run plenty fast enough but is a lot bigger than needed. From my project notes:The results, on the finished project (11/2024), boot block (max 32k) / main flash (1024k-32k)-O0 15.3k / 623k-Og 10.3k / 440k <- used for development and production-O1 11.5k / 493k-Os 9.4k / 391k-Ofast 15.5k / 490kOthers, not listed above, have not been tested.So you can see -O0 will fill up the FLASH pretty fast and for no good reason.

author: bson
author_rank: Unknown
timestamp: July 10, 2025, 06:54:04 pm
content: For non-trivial projects consider also concatenating all files together and compiling with -fwhole-file. It won't work miracles, but can shave off a little bit for a production release. More specifically it can identify unreachable code and identify inlining candidates in a call graph involving globals, and overall makes global calls (and variables) optimization non-barriers.

author: xvr
author_rank: Frequent Contributor
timestamp: July 10, 2025, 10:07:58 pm
content: I remember my old job at Sun Microsystems (over 30 years ago). We were maintaining the SunPro compiler, and we got a bug report. At some optimization level, some user-defined function terminates with a completely wrong result. Investigation revealed that the function code contains one undefined construct (which does not intersect with the execution flow for the given input parameters). The optimizer detects the undefined construct and removes the code that depends on it. Then it removes even more dependent code, and eventually removes the entire body of the function.It is very difficult for us to explain to the user that removing the body of his function is the expected behavior.

author: John Coloccia
author_rank: Super Contributor
timestamp: July 10, 2025, 11:02:19 pm
content: Quote from: xvr on July 10, 2025, 10:07:58 pmI've seen it work the other way around too. Compiler invents code because otherwise the behavior is would be undefined....so this Must happen. At a minimum, you're certainly allowed to infer it and assume the code is free of undefined behavior. It's a fascinating chain of reasoning that usually does something reasonable, even when the program itself is, technically, broken. Compiler designers are wicked clever these days.The customer is lucky he didn't end up with nasal demons...

author: ejeffrey
author_rank: Super Contributor
timestamp: July 11, 2025, 12:21:18 am
content: Quote from: EVblog1 on June 24, 2025, 03:42:57 pmYes.QuoteDon't think of it in terms of optimization.Instead think about the purpose of a program. Some programs compute a value (return value). Others perform IO, such as writing information to a file or a serial port, or reading from the keyboard. Some do both.From the perspective of a compiler, a source program is instructions that define the outputs in terms of the inputs. Those are the observable behavior of the program. Everything else is the implementation.Declaring a variable volatile defines read and write operations on that variable as observable behavior, rather than implementation details.Just like when you write:Code:printf("hello, world.\n");You don't care about where the string constant is stored, how the function is called, but you do care that the output isn't "world. hello," Volatile declares that the specific variable reads and writes need to occur in the order written because they are observable.That's all.

author: John Coloccia
author_rank: Super Contributor
timestamp: July 11, 2025, 12:21:56 am
content: Quote from: gf on July 10, 2025, 06:00:13 pmI don't remember. It was a while ago. I recall Linux added something....and then maybe c++ added something. I may have tried it or it didn't exist, and I also may have just guessed or looked at the list file, and realized what was happening.It was a bit of a tangent from this discussion, but I tossed it out there because optimization for a while was very important to the work I did, and it was almost never a number of instructions problem.But like I said, optimizing compilers are pretty good these days.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 11, 2025, 10:08:36 am
content: Quote from: John Coloccia on July 10, 2025, 05:46:36 pmI can totally relate to that, and am guilty to the very same pattern. If it so happens that I can't figure out a high-level algorithm for a complex problem, and it does as I'm not really a mathematician, I just write brute-force loops and, if needed, micro-optimize shit out of them. How much modern CPUs, even just microcontollers, can do nowadays when programmed in C, with some thought given to the structure of data representation in memory, is astonishing.Recent example is running 3 million(!) simulations of battery state-of-charge, grid energy import/export and energy cost with 1 hour resolution over 24 hours, with half a million of different control strategy combinations and six different simulation scenarios (different weather / energy use forecasting). These 3 million simulations take 5 minutes on Cortex-M4 at 64MHz, or < 1 second on a desktop PC. Competitors use neural networks, and we can read on social media how they sometimes make completely random decisions.Usually you are just minimizing some cost function, in which case you don't need to even store intermediate results. Just keep track what parameter set gives best score, and reconstruct after the iteration is done.Is this elegant? Probably not - but brute-forcing with some state space limitation works really well for complex multi-variable problems; computers are really good at doing small repetitive calculations, given that memory footprint is small enough that significant piece of state is kept in CPU registers or L1 cache. This sort of brute-forcing also tends to parallelize well, code being very easy to multi-thread.But you can't do that in Python or Matlab. This absolutely depends on least amount of bloat. You can do it in C; maybe a hand-crafted assembly version is 25% faster but that's not significant. But Python would be 100 to 1000 times slower and that's a no-go.

author: cv007
author_rank: Frequent Contributor
timestamp: July 11, 2025, 09:14:19 pm
content: Here is something that a lower optimization level (in gcc) may miss (and you may not realize you are not getting it)-https://godbolt.org/z/foe7Ga57fin this case array bounds checking. This is the gcc C++ compiler and is usually quite strict, but in -Og and -O1 array bounds checking is not active. In -O1 you can enable it via a few options, not sure how it gets enabled in -Og but probably can be if you can figure out the combo of options needed.I have some code now that is compiled with -Os with no errors, but if I use -O2 or -O3 will get an array bounds error. At the -Os level the compiler sees there is no actual array bounds problem or potential problem (at least not yet) where -O2 and -O3 have array bounds checking on but are not at a point where it can determine if the array bounds has or can be exceeded. In this case the code should be changed to eliminate the possibility of ever seeing this error, but even if unaware of it by using -Os, if the array is accessed out of bounds or get to a point where the compiler cannot know a value used at compile time, the compiler will start complaining. It would be better to know sooner rather than later even when the compiler was watching out for you.Maybe its not a bad idea to run through the optimization levels once in a while just to see how the compiler responds to each. It may not do much other than show different compile sizes, but it may also reveal as the above did that there could be a problem that is either not handled or the compiler was watching out for you all this time but the code could produce a compile error later.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 11, 2025, 09:28:52 pm
content: I always do most of my development and testing at the highest optimization level I'm aiming for for production.I only (rarely) lower it to the minimum usable for debugging with GDB if needed - and I fire GDB pretty rarely and only in hard-to-track-down cases.I have seen too many embedded devs doing what some seem to be doing here, doing most of their development at a low optimization level and "abusing" the debugger, only to get bitten late into the development process when switching to the production flags and starting to test the whole thing for real. I really advise against doing that. Just my 2 cents.

author: peter-h
author_rank: Super Contributor
timestamp: July 12, 2025, 09:22:18 am
content: You are probably a very good programmer.I've been doing code since the 1970s but still like to step through some stuff to make sure it is really doing what it seems to be doing. So much hardware interaction can work by luck...

author: brucehoult
author_rank: Super Contributor
timestamp: July 12, 2025, 09:41:41 am
content: Quote from: peter-h on July 12, 2025, 09:22:18 amThe problem with hardware interaction is it's often timing dependent and you CAN'T step through it -- it might be ok to delay 100 µs (to do a printf) instead of 1 µs, but 10 seconds is right out.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 12, 2025, 09:41:50 am
content: Quote from: xvr on July 10, 2025, 10:07:58 pmWhen large parts of code disappear on UB, that I think is one of the better outcomes. Best would be, of course, a guarantee of compiler always miraculously knowing the true intent of programmer, but since that is impossible, the second best is compiler detecting UB and printing a warning - which is what compilers are much better at today. Third best is a large and obvious difference against programmer's intent, such as removal of the whole function. It will cause major grieve the first time a C beginner has not even heard about existence of UB and can't understand why compiler removed the code, but after you have seen it once, you'll recognize the pattern quickly.It is hard to interact with people who A) are your customers so paid money to you, B) think they are right and experienced, C) really are not that experienced, D) are clearly wrong. This is a much more generic issue not limited to just compiler optimizations; just need to cope with this.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 12, 2025, 09:53:34 am
content: Quote from: peter-h on July 12, 2025, 09:22:18 amYou are overestimating geniusness of others.No matter how long you have been programming or how much code you have written, it's entirely normal you need to "step through" stuff, test with different inputs, see what outputs are produced, and scratch your head, "why?" - this is the hard work of programming. There probably are some true geniuses who just "see" the correct code in their minds but it would be 1 in a million or less.It's just that stepping through code in debugger is an extremely inefficient process, especially on microcontrollers. There seems to be some magical appeal to it, since some people start to resort to that as their default modus operandi. I feel sorry for them.Instead, copy-paste your iffy part to a fresh file and compile for PC (even better if your modules are small so that you can just add a testbench file and link it with the DUT - but don't worry if that's not the case). Generate inputs, and fprintf internal state and outputs into CSV file or similar. You can run on a lot of synthetic inputs you can modify ad hoc, instead of relying in real-world inputs on microcontroller. And investigate results all at once in spreadsheet software, or whatever you like best.It may take 15 extra minutes to "prepare" for this kind of test - maybe even hours for a complicated one -, but then you can be sure what you are doing "for good". Bonus points: save it in tests/ directory and you (and others) can refer to it later if questions arise how the logic works.

author: peter-h
author_rank: Super Contributor
timestamp: July 12, 2025, 10:29:46 am
content: QuoteOf course. But you get some hilarious cases like USB where a 1 second hold-up doesn't blow it upFor text output I find the SWV ITM console output is very fast - about 10us per char.QuoteThat's ok for testing algorithms, or even graphics (the Ramtex lib I use has a PC interface mode, USB connected I believe, but I've never used it because the issues I came across were mostly performance related) but mostly I don't find it useful. I do have a win32 build of MbedTLS (used in my last product) which was done by a colleague who knows x509 and which is pretty close to the embedded one and has been very handy for working out why some server won't talk.

author: tggzzz
author_rank: Super Contributor
timestamp: July 12, 2025, 10:34:58 am
content: Quote from: peter-h on July 12, 2025, 09:22:18 amNo surprises here...I like to spilt code into two parts: hardware specific, and hardware agnostic algorithm. Tie the two together with events for hardware->algorithm, and actions for algorithm->hardware. Then two separate test harnesses can unit test (1) the hardware on its own, and (2) the way the algorithm behaves in response to the hardware and controls the hardware. When trying to understand what the hardware is doing, (1) can be driven from the keyboard.Single stepping each on its own is practical and valuable, but trying to single step all simultaneously is likely to lead to timing problems.

author: John Coloccia
author_rank: Super Contributor
timestamp: July 12, 2025, 02:23:11 pm
content: I rarely use a debugger, but it doesn't mean I don't debug. It's just a personal preference. Most guys on my team use the debugger a lot. Honestly, though, we write our code so simply and un-cleverly that day to day stuff just mostly kind of works right out of the gate. Most stuff is either obviously correct, or if it's giving you trouble you ask for a second set of eyes and in 2 minutes they find something that's obviously incorrect.Getting interfaces going with peripherals is always a pain. That almost never works easily, and we attack it however we can. We spent a long time getting some single wire cryptographic EEPROMS working and playing with each other. Debuggers, scopes, logging, lots of cursing...probably some beer involved too.

author: brucehoult
author_rank: Super Contributor
timestamp: July 12, 2025, 02:36:58 pm
content: Quote from: John Coloccia on July 12, 2025, 02:23:11 pmThe thing is, you write that in a single small function (or two or three) in a small test program. You don't write thousands of lines of code with that as just part of it.Then when you get the function working properly you put it in your real program and never fiddle with it again.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 12, 2025, 04:37:10 pm
content: Quote from: peter-h on July 12, 2025, 10:29:46 amOr internal logic - everything except what happens inside peripherals. But you sure can test stuff that iscloseto peripherals, and should do that.All those iffy if-elses you have operating on some real-world inputs you easily cannot observe, and which are pain in the ass to produce on real HW.Last time I was becoming mad with my frequency counter code which had some conditions to filter out spurious edges outside expected frequency band. All really simple stuff, like take a timestamp when rising edge is detected, calculate dt between timestamps etc... Except with all tiny little real world details added (hold-off) and me not being a genius, it was PITA to see when this detection "algorithm" loses real edges.You can just say "oh this is close to hardware so cannot be tested" and give up, but exactly this is the case where testing is most valuable, since trying to get a real world mains frequency measurement produce those desired spikes and log them takes infinite amount of time. It is untestable on lab table and debugger.Instead, I wrote a small program which runs a counter and "generates" those "edges" - i.e., loop and call my function every time loop variable exceeds some value. Throw in some rand() to generate extra spikes (extra calls to the function), and another rand() for missing spikes. Log the indicated frequency to CSV. Calculate RMS error between the ideal input and "measured" output (of your "algorithm"). Adjust, run hours worth of synthetic test spikes in a second and see if RMS error went down. Look at the CSV output directly when investigating why.No microcontroller needed to test. This part of code is agnostic to which ADC is actually producing the input values.Yes, I ended up spending two days with it, but it allowed me to fix the relatively simple filtration code (a few lines of code) to a different, but similarly simple code, which performed like 100 000 times better or so. The original was a brainfart after all.

author: Nominal Animal
author_rank: Super Contributor
timestamp: July 12, 2025, 05:19:18 pm
content: Quote from: Siwastaja on July 12, 2025, 09:53:34 amI personally do this for every key algorithm and approach I do, saving each one in their own directory, with an explanatory README text file (and an executable that when run with-h, will output usage including a short summary of what the executable does). I have literally thousands of these by now.I don't write those in library form, but in a "fully featured" form, which can be pared down or refactored to what is appropriate for each use case. I often end up taking a look at the documentation, then the code, and then write an algorithmically optimized version with its own test cases, and only after testing, integrate it with the larger whole application/firmware, followed by more testing.Working with security-sensitive stuff has also "trained" me to switch viewpoints enough so that when I write tests, I can switch to thinking about the code from an adversarial viewpoint, instead of writing the tests to confirm my assumptions as the author of the code. I've caught many a off-by-one and pathological mishandling cases that way, too.As evidenced by this thread, I personally am definitely no genius; I'm just a good example of how diligent work can get one to the same end point.As I said before, my default GCC/Clang/ICC settings are-Wall -O2. For optimization, I learned over two decades ago to not try optimizing even the key partsperfectly. Instead, I want my code to behave acceptably even for surprisingly pessimistic and pathological inputs. So, I'm not making the best parts as good as possible, but making even the worst parts behave acceptably (for what the code is and does). It can still look like "perfectionism" for a casual observer, but I assure you, it made a huge difference in my development workflow: there is always parts to be enhanced/refactored, but the diminishing returns of that become easier to perceive.While I tend to use gdb less and less, I find it assuring that I can use it if necessary (evenimplementing my own Python accessors). For any kind of tree and graph manipulation, or pointer-based structures, Iveryoften implement a test case that generates pathological or random structures, and outputs their structure in Graphviz DOT format.Put simply, I've drifted from using a debugger, to exporting the key internal state in easily accessible and visualizable structures. Some call it "old printf style debugging", but because I do it also to the test cases, I do believe it helps me grasp the algorithms and approaches more thoroughly, and in a more intuitive manner. Plus, it does fit into the good ol' Unix Philosophy, of joining small modular tools to accomplish tasks, instead of using one monolithic tool for everything.

author: John Coloccia
author_rank: Super Contributor
timestamp: July 13, 2025, 12:19:01 am
content: Quote from: Siwastaja on July 12, 2025, 04:37:10 pmWe try to take even further. In most cases...not all...we can arrange it so that we read our inputs once at the beginning, abstract that into a structure, and set all our outputs once at the end....abstracted into a structureEveryone gets the same view of the system, as a cohesive frame, and everyone knows exactly when the outputs are set.Clearly not possible for all problems, or even 100‰ of any problem, but it's our goal.So the vast majority of the system just doesn't really interact with hardware at all. It's all very isolated, and is more or less very standard C that runs anywhere.We do our best to shove the rest into corners. I don't mind a bit of complicated, we'll understood, hacky nonsense in a corner or two. We get really nervous when we're dealing with complications at higher levels, and we try and figure out just what the helll we did wrong, and how to fix it.

author: hans
author_rank: Super Contributor
timestamp: July 13, 2025, 07:47:01 am
content: Quote from: Siwastaja on July 12, 2025, 04:37:10 pmDoesn't have to be this time costly. I have a GoogleTest CMake project secondary to my embedded code. It basically includes all source except for board BSP, and in addition, I also include all *_test.cpp files into the executable. Its really easy to switch between embedded and unittest environment.The case you're making is indeed excellent. The amount of code with hardware can often be made really really thin. If you just pass these register input/outputs around between higher level application code, then that is something you can test independent of hardware. It requires a tiny amount of extra code organization to seperate the pure hardware code from unittestable application code.The frustrating thing with debuggers is, they are so slow and clumsy to use. First they are mostly useful on lower optimization levels. Chances I have to recompile my whole project. Then what you can find is that a function call fails, but only on call 6. So you set a breakpoint, wait for it to hit 5 times (and click run 5 times).. but then this doesn't happen because the timing of your program has changed. Aargh so you wish you had some conditional breakpoints, but this is embedded, which means you don't. Instead you go in, add some messy global/static counter value and then increment its value on each call. Then if (callcount==6) { /* Breakpoint */ }. If you have a macro that calls the bkpt instruction on ARM great, but if you forget this exists you will likely write some volatile code statement which doesn't get optimized away.Okay so finally you can start single stepping and inspecting values inside your code. Presumably.. all this messing around took 5 to 10 minutes. And best of all, later on your forget you added this temporary callcount hack and it accidentally gets pushed to git.I'd much rather make a short trace which contain the function arguments and return values of the application layer calls. I can shove those into a unit test (just like you're generating fake edges) and test forever. Best of all: this code has now become an automated test enviroment, to prevent regressions.One could even test the arrival of certain events (e.g. from IRQ) on a larger application, and run a small integration test on your PC. Which then again takes over the work from manually pressing buttons on dev boards over and over to verify a certain scenario is still operational. We're programmers, so this is like automating ourselves out of the test equation.I done this kind of testing with RF DSP algorithms where you feed ADC data to symbol synchronizers, matched filters, demodulators, etc. Its easy to add a whole bunch of noise and test at what SNR or Eb/N0 the code breaks down. Having to do this on real hardware is a pure pain. Some people may do this kind of testing in MATLAB, but then you have to bridge code translation issues.

author: tggzzz
author_rank: Super Contributor
timestamp: July 13, 2025, 02:19:51 pm
content: My usual technique..1) Capture raw input in an ISR, put the raw info into a FIFO, return from interrupt.2) task reads FIFO, and converts raw input into a standard internal event, processes that in an FSM, outputs any appropriate actions, possibly puts new event in FIFO, possibly my pokes hardware. Frequently use the half-sync half-asynch design pattern.3) logging is largely keeping a very compact representation of events and states and actions. Brevity means it can be enabled during normal operation. Very convenient for field and remote maintenance.4) performance can be monitored by observing FIFO queue size.It5) log also convenient when arguing with other companies about who is responsible for fault found during system integration.Single stepping is used when checking event creation and actions poking hardware, but is rarely used otherwise: the log is better.

author: tggzzz
author_rank: Super Contributor
timestamp: August 06, 2025, 08:26:16 am
content: Just came across this while reorganising my bookmarks...Nine ways to break your systems code using volatilehttp://blog.regehr.org/archives/28

author: macboy
author_rank: Super Contributor
timestamp: August 09, 2025, 02:38:45 pm
content: Quote from: tggzzz on August 06, 2025, 08:26:16 amAnd his #1 way to break your system using volatile is ... "Not enough volatile". In other words, not using it when it really should be used.That is a great read, written by someone who seems to know a fair bit about the C standards and compilers, and who has a good amount of experience writing systems/kernel/driver code. Thanks for the link!

author: ejeffrey
author_rank: Super Contributor
timestamp: August 09, 2025, 03:22:42 pm
content: It's a good read but you have to keep in mind when it was written (2010) and what platform it mostly addresses (avr-gcc).C11 introduced atomics, and by now most 32 but microcontrollers can support them. Several of the listed ways to break your code should probably be updated to "use atomics if available" instead of using inline assembly to create memory barriers.Also most arm platforms have vendor provided headers that supply all the register definitions correctly marked volatile. If you can use those it will exclude 90% of problem #1 "not enough volatile"

author: tggzzz
author_rank: Super Contributor
timestamp: August 09, 2025, 03:35:02 pm
content: My attitudes...Most people who think they understand C's volatile, don't. (EDIT: a brain fart caused me to originally write "atomic".)Junk DNA in languages is a fertile source of subtle late surprises.Why doesn't anybody think of thechildrenpeople maintaining projects written in C89, C99,...

author: ejeffrey
author_rank: Super Contributor
timestamp: August 09, 2025, 04:04:19 pm
content: There are definitely plenty of pitfalls with atomics. C/C++ also definitely give you all the tools you need to get into trouble with e.g., relaxed atomics. And you can't really use them without knowing what types are lock free on your platform.For complex data structures outside an interrupt handler, locks remain the easiest and often best approach.But for simple uses like an atomic flag they mostly work. They are also pretty easy to use to implement locks if you aren't happy with your platform provided locks.On the other hand atomics solve a lot of the problems with people trying to abuse volatile for concurrency because they actually implement both a compiler and a memory barrier. Volatile access provides neither. Also while their primary purpose is for inter thread communication they can actually be used for IO registers in many cases. This is especially the useful if you are doing DMA, where atomics are absolutely the best way to ensure that the memory buffers stay coherent with the dma head/tail pointers.Yes they can be dangerous but not as dangerous as using the wrong tool for the job.

author: tggzzz
author_rank: Super Contributor
timestamp: August 09, 2025, 04:13:27 pm
content: Quote from: ejeffrey on August 09, 2025, 04:04:19 pmI've corrected my brain fart where I meant "volatile" but wrote "atomic".ItApologies

author: gf
author_rank: Super Contributor
timestamp: August 10, 2025, 12:43:22 pm
content: Quote from: ejeffrey on August 09, 2025, 04:04:19 pmBut even 'relaxed' has its justification sometimes.QuoteYou can, however, test if a particular atomic type is lock-free, using atomic_is_lock_free() and the ATOMIC_XXX_LOCK_FREE macros in C. In C++ there is also std::atomic<T>::is_always_lock_free, which is constexpr.In C or C++, the primary meaning of 'atomic' is 'atomic with regard to other threads'. This does not require lock-freeness, but it can be achieved with locks as well.A different (or additional) aspect is asynchronous signal safety, and it is only guaranteed for lock-free atomic data types; in other words, only lock-free atomic data types can be safely manipulated in signal or interrupt handlers.QuoteIn fact, atomic_flag is the only atomic type which is always guaranteed to be lock-free (on every platform that supports stdatomic).