Hidden gems of GCC and CLANG for a safer embedded development - Page 1
Forum section: electronics / general

author: YTusername
author_rank: Regular Contributor
timestamp: October 15, 2023, 07:31:04 am
content: Some of us may know some of the mentioned flags in theembeddedartistry's article. If you want to extend your knowledge about GCC and CLANG beyond '-Wall', then the article is for you. The presented flags are for GCC 12+ and Clang 18+. Some of the eye-catching flags:-Wthread-safety: warns about potential race conditions._FORTIFY_SOURCE=<n>: compiler built-ins to add in runtime bounds checking.-ftrivial-auto-var-init=[uninitialized|pattern|zero]: initialize all uninitialized variables.-fstack-protector: a “canary value” is added to the stack before other values are declared. Before returning from the function, the stored canary value in the stack is checked. If it has been changed, a stack buffer overflow has occurred, a callback is invoked, and the program is terminated.-fsanitize=signed-integer-overflow: checks to ensure the result of +, *, and both unary and binary - do not overflow in the signed arithmetics.

author: betocool
author_rank: Regular Contributor
timestamp: October 15, 2023, 09:57:04 am
content: The link is empty?Cheers,Alberto

author: YTusername
author_rank: Regular Contributor
timestamp: October 15, 2023, 10:39:06 am
content: Thanks. Fixed.

author: 5U4GB
author_rank: Super Contributor
timestamp: October 16, 2023, 01:07:27 pm
content: Some more very important ones that you should always enable:-fwrapv: Makes the compiler match what the hardware does rather than have the leeway to do whatever random thing the compiler writers feel like doing.-fno-delete-null-pointer-checks: What the name says, and yes, gcc will silently delete checks for null pointers if you don't use this.Reference: "How ISO C became unusable for operating systems development", Victor Yodaiken, Proceeedings of PLOS'21.

author: zilp
author_rank: Frequent Contributor
timestamp: October 16, 2023, 06:14:17 pm
content: Quote from: 5U4GB on October 16, 2023, 01:07:27 pmMind you that this can significantly cost you in performance. Also, that's a rather confused way to describe what the option does.Quote from: 5U4GB on October 16, 2023, 01:07:27 pmGCC will not silently delete checks for null pointers from C programs. Obviously.GCC will only eliminate dead code when you check that a pointer that you have already dereferenced (and that therefore can not be NULL in a C program) is not NULL.

author: 5U4GB
author_rank: Super Contributor
timestamp: October 21, 2023, 11:55:07 am
content: Quote from: zilp on October 16, 2023, 06:14:17 pmI've never noticed any difference in any code I've benchmarked. Perhaps you meant "under exactly the right conditions it may be possible to notice a slight difference in performance on special test cases"?For the description part, once the magic "UB" words are uttered the compiler can do whatever it wants, including reformat your hard drive. So "do whatever random thing the compiler writers feel like" seems accurate.Quote from: zilp on October 16, 2023, 06:14:17 pmIn other words it'll silently delete null pointer checks from programs if certain often nearly-impossible-to-detect conditions are met, exactly as I said I once spent the best part of a day tracking down a segfault in an OSS project that was caused by this, the developers still have no idea what it was that triggered gcc to delete the NULL pointer check. Another OSS project changed the way it uses gcc because they felt the presence of this behaviour made it too dangerous to compile code with.Other compilers (IAR, armcc, Keil, VS) don't do this, and even for gcc if you need to include a special compiler option just to fix something like this then the problem is the compiler, not its users. Or at least the compiler's developers.For a more detailed analysis, see for example the paper I referenced, "How ISO C became unusable for operating systems development", PLOS'21.

author: zilp
author_rank: Frequent Contributor
timestamp: October 22, 2023, 01:25:08 am
content: Quote from: 5U4GB on October 21, 2023, 11:55:07 amNo, I mean what I wrote.Quote from: 5U4GB on October 21, 2023, 11:55:07 amThat's not the confused part. Though it's kinda confused, too.Quote from: 5U4GB on October 21, 2023, 11:55:07 amIf your code is such that it's difficult to detect whether you are dereferencing NULL pointers, then ... your code should not be used in production anyway. What GCC does with your code is not your primary problem, and whether there is a NULL pointer check after the dereference is pretty much irrelevant.Quote from: 5U4GB on October 21, 2023, 11:55:07 amSo, which is it? They know that this caused the elimination of a NULL pointer check, or they don't know what caused the elimination of a NULL pointer check?Also, it's just obvious nonsense that somehow this could cause segfaults. If you dereference a pointer, then check it for NULL, and then dereference it again, it's completely irrelevant whether the check is eliminated or not, as the first dereference will segfault, so execution never reaches the NULL pointer check anyway if the pointer is NULL.Quote from: 5U4GB on October 21, 2023, 11:55:07 am... so what?Quote from: 5U4GB on October 21, 2023, 11:55:07 amYou don't need to include a special compiler option because there is nothing to fix. You are just completely confused about how compilers work. Also, other compilers also don't do anything defined when you dereference a NULL pointer, because there obviously aren't any sensible semantics for a NULL pointer dereference.(The exception being when you write code that can access data at address 0, which just isn't a thing normally ... and that's the whole reason for why the option exists, so that you can use GCC to compile code for these exceptional cases if you need to.)

author: ejeffrey
author_rank: Super Contributor
timestamp: October 22, 2023, 04:49:04 am
content: Quote from: zilp on October 22, 2023, 01:25:08 amThat's just false. The fact that you misunderstand this is a pretty good argument for the behavior being bad. It is 100% possible to have code segfault on a null pointer exception because of a compiler removed null pointer check.The key is that a null pointer dereference *may* cause a segfault but it isn't guaranteed to. For instance, if the first dereference itself is optimized away for having no observable side effects. Just doing pointer arithmetic on a null pointer is undefined even if you never derference it. Even though there is no illegal memory access and no segfault, it still is undefined behavior and the compiler can still assume it doesn't happen and use that optimize away future null checks.Of course the code is still by definition wrong. People aren't really upset about the segfault, they are upset about where the segfault happens: not in the code that actually has the bug but somewhere totally unrelated that has an explicit null pointer test.QuoteThe problem is it's not "your" code, it's someone else's code. If your code receives a pointer from "somewhere" that the compiler thinks it can prove is non-null, but the pointer is in fact null, that's where you have a bad day. And as compilers get more powerful they see more and more of the program and can make more inferences, especially with LTO. So it gets harder and harder to see where there might be a null pointer dereference.The solution to this is not to tell people they are wrong or call it nonsense, that only means you don't understand the issue. The only real (if partial) solution to this is ubsan tools that can find the undefined behavior where it occurs, rather than where the program fails. It's still hard to find bugs that only show up in rarely production, but ubsan has advantages over other tools in that it can find behavior that is undefined even in situations where it is apparently benign.

author: Siwastaja
author_rank: Super Contributor
timestamp: October 22, 2023, 09:49:45 am
content: What ejeffrey said. 5U4GB probably mispresented and oversimplified the issue, but arrogantly oversimplifying itthe opposite waydoes not solve anything. We should give them a benefit of doubt, it is well possible they had some very complex interaction where an esoteric bug somewhere goes unnoticed and then pops elsewhere, contributing factor being the compiler. People make mistakes, programming is difficult, and tools should be as helpful as possible.Take a look how air crashes are investigated. The answer almost never is "they did not follow the rules, their fault". Whether the design of the machines and processes help prevent mistakes, or drive small mistakes worse, is under careful investigation every time.

author: zilp
author_rank: Frequent Contributor
timestamp: October 22, 2023, 04:57:15 pm
content: Quote from: ejeffrey on October 22, 2023, 04:49:04 amOK, admittedly, they didn't talk specifically only about dereferences causing DCE, and also, of course, C doesn't guarantee a segfault in any case, even if you do dereference a NULL pointer ... but their posts here just have this strong "compiler writers are dumb because their compiler doesn't compile my code into what I've made up it should mean and instead strive to make code fail where the standard allows them to" vibe, which really isn't helpful for anything, and which also tends to be so vague, bordering on non-falsifiable, that you can't really address it fully in a reasonably short answer, but letting it stand as supposed wisdom isn't really helpful either.Quote from: ejeffrey on October 22, 2023, 04:49:04 amWhich is all true ... but the important point that needs to be pointed out to counter these compiler conspiracy myths is that compilers do all that in order to improve performance, not in order to make code malfunction, or just because the standard allows it.Compilers try to infer as much as possible about a program in order to find the most efficient way to express it in machine code, and finding checks that, according to the definition of the language, can not be true, and eliminating the code that expresses such checks, is done because that, obviously, improves performance. And not only does it improve performance, but rather, it allows for more reliable/more secure code, because it allows you to add more sanity checks in your code (like, say, bounds checks) and to rely on the compiler to prove where they are unnecessary so that the performance impact of those checks is minimal. So, it's a *good* thing that a modern compiler with LTO might be able to eliminate unnecessary bounds checks across compilation units, say, where a human might have a hard time figuring out why exactly the check is unnecessary.Now, all of this *sometimes* causes code to be eliminated that actually might prevent bad things from happening if it were kept. Which is unfortunate, and sometimes there might even be good reasons to make the compiler selectively less aggressive, but the important point is that there are usually babies in the bath water, and and no easy solution that reliably gets rid of just the water.

author: Siwastaja
author_rank: Super Contributor
timestamp: October 22, 2023, 06:15:01 pm
content: Quote from: zilp on October 22, 2023, 04:57:15 pmI pretty much agree with this. Even if it's frustrating, your program breaking because you made a mistake and compiler optimized something into unwanted territory is ultimatelya good thing; it exposes the bug (even if in suboptimal (tedious) way), you spend a lot of time hunting for it, learn something, and remember it the next time - hopefully, unless you choose to blame the compiler. I have long ago stopped blaming the compiler for my own mistakes as I see how the benefits of modern-day optimization outweigh the problems it causes when compiling buggy code. Ultimately, what needs to be done is admit mistakes and start writing less buggy code.Compiler writers have been extremely helpful in improving static analysis parts that cangenerate warningsabout so many common and even rare bugs during compile time. This goes side by side with optimization; increasing performance targets (a good thing!) drive the static analysis capabilities of the compiler, which as a side effect allows giving better warnings, too. So the commonly seen idea that better optimizations make programming more dangerous and more prone to bugs is very wrong on average, although in some isolated cases that seems to happen.

author: 5U4GB
author_rank: Super Contributor
timestamp: November 05, 2023, 02:07:46 pm
content: Quote from: zilp on October 22, 2023, 01:25:08 amAnd I meant what I wrote. Isn't it great how we can both mean what we wrote but get the opposite results simply by asserting something? Admittedly in my case I got that by taking measurements of actual production code, not sure where you got yours from.Quote from: zilp on October 22, 2023, 01:25:08 amIt's not my code. Also, the last time someone tried the the "all you need to do is write perfectly correct code" I ran some of the code he'd written through a source code analyser and found quite a lot of UB and other errors in there. He stopped responding to the thread after that. Care to post links to some of the code you've written so we can have a look? (See also the quote at the end by the main architect of LLVM).Quote from: zilp on October 22, 2023, 01:25:08 amIt was something like (greatly simplified):foo = bar->baz;if bar == NULL return -1;do_thing( foo );The generated code would have been:if bar == NULL return -1;do_thing( bar->baz );because foo was never used before the NULL pointer check. So the code wouldn't have segfaulted if gcc hadn't silently removed the NULL pointer check without ever issuing a warning.Quote from: zilp on October 22, 2023, 01:25:08 amI know how compilers work, but I also know how the gcc developers work because I've had to interact with some of them in the past. For compilers targeted at critical-applications use (the Keil, IAR, etc set I mentioned previously), the compiler developers tend to take the approach that the compiler should capture the intent of the software developers as much as possible. For gcc, the compiler developers take the approach that if you squint at the C specs just right and know just how to interpret the nuances correctly then technically the behaviour they've implemented is allowed, and they have a great opportunity to show how much smarter they are than the idiots who write the code that gcc has to process by silently causing breakage.Just out of curiosity, you're not a gcc developer are you? You have the exact attitude of other gcc developers I've interacted with.Finally, some quotes from the PLOS paper I cited:Quote

author: 5U4GB
author_rank: Super Contributor
timestamp: November 05, 2023, 02:15:17 pm
content: Quote from: Siwastaja on October 22, 2023, 09:49:45 amYou've pretty much captured the gcc design philosophy in one sentence there, or at least the attitude of the gcc developers. With a corollary that a lof of these rules are so obtuse and obscure that even the members of the JTC1/SC22/WG14 standards committee can spend months if not years arguing over them and, typically, not reaching any conclusion apart from adding another entry to the ever-growing UB catalogue.