GCC v11 32F4: compiler warning depends on optimisation level - Page 1
Forum section: electronics / general

author: peter-h
author_rank: Super Contributor
timestamp: November 22, 2024, 04:22:08 pm
content: This function produces no warning with -Og or -Os but with most others e.g. -O1 you get'ipaddr.addr' may be used uninitialized in this function [-Wmaybe-uninitialized] line 182How does that work?

author: Nominal Animal
author_rank: Super Contributor
timestamp: November 22, 2024, 04:57:08 pm
content: Such "maybe" warnings are based on the pattern in the intermediate representation (GIMPLE or GENERIC for GCC) after optimizations are applied, and are heuristic instead of deterministic.In your case,ipaddrindeed is used uninitialized if the value does not conform to numeric IPv4 dotted notation. Initializing it to all zeros would make a lot of sense in my opinion.

author: SiliconWizard
author_rank: Super Contributor
timestamp: November 22, 2024, 10:15:57 pm
content: As Nominal said, it does depend on the internal representation the compilers makes, but I would see it as a limitation of its static analyzer, which is still not very good for GCC. The CLang one is better.You simply have execution paths for which ipaddr is not set (if both sscanf fail), and the function returns this variable it in all cases. So the warning is definitely relevant.The fact it only appears in some optimization levels isn't great IMHO, but would again be explained with how GCC handles it internally.You should initialize ipaddr to some default value. Not sure what the type is, but simply:ip_addr_t ipaddr = {};should do (I think formally it's only correct in recent standard revisions - maybe strictly C2x? - but GCC has been accepting that for a long time for initializing almost anything to '0').I would suggest avoiding sscanf() too here, ideally. Not only is it relatively expensive (both in terms of code size and execution time), but it's also pretty unsafe.

author: coppice
author_rank: Super Contributor
timestamp: November 22, 2024, 10:27:37 pm
content: What platform is this? IP4_ADDR is usually a macro. So, depending exactly what that macro contains, ipaddr.addr might we well be used undefined.

author: golden_labels
author_rank: Super Contributor
timestamp: November 23, 2024, 08:41:01 am
content: Extending what Nominal Animal said about such best-effort warnings,-Ogdisables dead store elimination which is otherwise enabled. We may speculate, what is going on. Most notably that the difference lies outside▒▒▒_netconf_get_ip_config_value. Variableipaddris allocated not in this function, but in its caller. Something in the caller writes over storage used byipaddr, which is ignored with DSE enabled (hence seen as uninitialized), but is not ignored with DSE disabled (hence seen as initialized, even if not intentionally).That may be separate from the logical error in this function itself, where an execution path exists allowingipaddrto never be set before being returned.

author: peter-h
author_rank: Super Contributor
timestamp: November 24, 2024, 07:45:59 pm
content: Thank you allI can see that if both sscanf calls fail, the IP is undefined in this function. I just could not see how the optimisation level should affect this warning.I googled on Dead Store Elimination and while I can see why -Og disables it (it enables better single stepping) I can't see why -Os should disable it.I also can't see howunsigned int a, b, c, d;if (sscanf(value, "%u.%u.%u.%u", &a, &b, &c, &d) == 4) ...could make the sscanf fail. Surely any int (without a - sign) will be valid for %u. But yes it might fail if not seeing that x.x.x.x pattern, but file_get_config_value_default() does always return a valid string, in the form "169.254.75.75" (static const definitions).So maybe the compiler is trying too hard and getting it wrong.Probably I should haveip_addr_t ipaddr = {0};and sure enough that removes the warning with -O1.BTW, the defs aretypedef struct ip4_addr ip4_addr_t;typedef ip4_addr_t ip_addr_t;struct ip4_addr { u32_t addr; };For sure sscanf() is slow but this code just gets some IPs from a config file, and returns default values if the value is missing in the file. It seems to work fine. It's like snprintf() - it has its uses in the right places. I just hope no customer ever asks for IPV6

author: ejeffrey
author_rank: Super Contributor
timestamp: November 24, 2024, 07:51:59 pm
content: Quote from: peter-h on November 24, 2024, 07:45:59 pmYou may be right that sscanf can never fail, but you bothered to put the "if (sscanf(default_value, "%u.%u.%u.%u", &a, &b, &c, &d) == 4)" test in, so it certainly looks like you weren't *that* confident...

author: coppice
author_rank: Super Contributor
timestamp: November 24, 2024, 07:57:53 pm
content: Quote from: ejeffrey on November 24, 2024, 07:51:59 pmOf course sscanf can fail from the compiler's viewpoint. In his code he might be sure everything is sanitised, and only a minimum of 4 groups of digits may be present at that point, but there is no way the compiler can be expected to know for sure. They are doing pretty well these days with checking whether arithmetic has been sanitised, but I don't think they have any text to integer checking.

author: peter-h
author_rank: Super Contributor
timestamp: November 24, 2024, 08:05:22 pm
content: QuoteYes, I agree. FWIW I didn't write this code. It was done by the guy who worked on it a few years ago. May even be open source.What bothered me most was the effect of the -O level. I still don't actually get it. I am not seeing warnings from elsewhere, and probably should if file_get_config_value_default() could return an undefined IP.QuoteThat would suggest that sscanf() shouldalwaysbe assumed to fail, inside any function where the string is supplied as a parameter to that function. And in this case it istwocall depths removed...

author: coppice
author_rank: Super Contributor
timestamp: November 24, 2024, 08:11:35 pm
content: Quote from: peter-h on November 24, 2024, 08:05:22 pmUnless the string is guaranteed to have been sanitised, of course sscanf can fail. The depth would generally be less important than whether all the code is available in scope during compilation. If you are linking together various separately compiled pieces it seems unlikely such analysis will ever be performed by future compilers.

author: peter-h
author_rank: Super Contributor
timestamp: November 24, 2024, 08:55:17 pm
content: QuoteThat seems to depend on the optimisation level, and I can't see why.What I can see is that it would require an incredible amount of code to determine that the string passed to the sscanf() is OK - even though it is defined (several levels up the call hierarchy) with e.g.#define IP_STATIC_DEFAULT "169.254.75.75"and the buffer for it is 20 bytes.This is interesting:https://www.phoronix.com/news/GCC-12-Better-Securityre zeroing of stack variables but I am still on GCC v11.

author: golden_labels
author_rank: Super Contributor
timestamp: November 25, 2024, 04:13:16 am
content: Regarding value sanitization and validity checks: there isinet_pton(Linux,Beej’s). That also solves your worry about customers wanting support for IPv6 and any potential future addressing family.

author: SiliconWizard
author_rank: Super Contributor
timestamp: November 25, 2024, 04:23:14 am
content: Quote from: peter-h on November 24, 2024, 08:55:17 pmMy guess (while I would have to check, though) is that at < -O1 optimization level, the compiler may automatically initialize to zero all unitialized local variables, which would explain why it doesn't give a warning.Not, again, that I would find this warning behavior sensible. Even if the compiler was automatically initializing to zero, I do think it should still warn about the possible unitialized variable.And no, I don't buy either that it would do deep enough analysis to statically detect that the sscanf can never fail, and even so, why optimization would change anything about it. So the more likely reason to me is what I mentioned, although that would need again to be checked (you can look at the generated assembly in both opt levels, or use godbolt.org for that.)As to using sscanf(), I and others made some points, after that, it's obviously your code and your call.

author: peter-h
author_rank: Super Contributor
timestamp: November 25, 2024, 09:07:36 am
content: At -Og, I see no zeroing codeBut I probably don't understand the subtleties of arm32 asm since the code with = { 0 } produces the same asm. The bic.w instruction might be doing the job but the #7 operand makes no senseand this is with -O1 but not zeroing the IP; same code produced!

author: peter-h
author_rank: Super Contributor
timestamp: November 26, 2024, 09:56:49 am
content: Looks like the mystery remainsLooks to me like ipaddrisgetting zeroed in all 3 cases.

author: xvr
author_rank: Frequent Contributor
timestamp: November 26, 2024, 02:38:07 pm
content: QuoteI guess that all is a much simpler. To produce this kind of warning compiler have to run some data analyzing passes (at least it should build Control Flow Graph and run accessibility analyze on it). Compiler will build these graphs on high optimization level (not to produce warnings, but to perform required optimizations). But on -O0 it will not build them, so no possibility to perform warnings check (just not enough information). And gcc do not bother to run these data analyses passes only to produce warnings.

author: Siwastaja
author_rank: Super Contributor
timestamp: November 26, 2024, 04:21:06 pm
content: Quote from: xvr on November 26, 2024, 02:38:07 pmThis makes sense; it's well known how higher optimization levels compile slower. Probably not that significant today, but we used to see benchmarks ofcompilation speedsat different settings. So obviously compilers are doing much more work to analyze the code at higher optimization levels, and as side effect, would also emit more warnings.Remember, GCC is not a proper/perfect static analysis tool. Warnings are abonus; but you can't trust they are always generated, even when it's trivially easy for a human being (or better static analyzer) to prove that code has a problem.

author: peter-h
author_rank: Super Contributor
timestamp: November 26, 2024, 04:31:00 pm
content: However, several things:1) I am not using -O0 at all (at some +30%, the code is too big for the product)2) We are seeing the same code generated for -Og and -O1, but only -O1 generates the warning3) GCC claims (variously?) that -Og may zero stack variables but -O1 does not, yet the generated code does not confirm thisCan anyone confirm that ipaddr is actually being zeroed? I cannot find an asm language reference explaining some of the code.It would be quite useful to find out the reason for the warning, but I guess this is not possible?

author: Siwastaja
author_rank: Super Contributor
timestamp: November 27, 2024, 03:08:24 pm
content: Quote from: peter-h on November 26, 2024, 04:31:00 pmReason for warning is that the C code does not zero ipaddr and may use it uninitialized.Reason fornotalways getting warning can be complex, it makes no sense to delve deep into rabbit hole of GCC internals to understand exactly why. Unless you are a gcc developer and want to improve it.

author: peter-h
author_rank: Super Contributor
timestamp: November 27, 2024, 03:33:34 pm
content: What does this asm do?It is thesameif I use ipaddr={0} - the above screenshots show that.

author: rhodges
author_rank: Frequent Contributor
timestamp: November 27, 2024, 03:52:26 pm
content: Quote from: peter-h on November 27, 2024, 03:33:34 pmIt rounds r1 up to a 64-bit word, moves the stack pointer down for local variable space, pointed to by r5.This code does not clear any variable.

author: peter-h
author_rank: Super Contributor
timestamp: November 27, 2024, 04:08:58 pm
content: QuoteRight, so why did my ipaddr = {0} produce thesameasm?It is not as if ipaddr was a static and thus in BSS and thus zeroed at startup. This stuff is on the stack so could be anything.

author: Tation
author_rank: Regular Contributor
timestamp: November 27, 2024, 04:30:26 pm
content: Sure it is on the stack? Fitting in 32 bit it may well be in a register.

author: ejeffrey
author_rank: Super Contributor
timestamp: November 27, 2024, 04:47:12 pm
content: You need to look at the whole function. A very simple optimization would be to move the zero initialization into the conditional branch where scanf fails, since otherwise the initialized values are dead stores.One other complication: in most C ABIs, some struct return values are caller allocated with a pointer provided as a hidden parameter. In arm32 I think it's up to 32 bit size objects? I think ip_addr_t is large enough to hold either a v4 or v6 address so it probably qualifies. In addition, copy elision is performed whenever possible so the function doesn't create its own local variable then copy it to the caller provided buffer, instead it works on the provided buffer directly. In C++ where copy/move constructors can have side effects this behavior is required at any optimization level, so I wouldn't be surprised if gcc is doing this for C regardless of the optimization level.

author: peter-h
author_rank: Super Contributor
timestamp: November 27, 2024, 05:01:01 pm
content: OK here is the asm for a non-zeroed ipaddrCode:0805088c <netconf_get_ip_config_value>:static ip_addr_t netconf_get_ip_config_value(char *name,// InConfig option name to readchar *default_value,// InThe default value of the option as a stringchar *filename,// InThe filename of the config fileint number) {805088c:b5f0push{r4, r5, r6, r7, lr}805088e:b087subsp, #288050890:af02addr7, sp, #88050892:460emovr6, r1ip_addr_t ipaddr;unsigned int a, b, c, d;char value;8050894:1dd9addsr1, r3, #78050896:f021 0107bic.wr1, r1, #7805089a:ebad 0d01sub.wsp, sp, r1805089e:466dmovr5, sp//char value;file_get_config_value_default(name, value, default_value, filename, sizeof(value));80508a0:b29buxthr3, r380508a2:f845 3b08str.wr3, , #880508a6:4613movr3, r280508a8:4632movr2, r680508aa:4629movr1, r580508ac:f7fc f95ebl804cb6c <file_get_config_value_default>if (sscanf(value, "%u.%u.%u.%u", &a, &b, &c, &d) == 4) {80508b0:9701strr7, [sp, #4]80508b2:1d3baddsr3, r7, #480508b4:9300strr3, [sp, #0]80508b6:f107 0308add.wr3, r7, #880508ba:f107 020cadd.wr2, r7, #1280508be:4924ldrr1, [pc, #144]; (8050950 <netconf_get_ip_config_value+0xc4>)80508c0:4628movr0, r580508c2:f008 f943bl8058b4c <sscanf>80508c6:2804cmpr0, #480508c8:d11bbne.n8050902 <netconf_get_ip_config_value+0x76>IP4_ADDR(&ipaddr, a, b, c, d);80508ca:68bbldrr3, [r7, #8]80508cc:041blslsr3, r3, #1680508ce:f403 037fand.wr3, r3, #16711680; 0xff000080508d2:68faldrr2, [r7, #12]80508d4:ea43 6302orr.wr3, r3, r2, lsl #2480508d8:687aldrr2, [r7, #4]80508da:0212lslsr2, r2, #880508dc:b292uxthr2, r280508de:4313orrsr3, r280508e0:783aldrbr2, [r7, #0]80508e2:4313orrsr3, r280508e4:021clslsr4, r3, #880508e6:f404 047fand.wr4, r4, #16711680; 0xff000080508ea:ea44 6402orr.wr4, r4, r2, lsl #2480508ee:0a1alsrsr2, r3, #880508f0:f402 427fand.wr2, r2, #65280; 0xff0080508f4:4314orrsr4, r280508f6:ea44 6413orr.wr4, r4, r3, lsr #24IP4_ADDR(&ipaddr, a, b, c, d);}}return ipaddr;}and here is the asm for the zeroed ipaddr:Code:0805088c <netconf_get_ip_config_value>:static ip_addr_t netconf_get_ip_config_value(char *name,// InConfig option name to readchar *default_value,// InThe default value of the option as a stringchar *filename,// InThe filename of the config fileint number) {805088c:b5f0push{r4, r5, r6, r7, lr}805088e:b087subsp, #288050890:af02addr7, sp, #88050892:460emovr6, r1ip_addr_t ipaddr = {0};unsigned int a, b, c, d;char value;8050894:1dd9addsr1, r3, #78050896:f021 0107bic.wr1, r1, #7805089a:ebad 0d01sub.wsp, sp, r1805089e:466dmovr5, sp//char value;file_get_config_value_default(name, value, default_value, filename, sizeof(value));80508a0:b29buxthr3, r380508a2:f845 3b08str.wr3, , #880508a6:4613movr3, r280508a8:4632movr2, r680508aa:4629movr1, r580508ac:f7fc f95ebl804cb6c <file_get_config_value_default>if (sscanf(value, "%u.%u.%u.%u", &a, &b, &c, &d) == 4) {80508b0:9701strr7, [sp, #4]80508b2:1d3baddsr3, r7, #480508b4:9300strr3, [sp, #0]80508b6:f107 0308add.wr3, r7, #880508ba:f107 020cadd.wr2, r7, #1280508be:4924ldrr1, [pc, #144]; (8050950 <netconf_get_ip_config_value+0xc4>)80508c0:4628movr0, r580508c2:f008 f943bl8058b4c <sscanf>80508c6:2804cmpr0, #480508c8:d11bbne.n8050902 <netconf_get_ip_config_value+0x76>IP4_ADDR(&ipaddr, a, b, c, d);80508ca:68bbldrr3, [r7, #8]80508cc:041blslsr3, r3, #1680508ce:f403 037fand.wr3, r3, #16711680; 0xff000080508d2:68faldrr2, [r7, #12]80508d4:ea43 6302orr.wr3, r3, r2, lsl #2480508d8:687aldrr2, [r7, #4]80508da:0212lslsr2, r2, #880508dc:b292uxthr2, r280508de:4313orrsr3, r280508e0:783aldrbr2, [r7, #0]80508e2:4313orrsr3, r280508e4:021clslsr4, r3, #880508e6:f404 047fand.wr4, r4, #16711680; 0xff000080508ea:ea44 6402orr.wr4, r4, r2, lsl #2480508ee:0a1alsrsr2, r3, #880508f0:f402 427fand.wr2, r2, #65280; 0xff0080508f4:4314orrsr4, r280508f6:ea44 6413orr.wr4, r4, r3, lsr #24IP4_ADDR(&ipaddr, a, b, c, d);}}return ipaddr;}A comparison shows no change.Above is with -Og. With -O1 I do see a difference. Here are the two cases, same order as aboveCode:0804f40c <netconf_get_ip_config_value>:static ip_addr_t netconf_get_ip_config_value(char *name,// InConfig option name to readchar *default_value,// InThe default value of the option as a stringchar *filename,// InThe filename of the config fileint number) {804f40c:b5f0push{r4, r5, r6, r7, lr}804f40e:b087subsp, #28804f410:af02addr7, sp, #8804f412:460emovr6, r1ip_addr_t ipaddr;unsigned int a, b, c, d;char value;804f414:1dd9addsr1, r3, #7804f416:f021 0107bic.wr1, r1, #7804f41a:ebad 0d01sub.wsp, sp, r1804f41e:466dmovr5, sp//char value;file_get_config_value_default(name, value, default_value, filename, sizeof(value));804f420:b29buxthr3, r3804f422:f845 3b08str.wr3, , #8804f426:4613movr3, r2804f428:4632movr2, r6804f42a:4629movr1, r5804f42c:f7fc fa04bl804b838 <file_get_config_value_default>if (sscanf(value, "%u.%u.%u.%u", &a, &b, &c, &d) == 4) {804f430:9701strr7, [sp, #4]804f432:1d3baddsr3, r7, #4804f434:9300strr3, [sp, #0]804f436:f107 0308add.wr3, r7, #8804f43a:f107 020cadd.wr2, r7, #12804f43e:491aldrr1, [pc, #104]; (804f4a8 <netconf_get_ip_config_value+0x9c>)804f440:4628movr0, r5804f442:f008 f819bl8057478 <sscanf>804f446:2804cmpr0, #4804f448:d111bne.n804f46e <netconf_get_ip_config_value+0x62>IP4_ADDR(&ipaddr, a, b, c, d);804f44a:7839ldrbr1, [r7, #0]804f44c:68bbldrr3, [r7, #8]804f44e:021blslsr3, r3, #8804f450:687aldrr2, [r7, #4]804f452:0212lslsr2, r2, #8804f454:b292uxthr2, r2804f456:7b3cldrbr4, [r7, #12]804f458:ea44 6401orr.wr4, r4, r1, lsl #24804f45c:ea44 2402orr.wr4, r4, r2, lsl #8804f460:f403 437fand.wr3, r3, #65280; 0xff00804f464:431corrsr4, r3IP4_ADDR(&ipaddr, a, b, c, d);}}return ipaddr;}Code:0804f40c <netconf_get_ip_config_value>:static ip_addr_t netconf_get_ip_config_value(char *name,// InConfig option name to readchar *default_value,// InThe default value of the option as a stringchar *filename,// InThe filename of the config fileint number) {804f40c:b5b0push{r4, r5, r7, lr}804f40e:b086subsp, #24804f410:af02addr7, sp, #8804f412:460dmovr5, r1ip_addr_t ipaddr = {0};unsigned int a, b, c, d;char value;804f414:1dd9addsr1, r3, #7804f416:f021 0107bic.wr1, r1, #7804f41a:ebad 0d01sub.wsp, sp, r1804f41e:466cmovr4, sp//char value;file_get_config_value_default(name, value, default_value, filename, sizeof(value));804f420:b29buxthr3, r3804f422:f844 3b08str.wr3, , #8804f426:4613movr3, r2804f428:462amovr2, r5804f42a:4621movr1, r4804f42c:f7fc fa04bl804b838 <file_get_config_value_default>if (sscanf(value, "%u.%u.%u.%u", &a, &b, &c, &d) == 4) {804f430:9701strr7, [sp, #4]804f432:1d3baddsr3, r7, #4804f434:9300strr3, [sp, #0]804f436:f107 0308add.wr3, r7, #8804f43a:f107 020cadd.wr2, r7, #12804f43e:491aldrr1, [pc, #104]; (804f4a8 <netconf_get_ip_config_value+0x9c>)804f440:4620movr0, r4804f442:f008 f819bl8057478 <sscanf>804f446:2804cmpr0, #4804f448:d110bne.n804f46c <netconf_get_ip_config_value+0x60>IP4_ADDR(&ipaddr, a, b, c, d);804f44a:7839ldrbr1, [r7, #0]804f44c:68bbldrr3, [r7, #8]804f44e:021blslsr3, r3, #8804f450:687aldrr2, [r7, #4]804f452:0212lslsr2, r2, #8804f454:b292uxthr2, r2804f456:7b38ldrbr0, [r7, #12]804f458:ea40 6001orr.wr0, r0, r1, lsl #24804f45c:ea40 2002orr.wr0, r0, r2, lsl #8804f460:f403 437fand.wr3, r3, #65280; 0xff00804f464:4318orrsr0, r3IP4_ADDR(&ipaddr, a, b, c, d);}}return ipaddr;}The code looks similar but registers are used differently. The difference starts withsub sp, #28sub sp, #24and as posted before, the non-zeroing version with -O1 produces that warning.

author: Nominal Animal
author_rank: Super Contributor
timestamp: November 27, 2024, 06:19:18 pm
content: Quote from: ejeffrey on November 27, 2024, 04:47:12 pmIt is true that for composite objects (structures with multiple members) at most 32-bit ones are returned in registers, with larger ones stored at an address provided by the caller in r0, but for vectors up to 128 bits can be returned in registers (r0-r3). Seeherefor details.When software supports both IPv4 and IPv6 addresses, then the address part is 128-bit, and IPv4 addresses have the first 80 bits zeroes, next 16 bits ones, and the last 32 bits contain the IPv4 address. (This is standardIPv6-mapped IPv4 address, typically described using::ffff:a.b.c.dnotation, wherea.b.c.dis the corresponding IPv4 address.)Thus, if you haveCode:typedef uint32_t ip_addr_t __attribute__((__vector_size__(16)));ip_addr_t zero_address(void) { return (ip_addr_t){0}; }the function sets registers r0-r4 to zero; whereasCode:typedef union {uint64_t u64;uint32_t u32;uint16_t u16;uint8_t u8;} ip_addr_t;ip_addr_t zero_address(void) { return (ip_addr_t){ .u64 = { 0, 0 } }; }stores zeroes atr0+0..r0+15, inclusive.The non-standardvector_sizeattribute is supported by both GCC and Clang.If the vector definition is used, thenCode:typedef uint32_t ip_addr_t __attribute__((__vector_size__(16)));ip_addr_t ipv4_address(const uint8_t a, const uint8_t b, const uint8_t c, const uint8_t d) {return (ip_addr_t){ 0x00000000,0x00000000,0x0000FFFF,(((uint32_t)a) << 24) |(((uint32_t)b) << 16) |(((uint32_t)c) << 8) |(uint32_t)d };}returns the IPv6-mapped IPv4 address in registers r0-r3 on 32-bit arm-none-eabi. Clang generates nice code for it, but GCC usesr4andr6needlessly, and pushesr4-r10andfptemporarily to stack. (When using godbolt.org, remember to choose "unknown-eabi" or "none" GCC versions, as the "Linux" one uses SysV ABI instead.)

author: peter-h
author_rank: Super Contributor
timestamp: November 27, 2024, 07:21:41 pm
content: I posted the typedefs earlier e.g.Code:/** This is the aligned version of ip4_addr_t,used as local variable, on the stack, etc. */struct ip4_addr {u32_t addr;};So I am not using the more generic form of ipaddr which is a struct which can hold the single uint32_t (ipv4) or the ipv6 version.To support ipv6 I would need to rebuild LWIP with a different #define and re-code lots of code which processes and displays the IP. The decision was made to not support ipv6.

author: Nominal Animal
author_rank: Super Contributor
timestamp: November 27, 2024, 08:27:52 pm
content: Quote from: peter-h on November 27, 2024, 07:21:41 pmApologies, I missed that.Yes, this will be returned inr0onarm-none-eabiAKAaacs32(which is what I believe you are using), and passed in a single register when supplied as a parameter (ip4_addr_taddr).To repeat, the warning issue you are seeing isnotbecause the cause varies at different optimization levels; it is just that at some optimization levels the compiler just happens to not detect the pattern and thus omits the warning, most likely because the pattern is spread over too much code (which gets eliminated and thus pattern more compact when optimizations are enabled).You can see something similar when you examine the code GCC generates for the last function in my previous message: it pushes a lot of extra registers on stack, because when not optimized, itusesthose registers. Register use is a bit of a weak point in GCC, and has always been, so current optimization schemes still cannot fully optimize it in this case. Plus, it apparently does not notice those registers are no longer modified, so doesn't remove them from the saved+restored set. So, when looking at say-O2optimized code, those extra registers it pushes and pops look insane.Similarly, because GCC doesn't see the warnings as something ithas tocheck, but something itmaydo, it doesn't spend much resources in trying to detect the pattern causing the warning; therefore only when the pattern is easily discernible, will it detect and emit the warning. (In other words, the lack of the warning isnotan indication that the return value is initialized. It is only best-effort warning, nothing conclusive.)Also, whetheripaddris initialized in the caller or notdoes not and should not affect the warning, because the warning is about returning an uninitialized value from a function. It does not matter that the ABI might pass the result by reference and the immediate caller clears it to zero first,because the compiler must not assume it sees all callers while compiling this file. (If the function had only local linkage, i.e. it wasstatic, then it would be different, as the compiler would indeed see all callers while compiling this file.)

author: peter-h
author_rank: Super Contributor
timestamp: November 27, 2024, 08:40:15 pm
content: OK; I accept all this, as much of it is above my pay gradeLet me just say that file_get_config_value_default() is in the same .c file as the function under discussion above. But not everything that file_get_config_value_default() itself calls is in the same .c file.

author: Nominal Animal
author_rank: Super Contributor
timestamp: November 27, 2024, 09:38:46 pm
content: Quote from: peter-h on November 27, 2024, 08:40:15 pmNo, no, it really is a simple thing: GCC does not and cannot guarantee it emits the warning, as it is best-effort (or rather, "some effort") only. Basically, the warning is emitted only when GCC detects the pattern, but it doesn't really try that hard to detect it. Thus, the difference between optimization levels is the effort needed to detect the problem in the internal representation.So, it's just like a human proofreader: it may still miss it, especially if the code involved yields a complex internal representation (in GENERIC or GIMPLE), just like a proofreader may miss a minor problem in a complex sentence. If someone else cleans up the sentence structure first, that same proofreader is much more likely to detect that minor problem.Also, I forgot that##netconf_get_ip_config_value()was indeed markedstatic, so the last two sentences in my above post should be replaced with "Even if the ABI is such that the return value is allocated by the caller, the address passed to this function, this function just conditionally filling the value, does not change the fact that the problem is in this function and not the caller."You would need to change the function prototype tostatic void ##netconf_get_ip_config_value(ip_addr_t *dst, char *name, char *default_name, int len)replacingnumberwithlen,&ipaddrwithdst, and omitting thereturnline, for the warning to occur in the caller. The simple reason for this is that the warning is intended to help the developer to check thesource codevalidity (with respect to and in terms of theabstract machineas defined in the C standard), not to check whether the generated machine code is sane or not. Thus, the ABI used should not affect the warnings. (It will somewhat, though, because the ABI affects the complexity in the internal representation.)

author: SiliconWizard
author_rank: Super Contributor
timestamp: November 28, 2024, 01:27:18 am
content: I'll add that while GCC warnings should still not be ignored, they don't replace a full-blown static analyzer. Clang already does a much better job at it.Using a dedicated analyzer is always useful IMO. Some are free, others can be commercial and very expensive.One cheap (it's open & free & moderately good, but still very useful) option is CppCheck. Highly recommended. It will catch much more than GCC and (usually) than Clang as well, although on some specific cases, I've found Clang to catch stuff that CppCheck didn't.

author: peter-h
author_rank: Super Contributor
timestamp: November 28, 2024, 07:57:00 am
content: How does one run one of these? The project is after all a makefile, which is in this case hidden inside Cube IDE. Or maybe it just looks at all the .c and .h files it can find?

author: Siwastaja
author_rank: Super Contributor
timestamp: November 28, 2024, 08:48:41 am
content: Quote from: peter-h on November 28, 2024, 07:57:00 amcppcheck just processes the source files you give to it. It couldn't be much simpler to run:cppcheck .checks the source files in that directory and below.Note it doesn't catch everything either but is a good addition. If you are serious about product quality then adding as many tools as you can imagine is kinda low-hanging fruit (maybe except those that generate way too much false positives and therefore produce workload for you which could be better spent testing traditionally). It doesn't replace proper automated (and manual) testing, good planning, reviews etc. of course.

author: SiliconWizard
author_rank: Super Contributor
timestamp: November 28, 2024, 08:51:19 am
content: Yep. You can customize the files it checks, but with just a '.', it will check all .c/.cpp/.h files automatically. You may want to run it at the appropriate level of directory hierarchy if you want it to avoid checking some files, like maybe some third-party library that you don't intend to bother checking and even less so fixing.

author: peter-h
author_rank: Super Contributor
timestamp: November 28, 2024, 02:31:47 pm
content: I've installed an run cppcheck (the GUI mode) and it goes through the project (I added the inc and src directories) but it seems to report nothing. I see a few popups likebut no way to view anything. Under Warning details there is nothing, and under the Analysis log there is nothing but a list of modules. It doesn't find the .h files; a config of the inc path doesn't help.Edit: it does show some warnings; nothing important. Stuff like implicit casts. It finds some curious stuff e.g. "sportability: Shifting a negative value is technically undefined behaviour " but why is this a problem? Probably a new thread topicIt picks up a ton of stuff which has no effect e.g. "Variable 'long_deg' is assigned a value that is never used. " when I zero stack variables (good practice, surely?) but the init value (0) is never used.

author: Siwastaja
author_rank: Super Contributor
timestamp: November 28, 2024, 03:30:16 pm
content: Quote from: peter-h on November 28, 2024, 02:31:47 pmAre you seriously asking why UB is a problem? See nasal daemons.Finding UB is one of the primary reason tools like cppcheck would be used. Just fix if you have time. If not, freeze your tools and hope for the best.Otherwise than that, don't overthink it. Language rules are what they are and you have to live with that. Be happy it's standardized unlike some "replacements"!

author: peter-h
author_rank: Super Contributor
timestamp: November 28, 2024, 03:33:49 pm
content: I don't mean that UB is not worth checking out. I mean why an int32_t cannot be shifted. Isn't it just a pattern of 32 bits, for the purposes of the shift? Should it first be cast into a uint32_t?If the left shift was implemented as a x2 x4 etc signed multiply then yes it won't make sense. But I would expect it is done with a barrel shifter which just grabs the whole 32 bits and moves it left. That seems to be what actually happens e.g.int32_t pkt_value = -32767pkt_value = pkt_value << 13pkt_value &= 0x1fffe000;works as expected. How would this be done correctly? Presumably a cast before the shift. The application is weird: assembling arinc429 packets.I can see why somebody decided shifting signed ints should be UB but is there any CPU on which that is actually true, and why? I'd like to know since why would any compiler writer do anything other than just shift the 32 bit field presented? The intent of the programmer is a shift, simply.Every day is a school day

author: JPortici
author_rank: Super Contributor
timestamp: November 28, 2024, 03:54:12 pm
content: Quote from: peter-h on November 28, 2024, 03:33:49 pmYes.Because, what do you do with the sign bit? Do you keep it? Do you zero it? You are assuming that the underlying instruction set has an "arithmetic shift right" instruction (right shift, if previous MSB is 1 shift 1 in) which may not be the case, and that you are using >> for modulo division in this context, or to shift bits in another context, but when which is which? You know it, the compiler doesn't.var /= 2 is unambigous (and will definetly be mapped to an arithmetic right shift if such instruction is available). var >>= 1, not so much.this is why it's undefined behaviour (or implementation defined? on microchip's compilers they have an implementation-defined behaviour chapter, sign of right shift is there)

author: peter-h
author_rank: Super Contributor
timestamp: November 28, 2024, 04:02:46 pm
content: QuoteNo. AIUI, C shifts are always "shift in zeroes" so (taking an 8 bit example) if you shift 10000000 1 right, you get 01000000. I think that is called alogicalshift right.I had one C coder explain this to me. C doesn't have any of the handy asm shifts like shifts/rotates via the carry, etc. You have to hack them all. Shifts always fill in zeroes.The thing is that really both these lines should be UB for a signed intpkt_value = pkt_value << 13pkt_value &= 0x1fffe000;because both mess with the sign bit in a way which makes no sense if you want a meaningful signed int afterwards.Should it be like thispkt_value = (uint32_t) pkt_value << 13pkt_value &= 0x1fffe000;but then the result of line 1 becomes really subtle.OK here's a funny one. This piece of code is all over the internetCode:// Utility for the file listing - get file size units stringstatic void printsize(size_t size, char *out){static const char *SIZES[] = { "B", "kB", "MB", "GB" };size_t div = 0;size_t rem = 0;while (size >= 1024 && div < (sizeof SIZES / sizeof *SIZES)) {rem = (size % 1024);div++;size /= 1024;}if(div == 0)sprintf(out, "%iB\n", size);elsesprintf(out, "%.1f%s\n", (float)size + (float)rem / 1024.0f, SIZES);}but it doesn't like it:warning: Either the condition 'div<(sizeof(SIZES)/sizeof(*SIZES))' is redundant or the array 'SIZES' is accessed at index 4, which is out of bounds. What is wrong with it? Is it that it works only up to a certain number of GB (perhaps 1024) and then it bombs?

author: JPortici
author_rank: Super Contributor
timestamp: November 28, 2024, 04:05:49 pm
content: Quote from: peter-h on November 28, 2024, 04:02:46 pmNote that i explicitely mentioned "arithmetic" and not logical.Also, from my favourite compiler's manual "Implementation Defined Behavior" chapterQuote

author: Siwastaja
author_rank: Super Contributor
timestamp: November 28, 2024, 04:05:53 pm
content: Quote from: peter-h on November 28, 2024, 03:33:49 pmBut shifting is not just shifting bits! Many peopleexpectshifting to be equal to division or multiplication by 2^n, which for signed types is different than just the literal meaning of shifting bits. And this assumption is sometimes wrong, sometimes correct.If you really want to "just shift" the bits for whatever reason, then cast into unsigned. If you want to multiply/divide, then it's a good idea to use * or / operator for that, and let the compiler optimize it to the most suitable shift operation (if that exists on the target architecture).In the end, this does not matter. Standard saying it's UB means it's UB. You should not do it, if you do, you take the risk.

author: Siwastaja
author_rank: Super Contributor
timestamp: November 28, 2024, 04:08:06 pm
content: Quote from: peter-h on November 28, 2024, 04:02:46 pmThis is not true. Whoever taught you this was simply wrong.Don't use right shift on signed types if you care about which type of shift it is. And you probably do care.

author: peter-h
author_rank: Super Contributor
timestamp: November 28, 2024, 04:23:23 pm
content: So a right shift of a negative signed value introduces 1s. And a left shift of a negative signed value introduces zeroes, and just drops off bits at the left end.It turns out this is OK in my code because I mask off those bits anyway and merge in other ones (which are not related to the number in question). The reason this was not picked up earlier is because I exhaustively tested the various boundary conditions and all worked fine.pkt_value = pkt_value >> 3;pkt_value &= 0x1fffffff; // this clears any 1s introduced from the leftLike I say - every day one learns something.

author: JPortici
author_rank: Super Contributor
timestamp: November 28, 2024, 04:33:02 pm
content: Quote from: peter-h on November 28, 2024, 04:23:23 pmNoooooooooooooooooooooooooooooooooooit'sIMPLEMENTATION-DEFINED BEHAVIOURIt retains the sign for this particular compiler, for this particular version, for this target architecture. No guarantees that it could change in the future, no guarantees it's going to be the same for another target.

author: peter-h
author_rank: Super Contributor
timestamp: November 28, 2024, 04:42:59 pm
content: OK; last 2 posts crossed. I do actually mask all "new" bits after a right shift, so it doesn't matter if they are 1s or 0s. Just checked all the code...That is not the same as saying the right shift is not UB. But at least it doesn't matter what the new bits are.Cppcheck is very handy - thank you all!

author: coppice
author_rank: Super Contributor
timestamp: November 28, 2024, 05:16:17 pm
content: Quote from: JPortici on November 28, 2024, 04:33:02 pmPeople have gotten so used to modern C compilers treating signed values with arithmetic shifts and unsigned values with logical shifts, most have lost sight of this being undefined behaviour. it is, however, something you can rely on quite well.. A vast amount of code relies on it, and would break if compiler behaviour changes.

author: peter-h
author_rank: Super Contributor
timestamp: November 28, 2024, 05:52:48 pm
content: Interesting explanation:https://www.mathworks.com/help/fixedpoint/ug/use-of-shifts-by-c-coders.html

author: golden_labels
author_rank: Super Contributor
timestamp: November 28, 2024, 06:00:56 pm
content: A lot of arguing about howbitscould be shifted, if 0s or 1s should be added, what is “reasonable”, what is undefined, strictly-defined, or implementation-defined. But seemingly everybody forgotten, that in C operators<<and>>are not defined in terms of bits. While “bit shift” is mentioned, giving “the feel” of the operation, the actual behavior is defined in terms of numeric value and traditional arithmetic. All that reasoning and arguing is meaningless in C.Operationx << Bis multiplication by 2B, whilex >> Bis division by 2Band taking the quotient. Both cases are strictly defined exclusively for non-negative numbers, with the latter case also allowing implementation-defined behavior for negative ones. There is no bits involved at all.Both the original C manual and “The C programming language” described those operations in terms of bit shifts. However, also left them underspecified, allowing for either binary or arithmetic interpretation, limiting to particular type representations, and — in contradiction to bit shift perspective — putting a limitation on negative right operands. This notion has been dropped when C became adapted for a wider range of systems, choosing the arithmetic definition.

author: ejeffrey
author_rank: Super Contributor
timestamp: November 28, 2024, 06:27:12 pm
content: Quote from: golden_labels on November 28, 2024, 06:00:56 pmThat's a pretty silly argument. The operators are called right and left shift, and then further defined in a mathematically equivalent fashion.It's also incorrect. From the ISO standard, under the shift operator semantics heading:QuoteIt's clearly explained both in terms of bitwise shifts and the mathematically equivalent multiplication.

author: golden_labels
author_rank: Super Contributor
timestamp: November 28, 2024, 06:59:25 pm
content: That’s “pretty silly” because? Like with most “it’s silly” reactions: because it doesn’t agree with the mental model you hold?You know, what I find silly? You see: you may call me a total idiot, if you wish. But neither cherry picking phrases from the standard nor pretending I didn’t refer to those phrases is going to help. Because what we face is a much simpler thing: those operators don’t work as bit shifts would. We wouldn’t be even talking right now, if they would. They work in numerical sense only, and that is behavior chosen from two options offered by K&R. I may only speculate on the reasons for chosing the arithmetic “bit-shift,” but C having trap representations seems a good candidate for the culprit.

author: Nominal Animal
author_rank: Super Contributor
timestamp: November 28, 2024, 07:55:38 pm
content: It is interesting to note that givenint32_t x = -1,(int32_t)((((uint32_t)x) >> 1) | (1 << 31)) == -1. ForGCC, that expression is equivalent to(x >> 1) == -1(which is true).That is, GCC only supports two's complement integer types where all possible patterns are ordinary values, and shifting signed values right copies the most significant AKA sign bit, and the entire bit pattern is shifted. That is, forint32_t x, shifting right by one bit isx >> 1 == (int32_t)(((uint32_t)x) >> 1) | (0x80000000 & (uint32_t)x))and larger shifts are this repeated. In particular,x >> 31 == (x < 0) ? -1 : 0This also means that(x >> 1) != (x/2)for negative values.It is also notable that this isimplementation defined behaviour(see C99-C23 6.5.7p5),notundefined behaviour, so a build-time test suffices to verify the behaviour for portable code.In the case where you want to ensure zeros are shifted in, use a cast,((uint32_t)x) >>nwhich is guaranteed by the C standard to shift in zeroes, because((uint32_t)x) >>n== ((uint32_t)x) / (1 <<n)per standard definition.Then, in your source/build documentation, add a sectionExpected implementation-defined behaviour, and there describe this behaviour; for example,Compiler implementation is such that givenint32_t x = -1,(x >> 1) == -1.Not only is this useful for porting, it is useful to check this list first when you encounter an unexpected bug (that is, a bug that is not in the code you added or modified last).

author: peter-h
author_rank: Super Contributor
timestamp: November 28, 2024, 08:08:28 pm
content: QuoteWillint32_t x;x = ((uint32_t)x) >> n;be equivalent? It is surely a stupid question but some of the above got me worried. IOW does a cast back to int change any bits (for any value of x)? I don't think I have ever written code which casts a uint to an int,where the uint may be big enough to have the top bit set. I would regard that as really useless and dodgy.I've just documented all this as you suggest.In the meantime I've had more fun with CPPcheck:while ( (response!='=') && (idy<50) ) { ... }style: Defensive programming: The variable 'idy' is used as an array index before it is checked that is within limits. This can mean that the array might be accessed out of bounds.It works but would blow up if the array happened to be at the very end of physical RAMBut I believe this warning is wrong:How can they be sure len is not 0. Inbuf could contain any garbage.And I wonder if anyone has an idea re printsize() further back...

author: Nominal Animal
author_rank: Super Contributor
timestamp: November 28, 2024, 08:46:35 pm
content: Givenint32_tx;thenx= ((uint32_t)x) >> 5;is exactly equivalent tox>>= 5;x&= 0x07FFFFFF;based on standard C, no implementation-defined behaviour or differences between compilers, as long as they follow standard C (currently C99, C11, C17, or C2x/C23).(Remember, exact-width integer typesuintN_tandintN_talwaysuse two's complement representation for negative values with all possible bit patterns corresponding to ordinary values, no padding, no trap representations.)Quote from: peter-h on November 28, 2024, 08:08:28 pmIn the right shift case (by a positive, nonzero number of bits) that is not relevant, because the result is always positive, because the sign bit is the most significant bit and a zero will be shifted in when using the unsigned type cast. That is, if the exact-size (uint_N) cast value is shifted right by at least one bit, then the result is guaranteed to fit and be within the range of the correspondingintN_ttype, and the standard says that in that case that will be the resulting value.The standard leaves a cast from unsigned to signed integer typesimplementation definedonly whenthe value cannot be represented in the signed type. GCC will never change the bit pattern (seehere), and will simply re-interpret the sign bit as-is. This is another implementation-defined behaviour one can expect, and note in the documentation for portable code. For example,(int32_t)(0x80000008u) == -2147483640.I did some digging for Clang, but I couldn't find a similar implementation behaviour list, but Ibelieveit behaves the exact same way. It is actually quite possible that future C standards end up codifying this, because I personally have never used a C99-or-later compliant compiler that behaved differently.

author: golden_labels
author_rank: Super Contributor
timestamp: November 29, 2024, 04:41:25 am
content: That implementation-defined behavior for right shifts may be tracked down to early versions of C before standardization. It’s even mentioned in “The C programming language” (2.9):“Right shifting a signed quantity will fill with sign bits ("arithmetic shift") on some machines such as the PDP-11, and with 0-bits ("logical shift") on others.”The associatedgetbitsexample evades the issue entirely by just assuming the value is positive. A similar note may be found in the Ritchie’s memo, where it’s indicated that“the use of arithmetic rather than logical shift does not survive transportation between machines.”Both versions still assumed a more or less specific integer representation. In particular the 1974 version assumes 16-bit, 2’s complement integers.

author: Nominal Animal
author_rank: Super Contributor
timestamp: November 29, 2024, 05:44:57 am
content: Quote from: golden_labels on November 29, 2024, 04:41:25 am(To nitpick, only right shifts of negative values is implementation-defined. Positive values in signed integer types follow logical right-shift rules, being equivalent to division by a power of two. I know golden_labels knows this well; this is just a reminder for others reading this thread that might not be aware of this.)For exact-size unsigned integer types (uintN_t) right-shift is always the logical one, equivalent to division by a power of two. So, if casting to/from the corresponding signed type (intN_t) simply reinterprets theN-bit pattern –– the types being required by the standard to use two's complement format, with no padding bits or trap representations, all possible storage patterns and values corresponding to ordinary integer values ––, and the signed type right-shift is always the arithmetic one (filling new bits with the sign bit), programmers have full control and portable use of both. This is how GCC behaves on all architectures it supports. I believe, but have not been able to verify from the documentation, that Clang behaves the same way on all architectures. To select between logical/arithmetic right shift on expressions using these types, one needs at most two casts, which are almost always no-ops affecting only code generation, generating no machine code themselves.This is why I believe it likely that a future standard C revision will standardize the current GCC behaviour. I don't think there is any current architecture that supports C that would suffer from this, and the benefit to low-level and systems programmers is huge.

author: SiliconWizard
author_rank: Super Contributor
timestamp: November 29, 2024, 06:24:48 am
content: Quote from: peter-h on November 28, 2024, 08:08:28 pmHow much is INBUF_SEARCH_LEN?

author: peter-h
author_rank: Super Contributor
timestamp: November 29, 2024, 07:45:44 am
content: It is 7.So if inbuf[4,5] contain 0xffff, len=0.Admittedly if len=0xffff then the other condition will also failBut that is not the same thing as saying len==0 is always false.But this isn't worth worrying about because the ==0 check is just good programming, even if some weird analysis decides it cannot happen.

author: SiliconWizard
author_rank: Super Contributor
timestamp: November 29, 2024, 11:09:17 am
content: Quote from: peter-h on November 29, 2024, 07:45:44 amThe diagnostic is indeed wrong but it still points (possibly unwillingly) to a potential issue.I guess (at least for this particular type of diagnostic) CppCheck doesn't consider the modulo (wrap-around) in C arithmetics, or doesn't properly consider the integer width here. In which case, it wouldn't see the possibility of a zero (which would mean the sum has wrapped around).The potential issue I mentiioned is that checking the values in inbuf (so the value of 'len' *before* adding (8 - INBUF_SEARCH_LEN) ) would be a good idea (input validation) and would avoid a possible wraparound.So I would check len before the line 'len = len + ...'. You can also check it after the sum if that makes sense for your application. But I would not potentially let the sum wrap around.

author: coppice
author_rank: Super Contributor
timestamp: November 29, 2024, 11:30:42 am
content: Quote from: golden_labels on November 29, 2024, 04:41:25 amEarly C compilers were all over the place with shift behaviour. I think they mostly implemented whatever was quick to do with the instruction set available. This was a bad idea even than, as from day 1 C had signed and unsigned integers, and treating them in a natural way for the type was perfectly reasonable to do, even when the instruction set didn't directly support both arithmetic and logical shifts. Early C was not well thought through in many ways. Like not having integers which nailed their number of bits in a clear unambiguous way. Retrofitting that decades later is still gradually working its way through the system. In the last 30 years or so the only C compilation I have seen which doesn't obey the natural behaviour of signed and unsigned values when shifting have been DSPs. which often have a pretty quirkly form of C for many reasons.

author: peter-h
author_rank: Super Contributor
timestamp: November 29, 2024, 12:34:15 pm
content: What is needed is a dedicated CPPcheck threadIt does pick up some curious stuff e.g. I had a prototype for a function, 3 int parms, and in the prototype I had their names in the wrong order. I wondered for a moment... how can the code possibly work? The answer is that their names don't matter (obviously). I verified this with a binary comparison.It also disregards #defines, so e.g. the above len==0 code was not even included in the project.And it reports loads of functions as never called, because it doesn't seem to relate multiple source files. Probably there is a config for that...Many "scope of ... can be reduced" comments. Many"Local variable 'ch' shadows outer variable " but I see no issue with using "i" as a for loop counter; it is private to the loop.So far I have not found anything serious in this multi-year project. But I have not yet let it loose on MbedTLS

author: newbrain
author_rank: Super Contributor
timestamp: December 05, 2024, 03:43:51 pm
content: Quote from: peter-h on November 28, 2024, 08:08:28 pmJust a question: how is inbuf defined?If achararray has been used, does cppcheck know that, by default, char is unsigned on gcc for arm?If cppcheck thinks char to be signed, you'll never be able to obtain 0xFFFFu from the line:Code:len = inbuf + 256 * inbuf;for semi-obvious reasons.If inbuf is auint8_t(or equivalently, explicitunsigned char) array, then the warning is indeed spurious, as you suggest.

author: peter-h
author_rank: Super Contributor
timestamp: December 05, 2024, 03:48:58 pm
content: static uint8_t inbuf;and PKT_BUF_LEN is 2048 bytes.