Confusing use of Pointers, help me understand - Page 1
Forum section: electronics / general

author: Ground_Loop
author_rank: Frequent Contributor
timestamp: July 08, 2021, 10:57:38 pm
content: I have a line of code as such:*(volatile uint32_t *)(&LATB + (port * 0x10)) = (*(volatile uint32_t *)(&LATB + (port * 0x10)) & (~mask)) | (mask & value)The intent is to write a 'mask'ed 'value' to a 'port' on a PIC32. I get the gist of it, but the pointer operators are hurting my head. How is the multiplication operator '*' distinguished from the pointer operator '*'? Same for '&' as both a logical and pointer operator.

author: oPossum
author_rank: Super Contributor
timestamp: July 08, 2021, 11:04:36 pm
content: Pointer arithmetic is limited to +, -, ++, and --. It is not possible to multiply (*) or bitwise and (&) pointers.

author: Ground_Loop
author_rank: Frequent Contributor
timestamp: July 08, 2021, 11:14:20 pm
content: Quote from: oPossum on July 08, 2021, 11:04:36 pmWhat is this '(port * 0x10)'?

author: oPossum
author_rank: Super Contributor
timestamp: July 08, 2021, 11:22:36 pm
content: port is an integer, not a pointer. The result is added to a pointer (&LATB).

author: NorthGuy
author_rank: Super Contributor
timestamp: July 08, 2021, 11:24:36 pm
content: Quote from: Ground_Loop on July 08, 2021, 11:14:20 pmThis is multiplication intended to produce the desired offset which is then added to the address.Someone noticed that LATx registers are spaced at equal distance from each other. Thus there's possible to calculate the address of the LATx register knowing the number of the port. This is what it's doing.BTW: Using an array of structs would probably require less writing and would make the code more understandable.

author: oPossum
author_rank: Super Contributor
timestamp: July 08, 2021, 11:41:30 pm
content: Another way to write this would be...Code:(&LATB)[port << 4] = ((&LATB)[port << 4] & (~mask)) | (mask & value);

author: Ground_Loop
author_rank: Frequent Contributor
timestamp: July 08, 2021, 11:58:55 pm
content: Quote from: oPossum on July 08, 2021, 11:41:30 pmI was aware that an array is really a pointer, but didn't know a pointer offset could be syntaxed as an array. I'm not a programmer by trade or training, but hobby work these days requires it.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 09, 2021, 12:00:54 am
content: Quote from: Ground_Loop on July 08, 2021, 10:57:38 pmSame remarks as others above.But I'll add something. The PIC32 has SET, INV and CLR registers for its GPIOs. With those registers, you can probably do the above in a more elegant, and efficient way?

author: oPossum
author_rank: Super Contributor
timestamp: July 09, 2021, 12:09:23 am
content: Quote from: Ground_Loop on July 08, 2021, 11:58:55 pmThese are all the same...*(pointer + int)*(int + pointer)pointerint* and [] are just different ways of dereferencing a pointer

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 09, 2021, 12:26:14 am
content: Regarding pointer arithmetic, or playing with array indexing (which is the same as you said), I do not like this much here. It's based on assumptions that may not hold true on different models/generations of PIC32, and then your code becomes absolutely not portable within the same familty. I can't say I like this much. Especially if it's not clearly documented in the source code...

author: cv007
author_rank: Frequent Contributor
timestamp: July 09, 2021, 01:30:16 am
content: *(volatile uint32_t *)(&LATB + (port * 0x10)) = (*(volatile uint32_t *)(&LATB + (port * 0x10)) & (~mask)) | (mask & value)This does not make much sense, as the 'port' use makes little sense here, and hard to imagine what it is for.LATB is a uint32_t, so &LATB is the address of a uint32_t, so is a uint32_t*, and any math on it with be pointer math.Each register also has CLR/SET/INV registers, so each register (name) is spaced 16bytes apart, and presumably this is where the 0x10 comes in. Now port*16 + &LATB is pointer math on a uint32_t as &LATB is involved, so instead you now have another multiplication of 4* involved and your result is 4 registers away for every value of port (port=1, you get a register 64bytes away from LATB, or 4 register names away). Now how that can be useful, and where 'port' could possibly fit in, I'm not sure.You could provide more info on what port may be and its purpose. I would also make use of CLR/SET as much as possible so you get atomic masking/setting especially on pin registers.and a simple test to show pointer math is involved (port*16*4 is used, not port*16)-void test(uint32_t port){*(volatile uint32_t *)(&LATB + (port * 0x10)) = 0;}9d0000b2: 264c sll a0,a0,6a0 = a0*64 (a0 = port)9d0000b4: 41a2 bf80 lui v0,0xbf809d0000b8: 3042 2ce0 addiu v0,v0,11488v0 = 0xbf802ce0 (&LATB)9d0000bc: 0644 addu a0,v0,a0a0 = v0+a0 = 0xbf802ce0 + port*649d0000be: e840 sw zero,0(a0)a0 = 09d0000c0: 45bf jrc raWhile I'm at it, since LATB is already a volatile uint32_t, you can skip the casting-void test(uint32_t port){*(&LATB + (port * 0x10)) = 0;}Or put another way-void test(uint32_t port){(&LATB)[port * 0x10] = 0;}

author: Ground_Loop
author_rank: Frequent Contributor
timestamp: July 09, 2021, 01:52:55 am
content: It's clear now. Thanks all.

author: NorthGuy
author_rank: Super Contributor
timestamp: July 09, 2021, 03:38:34 am
content: Quote from: SiliconWizard on July 09, 2021, 12:26:14 amAnd by looking at this code you can even figure out what PIC32 family it is. This is one of the oldest PIC32MX795 family. LATB is at 0x1f886060, and LATC is at 0x1f8860a0. Exactly 0x10 32-bit words apart. New PICs are not like that.

author: cv007
author_rank: Frequent Contributor
timestamp: July 09, 2021, 06:33:08 am
content: QuoteWell, that explains what 'port' is doing, and it must be a 64pin variant if starting at latb. I'll add it the list of things I did not know, or do not know (its a big list, and I don't even know how big).Here is something I would tend to do in C, and I had something similar in C++ for a pic32mm which essentially did the same thing but more elaborate and complete. These pic32's have the set/clr registers, and you can/should take advantage of them, especially when atomic is required. Notice the lack of casting (because not needed) and pointer deference use, since array style use is what is needed here (and happens to be easier to read).Code:typedef volatile uint32_t vu32;typedef uint32_t u32;static void clrBits (vu32* reg, u32 v){ reg = v; } //regCLRstatic void setBits (vu32* reg, u32 v){ reg = v; } //regSETstatic void togBits (vu32* reg, u32 v){ reg = v; } //regINVstatic void setMask (vu32* reg, u32 mask, u32 val){ clrBits(reg,mask); setBits(reg,val); }//all pin functions based on PORTxstatic void pinOutput(vu32* reg, u32 pinbm){ clrBits(&reg[-4],pinbm); } //TRISstatic void pinInput (vu32* reg, u32 pinbm){ setBits(&reg[-4],pinbm); } //TRIS//will set to output when any of these are used, so no need to init pinsstatic void pinLow (vu32* reg, u32 pinbm){ pinOutput(reg,pinbm); clrBits(&reg,pinbm); } //LATstatic void pinHigh (vu32* reg, u32 pinbm){ pinOutput(reg,pinbm); setBits(&reg,pinbm); } //LATstatic void pinTog (vu32* reg, u32 pinbm){ pinOutput(reg,pinbm); togBits(&reg,pinbm); } //LAT//obviously incomplete for pin functions// &PORTx, pin bitmask (use defines in header, will restrict you to available pins)#define LED &PORTB, _PORTB_RB3_MASKint main(){while (1) {pinTog(LED);//delay}}

author: Ground_Loop
author_rank: Frequent Contributor
timestamp: July 09, 2021, 11:34:40 am
content: Wow, super sleuths. It is indeed a PIC32MX564F128 in QFP64 left over from a Unisolder build. I am attempting to repurpose it as a BLDC motor controller. No real end use for the motor, just goofing in the lab. Thanks again. My learn something new every day goal is satisfied.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 09, 2021, 04:53:25 pm
content: Don't hesitate to take a look at those SET, CLR and INV registers we mentioned for GPIOs. They are very handy!

author: peter-h
author_rank: Super Contributor
timestamp: July 10, 2021, 06:25:10 pm
content: Avoid pointers. They are at the root of all evil97.5% of bugs are caused by pointers(actually that may not be far off)

author: brucehoult
author_rank: Super Contributor
timestamp: July 11, 2021, 12:25:27 am
content: Quote from: peter-h on July 10, 2021, 06:25:10 pm99.9% of people who die have ingested dihydrogen-monoxide within the 7 days prior to death.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 11, 2021, 01:09:25 am
content: Quote from: brucehoult on July 11, 2021, 12:25:27 am

author: retiredfeline
author_rank: Frequent Contributor
timestamp: July 11, 2021, 01:19:20 am
content: Quote from: peter-h on July 10, 2021, 06:25:10 pmClose to 100% of software bugs involve the instruction pointer (aka program counter).

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 11, 2021, 02:10:41 am
content: This is entertaining:https://tylervigen.com/spurious-correlations

author: brucehoult
author_rank: Super Contributor
timestamp: July 11, 2021, 04:42:01 am
content: Quote from: SiliconWizard on July 11, 2021, 02:10:41 amReminds me of social scientists, climatologists, and statistical illiterates (but I repeat myself) who breathlessly publish papers saying their pet hypothesis is significant at the 95% level.So one hypothesis in twenty would show this degree or correlation by chance. How many hypotheses did they generate and test? Only enough for one of them to pass the tests...I like that the atomic physics people with their particle accelerators use five sigma, not two like the social scientists. "Significant at the 99.99994% level" is far more convincing.

author: peter-h
author_rank: Super Contributor
timestamp: July 11, 2021, 09:13:46 am
content: What I wrote is not totally BS.Look up some guidelines on programming of life-critical systems, like medical stuff, engine ECUs, etc.It starts with no auto variables (all storage is static).Of course pointers "work", but a statically declared array and using an explicit index is a lot more obvious, and probably generates exactly the same code in the end.Of course, everybody here is an expert

author: Siwastaja
author_rank: Super Contributor
timestamp: July 11, 2021, 09:32:36 am
content: It's ridiculous to say such widely used, completely basic language construct everybody uses all the time causes a large number of bugs.In reality, bugs are complex and related to thedesignof the software and interaction between modules.Bugs "caused by pointers" may be common in beginner code but in real professional work, such stupidly simple bugs are caught easily by the coders themselves, code reviews, and analysis tools (static and run-time).Of course, despite of that, every now and then a stupid beginner bug (for example, "caused by" a wrong way of using a pointer in C) sneaks in critical production code but the percentage is not some 97.5% but more like 0.1%. Most of the bugs are at higher level and surprisingly, language independent.In poor quality code, bugs related toinput validation, internalstate validation, andrange checksare very very typical. None of which requires pointers. Of course, you may set up a pointer at a wrong value and forget to check, but equally easily you can overindex a statically allocated array, actually very common mistake in beginner code with beginners who don't know how to use pointers yet. When they learn about pointers, then they start making mistakes with them, until they learn.But this reminds me of speed limits. Finland has lowest speed limits among Europe, and people, say 80-90%, routinely exceed them slightly, which is actually encouraged publicly by the police. Yet we get to read about statistics that "speeding causes 90% of accidents". If you look further, there are always more underlying reasons, for example alcohol use. But if an alcoholic DUI driver kills someoneandexceeds the speed limit, it's classified as caused by exceeding the speed limit. In the same way, a pointer "causes" bugs.Yes, it's true some embedded rule sets limit or completely forbid pointer use but that also limits the scope of such rule sets. In a complex project, you practically can't completely avoid pointers in C. Pointers are crucial for example when writing modular, reusable code, for example a data processing functionneedsa pointer for the data in order to avoid copying massive amounts of data around. Such pointer use can be made safe quite easily, as well. Good rule sets limit or add rules regarding some more dangerous patterns like typecasting a pointer to a different type.

author: peter-h
author_rank: Super Contributor
timestamp: July 11, 2021, 10:20:48 am
content: As I said, everyone here is an expert, so none of this appliesSure, if passing an array of say uint8_t fred  then you would pass it to a function by its address, which AIUI can befredwhich is the equivalent of&fredand one could call that a "pointer", but that isn't the same thing as less than expert C programmers screwing up by mistakes in the use of pointers elsewhere.

author: gf
author_rank: Super Contributor
timestamp: July 11, 2021, 11:38:16 am
content: Quote from: Siwastaja on July 11, 2021, 09:32:36 amWhen you mentiontypecasting, then yet another pitfall comes into my mind: I guess that quite some developers are not aware of "Strict Alising" semantics1)of recent C and C++ versions, but today's optimizers do exploit these semantics mercilessly, generating object code then, which does not necessarily do what the (unaware) programmers expect. Strict Aliasing semantincs are optimizer-friendly (enabling optimizations which were not possible otherwise), but I'd not call them programmer-friendly.Much existing legacy C/C++ code (written before Strict Aliasing was introduced in the C and C++ standards) does not work correctly with modern compilers, unless the compiler is explicitly instructednot to exploitstrict aliasing semantics (e.g. GCC's -fno-strict-aliasing option, or by explicity specifying an option to assume an older version of the language).1)Seehttps://en.cppreference.com/w/c/language/object, sections Effective Type and Strict Aliasing.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 11, 2021, 12:11:23 pm
content: Quote from: peter-h on July 11, 2021, 10:20:48 amA processing function taking in a pointer is maybe the most typical use of pointers, and avoiding it is extremely limiting.By reading your reply, it seems you somehow don't count this as a "pointer", but it is, you are just contradicting yourself. Even if you don't think of the array-pointer equivalency, inside the function, it's a very real and pure pointer.Sure, you can say that "only pass statically allocated arrays to functions expecting pointers and always accompany it with the length parameter and always use sizeof operator to obtain the array length to avoid mistyping length", but this isn't the same as "don't use pointers"; this is, "use pointers carefully and wisely".One point I forgot to mention; most of the "pointer mistakes" are actually mistakes caused by poor library interfaces. Things like sprintf family of functions overindexing the C strings, and even some of the *n* versions having traps. If you keep things simple and think about interfaces and interactions, "pointer mistakes" are really rare even when you do use pointers, and 99.99% of C code does, ignoring most trivially small programs.

author: NorthGuy
author_rank: Super Contributor
timestamp: July 12, 2021, 03:41:00 am
content: Quote from: peter-h on July 10, 2021, 06:25:10 pmImagine something simple. Say, you receive some serial data and you store them in a circular buffer. Now, these serial data may contain binary block which may be described by a C struct. So, you get the pointer to the first byte of it sitting in your circular buffer and cast it to the pointer to the said C struct. This gives you an easy way to access all the elements of the struct. Of course, circular buffer wraps while the struct doesn't, so instead of the struct the code may as well access complete garbage, may even crash. Terrible bug.So, what's the reason for it:A. pointer typecasting (many people will say it's bad)B. using C struct to access external data (often considered bad because of endiness, alignment, etc.)C. idiotic design (there's no rules against that)I argue it's C - ring buffer is not the best storage option for the task. What's important, after the bug is found, the programmer will blame it on A, on B, or similar. He will then go to forums and post how A and B are bad. But everyone seems to be content with C.

author: peter-h
author_rank: Super Contributor
timestamp: July 12, 2021, 07:54:00 am
content: My gripe with  use of pointers is thata) most C programmers don't comment their code much, and often don't comment it at all (there is a widespread view that good code doesn't need comments*)b) there is often a lot of pride invested in writing code which does clever (not illegal - I realise that adding 3 to a pointer pointing to an array of int32 actually increments the address by 12) things with pointer arithmetic and the resulting code is impenetrableI tend to use arrays and then I manipulate the index, which is more obvious when you come back to it after a year to change something. It also lends itself to more clear documentation e.g. if you have a buffer uint8_t inbuf which at some point has+0 length+1 packet ID+2 data...+9 checksumand the above is documented, then if you writex = inbuf;everybody can see you are picking up the checksum byte. Whereas somebody doing this with a pointer to inbuf will have probably wangled things so the pointer happens to point to  because one has already worked through the preceeding bytes and it just happens to work out right. I have seen tons of code like that and, without comments as it usually the case to show the programmer is a real man, it is unreadable. I've just seen a lot of it in a widely used embedded library (won't say which).The resulting code is probably identical, or so nearly identical that it doesn't matter, on a 168MHz CPU.But then, when I was at univ, 1970s, when I had to write something in Pascal (the then fashionable language which almost nobody used for anything real, apart from Delphi) I would write it in Fortran, punch out the cards, run and debug it on the ICL1904S, and then translate it back to Pascal to hand it in. The teachers hated that because it wasn't in the right "spirit" but it always worked* I spent many years programming in assembler and not commenting alotis just dumb; often every line of code would be commented

author: bson
author_rank: Unknown
timestamp: July 12, 2021, 11:34:31 am
content: Quote from: brucehoult on July 11, 2021, 04:42:01 am^ this100%and if it falls outside the 40% (!!!) error bound (hello climatologists) they just move the error bound. If you point out that maybe this means their model is cr*p and needs work, you get the usual "what do you know, you're not a scientist" type response. It's basically science as an ersatz religion.

author: bson
author_rank: Unknown
timestamp: July 12, 2021, 11:39:39 am
content: Quote from: peter-h on July 12, 2021, 07:54:00 amYou should really define a packed structure for memory layouts like this.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 12, 2021, 04:53:13 pm
content: Quote from: bson on July 12, 2021, 11:39:39 amYes, although you need to be careful with possible alignment issues. Now of course, if all fields are 8-bit integers, as seems to be the case in this small example, then there is no such problem.Apart from readability, using structs has the benefit of self-documenting their content, and making it easy to maintain (changing the structure layout will not break any code as long as you keep the existing member identifiers unmodified, as opposed to using arrays and constant indices. If using the latter, the minimum viable approach is at least to define macros for the indices - but that's still clunkier, and more work to maintain.

author: brucehoult
author_rank: Super Contributor
timestamp: July 13, 2021, 01:17:38 am
content: Quote from: SiliconWizard on July 12, 2021, 04:53:13 pmThe approved way to deal with that is to memcpy() the bytes to a variable of the struct type declared in the normal way (local or global).These days you're supposed to do that even for something as simple as casting a float to an int32. And the compiler should not only inline the memcpy but also deal with any potential alignment issues, if needed on that CPU type.

author: peter-h
author_rank: Super Contributor
timestamp: July 13, 2021, 10:49:35 am
content: I just googled on memcpy (which I have used lots of times) and surely it just copies over the actual bytes?So a memcpy of the address of a float to the address of a unit32_t will just give you garbage - unless you then want to use ANDs and shifts etc to extract the various bit fields e.g. the 1 byte exponent. I am told the more proper way to do that would be to use a union of the float and four bytes.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 13, 2021, 10:58:32 am
content: The point in using memcpy is that its interface works on void* pointers, i.e., 1-byte units which have no alignment restrictions on actual CPUs. Internally, it handles optimum way to access the data.So you can use memcpy to extract things arbitrarily aligned and copy them to somewhere to be properly aligned, allowing you to access for example the "float" inside.Because in most arcitectures, the float that resides at 0xacdc0000..0xacdc0003 can be accessed, but one at 0xacdc0001..0xacdc0004 cannot be accessed. Yet the latter may happen if you have a buffer collecting data byte by byte from UART, for example. Then the easiest way is to memcpy the relevant area to a properly aligned pointer, then read the float out of the latter.memcpy is rarely a library call today, it's such basic building block it's often implemented by compiler, and things happen like code containing no memcpy implicitly generating a call to memcpy, or code explicitly calling memcpy making compiler to use an inlined version instead, or optimize the whole thing away.

author: brucehoult
author_rank: Super Contributor
timestamp: July 13, 2021, 11:31:13 am
content: Quote from: peter-h on July 13, 2021, 10:49:35 amExactly. It's not garbage -- it's well defined.One thing it's useful to cast a float to int for is to add or subtract 1 from the int, and then convert back to float and ... hey presto ... you've got the next bigger or next smaller float.

author: peter-h
author_rank: Super Contributor
timestamp: July 13, 2021, 02:33:26 pm
content: Why would one do that? Int to float is a complicated conversion, with normalisation etc.

author: gf
author_rank: Super Contributor
timestamp: July 13, 2021, 03:13:57 pm
content: Quote from: brucehoult on July 13, 2021, 11:31:13 amThe C library (starting with C99) actually hasnextafter()to do this in a platform-independent way

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 13, 2021, 03:21:28 pm
content: Quote from: brucehoult on July 13, 2021, 01:17:38 amI'm assuming here you mean:- Defining a struct with no packing attribute (members all guaranteed to be properly aligned);- Copying data to each struct member separately one by one. Obviously copying the whole block wouldn't change anything.Yes this is the only portable way of dealing with this. But note that it defeats the benefit I was mentioning above, which was only requiring to define a struct (potentially packed), and not having to deal with data indices separately. Here, you have to maintain both.This is of course the only clean way of making this portable. But if you don't care about portability and can ensure you won't have any alignment problems on the target(s) you write for, then using a packed struct is still much simpler, easier to maintain, and more efficient.Quote from: brucehoult on July 13, 2021, 01:17:38 amI'm not sure I got this one. What exactly do you mean?Surely "casting" a float to an int32, at least in C and by definition of casting, is just a conversion like:Code:float x = yyy;int32_t n = (int32_t) x;This is what I call a cast. It can't be implemented with a memcpy. It will usually use either a call to a conversion function, or a FP instruction if there is one (and depending on optimizations, because the assembly instruction may be used only if you use the -ffast-math option in GCC, I guess?) You can't simply copy data for a float to int conversion in the general case.So, did you actually mean "aliasing a float to an int32 through pointers"? Such as:Code:float x = yyy;int32_t n = *(int32_t *) &x;(Not sure what purpose it would have, but that's possibly another question.)The above may have an alignment problem, although on typical FP implementations where a float is a 32-bit number, there shouldn't be an alignment issue there. Or, if by "float" you considered a double-precision FP number (or larger), I don't think there would be an alignment problem either, at least if said double is 64-bit wide, which is the standard case for IEEE FP numbers? Or, maybe you just considered a fully portable approach without assuming anything about FP numbers on a particular implementation, which could be neither 32-bit nor 64-bit wide?Or, did you mean something else entirely? Can you elaborate?

author: gf
author_rank: Super Contributor
timestamp: July 13, 2021, 04:19:40 pm
content: Quote from: SiliconWizard on July 13, 2021, 03:21:28 pmEven if there is no size or alignment problem, the behavior of such a pointer cast isundefined, according to recent versions of the C standard.Btw,hereis a wicked example, exploting the same kind of undefined behavior (although in a different context).

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 13, 2021, 04:43:26 pm
content: Quote from: gf on July 13, 2021, 04:19:40 pmIt isn't, of course. Generally speaking, aliasing data of different types through pointers is undefined if the types are not "compatible". Now of course, on many, if not most implementations, this will just blindly access memory through the same address. Obviously the result may vary. It's slippery and non-portable. But yes, you may still use this on some specific targets for specific reasons. Just make it clear to yourself and others that this isn't portable. (Just like for the packed structs.)

author: gf
author_rank: Super Contributor
timestamp: July 13, 2021, 05:13:45 pm
content: Quote from: SiliconWizard on July 13, 2021, 04:43:26 pmThe scurrility is that the optimizer is explicitly allowed to assume that an int* pointer and a float* pointer don't point to the same address (although the possibly do). Therefore test1() does not reload *pi from memory after storing a value to *pf, and the OR operator gets eliminated, too, by the optimizer. In test2(), using memcpy() to store the same value to *pf breaks the no-aliasing assumption, therefore *pi is reloaded from memory after memcpy() to *pf, and the OR operator is not eliminated, either. Consequenty, test1() and test2() are not equivalent, at the end (or they are only equivalent if pi and pf do not point to the same address).

author: NorthGuy
author_rank: Super Contributor
timestamp: July 13, 2021, 05:37:24 pm
content: Quote from: gf on July 13, 2021, 04:19:40 pmThis is simply a bug. It can be fixed:Code:int test1(volatile int *pi, float *pf){int i = *pi;// directly assign to *pf*pf = 5.0;return *pi | i;}

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 13, 2021, 05:42:55 pm
content: Quote from: gf on July 13, 2021, 05:13:45 pmYes, this is "interesting". So maybe that's the case Bruce had in mind (athough I would still not call that 'casting'.)Note that the compiler is assuming the pointers are different in Test1() while knowing nothing about that since they are parameters. If you modify the code like this:Code:int test3(int *pi){float *pf = (float *) pi;int i = *pi;// directly assign to *pf*pf = 5.0;return *pi | i;}Now it becomes correct. Of course, in this case, the compiler sees that the pointers are equal, so it can't assume they aren't. (Otherwise, I think it would be wise to throw it away...)

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 13, 2021, 05:45:14 pm
content: Quote from: NorthGuy on July 13, 2021, 05:37:24 pmThis isn't a "bug", since, as gf said, the compiler is allowed to make this assumption.Your version "fixes" it only because declaring pi volatile forces a memory access every time it's dereferenced. So it just can't optimize this access away.

author: NorthGuy
author_rank: Super Contributor
timestamp: July 13, 2021, 05:59:54 pm
content: Quote from: SiliconWizard on July 13, 2021, 05:45:14 pmThis is a bug precisely because the compiler is allowed to do this. Whoever wrote this code expected certain behaviour from the compiler and didn't know that to expect this behaviour he must use the "volatile" keyword. Hence, he wrote the code which doesn't do what he expected. If you look through the forums, there are hundreds of examples where people expect something where they shouldn't. This example is not any different. This example doesn't demonstrate any inherent dangers of pointers. It simply demonstrates that a programmer who doesn't know what he's doing will make bugs.

author: gf
author_rank: Super Contributor
timestamp: July 13, 2021, 06:19:47 pm
content: Quote from: NorthGuy on July 13, 2021, 05:59:54 pmExactly. That's just one particular pointer casting / aliasing pitfall I wanted to demonstrate.QuoteBut isn't this the root cause for a majority of so called "pointer bugs"?

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 13, 2021, 06:28:36 pm
content: Quote from: NorthGuy on July 13, 2021, 05:59:54 pmMy answer had assumed that you meant a compiler bug. It's not, even though it can certainly be confusing, especially since it happens only when a certain level of optimzation is enabled, possibly triggering the "it breaks when optimizations are on" syndrom.https://gcc.gnu.org/onlinedocs/gcc-4.6.1/gcc/Optimize-Options.html#index-fstrict_002daliasing-825But of course this is a programmer's bug.A "fix" would of course be not to write such horrible code. Using a volatile qualifier here just looks like a band-aid.As I showed, in all cases where the pointers are known to be the same by the compiler, there won't be a problem (at least, not a problem of this kind.)Another approach if you happen to know what you're doing and don't want to be bothered with strict aliasing rules is to pass the '-fno-strict-aliasing' option to GCC (or equivalent if any for whatever compiler you're using.) Not really recommended if this is just an excuse for writing sloppy and ultra-non-portable code, but could be useful for very specific and very low-level code for which you definitely don't want this kind of optimizations. I would still recommend this option for people very uncomfortable with strict aliasing rules and writing low-level code. As long again as they know this is again NOT portable.The problem with C programmers in general (as I already said in other threads) is that many assume C is a very simple language that you don't even need to learn, or barely. An approach that almost never works. This one thing is even rather tricky compared to most questions that flood forums, and which show the authors have probably never opened a book on C.As a PS: interestingly, a related question is when using pointers to compatible types, the compiler, not being able to assume they point to different things, will produce unoptimized code compared to what can be generated when it can assume they never point to the same thing. For this, the "restrict" qualifier has been added to C99. Of course when using it, the programmer has to make sure this assumption always holds true. This is pretty dangerous too.

author: gf
author_rank: Super Contributor
timestamp: July 13, 2021, 06:40:12 pm
content: Quote from: SiliconWizard on July 13, 2021, 06:28:36 pmThe language semantics do not guarantee this. It just happens, that gcc, llvm and others are clever enough not to enforce strict aliasing if they can prove that the pointers do alias, by tracking them to a common origin. As this is implementation dependent, you should not rely on it either.Edit:Quote from: SiliconWizard on July 13, 2021, 06:28:36 pmThere exists tons of legacy code which don't work correctly without -fno-strict-aliasing. Nobody wants to spend the effort to review thousands or even millions lines of code, in order to find and fix all strict aliasing violations, whille still missing 50% or more anyway. And the static analysis capabilities of the compilers are limited either, so they can only find a subset of the violations either. I'm in fact wondering whether the Linux kernel code is "strict-aliasing clean" in the meantime, or still compiled with -fno-sctrict-aliasing.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 13, 2021, 07:21:53 pm
content: Quote from: gf on July 13, 2021, 06:40:12 pmThe standard just says it's undefined behavior. That GCC and others handle this UB as an opportunity for optimizations is actually every bit as "odd" as not doing anything about it and just dereferencing the pointers "as if".Sure it's not portable, I said it a number of times already. So you shouldn't rely on anything about that *except* if you are actually not writing portable code for a reason, and you know your tools, something that is not uncommon again when writing very low-level code.One thing to note as well: if I'm not mistaken, strict aliasing rules were introduced in C11. I didn't see any mention of it in C99 (tell me if I missed it.) And many, many C developers are still not even using C99... But AFAIR, GCC introduced strict aliasing optimizations *before* C11 got released. Interestingly, even if you force C99 via 'std=c99', GCC will make this optimization. Please tell me if I missed the strict aliasing rules in C99.So I wouldn't totally blame developers for finding this particular point confusing.

author: gf
author_rank: Super Contributor
timestamp: July 13, 2021, 08:02:00 pm
content: Quote from: SiliconWizard on July 13, 2021, 07:21:53 pmIt is not really odd. IMO, strict aliasing semantings were clearly introduced into the standard on order to enable optimizations (by exploiting the newly introduced UB), which were not possible otherwise.I cannot imagine any other reason for introducing this kind of UB.The resulting confusion of programmers was certainly not intended, but obviously accepted as side-effect, in favor of better optimization.QuoteI don't remember exactly either whether it was introducced in C99 or C11. I rather had C11 in mind, but I may be wrong.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 13, 2021, 08:46:21 pm
content: Quote from: gf on July 13, 2021, 08:02:00 pmYep. You're right. It's just that UB is used throughout the standard with many different reasons as a rationale - but compilers are free to do whatever they want.As to favoring optimization rather than clarity for programmers, I'm not sure I'm completely on board with this. Oh well.Actually, the "restrict" qualifier bothers me much less - because in this case, the programmer's intent is much more clearly stated in written code.Quote from: gf on July 13, 2021, 08:02:00 pmYep. I think this is C11.Which would make using strict aliasing rules as default even for C99 in GCC, well... a bug IMHO. Programmers using std revisions earlier than C11 are not supposed to know about it.I'll have a look at GCC bugs to see if it has ever been mentioned or not.

author: NorthGuy
author_rank: Super Contributor
timestamp: July 13, 2021, 08:48:17 pm
content: Quote from: gf on July 13, 2021, 06:19:47 pmI think the majority of "pointer bugs" are simple mistakes. Since the effect may be everywhere, even in places which are not related to the "pointer code", these are perceived as extremely damaging, which causes pointer aversion.There may be some truth in it. But I think the benefits of using pointers outweigh the risk of bugs, by far.

author: gf
author_rank: Super Contributor
timestamp: July 13, 2021, 09:27:30 pm
content: Quote from: SiliconWizard on July 13, 2021, 08:46:21 pmWell, it does not bother, because it only takes effect if you actively use it.OTOH, with strict alising, a program which did work for many years may suddenly fail when compiled with a new compiler.How many experienced programmers which are using C since the 1970's are really aware of strict aliasing semantics, and the potential implications?Many legacy programs use pointer casts extensively.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 13, 2021, 09:52:22 pm
content: Quote from: gf on July 13, 2021, 09:27:30 pmYep.And, this is not too bad, if again your know your tools. For instance, if you use a certain version of GCC, you should know which version of C it supports by default. It's documented.And then, you should know the corresponding standard. But a reasonable approach for compiling "legacy" code is to force the compiler using the C std used when it was written. GCC (and CLANG I think) can do that by using the '-std=' option.Unfortunately, as we can see with GCC and this particular feature - strict aliasing optimizations - it seems to do it regardless of the std version you specify. Which, as I said above, I think is a bug. It looks pretty severe to me, even. Apparently (using the online compiler stuff), CLANG does the same thing. They may have used the same rationale - which I'm not sure I agree with. MSVC doesn't have this problem (but OTOH, it seems to optimize a lot less.)I re-read C99 on pointers and didn't see anything about this. Aliasing issues are not even mentioned really, apart for function pointers.I took a look at GCC's bugs. There are 225 currently with "strict aliasing" keywords. I haven't seen one exactly dealing with this std version issue so far from the titles, but I haven't read them all either apart from titles, so... I'd be tempted to file a bug and see what they say. But since there is a workaround (using the '-fno-strict-aliasing' option as I said earlier), they may not care one bit.

author: NorthGuy
author_rank: Super Contributor
timestamp: July 14, 2021, 01:16:54 am
content: Quote from: SiliconWizard on July 13, 2021, 09:52:22 pmThat's definitely not in C99. I tried your example and even with --std=c89 GCC still behaves like this. Nobody would think of strict aliasing in 1989. But it's been there for a while. I tried GCC 4.5 for i86 built in 2012. It's already there. May be this was their idea to push it into the standard. So, this wrong implementation has persisted for 10 years.Since they didn't bother to fix this, it's probably not too many occurrences. Of all of the uses of aliasing, the majority will be strict. Of those which are not strict, only handful will allow some sort of optimization. Of those which can be optimized, the majority will not be hurt by the optimization. So, this bug may not be discovered for a long time. And then it'll pop up in some super secure applicationIMHO, the compilers were nicely optimized around 20 years ago. Before then there were bugs in optimizations, unoptimized code was somewhat safer. After that, there's a lot of development, but not that much progress - diminishing returns. I wonder where it goes next.

author: brucehoult
author_rank: Super Contributor
timestamp: July 14, 2021, 01:32:55 am
content: Quote from: NorthGuy on July 14, 2021, 01:16:54 amgcc 2.95.x (1999-2001) was a good reliable compiler.The first few releases of gcc 3 were just awful, bugs everywhere. I seem to recall it was ok by 3.2 or 3.3.Modern gcc is fine.The biggest thing as far as I'm concerned is support for alternative ISAs. By 2006 or 2007 gcc was as good as or better than ARM's expensive compiler. Support for AVR is good. Support for RISC-V is good (though LLVM is now as good or better, and taking the lead for supporting new extensions, after being quite incomplete even three years ago)

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 14, 2021, 05:56:57 pm
content: Glad I'm not the only one thinking this strict aliasing thing being enabled by default regardless of the C std you specify is definitely BAD.As I said, they probably think that since there is an option flag for disabling it, this is good enough. And as I said, I think people not knowing/being comfortable with strict aliasing rules or working with older code should definitely disable this.But I still don't know why they would make strict aliasing optimizations enabled by default for all C std revisions. That doesn't make sense. A reasonable approach would have been to make it default only starting with C11, and disabled by default for earlier revisions. And of course, you could always enable it selectively with the '-fstrict-aliasing' option if you have checked this is safe for a given code base. Why make it default?As I said, IIRC, I think GCC introduced strict aliasing before C11 was officially released, so that may have been a reason why it has remained uncorrelated with specific C std revisions.I'm considering filing a bug for this. (I've already filed two bugs for GCC, one has been accepted and now fixed, the other is still pending.) I'm curious what they will have to say.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 15, 2021, 08:26:29 am
content: Strict aliasing rule is a massive blunder. It's one of the big mistakes in the recent history of C.I would understand it if it was accidentally introduced in 1970's to 1980's when C was originally designed as a then "high-level" language, along with other "great" ideas like implementation defined ranges and sizes of built-in types like int. But it isn't, it's a new idea that broke half of the world's code bases, making it inexcusable.Coming into late 1990's, C was already a low-level system programming language.Most nontrivial codebases used, and still use type aliasing through pointers becausethat makes a lot of sense. In assembly, no one questions using different instructions to access the data. In C, that is done either through union type punning,which was originally forbidden (or UB) by the original standard, or through pointer casting.Then, out of nowhere, the GCC folks just broke down most of the existing code. I'm not sure about the timeline, it seems the mistake in the standard was madelaterthat, basically retrospectively covering the language lawyers so they can now say "hey, we are allowed by standard to break your code, tough luck"!? This caused massive backlash in linux kernel community which is one of the largest, successful C code bases.It's not only a compiler bug, it's against common sense to use this inverted logic, "hey, we break your program, but by the way, there is a flag you can give on command line to prevent us breaking your program". It should work the opposite way: "hey, I as a programmer guarantee that different types of pointers always point to different memory areas so you can use this assumption everywhere to optimize". I.e.,, it should be -fstrict-aliasing instead of -fno-strict-aliasing, and it shouldn't be enabled by default on any optimization level.The thing is, strict aliasing rule is a crappy fix to performance issues regarding pointer assumptions. It is not a good optimization,because compatible types can often be guaranteed by programmer to not point to the same address, either! Strict aliasing rule misses this most obvious case.There simply is no other way than to let the programmer explicitly tell the compiler whenever this assumption can be made, and the plus side is, now the programmer can tell this even with compatible types, enabling much much better optimizations to be made. And hey, the restrict keyword was introduced for this reason! This should have prevented the addition of strict aliasing rule, because now everybody who needs the optimization can have it, while others retain correct operating of their programs.For example, a processing_function(int* data, int* n_processed) benefits from this assumption but strict aliasing rule doesn't help the slightest. Instead, processing_function (int* restrict data, int* restrict n_processed) does the trick.There is an easy workaround around the broken standard:1) Admit the standard is the problem. It's not a golden book of good design, never was, never supposed to be. In this case, the standard was originally fine; first it was abused by GCC, then broken.2) As a result of 1, always compile code with -fno-strict-aliasing, accepting the ugliness of this inverted logic of having to say "don't break my good code"3) Use therestrictkeyword extensively (but carefully) to allow optimizations explicitlyFinally, it's a funny argument to call strict aliasing related bugs "pointer bugs", or to say "hey, I told you! Pointers are dangerous!" after deliberately causing the bug. After all, the problem isn't in usage of pointers, the problem is in the fairly new, arbitrary and very debatable compiler feature, backed up by a late mistake in the standard.It's still true though that aliasing through pointers requires care, due toalignment reasons. So like goto, not every beginner should be using this feature on every five lines of their code; if that happens, it's likely a sign of lacking the knowledge how to use other (higher-level) language concepts.

author: Nominal Animal
author_rank: Super Contributor
timestamp: July 15, 2021, 10:46:51 am
content: Quote from: Siwastaja on July 15, 2021, 08:26:29 amYup. Type punning via an union (as opposed via pointer or copying the storage representation) has been explicitly mentioned as a valid approach in a footnote in C99 (6.5.2.3, page 77, footnote 83), C11 (6.5.2.3, page 83, footnote 95), and C17 (6.5.2.3, page 59, footnote 97):Quotewhere 6.2.6 Representations of types describes how object representation (memory contents) of various types are interpreted.Nevertheless, I still regularly encounter C language lawyers who insist that type punning via an union is still not allowed in C99 and later, due to some wording or other elsewhere in the standard. Because, and I repeat,"The footnotes are not normative; they are for information only, and you (Nominal Animal) are interpreting that footnote wrong".With C compilers used to compile BSD or POSIX C code, type punning via an union is the safest approach, because any "cleverness" here by the C compiler likely meansstruct sockaddrstops working. (It does not rely on type punning precisely, but on 6.5.2.3p6, common initial sequences, but it typically boils down to the same code in the compiler itself: whether it examines the memory, or tries to be "clever" about it and breaks real-world BSD/POSIX C code. So, consider them closely related details, where breaking one likely breaks the other, and breaking the latter breaks a LOT of existing code.)I no longer point to that footnote, just tostruct sockaddr, exactly because real world always wins over a paper standard. (Okay, I did above, but just to point out it has been there 20 years in pretty darn plain language, and still causes arguments.)I happen to need type punning surprisingly often. A very common use case is to convert IEEE-754 Binary32 (float) or Binary64 (double) to uint32_t or uint64_t, respectively (by inverting the most significant bit if unset, and all bits if set; this retains the same ordering. (That is, if you do that bit flip, then < ≤ = ≠ ≥ > in one form applies in the other too, assuming the floating-point value is finite. This is how e.g. radix sorting floating-point values is done.) When you don't need to operate on the values, but do need to compare them (say, select/find), it is sometimes faster to do it in the integer domain. In my binary storage formats, I also use a prototype IEEE-754 values (where each byte of the object storage has an unique value), so a quick test can detect and handle any byte order conversion issues, and that too requires type punning between the unsigned integer and floating point types.

author: peter-h
author_rank: Super Contributor
timestamp: July 15, 2021, 11:12:14 am
content: The complexity of this discussion (which is way above my pay grade) shows that I was right about pointers being ... dodgyAnd I don't think anything has really changed. There are some really good programmers ("developers" or even "artisans" might be current terminology) but they are rare and always have been rare.Go back to the first windows programs, win3.x. Nearly all the GPFs were null pointer related. That was early 1990s. And people still do it today. On simple CPUs, writing address 0 doesn't do anything, so if this is a side effect, it never gets picked up. I think one of the common functions like strtoul() does this, unless you enter NULL for one of the parameters

author: NorthGuy
author_rank: Super Contributor
timestamp: July 15, 2021, 12:58:01 pm
content: Actually, the strict aliasing may have been in C99 already. The ISO/IEC 9899:TC3 document (dated 2007) says:"An object shall have its stored value accessed only by an lvalue expression that has one ofthe following types:—a type compatible with the effective type of the object,—a qualified version of a type compatible with the effective type of the object,—a type that is the signed or unsigned type corresponding to the effective type of theobject,—a type that is the signed or unsigned type corresponding to a qualified version of theeffective type of the object,—an aggregate or union type that includes one of the aforementioned types among itsmembers (including, recursively, a member of a subaggregate or contained union), or—a character type."

author: Siwastaja
author_rank: Super Contributor
timestamp: July 15, 2021, 04:14:16 pm
content: Quote from: peter-h on July 15, 2021, 11:12:14 amThis is a ridiculous concept. Something (for example, a surgery) is not "dodgy" just becauseyoudon't understand it.C, like many powerful tools, is full of traps. Only some of them relate to pointers.And in C, pointers are everywhere, so any bug is likely to be somewhere where pointers are "nearby", just like car accidents happen when people drive normal driving speeds yet they are not caused by driving normal driving speeds.Also, you basically can't use C to write non-trivial programs without using pointers, so that makes whole C language "dodgy".I hear a lot of people downplaying C simply becausetheyare not proficient in it. For the same reason, you can't engineer bridges or do pretty much anything nice we have today.It's OK not to understand and not to use C, not everyone needs to do that. But claiming pointers are dangerous or dodgy or is just... stupid. Large part of our computer infrastructure, including the very Internet, works written on C, with alotof pointers everywhere, and it works just fine, large problems are at totally different level, higher design level.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 15, 2021, 05:18:42 pm
content: Quote from: peter-h on July 15, 2021, 11:12:14 amThey are not dodgy. But they are a powerful low-level tool, thus can be dangerous in the wrong hands. Like any other tool. A kitchen knife in the wrong/inexperienced hands can lead to death. Would you say they are dodgy?As I already said, the main problem with C is that many people assume it's so simple that it's obvious, and doesn't even need proper learning. And then they get all worked up as soon as they run into a problem. This just is plain wrong. But still, many people keep doing that. In a way, we could say that C is so 'simple' and permissive that it makes it too tempting to start using it without even opening a book - let alone read standards... with more complex languages - even C++ - you usually can't do anything useful before you have learned at least some of it properly, so it forces you to learn. Now I'll also admit that the teaching of C has been (except when you were very lucky with one great teacher) very poor in schools/universities, so that has never helped. And unfortunately, although still heavily used in embedded dev, C is not even taught anymore in prog. language courses, or barely. These days, you often run into it when you're taking an EE curriculum, and are never taught properly.And with that said, C is certainly not without its quirks, as some of us have said. But as long as those are well documented, this is "just" a matter of learning. Again.The bad is when there are quirks that could be undocumented - such as this strict aliasing thing made default on some compilers regardless of the C std. *This* isn't good.Now don't get me wrong - although strict aliasing rules were not defined until C11, I don't blame the commitee for introducing them clearly. Although of course confusing if you *don't know* about them, they IMHO are pretty much good sense when you know that pointer dereference in C is just an access and doesn't imply a conversion of the pointed to data if there's an object mismatch. So, except again in very particular situations where you'd do this for a very good and low-level reason, knowing well it wouldn't be portable, aliasing objects with different pointer types is something that any programmer should know, even if just intuitively when it was not clearly stated in the standard, is like shooting in the dark.Now there are certainly languages that allow you some very low-level stuff in a much cleaner way. ADA comes to mind. But ADA is not for everybody.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 15, 2021, 05:29:17 pm
content: Quote from: NorthGuy on July 15, 2021, 12:58:01 pmJust checked and I have this in my copy of C99 as well.I guess you could say that it implies the strict aliasing rules mentioned explicitely as such in C11.The oddity is that this is in the "expressions" paragraph, and nothing of the sort is mentioned in the part dealing with pointers. They may have possibly considered it was general enough not to require being repeated in the "pointers" section. But if so, it looks clear that the committee ended up realizing it required precisions in C11.Now as we said above, GCC does enable strict aliasing by default even for the C89/C90 revision. I unfortunately do not have a copy of it, so I can't tell if the same was already mentioned.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 15, 2021, 05:45:20 pm
content: I agree the almost total lack of proper teaching of C has been a devastating failure of our universities, as far as I understand it's a universal problem.C, after all, pretty much "runs" the backbones of the modern digital world. It's not C only, but C has a pretty big meaning including both understanding traditional projects (like UNIX, linux, networking), and, quite surprisingly, in new projects as well, not even limited to embedded / lowest level.Universitiesloveto go for either "easy" modern modern languages, or monsters like C++. And that's stupid, because they also limit the mindset. Similarly as you shouldn't ignore C++, you shouldn't ignore C either. It's pretty much needed.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 15, 2021, 06:11:11 pm
content: Quote from: Siwastaja on July 15, 2021, 05:45:20 pmFor at least the last 10 years, it's been mostly Java. I can guarantee you that when you expose students that started with Java to C, things get fun. (Not necessarily in a good way.)

author: brucehoult
author_rank: Super Contributor
timestamp: July 15, 2021, 08:06:11 pm
content: Quote from: SiliconWizard on July 15, 2021, 06:11:11 pmThe switch to Java was 20+ years ago. Everyone I know of has switched to Python 10 years ago. They have similar levels of abstraction and safety, but Python doesn't require you to write half a screenful toprint "HelloWorld\n".