C strange struct behavior, what happens to size? - Page 1
Forum section: electronics / general

author: legacy
author_rank: Super Contributor
timestamp: September 20, 2014, 01:12:03 am
content: Code:#include <stdio.h>typedef struct{unsigned long aadata; /* 4 byte */unsigned char aasign; /* 1 byte */unsigned char aasign2; /* 1 byte */} my_t;my_t my;void my_size(){printf("size=%lu byte\n", sizeof(my_t));printf("size=%lu byte\n", sizeof(my));}int main(){my_size();return 0;}Code:size=8 bytesize=8 byte4+1+1 = 8It seems it is aligning by 4 bytes, but it should not be a common default on linux-PC/x86what is happening ?

author: c4757p
author_rank: Super Contributor
timestamp: September 20, 2014, 01:17:13 am
content: This can differ from compiler to compiler, but a guess is this: my.aadata is aligned on 4-byte boundaries because it is 4 bytes long. If you were to create an array of my_t, this would require two bytes of padding to be added to the struct to maintain this alignment for subsequent array elements, so it is added. In other words, the struct is padded to a multiple of the alignment of its most coarsely aligned member.

author: IanB
author_rank: Super Contributor
timestamp: September 20, 2014, 01:18:14 am
content: The compiler is adding some padding so the struct aligns at suitable boundaries. Otherwise you could end up with inefficient memory accesses.There may be a compiler switch to prevent this behavior.

author: miguelvp
author_rank: Super Contributor
timestamp: September 20, 2014, 01:20:26 am
content: Quote from: legacy on September 20, 2014, 01:12:03 amTry with the lines I added, it's an alignment thing.Edit: maybe stdio includes a header that sets the packing to 4, or it might be the default packing of the compiler.Edit2: but you might want to leave it as 4 and live with it, data fetching is more efficient at the processor's default alignment.

author: IanB
author_rank: Super Contributor
timestamp: September 20, 2014, 01:21:14 am
content: By the way, this behavior is quite common, normal and expected. But it may perhaps be considered a "trap for young players".

author: legacy
author_rank: Super Contributor
timestamp: September 20, 2014, 01:23:19 am
content: understoodi am using these gcc version on my x86 laptop, they both have the same behavior i686-pc-linux-gnu-4.1.2 * i686-pc-linux-gnu-4.4.5also tested on arm/le machine with armv5tel-softfloat-linux-gnueabi-4.1.2 * armv5tel-softfloat-linux-gnueabi-4.3.4what is the switch to avoid padding ?

author: IanB
author_rank: Super Contributor
timestamp: September 20, 2014, 01:28:23 am
content: Quote from: legacy on September 20, 2014, 01:23:19 amRead the documentation?BTW, unless you are mapping to hardware addresses, you don't want to avoid padding. The compiler knows what it is doing.

author: legacy
author_rank: Super Contributor
timestamp: September 20, 2014, 01:32:46 am
content: Quote from: IanB on September 20, 2014, 01:21:14 ami was expecting something like in an embedded metal bare profiled toolchain with a linker file which forces alignmentMIPS has a bad reaction if you access to a data-address which is not good alignment, there are special instructions in MIPS ISA in order to access on misalignment data instead of accessing data with load/store (1) but they are not suggested to be used, btw, never considered on linux machine(1) which causes a bus exception, so you need an exception routine that fixes the problem, and about that, linux has the properly exception routine code while a bare bone firmware (e.g. the early releases of Yamon) may not have it, so this is the reason why you force a bare bone linker script to be good aligned for the must!

author: miguelvp
author_rank: Super Contributor
timestamp: September 20, 2014, 01:33:40 am
content: I guess you missed it on my previous post#pragma pack(push) // Store current packing.#pragma pack(1) // change to one byte packing.typedef struct{unsigned long aadata; /* 4 byte */unsigned char aasign; /* 1 byte */unsigned char aasign2; /* 1 byte */} my_t;#pragma pack(pop) // restore packing to previous setting.

author: legacy
author_rank: Super Contributor
timestamp: September 20, 2014, 01:36:02 am
content: Quote from: IanB on September 20, 2014, 01:28:23 amit's exactly what i am doing, unfortunately, then i got the unexpected behavior, so i isolated the "sizeof" code and i found what it is wrong!I think i will use a pretty ugly array of uint8_t (of the right size) to short up the story

author: miguelvp
author_rank: Super Contributor
timestamp: September 20, 2014, 01:37:21 am
content: I guess I'm on legacy's ignore list

author: legacy
author_rank: Super Contributor
timestamp: September 20, 2014, 01:39:15 am
content: @miguelvpyour pragma hints is perfectly working on both x86 and arm linux machinesthank you

author: miguelvp
author_rank: Super Contributor
timestamp: September 20, 2014, 01:47:54 am
content: I guess I'm not in your ignore list after allYou are welcome. But f you deal with an array of your struct there will be data fetching penalties. Processors pre-fetch data and miss-alignments will stall the pre-fetch queue. for for some structures you might not want to force it to 1 byte alignment.But for single instance structs there will be no penalty so you are good.

author: T3sl4co1l
author_rank: Super Contributor
timestamp: September 20, 2014, 01:55:29 am
content: Also... if you want any semblance of consistency in variable sizes, DON'T USE PLATFORM DEPENDENT SIZES.What's "long"? Since the 386 days, "int" was 32 bits, so "long" must be 64. So your struct should actually be 10 bytes (assuming packed).But what if "char" is Unicode? If it's UTF-16, your struct swells to 12 bytes!Maybe you copy the interface code to something embedded, like an Arduino. Now "long" is 32 or even 16 bits (I don't know offhand).Solution: always use (u)intNN_t types (NN = 8, 16, 32, etc.). Your struct should be,uint32_t aadata;uint8_t aasign;uint8_t aasign2;Now it is guaranteed to always take the same number of bytes (assuming identical packing, of course), and guaranteed to have the same numerical range, on any platform.(Yes, always writing these types is a pain, but you can typedef a whole lot of things for handy namespacing.)Tim

author: legacy
author_rank: Super Contributor
timestamp: September 20, 2014, 01:59:25 am
content: @miguelvpyou are welcome, toowhile you was posting i was trying your patch on both machines, x86 and arm, with different gcc versionsumm, so complex CPUs of now a days, that is the price we have to payi have also been developing a very simple RISC processor which has no issues like that, no fetching penalties, it's like the first MIPS R2000 but much more simple because it has no pipeline, so it's more like 68000, very pretty to be programmed.Unfortunately i am now dealing with an hyper complex Atom/x86 and with a pretty XScale-PXA270 arm processor (it's old, but my bos want it for his reasons), and they are both pipelined.btw, i have fixed the issues with the radio link! Everything is working and i can commit everything out

author: legacy
author_rank: Super Contributor
timestamp: September 20, 2014, 02:08:27 am
content: Quote from: T3sl4co1l on September 20, 2014, 01:55:29 amyes, in my final code i have pretty #include "types.h" in where i defined everything i am using.i was not expecting this behavior, I have found a bug in my code, then, to be sure it was a gcc problem, i "searched and replaced" my typedefs replacing them with C standard definitions, and to be more sure about the problem i have also removed every other includes (stdio.h and others) to be really really sure it was a padding or something like that.Thank you guys, the bug has been completely demystified

author: SirNick
author_rank: Frequent Contributor
timestamp: September 20, 2014, 03:50:18 am
content: Quote from: legacy on September 20, 2014, 01:23:19 amThat looks like eselect. O0

author: hans
author_rank: Super Contributor
timestamp: September 20, 2014, 07:54:05 am
content: Quote from: legacy on September 20, 2014, 02:08:27 amConsider #include <stdint.h>; it has uint8_t, uint16_t, uint32_t, int8_t etc. defined. It doesn't get much clearer than unsigned/signed + bit width.In addition to struct packing; that can be dangerous on some platforms if I'm correct. I believe x86 is pretty forgiving in that it will be slightly slower in accessing memory but will do it.Some platforms like ARM can choke on packed structs:http://infocenter.arm.com/help/topic/com.arm.doc.dui0473c/Cihdbfje.htmlQuoteNevertheless you can turn checks off, the hardware will not correct for it and may result in incorrect behavior.

author: nctnico
author_rank: Super Contributor
timestamp: September 20, 2014, 11:12:37 am
content: Quote from: legacy on September 20, 2014, 01:23:19 amYou can use something with packed-structs or assign an attribute to the variable using the array declaration. Still I strongly recommend to avoid using packed structs. ARM for example can access a 32 bit value only from addresses dividable by 4. If the compiler isn't sure where a struct is it will divide a 32 bit access into single bytes accesses and build the 32 bit value from that. It will make your code bloated and slow.

author: T3sl4co1l
author_rank: Super Contributor
timestamp: September 20, 2014, 11:30:54 am
content: You can also design the struct for the platform (unportable code, no, of course not!...), so, say, using uint32_t for the size number, then a uint32_t for flags which just happens to include the two byte-size things you had, plus either "reserved" (unused) bits, or even part of the next variable, if applicable. This way, instead of, say, checking byte variables (which are compiled as 32 bits anyway) for individual values and flags, you check the same 32 bit variable for more flags. That way, you're handling the same 32 bits as exactly that, 32 bits, and the compiler output follows much more closely from your input -- which helps reduce misunderstandings when inspecting the disassembled output, and misunderstandings between compiler and platform limitations when you do run into weird or unexpected behavior.Or if it's not random flags, but the bytes are actually numeric, you can byte-slice them into variables at the point of use (which can be in a variable, or an inline expression like "(int8_t)((var & 0x0000ff00) >>" or whatever), and let the compiler decide how it's going to approach that (32 bit processors often have byte and nibble swapping or selecting instructions). On very advanced processors, you'll even have the luxury of packed (SIMD) instructions, which if you're doing the same actions to all the bytes, can save even more execution time.The resulting code should still be portable, but will be suboptimal on other platforms. Applying full-width bitmasking operations on uint32_t variables on an 8-bit processor would be simply dreadful. So if you're well and truly designing the code for multi-platform use, maybe you'd add #ifdefs for different cores or WORD sizes or whatever. Which would be a little easier to maintain than separate implementations altogether, but not by much.Tim

author: miguelvp
author_rank: Super Contributor
timestamp: September 20, 2014, 05:00:34 pm
content: Quote from: hans on September 20, 2014, 07:54:05 amAn alignment exception doesn't mean incorrect behaviour just that it will incur a performance penalty.which of course you don't want. But for a single instance like his code there will be no exception since the packing gets restored and other variables will still be aligned, the struct packed as one byte has no internal missalignment either and with or without the packing pragma the two bytes left will be unused either way only difference being that it will report the actual struct size.

author: grumpydoc
author_rank: Super Contributor
timestamp: September 20, 2014, 06:36:25 pm
content: QuoteNo, "long" is generally 32 bits, unless your compiler makes plain "int" 64 bits."long long" is what you want for 64 bits.The rule is that char ? short ? int ? long ? long long with minimum sizes char = 8bits, short and int = 16 bits, long = 32 bits and long long = 64 bits.The reason as usual is history. On 16 bit platforms "int" could be 16 or 32 bits and you generally had to say "long" to get 32 bits. Then 32 bit platforms came along and "int"" became 32 bits, long also stuck at 32 bits for compatibility with older software written for long = 32 bits. long long was introduced as the "guaranteed 64 bit" type.QuoteOn x86 you just get poor performance, on many RISC architectures you get a bus error which can result in your process being terminated with prejudice

author: vvanders
author_rank: Regular Contributor
timestamp: September 21, 2014, 06:56:55 pm
content: Quote from: miguelvp on September 20, 2014, 05:00:34 pmDepends on the platform, some platforms will halt on an unaligned read(AKA don't do that).It's also worth noting that bitfields have similar undefined issues as the compiler is free to reorder where they are placed.

author: miguelvp
author_rank: Super Contributor
timestamp: September 21, 2014, 08:08:42 pm
content: Then stay away from MIPS and some ARM chips, problem solvedIn any event, the code:Code:#pragma pack(push) // Store current packing.#pragma pack(1) // change to one byte packing.typedef struct{unsigned long aadata; /* 4 byte */unsigned char aasign; /* 1 byte */unsigned char aasign2; /* 1 byte */} my_t;#pragma pack(pop) // restore packing to previous setting.Wont cause misalignment, if some obscure chip can't handle it, the compiler will surely scream at you.Instances of my_t will be aligned and everything is well.But let me know by all means a processor that doesn't support #pragma pack(1)

author: miguelvp
author_rank: Super Contributor
timestamp: September 21, 2014, 08:21:07 pm
content: Quote from: vvanders on September 21, 2014, 06:56:55 pmAlso I would like to know what compiler doesn't support unions of larger data types with bitfield descriptions and why will a bit field of 0 be part of C if the compiler can reorder it and don't force a storage alignment.Maybe endianness might be confusing to some, but then they have no place in using bitfields in the first place.

author: westfw
author_rank: Super Contributor
timestamp: September 21, 2014, 10:04:17 pm
content: QuoteOne normally hopes that the compiler is smart enough to use byte/halfword instructions to access bytes and halfwords, so that you can substitute "be careful when you define the data structure" for "avoid cpus that can't do unaligned accesses." As long as you have something like this (which will cover allmost all hardware register overlay applications):Code:struct {uint32_t longfield;uint16_t shortfield;uint8_t bytefield;uint8_t padding1;}but you would have problems (if structures are packed) with accessing "shortfield" in thisCode:struct {uint32_t longfield;uint8_t bytefield;uint16_t shortfield;}(and yet, for Computer-Science-y applications, you don't want to force programmers to manually align the structures.)

author: true
author_rank: Frequent Contributor
timestamp: September 22, 2014, 05:01:27 am
content: Quote from: SirNick on September 20, 2014, 03:50:18 amOr probably gcc-config -lBut probably without systemd either way, which is the best way

author: Jeroen3
author_rank: Super Contributor
timestamp: September 22, 2014, 02:09:57 pm
content: Play around with Offsetof.http://en.wikipedia.org/wiki/Offsetofto see how it packs stuff. (compilers can also warn you when they insert padding bytes)

author: legacy
author_rank: Super Contributor
timestamp: September 22, 2014, 02:30:21 pm
content: Quote from: true on September 22, 2014, 05:01:27 amI am running gentoo/linux on {MIPS, ARM, X86} machinesI can switch the C compiler with "gcc-config"followed by "source /etc/profile" to update the environment

author: Bassman59
author_rank: Super Contributor
timestamp: September 22, 2014, 05:19:23 pm
content: Quote from: miguelvp on September 20, 2014, 01:47:54 amIf you use the structure to encapsulate hardware registers or I/O access, it's gotta be in a memory space marked as not-prefetchable (unless the hardware was designed with that capability in mind).

author: Maxlor
author_rank: Frequent Contributor
timestamp: September 22, 2014, 09:20:57 pm
content: Quote from: grumpydoc on September 20, 2014, 06:36:25 pmThere were some Unix systems that used 64bit ints, but I'm not aware of any current one still doing that. Maybe there is an obscure one. But several different sizes are used for long on a 64bit machine. On most 64bit Unix-like systems, long is 64bits, whereas on 64bit Windows, long is 32bits.MCU toolchains might use weird sizes (I've encountered the C compiler for a DSP where char was 16 bits...), but they generally follow the rule: char <= short <= int <= long <= long long. That could mean that all types are 16bits. There's only one thing to do: read the documentation when using a new toolchain or platform for the first time.

author: legacy
author_rank: Super Contributor
timestamp: September 22, 2014, 10:39:50 pm
content: Quote from: Maxlor on September 22, 2014, 09:20:57 pmit happened to me, too, a lot of time ago, i think it was a Texas Instruments's DSP if i remember right

author: miguelvp
author_rank: Super Contributor
timestamp: September 22, 2014, 10:50:26 pm
content: Quote from: legacy on September 22, 2014, 10:39:50 pmKind of makes sense for UTF-16 Unicode, surprised it's not a default standard yet in this 21st century and 3rd millennium.

author: westfw
author_rank: Super Contributor
timestamp: September 22, 2014, 11:33:29 pm
content: "char" is 16bits in Java (and 16bits/character for strings.)They also apparently have "byte" to handle the 8bit case.

author: vvanders
author_rank: Regular Contributor
timestamp: September 23, 2014, 12:59:48 am
content: Don't even get me started on Java that abomination of a language doesn't even have the concept of unsigned because it's apparently too hard to handle the overflow edge case. Or the fact that every literal is an integer so you can't have a byte literal, argh.C# got so much of what Java was trying to do right, it's a shame it's so widely used and taught.

author: SirNick
author_rank: Frequent Contributor
timestamp: September 23, 2014, 08:32:29 pm
content: char shouldn't be 16-bits, because if that's what you need, there's often wchar just for that purpose. If we could turn back time, it would've been nice to have byte defined as an official type, and char could be whatever your platform prefers -- Unicode, ASCII, etc.Although, if we're time-traveling, just dump int, short, long, etc., and start with int8_t, uint8_t, and so on, instead. The size and endianness ambiguity of C is one of its biggest flaws WRT portability, in my not-so-expert opinion. Having intrinsic and reliable ways to solve those two issues would've been really nice to have from day one.