memset() and \0 termination - Page 1
Forum section: electronics / general

author: ricko_uk
author_rank: Super Contributor
timestamp: June 30, 2021, 02:10:18 am
content: Hi,couple of questions:1) if you do not initialise a char[], does the compiler automatically fill it with '\0'? Are there variations depending on the compiler standard/version?2) the following code snippets should yield the same result and NOT go out of boundaries or anything else correct? Asking because the debugger sometimes seems to show different behavious in the last char#define SIZE 4char sz_dest;memset(sz_dest, 'A', SIZE);sz_dest[SIZE - 1) = '\0"and#define SIZE 4char sz_dest;memset(sz_dest, 'A', SIZE-1);sz_dest[SIZE - 1) = '\0"Thank you

author: Fredderic
author_rank: Regular Contributor
timestamp: June 30, 2021, 02:23:42 am
content: I'm no expert… So just filling in until someone better comes along. But…"It depends". If your char is global, then the memory it occupies willprobablybe zero-initialised. If it's stack or heap allocated, then definitely not.Typo's aside, theyshouldhave the same result… the last char being determined by the last statement, which is the same in both cases. The first one would just set the last char twice.

author: ricko_uk
author_rank: Super Contributor
timestamp: June 30, 2021, 02:57:02 am
content: Thank you Fredderic1) yes, I assumed so but was wondering if there are differences between compiler standards2) found what was causing the strange behaviourThank you

author: ejeffrey
author_rank: Super Contributor
timestamp: June 30, 2021, 04:11:24 am
content: Initialization of global / static variables to zero is part of the C standard. That doesn't mean you can't muck around the the linker script to make it non compliant -- but initialization is standard and all compilers should do this in their normal configuration.Automatic variables and malloc() allocations don't do initialization but you have to watch out. The OS may zero-initialize memory before giving it to the application process. This can lead to people believing that their compiler is doing initialization until you do a few malloc/free cycles and are now reusing heap memory. calloc() does zero-initialize the memory.

author: ricko_uk
author_rank: Super Contributor
timestamp: June 30, 2021, 04:52:32 am
content: Thank you Ejeffreyyes I thought the linker might affect that but wasn't sure because I never played/modified linkers before.

author: Siwastaja
author_rank: Super Contributor
timestamp: June 30, 2021, 06:15:23 am
content: Quote from: Fredderic on June 30, 2021, 02:23:42 amNot probably,it is. Doesn't need to be global,staticinside a function is zero-initialized too.Exception to this rule would be a custom startup code intentionally breaking the C standard. You know if this is the case (basically, you have written the custom startup code.)Because this is basic 101 C and so widely trusted, I would be surprised to see buggy implementations out there. You can trust it. If it doesn't work, then ditch the niche compiler in question, it's likely completely unusable in other ways, too.The next question would be,whensuch "uninitialized" global/static are initialized to zero? The answer is,before main()by the startup code compiler inserts. Sometimes on MCUs you need extremely short startup times which could be a reason to write your own startup code, for example performing something importantbeforethe .data copy and .bss zeroing.Local variables need explicit initialization. Beware, contents can be anything that happens to be in that part of memory from previous operations, this is a classic security leak, maybe you happened to store some secret nuke launch code in the same memory area before!memset is a classic for initializing local (stack) variables. To reduce risk of bugs when modifying the code later, usage ofsizeofis recommended in the last argument.What comes to the code examples, functionally they do the same. The former just does unnecessary work by first memseting the last element to 'A', then overwriting it to 0.But please, copy-paste the same code which you have compiled and tested, this way you ensure there are no new typos included.

author: ajb
author_rank: Super Contributor
timestamp: June 30, 2021, 03:08:35 pm
content: What Siwastaja said. There is also a shorthand for zero initialization, which is to do:Code:char someArray = {0};In C, if an array is partially initialized, any extra elements that aren't covered by the initialization list will be initialized to zero. So you only need to be explicit about the first element and everything else will be zeroed implicitly.That's all about what the compiler will do for you, and anyone can reference your code against the C standard to determine what will or will not be initialized to zero, however this will not tell them what you WANTED to be initialized to zero versus what's getting zeroed just because that's the compiler's default behavior. So if you have an array that MUST be initialized to zero for the rest of the code to function properly, then it's a good idea to document that, which you can do using the zero initialization above as a hint that the zero initialization is deliberate--of course you can also just use a comment. That said, if your code depends on that zero initialization then you may want to include zeroing the array in your startup or initialization code. The implicit zero initialization may be adequate when the application starts up, but sometimes you need to be able to reset/reinitialize a module within your application, and if that module requires an array that starts out zeroed you will need to reinitialize it explicitly. This is one aspect of writing maintainable, well-decoupled software: each module of an application should be capable of independent and complete initialization, without depending on the rest of the application being in any particular state or providing any additional setup. That's the ideal anyway, of course in reality that's not always the way things happen for many good (and bad) reasons, but at the very least it's worth keeping in mind as you start laying out a software project.

author: SiliconWizard
author_rank: Super Contributor
timestamp: June 30, 2021, 05:46:18 pm
content: Quote from: ricko_uk on June 30, 2021, 02:10:18 amAs others have already said, it is guaranteed by the C standard, as long as said array is a global or static local array.OTOH, a non-static-qualified array local to a function is NOT automatically intialized with all zeroes. Some specific compiler options may allow this, but this isn't guaranteed by the standard, and would be compiler and option-specific. So, always initialize your non-static local arrays if they need to be.Quote from: ricko_uk on June 30, 2021, 02:10:18 amBoth of your examples are fine.Of course, a couple points:- The sz_dest[SIZE - 1) = '\0'; part is unnecessary if again the array is a global array. If it is local, then it is necessary.- The same thing could be just written: char sz_dest = "AAA";Your version has the benefit of being able to accomodate any length by just modifying the SIZE macro. But, unless you have a specific reason for specifying the size with a macro this way, note that a much simpler version would just be:char sz_dest[] = "AAA";This syntax is valid in gloabl as well as local contexts. The size of the array is determined by the constant at compile time, and in this case, it will be 4, as string literals always include the final zero in C.

author: westfw
author_rank: Super Contributor
timestamp: July 01, 2021, 08:14:45 am
content: QuoteWe had a system that initialized RAM to 0xFF (heap for sure. BSS as well, I think.) Because if you started followed wild pointer chains, 0 would be a valid pointer (in the sense that there was real memory there), but -1 definitely wasn't. I suspect that that was undone a long time ago, though. "cleverness" is one thing, but once you reach a certain size, "doing what every expects" becomes a lot more important.

author: harerod
author_rank: Frequent Contributor
timestamp: July 01, 2021, 04:12:02 pm
content: Quote from: westfw on July 01, 2021, 08:14:45 amInitialising RAM to a non-zero pattern may help to figure out memory usage. Its usefulness depends on the actual application. It can help with questions like:- How far are stack and heap apart? (non-os platforms)- Which portions of memory have been written? (wild pointers)

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 01, 2021, 04:48:53 pm
content: Quote from: westfw on July 01, 2021, 08:14:45 amAs others have said above, this just isn't a standard-compliant implementation if it does that with the .bss segment.But, certainly, some tools may do stuff that are not compliant. The important point is knowing your tools, and in particular knowing what it is they do that is not compliant.Also, as someone mentioned, this stuff can be, and is usually done in the "startup" code. In particular, zero-ing out the .bss segment is done in it. It's pretty easy to modify it and initialize this segment with other values. It's just non-compliant.As to the heap or stack, the initial content is unspecified in the standard AFAIK, so you're free to fill them with whatever you please. That would, again, usually be done in the startup code. Although most implementations just leave them as they are, some may indeed fill them with specific "magic" values. For instance to analyze usage, as was talked about in the thread in which we discussed stacks.Just a quick additional note: even though the standard ensures zero-initiliazed global/static variables (when compliant), I for one have never relied on this. if anything, not necessarily because I do not trust my tools to be compliant, but because I think it makes code less readable. So I usually always initialize them (when they need to be, otherwise I don't care) with some value. Even if it's just 0.Edit2: come to think of it, filling the .bss segment with something else than zero in the startup code would not work if the compiler itself is compliant. This is why: global/static variables that are explicitely initialized *with zero* will also end up in the .bss segment. At least that's what GCC does. So if the startup code fills the segment with anything else than 0, that would break the code badly. Unless you make it a rule NEVER to explicitely initialize those variables. That would be extremely confusing though. So I doubt your tools were doing this with the .bss segment. But, as I said above, that would be fine for the heap and the stack.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 01, 2021, 06:07:41 pm
content: Quote from: SiliconWizard on July 01, 2021, 04:48:53 pm... or not. I mean, it's a sane thing for a compiler to do, but in my experience, this doesn't necessarily happen.One of the reasons I picked up a habit ofnotexplicitly initializing globals/statics to zero was that exactly GCC (forgot which version, which platform - possibly 4.something on AVR or ARM) put explicitly zero-initialized in .data, consuming flash memory by storing all the zeroes. I also picked up the habit of commenting it "initialized to zero by C standard" but dropped it at some point assuming this is such basic feature in C anyone who works with my code should just know this, like they are supposed to know how the ?: operator works, and that bitshift operators for signed integers are arithmetic and not logical shifts, for example; those are not obvious to beginners, either, and no one wastes a comment how they work unless it's a tutorial. Comments are better used to describe design decisions ("why" and bigger-picture "how"), and "what" only when it's some really weird / rarely known.Indeed, the whole reason I can think of why .bss exists in the first place is optimization of program size because zero is the single most usual initialization value. Otherwise than that, zero-initialization of globals/statics could be fulfilled by automatically adding them to .data with zero values, by compiler. While arbitrary initialization is expensive, if startup time is of no consideration, initializing to zero is a very low cost thing to do, to the point of C standard writers not coming up with an alternative way to tell the compilernotto initialize globals/statics. Initializing stack objects to zero would insert the delays not to startup but all over the program so clearly they thought programmer needs to do that explicitly, which was not a bad idea.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 01, 2021, 06:14:22 pm
content: Quote from: Siwastaja on July 01, 2021, 06:07:41 pmI haven't seen that in GCC for many versions, but I admit I absolutely don't remember the exact behavior of GCC 4.x or older. It doesn't matter.My point was: it's reasonable for a compiler to do it, and as it is allowed to do so from the standard, any compiler *may* do this, and GCC has certainly been doing this for a number of versions now. Which is a good reason enough not to initiliaze the .bss segment in the startup code with anything else than 0. Otherwise you're bound to break code in very nasty ways. End of the story. That was just a rationale for never fiddling with the .bss segment in this way.And that said, whereas we can see a benefit of initializing the heap and the stack with magic values (and other than zero), I can't see any for the .bss segment. So point was, it's extremely unlikely any platform would have done this.And the main point of the .bss segment is indeed to save code size. Storing unnecessary zero values in code is a waste of code memory. It can make a lot of difference if you have big global arrays and data structures.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 01, 2021, 06:17:54 pm
content: Indeed, if you want to leave something uninitialized, or initialize to other weird values than zero, don't mess up with .bss init code, instead addanother, clearly named section like .noinit or .init_to_42 and do whatever you want with it in init code. This way you have the control, and still retain standard-compliant operation on everything which do not carry the very visible __attributes__ in code.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 01, 2021, 06:21:13 pm
content: Sure, but I still can't see any reason for doing this. Absolutely none.If you're doing this just to keep track of unitialized global variables, any decent static analysis tool will tell you that without requiring to get to the link phase and mess with segments.If you're doing this to automatically initiliaze global variables with a value other than zero, well, as I said earlier, I think it would be an atrocious decision from any readabilitu and code maintenance POV. Just never even think of that. Even the thought of it sounds twisted.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 01, 2021, 06:56:18 pm
content: One reason: Startup time. Often extremely important in embedded. Clearing .bss takes time. Not much, but measurable in microseconds, maybe even some single-digit milliseconds!Although, I prefer to add the Critical Things in the startup code before .bss zeroing / .data copy.But sometimes you don't have a few Critical Things but instead, want to run the whole thing i.e. enter main() as soon as possible.Especially worth noting is modern MCUs boot with some super slow internal RC oscillator. You may want to switch to an external clock sooner than at main(). ... and do .bss/.data loops while waiting for the PLL stabilization flagYou may also want to retain values through software reset for any weird reason.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 01, 2021, 07:17:18 pm
content: Quote from: Siwastaja on July 01, 2021, 06:56:18 pmBut. I'm not sure I get what you really meant then. I was more talking about automatically initializing global variables with a value other than zero. Here you are talking about not initiliazing them at all. That's different. And, in this particular case, yes, I can see the benefit (as long as you are cautious when writing code).Actually, possibly the most efficient approach would be:- Put global initializers in the .data segment as usual (those that are not all zeroes);- Put non-initialized global variables in the .bss segment;- Except those that you explicitely locate (with an attribute) in a separate dedicated segment.Now that you explained your rationale, that's probably exactly what you meant. And then, I agree.Quote from: Siwastaja on July 01, 2021, 06:56:18 pmWell yeah. On some MCUs, the "boot" time can be pretty long, and variable initialization at startup is actually pretty negligible in most cases. In some particular cases, as you mentioned, you may optimize things with the PLL, if using any. That would require putting the PLL config code in the startup code, which is OK but probably less "readable" than having it in your source files. But, for some MCUs which have no embedded Flash and run code from external Flash, really, the boot phase usually exceeds by a large margin anything you do in the startup code... so yeah.Quote from: Siwastaja on July 01, 2021, 06:56:18 pmThat would be a good reason, as long as said processor guarantees RAM integrity after a reset. On some, there are a number of dedicated registers instead. Or, a small RAM area dedicated to this. Of course, then, you need to define it in the linker script as a dedicated segment.Which leads us a bit astray, because I was of course not questioning the potential benefits of defining custom segments.

author: bson
author_rank: Unknown
timestamp: July 01, 2021, 10:11:43 pm
content: There are very good practical reasons for retaining memory across reset. I sometimes put aside a small data section in memory as a ring buffer, and define putc() or the equivalent delegate to stick trace messages there in a debug build. Then if it resets the first thing I do is if there's something in the ring stop and wait to attach a debugger with a flashing LED. (Same if the debug build panics.) I attach a debugger and can check what the last 2k or so of console output was to see what it was doing during the reset. This is very useful for debugging things like spurious watchdog and brownout resets. (And, of course, if the processor has a register indicating why it reset, I stash that away as well at startup.) For example, if you go to switch a relay and the relay switches and but the processor resets (which may be so fast you don't even notice it if you aren't looking for it) it's probably due to a supply drop.

author: westfw
author_rank: Super Contributor
timestamp: July 02, 2021, 12:12:07 am
content: QuoteKids today!The aforementioned hack was from back in the mid-80s, before gcc existed and probably before "decent static analysis" tools existed, either. (and it might have been back when the C specification was "weaker" as well; certainly before compiler "compliance" was a big issue.)

author: Siwastaja
author_rank: Super Contributor
timestamp: July 02, 2021, 02:05:28 pm
content: Quote from: SiliconWizard on July 01, 2021, 07:17:18 pmScroll up.Me: "if you want to leave something uninitialized ..."You: "Sure, but I still can't see any reason for doing this. Absolutely none."Replied to that.