who decides where a variable will be placed in c  - Page 1
Forum section: electronics / general

author: Dadu@
author_rank: Contributor
timestamp: February 05, 2022, 08:16:43 am
content: The value of the variable is stored in the hardware's memory. Memory is mainly divided into two parts Stack and Heap.who decides where a variable will be placed?Does the compiler decide where the variable should be placed?Do storage classes decide where variables should be kept

author: Siwastaja
author_rank: Super Contributor
timestamp: February 05, 2022, 08:38:08 am
content: You can control it, if you want.Local variables go into stack, which usually grows downwards.Heap allocations are by using malloc()/new. This means run-time algorithms decide where it goes; they might try to fill in gaps formed earlier by free(). Seeing this is a microcontoller subforum, I don't usually recommend using heap (malloc()) on microcontroller project at all, although this isn't a hard rule set in stone.Further, you can modify the linker script to configure your memory layout, create sections that are placed where you want them, and instruct the compiler to place variables in these sections using attributes.

author: FlyingDutch
author_rank: Regular Contributor
timestamp: February 05, 2022, 10:48:12 am
content: Hello @Siwastaja,I am wondering how it is in small microcontrollers like for example ATMega328, wich have only 2KB RAM. How this small memory is divided into stack and heap - where I can check how many memory is used for this both (stack and heap). It it possible to change sizes of stack and heap using "Arduino IDE? In programming STM32 (ARM-Cortex) I know how to perform such task (I am using "STM32CubeIDE").Thanks in advance and Regards

author: Siwastaja
author_rank: Super Contributor
timestamp: February 05, 2022, 11:26:40 am
content: I have never used heap in AVR with 2K of RAM, and just recommend not doing it. I don't know if Arduino library framework uses heap, but my guesstimate is no.The whole malloc()/free() business is really designed for larger allocations than a few bytes.Anyway, simple embedded projects that would be written in AVR rarely benefit from dynamic memory management. The project does mostly one thing, and has to keep the memory related to that thing. There rarely are paths in which you would free() some memory and then malloc() something to do a different thing. In other words, a refrigerator temperature controller does not suddenly change its role into a web browser.Instead, you just use the two classic memory allocation methods:* static allocation, by placing variables as globals, or if inside functions, qualified withstatickeyword. Now they exist in a fixed place in memory from the start. Compiler/linker decides what the exact address is, but you can affect this by modifying the linker script.* stack allocation, local variables (notstatic) inside functions. This is kind-of dynamic memory management, if you call a function, the stack size increases, but after returning from it, the stack size decreases again.There is one quite rare pattern which might be useful in embedded programming, alternative to heap (malloc()/free()): if there are two distinct operation modes in your product, which both need large memory buffers, but never run "in parallel", but in such way that the memory can be always cleared when swapping from one function to another, you can use the Cunionto define two sets of variables, buffers, whatever, so that they share the same physical memory.Code:union{struct{int a;float b;char buf;} operation_a_state;struct{char foo;float matrix;int bar;} operation_b_state;}In modern desktop computing, this isn't very useful as it makes more sense to just malloc() what you need, and when not needed anymore, free() it. But in simple embedded targets, this kind of semi-static manual memory management is a good alternative to "real" dynamic memory. You have all the control and can easily check you always have the memory what you need, because every variable has been assigned with an actual place in memory.Just be aware of the risks of stack. It's dynamic memory after all. Use -fstack-usage on GCC, but still, you'd need to guarantee the call graph.

author: Nominal Animal
author_rank: Super Contributor
timestamp: February 05, 2022, 11:34:49 am
content: On 8-bit AVRs in Arduino, the stack and heap form a single contiguous chunk of RAM, where stack grows down from the top, and heap is used starting at the bottom. (If those two cross, your code no longer works reliably. Yes, you can usemalloc()/free()even on AVRs with just 2k of RAM, because this part is actually provided by avr-libc the Arduino libraries build on top of. This also means you get the same behaviour if you write your own code using avr-gcc and avr-libc, and use a compatible linker script. It is the linker script that determines the various memory regions in the device memory.)In File > Preferences, Settings tab, make sure you have Show verbose output: During compilation selected/checked. (While you're there, make sure you also select Compiler warnings: All.)Then, after a successful compile, the two last lines of your output will beSketch usesFLASHSIZEbytes (FLASH%) of program space. Maximum isMAXFLASHSIZEbytes.Global variables useRAMUSEbytes (RAM%) of dynamic memory, leavingRAMFREEbytes for local variables. Maximum isRAMTOTALbytes.whereFLASHSIZEis the amount of Flash memory used by your program (includes code and constants);MAXFLASHSIZEis the amount of Flash available for your program (excluding bootloader etc.).RAMUSEis the amount of RAM statically allocated for global variables (including objects provided by the libraries you use);RAMTOTALis the total amount of RAM available for an Arduino program.RAMFREEis the amount of memory allocated for the shared stack and heap area. Again, the two are not split in Arduino for AVRs; they use the same area, one growing up and the other down.

author: hans
author_rank: Super Contributor
timestamp: February 05, 2022, 11:39:16 am
content: The memory layout is described by a linker file, as Siwastaja said.Typically the RAM layout will be: pre-initialized variables (.data section, the values are first stored in FLASH and copy over by the start-up routine), uninitialized variables (.bss section), and the heap. The heap is assigned a size in the linker file. The stack pointer is put at the end of RAM.If you make a variable const then that variable won't appear in RAM (data section) but only in FLASH.These are the standard classifications that exist in the C language. If you want to do more specific things with a variable, you can use attributes to put it at a specific memory address, or modify the linker file and add a new section for it.For example, an ARM processor typically has a TCM and multiple SRAM blocks. Typically the TCM is only connected to the CPU, so a DMA controller cannot reach the TCM memory and therefore any DMA buffers must be placed outside of TCM.@comment below: That's true, but AVR architecture was implicit by OP, and so there are many more exceptions. Also modifying the linker script with a certain intent is completely valid. E.g. running code from RAM. But that's getting a bit beyond the scope of the initial question.

author: Siwastaja
author_rank: Super Contributor
timestamp: February 05, 2022, 11:48:37 am
content: Quote from: hans on February 05, 2022, 11:39:16 amThis isn't always true, for example AVR copies consts to RAM because of the architecture where FLASH isn't simply memory mapped to the same address space but requires special instructions to access. This is also why string literals spend so much precious RAM, unless specified and accessed with special accessor macros. On ARM, this isn't a problem;constusually places the variable in flash only.But even on ARM targets, it's well possible some linker script places .rodata after .data, in RAM. I did that modification just a few days ago because a DMA couldn't access flash.

author: westfw
author_rank: Super Contributor
timestamp: February 07, 2022, 11:22:59 am
content: QuoteMemory is mostly divided into FOUR parts:Initialized data (.data segment)Uninitialized data (.bss segment)Stack for dynamic allocation of local variables. And maybe return addresses, saved state, and function arguments.Heap for explicit dynamic allocation of data by the programmer.QuoteThe compiler and linker conspire to place variables within .data, .bss, and where in the chip's memory those "segments" will go.The compiler determines how stack variables will be allocated on the stack (perhaps following some standard ABI for the chip in question.)The compiler, linker, and maybe the runtime setup code, conspire to decide where the stack and heap will go.Some library (and perhaps an OS) is responsible for carving up the memory allocated to the heap into pieces, as directed by the user program (malloc, free, new, delete, etc.)On chips with very limited memory, it pretty common to have a downward growing stack start at the "end" of memory, and an upward growing heap starting AFTER the .data and .bss areas, without specific size limits. It is possible that the stack and heap will collide (with very bad results), but that's seen as better than wasting memory. Also, in many small embedded systems, it's quite rare to actually use the heap; various standards (MISRA?) explicitly prohibit the use of dynamically allocated memory.Here's a page (with nice pictures!) describing how gcc works on AVRs:https://www.nongnu.org/avr-libc/user-manual/malloc.html

author: peter-h
author_rank: Super Contributor
timestamp: February 09, 2022, 05:55:43 pm
content: With just 2k RAM (I have programmed AVRs with 64 bytes) the best way is to make every "large" buffer static i.e.- declaredoutsideof any function (note main() is a function too!), or- if inside a function, putstaticin front of itWith the sort of default linker scripts one tends to get, that should work right.In these limited-RAM situations, one also tends to re-use buffers by different functions if necessary.Small variables declared inside functions are ok because they end up on the stack and provided you are careful with your stack size (stack pointer is initialised to the top of RAM plus 1, and the stack grows downwards) versus statics (which are allocated from bottom upwards) and the two never colliding (which should produce a warning at link-time, but not if your stack overruns) this should work fine.Never use the heap (malloc etc) in an embedded system unless it is one of- every malloc() is matched by a free() and there is no RTOS doing the same thing elsewhere**- a malloc() is used to allocate a block of RAM for a process which is an optional feature and, once started, never terminateswhich is ok.** many would argue this makes the heap pointless, and they would be generally right because the same thing can be achieved by declaring a local buffer inside a function, which neatly gets chucked away when the function exits

author: Siwastaja
author_rank: Super Contributor
timestamp: February 09, 2022, 06:45:12 pm
content: Quote from: peter-h on February 09, 2022, 05:55:43 pmFurther, if there is just one such optional feature, may as well statically allocate that block. The benefit is, the linker then reveals if there is memory for it. Also results in more available memory because you don't need to add the artificial barrier for static vs. heap storage.If there are many of those features, out of which only one is active at a time, usingunionfor static allocation has the same benefits.malloc() starts to make sense when any combination of those optional features, more than one at once, might be used, with not enough memory to statically allocate all of them. Even then, you could do it manually, but it is some real work and people want to use pre-existing solutions to avoid this work, and malloc is a well tested one. If you use it once just in init, fragmentation of heap won't be a problem.

author: peter-h
author_rank: Super Contributor
timestamp: February 09, 2022, 07:20:13 pm
content: QuoteI normally haveuint8_t odds_buf[whatever size];for that purposeThen it is obvious that this "odd jobs buffer" is shared between any number of functions. It will blow your code up equally well whether you have just the one, or a union of several

author: Siwastaja
author_rank: Super Contributor
timestamp: February 10, 2022, 10:23:11 am
content: Of course that does the trick if it's really a byte buffer.But often programs contain more complex data than single bytes, for example, 32-bit numbers! In such case, working around the uint8_t buffer by casting is pure horror.Variable namesis also quite handy feature in C, to make programs readable, instead of having to remember that hey, this variable is located at buffer! So just use structs and unions to your advantage.

author: Nominal Animal
author_rank: Super Contributor
timestamp: February 10, 2022, 11:30:32 am
content: It's not like the union is that different:Code:#define SCRATCH_BYTES 57union {unsigned char uc;unsigned short us[SCRATCH_BYTES / sizeof (unsigned short)];unsigned int ui[SCRATCH_BYTES / sizeof (unsigned int)];unsigned long ul[SCRATCH_BYTES / sizeof (unsigned long)];signed char sc;char c;float f[SCRATCH_BYTES / sizeof (float)];double d[SCRATCH_BYTES / sizeof (double)];} scratch;If you want to use the scratch buffer as chars, you usescratch.c. If you want to use it as doubles, you usescratch.d.If you need the number of elements you can put in sayscratch.ul, just use(sizeof scratch.ul / sizeof scratch.ul).The compiler will align the entire union so that it is sufficiently aligned for all types in the union, something that is not guaranteed if you just type-pun a pointer to a static char array.Converting such a scratch union into a stack, where "allocations" are strictly nested (freeing an earlier allocation frees all later allocations too) is very simple, and could be useful in some cases.

author: Siwastaja
author_rank: Super Contributor
timestamp: February 10, 2022, 11:43:23 am
content: I mean, in real program the "optional feature" contains whole set of variables, likeCode:struct{int num_things;double transformation_matrix;char name;float latest_temperature;};Put this struct inside the union, with all other alternative "optional feature" structs, and the code can be written normally, with sane variable names and normal type handling of C.