How do I enable link section overlap  - Page 1
Forum section: electronics / general

author: peter-h
author_rank: Super Contributor
timestamp: September 25, 2021, 09:41:34 pm
content: In my linkfile I have this bit which locates a RAM based loaderCode:/* RAM based loader is linked to execute at top of RAM minus 32k = 0x20018000 *//* In the finished KDE this is within the heap area, so there is no possible runtime conflict *//* Note that if you create enough static objects (which end up in BSS) to push the end of BSS past0x20018000 then you will get a linker error, which is due to this dumb linkfile */KDE_loader 0x20018000 : AT(_loader_loadaddr){_loader_start = .;*KDE_loader.o(.text*)*KDE_loader.o(.rodata*)*KDE_loader.o(.data*)*KDE_loader.o(.common*)*KDE_loader.o(.ARM.attributes). = ALIGN(4);_loader_end = .;}Unfortunately, as soon as I have statically allocated stuff (data+bss) reaching 0x20018000 I get a linker error re the section overlap.This isn't a real error because when the RAM based loader is active, none of the data+bss is being used . Well, only the first few k, starting at 0x20000000.The relevant rest of the linkfile which locates data+bss isCode:/* Initialized data sections goes into RAM, load LMA copy after code */.data :{. = ALIGN(4);_sdata = .; /* create a global symbol at data start */*(.data) /* .data sections */*(.data*) /* .data* sections */. = ALIGN(4);_edata = .; /* define a global symbol at data end */} >RAM AT >FLASH_BOOT/* used by the startup to initialize data */_sidata = LOADADDR(.data);/* dummy placeholder in flash for loader section, to count flash usage */.KDE_loader :{. = . + SIZEOF(KDE_loader);} AT >FLASH_BOOT_loader_loadaddr = LOADADDR(.KDE_loader);/* Uninitialized data section */. = ALIGN(4);.bss :{/* This is used by the startup in order to initialize the .bss section */_sbss = .; /* define a global symbol at bss start */__bss_start__ = _sbss;*(.bss)*(.bss*)*(COMMON). = ALIGN(4);_ebss = .; /* define a global symbol at bss end */__bss_end__ = _ebss;} >RAM/* The heap ends up after BSS in main RAM *//* This also checks that the top of the heap doesn't hit the bottom of the stack i.e. how much RAM left *//* User_heap_stack section, used to check that there is enough RAM left *//* ._user_heap_stack : *//* This check is basically disabled because _Min_Heap_Size=0 above */.main_heap :{. = ALIGN(8);PROVIDE ( end = . );PROVIDE ( _end = . );. = . + _Min_Heap_Size;/* . = . + _Min_Stack_Size; */ /* PH 14/5/2021 stack is in CCM, not here */. = ALIGN(8);} >RAMFor simplicity, I would like the first bit to be ignored by the linker (loader) for the purpose of checking for overlap. But maybe one has to do it on the subsequent bits (data+bss) instead?

author: peter-h
author_rank: Super Contributor
timestamp: September 27, 2021, 01:51:46 pm
content: I've been working through the GCC manual and trying various things like OVERLAY or NOLOAD but nothing works to stop the section conflict error.

author: abyrvalg
author_rank: Frequent Contributor
timestamp: September 27, 2021, 05:55:08 pm
content: You need to declare a separate overlapping region inside MEMORY {} and place overlaid sections there. See the example I've sent to you long time ago - it has two RAM regions both starting at 0x20000000.

author: peter-h
author_rank: Super Contributor
timestamp: September 27, 2021, 09:05:40 pm
content: Yes; thank you. I just don't understand the rest of the linkfileThere are too many things I don't get.Linkfiles are a real art. I was looking for a simple fix but maybe there isn't one.In most applications of this product the area where the loader goes will be used by the heap, and I am not allocating the heap in the linkfile (it was allocated originally in the ST example one but it is a bit dumb because it merely prevents you from declaring large statics) so no overlap errors will be generated.

author: peter-h
author_rank: Super Contributor
timestamp: September 28, 2021, 09:00:25 am
content: I created separate sections in MEMORY but while it works as before it doesn't suppress the linker/loader error.Have you actually tried compiling your code, with some huge static likeuint8_t fred [127*1024];to try to conflict with the loader location?Code:/********************************************************************************** File : LinkerScript.ld**** Author : Peter**** Abstract : Linker script for STM32F417VG Device with** 1024KByte FLASH, 128KByte RAM + 64kbyte in CCM****** Target : 32F407/417***18/7/2021 PHmods to support a loader which executes at base of RAM i.e. 0x20000000.* 21/7/21PHloader RAM address changed to base+64k i.e. 0x20010000, and heap check removed (reason unclear but didn't* do anything useful because the heap grows downwards at runtime anyway and the linker can't see that)*24/7/21PHGeneral cleanup and comments; added buffer1*27/7/21PH45dbxx.o linked into boot block* 29/7/21PHBoot block increased from 16k to 32k*6/8/21PHpreinit_array stuff put back in (for __libc_init_array()).*20/9/21PHLoader moved to top of RAM minus 32k = 0x20018000* 26/9/21PHLoader moved to top of RAM minus 8k = 0x2001E000*29/9/21PHRAM_L added for RAM based loader, to try to stop linker error due to section conflict***//* Entry Point */ENTRY(Reset_Handler)/* Reference boot block stuff to ensure that it gets linked-in - not sure if these do anything *//* EXTERN(b_boot) *//* EXTERN(KDE_loader) *//* Highest address of the main stack. This stack is only for startup code, boot loader, and ISRs. The RTOS has its own. *//* _estack = 0x20020000; */ /* stack in 128K RAM */_estack = 0x10010000;/* stack in 64k CCM - note: configTOTAL_HEAP_SIZE + min_stack_size must not exceed 64k) *//* top of RAM for _sbrk - top of heap check */_top = 0x20020000;/* Heap and stack sizes */_Min_Heap_Size = 0x0000; /* 21/7/21 set to zero to prevent heap/stack conflict check from blowing up due to boot loader area */_Min_Stack_Size = 0x4000; /* 16k stack - in CCM *//* General purpose buffer for loader usage, b_main.c and KDE_loader.c.Done this way to avoid spurious section overlap errors; the loader reboots when doneso it can use any RAM it wants */buffer1 = 0x10000000;/* base of CCM *//* Specify the memory areas *//* CCMRAM added PH 12/5/21 - cannot use with DMA *//* RAM_L added PH 28/9/21 - for where loader gets loaded */MEMORY{FLASH_BOOT (rx) : ORIGIN = 0x08000000, LENGTH = 32KFLASH_APP (rx) : ORIGIN = 0x08008000, LENGTH = 1024K-32KRAM (xrw) : ORIGIN = 0x20000000, LENGTH = 128KRAM_L (xrw): ORIGIN = 0x20000000, LENGTH = 128KMEMORY_B1 (rx) : ORIGIN = 0x60000000, LENGTH = 0KCCMRAM (rw) : ORIGIN = 0x10000000, LENGTH = 64K}/* Define output sections */SECTIONS{/* loader_bss and loader sections must come first, in order to override thewildcards in subsequent sections *//* RAM based loader is linked to execute at top of RAM minus 8k = 0x2001E000.In the finished KDE this is within the heap area, so there is no possible runtime conflict *//* Note that if you create enough static objects (which end up in BSS) to push the end of BSS past0x2001E000 then you will get a linker error. This issue remains unsolved despite the use ofseparate sections (RAM & RAM_L) in MEMORY */KDE_loader 0x2001E000 : AT(_loader_loadaddr){_loader_start = .;*KDE_loader.o(.text*)*KDE_loader.o(.rodata*)*KDE_loader.o(.data*)*KDE_loader.o(.common*)*KDE_loader.o(.ARM.attributes). = ALIGN(4);_loader_end = .;} >RAM_L /*AT >_loader_loadaddr */_loader_size = SIZEOF(KDE_loader);/* Loader BSS ends up right after the loader code, in RAM. Currently this is zero size */loader_bss _loader_end :{_loader_bss_start = .;*KDE_loader.o(.bss*). = ALIGN(4);_loader_bss_end = .;} >RAM_L/* Place modules into FLASH, starting at the bottom (0x08000000)The line "*b_main.o (.text .text* .rodata .rodata*)" is important, but for some reasonis not needed for isr_vector *//* Startup code *//* boot block vector table */.isr_vector :{. = ALIGN(4);KEEP(*(.isr_vector)). = ALIGN(4);} >FLASH_BOOT.text.Reset_Handler :{. = ALIGN(4);KEEP(*(.text.Reset_Handler))*text.Reset_Handler (.text .text* .rodata .rodata*). = ALIGN(4);} >FLASH_BOOT.b_main.o :{. = ALIGN(4);KEEP(*(.b_main.o))*b_main.o (.text .text* .rodata .rodata*). = ALIGN(4);} >FLASH_BOOT.45dbxx.o :{. = ALIGN(4);KEEP(*(.45dbxx.o))*45dbxx.o (.text .text* .rodata .rodata*). = ALIGN(4);} >FLASH_BOOT/* === other boot block stuff will go here === *//* KDE_loader.o - goes last in the bottom block which makes it easier to check the block usageby checking _loader_end_in_flash in the .map file */.KDE_loader.o :{. = ALIGN(4);KEEP(*(.KDE_loader.o))*KDE_loader.o (.text .text* .rodata .rodata*). = ALIGN(4);_loader_end_in_flash = .;} >FLASH_BOOT/* The rest of the KDE code goes here, loaded at base+32k, starting with a stub and then the real main() */.main_stub.o :{. = ALIGN(4);KEEP(*(.main_stub.o))*main_stub.o (.text .text* .rodata .rodata*). = ALIGN(4);} >FLASH_APP/* customer code vector table */.isr_vector2 :{. = ALIGN(0x200);KEEP(*(.isr_vector2))*isr_vector2.o (.text .text* .rodata .rodata*). = ALIGN(4);} >FLASH_APP.main.o :{. = ALIGN(4);KEEP(*(.main.o))*main.o (.text .text* .rodata .rodata*). = ALIGN(4);} >FLASH_APP/* This collects all other stuff, which gets loaded into FLASH after main.o above */.text :{. = ALIGN(4);*(.text) /* .text sections (code) */*(.text*) /* .text* sections (code) */*(.rodata) /* .rodata sections (constants, strings, etc.) */*(.rodata*) /* .rodata* sections (constants, strings, etc.) */*(.glue_7) /* glue arm to thumb code */*(.glue_7t) /* glue thumb to arm code */*(.eh_frame)KEEP (*(.init))KEEP (*(.fini)). = ALIGN(4);_etext = .; /* define a global symbol at end of code */} >FLASH_APP/* this is for __libc_init_array() *//* Not sure it is actually used for anything */.preinit_array :{PROVIDE_HIDDEN (__preinit_array_start = .);KEEP (*(.preinit_array*))PROVIDE_HIDDEN (__preinit_array_end = .);} >FLASH_APP.init_array :{PROVIDE_HIDDEN (__init_array_start = .);KEEP (*(SORT(.init_array.*)))KEEP (*(.init_array*))PROVIDE_HIDDEN (__init_array_end = .);} >FLASH_APP.fini_array :{PROVIDE_HIDDEN (__fini_array_start = .);KEEP (*(.fini_array*))KEEP (*(SORT(.fini_array.*)))PROVIDE_HIDDEN (__fini_array_end = .);} >FLASH_APP/* Initialized data sections goes into RAM, load LMA copy after code */.data :{. = ALIGN(4);_sdata = .; /* create a global symbol at data start */*(.data) /* .data sections */*(.data*) /* .data* sections */. = ALIGN(4);_edata = .; /* define a global symbol at data end */} >RAM AT >FLASH_BOOT/* used by the startup to initialize data */_sidata = LOADADDR(.data);/* dummy placeholder in flash for loader section, to set _loader_loadaddr value */.KDE_loader :{. = . + SIZEOF(KDE_loader);} AT >FLASH_BOOT_loader_loadaddr = LOADADDR(.KDE_loader);/* Uninitialized data section */. = ALIGN(4);.bss :{/* This is used by the startup in order to initialize the .bss section */_sbss = .; /* define a global symbol at bss start */__bss_start__ = _sbss;*(.bss)*(.bss*)*(COMMON). = ALIGN(4);_ebss = .; /* define a global symbol at bss end */__bss_end__ = _ebss;} >RAM/* The heap ends up after BSS in main RAM *//* This also checks that the top of the heap doesn't hit the bottom of the stack i.e. how much RAM left *//* User_heap_stack section, used to check that there is enough RAM left *//* ._user_heap_stack : *//* This check is basically disabled because _Min_Heap_Size=0 above */.main_heap :{. = ALIGN(8);PROVIDE ( end = . );PROVIDE ( _end = . );. = . + _Min_Heap_Size;/* . = . + _Min_Stack_Size; */ /* PH 14/5/2021 stack is in CCM, not here */. = ALIGN(8);} >RAM/* } >CCMRAM *//* MEMORY_bank1 section, code must be located here explicitly *//* Example: extern int foo(void) __attribute__ ((section (".mb1text"))); *//* Not used 14/5/2021 - was apparently used for LCD display on ST dev kit */.memory_b1_text :{*(.mb1text) /* .mb1text sections (code) */*(.mb1text*) /* .mb1text* sections (code) */*(.mb1rodata) /* read-only data (constants) */*(.mb1rodata*)} >MEMORY_B1/* CCM-RAM section** IMPORTANT NOTE!* If variables placed in this section must be zero initialized,* the startup code needs to be modified to initialize this section.* Done PH 12/5/2021*/.ccmram :{. = ALIGN(4);_sccmram = .; /* create a global symbol at ccmram start */*(.ccmram)*(.ccmram*). = ALIGN(4);_eccmram = .; /* create a global symbol at ccmram end */} >CCMRAM/* Remove information from the standard libraries *//DISCARD/ :{libc.a ( * )libm.a ( * )libgcc.a ( * )}.ARM.attributes 0 : { *(.ARM.attributes) }}The above is testing with a 62000 byte static. 63000 breaks it. I have >60k of other DATA+BSS stuff already. It's not a big deal because this whole issue is worth only 8k.Perhaps a clue is in RAM and RAM_L showing the same numbers. I don't get that at all.

author: peter-h
author_rank: Super Contributor
timestamp: September 29, 2021, 07:41:54 am
content: I spent a whole day on this yesterday and can't suppress the section overlap errorOne could probably suppress the error from LD, with some command line options, but I don't know what that would do to the output.The OVERLAY mode must work, and is thus documented as one solution, but I think you have to use OVERLAY on all overlaid sections, not just the loader, so it gets messy.

author: abyrvalg
author_rank: Frequent Contributor
timestamp: September 29, 2021, 01:12:42 pm
content: Sure, I've tested my example and it works fine. I suspect your problem is in too many duplicated KDE_loader section declarations (what is the purpose?).Clean your .ld up a bit, arrange everything in the order you want it to be placed in flash:boot VT >FLASH_BOOTboot code >FLASH_BOOTboot data >RAM_L AT>FLASH_BOOTboot bss >RAM_Lloader code >RAM_L AT>FLASH_BOOTloader data >RAM_L AT>BOOTloader bss >RAM_Lapp VT >FLASH_APPapp code >FLASH_APPapp data >RAM AT>FLASH_APPapp bss >RAMDraw each mode's memory map on a paper - you'll see how it is structured both in space (BOOT_FLASH vs APP_FLASH) and in time (boot/loader execution vs app execution). I.e. you don't need boot's data/bss to exist during app execution etc.

author: peter-h
author_rank: Super Contributor
timestamp: September 29, 2021, 07:31:34 pm
content: The first one causes loader.o to be linked to execute at 0x20001e00Code:KDE_loader 0x2001E000 : AT(_loader_loadaddr){_loader_start = .;*KDE_loader.o(.text*)*KDE_loader.o(.rodata*)*KDE_loader.o(.data*)*KDE_loader.o(.common*)*KDE_loader.o(.ARM.attributes). = ALIGN(4);_loader_end = .;} >RAM_LMaybe the above should just do "text" because it is only executable code which needs to be located to run from 0x2001E000. The loader does not actually use any uninitialised statics (COMMON) nor zeroed data (BSS) nor other initialised data (DATA), currently.The next one places loader.o into the boot blockCode:.KDE_loader.o :{. = ALIGN(4);KEEP(*(.KDE_loader.o))*KDE_loader.o (.text .text* .rodata .rodata*). = ALIGN(4);_loader_end_in_flash = .;} >FLASH_BOOT_loader_end_in_flash is just a symbol in the .map file which can be checked to make sure it isn't outside the boot block (base+32k).And this one just sets up _loader_loadaddr, so the code below can copy it into RAMCode:.KDE_loader :{. = . + SIZEOF(KDE_loader);} AT >FLASH_BOOT_loader_loadaddr = LOADADDR(.KDE_loader);Code:extern char _loader_start;extern char _loader_end;extern char _loader_loadaddr;// Copy loader code to RAM. This also sets up its initialised data.B_memcpy(&_loader_start, &_loader_loadaddr, &_loader_end - &_loader_start);Maybe there is a way to combine all these.

author: abyrvalg
author_rank: Frequent Contributor
timestamp: September 29, 2021, 08:05:39 pm
content: What’s the point of specifying that fixed address for the loader? Your code uses the _loader_start symbol instead of a fixed addr, so just place it >RAM_L AT>BOOT_FLASH in a single placement directive and drop those duplicate declarations. You don’t need that code to run from a known address, it must be "just somewhere in RAM_L" and you have a named label representing that place.

author: peter-h
author_rank: Super Contributor
timestamp: September 29, 2021, 08:53:08 pm
content: Yes; I did realise that the loader can just sit somewhere in RAM, with a label pointing to the start of it, another label pointing to the end of it, and there is no need to fix its address. I just didn't know how to specify all this, and still don't. I have tried various ways but none work.What does>RAM_L AT>BOOT_FLASHmean? Does it mean the code is located to execute from RAM_L and then placed in BOOT_FLASH? If so, why even have two "RAM" sections? The entire loader is under 2k bytes so it could be a permanent static object in the RAM.I found this seems to work, as the only bit of the linkfile referencing the loaderCode:. = ALIGN(4);_loader_loadaddr = .;.KDE_loader.o :{. = ALIGN(4);_loader_start = .;KEEP(*(.KDE_loader.o))*KDE_loader.o (.text .text* .rodata .rodata*). = ALIGN(4);_loader_end = .;} >RAM_L AT>FLASH_BOOT_loader_end_in_flash = .;but then a) one does not need two RAM sections, and b) the loader codein RAMalways takes up that space (which is not much - under 2k) like it was e.g. a piece of BSS.

author: abyrvalg
author_rank: Frequent Contributor
timestamp: September 29, 2021, 10:39:34 pm
content: Separate RAM and RAM_L allow the overlapping.What ld does: each area declared in MEMORY{} has it’s own "current location" pointer (accessed via dot symbol) initialized to the area start address. When you do a > placement the obect is placed starting from that "current" address and the current location pointer gets advanced by the object’s size.So, placing to RAM area will use RAM’s current location and will not affect RAM_L’s current location and vice versa.And yes, placing to >RAM_L AT>BOOT_FLASH will allocate the runtime address in RAM_L but storage address in BOOT_FLASH. Check any .elf file - each segment has two address/size pairs - "virtual" and "physical". A flasher writes segments of physisal sizes to physical addresses (so everything with non-zero physical sizes, i.e. code or initialized data goes to flash and everything with zero physical sizes, i.e. bss sections goes to nowhere (or "to flash, but occupying 0 bytes")). Then at runtime segments with VA=PA just stay where they are and segments with VA<>PA must be copied (or zero-initialized) to their VAs (either by a startup code or by your code) before being used. On a more advanved CPU such sections could be just remapped via MMU.A more simple >SOMEWHERE placement (w/o AT>) places the object in SOMEWHERE segment with VA=PA.

author: peter-h
author_rank: Super Contributor
timestamp: September 30, 2021, 07:35:44 am
content: OK this seems to be working. From the .map file:This is the loader in RAM_L:and this is the same address in RAM containing other statics:I have put the loader BSS after it, although currently there isn't any static data used; this section is zeroed withCode:B_memset(&_loader_bss_start, 0, &_loader_bss_end - &_loader_bss_start);Code:/* KDE_loader.o - goes ===LAST=== in the bottom block, because the location pointeris now pointing into RAM_L. You can check the boot block code usage by checking_loader_end_in_flash in the .map file */. = ALIGN(4);_loader_loadaddr = .;/* this is where the loader start is in FLASH */.KDE_loader.o :{. = ALIGN(4);_loader_start = .;KEEP(*(.KDE_loader.o))*KDE_loader.o (.text .text* .rodata .rodata*). = ALIGN(4);_loader_end = .;} >RAM_L AT>FLASH_BOOT/* These variables are not necessarily used but can be inspected in the .map file */_loader_size = _loader_end - _loader_start;_loader_end_in_flash = _loader_loadaddr + _loader_size;/* Place loader BSS right after the loader code, in RAM. */loader_bss _loader_end :{. = ALIGN(4);_loader_bss_start = .;*KDE_loader.o(.bss*). = ALIGN(4);_loader_bss_end = .;} >RAM_Land it does seem to actually run, too.Thanks very muchEDIT: I had to modify the 1st lineCode:.KDE_loader.o 0x2001e000 :to set the RAM address of the loader high up in RAM after all. Otherwise, any global statics set up by previous code would be trashed when the loader gets copied into RAM. There were a few which were needed by the loader.

author: abyrvalg
author_rank: Frequent Contributor
timestamp: September 30, 2021, 08:39:53 am
content: Great! BTW, I don’t see loader’s .data placement. Your current loader code has no initialized data? Consider appending .data .data* into *KDE_loader.o (.text .text* .rodata .rodata*). Otherwise if you declare any initialized data in the loader later it will be placed into RAM segment instead of RAM_L (by matching the generic *(*.data) rule placing app data) possibly at some overlapping addresses. Note that you don’t need to copy that data separately, just treat it as a part of loader image and your existing memcpy will take care of it.Also, while we are on this topic, consider separating all boot-related .data from the app’s .data, placing them separately (boot data >RAM_L AT>BOOT_FLASH, app data >RAM AT>APP_FLASH) and copying the app data separately (at app start). This will result in two things: 1. App will reuse all RAM used by boot, 2. If the app data will ever grow big it will not overflow the small BOOT_FLASH seg (currently app data’s storage address belongs there, together with boot data).

author: peter-h
author_rank: Super Contributor
timestamp: September 30, 2021, 09:32:59 am
content: That is correct; the loader uses no BSS DATA or COMMON, but I added your suggestion anywayCode:/* KDE_loader.o - goes ===LAST=== in the boot block, because the location pointeris now pointing into RAM_L. You can check the boot block code usage by checking_loader_end_in_flash in the .map file.The address specified below is not actually needed but if you don't specify it, theloader gets loaded into RAM at 0x20000000 which is fine except that it craps oversome globals set up by previous code e.g. systemcoreclock */. = ALIGN(4);_loader_loadaddr = .;/* this is where the loader start is in FLASH */.KDE_loader.o 0x20018000 :/* this is where the loader start is in RAM */{. = ALIGN(4);_loader_start = .;KEEP(*(.KDE_loader.o))*KDE_loader.o (.text .text* .rodata .rodata* .data .data*). = ALIGN(4);_loader_end = .;} >RAM_L AT>FLASH_BOOT/* These variables are not necessarily used but can be inspected in the .map file */_loader_size = _loader_end - _loader_start;_loader_end_in_flash = _loader_loadaddr + _loader_size;/* Place loader BSS right after the loader code, in RAM. */loader_bss _loader_end :{. = ALIGN(4);_loader_bss_start = .;*KDE_loader.o(.bss*). = ALIGN(4);_loader_bss_end = .;} >RAM_LI established, with careful testing, that with this compiler you get:With ARM GCC, statics are categorised thus:int fred=0; goes into BSS (which by definition is zeroed by startup code)int fred; goes into COMMONint fred=1; goes into DATA (statics initialised to nonzero)HOWEVER I have just tested for the overlap andstillget the overlap error... so back to square one after all thatc:\st\stm32cubeide_1.4.0\stm32cubeide\plugins\com.st.stm32cube.ide.mcu.externaltools.gnu-tools-for-stm32.9-2020-q2-update.win32_2.0.0.202105311346\tools\arm-none-eabi\bin\ld.exe: section .KDE_loader.o VMA [0000000020018000,00000000200186ff] overlaps section .bss VMA [0000000020000eb0,000000002001dc3f]It seems that if I don't specify the 0x20018000, there is no overlap detection, which is interesting. The loader then runs from 0x20000000 but has crapped over an statics configured by previous code. I can work around that. I suspect this is how the avoidance of overlap errors was actually achieved by those who did it previously.So next I tried removing the address and defining it in MEMORY as RAM_L of the appropriate size:Code:MEMORY{FLASH_BOOT (rx) : ORIGIN = 0x08000000, LENGTH = 32KFLASH_APP (rx) : ORIGIN = 0x08008000, LENGTH = 1024K-32KRAM (xrw) : ORIGIN = 0x20000000, LENGTH = 128KRAM_L (xrw) : ORIGIN = 0x20018000, LENGTH = 32KMEMORY_B1 (rx) : ORIGIN = 0x60000000, LENGTH = 0KCCMRAM (rw) : ORIGIN = 0x10000000, LENGTH = 64K}Nope, doesn't work. Same overlap error. This is what I am using to do the overlap test:Code:uint8_t fred1234;Fred1234 is correctly in COMMONeven though the LD error talks about BSS. Examining the .map file shows that they have BSS+COMMON as one single block so this makes sense.This issue is clearly more subtle, because having the loader at base of RAM, and having it crap all over bss, common and data also there, doesn't generate an error, while defining a big fred1234[] in main.c as above does get detected.For amusement, I also found thatCode:*KDE_loader.o (.text .text* .rodata .rodata* .data .data*)placed DATAbeforeTEXT in memory. That doesn't actually matter (because control is transferred to the loader by calling a function, not a fixed address) but it is daft.

author: peter-h
author_rank: Super Contributor
timestamp: September 30, 2021, 08:02:17 pm
content: Spending yet more time on this, I find all kinds of oddities. For example(*COMMON)does absolutely nothing to place a COMMON static somewhere. It has to be(*.common)The dot and the lowercase are both needed. Where did the 1st one come from? STM examples of course!No luck with enabling the section overlap though. I reckon it must be section-specific, plus some other magic factor. There is some old stuff online suggesting that older versions of LD allowed section overlap if one section was TEXT and the other was BCC, but that is not the case today.

author: SiliconWizard
author_rank: Super Contributor
timestamp: September 30, 2021, 08:08:21 pm
content: Well, I remember there was already a rather long thread about it all, but at this point I'm still not sure why you didn't just create a separate build with a distinct linker script...

author: peter-h
author_rank: Super Contributor
timestamp: September 30, 2021, 09:03:43 pm
content: Yes there was a thread; this one is specific on how to enable sections to overlap in GCC (LD specifically).The reason for not doing a separate build for the loader was to make the whole project easier to support in years to come; for example software fixes are likely to apply equally to both the boot block and the "other" code. Ideally the boot loader should be 100% bug-free and possible to freeze, but in reality these days things are way more complex. For example the loader reads data from a serial FLASH via SPI and there were various issues with getting all that hardware to be fully reliable. With a 168MHz CPU a lot of things (a lot of manufacturer code examples!) break, which worked fine at 16MHz.I spent a lot of time on this but it is probably time to draw a line under it. The RAM-resident part of the boot block is only 1.7k. I am allowing 4k and it is sitting at the top of the 128k RAM, so I am losing just 4k. The final product has the heap there anyway...The more time I spend playing with this GCC/ARM stuff the more I think there is a load of crap there. For example the compiler emits a vast array of error messages and warnings, while the linker is designed by real men for real men who don't need useful error messages, so "syntax error" is good enough. The syntax is absolutely appalling, with spaces needed around a = sign, etc. This is on the level of the worst CP/M stuff from c. 1976. All great stuff if you are paid by the hour

author: abyrvalg
author_rank: Frequent Contributor
timestamp: September 30, 2021, 11:35:03 pm
content: Ok, here is the summary and the solution:Indeed, the linker doesn't like overlapping MEMORY areas officially. Looks like I've abused some undocumented behavior in my exmaple: with two MEMORY areas starting at the same address (sizes could differ) ld happily accepts an overlapped placement. But with overlapping areas starting at different addresses it produces an error.A--no-check-sectionsparameter disables that overlap check completely - this is the wayto shoot yourself in the foot

author: peter-h
author_rank: Super Contributor
timestamp: October 01, 2021, 07:35:06 am
content: Aha that would explain itHowever the overlap detection gets re-enabled if you then add a value parameter to one of the sections e.g.So this works only if the loader is to be loaded into thebaseof the RAM (address 0x20000000).I am not bothered about disabling overlap checks, but whatisuseful is having a check on say the BSS running off the top of the RAM.

author: abyrvalg
author_rank: Frequent Contributor
timestamp: October 01, 2021, 08:29:10 am
content: > So this works only if the loader is to be loaded into the base of the RAM (address 0x20000000).With —no-check-sections you can declare an area starting at the desired address and place into it (and ld will not complain). But what is the purpose of specifying that address? You’ve mentioned some overwritten data there and now trying to move the loader away from it to an address looking “far enough”? - that’s another way to shoot in the foot. Who’s data is that? If it is boot’s or loader’s - place it together with the loader and let the linker choose the address. If it is app’s - you are doing something wrong, they don’t exist at the same run time. If it is something common (like boot state passed to the app) - declare it in a separate section and place into a separate non-overlapping MEMORY {} area.Sorry, all this starts looking a bit like the famous “We tried baseball and it didn’t work” - you are using a tool that automates the placement for you but sabotaging it by manual overrides of Z80 era style. After so many troubles are you still sure about maintainability of this solution? Why not just use a “single project with two build configs” approach suggested by harerod? That would fulfill your “single project” requirement but without fighting with a linker - the same code would be used in two independent link steps eliminating all those time-space conflicts.

author: peter-h
author_rank: Super Contributor
timestamp: October 01, 2021, 10:11:45 am
content: Yes; it is a fair point that it doesn't matter where the loader goes in RAM provided the region is definitely not used by something else.This could be achieved in 2 ways:1) Place it in RAM, high up, where DATA BSS COMMON will definitely never reach, and using the top 4k is a fair place for that since the heap goes there, but the heap will never be there when the loader is running (and it always reboots when finished). This is what I am using. It means that unless section overlap detection is disabled, you cannot have DATA BSS COMMON in the uppermost 4k of the 128k because you get the abort from LD.2) Place it in RAM, after DATA BSS COMMON. This avoids the need to document the placement address of 0x2001f000 but it still means you lose that space which could potentially be used for DATA BSS COMMON or heap. The drawback of this method is that youalwayslose the space (because the loader is like a piece of BSS) whereas with 1) you lose it only if there is no heap in use. OTOH with this method you lose only the actual loader size which in this case is 1.7k. I was going to give this another try (it worked ok previously, and it simplifies everything because the loader DATA BSS COMMON don't need to be separate, and the BSS+COMMON area will get zeroed at startup without need for extra code) but the gotcha is that if the loader needed more functionality in the future then the memory loss might be an issue. Method 1 makes it possible to have a much bigger loader, say 32k, which may be relevant if one had to do more security on code being installed, etc.I tried setting up multiple projects in Cube and it is really messy. It's OK if you spend your life with this tool.I went to look at where in Cube one can specify linker command line options and found thiswhich would be quite useful for a loader, but apart from some very early threads on this topic where it was mentioned, nobody seems to use it.

author: abyrvalg
author_rank: Frequent Contributor
timestamp: October 01, 2021, 11:50:35 am
content: But why are you solving the loader vs data/bss/common conflict at all if the loader never runs together with app (not calling any funcs from there) and reboots when finishes? What’s the point of all that overlaying then? You have some data coming from code before the loader which you want to use in loader?-fPIC - while it looks cool it has it’s own quirks. And it isreallyneeded only when the runtime address is truly dynamic (i.e. running a code from malloc’ed buf), not your case I guess.> I tried setting up multiple projects in CubeI mean multiplebuild configsinside thesame single project, not multiple projects. With this approach the same source file set (with exclusions/additions specified by you) can be built with different settings (i.e. two separate .ld files - one for boot, another one for app, separate #defines - so your code “knows” the build flavor and can #ifdef some parts).

author: peter-h
author_rank: Super Contributor
timestamp: October 01, 2021, 02:13:54 pm
content: The loader does call other stuff in the 32k boot block. It has to otherwise it would be a lot bigger. For example the main job of the loader is to program the CPU FLASH, from image(s) stored in a serial (SPI) FLASH. So it needs a load of code to access the serial FLASH. And it may need to check CRC, etc. Just programming the CPU FLASH from RAM is a tiny bit of code - a few hundred bytes max. It also has to preserve some statics previously set up, and not trash others e.g those related to the serial FLASH code. It isafterthe loader has finished that everything can be chucked away.Another angle is that, reportedly, you don't need RAM based code at all, because if you are running code out of FLASH you get wait states during the programming cycle. The CPU doesn't actually crash. I don't know if anybody has ever done this. What you cannot do is program the very address from which the code is being fetched (plus and minus the cache prefetch size, presumably), so to program the entire 1MB you would needtwoloaders, some distance apart. Also on the 2MB FLASH version of the 32F4 you can program one 1MB block while running from the other 1MB block. I haven't tried this method because it sounds too good to be true, and one could get burnt by the cache refill.Re multiple build configs, do you mean stuff like Debug versus Release? I have done that. I don't use it currently because -Og is perfectly good enough but generating a Release with say -O1 means a huge amount of re-testing everything.

author: SiliconWizard
author_rank: Super Contributor
timestamp: October 01, 2021, 06:21:43 pm
content: Quote from: peter-h on September 30, 2021, 09:03:43 pmI guess you probably don't want to go all over again at this - you apparently have made up your mind - but what you said above is just very common.There is absolutely no problem managing multiple projects based on shared source files, so that any modification in one place will be reflected when rebuilding the projects.Now if your main concern is not with the source files, but just the linker scripts themselves - which would have some common definitions, and some different - you can absolutely include files in linker scripts, so the common parts could just be maintained in one file, included in both linker scripts. See: "Commands dealing with files" :https://home.cs.colorado.edu/~main/cs1300/doc/gnu/ld_3.htmlLastly, if your main issue for all this is that you're using STM32Cube (I'm guessing, although I don't quite remember), then frankly think about using something else. What I suggest above is easy to do, to maintain and to build using Make, or any build tool you like better, such as CMake, and there are many others. Just my 2 cents. You can just ignore it if you really don't wanna hear this, but I'm just thinking you're making your life much harder here just because you don't wanna take the plunge.

author: peter-h
author_rank: Super Contributor
timestamp: October 01, 2021, 07:04:14 pm
content: Many thanks for that linker script reference, SiliconWizard. It's very clear and I will read through it. It didn't occur to me that one can include files in a linkfile.I am working with ST Cube for specific reasons and right now I am stuck with it. But the project is all working. I've today tested all the different static types DATA BSS COMMON to make sure that for both the base code and for the loader they end up in the right places; some surprises there which I can't explain e.g. ".common" works in some places while "COMMON" is needed elsewhere. So much time wasted on so many obscure things.

author: abyrvalg
author_rank: Frequent Contributor
timestamp: October 03, 2021, 10:04:20 pm
content: So your boot code calls the loader part for each data chunk? Then you need to place the boot data in the same RAM "view" as loader (and app’s data in the other overlaid view). But you have plenty of RAM to fit the SPI flash reading and MCU flash writing and run the entire update process from RAM w/o calling flash code. Your FLASH_BOOT area containing both parts of the updater is just 32K.> multiple build configs, do you mean stuff like Debug versus Release?Yes, those can be used to do much more than just changing the -O flag. You can i.e. create Boot and App configs, set different .ld files for each one in linker settings, exclude files not needed for this particular build flavor in each one and build two truly independent binaries from a single project (so you don’t need to hunt for boot dependencies drifted to the app part).

author: peter-h
author_rank: Super Contributor
timestamp: October 04, 2021, 07:43:21 pm
content: "So your boot code calls the loader part for each data chunk? "No; I jump to it and it never returns (it reboots).It is working fine. I never found out how to overlay the loader with BSS etc but the loader is now in the top 4k of RAM which is where the main stack (8k) is when the product is running normally, so no RAM is lost and the issue has gone away.Obviously I had to set SP elsewhere (to the top of CCM in this case) upon entry to the loader. Had a bit of fun with that, as you can see from the commentsCode:// This is the entry point for when this module ends up in RAM.// Should be the first thing here.void loader_entry(void){// Set SP to top of CCM. This is not where the KDE general stack is normally but it doesn't matter because// the loader always reboots at the end.// This asm code cannot be used if the function contains any stack based variables because they will have been// allocated already and now we lose them!asm volatile ("ldr sp, = 0x10010000 \n");real_loader();// never get herefor (;;);}In the running product, the CCM is used for the FreeRTOS workspace - all 64k of it.

author: Doctorandus_P
author_rank: Super Contributor
timestamp: October 05, 2021, 01:01:17 pm
content: The book "Beginning STM32" from Warren Gay has a chapter about using overlays for STM32 and GCC.Beginning STM32: Developing with FreeRTOS, libopencm3 and GCCWarren GayISBN-13 (pbk): 978-1-4842-3623-9ISBN-13 (electronic): 978-1-4842-3624-6https://doi.org/10.1007/978-1-4842-3624-6Library of Congress Control Number: 2018945101

author: peter-h
author_rank: Super Contributor
timestamp: October 08, 2021, 09:24:59 pm
content: Thanks for the tipI bought the book and have been working through it. The overlay stuff is on page 156 onwards.However it is pretty opaque - as indeed I suspected it would be. The syntax is weird and I am not going to spend time trying to unravel it since I have now bypassed the problem. But as I suspected the solution lies in the use of the keyword OVERLAY, and you need to thus declarealloverlapping regions; there isn't a single keyword which tells the linker to just ignore the overlap of one particular section with some other one(s).I think this topic is not well known because few people are using overlays today. It was a big thing in the old days, with e.g. the Z180 offering 64k addressing with a simple MMU giving you 1MB physical addressing, and compilers (we used IAR back then) having a "large model" mode where you had typically 60k of code+data and the last 4k was mapped to some address in a 1MB ROM, via an 8 bit overlay value. You had to make sure each C function fitted into this 4k (usually doable) but a 4k overlay could contain any number of small functions. Each function call consisted of loading the 8 bit overlay value into the register and then calling the function by its 16 bit address, so there was a bit of overhead but otherwise it was pretty transparent and a great productivity tool. Maybe some ARM users are doing overlays but it's not obvious when, other than if e.g. wanting to address a few GB.