ELF to binary for boot loader - Page 1
Forum section: electronics / general

author: peter-h
author_rank: Super Contributor
timestamp: July 29, 2021, 08:46:00 pm
content: There is a lot of stuff online for ELF to BIN etc. For example arm-none-eabi-objcopy can generate the binary.But not much on how this actually works.Let's say you are programming a 32F4 within Cube IDE, using STLINK V2/V3. Does the IDE just feed a binary block to the debugger, to program the FLASH with? The debugger has very little storage it in. It must be just programming the stuff straight in, 0x08000000 onwards.But there must be a way to specify gaps otherwise if you generated some text for bottom 100k and then top 40k, you would end up with a 1MB binary block, mostly empty. I don't think this is happening. So whatever is managing this process must be generating multiple binary blocks and programming these individually.Also, AIUI, for single stepping, you need to have a symbol table "somewhere" and it can't be in the CPU FLASH because a 1MB binary could be several MB with symbols.So the stuff loaded into the CPU must be just the binary block comprising of the sections 'text' and 'data'.IOW, same as what you would need to generate for a boot loader which just takes a binary block and programs the CPU FLASH with it.I am trying to work out how to generate a single binary block. The program will never have discontinuous text or data sections, and AFAIK data always follows text.

author: langwadt
author_rank: Super Contributor
timestamp: July 29, 2021, 09:00:40 pm
content: Quote from: peter-h on July 29, 2021, 08:46:00 pma bin file doesn't have gaps and you need to specify where to put it in memoryan intelhex file can have gaps and specify the address for each line

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 29, 2021, 09:16:49 pm
content: And... you can have a look here:https://stackoverflow.com/questions/19458031/how-to-create-a-executable-hex-from-elf-file-format

author: peter-h
author_rank: Super Contributor
timestamp: July 29, 2021, 09:27:38 pm
content: I saw that post and thought he was trying to generate a an .exe (a DOS, or windoze command line executable). These have a complicated format, with a load-time fixed-up table of jumps etc.I am also very familiar with Intel hex. But I don't want to use that, due to the bulk. For a 1MB CPU you would need ~2MB intel hex.So I guess the objective here is just a string of bytes representing the CPU FLASH, 0x08000000 onwards, as long as there is text+data. This must incidentally be what gets fed to the debugger too.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 29, 2021, 09:31:41 pm
content: Quote from: peter-h on July 29, 2021, 09:27:38 pmhex is a very simple format to deal with, much simpler than elf. So point is, you can use objcopy to generate a hex file from a elf file, and then do whatever you find convenient from the hex file. If a hex file is too big, you could either compress it, or convert it to some custom format.

author: ataradov
author_rank: Super Contributor
timestamp: July 29, 2021, 09:33:57 pm
content: I think you are confused. IDE issues very simple programming commands for the flash controller to the debugger/programmer, it does not store anything.Normally ELF files will be contiguous, unless you go out of your way to make them otherwise. And if you do, you need to come up with your own binary format that includes separate sections.

author: ajb
author_rank: Super Contributor
timestamp: July 29, 2021, 09:53:32 pm
content: You can certainly use objcopy to create a binary of the application and hand that off to a bootloader for installation. You'll want to provide at least a length for it and maybe some other metadata so it's probably a good idea to define some sort of header the bootloader can inspect, and a checksum or cryptographic signature would be a really good idea. Easy enough to create a script to build all of that into an update package. There are even tools to script the parsing of source files, so you can do things like extract version information or other metadata from the application files and put that into the update package for easy automated deployment. Of course if you don't have enough memory available to hold an entire binary image+header in addition to the installed software--whether in some sort of on-chip/soldered-on storage or a removable memory card or USB drive--then you'll need to drip-feed the binary to the bootloader from some other device. What that other device is would be up to you/your application.

author: westfw
author_rank: Super Contributor
timestamp: July 30, 2021, 01:40:49 am
content: QuoteAnd you probably should, because the elf format itself is relatively complex and probably subject to changing.There is also readelf (a program) and libelf (a library for parsing elf files.)

author: ataradov
author_rank: Super Contributor
timestamp: July 30, 2021, 01:56:34 am
content: Parsing ELF just to extract the programming data is trivial. I actually prefer doing that over parsing HEX files, since there is no need to deal with a ton of formatting options and parsing text files.But for the bootloaders, it makes no sense to directly accept either of them, you really want some dedicated format, possibly with check sums and stuff like that.

author: brucehoult
author_rank: Super Contributor
timestamp: July 30, 2021, 03:32:16 am
content: Quote from: peter-h on July 29, 2021, 09:27:38 pmWhat is the problem with that?

author: brucehoult
author_rank: Super Contributor
timestamp: July 30, 2021, 03:37:12 am
content: Quote from: ataradov on July 30, 2021, 01:56:34 amParsing Intel hex is easy. I do it in 50 lines for a simple RV32I emulator here:https://github.com/brucehoult/trv/blob/main/trv.cI'm sure I couldn't load an elf file with as little code -- certainly using my self-imposed no headers, no libraries constraint.I'd be happy to be proven wrong.

author: ataradov
author_rank: Super Contributor
timestamp: July 30, 2021, 05:06:30 am
content: Here is 50 lines for parsing ELF files. And most of this is just structure definitions. The actual parsing is 20 lines.Code:typedef struct{uint8_t e_ident;uint8_t e_type;uint16_t e_machine;uint16_t e_version;uint32_t e_entry;uint32_t e_phoff;uint32_t e_shoff;uint32_t e_flags;uint16_t e_ehsize;uint16_t e_phentsize;uint16_t e_phnum;uint16_t e_shentsize;uint16_t e_shnum;uint16_t e_shstrndx;} elf_ehdr_t;typedef struct{uint32_t sh_name;uint32_t sh_type;uint32_t sh_flags;uint32_t sh_addr;uint32_t sh_offset;uint32_t sh_size;uint32_t sh_link;uint32_t sh_info;uint32_t sh_addralign;uint32_t sh_entsize;} elf_shdr_t;bool read_elf(uint8_t *data){elf_ehdr_t *hdr = (elf_ehdr_t *)data;elf_shdr_t *shdr = (elf_shdr_t *)&data[hdr->e_shoff];if (hdr->e_type != 2/*ET_EXEC*/ || hdr->e_version != 1/*EV_CURRENT*/ || hdr->e_machine != 40/*EM_ARM*/)return false;for (int i = 0; i < hdr->e_shnum; i++){if (shdr.sh_type == 1/*SHT_PROGBITS*/ && (shdr.sh_flags & 2/*SHF_ALLOC*/)){printf("program %d bytes from file offset %d into memory address %d (0x%08x)\n",shdr.sh_size, shdr.sh_offset, shdr.sh_addr, shdr.sh_addr);}}return true;}Obviously replace printf() with the actual data extraction and programming functions.This does not verify input data and valid ELF file is assumed.And of course replace 40 (EM_ARM) in e_machine check with an actual target machine type. Or just remove the check and assume that ELF file is valid for the used processor.

author: peter-h
author_rank: Super Contributor
timestamp: July 30, 2021, 06:53:25 am
content: Quote from: ajb on July 29, 2021, 09:53:32 pmThat's exactly what I need to do - many thanks. Just a straight binary image of the CPU FLASH which can be programmed into it directly.Sounds like objcopy -O binary.dat will do this.Then I will need to get a little executable written which packages that binary block with a length value, a CRC and maybe some crypto hash. Has anyone done this within Cube? AIUI it needs to be written in Java, unless you are happy for a Windows-only setup.

author: gf
author_rank: Super Contributor
timestamp: July 30, 2021, 07:22:20 am
content: Quote from: peter-h on July 29, 2021, 08:46:00 pmExactly thisishappening. My understanding is that objcopy copies allloadablesections in the .elf file to the .bin file. Gaps betwen sections are filled. Non-loadable sections (like debug, etc.) are not copied.Quote from: man page

author: brucehoult
author_rank: Super Contributor
timestamp: July 30, 2021, 07:51:49 am
content: Quote from: ataradov on July 30, 2021, 05:06:30 amThanks, I'll give that a try.I assume there's supposed to be padding after e_type and e_version? Otherwise all the 16 bit and 32 bit fields are unaligned. It woul dbe better if padding had explicit fields.This can be handy for elf-in-ROM on small machines, or memory-mapped on big machines. Seems like potential for trouble if it wants to unpack over the top of the place you read it into...

author: peter-h
author_rank: Super Contributor
timestamp: July 30, 2021, 07:56:15 am
content: "Exactly this is happening. My understanding is that objcopy copies all loadable sections in the .elf file to the .bin file. Gaps betwen sections are filled. Non-loadable sections (like debug, etc.) are not copied."Fortunately this isn't a problem in my project.I do have a bit of code which is linked toexecutein RAM, halfway up ie 0x20010000, but it should not be loading that at that address.It was discussed here, and is working finehttps://www.eevblog.com/forum/microcontrollers/how-to-create-elf-file-which-contains-the-normal-prog-plus-a-relocatable-block/new/#new"IDE issues very simple programming commands for the flash controller to the debugger/programmer, it does not store anything."OK; that makes sense. That's why the more expensive debuggers claim to run faster. The debugger is probably programming every byte as a separate transaction. Although one should still be limited by the FLASH programming speed. STLINK V3 does appear to program a bit faster than V2. I think the only real advantage of a fast debugger is with stuff like breakpoints with an ignore count; these trigger a breakpoint on every hit and a fast debugger will be able to process this faster so slowing down the target less.Is there any way to incorporate objcopy -O binary.dat into the Cube script, so that along the .elf (which is used by the debugger) it pops out the binary?

author: ataradov
author_rank: Super Contributor
timestamp: July 30, 2021, 08:19:44 am
content: Quote from: brucehoult on July 30, 2021, 07:51:49 amNewer versions of the standard define e_type as uint16_t and e_version as uint32_t. Makes sense, I guess. This code is pretty old, I don't even remember where I've got this layout from.Quote from: brucehoult on July 30, 2021, 07:51:49 amEven stripped ELF files contain a lot more stuff, so if you are optimizing for size, it may not be the best idea to use ELF for anything.Another issue is that for RAM code you don't really get a good info. ELF is not designed to be converted into binary, it was designed to be loaded as is. So even with this code you get output like this if you have RAM functions:Quote0x00400000 in this case is the flash offset and 0x20400000 is SRAM. As you can see, the last allocated section is in the RAM, but the actual location of the data is in the flash.The way objcopy handles this is it takes all the parts within the section (as defined by SECTIONS part of the linker script) and concatenates them together without filling the gaps. So in this case to simulate objcopy you would ignore the address and copy the data sequentially.The only time objcopy fills the gaps is when there are multiple sections that contain loadable data. Those are the sections defined by the MEMORY part of the linker script. To extract that info add this:Code:typedef struct{uint32_t p_type;uint32_t p_offset;uint32_t p_vaddr;uint32_t p_paddr;uint32_t p_filesz;uint32_t p_memsz;uint32_t p_flags;uint32_t p_align;} elf_phdr_t;..........elf_phdr_t *phdr = (elf_phdr_t *)&data[hdr->e_phoff];for (int i = 0; i < hdr->e_phnum; i++){printf("--- %d\n", i);printf("p_type : 0x%08x\n", phdr.p_type);printf("p_offset : 0x%08x\n", phdr.p_offset);printf("p_vaddr : 0x%08x\n", phdr.p_vaddr);printf("p_paddr : 0x%08x\n", phdr.p_paddr);printf("p_filesz : 0x%08x\n", phdr.p_filesz);printf("p_memsz : 0x%08x\n", phdr.p_memsz);printf("p_flags : 0x%08x\n", phdr.p_flags);printf("p_align : 0x%08x\n", phdr.p_align);}Although it looks like it may be relying on the physical address from the program headers as well to convert things:QuoteHere 0x00401aec is the address of the RAM functions (part of the .data segment) in the flash. If you just go by "Section Headers", then you will have a huge gap. But "Program Headers" alone do not have enough information.

author: nctnico
author_rank: Super Contributor
timestamp: July 30, 2021, 08:21:51 am
content: Quote from: gf on July 30, 2021, 07:22:20 amAFAIK you need to tell objcopy to fill the gaps. There is an option for that as well which also specifies the byte value to use.

author: ataradov
author_rank: Super Contributor
timestamp: July 30, 2021, 08:23:15 am
content: Quote from: peter-h on July 30, 2021, 07:56:15 amThey claim it to justify the price. Past certain point (at least using USB HS for transfers) there is no real difference. Most of the time is spent waiting for the flash to program anyway.Quote from: peter-h on July 30, 2021, 07:56:15 amNo, because there are limitations on how you can program the flash. A debugger does the same exact things as you would from the code. So you can read the device datasheet and know exactly what debugger is doing. And it is not really a debugger doing those things, but the IDE. A debugger is just a pipe.Some debuggers have more complicated logic they can run on the device itself. This is helpful for devices with poorly designed flash controllers (like ST) where fast programming is a time-sensitive operation and if the data is not delivered in time, the whole operation fails.Quote from: peter-h on July 30, 2021, 07:56:15 amNo idea. Usually it is done through post-build events.

author: ajb
author_rank: Super Contributor
timestamp: July 30, 2021, 01:38:48 pm
content: Quote from: peter-h on July 30, 2021, 07:56:15 amCube IDE is based on eclipse, so unless they've really munged it up there should be a 'post build' section in the build configuration where you can tell it to invoke whatever external process you want. Off the top of my head I don't remember where exactly. You can throw the objcopy command in there, and even invoke your packager from there was well. Post build steps are per build configuration, so you can have this all set up on a 'release' build config and leave your debug config as is since you probably don't need it there.I'm not a huge fan of python in general but it's well suited to these sorts of things, being cross platform and having a ton of libraries. You could probably do a basic packager in <20 lines including hashing.

author: gf
author_rank: Super Contributor
timestamp: July 31, 2021, 07:51:23 am
content: Quote from: nctnico on July 30, 2021, 08:21:51 am$ objcopy -O binary a.out a.bin$ ls -l a.bin-rwxr-xr-x 1 fuer users8288Jul 31 09:35 a.bin$ objdump -h a.outa.out: Dateiformat elf64-x86-64Sektionen:Idx Name Size VMA LMA File off Algn0 text1 00000007 0000000000003000000000000000300000003000 2**0CONTENTS, ALLOC,LOAD, READONLY, CODE1 text2 00000007 0000000000005000 0000000000005000 00005000 2**0CONTENTS, ALLOC,LOAD, READONLY, CODE2 .eh_frame000000580000000000005008000000000000500800005008 2**3CONTENTS, ALLOC,LOAD, READONLY, DATA3 .comment 00000029 0000000000000000 0000000000000000 00005060 2**0CONTENTS, READONLYThe size of a.bin is 8288 = LMA(.eh_frame) + SIZEOF(.eh_frame) - LMA(text1).Obviously it does keep and fill the gap between text1 and text2, and between text2 and .eh_frame, preserving the offsets beween the copied sctions' load addresses also in the .bin file.

author: peter-h
author_rank: Super Contributor
timestamp: July 31, 2021, 08:23:40 am
content: Thank you everyone. This is great. Yes the post build steps can be found under Project / PropertiesI am going to put a job up on freelancer.com to get a win32 command line utility done which appends four crc32 bytes, little-endian, to a file supplied on a command lineI see some other IDE (IAR, I think) offers this as a built-in function. Or maybe it's been done before... well it must have been.

author: brucehoult
author_rank: Super Contributor
timestamp: July 31, 2021, 09:22:06 am
content: Quote from: peter-h on July 31, 2021, 08:23:40 amOn Mac or Linux the following ...Code:perl -e 'print pack("V",int($ARGV)),"\n"' $(cksum foo) >>foo... will literally append crc32 to foo, little endian. As in modify the given file. If you want big endian change "V" (Vax) to "N" (Network).On Windows you can (and should) install Linux in WSL.

author: abyrvalg
author_rank: Frequent Contributor
timestamp: July 31, 2021, 09:25:48 am
content: A Python tool appending CRC32 matching STM32 hw CRC:Code:#!/usr/bin/python3'''CRC32 matching STM32 hardware algo'''_table = {}_poly = 0x04C11DB7for i in range(256):c = i << 24for j in range(8):c = (c << 1) ^ _poly if (c & 0x80000000) else c << 1_table = c & 0xffffffffdef crc32stm(buf, prev=0xFFFFFFFF):crc = prevfor i in range(0, len(buf), 4):crc = ((crc << 8) & 0xffffffff) ^ _table[(crc >> 24) ^ buf[i+3]]crc = ((crc << 8) & 0xffffffff) ^ _table[(crc >> 24) ^ buf[i+2]]crc = ((crc << 8) & 0xffffffff) ^ _table[(crc >> 24) ^ buf[i+1]]crc = ((crc << 8) & 0xffffffff) ^ _table[(crc >> 24) ^ buf]return crcfrom sys import argv, exitfrom struct import packif len(argv)!=2:exit("Usage: "+argv+" <file>")with open(argv, 'rb+') as fp:crc = crc32stm(fp.read())fp.write(pack('<I', crc))To use in Cube:- install Python 3 (enable "install launcher" and "add to PATH")- add post-build command: python crc32stm.py yourbinary.bin

author: brucehoult
author_rank: Super Contributor
timestamp: July 31, 2021, 09:51:37 am
content: Oh crud .. cksum for some reason appends the length of the file to the data before calculating the CRC. That sucks.

author: peter-h
author_rank: Super Contributor
timestamp: August 01, 2021, 06:41:52 am
content: Here is the source for a win32 .exe which appends the CRC32 (the inverted JAMCRC) to a file:Code:#include <stdio.h>#include <stdlib.h>#include <stdint.h>#include <string.h>/*** CRC-32/JAMCRC. This is a "rolling" algorithm. Invert the *final* result for ISO-HDLC.* Returns 0x098494f3 from "123456789" (9 bytes). Inverting this (at the end) gives 0xcbf43926.* Polynomial is 0x04c11db7 and it holds it backwards (0xEDB88320).* This version accepts one byte at a time, and maintains the CRC in crcvalue. This makes it suitable* for calculating a CRC across a number of data blocks.* Speed is approx 400kbytes/sec.* crcvalue must be initialised to 0xffffffff by caller, and holds the accumulated CRC as you go along**/void crc32(uint8_t input_byte, uint32_t *crcvalue){for (uint32_t j = 0; j < 8; j++){uint32_t mask = (input_byte^*crcvalue) & 1;*crcvalue >>= 1;if (mask)*crcvalue = *crcvalue ^ 0xEDB88320;input_byte >>= 1;}}int main(int argc, char* argv[]){uint32_t crcvalue = 0xffffffff;FILE* file = NULL;if (argc > 1){if (argc > 2){int c = 0;if (!_strnicmp(argv, "0x", 2))c = 2;crcvalue = strtol(&argv, NULL, 16);}file = fopen(argv, "rb");if (file){while (!feof(file)){uint8_t input_byte;if (fread(&input_byte, sizeof(input_byte), 1, file)){crc32(input_byte, &crcvalue);}//else//{//printf("Before last zero byte = %08X\r\n", crcvalue);//crc32(0, &crcvalue);//printf("After last zero byte = %08X\r\n", crcvalue);//}}fclose(file);crcvalue = ~crcvalue;file = fopen(argv, "ab");fwrite(&crcvalue, sizeof(uint32_t), 1, file);fclose(file);}else{printf("Cannot open file\r\n");}file = NULL;}else{printf("Missing file name\r\n");}return 0;}An optional 2nd parameter is a different CRC initialisation value (default is 0xffffffff).Just curious: someone said that anything running out of Cube IDE should be written in Java.

author: peter-h
author_rank: Super Contributor
timestamp: August 08, 2021, 12:46:27 pm
content: I have finished the boot loader and it is programming the flash fine. I will post my code as an update to the other thread... took a few days to get it right.However I have problems producing the .bin file (to which I am appending the CRC). So currently I am flashing various jpeg files, which obviously don't run too wellThe batch file I am using, in the post build steps configuration, and which runs with no errors, is:Code:cd c:\kde420\project1\debugarm-none-eabi-objdump -h kde2020.elf > objdump.txtarm-none-eabi-objcopy --strip-debug -O binary kde2020.elf c:\kde420\project1\kde420.bincd c:\kde420\project1\c:\kde420\crcgen\addcrc kde420.bin 0xffffffffI noticed Cube is already generating a .bin file but that one is also 131MB. Hence I am generating my different kde420.bin file, but that is no smaller.Clearly objcopy is including stuff which is not in the final program (which is about 240k). The objdump.txt I am creating contains thisCode:kde2020.elf: file format elf32-littlearmSections:Idx Name Size VMA LMA File off Algn0 loader_bss 00000000 2001064c 2001064c 00090000 2**0CONTENTS1 KDE_loader 0000064c 20010000 08002c48 00030000 2**3CONTENTS, ALLOC, LOAD, READONLY, CODE2 .isr_vector 00000188 08000000 08000000 00010000 2**0CONTENTS, ALLOC, LOAD, READONLY, DATA3 .b_main.o 000011a4 08000188 08000188 00010188 2**2CONTENTS, ALLOC, LOAD, READONLY, CODE4 .45dbxx.o 000009b0 0800132c 0800132c 0001132c 2**2CONTENTS, ALLOC, LOAD, READONLY, CODE5 .KDE_loader.o 00000000 08001cdc 08001cdc 00090000 2**0CONTENTS6 .main_stub.o 00000008 08008000 08008000 00038000 2**1CONTENTS, ALLOC, LOAD, READONLY, CODE7 .main.o 000002c0 08008008 08008008 00038008 2**2CONTENTS, ALLOC, LOAD, READONLY, CODE8 .text 000382c0 080082d0 080082d0 000382d0 2**4CONTENTS, ALLOC, LOAD, READONLY, CODE9 .ARM.exidx 00000008 08040590 08040590 00070590 2**2CONTENTS, ALLOC, LOAD, READONLY, DATA10 .init_array 00000008 08040598 08040598 00070598 2**2CONTENTS, ALLOC, LOAD, DATA11 .fini_array 00000004 080405a0 080405a0 000705a0 2**2CONTENTS, ALLOC, LOAD, DATA12 .data 00000f6c 20000000 08001cdc 00020000 2**3CONTENTS, ALLOC, LOAD, DATA13 .KDE_loader 0000064c 08002c48 08002c48 00022c48 2**0ALLOC14 .bss 0000ddac 20000f70 20000f70 00090f70 2**3ALLOC15 .main_heap 00000004 2000ed1c 2000ed1c 00090f70 2**0ALLOC16 .ccmram 00010000 10000000 10000000 00080000 2**2CONTENTS, ALLOC, LOAD, DATA17 .ARM.attributes 00000030 00000000 00000000 00090000 2**0CONTENTS, READONLY18 .debug_info 00079159 00000000 00000000 00090030 2**0CONTENTS, READONLY, DEBUGGING, OCTETS19 .debug_abbrev 00010d81 00000000 00000000 00109189 2**0CONTENTS, READONLY, DEBUGGING, OCTETS20 .debug_aranges 00003eb8 00000000 00000000 00119f10 2**3CONTENTS, READONLY, DEBUGGING, OCTETS21 .debug_ranges 00003948 00000000 00000000 0011ddc8 2**3CONTENTS, READONLY, DEBUGGING, OCTETS22 .debug_macro 000417ae 00000000 00000000 00121710 2**0CONTENTS, READONLY, DEBUGGING, OCTETS23 .debug_line 00062b3d 00000000 00000000 00162ebe 2**0CONTENTS, READONLY, DEBUGGING, OCTETS24 .debug_str 001128f3 00000000 00000000 001c59fb 2**0CONTENTS, READONLY, DEBUGGING, OCTETS25 .comment 00000053 00000000 00000000 002d82ee 2**0CONTENTS, READONLY26 .debug_frame 0000d4c4 00000000 00000000 002d8344 2**2CONTENTS, READONLY, DEBUGGING, OCTETSIt looks like I should have only this (I do want initialised DATA included, obviously)Any ideas on how to achieve this?I've been looking at various references e.g.https://sourceware.org/binutils/docs/binutils/objcopy.htmlbut none of the --strip... options I have tried make any difference.

author: gf
author_rank: Super Contributor
timestamp: August 08, 2021, 01:19:52 pm
content: My understanding is that all sections tagged with "LOAD" are included in the bin file, and all gaps between the load addresses of the included sections are included, too, and filled.The section .ccmram is tagged with "LOAD" too, and there seems to be a larger gap between .data and .ccram.Edit: So if I did not overlook anything, I'd expect the .bin file to contain the load address range 0x08000000 - 0x1000ffff (which implies a file size of 134283264 bytes)

author: peter-h
author_rank: Super Contributor
timestamp: August 08, 2021, 01:53:47 pm
content: Yeah... after much hacking I found this gives me a 258k filearm-none-eabi-objcopy -S --strip-all -R .bss -R .main_heap -R .ccmram -O binary kde2020.elf output.binwhich seems to contain the right stuff - at least in the beginningFWIW I am running the Cube post build stuff out of a batch file. I didn't find any way to add multiple command lines. Some people online said they did but Cube didn't preserve their order, but putting them in a .bat sidestepped that.AND IT IS RUNNING THE CODE!Took a few days' coding.

author: gf
author_rank: Super Contributor
timestamp: August 08, 2021, 02:00:47 pm
content: Sure, with -R you can exclude sections. But the actual question is, why did .ccram become a loadable section in the first place, if it is not suposed to become part of the flash? I guess this is a bug in the linker script?

author: peter-h
author_rank: Super Contributor
timestamp: August 08, 2021, 02:10:05 pm
content: Yes course there is a bug in my linkfileI am right on the edge with those.I now need a reboot function, for the boot loader, which doesn't need calling any of the ST stuff. I condensed this out of one of the HAL... things but it just hangs. I will work on it but maybe somebody knows something right offCode:static void reboot(void){__ASM volatile ("dsb 0xF":::"memory");// Ensure all outstanding memory accesses included// buffered write are completed before resetSCB->AIRCR = (uint32_t)((0x5FAUL << 16) | (SCB->AIRCR & 0x700) | 0 );// Keep priority group unchanged__ASM volatile ("dsb 0xF":::"memory");// Ensure completion of memory accessfor(;;)// wait until reset{__ASM volatile ("nop");}}

author: cv007
author_rank: Frequent Contributor
timestamp: August 08, 2021, 03:07:18 pm
content: bit 3- SYSRESETREQ - needs to be set.

author: peter-h
author_rank: Super Contributor
timestamp: August 08, 2021, 04:00:36 pm
content: Nothing in the RM, strangely enough. This is in core_cm4.h and did eventually work. I originally tried to simplify it a bit - not a good ideaCode:__NO_RETURN __STATIC_INLINE void __NVIC_SystemReset(void){__DSB(); /* Ensure all outstanding memory accesses includedbuffered write are completed before reset */SCB->AIRCR = (uint32_t)((0x5FAUL << SCB_AIRCR_VECTKEY_Pos) |(SCB->AIRCR & SCB_AIRCR_PRIGROUP_Msk) |SCB_AIRCR_SYSRESETREQ_Msk ); /* Keep priority group unchanged */__DSB(); /* Ensure completion of memory access */for(;;) /* wait until reset */{__NOP();}}In case anyone is interested, this is my linkfileCode:/* Entry Point */ENTRY(Reset_Handler)/* Reference boot block stuff to ensure that it gets linked-in - not sure if these do anything *//* EXTERN(b_boot) *//* EXTERN(KDE_loader) *//* Highest address of the main stack. This stack is only for startup code, boot loader, and ISRs. The RTOS has its own. *//* _estack = 0x20020000; */ /* stack in 128K RAM */_estack = 0x10010000;/* stack in 64k CCM - note: configTOTAL_HEAP_SIZE + min_stack_size must not exceed 64k) *//* top of RAM for _sbrk - top of heap check */_top = 0x20020000;/* Heap and stack sizes */_Min_Heap_Size = 0x0000; /* 21/7/21 set to zero to prevent heap/stack conflict check from blowing up due to boot loader area */_Min_Stack_Size = 0x4000; /* 16k stack - in CCM *//* Specify the memory areas *//* CCMRAM added PH 12/5/2021 - cannot use with DMA */MEMORY{FLASH_BOOT (rx) : ORIGIN = 0x08000000, LENGTH = 32KFLASH_APP (rx) : ORIGIN = 0x08008000, LENGTH = 1024K-32KRAM (xrw) : ORIGIN = 0x20000000, LENGTH = 128KMEMORY_B1 (rx) : ORIGIN = 0x60000000, LENGTH = 0KCCMRAM (rw) : ORIGIN = 0x10000000, LENGTH = 64K}/* Define output sections */SECTIONS{/* loader_bss and loader sections must come first, in order to override thewildcards in subsequent sections */loader_bss _loader_end :{_loader_bss_start = .;*KDE_loader.o(.bss*). = ALIGN(4);_loader_bss_end = .;}/* General purpose buffer for loader usage, b_main.c and KDE_loader.c.Done this way to avoid spurious section overlap errors; the loader reboots when doneso it can use any RAM it wants */buffer1=0x10000000;/* base of CCM *//* RAM based loader is linked to execute here */KDE_loader 0x20010000 : AT(_loader_loadaddr){_loader_start = .;*KDE_loader.o(.text*)*KDE_loader.o(.rodata*)*KDE_loader.o(.data*)*KDE_loader.o(.common*)*KDE_loader.o(.ARM.attributes). = ALIGN(4);_loader_end = .;}_loader_size = SIZEOF(KDE_loader);/* Place modules into FLASH, starting at the bottom (0x08000000)The line "*b_main.o (.text .text* .rodata .rodata*)" is important, but for some reason(probably the reset_handler reference) is not needed for isr_vector *//* Startup code */.isr_vector :{. = ALIGN(4);KEEP(*(.isr_vector)). = ALIGN(4);} >FLASH_BOOT.b_main.o :{. = ALIGN(4);KEEP(*(.b_main.o))*b_main.o (.text .text* .rodata .rodata*). = ALIGN(4);} >FLASH_BOOT.45dbxx.o :{. = ALIGN(4);KEEP(*(.45dbxx.o))*45dbxx.o (.text .text* .rodata .rodata*). = ALIGN(4);} >FLASH_BOOT/* === other boot block stuff will go here === *//* KDE_loader.o - goes last in the bottom block which makes it easier to check the block usageby checking _loader_end_in_flash in the .map file */.KDE_loader.o :{. = ALIGN(4);KEEP(*(.KDE_loader.o))*KDE_loader.o (.text .text* .rodata .rodata*). = ALIGN(4);_loader_end_in_flash = .;} >FLASH_BOOT/* The rest of the KDE code goes here, loaded at base+32k, starting with a stub and then the real main() */.main_stub.o :{. = ALIGN(4);KEEP(*(.main_stub.o))*main_stub.o (.text .text* .rodata .rodata*). = ALIGN(4);} >FLASH_APP.main.o :{. = ALIGN(4);KEEP(*(.main.o))*main.o (.text .text* .rodata .rodata*). = ALIGN(4);} >FLASH_APP/* This collects all other stuff, which gets loaded into FLASH after main.o above */.text :{. = ALIGN(4);*(.text) /* .text sections (code) */*(.text*) /* .text* sections (code) */*(.rodata) /* .rodata sections (constants, strings, etc.) */*(.rodata*) /* .rodata* sections (constants, strings, etc.) */*(.glue_7) /* glue arm to thumb code */*(.glue_7t) /* glue thumb to arm code */*(.eh_frame)KEEP (*(.init))KEEP (*(.fini)). = ALIGN(4);_etext = .; /* define a global symbol at end of code */} >FLASH_APP/* this is for __libc_init_array() */.preinit_array :{PROVIDE_HIDDEN (__preinit_array_start = .);KEEP (*(.preinit_array*))PROVIDE_HIDDEN (__preinit_array_end = .);} >FLASH_APP.init_array :{PROVIDE_HIDDEN (__init_array_start = .);KEEP (*(SORT(.init_array.*)))KEEP (*(.init_array*))PROVIDE_HIDDEN (__init_array_end = .);} >FLASH_APP.fini_array :{PROVIDE_HIDDEN (__fini_array_start = .);KEEP (*(.fini_array*))KEEP (*(SORT(.fini_array.*)))PROVIDE_HIDDEN (__fini_array_end = .);} >FLASH_APP/* Initialized data sections goes into RAM, load LMA copy after code */.data :{. = ALIGN(4);_sdata = .; /* create a global symbol at data start */*(.data) /* .data sections */*(.data*) /* .data* sections */. = ALIGN(4);_edata = .; /* define a global symbol at data end */} >RAM AT >FLASH_BOOT/* used by the startup to initialize data */_sidata = LOADADDR(.data);/* dummy placeholder in flash for loader section, to count flash usage */.KDE_loader :{. = . + SIZEOF(KDE_loader);} AT >FLASH_BOOT_loader_loadaddr = LOADADDR(.KDE_loader);/* Uninitialized data section */. = ALIGN(4);.bss :{/* This is used by the startup in order to initialize the .bss section */_sbss = .; /* define a global symbol at bss start */__bss_start__ = _sbss;*(.bss)*(.bss*)*(COMMON). = ALIGN(4);_ebss = .; /* define a global symbol at bss end */__bss_end__ = _ebss;} >RAM/* The heap ends up after BSS in main RAM *//* This also checks that the top of the heap doesn't hit the bottom of the stack i.e. how much RAM left *//* User_heap_stack section, used to check that there is enough RAM left *//* ._user_heap_stack : *//* This check is basically disabled because _Min_Heap_Size=0 above */.main_heap :{. = ALIGN(8);PROVIDE ( end = . );PROVIDE ( _end = . );. = . + _Min_Heap_Size;/* . = . + _Min_Stack_Size; */ /* PH 14/5/2021 stack is in CCM, not here */. = ALIGN(8);} >RAM/* } >CCMRAM *//* MEMORY_bank1 section, code must be located here explicitly *//* Example: extern int foo(void) __attribute__ ((section (".mb1text"))); *//* Not used 14/5/2021 - was apparently used for LCD display on ST dev kit */.memory_b1_text :{*(.mb1text) /* .mb1text sections (code) */*(.mb1text*) /* .mb1text* sections (code) */*(.mb1rodata) /* read-only data (constants) */*(.mb1rodata*)} >MEMORY_B1/* CCM-RAM section** IMPORTANT NOTE!* If variables placed in this section must be zero initialized,* the startup code needs to be modified to initialize this section.* Done PH 12/5/2021*/.ccmram :{. = ALIGN(4);_sccmram = .; /* create a global symbol at ccmram start */*(.ccmram)*(.ccmram*). = ALIGN(4);_eccmram = .; /* create a global symbol at ccmram end */} >CCMRAM/* Remove information from the standard libraries *//DISCARD/ :{libc.a ( * )libm.a ( * )libgcc.a ( * )}.ARM.attributes 0 : { *(.ARM.attributes) }}When I get around to it I will post my flashing code herehttps://www.eevblog.com/forum/microcontrollers/32f417-best-way-to-program-the-flash-from-ram-based-code/new/#newA lot of work went into it.

author: ajb
author_rank: Super Contributor
timestamp: August 09, 2021, 06:26:56 am
content: The reference manual for ARM MCUs only covers the information specific to that part: bus architecture, memory layout, peripherals, that sort of thing. The operation of the core, of which the NVIC is part, is going to be the same as any other part with the same core (notwithstanding options like caches, FPU, etc). So there's no point in including those details because they're already adequate covered by the ARM architecture reference material--and being the ones who designed the core, ARM are in a better position to document that stuff anyway.So for a comprehensive understanding of an ARM part you really the datasheet for electrical/performance, the reference manual for operational and peripheral information, and the relevant ARM documentation for how the core works

author: gf
author_rank: Super Contributor
timestamp: August 09, 2021, 11:15:03 am
content: QuoteDid you possibly place anyinitializeddata in your .ccram section?Seems that the section does have some contents:Code:16 .ccmram 00010000 10000000 10000000 00080000 2**2CONTENTS, ALLOC, LOAD, DATA

author: peter-h
author_rank: Super Contributor
timestamp: August 09, 2021, 12:08:54 pm
content: There should be nothinginitialisedin the CCM.The CCM is 64k, which is allocated as48k for the FreeRTOS "buffer" (it runs all the processes, including their stacks, in there) and 16k general stack (used by main() before the RTOS starts, and by ISRs, and of course by the boot loader which basically gets control instead of (most of) main().I have no variables likeuint64_t fred=55;in the CCM.

author: gf
author_rank: Super Contributor
timestamp: August 09, 2021, 01:17:30 pm
content: Hmm. Tried a small test pogram:Code:char buffer __attribute__((section(".ccram")));Unllike regular uninitialized global variables (which either become common, or go into bss w/o "CONTENTS" or "LOAD" tag), gcc obviouslydoes initializebuffer[] with zeros.Code:$ objdump -h -s x.oSektionen:Idx Name Size VMA LMA File off Algn0 .text 00000000 0000000000000000 0000000000000000 00000040 2**0CONTENTS, ALLOC, LOAD, READONLY, CODE1 .data 00000000 0000000000000000 0000000000000000 00000040 2**0CONTENTS, ALLOC, LOAD, DATA2 .bss 00000000 0000000000000000 0000000000000000 00000040 2**0ALLOC3 .ccram 00000400 0000000000000000 0000000000000000 00000040 2**5CONTENTS, ALLOC, LOAD, DATA4 .comment 0000002a 0000000000000000 0000000000000000 00000440 2**0CONTENTS, READONLY5 .note.GNU-stack 00000000 0000000000000000 0000000000000000 0000046a 2**0CONTENTS, READONLYInhalt von Abschnitt .ccram:0000 00000000 00000000 00000000 00000000 ................0010 00000000 00000000 00000000 00000000 ................0020 00000000 00000000 00000000 00000000 ................0030 00000000 00000000 00000000 00000000 ................0040 00000000 00000000 00000000 00000000 ................0050 00000000 00000000 00000000 00000000 ................0060 00000000 00000000 00000000 00000000 ................0070 00000000 00000000 00000000 00000000 ................0080 00000000 00000000 00000000 00000000 ................0090 00000000 00000000 00000000 00000000 ................00a0 00000000 00000000 00000000 00000000 ................00b0 00000000 00000000 00000000 00000000 ................00c0 00000000 00000000 00000000 00000000 ................00d0 00000000 00000000 00000000 00000000 ................00e0 00000000 00000000 00000000 00000000 ................00f0 00000000 00000000 00000000 00000000 ................0100 00000000 00000000 00000000 00000000 ................0110 00000000 00000000 00000000 00000000 ................0120 00000000 00000000 00000000 00000000 ................0130 00000000 00000000 00000000 00000000 ................0140 00000000 00000000 00000000 00000000 ................0150 00000000 00000000 00000000 00000000 ................0160 00000000 00000000 00000000 00000000 ................0170 00000000 00000000 00000000 00000000 ................0180 00000000 00000000 00000000 00000000 ................0190 00000000 00000000 00000000 00000000 ................01a0 00000000 00000000 00000000 00000000 ................01b0 00000000 00000000 00000000 00000000 ................01c0 00000000 00000000 00000000 00000000 ................01d0 00000000 00000000 00000000 00000000 ................01e0 00000000 00000000 00000000 00000000 ................01f0 00000000 00000000 00000000 00000000 ................0200 00000000 00000000 00000000 00000000 ................0210 00000000 00000000 00000000 00000000 ................0220 00000000 00000000 00000000 00000000 ................0230 00000000 00000000 00000000 00000000 ................0240 00000000 00000000 00000000 00000000 ................0250 00000000 00000000 00000000 00000000 ................0260 00000000 00000000 00000000 00000000 ................0270 00000000 00000000 00000000 00000000 ................0280 00000000 00000000 00000000 00000000 ................0290 00000000 00000000 00000000 00000000 ................02a0 00000000 00000000 00000000 00000000 ................02b0 00000000 00000000 00000000 00000000 ................02c0 00000000 00000000 00000000 00000000 ................02d0 00000000 00000000 00000000 00000000 ................02e0 00000000 00000000 00000000 00000000 ................02f0 00000000 00000000 00000000 00000000 ................0300 00000000 00000000 00000000 00000000 ................0310 00000000 00000000 00000000 00000000 ................0320 00000000 00000000 00000000 00000000 ................0330 00000000 00000000 00000000 00000000 ................0340 00000000 00000000 00000000 00000000 ................0350 00000000 00000000 00000000 00000000 ................0360 00000000 00000000 00000000 00000000 ................0370 00000000 00000000 00000000 00000000 ................0380 00000000 00000000 00000000 00000000 ................0390 00000000 00000000 00000000 00000000 ................03a0 00000000 00000000 00000000 00000000 ................03b0 00000000 00000000 00000000 00000000 ................03c0 00000000 00000000 00000000 00000000 ................03d0 00000000 00000000 00000000 00000000 ................03e0 00000000 00000000 00000000 00000000 ................03f0 00000000 00000000 00000000 00000000 ................Inhalt von Abschnitt .comment:0000 00474343 3a202855 62756e74 7520372e .GCC: (Ubuntu 7.0010 352e302d 33756275 6e747531 7e31382e 5.0-3ubuntu1~18.0020 30342920 372e352e 3000 04) 7.5.0.Edit: I wonder whether.ccmram(NOLOAD):{...}in the linker script helps?

author: peter-h
author_rank: Super Contributor
timestamp: August 09, 2021, 07:46:08 pm
content: Do you mean that if one allocates say 16k of CCM to a buffer, you end up with a 16k block of 0x00 in the initialised DATA?It wouldn't actually get copied to CCM because my startup (the st... .s bit) doesn't have any code for that.I excluded that section from the binary output anyway, witharm-none-eabi-objcopy -S --strip-all -R .bss -R .main_heap -R .ccmram -O binary kde2020.elf c:\kde420\project1\factory0.datso I am ok with it... but this was probably responsible for the 131MB bin file

author: gf
author_rank: Super Contributor
timestamp: August 09, 2021, 08:09:04 pm
content: QuoteObviously, but the 16k of zeros are stored as contents of the .ccram section, not in the .data section.Just do an objdump -h -s on the .o file, in order to check.QuoteIn your objdump output, the section .ccram is tagged with LOAD, therefore it is included in the .bin file (unless you exclude it explicitly).And if both, .data and .ccram are included, then the huge gap between .data and .ccram is included as well in the .bin file.The .bin file represents a memory dump of contiguous load addresses, so any gaps between incuded sections have to be filled.

author: peter-h
author_rank: Super Contributor
timestamp: August 09, 2021, 09:38:49 pm
content: "Just do an objdump -h -s on the .o file, in order to check."This is the section listCode:project1.elf: file format elf32-littlearmSections:Idx Name Size VMA LMA File off Algn0 loader_bss 00000000 20010860 20010860 00090000 2**0CONTENTS1 KDE_loader 00000860 20010000 08002c98 00030000 2**3CONTENTS, ALLOC, LOAD, READONLY, CODE2 .isr_vector 00000188 08000000 08000000 00010000 2**0CONTENTS, ALLOC, LOAD, READONLY, DATA3 .b_main.o 000011f4 08000188 08000188 00010188 2**2CONTENTS, ALLOC, LOAD, READONLY, CODE4 .45dbxx.o 000009b0 0800137c 0800137c 0001137c 2**2CONTENTS, ALLOC, LOAD, READONLY, CODE5 .KDE_loader.o 00000000 08001d2c 08001d2c 00090000 2**0CONTENTS6 .main_stub.o 00000008 08008000 08008000 00038000 2**1CONTENTS, ALLOC, LOAD, READONLY, CODE7 .main.o 000002c0 08008008 08008008 00038008 2**2CONTENTS, ALLOC, LOAD, READONLY, CODE8 .text 000383f0 080082d0 080082d0 000382d0 2**4CONTENTS, ALLOC, LOAD, READONLY, CODE9 .ARM.exidx 00000008 080406c0 080406c0 000706c0 2**2CONTENTS, ALLOC, LOAD, READONLY, DATA10 .init_array 00000008 080406c8 080406c8 000706c8 2**2CONTENTS, ALLOC, LOAD, DATA11 .fini_array 00000004 080406d0 080406d0 000706d0 2**2CONTENTS, ALLOC, LOAD, DATA12 .data 00000f6c 20000000 08001d2c 00020000 2**3CONTENTS, ALLOC, LOAD, DATA13 .KDE_loader 00000860 08002c98 08002c98 00022c98 2**0ALLOC14 .bss 0000ddac 20000f70 20000f70 00090f70 2**3ALLOC15 .main_heap 00000004 2000ed1c 2000ed1c 00090f70 2**0ALLOC16 .ccmram 00010000 10000000 10000000 00080000 2**2CONTENTS, ALLOC, LOAD, DATA17 .ARM.attributes 00000030 00000000 00000000 00090000 2**0CONTENTS, READONLY18 .debug_info 0007935a 00000000 00000000 00090030 2**0CONTENTS, READONLY, DEBUGGING, OCTETS19 .debug_abbrev 00010da6 00000000 00000000 0010938a 2**0CONTENTS, READONLY, DEBUGGING, OCTETS20 .debug_aranges 00003ec0 00000000 00000000 0011a130 2**3CONTENTS, READONLY, DEBUGGING, OCTETS21 .debug_ranges 00003938 00000000 00000000 0011dff0 2**3CONTENTS, READONLY, DEBUGGING, OCTETS22 .debug_macro 000417ae 00000000 00000000 00121928 2**0CONTENTS, READONLY, DEBUGGING, OCTETS23 .debug_line 00062dcb 00000000 00000000 001630d6 2**0CONTENTS, READONLY, DEBUGGING, OCTETS24 .debug_str 001128fe 00000000 00000000 001c5ea1 2**0CONTENTS, READONLY, DEBUGGING, OCTETS25 .comment 00000053 00000000 00000000 002d879f 2**0CONTENTS, READONLY26 .debug_frame 0000d4d4 00000000 00000000 002d87f4 2**2CONTENTS, READONLY, DEBUGGING, OCTETSI see a 64k CCM block there, not 16k. Could be this bit in the linkfileCode:MEMORY{FLASH_BOOT (rx) : ORIGIN = 0x08000000, LENGTH = 32KFLASH_APP (rx) : ORIGIN = 0x08008000, LENGTH = 1024K-32KRAM (xrw) : ORIGIN = 0x20000000, LENGTH = 128KMEMORY_B1 (rx) : ORIGIN = 0x60000000, LENGTH = 0KCCMRAM (rw) : ORIGIN = 0x10000000, LENGTH = 64K}"The .bin file represents a memory dump of contiguous load addresses, so any gaps between incuded sections have to be filled."Sure; I see that. However this wheel must have been re-invented by so many already, because sure as hell Cube is not sending 131MB to the STLINK debuggerThey must be doing "something".

author: gf
author_rank: Super Contributor
timestamp: August 09, 2021, 09:53:58 pm
content: Quote from: peter-h on August 09, 2021, 09:38:49 pmI meant the .o file where the ccram buffer is defined, not the final .elf.The .map file generated by the linker may interesting too, as it tells in detail the input sections and .o files which have been included into each output section, and at which address.

author: abyrvalg
author_rank: Frequent Contributor
timestamp: August 09, 2021, 09:58:03 pm
content: .bss* section names are recognized by as/ld and treated as NOLOAD automatically (see GNU as manual section 4.2). You either need to specify NOLOAD type (as gf already advised, I would prefer this way too, explicit is better than implicit) or name it somehow like .bss.ccmram.Manuals are cool