Where has all my memory gone? STM32 - Page 1
Forum section: electronics / general

author: Boscoe
author_rank: Frequent Contributor
timestamp: March 04, 2018, 01:23:22 pm
content: Hi all,I've got a strange problem that I'm struggling to deal with due to lack of tools if nothing else. I'm working on a project with the STM32F030F4 with 4kB of RAM with Eclipse. I'm using it as a communication controller where I send it packets of data (up to 3kB long) and it does some work based on what's inside it. The packet is in ASCII format and is mostly made up of an array of 256 int32 numbers and is ended with a \n. The code will store all the packet in a buffer until it sees the newline and then decodes to get raw int32s into a 256 element array.So in my code I have two large data structures and then a few variables here and there (insignificant for my problem).I have in main.c,Code:int32_t data;Which I think will occupy 1024 bytes of memory.and then I have in serial_decoder.c,Code:char packet_buffer;I believe this should be 1000 bytes.In this case, all is fine and it compiles. But, if I increase the packet buffer length to 1100 bytes, I get a RAM overflow error which does not make sense to me, where did my other 2kB of RAM go?I have decreases the minimum stack size to 0 so it's all free for the heap. Have I calculated the sizes incorrectly? Are there any good tools to see how much memory all the variables have used? Any help or suggestions would be amazing.ThanksPS: This is the output from the size analysis:Code:textdatabssdechexfilename22000220dcsrc/gpio.o31802320140src/main.o958839693c9src/serial_decoder.o3330033314dsrc/spi.o84008454src/stm32f0xx_hal_msp.o82008252src/stm32f0xx_it.o3764038017csrc/system_stm32f0xx.o32700327147src/usart.o115964843348154283c44coef_update.elf

author: Jeroen3
author_rank: Super Contributor
timestamp: March 04, 2018, 01:50:45 pm
content: What does your linker script look like. You could just be overflowing a region since they aren't properly defined.You have 4096 bytes of SRAM at 0x20000000 up to 0x20000FFF. Meaning you could easily fit your buffers, including some and stack. (I don't think you need heap)You even have 1024 bytes of dedicated USB packet buffer SRAM, which usually takes a chunk of your regular SRAM.

author: andersm
author_rank: Super Contributor
timestamp: March 04, 2018, 02:19:14 pm
content: Quote from: Boscoe on March 04, 2018, 01:23:22 pmYou need some stack. How much depends on your application, but will needsome. Local variables, caller-saved variables, function return addresses, interrupt contexts all get saved on the stack.QuoteAccording to the size output, you have over 3K of uninitialized data, and close to 0.5K of initialized data. Start by figuring out where that goes. Large amounts of initialized data could be tables that aren't const, or a vector table relocated to RAM.

author: donotdespisethesnake
author_rank: Super Contributor
timestamp: March 04, 2018, 02:51:32 pm
content: Quote from: Boscoe on March 04, 2018, 01:23:22 pmThe only tool you need is the linker map file, it has a breakdown of where all the RAM goes.

author: andyturk
author_rank: Frequent Contributor
timestamp: March 04, 2018, 03:52:36 pm
content: Generally speaking, if you're building fewer than 100K gizmos, you shouldn't be attempting to use 75% of available SRAM at this stage. I.e., you'll be better off simply getting a processor with more resources. To be safe (and not waste time optimizing too early), figure out what minimum resources you'll need (SRAM, flash, GPIO, cpu speed, etc.), double those figures, and then pick your hardware.Koopmansays,"Software costs rise dramatically when system resources are more than 75% to 85% full."p. 178That said, the thing you're building seems simple enough and ought to fit (with some effort) the mcu you chose. I worked on a project where we ended up *way* too close to the limit and had to constantly squeeze SRAM. I ended up writinga script to calculate the 50 largest items in memory. It's a makefile, and it relies on readelf, egrep and sqlite3 to do the heavy lifting. The output looks like this:Code:Linking build/main.elftextdatabssdechexfilename922391332174281109991b197build/main.elfExtracting symbol data from build/main.elf into build/main_symbols.txtConverting build/main_symbols.txt to build/main_symbols.csvCreating build/main.db from build/main_symbols.csvCreating build/main.report from build/main.dbTotal RAM Used--------------16695size value Largest items in RAM---------- ---------- --------------------------------------------------------------------------------5912 200028f0 Application& akt::the<Application>()::memory4096 20001148 os_mem1024 20000d30 _acUpBuffer1008 2000454c hpcd_USB_OTG_FS548 20004048 hUsbDeviceFS512 2000434c USBD_StrDesc364 20000bc4 __global_locale200 20002210 os_idle_thread_stack200 200022d8 os_timer_thread_stack192 20000aa0 (anonymous namespace)::afs168 20000948 osRtxInfo168 200042a4 _SEGGER_RTT96 20000b64 impure_data80 200023d4 os_timer_mq_data68 20002188 os_idle_thread_cb68 200021cc os_timer_thread_cb67 20000904 USBD_CDC_OtherSpeedCfgDesc67 200008c0 USBD_CDC_CfgFSDesc67 2000087c USBD_CDC_CfgHSDesc64 20002148 os_isr_queue56 20000844 USBD_CDC52 200023a0 os_timer_mq_cb48 20000a04 (anonymous namespace)::modes48 20000a4c (anonymous namespace)::speeds36 20000a7c (anonymous namespace)::pulls32 20004270 akt::shell::ShellCommand::argv28 20000804 FS_Desc24 20000a34 (anonymous namespace)::outputs24 20002668 test::spi_mosi24 20002608 test::adc1024 200024e8 test::dac124 200027b8 test::uart_rts24 20002818 test::gpio224 200028d8 test::hse_out24 200028c0 test::hse_in24 20002470 test::usb_dp24 20002590 test::disp_miso24 20002698 test::lse_out24 20002680 test::lse_in24 20002770 test::gpio724 200025d8 test::scl24 20002488 test::swdio24 200026f8 test::spi_cs24 200024b8 test::tdi24 20002530 test::disp_dc24 20002440 test::led_g24 200025c0 test::swo24 200024d0 test::led_r24 20002740 test::gpio524 200027a0 test::uart_ctsTotal FLASH Used----------------90886size value Largest items in FLASH---------- ---------- --------------------------------------------------------------------------------2322 080028b1 HAL_PCD_IRQHandler2040 0800b409 __static_initialization_and_destruction_0(int, int)1232 0800ca59 akt::stm32::PinCommand::exec()1204 08001a45 HAL_RCC_OscConfig1156 08003f0d USB_EPStartXfer1114 0800a00d RTTThread::run()898 080108cd xvprintf(akt::Console&, char const*, std::__va_list)820 080022f9 HAL_GPIO_Init810 080008b5 SEGGER_RTT_vprintf788 08009559 svcRtxThreadNew732 080136ad __udivmoddi4700 080078cd svcRtxKernelInitialize700 08004391 USB_EP0StartXfer628 0800afb5 Application::run()620 08011933 akt::views::Canvas::draw_line(akt::views::Point, akt::views::Point, unsigned sho612 080038f5 USB_DevInit576 080127c9 _printf_i556 080124ad _vfiprintf_r556 080124ad _vfprintf_r548 08001ef9 HAL_RCC_ClockConfig528 0800ada5 Application::Application()528 0800ada5 Application::Application()526 0800265d HAL_PCD_Init488 080132f5 _svfprintf_r488 080132f5 _svfiprintf_r486 08006699 USBD_GetDescriptor484 0800eba1 akt::stm32::UART::UART(USART_TypeDef&, unsigned char, akt::stm32::DMA::Stream&,484 0800eba1 akt::stm32::UART::UART(USART_TypeDef&, unsigned char, akt::stm32::DMA::Stream&,466 08011405 akt::views::Rect::clip(akt::views::Point&, akt::views::Point&) const432 080064e7 USBD_StdEPReq416 08011bff akt::views::IO::interpret(unsigned char const*)412 0800aa7d UARTCommand::exec()392 08000000 cortex_vectors392 08000c11 MX_GPIO_Init388 08003d89 USB_DeactivateEndpoint364 08005009 LL_USART_SetBaudRate360 0800e01b akt::stm32::SPI::transfer(void*, void const*, unsigned short, unsigned long)356 0800fcc1 akt::stm32::USBDevice::cdc_control(unsigned char, unsigned char*, unsigned short344 0800060d _PrintUnsigned334 08000765 _PrintInt328 0800dd1d akt::stm32::SPI::SPI(SPI_TypeDef&, unsigned int, akt::stm32::DMA::Stream&, unsig328 0800cf4d akt::stm32::PinCommand::print_pin_config(akt::stm32::Pin const&)328 0800d821 akt::stm32::Pin::is_port_enabled() const328 0800dd1d akt::stm32::SPI::SPI(SPI_TypeDef&, unsigned int, akt::stm32::DMA::Stream&, unsig320 08008771 svcRtxMutexRelease318 0800f365 akt::stm32::UART::read(unsigned char*, unsigned int, unsigned long)312 080055f1 LL_TIM_Init304 0800efb9 akt::stm32::UART::start()296 0800a561 akt::shell::InfoCommand::exec()294 080082c3 svcRtxMessageQueuePut

author: hans
author_rank: Super Contributor
timestamp: March 04, 2018, 05:24:57 pm
content: Try running (if you're on a UNIX OS):QuoteThis prints all names in the project with it's size (-S) , and sorts them (--size-sort). Then grep $200 filters all lines that start with "200", which in my EFM32G project is the prefix of the SRAM memory (20000000). This for me then directly prints:QuoteWith address (hex), size (hex), type of allocation (see man nm, B=unitialized, d=initialized, ?=unknown due to sections)If you're not on Windows but do use the GNU toolchain, you can ofcourse run the nm tool manually and filter the results via a spreadsheet tool (export to CSV, import with spaces as seperator, and filter).---I suspect probably that you're using either 2K stack or heap in your program.Note that it's never wise to run a program to 100% RAM. The "used RAM" given of the size tool is not including the stack; which must have sufficient room in order to grow and shrink.If you really need the 3Kbyte buffer plus 256x4 byte second buffer, it sounds like you better get a 8KB RAM part, or perhaps re-use the serial buffer to also store the binary integers (but IMO more trouble than it's worth).

author: mikeselectricstuff
author_rank: Super Contributor
timestamp: March 04, 2018, 05:43:30 pm
content: Quote from: andyturk on March 04, 2018, 03:52:36 pmThat's a nonsensical statement - more so with a figure of 100K. MCU applications are way too diverse to make meaningful statements which are that generally applicable. Every application is different. It may be that RAM and other requirements are very well defined and finite, and as long as it's not being wasted somewhere, using 90% or more may be fine.

author: andyturk
author_rank: Frequent Contributor
timestamp: March 07, 2018, 05:04:32 am
content: Quote from: mikeselectricstuff on March 04, 2018, 05:43:30 pmI disagree. Koopman's advice makes sense in commercial projects where requirements change over time, and late-stage redesigns get really expensive. You might be able to target 90% resource usage in your projects and do just fine, but that's not one-size-fits-all advice either.These days, most mcu vendors have a variety of mcus to choose from with different configurations of RAM, flash, peripherals, etc. It's relatively easy to start with one chip (in a given family) and slide up or down the scale to match requirements when they're known. Personally, I like to start at the high end for early development and then slide down to cheaper/less capable chips once firmware is feature complete. Management likes it when you say, "hey, we found a way to build this cheaper."You can end up at 90% resource usage if you plan everything right. That's great. But if you start there and have to constantly count bytes throughout development, it's probably wasted effort.

author: mikeselectricstuff
author_rank: Super Contributor
timestamp: March 07, 2018, 10:24:18 am
content: Quote from: andyturk on March 07, 2018, 05:04:32 amNo argument there. Just saying that there are no generalisations and you need to look at it on a case-by-case business.Also no argument that for low volumes it's not worth squeezing the last cent, but 100K is a ridiculous figure. If the next biggest part is another 50 cents, $50K buys a hell of a lot of optimisation time.

author: rstofer
author_rank: Super Contributor
timestamp: March 07, 2018, 04:28:11 pm
content: My big concern would be 'feature creep' requiring more than the remaining resources.

author: mikeselectricstuff
author_rank: Super Contributor
timestamp: March 07, 2018, 04:31:26 pm
content: Quote from: rstofer on March 07, 2018, 04:28:11 pmThat can work both ways : "Sorry I can't implement the latest idiotic idea from the morons in the Sales dept as there's no room". End of discussion.

author: westfw
author_rank: Super Contributor
timestamp: March 08, 2018, 02:41:11 am
content: QuoteQuoteFWIW, I find "nm" to be much more useful than map files. The map files always seem to be hopelessly verbose, full of a lot of information that isn't particularly useful for any one case (and object size analysis uses a tiny part of it.) (also, for most "real" builds, the map file gets very cluttered with long path names to files you didn't care about, and information about the debug sections.) (Hmm. I wonder if there are more link options to help keep the noise in check?)Other useful "nm" expressions:Code:# Show only symbols from bss and data sections. De-mangle C++ symbols.arm-none-eabi-nm -SC --size-sort .elf | grep -i "  "# Show only symbols from codearm-none-eabi-nm -SC --size-sort .elf | grep -i " t "

author: rstofer
author_rank: Super Contributor
timestamp: March 08, 2018, 03:40:40 am
content: Quote from: mikeselectricstuff on March 07, 2018, 04:31:26 pmAnd your replacement will suggest a different chip, add the features and live happily ever after on his stock options.Engineering never runs a company - sad as it is. Mostly it is run by finance but sales carries a lot of weight. Engineers come and go, some stay longer than others.

author: donotdespisethesnake
author_rank: Super Contributor
timestamp: March 08, 2018, 10:22:49 am
content: Quote from: westfw on March 08, 2018, 02:41:11 amPerhaps you mean "easier to use", but clearly the map file is more useful, since it is more detailed. Granted it takes a little while to get used to, but is worth the effort.Usually, you can check a box in the IDE to generate a map file, then double click it to view in a text editor. A lot easier than fiddling around with Unix command line tools that may not even be available.I guess nm output is useful for Arduino type users who tend to prefer a more dumbed down output.YMMV!

author: fenugrec
author_rank: Frequent Contributor
timestamp: March 08, 2018, 03:07:21 pm
content: Quote from: evb149 on March 04, 2018, 08:24:57 pmI don't think this has been mentioned here yet, but gcc can produce stack usage statistics : adding the-fstack-usageflag produces a .su file for each module.

author: Jeroen3
author_rank: Super Contributor
timestamp: March 08, 2018, 08:03:15 pm
content: The compiler can probably also show the maximum stack depth, called callgraph. That is the deepest function call from main, and each loose entry point (interrupts).This way you can calculate the required stack. You take max depth of main + largest ISR depth (+ fault handler).

author: bson
author_rank: Unknown
timestamp: March 13, 2018, 10:45:29 pm
content: It's better to set up the MSP for interrupts and use the PSP for tasks/threads. This way the only overhead on the PSP is for the exception state (R0-R3,R12,LR,PC,PSR) and context switches (R4-R11 on top of the exc state), not a full nested exception stack - that gets its own stack.Code:// Set up stacks.// The current MSP becomes a thread PSP.// The MSP is set to the start of the interrupt stack.#define StackStrap(INTR_TOS) \asm volatile( \"mov r0, sp;" \"msr psp, r0;" /* PSP = MSP */ \"mrs r0, control;" \"orr r0, r0, #2;" \"msr control, r0;" /* CONTROL |= SPSEL, switch to PSP */ \"msr msp, %0;" /* Set up MSP */ \: : "r"(INTR_TOS) : "r0" )

author: Bassman59
author_rank: Super Contributor
timestamp: March 14, 2018, 04:59:58 pm
content: Quote from: mikeselectricstuff on March 07, 2018, 04:31:26 pmUsually those "idiotic ideas" come from discussions with customers, and when the engineer says, "Yes, we can implement that feature in our existing platform," well, that's a win for everyone.

author: Bruce Abbott
author_rank: Frequent Contributor
timestamp: March 14, 2018, 06:51:51 pm
content: Quote from: bson on March 13, 2018, 10:45:29 pmWhy is this better, and how does it reduce RAM usage?

author: Jeroen3
author_rank: Super Contributor
timestamp: March 14, 2018, 07:59:26 pm
content: It has no benefit without operating system. It's to prevent a process from creating a stack overflow that would affect the operating system, which runs in the interrupts.

author: bson
author_rank: Unknown
timestamp: March 21, 2018, 03:13:24 am
content: Quote from: Bruce Abbott on March 14, 2018, 06:51:51 pmIf you have ten threads, each with its own stack and you need 1k of interrupt stack, then you need 10k of stack space because all ten threads need to be able to process interrupts on their stack. If you have a single interrupt stack you only need 1k for it.