EEPROM emulation – STM32F4 flash vs external flash chip - Page 1
Forum section: electronics / general

author: 3dgeo
author_rank: Frequent Contributor
timestamp: December 15, 2020, 11:35:52 pm
content: Hi,I want to store few KB of settings/presets, due to 128KB external flash block size I was hoping to store presets in MCU flash. My idea is to use predefined memory amount per preset (for example 1KB) and define first byte on it as its ID for indentification while reading presets. If preset gets updated MCU finds new emty space, writes new preset there and sets old preset 1st byte to 0x00 to indicate that it's no longer valid.As far as I know external flash has no trouble writing to the same page without deleting it (writes only "0" – I know how flash works).But after googling I foundthis:"... However, the internal flash memory controller in the STM32's won't allow any writes unless theentire page is cleared."Is this true? STM controler will not allow multiple writes? If so saving to MCU flash goes out of the window, cuz rewriting 16KB for less than 1KB is a waste of flash life.

author: ve7xen
author_rank: Super Contributor
timestamp: December 16, 2020, 12:51:04 am
content: The reference manual for STM32F4xx says (p 86):QuoteSo it seems to explicitly permit this use, even within a single byte.Not sure what the ST EEPROM emulation library does, though. If you want this you probably need to implement it yourself.

author: techman-001
author_rank: Frequent Contributor
timestamp: December 16, 2020, 12:51:52 am
content: Quote from: 3dgeo on December 15, 2020, 11:35:52 pmIn my STM32F103 diagnostics program there is a mcu flash memory test which writes 0x55 and 0xAA to all of the 'hidden' 2nd 64KB flash block memory locations. I have to erase that block after writing 0x55 to each location and then testing they are all now 0x55. I do the same for 0xAA.In other words, after a mcu flash erase I can write anything to any flash location but to change the data in that location it must be erased first. The size of the erasure block is dependent upon the flash controller which varies between models. In the case of the STM32F103 the minimum mcu flash erasure block size is 1KB.The STM32L053C8 for instance allows a 128 Byte erasure block, but then the mcu flash is 64KB maximum.The F4 looks like the minimum erase flash block is 2KB.

author: lucazader
author_rank: Regular Contributor
timestamp: December 16, 2020, 12:54:19 am
content: While you can use the internal flash as nvm storage for settings etc, we have found it is often inconvenient for both development and firmware updates to have this store in the system flash, so matter how we partition it.We have recently switched over to adding a small cheap eeprom chip to our boards that require it. Much easier to deal with and critically for us during development and firmware updates the settings dont get wiped everytime the mcu is programmed. Pretty cheap in volume toohttps://www.digikey.com/en/products/detail/stmicroelectronics/M24C04-FMN6P/5402891I only suggest this since you did mention an external flash chip for storing these settings,If cost is a concern to your project then using the internal flash for this is definitely possible.St even have some example code (albeit not for the f4. only for the g0, g4 etc).https://www.st.com/en/embedded-software/x-cube-eeprom.htmlThey also have an application note for F4 processors.https://www.st.com/resource/en/application_note/dm00036065-eeprom-emulation-in-stm32f40xstm32f41x-microcontrollers-stmicroelectronics.pdfHope either of these approaches help you!

author: ataradov
author_rank: Super Contributor
timestamp: December 16, 2020, 01:53:52 am
content: I have written a short appnote on that topichttp://ww1.microchip.com/downloads/en/DeviceDoc/EEPROM_Emulation_for_Flash-Only_Devices_DS90003222A.pdf(nevermind Microchip, it is generic).Practical implementation of this approach is herehttps://github.com/ataradov/open-5012h/blob/master/config.cApproach to EEPROM on flash-based device is a bit different, but actually better than with a real EEPROM in many cases. At least I find it much better.

author: 3dgeo
author_rank: Frequent Contributor
timestamp: December 16, 2020, 12:39:58 pm
content: Apreshate replies,ST lib uses 2 (or more blocks) – it combines one block data with "new" data and writes it to other block and vice versa. It very inaficiant, but I'm coding my own lib anyway. The one thing I can't find/figure out is how to reserve those blocks for this purpose, I'm guessing it's done via IDE settings? I should be fine with coding other stuff.So it's possible to write multiple times to MCU flash after all, I guess I'll have to try both ways and pick the best one.Quote from: techman-001 on December 16, 2020, 12:51:52 amF4 has 16KB to 128KB blocks. I'm guessing you mistaken blocks with pages – I have no idea what size F411 pages are but 2KB seems right – you can only write by page and erace by block. As I've said – I know how flash works.Quote from: lucazader on December 16, 2020, 12:54:19 amWell, I'm only considering two options – internal flash or flash that already exists on the PCB. I can "fit" that EEPROM block between bootloader and actual firmware to avoid collisions. But I still don't have bootloader ready, so I would appreciate tips on how to reserve EEPROM block.Quote from: ataradov on December 16, 2020, 01:53:52 amThanks, will throw an eyeball at it.

author: Tagli
author_rank: Contributor
timestamp: December 16, 2020, 01:20:57 pm
content: Quote from: 3dgeo on December 16, 2020, 12:39:58 pmYou can reserve memory by modifying the linker script. You need to define a new section.You could also define a const array in a source file, but the compiler would probably optimize it out. So I recommend the linker script way.

author: 3dgeo
author_rank: Frequent Contributor
timestamp: December 16, 2020, 07:00:09 pm
content: Quote from: Tagli on December 16, 2020, 01:20:57 pmSo, if I want to reserve Sector 2 and Sector 3:Original:Code:/* Memories definition */MEMORY{RAM (xrw) : ORIGIN = 0x20000000, LENGTH = 128KFLASH (rx) : ORIGIN = 0x8000000, LENGTH = 512K}My mod:Code:/* Memories definition */MEMORY{RAM (xrw) : ORIGIN = 0x20000000, LENGTH = 128KFLASH (rx) : ORIGIN = 0x8000000, LENGTH = 32KFLASH1 (rx) : ORIGIN = 0x8010000, LENGTH = 448K}What other changes I need to make?Memory map:

author: ataradov
author_rank: Super Contributor
timestamp: December 16, 2020, 07:10:24 pm
content: Reserving things in the middle of the flash is pretty much impossible with the GNU linker. Your changes will not do anything, you would also need to change the rest of the script. But again, if you split the flash like this, you would have to manually place the code in different sections.The layout of this device is not ideal for this at all.I think the easiest thing you can do is reserve the first 16 KB for the vector table and some fixed code like reset handler and interrupt vectors, then allocate sectors 1 ans 2 for the EEPROM and then place the rest of the code.

author: Jeroen3
author_rank: Super Contributor
timestamp: December 16, 2020, 07:16:39 pm
content: You should also make a section for it.Code:/* Memories definition */MEMORY{RAM (xrw) : ORIGIN = 0x20000000, LENGTH = 128KFLASH (rx) : ORIGIN = 0x8000000, LENGTH = 512K - LENGTH(PRESET)PRESET (r) : ORIGIN = ORIGIN(FLASH)+LENGTH(FLASH), LENGTH = erase size}SECTIONS{/* put const from presets.c in PRESET flash sector */.presets :{KEEP(*(.presets))} >PRESET}@ataradov, why do you say it is impossible? I've just done it in a project. Or did I mis something important?replaced 32k for erase size

author: ataradov
author_rank: Super Contributor
timestamp: December 16, 2020, 07:21:23 pm
content: There is no point in reserving 32K at the end if the flash, since erase operation will erase 128K.This is for EEPROM emulation, not a static section.

author: Jeroen3
author_rank: Super Contributor
timestamp: December 16, 2020, 07:45:50 pm
content: True. 32k makes no sense in a 407.

author: Tagli
author_rank: Contributor
timestamp: December 16, 2020, 07:54:20 pm
content: I think we can forget about Sector 0, as we need it for the vector table. Sectors 1, 2 and 3 look promising. We can move .text section so it can start from Sector 4.BTW, is it possible to have more than one sections for .text? I mean, more than one output section for a single input section? Or does it need to be continuous?

author: 3dgeo
author_rank: Frequent Contributor
timestamp: December 16, 2020, 07:54:41 pm
content: Well, with working bootloader I think I can acheave this by using Sectos 0 and 1 for bootloader, skipping two sectors and burning firmware to Sector 4 and up – from my tests that seems straight forward.But as I have 0 knowledge on linker file I think I skipp emulating EEPROM till I have a bootloader. I'll use external flash till then.Don't get me wrong, I would be thankful if someone walk me true setting up linker file, but I realise that this is a huge thing to ask.

author: ataradov
author_rank: Super Contributor
timestamp: December 16, 2020, 08:04:28 pm
content: Yes, if there is a bootloader, then it makes sense to allocate 16K for the bootloader, then 32K for EEPROM, then the rest for the application.With GNU linker it is impossible to have split sections.

author: ataradov
author_rank: Super Contributor
timestamp: December 16, 2020, 08:50:45 pm
content: And for the bootloader modification, it is trivial. Just do this:Code:MEMORY{RAM (xrw) : ORIGIN = 0x20000000, LENGTH = 128KFLASH (rx) : ORIGIN = 0x8000000 + 48K, LENGTH = 512K - 48K}That's it, you don't need to do anything about about that 32K chunk, just use it directly from the EEPROM code. This linker script guarantees it will be empty.

author: techman-001
author_rank: Frequent Contributor
timestamp: December 17, 2020, 07:43:03 am
content: I'm a Forth user (technician not programmer) and have just done some tests on my F4 Discovery board (STM32F407) to see if I can overwrite a flash data with 00.Big FAIL.I isolated block 11 from the system for this test.Code:Sector 11 0x080E0000 - 0x080FFFFF 128 KbytesI was able to save 0xAA to 0x080F0000 without problems BUT the flash controller complained when I tried to save 00 to it directly afterwards leaving 0x080F0000 set to AA.I then tried saving 0xFF to 0x080F0000 and the flash controller didn't complain, but it didn't change the value either, which remains at 0xAA.The only way I could change 0x080F0000 to 00 was by erasing Sector 11.It's looking more and more sensible to me to use an external memory, perhaps something like a FM25CL64B-DG SPI FRAM for NV data or configs.

author: ataradov
author_rank: Super Contributor
timestamp: December 17, 2020, 07:46:06 am
content: The reason for not being able to do multiple writes into the same word (or more often 64 or 128 bit chunk) is ECC on the flash. It is impossible to "update" ECC bits with your new write. This is not fail, it is the result of higher memory densities. Most new high performance MCUs will have this.Again, this is not a real problem if you do things correctly.

author: techman-001
author_rank: Frequent Contributor
timestamp: December 17, 2020, 08:10:57 am
content: Quote from: ataradov on December 17, 2020, 07:46:06 amAre you saying that ECC needs to be disabled to be able to overwrite a MCU flash location ?I see no "ECC" bitfields in the reference manual for the STM32F407, only for the Flexible static memory controller (FSMC), so it appears that ECC for the mcu flash cannot be disabled ?Are you able to elaborate on the correct way to do this for the readers of this thread ?

author: ataradov
author_rank: Super Contributor
timestamp: December 17, 2020, 08:17:06 am
content: ECC often can not be disabled, and often is not even exposed to the user (except in strange limitations of programming ).But after checking the datasheet it appears this device does not have ECC. In fact the reference manual is explicit on this: "Successive write operations are possible without the need of an erase operation when changing bits from ‘1’ to ‘0’. Writing ‘1’ requires a Flash memory erase operation."So how exactly did the flash controller complain? STMs are very verbose with their error codes on flash operations.Oh, wait you have 407. I was looking in the manual for the original device. Ok, the same exact text is present in the 407 RM.

author: Nominal Animal
author_rank: Super Contributor
timestamp: December 17, 2020, 08:03:53 pm
content: Quote from: ataradov on December 16, 2020, 01:53:52 amVery interesting! I've suggested something very similar for storing data on microSD media without a filesystem, using a binary search to find the latest counter in logarithmic time – assuming the media is cleared to all zeros or all ones before first use. (If you use a 8 GB microSD with 512 byte sectors, you got 16M sectors, so a linear search is not feasible. If there are no invalid sectors, you need to read at most 27 counters (if I counted right) to find the latest one; the only "trick" in implementing the search is to handle both partially used and fully used cases by looking for the discontinuity.) With 512 byte sectors and a 64-bit counter (avoiding all longevity limits), you still have 504 bytes per sector for user data.

author: Jeroen3
author_rank: Super Contributor
timestamp: December 17, 2020, 08:17:23 pm
content: Quote from: ataradov on December 17, 2020, 07:46:06 amAh, of course. Why didn't I think of that when I tried thisThey often don't even talk about this in the manual. But they don't need to really.So you can't overwrite, only append.

author: techman-001
author_rank: Frequent Contributor
timestamp: December 17, 2020, 11:56:58 pm
content: Quote from: Jeroen3 on December 17, 2020, 08:17:23 pmI does look like only the STM32 L, G series and the STM303F actually have ECC flash onboard.https://en.wikipedia.org/wiki/STM32There is no mention of onboard ECC for the STM32F407 (OP's mcu) in the Ref Manual or the SVD's.

author: ataradov
author_rank: Super Contributor
timestamp: December 17, 2020, 11:58:47 pm
content: Yes, reference manual for OPs chip allows any writes which set 1 -> 0.

author: techman-001
author_rank: Frequent Contributor
timestamp: December 18, 2020, 12:57:50 am
content: Quote from: ataradov on December 17, 2020, 08:17:06 amA excellent question, I don't actually know right now as I'm waiting for feedback from the architect of the Forth system (mecrisp-Stellaris) I use.I grepped the Mecrisp-Stellaris source and it seems the error came from the Forth Kernel, not the F407 Flash_SR register which has the following Bitfields for flash issues.: Flash_SR_EOP ( -- x addr ) 5 bit Flash_SR ; \ Flash_SR_EOP, End of operation: Flash_SR_WRPRT ( -- x addr ) 4 bit Flash_SR ; \ Flash_SR_WRPRT, Write protection error: Flash_SR_PGERR ( -- x addr ) 2 bit Flash_SR ; \ Flash_SR_PGERR, Programming error: Flash_SR_BSY ( -- x addr ) 0 bit Flash_SR ; \ Flash_SR_BSY, BusyI expected that the Flash_SR_PGERR was the issue but I can't see it being raised ( or any of the others) or find it in the source, but then the source is mostly in German and not exactly CMSIS-SVD compliant, so I'll have to wait for advice from the architect.

author: techman-001
author_rank: Frequent Contributor
timestamp: December 18, 2020, 09:03:28 am
content: Quote from: techman-001 on December 18, 2020, 12:57:50 amThe architect pointed out that his assembly code had protection against altering flash and once this code was removed everything worked as expected from the User Manual.Code:Sector 11 0x080E 0000 - 0x080F FFFF = 128 KbytesCode:11 eraseflashsector Erase sector 11ok.read_$F0000 000000FF ok.flash_0xAA->$F0000 ok.read_$F0000 000000AA ok.flash_0x00->$F0000 ok.read_$F0000 00000000 ok.

author: techman-001
author_rank: Frequent Contributor
timestamp: December 18, 2020, 09:16:19 am
content: I want to store few KB of settings/presets, due to 128KB external flash block size I was hoping to store presets in MCU (STM32F407) flash. My idea is to use predefined memory amount per preset (for example 1KB) and define first byte on it as its ID for indentification while reading presets. If preset gets updated MCU finds new emty space, writes new preset there and sets old preset 1st byte to 0x00 to indicate that it's no longer valid.Summary:Yes, this can be done, a single flash location can have the data reflashed with "0"'s, (but not "1"'s).Minimum erasure size is the sector involved. My test used Sector 11.NOTE: the MCU (STM32F407) doesnot have ECC flash.Code:Sector 11 0x080E 0000 - 0x080F FFFF = 128 Kbytes11 eraseflashsector Erase sector 11 ok.Data from a STM32F407 Discovery Board running Mecrisp-Stellaris Forth showing the above.Code:read_$F0000 000000FF ok.flash_0xAA->$F0000 ok.read_$F0000 000000AA ok.flash_0x00->$F0000 ok.read_$F0000 00000000 ok.

author: ataradov
author_rank: Super Contributor
timestamp: December 18, 2020, 09:19:56 am
content: The only danger here with one sector is that power may go away while you are erasing and you will lose your settings. You need at least two sectors for reliable operation.

author: techman-001
author_rank: Frequent Contributor
timestamp: December 18, 2020, 10:14:19 am
content: Quote from: ataradov on December 18, 2020, 09:19:56 amIf a hobbyist is willing to forgo that professional level of data security, a 64KB logger could be built using the second "hidden" flash block in a "blue-pill" with the STM32F103C8 MCU found in most units.That would leave 64KB for the main program, which is tons for C and Forth.

author: S. Petrukhin
author_rank: Super Contributor
timestamp: December 20, 2020, 03:21:39 am
content: The logic of flash is that it can't accept data using MOV (like eeprom), it accepts data as AND. I.e. you can write 0 to each bit of flash, but you can't write 1. To write 1, you need to erase the entire sector at the same time - erased flash contains 1 in all bits.When I often needed to write just one byte, it seemed unnecessary to supplement the eeprom, often erase flash-a hole will be wiped. I organized a chain of data cells. The first byte was a signature with the values: FFh-cell is free, 0Fh-cell contains current data, 00h-cell is canceled. So by finding a cell with the signature 0Fh, I could find the actual data. To write new actual data, I wrote the signature 00h to the current cell, and the signature 0Fh to the next one. When the sector ended, I erased it and used a new circle.Code:constcEmptyFlashCell = 0xFF;cActualFlashCell = 0x0F;cCanceledFlashCell = 0x00;cEmptyFlashValue = 0;typetFlashCell = recordSingnatute : byte;Value : byte;end;constcLowFlashData = 1;cHighFlashData = cSectorSize div SizeOf(tFlashCell);typetFlashData = array[cLowFlashData..cHighFlashData] of tFlashCell;varFlashData : tFlashData absolute cFlashEnd-cFlashSectorSize;funtion GetFlashCurrPos:dword;var i: dword;begini:=cLowFlashData;while (i<=cHighFlashData) and (FlashData.Signature<>cActualFlashCell ) do Inc(i);if i>cHighFlashData then i:=cLowFlashData;Result:=i;end;function GetFromFlashValue:byte;var t: dword;begint:=GetFlashCurPos;if FlashData.Signature=cEmptyFlashCell then beginResult:=cEmptyFlashValue;and else beginResult:=FlashData.Value;end;end;procedure SetToFlashValue(aValue:byte);var t: dword;begint:=GetFlashCurPos;if t=cHighFlashData then beginEraseFlashSector;t:=cLowFlashData;end else beginif FlashData.Signature<>cEmptyFlashCell then beginSaveToFlash(t,cCanceledFlashCell,0);Inc(t);end;end;SaveToFlash(t,cActualFlashCell,aValue);end;