What is the best way to restore the last memory address written before shutdown - Page 1
Forum section: electronics / beginners

author: syntax333
author_rank: Regular Contributor
timestamp: September 25, 2020, 04:58:06 pm
content: Hi, I am working on a project where I need to collect ADC data and store the data to an external FLASH.I am using STM32F405RG with MT25LQ01GB FLASH. The problem is if the board loses for a brief moment and STM32F405 resets, I want it to continue to store data where it left off.For example:STM32 writes 0x0000 0000 addressSTM32 writes 0x0000 0001 addressSTM32 writes 0x0000 0002 address/*Power is gone for brief moment and STM32 ResetsSTM32 writes 0x0000 0000 addressSTM32 writes 0x0000 0001 addressSTM32 writes 0x0000 0002 addressSTM32 writes 0x0000 0003 addressI don't want the above example to happen.I want STM32 to continue from 0x0000 0002 address after Power is switched.What is the best way to restore the last memory address written before shutdown?

author: atmfjstc
author_rank: Regular Contributor
timestamp: September 25, 2020, 05:06:19 pm
content: One simple idea would to pre-format the flash with some special value, e.g. 0xffffffff, 0xdeadbeef etc. and, on boot, quickly scan the flash for the first address where the value was not filled with some proper data, and start writing from there.

author: syntax333
author_rank: Regular Contributor
timestamp: September 25, 2020, 05:58:11 pm
content: What is the best way to scan the FLASH because its 1Gb FLASH. It will take too much time I think

author: voltsandjolts
author_rank: Unknown
timestamp: September 25, 2020, 06:10:31 pm
content: If you are writing from the beginning of memory, then a binary search should be fairly quick to find the end of data.Be aware that if you are writing directly to the NAND FLASH pages (i.e. no filesystem, like LittleFS for example) then you either have to deal with bad memory pages/blocks or accept the possibility of some lost data.

author: iMo
author_rank: Super Contributor
timestamp: September 25, 2020, 06:15:36 pm
content: Write the last good address to an external i2c or spi fram/mram.. (ie. you write the "data item" to the flash at the "address" and always write the "address" to a nonvolatile location "last good address" as well).Or, an mcu of that type usually has got a few battery backup-ed registers handy - write the last good address there..

author: MosherIV
author_rank: Super Contributor
timestamp: September 25, 2020, 06:28:10 pm
content: Use a simple FAT table.By that I mean have a table in 1 sector/block of flash that represents that last block/sector written to.At power up, the table is read to find the last block/sector.Then scan through the sector for unused location.Flash erases to '1'. You can write to the same address over and over. 1s can be changed to 0 but 0 cannot be changed to 1.So you can use simple bits to represent each block in sectors.

author: langwadt
author_rank: Super Contributor
timestamp: September 25, 2020, 06:30:05 pm
content: Quote from: atmfjstc on September 25, 2020, 05:06:19 pmwhich is simple because you can only write zeros to flash, an erased flash is all Fs

author: DrG
author_rank: Super Contributor
timestamp: September 25, 2020, 06:39:33 pm
content: One possibility, which is an extension of what @imo said is to use EERAM e.g.,http://ww1.microchip.com/downloads/en/DeviceDoc/20005371C.pdfThese are I2C serial EEPROMS but they act like RAM on your end and the transfer to EEPROM is done internally.They are *very* easy to use as a simple cap takes care of the problem of keeping power long enough after a loss to finish a write to the chip. You could simply use two locations (bit width is up to you) ...STM32 writes 0000 to EERAM location #1 pre-writeSTM32 writes 0x0000 0000 addressSTM32 writes 0000 to EERAM location #2 post-writeSTM32 writes 0001 to EERAM location #1 pre-writeSTM32 writes 0x0000 0001 addressSTM32 writes 0000 to EERAM location #2 post-write/*Power is gone for brief moment and STM32 Resets// code checks the contents of EERAM. If #1 and #2 are the same address, that is the last address successfully written// If 1<>2 then 2 is the last address successfully written to AND power failed during the write to address #1 (may or may not be written)Maybe over-kill, and adds a buck but I thought they were pretty neat devices and they only write to EEPROM when there is a power down so they should last a long time.

author: nuclearcat
author_rank: Unknown
timestamp: September 25, 2020, 07:34:07 pm
content: Use large enough capacitor or battery on VBAT and RTC memory on STM to write current index where you write data. It will stay alive across reboot, if properly configured.

author: syntax333
author_rank: Regular Contributor
timestamp: September 25, 2020, 07:40:36 pm
content: Quote from: voltsandjolts on September 25, 2020, 06:10:31 pmDo I have to sort the data? I only used binary search for finding specific value. Its accaptable for my application to lose data in case of Power reset.Quote from: imo on September 25, 2020, 06:15:36 pmProblem with using external fram/mram is I am not allowed to modifiy or add anything else to PCB so I have to find a solution without adding components.I checked the datasheet of STM32F4 for battery backup-ed registers but in order to work those I need to add another battery I think, which is not allowed.Quote from: MosherIV on September 25, 2020, 06:28:10 pmErasing bytes takes too much time like order of ms which my application cannot afford unfortunately.

author: ledtester
author_rank: Super Contributor
timestamp: September 25, 2020, 07:46:56 pm
content: QuoteJust erase the pages ahead of time. Those flash chips can erase entire sectors with a single command.

author: atmfjstc
author_rank: Regular Contributor
timestamp: September 25, 2020, 08:20:59 pm
content: Quote from: syntax333 on September 25, 2020, 07:40:36 pmThe data is already sorted from the point of view of the thing we are looking for, i.e. only the fact that it is "blank" or not: all the non-blank values are at the beginning of the memory, followed by only blank values all the way to the end.Start in the middle of the memory, at 1GB / 2 = 512MB. If the value there is blank, then the first blank address is somewhere in the lower half of the memory, and you repeat the search from there. Otherwise it is in the upper half of the memory, etc.TBH this seems to me a lot simpler than any other solution requiring extra metadata.

author: ejeffrey
author_rank: Super Contributor
timestamp: September 25, 2020, 09:14:43 pm
content: Do you write the flash once through beginning to end or do you have a circular buffer where you need to wrap around when you get to the end of flash?

author: syntax333
author_rank: Regular Contributor
timestamp: September 25, 2020, 09:47:43 pm
content: Quote from: ejeffrey on September 25, 2020, 09:14:43 pmI scan ADC channels then write to FLASH, scan channels write to FLASH so on..When FLASH is full I stop.

author: JustMeHere
author_rank: Frequent Contributor
timestamp: September 26, 2020, 12:57:52 am
content: If you're using an external RTC chip like the MAX31343, there's some battery backed RAM on the chip that could solve your problem.

author: ledtester
author_rank: Super Contributor
timestamp: September 26, 2020, 02:29:15 am
content: Another non-volatile memory solution:https://www.adafruit.com/product/1895

author: Siwastaja
author_rank: Super Contributor
timestamp: September 26, 2020, 07:23:35 am
content: Supercap or a tiny coin cell on Vbat, use backup SRAM, done.

author: MosherIV
author_rank: Super Contributor
timestamp: September 26, 2020, 08:17:16 am
content: QuoteI do not think you have understood.Use 1 block or sector to note which blocks and sectors have been used. This is your 'fat' table.You map each block in flash to a but or byte or word.As you use each block, you mark in the fat table that block is used. Use a 1 for unused and 0 for used.You just write 0 for that block or sector.Depending on the ratio of bytes in a block/sector, you can used bits or bytes, 16bit words or 32bit words to represent each blockSo as you fill up the flash, more and more 0s are written.You only need to erase this 'fat table' when you want to start writing from the beginning again.

author: AndyC_772
author_rank: Super Contributor
timestamp: September 26, 2020, 03:49:22 pm
content: Do you need to assure that all values written are valid? Or just start writing again from where you left off, and it's OK if the value that was being written when power failed is corrupted?If the entire flash space you're using is initialised to FF, all you need to do is a binary search to find the first address which still contains FF. That only requires one read/compare for each address bit, so it's a very fast operation. (Start by reading the data that's halfway through the memory space, then depending on whether it's used or not, look at the data 1/4 or 3/4 through the space and so on - you discover one bit of the address per comparison).If you need to detect bad writes, then it's a bit more involved. Consider dividing your flash into blocks, each of which contains one flag that's set to zero before the data is written, and another which is cleared afterwards; any block in which both flags are clear is used and valid, if they're both set then the block is empty and available for use, and if only the first flag is clear then the data should be treated as invalid.More elaborate schemes can also include CRC checks, sequence numbers and the like.

author: DrG
author_rank: Super Contributor
timestamp: September 26, 2020, 04:27:07 pm
content: I have been following this thread and, after hearing more from the OP, I am seeing a problem.First off, not stated originally, but added is that the OP can not use any additional chips - so no FRAM, no EERAM, no external RTC chips...he can't even add a battery to use battery backed up SRAM on the chip!I admire the FAT type suggestion, but it seems a bit pointless in one sense...he is reading analog values and writing them out until the space is filled. If he loses power during the write, continuity of the readings are lost and he has no idea about the temporal parameters for what was written, regardless of his coded write intervals.One of the issues with this kind of thread is that some qualifications come in later - ok, no harm, no foul. But the sooner ALL of the requirements come out, the better.Seems like he might need to start over in case of a power outage regardless, because I would think one would want a meaningful stream, with respect to time - no? IOW, it could be that the first 50 bytes were taking a sec apart, the rest of them started at some indeterminate time later but are also a sec part, unless there were more power outages. Or is power loss recovery part of the assignment/requirement?If that is true, then some kind of FAT would serve at the least, as a power outage detection. But, so would writing a single byte before starting and re-writing the byte when finishing. Boot up would check that byte to determine if a power outage had occurred and allow a time interval-accurate stream to be written.

author: ledtester
author_rank: Super Contributor
timestamp: September 26, 2020, 04:53:42 pm
content: QuoteCould you clarify what you mean by "starting" and "finishing"? How often would this byte be written?If I were doing this project I would either:1) include some sort of timestamp in the recorded data; it could be the number of "ticks" since power-up or even just a serial number starting at 0 if a real-time clock wasn't available2) record the data in records which a special record indicating that the system started up

author: DrG
author_rank: Super Contributor
timestamp: September 26, 2020, 05:00:00 pm
content: Quote from: ledtester on September 26, 2020, 04:53:42 pmYeah, lots of possibilities. If he doesn't care about endurance, the first 4 bytes could simply act as the memory counter - of course this would more than double the write time.