EEPROM wear levelling techniques - Page 1
Forum section: electronics / general

author: HwAoRrDk
author_rank: Super Contributor
timestamp: October 07, 2022, 11:39:17 am
content: I'm working on a project where I'm storing some persistent settings in my microcontroller's built-in EEPROM. Because the settings will be written every time the user changes something, which could be several times a day, I've decided to try and employ some wear levelling techniques so that the EEPROM doesn't wear out too quickly. The EEPROM in my microcontroller is rated for 10k write/erase cycles with 20 year retention, and 100k cycles with 1 year retention.The primary technique I've decided to use is the common one of treating the entire EEPROM area as a cyclical array of entries, and each time the settings are written, it does so at the next entry to the last used one, wrapping around appropriately. In order to determine which entry is the latest, each entry has a monotonically incrementing sequence number, and the latest is whichever one has the greatest number (determined by scanning the entire EEPROM). I can store 16 entries, so that multiplies the most conservative rated cycle count to 160,000 writes. If the settings are written 20 times a day, that gives a lifespan of 8,000 days, or nearly 22 years, which should be suitable for my needs.One other thing I am doing is that because the datasheet states that the granularity of the EEPROM is 4 bytes ("cycling is performed on 4 bytes even when a write/erase operation addresses a single byte"), I have made the sequence number a uint32_t, and the settings struct is padded to be a multiple of 4 bytes in size.My first query is with the sequence number. Should I be concerned with roll-over? Now, a maximum possible value of 4,294,967,296 is capability fora lotof writes, and I'm confident that the EEPROM will have expired long before this sequence value ever gets near rolling over, but maybe there's something I'm not thinking of...The other question I have is whether it is worth doing something like inverting (i.e. taking one's compliment of) the sequence number when writing/reading so that, for a long while at least, the majority of the bits in the sequence value are 1s rather than 0s. That is, if the sequence number is 2000 it gets written as 0xFFFFF82F rather than 0x000007D0. As I understand things, because the natural default value of an EEPROM is 0xFF (all 1s), when you write a value only the 0s perform a wear-inducing operation, and writing of 1s is essentially a 'no-op'. And so always writing the zero-value MSBs will wear things out quicker. Am I right? Should I do this? Or is it pointless, because the LSBs of the sequence numbers will still wear out first, and if they are corrupted, then it doesn't matter if the MSBs are still fine?Also, is there any potential in using Gray code for the sequence number? IIRC, counting up with Gray code only ever changes one bit per increment.Anyone got any other advice or insights?

author: Psi
author_rank: Super Contributor
timestamp: October 07, 2022, 12:03:00 pm
content: Quote from: HwAoRrDk on October 07, 2022, 11:39:17 amDo you really need to store the sequence number?Can you write them in order and then just scan for the "latest" one by its position when you do a read?You'd need two blocks of eeprom for this approach, when one block is full you scan and copy all the "latest" entries into the other block (which becomes the new primary) and also erase the old block so it can be used later

author: newbrain
author_rank: Super Contributor
timestamp: October 07, 2022, 01:29:09 pm
content: I have implemented something similar, but geared to the handling of a number (15+1) of presets.Each preset has a fixed area in the EEPROM, and in each area wear leveling is done writing sequentially in 16 different slots.Due to space constraints, I use an 8 bit sequence number.Sequence numbers are checked at power-on, the highest is used as the current value, and the lowest to decide where to write next.To avoid rollover confusion there is some conditional comparisons: if the difference between the max and the min is greater than 16, then a rollover has occurred, and the highest of the lower numbers is selected.Presets can be manually loaded or stored by the user, with the exception of Preset 0, which is auto-saved 10 seconds after the last change to the settings.The delay avoids a lot of useless writes if the user is tuning something or selecting some mode in sequence.Each slot is CRC protected, to discard corrupted slots (e.g. if power went down while writing) when searching for the latest - they are rewritten at power-on with the latest values or with a default preset if all slots are corrupted.Quote from: Psi on October 07, 2022, 12:03:00 pmThat's somewhat wasteful - in this way you always have an empty block, effectively halving the storage, which is in any case written twice (when the values were saved, and then at erasure) so double the write cycles per each position.It's a bit simpler to implement, though.

author: rteodor
author_rank: Frequent Contributor
timestamp: October 07, 2022, 01:49:14 pm
content: Quote from: HwAoRrDkThis is correct for the single cell EEPROM's (vast majority of them). There are also differential cells with insane long retention duration that read a random value when they are in the erased state.Quote from: HwAoRrDkFrom what I'we worked with EEL (EEPROM Emulation Layer) there was no concern with leveling of individual bits.Much effort was put into assuring the consistency of data, i.e.: if a reset occurs during a write, make sure that that data block is detected as corrupted. This was done setting a variable at the start of a block to 0x55555555 after all writing was done.EEPROM's contain also a charge pump that raises voltages for writing/erasing. This circuit can sometimes fail a lot faster than individual bits.And there is read/write disturbance as well. That is one more reason to roll over the entire EEPROM in blocks.Quote from: newbrainThis makes a lot of sense in small apllications. One way is to keep a RAM image and only transfer that to flash in certain conditions: delay timer (as you said) or in an hourly/daily/weekly update.But I would keep a dual copy in flash and update them alternately.

author: AndyBeez
author_rank: Frequent Contributor
timestamp: October 07, 2022, 02:16:23 pm
content: NAND or NOR EEPROM?In the past on a NOR EEPROM I did 'bump and zero' for a frequently altering value: Byte n was set to 'nil' (FF) and byte n+1 had the new value. Retrieval was simply reading forward from address 1 to find the valid byte. When the write address pointer exceeded the boundary, the pointer wrapped back to 1. EEPROM byte 0 was never used, which was a recommendation in the datasheet. Obviously FF was not a valid number to store, otherwise the array would iterate forever!

author: HwAoRrDk
author_rank: Super Contributor
timestamp: October 07, 2022, 02:21:26 pm
content: Quote from: Psi on October 07, 2022, 12:03:00 pmHow exactly do I determine which one is the latest by position? It's cyclical, so the latest settings data might, for example, be in position 3, and positions 4-16 are old redundant data.Quote from: Psi on October 07, 2022, 12:03:00 pmThat seems to me like it doesn't help. You have two blocks so you're only multiplying the write endurance by 2, but then halving it again by erasing the block when you stop using it. Unless I'm misunderstanding something, because I don't know what you mean by "one block is full".Quote from: newbrain on October 07, 2022, 01:29:09 pmAh, that's a good idea. I was wondering whether I could use a smaller sequence number to have more entries and thus multiply the endurance further, but then found out about the write/erase granularity of 4 bytes, so figured I might as well keep with the uint32_t sequence number.Quote from: newbrain on October 07, 2022, 01:29:09 pmThat's a good point, I think I might add a CRC and use up some of the padding space in my struct. A CRC-8 should be sufficient for a few bytes of data, right?Quote from: AndyBeez on October 07, 2022, 02:16:23 pmI don't know. It's only described in the datasheet as "true data EEPROM" - meaning that it's not flash emulating EEPROM.I just realised that if I want to use an inverted sequence number, that's just a decrementing count from 0xFFFFFFFF!Then the logic to find the latest settings data is instead look for the smallest value, and I don't need to special-case ignoring sequence values of UINT32_MAX when scanning through (for handling case of blank EEPROM).

author: newbrain
author_rank: Super Contributor
timestamp: October 07, 2022, 02:59:44 pm
content: Quote from: HwAoRrDk on October 07, 2022, 02:21:26 pmCRC-8 is a bit meager, I would think (I remember it was used in ATM, but not on the full 53 octets packet, only on the first part of the header).I used a CRC-16 inside a header that also contains the sequence counter and a version byte.Code:typedef struct Header_{uint16_t crc16; /* 0 CRC-16/IBM of whole block */uint8_t version; /* 2 Version of this block */uint8_t counter; /* 3 Sequential counter to pick latest */} Header; /* 4 Total header size */The version byte is there to make an automatic conversion if I update the settings data structure in an incompatible way (e.g. adding fields, changing their size or rearranging them) - or, at the very least, not to load invalid data.

author: peter-h
author_rank: Super Contributor
timestamp: October 07, 2022, 04:09:55 pm
content: There are lots of ways to store data in a "rolling manner". Watch out for adjacent cell disturbance though; this can significantly reduce the apparent endurance.Also consider other ways e.g.Store the data in a RAM, and upon power loss save the data in the FLASH, into a dedicated page which has been pre-erased, so the (Adesto SPI FLASH case) 15ms programming time reduces to just 3ms which is easily achieved, without big capacitors and using a cheap VCC threshold detectorhttps://www.eevblog.com/forum/microcontrollers/ideas-for-storing-frequently-updated-data-in-a-flash-chip/msg3913820/#msg3913820https://www.eevblog.com/forum/microcontrollers/flash-memory-write-without-pre-erase/msg4030507/#msg4030507this shows you how in more detailhttps://www.eevblog.com/forum/microcontrollers/how-fast-does-st-32f417-enter-standby-mode/msg4062652/#msg4062652

author: HwAoRrDk
author_rank: Super Contributor
timestamp: October 07, 2022, 04:24:10 pm
content: Quote from: newbrain on October 07, 2022, 02:59:44 pmI think CRC-8 should be good for my needs. According toWikipedia, the AUTOSAR variant is good for detecting up to 4 bit errors for payloads of up to 119 bits (~14 bytes), which covers enough payload data for me.But yes, I think your observation holds in some cases - some CRC-8 implementations are only good forverylimited quantities of payload data. I recall now reading a paper about that ages ago, IIRC the Koopman one.Quote from: newbrain on October 07, 2022, 02:59:44 pmAnother good idea.Not sure if I will add a version number. If I need to change the settings structure, it'll almost certainly be because the hardware has changed, which will be de facto incompatible with previous firmware anyway.

author: Doctorandus_P
author_rank: Super Contributor
timestamp: October 07, 2022, 04:56:43 pm
content: I'd also add an CRC over the whole memory block (for one setting).Also, if your settings are expected to change multiple times a day, then you don't need years of retention.This is the first time I've seen two different retention time values quoted for different number of write cycles. This does make sense as leakage increases with write cycles.What do you do if the CRC fails? Do you go back to defaults, or do you attempt to read older settings (and generate some warning?)Maybe it's an option to just erase old data blocks, so there is only one (or a few) valid block(s) in EEprom. If you do this you also don't need sequence numbers. You can use the end of empty memory as a start for your data.If this is not good enough or you still have doubts, it's getting time to consider other technologies, such as Fram.

author: HwAoRrDk
author_rank: Super Contributor
timestamp: October 07, 2022, 06:08:54 pm
content: Quote from: Doctorandus_P on October 07, 2022, 04:56:43 pmI think there's a mis-interpretation here. When I say "settings", I meantthesettings, singular. I don't have multiple (e.g. a range of presets).And the EEPROM settings will be basically persisting how the user had things set up last time they used it. They may be changing and saving it during usage, but it may be days, weeks, months between occasions they use the device, or using the device regularly but not changing the settings, so I do need long periods of retention.Quote from: Doctorandus_P on October 07, 2022, 04:56:43 pmPerhaps revert to defaults, as it may be that older data is not what the user presently wants. Or perhaps just enter a general device-wide error state. Not sure yet.

author: AndyBeez
author_rank: Frequent Contributor
timestamp: October 07, 2022, 07:28:22 pm
content: How mission critical is your application?An alternate to EEPROM is Ferroelectric-RAM. FeRAM has very high RW endurance and has data retention measured in decades. Power consumption is battery friendly.https://www.electronics-notes.com/articles/electronic_components/semiconductor-ic-memory/fram-ferroelectric-ram-memory.php

author: eutectique
author_rank: Frequent Contributor
timestamp: October 07, 2022, 07:31:41 pm
content: Are you limited to internal EEPROM? For less than 2 quid you can haveFRAMchip with 100 trillion (10^14) read/writes and 151-year data retention.

author: SiliconWizard
author_rank: Super Contributor
timestamp: October 07, 2022, 08:15:00 pm
content: FRAM is great. But it *is* expensive compared to EEPROM. This $2 part is only... 512 *bytes*.At this price you can get a decent 256 *Kbytes* EEPROM.

author: woofy
author_rank: Frequent Contributor
timestamp: October 07, 2022, 08:34:24 pm
content: If you're only saving in a 16 entry set, you don't need 32 bits for the roll-over, a single byte is enough. The rollover, if there is one, is easily detected - its just a 0 and FF in the sequence numbers. So just pick the highest number before the break to the higher numbers. Do include a checksum/crc so you can fallback safely if there was a write failure.

author: Psi
author_rank: Super Contributor
timestamp: October 08, 2022, 01:14:59 am
content: Quote from: newbrain on October 07, 2022, 01:29:09 pmAgreed, it depends what you value more.Lots of advantages and disadvantages.In many cases the additional number of writes you want over a single EEPROM cell is not that much more than that its rated for.If it typically cell has 100,000 writes then a 10x leveling scheme to get 1mil writes is plenty for 90% of applications.Even using a raw cell with no leveling, 100,000 writes gets you 27 writes per day for 10 years.For an AVR with 100,000 cycle life, when tested by writing over and over again you get around 2-5million before actually seeing a failure.Of course it does also depend on how long there is between write and read and doing a write loop test doesn't take that into account.

author: SiliconWizard
author_rank: Super Contributor
timestamp: October 08, 2022, 01:44:49 am
content: Don't confuse Flash with EEPROM though. Completely different thing and number of write cycles.The ATMEGA series had a chunk of EEPROM (real EEPROM) separate from its Flash. That was also the case for older PICs.Newer MCUs very, very rarely have true EEPROM embedded. The max number of write cycles for EEPROM is one to two orders of magnitude higher than Flash.

author: Psi
author_rank: Super Contributor
timestamp: October 08, 2022, 02:24:02 am
content: Yes, it's good to make that point clear.I just assumed EEPROM because the topic is EEPROM.However some software libs may offer EEPROM support on flash only MCUs and that isn't EEROM so max write cycles is much less, and entire pages need to be erased at once, which makes things more compilated.

author: free_electron
author_rank: Super Contributor
timestamp: October 08, 2022, 07:37:36 pm
content: cyclic writes with tail-end erase.before you can write an eeprom block you need to erase it.so, if you write your structure and erase the NEXT block. you can do two thingsyou spread the data over multiple blocks for wear-leveling , you always have a freshly erased block ready for writing, and you know where the last configuration is (just before the empty block)

author: newbrain
author_rank: Super Contributor
timestamp: October 08, 2022, 11:18:00 pm
content: Quote from: free_electron on October 08, 2022, 07:37:36 pmNot for any EEPROM on my bench.24C32, 24C64, 24C256.That's, in fact, one of the major differences with flash.

author: Psi
author_rank: Super Contributor
timestamp: October 08, 2022, 11:32:09 pm
content: Quote from: newbrain on October 08, 2022, 11:18:00 pmPretty sure he knows that and was just referring to saving a struct and needing to erase all the bytes in that struct sized block before writing it.All EEPROM i've ever used is byte by byte erasable, unlike flash.But maybe there are some that have block erase only ?

author: SiliconWizard
author_rank: Super Contributor
timestamp: October 08, 2022, 11:56:14 pm
content: Usually writing byte by byte without erasing a whole page on EEPROM is much slower than erasing the page first and writing it after that. But yes, it is at least possible. That's something to be aware of though if write time is a concern.

author: Nominal Animal
author_rank: Super Contributor
timestamp: October 09, 2022, 05:26:21 pm
content: For rolling (monotonically increasing)N-bit sequence numbersaandb,if ((intN_t)((uintN_t)(a) - (uintN_t)(b))OP0) {/*aOPb, but taking possible rollover into account */}whereOPis one of<,<=,==,>=,>, or!=.This works, because in CuintN_tarithmetic is modular, andintN_tuses twos complement.In essence, it moves and wrapsaandbso thatbis at zero, and checks whereathen is relative tob, positive or negative. TheuintN_tcasts are not needed ifaandbare declared as that type; but theintN_tcast should not be omitted.For example, consideruint8_ta= 40anduint8_tb= 200.((int8_t)(a-b)) = 96, so in terms of the monotonic sequence,a=b+ 96.If we haveuint8_ta= 200anduint8_tb= 40, then((int8_t)(a-b)) = -96, so in terms of the monotonic sequence,a=b- 96.If you have a cyclical buffer withLentries, each with a sequentially assigned unsigned integer (of sizeNbits,2N>L+1), with no holes or missing values, you can do a binary search to discover the latest (followed by oldest or unused) entry, as long as unused entries are assigned a special integer (either0or2N-1) and are all contiguous (or in two parts at the beginning and end). In general, you need to examine about⌈log2(L+M)⌉ + 2entries, whereMis the number of unused entries. It is quite efficient, even when you do this for e.g. SD card sectors, without a filesystem.

author: fchk
author_rank: Frequent Contributor
timestamp: October 09, 2022, 05:44:39 pm
content: Have you seen the new Microchip EERAM chips?https://www.microchip.com/en-us/products/memory/serial-eeramThese are serial SRAM memory with automaic backup and restore to eeprom cells. These chips need a ceramic capacitor (22...68u) as shutdown energy storage.This eliminates wearout issues completely.fchk

author: peter-h
author_rank: Super Contributor
timestamp: October 10, 2022, 09:04:22 am
content: As would the software solution I suggested, and without dependence on some weird single sourced part whichwillone day bite you in the bumAs an extreme example, I can erase and program the bottom 32k of my 32F417 FLASH in300ms, and with a decent size capacitor you could achieve that, too.

author: free_electron
author_rank: Super Contributor
timestamp: October 10, 2022, 02:05:10 pm
content: Quote from: newbrain on October 08, 2022, 11:18:00 pmthe OP is not talking about external eeprom but on-chip eeprom.example : built-in eeprom in one of the "popular" atmel processors :"The EEPROM Programming mode bit setting defines which programming action that will be triggered whenwriting EEPE. It is possible to program data in one atomic operation (erase the old value and program the newvalue) or to split the Erase and Write operations in two different operations."Many processors (non-atmel) have such ability.the I2c or spi eeproms all have self timed erase/write cycles as there is no "command register"

author: NorthGuy
author_rank: Super Contributor
timestamp: October 13, 2022, 12:23:07 am
content: You don't need a 32-bit number. Just one bit somewhere within the data record is sufficient.When you write, flip the bit when the address rolls over.To find the last element, scan from the beginning and find the last element which has the same bit as the element at the 0 address.

author: DavidAlfa
author_rank: Super Contributor
timestamp: October 14, 2022, 06:31:45 pm
content: I hope someone didn't mention this already (I barely read the thread)If the stored data is a lot smaller than a sector erase size, which usually is, I'd try to avoid erasing blocks as much as possible, as this is what actually reduces lifespan.What I'd do is to divide sector size/data size, so if the sector size is 32K and your data is 88bytes, I'd make 32768/88=372 blocks.Starting with erased flash, everything is 0xFF.As we know, we can set any bit to 0 without erasing the flash, so we can take advantage of that.In this case, use the first 372bits (48 bytes for best alignment) as some sort of index that tells us what block to use.So we take the first byte and read the number of 0s, which will be 0, so we're at the first block.When data is changed, you know it must be written to the next block, so you set the next index bit to 0 and write the data to the next block.When all blocks were written, erase the whole block and start over.Let's simplify, say we have 8 blocks of 16 bytes each, so we need the first 8 bits=1byte to store the index.The index data would do:Code:Stored Our index is number bits set to 0b11111111 -> 0b11111110 -> 1b11111100 -> 2b11111000 -> 3b11110000 -> 4b11100000 -> 5b11000000 -> 6b10000000 -> 7b00000000 -> 8A basic example lacking a lot of stuff, but showing the concept:Code:mydata_t my_data; // Data in ram#define BLK_COUNT 8uint8_t get_flash_index(void){uint8_t index;read_flash_data(index_address, &index, index_size); // Read index: src addr, dst addr, sizefor(uint8_t i=0; i<BLK_COUNT; i++){if( (index & (1<<i) )({ // Find first bit set to 1return i; // This is our index}}return (BLK_COUNT-1);}void load_flash_data(void){read_flash_data(first_block_adr+(sizeof(mydata_t)*get_flash_index()), &my_data, sizeof(my_data_t)); // Read data: src addr, dst addr, size}void store_flash_data(void){uint8_t index=get_flash_index(), new_index=0;if(++index>(BLK_COUNT-1){ // Increase index, if all blocks are used then erase everything and start overindex=0;erase_flash_sector();}for(uint8_t i=0; i<index; i++){new_index |= 1<<i; // Set bits to 1 for each block used}new_index = ~new_index; // Invert bits, as we count 0s, not 1swrite_flash_data(&new_index, index_address, index_size); // Write index: src, dst, sizewrite_flash_data(&my_data, first_block_address+(sizeof(my_data_t)*index), sizeof(my_data_t)); // Write data: src, dst, size}