Seeking cheap embedded file storage solution - Page 1
Forum section: electronics / general

author: cheeseburger
author_rank: Contributor
timestamp: July 27, 2025, 09:59:46 pm
content: HelloI'm searching a way to save files in the total range of 64MB to 256MB in the kind of cheapest way possible. AliExpress parts allowedMy MCU is kind of capable in terms of computing and RAM, it is a ESP32-C6.The current implementation uses a W25Q256JVEIQ with LittleFS. This works great, but it "just" provides around 32MB storage for 0.5-1 USD in AliExpress pricing.With the bigger W25Q512JVEIQ, we would be in the 64MB storage range, but on AliExpress this part is >3USD.Both parts above are SPI NOR which work extremely well with LittleFS.A possible price optimization would be the use of a SPI NAND part.The W25N01GVZEIG is available for <0.5USD on AliExpress, which makes it very tempting as it is 128MB. But it is SPI NAND, so there is the need for much more intelligent software.Anyone any idea if there is a well working filesystem for such SPI NAND?I played around with GD5F1GQ4UBYIGR some time ago, this is a SPI NAND too. I was able to talk to it with raw SPI commands, but I never got it working together with a wear-leveling library (dhara).A different route would be the use of cheap microSD cards. They are available with 128MB for ~0.5USD on AliExpress. But they need a holder which needs board space. Also talking with SPI to cheap microSD cards showed kind of problematic in the past.

author: langwadt
author_rank: Super Contributor
timestamp: July 27, 2025, 10:21:09 pm
content: doesn't this work just fine with microSD?https://github.com/sparkfun/OpenLog

author: Fire Doger
author_rank: Regular Contributor
timestamp: July 27, 2025, 11:49:52 pm
content: There are a lot of questions that must be answered before making suggestions.For example what are those files? (if for example are json strings then your problem is not your flash size)Who generates them, who consumes them and how? (In some cases you could have compressed files, XZ works really well on embedded)Do you really need littleFS or any FS in general? (In some cases a single blob is much better, i.e. in wear leveling)How often do you expect the files to change? (maybe the flash lives more than you without wear leveling)Cheap microSDs are a pain, they may draw a ton of current and you may need 50+uF decoupling.Also they need a switch for power cycling because they can get stuck after days of operation.

author: cheeseburger
author_rank: Contributor
timestamp: July 28, 2025, 10:58:44 am
content: Quote from: Fire Doger on July 27, 2025, 11:49:52 pmWe are talking about compressed files which are mostly consumed (and modified) by calling them up with the filename.For example audio files like MP3. It is also possible that it needs to store compressed image files like JPEG.These files will also potentially be replaced via network functionality.Quote from: Fire Doger on July 27, 2025, 11:49:52 pmUsing xz, or a better audio/image codec, is indeed an interesting idea!Quote from: Fire Doger on July 27, 2025, 11:49:52 pmVery rarely, but when they do it should be a kind of reliable process. But ditching wear leveling might be a potential compromise.Quote from: Fire Doger on July 27, 2025, 11:49:52 pmDoesn't sound promising if you're the same opinion that I am

author: voltsandjolts
author_rank: Unknown
timestamp: July 28, 2025, 11:17:07 am
content: SPI NAND that you mentioned would likely be the cheapest option. How much erase/write cycles do you expect to do? I've used NAND in systems that only needed a few hundred writes over the device lifetime and didn't implement wear levelling. Lazy, perhaps. But it worked fine. Just build a list of bad blocks initially when the board is new, and avoid them.

author: cheeseburger
author_rank: Contributor
timestamp: July 28, 2025, 11:33:57 am
content: Quote from: voltsandjolts on July 28, 2025, 11:17:07 amMight work out for my use-case. I don't except more than a handful rewrites to the storage over the life-span of the device.Quote from: voltsandjolts on July 28, 2025, 11:17:07 amWere you able to implement this in a filesystem like LittleFS/FatFs/SPIFFS?

author: xvr
author_rank: Frequent Contributor
timestamp: July 28, 2025, 12:24:21 pm
content: ESP32 itself uses SPI NAND as code storage. So IDF includes all needed drivers for low level SPI NAND access and for all FS (FatFS and SPIFFS for sure)

author: voltsandjolts
author_rank: Unknown
timestamp: July 28, 2025, 12:36:38 pm
content: Quote from: cheeseburger on July 28, 2025, 11:33:57 amI wasn't using an FS, just writing raw. LittleFS must have some way of handling bad blocks though.I did notice that majority of the ST NAND chips I was using didn't have any bad blocks. So you could even pre-screen for NANDs without bad blocks.Note that the spare area in NAND pages is intended for CRC error correction. Some NANDs have built-in CRC/correction engines which also reduce fw burden.

author: peter-h
author_rank: Super Contributor
timestamp: July 28, 2025, 01:07:55 pm
content: For loads of storage you probably can't beat an SD or micro-SD card.At the basic level you can talk to it with SPI. It should do a few megabits/sec. There are threads here on this topic, also covering the (debatable) licensing requirements for the faster modes.FatFS works great. I am using it with Adesto SPI FLASH, 4MB.

author: Fire Doger
author_rank: Regular Contributor
timestamp: July 29, 2025, 07:31:51 am
content: Espressif has an xz library which you can try outhttps://components.espressif.com/components/espressif/xz/versions/1.1.0/examples/xz_decompress_fileI used it for ST. The benefit of xz is that you can decompress it in steps, with a few kb of RAM usage. (which can also be static allocated)But for already compressed files It won't do much of a difference. In addition, if the consumer is a browser XZ might not be the option.In my use case I reduced my .bmp images to 256 colors, and converted them to 8bpp with 256 32bit look up table (LUT).From 24MB of firmware size It went down to 2MB and firmware update was a lot faster.

author: peter-h
author_rank: Super Contributor
timestamp: July 29, 2025, 10:23:00 am
content: BMP compresses massively; far better to support jpeg and not bother with compression