Preserving settings over a firmware upgrade - Page 1
Forum section: electronics / general

author: AndyC_772
author_rank: Super Contributor
timestamp: July 25, 2023, 02:51:31 pm
content: Hi,I have a product which has been in production for a number of years, and which has a lot of configurable settings. Some of these are set at the factory and are hidden, while others are accessible to the end user.The environment is an ARM microcontroller using internal storage only. No file system or networking, but it can be connected to a PC via a serial link.Typically I store all of a product's settings in a C struct called 'nv' ('non-volatile'), which is stored in Flash whenever a change is made, and read back in as part of the boot process.Within this struct there are all kinds of integers, strings, other structs and so on, and then right at the very end there's a uint32_t which must always be set to a 'magic number' to indicate that the rest of the struct has been populated with valid data. This number could be a fixed value, a CRC, a cryptographic hash, checksum, or whatever.When the firmware boots, it checks for the magic number. If it's valid, then that's great, carry on as normal. If not, it assumes the configuration is broken, and restores factory default settings including a new magic number.This works well. It ensures that the unit always boots with valid settings whether it's brand new and is booting for the first time, or if it's been updated to new firmware which (usually) has a larger 'nv' struct with the magic number in a different place.Until quite recently, firmware updates have only ever been done at a very limited number of authorised sites, where having the unit reset to factory settings after an update isn't a problem. However, I'd now like to explore how to go about preserving settings when a unit is updated by the end user in the field.The issue is, of course, that new firmware may well require some new settings that weren't supported before. Insert new items into a struct, and everything beyond them has to move.A few options spring to mind, but I can't quite figure out which I dislike least.Keep the original nv struct exactly the same forever. For new settings introduced in the next firmware release, create a new struct called nv2. Then nv3, nv4 etc, each with their own magic number to show they're valid.As above, but add the new items to the end of nv so they sit after the magic number, which at least means I don't have to litter my code with nv3.this, nv5.that, nv2.the_other etcReserve space in nv between the last used setting and the magic number, put any new settings here. The magic number doesn't move, so it definitely has to be a CRC or hash, not a fixed value.Tag the nv struct with a value that indicates its version number. Include start-up code that recognises previous versions and knows how to migrate settings to the current version.Export settings in a readable format, with tags or commands to indicate what each value actually means. After an upgrade, reset to factory defaults, then parse the readable version to get back to a binary.The better idea that I haven't thought of yet...!Suggestions please, folks. What's the industry's solution to this problem?

author: tszaboo
author_rank: Super Contributor
timestamp: July 25, 2023, 03:27:53 pm
content: Think of it from the hardware perspective for a second. You can only erase sectors, that are for example 4KB in size, and you need to erase them to overwrite data in it. Is 4KB enough for you, than make the last 4 bytes eg. for your CRC and magic number, and place the data before, always.How you organize your data before it... there are so many ways to do that I'm sure you find something you like. I think last time I was doing something like this, I had a lightweight version control for the config file, and the config version was the first data saved.

author: Veteran68
author_rank: Frequent Contributor
timestamp: July 25, 2023, 03:56:40 pm
content: Several ways to go about this, but just to clarify: are youconsideredconcerned about the storage space available and ensuring it's contiguous, or are you simply concerned about the management of the code itself and how the struct is laid out?Assuming a contiguous storage area, my preferred approach would be something like your second bullet option, but to add a size value as the very first value in your configuration structure. Optionally (but not necessary since you now know the size of the struct), you can move the CRC/magic# to the second value in the struct, something like:struct Config {uint32_t sz, (or maybe size_t)uint32_t crc,...};Now for any new version you know exactly how much data to read from the old configuration. The CRC being second is only a convenience but not necessary, since you could always do the math (sz-sizeof(crc)) to get to the CRC field at the end, but if you're going to rework things for the future, I'd consider moving it to second position and reading it directly as well.Then any new configuration values for new versions are simply appended to the end and the size+CRC is updated to reflect that.EDIT: typo

author: Siwastaja
author_rank: Super Contributor
timestamp: July 25, 2023, 04:06:48 pm
content: Magic number for the whole configuration which indicates the "revision" (magic number changes whenever large changes are made). Keep CRC a separate concept.Increasing the configuration size at the end with something where all zeroes (or all ones) can be treated as "sane defaults" does not count as "large change", and hence does not require changing the magic number. This enables you to do maybe 95% of updates without breaking compatibility. Your usual feature creep.When you find out you need to restructure old stuff, then you change the magic number and add a special migration code (which is quite PITA to write and test) which takes conf version xyz and turns it into conf version åäö. This has to understand what the conf means in order to generate the new, functionally equivalent conf. Maybe this is mostly copying variables into different locations.To make your life easier, add dummy fields here and there unless you are very tight on flash. Have a convention so that unused bytes are written to 0 (or 0xff) and treat that as "sane default" in the new code. This way, you can avoid those migration updates.Current project of mine is now on firmware version 9 and there has been one (1) update which required migration so far and that was back when we had some 3 customers or so, it was between releases 1 and 2. This scheme is not perfect by any means but it sure is simple.

author: nctnico
author_rank: Super Contributor
timestamp: July 25, 2023, 04:17:10 pm
content: I always add versions numbers to configuration data. I use a header that has size, version and a checksum (can be CRC or one of the SHA algorithms). By using a fixed header, I can use the same software to deal with any kind of configuration data and put it somewhere on a flash or eeprom memory. The higher layers can figure out how to deal with the various versions.

author: tom66
author_rank: Super Contributor
timestamp: July 25, 2023, 05:13:06 pm
content: Build the data in the memory a little like a key-value struct, where each setting has a 'key ID' (this can just be decoded to flash address) and a size.The key ID can encode the type and size of variable. For instance, the first byte can signal type (int8, uint16, float) and the second byte might encode maximum size. Then the remaining 16 bits (assuming a 32-bit CPU) can effectively encode the address of the value.Then you write a number of functions that do things like:uint8_t getKeyAsU8(uint32_t key);uint16_t getKeyAsU16(uint32_t key);I don't know if you need to store strings. That can get tricky as they are variable length, and they might need to be packed in a different way.You could make it so that trying to get a key as an invalid type (e.g. can't convert U16 to U8) returns zero, throws exception (in C++) or passes an error state by reference.Future programs could access keys that have not yet been saved. This can be handled by robust checks on the values, or by checking a version number and preloading values with defaults if they don't match. If you go for the 'header' for each value, then the value can be loaded with a default if it is not yet set.This method is similar to ublox's method for their flash configuration, though they do note in other app notes that firmware updates do not guarantee configuration will be kept in Flash.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 25, 2023, 05:17:14 pm
content: The id-length-data scheme has an advantage that you can update settings (not unlimited times of course) without erase cycle - just append {id-length-data} at the end. Flash erase state (usually 0xff) signifies end-of-chain. To fetch a value, scan the whole chain and keep the last value for each id (you can do this at boot, plus during update, and just maybe update a pointer to each item). It's also quite efficient in storage space when there are different size items. If you then need to add a new feature, no problem, you just add a new ID for it; it can be physically at any memory address.

author: nctnico
author_rank: Super Contributor
timestamp: July 25, 2023, 05:38:19 pm
content: Next step is to store the config as a JSON formatted text blob. But you still might need to do some kind of conversion if a single setting becomes two settings or vice versa.

author: tom66
author_rank: Super Contributor
timestamp: July 25, 2023, 05:59:37 pm
content: Quote from: nctnico on July 25, 2023, 05:38:19 pmIf you've got a bigger applications processor (we use Zynq's a lot) then JSON absolutely makes sense, it's quite efficient to parse, is human readable, error resistant (invalid keys can return defaults or throw exceptions on most parsers) and handles backwards and forwards compatibility reasonably well.

author: nctnico
author_rank: Super Contributor
timestamp: July 25, 2023, 07:28:45 pm
content: You can use JSON on a small microcontroller just fine. In the end you only need a few simple text parsing functions to take the JSON data apart and extract the data you need.

author: AndyC_772
author_rank: Super Contributor
timestamp: July 25, 2023, 07:32:43 pm
content: Some good ideas there folks, thank you all.I think I'll go with:- keep each new variant as similar as possible to what went before. Insert dummy variables where there used to be meaningful data, add anything new at the end. Everything stays in the same place as far as possible, at least until I run out of storage and have to do something drastic. Fortunately I have a lot more space available for code than I do for settings.- add a revision number to the start of the nv struct, which increments each time there's a change that the code needs to know about. Come to think of it, I do this with register interfaces to FPGAs all the time, so it's hardly a foreign concept.- move the magic number to the start, so it's always in the same place; we can still tell the difference between old config vs new config vs erased flash.- at power up, check the revision number and run some one-time code to migrate settings as necessary. This might be as simple as setting factory defaults for new features, but it might include manipulating other data too. This is OK; better to update the settings once than forever have to cope with the fact that they might be stored in an out-of-date way.

author: tom66
author_rank: Super Contributor
timestamp: July 25, 2023, 07:33:45 pm
content: Quote from: nctnico on July 25, 2023, 07:28:45 pmThe issue I forsee is updating one key possibly requires rewriting the whole JSON. That possibly requires enough RAM to recreate the JSON as a string, then you need to overwrite all of the flash pages that change. If you only have a couple of settings, no issue, but a large number of settings quickly becomes a bottleneck.Not an issue on a Zynq running Linux. A headache on an STM32 with 32KB of RAM...

author: nctnico
author_rank: Super Contributor
timestamp: July 25, 2023, 07:49:38 pm
content: Quote from: tom66 on July 25, 2023, 07:33:45 pmNo problem at all. You erase the flash page you want to write the settings to (which needs to be done anyway *) and then create (snprintf) each piece of the JSON data one by-one and commit to flash. In case the flash can be written in chunks of several bytes (which is typical for ECC flash), then all you need is a small extra buffer to capture data before you have a full flash chunk. You'd need maybe 100 bytes at most for buffering (on stack) to have long key names and reasonable long text strings for settings.* Ofcourse you have to implement a double / multi buffering system in order to always retain a valid record with settings in case there is a power outage during write or the flash wears out.

author: Veteran68
author_rank: Frequent Contributor
timestamp: July 25, 2023, 08:03:36 pm
content: My how things have changed since my early embedded systems days. Sure, compared to the bloat of something like XML, JSON is certainly much lighter, but it's still not what I'd call firmware light. I use it all the time in PC application code where you're storing or transferring lots of strings or human readable data, but I can't see incurring its data storage and parsing logic overhead for something like a firmware configuration.High level markup languages are intended as self documenting data structures for data transfer and storage for high level business developers (many of which can barely code in Visual Basic or HTML/CSS/JS), not for an embedded systems developer dealing in bits and bytes, registers and interrupts. It's too easy to stream a series of bytes and lay them into a struct for use by the system. Then use the numerous tricks as discussed in this thread to manage the dynamic scalability of that data.Then again the MCUs and SOCs we have today have mountains of storage and power compared to the old days when we counted every clock cycle and data bit, while hand-optimizing the assembler produced by C compilers (or just wrote it in ASM to begin with).These kids today have it easy...

author: PlainName
author_rank: Super Contributor
timestamp: July 25, 2023, 09:18:51 pm
content: If the user is applying the upgrade then that implies some kind of connection (USB, WiFi, BT), in which case cannot the device send the config to the PC or phone (or website) that's sending the upgrade? In a perfect world it could save the config on demand (even if that meant emailing it to the user) so it could be saved 'just in case', or restored after the user has screwed it up.Although the on-chip config is no doubt fine as blob (actually a defined structure), converting to and from JSON for the purposes of saving and restoring would save lots of hassle when the format changes, perhaps by thing being inserted or removed.

author: nctnico
author_rank: Super Contributor
timestamp: July 25, 2023, 09:22:33 pm
content: Quote from: PlainName on July 25, 2023, 09:18:51 pmExactly! Conversion between configuration versions basically comes for free when using JSON to store the configuration.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 26, 2023, 05:57:31 am
content: Quote from: nctnico on July 25, 2023, 09:22:33 pmIt only avoids the fixed memory offset relationships, but does not magically understand functional differences which still need migration code to solve. And if you have no functional differences, then surely you don't need to change the layout either. If you do like I suggested, add new stuff at the end, then the only case where layout changes are large, fundamental changes where functional changes are to be expected too. In which case you need migration code anyway.Migration between two binary formats is actually simpler:Code:struct conf_v1{int16_t thing1; // 1 LSB = 0.1int32_t thing2;};struct conf_v2{double thing1; // add more range and resolution; fundamental change affecting both functionality and layoutint32_t thing2;}void migrate_v1_to_v2(struct conf_v1 const * const v1, struct conf_v2 * const v2){v2->thing1 = v1->thing1 * 0.1;v2->thing2 = v1->thing2;}With json, this is otherwise the same but instead of direct access with variable names and assignment operators, you use the json library to parse and generate the json. No automation nor simplification here, only a complication.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 26, 2023, 07:18:39 am
content: I would use JSON to store the settings in flash when1) the upstream, where the config is coming from, e.g. web UI or server, prefers JSON. Conversion into microcontroller memory layout has to be done somewhere; do it on the target itself and avoid multiple/intermediate formats2) there is enough flash & RAM on the target to do soOtherwise than that, it's the good old prefer data over code. If and when you can avoid conversions and directly use the memory layout and language syntax (in this case, probably C), do so. Some people love to add intermediate formats and write parsers for them, and it sure does help with job security because bugs sneak in and changes need modifications in multiple places - data structures themselves, parsers/generators, and you usuallystillcan't avoid versioning.What JSON actually provides? Two things basically:A) interoperability with some web UI / database / server frameworks (mapping directly into their data structure types, that's the whole point)B) decoupling of physical memory layout (lengths, offsets)If you don't need JSON for A, then B can be achieved much more efficiently by id-length-data pattern described above. Instead of arbitrary length string key, you would use for example a 16-bit ID. But you don't usually need even that. It can be easily shown that the struct member assignment pattern shown in my above post is never more complex or more difficult than the JSON equivalent, where you always need to parse and generate every member and need to map (manually, or maybe with preprocessor stringification) from JSON key strings to variable names, and forgetting some is as disastrous as it is on the direct binary mapping. Worst thing is, you always have to do all of this parsing, even if you never change revisions, while on versioned structs you just do it with major changes. Assignment operator on C between the members of the two structs does all the memory layout work for you.I can't really see any disadvantage on versioned structs. This is within the target itself so memory alignment and endianness are not even theoretical problems. Really it's the same struct where you would put the parsed JSON anyway / generate JSON from. Just write it as-is. RAM or FLASH, doesn't matter.

author: PlainName
author_rank: Super Contributor
timestamp: July 26, 2023, 10:59:51 am
content: Quote from: Siwastaja on July 26, 2023, 05:57:31 amWhere you have that kind of migration code you're polluting the product for a one-time (possibly never) migration. And next time it won't work so you'll need another lot of migration code to do it all again. Or never again.The idea with JSON (and it doesn't have to be JSON - that appears to be the format du jour, and previously might have been XML or INI or any other text-based thing) is that the parser on the product doesn't change to suit the data because the data is already OK. If there needs to be a migration you can do that off-product on a PC or phone or something, which has significant benefits. One is that a really tricky conversion is just invisible (to the product) code. Doesn't matter if it takes 2 weeks of an I7 to do it, or 20GB of RAM to store intermediate results. OK, a bit hyperbolic but that simple int to double may not be as simple as a 0.1 multiplier, particularly at the top and bottom of the range.Another benefit is that you can do partial export and restore - just do a couple of values instead of the entire flash. Think registry, and how you can alter bits without having to write the multi-MB monstrosity it is, and that's because it's no a binary copy/modify thing.Finally, and I'm sure we've all been there at some point, if all else fails you can manually create/modify the data to be changed/set. And it needs no support on the product beyond the standard save/restore function that doesn't change.Edit: Forgot to say that in most cases a conversion wouldn't be necessary in the first place. JSON (or whatever) says 'this_var' is 76 and the product code just puts 76 into whatever it uses as store: int, double, long, string, who knows or cares. So 'coversion' is easy: write out int as "76", read "76" into double. And the user can alter that to 74 without know or caring what the storage slot is, or where it is.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 26, 2023, 01:09:33 pm
content: Quote from: PlainName on July 26, 2023, 10:59:51 amThat is something you cannot completely avoid. Usage of JSON does not change this the slightest. You are just pushing the problem elsewhere: now youhaveto implement parsing, generation and mapping to internal variables anyway, even without any migration. This is at least the same or more work than the trivial case of migration using old and new structs where variables are assigned to each other.For more complex cases of migration, where functionality changes enough to prevent simple 1:1 mapping, then both approaches (direct struct vs. json intermediate) need similar amount of "throw-away" code for migration. By adding mystical cargo cult layers, you can hide this work, but finally someone has to do it somewhere.Remember we are discussing storage within a product, not interchange format. It's not like the CPU architecture (endianness) is changing. If the product also needs a configuration interchange format in ASCII for a server / web UI, then I wholeheartedly agree that JSON is one of the sanest choices, and if you just have resources on the MCU, bring it down to flash storage without another intermediate format, parse in one place.QuoteI agree, migration can be done on a server or something, it depends on the case which is the best, but throwaway code on an MCU is not necessarily that bad. If you have say 20 parameters needing remapping between two structs, that would be ~25 LoC with maybe 200 bytes of code generated. Much less than a JSON library.Generally it's a good idea to use tools for the intended purpose, JSON for data interchange; it's excessive for memory location independent code if that is the only specific need.QuoteI don't think you understand how the flash works. It has to be erased by sector and fully rewritten. There are ways to circumvent that to some extent like the accumulating id-length-data pattern I described above, but that's even less suitable territory to force JSON to.

author: DavidAlfa
author_rank: Super Contributor
timestamp: July 26, 2023, 02:45:26 pm
content: You could keep older settings struct definitions, then detect, check and import them.All the functions would need to be aware of older versions and how to handle them.A simple example, might contains some errors and possibly a lot of alignment issues, this is only a simple idea:Code:#define SETTINGS_VERSION settings_v2 // Current settings versiontypedef enum {settings_v0 = 0xFF00,settings_v1 = 0xFE01,settings_v2 = 0xFD02,}settings_version;typedef struct { // First int is always settings versionstruct{uint16_t version;uint16_t data0;uint16_t data1;}data;uint16_t checksum;} settings_v0_t;typedef struct {struct{uint16_t version;uint16_t data0;uint16_t data1;uint16_t data2;}data;uint16_t checksum;} settings_v1_t;typedef struct {struct{uint16_t version;uint16_t data0;uint16_t data1;uint16_t data2;uint16_t data3;}data;uint16_t checksum;} settings_v2_t;Code:void init(void){void * settings = some_flash_address;uint16_t version = *(uint16_t*)settings;if(!check_settings(settings))if(version != SETTINGS_VERSION)import_settings(settings);elsereset_settings();}Code:// Return 1 if error, 0 if OK.bool check_settings(void * settings){uint16_t new_checksum = checksum(settings);uint16_t checksum=0;switch (*(uint16_t*)settings){case settings_v0:checksum = (settings_v0_t*)settings->checksum;break;case settings_v1:checksum = (settings_v1_t*)settings->checksum;break;case settings_v2:checksum = (settings_v2_t*)settings->checksum;break;default:return 1;}if(new_checksum != checksum)return 1;return 0;}Code:// Import different system settings, reset new values to their default statevoid import_settings(void * settings){settings_v2_t s2; // Temporal buffer to import settings intoswitch (*(uint16_t*)settings){case settings_v0:s2.data.data0 = *(settings_v0_t*)settings->data.data0;s2.data.data1 = *(settings_v0_t*)settings->data.data1;s2.data.data2 = DEFAULT_DATA_2;s2.data.data3 = DEFAULT_DATA_3;break;case settings_v1:s2.data.data0 = *(settings_v0_t*)settings->data.data0;s2.data.data1 = *(settings_v0_t*)settings->data.data1;s2.data.data2 = *(settings_v0_t*)settings->data.data2;s2.data.data3 = DEFAULT_DATA_3;break;case settings_v2: // Nothing to do, same settigns version, we shouldn't reach thisbreak;default: // Unknown_settings_version !break;}s2.data.version = SETTINGS_VERSION;s2.checksum = checksum(s2);write_settings(s2);reboot();}Code:// Compute checksum for each settings versionuint16_t checksum(void * settings){uint16_t sz=0;uint16_t checksum=0;uint8_t * data = (uint8_t*)settings;switch (*(uint16_t*)settings){case settings_v0:sz = sizeof(settings_v0_t.data);break;case settings_v1:sz = sizeof(settings_v1_t.data);break;case settings_v2:sz = sizeof(settings_v2_t.data);break;default:return 0;}for(uint16_t i=0;i<sz;i++){checksum += *data++;}return checksum;}

author: PlainName
author_rank: Super Contributor
timestamp: July 26, 2023, 03:46:51 pm
content: QuoteI am fully conversant with how to program flash, thank you, and ways to retain existing data whilst changing parts that are smaller than a flash page.

author: thm_w
author_rank: Super Contributor
timestamp: July 26, 2023, 09:41:47 pm
content: Quote from: tom66 on July 25, 2023, 05:13:06 pmThis is a cool example of what you are talking about for EEPROM but sadly not open source:https://www.realtime.bc.ca/articles/nonvol.html

author: nctnico
author_rank: Super Contributor
timestamp: July 28, 2023, 08:28:25 am
content: Quote from: PlainName on July 26, 2023, 10:59:51 amThat is the cost of backward compatibility. For sure everyone will agree that it is better to avoid needing such migrations but either way it is better to prepare the way the data is stored so that software can detect the settings are compatible or not. When the configuration data is incompatible between versions, the next question is whether to use sensible defaults or try to convert.

author: Shonky
author_rank: Frequent Contributor
timestamp: July 28, 2023, 11:16:32 am
content: Quote from: thm_w on July 26, 2023, 09:41:47 pmFull source linked at the end?

author: Siwastaja
author_rank: Super Contributor
timestamp: July 28, 2023, 11:21:51 am
content: Quote from: nctnico on July 28, 2023, 08:28:25 amYeah. And while it would be nice to plan all the future features/fixes in advance and come up with perfect set of settings from day 1, it will only happen for a really simple product or a product which does not need much updates. In which case OP would not be asking. Better mentally prepare for significant changes. On the other hand, I also preferjust-in-timeprinciple for development: if you know you will be able to pull off migration code when needed, then do it only when needed. Simply, when the day comes that you have to break the compatibility, add the migration code then. Test it well before releasing the update in the wild. If you are lucky, you never need to do that. Even if you have to, you are then better off than trying to come up with the scheme 2 years earlier.And simple structs lend themselves very well for such migration functions, as shown above in code examples by me or DavidAlfa: all you need are the constructs available in the programming language you are using in that project anyway.

author: Kalvin
author_rank: Super Contributor
timestamp: July 28, 2023, 12:30:09 pm
content: The settings needs to be compatible both ways:- The settings need to be preserved when upgrading to a newer version.- The settings need to be backwards compatible when downgrading to an older version so that downgrading won't break those settings values that are shared between releases.If the settings cannot be preserved over upgrade/downgrade, the firmware should provide good default values for those settings that cannot be preserved.

author: DavidAlfa
author_rank: Super Contributor
timestamp: July 28, 2023, 12:31:04 pm
content: 50% simpler: Don't allow downgrading!

author: Siwastaja
author_rank: Super Contributor
timestamp: July 28, 2023, 12:38:42 pm
content: Quote from: Kalvin on July 28, 2023, 12:30:09 pmI would not generally agree. It's a sensible decision not to allow downgrading, to KISS, especially in a cheap commodity product. (Much more limiting decisions are made all the time.) For a special industrial expensive gizmo, that might be desirable for some weird reason customer might have, but then you can sell the downgrade work, along with generating new settings, as specialized work and the customer would pay for it.If downgrading is required for troubleshooting, then that can involve manually working out the correct settings. When the problem that required the downgrade is found, it can be applied to the new version so that the reason for downgrade goes away.

author: Kalvin
author_rank: Super Contributor
timestamp: July 28, 2023, 12:41:40 pm
content: Quote from: DavidAlfa on July 28, 2023, 12:31:04 pmIf the settings management is done rightTM, then it really doesn't increase complexity that much.

author: AndyC_772
author_rank: Super Contributor
timestamp: July 28, 2023, 12:53:16 pm
content: I'm inclined to favour:Code:if (nv.settings_version < CURRENT_SETTINGS_VERSION)migrate_settings();if (nv.settings_version > CURRENT_SETTINGS_VERSION)factory_default();Downgrading firmware isn't something I'd look to explicitly lock out, but it doesn't need to be so graceful.

author: PlainName
author_rank: Super Contributor
timestamp: July 28, 2023, 02:46:09 pm
content: I've upgraded a lot of stuff I wish could be downgraded to fix new bugs or unwanted features. Altium is a great example: doesn't care what version of software you're running and will tell you if a new feature can't be handled in an old version (obviously, with an embedded device it can't do that, but it can ignore the new data). Altium 09 can be used to view Altium 23 documents perfectly well, despite the features introduced in the intervening 16 years.I have similar software, and I love 'em. For the user they are pain free, and these things should be designed for the user, not the developer. In contrast, there is stuff I can't allow to be updated because there is no way back, and I hate 'em.pfSense is middle of the road. You can export settings, update (or install somewhere else) and import. The worst I've had has been the Ethernet interfaces changing names, so the low level network IDs have to be frigged about with. But the bulk of the settings have been fine. But look at Tweetduck - minor update and it can't import settings from the previous version. Another one I used to not allow to update (pointless now since it no longer works, thanks to Musk).

author: Siwastaja
author_rank: Super Contributor
timestamp: July 28, 2023, 02:52:15 pm
content: Quote from: Kalvin on July 28, 2023, 12:41:40 pmSettings management (how to store them) is the trivial part (unless you colossally fucked it up, of course). The problem is always in functionality (what the settings signify and how). There must be a reason why a new version with incompatible conf came out: it does something the old version did not do at all, or does some new thing better, and conf is somehow related to these changes. Now there also must be some reason why the new version is in use, and configured to do something. You would need to transform these new settings to something that does as close to the right thing as possible, using that old version. This is a lot of work for little gain, because going back in versions would be quite rare.Therefore, I second AndyC's suggestion that downgrading would be handled with factory defaults. As downgrading is a special measure against something anyway, manually changing settings is not a big deal.

author: PlainName
author_rank: Super Contributor
timestamp: July 28, 2023, 06:35:09 pm
content: QuoteDepends on how many, what they are, and the device. Some devices are so bleeding tedious to change anything you really don't want to do that very often, and with many devices you can't remember what you've changed since you spent that week setting it up 2 years ago (because, no export remember). That viewpoint - the user won't mind doing it all again - is what keeps stuff on old versions. If you're lucky the user will update once before finding out the best thing is to not do it again.

author: thm_w
author_rank: Super Contributor
timestamp: July 28, 2023, 09:14:43 pm
content: Quote from: Shonky on July 28, 2023, 11:16:32 amNot licensed as open source.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 29, 2023, 10:18:56 am
content: Quote from: PlainName on July 28, 2023, 06:35:09 pmYou missed the context. The discussion was about downgrading. That is something which doesn't happen on its own. If adjusting settings is tedious to user, manual firmware changes in the opposite direction is surely tedious as well. There must be some specific reason why user wants to do that.Your claim that difficulty indowngradingkeeps people in old versions is weird. I think it's the opposite; difficulty in downgrading but ease of upgrading would drive people into trying the new versions instead to see if they fix whatever problem they have (or think they have).

author: Kalvin
author_rank: Super Contributor
timestamp: July 29, 2023, 10:40:11 am
content: Quote from: AndyC_772 on July 28, 2023, 12:53:16 pmThis scheme has been working pretty well over the years:Firmware settings are stored as C structs. The attachment shows three different versions of the same settings structure evolving over firmware versions.When the firmware is run for the first time, the system doesn't find any existing settings, so it will just copy the default settings into the actual settings structure.When the firmware is upgraded from version 1 to version 2, the new firmware version will be able to restore the version 1 settings. The firmware will then copy the default settings from the version 2 defaults to the version 2 settings.When the firmware is upgraded from version 2 to version 3, the new firmware version will be able to restore the version 1 settings and version 2 settings. The firmware will then copy the default settings from the version 3 defaults to the version 3 settings.When the firmware is downgraded from version 3 to version 2, the new firmware version will find the old settings version 1 settings, version 2 settings, and version 3 settings. However, it will only restore the version 1 settings and version 2 settings.Typically, there are separate settings structs for different software components. This helps to keep the settings organized throughout the product's lifetime.

author: PlainName
author_rank: Super Contributor
timestamp: July 29, 2023, 02:15:49 pm
content: Quote from: Siwastaja on July 29, 2023, 10:18:56 amI was pointing out that changing settings can be a drag. Earlier, I pointed out that sometimes you want to downgrade because of new features/bugs. Both of these are in the context of "manually changing settings is not a big deal". Admittedly, the upgrade rant that flowed from that may have confused anyone not, ah, going with the flow and fixating on the part Ididn'tquote (because that quote was the entire context I was discussing).QuoteThat's not what I claimed.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 29, 2023, 02:46:04 pm
content: OK, so you just take words out of context to turn their meaning around, creating a strawman which you then argue against, without making it obvious you do so, wasting everyone's time. You are ignored from now on.