Change:
<ins> import ./make-test-python.nix ({ lib, pkgs, ... }: # This nixosTest is supposed to check the following: # # - Whether syncthing's API handles multiple requests for many devices, see #   https://github.com/NixOS/nixpkgs/issues/260262 # # - Whether syncthing-init.service generated bash script removes devices and #   folders that are not present in the user's configuration, which is partly #   injected into the script. See also: #   https://github.com/NixOS/nixpkgs/issues/259256 # let # Just a long path not to copy paste configPath = "/var/lib/syncthing/.config/syncthing/config.xml"; # We will iterate this and more attribute sets defined here, later in the # testScript. Start with this, and distinguish these settings from other # settings, as we check these differently with xmllint, due to the ID. settingsWithId = { devices = { # All of the device IDs used here were generated by the following command: # #    (${pkgs.syncthing}/bin/syncthing generate --home /tmp/foo #       | grep ID: | sed 's/.*ID: *//') && rm -rf /tmp/foo # # See also discussion at: # https://forum.syncthing.net/t/how-to-generate-dummy-device-ids/20927/8 test_device1.id  = "IVTZ5XF-EF3GKFT-GS4AZLG-IT6H2ZP-6WK75SF-AFXQXJJ-BNRZ4N6-XPDKVAU"; test_device2.id  = "5C35H56-Z2GFF4F-F3IVD4B-GJYVWIE-SMDBJZN-GI66KWP-52JIQGN-4AVLYAM"; test_device3.id  = "XKLSKHE-BZOHV7B-WQZACEF-GTH36NP-6JSBB6L-RXS3M7C-EEVWO2L-C5B4OAJ"; test_device4.id  = "APN5Q7J-35GZETO-5KCLF35-ZA7KBWK-HGWPBNG-FERF24R-UTLGMEX-4VJ6PQX"; test_device5.id  = "D4YXQEE-5MK6LIK-BRU5QWM-ZRXJCK2-N3RQBJE-23JKTQQ-LYGDPHF-RFPZIQX"; test_device6.id  = "TKMCH64-T44VSLI-6FN2YLF-URBZOBR-ATO4DYX-GEDRIII-CSMRQAI-UAQMDQG"; test_device7.id  = "472EEBG-Q4PZCD4-4CX6PGF-XS3FSQ2-UFXBZVB-PGNXWLX-7FKBLER-NJ3EMAR"; test_device8.id  = "HW6KUMK-WTBG24L-2HZQXLO-TGJSG2M-2JG3FHX-5OGYRUJ-T6L5NN7-L364QAZ"; test_device9.id  = "YAE24AP-7LSVY4T-J74ZSEM-A2IK6RB-FGA35TP-AG4CSLU-ED4UYYY-2J2TDQU"; test_device10.id = "277XFSB-OFMQOBI-3XGNGUE-Y7FWRV3-QQDADIY-QIIPQ26-EOGTYKW-JP2EXAI"; test_device11.id = "2WWXVTN-Q3QWAAY-XFORMRM-2FDI5XZ-OGN33BD-XOLL42R-DHLT2ML-QYXDQAU"; }; # Generates a few folders with IDs and paths as written... folders = lib.pipe 6 [ (builtins.genList (x: { name = "/var/lib/syncthing/test_folder${builtins.toString x}"; value = { id = "DontDeleteMe${builtins.toString x}"; }; })) builtins.listToAttrs ]; }; # Non default options that we check later if were applied settingsWithoutId = { options = { autoUpgradeIntervalH = 0; urAccepted = -1; }; gui = { theme = "dark"; }; }; # Used later when checking whether settings were set in config.xml: checkSettingWithId = { t # t for type , id , not ? false }: '' print("Searching for a ${t} with id ${id}") configVal_${t} = machine.succeed( "${pkgs.libxml2}/bin/xmllint " "--xpath 'string(//${t}[@id="${id}"]/@id)' ${configPath}" ) print("${t}.id = {}".format(configVal_${t})) assert "${id}" ${if not then "not" else ""} in configVal_${t} ''; # Same as checkSettingWithId, but for 'options' and 'gui' checkSettingWithoutId = { t # t for type , n # n for name , v # v for value , not ? false }: '' print("checking whether setting ${t}.${n} is set to ${v}") configVal_${t}_${n} = machine.succeed( "${pkgs.libxml2}/bin/xmllint " "--xpath 'string(/configuration/${t}/${n})' ${configPath}" ) print("${t}.${n} = {}".format(configVal_${t}_${n})) assert "${v}" ${if not then "not" else ""} in configVal_${t}_${n} ''; # Removes duplication a bit to define this function for the IDs to delete - # we check whether they were added after our script ran, and before the # systemd unit's bash script ran, and afterwards - whether the systemd unit # worked. checkSettingsToDelete = { not }: lib.pipe IDsToDelete [ (lib.mapAttrsToList (t: id: checkSettingWithId { inherit t id; inherit not; } )) lib.concatStrings ]; # These IDs are added to syncthing using the API, similarly to how the # generated systemd unit's bash script does it. Only we add it and expect the # systemd unit bash script to remove them when executed. IDsToDelete = { # Also created using the syncthing generate command above device = "LZ2CTHT-3W2M7BC-CMKDFZL-DLUQJFS-WJR73PA-NZGODWG-DZBHCHI-OXTQXAK"; # Intentionally this is a substring of the IDs of the 'test_folder's, as # explained in: https://github.com/NixOS/nixpkgs/issues/259256 folder = "DeleteMe"; }; addDeviceToDeleteScript = pkgs.writers.writeBash "syncthing-add-device-to-delete.sh" '' set -euo pipefail export RUNTIME_DIRECTORY=/tmp curl() { # get the api key by parsing the config.xml while ! ${pkgs.libxml2}/bin/xmllint  --xpath 'string(configuration/gui/apikey)'  ${configPath}  >"$RUNTIME_DIRECTORY/api_key" do sleep 1; done (printf "X-API-Key: "; cat "$RUNTIME_DIRECTORY/api_key") >"$RUNTIME_DIRECTORY/headers" ${pkgs.curl}/bin/curl -sSLk -H "@$RUNTIME_DIRECTORY/headers"  --retry 1000 --retry-delay 1 --retry-all-errors  "$@" } curl -d ${lib.escapeShellArg (builtins.toJSON { deviceID = IDsToDelete.device;})}  -X POST 127.0.0.1:8384/rest/config/devices curl -d ${lib.escapeShellArg (builtins.toJSON { id = IDsToDelete.folder;})}  -X POST 127.0.0.1:8384/rest/config/folders ''; in { name = "syncthing-init"; meta.maintainers = with lib.maintainers; [ doronbehar ]; nodes.machine = { services.syncthing = { enable = true; overrideDevices = true; overrideFolders = true; settings = settingsWithoutId // settingsWithId; }; }; testScript = '' machine.wait_for_unit("syncthing-init.service") '' + (lib.pipe settingsWithId [ # Check that folders and devices were added properly and that all IDs exist (lib.mapAttrsRecursive (path: id: checkSettingWithId { # plural -> solitary t = (lib.removeSuffix "s" (builtins.elemAt path 0)); inherit id; } )) # Get all the values we applied the above function upon (lib.collect builtins.isString) lib.concatStrings ]) + (lib.pipe settingsWithoutId [ # Check that all other syncthing.settings were added properly with correct # values (lib.mapAttrsRecursive (path: value: checkSettingWithoutId { t = (builtins.elemAt path 0); n = (builtins.elemAt path 1); v = (builtins.toString value); } )) # Get all the values we applied the above function upon (lib.collect builtins.isString) lib.concatStrings ]) + '' # Run the script on the machine machine.succeed("${addDeviceToDeleteScript}") '' + (checkSettingsToDelete { not = false; }) + '' # Useful for debugging later machine.copy_from_vm("${configPath}", "before") machine.systemctl("restart syncthing-init.service") machine.wait_for_unit("syncthing-init.service") '' + (checkSettingsToDelete { not = true; }) + '' # Useful for debugging later machine.copy_from_vm("${configPath}", "after") # Copy the systemd unit's bash script, to inspect it for debugging. mergeScript = machine.succeed( "systemctl cat syncthing-init.service | " "${pkgs.initool}/bin/initool g - Service ExecStart --value-only" ).strip() # strip from new lines machine.copy_from_vm(mergeScript, "") ''; }) </ins>

Discussion:
If you add enough (minimum number I haven't searched for) settings, then the script of applying these settings via api will not be able to execute completely and curl will generate "CSRF Error" error, this happens because syncthing resets api key after applying a certain number of settings Steps to reproduce the behavior: services.syncthing several devices and options logs of syncthing-init.service, there will be errors syncthing-init.service will execute entirely with no errors and will apply the entire configuration without errors ! I made a vm with the minimal configuration at which the error is reproduced I used nixpkgs commit Config for vm - Steps to reproduce nixpkgs - vm config - vm - vm, login as with password "CSRF Error" and parse error from jq Also during testing I managed to fix this by parsing the api key before each curl call, this helps but looks like a dirty solution <!-- Please @ people who are in the list of the offending package or module. If in doubt, check for whoever last touched something. --Please run and paste the result.
Hmm indeed we recently changed a few details in the script in: I tried to write a nixos test to test some of these issues, and I experienced numerous other bugs, and I sort of reproduced the issue as well. I Reverted In: And reopened: cc .
I have this issue with (vm config, that i provided, uses this values too) So i dont think reverting will help
Could you try to merge locally and test?
I tested vanila with as patch on top. In both cases, the issue persists
OK. Thanks for the report!
As I said before, getting the api key from config before each curl call solves the problem (an example of this solution is in the patch), but I don't think it's a good solution, probably we should check the result (or/and stdout) of curl, reparsing api key if needed, call curl again, before trying to parse it with jq Patch:
I also found that to add several different devices you need to have different id's, so I tested with four devices with different id's. With my patch, all 4 devices were to correctly and syncthing-init.service ran correctly and without errors
I also found out that during my testings in . You are invited to submit that patch, with better formatting.

I am also looking into finding a way to add dummy, but valid devices ID, for testings.
OK I managed to fix the test, regarding checking that the script indeed deletes devices that need to be deleted. I haven't shared it in , because I use there private (sort-of) syncthing's devices' IDs I took from my personal configuration. I'm still looking for a way to generate valid dummy devices IDs. My help request is at syncthing's forum: I also applied a better version of your patch to the (currently draft) . I plan to add many more devices to that test, and maybe do the same for folders. Then I'll also test issue . In the meantime, I'm waiting for a dummy device ID generator.
I managed to create dummy syncthing device IDs and I am using them now in my PR . I saw your PR , and I think we can concentrate our efforts together on my PR at , where I'm currently failing to reproduce your issue, your comments will be appreciated. I am testing your issue without the 2nd commit in my PR (that is identical to your commit at ), and I'm testing that is fixed by reverting commit just for testing (this works :)).