Change:
is documented as being an executable file, but what's not immediately obvious is that this file is copied into the nix store and therefore if it does anything relative to its own location, that's not going to be very effective. Looking through the repo I found at least one script that appears to be broken by this behavior; namely, the update script writes to , which presumably will fail if it's actually attempted because that path is going to be in the nix store when it really wants to write to the nixpkgs working copy. It would have been nice if had a mechanism by which the current directory could be set to the package's directory, but there's no existing means to do that. After some experimentation I determined that I could write something like the following: This writes a new file to the nix store whose only job is to back to the working tree so it can run the real update script from the correct directory. I feel like this sort of hack really shouldn't be necessary. I feel like the ideal behavior here is for the update script to be invoked with the current directory set to the containing directory of the file that set the attribute (using ). However this would likely be a backwards-incompatible change as there probably exist some update scripts that assume the current directory is the root of the repo (even though that's not at all guaranteed as I could run the update script from a subdirectory; I haven't actually confirmed but I assume the current directory is whatever the current directory was when I ran ). I did find at least one script that uses so that script wouldn't be broken by this, but I haven't done an exhaustive survey of update scripts.
It occurred to me this morning that if I set the to a string, it won't be moved into the nix store. Experimentally, the following seems to work: Of course, the script itself still needs to handle the being some arbitrary location, but at least it can then use to find its containing folder. In any case, this is non-obvious. It might be worth updating the manual to mention the behavioral distinction between providing a path and providing a string.
In we have essentially where is defined in .
Another itch is the way we use nix-shell shebangs in a lot of the updater scripts - the nix-shell invoked there might use another . In my case, I ended up with an that didn't know how to parse nix 2.0-style hash output, resulting in lines. I guess we should add some documentation/examples about how things should be used, and update the existing scripts to honor these recommendations.
Sorry, wanted to reply sooner but did not have time and then forgot about this. Yeah, this is something we are so accustomed that we do not even realize someone might not know that. IMHO this feature of Nix makes sense to ensure consistency of build trees but benefits are dubious in case of single-file update scripts. Hmm, can confirm produces an error. Apparently, it is a side effect of JSONification from Maybe we should stringify the paths there first. Yeah, using Nix’s script-relative semantics would work for individual packages but not for high-order updaters (e.g. the one used by GNOME packages). Indeed, update scripts and other service info should be as minimal as possible. Actually, I would expect a bash script to work relative to when run manually. But I can see how the knowledge of can become blurred when it is invoked by another program. expects to be called from nixpkgs root so this would break almost every update script. Though I am not against making work in subdirs and break BC if we specify the expected behaviour and fix all the in-tree s. If we do not want to introduce BC break, we could implement it using from Ideally, would work anywhere in the tree but since 99.9% of people will probably call it from the root, I am not sure making the scripts more robust will be worth the effort. It is documented in the : I see this issue quite frequently: people miss some detail in Nix manual and then a part of nixpkgs that relies on that knowledge confounds them. Unfortunately, I do not see a solution; the Nix manual is huge and many people did not read it cover to cover (I certainly did not not). And you need to know 90% of Nix manual so we cannot just copy the relevant parts to nixpkgs manual. I have set up to point to my nixpkgs checkout and I actually had the opposite problem – some update scripts depended on a package I updated previously so building the was blocked until everything built. Then I was glad that s depended on so I could point it to a cached revision. I agree though, that this behaviour is unintuitive and catering to my minor issue is probably not necessary. :+1:
It's probably safe to say that we should stringify all paths when converting to JSON; I can't imagine any scenario where we'd actually want the update script to be copied into the Nix store, given that we're invoking it immediately (if we're invoking it at all) and so long-term storage or consistency is unnecessary. On another note, I realized today that doesn't provide overlays when it imports the nixpkgs root, meaning this script will go ahead and try to invoke any attributes defined in overlays. This isn't objectively a bad thing, but I'm inclined to say that the nixpkgs maintainer update script shouldn't be touching overlays.
Yeah, it was designed before the advent of overlays. I can reproduce it by creating and then running
I realized there are some uses of outside of : so perhaps we do not want to limit to as in What do you think
I realized the overlay updateScript issue tonight because I was writing my own command to update my overlays. I ended up picking a different attribute name to avoid problems, but my solution to detecting just the overlay update scripts was to reimport nixpkgs with every overlay wrapped in a function that copied the overlay attributes into a brand new top-level attribute set, so I could examine them later. That said, if someone does want to update overlays with the maintainer script, maybe we could disable that by default but add a new argument that means “update overlays too”?
Apparently they use of so we probably do not need to worry about overlays.
looks great and will solve most of this. I would still like the manual to explicitly mention that will be run from an arbitrary PWD (frequently, but not always, the root of ).
Thank you for your contributions. This has been automatically marked as stale because it has had no activity for 180 days. If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity. Here are suggestions that might help resolve this more quickly: for maintainers and people that previously touched the related code and @ mention them in a comment. on the . on the on .
I believe this was fixed by The script name is stringified and the manual was updated to note that it may be run from arbitrary locations.

Discussion:
<programlisting> passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]; </programlisting> <del> Note that the update scripts will be run in parallel by default; you should avoid running <command>git commit</command> or any other commands that cannot handle that. </del> <ins> </para> <para> The script will be usually run from the root of the Nixpkgs repository but you should not rely on that. Also note that the update scripts will be run in parallel by default; you should avoid running <command>git commit</command> or any other commands that cannot handle that. </ins> </para> <para> For information about how to run the updates, execute <del> <cmdsynopsis> <command>nix-shell</command> <arg> maintainers/scripts/update.nix </arg> </cmdsynopsis> . </del> <ins> <command>nix-shell maintainers/scripts/update.nix</command>. </ins> </para> </listitem> </varlistentry>

in [x] ++ nubOn f xs; <del> pkgs = import ./../../default.nix { }; </del> <ins> pkgs = import ./../../default.nix { overlays = []; }; </ins> packagesWith = cond: return: set: nubOn (pkg: pkg.updateScript)

let attrSet = pkgs.lib.attrByPath (pkgs.lib.splitString "." path) null pkgs; in <del> packagesWith (name: pkg: builtins.hasAttr "updateScript" pkg) (name: pkg: pkg) attrSet; </del> <ins> if attrSet == null then builtins.throw "Attribute path `${path}` does not exists." else packagesWith (name: pkg: builtins.hasAttr "updateScript" pkg) (name: pkg: pkg) attrSet; </ins> packageByName = name: let

packageData = package: { name = package.name; pname = (builtins.parseDrvName package.name).name; <del> updateScript = pkgs.lib.toList package.updateScript; </del> <ins> updateScript = map builtins.toString (pkgs.lib.toList package.updateScript); </ins> }; <ins> packagesJson = pkgs.writeText "packages.json" (builtins.toJSON (map packageData packages)); optionalArgs = pkgs.lib.optional (max-workers != null) "--max-workers=${max-workers}" ++ pkgs.lib.optional (keep-going == "true") "--keep-going"; args = [ packagesJson ] ++ optionalArgs; </ins> in pkgs.stdenv.mkDerivation { name = "nixpkgs-update-script"; buildCommand = ''

''; shellHook = '' unset shellHook # do not contaminate nested shells <del> exec ${pkgs.python3.interpreter} ${./update.py} ${pkgs.writeText "packages.json" (builtins.toJSON (map packageData packages))}${pkgs.lib.optionalString (max-workers != null) " --max-workers=${max-workers}"}${pkgs.lib.optionalString (keep-going == "true") " --keep-going"} </del> <ins> exec ${pkgs.python3.interpreter} ${./update.py} ${builtins.concatStringsSep " " args} </ins> ''; }

die "Couldn't locate old source hash '$oldHash' (or it appeared more than once) in '$nixFile'!" fi <del> oldUrl=$(nix-instantiate $systemArg --eval -E "with import ./. {}; builtins.elemAt $attr.src.drvAttrs.urls 0" | tr -d '"') </del> <ins> oldUrl=$(nix-instantiate $systemArg --eval -E "with import ./. {}; builtins.elemAt ($attr.src.drvAttrs.urls or [ $attr.src.url ]) 0" | tr -d '"') </ins> if [ -z "$oldUrl" ]; then <del> oldUrl=$(nix-instantiate $systemArg --eval -E "with import ./. {}; $attr.src.url" | tr -d '"') if [ -z "$oldUrl" ]; then die "Couldn't evaluate source url from '$attr.src'!" fi </del> <ins> die "Couldn't evaluate source url from '$attr.src'!" </ins> fi drvName=$(nix-instantiate $systemArg --eval -E "with import ./. {}; (builtins.parseDrvName $attr.name).name" | tr -d '"')

fi # Escape regex metacharacter that are allowed in store path names <del> oldVersion=$(echo "$oldVersion" | sed -re 's|[.+]|&|g') oldUrl=$(echo "$oldUrl" | sed -re 's|[${}.+]|&|g') </del> <ins> oldVersionEscaped=$(echo "$oldVersion" | sed -re 's|[.+]|&|g') oldUrlEscaped=$(echo "$oldUrl" | sed -re 's|[${}.+]|&|g') </ins> <del> if [ $(grep -c -E "^s*(letb)?s*$versionKeys*=s*"$oldVersion"" "$nixFile") = 1 ]; then pattern="/b$versionKeybs*=/ s|"$oldVersion"|"$newVersion"|" elif [ $(grep -c -E "^s*(letb)?s*names*=s*"[^"]+-$oldVersion"" "$nixFile") = 1 ]; then pattern="/bnamebs*=/ s|-$oldVersion"|-$newVersion"|" </del> <ins> if [ $(grep -c -E "^s*(letb)?s*$versionKeys*=s*"$oldVersionEscaped"" "$nixFile") = 1 ]; then pattern="/b$versionKeybs*=/ s|"$oldVersionEscaped"|"$newVersion"|" elif [ $(grep -c -E "^s*(letb)?s*names*=s*"[^"]+-$oldVersionEscaped"" "$nixFile") = 1 ]; then pattern="/bnamebs*=/ s|-$oldVersionEscaped"|-$newVersion"|" </ins> else die "Couldn't figure out where out where to patch in new version in '$attr'!" fi

# Replace new URL if [ -n "$newUrl" ]; then <del> sed -i "$nixFile" -re "s|"$oldUrl"|"$newUrl"|" </del> <ins> sed -i "$nixFile" -re "s|"$oldUrlEscaped"|"$newUrl"|" </ins> if cmp -s "$nixFile" "$nixFile.bak"; then die "Failed to replace source URL '$oldUrl' to '$newUrl' in '$attr'!"

<del> { stdenv, lib, writeScript, python3, common-updater-scripts, coreutils, gnugrep, gnused }: </del> <ins> { stdenv, lib, writeScript, python3, common-updater-scripts }: </ins> { packageName, attrPath ? packageName, versionPolicy ? "odd-unstable" }: let

package_name="$1" attr_path="$2" version_policy="$3" <del> PATH=${lib.makeBinPath [ common-updater-scripts coreutils gnugrep gnused python ]} </del> <ins> PATH=${lib.makeBinPath [ common-updater-scripts python ]} </ins> latest_tag=$(python "${./find-latest-version.py}" "$package_name" "$version_policy" "stable") update-source-version "$attr_path" "$latest_tag" '';