Repository: NixOS/nixpkgs
Pull request #386789: pulumiPackages: move implementation to pkgs/by-name
Author: login: tie
Labels: name: 6.topic: python
description: Python is a high-level, general-purpose programming language.
name: 10.rebuild-linux: 11-100
description: This PR causes between 11 and 100 packages to rebuild on Linux.
name: 10.rebuild-darwin: 11-100
description: This PR causes between 11 and 100 packages to rebuild on Darwin.
name: 12.approvals: 1
description: This PR was reviewed and approved by one person.
Created: 2025-03-03T17:03:44Z | Merged: 2025-03-28T16:28:13Z

Second part of #382594.

- Moves pulumiPackages to `pkgs/by-name/pu/pulumi/plugins`.
- Updates pulumiPackages implementation to use `makeScopeWithSplicing`.
- Renames `pulumi-{go,nodejs,python}` for consistency with other plugins (with an alias if `config.allowAliases` option is enabled in Nixpkgs).
- Adds `testResourceSchema` and `pulumiTestHook` for plugin version bumps in the next PR.

Note that Pulumi requires `protobuf4` for Python packages. Build all Pulumi packages, SDKs and tests with:
<details>

```
nix build --impure --keep-going --expr 'with import ./. {
  config.allowAliases = false;
  overlays = [
    (_: prev: {
      pythonPackagesExtensions = prev.pythonPackagesExtensions ++ [
        (final: _: {
          protobuf = final.protobuf4;
        })
      ];
    })
  ];
};
linkFarmFromDrvs "pulumi-packages" (
  lib.concatLists (
    lib.mapAttrsToList (
      _: p:
      lib.optionals (lib.isDerivation p) (
        [ p ] ++ lib.attrValues (p.sdks or { }) ++ lib.attrValues (p.tests or { })
      )
    ) (pulumiPackages // { inherit pulumi; })
  )
)'
```
</details>

Note: pulumi-yandex-unofficial Python SDK fails to build on master; this will be fixed in the version bump PR.

## Things done

- Built on platform(s)
  - [ ] x86_64-linux
  - [x] aarch64-linux
  - [ ] x86_64-darwin
  - [ ] aarch64-darwin
- For non-Linux: Is sandboxing enabled in `nix.conf`? (See [Nix manual](https://nixos.org/manual/nix/stable/command-ref/conf-file.html))
  - [ ] `sandbox = relaxed`
  - [ ] `sandbox = true`
- [ ] Tested, as applicable:
  - [NixOS test(s)](https://nixos.org/manual/nixos/unstable/index.html#sec-nixos-tests) (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
  - and/or [package tests](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#package-tests)
  - or, for functions and "core" functionality, tests in [lib/tests](https://github.com/NixOS/nixpkgs/blob/master/lib/tests) or [pkgs/test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/test)
  - made sure NixOS tests are [linked](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#linking-nixos-module-tests-to-a-package) to the relevant packages
- [x] Tested compilation of all packages that depend on this change using `nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"`. Note: all changes have to be committed, also see [nixpkgs-review usage](https://github.com/Mic92/nixpkgs-review#usage)
- [ ] Tested basic functionality of all binary files (usually in `./result/bin/`)
- [25.05 Release Notes](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2505.section.md) (or backporting [24.11](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2411.section.md) and [25.05](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2505.section.md) Release notes)
  - [ ] (Package updates) Added a release notes entry if the change is major or breaking
  - [ ] (Module updates) Added a release notes entry if the change is significant
  - [ ] (Module addition) Added a release notes entry if adding a new NixOS module
- [x] Fits [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).

<!--
To help with the large amounts of pull requests, we would appreciate your
reviews of other pull requests, especially simple package updates. Just leave a
comment describing what you have tested in the relevant package/service.
Reviewing helps to reduce the average time-to-merge for everyone.
Thanks a lot if you do!

List of open PRs: https://github.com/NixOS/nixpkgs/pulls
Reviewing guidelines: https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#reviewing-contributions
-->

---

Add a :+1: [reaction] to [pull requests you find important].

[reaction]: https://github.blog/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/
[pull requests you find important]: https://github.com/NixOS/nixpkgs/pulls?q=is%3Aopen+sort%3Areactions-%2B1-desc



Diff:
diff --git a/pkgs/by-name/pu/pulumi/extra/mk-pulumi-package.nix b/pkgs/by-name/pu/pulumi/extra/mk-pulumi-package.nix
new file mode 100644
index 0000000000000..9981bb8d22516
--- /dev/null
+++ b/pkgs/by-name/pu/pulumi/extra/mk-pulumi-package.nix
@@ -0,0 +1,187 @@
+{
+  buildGoModule,
+  fetchFromGitHub,
+  python3Packages,
+}:
+let
+  mkBasePackage =
+    {
+      pname,
+      src,
+      version,
+      vendorHash,
+      cmd,
+      extraLdflags,
+      env,
+      ...
+    }@args:
+    buildGoModule (
+      rec {
+        inherit
+          pname
+          src
+          vendorHash
+          version
+          env
+          ;
+
+        sourceRoot = "${src.name}/provider";
+
+        subPackages = [ "cmd/${cmd}" ];
+
+        doCheck = false;
+
+        ldflags = [
+          "-s"
+          "-w"
+        ] ++ extraLdflags;
+      }
+      // args
+    );
+
+  mkPythonPackage =
+    {
+      meta,
+      pname,
+      src,
+      version,
+      ...
+    }:
+    python3Packages.callPackage (
+      {
+        buildPythonPackage,
+        pythonOlder,
+        parver,
+        pip,
+        pulumi,
+        semver,
+        setuptools,
+      }:
+      buildPythonPackage {
+        inherit
+          pname
+          meta
+          src
+          version
+          ;
+        format = "pyproject";
+
+        disabled = pythonOlder "3.7";
+
+        sourceRoot = "${src.name}/sdk/python";
+
+        propagatedBuildInputs = [
+          parver
+          pulumi
+          semver
+          setuptools
+        ];
+
+        postPatch = ''
+          if [[ -e "pyproject.toml" ]]; then
+            sed -i \
+              -e 's/^  version = .*/  version = "${version}"/g' \
+              pyproject.toml
+          else
+            sed -i \
+               -e 's/^VERSION = .*/VERSION = "${version}"/g' \
+               -e 's/^PLUGIN_VERSION = .*/PLUGIN_VERSION = "${version}"/g' \
+               setup.py
+          fi
+        '';
+
+        # Auto-generated; upstream does not have any tests.
+        # Verify that the version substitution works
+        checkPhase = ''
+          runHook preCheck
+
+          ${pip}/bin/pip show "${pname}" | grep "Version: ${version}" > /dev/null \
+            || (echo "ERROR: Version substitution seems to be broken"; exit 1)
+
+          runHook postCheck
+        '';
+
+        pythonImportsCheck = [
+          (builtins.replaceStrings [ "-" ] [ "_" ] pname)
+        ];
+      }
+    ) { };
+in
+{
+  owner,
+  repo,
+  rev,
+  version,
+  hash,
+  vendorHash,
+  cmdGen,
+  cmdRes,
+  extraLdflags,
+  env ? { },
+  meta,
+  fetchSubmodules ? false,
+  ...
+}@args:
+let
+  src = fetchFromGitHub {
+    name = "source-${repo}-${rev}";
+    inherit
+      owner
+      repo
+      rev
+      hash
+      fetchSubmodules
+      ;
+  };
+
+  pulumi-gen = mkBasePackage rec {
+    inherit
+      src
+      version
+      vendorHash
+      extraLdflags
+      env
+      ;
+
+    cmd = cmdGen;
+    pname = cmdGen;
+  };
+in
+mkBasePackage (
+  {
+    inherit
+      meta
+      src
+      version
+      vendorHash
+      extraLdflags
+      env
+      ;
+
+    pname = repo;
+
+    nativeBuildInputs = [
+      pulumi-gen
+    ];
+
+    cmd = cmdRes;
+
+    postConfigure = ''
+      pushd ..
+
+      chmod +w sdk/
+      ${cmdGen} schema
+
+      popd
+
+      VERSION=v${version} go generate cmd/${cmdRes}/main.go
+    '';
+
+    passthru.sdks.python = mkPythonPackage {
+      inherit meta src version;
+
+      pname = repo;
+    };
+  }
+  // args
+)
diff --git a/pkgs/by-name/pu/pulumi/extra/pulumi-test-hook.sh b/pkgs/by-name/pu/pulumi/extra/pulumi-test-hook.sh
new file mode 100644
index 0000000000000..f62a9418c46c9
--- /dev/null
+++ b/pkgs/by-name/pu/pulumi/extra/pulumi-test-hook.sh
@@ -0,0 +1,15 @@
+# shellcheck shell=bash
+
+appendToVar preCheckHooks pulumiTestHook
+
+pulumiTestHook() {
+    local tmpdir
+    tmpdir=$(mktemp -d)
+    export USER=${USER-nixbld} HOME=${HOME-$tmpdir}
+    export PULUMI_HOME=$tmpdir/.pulumi
+    export PULUMI_CONFIG_PASSPHRASE=5up3r53cr37
+    export PULUMI_SKIP_UPDATE_CHECK=1
+    export PULUMI_DISABLE_AUTOMATIC_PLUGIN_ACQUISITION=1
+    pulumi login "file://$tmpdir"
+    pulumi stack init -- "${pulumiStackName-nixpkgs}"
+}
diff --git a/pkgs/by-name/pu/pulumi/extra/test-resource-schema.nix b/pkgs/by-name/pu/pulumi/extra/test-resource-schema.nix
new file mode 100644
index 0000000000000..99ca01b0b7bf6
--- /dev/null
+++ b/pkgs/by-name/pu/pulumi/extra/test-resource-schema.nix
@@ -0,0 +1,35 @@
+{
+  lib,
+  runCommand,
+  pulumi,
+  jq,
+}:
+{
+  package,
+  name ? lib.removePrefix "pulumi-" (lib.getName package),
+  version ? lib.getVersion package,
+}:
+runCommand "pulumi-resource-${name}-schema-version-check"
+  {
+    resourceName = name;
+    expectedVersion = if version != null then version else "null";
+    nativeBuildInputs = [
+      jq
+      pulumi
+      package
+    ];
+    env = {
+      PULUMI_SKIP_UPDATE_CHECK = "1";
+      PULUMI_DISABLE_AUTOMATIC_PLUGIN_ACQUISITION = "1";
+    };
+    __darwinAllowLocalNetworking = true;
+    meta.timeout = 120;
+  }
+  ''
+    actualVersion=$(pulumi package get-schema -- "$resourceName" | jq -j .version)
+    if [[ $expectedVersion != "$actualVersion" ]]; then
+      echo "Expected schema version $expectedVersion, but got $actualVersion" >&2
+      false
+    fi
+    mkdir -p "$out"
+  ''
diff --git a/pkgs/by-name/pu/pulumi/package.nix b/pkgs/by-name/pu/pulumi/package.nix
index a7eaaf0512734..219a23859dec3 100644
--- a/pkgs/by-name/pu/pulumi/package.nix
+++ b/pkgs/by-name/pu/pulumi/package.nix
@@ -7,11 +7,9 @@
   git,
   buildPackages,
   # passthru
-  runCommand,
-  makeWrapper,
+  callPackage,
   testers,
   pulumi,
-  pulumiPackages,
 }:
 buildGoModule rec {
   pname = "pulumi";
@@ -117,25 +115,18 @@ buildGoModule rec {
   '';
 
   passthru = {
-    pkgs = pulumiPackages;
-    withPackages =
-      f:
-      runCommand "${pulumi.name}-with-packages"
-        {
-          nativeBuildInputs = [ makeWrapper ];
-        }
-        ''
-          mkdir -p $out/bin
-          makeWrapper ${pulumi}/bin/pulumi $out/bin/pulumi \
-            --suffix PATH : ${lib.makeBinPath (f pulumiPackages)} \
-            --set LD_LIBRARY_PATH "${lib.getLib stdenv.cc.cc}/lib"
-        '';
+    pkgs = callPackage ./plugins.nix { };
+    withPackages = callPackage ./with-packages.nix { };
     tests = {
       version = testers.testVersion {
         package = pulumi;
         version = "v${version}";
         command = "PULUMI_SKIP_UPDATE_CHECK=1 pulumi version";
       };
+      pulumiTestHookShellcheck = testers.shellcheck {
+        name = "pulumi-test-hook-shellcheck";
+        src = ./extra/pulumi-test-hook.sh;
+      };
     };
   };
 
diff --git a/pkgs/by-name/pu/pulumi/plugins.nix b/pkgs/by-name/pu/pulumi/plugins.nix
new file mode 100644
index 0000000000000..194d8483c1e56
--- /dev/null
+++ b/pkgs/by-name/pu/pulumi/plugins.nix
@@ -0,0 +1,26 @@
+{
+  lib,
+  config,
+  generateSplicesForMkScope,
+  makeScopeWithSplicing',
+  attributePathToSplice ? [ "pulumiPackages" ],
+}:
+makeScopeWithSplicing' {
+  otherSplices = generateSplicesForMkScope attributePathToSplice;
+  extra = self: {
+    mkPulumiPackage = self.callPackage ./extra/mk-pulumi-package.nix { };
+    testResourceSchema = self.callPackage ./extra/test-resource-schema.nix { };
+    pulumiTestHook = ./extra/pulumi-test-hook.sh;
+  };
+  f =
+    self:
+    lib.packagesFromDirectoryRecursive {
+      inherit (self) callPackage;
+      directory = ./plugins;
+    }
+    // lib.optionalAttrs config.allowAliases {
+      pulumi-language-go = lib.warnOnInstantiate "pulumi-language-go has been renamed to pulumi-go" self.pulumi-go;
+      pulumi-language-nodejs = lib.warnOnInstantiate "pulumi-language-nodejs has been renamed to pulumi-nodejs" self.pulumi-nodejs;
+      pulumi-language-python = lib.warnOnInstantiate "pulumi-language-python has been renamed to pulumi-python" self.pulumi-python;
+    };
+}
diff --git a/pkgs/tools/admin/pulumi-packages/pulumi-aws-native.nix b/pkgs/by-name/pu/pulumi/plugins/pulumi-aws-native/package.nix
similarity index 100%
rename from pkgs/tools/admin/pulumi-packages/pulumi-aws-native.nix
rename to pkgs/by-name/pu/pulumi/plugins/pulumi-aws-native/package.nix
diff --git a/pkgs/tools/admin/pulumi-packages/pulumi-azure-native.nix b/pkgs/by-name/pu/pulumi/plugins/pulumi-azure-native/package.nix
similarity index 100%
rename from pkgs/tools/admin/pulumi-packages/pulumi-azure-native.nix
rename to pkgs/by-name/pu/pulumi/plugins/pulumi-azure-native/package.nix
diff --git a/pkgs/tools/admin/pulumi-packages/pulumi-command.nix b/pkgs/by-name/pu/pulumi/plugins/pulumi-command/package.nix
similarity index 100%
rename from pkgs/tools/admin/pulumi-packages/pulumi-command.nix
rename to pkgs/by-name/pu/pulumi/plugins/pulumi-command/package.nix
diff --git a/pkgs/tools/admin/pulumi-packages/pulumi-language-go.nix b/pkgs/by-name/pu/pulumi/plugins/pulumi-go/package.nix
similarity index 96%
rename from pkgs/tools/admin/pulumi-packages/pulumi-language-go.nix
rename to pkgs/by-name/pu/pulumi/plugins/pulumi-go/package.nix
index eb60942775694..a7371c0e6b677 100644
--- a/pkgs/tools/admin/pulumi-packages/pulumi-language-go.nix
+++ b/pkgs/by-name/pu/pulumi/plugins/pulumi-go/package.nix
@@ -4,7 +4,7 @@
   pulumi,
 }:
 buildGoModule rec {
-  pname = "pulumi-language-go";
+  pname = "pulumi-go";
   inherit (pulumi) version src;
 
   sourceRoot = "${src.name}/sdk/go/pulumi-language-go";
diff --git a/pkgs/tools/admin/pulumi-packages/pulumi-hcloud.nix b/pkgs/by-name/pu/pulumi/plugins/pulumi-hcloud/package.nix
similarity index 100%
rename from pkgs/tools/admin/pulumi-packages/pulumi-hcloud.nix
rename to pkgs/by-name/pu/pulumi/plugins/pulumi-hcloud/package.nix
diff --git a/pkgs/tools/admin/pulumi-packages/pulumi-language-nodejs.nix b/pkgs/by-name/pu/pulumi/plugins/pulumi-nodejs/package.nix
similarity index 97%
rename from pkgs/tools/admin/pulumi-packages/pulumi-language-nodejs.nix
rename to pkgs/by-name/pu/pulumi/plugins/pulumi-nodejs/package.nix
index 835e46e43c9da..c3dd0f0502397 100644
--- a/pkgs/tools/admin/pulumi-packages/pulumi-language-nodejs.nix
+++ b/pkgs/by-name/pu/pulumi/plugins/pulumi-nodejs/package.nix
@@ -7,7 +7,7 @@
   python3,
 }:
 buildGoModule rec {
-  pname = "pulumi-language-nodejs";
+  pname = "pulumi-nodejs";
   inherit (pulumi) version src;
 
   sourceRoot = "${src.name}/sdk/nodejs/cmd/pulumi-language-nodejs";
diff --git a/pkgs/tools/admin/pulumi-packages/pulumi-language-python.nix b/pkgs/by-name/pu/pulumi/plugins/pulumi-python/package.nix
similarity index 90%
rename from pkgs/tools/admin/pulumi-packages/pulumi-language-python.nix
rename to pkgs/by-name/pu/pulumi/plugins/pulumi-python/package.nix
index 45a346cd0bf3d..74ff53ad2b08e 100644
--- a/pkgs/tools/admin/pulumi-packages/pulumi-language-python.nix
+++ b/pkgs/by-name/pu/pulumi/plugins/pulumi-python/package.nix
@@ -1,12 +1,13 @@
 {
   lib,
   buildGoModule,
+  callPackage,
   pulumi,
   bash,
   python3,
 }:
 buildGoModule rec {
-  pname = "pulumi-language-python";
+  pname = "pulumi-python";
   inherit (pulumi) version src;
 
   sourceRoot = "${src.name}/sdk/python/cmd/pulumi-language-python";
@@ -45,6 +46,8 @@ buildGoModule rec {
       ../../dist/pulumi-analyzer-policy-python
   '';
 
+  passthru.tests.smokeTest = callPackage ./smoke-test/default.nix { };
+
   meta = {
     homepage = "https://www.pulumi.com/docs/iac/languages-sdks/python/";
     description = "Language host for Pulumi programs written in Python";
diff --git a/pkgs/by-name/pu/pulumi/plugins/pulumi-python/smoke-test/Pulumi.yaml b/pkgs/by-name/pu/pulumi/plugins/pulumi-python/smoke-test/Pulumi.yaml
new file mode 100644
index 0000000000000..94072ba5e3aa2
--- /dev/null
+++ b/pkgs/by-name/pu/pulumi/plugins/pulumi-python/smoke-test/Pulumi.yaml
@@ -0,0 +1,2 @@
+name: smoke-test
+runtime: python
diff --git a/pkgs/by-name/pu/pulumi/plugins/pulumi-python/smoke-test/__main__.py b/pkgs/by-name/pu/pulumi/plugins/pulumi-python/smoke-test/__main__.py
new file mode 100644
index 0000000000000..43b7d1a8e8959
--- /dev/null
+++ b/pkgs/by-name/pu/pulumi/plugins/pulumi-python/smoke-test/__main__.py
@@ -0,0 +1,18 @@
+from binascii import b2a_hex
+from os import urandom
+
+from pulumi import export, ResourceOptions
+from pulumi.dynamic import Resource, ResourceProvider, CreateResult
+
+
+class RandomProvider(ResourceProvider):
+    def create(self, inputs):
+        return CreateResult(b2a_hex(urandom(16)), outs={})
+
+
+class Random(Resource):
+    def __init__(self, name, opts = None):
+         super().__init__(RandomProvider(), name, {}, opts)
+
+
+export("out", Random(name="random_test").id)
diff --git a/pkgs/by-name/pu/pulumi/plugins/pulumi-python/smoke-test/default.nix b/pkgs/by-name/pu/pulumi/plugins/pulumi-python/smoke-test/default.nix
new file mode 100644
index 0000000000000..903461878342e
--- /dev/null
+++ b/pkgs/by-name/pu/pulumi/plugins/pulumi-python/smoke-test/default.nix
@@ -0,0 +1,37 @@
+{
+  lib,
+  stdenvNoCC,
+  pulumiTestHook,
+  pulumi,
+  pulumi-python,
+  python3Packages,
+}:
+stdenvNoCC.mkDerivation {
+  name = "pulumi-python-smoke-test";
+  src = builtins.filterSource (name: _: !(lib.hasSuffix ".nix" name)) ./.;
+
+  doCheck = true;
+
+  nativeCheckInputs = [
+    pulumiTestHook
+    pulumi
+    pulumi-python
+    python3Packages.pulumi
+  ];
+
+  __darwinAllowLocalNetworking = true;
+
+  checkPhase = ''
+    runHook preCheck
+    pulumi update --skip-preview
+    stackOutput=$(pulumi stack output out)
+    [[ $stackOutput =~ ^[0-9a-f]{32}$ ]]
+    runHook postCheck
+  '';
+
+  installPhase = ''
+    runHook preInstall
+    mkdir -p "$out"
+    runHook postInstall
+  '';
+}
diff --git a/pkgs/tools/admin/pulumi-packages/pulumi-random.nix b/pkgs/by-name/pu/pulumi/plugins/pulumi-random/package.nix
similarity index 100%
rename from pkgs/tools/admin/pulumi-packages/pulumi-random.nix
rename to pkgs/by-name/pu/pulumi/plugins/pulumi-random/package.nix
diff --git a/pkgs/tools/admin/pulumi-packages/pulumi-yandex-unofficial.nix b/pkgs/by-name/pu/pulumi/plugins/pulumi-yandex-unofficial/package.nix
similarity index 100%
rename from pkgs/tools/admin/pulumi-packages/pulumi-yandex-unofficial.nix
rename to pkgs/by-name/pu/pulumi/plugins/pulumi-yandex-unofficial/package.nix
diff --git a/pkgs/by-name/pu/pulumi/with-packages.nix b/pkgs/by-name/pu/pulumi/with-packages.nix
new file mode 100644
index 0000000000000..68b98228fce91
--- /dev/null
+++ b/pkgs/by-name/pu/pulumi/with-packages.nix
@@ -0,0 +1,35 @@
+{
+  lib,
+  runCommand,
+  pulumi,
+  pulumiPackages,
+  makeBinaryWrapper,
+}:
+f:
+# Note that Pulumi prints a warning for “ambient” plugins (i.e. from PATH). E.g.
+#   warning: using pulumi-resource-* from $PATH at /nix/store/*
+# See also https://github.com/pulumi/pulumi/issues/14525
+#
+# We can install packages alongside pulumi executable, but that would only
+# suppress the warning for packages that are bundled with the official
+# distribution. Refer to the implementation of workspace.IsPluginBundled:
+# https://pkg.go.dev/github.com/pulumi/pulumi/sdk/v3@v3.150.0/go/common/workspace#IsPluginBundled
+# https://github.com/pulumi/pulumi/blob/v3.150.0/sdk/go/common/workspace/plugins.go#L2202-L2210
+runCommand "pulumi-with-packages"
+  {
+    inherit pulumi;
+    makeWrapperArgs = [
+      "--suffix"
+      "PATH"
+      ":"
+      (lib.makeBinPath (f pulumiPackages))
+    ];
+    __structuredAttrs = true;
+    nativeBuildInputs = [ makeBinaryWrapper ];
+    meta.mainProgram = "pulumi";
+  }
+  ''
+    mkdir -p "$out/bin"
+    ln -s -t "$out" "$pulumi/share"
+    makeWrapper "$pulumi/bin/pulumi" "$out/bin/pulumi" "''${makeWrapperArgs[@]}"
+  ''
diff --git a/pkgs/development/python-modules/pulumi/default.nix b/pkgs/development/python-modules/pulumi/default.nix
index 89ae0e6193d41..c05401d20f747 100644
--- a/pkgs/development/python-modules/pulumi/default.nix
+++ b/pkgs/development/python-modules/pulumi/default.nix
@@ -20,7 +20,7 @@
 }:
 let
   inherit (pkgs.pulumi) pname version src;
-  inherit (pulumiPackages) pulumi-language-python;
+  inherit (pulumiPackages) pulumi-python;
   sourceRoot = "${src.name}/sdk/python";
 in
 buildPythonPackage {
@@ -63,7 +63,7 @@ buildPythonPackage {
     pytest
     pytest-asyncio
     pytest-timeout
-    pulumi-language-python
+    pulumi-python
   ];
 
   # https://github.com/pulumi/pulumi/blob/0acaf8060640fdd892abccf1ce7435cd6aae69fe/sdk/python/scripts/test_fast.sh#L10-L11
diff --git a/pkgs/tools/admin/pulumi-packages/base.nix b/pkgs/tools/admin/pulumi-packages/base.nix
deleted file mode 100644
index 5c5e9c6874693..0000000000000
--- a/pkgs/tools/admin/pulumi-packages/base.nix
+++ /dev/null
@@ -1,137 +0,0 @@
-{ buildGoModule
-, fetchFromGitHub
-, python3Packages
-}:
-let
-  mkBasePackage =
-    { pname
-    , src
-    , version
-    , vendorHash
-    , cmd
-    , extraLdflags
-    , env
-    , ...
-    }@args: buildGoModule (rec {
-      inherit pname src vendorHash version env;
-
-      sourceRoot = "${src.name}/provider";
-
-      subPackages = [ "cmd/${cmd}" ];
-
-      doCheck = false;
-
-      ldflags = [
-        "-s"
-        "-w"
-      ] ++ extraLdflags;
-    } // args);
-
-  mkPythonPackage =
-    { meta
-    , pname
-    , src
-    , version
-    , ...
-    }: python3Packages.callPackage
-      ({ buildPythonPackage, pythonOlder, parver, pip, pulumi, semver, setuptools }:
-      buildPythonPackage {
-        inherit pname meta src version;
-        format = "pyproject";
-
-        disabled = pythonOlder "3.7";
-
-        sourceRoot = "${src.name}/sdk/python";
-
-        propagatedBuildInputs = [
-          parver
-          pulumi
-          semver
-          setuptools
-        ];
-
-        postPatch = ''
-          if [[ -e "pyproject.toml" ]]; then
-            sed -i \
-              -e 's/^  version = .*/  version = "${version}"/g' \
-              pyproject.toml
-          else
-            sed -i \
-               -e 's/^VERSION = .*/VERSION = "${version}"/g' \
-               -e 's/^PLUGIN_VERSION = .*/PLUGIN_VERSION = "${version}"/g' \
-               setup.py
-          fi
-        '';
-
-        # Auto-generated; upstream does not have any tests.
-        # Verify that the version substitution works
-        checkPhase = ''
-          runHook preCheck
-
-          ${pip}/bin/pip show "${pname}" | grep "Version: ${version}" > /dev/null \
-            || (echo "ERROR: Version substitution seems to be broken"; exit 1)
-
-          runHook postCheck
-        '';
-
-        pythonImportsCheck = [
-          (builtins.replaceStrings [ "-" ] [ "_" ] pname)
-        ];
-      })
-      { };
-in
-{ owner
-, repo
-, rev
-, version
-, hash
-, vendorHash
-, cmdGen
-, cmdRes
-, extraLdflags
-, env ? { }
-, meta
-, fetchSubmodules ? false
-, ...
-}@args:
-let
-  src = fetchFromGitHub {
-    name = "source-${repo}-${rev}";
-    inherit owner repo rev hash fetchSubmodules;
-  };
-
-  pulumi-gen = mkBasePackage rec {
-    inherit src version vendorHash extraLdflags env;
-
-    cmd = cmdGen;
-    pname = cmdGen;
-  };
-in
-mkBasePackage ({
-  inherit meta src version vendorHash extraLdflags env;
-
-  pname = repo;
-
-  nativeBuildInputs = [
-    pulumi-gen
-  ];
-
-  cmd = cmdRes;
-
-  postConfigure = ''
-    pushd ..
-
-    chmod +w sdk/
-    ${cmdGen} schema
-
-    popd
-
-    VERSION=v${version} go generate cmd/${cmdRes}/main.go
-  '';
-
-  passthru.sdks.python = mkPythonPackage {
-    inherit meta src version;
-
-    pname = repo;
-  };
-} // args)
diff --git a/pkgs/tools/admin/pulumi-packages/default.nix b/pkgs/tools/admin/pulumi-packages/default.nix
deleted file mode 100644
index f46dffbfdfe89..0000000000000
--- a/pkgs/tools/admin/pulumi-packages/default.nix
+++ /dev/null
@@ -1,16 +0,0 @@
-{ callPackage }:
-let
-  mkPulumiPackage = callPackage ./base.nix { };
-  callPackage' = p: args: callPackage p (args // { inherit mkPulumiPackage; });
-in
-{
-  pulumi-aws-native = callPackage' ./pulumi-aws-native.nix { };
-  pulumi-azure-native = callPackage' ./pulumi-azure-native.nix { };
-  pulumi-command = callPackage' ./pulumi-command.nix { };
-  pulumi-hcloud = callPackage' ./pulumi-hcloud.nix { };
-  pulumi-language-go = callPackage ./pulumi-language-go.nix { };
-  pulumi-language-nodejs = callPackage ./pulumi-language-nodejs.nix { };
-  pulumi-language-python = callPackage ./pulumi-language-python.nix { };
-  pulumi-random = callPackage' ./pulumi-random.nix { };
-  pulumi-yandex-unofficial = callPackage' ./pulumi-yandex-unofficial.nix { };
-}
diff --git a/pkgs/top-level/all-packages.nix b/pkgs/top-level/all-packages.nix
index 69b69cea9e581..ecd69a09d0a6e 100644
--- a/pkgs/top-level/all-packages.nix
+++ b/pkgs/top-level/all-packages.nix
@@ -4511,9 +4511,7 @@ with pkgs;
 
   pipecontrol = libsForQt5.callPackage ../applications/audio/pipecontrol { };
 
-  pulumiPackages = recurseIntoAttrs (
-    callPackage ../tools/admin/pulumi-packages { }
-  );
+  pulumiPackages = recurseIntoAttrs pulumi.pkgs;
 
   pulumi-bin = callPackage ../tools/admin/pulumi-bin { };
 


Reviews:
user: login: tie
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-03T17:05:33Z
user: login: wolfgangwalther
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-14T22:25:28Z
user: login: wolfgangwalther
body: Some nits.

Very well structured commits, very well readable.

I did not go into the details of pulumi.. because I don't know it at all.
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-14T22:37:30Z
user: login: tie
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-25T15:37:01Z
user: login: tie
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-25T15:37:15Z
user: login: teto
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-25T15:54:04Z
user: login: tie
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-25T17:29:21Z

Review comments:
path: pkgs/by-name/pu/pulumi/extra/mk-pulumi-package.nix
user: login: tie
body: Unfortunately, git does not recognize this as a rename after `nixfmt`.
created_at: 2025-03-03T17:05:26Z
author_association: MEMBER
path: pkgs/by-name/pu/pulumi/extra/mk-pulumi-package.nix
user: login: wolfgangwalther
body: I think it's fine - it's renamed in an earlier commit and formatted later. It's just the summary view in GitHub that doesn't recognize the rename. It's good as-is regarding `git blame` & co, though.
created_at: 2025-03-14T22:25:28Z
author_association: CONTRIBUTOR
diff_hunk: @@ -118,18 +117,7 @@ buildGoModule rec {
 
   passthru = {
     pkgs = pulumiPackages;
-    withPackages =
-      f:
-      runCommand "${pulumi.name}-with-packages"
-        {
-          nativeBuildInputs = [ makeWrapper ];
-        }
-        ''
-          mkdir -p $out/bin
-          makeWrapper ${pulumi}/bin/pulumi $out/bin/pulumi \
-            --suffix PATH : ${lib.makeBinPath (f pulumiPackages)} \
-            --set LD_LIBRARY_PATH "${lib.getLib stdenv.cc.cc}/lib"
path: pkgs/by-name/pu/pulumi/package.nix
user: login: wolfgangwalther
body: The LD_LIBRARY_PATH seems to be removed while splitting this into a separate file in the first commit. It's unclear whether that's on purpose, because nothing in the commit message hints at it.
created_at: 2025-03-14T22:27:56Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,13 @@
+appendToVar preCheckHooks pulumiTestHook
+
+pulumiTestHook() {
+    local tmpdir
+    tmpdir=$(mktemp -d)
+    export USER=${USER-nixbld} HOME=${HOME-$tmpdir}
+    export PULUMI_HOME=$tmpdir/.pulumi
+    export PULUMI_CONFIG_PASSPHRASE=5up3r53cr37
+    export PULUMI_SKIP_UPDATE_CHECK=1
+    export PULUMI_DISABLE_AUTOMATIC_PLUGIN_ACQUISITION=1
+    pulumi login "file://$tmpdir"
+    pulumi stack init -- "${pulumiStackName-nixpkgs}"
+}
path: pkgs/by-name/pu/pulumi/extra/pulumi-test-hook.sh
user: login: wolfgangwalther
body: We've been trying to run shellcheck on the hook files that we worked on in the last couple of months, could you do the same here? I assume it will complain about a few missing quotes.
created_at: 2025-03-14T22:34:40Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,13 @@
+appendToVar preCheckHooks pulumiTestHook
+
+pulumiTestHook() {
+    local tmpdir
+    tmpdir=$(mktemp -d)
+    export USER=${USER-nixbld} HOME=${HOME-$tmpdir}
+    export PULUMI_HOME=$tmpdir/.pulumi
+    export PULUMI_CONFIG_PASSPHRASE=5up3r53cr37
+    export PULUMI_SKIP_UPDATE_CHECK=1
+    export PULUMI_DISABLE_AUTOMATIC_PLUGIN_ACQUISITION=1
+    pulumi login "file://$tmpdir"
+    pulumi stack init -- "${pulumiStackName-nixpkgs}"
+}
path: pkgs/by-name/pu/pulumi/extra/pulumi-test-hook.sh
user: login: tie
body: Done. Quotes are fine. Note that this hook is defined in [`extra` argument](https://github.com/NixOS/nixpkgs/blob/7105ae3957700a9646cc4b766f5815b23ed0c682/lib/customisation.nix#L623-L627) of `makeScopeWithSplicing'` and therefore is not exposed in the resulting package set, so I’ve added `testers.shellcheck` to the `pulumi.passthru.tests`.
created_at: 2025-03-25T15:37:01Z
author_association: MEMBER
diff_hunk: @@ -118,18 +117,7 @@ buildGoModule rec {
 
   passthru = {
     pkgs = pulumiPackages;
-    withPackages =
-      f:
-      runCommand "${pulumi.name}-with-packages"
-        {
-          nativeBuildInputs = [ makeWrapper ];
-        }
-        ''
-          mkdir -p $out/bin
-          makeWrapper ${pulumi}/bin/pulumi $out/bin/pulumi \
-            --suffix PATH : ${lib.makeBinPath (f pulumiPackages)} \
-            --set LD_LIBRARY_PATH "${lib.getLib stdenv.cc.cc}/lib"
path: pkgs/by-name/pu/pulumi/package.nix
user: login: tie
body: Updated commit message. Pulumi and its plugins are self-contained executables, so it does not make sense to set `LD_LIBRARY_PATH`, even for plugins acquired from the Pulumi package registry.
created_at: 2025-03-25T15:37:15Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,35 @@
+{
+  lib,
+  runCommand,
+  pulumi,
+  pulumiPackages,
+  makeBinaryWrapper,
+}:
+f:
+# Note that Pulumi prints a warning for “ambient” plugins (i.e. from PATH). E.g.
+#   warning: using pulumi-resource-* from $PATH at /nix/store/*
+# See also https://github.com/pulumi/pulumi/issues/14525
+#
+# We can install packages alongside pulumi executable, but that would only
+# suppress the warning for packages that are bundled with the official
+# distribution. Refer to the implementation of workspace.IsPluginBundled:
+# https://pkg.go.dev/github.com/pulumi/pulumi/sdk/v3@v3.150.0/go/common/workspace#IsPluginBundled
+# https://github.com/pulumi/pulumi/blob/v3.150.0/sdk/go/common/workspace/plugins.go#L2202-L2210
path: pkgs/by-name/pu/pulumi/with-packages.nix
user: login: teto
body: It's unclear to me what you are trying to say: so we still get the warning.
instead of symlinking we could copy the executable.
It would be neat to have this in a small `doc/languages-frameworks/pulumi.section.md`, it could start with just 3 lines showing an example of `pulumi.withPackages`.
This can be done later by whoever. Just that you look motivated :)
created_at: 2025-03-25T15:54:04Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,35 @@
+{
+  lib,
+  runCommand,
+  pulumi,
+  pulumiPackages,
+  makeBinaryWrapper,
+}:
+f:
+# Note that Pulumi prints a warning for “ambient” plugins (i.e. from PATH). E.g.
+#   warning: using pulumi-resource-* from $PATH at /nix/store/*
+# See also https://github.com/pulumi/pulumi/issues/14525
+#
+# We can install packages alongside pulumi executable, but that would only
+# suppress the warning for packages that are bundled with the official
+# distribution. Refer to the implementation of workspace.IsPluginBundled:
+# https://pkg.go.dev/github.com/pulumi/pulumi/sdk/v3@v3.150.0/go/common/workspace#IsPluginBundled
+# https://github.com/pulumi/pulumi/blob/v3.150.0/sdk/go/common/workspace/plugins.go#L2202-L2210
path: pkgs/by-name/pu/pulumi/with-packages.nix
user: login: tie
body: > instead of symlinking we could copy the executable.

Please see the referenced implementation. Plugins alongside the executable are used only if `IsPluginBundled` returns true.

https://github.com/pulumi/pulumi/blob/v3.150.0/sdk/go/common/workspace/plugins.go#L2202-L2210
https://github.com/pulumi/pulumi/blob/v3.150.0/sdk/go/common/workspace/plugins.go#L2063-L2073

In other cases, `bundledPath` is an empty string, so a result of `PATH` lookup is used.

That is, the suggested workaround in https://github.com/pulumi/pulumi/issues/14525#issuecomment-2322723229 and https://github.com/NixOS/nixpkgs/issues/376137 (i.e. copying instead of symlinking) does not work for plugins that are not bundled with the official distribution.

Instead, I think it’d be reasonable to support e.g. `filepath.SplitList(os.Getenv("PULUMI_BUNDLED_PLUGINS_PATH"))` lookup in Pulumi to override this behavior. But for now, the warning about `PATH` lookup is harmless.
created_at: 2025-03-25T17:29:21Z
author_association: MEMBER

PR comments:
user: login: teto
created_at: 2025-03-25T15:55:09Z
body: This looks ok and you have wolfgang's approval which counts for a lot :) I will try it tomorrow and let's merge before 25.05
author_association: MEMBER
user: login: tie
created_at: 2025-03-26T00:59:54Z
body: > I will try it tomorrow and let's merge before 25.05

Note that there will be 2–3 follow-up PRs with version bumps, update script for Pulumi and more plugin packages (see my https://github.com/tie/nixpkgs/tree/pulumi-bump3 branch).
author_association: MEMBER
user: login: wolfgangwalther
created_at: 2025-03-26T17:56:31Z
body: ## `nixpkgs-review` result

Generated using [`nixpkgs-review`](https://github.com/Mic92/nixpkgs-review).

Command: `nixpkgs-review pr 386789`

---
### `x86_64-linux`
<details>
  <summary>:fast_forward: 20 packages marked as broken and skipped:</summary>
  <ul>
    <li>python312Packages.pulumi</li>
    <li>python312Packages.pulumi-aws</li>
    <li>python312Packages.pulumi-aws-native</li>
    <li>python312Packages.pulumi-aws-native.dist</li>
    <li>python312Packages.pulumi-aws.dist</li>
    <li>python312Packages.pulumi-azure-native</li>
    <li>python312Packages.pulumi-azure-native.dist</li>
    <li>python312Packages.pulumi-command</li>
    <li>python312Packages.pulumi-command.dist</li>
    <li>python312Packages.pulumi-hcloud</li>
    <li>python312Packages.pulumi-hcloud.dist</li>
    <li>python312Packages.pulumi-random</li>
    <li>python312Packages.pulumi-random.dist</li>
    <li>python312Packages.pulumi.dev</li>
    <li>python312Packages.pulumi.dist</li>
    <li>python313Packages.pulumi</li>
    <li>python313Packages.pulumi-aws</li>
    <li>python313Packages.pulumi-aws.dist</li>
    <li>python313Packages.pulumi.dev</li>
    <li>python313Packages.pulumi.dist</li>
  </ul>
</details>
<details>
  <summary>:white_check_mark: 3 packages built:</summary>
  <ul>
    <li>pulumiPackages.pulumi-go</li>
    <li>pulumiPackages.pulumi-nodejs</li>
    <li>pulumiPackages.pulumi-python</li>
  </ul>
</details>

---
### `aarch64-linux`
<details>
  <summary>:fast_forward: 20 packages marked as broken and skipped:</summary>
  <ul>
    <li>python312Packages.pulumi</li>
    <li>python312Packages.pulumi-aws</li>
    <li>python312Packages.pulumi-aws-native</li>
    <li>python312Packages.pulumi-aws-native.dist</li>
    <li>python312Packages.pulumi-aws.dist</li>
    <li>python312Packages.pulumi-azure-native</li>
    <li>python312Packages.pulumi-azure-native.dist</li>
    <li>python312Packages.pulumi-command</li>
    <li>python312Packages.pulumi-command.dist</li>
    <li>python312Packages.pulumi-hcloud</li>
    <li>python312Packages.pulumi-hcloud.dist</li>
    <li>python312Packages.pulumi-random</li>
    <li>python312Packages.pulumi-random.dist</li>
    <li>python312Packages.pulumi.dev</li>
    <li>python312Packages.pulumi.dist</li>
    <li>python313Packages.pulumi</li>
    <li>python313Packages.pulumi-aws</li>
    <li>python313Packages.pulumi-aws.dist</li>
    <li>python313Packages.pulumi.dev</li>
    <li>python313Packages.pulumi.dist</li>
  </ul>
</details>
<details>
  <summary>:white_check_mark: 3 packages built:</summary>
  <ul>
    <li>pulumiPackages.pulumi-go</li>
    <li>pulumiPackages.pulumi-nodejs</li>
    <li>pulumiPackages.pulumi-python</li>
  </ul>
</details>

---
### `x86_64-darwin`
<details>
  <summary>:fast_forward: 20 packages marked as broken and skipped:</summary>
  <ul>
    <li>python312Packages.pulumi</li>
    <li>python312Packages.pulumi-aws</li>
    <li>python312Packages.pulumi-aws-native</li>
    <li>python312Packages.pulumi-aws-native.dist</li>
    <li>python312Packages.pulumi-aws.dist</li>
    <li>python312Packages.pulumi-azure-native</li>
    <li>python312Packages.pulumi-azure-native.dist</li>
    <li>python312Packages.pulumi-command</li>
    <li>python312Packages.pulumi-command.dist</li>
    <li>python312Packages.pulumi-hcloud</li>
    <li>python312Packages.pulumi-hcloud.dist</li>
    <li>python312Packages.pulumi-random</li>
    <li>python312Packages.pulumi-random.dist</li>
    <li>python312Packages.pulumi.dev</li>
    <li>python312Packages.pulumi.dist</li>
    <li>python313Packages.pulumi</li>
    <li>python313Packages.pulumi-aws</li>
    <li>python313Packages.pulumi-aws.dist</li>
    <li>python313Packages.pulumi.dev</li>
    <li>python313Packages.pulumi.dist</li>
  </ul>
</details>
<details>
  <summary>:white_check_mark: 3 packages built:</summary>
  <ul>
    <li>pulumiPackages.pulumi-go</li>
    <li>pulumiPackages.pulumi-nodejs</li>
    <li>pulumiPackages.pulumi-python</li>
  </ul>
</details>

---
### `aarch64-darwin`
<details>
  <summary>:fast_forward: 20 packages marked as broken and skipped:</summary>
  <ul>
    <li>python312Packages.pulumi</li>
    <li>python312Packages.pulumi-aws</li>
    <li>python312Packages.pulumi-aws-native</li>
    <li>python312Packages.pulumi-aws-native.dist</li>
    <li>python312Packages.pulumi-aws.dist</li>
    <li>python312Packages.pulumi-azure-native</li>
    <li>python312Packages.pulumi-azure-native.dist</li>
    <li>python312Packages.pulumi-command</li>
    <li>python312Packages.pulumi-command.dist</li>
    <li>python312Packages.pulumi-hcloud</li>
    <li>python312Packages.pulumi-hcloud.dist</li>
    <li>python312Packages.pulumi-random</li>
    <li>python312Packages.pulumi-random.dist</li>
    <li>python312Packages.pulumi.dev</li>
    <li>python312Packages.pulumi.dist</li>
    <li>python313Packages.pulumi</li>
    <li>python313Packages.pulumi-aws</li>
    <li>python313Packages.pulumi-aws.dist</li>
    <li>python313Packages.pulumi.dev</li>
    <li>python313Packages.pulumi.dist</li>
  </ul>
</details>
<details>
  <summary>:white_check_mark: 3 packages built:</summary>
  <ul>
    <li>pulumiPackages.pulumi-go</li>
    <li>pulumiPackages.pulumi-nodejs</li>
    <li>pulumiPackages.pulumi-python</li>
  </ul>
</details>
author_association: CONTRIBUTOR
user: login: wolfgangwalther
created_at: 2025-03-26T18:13:35Z
body: > Note that Pulumi requires `protobuf4` for Python packages. Build all Pulumi packages, SDKs and tests with:

I am running the snippet from the description on x86_64-linux and I get this:

```
       > Executing pythonRuntimeDepsCheck
       > Checking runtime dependencies for pulumi-3.156.0-py3-none-any.whl
       >   - pytest-watch not installed
       For full logs, run 'nix log /nix/store/ry4rkvq95w12sryzjcngcj6ayazhfs7a-python3.12-pulumi-3.156.0.drv'.
```
author_association: CONTRIBUTOR
user: login: tie
created_at: 2025-03-26T22:03:31Z
body: > I am running the snippet from the description on x86_64-linux and I get this:

Bug in Pulumi Python package, see https://github.com/pulumi/pulumi/issues/18965. I’ll be adding an update script in the next PR along with a version bump to fix this.
author_association: MEMBER
user: login: teto
created_at: 2025-03-27T10:44:16Z
body: Do you have the next PR ready ? I consider merging this one since it's already broken but it would be reassuring to see the follow up PR such that I could test it directly.
author_association: MEMBER
user: login: tie
created_at: 2025-03-27T13:49:57Z
body: @teto created https://github.com/NixOS/nixpkgs/pull/393740 for python3Packages.pulumi fix.
Plugin version bumps and new plugins are available in https://github.com/tie/nixpkgs/tree/pulumi-bump3 branch (~15 commits total, not counting the PR above).
author_association: MEMBER