Repository: dependabot/dependabot-core
Pull request #12226: Use private registries when updating uv lockfile dependencies
Author: login: michael-booth
Labels: name: L: python:uv
Created: 2025-05-08T10:41:02Z | Merged: 2025-05-21T21:46:37Z

### What are you trying to accomplish?

Addresses: https://github.com/dependabot/dependabot-core/issues/11889

Private registries specified by user config are included as indexes/default-indexes in the `uv lock` command allowing dependencies to be resolved from sources other than the default public registry


Accomplished by
* Generating lock command options based on python-index credentials provivded
* Passing lock command options to pre-existing `uv lock` execution.


### Anything you want to highlight for special attention from reviewers?

It appears the uv code can already determine the latest version of a dependency hosted on a private registry, it just doesn't pass the relevant indexes during `uv lock` (per example in linked issue)

Implementation is essentially based off of [uv/file_updater/compile_file_updater.rb](https://github.com/dependabot/dependabot-core/blob/main/uv/lib/dependabot/uv/file_updater/compile_file_updater.rb#L412-L431)

I was unsure of the level of testing required here, feedback if you need more than has been provided is appreciated!

I've also added typing to this file as it was required for a passing CI. Changes have been made in a separate commit.

### How will you know you've accomplished your goal?

<!--
  * If you've reproduced an error, can you link to, or demonstrate the reproduction?
  * If you've added a new feature, how will you demonstrate it to others?
  * If you've refactored code, how will you demonstrate that the new code is functionally equivalent to the old code?
-->

### Checklist

<!-- Before requesting review, please ensure that your pull request fulfills the following requirements: -->

- [x] I have run the complete test suite to ensure all tests and linters pass.
- [x] I have thoroughly tested my code changes to ensure they work as expected, including adding additional tests for new functionality.
- [x] I have written clear and descriptive commit messages.
- [x] I have provided a detailed description of the changes in the pull request, including the problem it addresses, how it fixes the problem, and any relevant details about the implementation.
- [x] I have ensured that the code is well-documented and easy to understand.



Diff:
diff --git a/uv/lib/dependabot/uv/file_updater/lock_file_updater.rb b/uv/lib/dependabot/uv/file_updater/lock_file_updater.rb
index dfc86d74f5c..9a03edb371b 100644
--- a/uv/lib/dependabot/uv/file_updater/lock_file_updater.rb
+++ b/uv/lib/dependabot/uv/file_updater/lock_file_updater.rb
@@ -1,4 +1,4 @@
-# typed: true
+# typed: strict
 # frozen_string_literal: true
 
 require "toml-rb"
@@ -17,33 +17,56 @@ module Dependabot
   module Uv
     class FileUpdater
       class LockFileUpdater
+        extend T::Sig
         require_relative "pyproject_preparer"
 
         REQUIRED_FILES = %w(pyproject.toml uv.lock).freeze # At least one of these files should be present
 
+        sig { returns(T::Array[Dependency]) }
         attr_reader :dependencies
+
+        sig { returns(T::Array[DependencyFile]) }
         attr_reader :dependency_files
+
+        sig { returns(T::Array[Dependabot::Credential]) }
         attr_reader :credentials
+
+        sig { returns(T.nilable(T::Array[String])) }
         attr_reader :index_urls
 
+        sig do
+          params(
+            dependencies: T::Array[Dependency],
+            dependency_files: T::Array[DependencyFile],
+            credentials: T::Array[Dependabot::Credential],
+            index_urls: T.nilable(T::Array[String])
+          ).void
+        end
         def initialize(dependencies:, dependency_files:, credentials:, index_urls: nil)
           @dependencies = dependencies
           @dependency_files = dependency_files
           @credentials = credentials
           @index_urls = index_urls
+          @prepared_pyproject = T.let(nil, T.nilable(String))
+          @updated_lockfile_content = T.let(nil, T.nilable(String))
+          @pyproject = T.let(nil, T.nilable(Dependabot::DependencyFile))
         end
 
+        sig { returns(T::Array[Dependabot::DependencyFile]) }
         def updated_dependency_files
-          @updated_dependency_files ||= fetch_updated_dependency_files
+          @updated_dependency_files ||= T.let(fetch_updated_dependency_files,
+                                              T.nilable(T::Array[Dependabot::DependencyFile]))
         end
 
         private
 
+        sig { returns(T.nilable(Dependabot::Dependency)) }
         def dependency
           # For now, we'll only ever be updating a single dependency
-          dependencies.first
+          T.must(dependencies.first)
         end
 
+        sig { returns(T::Array[Dependabot::DependencyFile]) }
         def fetch_updated_dependency_files
           return [] unless create_or_update_lock_file?
 
@@ -52,32 +75,33 @@ def fetch_updated_dependency_files
           if file_changed?(pyproject)
             updated_files <<
               updated_file(
-                file: pyproject,
-                content: updated_pyproject_content
+                file: T.must(pyproject),
+                content: T.must(updated_pyproject_content)
               )
           end
 
           if lockfile
             # Use updated_lockfile_content which might raise if the lockfile doesn't change
             new_content = updated_lockfile_content
-            raise "Expected lockfile to change!" if lockfile.content == new_content
+            raise "Expected lockfile to change!" if T.must(lockfile).content == new_content
 
-            updated_files << updated_file(file: lockfile, content: new_content)
+            updated_files << updated_file(file: T.must(lockfile), content: new_content)
           end
 
           updated_files
         end
 
+        sig { returns(T.nilable(String)) }
         def updated_pyproject_content
-          content = pyproject.content
-          return content unless file_changed?(pyproject)
+          content = T.must(pyproject).content
+          return content unless file_changed?(T.must(pyproject))
 
           updated_content = content.dup
 
-          dependency.requirements.zip(dependency.previous_requirements).each do |new_r, old_r|
-            next unless new_r[:file] == pyproject.name && old_r[:file] == pyproject.name
+          T.must(dependency).requirements.zip(T.must(T.must(dependency).previous_requirements)).each do |new_r, old_r|
+            next unless new_r[:file] == T.must(pyproject).name && T.must(old_r)[:file] == T.must(pyproject).name
 
-            updated_content = replace_dep(dependency, updated_content, new_r, old_r)
+            updated_content = replace_dep(T.must(dependency), T.must(updated_content), new_r, T.must(old_r))
           end
 
           raise DependencyFileContentNotChanged, "Content did not change!" if content == updated_content
@@ -85,6 +109,14 @@ def updated_pyproject_content
           updated_content
         end
 
+        sig do
+          params(
+            dep: Dependabot::Dependency,
+            content: String,
+            new_r: T::Hash[Symbol, T.untyped],
+            old_r: T::Hash[Symbol, T.untyped]
+          ).returns(String)
+        end
         def replace_dep(dep, content, new_r, old_r)
           new_req = new_r[:requirement]
           old_req = old_r[:requirement]
@@ -93,28 +125,29 @@ def replace_dep(dep, content, new_r, old_r)
           declaration_match = content.match(declaration_regex)
           if declaration_match
             declaration = declaration_match[:declaration]
-            new_declaration = declaration.sub(old_req, new_req)
-            content.sub(declaration, new_declaration)
+            new_declaration = T.must(declaration).sub(old_req, new_req)
+            content.sub(T.must(declaration), new_declaration)
           else
             content
           end
         end
 
+        sig { returns(String) }
         def updated_lockfile_content
           @updated_lockfile_content ||=
             begin
-              original_content = lockfile.content
+              original_content = T.must(lockfile).content
               # Extract the original requires-python value to preserve it
-              original_requires_python = original_content
-                                         .match(/requires-python\s*=\s*["']([^"']+)["']/)&.captures&.first
+              original_requires_python = T.must(original_content)
+                                          .match(/requires-python\s*=\s*["']([^"']+)["']/)&.captures&.first
 
               # Store the original Python version requirement for later use
-              @original_python_version = original_requires_python
+              @original_python_version = T.let(original_requires_python, T.nilable(String))
 
               new_lockfile = updated_lockfile_content_for(prepared_pyproject)
 
               # Normalize line endings to ensure proper comparison
-              new_lockfile = normalize_line_endings(new_lockfile, original_content)
+              new_lockfile = normalize_line_endings(new_lockfile, T.must(original_content))
 
               result = new_lockfile
 
@@ -129,6 +162,7 @@ def updated_lockfile_content
         end
 
         # Helper method to normalize line endings between two strings
+        sig { params(content: String, reference: String).returns(String) }
         def normalize_line_endings(content, reference)
           # Check if reference has escaped newlines like "\n" +
           if reference.include?("\\n")
@@ -138,33 +172,24 @@ def normalize_line_endings(content, reference)
           end
         end
 
-        def with_original_python_version(original_requires_python)
-          if original_requires_python
-            original_python_version = @original_python_version
-            @original_python_version = original_requires_python
-            result = yield
-            @original_python_version = original_python_version
-            result
-          else
-            yield
-          end
-        end
-
+        sig { returns(String) }
         def prepared_pyproject
           @prepared_pyproject ||=
             begin
               content = updated_pyproject_content
-              content = sanitize(content)
+              content = sanitize(T.must(content))
               content
             end
         end
 
+        sig { params(pyproject_content: String).returns(String) }
         def sanitize(pyproject_content)
           PyprojectPreparer
             .new(pyproject_content: pyproject_content)
             .sanitize
         end
 
+        sig { params(pyproject_content: String).returns(String) }
         def updated_lockfile_content_for(pyproject_content)
           SharedHelpers.in_a_temporary_directory do
             SharedHelpers.with_git_configured(credentials: credentials) do
@@ -180,19 +205,25 @@ def updated_lockfile_content_for(pyproject_content)
           end
         end
 
+        sig { returns(T.nilable(String)) }
         def run_update_command
+          options = lock_options
+          options_fingerprint = lock_options_fingerprint(options)
+
           # Use pyenv exec to ensure we're using the correct Python environment
-          command = "pyenv exec uv lock --upgrade-package #{dependency.name}"
-          fingerprint = "pyenv exec uv lock --upgrade-package <dependency_name>"
+          command = "pyenv exec uv lock --upgrade-package #{T.must(dependency).name} #{options}"
+          fingerprint = "pyenv exec uv lock --upgrade-package <dependency_name> #{options_fingerprint}"
 
           run_command(command, fingerprint:)
         end
 
+        sig { params(command: String, fingerprint: T.nilable(String)).returns(String) }
         def run_command(command, fingerprint: nil)
           Dependabot.logger.info("Running command: #{command}")
           SharedHelpers.run_shell_command(command, fingerprint: fingerprint)
         end
 
+        sig { params(pyproject_content: String).returns(Integer) }
         def write_temporary_dependency_files(pyproject_content)
           dependency_files.each do |file|
             path = file.name
@@ -204,6 +235,7 @@ def write_temporary_dependency_files(pyproject_content)
           File.write("pyproject.toml", pyproject_content)
         end
 
+        sig { void }
         def setup_python_environment
           # Use LanguageVersionManager to determine and install the appropriate Python version
           Dependabot.logger.info("Setting up Python environment using LanguageVersionManager")
@@ -225,10 +257,12 @@ def setup_python_environment
           end
         end
 
+        sig { params(url: String).returns(String) }
         def sanitize_env_name(url)
           url.gsub(%r{^https?://}, "").gsub(/[^a-zA-Z0-9]/, "_").upcase
         end
 
+        sig { params(dep: T.untyped, old_req: T.untyped).returns(Regexp) }
         def declaration_regex(dep, old_req)
           escaped_name = Regexp.escape(dep.name)
           # Extract the requirement operator and version
@@ -246,10 +280,43 @@ def declaration_regex(dep, old_req)
           /x
         end
 
+        sig { returns(String) }
+        def lock_options
+          options = lock_index_options
+
+          options.join(" ")
+        end
+
+        sig { returns(T::Array[String]) }
+        def lock_index_options
+          credentials
+            .select { |cred| cred["type"] == "python_index" }
+            .map do |cred|
+            authed_url = AuthedUrlBuilder.authed_url(credential: cred)
+
+            if cred.replaces_base?
+              "--default-index #{authed_url}"
+            else
+              "--index #{authed_url}"
+            end
+          end
+        end
+
+        sig { params(options: String).returns(String) }
+        def lock_options_fingerprint(options)
+          options.sub(
+            /--default-index\s+\S+/, "--default-index <default_index>"
+          ).sub(
+            /--index\s+\S+/, "--index <index>"
+          )
+        end
+
+        sig { params(name: T.any(String, Symbol)).returns(String) }
         def escape(name)
           Regexp.escape(name).gsub("\\-", "[-_.]")
         end
 
+        sig { params(file: T.nilable(DependencyFile)).returns(T::Boolean) }
         def file_changed?(file)
           return false unless file
 
@@ -259,56 +326,71 @@ def file_changed?(file)
           end
         end
 
+        sig do
+          params(file: T.nilable(DependencyFile), dependency: Dependency)
+            .returns(T::Boolean)
+        end
         def requirement_changed?(file, dependency)
           changed_requirements =
-            dependency.requirements - dependency.previous_requirements
+            dependency.requirements - T.must(dependency.previous_requirements)
 
-          changed_requirements.any? { |f| f[:file] == file.name }
+          changed_requirements.any? { |f| f[:file] == T.must(file).name }
         end
 
+        sig { params(file: Dependabot::DependencyFile, content: String).returns(Dependabot::DependencyFile) }
         def updated_file(file:, content:)
           updated_file = file.dup
           updated_file.content = content
           updated_file
         end
 
+        sig { params(name: String).returns(String) }
         def normalise(name)
           NameNormaliser.normalise(name)
         end
 
+        sig { returns(Dependabot::Uv::FileParser::PythonRequirementParser) }
         def python_requirement_parser
-          @python_requirement_parser ||=
+          @python_requirement_parser ||= T.let(
             FileParser::PythonRequirementParser.new(
               dependency_files: dependency_files
-            )
+            ), T.nilable(FileParser::PythonRequirementParser)
+          )
         end
 
+        sig { returns(Dependabot::Uv::LanguageVersionManager) }
         def language_version_manager
-          @language_version_manager ||=
+          @language_version_manager ||= T.let(
             LanguageVersionManager.new(
               python_requirement_parser: python_requirement_parser
-            )
+            ), T.nilable(LanguageVersionManager)
+          )
         end
 
+        sig { returns(T.nilable(Dependabot::DependencyFile)) }
         def pyproject
-          @pyproject ||=
-            dependency_files.find { |f| f.name == "pyproject.toml" }
+          @pyproject ||= T.let(dependency_files.find { |f| f.name == "pyproject.toml" },
+                               T.nilable(Dependabot::DependencyFile))
         end
 
+        sig { returns(T.nilable(Dependabot::DependencyFile)) }
         def lockfile
-          @lockfile ||= uv_lock
+          @lockfile ||= T.let(uv_lock, T.nilable(Dependabot::DependencyFile))
         end
 
+        sig { returns(String) }
         def python_helper_path
           NativeHelpers.python_helper_path
         end
 
+        sig { returns(T.nilable(Dependabot::DependencyFile)) }
         def uv_lock
           dependency_files.find { |f| f.name == "uv.lock" }
         end
 
+        sig { returns(T::Boolean) }
         def create_or_update_lock_file?
-          dependency.requirements.select { _1[:file].end_with?(*REQUIRED_FILES) }.any?
+          T.must(dependency).requirements.select { _1[:file].end_with?(*REQUIRED_FILES) }.any?
         end
       end
     end
diff --git a/uv/spec/dependabot/uv/file_updater/lock_file_updater_spec.rb b/uv/spec/dependabot/uv/file_updater/lock_file_updater_spec.rb
index c3b1105a4cc..9e469f7490f 100644
--- a/uv/spec/dependabot/uv/file_updater/lock_file_updater_spec.rb
+++ b/uv/spec/dependabot/uv/file_updater/lock_file_updater_spec.rb
@@ -251,4 +251,83 @@
       expect(match[:declaration]).to eq("redis~=4.5.4")
     end
   end
+
+  describe "#lock_index_options" do
+    subject(:lock_index_options) { updater.send(:lock_index_options) }
+
+    let(:credentials) do
+      [
+        Dependabot::Credential.new({
+          "type" => "python_index",
+          "index-url" => "https://example.com/simple",
+          "token" => "token",
+          "replaces-base" => false
+        }),
+        Dependabot::Credential.new({
+          "type" => "python_index",
+          "index-url" => "https://another.com/simple",
+          "token" => "another_token",
+          "replaces-base" => true
+        })
+      ]
+    end
+
+    it "matches authed urls to correct option index flags" do
+      expect(lock_index_options).to include("--default-index https://another_token@another.com/simple")
+      expect(lock_index_options).to include("--index https://token@example.com/simple")
+    end
+  end
+
+  describe "#lock_options_fingerprint" do
+    subject(:lock_options_fingerprint) { updater.send(:lock_options_fingerprint, options) }
+
+    let(:options) do
+      "--default-index https://another.com/simple --index https://example.com/simple"
+    end
+
+    it "replaces sensitive information in the fingerprint with placeholders" do
+      expect(lock_options_fingerprint).to eq("--default-index <default_index> --index <index>")
+    end
+  end
+
+  describe "#run_update_command" do
+    subject(:run_update_command) { updater.send(:run_update_command) }
+
+    let(:credentials) do
+      [
+        Dependabot::Credential.new({
+          "type" => "python_index",
+          "index-url" => "https://example.com/simple",
+          "token" => "token",
+          "replaces-base" => false
+        }),
+        Dependabot::Credential.new({
+          "type" => "python_index",
+          "index-url" => "https://another.com/simple",
+          "token" => "another_token",
+          "replaces-base" => true
+        })
+      ]
+    end
+
+    before do
+      allow(updater).to receive(:run_command)
+    end
+
+    it "includes the expected options in the command and fingerprint" do
+      expected_command = "pyenv exec uv lock --upgrade-package requests " \
+                         "--index https://token@example.com/simple " \
+                         "--default-index https://another_token@another.com/simple"
+      expected_fingerprint = "pyenv exec uv lock --upgrade-package <dependency_name> " \
+                             "--index <index> " \
+                             "--default-index <default_index>"
+
+      run_update_command
+
+      expect(updater).to have_received(:run_command).with(
+        expected_command,
+        fingerprint: expected_fingerprint
+      )
+    end
+  end
 end


Reviews:
user: login: michael-booth
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-08T10:45:07Z
user: login: michael-booth
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-12T09:02:58Z
user: login: randhircs
body: I approve it.
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-21T20:35:41Z

Review comments:
diff_hunk: @@ -246,6 +249,36 @@ def declaration_regex(dep, old_req)
           /x
         end
 
+        def lock_options
+          options = lock_index_options
+          puts options.inspect
+
+          options.join(" ")
+        end
+
+        def lock_index_options
+          puts credentials.inspect
+          credentials
+            .select { |cred| cred["type"] == "python_index" }
+            .map do |cred|
+            authed_url = AuthedUrlBuilder.authed_url(credential: cred)
+
+            if cred.replaces_base?
+              "--default-index #{authed_url}"
+            else
+              "--index #{authed_url}"
path: uv/lib/dependabot/uv/file_updater/lock_file_updater.rb
user: login: michael-booth
body: Flag selection based off of the `uv lock` help description

```
--index <INDEX>                        The URLs to use when resolving dependencies, in addition to the default index [env: UV_INDEX=]
--default-index <DEFAULT_INDEX>        The URL of the default package index (by default: <https://pypi.org/simple>) [env: UV_DEFAULT_INDEX=]
```
created_at: 2025-05-08T10:45:07Z
author_association: CONTRIBUTOR
diff_hunk: @@ -138,33 +172,24 @@ def normalize_line_endings(content, reference)
           end
         end
 
-        def with_original_python_version(original_requires_python)
path: uv/lib/dependabot/uv/file_updater/lock_file_updater.rb
user: login: michael-booth
body: This function was causing some issues when applying typing. It appears entirely unused within the code base so i opted to remove it instead. Happy to take some steer if you have a preferred approach
created_at: 2025-05-12T09:02:58Z
author_association: CONTRIBUTOR

PR comments:
user: login: michael-booth
created_at: 2025-05-08T12:23:43Z
body: Currently the sorbet check is failing (because the modified file has no typing applied as far as i can see).
Is it desirable for me to apply some basic type annotations?
author_association: CONTRIBUTOR
user: login: abdulapopoola
created_at: 2025-05-09T15:42:38Z
body: > Currently the sorbet check is failing (because the modified file has no typing applied as far as i can see). Is it desirable for me to apply some basic type annotations?

Yes please
author_association: CONTRIBUTOR
user: login: michael-booth
created_at: 2025-05-12T09:11:56Z
body: Thanks @abdulapopoola - i've pushed an additional commit which adds typing (to the best of my ability) to the file.
CI checks all passing so hopefully this is in a good place for someone to have a look. Thanks!
author_association: CONTRIBUTOR