internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: pantsbuild/pants

Issue:
TODO: Document synthetic targets API
What it says on the tin.


Discussion:

Resolving patch:
diff --git a/src/python/pants/engine/internals/synthetic_targets.py b/src/python/pants/engine/internals/synthetic_targets.py
--- a/src/python/pants/engine/internals/synthetic_targets.py
+++ b/src/python/pants/engine/internals/synthetic_targets.py
@@ -22,7 +22,6 @@
     class SyntheticExampleTargetsRequest(SyntheticTargetsRequest):
         path: str = SyntheticTargetsRequest.REQUEST_TARGETS_PER_DIRECTORY
         path: str = SyntheticTargetsRequest.SINGLE_REQUEST_FOR_ALL_TARGETS
-        path: str = "fixed/path/..."  # Get a single request for named directory only.
 
 
     @rule


Test patch:
Repository: pantsbuild/pants

Issue:
Buf 1.9 isn't compatible with the buf backend
Running Buf 1.9 results in the following error during `pants fmt`:
```
Failure: exec: "diff": cannot run executable found relative to current directory
```

From the [Go docs](https://pkg.go.dev/os/exec@master):
> To avoid those security problems, as of Go 1.19, this package will not resolve a program using an implicit or explicit path entry relative to the current directory. 

`buf format` needs `diff` in order to run, and our buf backend uses `BinaryShims` to get `diff` into the sandbox. Taking a look at `BinaryShims.bin_directory`, which we add to `PATH` before running `buf`, it is very much a relative path.


The easy fix would be to make `BinaryShims.bin_directory` an absolute path instead, but we don't know about the sandbox and its path at the time `bin_directory` is set. Instead we can patch the sandbox similar to the [`GoSdkProcess` rule](https://github.com/pantsbuild/pants/blob/abd3bfecea3cd9b35d310684f973e96dc04df721/src/python/pants/backend/go/util_rules/sdk.py#L89-L92) to make `PATH` absolute.


Discussion:
> The easy fix would be to make BinaryShims.bin_directory an absolute path instead, but we don't know about the sandbox and its path at the time bin_directory is set. Instead we can patch the sandbox similar to the [GoSdkProcess rule](https://github.com/pantsbuild/pants/blob/abd3bfecea3cd9b35d310684f973e96dc04df721/src/python/pants/backend/go/util_rules/sdk.py#L89-L92) to make PATH absolute.

Also, even if Pants knows the absolute path to the local sandbox directory for local execution, Pants will not know the absolute path of the "input root" directory used with remote execution (except in the actual wrapper script running in the remote environment).

Resolving patch:
diff --git a/src/python/pants/backend/codegen/protobuf/lint/buf/format_rules.py b/src/python/pants/backend/codegen/protobuf/lint/buf/format_rules.py
--- a/src/python/pants/backend/codegen/protobuf/lint/buf/format_rules.py
+++ b/src/python/pants/backend/codegen/protobuf/lint/buf/format_rules.py
@@ -1,5 +1,6 @@
 # Copyright 2022 Pants project contributors (see CONTRIBUTORS.md).
 # Licensed under the Apache License, Version 2.0 (see LICENSE).
+import os
 from dataclasses import dataclass
 
 from pants.backend.codegen.protobuf.lint.buf.skip_field import SkipBufFormatField
@@ -98,7 +99,7 @@ async def run_buf_format(
             output_files=request.files,
             description=f"Run buf format on {pluralize(len(request.files), 'file')}.",
             level=LogLevel.DEBUG,
-            env={"PATH": binary_shims.bin_directory},
+            env={"PATH": os.path.join("{chroot}", binary_shims.bin_directory)},
         ),
     )
     return await FmtResult.create(request, result)


Test patch:
Repository: pantsbuild/pants

Issue:
`pex_binary` needs an `environment=` field



Discussion:

Resolving patch:
diff --git a/src/python/pants/backend/python/target_types.py b/src/python/pants/backend/python/target_types.py
--- a/src/python/pants/backend/python/target_types.py
+++ b/src/python/pants/backend/python/target_types.py
@@ -625,6 +625,7 @@ class PexVenvSitePackagesCopies(BoolField):
 
 
 _PEX_BINARY_COMMON_FIELDS = (
+    EnvironmentField,
     InterpreterConstraintsField,
     PythonResolveField,
     PexBinaryDependenciesField,


Test patch:
diff --git a/src/python/pants/backend/awslambda/python/rules_test.py b/src/python/pants/backend/awslambda/python/rules_test.py
--- a/src/python/pants/backend/awslambda/python/rules_test.py
+++ b/src/python/pants/backend/awslambda/python/rules_test.py
@@ -95,7 +95,7 @@ def complete_platform(rule_runner: RuleRunner) -> bytes:
         {
             "pex_exe/BUILD": dedent(
                 """\
-                python_requirement(name="req", requirements=["pex==2.1.66"])
+                python_requirement(name="req", requirements=["pex==2.1.112"])
                 pex_binary(dependencies=[":req"], script="pex")
                 """
             ),
diff --git a/src/python/pants/backend/python/goals/package_pex_binary_integration_test.py b/src/python/pants/backend/python/goals/package_pex_binary_integration_test.py
--- a/src/python/pants/backend/python/goals/package_pex_binary_integration_test.py
+++ b/src/python/pants/backend/python/goals/package_pex_binary_integration_test.py
@@ -145,7 +145,7 @@ def pex_executable(rule_runner: RuleRunner) -> str:
         {
             "pex_exe/BUILD": dedent(
                 """\
-                python_requirement(name="req", requirements=["pex==2.1.66"])
+                python_requirement(name="req", requirements=["pex==2.1.112"])
                 pex_binary(dependencies=[":req"], script="pex")
                 """
             ),
Repository: pantsbuild/pants

Issue:
Weird stdout redirection behavior
```
$ ./pants version > version.stdout 2> version.stderr
$ wc -l version.stdout 
   1 version.stdout
$ wc -l version.stderr
   0 version.stderr
$ # Makes sense, version emits to stdout
$
$ ./pants version > version.stdout 
$ wc -l version.stdout 
   0 version.stdout
$ wc -l version.stderr
   0 version.stderr
$ # What? Where did the output go? 
```

This is due to https://github.com/pantsbuild/pants/pull/16822, although it is unclear why. Reverting that change fixes this.

Note that it doesn't reproduce when running pants in an integration test.


Discussion:

Resolving patch:
diff --git a/build-support/bin/generate_github_workflows.py b/build-support/bin/generate_github_workflows.py
--- a/build-support/bin/generate_github_workflows.py
+++ b/build-support/bin/generate_github_workflows.py
@@ -408,7 +408,14 @@ def bootstrap_pants(self, *, install_python: bool) -> Sequence[Step]:
             *setup_primary_python(install_python=install_python),
             *self.bootstrap_caches(),
             setup_toolchain_auth(),
-            {"name": "Bootstrap Pants", "run": self.wrap_cmd("./pants --version\n")},
+            {
+                "name": "Bootstrap Pants",
+                # Check for a regression of https://github.com/pantsbuild/pants/issues/17470.
+                "run": self.wrap_cmd(
+                    f"./pants version > {gha_expr('runner.temp')}/_pants_version.stdout && "
+                    f"[[ -s {gha_expr('runner.temp')}/_pants_version.stdout ]]"
+                ),
+            },
             {
                 "name": "Run smoke tests",
                 "run": dedent(


Test patch:
diff --git a/.github/workflows/test.yaml b/.github/workflows/test.yaml
--- a/.github/workflows/test.yaml
+++ b/.github/workflows/test.yaml
@@ -70,9 +70,8 @@ jobs:
 
         '
     - name: Bootstrap Pants
-      run: './pants --version
-
-        '
+      run: ./pants version > ${{ runner.temp }}/_pants_version.stdout && [[ -s ${{
+        runner.temp }}/_pants_version.stdout ]]
     - name: Run smoke tests
       run: './pants list ::
 
@@ -184,9 +183,8 @@ jobs:
 
         '
     - name: Bootstrap Pants
-      run: './pants --version
-
-        '
+      run: ./pants version > ${{ runner.temp }}/_pants_version.stdout && [[ -s ${{
+        runner.temp }}/_pants_version.stdout ]]
     - name: Run smoke tests
       run: './pants list ::
Repository: pantsbuild/pants

Issue:
Weird stdout redirection behavior
```
$ ./pants version > version.stdout 2> version.stderr
$ wc -l version.stdout 
   1 version.stdout
$ wc -l version.stderr
   0 version.stderr
$ # Makes sense, version emits to stdout
$
$ ./pants version > version.stdout 
$ wc -l version.stdout 
   0 version.stdout
$ wc -l version.stderr
   0 version.stderr
$ # What? Where did the output go? 
```

This is due to https://github.com/pantsbuild/pants/pull/16822, although it is unclear why. Reverting that change fixes this.

Note that it doesn't reproduce when running pants in an integration test.


Discussion:
@stuhood any thoughts on why a patch upgrade of rust futures would cause this?

Resolving patch:
diff --git a/build-support/bin/generate_github_workflows.py b/build-support/bin/generate_github_workflows.py
--- a/build-support/bin/generate_github_workflows.py
+++ b/build-support/bin/generate_github_workflows.py
@@ -408,7 +408,14 @@ def bootstrap_pants(self, *, install_python: bool) -> Sequence[Step]:
             *setup_primary_python(install_python=install_python),
             *self.bootstrap_caches(),
             setup_toolchain_auth(),
-            {"name": "Bootstrap Pants", "run": self.wrap_cmd("./pants --version\n")},
+            {
+                "name": "Bootstrap Pants",
+                # Check for a regression of https://github.com/pantsbuild/pants/issues/17470.
+                "run": self.wrap_cmd(
+                    f"./pants version > {gha_expr('runner.temp')}/_pants_version.stdout && "
+                    f"[[ -s {gha_expr('runner.temp')}/_pants_version.stdout ]]"
+                ),
+            },
             {
                 "name": "Run smoke tests",
                 "run": dedent(


Test patch:
diff --git a/.github/workflows/test.yaml b/.github/workflows/test.yaml
--- a/.github/workflows/test.yaml
+++ b/.github/workflows/test.yaml
@@ -70,9 +70,8 @@ jobs:
 
         '
     - name: Bootstrap Pants
-      run: './pants --version
-
-        '
+      run: ./pants version > ${{ runner.temp }}/_pants_version.stdout && [[ -s ${{
+        runner.temp }}/_pants_version.stdout ]]
     - name: Run smoke tests
       run: './pants list ::
 
@@ -184,9 +183,8 @@ jobs:
 
         '
     - name: Bootstrap Pants
-      run: './pants --version
-
-        '
+      run: ./pants version > ${{ runner.temp }}/_pants_version.stdout && [[ -s ${{
+        runner.temp }}/_pants_version.stdout ]]
     - name: Run smoke tests
       run: './pants list ::
Repository: pantsbuild/pants

Issue:
`CoarsenedTargets.closure()` returns duplicate targets
**Describe the bug**

While investigating `pylint` perf regressions in 2.15.x, I found that [this call to `closure`](https://github.com/pantsbuild/pants/blob/main/src/python/pants/backend/python/lint/pylint/rules.py#L132) is yielding duplicate elements (sometimes 10s of thousands of duplicates in our very tangled repo). @stuhood confirmed it should not be doing that.

**Pants version**

2.15.x

In 2.14.x the `closure` call for `pylint` wrapped the result in a `FrozenOrderedSet`, so the bug could be present there too

**OS**

Both


Discussion:
Have a repro of this. Sorry for the trouble!

Resolving patch:
diff --git a/src/python/pants/engine/target.py b/src/python/pants/engine/target.py
--- a/src/python/pants/engine/target.py
+++ b/src/python/pants/engine/target.py
@@ -816,7 +816,7 @@ def coarsened_closure(
     ) -> Iterator[CoarsenedTarget]:
         """All CoarsenedTargets reachable from this root."""
 
-        visited = visited or set()
+        visited = set() if visited is None else visited
         queue = deque([self])
         while queue:
             ct = queue.popleft()


Test patch:
diff --git a/src/python/pants/engine/target_test.py b/src/python/pants/engine/target_test.py
--- a/src/python/pants/engine/target_test.py
+++ b/src/python/pants/engine/target_test.py
@@ -5,7 +5,7 @@
 from collections import namedtuple
 from dataclasses import dataclass
 from enum import Enum
-from typing import Any, Dict, Iterable, List, Optional, Set, Tuple, cast
+from typing import Any, Dict, Iterable, List, Optional, Sequence, Set, Tuple, cast
 
 import pytest
 
@@ -15,6 +15,7 @@
     AsyncFieldMixin,
     BoolField,
     CoarsenedTarget,
+    CoarsenedTargets,
     DictStringToStringField,
     DictStringToStringSequenceField,
     ExplicitlyProvidedDependencies,
@@ -498,6 +499,27 @@ def nested():
     assert nested() == nested()
 
 
+def test_coarsened_target_closure() -> None:
+    all_targets = [FortranTarget({}, Address(name)) for name in string.ascii_lowercase[:5]]
+    a, b, c, d, e = all_targets
+
+    def ct(members: List[Target], dependencies: List[CoarsenedTarget] = []) -> CoarsenedTarget:
+        return CoarsenedTarget(members, dependencies)
+
+    def assert_closure(cts: Sequence[CoarsenedTarget], expected: Sequence[Target]) -> None:
+        assert sorted(t.address for t in CoarsenedTargets(cts).closure()) == sorted(
+            t.address for t in expected
+        )
+
+    ct1 = ct([a], [])
+    ct2 = ct([b, c], [ct1])
+    ct3 = ct([d, e], [ct1, ct2])
+
+    assert_closure([ct1], [a])
+    assert_closure([ct1, ct2], [a, b, c])
+    assert_closure([ct1, ct2, ct3], all_targets)
+
+
 # -----------------------------------------------------------------------------------------------
 # Test file-level target generation
 # -----------------------------------------------------------------------------------------------
Repository: pantsbuild/pants

Issue:
`"chosen_resolve.name"` string printed to logs when exporting a resolve
**Describe the bug**

`"chosen_resolve.name"` string printed to logs when exporting a resolve.

It seems like the intent here was probably to print the resolve name

The related source code line is here:

https://github.com/pantsbuild/pants/blob/4bbd0c7aab9e5d644450e6d4664e6ff9d7ffd037/src/python/pants/backend/python/goals/export.py#L240

**Pants version**

2.15.a0

**OS**

Linux

**Additional info**
```bash
./pants export --resolve=python-default  
15:57:36.96 [INFO] waiting for pantsd to start...
15:57:41.08 [INFO] pantsd started
15:57:44.30 [INFO] Initializing scheduler...
15:57:50.58 [INFO] Scheduler initialized.
15:57:51.60 [INFO] Preserving local process execution dir /tmp/pants-sandbox-Z6lBpI for Test binary /usr/bin/python.
15:57:51.60 [INFO] Preserving local process execution dir /tmp/pants-sandbox-aSSJJy for Test binary /bin/python.
16:00:55.31 [INFO] Completed: chosen_resolve.name
16:01:04.26 [INFO] Completed: Get interpreter version
Wrote mutable virtualenv for python-default (using Python 3.8.10) to dist/export/python/virtualenvs/python-default
```



Discussion:

Resolving patch:
diff --git a/src/python/pants/backend/python/goals/export.py b/src/python/pants/backend/python/goals/export.py
--- a/src/python/pants/backend/python/goals/export.py
+++ b/src/python/pants/backend/python/goals/export.py
@@ -236,7 +236,7 @@ async def export_virtualenv_for_resolve(
         )
 
         pex_request = PexRequest(
-            description="chosen_resolve.name",
+            description=f"Build pex for resolve `{resolve}`",
             output_filename=f"{path_safe(resolve)}.pex",
             internal_only=True,
             requirements=EntireLockfile(lockfile),


Test patch:
Repository: pantsbuild/pants

Issue:
deprecation warning for `python-dump-source-analysis` goal
Deprecation warning encountered after https://github.com/pantsbuild/pants/pull/17057:

```
19:51:58.21 [WARN] DEPRECATED: Setting `Goal.environment_behavior=EnvironmentBehavior.UNMIGRATED` for `Goal` `python-dump-source-analysis` is scheduled to be removed in version 2.17.0.dev0.
```


Discussion:
cc @lilatomic @stuhood 
I don't know much about environments. This goal is just dumping the information from python dependency inference. I think this is similar to the `Peek` goal, which uses `environment_behavior=Goal.EnvironmentBehavior.LOCAL_ONLY`. The other dump goals use that, but have a TODO referencing #17129 with the comment "Migrate this".

I can just do the same?
`environment_behavior=Goal.EnvironmentBehavior.LOCAL_ONLY` should be fine for now. At the very least, it will remove the deprecation warning.

Resolving patch:
diff --git a/src/python/pants/backend/python/goals/debug_goals.py b/src/python/pants/backend/python/goals/debug_goals.py
--- a/src/python/pants/backend/python/goals/debug_goals.py
+++ b/src/python/pants/backend/python/goals/debug_goals.py
@@ -65,6 +65,7 @@ class DumpPythonSourceAnalysisSubsystem(GoalSubsystem):
 
 class DumpPythonSourceAnalysis(Goal):
     subsystem_cls = DumpPythonSourceAnalysisSubsystem
+    environment_behavior = Goal.EnvironmentBehavior.LOCAL_ONLY  # TODO(#17129) — Migrate this.
 
 
 @dataclass(frozen=True)


Test patch:
Repository: pantsbuild/pants

Issue:
`./pants run company/pyfile.py` always uses repo-path if source-roots are `"/"`.
**Describe the bug**
`./pants run company/pyfile.py` will always use the in-repo root if source-roots are "/", because `cwd` is still the repo-root and `PYTHONPATH` will have `""` as an early entry meaning `company.pyfile` is loaded from repo-root before we ever even consider the entries we add using `PEX_EXTRA_SYS_PATH`.

**Pants version**
2.14.

**OS**
N/A

**Additional info**
N/A



Discussion:
If I leak and enter the sandbox and run the PEX everything works fine.
~I'm thinking this is more of an issue of `run_goal_use_sandbox=False` running in CWD of the project root.~
I can confirm this bug goes back to 2.13.x when running a python source was introduced.
So the issue stems from `--venv` in the PEX args. I think thats an OK thing to do. Likely aligns with the element of least surprise.

So to "fix" I filed https://github.com/pantsbuild/pex/issues/1981 and we can just upgrade PEX, use it, and not backport the "fix".

Resolving patch:
diff --git a/src/python/pants/backend/python/util_rules/pex_cli.py b/src/python/pants/backend/python/util_rules/pex_cli.py
--- a/src/python/pants/backend/python/util_rules/pex_cli.py
+++ b/src/python/pants/backend/python/util_rules/pex_cli.py
@@ -38,9 +38,9 @@ class PexCli(TemplatedExternalTool):
     name = "pex"
     help = "The PEX (Python EXecutable) tool (https://github.com/pantsbuild/pex)."
 
-    default_version = "v2.1.113"
+    default_version = "v2.1.116"
     default_url_template = "https://github.com/pantsbuild/pex/releases/download/{version}/pex"
-    version_constraints = ">=2.1.113,<3.0"
+    version_constraints = ">=2.1.116,<3.0"
 
     @classproperty
     def default_known_versions(cls):
@@ -49,8 +49,8 @@ def default_known_versions(cls):
                 (
                     cls.default_version,
                     plat,
-                    "cde8eecc5e8e1c9fcca89f36c125a2c4b1c55cdb1073220b1be645e60c0c36e6",
-                    "4067306",
+                    "3158c6ab70272f836daade412974e791a069af2eb1d43aee9c10cfdf63154385",
+                    "4068322",
                 )
             )
             for plat in ["macos_arm64", "macos_x86_64", "linux_x86_64", "linux_arm64"]


Test patch:
Repository: pantsbuild/pants

Issue:
`deploy_jar` attempts to build Java source files that do not exist in the package
Attempting to build a `deploy_jar` results in:

```
FileNotFoundError: [Errno 2] No such file or directory: '/Users/chrisjrn/src/pants/src/python/pants/jvm/jar_tool/src/java/org/pantsbuild/args4j'
```

Took a look through the unzipped pants wheel, and the relevant files are nowhere to be found.


Discussion:
perhaps the same kind of ops I did address in #17570 
@kaos Doesn't look like it -- there's already a `relocated_files` target in the relevant python package
OK 🤷🏽 

Resolving patch:
diff --git a/src/python/pants/jvm/jar_tool/jar_tool.py b/src/python/pants/jvm/jar_tool/jar_tool.py
--- a/src/python/pants/jvm/jar_tool/jar_tool.py
+++ b/src/python/pants/jvm/jar_tool/jar_tool.py
@@ -193,20 +193,20 @@ async def run_jar_tool(
     return await Get(Digest, RemovePrefix(result.output_digest, output_prefix))
 
 
-_JAR_TOOL_SRC_PACKAGES = ["org.pantsbuild.args4j", "org.pantsbuild.tools.jar"]
+_JAR_TOOL_SRC_PACKAGES = ["args4j", "jar_tool_source"]
 
 
 def _load_jar_tool_sources() -> list[FileContent]:
     result = []
     for package in _JAR_TOOL_SRC_PACKAGES:
-        pkg_path = package.replace(".", os.path.sep)
-        relative_folder = os.path.join("src", pkg_path)
-        for basename in pkg_resources.resource_listdir(__name__, relative_folder):
+        # pkg_path = package.replace(".", os.path.sep)
+        # relative_folder = os.path.join("src", pkg_path)
+        for basename in pkg_resources.resource_listdir(__name__, package):
             result.append(
                 FileContent(
-                    path=os.path.join(pkg_path, basename),
+                    path=os.path.join(package, basename),
                     content=pkg_resources.resource_string(
-                        __name__, os.path.join(relative_folder, basename)
+                        __name__, os.path.join(package, basename)
                     ),
                 )
             )


Test patch:
Repository: pantsbuild/pants

Issue:
Pants 2.16.0.dev1 has partially reversioned wheels
Looking at a wheel:
```
$ curl -sSL https://files.pythonhosted.org/packages/26/b4/356748c06781caabbc8f17d71b69a06a0efe14dd63dd199b9c29d1dbc85f/pantsbuild.pants-2.16.0.dev1-cp39-cp39-manylinux2014_x86_64.whl -O
$ unzip -qc pantsbuild.pants-2.16.0.dev1-cp39-cp39-manylinux2014_x86_64.whl pantsbuild.pants-2.16.0.dev1.dist-info/METADATA | grep -E "^Version:"
Version: 2.16.0.dev1
$ unzip -qc pantsbuild.pants-2.16.0.dev1-cp39-cp39-manylinux2014_x86_64.whl pants/_version/VERSION
2.16.0.dev1+git57fecb6c
```

It appears this may be due to #17582 since there is no longer a `pants/VERSION`:
```
$ zipinfo pantsbuild.pants-2.16.0.dev1-cp39-cp39-manylinux2014_x86_64.whl | grep -E "pants/(_version|version)"
-rw-r--r--  2.0 unx     1310 b- defN 22-Nov-27 18:01 pants/version.py
-rw-r--r--  2.0 unx       23 b- defN 22-Nov-27 18:01 pants/_version/VERSION
-rw-r--r--  2.0 unx        0 b- defN 22-Nov-27 18:01 pants/_version/__init__.py
```
And that is what we look for here:
https://github.com/pantsbuild/pants/blob/15750dbd534028ee17b16401eb49cd79d8495896/build-support/bin/_release_helper.py#L984-L990

The fix may be as simple as adjusting `extra_globs` there.



Discussion:

Resolving patch:
diff --git a/build-support/bin/_release_helper.py b/build-support/bin/_release_helper.py
--- a/build-support/bin/_release_helper.py
+++ b/build-support/bin/_release_helper.py
@@ -986,7 +986,7 @@ def reversion_prebuilt_wheels() -> None:
             whl_file=str(whl),
             dest_dir=str(stable_wheel_dir),
             target_version=CONSTANTS.pants_stable_version,
-            extra_globs=["pants/VERSION"],
+            extra_globs=["pants/_version/VERSION"],
         )
 
 


Test patch: