internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: pantsbuild/pants

Issue:
codegen: Scala codegen fails with missing jdk field
There is a missing `jdk` field on some Protobuf target that is causing compilation of a Scala target with a protobuf dependency to fail. I used example-codegen with https://github.com/pantsbuild/example-codegen/pull/10 to test compilation of Scala Protobuf code. I then switched the dependency to a file-level target using `scala_sources(dependencies=["src/protobuf/simple_example/person.proto"])` in the relavent BUILD file.

Running `./pants_from_sources check src/jvm/org/pantsbuild/example/codegen/scala::` results in: 

```
  ValueError: Cannot construct a JDK request for src/protobuf/simple_example/person.proto, since none of its members have a `jdk=` field:
  * src/protobuf/simple_example/person.proto	(protobuf_source)
```


Discussion:

Resolving patch:
diff --git a/src/python/pants/backend/codegen/protobuf/scala/rules.py b/src/python/pants/backend/codegen/protobuf/scala/rules.py
--- a/src/python/pants/backend/codegen/protobuf/scala/rules.py
+++ b/src/python/pants/backend/codegen/protobuf/scala/rules.py
@@ -11,6 +11,8 @@
 from pants.backend.codegen.protobuf.target_types import (
     ProtobufDependenciesField,
     ProtobufSourceField,
+    ProtobufSourcesGeneratorTarget,
+    ProtobufSourceTarget,
 )
 from pants.backend.scala.target_types import ScalaSourceField
 from pants.core.goals.generate_lockfiles import GenerateToolLockfileSentinel
@@ -49,6 +51,7 @@
 from pants.jvm.resolve.common import ArtifactRequirements, Coordinate, GatherJvmCoordinatesRequest
 from pants.jvm.resolve.coursier_fetch import ToolClasspath, ToolClasspathRequest
 from pants.jvm.resolve.jvm_tool import GenerateJvmLockfileFromTool
+from pants.jvm.target_types import JvmJdkField
 from pants.source.source_root import SourceRoot, SourceRootRequest
 from pants.util.logging import LogLevel
 from pants.util.ordered_set import FrozenOrderedSet
@@ -333,10 +336,16 @@ def generate_scalapbc_lockfile_request(
     return GenerateJvmLockfileFromTool.create(tool)
 
 
+class PrefixedJvmJdkField(JvmJdkField):
+    alias = "jvm_jdk"
+
+
 def rules():
     return [
         *collect_rules(),
         *lockfile.rules(),
         UnionRule(GenerateSourcesRequest, GenerateScalaFromProtobufRequest),
         UnionRule(GenerateToolLockfileSentinel, ScalapbcToolLockfileSentinel),
+        ProtobufSourceTarget.register_plugin_field(PrefixedJvmJdkField),
+        ProtobufSourcesGeneratorTarget.register_plugin_field(PrefixedJvmJdkField),
     ]


Test patch:
Repository: pantsbuild/pants

Issue:
SEGFAULT in internal JVM when running scalafmt in large filesets
**Describe the bug**
When running `./pants fmt ::` on a codebase with a large set of Scala files (more than 1000), the internal JVM used by Pants (AdoptJDK 11) occasionally segfaults with the following error message:

```
12:19:06.67 [ERROR] Completed: Lint with scalafmt - scalafmt failed (exit code -6).
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f8473fed6f1, pid=6524, tid=6567
#
# JRE version: OpenJDK Runtime Environment (11.0+28) (build 11+28)
# Java VM: OpenJDK 64-Bit Server VM (11+28, mixed mode, tiered, compressed oops, g1 gc, linux-amd64)
# Problematic frame:
# V  [libjvm.so+0xbf46f1]  Node::add_req(Node*)+0xc1
#
# Core dump will be written. Default location: Core dumps may be processed with "/usr/share/apport/apport %p %s %c %d %P %E" (or dumping to /tmp/process-executionzWXy23/core.6524)
#
# An error report file with more information is saved as:
# /tmp/process-executionzWXy23/hs_err_pid6524.log
#
# Compiler replay data is saved as:
# [thread 6588 also had an error]
/tmp/process-executionzWXy23/replay_pid6524.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
```

Changing the internal JDK to the Temurin 11 distribution seems to remove the problem permanently:

```toml
[jvm]
tool_jdk = "temurin:1.11"
```

**Pants version**
2.11.0rc1

**OS**
MacOS

**Additional info**
Initially mentioned here: https://pantsbuild.slack.com/archives/C046T6T9U/p1649158967074989



Discussion:

Resolving patch:
diff --git a/src/python/pants/jvm/subsystems.py b/src/python/pants/jvm/subsystems.py
--- a/src/python/pants/jvm/subsystems.py
+++ b/src/python/pants/jvm/subsystems.py
@@ -25,7 +25,7 @@ class JvmSubsystem(Subsystem):
 
     tool_jdk = StrOption(
         "--tool-jdk",
-        default="adopt:1.11",
+        default="temurin:1.11",
         help=softwrap(
             """
             The JDK to use when building and running Pants' internal JVM support code and other
@@ -36,7 +36,7 @@ class JvmSubsystem(Subsystem):
     )
     jdk = StrOption(
         "--jdk",
-        default="adopt:1.11",
+        default="temurin:1.11",
         help=softwrap(
             """
             The JDK to use.


Test patch:
Repository: pantsbuild/pants

Issue:
Docker publish fails with additional tools
**Describe the bug**

```
ValueError: Process has immutable_input_digests, so it cannot be converted to an InteractiveProcess by calling from_process().  Use an async InteractiveProcessRequest instead.
```



Discussion:

Resolving patch:
diff --git a/src/python/pants/backend/docker/goals/publish.py b/src/python/pants/backend/docker/goals/publish.py
--- a/src/python/pants/backend/docker/goals/publish.py
+++ b/src/python/pants/backend/docker/goals/publish.py
@@ -21,8 +21,8 @@
     PublishRequest,
 )
 from pants.engine.environment import Environment, EnvironmentRequest
-from pants.engine.process import InteractiveProcess
-from pants.engine.rules import Get, collect_rules, rule
+from pants.engine.process import InteractiveProcess, InteractiveProcessRequest
+from pants.engine.rules import Get, MultiGet, collect_rules, rule
 
 logger = logging.getLogger(__name__)
 
@@ -72,8 +72,10 @@ async def push_docker_images(
         )
 
     env = await Get(Environment, EnvironmentRequest(options.env_vars))
-    processes = []
     skip_push = defaultdict(set)
+    jobs: list[PublishPackages] = []
+    refs: list[str] = []
+    processes: list[Get] = []
 
     for tag in tags:
         for registry in options.registries().registries.values():
@@ -81,22 +83,30 @@ async def push_docker_images(
                 skip_push[registry.alias].add(tag)
                 break
         else:
+            refs.append(tag)
             processes.append(
-                PublishPackages(
-                    names=(tag,),
-                    process=InteractiveProcess.from_process(docker.push_image(tag, env)),
-                )
+                Get(InteractiveProcess, InteractiveProcessRequest(docker.push_image(tag, env)))
+            )
+
+    interactive_processes = await MultiGet(processes)
+    for ref, process in zip(refs, interactive_processes):
+        jobs.append(
+            PublishPackages(
+                names=(ref,),
+                process=process,
             )
+        )
+
     if skip_push:
         for name, skip_tags in skip_push.items():
-            processes.append(
+            jobs.append(
                 PublishPackages(
                     names=tuple(skip_tags),
                     description=f"(by `skip_push` on registry @{name})",
                 ),
             )
 
-    return PublishProcesses(processes)
+    return PublishProcesses(jobs)
 
 
 def rules():


Test patch:
Repository: pantsbuild/pants

Issue:
Add `update_env` to `process_execution::local`.
Replaces any `{chroot}` placeholders in the requested process env with the sandbox workdir.

This reflects the behavior for interactive processes executed with the `run` goal.


Discussion:
> What milestone should this be cherry-picked to?

2.11 :) 
(Andreas asked me to merge)

Resolving patch:
diff --git a/src/python/pants/core/goals/publish.py b/src/python/pants/core/goals/publish.py
--- a/src/python/pants/core/goals/publish.py
+++ b/src/python/pants/core/goals/publish.py
@@ -240,6 +240,7 @@ async def run_publish(console: Console, publish: PublishSubsystem) -> Publish:
             outputs.append(pub.get_output_data(published=False, status=status))
             continue
 
+        logger.debug(f"Execute {pub.process}")
         res = await Effect(InteractiveProcessResult, InteractiveProcess, pub.process)
         if res.exit_code == 0:
             sigil = console.sigil_succeeded()


Test patch:
Repository: pantsbuild/pants

Issue:
Required fields should sort first on the docsite
Currently `required` fields of a target definition are not necessarily listed first on the docsite, which makes it less obvious how to write a target definition from scratch.


Discussion:
Good idea. We should do this for `./pants help $tgt` too.

For `help`, you modify:

https://github.com/pantsbuild/pants/blob/a19d4403d3f68bf762187ca70ad027a619f7929a/src/python/pants/help/help_printer.py#L288-L296

For docs:

https://github.com/pantsbuild/pants/blob/a19d4403d3f68bf762187ca70ad027a619f7929a/build-support/bin/generate_docs.py#L390

Resolving patch:
diff --git a/build-support/bin/generate_docs.py b/build-support/bin/generate_docs.py
--- a/build-support/bin/generate_docs.py
+++ b/build-support/bin/generate_docs.py
@@ -424,7 +424,9 @@ def process_targets_input(cls, help_info: dict[str, Any]) -> dict[str, dict[str,
                     "required" if field["required"] else f"default: <code>{default_str}</code>"
                 )
                 field["description"] = str(field["description"])
-            target["fields"] = sorted(target["fields"], key=lambda fld: cast(str, fld["alias"]))
+            target["fields"] = sorted(
+                target["fields"], key=lambda fld: (-fld["required"], cast(str, fld["alias"]))
+            )
             target["description"] = str(target["description"])
 
         return cast(Dict[str, Dict[str, Any]], target_info)
diff --git a/src/python/pants/help/help_printer.py b/src/python/pants/help/help_printer.py
--- a/src/python/pants/help/help_printer.py
+++ b/src/python/pants/help/help_printer.py
@@ -293,7 +293,7 @@ def _print_target_help(self, target_alias: str) -> None:
             print(formatted_desc)
         print(f"\n\nActivated by {self.maybe_magenta(tinfo.provider)}")
         print("Valid fields:")
-        for field in sorted(tinfo.fields, key=lambda x: x.alias):
+        for field in sorted(tinfo.fields, key=lambda x: (-x.required, x.alias)):
             print()
             print(self.maybe_magenta(field.alias))
             indent = "    "


Test patch:
Repository: pantsbuild/pants

Issue:
inconsistent deprecation messaging for `use_deprecated_pex_binary_run_semantics`
With Pants v2.13.0a1, not having the `use_deprecated_pex_binary_run_semantics` config displays:

```
15:44:49.52 [WARN] DEPRECATED: the option --use-deprecated-pex-binary-run-semantics defaulting to true is scheduled to be removed in version 2.14.0.dev1.

Currently, running a `pex_binary` by default will not include the source files in the PEX, and will instead put them in a temporary sandbox.

In Pants 2.14, the default will change to instead build the PEX like you had run the `package` goal, and then execute that PEX. This is more consistent and intuitive behavior.

To fix this deprecation, explictly set `use_deprecated_pex_binary_run_semantics` in the `[GLOBAL]` section of `pants.toml`. Set it to `true` to use the "old" behavior. Set it to `false` to use the "new" behavior.

When set to `false`, you can still run the binary as before because you can now run on a `python_source` target. The simplest way to do this is to use `./pants run path/to/file.py`, which will find the owning `python_source`. Pants will run the file the same way it used to with `pex_binary` targets.
```

Setting the config to `False` then causes a different deprecation message to appear suggesting to remove the config option:

```
15:44:19.29 [WARN] DEPRECATED: option 'use_deprecated_pex_binary_run_semantics' in global scope is scheduled to be removed in version 2.15.0.dev0.

If `use_deprecated_pex_binary_run_semantics` is already set explicitly to `false`, simply delete the option from `pants.toml` because `false` is now the default.

If set to `true`, removing the option will cause `run` on a `pex_binary` to package and run the built PEX file. Additionally, the `pex_binary` must be referred to by its address. To keep the old `run` semantics, use `run` on the relevant `python_source` target.
```

The second config message should not appear until after 2.13.0 is released.


Discussion:

Resolving patch:
diff --git a/src/python/pants/option/global_options.py b/src/python/pants/option/global_options.py
--- a/src/python/pants/option/global_options.py
+++ b/src/python/pants/option/global_options.py
@@ -1732,17 +1732,6 @@ class GlobalOptions(BootstrapOptions, Subsystem):
             that exist solely for running Python code (and aren't meant to be packaged).
             """
         ),
-        removal_version="2.15.0.dev0",
-        removal_hint=softwrap(
-            """
-            If `use_deprecated_pex_binary_run_semantics` is already set explicitly to `false`,
-            simply delete the option from `pants.toml` because `false` is now the default.
-
-            If set to `true`, removing the option will cause `run` on a `pex_binary` to package and
-            run the built PEX file. Additionally, the `pex_binary` must be referred to by its address.
-            To keep the old `run` semantics, use `run` on the relevant `python_source` target.
-            """
-        ),
     )
 
     @property


Test patch:
Repository: pantsbuild/pants

Issue:
Pants poetry-based lockfiles fail to include hashes.
This was detected in a unit test in the Pants repo, but is a wider problem for all versions of Pants that support generating lockfiles using Poetry.

The proximal cause is this announcement from PyPI:
  https://discuss.python.org/t/backwards-incompatible-change-to-pypi-json-api/17154

And the root cause is this Poetry code:
  https://github.com/python-poetry/poetry/blob/bce13c14f73060b3abbb791dea585d8fde26eaef/poetry/repositories/pypi_repository.py#L272-L283

There was a Poetry fix released and backported to the 1.1. branch here:
  https://github.com/python-poetry/poetry/pull/5973

Users can fix with 2 steps:
1. Update Pants config
```toml
[poetry]
# N.B.: Works around issue described at https://github.com/pantsbuild/pants/issues/16111
# Undo once on a Pants with this version or greater as the default.
version = "poetry==1.1.14"
```
2. Clear Poetry caches with `rm -rf ~/.cache/pypoetry` on Linux and `rm -rf ~/Library/Caches/pypoetry` on Mac.

This issue tracks bumping Pants default to this fixed Poetry version.



Discussion:

Resolving patch:
diff --git a/src/python/pants/backend/python/subsystems/poetry.py b/src/python/pants/backend/python/subsystems/poetry.py
--- a/src/python/pants/backend/python/subsystems/poetry.py
+++ b/src/python/pants/backend/python/subsystems/poetry.py
@@ -24,7 +24,7 @@ class PoetrySubsystem(PythonToolRequirementsBase):
     options_scope = "poetry"
     help = "Used to generate lockfiles for third-party Python dependencies."
 
-    default_version = "poetry==1.1.8"
+    default_version = "poetry==1.1.14"
 
     register_interpreter_constraints = True
     default_interpreter_constraints = ["CPython>=3.7,<4"]


Test patch:
Repository: pantsbuild/pants

Issue:
pluralizing "dependency" produces "dependencys"
**Describe the bug**
`pluralize(len(dependencies), 'dependency')` -> `"dependencys"`

**Pants version**
`main`

**OS**
N/A

**Additional info**
hehehe



Discussion:

Resolving patch:
diff --git a/src/python/pants/util/strutil.py b/src/python/pants/util/strutil.py
--- a/src/python/pants/util/strutil.py
+++ b/src/python/pants/util/strutil.py
@@ -102,6 +102,8 @@ def pluralize(count: int, item_type: str, include_count: bool = True) -> str:
     def pluralize_string(x: str) -> str:
         if x.endswith("s"):
             return x + "es"
+        elif x.endswith("y"):
+            return x[:-1] + "ies"
         else:
             return x + "s"
 


Test patch:
diff --git a/src/python/pants/util/strutil_test.py b/src/python/pants/util/strutil_test.py
--- a/src/python/pants/util/strutil_test.py
+++ b/src/python/pants/util/strutil_test.py
@@ -28,6 +28,8 @@ def test_pluralize() -> None:
     assert "2 bosses" == pluralize(2, "boss")
     assert "0 bats" == pluralize(0, "bat")
     assert "0 bosses" == pluralize(0, "boss")
+    assert "1 dependency" == pluralize(1, "dependency")
+    assert "2 dependencies" == pluralize(2, "dependency")
 
 
 def test_ensure_text() -> None:
Repository: pantsbuild/pants

Issue:
Please add Vicara Solutions Inc to the public list of Pants Users
### Company name

Vicara Solutions Inc

### Company website

https://vicarasolutions.com

### Company logo

![vicara_red](https://user-images.githubusercontent.com/3823045/177662904-c939d2cd-954d-41e5-9f5a-365e9d680755.png)



Discussion:

Resolving patch:
diff --git a/build-support/bin/generate_user_list.py b/build-support/bin/generate_user_list.py
--- a/build-support/bin/generate_user_list.py
+++ b/build-support/bin/generate_user_list.py
@@ -103,6 +103,11 @@ class Org:
         "https://files.readme.io/43d674d-toolchain_logo_small.png",
     ),
     Org("Valon", "https://valon.com/", "https://files.readme.io/df5216a-valon-small.png"),
+    Org(
+        "Vicara Solutions",
+        "https://vicarasolutions.com/",
+        "https://files.readme.io/1748a22-vicara-solutions.png",
+    ),
 )
 
 


Test patch:
Repository: pantsbuild/pants

Issue:
Parametrized binary targets collide when `package`d simultaneously
`deploy_jar` and `pex_binary` use the `output_path` field, which defaults to the target name. But the current calculation of that field does not include parameter values: https://github.com/pantsbuild/pants/blob/1ef5a17f2bd331cc255c0af69fae18f808853115/src/python/pants/core/goals/package.py#L72-L85

That means that when a binary target is parametrized (on the `resolve` field, for example), both instances of the target will use the same computed `output_path`, which will cause a digest merge failure.

It should likely delegate to a new method of `Address` which uses the relevant portion of [`path_safe_spec`](https://github.com/pantsbuild/pants/blob/1ef5a17f2bd331cc255c0af69fae18f808853115/src/python/pants/build_graph/address.py#L496-L500), so that the parameters are included.


Discussion:

Resolving patch:
diff --git a/src/python/pants/build_graph/address.py b/src/python/pants/build_graph/address.py
--- a/src/python/pants/build_graph/address.py
+++ b/src/python/pants/build_graph/address.py
@@ -514,10 +514,7 @@ def sanitize(s: str) -> str:
         else:
             target = f"{parent_prefix}{self.target_name}"
         if self.parameters:
-            key_value_strs = ",".join(
-                f"{sanitize(k)}={sanitize(v)}" for k, v in self.parameters.items()
-            )
-            params = f"@@{key_value_strs}"
+            params = f"@{sanitize(self.parameters_repr)}"
         else:
             params = ""
         generated = f"@{sanitize(self.generated_name)}" if self.generated_name else ""
diff --git a/src/python/pants/core/goals/package.py b/src/python/pants/core/goals/package.py
--- a/src/python/pants/core/goals/package.py
+++ b/src/python/pants/core/goals/package.py
@@ -72,11 +72,14 @@ class OutputPathField(StringField, AsyncFieldMixin):
     def value_or_default(self, *, file_ending: str | None) -> str:
         if self.value:
             return self.value
-        file_prefix = (
+        target_name_part = (
             self.address.generated_name.replace(".", "_")
             if self.address.generated_name
             else self.address.target_name
         )
+        params_sanitized = self.address.parameters_repr.replace(".", "_")
+        file_prefix = f"{target_name_part}{params_sanitized}"
+
         if file_ending is None:
             file_name = file_prefix
         else:


Test patch:
Repository: pantsbuild/pants

Issue:
Please add Brand24 to the public list of Pants Users
### Company name

Brand24

### Company website

https://brand24.com

### Company logo

![logo_brand24_black_small](https://user-images.githubusercontent.com/36135334/187741686-89d1e4f7-37ea-4827-91c0-c90d130e751c.png)



Discussion:
Thank you! The person who manages the Who Uses Pants page is OOO for a couple days, so expect this issue to land next week.

Resolving patch:
diff --git a/build-support/bin/generate_user_list.py b/build-support/bin/generate_user_list.py
--- a/build-support/bin/generate_user_list.py
+++ b/build-support/bin/generate_user_list.py
@@ -40,6 +40,7 @@ class Org:
 # Orgs will be displayed in case-insensitive alphabetical order, but it's useful for human readers
 # to keep this list in that order too.
 _orgs = (
+    Org("Brand24", "https://brand24.com/", "https://files.readme.io/e3203d1-brand24-small.png"),
     Org(
         "Chartbeat", "https://chartbeat.com/", "https://files.readme.io/861ace7-chartbeat-small.png"
     ),


Test patch:
Repository: pantsbuild/pants

Issue:
Double-logging results when FmtResult is promoted to LintResult
**Describe the bug**
Since both are marked as engine aware, we get the double-logging during `lint`.

**Pants version**
`main`



Discussion:
If there's no obvious clean solution, I'm liable to add a `_render` field to `LintResult` and set it to `False` when converting and wipe my hands clean. :smile: 
I am not aware of a clean solution currently, BUT I did have a realization about our existing facilities:

Currently, both `EngineAwareParameter` and `EngineAwareReturnType` are "node metadata"... i.e., they can be used to attach/control the metadata of a node in the graph (which has an associated workunit).

What we don't have currently, and which might might be useful for this use case is "edge metadata"... i.e. each consumer of a node might have different metadata about how/why it is consuming it. Even if this were not truly "metadata" and instead just "a streaming logging statement attached to the edge", I think it's an interesting point. cc @Eric-Arellano 
Yeah, I think that would be super helpful and it avoids weird side-effects of things like refactoring a helper rule to return an `EngineAwareReturnType`
I think, for posterity, part of the reason we're thinking about this too is because `LintResult` and `FmtResult` have metadata solely for logging sake (tool name, partition description). Which means the rule has to take information from the request and construct the result with it just for the sake of logging it :unamused: 
Although @stuhood might be best to capture this in a new ticket, as it'll get lost in this one.

Resolving patch:
diff --git a/src/python/pants/core/goals/fmt.py b/src/python/pants/core/goals/fmt.py
--- a/src/python/pants/core/goals/fmt.py
+++ b/src/python/pants/core/goals/fmt.py
@@ -345,6 +345,7 @@ async def convert_fmt_result_to_lint_result(fmt_result: FmtResult) -> LintResult
         fmt_result.stdout,
         fmt_result.stderr,
         linter_name=fmt_result.formatter_name,
+        _render_message=False,  # Don't re-render the message
     )
 
 
diff --git a/src/python/pants/core/goals/lint.py b/src/python/pants/core/goals/lint.py
--- a/src/python/pants/core/goals/lint.py
+++ b/src/python/pants/core/goals/lint.py
@@ -58,6 +58,7 @@ class LintResult(EngineAwareReturnType):
     linter_name: str
     partition_description: str | None = None
     report: Digest = EMPTY_DIGEST
+    _render_message: bool = True
 
     @classmethod
     def create(
@@ -84,7 +85,11 @@ def metadata(self) -> dict[str, Any]:
         return {"partition": self.partition_description}
 
     def level(self) -> LogLevel | None:
-        return LogLevel.ERROR if self.exit_code != 0 else LogLevel.INFO
+        if not self._render_message:
+            return LogLevel.TRACE
+        if self.exit_code != 0:
+            return LogLevel.ERROR
+        return LogLevel.INFO
 
     def message(self) -> str | None:
         message = self.linter_name


Test patch: