Repository: pantsbuild/pants

Issue:
`KeyError` in `StreamingWorkunitContext.get_expanded_specs()`
**Describe the bug**
We've started to observe this error at work, occasionally:
```
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/github/home/.cache/nce/cdc3a4cfddcd63b6cebdd75b14970e02d8ef0ac5be4d350e57ab5df56c19e85e/cpython-3.9.15+20221106-x86_64-unknown-linux-gnu-install_only.tar.gz/python/lib/python3.9/threading.py", line 980, in _bootstrap_inner
    self.run()
  File "/github/home/.cache/nce/6621291ed4120fa45607b367abf7786ce496f389f27240712a63c31f1c438c21/bindings/venvs/2.16.0a0/lib/python3.9/site-packages/pants/engine/streaming_workunit_handler.py", line 298, in run
    self.poll_workunits(finished=True)
  File "/github/home/.cache/nce/6621291ed4120fa45607b367abf7786ce496f389f27240712a63c31f1c438c21/bindings/venvs/2.16.0a0/lib/python3.9/site-packages/pants/engine/streaming_workunit_handler.py", line 281, in poll_workunits
    callback(
  File "/github/home/.cache/pants/named_caches/pex_root/venvs/s/aad2bc99/venv/lib/python3.9/site-packages/toolchain/pants/buildsense/reporter.py", line 98, in __call__
    self.handle_workunits(
  File "/github/home/.cache/pants/named_caches/pex_root/venvs/s/aad2bc99/venv/lib/python3.9/site-packages/toolchain/pants/buildsense/reporter.py", line 126, in handle_workunits
    self._on_finish(context, self._call_count, work_units_map)
  File "/github/home/.cache/pants/named_caches/pex_root/venvs/s/aad2bc99/venv/lib/python3.9/site-packages/toolchain/pants/buildsense/reporter.py", line 152, in _on_finish
    run_tracker_info = self._get_run_tracker_info(context)
  File "/github/home/.cache/pants/named_caches/pex_root/venvs/s/aad2bc99/venv/lib/python3.9/site-packages/toolchain/pants/buildsense/reporter.py", line 183, in _get_run_tracker_info
    targets_specs = _get_expanded_specs(context)
  File "/github/home/.cache/pants/named_caches/pex_root/venvs/s/aad2bc99/venv/lib/python3.9/site-packages/toolchain/pants/buildsense/reporter.py", line 201, in _get_expanded_specs
    targets = context.get_expanded_specs().targets
  File "/github/home/.cache/nce/6621291ed4120fa45607b367abf7786ce496f389f27240712a63c31f1c438c21/bindings/venvs/2.16.0a0/lib/python3.9/site-packages/pants/engine/streaming_workunit_handler.py", line 119, in get_expanded_specs
    source = targets_dict[str(target.address.maybe_convert_to_target_generator())]
KeyError: 'src/python/.../files:files'
Error: Process completed with exit code 1.
```

It is for different target addresses, but so far they've all been for a `python_sources` target.


**Pants version**
`2.16.0a0`

**OS**
Linux (ubuntu, GitHub CI runner)

**Additional info**
Started when we switched from the old pants bootstrap script to use `scie-pants`.

We do use parametrizations for multiple resolves.

https://github.com/pantsbuild/pants/blob/d37d57a0640e7171d9174859d8fc69f5cde8b975/src/python/pants/engine/streaming_workunit_handler.py#L115-L120



Discussion:

Resolving patch:
diff --git a/src/python/pants/engine/streaming_workunit_handler.py b/src/python/pants/engine/streaming_workunit_handler.py
--- a/src/python/pants/engine/streaming_workunit_handler.py
+++ b/src/python/pants/engine/streaming_workunit_handler.py
@@ -23,6 +23,7 @@
 from pants.goal.run_tracker import RunTracker
 from pants.option.options_bootstrapper import OptionsBootstrapper
 from pants.util.logging import LogLevel
+from pants.util.strutil import softwrap
 
 logger = logging.getLogger(__name__)
 
@@ -114,9 +115,33 @@ def get_expanded_specs(self) -> ExpandedSpecs:
 
         targets_dict: dict[str, list[TargetInfo]] = {str(addr): [] for addr in unexpanded_addresses}
         for target in expanded_targets:
-            source = targets_dict.get(str(target.address.spec), None)
+            target_spec = str(target.address.spec)
+            source = targets_dict.get(target_spec)
             if source is None:
-                source = targets_dict[str(target.address.maybe_convert_to_target_generator())]
+                target_gen_spec = str(target.address.maybe_convert_to_target_generator())
+                source = targets_dict.get(target_gen_spec)
+                if source is None:
+                    # This is a thing, that may need investigating to be fully understood.
+                    # merely patches over a crash here. See #18564.
+                    logger.warn(
+                        softwrap(
+                            f"""
+                            Unknown source address for target: {target_spec}
+                            Target address generator: {target_gen_spec}
+
+                            Input params:
+                            {params}
+
+                            Unexpanded addresses:
+                            {unexpanded_addresses}
+
+                            Expanded targets:
+                            {expanded_targets}
+                            """
+                        )
+                    )
+                    targets_dict[target_gen_spec or target_spec] = source = []
+
             source.append(
                 TargetInfo(
                     filename=(


Test patch: