internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: spack/spack

Issue:
HIP_PATH and other HIP variables added to modulefiles with HIP dependency
### Steps to reproduce

I created an environment that solely builds the following spec: `blaspp +rocm amdgpu_target=gfx90a`, and finds all dependencies as externals. Spack.yaml is provided at the bottom of this message. Build & install runs fine, but when creating the Lmod modulefile, variables like `ROCM_PATH`, `HIP_PATH`, `HSA_PATH`, etc, are being added to the `blaspp` modulefile. `HIP_PATH` is coming from here: https://github.com/spack/spack/blob/7d4dcd52d96b593fb5e1557794ad57a7c5eb2304/var/spack/repos/builtin/packages/hip/package.py#L565

`HIP_PATH` causes minor breakages beginning in ROCm/5.5.1 and major breakages beginning in 5.7.1.
Apparently .yaml's can't be uploaded, so here's the yaml file with a .txt prefix :) 
[spack.yaml.txt](https://github.com/spack/spack/files/14285413/spack.yaml.txt)

Here is the problematic section of the blaspp modulefile:
```
setenv("ROCM_PATH", "/opt/rocm-5.3.0")
setenv("HIP_PLATFORM", "amd")
setenv("HIP_COMPILER", "clang")
setenv("HIP_CLANG_PATH", "/opt/rocm-5.3.0/llvm/bin")
setenv("HSA_PATH", "/opt/rocm-5.3.0/hsa")
setenv("ROCMINFO_PATH", "/opt/rocm-5.3.0")
setenv("DEVICE_LIB_PATH", "/opt/rocm-5.3.0/amdgcn/bitcode")
setenv("HIP_DEVICE_LIB_PATH", "/opt/rocm-5.3.0/amdgcn/bitcode")
setenv("HIP_PATH", "/opt/rocm-5.3.0/hip")
setenv("LLVM_PATH", "/opt/rocm-5.3.0/llvm")
setenv("COMGR_PATH", "/opt/rocm-5.3.0")
append_path("HIPCC_COMPILE_FLAGS_APPEND", "--rocm-path=/opt/rocm-5.3.0", " ")
setenv("HCC_AMDGPU_TARGET", "gfx90a")
prepend_path("LD_LIBRARY_PATH", "/opt/rocm-5.3.0/llvm/lib", ":")
```

### Error message

_No response_

### Information on your system

Frontier @ OLCF:
$ spack debug report
* **Spack:** 0.21.1 (e30fedab102f9281a220fb4fae82e3f8c43a82ac)
* **Python:** 3.6.15
* **Platform:** linux-sles15-zen3
* **Concretizer:** clingo

### General information

- [X] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [X] I have searched the issues of this repo and believe this is not a duplicate
- [X] I have run the failing commands in debug mode and reported the output


Discussion:

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/hip/package.py b/var/spack/repos/builtin/packages/hip/package.py
--- a/var/spack/repos/builtin/packages/hip/package.py
+++ b/var/spack/repos/builtin/packages/hip/package.py
@@ -415,7 +415,9 @@ def set_variables(self, env):
             env.set("HIP_DEVICE_LIB_PATH", paths["bitcode"])
 
             # Just the prefix of hip (used in hipcc)
-            env.set("HIP_PATH", paths["hip-path"])
+            # Deprecated in 5.1.0 and breaks hipcc in 5.5.1+
+            if self.spec.satisfies("@:5.4"):
+                env.set("HIP_PATH", paths["hip-path"])
 
             # Used in comgr and seems necessary when using the JIT compiler, e.g.
             # hiprtcCreateProgram:


Test patch:
Repository: spack/spack

Issue:
Internal error when requesting a compiler that is not available
### Steps to reproduce

Try to concretize:
```console
$ spack spec zlib-ng %nvhpc
```
with no `nvhpc` compiler available

### Error message

Spack will prompt the following:
```console
$ spack spec zlib-ng %nvhpc
==> Error: Internal Spack error: the solver completed but produced specs that do not satisfy the request.
	Unsatisfied input specs:
	Input spec: zlib-ng%nvhpc
	(No candidate specs from solver)
```
while it should state that no compiler is available to satisfy the request.

### Information on your system

* **Spack:** 0.22.0.dev0 (5f5fc78236f8ad84121ac1d9aabf47d509676349)
* **Python:** 3.11.5
* **Platform:** linux-ubuntu20.04-icelake
* **Concretizer:** clingo


### General information

- [X] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [X] I have searched the issues of this repo and believe this is not a duplicate
- [X] I have run the failing commands in debug mode and reported the output


Discussion:
@scheibelp We likely introduced this in #42655 

Resolving patch:
diff --git a/lib/spack/spack/solver/asp.py b/lib/spack/spack/solver/asp.py
--- a/lib/spack/spack/solver/asp.py
+++ b/lib/spack/spack/solver/asp.py
@@ -823,11 +823,12 @@ def on_model(model):
             print("Statistics:")
             pprint.pprint(self.control.statistics)
 
-        if result.unsolved_specs and setup.concretize_everything:
+        if result.satisfiable and result.unsolved_specs and setup.concretize_everything:
             unsolved_str = Result.format_unsolved(result.unsolved_specs)
             raise InternalConcretizerError(
                 "Internal Spack error: the solver completed but produced specs"
-                f" that do not satisfy the request.\n\t{unsolved_str}"
+                " that do not satisfy the request. Please report a bug at "
+                f"https://github.com/spack/spack/issues\n\t{unsolved_str}"
             )
 
         return result, timer, self.control.statistics


Test patch:
diff --git a/lib/spack/spack/test/concretize.py b/lib/spack/spack/test/concretize.py
--- a/lib/spack/spack/test/concretize.py
+++ b/lib/spack/spack/test/concretize.py
@@ -2114,6 +2114,15 @@ def test_unsolved_specs_raises_error(self, monkeypatch, mock_packages, config):
         ):
             solver.driver.solve(setup, specs, reuse=[])
 
+    @pytest.mark.regression("43141")
+    @pytest.mark.only_clingo("Use case not supported by the original concretizer")
+    def test_clear_error_when_unknown_compiler_requested(self, mock_packages, config):
+        """Tests that the solver can report a case where the compiler cannot be set"""
+        with pytest.raises(
+            spack.error.UnsatisfiableSpecError, match="Cannot set the required compiler: a%foo"
+        ):
+            Spec("a %foo").concretized()
+
     @pytest.mark.regression("36339")
     def test_compiler_match_constraints_when_selected(self):
         """Test that, when multiple compilers with the same name are in the configuration
Repository: spack/spack

Issue:
Revert "ci.py: simplify, and dont warn excessively about externals"
Reverts spack/spack#43759

There is some error computing stages. We can merge this PR if we don't get a fix quickly.


Discussion:

Resolving patch:
diff --git a/lib/spack/spack/ci.py b/lib/spack/spack/ci.py
--- a/lib/spack/spack/ci.py
+++ b/lib/spack/spack/ci.py
@@ -205,7 +205,7 @@ def _extract_dag(specs: List[spack.spec.Spec]) -> Tuple[PlainNodes, PlainEdges]:
     nodes: PlainNodes = {}
     edges: PlainEdges = defaultdict(set)
 
-    for edge in traverse.traverse_edges(specs):
+    for edge in traverse.traverse_edges(specs, cover="edges"):
         if (edge.parent and edge.parent.external) or edge.spec.external:
             continue
         child_id = _spec_ci_label(edge.spec)


Test patch:
Repository: spack/spack

Issue:
libc lost iconv variant
### Steps to reproduce

`libc` used to be an actual package with an `iconv` variant, which was used as an external.

Now it's virtual and provided by `glibc` and `musl`, which are automatic externals, and do not have the variant.

We may need to restore this for glibc and maybe fix some packages.

### Error message

_No response_

### Information on your system

.

### General information

- [X] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [X] I have searched the issues of this repo and believe this is not a duplicate
- [X] I have run the failing commands in debug mode and reported the output


Discussion:
The old `libc` concrete recipe not only had `iconv` and `rpc` variants, but also had `provides` clauses for the corresponding virtual packages. `iconv` could be provided by `libc` or `iconv`, and `rpc` could be provided by `libc` or `libtirpc`. RedHat-like distributions are known for having Glibc built this way, and multiple packages have features in their build systems (usually via autotools) to handle this.

This is analogous to (but separable from) the `gettext`/`libintl` issue.

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/glibc/package.py b/var/spack/repos/builtin/packages/glibc/package.py
--- a/var/spack/repos/builtin/packages/glibc/package.py
+++ b/var/spack/repos/builtin/packages/glibc/package.py
@@ -29,6 +29,7 @@ class Glibc(AutotoolsPackage, GNUMirrorPackage):
     license("LGPL-2.1-or-later")
 
     provides("libc")
+    provides("iconv")
 
     version("master", branch="master")
     version("2.39", sha256="97f84f3b7588cd54093a6f6389b0c1a81e70d99708d74963a2e3eab7c7dc942d")
diff --git a/var/spack/repos/builtin/packages/musl/package.py b/var/spack/repos/builtin/packages/musl/package.py
--- a/var/spack/repos/builtin/packages/musl/package.py
+++ b/var/spack/repos/builtin/packages/musl/package.py
@@ -34,6 +34,7 @@ class Musl(MakefilePackage):
     representative_headers = ["iso646.h"]
 
     provides("libc")
+    provides("iconv")
 
     version("1.2.4", sha256="7a35eae33d5372a7c0da1188de798726f68825513b7ae3ebe97aaaa52114f039")
     version("1.2.3", sha256="7d5b0b6062521e4627e099e4c9dc8248d32a30285e959b7eecaa780cf8cfd4a4")


Test patch:
Repository: spack/spack

Issue:
Fix csh patch() for ncl
This PR adds `@run_before("install")` for the patch() function in the ncl recipe. Without this, on a system without /bin/csh, the build fails because the replacement doesn't get applied, so the Configure and config/* scripts can't run.


Discussion:
That was my understanding as well, so I was a little thrown off by this. I'm not sure if it reflects something that might have changed in the backend, like which step the patch() function gets run in. In any case, one alternative would be to consolidate what's there into the existing filter_sbang() function.

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/ncl/package.py b/var/spack/repos/builtin/packages/ncl/package.py
--- a/var/spack/repos/builtin/packages/ncl/package.py
+++ b/var/spack/repos/builtin/packages/ncl/package.py
@@ -136,18 +136,26 @@ def patch(self):
         # Make configure scripts use Spack's tcsh
         files = ["Configure"] + glob.glob("config/*")
 
-        filter_file("^#!/bin/csh -f", "#!/usr/bin/env csh", *files)
-
-    @run_before("install")
-    def filter_sbang(self):
         # Filter sbang before install so Spack's sbang hook can fix it up
-        files = glob.glob("ncarg2d/src/bin/scripts/*")
+        files += glob.glob("ncarg2d/src/bin/scripts/*")
         files += glob.glob("ncarview/src/bin/scripts/*")
         files += glob.glob("ni/src/scripts/*")
 
         csh = join_path(self.spec["tcsh"].prefix.bin, "csh")
 
-        filter_file("^#!/bin/csh", "#!{0}".format(csh), *files)
+        filter_file("^#!/bin/csh.*", "#!{0}".format(csh), *files)
+
+        if self.spec.satisfies("+grib"):
+            # Newer versions of libjasper do not provide the inmem property
+            if self.spec.satisfies("^jasper@2"):
+                filter_file("image.inmem_=1;", "", "external/g2clib-1.6.0/enc_jpeg2000.c")
+
+            filter_file("SUBDIRS = ", "SUBDIRS = g2clib-1.6.0 ", "external/yMakefile")
+            filter_file(
+                "INC=.*",
+                "INC=%s" % self.spec["jasper"].prefix.include,
+                "external/g2clib-1.6.0/makefile",
+            )
 
     def install(self, spec, prefix):
         if (self.compiler.fc is None) or (self.compiler.cc is None):
@@ -398,16 +406,3 @@ def delete_files(*filenames):
                     os.remove(filename)
                 except OSError as e:
                     raise InstallError("Failed to delete file %s: %s" % (e.filename, e.strerror))
-
-    @when("+grib")
-    def patch(self):
-        # Newer versions of libjasper do not provide the inmem property
-        if self.spec.satisfies("^jasper@2"):
-            filter_file("image.inmem_=1;", "", "external/g2clib-1.6.0/enc_jpeg2000.c")
-
-        filter_file("SUBDIRS = ", "SUBDIRS = g2clib-1.6.0 ", "external/yMakefile")
-        filter_file(
-            "INC=.*",
-            "INC=%s" % self.spec["jasper"].prefix.include,
-            "external/g2clib-1.6.0/makefile",
-        )


Test patch:
Repository: capitalone/DataProfiler

Issue:
Windows Install error - ValueError: path 'resources/' cannot end with '/
https://github.com/capitalone/DataProfiler/blob/5b04b7fe5ee3556235c397efb69b32cd5d364a3b/setup.py#L33

Ran into an install isue 
ValueError: path 'resources/' cannot end with '/

As per 
https://stackoverflow.com/questions/20356482/valueerror-path-conf-cannot-end-with

resource_dir = "resources/"
needs to change to 
resource_dir = "resources"

Thank you. 


Discussion:
@rxm7706 If you are interested in making a PR to fix, we'd be happy to support. Or we can add it to our backlog.

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -30,7 +30,7 @@
 with open(path.join(here, "requirements-reports.txt"), encoding="utf-8") as f:
     reports_packages = f.read().splitlines()
 
-resource_dir = "resources/"
+resource_dir = "resources"
 default_labeler_files = [
     (d, [os.path.join(d, f) for f in files]) for d, _, files in os.walk(resource_dir)
 ]


Test patch:
Repository: PaddlePaddle/models

Issue:
Add loading model function for train.py



Discussion:

Resolving patch:
diff --git a/deep_speech_2/train.py b/deep_speech_2/train.py
--- a/deep_speech_2/train.py
+++ b/deep_speech_2/train.py
@@ -11,6 +11,7 @@
 from model import deep_speech2
 from audio_data_utils import DataGenerator
 import numpy as np
+import os
 
 #TODO: add WER metric
 
@@ -78,6 +79,13 @@
     default='data/eng_vocab.txt',
     type=str,
     help="Vocabulary filepath. (default: %(default)s)")
+parser.add_argument(
+    "--init_model_path",
+    default=None,
+    type=str,
+    help="If set None, the training will start from scratch. "
+    "Otherwise, the training will resume from "
+    "the existing model of this path. (default: %(default)s)")
 args = parser.parse_args()
 
 
@@ -114,8 +122,14 @@ def train():
         rnn_size=args.rnn_layer_size,
         is_inference=False)
 
-    # create parameters and optimizer
-    parameters = paddle.parameters.create(cost)
+    # create/load parameters and optimizer
+    if args.init_model_path is None:
+        parameters = paddle.parameters.create(cost)
+    else:
+        if not os.path.isfile(args.init_model_path):
+            raise IOError("Invalid model!")
+        parameters = paddle.parameters.Parameters.from_tar(
+            gzip.open(args.init_model_path))
     optimizer = paddle.optimizer.Adam(
         learning_rate=args.adam_learning_rate, gradient_clipping_threshold=400)
     trainer = paddle.trainer.SGD(


Test patch:
Repository: PaddlePaddle/models

Issue:
Modify inference script



Discussion:

Resolving patch:
diff --git a/hsigmoid/infer.py b/hsigmoid/infer.py
--- a/hsigmoid/infer.py
+++ b/hsigmoid/infer.py
@@ -41,9 +41,8 @@ def decode_res(infer_res, dict_size):
     return predict_lbls
 
 
-def predict(batch_ins, idx_word_dict, dict_size, prediction_layer, parameters):
-    infer_res = paddle.infer(
-        output_layer=prediction_layer, parameters=parameters, input=batch_ins)
+def predict(batch_ins, idx_word_dict, dict_size, inferer):
+    infer_res = inferer.infer(input=batch_ins)
 
     predict_lbls = decode_res(infer_res, dict_size)
     predict_words = [idx_word_dict[lbl] for lbl in predict_lbls]  # map to word
@@ -66,6 +65,8 @@ def main(model_path):
     with gzip.open(model_path, "r") as f:
         parameters = paddle.parameters.Parameters.from_tar(f)
 
+    inferer = paddle.inference.Inference(
+        output_layer=prediction_layer, parameters=parameters)
     idx_word_dict = dict((v, k) for k, v in word_dict.items())
     batch_size = 64
     batch_ins = []
@@ -74,13 +75,11 @@ def main(model_path):
     for ins in ins_iter():
         batch_ins.append(ins[:-1])
         if len(batch_ins) == batch_size:
-            predict(batch_ins, idx_word_dict, dict_size, prediction_layer,
-                    parameters)
+            predict(batch_ins, idx_word_dict, dict_size, inferer)
             batch_ins = []
 
     if len(batch_ins) > 0:
-        predict(batch_ins, idx_word_dict, dict_size, prediction_layer,
-                parameters)
+        predict(batch_ins, idx_word_dict, dict_size, inferer)
 
 
 if __name__ == "__main__":


Test patch:
Repository: PaddlePaddle/models

Issue:
DS2 reader: Move local data variable from global into class DataGenerator.



Discussion:

Resolving patch:
diff --git a/deep_speech_2/data_utils/data.py b/deep_speech_2/data_utils/data.py
--- a/deep_speech_2/data_utils/data.py
+++ b/deep_speech_2/data_utils/data.py
@@ -17,11 +17,6 @@
 from data_utils.speech import SpeechSegment
 from data_utils.normalizer import FeatureNormalizer
 
-# for caching tar files info
-local_data = local()
-local_data.tar2info = {}
-local_data.tar2object = {}
-
 
 class DataGenerator(object):
     """
@@ -89,6 +84,10 @@ def __init__(self,
         self._num_threads = num_threads
         self._rng = random.Random(random_seed)
         self._epoch = 0
+        # for caching tar files info
+        self.local_data = local()
+        self.local_data.tar2info = {}
+        self.local_data.tar2object = {}
 
     def process_utterance(self, filename, transcript):
         """Load, augment, featurize and normalize for speech data.
@@ -241,16 +240,16 @@ def _get_file_object(self, file):
         """
         if file.startswith('tar:'):
             tarpath, filename = file.split(':', 1)[1].split('#', 1)
-            if 'tar2info' not in local_data.__dict__:
-                local_data.tar2info = {}
-            if 'tar2object' not in local_data.__dict__:
-                local_data.tar2object = {}
-            if tarpath not in local_data.tar2info:
+            if 'tar2info' not in self.local_data.__dict__:
+                self.local_data.tar2info = {}
+            if 'tar2object' not in self.local_data.__dict__:
+                self.local_data.tar2object = {}
+            if tarpath not in self.local_data.tar2info:
                 object, infoes = self._parse_tar(tarpath)
-                local_data.tar2info[tarpath] = infoes
-                local_data.tar2object[tarpath] = object
-            return local_data.tar2object[tarpath].extractfile(
-                local_data.tar2info[tarpath][filename])
+                self.local_data.tar2info[tarpath] = infoes
+                self.local_data.tar2object[tarpath] = object
+            return self.local_data.tar2object[tarpath].extractfile(
+                self.local_data.tar2info[tarpath][filename])
         else:
             return open(file, 'r')
 


Test patch:
Repository: PaddlePaddle/models

Issue:
Rename self.local_data to self._local_data in class DataGenerator.



Discussion:

Resolving patch:
diff --git a/deep_speech_2/data_utils/data.py b/deep_speech_2/data_utils/data.py
--- a/deep_speech_2/data_utils/data.py
+++ b/deep_speech_2/data_utils/data.py
@@ -85,9 +85,9 @@ def __init__(self,
         self._rng = random.Random(random_seed)
         self._epoch = 0
         # for caching tar files info
-        self.local_data = local()
-        self.local_data.tar2info = {}
-        self.local_data.tar2object = {}
+        self._local_data = local()
+        self._local_data.tar2info = {}
+        self._local_data.tar2object = {}
 
     def process_utterance(self, filename, transcript):
         """Load, augment, featurize and normalize for speech data.
@@ -240,16 +240,16 @@ def _get_file_object(self, file):
         """
         if file.startswith('tar:'):
             tarpath, filename = file.split(':', 1)[1].split('#', 1)
-            if 'tar2info' not in self.local_data.__dict__:
-                self.local_data.tar2info = {}
-            if 'tar2object' not in self.local_data.__dict__:
-                self.local_data.tar2object = {}
-            if tarpath not in self.local_data.tar2info:
+            if 'tar2info' not in self._local_data.__dict__:
+                self._local_data.tar2info = {}
+            if 'tar2object' not in self._local_data.__dict__:
+                self._local_data.tar2object = {}
+            if tarpath not in self._local_data.tar2info:
                 object, infoes = self._parse_tar(tarpath)
-                self.local_data.tar2info[tarpath] = infoes
-                self.local_data.tar2object[tarpath] = object
-            return self.local_data.tar2object[tarpath].extractfile(
-                self.local_data.tar2info[tarpath][filename])
+                self._local_data.tar2info[tarpath] = infoes
+                self._local_data.tar2object[tarpath] = object
+            return self._local_data.tar2object[tarpath].extractfile(
+                self._local_data.tar2info[tarpath][filename])
         else:
             return open(file, 'r')
 


Test patch:
Repository: PaddlePaddle/models

Issue:
Encounter exception when running tools/compute_mean_std.py
```
Traceback (most recent call last):
  File "tools/compute_mean_std.py", line 64, in <module>
    main()
  File "tools/compute_mean_std.py", line 59, in main
    num_samples=args.num_samples)
  File "/home/disk1/yangyaming/workspace/paddle/ds2/mandarin_unicom/default_conf/tools/../data_utils/normalizer.py", line 46, in __init__
    self._compute_mean_std(manifest_path, featurize_func, num_samples)
  File "/home/disk1/yangyaming/workspace/paddle/ds2/mandarin_unicom/default_conf/tools/../data_utils/normalizer.py", line 84, in _compute_mean_std
    AudioSegment.from_file(instance["audio_filepath"])))
  File "tools/compute_mean_std.py", line 53, in augment_and_featurize
    return audio_featurizer.featurize(audio_segment)
  File "/home/disk1/yangyaming/workspace/paddle/ds2/mandarin_unicom/default_conf/tools/../data_utils/featurizer/audio_featurizer.py", line 79, in featurize
    allow_upsampling)):
NameError: global name 'allow_upsampling' is not defined
```
Look like a typo problem.


Discussion:

Resolving patch:
diff --git a/deep_speech_2/data_utils/featurizer/audio_featurizer.py b/deep_speech_2/data_utils/featurizer/audio_featurizer.py
--- a/deep_speech_2/data_utils/featurizer/audio_featurizer.py
+++ b/deep_speech_2/data_utils/featurizer/audio_featurizer.py
@@ -57,7 +57,7 @@ def __init__(self,
     def featurize(self,
                   audio_segment,
                   allow_downsampling=True,
-                  allow_upsamplling=True):
+                  allow_upsampling=True):
         """Extract audio features from AudioSegment or SpeechSegment.
 
         :param audio_segment: Audio/speech segment to extract features from.


Test patch: