internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: google/jax

Issue:
jnp.select should use jax.lax.select_n
### Description

jnp.select is a function that's like jnp.where, but selects between n different arrays. jnp.where is a wrapper around jax.lax.select, which makes it more flexible in terms of input shapes and dtypes.

jax.lax.select_n (added in #9482 ) is a generalization of jax.lax.select to pick from n arrays rather than 2; it translates at lower time into a binary tree of jax.lax.select's, which is helpful for efficiency when there are many branches. jnp.select doesn't use this: it linearly chains jnp.where's across the different arrays. It would likely help performance if it did, particularly in the case where the condlist entries have lower ndim than the choicelist.

### System info (python version, jaxlib version, accelerator, etc.)

jax:    0.4.27
jaxlib: 0.4.27
numpy:  1.26.3
python: 3.11.8 (stable, redacted, redacted) [Clang google3-trunk (3b5e7c83a6e226d5bd7ed2e9b67449b64812074c)]
jax.devices (16 total, 16 local): [TpuDevice(id=0, process_index=0, coords=(0,0,0), core_on_chip=0) TpuDevice(id=1, process_index=0, coords=(0,0,0), core_on_chip=1) ... TpuDevice(id=14, process_index=0, coords=(3,1,0), core_on_chip=0) TpuDevice(id=15, process_index=0, coords=(3,1,0), core_on_chip=1)]
process_count: 1
platform: uname_result(system='Linux', node='ebe6e776479e64ea-65d7388ae36.borgtask.google.com', release='5.10.0-smp-1101.33.0.0', version='#1 [v5.10.0-1101.33.0.0] SMP @1708585970', machine='x86_64')



Discussion:

Resolving patch:
diff --git a/jax/_src/numpy/lax_numpy.py b/jax/_src/numpy/lax_numpy.py
--- a/jax/_src/numpy/lax_numpy.py
+++ b/jax/_src/numpy/lax_numpy.py
@@ -1162,12 +1162,12 @@ def select(
     raise ValueError(msg.format(len(condlist), len(choicelist)))
   if len(condlist) == 0:
     raise ValueError("condlist must be non-empty")
-  choices = util.promote_dtypes(default, *choicelist)
-  choicelist = choices[1:]
-  output = choices[0]
-  for cond, choice in zip(condlist[::-1], choicelist[::-1]):
-    output = where(cond, choice, output)
-  return output
+  # Put the default at front with condition False because
+  # argmax returns zero for an array of False values.
+  choicelist = util.promote_dtypes(default, *choicelist)
+  conditions = stack(broadcast_arrays(False, *condlist))
+  idx = argmax(conditions.astype(bool), axis=0)
+  return lax.select_n(*broadcast_arrays(idx, *choicelist))
 
 
 @util.implements(np.bincount, lax_description="""\


Test patch:
diff --git a/tests/lax_numpy_test.py b/tests/lax_numpy_test.py
--- a/tests/lax_numpy_test.py
+++ b/tests/lax_numpy_test.py
@@ -4532,6 +4532,7 @@ def testWhereScalarPromotion(self):
     # maximal set of dtypes.
     dtypes=itertools.combinations_with_replacement(all_dtypes, 3),
   )
+  @jax.numpy_rank_promotion('allow')
   def testSelect(self, n, shapes, dtypes):
     dtypes = dtypes[:n+1]
     rng = jtu.rand_default(self.rng())
Repository: google/jax

Issue:
`jit(pmap)` on key arrays raises an error in `pmap` lowering
### Description

While not really a bug, I was surprised to find a rogue variable `jax.random.key` that seems to be similar to PRNGKey but fails for the case`jit(pmap)`. Copilot dropped this surprise, as an autocomplete that I didn't notice until later on during debugging.

```python
import os

num_devices = 8

os.environ['XLA_FLAGS'] = f'--xla_force_host_platform_device_count={num_devices}'

import jax


def test():
    def single_fn(key):
        return jax.random.normal(key, ())

    def pmap_batched_fn(key):
        batch_size = num_devices
        keys = jax.random.split(key, batch_size)
        pmap_fn = jax.pmap(single_fn)
        return pmap_fn(keys)

    key = jax.random.key(0) # Some rogue jax.random.key
    # key = jax.random.PRNGKey(0) # Works of course

    # Passes pmap(...)
    result_pmap_no_jit = pmap_batched_fn(key)

    # Fails jit(pmap(...))
    result_pmap_jit = jax.jit(pmap_batched_fn)(key)


```

### System info (python version, jaxlib version, accelerator, etc.)

Present at least in JAX versions 0.4.20 to 0.4.25


Discussion:
`jax.random.key` is far from rogue. See:
* https://jax.readthedocs.io/en/latest/jax.random.html
* https://jax.readthedocs.io/en/latest/jep/9263-typed-keys.html
* #9263

The function was introduced quietly a few versions ago. We only recently switched the module documentation to mention it (in #17741), so it isn't in the documentation for a release yet.

So, this looks like a bug! Thanks for filing.

Also, since we changed the docs in the current release, we should remember to mention it in the changelog.

fyi @jakevdp 
Then I'm glad I used the correct issue template :)

Resolving patch:
diff --git a/jax/_src/interpreters/pxla.py b/jax/_src/interpreters/pxla.py
--- a/jax/_src/interpreters/pxla.py
+++ b/jax/_src/interpreters/pxla.py
@@ -1319,6 +1319,8 @@ def _hlo_shard(aval, axis_env, xs, in_axis):
   if aval is core.abstract_token:
     return xs
   elif isinstance(aval, core.ShapedArray):
+    if dtypes.issubdtype(aval.dtype, dtypes.extended):
+      aval = aval.dtype._rules.physical_element_aval(aval.dtype)
     x, = xs
     dims = list(aval.shape)
     zero = mlir.ir_constant(np.zeros((), dtype=np.uint32))


Test patch:
diff --git a/tests/pmap_test.py b/tests/pmap_test.py
--- a/tests/pmap_test.py
+++ b/tests/pmap_test.py
@@ -2207,6 +2207,15 @@ def test_pmap_stack_size(self):
     y = jax.pmap(jax.scipy.linalg.expm)(jnp.array([x, x]))
     y.block_until_ready()  # doesn't crash
 
+  def test_pmap_of_prng_key(self):
+    # Regression test for https://github.com/google/jax/issues/20392
+    keys = jax.random.split(jax.random.key(0), jax.device_count())
+    result1 = jax.pmap(jax.random.bits)(keys)
+    with jtu.ignore_warning(
+        category=UserWarning, message="The jitted function foo includes a pmap"):
+      result2 = jax.jit(jax.pmap(jax.random.bits))(keys)
+    self.assertArraysEqual(result1, result2)
+
 
 @jtu.pytest_mark_if_available('multiaccelerator')
 class CppPmapTest(PythonPmapTest):
Repository: google/jax

Issue:
Behavior of `jax.experimental.array_api.var` w/ `correction > n`
### Description

`jax.experimental.array_api.var` does not follow the special case specified in the [array API documentation of `var`](https://data-apis.org/array-api/latest/API_specification/generated/array_api.var.html) for `correction > N`.

> Let `N` equal the number of elements over which to compute the variance.
>
> - If `N - correction` is less than or equal to 0, the variance is NaN.

```python3
import jax.experimental.array_api as xp
xp.var(xp.asarray([1.]), correction=1)  # Array(nan, dtype=float32)
xp.var(xp.asarray([]), correction=1)  # Array(-0., dtype=float32)
```

In most cases, `-0` is be the result of following the usual formula. Whether it is applicable for `correction > n` is debatable, but at least in the case above, the mean is undefined, so it might be desirable to update `jax.numpy`, too.

```python3
import jax.numpy as jnp
jnp.mean(jnp.asarray([]))  # Array(nan, dtype=float32)
jnp.var(jnp.asarray([]), ddof=1)  # Array(-0., dtype=float32)
```

For context, I ran into this when adding array API support to `scipy.stats.bartlett` (scipy/scipy#20751).

### System info (python version, jaxlib version, accelerator, etc.)

```
jax:    0.4.28
jaxlib: 0.4.28
numpy:  1.25.2
python: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
jax.devices (1 total, 1 local): [CpuDevice(id=0)]
process_count: 1
platform: uname_result(system='Linux', node='2e6238110dfb', release='6.1.85+', version='#1 SMP PREEMPT_DYNAMIC Sun Apr 28 14:29:16 UTC 2024', machine='x86_64')
```


Discussion:

Resolving patch:
diff --git a/jax/_src/numpy/reductions.py b/jax/_src/numpy/reductions.py
--- a/jax/_src/numpy/reductions.py
+++ b/jax/_src/numpy/reductions.py
@@ -467,7 +467,7 @@ def _var(a: ArrayLike, axis: Axis = None, dtype: DTypeLike | None = None,
                      dtype=computation_dtype, keepdims=keepdims)
   normalizer = lax.sub(normalizer, lax.convert_element_type(ddof, computation_dtype))
   result = sum(centered, axis, dtype=computation_dtype, keepdims=keepdims, where=where)
-  return lax.div(result, normalizer).astype(dtype)
+  return _where(normalizer > 0, lax.div(result, normalizer).astype(dtype), np.nan)
 
 
 def _var_promote_types(a_dtype: DTypeLike, dtype: DTypeLike | None) -> tuple[DType, DType]:


Test patch:
diff --git a/tests/lax_numpy_reducers_test.py b/tests/lax_numpy_reducers_test.py
--- a/tests/lax_numpy_reducers_test.py
+++ b/tests/lax_numpy_reducers_test.py
@@ -571,6 +571,17 @@ def np_fun(x):
       self._CompileAndCheck(jnp_fun, args_maker, rtol=tol,
                             atol=tol)
 
+  @jtu.sample_product(
+    jnp_fn=[jnp.var, jnp.std],
+    size=[0, 1, 2]
+  )
+  def testStdOrVarLargeDdofReturnsNan(self, jnp_fn, size):
+    # test for https://github.com/google/jax/issues/21330
+    x = jnp.arange(size)
+    self.assertTrue(np.isnan(jnp_fn(x, ddof=size)))
+    self.assertTrue(np.isnan(jnp_fn(x, ddof=size + 1)))
+    self.assertTrue(np.isnan(jnp_fn(x, ddof=size + 2)))
+
   @jtu.sample_product(
     shape=[(5,), (10, 5)],
     dtype=all_dtypes,
Repository: cleanlab/cleanlab

Issue:
Add CI to ensure that Jupyter notebooks don't include output
Preserving the output cells in Jupyter notebooks that are checked in to this repo bloat the repo size, and keeping the output is not necessary because the notebooks are run as part of the CI for producing docs.

For example, in #165, the `audio.ipynb` is 112K (and now part of the repo history forever), while clearing the output cells reduces that size down to 24K. I suspect that Git's delta compression will also work better without the output cells there as we make changes to the notebook. Also, the diffs will be more readable.

We should add CI that enforces that Jupyter notebooks checked in to the repo don't have any output cells. In the meantime, let's be careful and do this manually. CC @weijinglok @JohnsonKuan @jwmueller


Discussion:

Resolving patch:
diff --git a/.ci/nblint.py b/.ci/nblint.py
new file mode 100755
--- /dev/null
+++ b/.ci/nblint.py
@@ -0,0 +1,61 @@
+#!/usr/bin/env python
+
+"""
+Lint Jupyter notebooks being checked in to this repo.
+
+Currently, this "linter" only checks one property, that the notebook's output
+cells are empty, to avoid bloating the repository size.
+"""
+
+
+import argparse
+import json
+import os
+import sys
+
+
+def main():
+    opts = get_opts()
+    notebooks = find_notebooks(opts.dir)
+    for notebook in notebooks:
+        check(notebook)
+
+
+def get_opts():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("dir", help="Directories to search for notebooks", type=str, nargs="+")
+    return parser.parse_args()
+
+
+def find_notebooks(dirs):
+    notebooks = set()
+    for d in dirs:
+        for dirname, _, filenames in os.walk(d):
+            for filename in filenames:
+                if not filename.endswith(".ipynb"):
+                    continue
+                full_path = os.path.join(dirname, filename)
+                notebooks.add(full_path)
+    return notebooks
+
+
+def check(notebook):
+    with open(notebook) as f:
+        contents = json.load(f)
+    check_outputs_empty(notebook, contents)
+
+
+def check_outputs_empty(path, contents):
+    for i, cell in enumerate(contents["cells"]):
+        if "outputs" in cell and cell["outputs"] != []:
+            fail(path, "output is not empty", i)
+
+
+def fail(path, message, cell=None):
+    cell_msg = f" [cell {cell}]" if cell is not None else ""
+    print(f"{path}{cell_msg}: {message}")
+    sys.exit(1)
+
+
+if __name__ == "__main__":
+    main()


Test patch:
Repository: cleanlab/cleanlab

Issue:
Change labels arg -> y in CleanLearning.fit()
`CleanLearning.fit()` argument names should match those of sklearn fit()
Change labels arg -> y in CleanLearning.fit()
`CleanLearning.fit()` argument names should match those of sklearn fit()


Discussion:

Resolving patch:
diff --git a/cleanlab/classification.py b/cleanlab/classification.py
--- a/cleanlab/classification.py
+++ b/cleanlab/classification.py
@@ -263,7 +263,7 @@ def __init__(
     def fit(
         self,
         X,
-        labels,
+        labels=None,
         *,
         pred_probs=None,
         thresholds=None,
@@ -274,6 +274,7 @@ def fit(
         clf_kwargs={},
         clf_final_kwargs={},
         validation_func=None,
+        y=None,
     ):
         """
         Train the model `clf` with error-prone, noisy labels as if
@@ -388,6 +389,9 @@ class 0, 1, ..., K-1. `pred_probs` should have been computed using 3 (or
           on the cleaned data subset, you should explicitly pass in that data yourself
           (eg. via `clf_final_kwargs` or `clf_kwargs`).
 
+        y: np.array or pd.Series, optional
+          Alternative argument that can be specified instead of `labels`. Specifying `y` has the same effect as specifying `labels`, and is offered as an alternative for compatibility with sklearn.
+
         Returns
         -------
         CleanLearning
@@ -416,6 +420,13 @@ class 0, 1, ..., K-1. `pred_probs` should have been computed using 3 (or
             not just the subset of cleaned data left after pruning out the label issues.
         """
 
+        if labels is not None and y is not None:
+            raise ValueError("You must specify either `labels` or `y`, but not both.")
+        if y is not None:
+            labels = y
+        if labels is None:
+            raise ValueError("You must specify `labels`.")
+
         self.clf_final_kwargs = {**clf_kwargs, **clf_final_kwargs}
 
         if "sample_weight" in clf_kwargs:


Test patch:
diff --git a/tests/test_classification.py b/tests/test_classification.py
--- a/tests/test_classification.py
+++ b/tests/test_classification.py
@@ -491,6 +491,29 @@ def test_clf_fit_nm_inm(format):
     assert score < score_nm_inm + 1e-4
 
 
+@pytest.mark.parametrize("format", list(DATA_FORMATS.keys()))
+def test_clf_fit_y_alias(format):
+    data = DATA_FORMATS[format]
+    cl = CleanLearning(seed=SEED)
+
+    # Valid signature
+    cl.fit(data["X_train"], data["labels"])
+
+    # Valid signature for labels/y alias
+    cl.fit(data["X_train"], labels=data["labels"])
+    cl.fit(data["X_train"], y=data["labels"])
+    cl.fit(X=data["X_train"], labels=data["labels"])
+    cl.fit(X=data["X_train"], y=data["labels"])
+
+    # Invalid signatures
+    with pytest.raises(ValueError):
+        cl.fit(data["X_train"])
+    with pytest.raises(ValueError):
+        cl.fit(data["X_train"], data["labels"], y=data["labels"])
+    with pytest.raises(ValueError):
+        cl.fit(X=data["X_train"], labels=data["labels"], y=data["labels"])
+
+
 @pytest.mark.parametrize("format", list(DATA_FORMATS.keys()))
 def test_pred_and_pred_proba(format):
     data = DATA_FORMATS[format]
Repository: cleanlab/cleanlab

Issue:
CI: Suppress complex mypy typing in docs.cleanlab.ai
 If docs.cleanlab.ai shows the mypy type annotations, the current signatures look a bit intimidating, eg. for: `cleanlab.internal.validation`
https://docs.cleanlab.ai/master/cleanlab/internal/validation.html 


Plan is to use https://github.com/tox-dev/sphinx-autodoc-typehints to auto-tag parameters in docstrings based on the function signatures.

Consider the docstring for `cleanlab.internal.validation.labels_to_array`:
https://docs.cleanlab.ai/master/cleanlab/internal/validation.html#cleanlab.internal.validation.labels_to_array

The docstring would not change as the tags have been hard-coded. But the signature at the top would read:
```
cleanlab.internal.validation.labels_to_array(y)
```
instead of
```
cleanlab.internal.validation.labels_to_array(y: Union[list, ndarray, Series, DataFrame, generic]) → ndarray
```

This plan requires ironing out details with unrolling/expanding custom types like `LabelLike`. 
It will take some effort to review all hard-coded types/tags in docstrings, because they won't be replaced with the auto-tag.


Discussion:
Related to this, we might want to show `LabelLike` instead of expanding the type alias. See [this StackOverflow thread](https://stackoverflow.com/questions/60028577/keeping-alias-types-simple-in-python-documentation) for some approaches.

Resolving patch:
diff --git a/docs/source/conf.py b/docs/source/conf.py
--- a/docs/source/conf.py
+++ b/docs/source/conf.py
@@ -39,6 +39,7 @@
     "sphinx_multiversion",
     "sphinx_copybutton",
     "sphinxcontrib.katex",
+    "sphinx_autodoc_typehints",
 ]
 
 numpy_show_class_members = True
@@ -78,7 +79,7 @@
 napoleon_use_admonition_for_references = False
 napoleon_use_ivar = False
 napoleon_use_param = True
-napoleon_use_rtype = True
+napoleon_use_rtype = False
 napoleon_preprocess_types = True
 napoleon_type_aliases = None
 napoleon_attr_annotations = True


Test patch:
Repository: cleanlab/cleanlab

Issue:
fix return type of get_label_quality_scores in token classification
The function only returns a `Tuple[np.ndarray, list]`, but it is annotated with:

https://github.com/cleanlab/cleanlab/blob/fad4eb266dee8b9e2925d3f0d74fe4a81939eb8a/cleanlab/token_classification/rank.py#L36


Discussion:

Resolving patch:
diff --git a/cleanlab/token_classification/rank.py b/cleanlab/token_classification/rank.py
--- a/cleanlab/token_classification/rank.py
+++ b/cleanlab/token_classification/rank.py
@@ -33,7 +33,7 @@ def get_label_quality_scores(
     token_score_method: str = "self_confidence",
     sentence_score_method: str = "min",
     sentence_score_kwargs: dict = {},
-) -> Union[np.ndarray, Tuple[np.ndarray, list]]:
+) -> Tuple[np.ndarray, list]:
     """
     Returns overall quality scores for the labels in each sentence, as well as for the individual tokens' labels in a token classification dataset.
 


Test patch:
Repository: cleanlab/cleanlab

Issue:
Docs: Fix math in EMA docstring
I may have broken the math display format in EMA label-score docstring by adding extra slashes, but the docs build was also broken the way it was before with a single slash (there was some complaint about escaping this character that I don't precisely recall).

![Image](https://user-images.githubusercontent.com/1390638/205741893-00016696-0525-4789-be66-155307e9ab7c.png)

After this issue is fixed, we will want to replace the relevant section of the HTML file of the v2.2 docs in cleanlab-docs repo to update the stable documentation as well. I would be very cautious about replacing the entire HTML file and rather manually replace just the relevant section of this file.

## Code
https://github.com/cleanlab/cleanlab/blob/a14d810b9bf9f4f0ed7aae38a397b57e630f73be/cleanlab/internal/multilabel_scorer.py#L168



Discussion:

Resolving patch:
diff --git a/cleanlab/internal/multilabel_scorer.py b/cleanlab/internal/multilabel_scorer.py
--- a/cleanlab/internal/multilabel_scorer.py
+++ b/cleanlab/internal/multilabel_scorer.py
@@ -165,11 +165,11 @@ def exponential_moving_average(
 
     .. math::
 
-        \\text{EMA}_t = \\alpha \cdot s_t + (1 - \\alpha) \cdot \\text{EMA}_{t-1}, \\qquad 0 \\leq \\alpha \\leq 1
+        \text{EMA}_t = \alpha \cdot s_t + (1 - \alpha) \cdot \text{EMA}_{t-1}, \qquad 0 \leq \alpha \leq 1
 
-    We set :math:`\\text{EMA}_1 = s_1` as the largest score in the sorted vector s.
+    We set :math:`\text{EMA}_1 = s_1` as the largest score in the sorted vector s.
 
-    :math:`\\alpha` is the "forgetting factor" that gives more weight to the
+    :math:`\alpha` is the "forgetting factor" that gives more weight to the
     most recent scores, and successively less weight to the previous scores.
 
     Parameters


Test patch:
Repository: cleanlab/cleanlab

Issue:
Revert #961 before release
Tensorflow version temporarily has an upper bound (`tensorflow<2.16.0`) in requirements-dev.txt.
scikit-learn version temporarily has an upper bound (`scikit-learn>=1.0,<1.4.0`) in setup.py

This needs to be reverted before releasing v2.6.0.


 _Originally posted by @elisno in https://github.com/cleanlab/cleanlab/issues/961#issuecomment-1898968097_
            


Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -102,7 +102,7 @@ def run(self):
     # https://packaging.python.org/en/latest/discussions/install-requires-vs-requirements/
     install_requires=[
         "numpy>=1.20.0",
-        "scikit-learn>=1.0,<1.4.0",
+        "scikit-learn>=1.0",
         "tqdm>=4.53.0",
         "pandas>=1.1.5",
         "termcolor>=2.0.0,<2.4.0",


Test patch:
Repository: cleanlab/cleanlab

Issue:
Class Imbalance issue checker should not run if labels are not provided in Datalab
```
from cleanlab import Datalab

lab = Datalab(data=df_without_labels)
lab.find_issues()
```

It should not run the ClassImbalanceIssueManager, but it tries to anyway.

Just add a check that the Datlab had labels specified, then it can run the ClassImbalanceIssueManager in find_issues.


Discussion:

Resolving patch:
diff --git a/cleanlab/datalab/internal/issue_finder.py b/cleanlab/datalab/internal/issue_finder.py
--- a/cleanlab/datalab/internal/issue_finder.py
+++ b/cleanlab/datalab/internal/issue_finder.py
@@ -472,4 +472,12 @@ def get_available_issue_types(self, **kwargs):
                 )
                 issue_types_copy.pop("outlier")
 
+        drop_class_imbalance_check = (
+            "class_imbalance" in issue_types_copy
+            and not self.datalab.has_labels
+            and self.task == Task.CLASSIFICATION
+        )
+        if drop_class_imbalance_check:
+            issue_types_copy.pop("class_imbalance")
+
         return issue_types_copy


Test patch:
Repository: cleanlab/cleanlab

Issue:
exact issue name should be listed for each issue type in the Datalab Issue Type Guide
Otherwise it's hard to know how to run an audit for this issue type (eg. data valuation say).


Discussion:

Resolving patch:
diff --git a/docs/source/conf.py b/docs/source/conf.py
--- a/docs/source/conf.py
+++ b/docs/source/conf.py
@@ -54,6 +54,7 @@
     "sphinx_copybutton",
     "sphinxcontrib.katex",
     "sphinxcontrib.gtagjs",
+    "sphinx_jinja",
     "sphinx_autodoc_typehints",
     "sphinx.ext.doctest",
     "sphinxext.opengraph",


Test patch:
Repository: djangopackages/djangopackages

Issue:
Add errormator.com



Discussion:
https://errormator.com/page/python/django-exception-logging


Resolving patch:
diff --git a/settings/heroku.py b/settings/heroku.py
--- a/settings/heroku.py
+++ b/settings/heroku.py
@@ -168,3 +168,14 @@
 )
 
 ########## end templates
+
+#-------------------
+# appenlight-client
+#------------------
+
+import appenlight_client.client as e_client
+APPENLIGHT = e_client.get_config({'appenlight.api_key': os.environ.get('APPENLIGHT_KEY', '')})
+
+MIDDLEWARE_CLASSES += (
+    'appenlight_client.django_middleware.AppenlightMiddleware',
+)
\ No newline at end of file


Test patch: