internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: scverse/scanpy

Issue:
String conversion of `qc_vars` into a `Collection` when a `str` input is provided in `calculate_qc_metrics`
### What kind of feature would you like to request?

Additional function parameters / changed functionality / changed defaults?

### Please describe your wishes

I often make the mistake of providing a single string input to the `qc_vars` argument in `calculate_qc_metrics` which always fails with a `KeyError` because it beings to iterate over the string. This could be a very simple addition to just convert the argument into a `list` if a `str` is provided as input. 


Discussion:

Resolving patch:
diff --git a/scanpy/preprocessing/_qc.py b/scanpy/preprocessing/_qc.py
--- a/scanpy/preprocessing/_qc.py
+++ b/scanpy/preprocessing/_qc.py
@@ -233,7 +233,7 @@ def calculate_qc_metrics(
     *,
     expr_type: str = "counts",
     var_type: str = "genes",
-    qc_vars: Collection[str] = (),
+    qc_vars: Collection[str] | str = (),
     percent_top: Collection[int] | None = (50, 100, 200, 500),
     layer: str | None = None,
     use_raw: bool = False,
@@ -308,6 +308,10 @@ def calculate_qc_metrics(
     if issparse(X):
         X.eliminate_zeros()
 
+    # Convert qc_vars to list if str
+    if isinstance(qc_vars, str):
+        qc_vars = [qc_vars]
+
     obs_metrics = describe_obs(
         adata,
         expr_type=expr_type,


Test patch:
diff --git a/scanpy/tests/test_qc_metrics.py b/scanpy/tests/test_qc_metrics.py
--- a/scanpy/tests/test_qc_metrics.py
+++ b/scanpy/tests/test_qc_metrics.py
@@ -143,6 +143,16 @@ def test_qc_metrics_format(cls):
         assert np.allclose(adata.var[col], adata_dense.var[col])
 
 
+def test_qc_metrics_format_str_qc_vars():
+    adata_dense, init_var = adata_mito()
+    sc.pp.calculate_qc_metrics(adata_dense, qc_vars="mito", inplace=True)
+    adata = AnnData(X=adata_dense.X, var=init_var.copy())
+    sc.pp.calculate_qc_metrics(adata, qc_vars="mito", inplace=True)
+    assert np.allclose(adata.obs, adata_dense.obs)
+    for col in adata.var:  # np.allclose doesn't like mix of types
+        assert np.allclose(adata.var[col], adata_dense.var[col])
+
+
 def test_qc_metrics_percentage():  # In response to #421
     adata_dense, init_var = adata_mito()
     sc.pp.calculate_qc_metrics(adata_dense, percent_top=[])
Repository: scverse/scanpy

Issue:
scale max value only cuts high values
### Please make sure these conditions are met

- [X] I have checked that this issue has not already been reported.
- [X] I have confirmed this bug exists on the latest version of scanpy.
- [X] (optional) I have confirmed this bug exists on the main branch of scanpy.

### What happened?

Shouldn't max Value for `scale` and zero_center also clip the negative values?

### Minimal code sample

```python
bdata = sc.datasets.pbmc3k()
sc.pp.scale(bdata,max_value= 1)
print(bdata.X.min(),bdata.X.max())
```


### Error output

```pytb
-2.62718 1.0

shouldn't this be -1,1
```


### Versions

scanpy build from github main branch


Discussion:

Resolving patch:
diff --git a/scanpy/preprocessing/_simple.py b/scanpy/preprocessing/_simple.py
--- a/scanpy/preprocessing/_simple.py
+++ b/scanpy/preprocessing/_simple.py
@@ -871,7 +871,10 @@ def scale_array(
     # do the clipping
     if max_value is not None:
         logg.debug(f"... clipping at max_value {max_value}")
-        X[X > max_value] = max_value
+        if zero_center:
+            X = np.clip(X, a_min=-max_value, a_max=max_value)
+        else:
+            X[X > max_value] = max_value
     if return_mean_std:
         return X, mean, std
     else:


Test patch:
diff --git a/scanpy/tests/test_scaling.py b/scanpy/tests/test_scaling.py
--- a/scanpy/tests/test_scaling.py
+++ b/scanpy/tests/test_scaling.py
@@ -104,3 +104,12 @@ def test_mask_string():
     sc.pp.scale(adata, mask_obs="some cells")
     assert np.array_equal(adata.X, X_centered_for_mask)
     assert "mean of some cells" in adata.var.keys()
+
+
+@pytest.mark.parametrize("zero_center", [True, False])
+def test_clip(zero_center):
+    adata = sc.datasets.pbmc3k()
+    sc.pp.scale(adata, max_value=1, zero_center=zero_center)
+    if zero_center:
+        assert adata.X.min() >= -1
+    assert adata.X.max() <= 1
Repository: scverse/scanpy

Issue:
Add layer argument to `score_genes()`/`score_genes_cell_cycle()`
### What kind of feature would you like to request?

Additional function parameters / changed functionality / changed defaults?

### Please describe your wishes

These functions are some of the few to still not have this argument. Would also help with #2909.


Discussion:
Sounds good! Would you be up for a PR?

Resolving patch:
diff --git a/scanpy/tools/_score_genes.py b/scanpy/tools/_score_genes.py
--- a/scanpy/tools/_score_genes.py
+++ b/scanpy/tools/_score_genes.py
@@ -66,6 +66,7 @@ def score_genes(
     random_state: AnyRandom = 0,
     copy: bool = False,
     use_raw: bool | None = None,
+    layer: str | None = None,
 ) -> AnnData | None:
     """\
     Score a set of genes :cite:p:`Satija2015`.
@@ -101,6 +102,8 @@ def score_genes(
 
         .. versionchanged:: 1.4.5
            Default value changed from `False` to `None`.
+    layer
+        Key from `adata.layers` whose value will be used to perform tests on.
 
     Returns
     -------
@@ -145,7 +148,7 @@ def score_genes(
     # interval of expression.
 
     def get_subset(genes: pd.Index[str]):
-        x = _get_obs_rep(adata, use_raw=use_raw)
+        x = _get_obs_rep(adata, use_raw=use_raw, layer=layer)
         if len(genes) == len(var_names):
             return x
         idx = var_names.get_indexer(genes)


Test patch:
diff --git a/scanpy/tests/test_score_genes.py b/scanpy/tests/test_score_genes.py
--- a/scanpy/tests/test_score_genes.py
+++ b/scanpy/tests/test_score_genes.py
@@ -239,6 +239,23 @@ def test_use_raw_None():
     sc.tl.score_genes(adata, adata_raw.var_names[:3], use_raw=None)
 
 
+def test_layer():
+    adata = _create_adata(100, 1000, p_zero=0, p_nan=0)
+
+    sc.pp.normalize_per_cell(adata, counts_per_cell_after=1e4)
+    sc.pp.log1p(adata)
+
+    # score X
+    gene_set = adata.var_names[:10]
+    sc.tl.score_genes(adata, gene_set, score_name="X_score")
+    # score layer (`del` makes sure it actually uses the layer)
+    adata.layers["test"] = adata.X.copy()
+    del adata.X
+    sc.tl.score_genes(adata, gene_set, score_name="test_score", layer="test")
+
+    np.testing.assert_array_equal(adata.obs["X_score"], adata.obs["test_score"])
+
+
 @pytest.mark.parametrize("gene_pool", [[], ["foo", "bar"]])
 def test_invalid_gene_pool(gene_pool):
     adata = _create_adata(100, 1000, p_zero=0, p_nan=0)
Repository: scverse/scanpy

Issue:
Docs for aggregate are incorrect

![screenshot](https://github.com/scverse/scanpy/assets/8238804/5d21ca96-6186-4fee-8a40-fe258f7722d3)

The latter sentences of this docstring are totally incorrect, plus `AggType` is just an alias for the string literal values and isn't one we expose to users.


Discussion:

Resolving patch:
diff --git a/scanpy/get/_aggregated.py b/scanpy/get/_aggregated.py
--- a/scanpy/get/_aggregated.py
+++ b/scanpy/get/_aggregated.py
@@ -179,8 +179,6 @@ def aggregate(
     in the output `AnnData` object.
 
     If none of `layer`, `obsm`, or `varm` are passed in, `X` will be used for aggregation data.
-    If `func` only has length 1 or is just an `AggType`, then aggregation data is written to `X`.
-    Otherwise, it is written to `layers` or `xxxm` as appropriate for the dimensions of the aggregation data.
 
     Params
     ------


Test patch:
Repository: facebookresearch/ParlAI

Issue:
Return type ambiguous when extracting image features
The return type of the extracted image features (if the features file is present or not) is different. If the file is present then it returns a numpy.ndarray type object otherwise it returns a torch.autograd.variable.Variable object.
( https://github.com/facebookresearch/ParlAI/blob/3d86ccdbb4d87002cc6c4782afd0ee5277e742f1/parlai/core/image_featurizers.py#L149 )


Discussion:

Resolving patch:
diff --git a/parlai/core/image_featurizers.py b/parlai/core/image_featurizers.py
--- a/parlai/core/image_featurizers.py
+++ b/parlai/core/image_featurizers.py
@@ -74,7 +74,6 @@ def cuda(self):
         self.netCNN.cuda()
 
     def save(self, feature, path):
-        feature = feature.cpu().data.numpy()
         np.save(path, feature)
 
     def image_mode_switcher(self):
@@ -105,6 +104,7 @@ def extract(self, image, path):
         self.xs.data.copy_(self.transform(image))
         # extract the image feature
         feature = self.netCNN(self.xs)
+        feature = feature.cpu().data.numpy()
         # save the feature
         self.save(feature, path)
         return feature


Test patch:
Repository: facebookresearch/ParlAI

Issue:
Return type ambiguous when extracting image features
The return type of the extracted image features (if the features file is present or not) is different. If the file is present then it returns a numpy.ndarray type object otherwise it returns a torch.autograd.variable.Variable object.
( https://github.com/facebookresearch/ParlAI/blob/3d86ccdbb4d87002cc6c4782afd0ee5277e742f1/parlai/core/image_featurizers.py#L149 )


Discussion:

Resolving patch:
diff --git a/parlai/tasks/vqa_v1/agents.py b/parlai/tasks/vqa_v1/agents.py
--- a/parlai/tasks/vqa_v1/agents.py
+++ b/parlai/tasks/vqa_v1/agents.py
@@ -157,6 +157,30 @@ def act(self):
         return action
 
 
+class AllTeacher(OeTeacher):
+    """
+    VQA Teacher, which inherits from OeTeacher and gives access to
+    the multiple choices and the multiple choice answer.
+    """
+
+    def act(self):
+        action = super().act()
+
+        qa = self.ques['questions'][self.episode_idx]
+        multiple_choices = qa['multiple_choices']
+
+        action['label_candidates'] = multiple_choices
+
+        if not self.datatype.startswith('test'):
+            anno = self.annotation['annotations'][self.episode_idx]
+            self.mclabel = [anno['multiple_choice_answer']]
+
+        if self.datatype.startswith('train'):
+            action['mc_label'] = self.mclabel
+
+        return action
+
+
 class DefaultTeacher(McTeacher):
     # default to Multiple-Choice Teacher
     pass
diff --git a/parlai/tasks/vqa_v2/agents.py b/parlai/tasks/vqa_v2/agents.py
--- a/parlai/tasks/vqa_v2/agents.py
+++ b/parlai/tasks/vqa_v2/agents.py
@@ -135,5 +135,24 @@ def _setup_data(self, data_path, annotation_path):
                 self.annotation = json.load(data_file)
 
 
+class AllTeacher(OeTeacher):
+    """
+    VQA v2.0 Open-Ended teacher, which inherits from OeTeacher and 
+    gives access to the multiple choice answer.
+    """
+
+    def act(self):
+        action = super().act()
+
+        if not self.datatype.startswith('test'):
+            anno = self.annotation['annotations'][self.episode_idx]
+            self.mclabel = [anno['multiple_choice_answer']]
+
+        if self.datatype.startswith('train'):
+            action['mc_label'] = self.mclabel
+
+        return action
+
+
 class DefaultTeacher(OeTeacher):
     pass


Test patch:
Repository: facebookresearch/ParlAI

Issue:
Slow loading of image features during training
For VQA tasks, it takes massive amount of time to load the image features for training. The overhead is so much that for the same model it increases the training time by a factor of 50.


Discussion:

Resolving patch:
diff --git a/parlai/core/image_featurizers.py b/parlai/core/image_featurizers.py
--- a/parlai/core/image_featurizers.py
+++ b/parlai/core/image_featurizers.py
@@ -9,6 +9,7 @@
 import copy
 import numpy as np
 from PIL import Image
+from functools import lru_cache
 
 _greyscale = '  .,:;crsA23hHG#98&@'
 
@@ -121,6 +122,7 @@ def img_to_ascii(self, path):
             asc.append('\n')
         return ''.join(asc)
 
+    @lru_cache(maxsize=None)
     def load(self, path):
         opt = self.opt
         mode = opt.get('image_mode', 'raw')


Test patch:
Repository: facebookresearch/ParlAI

Issue:
Training hanged at loss.backward() of movie-dialog-qa
Hi, I am interested in Memory Networks and Movie Dialog QA. Although I tried below command, training stopped at first `loss.backward()` at `memnn.py`. I waited almost one day, but `loss.backward()` didn't finish. I have checked this by printing debug and `[Using Cuda]` printing. Actually my GPU was working because it used some memory. I checked this by `nvidia-smi -l 1`.

`python examples/train_model.py -m memnn -t "#moviedd-qa" -bs 32 --gpu 0 -e 10`

Then, I switched to simple task, and it finished a few minutes.
`python examples/train_model.py -m memnn -t "babi:task1k:1" -bs 32 --gpu 0 -e 10 `

I realize `#moviedd-qa` is more complicated compared to babi task. But how long does it usually take to train this model in my setting? Does anyone try to train this model via ParlAI? I am afraid this is not bug of ParlAI. Could you advise me to proceed my work?

### My Environment
- Ubunt 16.04.03 LTS, 64 bit
- python 3.6.1 (Anaconda 4.4.0 (64-bit))
- GPU: GTX 1080 ti
- CPU: Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz
- torch.\_\_version\_\_: '0.2.0_3'


Discussion:
Hi @jefst, thanks for flagging! We'll take a look when we get the chance. We've tested in quite a bit on babi, but we hadn't yet tried to reproduce the moviedd-qa results with this new pytorch implementation of the memory network.
Hi, thank you for your reply. I hope there will be some progress on this task, including #313 . 
So far, I am struggling to reproduce this task. 
[Evaluating Prerequisite Qualities for Learning End-to-End Dialog Systems](https://arxiv.org/abs/1511.06931)

@jefst is exactly me. I posted at my wrong account. I am sorry for the confusing.

Resolving patch:
diff --git a/parlai/agents/memnn/memnn.py b/parlai/agents/memnn/memnn.py
--- a/parlai/agents/memnn/memnn.py
+++ b/parlai/agents/memnn/memnn.py
@@ -129,6 +129,8 @@ def observe(self, observation):
         return observation
 
     def predict(self, xs, cands, ys=None):
+        # Subsample to reduce training time
+        cands = [list(set(random.sample(c, min(32, len(c))) + self.labels)) for c in cands]
         is_training = ys is not None
         self.model.train(mode=is_training)
         # Organize inputs for network (see contents of xs and ys in batchify method)


Test patch:
Repository: facebookresearch/ParlAI

Issue:
Bug report.
https://github.com/facebookresearch/ParlAI/blob/c6f03dc8d8f1df8a237542ce76e7d9be8d9e338d/parlai/core/utils.py#L435
This line, comparing Torch Variable to Int type, will issue this error: 
_RuntimeError: bool value of Variable objects containing non-empty torch.ByteTensor is ambiguous._

Modifying the former line as: `for c in predictions[i].data:` will solve this problem.


Discussion:
@alexholdenmiller  or @klshuster 

Resolving patch:
diff --git a/parlai/core/utils.py b/parlai/core/utils.py
--- a/parlai/core/utils.py
+++ b/parlai/core/utils.py
@@ -209,7 +209,7 @@ def make_batches(data, bsz):
 
 def maintain_dialog_history(history, observation, reply='',
                             historyLength=1, useReplies="labels",
-                            dict=None, useStartEndIndices=True, 
+                            dict=None, useStartEndIndices=True,
                             splitSentences=False):
     """Keeps track of dialog history, up to a truncation length.
     Either includes replies from the labels, model, or not all using param 'replies'."""
@@ -424,7 +424,7 @@ def map_predictions(cls, predictions, valid_inds, batch_reply, observations, dic
            using valid_inds.
            report_freq -- how often we report predictions
         """
-        predictions = predictions.cpu()
+        predictions = predictions.cpu().data
         for i in range(len(predictions)):
             # map the predictions back to non-empty examples in the batch
             # we join with spaces since we produce tokens one at a timelab


Test patch:
Repository: facebookresearch/ParlAI

Issue:
Can we keep a mturk task from outside parlai/mturk/tasks?
Hi @JackUrb, I have a few questions regarding the mturk evaluation:

1. This link (http://parl.ai/static/docs/mturk.html#running-a-task) says that 

> to run an MTurk task, first ensure that the task directory is in `parlai/mturk/tasks/`. 

 Is it by design? I tried to keep my task in some other directory (outside root parlai directory) and tried to import parlai as a package but that doesn't seem to work. Basically I am trying to use ParlAI as one of the dependency for one of my project for Human In the loop evaluation.

2. How easy/hard it is to provide support for keeping the mturk task anywhere?



Discussion:
This is by design within the ParlAI platform, as it allows use of the default ParlAI datapath (say you want to load up a model like in the QA_data_collection task, it uses the ParlAI datapath to locate and load the squad agent) and would make it easier to convert whatever you create into a ParlAI task, however it isn't necessarily required. You should be able to import and use ParlAI-MTurk elsewhere so I'm not 100% sure what is blocking you. Can you further explain what the issue with your import is?

You should be able to use:
```
from parlai.core.params import ParlaiParser
from <your_task_dir>.worlds import <your_world_files>
from parlai.mturk.core.mturk_manager import MTurkManager
```
as long as you've installed the parlai package, and then your run file should be able to be similar to the ones in the existing demos.

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -30,4 +30,5 @@
     packages=find_packages(exclude=(
         'data', 'docs', 'downloads', 'examples', 'logs', 'tests')),
     install_requires=reqs.strip().split('\n'),
+    include_package_data=True,
 )


Test patch:
Repository: facebookresearch/ParlAI

Issue:
Syntax Error dict.py
File "/home/cafagna/ParlAI/parlai/core/dict.py", line 653
    dictionary = (f'{k} {v}' for k, v in frequencies.items())

Syntax Error on execution


Discussion:
Are you using Python 3.5?

This change depends on Python 3.6, but I don't think that was intentional. @alexholdenmiller ?
Yes, I'm using Python 3.5, 
since I need this version and I don't know this particular sintax, you know how to correct the line in order to let it work?

Resolving patch:
diff --git a/parlai/core/dict.py b/parlai/core/dict.py
--- a/parlai/core/dict.py
+++ b/parlai/core/dict.py
@@ -660,9 +660,9 @@ def finalize(self, frequencies, num_symbols=30000, minfreq=2):
             # we already finalized the codecs
             return False
 
-        print('Dictionary: saving bpe codecs to {}'.format(self.codes))
+        print('Dictionary: saving bpe codecs to {}'.format(self.codecs))
 
-        dictionary = (f'{k} {v}' for k, v in frequencies.items())
+        dictionary = ("{} {}".format(k, v) for k, v in frequencies.items())
 
         if num_symbols <= 0:
             num_symbols = 30000


Test patch:
Repository: facebookresearch/ParlAI

Issue:
profilememory eval_ppl.py fails
from #1066 

when running eval_ppl.py I get:

Traceback (most recent call last):
  File "eval_ppl.py", line 147, in <module>
    eval_ppl(opt)
  File "/ParlAI/projects/convai2/eval_ppl.py", line 100, in eval_ppl
    return run_eval_ppl(opt, build_dict)
  File "/ParlAI/parlai/scripts/eval_ppl.py", line 214, in eval_ppl
    world.parley()  # process an example
  File "/ParlAI/parlai/scripts/eval_ppl.py", line 123, in parley
    probs = self.agent.next_word_probability(parsed[:i])
  File "/ParlAI/projects/convai2/baselines/profilememory/eval_ppl.py", line 66, in next_word_probability
    attn_mask = Variable(xs_persona.data.sum(2).ne(0), requires_grad=False)
RuntimeError: dimension out of range (expected to be in range of [-2, 1], but got 2)


Discussion:

Resolving patch:
diff --git a/projects/personachat/persona_seq2seq.py b/projects/personachat/persona_seq2seq.py
--- a/projects/personachat/persona_seq2seq.py
+++ b/projects/personachat/persona_seq2seq.py
@@ -1599,6 +1599,7 @@ def observe(self, obs):
                 self.persona_given = ''
             text_split = observation['text'].split('\n')
             if self.usepersona:
+                self.persona_given = ''
                 for t in text_split:
                     if 'persona' in t:
                         t = t.replace('your persona: ', '').replace('their persona: ', '')


Test patch:
Repository: facebookresearch/ParlAI

Issue:
fix predict function in s2s 
- In predict function, rename valid_cands => cand_indices



Discussion:

Resolving patch:
diff --git a/parlai/agents/seq2seq/seq2seq.py b/parlai/agents/seq2seq/seq2seq.py
--- a/parlai/agents/seq2seq/seq2seq.py
+++ b/parlai/agents/seq2seq/seq2seq.py
@@ -483,7 +483,7 @@ def observe(self, observation):
         self.answers[self.batch_idx] = None
         return obs
 
-    def predict(self, xs, ys=None, cands=None, valid_cands=None, is_training=False):
+    def predict(self, xs, ys=None, cands=None, cand_indices=None, is_training=False):
         """Produce a prediction from our model.
 
         Update the model using the targets if available, otherwise rank
@@ -523,7 +523,7 @@ def predict(self, xs, ys=None, cands=None, valid_cands=None, is_training=False):
             self.update_params()
         else:
             self.model.eval()
-            out = self.model(xs, ys=None, cands=cands, valid_cands=valid_cands, beam_size=self.beam_size, topk=self.topk)
+            out = self.model(xs, ys=None, cands=cands, cand_indices=cand_indices, beam_size=self.beam_size, topk=self.topk)
             predictions, cand_preds = out[0], out[2]
 
             if ys is not None:


Test patch:
Repository: facebookresearch/ParlAI

Issue:
Wrong error message in tfidf retriever
https://github.com/facebookresearch/ParlAI/blob/24b49f1c6921cd2a80f4c99a980c438b318e2b5a/parlai/agents/tfidf_retriever/tfidf_retriever.py#L14

should check other places as well

(should be `pip install scikit-learn`)


Discussion:

Resolving patch:
diff --git a/parlai/agents/tfidf_retriever/tfidf_retriever.py b/parlai/agents/tfidf_retriever/tfidf_retriever.py
--- a/parlai/agents/tfidf_retriever/tfidf_retriever.py
+++ b/parlai/agents/tfidf_retriever/tfidf_retriever.py
@@ -13,7 +13,7 @@
     import unicodedata  # noqa: F401
     import pexpect  # noqa: F401
 except ImportError:
-    raise ImportError('Please `pip install regex scipy sklearn pexpect`'
+    raise ImportError('Please `pip install regex scipy scikit-learn pexpect`'
                       ' to use the tfidf_retriever agent.')
 
 from parlai.core.agents import Agent


Test patch:
Repository: facebookresearch/ParlAI

Issue:
Pytorch Data Teacher seems to hang
I observe a weird hang when using the pytorch data teacher:

```
doesn't hang
PYTHONPATH=. python examples/display_data.py -t wizard_of_wikipedia:WizardDialogKnowledgeTeacher

seems to hang indefinitely
PYTHONPATH=. python examples/display_data.py -pyt wizard_of_wikipedia:WizardDialogKnowledgeTeacher
```


Discussion:

Resolving patch:
diff --git a/parlai/core/pytorch_data_teacher.py b/parlai/core/pytorch_data_teacher.py
--- a/parlai/core/pytorch_data_teacher.py
+++ b/parlai/core/pytorch_data_teacher.py
@@ -29,6 +29,13 @@
 from threading import Thread, Condition, RLock
 
 
+if torch.version.__version__.startswith('0.'):
+    raise ImportError(
+        "Please upgrade to PyTorch >=1.0; "
+        "visit https://pytorch.org for instructions."
+    )
+
+
 class BatchSortCache(object):
     """
         Object that encapsulates the functionality of the batch sort cache.


Test patch: