internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: google-research/text-to-text-transfer-transformer

Issue:
Run T5 on a CPU for inference
Hi, great work on T5!  

I'm looking to run T5 on a CPU for interactive predictions only, no fine-tuning.  The given notebook provides great instructions for using T5 with a TPU, but I'm struggling to find how I could use it with a CPU?

I've tried changing the notebook similar to this:
```
model = t5.models.MtfModel(
    model_dir=MODEL_DIR,
    tpu=None,
    model_parallelism=model_parallelism,
    batch_size=train_batch_size,
    layout_rules="ensemble:ensemble,batch:batch,d_ff:model,heads:model,vocab:model,experts:batch,model1:batch1,cpu:0", # sometimes I include this, sometimes I don't - it doesn't seem to matter
    sequence_length={"inputs": 128, "targets": 32},
    learning_rate_schedule=0.003,
    save_checkpoints_steps=5000,
    keep_checkpoint_max=keep_checkpoint_max if ON_CLOUD else None,
    iterations_per_loop=100,
)
```

But I get these errors:
```
ValueError: Tensor dimension size not divisible by mesh dimension size: tensor_shape=Shape[outer_batch=1, batch=4, length=128] tensor_layout=TensorLayout(None, 0, None)
```
It seems likely that it has something to do with my TensorLayout being `none`.  Would you mind giving me some tips for this?  Thanks a bunch in advance.


Discussion:
@nshazeer do you have any ideas?
I think that the mesh_shape needs to be empty "" for single-device CPU execution.  It looks like the MtfModel constructor is hard-coded to always call utils.tpu_mesh_shape().  That should probably be changed. 

Resolving patch:
diff --git a/t5/models/mtf_model.py b/t5/models/mtf_model.py
--- a/t5/models/mtf_model.py
+++ b/t5/models/mtf_model.py
@@ -126,7 +126,9 @@ def __init__(
       iterations_per_loop: integer, steps per train loop
     """
 
-    mesh_shape = utils.tpu_mesh_shape(tpu_topology, model_parallelism)
+    # TODO(adarob): Allow setting mesh_shape and mesh_devices directly for GPU.
+    mesh_shape = (
+        utils.tpu_mesh_shape(tpu_topology, model_parallelism) if tpu else "")
 
     sequence_length = sequence_length or {"inputs": 512, "targets": 512}
 


Test patch:
Repository: google-research/text-to-text-transfer-transformer

Issue:
Regarding masking function
@adarob Thank you for your quickly answer.

I don't understand one more thing in your code.

In 

https://github.com/google-research/text-to-text-transfer-transformer/blob/4598ccbda22ba051e565b8d13326afe1cbd1de33/t5/data/preprocessors.py#L2115,

you describe as below:

We want to generate training examples like
  "We hold X to be Y that Z" -> "X these truths Y self evident Z that"

In my assumption, its original sentence is "We hold these truths to be self evident that".
Following the 'T5' paper, it is converted as

Input: We hold X to be Y that
Target: X these truths Y self evident Z

I would like to know why the target has 'that'.
Additionally, in the language processing, there should be '(BOS)' and '(EOS)' tokens in the input and target.

In my best knowledge, the input and target will be

Input: We hold X to be Y that (EOS)
target: (BOS) X these truths Y self evident Z (EOS)

Is my assumption correct?

_Originally posted by @h19920918 in https://github.com/google-research/text-to-text-transfer-transformer/issues/147#issuecomment-607605929_


Discussion:
Yes, there is a typo in the docstring, the "that" should not be there at the end of the target.

The token sequences will eventually have EOS and BOS tokens added; the example is just to illustrate what the function does, not the actual token sequences that result at the end of our data pipeline.

Resolving patch:
diff --git a/t5/data/preprocessors.py b/t5/data/preprocessors.py
--- a/t5/data/preprocessors.py
+++ b/t5/data/preprocessors.py
@@ -2113,7 +2113,7 @@ def noise_span_to_unique_sentinel(tokens, noise_mask, vocabulary):
   with the markers in the targets.
 
   We want to generate training examples like
-  "We hold X to be Y that Z" -> "X these truths Y self evident Z that"
+  "We hold X to be Y that" -> "X these truths Y self evident Z"
 
   Sentinels assigned in decreasing order within the sequence starting at
   vocabulary.size - 1.  That is, we appropriate the last tokens in the


Test patch:
Repository: google-research/text-to-text-transfer-transformer

Issue:
rate_num_examples Doesn't work with new tasks
Hi,

I am training new mixture tasks (new tasks from tsv files), but it only works with "default_rate=1.0".
When I try to use "default_rate=rate_num_examples", it doesn't work because it can't find the tasks in the cache.

In issue #15 you mentioned the cache is only supported in your internal infrastructure.

So, how we can use the rate_num_examples in new mixture tasks ?



Discussion:
This is a known issue that we are working on. For now, you can hardcode the number of examples in each task when constructing the mixture.
@craffel Thanks for the reply.

For now, I set it manually to:

t5.data.MixtureRegistry.add(
    "mixtask",
    [["task1",0.34], ["task2",0.34],["task3",0.21],["task3",0.11]],
)

Hopefully, you fix it soon.
Thanks, yes, it is at the top of our to-do list!
@craffel Is the rate is correct as in the above example or should we use the actual number of samples?

I have checked the code and the task sampling is generated using "tf.data.experimental.sample_from_datasets" which uses "math_ops.log".

I am not sure should we pass the actual number of examples for each task or the rate of each task which is sum of 1 ?

using rates the math_ops.log leads to :
[0.34, 0.34, 0.21, 0.11] -> [-1.0788096 -1.0788096 -1.5606477 -2.207275 ]
using actual number of samples :
[10791.0, 10791.0, 6621.0, 3744.0] -> [9.286468 9.286468 8.798001 8.22791 ]

each one of them leads to totally different results using the multi-task training.
It uses a log to convert the weights to logits before feeding them into a multinomial distribution for sampling. This is not a detail you need to worry about. It will sample proportionally according to the weights as you'd expect.
test

Resolving patch:
diff --git a/t5/data/utils.py b/t5/data/utils.py
--- a/t5/data/utils.py
+++ b/t5/data/utils.py
@@ -1032,10 +1032,20 @@ def filter_features(ex):
 
 
 @gin.configurable
-def rate_num_examples(task, maximum=None, temperature=1.0, scale=1.0):
+def rate_num_examples(
+    task, maximum=None, temperature=1.0, scale=1.0,
+    fallback_to_num_input_examples=True):
   """Mixing rate equal to the number of examples for the task."""
-  # TODO(adarob): Support case when there are no cached stats.
-  ret = task.get_cached_stats("train")["examples"]
+
+  if task.cache_dir or not fallback_to_num_input_examples:
+    ret = task.get_cached_stats("train")["examples"]
+  else:
+    logging.warning(
+        "Task '%s' not cached so using number of input examples instead of "
+        "preprocessed examples to compute rate.",
+        task.name)
+    ret = task.num_input_examples("train")
+
   ret *= scale
   if maximum:
     ret = min(ret, maximum)


Test patch:
Repository: google-research/text-to-text-transfer-transformer

Issue:
Issue  Running T5 in colab TPU
Hi Team,

I was trying to do a pre training of T5 from scratch on colab. I could see if i install t5 using (pip install t5[gcp]), and tried to connect to execute ` tf.tpu.experimental.initialize_tpu_system(tpu)`, getting  below error.

`InvalidArgumentError: NodeDef expected inputs 'string' do not match 0 inputs specified; Op<name=_Send; signature=tensor:T -> ; attr=T:type; attr=tensor_name:string; attr=send_device:string; attr=send_device_incarnation:int; attr=recv_device:string; attr=client_terminated:bool,default=false; is_stateful=true>; NodeDef: {{node _Send}}`

If install/ upgrade tensorflow, it gets resolved, however import of t5 does not work as below.
`
import t5`

`NotFoundError: /usr/local/lib/python3.6/dist-packages/tensorflow_text/python/metrics/_text_similarity_metric_ops.so: undefined symbol: _ZN10tensorflow14kernel_factory17OpKernelRegistrar12InitInternalEPKNS_9KernelDefEN4absl11string_viewESt10unique_ptrINS0_15OpKernelFactoryESt14default_deleteIS8_EE`

Please let me know how if there is a way to resolve this.
Thanks.



Discussion:
I have a similar issue on Colab.

My workaround is to install the following commit:
!pip install -q git+https://github.com/google-research/text-to-text-transfer-transformer.git@1e269e72a981fde4ea64a88a0a0d8cc88871e20a

I think the problem lies in the conflict of the current TensorFlow 2.3 version which installed on Colab by default and the new T5 commit that force reinstalling TensorFlow 2.2.

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -56,7 +56,7 @@
         'scipy',
         'sentencepiece',
         'six>=1.14',  # TODO(adarob): Remove once rouge-score is updated.
-        'tensorflow-text<2.3',  # TODO(adarob): Unpin once #320 is resolved.
+        'tensorflow-text',
         'tfds-nightly',
         'torch',
         'transformers>=2.7.0',


Test patch:
Repository: google-research/text-to-text-transfer-transformer

Issue:
Unable to import tensorflow_gcs_config in t5_trivia colab notebook
Upon running line `import tensorflow_gcs_config` (in t5_trivia colab notebook, setup section) I get this error,
```
---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
<ipython-input-2-3bb7f36f8553> in <module>()
----> 1 import tensorflow_gcs_config

1 frames
/usr/local/lib/python3.6/dist-packages/tensorflow_gcs_config/__init__.py in _load_library(filename, lib)
     55   raise NotImplementedError(
     56       "unable to open file: " +
---> 57       "{}, from paths: {}\ncaused by: {}".format(filename, filenames, errs))
     58 
     59 _gcs_config_so = _load_library("_gcs_config_ops.so")

NotImplementedError: unable to open file: _gcs_config_ops.so, from paths: ['/usr/local/lib/python3.6/dist-packages/tensorflow_gcs_config/_gcs_config_ops.so']
caused by: ['/usr/local/lib/python3.6/dist-packages/tensorflow_gcs_config/_gcs_config_ops.so: undefined symbol: _ZN10tensorflow15OpKernelContext5inputEN4absl11string_viewEPPKNS_6TensorE']
```
`tf.__version__` is '2.3.0'


Discussion:
this solved my problem
`!pip install -U tensorflow-gcs-config==2.1.3`
Hi @Voil8, 

`!pip install -U tensorflow-gcs-config==2.1.3` seems to solve this problem last week but it doesn' t work today.

It raises the following error:
```
InvalidArgumentError: NodeDef expected inputs 'string' do not match 0 inputs specified; Op<name=GcsConfigureCredentials; signature=json:string -> >; NodeDef: {{node GcsConfigureCredentials}}
``` 

Do you face the same problem?
> Hi @Voil8,
> 
> `!pip install -U tensorflow-gcs-config==2.1.3` seems to solve this problem last week but it doesn' t work today.
> 
> It raises the following error:
> 
> ```
> InvalidArgumentError: NodeDef expected inputs 'string' do not match 0 inputs specified; Op<name=GcsConfigureCredentials; signature=json:string -> >; NodeDef: {{node GcsConfigureCredentials}}
> ```
> 
> Do you face the same problem?

Try this
`!pip install -U tensorflow-gcs-config==2.3.0`

I have the same issue as @Kevinweisl 
`!pip install -U tensorflow-gcs-config==2.3.0` gives me the original `NotImplementedError: unable to open file: _gcs_config_ops.so` error on the hosted runtime.

Should we reopen this?
I agree. As a workaround you may try various combination of TF version and gcs config version. For me TF 2.3.0 + gcs config 2.3.0 worked, hence I suggested.
It looks like one of our dependencies is downgrading tf.

Resolving patch:
diff --git a/t5/version.py b/t5/version.py
--- a/t5/version.py
+++ b/t5/version.py
@@ -18,4 +18,4 @@
 Stored in a separate file so that setup.py can reference the version without
 pulling in all the dependencies in __init__.py.
 """
-__version__ = '0.6.3'
+__version__ = '0.6.4'


Test patch:
Repository: google-research/text-to-text-transfer-transformer

Issue:
Running hf_model.py 
I am trying to run your models with [`hf_model`](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/models/hf_model.py). The current blocker issue is that the code is using `num_parallel_calls` in  [in multiple places](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/models/hf_model.py#L128), however, this function seems to be [deprecated](https://github.com/google-research/text-to-text-transfer-transformer/blob/838157d433995473e96b773c9c761b6aadf01e37/t5/data/preprocessors.py#L2651).

Wondering if there is a replacement for this function I can use as a quick fix.  


Discussion:
Ah, they should all be set to `tf.data.experimental.AUTOTUNE`. I will fix this. Thanks for the heads up.

Resolving patch:
diff --git a/t5/models/hf_model.py b/t5/models/hf_model.py
--- a/t5/models/hf_model.py
+++ b/t5/models/hf_model.py
@@ -125,7 +125,7 @@ def _map_fn(ex):
 
   dataset = dataset.map(
       _map_fn,
-      num_parallel_calls=t5.data.preprocessors.num_parallel_calls()
+      num_parallel_calls=tf.data.experimental.AUTOTUNE,
   )
 
   dataset = dataset.batch(batch_size, drop_remainder=False)
@@ -546,7 +546,7 @@ def predict(
     dataset = tf.data.Dataset.from_tensor_slices(inputs)
     dataset = dataset.map(
         lambda x: {"inputs": tf.cast(vocabs["inputs"].encode_tf(x), tf.int64)},
-        num_parallel_calls=t5.data.preprocessors.num_parallel_calls()
+        num_parallel_calls=tf.data.experimental.AUTOTUNE,
     )
     dataset = tokens_to_batches(
         dataset, sequence_length, batch_size, ["inputs"]


Test patch:
Repository: google-research/text-to-text-transfer-transformer

Issue:
Running hf_model.py 
I am trying to run your models with [`hf_model`](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/models/hf_model.py). The current blocker issue is that the code is using `num_parallel_calls` in  [in multiple places](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/models/hf_model.py#L128), however, this function seems to be [deprecated](https://github.com/google-research/text-to-text-transfer-transformer/blob/838157d433995473e96b773c9c761b6aadf01e37/t5/data/preprocessors.py#L2651).

Wondering if there is a replacement for this function I can use as a quick fix.  


Discussion:
Ah, they should all be set to `tf.data.experimental.AUTOTUNE`. I will fix this. Thanks for the heads up.
Thanks @craffel! 

Now that we're on this thread: 
Might be helpful to mention that the model specified by [`model_dir`](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/models/hf_model.py#L177) is a Pytorch mode, not a TF model (just in case there is any confusion). 

Even better, if you provide an example script for converting TF models to PyTorch ones: 
```python
from transformers import T5Config, T5Tokenizer, T5ForConditionalGeneration
from transformers.modeling_t5 import load_tf_weights_in_t5

base_model = "t5-small"
tokenizer = T5Tokenizer.from_pretrained(base_model)
model = T5ForConditionalGeneration(T5Config.from_pretrained(base_model))
tf_model_path = ... 
load_tf_weights_in_t5(model, None, path)
model.save_pretrained(path + "pytorch_model/")
tokenizer.save_pretrained(path + "pytorch_model/")
```


Hi this issue is not yet fixed when one install your repo with pip, could you please have a look? thanks 
@danyaljj PRs welcome :) 
@juliahane This has not made it into the pip release yet, you need to install the current code from GitHub. You can do this with pip via `pip install git+https://github.com/google-research/text-to-text-transfer-transformer.git`

Resolving patch:
diff --git a/t5/version.py b/t5/version.py
--- a/t5/version.py
+++ b/t5/version.py
@@ -18,4 +18,4 @@
 Stored in a separate file so that setup.py can reference the version without
 pulling in all the dependencies in __init__.py.
 """
-__version__ = '0.7.0'
+__version__ = '0.7.1'


Test patch:
Repository: google-research/text-to-text-transfer-transformer

Issue:
No values in Mixture Registry
**THE ISSUES SECTION IS ONLY FOR FILING BUGS. PLEASE ASK YOUR QUESTION ON THE DISCUSSION TAB.**

I ran the script provided [here](https://github.com/google-research/text-to-text-transfer-transformer/blob/main/t5/models/hf_model.py#L39), after installing T5 in my environment, but got:
```
  File "run_t5_glue_test.py", line 12, in <module>
    model.eval(
  File "/home/paulwu/miniconda3/envs/t5/lib/python3.8/site-packages/t5/models/hf_model.py", line 445, in eval
    utils.run_eval(
  File "/home/paulwu/miniconda3/envs/t5/lib/python3.8/site-packages/t5/models/utils.py", line 288, in run_eval
    vocabulary = get_vocabulary(mixture_or_task_name)
  File "/home/paulwu/miniconda3/envs/t5/lib/python3.8/site-packages/t5/models/utils.py", line 118, in get_vocabulary
    provider = t5.data.get_mixture_or_task(mixture_or_task_name)
  File "/home/paulwu/miniconda3/envs/t5/lib/python3.8/site-packages/seqio/dataset_providers.py", line 1517, in get_mixture_or_task
    raise ValueError(
ValueError: No Task or Mixture found with name 'glue_cola_v002'. Available:
```
When I try the command `python -c "import t5; print(t5.data.MixtureRegistry.names())"`, I got this:
```
2022-02-18 19:36:08.448536: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2022-02-18 19:36:08.448560: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
dict_keys([])
```
Could anyone help me to get mixture registry, please?



Discussion:
You'll need to also include `import t5.data.mixtures` to register those mixtures.
I've created a PR to update the example string.

Resolving patch:
diff --git a/t5/models/hf_model.py b/t5/models/hf_model.py
--- a/t5/models/hf_model.py
+++ b/t5/models/hf_model.py
@@ -27,6 +27,7 @@
 import functools
 
 import t5
+import t5.data.mixtures
 import t5.models
 import torch
 import transformers


Test patch:
Repository: carltongibson/django-filter

Issue:
Incompatible with django-debug-toolbar versions panel
django-debug-toolbar versions panel expects a parameterless get_version() method.

https://github.com/django-debug-toolbar/django-debug-toolbar/blob/master/debug_toolbar/panels/versions.py#L67

Your get_version method requires a parameter (version)

https://github.com/alex/django-filter/blob/develop/django_filters/__init__.py#L9

Is there a way you could change your get_version method signature to be compatible?  Been running django-debug-toolbar for a long time and I have never run into this with any other 3rd party apps.



Discussion:

Resolving patch:
diff --git a/django_filters/__init__.py b/django_filters/__init__.py
--- a/django_filters/__init__.py
+++ b/django_filters/__init__.py
@@ -6,7 +6,7 @@
 __version__ = '0.9.0'
 
 
-def get_version(version):
+def parse_version(version):
     '''
     '0.1.2-dev' -> (0, 1, 2, 'dev')
     '0.1.2' -> (0, 1, 2)
@@ -21,4 +21,4 @@ def get_version(version):
             ret.append(p)
     return tuple(ret)
 
-VERSION = get_version(__version__)
+VERSION = parse_version(__version__)


Test patch:
Repository: carltongibson/django-filter

Issue:
Django 1.8 support
Django 1.8 `replaces django.db.models.related.RelatedObject`  with `django.db.models.fields.related.ForeignObjectRel`. 

https://docs.djangoproject.com/en/1.8/releases/1.8/#model-field-related

`RelatedObject` is used in `django_filter/filterset.py`. However, even when I fix the import and usage, tests related to relations remains broken.



Discussion:

Resolving patch:
diff --git a/django_filters/filterset.py b/django_filters/filterset.py
--- a/django_filters/filterset.py
+++ b/django_filters/filterset.py
@@ -8,7 +8,6 @@
 from django.core.validators import EMPTY_VALUES
 from django.db import models
 from django.db.models.fields import FieldDoesNotExist
-from django.db.models.related import RelatedObject
 from django.utils import six
 from django.utils.text import capfirst
 from django.utils.translation import ugettext as _
@@ -26,6 +25,12 @@
     # Django < 1.5 fallback
     from django.utils.datastructures import SortedDict as OrderedDict  # noqa
 
+try:
+    from django.db.models.related import RelatedObject as ForeignObjectRel
+except ImportError:  # pragma: nocover
+    # Django >= 1.8 replaces RelatedObject with ForeignObjectRel
+    from django.db.models.fields.related import ForeignObjectRel
+
 
 from .filters import (Filter, CharFilter, BooleanFilter,
     ChoiceFilter, DateFilter, DateTimeFilter, TimeFilter, ModelChoiceFilter,
@@ -73,7 +78,7 @@ def get_model_field(model, f):
             rel = opts.get_field_by_name(name)[0]
         except FieldDoesNotExist:
             return None
-        if isinstance(rel, RelatedObject):
+        if isinstance(rel, ForeignObjectRel):
             model = rel.model
             opts = rel.opts
         else:
@@ -103,7 +108,7 @@ def filters_for_model(model, fields=None, exclude=None, filter_for_field=None,
         if field is None:
             field_dict[f] = None
             continue
-        if isinstance(field, RelatedObject):
+        if isinstance(field, ForeignObjectRel):
             filter_ = filter_for_reverse_field(field, f)
             if filter_:
                 field_dict[f] = filter_
@@ -426,7 +431,7 @@ def filter_for_field(cls, f, name, lookup_type='exact'):
     @classmethod
     def filter_for_reverse_field(cls, f, name):
         rel = f.field.rel
-        queryset = f.model._default_manager.all()
+        queryset = f.field.model._default_manager.all()
         default = {
             'name': name,
             'label': capfirst(rel.related_name),


Test patch: