noamaanMulla-03/transformers-issues huggingface dataset:
[examples] run_clm re-processes dataset on every run
Author: stas00
Pull request: no

developing with `run_clm` is difficult since its startup is very slow - it rebuilds the dataset on each start.

@VictorSanh says it started to do that recently...

I think it's because it has to chunk the existing dataset into smaller pieces, it's a slow start everytime and it doesn't save these results.
So the original dataset has already been preprocessed, but it's not good enough for `run_clm.py`.

So I'm thinking perhaps for dev needs we need a dataset with short  <512 entries? and then it could use it w/o additional preprocessing?

But I could be wrong I haven't investigated the reason for the slow start.

to reproduce:
```
USE_TF=0 python  examples/pytorch/language-modeling/run_clm.py \
    --model_name_or_path gpt2 \
    --dataset_name "stas/openwebtext-10k" \
    --output_dir output_dir \
    --overwrite_output_dir \
    --do_train \
    --do_eval \
    --max_train_samples 1000 \
    --max_eval_samples 200 \
    --per_device_train_batch_size 4 \
    --per_device_eval_batch_size 4 \
    --num_train_epochs 1 \
    --warmup_steps 8 \
    --block_size 64 \
    --fp16 \
    --report_to none
```

So look at the tqdm bars before training starts to see the symptom. And this is already a very truncated dataset.

@VictorSanh, @sgugger 


Comments:
The dataset caching is all relying on the datasets library, so the issue should probably be tracked here. Especially if this is a new change: since there was no change I'm aware of in `run_clm` recently it may be coming from a change there.

Thank you! I will ask on the `datasets` side.

you scooped me Sylvain.

I downgraded datasets to `1.5.0` and printed `tokenized_datasets.cache_files` (L335):

> `{'train': [{'filename': '/home/victor/.cache/huggingface/datasets/openwebtext10k/plain_text/1.0.0/3a8df094c671b4cb63ed0b41f40fb3bd855e9ce2e3765e5df50abcdfb5ec144b/cache-c6aefe81ca4e5152.arrow'}], 'validation': [{'filename': '/home/victor/.cache/huggingface/datasets/openwebtext10k/plain_text/1.0.0/3a8df094c671b4cb63ed0b41f40fb3bd855e9ce2e3765e5df50abcdfb5ec144b/cache-97cf4c813e6469c6.arrow'}]}`

while the same command with the latest version of datasets (actually starting at `1.6.0`) gives:
> `{'train': [], 'validation': []}`

Does it ring any bell @lhoestq ?

OK, moved this to `datasets` https://github.com/huggingface/datasets/issues/2387


Reopening and bringing it back here:

According to this https://github.com/huggingface/datasets/issues/2387#issuecomment-845781874

we need to change examples to add `keep_in_memory=False` - load_dataset otherwise there is no caching.

here:
https://github.com/huggingface/transformers/blob/223943872e8c9c3fc11db3c6e93da07f5177423f/examples/pytorch/language-modeling/run_clm.py#L233



ok, `datasets` reverted the in-memory-datasets by default in master, so this is no longer a problem.
[trainer] the noisy tensorflow loaded when asked explicitly not to load it
Author: stas00
Pull request: no

Unrequested TF loading and its noisy disrespectful logging is back it seems:

```
USE_TF=0 python  examples/pytorch/language-modeling/run_clm.py \
    --model_name_or_path $MODEL \
    --dataset_name $DATASET \
    --output_dir output_dir \
    --overwrite_output_dir \
    --do_train \
    --do_eval \
    --max_train_samples 1000 \
    --max_eval_samples 200 \
    --per_device_train_batch_size 4 \
    --per_device_eval_batch_size 4 \
    --num_train_epochs 1 \
    --warmup_steps 8 \
    --block_size 64 \
    --fp16 \
    --report_to none
```

```
r10i6n8: 2021-05-20 19:52:04.357654: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: [...]
r10i6n8: 2021-05-20 19:52:04.357677: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
```

I am testing multinode setups so am I'm getting hundreds of these! 256 gpus - 512 of these warnings!

How can we make sure that `USE_TF=0` is respected and `tensorflow` doesn't get loaded - I can't uninstall it since it's a shared environment.

Thank you!

@sgugger 

Comments:
Indeed, the `import Trainer` seems to be importing TensorFlow again. Let me try to see if I can remove that.

I messed up my branch and pushed directly on master by mistake, but I don't think it needs reverting and doing a PR since it's a short fix.
Short story is that I locally have no tensorflow import with `USE_TF=0` after [this commit](https://github.com/huggingface/transformers/commit/b8697bc62216b9e2ca60811626c6a6ca992b0d34). Can you confirm?

I confirm. Apologies I missed that request.

Thank you for fixing it, @sgugger!
unable to save model1.h5 .I am using huggingface distilbert
Author: Somabhadra
Pull request: no

Hi Team,
I was trying to save model but i got error.
model.save("model1.h5")
but it didn't work.
I tried different method like model.save("model1") so it will save as .pb file. But when i was trying to load I am getting below error. 
please let me know how to solved it. how can save model after using distil-bert with my model.
~\.conda\envs\bot2\lib\site-packages\tensorflow\python\keras\engine\training.py in get_config(self)
   2229 
   2230   def get_config(self):
-> 2231     raise NotImplementedError
   2232 
   2233   @classmethod

NotImplementedError: 

Comments:
You should use the `save_pretrained` method to save the models.

when i was trying to save model model.save_pretrained("./model/").
I got error
AttributeError: 'Functional' object has no attribute 'save_pretrained'

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
no connection error
Author: Aniruddha-JU
Pull request: no


- `transformers` version: 4.5.0
- Platform: linux
- Python version: 3.8
- PyTorch version (GPU?): 1.7.1
- Tensorflow version (GPU?):
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?:



Models: mbart and mt5

 @patrickvonplaten, @patil-suraj


output: ValueError: Connection error, and we cannot find the requested files in the cached path. Please try again or make sure your Internet connection is on.

Comments:
Please put the code you used that led to this error.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
reformer-enwik8 Fine-tuning
Author: WHU-Peter
Pull request: no

Could you pls provide a demo for training and Fine-tuning the model of 'reformer-enwik8'? @patrickvonplaten 

Comments:
Hi, you could use [this ](https://colab.research.google.com/github/patrickvonplaten/notebooks/blob/master/PyTorch_Reformer.ipynb) notebook as a reference, it shows how to train reformer on the crime-and-punishment data. 

You could usually find such examples in the community notebooks section [here](https://huggingface.co/transformers/community.html). Also, please use [forum](https://discuss.huggingface.co/) to ask such questions.

@patil-suraj Thank you for your reply. Actually, I do not know how to build the enwik8 dataset, it do not need a tokenizer, which is different from others. I am an beginner 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
why run_translation.py automatically is running on cpu?
Author: Aniruddha-JU
Pull request: no



-`transformers` version: 4.5.0
- Platform: linux
- Python version: 3.8
- PyTorch version (GPU?): 1.7.1
- Tensorflow version (GPU?):
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?:yes

@patrickvonplaten, @patil-suraj
Models: mbart-large-50




Comments:
Could you please post the command that you used?

I run the command for telugu to telugu


 python examples/pytorch/translation/run_translation.py     --model_name_or_path  facebook/mbart-large-50     --do_train     --do_eval     --do_predict     --source_lang te_IN     --target_lang te_IN     --test_file /home/aniruddha/mbart/     --train_file "/content/te_train_mbart.json"     --validation_file "/content/te_dev_mbart.json"     --source_prefix "translate Telugu to Telugu: "     --output_dir "/content/transformers/examples/pytorch/summarization/chkpt"     --overwrite_output_dir yes     --per_device_train_batch_size=1     --per_device_eval_batch_size=2     --predict_with_generate yes

@patil-suraj 

Hi! Is your CUDA environment correctly set up? What is the output of the following in your environment?
```
python -c "import torch;print(torch.cuda.is_available())"   
```

True

Then it should run on GPU. Why do you say it doesn't?

I have same issues. How do you solve it ?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
wav2vec2 not converging when finetuning
Author: cheongalc
Pull request: no

## Environment info

- `transformers` version: 4.4.0
- Platform: Linux-5.4.72-microsoft-standard-WSL2-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.8.1+cu111 (True)
- Tensorflow version (GPU?): 2.5.0 (True)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
@patrickvonplaten 

## Information

Model I am using: wav2vec2

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

I have a dataset of single English word, 1 second long audio files sampled at 16kHz. I wanted to use wav2vec2 for speech recognition instead of just doing audio classification because I wanted the model to be able to generalize to longer audio samples with more words. I followed [the official wav2vec2 guide](https://huggingface.co/blog/fine-tune-wav2vec2-english) almost exactly (the only difference was the dataset used, but I made sure the dataset format and vocab list format was identical as well) but the model does not seem to be converging. The loss would decrease to approx. 3 and stay around there. Checking the predictions made during evaluation, I realized that the model just kept outputting the padding token regardless of the input.

Other issues with similar behaviour are #10884 and #10983. I have tried suggestions there such as increasing the learning rate with no success. 

## Expected behavior

The model should show signs of convergence, such as slowly starting to output sensible prediction strings.

Any help is greatly appreciated!


Comments:
Hey @meeps123,

We try to keep the github issues for code related bugs. For such questions, could you please the [forum](https://discuss.huggingface.co/) instead? :-) Feel free to tag me there! 

Also could you attach a google colab so that I can take a look at your training script? It is very difficult to draw any conclusions just from reading the text. 

Cheers,
Patrick

Hi @patrickvonplaten,

Sure thing! I have opened a topic [here](https://discuss.huggingface.co/t/wav2vec2-not-converging-when-finetuning/6773). The Colab notebook is linked there. Thank you for the assistance!
Fix t5 error message
Author: changjonathanc
Pull request: yes

# What does this PR do?

Change `inputs` to `input_ids` in error message.

```diff
-                f"You cannot specify both {err_msg_prefix}inputs and {err_msg_prefix}inputs_embeds at the same time"
+                f"You cannot specify both {err_msg_prefix}inputs_ids and {err_msg_prefix}inputs_embeds at the same time"

```

- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj

Comments:
Adding new Jax Models.
Author: jayendra13
Pull request: no
Labels: name: New model

Is there any board to track that how many current models have jax implementation ?
I would like to contribute to add jax implementation for the remaining ones, which model I can take to start ?  

Comments:
Hi there, thank you for your interest in JAX.

We plan to add as many models as possible in JAX/Flax. Right now we are working on improving the JAX support in the lib, better JAX/Flax tests, generation, cookie-cutter templates etc so that it'll become easier to add more models faster.

Please stay tuned, we'll soon share more details :)
Is the "dummy inputs and standard lengths" implication, when using tracing for exporting a model, still true?
Author: yaireclipse
Pull request: no

Hi,

We're experimenting with exporting a CamemBERT model via tracing. We used a relatively short sentence as sample input. Then we used the resulting traced model for inference of a sentence with greater length than the sample input's. According to https://huggingface.co/transformers/serialization.html#dummy-inputs-and-standard-lengths, it should've raised an error. However, it seems to just work for us 🤔 .

Are we missing something? Or that section in the docs isn't up-to-date and that implication no longer applies?

Thanks,
Yair

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Windows Test Errors 
Author: kere-nel
Pull request: no

There are some errors that I encounter when running tests on Windows. 

**tests\test_benchmark.py**
```
obj = <MemoryMeasureProcess(MemoryMeasureProcess-8, initial)>
 file = <_io.BufferedWriter name=10>, protocol = None

 def dump(obj, file, protocol=None):
     '''Replacement for pickle.dump() using ForkingPickler.'''
      ForkingPickler(file, protocol).dump(obj)
      AttributeError: Can't pickle local object 'measure_peak_memory_cpu.<locals>.MemoryMeasureProcess'
```

**tests/test_modeling_led.py**
`RuntimeError: 0INTERNAL ASSERT FAILED at "..\\torch\\csrc\\jit\\ir\\alias_analysis.cpp":532, please report a bug to PyTorch. We don't have an op for aten::constant_pad_nd but it isn't a special case.  Argument types: Tensor, int[], bool,`

**tests/test_modeling_bart.py**
The timeout-decorator uses SIGALRM, which causes an error on Windows. Can we skip this tests on Windows? 

**test_tokenization_wav2vec2.py** - makes the assumption that paths are separated with forward slashes, but Windows uses backslash. 
`    AssertionError: Sequences differ: ['added_tokens.json', 'special_tokens_map.j[39 chars]son'] != ['C:\\Users\\AzureUser\\AppData\\Local\\Tem[267 chars]son']`


**tests/test_utils_check_copies.py**
```
tests\test_utils_check_copies.py:76: in check_copy_consistency
  self.assertTrue(len(check_copies.is_copy_consistent(fname)) == 0)
   AssertionError: False is not true
```

**tests/extended/test_trainer_ext.py**
```
 def encode(self, input, final=False):
     return codecs.charmap_encode(input,self.errors,encoding_table)[0]
    UnicodeEncodeError: 'charmap' codec can't encode characters in position 0-59: character maps to <undefined>
```

[additional-logs.txt](https://github.com/huggingface/transformers/files/6422688/windows-logs.txt)

## Environment info
- `transformers` version: master
- Platform: Windows
- Python version:
- PyTorch version (GPU?): 1.8.1 10.2cuda 
- Tensorflow version (GPU?): none
- Using GPU in script?: 10.2cuda 

 @sgugger, @patil-suraj





Comments:
The tests are not run on Windows, so there is absolutely no guarantee any of them passes there.

> The tests are not run on Windows, so there is absolutely no guarantee any of them passes there.

Isn't Windows a supported platform? If so, would it be helpful to run HF tests on Windows as well? 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Speedup batch matmul in pytorch
Author: AngThanos
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
Hi there,

I'm trying to combine Sparse Transformer into Vision Transformer to speed up the time-consuming training also inference.
However, the speed is really slow with my code which is shown below. Can someone help me point out where I was wrong?

Thanks.
`import torch
import math

query_layer = torch.randn(32, 12, 197, 64)
key_layer = torch.randn(32, 12, 197, 64)
key_layer_transpose = key_layer.transpose(-1, -2)

dim0 = query_layer.shape[0]
dim1 = query_layer.shape[1]
dim2 = query_layer.shape[2]
dim3 = query_layer.shape[3]

print(dim0, dim1, dim2, dim3)

#orignal transformer attention score calculation
attention_scores = torch.matmul(query_layer, key_layer_transpose)
print(attention_scores)

#my modified based on Sparse Transformer for speeding up the training (but actually lower than at least 20 times in comparison with the original transformer
N = math.sqrt(dim3)
attention_scores = torch.zeros(dim0, dim1, dim2, dim2, device='cuda:0')
for i_dim0 in range(dim0):
  for i_dim1 in range(dim1):
    for i in range(dim2):
      for j in range(dim2):
        if (i == j) or ((i - j) % N == 0 and i - j > 0):
          attention_scores[i_dim0, i_dim1, i, j] = torch.matmul(query_layer[i_dim0, i_dim1, i, :], key_layer_transpose[i_dim0, i_dim1, :, j])
attention_scores.shape
print(attention_scores)
`



Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co) instead?

Thanks!

I've found the solution by using deepspeed.

However, I encountered the error and I opened an issue here. Please help me
with this issue. Thanks.

https://github.com/microsoft/DeepSpeed/issues/1153

On Thu, Jun 10, 2021 at 2:04 PM Lysandre Debut ***@***.***>
wrote:

> Hello, thanks for opening an issue! We try to keep the github issues for
> bugs/feature requests.
> Could you ask your question on the forum <https://discuss.huggingface.co>
> instead?
>
> Thanks!
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/12093#issuecomment-858369696>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AJY7KIWLCP5AOY7CA3JI6ZDTSBPY3ANCNFSM46ND2SZQ>
> .
>
BigBird on TPU
Author: thevasudevgupta
Pull request: yes

# What does this PR do?

This PR will enable BigBird working on TPUs. Problem was happening because we were concatenating tensors of different dtypes. This PR will fix it.

See this notebook to infer BigBird on TPUs: https://colab.research.google.com/drive/1ptZlDuEgmoElWmPmrZXHA7uWjvra9G8T#scrollTo=sR2Yk-HzmGnw

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #11363

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->
@patrickvonplaten 

Comments:
Hi, do you guys have any idea whether we can still train with `trainer` on TPU with BigBird? I am still facing this same error.

Is there any tentative timeline by which this problem can be fixed?

BigBird will be merged in Flax soon. It is recommend to use `FlaxBigBird` for TPU 

That's great news! Till when can we expect it to be available?

It will get merged this week (may be in a day or two).