noamaanMulla-03/transformers-issues huggingface dataset:
ONNX export: move sample input to same device as model when inferring shapes
Author: severinsimmler
Pull request: yes

# What does this PR do?

Training a model on a GPU and exporting it afterwards to ONNX has raised a `RuntimeError`, because the model and the [sample input](https://github.com/huggingface/transformers/blob/1a3e0c4fe6868b4eb1105dfe601a79d7e5d11a0f/src/transformers/convert_graph_to_onnx.py#L196) in [`transformers.convert_graph_to_onnx.infer_shapes()`](https://github.com/huggingface/transformers/blob/1a3e0c4fe6868b4eb1105dfe601a79d7e5d11a0f/src/transformers/convert_graph_to_onnx.py#L161-L222) were not on the same device. This PR moves the output to the same device as the model.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests) Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@mfuntowicz (according to `git blame`)


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
[WIP] CLIP
Author: patil-suraj
Pull request: yes
Labels: name: WIP

# What does this PR do?

This PR adds OpenAI's CLIP model.
original repo: https://github.com/openai/CLIP
initial demo: https://colab.research.google.com/drive/1hwiCuKvw7hwSlE8yv7J1dh280PlYgPef?usp=sharing

Comments:
Awesome effort. Is the current version already compatible with (now merged) #10594 ?

Thanks @dribnet 

I haven't added a feature extractor class for CLIP yet. We first need to finish the `ImageFeatureExtractor` (#01608)
Then the `ClipFeatureExtractor` can inherit from that. This PR will be ready to merge by the end of next week.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Unstale

continuing this in #11445
squad data preprocessor error (list index out of range) while finetuning bert on squad 1.1
Author: dineshggaonkar
Pull request: no
Labels: name: wontfix

run_squad.py throws this error on squad v1.1 dataset 

Traceback (most recent call last):
  File "run_squad.py", line 820, in <module>
    main()
  File "run_squad.py", line 762, in main
    train_dataset = load_and_cache_examples(args, tokenizer, evaluate=False, output_examples=False)
  File "run_squad.py", line 446, in load_and_cache_examples
    examples = processor.get_train_examples(args.data_dir, filename=args.train_file)
  File "/home/din/question_answering_deepQA/venv_indic_deepQA/lib/python3.6/site-packages/transformers/data/processors/squad.py", line 602, in get_train_examples
    return self._create_examples(input_data, "train")
  File "/home/din/question_answering_deepQA/venv_indic_deepQA/lib/python3.6/site-packages/transformers/data/processors/squad.py", line 656, in _create_examples
    answers=answers,
  File "/home/din/question_answering_deepQA/venv_indic_deepQA/lib/python3.6/site-packages/transformers/data/processors/squad.py", line 729, in __init__
    self.start_position = char_to_word_offset[start_position_character]
IndexError: list index out of range


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


@dineshggaonkar  Did you fix it?
TF BART models - Add `cross_attentions` to model output and fix cross-attention head masking
Author: stancld
Pull request: yes

This PR fixes some missing and invalid things around `cross_attentions` for the TensorFlow implementation of BART models:

- `Bart`,
- `Blenderbot` / `Blenderbot_small`,
- `Marian`,
- `MBart`,
- `Pegasus`.

More specifically, this PR includes:
- Enable returning `cross_attentions`
- Add class `TFBaseModelOutputWithCrossAttentions` (according to the PyTorch counterpart) to support output containing `cross_attentions`
- Fix attention head masking for the cross-attention module (by the introduction of `cross_attn_head_mask` and `cross_attn_layer_head_mask`)
- Implement `test_head_masking` for `cross_attn_head_mask`
- Fix some little typos in docs
- Update model templates - implement `head_mask`, `decoder_head_mask`, `cross_attn_head_mask` and code around `cross_attentions` to the TF encoder-decoder models

<hr>
Partially fixes: #10698 

<hr>

**Reviewers:** @jplu @patrickvonplaten @LysandreJik @sgugger 

Comments:
Currently, there are some Torch/TF equivalent tests failing but it should be settled down once #10605 be merged.

@jplu - I'm gonna run all slow tests today and will let you know if everything works or not.

@jplu - I ran all (slow) non-GPU tests and it seems to me everything is passing :) 

Ok, if all the tests for the involved models, including the slow ones, are passing, it is fine to merge for me.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

still open 

@jplu @patrickvonplaten After merging #10605 and rebasing the branch, all tests have passed now :)

Great, thanks @stancld! 

This looks good to merge for me.

Pinging @Rocketknight1 for a final review here. Seq2Seq models like BART should also return cross attentions in TF (not just in PT)

This is a big PR but LGTM! I haven't exhaustively checked everything but the changes seem correct and innocuous, so if it passes tests I'm happy to merge it.
Add basic support for FP16 in SageMaker model parallelism
Author: sgugger
Pull request: yes

# What does this PR do?

**Note:** This is not full support yet as SageMaker Model Parallelism does not support gradient clipping, so a user has to change the default of `max_grad_norm` to 0 if they want to use it.

Otherwise, this adds support for mixed precision training in SageMaker Model Parallelism mode. The script has been tested on `run_glue.py` without error (as long as the caveat above is respected).
A defensive check is added so that the user gets an obvious error message if they don't change the default value of `max_grad_norm`.

Comments:
Typo fixes
Author: LSinev
Pull request: yes

# What does this PR do?

Fix some typos in docs, comments, logging/errors

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@sgugger


Comments:
Pass along seed to DistributedSampler
Author: sgugger
Pull request: yes

# What does this PR do?

This PR passes along the seed to `DistributedSampler` otherwise it always uses 0 for setting its RNG. See #11389 for more context.

Fixes #11389

Comments:
Should also be passed to the constructor for `DistributedLengthGroupedSampler`?

Good point, added it!
ONNX export outputs many warnings
Author: NebelAI
Pull request: no


I was testing ONNX export via your **04-onnx-export.ipynb** notebook and when calling `!python -m transformers.convert_graph_to_onnx --framework pt --model bert-base-cased --opset 11 --quantize onnx/bert-base-cased2.onnx` I get many Warnings like:
```
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator Attention. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator Gelu. No schema registered for this operator.
...
```

They only appear when using `--quantize` flag.

I know it is just a warning ... but still ... does it affect the exporting process in any way? 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Initializing ddp is extremely slow when finetuning RAG
Author: tangxiangru
Pull request: no

Hi, when I am finetuning the RAG model, it seems that the DDP process is extremely slow. I waited 1 day but still did not see the training process.


loading file None
loading file None
loading file /nfs/users/s_xiangru/model/rag/rag/rag-sequence-nq/generator_tokenizer/special_tokens_map.json
loading file /nfs/users/s_xiangru/model/rag/rag/rag-sequence-nq/generator_tokenizer/tokenizer_config.json
Global seed set to 42
Global seed set to 42
LOCAL_RANK: 2 - CUDA_VISIBLE_DEVICES: [0,1,2,3]
Using native 16bit precision.
Global seed set to 42
LOCAL_RANK: 1 - CUDA_VISIBLE_DEVICES: [0,1,2,3]
Using native 16bit precision.
Global seed set to 42
INFO:__main__:Custom init_ddp_connection.
initializing ddp: GLOBAL_RANK: 2, MEMBER: 3/4
INFO:__main__:Custom init_ddp_connection.
initializing ddp: GLOBAL_RANK: 1, MEMBER: 2/4

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
AlbertForMaskedLM always has bad results
Author: Twinparadox
Pull request: no

I am the one who is using your great project.

I'm trying to make my own ALBERT language model using scratch from [here](https://mlcom.github.io/Create-Language-Model/)
This article seems similar to [your tutorial notebook.](https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/01_how_to_train.ipynb#scrollTo=YZ9HSQxAAbme)
I have already made a BERT model that fits my language by referring to this, and it has shown satisfactory results.

```
BertForMaskedLM : about 0.6 loss
BertForSequenceClassification : accuracy 0.88 in my dataset(binary classification)
```

In order to use ALBERT, I trained tokenizer by using Sentencepiece and trained the pretrain model.
My tokenizer had good results, but my language model had higher loss than my BERT model.(about 2.7~2.8)
I thought the result was bad after seeing the loss, so I checked the result with pipeline and fill-mask tasks, and all the results came out the same.

Sentence A
```json
[{'score': 0.7783917188644409, 'token': 32002, 'token_str': '<pad>'},  {'score': 0.008062483742833138, 'token': 3, 'token_str': '.'}, {'score': 0.0054806191474199295, 'token': 4, 'token_str': ','},  ...
```

Sentence B
```json
[{'score': 0.7783915400505066, 'token': 32002, 'token_str': '<pad>'}, {'score': 0.008062485605478287, 'token': 3, 'token_str': '.'}, {'score': 0.005480623338371515, 'token': 4, 'token_str': ','},  ...
```

I want to solve this problem, but I couldn't find the answer even though I found a lot of articles.
I will look forward to the opinions of the contributors and users of this project.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Even slower when using multiple gpus with sharded_ddp
Author: yana-xuyan
Pull request: no

## Environment info

- `transformers` version: 4.2.2
- Platform: Linux-3.10.0-1062.el7.x86_64-x86_64-with-centos-7.7.1908-Core
- Python version: 3.7.10
- PyTorch version (GPU?): 1.8.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: True
- Using distributed or parallel set-up in script?: True

### Who can help

Library:
- trainer: @sgugger

 -->

## Information

Model I am using (Bert, XLNet ...): facebook/bart-large-cnn

## To reproduce

Steps to reproduce the behavior:

When I was trying to use sharded_ddp and multiple GPUs to accelerate the training process, my command I used is as follows:

CUDA_VISIBLE_DEIVCES=6,7 python -m torch.distributed.launch --nproc_per_node=2 finetune_trainer.py \
--sharded_ddp --[other_args]

However, the experiment even took a longer time than the experiment using a single GPU. May I ask which part I did was wrong?



Comments:
`--sharded_ddp` is not there to accelerate your training, it's there to save GPU memory (for very large models) at some cost on the training time. So if you can finetune on one GPU, you should definitely use this option.

@sgugger  as long as I know sharded_ddp is only for distributed training, not sure why you have suggested to use sharded_ddp on one GPU? 

I have not suggested that. I have said to just fine-tune your model on one GPU without any kind of DDP (so no `--sharded_ddp`). It does not make any sense to use this option if your model and its training can fit on one GPU as it is there to reduce GPU memory, not speed up training.

thanks a lot, now I understood what you meant.

On Mon, Mar 22, 2021 at 3:08 AM Sylvain Gugger ***@***.***>
wrote:

> I have not suggested that. I have said to just fine-tune your model on one
> GPU without any kind of DDP (so no --sharded_ddp). It does not make any
> sense to use this option if your model and its training can fit on one GPU
> as it is there to reduce GPU memory, not speed up training.
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/10758#issuecomment-803714561>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AS37NMRT24IR4P3H5QAT753TE2RCVANCNFSM4ZJW56GA>
> .
>


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
head mask issue  transformers==3.5.1
Author: jingyonglin
Pull request: no

## Environment info
     
- `transformers` version: 3.5.1
- Platform: windows & linux
- Python version: python 3.7
- PyTorch version (GPU?): 1.7.0  
- Tensorflow version (GPU?): 2.3.1
- Using GPU in script?: Yes  both CPU and GPU
- Using distributed or parallel set-up in script?: No

### Who can help
albert, bert, GPT2, XLM: @LysandreJik 

## Information
Hi 
I am using  tiny albert Chinese as an encoder. and I've also tried to use albert transformer in my code.
Thing is I have to change  a little bit source code to avoid some head mask issues.
watch transformers/modeling_albert.py 
line 387:  layer_output = albert_layer(hidden_states, attention_mask, **head_mask[layer_index]**, output_attentions)
however a few lines above, the defualt head_mask is None. So **TypeError: 'NoneType' object is not subscriptable** would be raised

It's not a deep bug and could be easily avoided if making a torch.ones  head_mask. Just want to bring it up so it might probably help the others who encounter the same problem.  


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Can't load mt5 model after resizing token embedding
Author: alecoutre1
Pull request: no

## Environment info

- `transformers` version: 4.0.1
- Platform: macOS-10.15.6-x86_64-i386-64bit
- Python version: 3.8.3
- PyTorch version (GPU?): 1.7.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

 @patrickvonplaten 

## Description

I am having issues to reload a saved mt5 model when the token embedding has been resized. This error doesn't appear with the t5 model. I receive the following error : 

`Error(s) in loading state_dict for MT5ForConditionalGeneration:
	size mismatch for lm_head.weight: copying a param with shape torch.Size([250112, 768]) from checkpoint, the shape in current model is torch.Size([250102, 768]).`

Is there something different between the models that I am missing ?

## To reproduce : 

```python
from transformers import MT5ForConditionalGeneration, AutoTokenizer, T5ForConditionalGeneration

model_class = MT5ForConditionalGeneration #T5ForConditionalGeneration
model_path = "google/mt5-base" # "t5-base"

model = model_class.from_pretrained(model_path)
tokenizer = AutoTokenizer.from_pretrained(model_path)

tokenizer.add_tokens(['<tok1>', '<tok2>'])
model.resize_token_embeddings(len(tokenizer))

SAVING_PATH = "/tmp/test_model"

model.save_pretrained(SAVING_PATH)
tokenizer.save_pretrained(SAVING_PATH)

new_model = model_class.from_pretrained(SAVING_PATH)
```



Comments:
Hey @alecoutre1  I think this was fixed very recently. 

I cannot reproduce your error on master -> could you try to pip install the master version and see if the error persists? 

```
pip install git+https://github.com/huggingface/transformers
```

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Add encoder-decoder word embeddings tying by default
Author: alexyalunin
Pull request: yes

As discussed in #8158 config has `tie_encoder_decoder_word_embeds =True` parameter. `_tie_encoder_decoder_word_embeddings ` is called when Encoder Decoder model is initialized, if sizes are the same, the encoder word embedding matrix is assigned to decoder one, this may cause unexpected behavior if e.g. user chooses to init model with bert and gpt and their vocabs have same sizes but different words. 

@patrickvonplaten 

Comments:
@alexyalunin - this is a great PR, thanks a lot! In general the function does exactly what I had in mind :-) I added some changes that I'd suggest we apply.

Also it would be great if we could add a test analogues to this one: https://github.com/huggingface/transformers/blob/93354bc7790ecf768690745db2407b7542264304/tests/test_modeling_encoder_decoder.py#L306 . 

If you have any questions or need help, let me know! 

Looking forward to merge this soon 

> Thanks for the PR! Though I think the name of this option is kinda confusing, I can't think of a better one :)

I'm fine with the name

@patrickvonplaten finally I have found some time to finish this PR. I couldn't finish tests, you see I tried to initialize EncoderDecoder with a model and its copy and tie word embeddings, it seems like they are tied (I check by looking at `model.named_parameters()`, i.e these parameters do not have `decoder.word_embeddings`), but when I save the model and load it `decoder.word_embeddings` now appear in `model.named_parameters()`. I trained such a model for my project and after few epochs word embs for encoder and decoder are the same but they both appear in `model.named_parameters()`. Pls take a look. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
XLMRobertaTokenizer potential bug 
Author: arahusky
Pull request: no

## Environment info

- `transformers` version: 3.4.0
- Platform: Linux-5.4.60-1-pve-x86_64-with-debian-buster-sid
- Python version: 3.6.3
- PyTorch version (GPU?): 1.7.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help
@LysandreJik 
@mfuntowicz

## Information

I need to align tokens obtained with XLMRobertaTokenizer to original text. Normally, I use fast tokenizer API and token_to_chars method to get the mapping. However, when I use it with XLMRobertaTokenizer, there seems to be bug in the output (the returned indices of tokens tend to skip some tokens and sometimes do not correspond).

## To reproduce

```
from transformers import AutoTokenizer
input='walnut , 17.6.2007 22:20:59 , ip : *** . ***.108.25 , # 10305dobry den , nedavno me znicehonic zacal bolet nart prave nohy-spatne se s nim pohybuje , boli me pri chuzi .'
tokenizer = AutoTokenizer.from_pretrained('xlm-roberta-base', use_fast = True)
res = tokenizer(input_line)
print(res.tokens()[:5]
```

```
['
', '▁wal', 'nut', '▁', ',']
```

```
for i in range(15):
  cur_char_word_ind = res.char_to_token(i)
  print(input_line[i], cur_char_word_ind)
```

```
w 1
a 1
l 1
n 2
u 2
t 2
  None
, 3 
  None
1 5
7 5
. 6
6 6
. 6
2 7
```

The line ```, 3 ``` is wrong as ```,``` should be aligned to fourth token ```,```. The fourth token is not used and is skipped.

## Expected behavior

Possibly
```
w 1
a 1
l 1
n 2
u 2
t 2
  3
, 4 
  None
1 5
7 5
. 6
6 6
. 6
2 7
```


Comments:
Indeed this seems a bit strange.

Pining @n1t0 and @Narsil here (actually this should probably rather be an issue in the https://github.com/huggingface/tokenizers repo)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


@n1t0 Any update on this?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.