noamaanMulla-03/transformers-issues huggingface dataset:
can't load flax weights in PyTorch if flax model is saved with dtype `bfloat16`
Author: iliemihai
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform: linux
- Python version: 3,8
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:

### Who can help

- research_projects/r./run_clm_flax.py : @patrickvonplaten @patil-suraj


 -->

## Information

Model I am using (gpt2):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

Script:

1. run conversion script
2. receive error

`path = "./romanian-gpt2_80000/ckpt-80000"config = AutoConfig.from_pretrained(path)
model = AutoModelForCausalLM.from_config(config)
load_flax_checkpoint_in_pytorch_model(model, path+"/flax_model.msgpack")
model.save_pretrained("./romanian-gpt2-large_80000"). `

Converting FLAX model to Pytorch gives the following error:
TypeError: can't convert np.ndarray of type bfloat16. The only supported types are: float64, float32, float16, complex64, complex128, int64, int32, int16, int8, uint8, and bool.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
Convert flax to pytorch

<!-- A clear and concise description of what you would expect to happen. -->

Adding argument --dtype="bfloat16" to run_clm_flax.py   converts some of parameters to fp16, but it gives error when trying to convert the saved flax model to pytorch. A workaround is to convert all parameters of flax model to fp32 and convert then convert the flax model to pytorch.

`def to_f32(t):
    return jax.tree_map(lambda x: x.astype(jnp.float32) if x.dtype == jnp.bfloat16 else x, t)
model.params = to_f32(model.params)`

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
[Benchmark]
Author: Giahuynh1402
Pull request: no

# 🖥 Benchmarking `transformers`

## Benchmark

Which part of `transformers` did you benchmark?

## Set-up

What did you run your benchmarks on? Please include details, such as: CPU, GPU? If using multiple GPUs, which parallelization did you use?

## Results

Put your results here!

Comments:
Closing this for now as it doesn't contain any information
Update namespaces inside torch.utils.data to the latest.
Author: qqaatw
Pull request: yes

# What does this PR do?

Address #13036 .
1. Replace `torch.utils.data.dataset` with `torch.utils.data`
2. Replace `torch.utils.data.sampler` with `torch.utils.data`
3. Replace `torch.utils.data.dataloader` with `torch.utils.data`

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger 


Comments:
Is this compatible with older versions of PyTorch? Given the issue history this looks good though

@patrickvonplaten Yes, it's backward compatible with PyTorch 1.2.0+, which meets transformers' PyTorch requirements: 1.3.1+.

Ref:
https://pytorch.org/docs/1.2.0/data.html
https://github.com/pytorch/pytorch/blob/v1.2.0/torch/utils/data/__init__.py
How do i get the CLS token from the model output?
Author: mosh98
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->


- `transformers` version: 4.8.2
- Platform: Linux-5.4.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.11
- PyTorch version (GPU?): 1.9.0+cu102 (False)- Tensorflow version (GPU?): 2.6.0 (False)- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- funnel: @sgugger
- rag: @patrickvonplaten, @lhoestq

Library:
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

Examples:
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Information

Model I am using (Bert, XLNet ...): roBerta

The problem arises when using:
* [X ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ X] my own task or dataset: (give details below)

My dataset is like a bunch of sentences with labels with them ranging from 0, 1, 2 , 3

## To reproduce

Steps to reproduce the behavior:

1. Looping throw each sentence
2. getting input Id's
3. trying to get the CLS vector for each example

My questions are;
Am i doing it right?
How do i know i am getting the CLS token?

```
for idx, row in df.iterrows(): #Looping through each Row
  input_ids = torch.tensor(tokenizer.encode(row.Sentence)).unsqueeze(0) #Getting the input id's of the sentence
  output = model(input_ids) #Passing it to model
  print( output.last_hidden_state) #Here is where i want to get the [CLS] token vector
```
 


## Expected behavior

How do i get the [CLS] token for each example of Sentence?

Asking this becuase, the transformer [documentation](https://huggingface.co/transformers/main_classes/output.html) does not specify how to get the [CLS] token vector

Any Help is much Appreciated



Comments:
You can get the final hidden state of the [CLS] token as follows:

`cls_token_final_hidden_state = output.last_hidden_state[:,0,:]`

This is because the last hidden states are of shape (batch_size, sequence_length, hidden_size), and the [CLS] token is the first element across the sequence (also called time) dimension.

Ah i see that makes sense.

Huge thanks again Niels!
TF TransfoXL doesn't work with the `generate` method
Author: LysandreJik
Pull request: no

The TF TransfoXL model does not output `logits` but `prediction_scores` which are different due to the `AdaptiveEmbedding`.

The TF version of `generate` requires the `logits` to be output, therefore the model doesn't work with the `generate` method.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
[doc] fix distil* example link
Author: songyouwei
Pull request: yes

fix broken links

Comments:
Great, thanks @songyouwei !

Could you run `make fixup` at the root of your clone to fix the code quality issue? Thank you!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
translation with identical source and target language, for text normalization
Author: desothier1
Pull request: no

Hi,

This is rather a general question about translation and I am aware that I don't follow exactly your guidelines, so I am sorry for that.
(We could run the examples mentioned in your readme, great tool!)

We try to conceive normalization for Dutch, as a 'translation' task.

So, is it for instance possible to use source + target language, defined as the same language, for instance

    --source_lang nl_XX \
    --target_lang nl_XX \

{"translation": {"nl_XX": "liefst geen energie vandaag . waar is **m'n** oplaadstation ?", "nl_XX": "liefst geen energie vandaag . waar is **mijn** oplaadstation ?"}}

or

    --source_lang source\
    --target_lang target \

{"translation": {"source": "liefst geen energie vandaag . waar is **m'n** oplaadstation ?", "target": "liefst geen energie vandaag . waar is **mijn** oplaadstation ?"}}


## Environment info
https://github.com/huggingface/transformers/tree/master/examples/pytorch/translation

--model_name_or_path facebook/mbart-large-50-many-to-many-mmt

Thanks for your answer!



Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co) instead?

Thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
'TransfoXLLMHeadModelOutput' object has no attribute 'loss'
Author: phfaustini
Pull request: no

## Environment info

- `transformers` version: 4.6.1
- Platform: Linux-5.11.0-7620-generic-x86_64-with-glibc2.10
- Python version: 3.8.10
- PyTorch version (GPU?): 1.8.1 (True)
- Tensorflow version (GPU?): not installed (NA)

Models:
- Transformer XL



## To reproduce

Just to run the example from the documentation: https://huggingface.co/transformers/model_doc/transformerxl.html#transfoxllmheadmodel


Steps to reproduce the behavior:

```python
import torch
from transformers import TransfoXLTokenizer, TransfoXLLMHeadModel
tokenizer = TransfoXLTokenizer.from_pretrained('transfo-xl-wt103')
model = TransfoXLLMHeadModel.from_pretrained('transfo-xl-wt103')
inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
outputs = model(**inputs, labels=inputs["input_ids"])
loss = outputs.loss
```

## Expected behavior

I should get a loss, but an exception is thrown instead: 

AttributeError: 'TransfoXLLMHeadModelOutput' object has no attribute 'loss'



Comments:
Indeed, there's an issue with the docstring! `TransfoXL` has two losses, here's the correct snippet:
```py
import torch
from transformers import TransfoXLTokenizer, TransfoXLLMHeadModel
tokenizer = TransfoXLTokenizer.from_pretrained('transfo-xl-wt103')
model = TransfoXLLMHeadModel.from_pretrained('transfo-xl-wt103')
inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
outputs = model(**inputs, labels=inputs["input_ids"])
losses = outputs.losses
```

Would you like to open a PR to fix the docstring?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Autotokenizer error "Already borrowed" when used on thread pool
Author: Warra07
Pull request: no

## Environment info
- `transformers` version: 4.8.2
- Platform: Databricks
- Python version: 3.7.10
- PyTorch version (GPU?): GPU
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: Yes

## Information

Model I am using : camembert-base:

The problem arises when i try to use a tokenizer (from whatever model in my experiments) on multiple thread pools with an Autotokenizer, the error **RuntimeError: Already borrowed** get raised, i haven't tried if the same issues occure whith AutoModel, but i suspect it would, this makes it completly inefficient as it requires to duplicate the tokenizer on each thread (same for model), and is a real problem for packages like Petastorm  /  Horovod.

## To reproduce
Below you'll find a simple snippet of code to reproduce the error:
Steps to reproduce the behavior:
```Python
 from multiprocessing.dummy import Pool as ThreadPool
  
tokenizer =   AutoTokenizer.from_pretrained(pretrained_model_name_or_path="camembert-base")
def tokenizer_test(text):
    print(tokenizer(text))
    
pool = ThreadPool(10)  
data_list = ['this is a test'] * 10
pool.map(tokenizer_test, data_list)
pool.close()
pool.join()
```
However this works fine if i switch the Autotokenizer with CamembertTokenizer.from_pretrained("camembert-base") for example.

Comments:
Duplicate of https://github.com/huggingface/tokenizers/issues/537

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
[Examples]Flax Seq2Seq example fails when doing only eval or predict
Author: bhadreshpsavani
Pull request: no

## Descriptions

In the flax example if we only do predict or eval step it is not flexible enough to work. It will fail at this line
https://github.com/huggingface/transformers/blob/5dd0c956a8eb492c8597e9673cc1d818f0e6b501/examples/flax/summarization/run_summarization_flax.py#L569

because the current script is written in such a way it will always need a training dataset
I have modified a version of the same file which works but I need to remove the below line in that case
https://github.com/huggingface/transformers/blob/5dd0c956a8eb492c8597e9673cc1d818f0e6b501/examples/flax/summarization/run_summarization_flax.py#L467
and i always need to pass training data and create train_dataset by preprocessing even if i am not doing training

### Who can help
@patrickvonplaten @patil-suraj @sgugger 

Comments:
yes, right now for simplicity the scripts are written such that they always expect train datasets.

Feel free to open a PR :)

Sure, I will open PR!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
No docs for v2.3.0
Author: alexcoca
Pull request: no

## Environment info

- `transformers` version: 2.3.0

### Who can help



Documentation: @sgugger


## To reproduce

Steps to reproduce the behavior:

Click [here](https://huggingface.co/transformers/v2.3.0/model_doc/gpt2.html#gpt2doubleheadsmodel) and see there are no docs for 2.3.0.

## Expected behavior

Documentation should be displayed when clicking [here](https://huggingface.co/transformers/v2.3.0/model_doc/gpt2.html#gpt2doubleheadsmodel)

Comments:
@sgugger , I think all docs up to 2.9 are gone (checked 2.8 randomly and it was missing) so there might be a broader isssue.



It might be linked to use more recent versions of sphinx when building it, though I'm not sure. This is not a priority for us, especially for such an older version (it would be different if the docs of the last major version were down), so I don't think anyone on our side will investigate this further.

@sgugger, you are making a valid point. 

However, I just wanted to highlight that _a lot_ of research hinges on these older versions as, unfortunately, people do not maintain their research code once it's out there. It would be helpful if the docs did not disappear so we can still work with others' legacy code when we have to. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
confusing description in prepare_seq2seq_batch of MBart
Author: XuhuiZhou
Pull request: no

## Information

The model that I am using is `MBart-50`

In the description of `prepare_seq2seq_batch`, it says _Prepare model inputs for translation. For best performance, translate one sentence at a time._. Does this mean we should not do batching if we want to obtain the best performance? I am curious why it is the case since the paper itself does not mention that.

## Expected behavior

The performance should be the same doing batching or not.

@patil-suraj 

Comments:
Hi @XuhuiZhou, the `prepare_seq2seq_batch` method is now deprecated and the description is a bit outdated.
we don't recommend using it anymore. You could refer to this section to see how to prepare data for mbart-50 https://huggingface.co/transformers/model_doc/mbart.html#training-of-mbart-50

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
[Wav2Vec2] Better names for internal classes
Author: patrickvonplaten
Pull request: no
Labels: name: WIP

Wav2Vec2's classes have too many names, *e.g.*: FlaxWav2Vec2EncoderLayerStableLayerNormCollection. 

We should make those names easier (reminder for myself @patrickvonplaten to do this)

Comments:
This also concerns a badly chosen duplicate of `Wav2Vec2FeatureExtractor` as part of `modeling_wav2vec2.py` since it's also the "FeatureExtractor" in `feature_extraction_wav2Vec2.py` -> should be corrected as well.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Fix LUKE tests
Author: NielsRogge
Pull request: yes

# What does this PR do?

3 tests defined in `test_tokenization_luke.py` were having a timeout because they were too slow:

```
FAILED tests/test_tokenization_luke.py::Luke::test_add_special_tokens
FAILED tests/test_tokenization_luke.py::Luke::test_maximum_encoding_length_pair_input
FAILED tests/test_tokenization_luke.py::Luke::test_maximum_encoding_length_single_input
```

This was caused by the `get_clean_sequence` method (used in each of those methods), which is defined in `test_tokenization_common.py` and was inherited by default. By overwriting this method with a much simpler one, the tests are significantly faster. No bottleneck anymore.

Comments:
Add min and max question length options to TapasTokenizer
Author: NielsRogge
Pull request: yes

# What does this PR do?

Fixes #12790 

This PR adds 2 additional attributes to `TapasTokenizer`, namely `min_question_length` and `max_question_length`. These allow the user to skip questions that, when tokenized, are longer or shorter than the specified values. This also better reflects the original implementation (which can be found [here](https://github.com/google-research/tapas/blob/117f3bbed085f1dd0801f9ab3a4d10d2361facd0/tapas/utils/tf_example_utils.py#L941)). 

I have also added a corresponding test.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

unstale
run_mlm_no_trainer.py requires accelerate but not in requirements.txt
Author: david-waterworth
Pull request: no

I just installed Transformers 4.9.0 as I'm really excited to investigate the tokeniser free CANINE model.

I noticed that the `examples/pytorch/language-modeling/run_mlm_no_trainer.py` script requires the `accelerate` library but that doesn't appear to be included in `examples/pytorch/language-modeling/requirements.txt` or the main `setup.py`


Comments:
Thanks for flagging, I added those to all examples in the PR mentioned above!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Cosed by #12888
wav2vec pretrain and fine-tune with huge data
Author: lixx0105
Pull request: no

Hi,
Thanks for the great efforts for wav2vec!

is there good example to fine-tune and pretrain wav2vec with huge data? 
it seems the official examples works fine with one GPU but not so good for multigpus.

Thanks.

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co) instead?

Thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
correcting group beam search function output score bug
Author: sourabh112
Pull request: yes

#13177 

This PR Fixes [#13177](https://github.com/huggingface/transformers/issues/13177)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.
@patrickvonplaten 

Comments: