noamaanMulla-03/transformers-issues huggingface dataset:
fill-mask pipeline: fix handling topk() indices
Author: guyrosin
Pull request: yes

# What does this PR do?

Fixes #12113, where indices in the targets array were used instead of their corresponding token ids. 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@Narsil @LysandreJik

Comments:
Hi, thanks for letting us know! We just reverted the PR in the meantime.

Fixed by #12330
Killed Message
Author: adnan-fakahr-pk-90
Pull request: no

Describe the bug
When I run this command
generator = pipeline('text-generation', model='EleutherAI/gpt-neo-2.7B')
or even download the model in a folder like this
generator = pipeline('text-generation', model=neo-models/')

It is not loading and produce the result as "Killed" text.
which usually means "out of memory"

even though I have nothing loaded except pycharm GUI

I have tested this on Ubuntu and Centos Server. Same result

below is the whole code:

import gc
import os
from transformers import pipeline
import torch

gc.collect()
print("================1")
**generator = pipeline('text-generation', model='EleutherAI/gpt-neo-2.7B') ====> here comes the KILLED**
#generator = pipeline('text-generation', model='neo-models/'')
print("================2")
prompt = "what is the meaning of life"
res = generator(prompt, max_length=50, do_sample=True, Temperature=0.9)
print("================")
print(res)

Comments:
The `EleutherAI/gpt-neo-2.7B` is a large model. Loading it in memory takes  more than 10GB. 
Do you have the same results when trying to use the 1.3B or the 125M variants?

> The `EleutherAI/gpt-neo-2.7B` is a large model. Loading it in memory takes more than 10GB.
> Do you have the same results when trying to use the 1.3B or the 125M variants?

yes I did with 2.7 and then 1.3 both have the same result....
Then how we use it, if a person has 8GB or 16GB memory installed in a PC...


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Encoding/decoding NLP model in tensorflow lite (fine-tuned GPT2)
Author: Guillaume-slize
Pull request: no

We are in the process of building a small virtual assistant and would like it to be able to run a fine-tuned version of GPT-2 on a raspberry-pi with a coral accelerator. 

So far, we managed to convert our model to a tflite and to get first results. We know how to convert from words to indices with the previous tokenizer but then we need a bigger tensor as input to the interpreter. We miss the conversion from indices to tensors. Is there a way to do this simply?

You can find our pseudo-code here, we are stuck at step 2 and 6 :

```
import tensorflow as tf
 
#Prelude
TF_MODEL_PATH_LITE = "/path/model.tflite"
 
interpreter = tf.lite.Interpreter(model_path=TF_MODEL_PATH_LITE)
interpreter.allocate_tensors()
tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
input_shape = input_details[0]['shape']
 
#1-Encode input, giving you indices
context_idx = tokenizer.encode("Hello world.", return_tensors = "tf")
 
#2-How to convert the context_idx to appropriate np.array ?
input_data = np.array(np.random.random_sample(input_shape), dtype=np.int32) #dummy input for now
 
#3- feed input
interpreter.set_tensor(input_details[0]['index'], input_data)
 
#4- Run model
interpreter.invoke()
 
#5- Get output as tensor
output_data = interpreter.get_tensor(output_details[0]['index'])
 
#6- How decode this np array to idx
output_idx=np.random.randint(100) #dummy for now ...
 
#7- Decode Output from idx to word
string_tf = tokenizer.decode(output_idx, skip_special_tokens=True)

```

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co) instead?

Thanks!

Sure,
Thanks for your answer :)

On Mon, May 31, 2021 at 9:51 AM Lysandre Debut ***@***.***>
wrote:

> Hello, thanks for opening an issue! We try to keep the github issues for
> bugs/feature requests.
> Could you ask your question on the forum <https://discuss.huggingface.co>
> instead?
>
> Thanks!
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/11947#issuecomment-851283473>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ADRJZ5KO6QL4EYWKMDXCXADTQM5WVANCNFSM45ZFOJYQ>
> .
>


-- 

Guillaume Slizewicz
www.guillaumeslizewicz.com
+32 496 53 6666


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Loading mbart-large-50-one-to-many-mmt is very slow
Author: MK096
Pull request: no

Whenever i try to run :
model = MBartForConditionalGeneration.from_pretrained(" [local path]/mbart-large-50-one-to-many-mmt")
My computer ether freezes or it takes 15-20 minutes to load the model.

I am using it for translation 
Code: https://huggingface.co/facebook/mbart-large-50-one-to-many-mmt

Any solution fo this?

-Thanks

Comments:
Hi there, `mbart-50` is actlly a big model and takes a while to load. But 15-20 min seems a lot, it's could probably an issue with your system. You could try to load it using a colab and see how much time it takes.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
[docs] XLNETModel forward returns last_hidden_state 3rd dim should be d_model instead of hidden_size 
Author: Muktan
Pull request: no

**Link to doc** - https://huggingface.co/transformers/model_doc/xlnet.html#xlnetmodel

**Problem description** - forward method of XLNETModel returns `last_hidden_state` with dimension `(batch_size, num_predict, hidden_size)`. However, `hidden_size` is not an XLNET config (unlike for BERT) instead that dimension should be `d_model` which is a config of XLNETModel.

**current shape** - (batch_size, num_predict, hidden_size)
**expected shape** - (batch_size, num_predict, d_model)

## Environment info
Not required
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

### Who can help
@sgugger
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.


Comments:
Hi,

Actually, `hidden_size` is an alias for `d_model`, as can be seen [here](https://github.com/huggingface/transformers/blob/7e73601f3240b99d952c34b63bf4f8b78ca1462d/src/transformers/models/xlnet/configuration_xlnet.py#L233).

They mean the same thing. But I get your confusion, as some models use `hidden_size`, others `d_model`, or other names.

Feel free to open a PR to fix this in the docs of XLNet.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Predict masked word at the beginning of the sentence
Author: lalchand-pandia
Pull request: no

Case 1: I have a sentence "After the campaign finance laws changed, Albert ran for mayor of his city."  which I have modified as
\<s\> \<mask\>  the campaign finance laws changed, he ran for mayor of his city. \</s\>.
This input is passed through pre-trained RobertaBase and RobertaLarge (RobertaTokenizer, RobertaForMaskedLM).
The predicted top-10 words are:
['When', 'After', 'Before', 'Once', 'As', 'Until', 'While', 'With', 'Since', 'Then']
Case 2: But when we have masked word in the middle of the sentence:
\<s\> He ran for mayor of his city <mask> the campaign finance laws changed. \</s\>
The predicted top-10 words are:
['Ġbefore', 'Ġafter', 'Ġwhen', 'Ġuntil', 'Ġas', 'Ġand', 'Ġbut', 'Ġonce', 'Ġbecause', ',']

Why the special character is added in the 2nd case and not case 1?



Comments:
The special characters mean that these words are preceded by a space. This is not the case when the words are the start of sentences!

So, if I insert a space at the beginning of the sentence before [MASK], I should words beginning with special character or the tokenizer removes those extra spaces at the beginning?

Yes, the encoding of the first word will change if you add a space. The tokeniser does not normalise whitespace. Try adding 2 spaces, a tab or a newline character. Please read about `add_prefix_space=True` and `is_split_into_words=True` on https://huggingface.co/transformers/model_doc/roberta.html#robertatokenizer

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
'MT5Tokenizer' is not defined (on Google colab) 
Author: rashmibanthia
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: `transformers-4.9.0.dev0`
- Platform: Google Colab
- Python version: `3.7.10` 
- PyTorch version (GPU?):  `1.9.0+cu102`
- Tensorflow version (GPU?): `2.5.0`
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@sgugger , @patil-suraj 

## Information

Trying to run_mlm.py  - 

The problem arises when using:
- Just importing packages. 

## To reproduce

Steps to reproduce the behavior:

1. Run the following on Google colab 
2. 

```
!pip install git+https://github.com/huggingface/transformers.git

from transformers import (
    CONFIG_MAPPING,
    MODEL_FOR_MASKED_LM_MAPPING,
    AutoConfig,
    AutoModelForMaskedLM,
    AutoTokenizer,
    DataCollatorForLanguageModeling,
    HfArgumentParser,
    Trainer,
    TrainingArguments,
    set_seed,
)
```
Error message - 
```
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-1-a06d7b6932ed> in <module>()
      2 
      3 
----> 4 from transformers import (
      5     CONFIG_MAPPING,
      6     MODEL_FOR_MASKED_LM_MAPPING,

5 frames
/usr/local/lib/python3.7/dist-packages/transformers/models/mt5/__init__.py in <module>()
     94         globals()["__file__"],
     95         _import_structure,
---> 96         extra_objects={"MT5Tokenizer": MT5Tokenizer, "MT5TokenizerFast": MT5TokenizerFast},
     97     )

NameError: name 'MT5Tokenizer' is not defined
```

## Expected behavior
No error when importing.

Thank you for your help! 

Comments:
Fixed it for now - `!pip install git+https://github.com/huggingface/transformers.git@b29c394` 

Should be fixed now, thanks for reporting!
Git LFS bug when uploading to hub
Author: BirgerMoell
Pull request: no

After running a MLM for 69000 steps
https://huggingface.co/birgermoell/roberta-swedish-scandi/tree/main

 the model crashed and now I get an error when trying to upload to the hub. The same error was responsible for stopping the training.

```
Uploading LFS objects: 100% (2/2), 998 MB | 0 B/s, done.                          
Enumerating objects: 13, done.
Counting objects: 100% (13/13), done.
Delta compression using up to 96 threads
Compressing objects: 100% (9/9), done.
Writing objects: 100% (9/9), 33.99 KiB | 308.00 KiB/s, done.
Total 9 (delta 3), reused 0 (delta 0)
remote: -------------------------------------------------------------------------
remote: Your push was rejected because it contains files larger than 10M.
remote: Please use https://git-lfs.github.com/ to store larger files.
remote: -------------------------------------------------------------------------
remote: Offending files:
remote:  - events.out.tfevents.1625668537.t1v-n-98937c84-w-0.121638.3.v2 (ref: refs/heads/main)
To https://huggingface.co/birgermoell/roberta-swedish-scandi
 ! [remote rejected] main -> main (pre-receive hook declined)
error: failed to push some refs to 'https://huggingface.co/birgermoell/roberta-swedish-scandi'
```

Git lfs is installed in the repository.

Perhaps the files stored in git are too large?

Comments:
You need to explicitly `git lfs track` the files (of file name patterns) that are to be stored in LFS

Here it might be your tfevent files, so you can `git lfs track "*tfevents*"`?

Solved it by doing the following.


1.Tracking the file with LFS 
```
git lfs track filename
```
2 Assuring that it is tracked by 
```
git lfs status
```
3.
```
git lfs migrate import --include="*.v2"
```
[model.from_pretrained] raise exception early on failed load
Author: stas00
Pull request: yes

Currently if `load` pretrained weights fails in `from_pretrained`, we first print a whole bunch of successful messages and then fail - this PR puts the exception first to avoid all the misleading messages.

(github produces some weird replay effects when re-using a branch that it assigns by default when editing on github)

@sgugger, @LysandreJik 

Comments:
AttributeError for DataCollatorForLanguageModelling with tokenizers.Tokenizer
Author: lewisbails
Pull request: no

## Environment info
- `transformers` version: 4.8.2
- Platform: Linux-5.3.0-53-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.10
- PyTorch version (GPU?): 1.9.0+cu102 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
- tokenizers: @LysandreJik
- trainer: @sgugger


## Information

Model I am using (Bert, XLNet ...): Bert

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

```
tokenizer = Tokenizer.from_file("my-tokenizer.json")
config = AutoConfig.from_pretrained("bert-base-cased", vocab_size=tokenizer.get_vocab_size())
model = AutoModelForMaskedLM.from_config(config)

tokenizer.enable_truncation(max_length=model.config.max_position_embeddings)
dataset = LMDataset(tokenizer, files=['train_1.txt', 'train_2.txt'])
data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, **cfg.data_collator_kwargs)
```
```
Traceback (most recent call last):
...
File "/home/leb/lang-models/scripts/train_lm.py", line 25, in train_lm
  data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, **cfg.data_collator_kwargs)
File "<string>", line 7, in __init__
File "/home/leb/anaconda3/envs/lang-models/lib/python3.7/site-packages/transformers/data/data_collator.py", line 333, in __post_init__
  if self.mlm and self.tokenizer.mask_token is None:
AttributeError: 'tokenizers.Tokenizer' object has no attribute 'mask_token'
```

## Expected behavior

Expected to be able to use tokenizers.Tokenizer in the tokenizer parameter to DataCollatorForLanguageModelling.


Comments:
Hi there! I am unsure why you thought you could use a `tokenizers.Tokenizer` object here. The [documentation](https://huggingface.co/transformers/main_classes/data_collator.html#datacollatorforlanguagemodeling) clearly states it has to be a `PreTrainedTokenizerBase`, so either a `PreTrainedTokenizer` or a `PreTrainedTokenizerFast`. You can instantiate one with
```
from transformers import PreTrainedTokenizerFast

tokenizer = PreTrainedTokenizerFast(tokenzier_file=path_to_json)
```

Sorry about that, I get confused what to use where between the two projects sometimes.
I've also found [this](https://huggingface.co/transformers/fast_tokenizers.html#) to help me.
Although the [documentation](https://huggingface.co/transformers/main_classes/tokenizer.html#transformers.PreTrainedTokenizerFast) for `PreTrainedTokenizerFast` doesn't show `tokenizer_file` as a valid parameter to `__init__`

Oh very true, it's definitely missing! Do you want to make a PR to fix it?
text-classification example broken; version issue / inconsistency issues
Author: pasta41
Pull request: no

## Environment info
- `transformers` version: 4.8.2 (latest on master)
- Platform: Ubuntu 20.04
- Python version: 3.8.5
- PyTorch version (GPU?): 1.9
- Tensorflow version (GPU?): N/A
- Using GPU in script?: Didn't get that far
- Using distributed or parallel set-up in script?: No

### Who can help


@sgugger (because they have a commit that mentions to the 4.9dev release; and cross-checked that they are listed under the examples)
@patil-suraj (because they are listed under the examples)

## Information

Model I am using (Bert, XLNet ...): Bert

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The text-classification example crashes because it wants version 4.9.dev0 or more recent, but the latest on master is 4.8.2. I tried commenting this line out and running with 4.8.2, but there are arguments used that I'm guessing were introduced in 4.9 that 4.8.2 does not understand. In other words, the example crashes immediately.

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

Any of the tasks. I tried the example mrpc task in the text-classification readme.

## To reproduce

Steps to reproduce the behavior:


Copied from the readme / what is currently on master in terms of the code, run:

```
export TASK_NAME=mrpc

python run_glue.py \
  --model_name_or_path bert-base-cased \
  --task_name $TASK_NAME \
  --do_train \
  --do_eval \
  --max_seq_length 128 \
  --per_device_train_batch_size 32 \
  --learning_rate 2e-5 \
  --num_train_epochs 3 \
  --output_dir /tmp/$TASK_NAME/
```

## Expected behavior

Crashes because 4.8.2 is the latest in master and at least 4.9 is expected; when forcing to run with 4.8.2, crashes due to unrecognized arguments.


Comments:
Hello! Installing from `master` should yield the appropriate v4.9.0dev0 version, as this is the current version: https://github.com/huggingface/transformers/blob/master/src/transformers/__init__.py#L25

Did you install it from Pypi? `pip install transformers`
or from source? `pip install git+https://github.com/huggingface/transformers`

The latter should be done to install from source.

If you're looking for v4.8.2 examples, you can find them by going to that tag version: https://github.com/huggingface/transformers/tree/v4.8.2/examples

Thank you so much for the quick response. I suppose the issue was installing via conda instead of pip. I will give this a try and get back to you. Thanks again!

Thank you! Seems to be working. I am going to double check a few things tomorrow before closing. Thanks again!
How to transfer fine-tuned model from python to rust?
Author: plutonium-guy
Pull request: no

# 🚀 Feature request
Since the maximum model in huggingface is in pytorch or tf. But I have used to BART-Large in rust-bert find out that the overall execution of BART-Large in rust is significantly lower than python. But don't know how to transfer fine-tuned pytorch model to rust env.
Feature needed:-
1 . Need to make a copy of all the models of huggingface to rust-bert to gain performance in execution.
2 . Need to develop a standard way to transfer huggingface python model to rust env.


Comments:
Facing Issue while loading pytorch model as flax model
Author: bhadreshpsavani
Pull request: no

I am trying to convert pytorch model in flax so i can train in downstream task
I wrote converting script like this
```python
from transformers import AutoConfig, FlaxAutoModelForMaskedLM
config = AutoConfig.from_pretrained("./")
model = FlaxAutoModelForMaskedLM.from_pretrained("./", from_pt=True,  config=config)
model.save_pretrained("./")
```
By taking this [reference](https://huggingface.co/transformers/model_doc/auto.html#transformers.FlaxAutoModelForSeq2SeqLM)
This was the logs
```
Traceback (most recent call last):
  File "convert_to_flax.py", line 3, in <module>
    model = FlaxAutoModelForMaskedLM.from_pretrained("./", from_pt=True,  config=config)
  File "/home/bhadresh/transformers/src/transformers/models/auto/auto_factory.py", line 387, in from_pretrained
    raise ValueError(
ValueError: Unrecognized configuration class <class 'transformers.models.t5.configuration_t5.T5Config'> for this kind of AutoModel: FlaxAutoModelForMaskedLM.
Model type should be one of RobertaConfig, BertConfig, BigBirdConfig, BartConfig, ElectraConfig.
```

Comments:
using `FlaxAutoModelForSeq2SeqLM` solved the issue, It was typing mistake
Streaming mode in training examples
Author: gsarti
Pull request: no

# 🚀 Feature request

The v1.8,0 of `datasets` introduced [streaming mode](https://huggingface.co/docs/datasets/master/dataset_streaming.html). It would be very useful to include this mode in the training examples, either with a parameter `--stream_dataset` or an ad-hoc training script if radical changes are required to make it work.

## Motivation

Besides showcasing the potential of the new streaming feature, this would be very useful in the context of pre-training scripts (e.g., the new [`run_t5_mlm_flax.py`](https://github.com/huggingface/transformers/blob/master/examples/flax/language-modeling/run_t5_mlm_flax.py)) where large dataset like OSCAR and C4 are commonly leveraged. It would be especially useful to have it for the Flax Community Event!

## Your contribution

I can submit a PR and work to integrate the feature in some of the Flax examples.

Would love to hear the opinion of @patrickvonplaten and @patil-suraj on whether this is relatively easy to pull off! 


Comments:
Hey @gsarti,

That's a great issue! We will provide at least one streaming example until Friday :-)

For future watchers, @patrickvonplaten is working on this in #12470! Thanks!

Note that it's actually `datasets` 1.9.0 that will probably be released on Monday that will feature Streaming.
For now it's still available on the `master` branch only !