noamaanMulla-03/transformers-issues huggingface dataset:
Fast imports part 3
Author: sgugger
Pull request: yes

# What does this PR do?

This is the last PR to make the import of transformers and defer the imports of torch/tensorflow to when is necessary. It does the same work as #9446 but in each itnermediate init, so that
```
from transformers import BertModel
```
only imports torch and not TensorFlow (ans is thus very fast).

The templates are adapted to the new init format, so users adding models don't have to worry about this.
In passing, I noticed that the `tokenization_utils_base` was importing everything at init, so I deferred imports there to only do them when necessary. There might be a few places like this left, but we can address those later on.

Fixes #8733

Comments:
max_target length for question answering system
Author: Arij-Aladel
Pull request: no

Please could you tell the max target length for question answering systems?

I was trying and it does not work if the target length is more than 47 

this is the   [notebook](https://colab.research.google.com/drive/1JzsuPb68L-G4nsMXu57XkwbmzAVrizSS?usp=sharing)

Comments:
This doesn't seem to be related to your length, but rather to this:

```py
data['labels'].squeeze():  60
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-12-33cc96af719d> in <module>()
      2   input = tokenizer.decode(data['input_ids'].squeeze(), skip_special_tokens=True)
      3   print("data['labels'].squeeze(): ", len(data['labels'].squeeze()))
----> 4   label = tokenizer.decode(data['labels'].squeeze(), skip_special_tokens=True)
      5   print("data keys: ", data.keys(),"\n")
      6   lines = textwrap.wrap("Query:\n%s\n" % data['question'], width=150)

5 frames
/usr/local/lib/python3.6/dist-packages/sentencepiece/__init__.py in _func(v, n)
    492   def _func(v, n):
    493     if type(n) is int and (n < 0 or n >= v.piece_size()):
--> 494       raise IndexError('piece id is out of range.')
    495     return func(v, n)
    496 

IndexError: piece id is out of range.
```

Your tokenizer doesn't manage to decode your label.

@LysandreJik  yes understand that buy I did not understand what I did wrong first. Then I realized that labels that are -100 should be 0 again.

Anyway good to figure out late than not doing that
Makes HfArgumentParser compatible with Python 3.9
Author: Tpt
Pull request: yes

Python 3.9 changed the format of the string serialization of `typing.Optional`.

For example, `str(typing.Optional[str])` is `typing.Union[str, NoneType]` in python 3.8 and `typing.Optional[str]` in Python 3.9.

Comments:
LGTM
Fix TF input for np.ndarray
Author: jplu
Pull request: yes

# What does this PR do?

This PR allows `np.ndarray` datatype as input of the models.

# Fixes
#9248 


Comments:
I commented on the corresponding issue, I don't fully understand what's going on there in the error

This is on purpose because all the methods of a Keras Model allow to have `np.ndarray` as input. You can check for example `fit`, `predict` or `evaluate` here https://www.tensorflow.org/api_docs/python/tf/keras/Model. They all take a numpy array as possible input

Okey, I see. I still don't think we should provide this feature, just because keras has some automatic conversion internally. Is there a use case where one cannot forward a TF tensor and has to forward a `nd.array`? The general philosophy of the lib is to "not add too many magic functions" and allowing `nd.arrays` as inputs for TF seems like opening the door for lots of future issues to me. Let's see what @sgugger @LysandreJik think about it

On my side I would prefer to keep as much compliancy as possible with TF. But if everyone are not confident because of this Keras magic, I'm ok to do not provide it :)

The TF models do not accept numpy arrays inputs, so this would allow bad inputs to be passed to TF models. I think we should stick with inputs accepted by TF models only.

The TF models allow to have numpy arrays as inputs (if we had the `np.ndarray` type among the allowed ones). As example:
```python
from transformers import TFBertForSequenceClassification, BertTokenizer
tokenizer = BertTokenizer.from_pretrained("bert-base-cased")
inputs = tokenizer("Hello", return_tensors="np")
model = TFBertForSequenceClassification.from_pretrained("bert-base-cased")
model(inputs)
```
Gives:
```
TFSequenceClassifierOutput(loss=None, logits=<tf.Tensor: shape=(1, 2), dtype=float32, numpy=array([[0.08065815, 0.58226204]], dtype=float32)>, hidden_states=None, attentions=None)
```

Keras layers/models are by default compliant with numpy arrays.

Ah my bad, I tried but forgot to checkout the PR before :facepalm:
If TF models do accept those inputs then, I have no strong objection.

@LysandreJik you mean adding a test in `test_modeling_tf_common` with numpy array as input for each model?

Yes, for example !

There is a now a new test to be sure that the models can be properly executed with numpy inputs.

Does it looks ok to be merged?
Cannot Evaluate While Training Using the Trainer
Author: AliOskooeiTR
Pull request: no

@sgugger 

## Environment info
   
- `transformers` version: 4.0.0
- Platform: AWS Amazon Linux 
- Python version: 3.6.10
- PyTorch version (GPU?): 1.5.1
- Tensorflow version (GPU?): N/A
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


## Information

Model I am using (Bert, XLNet ...): Third Party Model ( Routing Transformer)

The problem arises when using:
* [ ] my own modified scripts:
I am using a custom Trainer to train the third party model. The training loops runs smoothly without evaluation. But as soon as I try to do evaluation while training, it stops training after a couple of evaluations even though it has not reached the max_steps. I copy my args and trainer below.  I understand in previous versions there was a evaluate_during_training flag that some have suggested as a fix until Sep 20. But that flag doesn't seem to exist anymore. Any help or pointers would be appreciated. 


The tasks I am working on is:
* [ ] my own task or dataset: Masked Language Modeling 

## To reproduce

Steps to reproduce the behavior:

```
custom_args = MLMArguments(
        output_dir='../models/', 
        mask_ratio=0.2, 
        do_train=True,
        do_eval=True,
        max_steps=2000000, 
        save_steps=50,
        logging_steps=5,
        per_device_train_batch_size=1,
        per_device_eval_batch_size=1,
        logging_dir = '../logs/',
        evaluation_strategy="steps",
        prediction_loss_only=True

)

checkpoint_callback=TrainerCallback()
tb_callback = TensorBoardCallback()

custom_trainer = MLMTrainer(
        rt_model, 
        args=custom_args,
        train_dataset=train_dataset,
        eval_dataset=eval_dataset,
        callbacks=[checkpoint_callback, tb_callback]
        
)

custom_trainer.train()
```


Step | Training Loss | Validation Loss
-- | -- | --
5 | 10.033914 | 10.017973
10 | 10.028201 | 9.935969

and at this point it prints the output and stops training.

Comments:
Hi there, I would like to help you, but the code you are providing is not runable on my side (MLMTrainer, MLMArguments, train_dataset, eval_dataset and rt_model for instance are not defined). Could you please post a complete and short reproducer of the bug?
Upgrade styler to better handle lists
Author: sgugger
Pull request: yes

# What does this PR do?

This PR upgrades the doc styling script to automatically add new lines before lists. This makes the script more robust as it will avoid reformatting those lists and make them appear properly once sphinx has done its thing.

In passing a few badly formatted docstrings/doc pages are fixed, just waiting for some input from @patrickvonplaten for the problems in LED/Longformer.

Fixes #9408



Comments:
I suppose I need to stick to the common bullet format, as it couldn't handle this `\d\) ` style of bullets. Leading to this rewrite:
```
-1) Optimizer State Partitioning (stage 1)
-2) Add Gradient Partitioning (stage 2)
+1) Optimizer State Partitioning (stage 1) 2) Add Gradient Partitioning (stage 2)
```
This is not a problem - will fix the style.

We also need a new line injector for bulleted lists in .rst checker pretty please.

 In .rst I had:

```
Miscellaneous notes:
- DeepSpeed works with the PyTorch Trainer but not TF Trainer.
- While DeepSpeed has a pip installable PyPI package, 
```
the style wrapper broke the bullets and made them into one paragraph/line.
```
Miscellaneous notes: - DeepSpeed works with the PyTorch Trainer but not TF Trainer. - While DeepSpeed has a pip installable PyPI package, 
```

same problem as with docstring - it's missing a new line again. Could we do the same fix for .rst to inject a new line before bullets if an unwary writer forgot to add one? 

Thank you!



Mmm, the patch should be applied to the rst files too (can't link to the diff but it's line 384 of the last file in the diff shown by GitHub).

I re-based just in case, and no, it still doesn't insert the line. Here is the exact para:
```
Miscellaneous notes:
* DeepSpeed works with the PyTorch Trainer but not TF Trainer.
* While DeepSpeed has a pip installable PyPI package, it is highly recommended that it be `installed from source
  <https://github.com/microsoft/deepspeed#installation>`__ to best match your hardware and also to enable features like
  1-bit Adam, which aren't available in the pypi distribution.
```

Indeed, I made some stupid mistake, #9488 should fix this.
Issue with XLM-R for multiple-choice questions 
Author: danyaljj
Pull request: no
Labels: name: wontfix

Hi there, 

I am not able to reasonable numbers with XLM models ("xlm-roberta-base" "xlm-roberta-large") when I test them on [multiple choice questions](https://github.com/huggingface/transformers/tree/master/examples/multiple-choice). 

I suspect that it's related to issue #7774. 

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.3.1
- Platform: Ubuntu 
- Python version: 3.7 
- PyTorch version (GPU?): 3.6.0 (yes) 
- Tensorflow version (GPU?): 
- Using GPU in script?: yes 
- Using distributed or parallel set-up in script?:

### Who can help
@LysandreJik  @sgugger @VictorSanh


## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

Here is the script you can try,  based on [these instructions](https://github.com/huggingface/transformers/tree/master/examples/multiple-choice): 
```
export SWAG_DIR=/path/to/swag_data_dir
python ./examples/multiple-choice/run_multiple_choice.py \
--task_name swag \
--model_name_or_path xlm-roberta-base \
--do_train \
--do_eval \
--data_dir $SWAG_DIR \
--learning_rate 5e-5 \
--num_train_epochs 3 \
--max_seq_length 80 \
--output_dir models_bert/swag_base \
--per_gpu_eval_batch_size=16 \
--per_device_train_batch_size=16 \
--gradient_accumulation_steps 2 \
--overwrite_output
```

It should give you under 30%  -- near random. 



Comments:
Thanks for reporting, will investigate.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Hi @danyaljj  and @LysandreJik I have the same issue. Are there any suggestions for running xlm-r (base/large) with multiple-choice qa?
tokenizer.vocab key and values is change begin line 261?
Author: zepen
Pull request: no
Labels: name: wontfix

transformers version 3.0.0

```python
tokenizer = BertTokenizer.from_pretrained('bert-base-chinese')

for v, i in tokenizer.vocab.items():
    print(v, i)
```

![image](https://user-images.githubusercontent.com/18568380/97958832-04cc1080-1de9-11eb-8c2b-0463c902853e.png)
i find key and values maybe wrong position.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
AutoTokenizer.from_pretrained function cannot be customized 
Author: ismymajia
Pull request: no
Labels: name: wontfix

A customized tokenizer has been provided in the tokenizer library, so that I can directly use the word segmentation data and vocab in the previous fairseq, but there is AutoTokenizer.from_pretrained function in the transformer, which cannot be customized like the tokenizer library, so I have no choice to directly use fairseq's vocab and word segmentation data in transformer. 

What needs to be done？

Comments:
Hi! Could you provide an example of the usage that you would like to see with the `transformers` library, so that we may see what can be done?

Is this the same as #8125 ?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
_shift_right when to use
Author: rabeehkarimimahabadi
Pull request: no
Labels: name: wontfix

Hi
In modeling_t5 there is a function called shift_right I wonder when it needs to be used, for which tasks? I sometimes see T5 finetuning without using it, not sure when this is suitable to use it thanks

Comments:
Hey @rabeehkarimimahabadi, it's a convenience function that is used if `input_ids` and `labels` are provided but no `decoder_input_ids`. In this case this function automatically creates the correct `decoder_input_ids` as described here: https://huggingface.co/transformers/model_doc/t5.html?highlight=t5#training

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Addition of MMI-antiLM decoding
Author: AADeLucia
Pull request: no
Labels: name: wontfix

# 🚀 Feature request

Hugging Face does a great job of including popular decoding strategies such as nucleus sampling, top-k, and temperature. There are also other really interesting decoding strategies for chatbots to fix the response "blandness" or "I don't know" problem, such as using the **Maximum Mutual Information anti-Language Model objective (MMI anti-LM)**. The algorithm is defined in [A Diversity-Promoting Objective Function for Neural Conversation Models](https://www.aclweb.org/anthology/N16-1014.pdf). 


## Motivation

I'm requesting this as a feature because I used this in a narrative generation paper. From this point forward I will be using "we" to reference my co-authors (@XiangLi1999 also contributed to the code) and I. The results of our work show that antiLM decoding does in fact help make the generated output more interesting without hurting fluency. Our work is [Decoding Methods for Neural Narrative Generation](https://arxiv.org/abs/2010.07375) and the rest of our code is in our [paper repo](https://github.com/AADeLucia/gpt2-narrative-decoding).

We think others would also be interested in using this decoding method for their work.

## Your contribution

We have a working implementation in a hacked version of the `generation_utils.py` file. It's not pretty (sorry) but maybe a good starting point? The code is in PR #7931. 

Also the author's implementation (not huggingface-based) is in [Jiwei Li's repo](https://github.com/jiweil/Neural-Dialogue-Generation).

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Train BERT with CLI commands
Author: Stimmot
Pull request: no
Labels: name: wontfix

I have downloaded the HuggingFace BERT model from the transformer repository found [here][1] and would like to train the model on custom NER labels by using the run_ner.py script as it is referenced [here][2] in the section "Named Entity Recognition".

I define model ("bert-base-german-cased"), data_dir ("Data/sentence_data.txt") and labels ("Data/labels.txt)" as defaults in the code.

Now I'm using this input for the command line:

```
python run_ner.py --output_dir="Models" --num_train_epochs=3 --logging_steps=100 --do_train --do_eval --do_predict
```

But all it does is telling me: 

```
Some weights of the model checkpoint at bert-base-german-cased were not used when initializing BertForTokenClassification: ['cls.predictions.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.w
eight', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.decoder.weight', 'cls.seq_relationship.weight', 'cls.seq_relationship.bias']
- This IS expected if you are initializing BertForTokenClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPretraining model).
- This IS NOT expected if you are initializing BertForTokenClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of BertForTokenClassification were not initialized from the model checkpoint at bert-base-german-cased and are newly initialized: ['classifier.weight', 'classifier.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
```

After that it just stops, not ending the script, but simply waiting. 

Does anyone know what could be the problem here? Am I missing a parameter?

My sentence_data.txt in CoNLL format looks like this (small snippet):
```
Strafverfahren O
gegen O
; O
wegen O
Diebstahls O
hat O
das O
Amtsgericht Ort
Leipzig Ort
- O
Strafrichter O
```
And that's how I defined my labels in labels.txt:
```
"Date", "Delikt", "Strafe_Tatbestand", "Schadensbetrag", "Geständnis_ja", "Vorstrafe_ja", "Vorstrafe_nein", "Ort",
"Strafe_Gesamtfreiheitsstrafe_Dauer", "Strafe_Gesamtsatz_Dauer", "Strafe_Gesamtsatz_Betrag"
```


  [1]: https://github.com/huggingface/transformers
  [2]: https://huggingface.co/transformers/task_summary.html

Comments:
It's probably tokenizing. How big is your dataset?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
when the txt file has 5GB, a Killed prompt appears.
Author: ismymajia
Pull request: no
Labels: name: wontfix

I am running run_language_modeling.py, 
python run_language_modeling.py \
	    --output_dir ${model_dir} \
	    --tokenizer_name $data_dir/wordpiece-custom.json \
	    --config_name $data_dir/config.json \
            --train_data_file "$data_dir/train.txt" \
            --eval_data_file $data_dir/valid.txt \
            --block_size=128 \
            --do_train \
            --per_device_train_batch_size 1 \
            --gradient_accumulation_steps 64 \
            --learning_rate 6e-4 \
            --weight_decay 0.01 \
            --adam_epsilon 1e-6 \
            --adam_beta1 0.9 \
            --adam_beta2 0.98 \
            --max_steps 500_000 \
            --warmup_steps 24_000 \
            --fp16 \
            --logging_dir ${model_dir}/tensorboard \
            --save_steps 1000 \
            --save_total_limit 20 \
            --seed 108 \
            --max_steps -1 \
            --num_train_epochs 20 \
            --overwrite_output_dir

when the txt file has 5GB, a Killed prompt appears.


Comments:
What is your machine's specs? It's probably an out of memory error.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Is there a pre-trained BERT model with the sequence length of 2048?
Author: h56cho
Pull request: no
Labels: name: wontfix

Hello,
I want to use the pre-trained BERT model because I do not want to train the entire BERT model to analyze my data. Is there a pre-trained BERT model with sequence length 2048?

or are all pre-trained BERT model only have the sequence length of 512?

Thank you.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
should be BartConfig.prefix  None?
Author: sshleifer
Pull request: no
Labels: name: wontfix

For bart-large-xsum/bart-large-cnn, currently set to `' '`

+  does not help evaluation performance vs setting to `None` (checked xsum and CNN)

+ It could help fine-tuning performance by serving as a work-around for the `add_prefix_space` issue?
Putting here in case others have thoughts.



Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
segmentation fault (core dumped)  proxychains4 python xxx.py
Author: Jasonsey
Pull request: no
Labels: name: wontfix

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.4.0
- Platform: ubuntu 18.04
- Python version: 3.6
- PyTorch version (GPU?): 1.4.0 GPU
- Tensorflow version (GPU?): 2.2.0 GPU
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no
- Network: need to use http proxy to download files and I use the tool ProxyChains4 


## To reproduce

Steps to reproduce the behavior:

1. Save the following code to file test.py

```python

from transformers import pipeline

classifier = pipeline('sentiment-analysis')


classifier('We are very happy to include pipeline into the transformers repository.')

```

2.  Exec `proxychains4 python test.py`
3. The following error was raised

```shell
(test) ➜  test-transformer proxychains4 python test.py
[proxychains] config file found: /etc/proxychains4.conf
[proxychains] preloading /usr/lib/x86_64-linux-gnu/libproxychains.so.4
[proxychains] DLL init: proxychains-ng 4.12
[proxychains] DLL init: proxychains-ng 4.12
[proxychains] Strict chain  ...  10.74.193.90:80  ...  s3.amazonaws.com:443  ...  OK
[proxychains] Strict chain  ...  10.74.193.90:80  ...  s3.amazonaws.com:443  ...  OK
[proxychains] Strict chain  ...  10.74.193.90:80  ...  s3.amazonaws.com:443  ...  OK
[proxychains] Strict chain  ...  10.74.193.90:80  ...  s3.amazonaws.com:443  ...  OK
[proxychains] Strict chain  ...  10.74.193.90:80  ...  s3.amazonaws.com:443  ...  OK
[proxychains] Strict chain  ...  10.74.193.90:80  ...  s3.amazonaws.com:443  ...  OK
[proxychains] Strict chain  ...  10.74.193.90:80  ...  s3.amazonaws.com:443  ...  OK
[proxychains] Strict chain  ...  10.74.193.90:80  ...  s3.amazonaws.com:443  ...  OK
[proxychains] Strict chain  ...  10.74.193.90:80  ...  cdn.huggingface.co:443  ...  OK
[1]    9790 segmentation fault (core dumped)  proxychains4 python test.py
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

The model files can be downloaded without error


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.