noamaanMulla-03/transformers-issues huggingface dataset:
[tokenizers] convert_to_tensors: don't reconvert when the type is already right
Author: stas00
Pull request: yes

I was trying to fix this warning:
```
src/transformers/tokenization_utils_base.py:608: UserWarning: To copy construct from a tensor, 
it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), 
rather than torch.tensor(sourceTensor).
  tensor = as_tensor(value)
```
which appeared when running:
```
python eval_rag.py --model_name_or_path facebook/rag-sequence-nq --model_type rag_sequence --evaluation_set output/biencoder-nq-dev.questions --gold_data_path output/biencoder-nq-dev.pages --predictions_path output/retrieval_preds.tsv --eval_mode retrieval --k 1
```

This appears to have happened since `convert_to_tensors` was called with data which was already a tensor of the right type. 

* [x] and ended up fixing it for pt and also adding the same fix for tf/jax/np. basically skip the conversion if the value is already of the required type and avoid the pytorch warning.
* [x] added tests for converting the already converted
* [x] while at it added a missing test for `test_batch_encoding_with_labels_jax`

I understand `lambda` isn't welcome, so I had to define a few helper functions for numpy/jax. `partial` would have done the trick, but `isinstance` doesn't accept keyword args.

@LysandreJik, @mfuntowicz

Comments:
ping

Looks good to me. Thanks for handling this one @stas00 and sorry for the delay.
[examples/seq2seq] fix PL deprecation warning
Author: stas00
Pull request: yes

This PR fixes a PL deprecation warning, since we require PL-1.0.4 this is a safe switch.

Reference: https://pytorch-lightning.readthedocs.io/en/latest/generated/pytorch_lightning.callbacks.ModelCheckpoint.html#pytorch_lightning.callbacks.ModelCheckpoint.params.filepath

@patrickvonplaten

Comments:
Add sentencepiece to the CI and fix tests
Author: sgugger
Pull request: yes

# What does this PR do?

When removing sentencepiece from the dependencies of Transformers, the CI started to skip all tests that required sentencepiece. As a result some failures due to new breaking changes (switch to fast tokenizer by default and the fact we removed `max_len`, deprecated argument of tokenizers) were unnoticed. 

This PR adds back sentenpiece install on all CI checks and fix the resulting failing tests.

Comments:
REALM checkpoints to pytorch checkpoints
Author: mchari
Pull request: no
Labels: name: New model

Will it be possible to convert checkpoints in https://console.cloud.google.com/storage/browser/realm-data/cc_news_pretrained to pytorch implementations ? 
I am not very familiar with Tensorflow....
I tried using convert_bert_original_tf_checkpoint_to_pytorch.py but I am not sure I am invoking it correctly.
Here is how I am invoking the python script...
--tf_checkpoint_path="./cc_news_pretrained/embedder/encoded/" \
 --bert_config_file="./bert_config.json" \
 --pytorch_dump_path="./pytorch"

I am using tensorflow 2.3.0.

The checkpoint file has the following entries which are probably internal developer files(?):  
model_checkpoint_path: "/cns/li-d/home/lumiere/public/models/gatoatigrado/ner-with-dates/10923195/1-active_losses=mlm_loss/export/temp/1580364602/retriever/encoded/encoded.ckpt"
all_model_checkpoint_paths: "/cns/li-d/home/lumiere/public/models/gatoatigrado/ner-with-dates/10923195/1-active_losses=mlm_loss/export/temp/1580364602/retriever/encoded/encoded.ckpt"


1)When I set tf_checkpoint_path to the directory containing the checkpoint, I get the error : 
tensorflow.python.framework.errors_impl.NotFoundError: /cns/li-d/home/lumiere/public/models/gatoatigrado/ner-with-dates/10923195/1-active_losses=mlm_loss/export/temp/1580364602/retriever/encoded; No such file or directory


2)When I set tf_checkpoint_path to the checkpoint file encode.ckpt.data-00000-of-00001, I get the error:
env/lib/python3.6/site-packages/tensorflow/python/training/py_checkpoint_reader.py", line 95, in NewCheckpointReader
    return CheckpointReader(compat.as_bytes(filepattern))
RuntimeError: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for ./cc_news_pretrained/embedder/encoded/encode.ckpt.data-00000-of-00001

Thanks!

Comments:
Why use 'BertLayerNorm'  instead of torch.nn.LayerNorm ?
Author: daydayfun
Pull request: no
Labels: name: New model

# 🌟 New model addition
What's the difference between 'BertLayerNorm' and  torch.nn.LayerNorm
## Model description  
1.pytorch    torch.nn.LayerNorm
https://pytorch.org/docs/stable/generated/torch.nn.LayerNorm.html?highlight=layernorm#torch.nn.LayerNorm

2.modeling.py
class BertLayerNorm(Module):
    def __init__(self, hidden_size, eps=1e-12):
        super(BertLayerNorm, self).__init__()
        self.shape = torch.Size((hidden_size,))
        self.eps = eps
        self.weight = nn.Parameter(torch.ones(hidden_size))
        self.bias = nn.Parameter(torch.zeros(hidden_size))
        self.apex_enabled = APEX_IS_AVAILABLE
             
    @torch.jit.unused
    def fused_layer_norm(self, x):
        return FusedLayerNormAffineFunction.apply(
                    x, self.weight, self.bias, self.shape, self.eps)
             
             
    def forward(self, x):
        if self.apex_enabled and not torch.jit.is_scripting():
            x = self.fused_layer_norm(x)                            
        else:
            u = x.mean(-1, keepdim=True)
            s = (x - u).pow(2).mean(-1, keepdim=True)
            x = (x - u) / torch.sqrt(s + self.eps)
            x = self.weight * x + self.bias
        return x


<!-- Important information -->
It seems like torch.nn.LayerNorm has the same function of  belows ops in BertLayerNorm
 u = x.mean(-1, keepdim=True)
 s = (x - u).pow(2).mean(-1, keepdim=True)
 x = (x - u) / torch.sqrt(s + self.eps)
 x = self.weight * x + self.bias

Why we don't use  torch.nn.LayerNorm ?

Thanks a lot for answering my question


## Open source status

* [ ] the model implementation is available: (give details)
* [ ] the model weights are available: (give details)
* [ ] who are the authors: (mention them, if possible by @gh-username)


Comments:
There once was a difference, but there is none anymore. I believe the `BertLayerNorm` was removed and is not available anymore in recent versions.

I have done some test,  
self.assertTensorsEqual(
out_BertLayerNorm, out_mlu_nativeLayerNorm, 0, use_RAE=True)
and the diff is   2.8356e-07
Model parallel v4
Author: alexorona
Pull request: yes

# Model Parallelism for GPT2 and T5

Note: this is a clean pull request for [PR # 7772](https://github.com/huggingface/transformers/pull/7772) that uses code from transformers v4.0.0.

Adds two new methods to `GPT2LMHead` and the `GPT2Model` classes to enable you to generate and fine-tune models using model parallelism. This feature is most applicable for `gpt2-large` and `gpt2-xl`. Minor modifications are made to the `TrainingArguments` and `Trainer` classes to avoid conflicting data parallelism behavior and related batch_size increases which would negate model parallelism. Note that nearly 64GB of GPU (4 Tesla v100s)  are needed to fine-tune `gpt2-xl` @ 1024 tokens.

It is critically important to provide users the ability to specify where to put the blocks of a model because the GPU sizes and numbers are likely to be very diverse. This is done with a dictionary called `device_map`. I am planning on providing some examples and guidelines for the p3, p2 and g3 AWS instances. 

Model parallelism has to be baked into the model class itself. Currently working on the T5 model. From my calculations the 11B model cannot fit on the largest p3 instance that I have access to (8 Tesla v100 GPUs). The 3B model can.

The methods are:
- `parallelize`, which will distribute the attention blocks of the model across several devices according to a device map
- `deparallelize`, which will move the model back to cpu

# Example

```
model = GPT2LMHeadModel.from_pretrained('gpt2-xl')
device_map = {0: [0, 1, 2, 3, 4, 5, 6, 7, 8],
                              1: [9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21],
                              2: [22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34],
                              3: [35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47]}
model.parallelize(device_map) # Distributes the model's attention blocks across several devices
model.deparallelize() # Puts the model back on cpu and calls torch.cuda.empty_cache() to liberate GPU memory
```

## Reviewers
@LysandreJik 


Comments:
Cannot load 'bert-base-german-cased'
Author: laifi
Pull request: no

`tokenizer = BertTokenizer.from_pretrained('bert-base-german-cased')`

**Output:**

> Model name 'bert-base-german-cased' was not found in model name list (bert-base-uncased, bert-large-uncased, bert-base-cased, bert-large-cased, bert-base-multilingual-uncased, bert-base-multilingual-cased, bert-base-chinese). We assumed 'bert-base-german-cased' was a path or url but couldn't find any file associated to this path or url.

Comments:
Hi @laifi, 

I cannot reproduce this issue. Are you sure that you run with the latest code from master branch? It looks suspicious to me that `tokenizer = BertTokenizer.from_pretrained('bert-base-german-cased')` doesn't find the model. 
Can you please check if you have [the according line](https://github.com/huggingface/pytorch-pretrained-BERT/blob/78462aad6113d50063d8251e27dbaadb7f44fbf0/pytorch_pretrained_bert/tokenization.py#L37) in your PRETRAINED_VOCAB_ARCHIVE_MAP? 

For your second approach with downloaded files: 
- be aware that model packaging changed lately from archives to individual files for vocab, model and config (see [here](https://github.com/huggingface/pytorch-pretrained-BERT/pull/688#issuecomment-502991015)). If you really want to download manually you should download the [.bin](https://int-deepset-models-bert.s3.eu-central-1.amazonaws.com/pytorch/bert-base-german-cased-pytorch_model.bin), [bert_config.json](https://int-deepset-models-bert.s3.eu-central-1.amazonaws.com/pytorch/bert-base-german-cased-config.json) and the [vocab file](https://int-deepset-models-bert.s3.eu-central-1.amazonaws.com/pytorch/bert-base-german-cased-vocab.txt) to a folder called "bert-base-german-cased"
- `from_pretrained` expects a model name or path not a .bin . You should try:  BertTokenizer.from_pretrained('YOUR_PATH_TO/bert-base-german-cased') 

Hope that helps!

Thank you @tholor  , i installed the package with pip and i cannot find 'bert-german-cased' in PRETRAINED_VOCAB_ARCHIVE_MAP
Now , i tried to reinstall the package from source and it's working .


@laifi  I am keep getting the same error as the one that you got:

> UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte

I also tried to reinstall it, how did you fix it?


> @laifi I am keep getting the same error as the one that you got:
> 
> > UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte
> 
> I also tried to reinstall it, how did you fix it?

@shaked571 , i have just uninstalled the pip package  and  installed it again from source (try to not keep any cache for the package).
**PS: the issue is fixed in the last migration from pytorch-pretrained-bert to pytorch-transformers .**

Hi,
I also run into the same issue when I try this piece of code in google colab. 
tokenizer = BertTokenizer.from_pretrained('bert-base-german-cased')

Hi,
I also have the same issue. Using 
```
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("bert-base-german-cased")
```
solves the problem for me
Error in RAG finetuning script
Author: shamanez
Pull request: no

## Environment info
ut the missing fields in that output! -->
     
- `transformers` version: 3.4.0
- Platform: Linux-4.18.0-147.5.1.el8_1.x86_64-x86_64-with-centos-8.1.1911-Core
- Python version: 3.6.8
- PyTorch version (GPU?): 1.7.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>


### Who can help
 @patrickvonplaten, @lhoestq


## Information

I am using RAG fine-tuning script. During the fine-tuning process, it says

**torch.nn.modules.module.ModuleAttributeError: 'GenerativeQAModule' object has no attribute 'opt'**

The bug exactly appears in [line 332 in finetune.py (https://github.com/huggingface/transformers/blob/master/examples/rag/finetune.py#L332)

## To reproduce
I have installed the transformers library from the source. Not the pip.

Comments:
It is related with the optimizer initialization in the finetune.py script. Seems like even in the lightning_base.py there is no initialization for the optimizer.

@lhoestq 

any idea on this?

I managed to work around by calling the optimizer initialization inside the train_dataloader function in finetune.py. 

Well the optimizer/scheduler is already defined in examples/lightningbase.py in `BaseTransoformer.configure_optimizers`. Not sure why the train_dataloader() function in finetune.py tries to define the scheduler. This must have been a bad copy paste...

I think we should remove those lines
https://github.com/huggingface/transformers/blob/17b1fd804f2ade052e40505a695ec7c9996178a9/examples/rag/finetune.py#L326-L336

I just tried to remove them and now I'm getting this other issue #7816 
I'll fix this one as well and make a PR

That what I was thinking since there is a specific def in lightningbase.py.
moved temperature warper before topP/topK warpers
Author: theorm
Pull request: yes

# What does this PR do?

Moves the `temperature` warper in `generation_utils.py` before `top_p` and `top_k` warper so that temperature affects sampling. This is how it used to be [before refactoring](https://github.com/huggingface/transformers/blob/v3.4.0/src/transformers/generation_utils.py#L571-L575) in `v.3.5.x`.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

@patrickvonplaten

Comments:
the time of loading different models to GPU is nearly the same?
Author: cmdllx
Pull request: no
Labels: name: wontfix

I once used the old script for text-classification,the time of loading different models(mode.to_device) to GPU is different,
However, when I run the lastest version of script, I find that loading different models to GPU nearly takes the same time(which is in the process of initing the Trainer)
I want to know the reason. 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
"You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference" when I am finetuning on distilert pretrained model, After printing this it is taking a lot of time and using only one CPU, how can we parallelized to all the cores in the system ( even I hve 8 GPU's but it is not using tht)
Author: trkece
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:

Comments:
Hi, could you fill in the issue template rather than the issue title?

python ./examples/text-classification/run_glue.py \
    --model_name_or_path ./examples/language-modeling/output/SIA_Ontopof_RISK/DistilBERT/Trained_on_iRisk/ \
    --do_train  \
    --do_predict \
    --task_name=mrpc \
    --data_dir=/mnt/share/app/proj/ph/com/gbl/irisk/project_ph_com_irisk/Udita/Code_evaluation/Data//Train_Test_Data_BERTformat_Biz13Aug/BusinessRules_more_0s/ \
    --output_dir=./proc_data/mrpc/BusinessRules_more_0s   \
    --max_seq_length=512   \
    --per_device_train_batch_size=24   \
    --per_device_eval_batch_size=32 \
    --gradient_accumulation_steps=1 \
    --max_steps=16323 \
    --save_steps=10000 \


This is the command i am executing (finetuning a distilbert on a customer pre trained on a specific dataset)

Train.tsv file is of 1GB.

Feature creation is taking longer time(more than 24 hours and it is still running)

![image](https://user-images.githubusercontent.com/36273175/92735019-abbaa000-f396-11ea-86c4-bc40fe6bc921.png)

 

![image](https://user-images.githubusercontent.com/36273175/92735247-e15f8900-f396-11ea-92cf-dcbcc72caa6f.png)


This warning means that the weights of `pre_classifier` and `classifier` have not been initialized by your checkpoint. This is normal since you seem to be loading a checkpoint right after a language modeling training. These layers will now be trained for sequence classification using the command you've shown.

Hi, Can we use multi core CPU's here, if yes, where to change and in which function?

If i look at all the cores are not utilized. 

![image](https://user-images.githubusercontent.com/36273175/92739146-35b83800-f39a-11ea-9166-f68df50b528a.png)


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
How to save the whole model as SavedModel format for inference?
Author: HX-idiot
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
Hi, I want to save the mode as SavedModel format using model.save(), but when i load it, the input_format is fixed and only input_ids can be used, how can i pass the inputs like {'input_dis':XX, 'attention_mask':XX}?
codes:
![image](https://user-images.githubusercontent.com/38073340/91794928-df871e80-ec4e-11ea-86a8-9c264913b5b7.png)
and then it reports:
![image](https://user-images.githubusercontent.com/38073340/91794969-f168c180-ec4e-11ea-98e1-2e1db94632ea.png)


Comments:
The model  architecture is simple:
![image](https://user-images.githubusercontent.com/38073340/91795291-db0f3580-ec4f-11ea-814f-2342ecfd1b1a.png)


Sorry for the late reply. This is because you did not respect the signature of `TFBertMainLayer` in order to properly use it you can do:

```python
import tensorflow as tf
from transformers import TFBertForSequenceClassification

a = tf.constant([[1,2,3,4,5]])
b = tf.constant([[1,1,1,1,1]])
inp = {"input_ids": a, "attention_mask": b}
model = TFBertForSequenceClassification.from_pretrained("bert-base-cased")
model._saved_model_inputs_spec = None
model._set_save_spec(inp)
tf.saved_model.save(model, "/tmp")
model = tf.keras.models.load_model("/tmp")
```

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Typo in modling_tf_bert
Author: AileenLin
Pull request: no
Labels: name: wontfix

end_pos will be overwritten in some case
https://github.com/huggingface/transformers/blob/858b7d5873577d7997918439eb7429d424d07dd5/src/transformers/modeling_tf_bert.py#L1422

Comments:
Good catch, thanks a lot! We will fix it ASAP.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
gradient_accumulation_steps in trainer_tf
Author: krislc
Pull request: no
Labels: name: wontfix

    part-1:
    self.total_train_batch_size = self.args.train_batch_size * self.args.gradient_accumulation_steps
    ds = (
            self.train_dataset.repeat()
            .shuffle(self.num_train_examples, seed=self.args.seed)
            .batch(self.total_train_batch_size, drop_remainder=self.args.dataloader_drop_last)
            .prefetch(tf.data.experimental.AUTOTUNE)
        )
     part-2:
     for _ in tf.range(self.args.gradient_accumulation_steps):
                reduced_features = {
                    k: ft[: self.args.train_batch_size // self.args.n_replicas] for k, ft in features.items()
                }
                reduced_labels = labels[: self.args.train_batch_size // self.args.n_replicas]

                self.training_step(reduced_features, reduced_labels)

                features = {
                    k: tf.concat(
                        [ft[self.args.train_batch_size // self.args.n_replicas :], reduced_features[k]],
                        axis=0,
                    )
                    for k, ft in features.items()
                }

                labels = tf.concat(
                    [labels[self.args.train_batch_size // self.args.n_replicas :], reduced_labels], axis=0
                )
 
the implementation of gradient_accumulation seems not friendly to users who have small gpu memory.


Comments:
What costs the most is the gradient computation, storing few predictions is ok general. I can run a sequence classification training with a batch of 32 of 128 sequence length and an acummulation of 3 with a 8GB GPU.

Did you encounter during your experiments a memory issue? If yes, let me know and I will look at it.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.