noamaanMulla-03/transformers-issues huggingface dataset:
from_pretrained does not load the modified part of model 
Author: ghost
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version:
- Platform: Linux
- Python version: 3.7
- PyTorch version (GPU?): 1.7
- Tensorflow version (GPU?): - 
- Using GPU in script?: no 
- Using distributed or parallel set-up in script?: no 

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.
 
 Model Cards: @julien-c
 -->
 Model Cards: @julien-c
## Information

Hi
1) I am observing that if one modifies a model lest say T5ForConditionalGeneration and then use  T5ForConditionalGeneration.from_pretrained(...) then not all components of the models are loaded, meaning that the 
parts of the model the user has modified are initialized to random! 

2) I observe this from accuracy, could you tell me how I can check which weights from_pretrained is loading, I am a bit lost in the repository. thanks
## Expected behavior

If the model has been changed to have more layers, .... all the trained weights need to be loaded. 

Comments:
In the documentation, this is written how from_pretrained works for untouched models, but I cannot see how this works when one modifies the model. 

Hey @juliahane,

could you please provide a code snippet showcasing the unintended behavior? Thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
SummarizationModule, Trainer and BertPreTrainedModel
Author: ziqi-zhang
Pull request: no

Hi,

I wonder what's the relationship between SummarizationModule (SummarizationDistiller), Trainer and BertPreTrainedModel? I want to reimplement the distillation.py of seq2seq example and run it on the glue dataset. But I'm confused by the relationship between these three classes. 

The model parameter of Trainer is BertPreTrainedModel and the Trainer train the model. But in the SummarizationDistiller there isn't a training function. I didn't find the training process in the distillation.py and finetune.py. Should I pass SummarizationDistiller object to Trainer to train the model? Or how should I train my custom SummarizationDistiller.

Comments:
Hi @ziqi-zhang,
the distillation code is written using the `pytorch-lightning` framework and `SummarizationModule` is a lightning module. You should go through the lightning docs to see how training works for lightning modules and how to customize it.

`Trainer` is transformers training helper which is different from lightning and there is no connection between the two.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Loss printed by tensorflow fit() differs from loss using custom loop for RoBERTa
Author: brand17
Pull request: no

## Environment info
- `transformers` version: 3.3.1
- Platform: Windows 10
- Python version: 3.6
- PyTorch version (GPU?): N/A
- Tensorflow version (GPU?): 2.3.0
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@jplu, @LysandreJik

## Information

Model I am using (Bert, XLNet ...): RoBERTa

The problem arises when using:
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Run the following script
2. The printed out losses are different

```
import tensorflow as tf
from transformers import RobertaConfig, TFRobertaMainLayer

# 1. Create a class to be able to use fit()

class Transformer(tf.keras.Model):
  def __init__(self):
    super(Transformer, self).__init__()

    config = RobertaConfig(
        vocab_size=100,
        hidden_size=128,
        intermediate_size=128, 
        max_position_embeddings=514,
        num_attention_heads=8,
        num_hidden_layers=6,
        type_vocab_size=1,
    )  
    self.encoder = TFRobertaMainLayer(config)

  def call(self, inp, training=False):
    return self.encoder(inp)[0]

model = Transformer()

# 2. Calculating loss manually for dummy input

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
x = tf.constant([[1, 0]])
y_true = tf.constant([[1, 0]])
y_pred = model((x, x))
loss = loss_fn(y_true, y_pred)
print(loss) #                              printing 4.8093767

# 3. Run fit()

model.compile(loss=loss_fn)
model.fit((x, x), y_true) #                printing 4.7854

```
## Expected behavior

The losses should be equal.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
[ci] install fairscale on self-runner CIs
Author: stas00
Pull request: no

The tests for the new sharded ddp fairscale integration are in place, https://github.com/huggingface/transformers/pull/9177
but CIs don't have `fairscale` installed so they won't run on CI. 

This is an issue to track this need so we will eventually run these tests.

Blocking event: needing a more reliable/quick way of installing fairscale - need to track the following issue for when this happens:
https://github.com/facebookresearch/fairscale/issues/264

Then we can add `pip install fairscale` to the self-runner CIs with multigpus.

Comments:
Probably should change CI to do:
```
pip install fairscale --no-build-isolation
```
which is much faster then the new pip system, as it doesn't need to fetch dependent packages


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
RAGRetriever loads dataset in the default cache dir even if a different one is specified
Author: JamesDeAntonis
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: latest
- Platform: any
- Python version: 3.8
- PyTorch version (GPU?): any
- Tensorflow version (GPU?): any
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.
 
 RAG: @patrickvonplaten, @lhoestq
 -->

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. load a `RagRetriever` model with `cache_dir='/mnt/.cache/huggingface/'
2. notice that the dataset is still downloaded to `'~/.cache'`

```{python
from transformers import RagRetriever
rag_retriever = RagRetriever.from_pretrained('facebook/rag-token-base', cache_dir='/mnt/.cache/huggingface')
```

## Expected behavior
The dataset is still downloaded in `'~/.cache'` even though we want it to download to the cache in '/mnt'

The reason this is happening is because, in `retrieval_rag.py`, `config.cache_dir` isn't passed through to `load_dataset` on line 273, for example

Comments:
Indeed! Would you like to open a PR with your fix?

I am dealing with this issue too, would love to find a way around this. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Zero Shot Classification Pipeline fails when running in CPU-only Docker container
Author: ravisurdhar
Pull request: no

## Environment info
- `transformers` version: 4.0.0
- Platform: MacOS 10.15.7 (2018 MacBook Pro 15")
- Python version: 3.6.9
- PyTorch version (GPU?): 1.7.0 CPU Only
- Tensorflow version (GPU?): N/A
- Using GPU in script?: N/A
- Using distributed or parallel set-up in script?: N/A

### Who can help
Maybe @LysandreJik ?

## Information

Model I am using (Bert, XLNet ...): facebook/bart-large-mnli

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [X] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Pull and start the official [transformers-pytorch-cpu](https://hub.docker.com/r/huggingface/transformers-pytorch-cpu/dockerfile) container.
2. `docker exec -it huggingface bash`
3. `python3`
4. `from transformers import pipeline`
5. `classifier = pipeline("zero-shot-classification", model='facebook/bart-large-mnli', tokenizer='facebook/bart-large-mnli', device=-1)`

Step 4 above results in the following warning being printed:
```
/usr/local/lib/python3.6/dist-packages/torch/cuda/__init__.py:52: UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at  /pytorch/c10/cuda/CUDAFunctions.cpp:100.)
  return torch._C._cuda_getDeviceCount() > 0
```

Step 5 above results in the models being downloaded, then `Killed` is printed and the Python interpreter exits.

## Expected behavior
When running locally in a Jupyter Notebook or directly in the terminal (not in a container), the following works correctly and the warning about the CUDA initialization isn't printed:
```
from transformers import pipeline
classifier = pipeline("zero-shot-classification", model='facebook/bart-large-mnli', tokenizer='facebook/bart-large-mnli', device=-1)
```

The problem seems to be limited to either the zero shot classification pipeline, or the facebook/bart-large-mnli model, since the following works correctly in the container (though the warning from Step 4 about the CUDA initialization is still printed):
```
from transformers import pipeline
classifier = pipeline('sentiment-analysis')
```


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
TransfoXL Slow Test Fails
Author: patrickvonplaten
Pull request: no

This test needs to be fixed:

```
pytest -s tests/test_modeling_tf_transfo_xl.py::TFTransfoXLModelLanguageGenerationTest::test_lm_generate_transfo_xl_wt103
```

@patrickvonplaten pinging myself.

cc @jplu (for notice)

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
"No log" when training RobertaForSequenceClassification using Trainer
Author: BryanWBear
Pull request: no

When training, for the first few logging steps I get "No log".

Looks like this:

Step | Training Loss | Validation Loss | Accuracy | F1
-- | -- | -- | -- | --
150 | No log | 0.695841 | 0.503277 | 0.410575
300 | No log | 0.696622 | 0.488860 | 0.298561
450 | No log | 0.694300 | 0.499345 | 0.356902

What does this mean? My classifier is performing poorly and I am wondering if this is related. I am finetuning roberta-base using 3k question-answer pairs, 50% positively labelled, 50% negatively labelled.

Thanks,
Bryan
 


Comments:
Could you provide the environment information as mentioned in the issue template, alongside the a reproducible that outputs this so that we may check what's going on? Thank you.

Hi @BryanWBear , 

I am facing this issue too. In the meantime, did you find a solution?

Thank you so much in advance!

the default `logging_steps` in `TrainingArguments` is set to `500` steps, so no loss is reported before 500 steps

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Question: What's the difference between tokenizer_utils, tokenizer_utils_base & tokenizer_utils_fast
Author: BrandonLiang
Pull request: no

As titled. Thanks!

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Using doc chunks without answer token during training ( BertForQuestionAnswering )
Author: nikoletta-toth
Pull request: no

Hi!

When creating features from the squad examples, you use sliding window approach to generate doc chunks, when the input data is too long. There you state, **_if the document chunk does not contain an annotation, you throw it out, since there is nothing to predict._** Hence you set the start_position=0 and end_position=0. Then you re-set it to cls_index in training mode. When you do **CLS token at the beginning, then your cls_index = 0**. Finishing up, you add this item to the InputFeatures. 
https://github.com/huggingface/transformers/blob/e768f2322abd2a2f60a3a6d64a6a94c2d957fe89/examples/utils_squad.py#L332-L351

During training - forward step, you calculate loss on these items:
Let's say the max seq length is 512, then the ignored_index = 512, meaning you only ignore those start/end positions which are >= 512. With cls_index = 0 we have start_position=0 and end_position=0. So we end up getting a loss calculated using the start_position with start_logits and end_position with end_logits.
https://github.com/huggingface/transformers/blob/a8c3f9aa760ed7b516ee00f602e8efc0e5d80285/src/transformers/models/bert/modeling_bert.py#L1651-L1665

Then **you return with the calculated loss and in run_squad.py you add this new loss to the training loss**:
https://github.com/huggingface/transformers/blob/a8c3f9aa760ed7b516ee00f602e8efc0e5d80285/examples/question-answering/run_squad.py#L219

**So actually you do not throw these chunks out, but use them for training.** 

**Solution maybe:**
Why not set the start_position = end_position = **max_seq_length** instead of cls_index in the utils_squad.py, making sure they will be ignored during training and the loss calculated with them will be 0 ??

Hope you will understand my point, let me know what you think!

**Update:**
I got this idea from another repo using bits and pieces from your implementation (they only used labels with answer, so squad-v1 like dataset) :
In case of too long sequences the sliding window approach creates multiple doc chunks, some of those are without an annotated answer. The model learns to predict start and end positions as 0  (if CLS is on 0. pos) when the answer is NOT present in the document chunk. This will help reduce false predictions on test data where documents are too long and split into multiple chunks.

**So either you actually want to use those doc chunks with no annotation, but your comments are misleading, 
OR you don't want to use those chunks, hence the comment, but you failed to implement it.**

Hope it helps!


Comments:
@sgugger you might be interested in that issue given that you're refactoring the squad example!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Wrong Length of Dataset in examples/seq2seq/finetune_trainer.py
Author: iseesaw
Pull request: no

`train/validation/test examples. -1 means use all` may be not correct

https://github.com/huggingface/transformers/blob/693ac3594b96e86dd282fdf8e413f3a48b176892/examples/seq2seq/finetune_trainer.py#L97-L99

`n_train/val/test` is used to compute the length of the dataset, there will lack one line if set to -1. It should be None to use all examples.

https://github.com/huggingface/transformers/blob/693ac3594b96e86dd282fdf8e413f3a48b176892/examples/seq2seq/utils.py#L136-L137

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
providing an example with a dummy iterative dataloaders
Author: rabeehkarimimahabadi
Pull request: no

Hi
I have tested the trainer.py with iterative datasets and this does not work in distributed case, I shard the data across the cores. Could you please assist me in providing me with a dummy iterative dataloader for finetune_seq2seq.py model which runs fine with xla_spawn.py on TPU so I get some understanding which functions needs to be implemented. I really need to make this work, and trainer.py does not seem to work with iterative datasets or I am missing how to do it properly. thanks 

@sgugger 

Comments:
Trainer does indeed not work in distributed fashion with iterative datasets. You need to convert your iterative dataset to a regular dataset for the time being.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
trainer.py does not handle distributed training for iterative datasets and is very slow
Author: rabeehkarimimahabadi
Pull request: no

## Environment info
- `transformers` version: 3.5.1
- Platform: TPU
- Python version: 3.7
- using xla_spawn.py 

### Who can help
@sgugger
@patrickvonplaten
@patrickvonplaten
@patil-suraj

## Information
I am running seq2seq_finetune.py with iterative datasets and I do not get any speed up for 8 TPU cores versus 1 TPU cores, the code is also even slower than 1 GPU.

## To reproduce
```
git clone git@github.com:google-research/ruse.git
go to iter branch 
pip install -r requirements.txt
python setup.py develop
cd seq2seq
python xla_spawn.py finetune_t5_trainer.py  configs/mrpc_adapter_tpu.json

```

## Expected behavior
Being faster on TPU, to me trainer.py does not handle iterative datasets properly. could you have a look please? thank you for your help. 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
'Some weights of BertModel were not initialized from the model checkpoint at ./model and are newly initialized: ['bert.pooler.dense.weight', 'bert.pooler.dense.bias']'
Author: AlexanderTekle
Pull request: no

Hi everyone,

I ran [ run_mlm.py ](https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_mlm.py) to continue pertaining uncased BERT directly from the examples on this repo, but once I load the newly saved pretrained Bert Model, I receive a warning - "'Some weights of BertModel were not initialized from the model checkpoint at ./model and are newly initialized: ['bert.pooler.dense.weight', 'bert.pooler.dense.bias']'"

I'm trying to fine-tune the model on a sentiment analysis task, but I'm getting horrible results and I wonder if it has something to do with this? Thanks for your help.

Comments:
maybe related to #8793 , hope could help.

> maybe related to #8793 , hope could help.

seems to be related. I get high variance in accuracy, I guessed it was probably because of the random initialization of those two weights.

If you're using the `run_mlm.py`, then you're doing masked language modeling with the `BertForMaskedLM` model. This model does not make use of the pooler, hence why those two layers are randomly initialized. They're not used for predictions or training.

@LysandreJik would it make more sense to load the saved model using BertModel.load_pretrained(saved_mlm_model) or would it be better to use BertModel.load("bert-base-uncased") and copy the weights over from the saved model? 

I also met this problem. Have you solved it?

@wenHK It's actually not really relevant. The BertForMaskedLM model doesn't use the pooler layer, so thus why there are no weights assigned. You don't really need to worry about the warning.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Exporting ALBERT model to onnx increases model size by 7x
Author: unography
Pull request: no

I'm trying to export `albert-base-v2` model to onnx using

`python -m transformers.convert_graph_to_onnx --framework pt --model albert-base-v2 --quantize albert.onnx --opset 12`

The original pytorch model size is around 45 MB (https://huggingface.co/albert-base-v2), but the exported model size is around 340 MB.

How do I keep the model's size same?

Comments:
@mfuntowicz might have an idea. I'm guessing that what it does is copy each layer, while all the layers have shared weights and should point to the same tensor.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.