noamaanMulla-03/transformers-issues huggingface dataset:
How to pass `past_key_values` to GPTNeo model?
Author: sooftware
Pull request: no

How to pass `past_key_values` to the GPTNeo model?
  
I want to pass `past_key_values` to the GPTNeo model. I set `past_key_values` as a size Tuple[Tuple[torch.Tensor]] => `(num_layers, 2, batch_size, seq_length, num_heads, d_head)`. But I got a below error message.

```
Original Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py", line 61, in _worker
    output = module(*input, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/kaki_ai/test/big-lm/prefix_tuning/models/prefix_tuning_gpt_neo.py", line 61, in forward
    use_cache=True
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/kaki_ai/test/big-lm/prefix_tuning/models/gpt_neo_for_causal_lm_wrapper.py", line 87, in forward
    return_dict=return_dict,
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/kaki_ai/test/big-lm/prefix_tuning/transformers/models/gpt_neo/modeling_gpt_neo.py", line 866, in forward
    output_attentions=output_attentions,
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/kaki_ai/test/big-lm/prefix_tuning/transformers/models/gpt_neo/modeling_gpt_neo.py", line 563, in forward
    output_attentions=output_attentions,
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/kaki_ai/test/big-lm/prefix_tuning/transformers/models/gpt_neo/modeling_gpt_neo.py", line 505, in forward
    output_attentions=output_attentions,
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/kaki_ai/test/big-lm/prefix_tuning/transformers/models/gpt_neo/modeling_gpt_neo.py", line 412, in forward
    key_value_hidden_states = torch.cat([past, hidden_states], dim=1)
RuntimeError: Tensors must have same number of dimensions: got 3 and 4
```

@patil-suraj 

Comments:
never mind. I solved it.

How? I find the shape of `past_key_values` is very strange.
T ** 2 in distillation process
Author: DA-L3
Pull request: no

Hello, 

I am confused about one part in the https://github.com/huggingface/transformers/blob/master/examples/research_projects/distillation/distiller.py script by VictorSanh et. al. 

![image](https://user-images.githubusercontent.com/33768245/122886299-2100cb80-d340-11eb-8118-0c0888c05fb1.png)

Why does the script weight the KL-Divergence between the student and the teacher distribution with an additional `T ** 2` namely the ```* (self.temperature) ** 2``` part in line 417.
The Hinton-paper says something about weighting with the squared temperature value but in this blog post: https://medium.com/huggingface/distilbert-8cf3380435b5 by VictorSanh (the same author) the KL value (can be found in about the middle of the page) does not seem to be weighted.

What am I missing here? Thank you.

@VictorSanh 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
BART infilling example? 
Author: suzyahyah
Pull request: no

Hi, 

I'm trying this official example from the documentation (v4.7.0) for BART mask infilling :

 "https://huggingface.co/transformers/model_doc/bart.html#bartforconditionalgeneration"

```
from transformers import BartForConditionalGeneration, BartTokenizer
model = BartForConditionalGeneration.from_pretrained("facebook/bart-large", force_bos_token_to_be_generated=True)
tok = BartTokenizer.from_pretrained("facebook/bart-large")
example_english_phrase = "UN Chief Says There Is No <mask> in Syria"
batch = tok(example_english_phrase, return_tensors='pt')
generated_ids = model.generate(batch['input_ids'])
assert tok.batch_decode(generated_ids, skip_special_tokens=True) == ['UN Chief Says There Is No Plan to Stop Chemical Weapons in Syria']
```

First, I get this `TypeError: __init__() got an unexpected keyword argument 'force_bos_token_to_be_generated`
After removing that argument, the output is:` ['UNALSO SEE']`, which is unexpected. 

I also tried several other examples but get behavior which does not seem like infilling:

```
example_english_phrase = "They are having a <mask> in a park."
batch = tok(example_english_phrase, return_tensors='pt')
generated_ids = model.generate(batch['input_ids'])
tok.batch_decode(generated_ids, skip_special_tokens=True)  

```
Output: `"'They are in a park.They are having a party.'"`



**Do I have the wrong model or documentation?** Grateful for any pointers to help resolve this. 

@patrickvonplaten, @patil-suraj
- `transformers` version: 4.7.0
- Python version: 3.7
- PyTorch version (GPU?): 1.7.1+cu110



Comments:
(Resolved,  but documentation needs to be updated)

```
from transformers import BartConfig
config = BartConfig(force_bos_token_to_be_generated=True)
model = BartForConditionalGeneration.from_pretrained("facebook/bart-large", config=config)
...
```

will let the model do infilling. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
RAG with T5 in a multitask setting
Author: sb1992
Pull request: no

I was trying to run RAG with t5 for multiple tasks i.e fact checking and QA. As i understand only with t5 you can do that by adding a unique prefix for the particular task before the input I was wondering what could be the best way to handle that ? Do i preprocess  train.source, val.source so that they already have these prefixes or is their an easier way as well?


Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co) instead?

Thanks!

@sb1992 

No, it doesn't matter whether you use T5 or BART. You can actually use a special token in front of every line in the target files.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Getting 404 Client Error when loading BaptisteDoyen/camembert-base-xnli
Author: guibaure
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.6.1
- Platform: Linux-5.11.16-arch1-1-x86_64-with-glibc2.33
- Python version: 3.9.5
- PyTorch version (GPU?): 1.9.0+cu111 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

## Information

The problem arises when using:
* [ ] the official example scripts: https://huggingface.co/BaptisteDoyen/camembert-base-xlni

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task

## To reproduce

Steps to reproduce the behavior:

```
from transformers import pipeline
classifier = pipeline("zero-shot-classification", 
                      model="BaptisteDoyen/camembert-base-xnli")
```

returns:

```
404 Client Error: Not Found for url: https://huggingface.co/BaptisteDoyen/camembert-base-xnli/resolve/main/config.json
```


Comments:
There's a typo in your model identifier:
```diff
- BaptisteDoyen/camembert-base-xlni
+ BaptisteDoyen/camembert-base-xnli
```

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Add callback method for substeps during gradient accumulation.
Author: wulu473
Pull request: no

# 🚀 Feature request

Add a callback method which is called between `on_step_begin` and `on_step_end` i.e. during gradient accumulation steps.
Something like `on_substep` which is called after each gradient accumulation step.

## Motivation

Some training techniques require custom code to be run after each substep during gradient accumulation . A commonly used tool is Opacus for differentially private training. It introduces a `privacy_engine` and requires `privacy_engine.virtual_step()` to be called during gradient accumulation substeps and `privacy_engine.step()` when accumulation is done. For example see https://github.com/pytorch/opacus/blob/master/tutorials/building_text_classifier.ipynb

With this in place we could quite easily extend the trainer to support differentially private training with Opacus.

## Your contribution

This should be fairly straight forward as we just need to add one method call to `trainer.Trainer` and a new method to `trainer_callback.TrainerCallback`. Happy to provide a PR.


Comments:
We can definitely accept a PR with this new method, as it seems there is a clear use case for it.
ZeroDivisionError in NotebookProgressBar.update with small dataset
Author: Rik-de-Kort
Pull request: no

I don't know the specifics, but during training (details below) NotebookProgressBar's update function got called with `force_update` is true, while no progress had been made (i.e. `value == self.start_value`). This leads directly to a ZeroDivisionError on line 151 in src/transformers/util/notebook.py.

## Environment info
- `transformers` version: 4.6.1
- Platform: Linux-5.4.0-1051-azure-x86_64-with-glibc2.17
- Python version: 3.8.10
- PyTorch version (GPU?): 1.8.1+cu102 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help
@sgugger based on Git blame.

## Information
Model I am using (Bert, XLNet ...): Roberta-Base

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [X] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: (give details below)

## To reproduce
Managed to reproduce based on the token classification notebook ([here](https://github.com/huggingface/notebooks/blob/master/examples/token_classification.ipynb)).
Steps to reproduce the behavior from this notebook:
After loading the dataset, apply the following code to make small:
```python
n = 14
datasets["train"] = datasets["train"].filter(lambda x: int(x["id"]) < n)
datasets["validation"] = datasets["validation"].filter(lambda x: int(x["id"]) < n)
datasets["test"] = datasets["test"].filter(lambda x: int(x["id"]) < n)
```

Replace the TrainingArguments in "Fine-tuning the model" with:
```python
model_name = model_checkpoint.split("/")[-1]
args = TrainingArguments(
    output_dir='./deletepls',
    num_train_epochs=1,
    per_device_train_batch_size=8,
    per_device_eval_batch_size=4,
    warmup_steps=500,
    weight_decay=0.01,
    learning_rate=5e-4,
    logging_dir='./logs',
    logging_steps=7,
)
```
After executing `trainer.train()` I get the ZeroDivisionError.

## Expected behavior
Some more descriptive error related to my logging steps, dataset size, or batch size. I'm still not sure what exactly causes this error.


Comments:
Thanks for reporting! I could reproduce and extracted a shorter reproducer (see the PR above). Fix is on its way :-)

Excellent!
Prevent `Trainer.evaluate()` crash when using only tensorboardX
Author: aphedges
Pull request: yes

# What does this PR do?

Fixes #12962

I did not write any tests because it seems like the logging integration callbacks have absolutely no testing at all, and I'm not creating a whole set of tests for a one-line fix.

## Who can review?

trainer: @sgugger


Comments:
Thanks for the fix!
Unable to convert output to interpretable format
Author: markrogersjr
Pull request: no

# 🚀 Feature request
There is no way to convert model outputs to a human-interpretable format, such as a list of token strings. Without this feature, there is no practicality other than benchmarking model performance, which does not require such a feature.

## Motivation
I'm trying to use the T5 CoLA task to determine whether an input sentence is grammatical.

## Your contribution
See #12969 


Comments:
How to get offset mapping then decoding wav2vec?
Author: hadaev8
Pull request: no


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Default Parameters for training DistillBERT and DistillGPT2
Author: umgupta
Pull request: no

https://github.com/huggingface/transformers/blob/cf3c9198aad5e2ea02e778aa9b04d27c216d1a35/examples/research_projects/distillation/train.py#L188

Hi @VictorSanh , 

I was going through your distillation code. Can you share what are the most suitable hyperparameters for training Distillation models, mainly DistillGPT2? Are the default parameters best to use? I am confused because the default batch size of 5 and 50 gradient accumulation step (i.e., 5X8X50 = 2000 examples) do not align with number reported in the [paper](https://arxiv.org/abs/1910.01108) (4K examples). 
 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Declaring `classifier_dropout` in model config but not using it
Author: PhilipMay
Pull request: no

The PEGASUS model is declaring `classifier_dropout` in the model config:

https://github.com/huggingface/transformers/blob/546dc24e0883e5e9f5eb06ec8060e3e6ccc5f6d7/src/transformers/models/pegasus/configuration_pegasus.py#L128

But the model never uses it. This is a bug IMO.

In contrast mBART defines it:

https://github.com/huggingface/transformers/blob/546dc24e0883e5e9f5eb06ec8060e3e6ccc5f6d7/src/transformers/models/mbart/configuration_mbart.py#L127

and uses it:

https://github.com/huggingface/transformers/blob/546dc24e0883e5e9f5eb06ec8060e3e6ccc5f6d7/src/transformers/models/mbart/modeling_mbart.py#L1395

If wanted I can provide a fix (non breaking) as a PR.

Comments:
Tagging @NielsRogge

also see this issue: #12781

@NielsRogge what do you think?

Well, since Pegasus does not have SequenceClassification or TokenClassification the `classifier_dropout` could be just removed from the config.

fixed in PR - closing
403 error in colab to download tokenizer
Author: moghadas76
Pull request: no

```
    647 class HTTPDefaultErrorHandler(BaseHandler):
    648     def http_error_default(self, req, fp, code, msg, hdrs):
--> 649         raise HTTPError(req.full_url, code, msg, hdrs, fp)
    650 
    651 class HTTPRedirectHandler(BaseHandler):
HTTPError: HTTP Error 403: rate limit exceeded
```

Comments:
Please follow the issue template, there is nothing we can do to help otherwise.
Trainer evaluation doesn't return eval loss for question-answering.
Author: avacaondata
Pull request: no
Labels: name: wontfix

As mentioned in this issue:
"""
Just a note that I tried `python run_squad_trainer.py --model_name_or_path bert-base-uncased --model_type bert --data_dir squad --output_dir /tmp/debug_squad/ --overwrite_output_dir --do_train --do_eval --evaluate_during_training --logging_steps 100`.

For some reason I don't get any evaluation metric during training (I was expecting `loss` or `eval_loss`).

_Originally posted by @borisdayma in https://github.com/huggingface/transformers/pull/4829#issuecomment-644338712_
"""
I'm facing the same problem. I'm trying to train with Trainer class over a QA dataset different from SQUAD. Everything works fine, the model learns based on train loss. However, I haven't been able to get the eval loss. I hope these pieces of code show how I'm configuring Trainer. Can somebody tell me if I'm doing something wrong?
```
training_args = TrainingArguments(
    output_dir="./models/prueba_2",  
    per_gpu_train_batch_size=16,
    per_gpu_eval_batch_size=32,  
    num_train_epochs=10, 
    logging_steps=10,  
    save_steps=25, 
    do_eval=True,
    evaluate_during_training=True,
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
    data_collator=collator,
    prediction_loss_only=True,
    compute_metrics = EvalPrediction
)
```
I also tried without EvalPrediction in compute_metrics.
Thanks in advance !


Comments:
I solved it editing trainer.py, in this line https://github.com/huggingface/transformers/blob/master/src/transformers/trainer.py#L765 . I added start_positions and end_positions to that "possible labels names" list, and it worked. Review the bug, please.

Thanks @alexvaca0 !

cc @julien-c , @sgugger 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


> I solved it editing trainer.py, in this line https://github.com/huggingface/transformers/blob/master/src/transformers/trainer.py#L765 . I added start_positions and end_positions to that "possible labels names" list, and it worked. Review the bug, please.

I am having the same issue. Can you elaborate on how you solved this problem? I could not locate a possible label list and start/end positions?

#7191 should solve the problem described here.

@sgugger How can we calculate validation and evaluation loss for question answering finetuning pipeline (run_qa.py)
Downloading the models is getting slower than before
Author: stepbystep88
Pull request: no


Downloading the models (using the **model.pre_trained('')** function) is getting slower than before, now the download speed is stable at 1MB/s, but it could reach 10MB/s a month ago.

I wonder if the official limits the download speed, please let me know if so.

Thanks for your supports.

Comments:
Nothing changed on our side.

Can you paste the output of e.g. `wget -O /dev/null https://huggingface.co/bert-base-uncased/resolve/main/pytorch_model.bin`?

![image](https://user-images.githubusercontent.com/47015435/123083858-16832680-d453-11eb-94c6-6f18104b1376.png)

Of course, please check the above picture.

Yep, that's downloading directly from [AWS Cloudfront](https://aws.amazon.com/cloudfront/), there's not much we can do on our side unfortunately

cc @osanseviero 

Okay, thanks for your attention to this issue.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Fix template for inputs docstrings
Author: sgugger
Pull request: yes

# What does this PR do?

The templates for the PyTorch model has a mistake in the input dosctrings (parenthesis should be inside the docstring and not the format) and several models had the same mistake (I realized it while reviewing Splinter actually). This PR fixes all of those and cleans up a few problems I spotted at the same time:
- image models don't need a format because there is nothing to format in the input docstrings
- some models that had the correct template for the input docstrings also add the parenthesis in some of the formats.

Comments:
Control sequence length for Token Classification with Trainer
Author: DimaClaudiu
Pull request: no

In the [new examples](https://github.com/huggingface/transformers/tree/master/examples/pytorch/token-classification) for running token classification training with pytorch and Trainer, there doesn't seem to be an option to control `max_seq_length`.

In the legacy version of the example, `DataTrainingArgs` has such an option, and for the example script without `Trainer` the option is also present.

Am I missing something? Or should I simply provide data samples pre-tokenized to my desired sequence length if I wish to use `Trainer`?

This was tested on v4.9.1.

Comments:
There is an option to set the `max_seq_length`, introduced in #12929

That is it! Thank you.