noamaanMulla-03/transformers-issues huggingface dataset:
[Benchmark]
Author: Giahuynh1402
Pull request: no

# 🖥 Benchmarking `transformers`

## Benchmark

Which part of `transformers` did you benchmark?

## Set-up

What did you run your benchmarks on? Please include details, such as: CPU, GPU? If using multiple GPUs, which parallelization did you use?

## Results

Put your results here!

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
"test" prefix for rewrite_logs() function of WandbCallback
Author: bablf
Pull request: no

# 🚀 Feature request

I would like it if the function `rewrite_logs()` of WandbCallback() would be compatible with the trainers `.predict()` function. 

## Motivation

Currently the `rewrite_logs()` function looks like this: 

```python
def rewrite_logs(d):
    new_d = {}
    eval_prefix = "eval_"
    eval_prefix_len = len(eval_prefix)
    for k, v in d.items():
        if k.startswith(eval_prefix):
            new_d["eval/" + k[eval_prefix_len:]] = v
        else:
            new_d["train/" + k] = v
    return new_d

```

but `.predict()` has an argument which is called `metric_key_prefix: str = "test"` and sets the prefix to `test`, 
This leads to the kind of ugly metric name `train/test_f1` in wandb. Better would be `test/f1` 
 

## Your contribution
This should do the trick? 

```python
def rewrite_logs(d):
    new_d = {}
    eval_prefix = "eval_"
    eval_prefix_len = len(eval_prefix)
    test_prefix = "test_"
    test_prefix_len = len(test_prefix)
    for k, v in d.items():
        if k.startswith(eval_prefix):
            new_d["eval/" + k[eval_prefix_len:]] = v
        elif k.startswith(test_prefix):
            new_d["test/" + k[test_prefix_len:]] = v
        else:
            new_d["train/" + k] = v
    return new_d

```


Comments:
fix: replace asserts by value error
Author: m5l14i11
Pull request: yes

# What does this PR do?
The PR addresses issue #12789.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

- @willfrey
- @LysandreJik
- Anyone in the community is free to review the PR once the tests have passed.


Comments:
Please add GPT Jaaye model ine right with Transformers website
Author: danish9129
Pull request: no

Please add GPT Jaaye AI model to use in right with Transformers website to write text because this model has 6 billion parameters and you can scale it up to 40 billion parameters to find this model you can go to GitHub and search for GPTJ

Comments:
GPT J is already part of the transformer library  https://huggingface.co/EleutherAI/gpt-j-6B - you may need to install the master branch (`pip install git+https://github.com/huggingface/transformers` ) if it's not yet in a release

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Zero-shot classification pipeline change in 4.11.2
Author: davidmezzetti
Pull request: no

The latest version of the ZSL classification pipeline appears to give slightly different output for the following test code:

```python
from transformers import pipeline

nlp = pipeline("zero-shot-classification")
print(nlp(["I am happy", "I am sad"], ["negative", "positive"]))
```

Version 4.10.2 output
```
[{'sequence': 'I am happy', 'labels': ['positive', 'negative'], 'scores': [0.9984542727470398, 0.0015457083936780691]}, {'sequence': 'I am sad', 'labels': ['negative', 'positive'], 'scores': [0.9985381364822388, 0.0014618091518059373]}]
```

Version 4.11.2 output
```
[[{'sequence': 'I am happy', 'labels': ['positive', 'negative'], 'scores': [0.9984542727470398, 0.0015457083936780691]}], [{'sequence': 'I am sad', 'labels': ['negative', 'positive'], 'scores': [0.9985381364822388, 0.00146180868614465]}]]
```

It appears that 4.11.2 is adding an extra list around each element. This can be manually processed out but I also wanted to pass this along in case this wasn't expected new behavior. 


Comments:
@Narsil @LysandreJik fyi
Facing similar issue as well.

Thanks for letting us know, 

This is unknown breaking backward compatible, fortunately it's an easy fix ! 

@Narsil Thank you for quick fix.
Any idea when this fix will be released?

I will let core maintainers answer there but first the PR needs to be reviewed.
Small typo
Author: mehmetsinanyildirim
Pull request: no

https://github.com/huggingface/transformers/blob/4475f1dc2aa7153adb5e66b361c48aed1321fe3d/examples/pytorch/question-answering/run_qa.py#L423

On this line, the word 'agument' has a typo. I believe it should be argument. While we are at it, I think changing it to 'the argument' would be better as well. 

Comments:
If there's a typo, feel free to open a PR! Thanks.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Just spotted this issue again, creating a PR
T5ForConditionalGeneration: enabling using past_key_values and labels in training
Author: yssjtu
Pull request: yes

In T5ForConditionalGeneration, when training, we're not able to use the parameter _past_key_values_ together with _labels_. 
And even if we use decoder_input_ids rather than labels, only the last target token is used, if the _past_key_values_ parameter is provided.  
These limitations do not exist in other models like BART and BERT. 
Fixes:
- Delete the whole if statement to enabling using past_key_values in the T5Decoder when training.


These changes will not affect the `forward()` and `generate()` under normal conditions. And they make it possible to modify _past_key_values_ in the training of T5 the same way I've done in BARTForConditionalGeneration.



# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes https://github.com/huggingface/transformers/issues/13711





## Who can review?

@patrickvonplaten, @patil-suraj

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
@patil-suraj - can you take a final look here?
problem with using past_key_values in T5ForConditionalGeneration
Author: yssjtu
Pull request: no

Hello! Here's my problem, I'm trying to add some modifications to past_key_values(like adding some tunable parameters) during training. To do this, I must pass both the _past_key_values_ and _labels_ parameters during training. I have implemented this in the BARTForConditionalGeneration model, but when I'm trying to do this in T5ForConditionalGeneration,  I find the following limitations.

In T5ForConditionalGeneration, when training, we're not able to use the parameter _past_key_values_ together with _labels_. 

And even if we use decoder_input_ids rather than labels, only the last target token is used, if the _past_key_values_ parameter is provided.

https://github.com/huggingface/transformers/blob/62832c962f85b5a554ebf8b930d13b76b9028a8d/src/transformers/models/t5/modeling_t5.py#L1596-L1603

However,  this limitation doesn't exist in other models like BART (which I think has a similar structure with T5).
https://github.com/huggingface/transformers/blob/62832c962f85b5a554ebf8b930d13b76b9028a8d/src/transformers/models/bart/modeling_bart.py#L1293-L1298

As far as I can see, the limitition in T5ForConditionalGeneration is not necessary.  Even if we pass both the _past_key_values_ and the _labels_, there'll be no errors since the inner modules like T5Block and T5Attention can handle the past_key_values properly(I'm trying to delete this limitation in T5 in my own fork, but I've not experimented yet, so I'm not 100% sure). Or are there some other purposes of this limitation in T5?
  


Comments:
cc @patrickvonplaten 

Hey @yssjtu, 

I agree with you! Would you like to open a PR to fix this? :-)
Fixing Backward compatiblity for zero-shot
Author: Narsil
Pull request: yes

Fixes #13846


# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@LysandreJik

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

Comments:
Error while running GPT-J 6B with revision="float16"
Author: zgerrard
Pull request: no

Hi, I am getting `RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'` while running the following snippet of code on the latest master. Suspect that it is because of the `revision="float16"`.

## Code
```
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model = AutoModelForCausalLM.from_pretrained("EleutherAI/gpt-j-6B", revision="float16", torch_dtype=torch.float16)
tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-j-6B")
prompt = "In a shocking finding, scientists discovered a herd of unicorns living in a remote, " \
"previously unexplored valley, in the Andes Mountains. Even more surprising to the " \
"researchers was the fact that the unicorns spoke perfect English."
input_ids = tokenizer(prompt, return_tensors="pt").input_ids
gen_tokens = model.generate(input_ids, do_sample=True, temperature=0.9, max_length=100,)
gen_text = tokenizer.batch_decode(gen_tokens)[0]
```

The error happens in `model.generate`. Stacktrace:

```
/usr/local/lib/python3.7/dist-packages/transformers/models/gptj/modeling_gptj.py in forward(self, hidden_states, layer_past, attention_mask, head_mask, use_cache, output_attentions)
    272     ):
    273         residual = hidden_states
--> 274         hidden_states = self.ln_1(hidden_states)
    275         attn_outputs = self.attn(
    276             hidden_states,
```

### Who can help

Models:

- gptj: @patrickvonplaten, @EricHallahan, @StellaAthena 

## Information

Model I am using (Bert, XLNet ...): GPT-J 6B

## To reproduce

Steps to reproduce the behavior:

Run the provided code snippet


Comments:
You need hardware that supports half-precision to run the model in half-precision. If you have a GPU with float16 support, load the model to GPU. If you only have a CPU without float16 support, you must specify `torch_dtype=torch.float32` instead.

I got the same error though I use p3.2xlarge on AWS, which have half precision support.
any idea what to do?
🐛 BART : Same representations for different `
` tokens 
Author: astariul
Pull request: no
Labels: name: Help wanted

name: wontfix

# 🐛 Bug

## Context

I'm trying to use BART for a sentence classification task. So I encode the input with the following format :

```

 Sen1 
 
 Sen2 
 
 Sen3 
 ...
```

And use `
` as sentence representation (from the encoder, not decoder). Then I classify these representations.

---

I trained my model but the classification give random choice. After debugging, I noticed that the encoder produce always the same representation for `
` token. 

## Bug

**The representation of `
` token is always the same, no matter where they appear in the input.**

Here is a notebook reproducing the issue : [Colab](https://colab.research.google.com/drive/1mqKKFAEGEwa5XbkJtm7_VrmQ8L3_0Bnt?usp=sharing)

In this notebook I simply encode an input, modify the input to add an additional `
` token, forward it through BART and compare the encoder representation of `
`. It gives me :

```
tensor([-0.0097,  0.0075,  0.0086,  ...,  0.0041, -0.0085, -0.0011],
       grad_fn=<SelectBackward>)
tensor([-0.0097,  0.0075,  0.0086,  ...,  0.0041, -0.0085, -0.0011],
       grad_fn=<SelectBackward>)
```

---

Now if I do the same with `
` token, their representation is different :

```
tensor([ 0.0658,  0.0161, -0.0062,  ..., -0.0536, -0.0515,  0.1837],
       grad_fn=<SelectBackward>)
tensor([ 0.0627,  0.0576,  0.0408,  ..., -0.0406, -0.0765,  0.1689],
       grad_fn=<SelectBackward>)
```

Which makes sense because they represent different thing...

## Note 1

I think this is a bug because I remember a few months ago, I tried to do the same (use `
` representation for classification) with fairseq version of BART, and it worked...

## Note 2

I'm aware the example in the Notebook is using just the pre-trained version of BART, so the `
` representation does not represent sentence. But even after training the model, the exact same behavior arise.

@sshleifer 

Comments:
Very strange indeed. Nothing comes to mind, but if you can show the fairseq discrepancies I can take a look.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


@astariul Hi I also encounter this problem. The finetuned BART just gives the same output for `
` under eval mode, have you fixed it? 

@turing-yfqiu I'm sorry it's long time ago, I don't remember how I did....
mBART50 finetuned many-to-many model failed to generate translation
Author: Mao-KU
Pull request: no

@patrickvonplaten
Hello, I downloaded mBART50 finetuned many-to-many model. When generating translations, it just repeats the same word over and over again.
This problem is also reported in https://github.com/huggingface/transformers/issues/7060#issuecomment-765648270.

I used mBART50 finetuned many-to-many model to do the ar_AR-en_XX translation on the IWSLT17 ar-en test set.
No bug occurred during the generation.
Also, I have done the multilingual fine-tuning for mBART25 successfully, so I suppose I did the preprocessing correctly.

The file that mBART50 fine-tuned N-N generated is put here:
https://lotus.kuee.kyoto-u.ac.jp/~zhuoyuanmao/ar_AR-en_XX

Comments:
sorry, I will go to fairseq to ask this question
pre-training roberta from scratch
Author: ozyurtf
Pull request: no

Hi,

I am currently pre-training RoBERTa model from scratch based on the tutorial in here: https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/01_how_to_train.ipynb and I got 3 questions.

1) I see that the data used in this tutorial is separated line by line and the train and test data is put into LineByLineTextDataset function before training. What is the usage of this function ? I couldn't find an explanation of it in Huggingface documents. Even though I have guesses, I would like to know better.  As I see the data in the tutorial is already split line by line. So, what would have been lost if these data are not put LineByLineTextDataset function before starting the training process ? 

2) The data I will use in pre-training process is not split line by line. In my data, each example have multiple sentences. So, I assume that I shouldn't use LineByLineTextDataset function. Is there any other function I can use instead ? Or would it be wrong if I would join the whole dataset in one very long text and then split it into sentences and then use LineByLineTextDataset function ?

3) This will be a little bit technical question. In RoBERTa paper, it was written that the size of the data they used for pre-training process is 160 GB.  If I would pre-train the model with 10-20 GB of data, would this decrease the performance of the pre-trained model drastically or is it okay to use as long as the training arguments are set accordingly ? 

Comments:
Hi @ozyurtf !

Thank you for opening the issue. It would be nice if you could ask such general questions on the [forum](https://discuss.huggingface.co/). We use issues for bug reports and feature requests.

Okay, I will. Thank you Suraj!
DestilGTP2 code from pytorch-transformers does not work in transformers, I made a basic example
Author: Oxi84
Pull request: no
Labels: name: Migration

How would i convert this to new version of transformers. Or is it possible to somehow use DestilGTP2 with pytorch-transformers.

    use_transformers = True
    if use_transformers: 
        import torch   
        from transformers import GPT2Tokenizer, GPT2Model, GPT2LMHeadModel                       
        tokenizer1 = GPT2Tokenizer.from_pretrained('distilgpt2',cache_dir="/var/software/Models/")
        model1 = GPT2LMHeadModel.from_pretrained('distilgpt2',cache_dir="/var/software/Models/") 
        model1.eval()
        model1.to('cuda')
        text = "Who was Jim Henson ?"
        indexed_tokens = tokenizer1.encode(text)
        tokens_tensor = torch.tensor([indexed_tokens])
        tokens_tensor = tokens_tensor.to('cuda')
       with torch.no_grad():
            predictions_1 = model1(tokens_tensor) 
       print(predictions_1) 
   
    else:  
        import torch  
        from pytorch_transformers import GPT2Tokenizer, GPT2Model, GPT2LMHeadModel
        tokenizer1 = GPT2Tokenizer.from_pretrained('gpt2',cache_dir="/var/software/Models/") # cache_dir=None
        model1 = GPT2LMHeadModel.from_pretrained('gpt2',cache_dir="/var/software/Models/")
        model1.eval()
        model1.to('cuda')
        text = "Who was Jim Henson ?"
        indexed_tokens = tokenizer1.encode(text)
        tokens_tensor = torch.tensor([indexed_tokens])
        tokens_tensor = tokens_tensor.to('cuda')
        with torch.no_grad():
            predictions_1 = model1(tokens_tensor) 
        print(predictions_1) 

When i try i get an error, and tried to follow the guide but do not get what the new tokeniser does differently. 

    

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.