noamaanMulla-03/transformers-issues huggingface dataset:
I want to train a BART model for conditional text generation. I want to train the encoder and the decoder separately for a specific task. Can anyone help with the code? I am new to this.  @patrickvonplaten
Author: Sai-Ashish
Pull request: no

@## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform:
- Python version:
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten @TevenLeScao
 Blenderbot: @patrickvonplaten
 Bart: @patrickvonplaten
 Marian: @patrickvonplaten
 Pegasus: @patrickvonplaten
 mBART: @patrickvonplaten
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 RAG: @patrickvonplaten, @lhoestq
 FSMT: @stas00
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 ray/raytune: @richardliaw @amogkam
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1.
2.
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->


Comments:
Hi @Sai-Ashish 

Please use the forum https://discuss.huggingface.co/ for asking such questions. Use issues to report bugs, feature requests, etc. Thanks!

Closing it

have you came up with solution?
BART
Author: AbbyYin1230
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform:
- Python version:
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @Rocketknight1

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1.
2.
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->


Comments:
[ci] nightly: add deepspeed master
Author: stas00
Pull request: yes

We recently saw bugs introduced in Deepspeed master that were caught only on their new release. Which creates a gap in time where deepspeed integration doesn't work. To avoid that let's also test deepspeed master with the pytorch-nightly as there both pushing the bleeding edge.

Additionally this PR fixes the nightly pip install line that was misplaced originally. and pushes it first, before any other installs. Otherwise we end up first installing the current pytorch via dependencies of sub-packages, and then again the nightly.

@LysandreJik 

Comments:
Q About BART bart-large-cnn
Author: epsilon-deltta
Pull request: no

I'm using this BART tokenizer code
```python
from transformers import BartForConditionalGeneration, BartTokenizer
model_name = "facebook/bart-large-cnn"
tokenizer = BartTokenizer.from_pretrained(model_name)
```
I wanna read the explanation of bart-large-cnn. 
I guess this BPE tokenization (the paper said it uses BPE same as GPT-2) is trained on CNN/DM (Hermann et al., 2015). is that right?
Q1. Which dataset did the above tokenization use?
Q2 If there is a documentation about 'bart-large-cnn, bart-large-mnli, bart-large-xsum, bart-eli5, and bart-base', could you leave the link.

Thanks in advance.

Comments:
Hi there!

Thank you for opening the issue. Please use the [forum](https://discuss.huggingface.co/)  for such general questions. Issues are used to report bugs or feature requests.

So sorry I didn't know the forum exists
T5 pre training on different languages from scratch
Author: ashispapu
Pull request: no
Labels: name: wontfix

Hi Team,

I was exploring the pre training procedure/document for T5 models(small,base, large) models on different languages from scratch. However did not come across anything which could help me. Please share if there is any resource for the same. If not, could you please consider to include it .

Thank you .

Comments:
Hi @ashispapu, T5 pre-training is not yet available in transformers, I'm working on it but might take some time. Feel free to take a stab.

Hi @patil-suraj Thanks for the response. Let me try it.

@patil-suraj, thanks for working on the different T5 ropes. Can you point to the branch you 're working for pre-training? I'd be interested to contribute.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


I'm interested in T5 pre-training from scratch. Any update on this project?
Do the Trainer docs need an update?
Author: UrosOgrizovic
Pull request: no

On [this](https://huggingface.co/transformers/main_classes/trainer.html) documentation page regarding `Trainer`, `torch.utils.data.dataset.Dataset` is mentioned. However, I can only seem to find `torch.utils.data.Dataset` [here](https://pytorch.org/docs/stable/data.html#torch.utils.data.Dataset). Do the docs require an update?

The same goes for `IterableDataset`, on the same page.

Comments:
cc @sgugger 

Sure, would you mind making a PR with that?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Inconsistency between GPTNeo and GPT2 config classes
Author: leogao2
Pull request: no
Labels: name: WIP

The config classes for GPTNeo and GPT2 have a bunch of differences that are seemingly unnecessary. This makes it harder for downstream users to write code that depends on accessing these attributes. See below:

![image](https://user-images.githubusercontent.com/54557097/122113739-bbf92300-cddf-11eb-9ac6-a0ea1a30055b.png)
It seems that max_position_embeddings, hidden_size, num_layers, num_heads, intermediate_size, resid_dropout, embed_dropout, and attention_dropout should be renamed for sonsistency with the GPT2 config class.

### Who can help
@LysandreJik @patil-suraj 

Comments:
Seconding this.

Last month I swapped out GPT-2 for GPT-Neo in a [project](https://github.com/nostalgebraist/nostalgebraist-autoresponder/), and these differences made it more difficult to adapt my existing code.

Hi @leogao2 and @nostalgebraist, thanks for opening an issue! You're correct that the way this is currently implemented it prevents a few use-cases. Namely this is authorized:

```py
from transformers import GPT2Config

config = GPT2Config()
config.hidden_size
```

But these are not:

```py
from transformers import GPT2Config

config = GPT2Config()
config.hidden_size = 4
# Fails

config = GPT2Config(hidden_size=4)
# Fails
```

Unfortunately we can't just rename arguments - as this would break both checkpoints on the hub as well as local checkpoints. We're thinking of a way to enable this with a convention set across configurations for the attributes you mention - this convention would allow getting and setting attributes that are defined in this convention, such as the ones you mention.

Let us explore a bit and we'll come back to you. cc @patil-suraj @patrickvonplaten @sgugger 

> This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
> 
> Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

This still needs to be addressed.

Is there any progress on this?

Hey @leogao2! Yes, a proposal is available here: https://github.com/nreimers/transformers/commit/2198ee719c6101ef47de00cd6d53da3a8f938fb4 but there are still a few rough edges to polish. We'll try to have it merged in the next few weeks, will let you know.

This was fixed in #13026 which will be in the next release alongside GPT-J. Thank you for opening an issue!
Fix make fix-copies with type annotations
Author: sgugger
Pull request: yes

# What does this PR do?

As pointed out by #13583, there is a bug in the current check_copies utils when a function has a very long signature and a type annotation. This was due to a regex only checking for `):` instead of `):` or `) -> type:`. This PR fixes that and pushes some changes due to divergences that went undetected.

Fixes #13583 

Comments:
@patrickvonplaten Can you double-check the changes in HuBERT are good?

@sgugger I pulled this PR and found there is 1 edge case remained:

https://github.com/huggingface/transformers/blob/95f933ea855bce0c18a665f7a6a3b8ae9ab11739/src/transformers/models/bert/modeling_tf_bert.py#L902

Change the above line to

```
def serving_output(self, output: TFBaseModelOutputWithPooling):
```
or
```
def serving_output(self, inputs, output: TFBaseModelOutputWithPooling) -> TFBaseModelOutputWithPooling:
```
or
```
def serving_output(self, output2: TFBaseModelOutputWithPooling) -> TFBaseModelOutputWithPooling:
```

Currently (with the work in this PR), these changes won't be copied to 
https://github.com/huggingface/transformers/blob/95f933ea855bce0c18a665f7a6a3b8ae9ab11739/src/transformers/models/roberta/modeling_tf_roberta.py#L762


Ok for HuBERT

@ydshieh This is not a edge case but the way the check copies utility has been implemented. It does not check for the class name, function name, just the code inside. This is why the introduction line is ignored in the check.

@sgugger , Thanks, I understand now. However, I personally feel this is a bit strange. In my work on encoder-decoder, I need to add cross-attention and cache mechanism to some models, and these involve changing some return value types.
Like `TFBaseModelOutput` to `TFBaseModelOutputWithPastAndCrossAttentions`.

And the way `check copies` is implemented won't copy this kind of changes (single-line function name & signature) as shown in the provided example.

Moreover, if the function name & signature have multiple lines, the added or removed parameters will be copied (suppose they are not in the same line as the function name). This is somehow inconsistent with the single-line case above.

I will let Hugging Face to decide if it is necessary to address this :)
fix-copies doesn't work well in some cases
Author: ydshieh
Pull request: no

## Environment info

- `transformers` version: 4.11.0.dev0
- Platform: Windows-10-10.0.19042-SP0
- Python version: 3.9.5
- PyTorch version (GPU?): 1.9.0+cpu (False)
- Tensorflow version (GPU?): 2.5.0 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help
@sgugger @patrickvonplaten @LysandreJik 

## Information

When I worked on `TFEncoderDecoderModel`, I found in some cases, `fix-copies` doesn't work well. See below

## To reproduce

Steps to reproduce the behavior:

Case 1: 

  a. Put `inputs2 = inputs` after this line in `modeling_tf_bert.py` (just a dummy change to show the issues).

https://github.com/huggingface/transformers/blob/95f933ea855bce0c18a665f7a6a3b8ae9ab11739/src/transformers/models/bert/modeling_tf_bert.py#L628

  b. run `make fix-copies`, you will find nothing is changed. However, `TFRoBertaMainLayer.call` indicates it is copied from `TFBertMainLayer.call`, as shown

https://github.com/huggingface/transformers/blob/95f933ea855bce0c18a665f7a6a3b8ae9ab11739/src/transformers/models/roberta/modeling_tf_roberta.py#L477

Therefore, the (dummy) change on `TFBertMainLayer.call` (1a) is not applied to  `TFRoBertaMainLayer.call`.

Case 2:

  c. If we continue, and also change

https://github.com/huggingface/transformers/blob/95f933ea855bce0c18a665f7a6a3b8ae9ab11739/src/transformers/models/bert/modeling_tf_bert.py#L613
to 
```
):
```
and run `make fix-copies`, `TFRoBertaMainLayer.call` will be modified, but the result is very strange. See the attached file.
[results-2c.txt](https://github.com/huggingface/transformers/files/7172805/results-2c.txt)

Case 3:

  d. Revert the changes in `TFRoBertaMainLayer.call` (made by `fix-copies`), and keep the changes on  `TFBertMainLayer.call` (1a & 2c). Now change

https://github.com/huggingface/transformers/blob/95f933ea855bce0c18a665f7a6a3b8ae9ab11739/src/transformers/models/roberta/modeling_tf_roberta.py#L491
to
``` 
):
```
(so it becomes the same as L613 in `modeling_tf_bert.py`).
Run `make fix-copies`, you will see the change in `1a` is correctly copied to `TFRoBertaMainLayer.call`

## Expected behavior

It seems that type hints like `) -> Union[TFBaseModelOutputWithPooling, Tuple[tf.Tensor]]: ` doesn't work well with `fix-copies`.

Furthermore, single-line or multiple-lines will give different issues. For examples, changes on

https://github.com/huggingface/transformers/blob/95f933ea855bce0c18a665f7a6a3b8ae9ab11739/src/transformers/models/bert/modeling_tf_bert.py#L902

I expect the changes on `modeling_tf_bert.py` will be correctly copied to `modeling_tf_roberta.py`.

Comments:
[deepspeed] replaced deprecated init arg
Author: stas00
Pull request: yes

deepspeed 0.5.2 deprecated `config` in `zero.Init()` in favor of `config_dict_or_path` - deepspeed 0.5.3 fixed a bug in the previous version related to that https://github.com/microsoft/DeepSpeed/pull/1373. So this PR updates the arg and ups the minimal requirement.

0.5.3 fixes a bug in `zero_to_fp32.py` introduced by 0.5.2

The ci failure is unrelated.

@sgugger, @LysandreJik 



Comments:
Properly use test_fetcher for examples
Author: sgugger
Pull request: yes

# What does this PR do?

This PR fixes the `test_fetcher` util script for examples. It does so by adding the example test file whenever an example file is modified, or if said example file is modified (anything in examples/pytorch).
To avoid this being launched by the other test jobs, the default for the filters arguments is changed to "tests".

An example modification will now run the example test file: on the commit "Fake example modification", the test fetcher for the example script returns:
```
Master is at 421929b556aedf022a1c4a1f3b2f116b14a7b88a
Current head is at 5ba342cbf98664f5a7a98d6f02b798d78ee523f3
Branching commit: 421929b556aedf022a1c4a1f3b2f116b14a7b88a

### DIFF ###


### MODIFIED FILES ###
- examples/pytorch/language-modeling/run_clm.py
- utils/tests_fetcher.py

### IMPACTED FILES ###
- examples/pytorch/language-modeling/run_clm.py
- utils/tests_fetcher.py

### TEST TO RUN ###
- examples/pytorch/test_examples.py
```

In practice the examples job is run when any modification triggers any tests (so it's run when there is a real modification in the examples or the source code of transformers). Before this PR, the example modifications were ignored.

Comments:
XLMR tokenizer is fully picklable
Author: ben-davidson-6
Pull request: yes

# What does this PR do?
This addresses the issue here https://github.com/huggingface/transformers/issues/13200 to summarize:
- unpickling was dependant on what was on disk
the tokenizer is now unpickled only with the serialised proto. 

This is needed if you want to write a pyspark udf which tokenizes a column, as the tokenizer needs to be pickled and sent to other nodes.
# Who can help
@LysandreJik 

Comments:
> This looks good to me - do you think you could implement a test in `tests/test_tokenization_xlm_roberta.py`?

done
seq2seq trainer gives OOM error while evaluating
Author: saichandrapandraju
Pull request: no

Hi,

I'm trying t5-base for translation with source and target lengths of 320 and 256 respectively. I'm using Seq2SeqTrainer on A100-40GB GPU. For training, it is consuming not more than 20GB of GPU memory with batch_size of 8. But for evaluation with batch_size of 4, it is giving CUDA Out Of Memory error.

Later I figured that if I remove `compute_metrics` from trainer, everything is working fine. So could you plz suggest what's wrong with my below `compute_metrics` function ..?

```
def compute_metrics(pred):
    sacrebleu = datasets.load_metric("metrics/sacrebleu.py")    
    labels_ids = pred.label_ids
    pred_ids = pred.predictions

    labels_ids[labels_ids == -100] = tokenizer.pad_token_id
    label_str = tokenizer.batch_decode(labels_ids, skip_special_tokens=True)
    pred_str = tokenizer.batch_decode(np.argmax(pred_ids[0],axis=-1), skip_special_tokens=True)

    label_string = []
    for l in label_str:
        label_string.append([l])

    sacrebleu_output = sacrebleu.compute(predictions=pred_str, references=label_string)

    return {
        "bleu": sacrebleu_output["score"]
    }
```

Comments:
Hi, this could be because during evaluation `.generate` is used with beam search which uses more memory. Also by default, all predictions are stored on the device, so if the eval dataset is large it could result in OOM.

You could use `--eval_accumulation_steps` argument if it's passed the predictions are offloaded on CPU after every `eval_accumulation_steps` which reduced GPU memory.

Thanks a lot @patil-suraj , it worked!
Converting Transformers model to Tensorflow model
Author: hakanbogan
Pull request: no

I am trying to convert `dbmdz/bert-base-turkish-uncased` to tensorflow model (`.pb`). It contains `tf_model.h5` file. 

I tried to convert from `tf_model.h5` to tensorflow model. However, I couldn't handle it. Is `tf_model.h5` file keras model, isn’t it?

Is there any instruction to convert huggingface non-tf models to tensorflow model(`.pb`)?

Comments:
I don't believe we have a way of converting keras models from .h5 to .pb. What is your use case, so that we can see if we may help further?

I want to improve the inference performance. I fine-tuned the pretrained model (`dbmdz/bert-base-turkish-uncased`). TensorRT can be helpful to reduce inference time. Use case is converting Keras model to TensorRT model. How can I achieve this?

P.S.: I am open to any advice. Tensorflow Lite or other things can be used. I just need some guidance.

@mfuntowicz I believe you have experience with using TensorRT with the `transformers` models. Do you have any idea of how to enable this?

You can try a couple of ways : 
1. Keras -> Onnx -> TensorRT :  You can use keras2onnx . You might have some operations that are not directly supported in onnx so you'll have to remove/edit those operations to convertible terms. Converting onnx to TRT can be done with a tool called "trtexec" or there are readymade scripts to do that. Do check model graph and accuracy at every step.
2. Keras -> Frozen graph(.pb) -> onnx -> TensorRT

You can also use TF-TRT which again optimizes but less optimized than TensorRT. 

There is another way you can explore Keras -> UFF -> TensorRT . 

Thank you for leading @zerocool95 

 i also face this problem, and i find a exits onnx file on onnx github, and it said it was transfered from huggingface, but when i use trtexec to covert onnx ---> trt . it say it don;'t support some ops like 'NoZero'
Advice needed: Adding more FSMT models
Author: jvamvas
Pull request: no
Labels: name: New model

# 🌟 New model addition

## Model description

I am planning to contribute a series of FSMT models to the model hub. The models have been trained for a paper that is currently under review.

Before working on a PR I wanted to ask for some advice:

### normalize_before
The new models have been trained with Fairseq's option `normalize_before=True`, while the existing FSMT implementation uses `normalize_before=False`. I understand that copy-pasting model code is preferred to extending the configuration. This would mean that a near-duplicate module `fsmt_prenorm` needs to be created. Is this correct?

### Adequate base branch
The FSMT module is currently being refactored (https://github.com/huggingface/transformers/pull/11218). Do you recommend that I start from the master branch or from the PR's feature branch, which is nearly completed?

Comments:
@patil-suraj I am still very motivated to work on the pull request :)  Just let me know if you need more information to answer my question.

In case you're interested, the paper describing our models is now public (https://openreview.net/forum?id=RvO9DqoWI9V). I believe the models could be of value to others in the community.