noamaanMulla-03/transformers-issues huggingface dataset:
RAG Model without DPR
Author: krishanudb
Pull request: no

Hello everyone,
I am interesting in studying how RAG generator answers questions without the DPR retriever but with other passages / contexts identified by other methods.

For example in the code below

```from transformers import RagRetriever
from transformers import RagTokenizer, RagRetriever, RagTokenForGeneration

retriever = RagRetriever.from_pretrained(’./rag-token-nq’, indexed_dataset=dataset)
tokenizer = RagTokenizer.from_pretrained("./rag-token-nq")
model = RagTokenForGeneration.from_pretrained("./rag-token-nq", retriever=retriever)

input_dict = tokenizer.prepare_seq2seq_batch(“How many people live in Paris?”, “In Paris, there are 10 million people.”, return_tensors=“pt”)
input_ids = input_dict[“input_ids”]

model = RagTokenForGeneration.from_pretrained(“facebook/rag-token-nq”, retriever=retriever)

generated_ids = model.generate(input_ids=input_ids, labels=input_dict[“labels”])

generated_string = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)
print(generated_string) 
```

In the line 
```input_dict = tokenizer.prepare_seq2seq_batch(“How many people live in Paris?”, “In Paris, there are 10 million people.”, return_tensors=“pt”) ```
, I want to use “How many people live in Paris ?” as the question and “In Paris, there are 10 million people.” the passage/context which should be used to generate the answer.

Kindly let me know how to do this?

Is my understanding of the code correct and if not, how to go about it?

Thanks,
Krishanu

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discusss.huggingface.co) instead? Feel free to tag @patrickvonplaten or @lhoestq on the forum, they'll probably be able to answer your quesitons :).

Thanks!
Fixes to run_seq2seq and instructions
Author: sgugger
Pull request: yes

# What does this PR do?

This PR fixes two issues in the new `run_seq2seq` script and adds instructions on how to run it. The fixes are:
- default of `val_max_target_length` to `max_target_length` (had forgotten to do this in the initial PR)
- add an optional prefix to the source text (for T5 models)

Comments:
Add `report_to` training arguments to control the integrations used
Author: sgugger
Pull request: yes

# What does this PR do?

This PR introduces a new `report_to` training argument that controls which of the multiple reporting tools to use in a training round. Currently, `Trainer` automatically uses everything installed, which can cause trouble when:
- one platform is installed but not properly set up.
- one platform is installed but the user doesn't want to use it today.

In my opinion the current behavior is too magical and does not fit our philosophy. To avoid any breaking change, the current default for this `report_to` argument is to use everything installed, but I would like to switch this to an empty list at the next major release, so the user has to opt-in the platforms they want to use.


Comments:
[fsmt] token_type_ids isn't used
Author: stas00
Pull request: yes
Labels: name: fsmt

This PR fixes a bug discovered in https://github.com/huggingface/transformers/issues/9722

`token_type_ids` was returned by default by the tokenizer, but it isn't used by the model.

The original fsmt port was a frankenstein of bart for the model and xlm for the tokenizer, hence the discrepancy.

I thought the tests had common onnx tests, but it doesn't seem to be the case. I added a local test in the related PR:  https://github.com/huggingface/transformers/pull/9738

With this fix `convert_graph_to_onnx.convert` still fails with:
```
RuntimeError: Exporting the operator triu to ONNX opset version 12 is not supported. Please open a bug to request ONNX export support for the missing operator
```
but that's a totally different issue. Fixed in https://github.com/huggingface/transformers/pull/9738

Fixes: https://github.com/huggingface/transformers/issues/9722

@LysandreJik, @patrickvonplaten 


Comments:
[trainer] `model` argument is not the same depending on n_gpus
Author: stas00
Pull request: no
Labels: name: wontfix

Extracting the discussion from https://github.com/huggingface/transformers/pull/8716

Summary of the issue: `prediction_step()` has a `model` argument which is a normal model with n_gpu < 2, and a wrapped DataParallel model with n_gpu > 1. So the API suffers from ambiguity here.

The user has to really use `self.model` to be able to call methods like `model.config()` or `model.generate()`, which can't be called on the wrapped model. But it's very likely they will use `model` instead since it'll act like `self.model` unless under multi_gpu. And why do we even have that `model` argument then?

Possible solutions discussed:

1. monkeypatch `torch.nn.DataParallel` to expand its API to support all the methods of the original model transparently by installing a catch all `__getattr__` and remap all the failed method look ups to delegate to `self.module`.

2. not to call the function argument `model` anymore, since it isn't under multi gpu, but is something else. 

3. remove the `model` argument completely + document to always use `self.model` - currently in `seq2seq_trainer.py `once we switch to `self.model`, `prediction_step()` no longer needs `model` as an argument (but is it always the case?)

4. pass `self.model `as the `model` arg, and making the wrapped model available via `self.wrapped_model` if the user needs it.

Summary of discussion around proposed solutions:

1. too magical

2. proposed calling it `wrapped_model`, but it's just as confusing since most of the time it's not.

3. need to check whether wrapped model is every needed inside user functions.

4. was not discussed yet

@sgugger, @LysandreJik 

Comments:
Note that this all internal and a user only interacts/get used with that if they subclass `Trainer` and override the `prediction_step` method. I would keep it simple since it's touching a small part of our users that should be experienced enough to be able to read the docstrings, and just detail in the docstrings with a proper warning what this `model` argument represents.

I can also live with 4 if it's the solution selected.

As I didn't participate in the design of Trainer, and I don't know whether it's meant to be easily sub-classable or not - I currently can only think of some ideas and I trust you guys to choose the most suitable solution. I hope it makes sense.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Well, we have implemented number 4, so closing this.
[Open in Colab] links not working in examples/README.md
Author: wilcoln
Pull request: no

For the following tasks below, the ![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg) button contains github links instead of colab links.

- question-answering
- text-classification
- token-classification

@sgugger




Comments:
Hi @wilcoln 

Yes, the links point to Github, feel free to open a PR to replace the GitHub links with colab :). Thanks!
How to pretrain the model (like Roberta) again?
Author: drxmy
Pull request: no
Labels: name: wontfix

I don't want to pretrain the model from scratch. I have some dataset related to my task. I want to pretrain the model from transformers the second time. Could someone give me some advice on how to do it or which document to read?  Thank you!

Comments:
The authors of the Transformers library wrote a script for this. You can find it under the examples folder -> language modeling [here](https://github.com/huggingface/transformers/tree/master/examples/language-modeling#robertabertdistilbert-and-masked-language-modeling). 

> The authors of the Transformers library wrote a script for this. You can find it under the examples folder -> language modeling [here](https://github.com/huggingface/transformers/tree/master/examples/language-modeling#robertabertdistilbert-and-masked-language-modeling).

Thank you! I will check it out.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
attention_mask size
Author: Jiaxin-Wen
Pull request: no

# 🚀 Feature request

current attention_mask argument is a tensor of shape [batch_size, sequence_length], 

I'd like it to be a tensor of shape [batch_size, from_seq_length, to_seq_length], as I want to set a different attention mask for a different position.


Comments:
have implemented that by myself.
If needed, I can make a pull request

Hi, I'm also trying to utilize such a customized mask. Would you mind sharing your implementation? Thank you!
save tokenizer and model from fine tuned LED model
Author: mmoya01
Pull request: no

Hello, I have been following this [notebook](https://colab.research.google.com/drive/12LjJazBl7Gam0XBPy_y0CTOJZeZ34c2v?usp=sharing#scrollTo=jpUr9QeebZ-n) to fine the `patrickvonplaten/led-large-16384-pubmed` model on my own data. However,after training, when I tried doing:

```python
model.save_pretrained("new_model")
tokenizer.save_pretrained("new_model")

```


to save the model and tokenizer, I noticed when I check out the `config.json` for it, it says 
`"_name_or_path": "patrickvonplaten/led-large-16384-pubmed",`

that said, is it actually saving the fine tuned model or just resaving `patrickvonplaten/led-large-16384-pubmed`? I'd greatly appreciate any feedback on this

Comments:
Trainer object empties dataset
Author: AlexBella365
Pull request: no

## Environment info
- `transformers` version: 4.2.2
- Platform: Linux-4.14.81.bm.20-amd64-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.7.0 (True)
- Tensorflow version (GPU?): 2.3.1 (True)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

## Information

Model I am using (Bert, XLNet ...): XLM-Roberta

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [X] an official GLUE/SQUaD task: NER
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Create a multilingual dataset by concatenating a handful of languages from wikiann
2. Instantiate the model
3. Instantiate the Trainer object

![image](https://user-images.githubusercontent.com/22292468/105478439-79ee0a00-5ca3-11eb-99f3-a648246187af.png)


## Expected behavior
The dataset object should not have been modified by the Trainer object

Comments:
Your dataset is not really emptied (I agree it looks like this however). It's just viewed without the columns the model can accept. You can restore all your columns with
```
dataset["train"].set_format(columns=list(dataset["train"].features.keys())
``` 
or more simply
```
dataset["train"].reset_format()
```
I agree that this is strange and we're seeing how we can have the same behavior without changing the dataset you pass to Trainer.

Note that you didn't preprocess your data, so you won't be able to train in this example.


Hi Sylvain,
Thanks so much for prompt response.

I was not aware of this behaviour so was left very puzzled. I tried what you suggested and indeed got back all my data.

Yes, I agree with you, the data was not yet ready for training. I was just trying to pinpoint at which stage in my script the dataset became "empty".

Thanks again :)
AttributeError: 'Trainer' object has no attribute 'is_world_master'
Author: blagav
Pull request: no

## Environment info
     
- `transformers` version: 4.0.0
- Platform: Linux-4.15.0-96-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.8
- PyTorch version (GPU?): 1.7.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No
- Ray version: 1.0.1.post1

### Who can help
@sgugger — Would you be able to offer any insight?

## Information

Model I am using (Bert, XLNet ...): BertForSequenceClassification

The problem arises when using:
* [ ] the official example scripts:
* [ x] my own modified scripts:

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task:
* [ x] my own task or dataset:

## To reproduce
The error occurs when I try to run a hyperparameter search for my finetuning step using Ray Tune. I am able to successfully finetune the BertForSequenceClassification model normally — the error only arises when running hyperparameter search.

```
config = BertConfig.from_pretrained(pretrained_model_path, num_labels=num_labels, finetuning_task ='text-classification')

def model_init():
    return BertForSequenceClassification.from_pretrained(pretrained_model_path, config=config)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
    model_init=model_init
)

trainer.hyperparameter_search(
    direction="minimize", 
    backend="ray",
    n_trials=20,
    keep_checkpoints_num=1,
    resources_per_trial = {'gpu':1, 'cpu':1}
)
```

## Expected behavior

I am trying to run Ray Tune from Huggingface as per these instructions: https://huggingface.co/blog/ray-tune

If anyone has any insight as to what could be causing this error, it would be greatly appreciated, thank you!


Comments:
you may replace is_world_master by is_world_process_zero.

Any solutions to this? I am not able to use ray tune backend in any way

If you get this error still it's probably because you're using mismatched library versions/example versions. Using the latest examples or a more recent version (v4.1.x) should patch this.

If it doesn't, then please open a new issue and fill in the issue template so that we may help you. Thank you.
Fix a typo in `Trainer.hyperparameter_search` docstring
Author: sorami
Pull request: yes

`compute_objectie` => `compute_objective`

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes a typo in `Trainer.hyperparameter_search` docstring: `"compute_objectie"` => `"compute_objective"`


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.


Comments:
Link Not Working
Author: akshat311
Pull request: no


 @patrickvonplaten 

I was looking at the MarianMT docs, and in the "Examples" section, the link for "Fine-Tune on GPU" and "Fine-Tune on GPU with pytorch-lightning" is broken.
Kindly look into this issue.



Comments:
Thanks for pointing it out, the files are now renames to `distil_marian_enro_teacher.sh`, `distil_marian_no_teacher.sh` and are available here 
https://github.com/huggingface/transformers/blob/master/examples/research_projects/seq2seq-distillation/distil_marian_enro_teacher.sh

https://github.com/huggingface/transformers/blob/master/examples/research_projects/seq2seq-distillation/distil_marian_no_teacher.sh

Feel free to open a PR to fix the links if you want to contribute :) 

Also, we now recommend using the new `Seq2SeqTrainer` for fine-tuning seq2seq models, https://github.com/huggingface/transformers/tree/master/examples/seq2seq

Thanks a lot. Also, there a way to train the MarianMT model on my dataset ?

Sure, the script let's you pass your own dataset as well, have a look at the readme.

Thanks a lot for the help.
fix text summarization evaluation bugs when calculate rouge
Author: ShichaoSun
Pull request: yes


# What does this PR do?

fix text summarization evaluation bugs when calculate rouge

It is important for calculating rouge

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?
@patil-suraj @sshleifer @patrickvonplaten




Comments:
only two lines changed but it take much effort to pass the check!
I think it may be important for the evaluation result.
They are simple but true bugs

Great catch @ShichaoSun  and thank you for working on this!
We are in the process of finishing the [new standalone seq2seq script](https://github.com/huggingface/transformers/blob/master/examples/seq2seq/run_seq2seq.py) with the same functionality but without depending on the utils or other helpers. The utils and other helpers will probably be removed once we completely test the new script.

Great job!  Looking forward to seeing that.
Fix model parallel definition in superclass
Author: LysandreJik
Pull request: yes

The `model_parallel` attribute should be obtainable from every class instance that has the `is_parallelized` class attribute. Otherwise the following line in the trainer crashes:

https://github.com/huggingface/transformers/blob/626116b7d76efef5137c3b4a92e64e3bb57a6882/src/transformers/trainer.py#L244

cc @stas00 @alexorona 

Comments:
Add BlenderbotSmallForCausalLM for EncoderDecoder
Author: sadakmed
Pull request: yes

# What does this PR do?

Implementing BlenderbotSmallForCausalLM 
 
Issue #9066
PR #9128

@patrickvonplaten


Comments:
Hey @sadakmed, sorry I might have been a bit unclear here. Could you instead of opening new PRs add your added code directly to your PR here: https://github.com/huggingface/transformers/pull/9128? The `#Copy from` statements force us to have all the code in the same PR.

Also we need a decoder-only test for this.
Implementation of BlenderbotForCausalLM
Author: sadakmed
Pull request: yes

# What does this PR do?
implementing BlenderbotForCausalLM for EncoderDecoder use, like ProphetNetForCausalLM

Fixes # (issue)
#9066 
#9128 

@patrickvonplaten


Comments:
Hey @sadakmed, sorry I might have been a bit unclear here. Could you instead of opening new PRs add your added code directly to your PR here: https://github.com/huggingface/transformers/pull/9128? The `#Copy from` statements force us to have all the code in the same PR.

Also we need a decoder-only test for this.
PegasusForCausalLM, analog to `ProphetNetForCausalLM`
Author: sadakmed
Pull request: yes

# What does this PR do?

this PR is implementing PegasusForCausalLM for EncoderDecoder use like ProphetNetForCausalLM  and it follow work of BartForCausalLM #9128 
issue #9066 

for a organisation matter, each one is in its own PR. @patrickvonplaten


Comments:
Hey @sadakmed, sorry I might have been a bit unclear here. Could you instead of opening new PRs add your added code directly to your PR here: https://github.com/huggingface/transformers/pull/9128? The `#Copy from` statements force us to have all the code in the same PR.

Also we need a decoder-only test for this.
adding MarianForCausalLM for EncoderDecoder use
Author: sadakmed
Pull request: yes

# What does this PR do?
adding MarianForCausalLM for EncoderDecoder use

Fixes #9066 
PR #9128 

@patrickvonplaten


Comments:
Hey @sadakmed, sorry I might have been a bit unclear here. Could you instead of opening new PRs add your added code directly to your PR here: https://github.com/huggingface/transformers/pull/9128? The `#Copy from` statements force us to have all the code in the same PR.

Also we need a test for this.

Hey @sadakmed, sorry I might have been a bit unclear here. Could you instead of opening new PRs add your added code directly to your PR here: https://github.com/huggingface/transformers/pull/9128? The `#Copy from` statements force us to have all the code in the same PR.

Also we need a decoder-only test for this.
does LED use distributed training by default?
Author: mmoya01
Pull request: no

Hello, I'm currently fine tuning the `allenai/led-base-16384` model and following allow this [notebook](https://colab.research.google.com/drive/12LjJazBl7Gam0XBPy_y0CTOJZeZ34c2v?usp=sharing#scrollTo=jpUr9QeebZ-n) .The node that I'm using has a couple of V100-SXM2-16GB GPUs in it. That said, does `Seq2SeqTrainer` automatically use distributed training by default? 
I noticed it says 
```
the inner model is wrapped in ``DeepSpeed`` and then again in ``torch.nn.DistributedDataParallel``.
```

but I just wanted to triple check that the training job would distributed across GPUs. I'd greatly appreciate the feedback

Comments:
Ah, I just noticed `Seq2SeqTrainingArguments` sets `local_rank=-1` by default