noamaanMulla-03/transformers-issues huggingface dataset:
[deepspeed tests] transition to new tests dir
Author: stas00
Pull request: yes

as discussed at https://github.com/huggingface/transformers/issues/10076 relocating deepspeed tests a dedicated area and out of the scripts area.

I went right ahead and create a dedicated sub-folder for deepspeed tests.

I no longer can use the libraries from `seq2seq` since I can't do a relative import from the script. 

The only thing is that I will need to update all of my comments/posts to mention that `ds_config.json` has moved.

fairscale will be probably next if this looks good.

Fixes: https://github.com/huggingface/transformers/issues/10076

@sgugger 

Comments:
> I no longer can use the libraries from seq2seq since I can't do a relative import from the script.

We can add things to the sys path if needed (see the [general text examples](https://github.com/huggingface/transformers/blob/master/examples/test_examples.py)).

Thanks for doing this, it looks good to me!

Indeed, that's what we have been doing, but having a library named "`utils.py`" and importing it from far away is too ambiguous. So will probably need to rename such libraries, or start moving their functionality into a more central area.
[Good first issue] ELECTRA PyTorch Integration tests
Author: LysandreJik
Pull request: no
Labels: name: PyTorch

name: Tests

name: Good First Issue

The PyTorch implementation of the ELECTRA model currently has no integration tests. This is problematic as the behavior can diverge without being noticed.

The [test_modeling_electra.py](https://github.com/huggingface/transformers/blob/master/tests/test_modeling_electra.py) file should be updated to include integration testing.

An example of a good modeling integration test is visible in the [test_modeling_bert.py#L552-L565](https://github.com/huggingface/transformers/blob/master/tests/test_modeling_bert.py#L552-L565) file:

https://github.com/huggingface/transformers/blob/1809de5165804666ba6c6a02a9d177f6683869cc/tests/test_modeling_bert.py#L552-L565

Some additional tips:
- The test must be marked as slow using the `@slow` decorator, so as to be run *daily*, and not on every commit of every branch/pull request of this repository.
- The test must be decorated with the `@require_torch` decorator so as to only be run in environments using PyTorch.
- A single test is necessary. If you feel like implementing multiple of these, then sharing the same checkpoint would be ideal so as to reduce download time.
- The TensorFlow implementation already has an integration test, which is visible here:

https://github.com/huggingface/transformers/blob/1809de5165804666ba6c6a02a9d177f6683869cc/tests/test_modeling_tf_electra.py#L253-L267

This test can be translated to PyTorch.

Comments:
Added integration tests for Pytorch implementation of the ELECTRA model
Author: spatil6
Pull request: yes

Added integration tests for Pytorch implementation of the ELECTRA model

Fixes #9949 

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

@LysandreJik 

Comments:
What does 'output of the embeddings' mean?
Author: h56cho
Pull request: no

Hello,

According to Hugging Face Transformers documentation, (https://huggingface.co/transformers/model_doc/gpt2.html#gpt2doubleheadsmodel)
the transformer's output ```hidden_state``` is defined as the following:

```
hidden_states: (optional, returned when config.output_hidden_states=True)
list of torch.FloatTensor (one for the output of each layer + the output of the embeddings) 
of shape (batch_size, sequence_length, hidden_size): Hidden-states of the model 
at the output of each layer plus the initial embedding outputs.
```

I am a bit confused by the statement ```list of torch.FloatTensor (one for the output of each layer + the output of the embeddings) ```. Does the ```output of the embeddings``` at the end of the statement refer to 'output of the uppermost output layer'?

Thank you,


Comments:
The output of the embeddings is the sum of the token embeddings + the segment embeddings + the position embeddings. This value is the value that will be fed to the first layer of the transformer.

@LysandreJik 

Hello,

Thank you very much for your reply.

So according to the Hugging Face Transformer documentation for the ```GPT2DoubleHeadsModel``` (under the 'output' section)

```
hidden_states: (optional, returned when config.output_hidden_states=True)
list of torch.FloatTensor (one for the output of each layer + the output of the embeddings) 
```
So in this case, would the first ```hidden_states``` tensor (index of 0) that is returned be the output of the embeddings, or would the very last ```hidden_states``` tensor that is returned be the output of the embeddings? 

I am confused about the order in which the ```hidden_states``` tensors are returned, because the documentation seem to indicate that the output of the embeddings is the last ```hidden_state``` tensor that is returned.

Thank you,

Indeed, the documentation might be misleading in that regard. The first value is the embedding output, every following value is the result of the preceding value being passed through an additional layer. I'll update the documentation shortly.

I remain confused by this and will be posting on the Disqus.
Removing run_pl_glue.py from text classification docs, include run_xnli.py & run_tf_text_classification.py
Author: cbjuan
Pull request: yes

# What does this PR do?

Since `run_pl_glue.py` is not part of `text-classification` examples after #9010, this PR removes it from the text-classification docs. Also, it adds `run_xnli.py` and `run_tf_text_classification.py` scripts, which are in that folder now.
<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@sgugger may be interested in the PR, he's responsible for docs and the author of #9010
 



Comments:
Sure! Change applied

Done, command `make style` applied. Thanks for the guidance

Build still not succeeding. I will check 

Ah yes, those links don't need the underscores! Good catch and sorry for giving you the wrong example to follow. Just waiting for the last tests and we can merge :-)

No worries, @sgugger. Thanks for the guidance :)
How to use BertModel ?
Author: g-jing
Pull request: no
Labels: name: wontfix

I want to use bert-crf in my NER task. But this github only provide softmax as the classifier, I decided to write my own crf. But I am not sure how to use it. Here is an example. Please correct me if I am wrong.

sentence: Here is some text to encode
input: torch.tensor([tokenizer.encode("[CLS]" + "Here is some text to encode" + "[SEP]")]), which is 
tensor([[  101,  3446,  1110,  1199,  3087,  1106,  4035, 13775,   102]])

output shape: [1, 9, 768],

Becasue "encode" is devided into two pieceword.
Then I select output[1, 2, 3, 4, 5, 6] from output[0,1,2,3,4,5,6,7,8]to get the crf_input that is of shape [1, 6, 768]. 

That is how I think it should work. Any suggestion will be appreciated.

Comments:
This issue has been discussed at [#64](https://github.com/huggingface/pytorch-transformers/issues/64#issuecomment-443703063).

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Hi - I tried to implement it, you can have a look at my implementation here: https://github.com/chnsh/BERT-NER-CoNLL, hope that helps

@chnsh  I don't see any crf layer on your github repo.

@RoderickGu Were you able to implement bert-crf?
Pre-Training for Question Generation
Author: saichandrapandraju
Pull request: no

Hi,
How to pre-train any of language generation models (T5, BART or GPT ) for Question Generation task where I had passage, question and answer?


Comments:
I guess @patil-suraj is the expert in this, check out his [repo](https://github.com/patil-suraj/question_generation) explaining all the details.

ok..
@patil-suraj , plz revert on [this](https://github.com/patil-suraj/question_generation/issues/69) issue I raised in your repo
[examples/s2s] add test set predictions
Author: patil-suraj
Pull request: yes

# What does this PR do?

This PR adds the `do_predict` option to the `run_seq2seq.py` script for test set predictions.

Fixes #10032

cc. @stas00

Comments:
> I propose that the best approach would be to finish everything that is planned and then we will run tests side by side and note any small discrepancies if any and fix them in one go? Does that work?

Yes, this was the last major missing piece from this script. Now I'm going to start running both scripts side by side (manually converting the old datasets to new datasets format) and note the discrepancies, I'll also wait for your tests.

> I'm waiting for the datasets hub to port the datasets to be able to compare the old and the new.

Let's not wait for the hub, for now, we could just manually convert the datasets for tests and later upload them to the hub once it's ready. 

> > I propose that the best approach would be to finish everything that is planned and then we will run tests side by side and note any small discrepancies if any and fix them in one go? Does that work?
> 
> Yes, this was the last major missing piece from this script. Now I'm going to start running both scripts side by side (manually converting the old datasets to new datasets format) and note the discrepancies, I'll also wait for your tests.

That works.

> > I'm waiting for the datasets hub to port the datasets to be able to compare the old and the new.
> 
> Let's not wait for the hub, for now, we could just manually convert the datasets for tests and later upload them to the hub once it's ready.

Sure - I already wrote the code for wmt en-ro https://github.com/huggingface/transformers/issues/10044#issuecomment-774413928 need to adapt to others.

Changed `eval_beams` to `num_beams`. Hopefully final name change. Merging!
Logging propagation
Author: LysandreJik
Pull request: yes

This PR enables logs propagation by default with transformers' logging system, in a similar fashion to https://github.com/huggingface/datasets/pull/1845.

Unlike `datasets` however, we will not remove the default handler in transformers' logging system: this handler is heavily used in all examples, and removing the default handler would prevent the formatting from being correctly applied to the examples.
Since this is the best practice shown in examples, I believe this change would be breaking for users that have copy/pasted this change across their codebases, and this change would therefore be breaking for these users.

Furthermore, any user that does not want the default handler may use the `disable_default_handler` method in order to disable that behavior. These two methods are added to the documentation in this PR.

cc @lhoestq @sgugger @patrickvonplaten 

Comments:
Fix some edge cases in report_to and add deprecation warnings
Author: sgugger
Pull request: yes

# What does this PR do?

This PR adds two new values for the `report_to` TrainingArguments:
- "all" for all integrations installed
- "none" for none (necessary when using in the CLI and we can't pass an empty list)

It also starts warning the user (with an info to not be too spammy) of the upcoming change of default in v5.

Comments:
Add head_mask and decoder_head_mask to TF LED
Author: stancld
Pull request: yes

This PR implements `head_mask` and `decoder_head_mask` for TF LED (and Longformer as there's a copy dependency) and it is the follow-up to the open issue #9814. 

**Motivation:** This PR is a part of an endeavour to enable the usage of `head_mask` and `decoder_head_mask` for all encoder-decoder transformers following the recent work on BART-like models (#9639).

<hr>

Fixes: https://github.com/huggingface/transformers/issues/9814

Reviewers: @jplu @patrickvonplaten @LysandreJik @sgugger 

Comments:
Replace strided slice with tf.expand_dims
Author: jplu
Pull request: yes

# What does this PR do?

This PR aims to replace the strided slice notation by its TF operator counterpart. As proposed by @mfuntowicz https://github.com/huggingface/transformers/pull/9890#discussion_r571939682

Comments:
> When two new dimensions are need, can we use a different operator to make the code more readable? Suggested tf.reshape but there might be something else available?

Yes, this is doable with `tf.reshape`.

All the slow tests of the concerned models are ok!
Fix TFConvBertModelIntegrationTest::test_inference_masked_lm Test
Author: abhishekkrthakur
Pull request: yes


Comments:
@abhishekkrthakur - this doesn't look good to me. 

Just changing the hardcoded integration test to values that make the test pass does not seem like the way to go here. The PyTorch integration test: https://github.com/huggingface/transformers/blob/7c7962ba891864f9770b9e9424f87d158b839a59/tests/test_modeling_convbert.py#L430 still has the old values and passes, which to me is an indicator that the TF implementation or the PyTorch implementation is not correct.

Also, it would be great if we could not merge PRs that have no description and that neither @sgugger, @LysandreJik or I approved.
Wrong offsets mapping in XLMRobertaTokenizerFast
Author: nikitakit
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.2.2
- Platform: Linux-4.15.0-124-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.6.0 (False)
- Tensorflow version (GPU?): 2.3.1 (False)
- Using GPU in script?: False
- Using distributed or parallel set-up in script?: False

### Who can help

@mfuntowicz @thomwolf 

## Information

Model I am using (Bert, XLNet ...): XLMRobertaTokenizerFast

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

```python
from transformers import XLMRobertaTokenizerFast
tokenizer = XLMRobertaTokenizerFast.from_pretrained('xlm-roberta-large')
text = "？……"
tokenized = tokenizer(text, return_offsets_mapping=True)
print('Text:', text)
print('Tokens:', tokenizer.convert_ids_to_tokens(tokenized.input_ids))
print('Mapped to:', [text[start:end] for start, end in tokenized.offset_mapping])
```

Observed behavior:

```
Text: ？……
Tokens: ['
', '▁?', '......', '
']
Mapped to: ['', '？', '？', '']
```

Expected behavior:

```
Text: ？……
Tokens: ['
', '▁?', '......', '
']
Mapped to: ['', '？', '……', '']
```

## Expected behavior

I'm using XLM-R for Chinese text, and I would expect offset mappings to work correctly even in the presence of various Unicode punctuation symbols. It looks like XLM-R tokenizes "？……" as two tokens  ('▁?' and '......'), which I would expect to map back to the appropriate locations in the input. Instead, the offset mapping from these tokens is identical.

This example is an ending of an actual sentence in the Chinese Treebank -- I removed the sentence itself because it doesn't matter for reproducing the bug.


Comments:
The wrong alignments are caused by the `Precompiled` normalizer in `tokenizers`. This will be fixed in version 0.10.1 of the library.

This is fixed for any version of `transformers>=4.3.0`
Remove speed metrics from default compute objective [WIP]
Author: shiva-z
Pull request: yes

# What does this PR do?

This PR removes speed metrics (e.g. `eval_runtime`) from the default compute objective (`default_compute_objective`). `default_compute_objective` is used when no `compute_objective` is passed to `Trainer.hyperparameter_search`. `Trainer` adds speed metrics such as `eval_runtime` and `eval_samples_per_second` to the metrics and `default_compute_objective` returns the sum of metrics as the objective so these speed metrics will be included in the objective. 

I still need to add unit test for `default_compute_objective` to avoid having such metrics in the objective in the future. 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@sgugger 


Comments:
Oh boy, that is rather bad! Thanks a lot for fixing this!
Did you want to add the test in this PR?

> Oh boy, that is rather bad! Thanks a lot for fixing this!
> Did you want to add the test in this PR?

I can also create a follow up PR if you want to merge this asap. I can't implement the test cases right away. Maybe in ~4 days. @sgugger

In that case maybe a follow-up PR, this fix is needed badly so I will merge. Thanks again!
How to run distributed training on multiple machines?
Author: allanj
Pull request: no

## Environment info


- `transformers` version: 4.3.0
- Platform: PyTorch
- Python version: 3.7
- PyTorch version (GPU?): 1.7.1
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Yes

##Who can help: 
Examples:
- maintained examples (not research project or legacy): @sgugger, @patil-suraj

## Information

Model I am using (Bert, XLNet ...): Roberta

## To reproduce
The script I'm working with is https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_mlm.py
I know that we can run the distributed training on multiple GPUs in a single machine by
`python -m torch.distributed.launch --nproc_per_node=8 run_mlm.py --sharded_dpp`

But what if I can multiple machines with multiple GPUs, let's say I have two machines and each is with 8 GPUs, what is the expected command to run on these 16 GPUs?


Comments:
I'm only aware that the PyTorch official documentation is using the RPC: https://pytorch.org/tutorials/intermediate/dist_pipeline_parallel_tutorial.html

This is more of a question for the PyTorch GitHub then ours since this is a question about to use `torch.distributed.launch`. That's why I'll close the issue. Still, I can share the command I run on my side:
```
python -m torch.distributed.launch --nproc_per_node 8 \
    --nnodes 2 \
    --node_rank rank_of_your_machine \
    --master_addr main_machine_ip \
    --master_port open_port_on_main_machine \
    run_mlm.py \
    --sharded_ddp \
    --all_other_args_to_script
```
where `rank_of_your_machine` should be 0 for the main machine and 1 for the other one, `main_machine_ip` the IP of the machine of rank 0 and `open_port_on_main_machine` the port to use to communicate between the two machines.

Thanks, this is really helpful
[RAG] fix generate
Author: patil-suraj
Pull request: yes

# What does this PR do?

#9984 introduced a new `encoder_no_repeat_ngram_size`  `generate` param, but it was missing from `RagTokenForGeneration.generate`,  it's a required argument for `_get_logits_processor` which is called inside `RagTokenForGeneration.generate`.

This PR adds the argument to `RagTokenForGeneration.generate` and passes it to `_get_logits_processor`.

Comments:
Wow this is a huge bug actually! Thanks a lot for fixing it @patil-suraj! 

@LysandreJik @sgugger - Sadly Circle CI is skipping all RAG tests at the moment -> therefore we should first fix the faiss import (#10103), then rebase this PR to see that everything is correctly solved, merge it and then do a patch
[DeepSpeed docs] new information
Author: stas00
Pull request: yes
Labels: name: DeepSpeed

As I bombarded the DeepSpeed with multiple Issues, as the answers are starting to percolate back, so I will gather them in this PR. So I will let it sit for a while collecting updates, unless users will need those answers sooner.

* [x] how to run DeepSpeed with a 1 gpu which is not GPU 0 (`CUDA_VISIBLE_DEVICES` can't be used)
* [x] add a newly published paper to resources 
* [x] various small additions/improvements 

Comments: