noamaanMulla-03/transformers-issues huggingface dataset:
How can I tweak the `Longformer` code to control the input of a `Longformer`'s layer?
Author: h56cho
Pull request: no

Hello,
I have asked a similar question on the HuggingFace forum, but I didn't get a clear answer that I was hoping for.
I tried the following to control the input of a `Longformer` layer:
`best_model_longformer.longformer.encoder.layer[layer_index](my_input_hidden_vector)` 

which of course, does not work.


How can I tweak the `Longformer` (or `BERT`) code to control the input of a `Longformer`'s layer?
Would this do the trick?:
```Python

  self_attention_outputs = Longformer_model.LongformerLayer.forward(hidden_states,
        attention_mask=None,
        output_attentions=False)
 Longformer_model.LongformerLayer.ff_chunk(self, self_attention_outputs)
```
But in this case, I don't know how to properly access the `LongformerLayer` class. I don't know if `Longformer_model.LongformerLayer` is the proper way.

I am not a programmer and I really do need more help on this. I know this can be a bit cumbersome to answer, but could you please help me on this? Your help is much appreciated.

Thank you,

Comments:
The answer you got on the forum is pretty much the only one we have: copy paste the model code and customize it to your needs. If you're not a programmer, then you will need to learn a bit of Python/PyTorch to do this, but apart from making sure each model file contains the full code of each model, there is little more we can do to help you. There is an API to use the models in the common cases, users are expected to make the customizations they want if that does not suit their needs.

Hello, thank you for your reply.
Sorry if my question sounds a bit silly, but I just have a question:

When I do `my_Longformer_model.encoder.layer[layer_index].forward(hidden_output,my_attention_mask,output_attention=False)`, this error is generated:
```python

  File "/Users/hyunjindominiquecho/opt/anaconda3/lib/python3.7/site-packages/transformers/modeling_longformer.py", line 384, in _sliding_chunks_query_key_matmul
    batch_size, seq_len, num_heads, head_dim = query.size()

ValueError: too many values to unpack (expected 4)
```

Here, `my_attention_mask` is the same attention mask that I would specify under the regular
```python
longformer_output= my_longformer_model(input_ids=input_ids,....,attention_mask=my_attention_mask)
```

why exactly is the above error generated, and how can I remedy it?


Thank you,
Create README.md
Author: sarahlintang
Pull request: yes
Labels: name: model card

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), 
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? 


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao 
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao 
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

Comments:
Thank you! You can add more metadata and/or links to eval results if necessary.
[stas/sam] Newsroom dataset wierdness
Author: sshleifer
Pull request: no

#### get data
```bash
cd examples/seq2seq/
curl -L -o stas_data.tgz https://www.dropbox.com/sh/ctpx2pflb9nmt0n/AABRTDak-W06RD8KxuCOUdXla\?dl\=0 && unzip stas_data.tgz
tar -xzvf newsroom-test.tgz
```

```python
from utils import Seq2SeqDataset
tok = PegasusTokenizer.from_pretrained('google/pegasus-newsroom')
ds = Seq2SeqDataset(tok, 'newsroom/data', tok.model_max_length, tok.model_max_length, type_path='test')
ds[659]['tgt_texts']
# "Insomniac's Pasquale Rotella has gone from throwing illegal raves in warehouses to throwing the nation's most iconic dance music festival in Las Vegas' Electric Daisy Carnival. "
ds[660]
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-17-7fbeab38f815> in <module>
----> 1 ds[660]

~/transformers_fork/examples/seq2seq/utils.py in __getitem__(self, index)
    248         tgt_line = linecache.getline(str(self.tgt_file), index).rstrip("\n")
    249         assert source_line, f"empty source line for index {index}"
--> 250         assert tgt_line, f"empty tgt line for index {index}"
    251         return {"tgt_texts": tgt_line, "src_texts": source_line, "id": index - 1}
    252 

AssertionError: empty tgt line for index 661

```

Clue:

In vim, the "Pasquale Rotella" line is 654 (off by 7/possible other bug), but it is 659/660 in the ds.
similarly, `linecache` disagrees with `wc -l` about file lengths.

```python
import linecache
src_lns = linecache.getlines(str(ds.src_file))
tgt_lns = linecache.getlines(str(ds.tgt_file))
assert len(src_lns) == len(tgt_lns),f'{ len(src_lns)} != {len(tgt_lns)}'
AssertionError: 108717 != 110412
```



Comments:
Here is part of the problem

![image](https://user-images.githubusercontent.com/6045025/96030008-6e2fb380-0e29-11eb-96c6-994955fa588d.png)

all one line (221  in vim)


Oh, I know - it's \cM characters. Let me take care of it.

```
Google is still the best company to work for, according to Fortune
.^M<n>^M<n>The Mountain View-based tech giant earned the top 
^^^^^^^^^^^^^
```

Easiest way to clarify will be to say how I fixed (in vim)

(1) `%s/^M//g`  # This is not ctrl-m it must be typed by following these [instructions](https://stackoverflow.com/questions/5843495/what-does-m-character-mean-in-vim#:~:text=Windows%20uses%20a%20combination%20of,letter%20in%20the%20English%20alphabet).&text=Where%20%5EM%20is%20entered%20by,m%20%2C%20and%20then%20releasing%20Ctrl%20.)
 
(2)`%s/<n><n>/<n>/g` (probably not necesarry, but did it anyway).


```
dos2unix filename
```

I will fix that in the build script

```
            src = re.sub(r'[\r\n]+', '<n>', src)
            tgt = re.sub(r'[\r\n]+', '<n>', tgt)
```
fix examples/rag imports, tests
Author: sshleifer
Pull request: yes

Before
```bash
pytest examples/rag
```
fails with
```
================================================================ ERRORS =================================================================
______________________________________ ERROR collecting examples/rag/test_distributed_retriever.py ______________________________________
ImportError while importing test module '/Users/shleifer/transformers_fork/examples/rag/test_distributed_retriever.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
examples/rag/test_distributed_retriever.py:26: in <module>
    from examples.rag.distributed_retriever import RagPyTorchDistributedRetriever  # noqa: E402 # isort:skip
E   ModuleNotFoundError: No module named 'examples.rag'
```

After, the same command passes.

The fix was to change
`from examples.rag.file_name -> from filename` 
and add the same `sys.path` magic that `examples/seq2seq/` uses.

### TODO
- test the scripts in `examples/rag/README.md` after this change


Comments:
@stas00 does this look reasonable? Am I missing anything you did for `examples/seq2seq`?

yes, except you shouldn't need 
```
sys.path.append(os.path.join(os.getcwd()))  # noqa: E402 # noqa: E402 # isort:skip
```
that's what `__init__.py` already did for you.

and once removed, all those subsequent import `# noqa:` comments can be removed too. (except PL import)

besides using `cwd` is a bad idea - who knows where the script is invoked from and not just from the same dir as the script itself. Use `__file__` instead, which is deterministic. but it's not needed here.


Scripts work, will merge on CI success.

btw, you can also remove most of the `# noqa: E402 # isort:skipq` as they are no longer needed.
tokenizers dependency warning: `transformers 3.3.1 has requirement tokenizers==0.8.1.rc2, but you'll have tokenizers 0.9.0`
Author: morganmcg1
Pull request: no

## Environment info
- `transformers` version: 3.3.1
- Platform: Darwin-19.6.0-x86_64-i386-64bit
- Python version: 3.7.5
- PyTorch version (GPU?): 1.4.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help

 tokenizers: @mfuntowicz

## Information

Hey all, it'll probably be fixed soon but when updating to `tokenziers` 0.9.0 with `pip install tokenizers --upgrade` I get:

`ERROR: transformers 3.3.1 has requirement tokenizers==0.8.1.rc2, but you'll have tokenizers 0.9.0 which is incompatible.`

## To reproduce

Steps to reproduce the behavior:

1. pip Install transformers --upgrade
2. pip install tokenizers --upgrade


## Expected behavior

Expected compatibility between transformers 3.3.1 and tokenizers 0.9.0


Comments:
Also stumbled upon this package. I was surprised we were forced to add the RC version of tokenizers. I would expect this to be pinned to "0.8.1" or "0.9.1"

https://github.com/huggingface/transformers/pull/7794 to update to release version 0.9.1

Hi, we have a strict requirement on `tokenizers==0.8.1rc2`. We're updating it in https://github.com/huggingface/transformers/pull/7659 but the current `transformers` `master` branch will stay pinned until that PR is merged.

Both libraries evolve quickly and generally evolve together, so having a strict `==` dependency is necessary until tokenizers version 1.0.0 is released.
Updated Tokenizer to 0.9.1 from prerelease version
Author: dciborow
Pull request: yes

Use latest stable version instead of RC prerelease.

# What does this PR do?

Upgrades Tokenizer to update release version, from pre-release version.

<!-- Remove if not applicable -->

Fixes # (issue)
#7794 

## Before submitting
- [ x] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [ x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), 
      Pull Request section?
- [x ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? 


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik 
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Comments:
Why not `>= 0.9.1` or `>=0.8.1.rc2`?

Hi, we have a strict requirement on `tokenizers==0.8.1rc2`. We're updating it in https://github.com/huggingface/transformers/pull/7659 but the current `transformers` `master` branch will stay pinned until that PR is merged.

Both libraries evolve quickly and generally evolve together, so having a strict `==` dependency is necessary until tokenizers version 1.0.0 is released.
Upgrading in pipelines TFAutoModelWithLMHead to new Causal/Masked/Seq2Seq LM classes
Author: Narsil
Pull request: yes

# What does this PR do?

Updates code to remove deprecation warning. 
No tests as it simply removes current warnings from tests.



## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), 
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? 


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

@mfuntowicz

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao 
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao 
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

Comments:
Fixing beam search output shapes
Author: nicola-decao
Pull request: yes

# What does this PR do?

`generate` in `generation_utils.py` returns a list of size `batch_size * num_beams` where it is much more practical if it returns a list of lists. The first list of size `batch_size` and all internal lists of size `num_beams`. In this way, one can generate using mini-batches of variable size and no do reshapes each time.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), 
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? 


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

@patrickvonplaten , @TevenLeScao 

Comments:
error when using the forward() function of the LongformerLayer class from the LongformerForMultipleChoice model
Author: h56cho
Pull request: no

Hello, 

Sorry if my question sounds a bit silly, but I just have a question:
I am trying to feed in the hidden output of the embedding layer of the `LongformerForMultipleChoice` model directly into the m-th layer of the same model. Each of my multiple-choice question that has 4 options.

When I do:
```Python
my_Longformer_multiple_choice_model.encoder.layer[layer_index].forward(hidden_output, 
                                                 attention_mask=my_attention_mask,output_attention=False)
```
, an this error is generated:
```Python
File "/Users/hyunjindominiquecho/opt/anaconda3/lib/python3.7/site-packages/transformers/modeling_longformer.py", line 384, in _sliding_chunks_query_key_matmul
    batch_size, seq_len, num_heads, head_dim = query.size()

ValueError: too many values to unpack (expected 4)
```
Here, `my_attention_mask` is the same attention mask that I would specify under the regular `LongformerForMultipleChoice` command. `my_attention_mask` was generated by:
```Python
# I am using the LongformerForMultipleChoice model, where each multiple choice question has 4 options.
encoded_dict = longformer_tokenizer(question_list, option_list,
                             return_tensors = 'pt',
                             padding ='max_length')
my_attention_mask = {k: v.unsqueeze(0) for k,v in encoded_dict.items()}['attention_mask']
my_attention_mask 
>>> tensor([[[1, 1, 1,  ..., 0, 0, 0],
         [1, 1, 1,  ..., 0, 0, 0],
         [1, 1, 1,  ..., 0, 0, 0],
         [1, 1, 1,  ..., 0, 0, 0]]])

# I can use this my_attention_mask in the regular command without an error, as below:
longformer_output= my_Longformer_multiple_choice_model(input_ids=input_ids,....,attention_mask=my_attention_mask)
```
Also, the `hidden_output` in my command was generated by the following:
```Python
encoded_dict = longformer_tokenizer(question_list, option_list,
                             return_tensors = 'pt',
                             padding ='max_length')
hidden_output = my_Longformer_multiple_choice_model(**{k: v.unsqueeze(0) for k,v in encoded_dict.items()}, labels = mc_labels)[2][0][:,:,:]
hidden_output.size()
>>> torch.Size([4, 4096, 768])
```

I am suspecting the value error is generated because the form of `my_attention_mask` is wrong. What should I pass for the `attention_mask` parameter in the command `my_Longformer_multiple_choice_model.encoder.layer[layer_index].forward(hidden_output, 
                                                 attention_mask,output_attention=False)`?

Thank you,
@LysandreJik 
@NielsRogge 
@sgugger 

Comments:
Add specific notebook ProgressCalback
Author: sgugger
Pull request: yes

# What does this PR do?

This PR introduces a new `NotebookProgressCallback` that is more suitable for trainings in notebook. There are two problems with the current use of tqdm:
- tqdm uses widgets, which disappear when you close and reopen your notebook, or download it from github. Instead of seeing 
  the full progress bar, a message "A Jupyter widget could not be displayed because the widget state could not be found. This 
  could happen if the kernel storing the widget is no longer available, or if the widget state was not saved in the notebook. You 
  may be able to create the widget by running the appropriate cells." appears
- tqdm creates a new widget each time you open a new progress bar, which, when closed, leaves a blank line there is absolutely 
  no way to remove (and I have tried!) This means we have one such blank line for every evaluation.

What's more, notebooks can properly render html code, so we can structure the output displayed during and at the end of training a bit better, using a table for instance.

This PR aims at tackling the issues above by:
- writing its own progress bar in pure HTML and using the `IPython.display` module to display and update it
- adding a table of results also in pure HTML to that progress bar.

It adds no dependency and just add a test (taken from tqdm.auto) to determine if the user is executing code in a notebook environment or not, and picking the best `ProgressCallback` accordingly.

It goes from the previous results:

![](https://i.ibb.co/W5zqv6n/old-progress.png)

To those new ones:

![](https://i.ibb.co/m8zpmmk/new-progress.png)

With a bit more work, it's also possible to add a graph of the losses/metrics that gets updated as the training progresses.


Comments: