noamaanMulla-03/transformers-issues huggingface dataset:
unshared Albert
Author: guowenying111
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
<!-- Description of your issue -->
How can I remove ALBERT's parameter sharing during fine-tuning? (just use the albert as bert) 
<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:

Comments:
Hello! The layers are shared in ALBERT because even though there are `config.num_hidden_layers=n`, there's usually `config.num_hidden_groups = 1`. Change that value to `n` to have `n` different layers.

Thanks for your reply, I have another question about that
How can I load the AlbertModel parameter(in transformer block) to unshared Albert, When I load just 
![image](https://user-images.githubusercontent.com/47688495/96669644-388e4b80-1390-11eb-8766-0d06941e7ea0.png)
###_IncompatibleKeys(missing_keys=['embeddings.position_ids'], unexpected_keys=[])###
whether is it wrong? can you tell me how I can load parameters correctly?
looking forward to your reply

Could you show me the full code used that generated this error? Or can you give an example so I can reproduce it?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
dropping "," in date because of Tokenization
Author: vkaul11
Pull request: no
Labels: name: wontfix

I had my model where "May 7, 2020" is split like this
`['▁may', b'\xe2\x96\x818', ',', '▁2017']`
 I saw that this is a problem here. Any reason why this code was thrown in for bytes string insertion where we have a comma in the token of len > 1? 
```
for piece in pieces:
            if len(piece) > 1 and piece[-1] == str(",") and piece[-2].isdigit():
                cur_pieces = self.sp_model.EncodeAsPieces(piece[:-1].replace(SPIECE_UNDERLINE, ""))
                if piece[0] != SPIECE_UNDERLINE and cur_pieces[0][0] == SPIECE_UNDERLINE:
                    if len(cur_pieces[0]) == 1:
                        cur_pieces = cur_pieces[1:]
                    else:
                        cur_pieces[0] = cur_pieces[0][1:]
                cur_pieces.append(piece[-1])
                new_pieces.extend(cur_pieces)
            else:
                new_pieces.append(piece)

        return new_pieces
```

Comments:
Hi, I'm sorry but I really don't understand what the issue is. Could you clarify?

I think this comes from the original Albert tokenization, maybe you want to ask on google repository about this?
https://github.com/google-research/albert/blob/master/tokenization.py#L67

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Code bug in tokenization_utils.py?
Author: liwei-cpp
Pull request: no
Labels: name: wontfix

In the function split_on_tokens in tokenization_utils.py, it contains the following logic:

if not text.strip():
    return []
if not tok_list:
    return self._tokenize(text)

So if the text contains just white space, '[]' will be returned. However, if the text contains white space and visible characters, the white space will be used in tokenization. For example, for the following code:

from transformers import GPT2Tokenizer
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
print(tokenizer("\nNorth")['input_ids'])  # output [198, 14157], since 198 <-> \n and 14157 <-> North
print(tokenizer("\n")['input_ids'])            # output [], even if 198 <-> \n

Is this behavior that we expected?

Comments:
Pinging @mfuntowicz, @n1t0 for their opinions

I think we should delegate to the underlying tokenization algorithm and never strip like we do here. This what is done in `tokenizers` and thus one of the source of discrepancies between fast and slow tokenizers.

For most tokenizers this is not a problem since these white space get removed later anyway, but for some others (like gpt2) it is.

Note: it is impossible to build tokenizers relying on the formatting with such rules. For example it would be impossible to tokenize some Python.

(Cc @thomwolf)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Make sure to use return dict for the encoder call inside RagTokenForGeneration
Author: dblakely
Pull request: yes

## What does this PR do?

At some point, `return_dict` was set to be `False` by default inside BART. However, this created a type error inside `RagTokenForGeneration`, which was still written with the expectation that `return_dict=True` by default. This PR simply adds `return_dict=True` to the call to the BART encoder inside the `RagTokenForGeneration` code.

## Tests
I did not create new tests because this change is very minor. I did run all the existing tests and they pass.

## Who can review?

Anyone can review, but I tagged these two because this involves RAG and BART:

@patrickvonplaten @lhoestq


Comments:
Thanks!
[docs] Fix TF base model examples: outputs.last_hidden_states -> state
Author: ck37
Pull request: yes

# What does this PR do?

Fixes a typo in the examples of Tensorflow-based base models, in which the returned last_hidden_state attribute of model output is incorrectly listed as "last_hidden_states".

Fixes #9376 

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?

@julien-c @patrickvonplaten


Comments:
[docs] TFRobertaModel example: last_hidden_states -> last_hidden_state
Author: ck37
Pull request: no

This is a documentation error on the currently deployed version of https://huggingface.co/transformers/

### Who can help

 examples/distillation: @VictorSanh
 documentation: @sgugger

## Information

Model I am using: TFRoberta

The problem arises when using:
* [x] the official example scripts

## To reproduce

Steps to reproduce the behavior:

1. View the code example at https://huggingface.co/transformers/model_doc/roberta.html#tfrobertamodel
2. `last_hidden_states = outputs.last_hidden_states` should be `last_hidden_states = outputs.last_hidden_state`

The current incorrect spelling will yield an error. I apologize that I was not able to find that line in the repo, otherwise I would submit a PR.


Comments:
looks like this is what you're looking for: https://github.com/huggingface/transformers/blob/ae333d04b29a25be1a70eaccd6260c294c243c5b/src/transformers/file_utils.py#L842-L855

Hey @ck37,

Thanks for your issue! Yes, this typo should be corrected -> it would be great if you could open a PR :-)
gpt2 and t5 model parallelism with tests
Author: alexorona
Pull request: yes
Labels: name: Model Parallel

# Model Parallelism for GPT2 and T5
Note: version compatible with v4

Adds two new methods to t5 and gpt2 models to enable you to generate and fine-tune models using model parallelism. This feature is most applicable for `gpt2-large` and `gpt2-xl`. Minor modifications are made to the `TrainingArguments` and `Trainer` classes to avoid conflicting data parallelism behavior and related batch_size increases which would negate model parallelism. Note that nearly 64GB of GPU (4 Tesla v100s)  are needed to fine-tune `gpt2-xl` @ 1024 tokens.

It is critically important to provide users the ability to specify where to put the blocks of a model because the GPU sizes and numbers are likely to be very diverse. This is done with a dictionary called `device_map`. I am planning on providing some examples and guidelines for the p3, p2 and g3 AWS instances. 

Model parallelism has to be baked into the model class itself. Currently working on the T5 model. From my calculations the 11B model cannot fit on the largest p3 instance that I have access to (8 Tesla v100 GPUs). The 3B model can.

The methods are:
- `parallelize`, which will distribute the attention blocks of the model across several devices according to a device map
- `deparallelize`, which will move the model back to cpu

# Example

```
model = GPT2LMHeadModel.from_pretrained('gpt2-xl')
device_map = {0: [0, 1, 2, 3, 4, 5, 6, 7, 8],
                              1: [9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21],
                              2: [22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34],
                              3: [35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47]}
model.parallelize(device_map) # Distributes the model's attention blocks across several devices
model.deparallelize() # Puts the model back on cpu and calls torch.cuda.empty_cache() to liberate GPU memory
```

## Reviewers
@LysandreJik 

Comments:
Use of global attention of Longformer when generating
Author: alexyalunin
Pull request: no
Labels: name: wontfix

I'm training Longformer2Roberta, the encoder part of this Seq2Seq model is Longformer. The one feature Longformer brings is global attention, I found the use of it during training, but it is never used at the point of generation. I guess it should be used somewhere here https://github.com/huggingface/transformers/blob/03e46c1de3864b8464a1b40d2a414b35f6b7f0df/src/transformers/generation_utils.py#L402. 

I guess @patrickvonplaten is working on these models.

Comments:
Hey @alexyalunin - you are completely right! I'm working on a bigger generation refactor at the moment to better handle cases like this. 

For this case, I would propose to modify the code yourself and do a little hack, something along the lines 

```python 
if "global_attention_mask" in model_kwargs:
    encoder_outputs: ModelOutput = encoder(input_ids, attention_mask=attention_mask, global_attention_mask=global_attention_mask, return_dict=True)
else:
    encoder_outputs: ModelOutput = encoder(input_ids, attention_mask=attention_mask, return_dict=True) 
```

I don't want to merge this into master because it's quite hacky and the `generate()` function needs a refactor before we start adding more and more hacks. Hope this works for you for now.

This should be solved soon by the new generate() design: #6949 in like ~1,2 weeks

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Longformer2Roberta: global_attention_mask is never used
Author: alexyalunin
Pull request: no
Labels: name: wontfix

I was following the Longformer2Roberta tutorial https://github.com/huggingface/transformers/blob/master/model_cards/patrickvonplaten/longformer2roberta-cnn_dailymail-fp16/README.md, it seems like 'global_attention_mask' is never used because this column is removed after https://github.com/huggingface/transformers/blob/9bdce3a4f91c6d53873582b0210e61c92bba8fd3/src/transformers/trainer.py#L301
so you have to add this column to the signature (whatever it is)
https://github.com/huggingface/transformers/blob/9bdce3a4f91c6d53873582b0210e61c92bba8fd3/src/transformers/trainer.py#L324
or set `remove_unused_columns=False` in TrainingArguments 

@patrickvonplaten


Comments:
This should be solved soon by the new `generate()` design: https://github.com/huggingface/transformers/pull/6949

Probably still takes ~1,2 weeks until merge

@patrickvonplaten are you sure you mentioned the correct issue? The issue about the `generate()` function was this one #7489 
In the current issue I mention that `global_attention_mask` is never used during training. 

You are 100% correct @alexyalunin :D - sorry my bad! Thanks for linking the correct issue!

Regarding this issue, I will add more scripts showing how Longformer2Roberta can be trained. I'll pay special attention to your issue here then :-)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
T5-large on multiple gpus.
Author: Palipoor
Pull request: no
Labels: name: wontfix

Hi! I'm trying to fine-tune a T5-large on multiple gpus, so I basically use `torch.nn.DataParallel`, but when I get the output of the model which contains the loss and I do `loss.mean().backward()` I run into `cuda out of memory` as I think the loss is just on the first gpu and that's taken. What should I do?

Comments:
For OOM errors, I guess you need to reduce batch_size or parallelize over more GPUs.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
T5 on multiple datasets
Author: rabeehk
Pull request: no
Labels: name: wontfix

Hi Everyone,
Is there an example showing how to run T5 on multiple datasets? Greatly appreciated.
thanks.
Best
Rabeeh

Comments:
pinging the T5 master @patrickvonplaten 

Hey @rabeehk - not sure about that. You can check the T5 notebooks we provide or `https://discuss.huggingface.co/`.

Hi
I could not find notebooks in the link you said.
I am looking for a way to train multiple tasks at once in T5, similar to
this script:
https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/models/hf_model.py
thanks for your help.
Best
Rabeeh

On Thu, Oct 22, 2020 at 10:51 PM Patrick von Platen <
notifications@github.com> wrote:

> Hey @rabeehk <https://github.com/rabeehk> - not sure about that. You can
> check the T5 notebooks we provide or https://discuss.huggingface.co/.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/7956#issuecomment-714754836>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABP4ZCFCMFQC5USZT3FRAFLSMCLNZANCNFSM4S2MCL7Q>
> .
>


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
T5 on multiple tasks
Author: yes1234man
Pull request: no
Labels: name: wontfix

Dear Huggingface team,
I am looking for a code to pretrain T5 on multiple tasks, the best I could find is the code released with wrapper around huggingface T5 model in the original author's repo:
https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/models/hf_model.py

On line 308-310, they create tensorflow datasets I think, and pass it to huggingface model, I was wondering if I could ask for your help, if one wants to add data parallelism to this code to make it efficient in pytorch, could you help me how I can do it in pytorch? thanks a lot, and I appreciate your help.


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
cannot load customized tokenizer with modified vocabulary
Author: CharizardAcademy
Pull request: no
Labels: name: wontfix

I have customized a tokenizer saved as tokenization_new.py and modified the vocab.txt from s3 server. I tried
`from transformers import NewBertTokenizer`
`tokenizer = NewBertTokenizer.from_pretrained('bert-base-uncased', do_lower_case=True)`
where I modified `PRETRAINED_VOCAB_FILES_MAP = {"vocab_file":{"bert-base-uncased": /vocab/vocab.txt},}`
where `/vocab/` is a directory parallel to tokenization_new.py that contains my customized vocabulary. However, I got an error raised as 
`Model name 'bert-base-uncased' was not found in tokenizers model name list (bert-base-uncased). We assumed 'bert-base-uncased' was a path, a model identifier, or url to a directory containing vocabulary files named ['vocab.txt'] but couldn't find such vocabulary files at this path or url.`
What should I do to use my own customized tokenizer? 
Thanks for the help!

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Add PRADO model
Author: ierezell
Pull request: no
Labels: name: wontfix

name: New model

# 🌟 New model addition

## Model description
PRADO is a model made by google, performing as bert with 100x less parameters

[link to the paper](https://www.aclweb.org/anthology/D19-1506.pdf)
[git to the model code](https://github.com/tensorflow/models/tree/master/research/sequence_projection)

## Open source status

* [X] the model implementation is available: (give details)
* [ ] the model weights are available: (give details)
* [X] who are the authors: (mention them, if possible by @gh-username)
Prabhu Kaliamoorthi / Sujith Ravi / Zornitsa Kozareva


Comments:
Yeah, This is a good model to go with if the the text sequence is too long.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Bump notebook from 6.1.4 to 6.1.5 in /examples/research_projects/lxmert
Author: dependabot[bot]
Pull request: yes
Labels: name: dependencies

Bumps [notebook](https://github.com/jupyter/jupyterhub) from 6.1.4 to 6.1.5.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a href="https://github.com/jupyter/jupyterhub/commits">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=notebook&package-manager=pip&previous-version=6.1.4&new-version=6.1.5)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/huggingface/transformers/network/alerts).

</details>

Comments:
Thanks dependabot!
[trainer] parametrize default output_dir
Author: stas00
Pull request: yes

This PR:

* fixes trainer to have the logger agree with the actual default `output_dir`, by setting it in one place and passing it as an argument to both places. The current logger falsely informs the user that `output_dir` is the current path, while using `tmp_trainer` as the path.

@patrickvonplaten, @sgugger


Comments:
Fix TF DPR
Author: jplu
Pull request: yes

# What does this PR do?

This PR rework the DPR architecture for its TF version. This rework allows DPR models to be saved as proper saved model.


Comments:
@patrickvonplaten , I reworked a bit the approach. Now `TFDPREncoder` and `TFDPRSpanPredictor` are still models and keep their features from `TFPreTrainedModel` while all the DPR models benefit of the serving.

All the slow/quick tests are still passing.

All @sgugger comments have been addressed.
Possible bug in `train_batch_size`
Author: EmilyAlsentzer
Pull request: no

## Environment info
- `transformers` version: 4.1.1
- Platform: Linux-4.4.0-62-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.7.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help
 Trainer: @sgugger 

## Information

Model I am using (Bert, XLNet ...):

BERT 

The problem arises when using:
* [X ] my own modified scripts: (give details below)

I'm running a model on a toy dataset with only 2 examples and a batch size of 2. In trainer, `num_examples` is 2, but `total_train_batch_size` is 12 even though I do not have the `model_parallel` flag set to `True` (Note I do have 6 GPUs available on the machine). This doesn't seem to impact my code because `train_dataset_is_sized=True`, but it seems strange. 

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ X] my own task or dataset: (give details below)

toy classification jsonl dataset with 2 examples

## To reproduce

I think that [this line](https://github.com/huggingface/transformers/blob/64103fb6beac8cc865945d3956266fd80b44f18f/src/transformers/training_args.py#L454) has an unnecessary `not`. Should this be `if self.model_parallel` instead of `if not self.model_parallel`? Thanks!


Comments:
Think @sgugger can best answer here when he's back from holiday :-) 

You misunderstand the flag `model_parallel`, it's not there to enable the use of several GPUs as this is done automatically by the `Trainer` (you have to set `CUDA_VISIBLE_DEVICES` to just one GPU if you don't want the Trainer to use them all). That flag is there to split the model layers on the various GPUs available (only available for a few models).

Got it, I didn't realize that the Trainer automatically uses multiple GPUs if visible. Thanks!