noamaanMulla-03/transformers-issues huggingface dataset:
Getting Bert Embeddings in Batch
Author: ChawlaAvi
Pull request: no
Labels: name: wontfix

Hi all

I have a list of sentences(a batch during training) and for every word in each sentence, I need an aligned bert embedding which should be the mean of every word-piece that word was split into.

Right now, I am doing it sentence by sentence and obtain the aligned embedding for every word by reiterating over the sentence, tokenise the individual word, note the number of word-pieces it was split into and lookup into the Bert embedding matrix to average out those rows of the matrix. Following is the code to get the aligned embeddings:

`
  
       def get_bert_aligned_embeddings(self, last_hidden_states, tokens):     

            count = 0
            aligned_embeddings = []

            for i in tokens:

                 tokenisation_length = len(self.tokenizer.tokenize(i))

                 emb = torch.mean(last_hidden_states[count:count+tokenisation_length], axis = 0)

                 count += tokenisation_length

                 aligned_embeddings.append(emb)


            aligned_embeddings = torch.stack(aligned_embeddings)

            return aligned_embeddings
`

tokens is the list of words in a sentence, last_hidden_states are the embeddings I obtained from Bert. 

So, the above function runs for every sentence and every sentence is passed one by one to Bert Model. 

I want to know if there is any faster way of doing this? Can this entire process be done in batches? Any suggestions that could help me speed up this process would be great. Thanks

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
What's the most straightforward way to initialise BertForSequenceClassification for different token rather than [CLS]?
Author: amirj
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

## Details
BertForSequenceClassification uses [CLS] token's representation to feed a linear classifier. I want to leverage another token (say [X] in the input sequence) rather than [CLS]. What's the most straightforward way to implement that in Transformers?

**A link to original question on the forum/Stack Overflow**: https://stackoverflow.com/questions/64094098/how-to-initialize-bertforsequenceclassification-for-different-input-rather-than

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Longformer run error
Author: Yangxiaojun1230
Pull request: no

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
When I train on a classification model by Longformer
def forward(self,input):
        embding=input['enc']
        att_mask=input['mask']
        att_mask[:,[100,300,500,800,1200,]]=2
        labels=input['targets']
        print('jeff:',embding.device,att_mask.device,self.l1.device,embding.shape,att_mask.shape,self.maxlen)
        
        logit=self.l1(inputs_embeds=embding,attention_mask=att_mask)#[:2]
        return [logit,labels]

Meet error:
pytorch/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = 2, SrcDim = 2, IdxDim = -2, IndexIsMajor = true]: block: [205,0,0], thread: [95,0,0] Assertion `srcIndex < srcSelectDimSize` failed

max_num_extra_indices_per_batch = num_extra_indices_per_batch.max()
RuntimeError: cuda runtime error (710) : device-side assert triggered at /pytorch/aten/src/THC/THCReduceAll.cuh:327

I checked the length of the attention_mask is same as the config.max_len which is like [bs,max_len]
Do anyone meet the same issue?
<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:

Comments:
i fixed finally

> i fixed finally

What was the solution?

@Yangxiaojun1230 How to fix it? I meet this problem too.

For my case, the problem is due to the miss correct of the length padding
BertForSequenceClassification finetune training loss and accuracy have some problem
Author: good74152
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: v4.0.0
- Platform: colab pro
- Python version: 3.6.9
- PyTorch version (GPU?): 1.7.0+cu101
- Tensorflow version (GPU?): 2.3.0
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

@sgugger
@JetRunner


## Information
I follow the paper https://arxiv.org/pdf/2003.02245.pdf to do augmentation and test the performance

Model I am using Bert for BertTokenizer, BertForMaskedLM, BertForSequenceClassification

The problem arises when using:
Use Trainer to fine-tuning on both training set and concatenated set of training set and augmentation set, the Training log is No log or 0.683592, and accuracy is always 0.8

The tasks I am working on is:
An official GLUE  task: sst2, using by huggingface datasets package

The details:
Trainer setting I follow the examples/text_classification.ipynb to build the compute_metrics function and tokenize mapping function, but the training loss and accuracy have bug

my tokenized datasets format:
![擷取](https://user-images.githubusercontent.com/39672039/102020487-7d44ca80-3db4-11eb-8f48-590f885f6957.JPG)

compute_function, little modify by examples/text_classification.ipynb
![image](https://user-images.githubusercontent.com/39672039/102020602-24c1fd00-3db5-11eb-9ed3-5c3a55e50ac9.png)

bert_finetuned_setting
![image](https://user-images.githubusercontent.com/39672039/102020585-05c36b00-3db5-11eb-9440-aa715b3f4c36.png)


fine_tuned result
![image](https://user-images.githubusercontent.com/39672039/102020616-3c00ea80-3db5-11eb-9bcd-95380c8166fc.png)


Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests, rather than help with training.
Could you ask your question on the [forum](https://discusss.huggingface.co) instead?

Thanks!
Chinese
Author: Cheng-Lily
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version:
- Platform:
- Python version:
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.
 
 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 TextGeneration: @TevenLeScao 
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten @TevenLeScao
 Blenderbot: @patrickvonplaten
 Bart: @patrickvonplaten
 Marian: @patrickvonplaten
 Pegasus: @patrickvonplaten
 mBART: @patrickvonplaten
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 RAG: @patrickvonplaten, @lhoestq
 FSMT: @stas00
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1.
2.
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->


Comments:
Fix variable name in TrainingArguments docstring
Author: navjotts
Pull request: yes

# What does this PR do?
Corrects a var name in the docstring for `TrainingArguments` (there is no `ignore_skip_data`)

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?
@sgugger 

Comments:
Fix a broken link in documentation
Author: SBrandeis
Pull request: yes
Labels: name: Documentation

# What does this PR do?

Fixes a broken link to the BERTology example in documentation

Fixes #9100 

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?

 documentation: @sgugger 


Comments:
Cannot load custom tokenizer for Trainer
Author: ierezell
Pull request: no

## Environment info
- `transformers` version: 4.0.0
- Platform: Linux-5.9.13-zen1-1-zen-x86_64-with-glibc2.2.5
- Python version: 3.8.6
- PyTorch version (GPU?): 1.7.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


### Who can help
@mfuntowicz

## Information

Model I am using: My own

The problem arises when using:
* [X] the official example scripts: (give details below)

The tasks I am working on is:
* [X] my own task or dataset: (give details below)

I want to finetune a model on my own dataset. For now it doesn't matter if i finetune bert, DistilBert or other, I just want good embeddings for text similarity (cosine distance)
## To reproduce

Steps to reproduce the behavior:

1. Read the [How to train tutorial](https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/01_how_to_train.ipynb)
2. Train your own tokenizer (working perfectly) and got `model-vocab.json` and `model-merges.txt`
3. Load and encode with :
```
tokenizer = ByteLevelBPETokenizer(./models/custom/my_model-vocab.json","./models/custom/my_model-merges.txt")
``` 
This works nicelly !
4. Try to do the same with a DistilBertTokenizerFast to use with the `Trainer` class
```
tokenizer = DistilBertTokenizerFast.from_pretrained('./models/custom', max_len=512)
```

5. Get the error `check that './models/custom' is the correct path to a directory containing relevant tokenizer files`

Note : I also tried to add a `config.json` file next to the merge and vocab which seemed missing but it doesn't change anything. 
I also tried a RobertaTokenizerFast (and the 'not fast' version) but same problem

## Expected behavior
Train a custom tokenizer and be able to load it with a ModelTokenizer for the Trainer. 
(The BPE tokenizer which works do not have the  `mask_token` attribute to work with the dataset loader)


Comments:
After playing with it more and changing files, names etc... I managed to made it work with Roberta. I guess it was a stupid name error... Sorry for taking 5mn of you time reading this. 
 
I realize it couldn't work with distilbert (bert) as the tokenizers are differents. 
In the end, the model is training. 

Maybe it will help someone else one day. 
Have a good day. 

Glad you could resolve your issue!
Cannot load community model on local machine
Author: logancyang
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.4.0
- Platform: Darwin-19.6.0-x86_64-i386-64bit
- Python version: 3.7.6
- PyTorch version (GPU?): 1.7.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
Probably @LysandreJik 

## Information

Model I am using: https://huggingface.co/huggingtweets/xinqisu

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

## To reproduce

Steps to reproduce the behavior: (this is the instruction on the model page)

```
from transformers import pipeline
generator = pipeline('text-generation', model='huggingtweets/xinqisu')
generator("My dream is", num_return_sequences=5)
```

It gives me

```
OSError: Can't load config for 'huggingtweets/xinqisu'. Make sure that:

- 'huggingtweets/xinqisu' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'huggingtweets/xinqisu' is the correct path to a directory containing a config.json file
```

## Expected behavior

The generator should work with the snippet above. I have trained other `huggingtweets` models and they still work with the same code, for example, the following still works, it downloaded the model successfully.

```
from transformers import pipeline
generator = pipeline('text-generation', model='huggingtweets/billgates')
generator("My dream is", num_return_sequences=5)
```


Comments:
Hello! I believe this is so because this model uses the new weights system, which was introduced in v3.5.1. Please upgrade your transformers version to at least v3.5.1, we recommand the latest (v4.0.1):

```
pip install -U transformers==4.0.1
```

@LysandreJik Thanks for the quick reply! It works now 👍
Add parallelization support for T5EncoderModel
Author: agemagician
Pull request: yes

# What does this PR do?

Extend T5EncoderModel to support model parallization across different GPUs.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

T5: @patrickvonplaten



Comments:
Very cool! Could you also enable the parallelization tests for these models? You can check how it was done in the initial model parallel PR, [here's the commit](https://github.com/huggingface/transformers/pull/8696/commits/cde47f0d110176d3834b736dac27bc9bc2a4de43) related to the tests. You can just add the `T5EncoderModel` to the `all_parallelizable_model_classes` attribute of the `T5ModelTester` class.

> Very cool! Could you also enable the parallelization tests for these models? You can check how it was done in the initial model parallel PR, [here's the commit](https://github.com/huggingface/transformers/pull/8696/commits/cde47f0d110176d3834b736dac27bc9bc2a4de43) related to the tests. You can just add the `T5EncoderModel` to the `all_parallelizable_model_classes` attribute of the `T5ModelTester` class.

Thanks for the tip.
Done, please let me know if anything else is needed from my side.

Also it would be great if you could run `make style && make quality` or `make fixup` to solve the quality issues.

> This LGTM. Looking into it it seems we have an error in `T5Stask` as it is creating the device map with `torch.cuda.device_count()`, rather than the `range` of that value like you're doing it here. Since we're always passing the device map to `T5Stack` (it's never used as a standalone model) we don't see it, but it doesn't seem correct.
> 
> What do you think? If you think this is true, do you mind adding a `range` in `T5Stack` so that we can merge it together? Thanks!

Yes, you are correct, T5Stack should also use range. Since "get_device_map" function apply len to it .
I have updated T5Stack using range.

> Also it would be great if you could run `make style && make quality` or `make fixup` to solve the quality issues.

Done and passed the code quality testing.

Wonderful!
Getting a 404 error when loading TFXLMRobertaModel from 'xlm-roberta-large'
Author: vinaygeorgeroy
Pull request: no

Getting a 404 when trying to load the model. 
Manyually  checked the https://huggingface.co repository for the xlm-roberta-large, was only able to find the Pytorch models, why aren't the TF models available for this, if its not, why is it not explicitly mentioned in the documentation?

Comments:
Could you try with the flag `from_pt=True` when using `from_pretrained`? 

That worked, thanks
Fix T5 and BART for TF
Author: jplu
Pull request: yes

# What does this PR do?

This PR fix the TensorFlow implementation of T5 and BART to make them graph compilation+execution compliant and then be able to create a savedmodel for each them.

The slow tests `test_saved_model_with_hidden_states_output` and `test_saved_model_with_attentions_output` are now passing for both models.


Comments:
I should have addressed everybody's comments :)
[WIP] Add Tapas (bis)
Author: NielsRogge
Pull request: yes
Labels: name: model card

This is a clean branch based on #8113 which is up-to-date with master. To do:

- [x] Make sure all tests pass (currently 44 passed, 4 skipped for `test_modeling_tapas.py`) cc @LysandreJik 
- [x] `make style` & `make quality`
- [x] Investigating forward/backward pass => weird issue with fine-tuning already-finetuned WTQ checkpoint, I guess people should just not do it
- [x]  Add notebooks to show how to use: 
- `tapas-base-finetuned-sqa`: https://colab.research.google.com/drive/1zMW-D2kYrpDA-cvpNJ-ctGD-tDXWebZa?usp=sharing
- `tapas-base-finetuned-tabfact`: https://colab.research.google.com/drive/1Ug6gzPFgf3J0dR-0f4spt0eyPS10dD1l?usp=sharing

Once they all pass, I'll start uploading more checkpoints to the model hub.


Comments:
Closing this PR and opening a new one on the same branch due to Github issues.
CUDA Error when running run_language_modeling.py
Author: IreneZihuiLi
Pull request: no
Labels: name: wontfix

I am trying to run this script[run_language_modeling.py](https://github.com/huggingface/transformers/blob/master/examples/run_language_modeling.py)

I can load the pre-trained roberta model without a problem. However, when it starts training (loss.backward()), then there are issues like these:

> /pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:106: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [30,0,0] Assertion `t >= 0 && t < n_classes` failed.
> /pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:106: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [31,0,0] Assertion `t >= 0 && t < n_classes` failed.

I assume it was caused by the `loss.backward()` line:

> 
> File "2_fine_tune_bert.py", line 386, in train
>     loss.backward()
>   File "/home/lily/zl379/anaconda2/envs/py36/lib/python3.7/site-packages/torch/tensor.py", line 195, in backward
>     torch.autograd.backward(self, gradient, retain_graph, create_graph)
>   File "/home/lily/zl379/anaconda2/envs/py36/lib/python3.7/site-packages/torch/autograd/__init__.py", line 99, in backward
>     allow_unreachable=True)  # allow_unreachable flag
> RuntimeError: CUDA error: CUBLAS_STATUS_ALLOC_FAILED when calling `cublasCreate(handle)

Is it caused by CUDA version or pytorch version? My pytorch is 1.4.0, CUDA: release 10.1, V10.1.243.
Thanks.

Comments:
This seems related to the classes that you use in NLLLoss, That loss function expects a torch.LongTensor with values in the range [0, nb_classes-1] with no values left out in between.

Which version of `transformers` do you have installed? There was a recent change from using -1 to -100 for tokens that should be ignored during the calculation of loss. For example in this line: https://github.com/huggingface/transformers/blob/master/examples/run_language_modeling.py#L218. Therefore, running the latest version of run_language_modeling.py with older versions of `transformers` will give an error similar to what you are seeing.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Problem solved, it was run with Python 3.6+. Return to Python 3.5, no more errors.

> Problem solved, it was run with Python 3.6+. Return to Python 3.5, no more errors.

That should not be the problem. The repo officially only supports 3.6+ any way. 

I'm seeing exactly the same thing training Roberta using `run_mlm.py`. It's at the same step in the training cycle so I can reproduce but I've not tracked down what the issue is, either there's a problem with my input data in a single batch, or perhaps the training has diverged so forward() produces NaN's.

I'll keep digging.
Which dataset is used for training GPT, GPT2 from scratch?
Author: arisohn
Pull request: no

Hi,

I checked the model card of GPT and GPT2, but I can't find the dataset which was used for training.
Where can I find the datasets which is used for?

https://huggingface.co/openai-gpt
https://huggingface.co/gpt2

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discusss.huggingface.co) instead?

Thanks!
seq2seq finetuning scripts break before training (cannot import name ParallelMode)
Author: apostolis19
Pull request: no

File "finetune_trainer.py", line 24, in <module>
    from seq2seq_trainer import Seq2SeqTrainer
  File "/home/---/transformers/examples/seq2seq/seq2seq_trainer.py", line 35, in <module>
    from transformers.training_args import ParallelMode
ImportError: cannot import name 'ParallelMode' from 'transformers.training_args'



Comments:
Make sure you install a recent version of transformers, `ParallelMode` was added to the master branch some ~2 weeks ago.

Yes, as @KDercksen points out, you need an up-to-date install from source to be able to run the examples (as mentioned in the main examples folder README).