noamaanMulla-03/transformers-issues huggingface dataset:
Wrong model_max_length for BERTOverflow tokenizer
Author: greav
Pull request: no

## Environment info

- `transformers` version: 3.4.0
- Platform: Linux-5.4.0-52-generic-x86_64-with-glibc2.29
- Python version: 3.8.5
- PyTorch version (GPU?): 1.6.0 (False)
- Tensorflow version (GPU?): 2.3.1 (False)
- Using GPU in script?: N
- Using distributed or parallel set-up in script?: N

## Information

Hi. I used [BERTOverflow](https://huggingface.co/jeniya/BERTOverflow) and found strange behavior for the tokenizer property `model_max_length`. It is equal 1000000000000000019884624838656, although it should be 512.

## To reproduce
```python
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained('jeniya/BERTOverflow')
print(tokenizer.model_max_length)
```

Comments:
Ah, this is an error, indeed. The uploader should have uploaded a `tokenizer_config.json` containing the model maximum length. I see it only contains the `do_lower_case` argument right now.

@julien-c what should we do here? Should we update that configuration ourselves?

Also @thomwolf @julien-c, I feel like we should have sensible defaults for each model. This is a `BertModel` which currently only has access to absolute positional embeddings. It doesn't make sense to have an unlimited `model_max_length`. I think `BertTokenizer` and all its relatives should have a default of `512`.

@LysandreJik Yes, our policy here is that we fix the config ourselves and notify the model author (via email/GitHub mention/anything). Feel free to do it and link the resulting commit from here.

 (use your hf.co email/author name for your commit to be linked to your hf.co user profile)

Updated with [212cd3e](https://huggingface.co/jeniya/BERTOverflow/commit/212cd3ef9615e7292da65a49162e0beb0cd3d604) and sent an email.

see [`huggingface.co:212cd3e`](https://huggingface.co/jeniya/BERTOverflow/commit/212cd3ef9615e7292da65a49162e0beb0cd3d604)

I happen to get the same error for all of:

```
dmis-lab/biobert-base-cased-v1.1
dmis-lab/biobert-large-cased-v1.1
microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract
emilyalsentzer/Bio_ClinicalBERT
```

I understand that in user code I should just do a fallback to sensible default if I get that large value, but that seem like a problem not related to a single model.

This issue has been stale for 1 month.
squad_convert_example_to_features is broken
Author: brian8128
Pull request: no

## Environment info
- `transformers` version: 4.2.2
- Platform: Linux-4.15.0-132-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.7.1+cu101 (True)
- Tensorflow version (GPU?): No
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@sgugger 

Models:

NA

## Information

The squad_convert_example_to_features function requires a tokenizer but there is no way to give it one so you always get `NameError: name 'tokenizer' is not defined`

## To reproduce

Call `squad_convert_example_to_features` with any input. 

## Expected behavior

Convert a squad example to features. 


Comments:
Please use `squad_convert_example_to_features_init(yourtokenizer)` to set the tokenizer.
Workflow fixes
Author: LysandreJik
Pull request: yes

Fixes some workflow issues:
- Installs torch scatter in the CI with the appropriate pre-compiled version
- Removes DeepSpeed and Fairscale from the non-cuda-extension workflows
- Adds the forgotten reports for cuda-extension workflows
- Adds the result of the cuda-extension workflows to be sent to Slack

Also it updates the `deepspeed` dependency in the dependency table, because they seem mismatched on `master` (running `make fixup` fixed it for me)

Comments:
That's one advantage of using the docker images! We control the exact CUDA and torch versions by controlling the images directly, so it won't break until we manually update it, at which point we should remember to be extra careful about this dependency.
DataCollatorForSOP marked as deprecated but DataCollatorForLanguageModeling does not offer the same functionality
Author: xamm
Pull request: no

The `DataCollatorForSOP` is marked as deprecated and it is recommended to use the `DataCollatorForLanguageModeling` instead. [Link to the data_collator.py](https://github.com/huggingface/transformers/blob/4bae96ec2bee265f938fc262201538819419089a/src/transformers/data/data_collator.py)

As far as I can tell, the labels for the sentence order prediction task (`sentence_order_label`) are not set in `DataCollatorForLanguageModeling`. 

Will this be added to `DataCollatorForLanguageModeling` in a future release, or what is the correct procedure when a data collator is needed for both the mlm and sop tasks at the same time, as in ALBERT training?



Comments:
The `sentence_order_label` will be left as is and collated if your dataset provides them. This is tested [here](https://github.com/huggingface/transformers/blob/2550b41aa2ec34f05ddfd3ec5875ddb32ad78d58/tests/test_data_collator.py#L268) which is adapted from the old test of `DataCollatorForSOP`.

Oh, now I see it too. I was kind of under the assumption that the DataCollator could also be used on non-preprocessed data to tokenize and preprocess batches on demand.
[TF T5] Possible Error using TF T5 with Keras
Author: patrickvonplaten
Pull request: no

See: https://discuss.huggingface.co/t/how-to-train-tft5forconditionalgeneration-model/888. 



Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This issue has been stale for 1 month.
failed to import BertModel
Author: masterbo98
Pull request: no
Labels: name: Migration

# 📚 Migration

## Information

The version of my torch is 1.6.0. When I want to import BertModel from transformers, it raised an error: ModuleNotFoundError: No module named '_sentencepiece'
I firstly activate my envs and used 'conda install transformers'.
Please help me, how can I address this problem?

Comments:
I don't know what your transformers version is, but from your error message, you should install `sentencepiece`: `pip install sentencepiece` or `conda install sentencepiece` with the appropriate channel (probably `conda-forge`)

Thank u very much, I have solved this problem. Your work is really remarkable.&nbsp;



---Original---
From: "Lysandre ***@***.***&gt;
Date: Fri, Apr 16, 2021 21:17 PM
To: ***@***.***&gt;;
Cc: ***@***.******@***.***&gt;;
Subject: Re: [huggingface/transformers] failed to import BertModel (#11280)




 
I don't know what your transformers version is, but from your error message, you should install sentencepiece: pip install sentencepiece or conda install sentencepiece with the appropriate channel (probably conda-forge)
 
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Happy to help!

The
Trainer support for IterableDataset for evaluation and predict
Author: sgugger
Pull request: yes

# What does this PR do?

This PR rewrites the entirety of the evaluation loop to add support for `IterableDataset`. The main problem with the current training loop is that, in distributed settings, it expects the indices of the evaluation set to come like this:
- `[0, 1, 2, 3, 4, 5, 6, 7, ...., 99]` for process 0
- `[100, 101, 102, 103, 104, 105, 106, 107, ...., 199]` for process 1
(if we have 200 samples)

In an `IterableDataset` we don't know the length at the beginning of the process, so we can cleanly cut the indices in half like that. Therefore, the indices will come like this (with a batch size of 4):
- `[0, 1, 2, 3, 8, 9, 10, 11, ...., 192, 193, 194, 195]` for process 0
- `[4, 5, 6, 7, 12, 13, 14, 15, ...., 196, 197, 198, 199]` for process 1

The rewrite of the evaluation loop is done to:
- change the sampling indices in a normal `Dataset` to be the same as an `IterableDataset`
- change the way predictions and labels are gathered accordingly
- avoid having one evaluation loop for `Dataset` and one for `IterableDataset`

To avoid any breaking change:
- the old evaluation loop is still there with the same name (for people who subclass Trainer) and can be used if one passes the flag `--use_legacy_prediction_loop`.
- the old `DistributedSequentialSampler` and `DistributedTensorGatherer` are left and deprecated

Comments:
Pr2keep encoder decoder synced
Author: cronoik
Pull request: yes

# What does this PR do?

Fixes #11285 + adds an implementation for the `resize_token_embeddings method` (currently the parent class implementation is used which throws an error). 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

 @patrickvonplaten, @patil-suraj

Comments:
Fast Tokenizers instantiated via vocab/merge files do not respect skip_special_tokens=True
Author: minimaxir
Pull request: no

## Environment info
- `transformers` version: 4.3.2
- Platform: macOS-11.2.1-x86_64-i386-64bit
- Python version: 3.9.1
- PyTorch version (GPU?): 1.7.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

## Information

See title; this issue does not reproduce with slow tokenizers. Does not reproduce with serialized tokenizers.

Found while investigating https://github.com/minimaxir/aitextgen/issues/88

## To reproduce

Using [gpt2_merges.txt](https://github.com/minimaxir/aitextgen/blob/master/aitextgen/static/gpt2_merges.txt) and [gpt2_vocab.json](https://github.com/minimaxir/aitextgen/blob/master/aitextgen/static/gpt2_vocab.json) as linked:

```py
from transformers import AutoModelForCausalLM, GPT2Tokenizer, GPT2TokenizerFast

model = AutoModelForCausalLM.from_pretrained("distilgpt2")

outputs = model.generate(max_length=40)

# tensor([[50256,   383,   471,    13,    50,    13,  2732,   286,  4796,   468,
#           587, 10240,   262,  1918,   286,   257,  1966,  5349,  5797,   508,
#           373,  2823,   290,  2923,   416,   257, 23128,   287,   262,   471,
#            13,    50,    13, 13241,   319,  3583,    13,   198,   198,   198]])

tokenizer_fast = GPT2TokenizerFast(vocab_file="gpt2_vocab.json", merges_file="gpt2_merges.txt")
tokenizer_fast.decode(outputs[0], skip_special_tokens=True)

# '
 The U.S. Department of Justice has been investigating the death of a former FBI agent who was shot and killed by a gunman in the U.S. Capitol on Wednesday.\n\n\n'

tokenizer_slow = GPT2Tokenizer(vocab_file="gpt2_vocab.json", merges_file="gpt2_merges.txt")
tokenizer_slow.decode(outputs[0], skip_special_tokens=True)

# ' The U.S. Department of Justice has been investigating the death of a former FBI agent who was shot and killed by a gunman in the U.S. Capitol on Wednesday.\n\n\n'

```


Comments:
Indeed, I can reproduce! Do you know what might be causing this @n1t0?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

I think this is happening because when you load it from the vocab and merge files, it doesn't know `
` is a special token. For the `skip_special_tokens` to work, I believe it would be necessary to add them to the tokenizer:
```python
tokenizer_fast.add_special_tokens({
    "additional_special_tokens": "
"
})
```

The `tokenizer.json` file on the hub, available for `gpt2` does have this special token registered, that's why it works in this case.

That workaround is sufficient for my needs and appears to have done the trick. Thanks!
This class and module cannot be found
Author: bboyxu5928
Pull request: no
Labels: name: Core: Tokenization

name: Installation

I want to use BertForNextSentencePrediction in ' modeling_bert.py' line 1020  ,but when I run Examples in line 1065,  error happens

  File "E:\work\pycharm\transformers-master\src\transformers\tokenization_bert.py", line 24, in <module>
    from tokenizers import BertWordPieceTokenizer
ImportError: No module named 'tokenizers'

where I can find this module tokenizers,
thanks !!


Comments:
You can do `pip install tokenizers` or [install from source](https://github.com/huggingface/tokenizers).

> You can do `pip install tokenizers` or [install from source](https://github.com/huggingface/tokenizers).

OK， I've got it ,thank you

> pip install tokenizers

```bash
Requirement already satisfied: tokenizers in ./anaconda3/envs/dnabert/lib/python3.6/site-packages (0.8.1rc2)
```

I did it,but it also has the error

in my another computer, the env displays its version is 0.5.0, so why give me 0.8.1rc2? sometimes i want to say, the doc is hard to understand, when understanded, the errors occurs one by one because of some small tips, is it my fault? maybe, but why not write it straightly?
env about run longformer model downloaded from https://github.com/allenai/longformer 
Author: BinchaoPeng
Pull request: no

1. just use `conda install transformers`，the transformers version is 4.4.2,  can't run
    1.1. error:
           ```bash
          can't import pipline
          
         # then use tokenizer and model to get my feature vector，ERROR：

           RuntimeError: Error(s) in loading state_dict for BartModel:
	size mismatch for model.encoder.embed_positions.weight: copying a param with shape torch.Size([16386, 768]) from 
       checkpoint, the shape in current model is torch.Size([1026, 768]).
          ```
2. change to use `pip install transformers`, the transformers version is 3.3.0,can't run
   2.1. error:
         ```bash
         can't import name tokenizer
        
         # then i found it in issue that use "pip install tokenizer", it has exited, and tokenizer version is 0.8.0rc2. I find another env that can work,the tokenizer version is 0.5.0,so i use pip to change its version from 0.8.0rc2 to 0.5.0.ERROR:
         
         pip's dependency ..... which is incompatibe
         
        # however the version of toikenizer in only one env which can run is 0.5.0 . 
         ```
3. So the only one worked env is:
 ```bash
conda create -n dnabert python=3.6

# pytorch-transformers 
pip install pytorch-transformers

conda install pytorch torchvision cudatoolkit=10.0 -c pytorch

# dnabert
git clone https://github.com/jerryji1993/DNABERT
cd DNABERT
python3 -m pip install --editable .
cd examples
python3 -m pip install -r requirements.txt 

# allenai 
conda install cudatoolkit=10.0
pip install git+https://github.com/allenai/longformer.git

# huggingface
pip install transformers
```
# 我做错了什么才会到这种局面？

Comments:
I created an env that cannot be copied

Please do not create duplicates.
Duplicate of #11301 

sorry, i close it now
Why Bert-chinese use do_lower_case=False?
Author: Fei-Wang
Pull request: no

Some Chinese Text has some English words, for example: "Apples是苹果的复数形式。". I have questions about how to tokenize the text:

1.  why Chinese Bert Case sensitive, but I can't find even 'A' in vocab.txt
2.  Because English words in Chinese vocab.txt is few, should I use wordpiece tokenizer as default, like "['apple', '##s', '是', '苹', ...]"or split to char to tokenize, like "['a', 'p', 'p', 'l', 'e', 's', '是', '苹', ...]"?

Comments:
Hmm, I don't really know how to best answer your question here...maybe @JetRunner ?

using lowercase=False preserves the information about casing and this information maybe helpful in the context of the some work. such as for sentiment analysis i think casing is not important there but for the task like NER it maybe useful. Again but for many task where we deal with multiple languages is it recommended to use cased model because i think every language has its own grammar and syntax and maybe casing helps in some or the other way?

Could this be the reason why Bert-chinese uses lowercase=False by default?

I think your explanation makes sense. 
But there are no capital letters in the Chinese vocab.txt, so all words contain capitals will be regarded as [unk].

Sent from my iPhone

> On Dec 25, 2020, at 4:57 AM, Shubham kumar <notifications@github.com> wrote:
> 
> ﻿
> using lowercase=False preserves the information about casing and this information maybe helpful in the context of the some work. such as for sentiment analysis i think casing is not important there but for the task like NER it maybe useful. Again but for many task where we deal with multiple languages is it recommended to use cased model because i think every language has its own grammar and syntax and maybe casing helps in some or the other way?
> 
> Could this be the reason why Bert-chinese uses lowercase=False by default?
> 
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub, or unsubscribe.


Well I would say the design of Chinese BERT is not necessarily the best. It makes sense to use only lower cases to resolve the data sparsity since there are not many English sentences in Chinese Wikipedia.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Zero-shot pipeline feature extraction
Author: rodrigoheck
Pull request: no

Is it possible to extract the hidden states representation from the zero-shot pipeline? I have these two tasks: feature extraction and zero-shot classification. But I don't want to load the same model twice, since it is a major burden on GPU memory. Any suggestions to how I can do both tasks without having to load it twice?

Comments:
The 

> The

Yes?

Answering my question: pipeline returns the model as well. You just have to use it directly to extract the hidden states.
Module Not found: datasets_modules.datasets.output
Author: ashleylew
Pull request: no

## Environment info

- `transformers` version: 4.5.0.dev0
- Platform: Linux-3.10.0-1160.15.2.el7.x86_64-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.8.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: not sure
- Using distributed or parallel set-up in script?: <fill in> ? 

### Who can help
@patil-suraj

## Information

Model I am using (Bert, XLNet ...): BART seq2seq

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. "Install from source" method
2. ran this command, where "data/output.jsonl" is my dataset: 
```
python examples/seq2seq/run_translation.py \
    --model_name_or_path t5-small \
    --do_train \
    --do_eval \
    --source_lang en \
    --target_lang de \
    --source_prefix "Translate English to Logical Forms: " \
    --dataset_name data/output.jsonl \
    --output_dir /tmp/tst-translation \
    --per_device_train_batch_size=4 \
    --per_device_eval_batch_size=4 \
    --overwrite_output_dir \
    --predict_with_generate
```
3. Got the following error: 
```
ModuleNotFoundError: No module named 'datasets_modules.datasets.output'
```

At first it told me that "datasets" was not installed, so I did ```pip install datasets``` and that worked fine. Then I got this error and haven't been able to figure out what it means or how to fix it. 

Comments:
@ashleylew were you able to fix this? Could you please share what did?


I was just loading the data incorrectly. I misunderstood how to load custom data in and was treating it like one of Hugging Face's datasets rather than a custom one, but once I fixed that, everything was fine! I can give more details if necessary.

I'll check that part out in my script then, thanks!


@ashleylew Hi I'm getting the same error, can you share the data loading script? I tried doing stuff but it doesn't work for me. 
Thanks


Yeah! So in my command script that I posted here, I used `    --dataset_name data/output.jsonl \ ` but that command is for pre-loaded data sets, whereas mine is a custom one. So instead you'll want to use: 

```
    --train_file train.json \
    --validation_file validation.json \
```
Why couldn't I use encoder_hidden_states when position_ids is not None? GPT2Model.foward()
Author: e-yi
Pull request: no

`device` is required in GPT2Model.foward() if I'd like to use encoder_hidden_states.
```
          if self.config.add_cross_attention and encoder_hidden_states is not None:
              encoder_batch_size, encoder_sequence_length, _ = encoder_hidden_states.size()
              encoder_hidden_shape = (encoder_batch_size, encoder_sequence_length)
              if encoder_attention_mask is None:
                  encoder_attention_mask = torch.ones(encoder_hidden_shape, device=device)
              encoder_attention_mask = self.invert_attention_mask(encoder_attention_mask)
          else:
              encoder_attention_mask = None
  ```
https://github.com/huggingface/transformers/blob/26212c14e5570aff40b90c11495d97dada4272fb/src/transformers/models/gpt2/modeling_gpt2.py#L682

But the only place that sets `device` is in another *if statement*.
```python
        if position_ids is None:
            device = input_ids.device if input_ids is not None else inputs_embeds.device
            position_ids = torch.arange(past_length, input_shape[-1] + past_length, dtype=torch.long, device=device)
            position_ids = position_ids.unsqueeze(0).view(-1, input_shape[-1])
```
https://github.com/huggingface/transformers/blob/26212c14e5570aff40b90c11495d97dada4272fb/src/transformers/models/gpt2/modeling_gpt2.py#L653

And I was wondering why is it required to have 'position_ids==None' when I just want to use `encoder_hidden_states` .
Am I missing something?

I ran into this problem when trying to use GPT2LMHeadModel for image captioning tasks.

Comments:
Ah, I think this is an issue indeed, the device statement shouldn't be inside that `if` statement. Do you want to open a PR to fix this?

Sure. 

Fixed by #11292