noamaanMulla-03/transformers-issues huggingface dataset:
"token_type_ids" is discarded when using GenerationMixin in ”generation_utils.py“
Author: nanzhao
Pull request: no

## Environment info
- `transformers` version: 4.8.2
- Platform: Darwin-19.6.0-x86_64-i386-64bit
- Python version: 3.6.10
- PyTorch version (GPU?): 1.9.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed

### Who can help
@patrickvonplaten @yjernite 


## Information

Model I am using OpenAIGPTLMHeadModel:

I try to use class GenerationMixin in ”generation_utils.py“ to generate words for my pre-trained openai-gpt model, but I find a model performance degradation.

My generation code segment is like this and I need put "input_ids" and "token_type_ids" for my gpt model:
     ` input_ids = torch.tensor(instance["input_ids"], dtype=torch.long, device=args.device).unsqueeze(0)` 
      ` token_type_ids = torch.tensor(instance["token_type_ids"], dtype=torch.long, device=args.device).unsqueeze(0)` 
        ` paras = {"token_type_ids": token_type_ids}` 
       `  bos, eos, pad, speaker1, speaker2 = tokenizer.convert_tokens_to_ids(SPECIAL_TOKENS)` 
        ` chat_history_ids = model.generate(
            input_ids,
            **paras, max_length=128, min_length=5, num_beams=1,
            pad_token_id=pad, use_cache=True,
            eos_token_id=eos, temperature=0.7,
            bos_token_id=bos,
            top_p=0.9, top_k=30, do_sample=True, repetition_penalty=1.03).cpu() `
        

But I find when it call forward in class OpenAIGPTLMHeadModel(OpenAIGPTPreTrainedModel), token_type_ids in forward is None. Although I already put "token_type_ids" in model.generate() with **paras.


## Expected behavior
it seems the bug is here. This function in generation_utils.py discards my token_type_ids here:
`def prepare_inputs_for_generation(self, input_ids: torch.LongTensor, **kwargs) -> Dict[str, Any]:`
 `return {"input_ids": input_ids}`
    



Comments:
Thanks for your issue @nanzhao! In order make `OpenAIGPTLMHeadModel` work with `token_type_ids` we should add this line to `prepare_inputs_for_generation`:

https://github.com/huggingface/transformers/blob/790f1c9545f4a83b97bf75640be82b2112c3efe7/src/transformers/models/gpt2/modeling_gpt2.py#L884

Would you like to open a PR and give it a try? :-)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Adding a test for multibytes unicode.
Author: Narsil
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

Comments:
rum_mlm crashes with bookcorpus and --preprocessing_num_workers
Author: shairoz-deci
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.9.0
- Platform: Linux-5.8.0-63-generic-x86_64-with-debian-bullseye-sid
- Python version: 3.7.10
- PyTorch version (GPU?): 1.8.1+cu111 (True)
- Tensorflow version (GPU?): 2.5.0 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: yes

### Who can help
@LysandreJik
Models:

- albert, bert, xlm: @LysandreJik @sgugger @patil-suraj

Library:
- trainer: @sgugger
- pipelines: @LysandreJik


Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj 


## Information
Trying to train BERT from scratch on wikipedia and bookcorpus using the run_mlm.py example.
As the dataset is large and I am using a strong machine (80 CPU cores 350GB RAM) I set the --preprocessing_num_workers flag to 64 to accelerate the preprocessing. 
When running a wikipedia or squad for sanity check, everything works fine but with bookcorpus, after dataset mapping is supposedly completed (all three occurrences), it gets stuck on with the info:
`Spawning 64 processes `
for a while and crashes with 
`BrokenPipeError: [Errno 32] Broken pipe`
This does not occur when dropping the --preprocessing_num_workers flag but then processing wiki + bookcorpus will take nearly two days.

I tried changing the transformer version or upgrading/downgrading the multiprocessing and dill packages and it didn't help

The problem arises when using:
* [ x] the official example scripts: (give details below)


## To reproduce

Steps to reproduce the behavior:

run:
`python transformers/examples/pytorch/language-modeling/run_mlm.py --output_dir transformers/trained_models/bert_base --dataset_name bookcorpus --model_type bert --preprocessing_num_workers 64 --tokenizer_name bert-base-uncased --do_train --do_eval --per_device_train_batch_size 16 --overwrite_output_dir --dataloader_num_workers 64 --max_steps 1000000 --learning_rate 1e-4 --warmup_steps 10000 --save_steps 25000 --adam_epsilon 1e-6 --adam_beta1 0.9 --adam_beta2 0.999 --weight_decay 0.0'


## Expected behavior

Training should begin as done properly when loading wiki and other datasets

Thanks is advance,

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
How to fuse copy mechnism into the GenerationMixin？
Author: Hannibal046
Pull request: no

Hello， is there any way to directly fuse copy mechanism into beamsearch? since the beam_search function receive model_output.logit rather than probabily of vocab?
https://github.com/huggingface/transformers/blob/72aee83ced5f31302c5e331d896412737287f976/src/transformers/generation_utils.py#L1801

Comments:
Pinging @patrickvonplaten and @patil-suraj 

Hey @Hannibal046,

Could you clarify a bit what you mean by "copy-mechanism" ?
Maybe a code example of what you want to do?

Hello,I also find others to talk about the `copy mechanism`. in this [link](https://discuss.huggingface.co/t/copying-mechanism-for-transformer/5025)
BTW,could you please check my another issue about BART Generation? It confused me a long time, https://github.com/huggingface/transformers/issues/12870,  thanks so much.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
LEDForSequenceClassification and LEDForQuestionAnswering example codes don't work.
Author: jacklxc
Pull request: no

## Environment info
Tried on both transformers=4.2.0 and the latest transformer package.

### Who can help
@patrickvonplaten

Models:

LED

## Information

LEDForSequenceClassification and LEDForQuestionAnswering example code doesn't work. Please fix these bugs. LEDForConditionalGeneration works though. [here](https://huggingface.co/transformers/model_doc/led.html#ledforsequenceclassification)

The example [notebook](https://colab.research.google.com/drive/12LjJazBl7Gam0XBPy_y0CTOJZeZ34c2v?usp=sharing) only works for transformers=4.2.0. Specifically, there will be an error of in-place operation during the training. It will be helpful if you can update the code to adapt to the latest packages.

Comments:
Actually we should probs just remove those examples since there is no fine-tuned model anyways...@jacklxc would you like to make a PR? :-)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
tokenizers add_token bug
Author: ssoltan88
Pull request: no

The way `add_token` is implemented results is problematic tokenization when added tokens are substring of each other. Example:
```
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("google/mt5-large")

tokenizer.add_tokens(['my_token_1', 'my_token_12', 'my_token_123'])

tokenizer.tokenize("my_token_1 and my_token_12 and my_token_123")
```

output:

```
['my_token_1',
 '▁and',
 '▁',
 'my_token_1',
 '▁2',
 '▁and',
 '▁',
 'my_token_1',
 '▁23']
```

Because of implementation (i.e., breaking text on added tokens), adding new tokens in the reversed order (i.e., `tokenizer.add_tokens(['my_token_123', 'my_token_12', 'my_token_1'])`) results in the correct tokenization. So one solution is always order the added tokens in reversed.

Comments:
Pinging @n1t0 and @SaulLu for advice

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Can we use trainer api with custom model layer?
Author: monk1337
Pull request: no

I am trying to use huggin face trainer API, For example, my model looks like this:

```
class BERTClass(torch.nn.Module):
    def __init__(self):
        super(BERTClass, self).__init__()
        self.l1 = AutoModel.from_pretrained("distilbert-base-uncased",return_dict=False)
        self.l2 = torch.nn.Dropout(0.2)
        self.l3 = torch.nn.Linear(768, 50)
    
    def forward(self, ids, mask, token_type_ids):
        _, output_1= self.l1(ids, attention_mask = mask, token_type_ids = token_type_ids)
        output_2 = self.l2(output_1)
        output = self.l3(output_2)
        return output

model = BERTClass()
```

Is it possible to utilize the trainer API for custom models? such as :

trainer = Trainer(
    model=model,                         # the instantiated 🤗 Transformers model to be trained
    args=training_args,                  # training arguments, defined above
    train_dataset=train_dataset,         # training dataset
    eval_dataset=val_dataset             # evaluation dataset
)

trainer.train()



Comments:
Yes you can. As mentioned in the [docs](https://huggingface.co/transformers/main_classes/trainer.html):

> When using it on your own model, make sure:
> your model always return tuples or subclasses of ModelOutput.
> your model can compute the loss if a labels argument is provided and that loss is returned as the first element of the tuple (if your model returns tuples)
> your model can accept multiple label arguments (use the label_names in your TrainingArguments to indicate their name to the Trainer) but none of them should be named "label".
Add TAPAS MLM-only models
Author: NielsRogge
Pull request: yes

# What does this PR do?

As requested by #12916, I've converted the TAPAS checkpoints which were pre-trained on masked language modeling (MLM) only. `TapasForMaskedLM` was previously defined, but there were no actual checkpoints available with the language modeling head. They're available now on the hub:

* [google/tapas-large-masklm](https://huggingface.co/google/tapas-large-masklm)
* [google/tapas-base-masklm](https://huggingface.co/google/tapas-base-masklm)
* [google/tapas-medium-masklm](https://huggingface.co/google/tapas-medium-masklm)
* [google/tapas-small-masklm](https://huggingface.co/google/tapas-small-masklm)
* [google/tapas-mini-masklm](https://huggingface.co/google/tapas-mini-masklm)
* [google/tapas-tiny-masklm](https://huggingface.co/google/tapas-tiny-masklm).

It also cleans up the conversion script of TAPAS a bit.

Fixes #12916 

Comments:
Thanks for this @NielsRogge !
Huggingface Inference API
Author: robertalanm
Pull request: no

Hello,

I recently bought the startup subscription for my organization. However, it appears I cannot add large models to inference API. I am trying to serve GPT-J-6B model on inference API. When I emailed "api-inference@huggingface.co" it returns an error message:
```host aspmx.l.google.com[142.251.4.27] said:
The email account that you tried to reach does not exist. 
Please try double-checking the recipient's email address for typos or unnecessary spaces.
```
What is the updated support email for startup plan users?

Comments:
Hi, it is `api-enterprise@huggingface.co` – I think there was an incorrect reference to the other email address somewhere that we fixed recently (cc @jeffboudier)

I sent this address an email but have not received a response. Is there another way to contact the engineers in charge of setting up large models for inference api endpoints?

I'm just going to cancel the sub. Sorry for taking up anyone's time.
Fix scheduled TF Speech tests
Author: anton-l
Pull request: yes

CI logs: https://github.com/huggingface/transformers/runs/3501334018?check_suite_focus=true

**Context**
Doing `import soundfile as sf` was causing `OSError: sndfile library not found` on all integration tests for `TFWav2Vec2` and `TFHubert`

**Solution**
This PR adds `libsndfile` as a dependency for the TF jobs, like it was done for the Pytorch counterparts.

CC @patrickvonplaten 

Comments:
Thanks a lot for taking care of it
Update version of `packaging` package
Author: shivdhar
Pull request: yes

This fixes an error when using packaging<20.0 with transformers.

With `packaging==19.0`:

```python
ipython3
Python 3.6.14 (default, Jun 29 2021, 00:00:00) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.6.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import transformers                                                                                                                                             
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-1-279c49635b32> in <module>
----> 1 import transformers

~/.local/lib/python3.6/site-packages/transformers/__init__.py in <module>
     41 
     42 # Check the dependencies satisfy the minimal versions required.
---> 43 from . import dependency_versions_check
     44 from .file_utils import (
     45     _LazyModule,

~/.local/lib/python3.6/site-packages/transformers/dependency_versions_check.py in <module>
     34         if pkg == "tokenizers":
     35             # must be loaded here, or else tqdm check may fail
---> 36             from .file_utils import is_tokenizers_available
     37 
     38             if not is_tokenizers_available():

~/.local/lib/python3.6/site-packages/transformers/file_utils.py in <module>
    310 if _torch_available:
    311     torch_version = version.parse(importlib_metadata.version("torch"))
--> 312     _torch_fx_available = (torch_version.major, torch_version.minor) == (
    313         TORCH_FX_REQUIRED_VERSION.major,
    314         TORCH_FX_REQUIRED_VERSION.minor,

AttributeError: 'Version' object has no attribute 'major'
```
This is fixed with `packaging>=20.0`


The `major` attribute was added in v20.0 of `packaging`. [Relevant changelog](https://packaging.pypa.io/en/latest/changelog.html#id11) excerpt:

> 20.0 - 2020-01-06¶
> Add major, minor, and micro aliases to packaging.version.Version (#226)

Therefore, have constrained version of packaging to '>=20.0'.

Please let me know if there are any setup.py-related tests to be updated.

CC @stas00 @sgugger 

Comments:
skip image classification example test
Author: patil-suraj
Pull request: yes

# What does this PR do?

The image classification example test is still faling, https://app.circleci.com/pipelines/github/huggingface/transformers/27636/workflows/fad6d0a8-e8fa-413b-a519-a4794673749a/jobs/268590

Skip the test for now, until fixed (cc @nateraw)


Comments:
Thanks!

Thank you! Will take a look
T5Tokenizer.from_pretrained("t5-small") returning NoneType
Author: santoshnlp
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform:
- Python version:  Python 3.7.11
- PyTorch version (GPU?):  1.9.0+cu102
- Tensorflow version (GPU?): 2.6.0

-Google colab

### Who can help
@patrickvonplaten, @patil-suraj @sgugger @LysandreJik

Models:

-   t5: @patrickvonplaten, @patil-suraj


Library:


- text generation: @patrickvonplaten
- tokenizers: @LysandreJik


Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj


 -->

## Information

Model I am using (Bert, XLNet ...): t5

The problem arises when using:
* [ ] the official example scripts: (give details below)
tokenizer = T5Tokenizer.from_pretrained("t5-small")




## To reproduce

Steps to reproduce the behavior:
using google colab
1.  tokenizer = T5Tokenizer.from_pretrained("t5-small")  as in
https://huggingface.co/transformers/model_doc/t5.html#transformers.T5Tokenizer
2. type(tokenizer)  -->NoneType 




## Expected behavior
tokenizer should not be NoneType
<!-- A clear and concise description of what you would expect to happen. -->


Comments:
Hi there, 

PLEASE, do not tag so many people in an issue, usually just tag one or two maintainers. Tagging everyone will unnecessarily disturb. While we  are happy to answer, we would expect people to consider this before opening an issue. Thanks.

Now regarding your issue, did you install `sentencpiece` ? If not the you could
```pip install sentencepiece```
and restart the colab.

hi Suraj,
       Thanks. It is working.
In ViT model,  last_hidden_state is not equal to hidden_states[-1]
Author: youyuge34
Pull request: no

When input the same image, in Google ViT model  output.last_hidden_state is not equal to output.hidden_states[-1] ?
I tried in Bert， the outputs are the same.

```
feature_extractor = ViTFeatureExtractor.from_pretrained('google/vit-base-patch16-224-in21k')
model = ViTModel.from_pretrained('google/vit-base-patch16-224-in21k')
inputs = feature_extractor(images=[image], return_tensors="pt")
outputs = model(pixel_values=inputs['pixel_values'], output_hidden_states=True)

vec1 = outputs.hidden_states[-1][0, 0, :]
vec2 = outputs.last_hidden_state[0, 0, :]
```

in my mind, vec1 should be the same as vec2. But the fact is they are not the same at all.

Comments:
Hi,

The Vision Transformer also applies a final layer norm after the several encoder layers (blocks), as can be seen [here](https://github.com/huggingface/transformers/blob/92d4ef9ab038dfbbe02556375c4c3c14215b37d2/src/transformers/models/vit/modeling_vit.py#L553). This explains why `outputs.last_hidden_state` is not equal to `outputs.hidden_states[-1]`.

@NielsRogge THX~