noamaanMulla-03/transformers-issues huggingface dataset:
Sentencepiece Unigram tokenizer add tokens
Author: sooftware
Pull request: no

I use sentencepiece unigram tokenizer that trained by `tokenizers` library.  
I'm trying to combine the two languages of tokenizer.  
So I tried like following:

```python
from transformers import AutoTokenizer

tokenizer1 = AutoTokenizer.from_pretrained('TOKENIZER1')
tokenizer2 = AutoTokenizer.from_pretrained('TOKENIZER2')

vocab1 = tokenizer1.vocab.keys()
vocab2 = tokenizer2.vocab.keys()

new_tokens = list()

for v in vocab2:
    if v not in vocab1:
        new_tokens.append(v)

tokenizer1.add_tokens(new_tokens)

tokenizer1.save_pretrained('NEW_TOKENIZER')
```

Will there be a problem when combined like this?
Thank you.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Use different data collator for train and eval dataset in trainer
Author: RyanWangZf
Pull request: no

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

I am trying pretraining a BART model from scratch on my own data. Since the noises added during the training phase are more than MLM, I need a different data collator during evaluation that differs from the one used in training. However, in `trainer`, there is only one arg for setting data collator that is used for both train and eval dataset. How do I achieve that?


Comments:
Hi there. You will need to subclass the Trainer and change the `get_eval_dataloader` method.

thanks a lot!
Add strong test for configuration attributes
Author: sgugger
Pull request: yes

# What does this PR do?

This PR adds tests to make sure every possible common kwargs is properly set in all the configurations. This is done by adding a common test in the config tests that tries to set all common attributes to a value different form the default, as set in the dictionary `config_common_kwargs`.

That dictionary is itself tested against a base `PretrainedConfig` to make sure that if we change the defaults or add new attributes, `config_common_kwargs` is also updated.

This should prevent #13992 from happening again.

Fixes #13992

Comments:
It has been run on all test model files in the second to last commit (before removing a fake modif introduced to trigger all modeling tests). The only failure there was the SQUAD example, failing on master because of the Datasets release.
where processor should i put in a training code?
Author: lycfight
Pull request: no

Hi @lycfight  could you please open an issue with a minimal code snippet so we could take a look. Thanks :)

_Originally posted by @patil-suraj in https://github.com/huggingface/transformers/issues/11445#issuecomment-909943670_

Comments:
I have a problem during writing a train code in pytorch.I want to create a custom Dataset for coco image_caption dataset such as follow:
1. inherit torch.utils.data:
```
from torch.utils.data import Dataset
Class Image_textDataset(Dataset):
```
2.then overwrite __getiem__(self,idx) where i used processor to process a tuple of (image, text) sample
but it seems that CLIPProcessor could't process a tuple of (image, text) sample to same shape for dataloader to make a batch as follow:
```
def __getitem__(self, idx):
        img_id = self.img_ids[idx]
        # randomly pick one caption from the image captions
        text = random.choice(self.img_id_to_captions[img_id])
        img_filename = self.img_id_to_filename[img_id]
        img_path = op.join(self.img_dir, img_filename)
        img = Image.open(img_path)
        inputs = processor(text = text, images = img, return_tensors="pt", padding="max_length",truncation=True)
        return inputs
```
3.or __getitem__return a pair of raw (image, text) sample, and use processor in a custom collate_fn like:
```
def collate_fn(examples):
        images = [example[0] for example in examples]
        captions = [example[1] for example in examples]
        inputs = processor(
            text=captions,
            images=images,
            max_length=77,
            padding="max_length",
            truncation=True,
            return_tensors="pt",
        )

        batch = {
            "pixel_values": inputs["pixel_values"],
            "input_ids": inputs["input_ids"],
            "attention_mask": inputs["attention_mask"],
        }

        return batch
```
then pass collate_fn as a parameter to dataloader

This issue is not of relevance to transformers repository please post this in pytorch forums for quick help

> This issue is not of relevance to transformers repository please post this in pytorch forums for quick help

I think processor is a base method of transformers which should be concerned in transformers' tutorials

You could put the processor anywhere you want either in the dataset or collate_fn. If processing on the fly, then I would  put it in the `collate_fn` as it would process the whole batch with single call, which is usually faster than processing single examples.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Saving/Reloading the Flax T5 model
Author: kaushalshetty
Pull request: no

@patil-suraj @patrickvonplaten 

Hey,
I was able to train the FlaxT5Model using ```run_t5_mlm_flax.py``` on a small custom dataset as per the flax doc. How do I save and reuse the models to get the last hidden states? Would the following code work?

```
auto_tokenizer = AutoTokenizer.from_pretrained("my_output_folder")
model = FlaxT5Model.from_pretrained('my_output_folder')
input_ids = auto_tokenizer("my encoder input", return_tensors="np").input_ids
decoder_input_ids = auto_tokenizer("my decoder input", return_tensors="np").input_ids
outputs = model(input_ids=input_ids, decoder_input_ids=decoder_input_ids)
last_hidden_states = outputs.last_hidden_state
print(numpy.array(last_hidden_states))
```




Comments:
You can save the model with `model.save_pretrained('my_saved_folder')`
How should I try to feed new memory states to the next segments
Author: Xelawk
Pull request: no

Here is a simple exemple for my training loop based on Trainer class
```python
transfo_xl_config = transformers.TransfoXLConfig(**hparams)
model = transformers.TransfoXLForSequenceClassification(transfo_xl_config)

training_args = transformers.TrainingArguments("test_trainer", no_cuda=True)
trainer = transformers.Trainer(
    model=model,
    args=training_args,
    train_dataset=MyDataset('train'),
    eval_dataset=MyDataset('test')
)
trainer.train()
```
The question is how should I feed new memory states to the next segments in the next batch when using TransfoXLModel

Comments:
cc @sgugger 

The `Trainer` does not support the transformer XL model, so you should run a manual training loop that you can power by [Accelerate](https://github.com/huggingface/accelerate) if you need distributed training.

> The `Trainer` does not support the transformer XL model, so you should run a manual training loop that you can power by [Accelerate](https://github.com/huggingface/accelerate) if you need distributed training.

Thanks
Replace assertion with ValueError exception
Author: ddrm86
Pull request: yes

# What does this PR do?

Replaces the assertion in generation_flax_logits_process.py with a ValueError exception.

Contributes towards fixing issue #12789

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
Replace assert statements with exceptions
Author: ddrm86
Pull request: yes

# What does this PR do?

Replaces the assertions in file_utils.py used in class ModelOutput for safety and consistency checks with ValueError exceptions.

Contributes towards fixing issue #12789


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
Remove wrong model_args supplied
Author: qqaatw
Pull request: yes

# What does this PR do?

`PretrainedConfig.from_pretrained` method doesn't accept positional arguments, while the `PreTrainedModel.from_pretrained` supplies `model_args` as positional arguments to it when a config is not provided.

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@LysandreJik @sgugger 

Comments:
The failure seems not related to this PR.

@sgugger Fixed, thanks for pointing it out.

There seems to be an issue with the code quality issues! Do you mind fixing these by running the following at the root of your clone?

```
pip install -e .[quality]
make fixup
```

@LysandreJik the tests are passing. Let's merge it?

Thanks again!
Fixed horizon_length for PPLM
Author: jacksukk
Pull request: yes

# What does this PR do?
The current code can only run at horizon_length=1.
However, It's not applicable if we want to run at horizon_length > 1 since the curr_probs is not updated in the for_loop.
This PR is to update the curr_probs in the horizon_length in the for_loop.

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


Comments:
@sgugger sorry for bothering you, could you help me review my PR?
Hope to see your review, thanks so much!

Hi @jacksukk, thanks for your PR but as mentioned in the [README](https://github.com/huggingface/transformers/tree/master/examples/research_projects) of the research projects, you need to tag the author of the project. We don't maintain that code ourselves.

Hi @sgugger, Thanks a lot for your reply! QQ

Let me tag @dathath, could you help to review my code? 
Sorry for bothering and hope to see your comment. Thanks!

Hi @jacksukk,
Thanks for the ping! I've seen this, and will review shortly.

Thanks a lot both!

Thanks a lot! I'm a big fan of both huggingface and PPLM. 
I'm delighted to have some commit here!
Add the SEW and SEW-D speech models
Author: anton-l
Pull request: yes

# What does this PR do?

This PR adds the SEW and SEW-D model from the paper "[Performance-Efficiency Trade-offs in Unsupervised Pre-training for Speech Recognition](https://arxiv.org/abs/2109.06870)" 

Source of the models: https://github.com/asappresearch/sew/

* **SEW** is based on Wav2Vec2, but with time frame downsampling and upsampling around the transformer layers.
* **SEW-D** replaces the transformer layers in SEW with a DeBERTa-v2 encoder.

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in this PR.

## TODO
- [x] model docs
- [x] checkpoints conversion
- [ ] fintetuned CTC checkpoints?

Comments:
The VQ pretraining modules aren't ported yet. After #13877 is merged they'll be added in a separate PR.

Let's try to get this PR merged by Thursday/Friday - anything I can help with? :-)

@patrickvonplaten in the end I removed the `feature_projection` if-else and left the modules only in SEW-D.
The checkpoints are all uploaded now :tada: 
https://huggingface.co/models?other=sew
https://huggingface.co/models?other=sew-d
minor issues in modeling_clip.py
Author: ydshieh
Pull request: no

While working on the PR [Add TFCLIPModel](https://github.com/huggingface/transformers/pull/13967), I observed a few minor issues in [modeling_clip.py](https://github.com/huggingface/transformers/blob/master/src/transformers/models/clip/modeling_clip.py):

1. Miss `@dataclass` decorators before https://github.com/huggingface/transformers/blob/d5b82bb70c2e8c4b184a6f2a7d1c91d7fd156956/src/transformers/models/clip/modeling_clip.py#L74
This causes `None` elements included in the tuple form. 
2. typo in `CLIPVisionTransformer.pre_layrnorm`
https://github.com/huggingface/transformers/blob/d5b82bb70c2e8c4b184a6f2a7d1c91d7fd156956/src/transformers/models/clip/modeling_clip.py#L738
3. In
https://github.com/huggingface/transformers/blob/d5b82bb70c2e8c4b184a6f2a7d1c91d7fd156956/src/transformers/models/clip/modeling_clip.py#L291
- `layer_head_mask` not in the arguments
- shape of `hidden_states` is incorrect, I think
4. https://github.com/huggingface/transformers/blob/d5b82bb70c2e8c4b184a6f2a7d1c91d7fd156956/src/transformers/models/clip/modeling_clip.py#L500
 I am not sure why there is  `embed_tokens` here.
5. https://github.com/huggingface/transformers/blob/d5b82bb70c2e8c4b184a6f2a7d1c91d7fd156956/src/transformers/models/clip/modeling_clip.py#L520
Here `inputs_embeds` is a required argument instead of being `optional`.

6. There are a few places `model(...)` in `test_modeling_clip.py` without being inside `with torch.no_grad():`.

## Who can help
@patil-suraj 



Comments:
Hi @ydshieh !
Wow,  thank you for finding these issues 😄 

1. Yes it should have `@dataclass` decorator

2. Unfortunately we can't fix the typo now because if we change it we won't be able to load the weights 😕 

3.  `layer_head_mask` is not implemented for CLIP. And yes `hidden_states` shape should be `(bs, seq_len, hidden_size)`

4. `embed_tokens` shouldn't be there

5.  `inputs_embeds` is required because the encoder is used by both the vision and text modules, where we compute the embeddings and then pass that to the encoder. This is because the encoder is similar for both modalities.

6. Argg, this should be fixed.

About 5, yes, but on the doc, it's mentioned as optional. That's what I mean.

About 2, I agree. So I am going to use the same name for TFCLIP.
Don't duplicate the elements in dir
Author: sgugger
Pull request: yes

# What does this PR do?

There is a tiny bug in the way the `__dir__` method of our lazy modules is implemented. To see it run:

```
import transformers
dir(transformers.models.bert)
```

which will give a list ending with
```
[...,
 'configuration_bert',
 'configuration_bert',
 'load_tf_weights_in_bert',
 'modeling_bert',
 'modeling_flax_bert',
 'modeling_tf_bert',
 'tokenization_bert',
 'tokenization_bert',
 'tokenization_bert_fast'
]
```
You can already see some elements are duplicated, and accessing some other submodules (like the modeling ones) will grow the list of duplicates.

This comes because we always add all the submodules to the `__dir__` even if they are already there (the fact they are there or not depends on whether some element of the submodule was accessed to, because of the lazy init).

This PR fixes that.

Thanks to @NielsRogge for reporting the bug!

Comments:
Replace assertions with ValueError exceptions
Author: ddrm86
Pull request: yes

# What does this PR do?

Replaces the assertions in generation_logits_process.py with ValueError exceptions.

Contributes towards fixing issue #12789

## Before submitting
- [] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
Can someone please help me understand this failed test (test_push_to_hub)? I cannot see how my changes can affect this...

Anyways, now with an explicit length check using len it works.
T5
Author: NawressMekki
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform:
- Python version:
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @Rocketknight1

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1.
2.
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Add `tokenizer_max_length` to `cardiffnlp/twitter-roberta-base-sentiment`
Author: philschmid
Pull request: no

[cardiffnlp/twitter-roberta-base-sentiment](https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment) is one of the most downloaded `text-classification` models on the hub. 
But there is a small little bug, when using the model with `seq_len` > 515, see image
![image](https://user-images.githubusercontent.com/32632186/132306231-60ba797c-baf6-434d-bcf7-0229f2e1cd80.png)

To fix this we would need to add `tokenizer_max_length` attribute in a `tokenizer_config.json` on the hub for that model. 
@cardiffnlp would it be okay for you to add it? I am happy to make the change. 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Can I use input embeds to generate text without input ids
Author: LemonQC
Pull request: no

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

## Your contribution

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->


Comments:
this might help #10599

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.