noamaanMulla-03/transformers-issues huggingface dataset:
🐛 `model_init` fails when its a partially evaluated funtion.
Author: Guillem96
Pull request: no

https://github.com/huggingface/transformers/blob/65e27215ba991450e30aac1bf06f7f4e889e77fb/src/transformers/trainer.py#L908

Inspect module usage here does not take into account the cases when the user provides a partially evaluated function, causing an exception.

Since the issue is related to the `trainer` API I tag you @sgugger 

Example:

```python
import functools
import inspect

checkpoint = "..."
fn = lambda: AutoModel.from_pretrained(checkpoint )
print(len(inspect.signature(fn).parameters))
# Outpus: 0, then no trial expected everything works fine 

def fn1(model_base_checkpoint):
  return AutoModel.from_pretrained(model_base_checkpoint)

model_init_fn = functools.partial(fn1, model_base_checkpoint=checkpoint )
print(len(inspect.signature(fn).parameters))
# Outputs 1, then the call_model_init tries to pass the ray or optuna trial which results in
# model_init() got multiple values for argument 'model_base_checkpoint' exception
```

Comments:
I can reproduce. I would argue that this is more a bug in Python than the `Trainer` but will try to find a way to fix this. In the meantime, you should avoid using partials for `model_init` :-)

Thanks for the quick answer! I'll avoid using partials for now 👍🏼

Should be fixed in the PR above!
[Examples][Flax] AttributeError: 'DataTrainingArguments' object has no attribute 'test_file'
Author: bhadreshpsavani
Pull request: no

## Description

While running run_summarization_flax.py with local files, currently we have only two DataTrainingArguments, one for training and another for validation file still we are validating test_file which is producing an error
```
Traceback (most recent call last):
  File "transformers/examples/flax/summarization/run_summarization_flax.py", line 808, in <module>
    main()
  File "transformers/examples/flax/summarization/run_summarization_flax.py", line 352, in main
    if data_args.test_file is not None:
AttributeError: 'DataTrainingArguments' object has no attribute 'test_file'
```

## Environment info
- `transformers` version: 4.9.0(master branch)
- Platform: TPU VM
- Python version: 3.9
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:



### Who can help
@sgugger, @patil-suraj

### Possible Fix:
Either we can add test_file argument or remove test file validation section https://github.com/huggingface/transformers/blob/7d6285a921a23c06169e2d90c94faa0d92d00d78/examples/flax/summarization/run_summarization_flax.py#L352-L354



Comments:
[Examples][Flax] added test file in summarization example 
Author: bhadreshpsavani
Pull request: yes

# What does this PR do?

Fixes #12527

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?
@sgugger, @patil-suraj


Comments:
[Flax]Not able to Run Hugging Face GPT2 model for jax on TPU's
Author: vivekvkashyap
Pull request: no

 Hi,Trying to do FlaxGPT2ForMultipleChoice.I am trying to run GPT2 using hugging face for TPU's.It is showing tuple out of index.But when i run it in CPU there is no such error.Even when i run a simple model without any code of mine it is also behaving the same in TPU.


![image](https://user-images.githubusercontent.com/58116635/124922271-f5a3ff00-e016-11eb-8679-3258fc3727e9.png)


This below example is for a basic model 


![image](https://user-images.githubusercontent.com/58116635/124922511-313ec900-e017-11eb-9d2c-7473fe8b21af.png)
To Reproduce
This is the colab notebook:
https://colab.research.google.com/drive/1h8CeTM5NUpHeS1oGHONX1YbwtGHf3nyU?usp=sharing


Comments:
@patrickvonplaten 


Hey @vivekvkashyap,
instead of putting a screenshot here - could you maybe copy paste the link to your google colab instead so that we can reproduce the error?

Thank you!

@patrickvonplaten i have added the  colab notebook in the section To Reproduce


@patil-suraj


Similar to #12578 - we are working on it :-)
[flax]fix jax array type check
Author: patil-suraj
Pull request: yes

# What does this PR do?

Fixes #12584, #12578

On colab the `ModelOutput` class is returning empty tuples for jax arrays. This is because on colab TPU the type of jax array is `jax.interpreters.xla._DeviceArray` and the `is_tensor` function here

https://github.com/huggingface/transformers/blob/2dd9440d0835782e41ae415a68e71fd15051c428/src/transformers/file_utils.py#L1796-L1798

expects `jaxlib.xla_extension.DeviceArray` or `jax.core.Tracer`. If the first argument is an array, the `is_tensor` returns `None` in which case the  `ModelOutput` class expects the first argument to be a key-value container which is not the case here. So at the end, everything becomes `None` and the `ModelOutput` returns an empty tuple.

Instead the `jnp.ndarray` type check works for  jax array types `jaxlib.xla_extension.DeviceArray`, `jax.interpreters.xla._DeviceArray`  and also the `ShardedDeviceArray`



Comments:
`TFHubertModelTest.test_model_from_pretrained` is failing
Author: LysandreJik
Pull request: no

The `TFHubertModelTest.test_model_from_pretrained` is failing because the TensorFlow checkpoint isn't available under `facebook/hubert-base-ls960`.

Same with `TFHubertRobustModelTest.test_model_from_pretrained`

Comments:
Added in [`cbd73bd#huggingface.co`](https://huggingface.co/facebook/hubert-base-ls960/commit/cbd73bd518689ab98a9370c74165f4875ccd97d5)
Allow registerable Components
Author: david-waterworth
Pull request: no

# 🚀 Feature request

Currently, the Auto* classes use hardcoded mappings from model_name to class. This means they cannot be used to extend transformers with custom implementations, which in turn makes it difficult to perform experiments for example using alternate tokenisation particularly integration with other frameworks such as Allen NLP.

I have a custom tokeniser that is based on the tokenizers library. Since it uses a custom python pre-tokenisation step it cannot be serialised to the tokenizer.json file. To get around this I created a custom class that builds the pipeline on the fly. i.e.

```
class CustomUnigramTokenizer(PreTrainedTokenizerFast,FromParams):
    def __init__(
        self,
        tokenizer_file:str=None,
        bos_token:str="
",
        eos_token:str="
",
        sep_token:str="
",
        cls_token:str="
",
        unk_token:str="<unk>",
        pad_token:str="<pad>",
        mask_token:str="<mask>",
        **kwargs
    ):
        super().__init__(
            tokenizer_file=tokenizer_file,
            bos_token=bos_token,
            eos_token=eos_token,
            unk_token=unk_token,
            sep_token=sep_token,
            cls_token=cls_token,
            pad_token=pad_token,
            mask_token=mask_token,
            **kwargs,
        )
        self.backend_tokenizer.pre_tokenizer = self._pre_tokenizer()
```

The issue is I have to use `CustomUnigramTokenizer.from_pretrained()` rather than `AutoTokenizer.from_pretrained()`

Most frameworks, i.e. the trainers in the transformers library, or the allennlp library use `AutoTokenizer.from_pretrained()`

So it would be nice if I could register my implementation, i.e.

AutoTokenizer.register(model_name="my_custom_model", CustomUnigramTokenizer)

Allowing AutoTokenizer.from_pretrained("/path_to_my_custom_tokeniser")

I'm assuming I have a config.json in /path_to_my_custom_tokeniser with model_name="my_custom_model"

Even better would be an allennlp registration system, i.e.

```
PreTrainedTokenizer.register("my_custom_model")
class CustomUnigramTokenizer(PreTrainedTokenizerFast,FromParams):
    def __init__():
```

## Motivation

It's quite difficult to extend transformers and experiment with different configurations.

Comments:
Similar question to https://github.com/huggingface/transformers/issues/10256, and it is indeed a nice proposal!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
EOFError("No valid references for a sentence!") for run_translation example
Author: puraminy
Pull request: no

I tried to apply `run_translation` on a dataset I created. I tried to get sure there is no blank target. 

example of test file:
`
{"data": [{"translation": {"en": "You're quite impatient with the rest of humanity, in fact.", "fa": "ﺩﺭ ﻭﺎﻘﻋ ﺩﺭ ﻢﻗﺎﺒﻟ ﺱﺎﯾﺭ ﺎﻨﺳﺎﻧ<200c>ﻫﺍ ﮎﺎﻣﻻ ﺐﯾ<200c>ﺣﻮﺼﻠﻫ ﻪﺴﺘﯾﺩ."}}, {"translation": {"en": "Now despite this history of distrust, I still believe that indigenous people can benefit from genetic research.", "fa": "ﺡﺍﻻ ﺏﺍ ﻮﺟﻭﺩ ﺎﯿﻧ ﺐﯾ ﺎﻌﺘﻣﺍﺪﯾ ﺕﺍﺮﯿﺨﯾ ﻢﻧ ﻪﻧﻭﺯ ﻢﻌﺘﻗﺪﻣ ﻡﺭﺪﻣﺎﻧ ﺏﻮﻤﯾ ﻢﯾ ﺕﻭﺎﻨﻧﺩ ﺍﺯ ﺖﺤﻘﯿﻗﺎﺗ ﮋﻨﺘﯿﮐ ﺱﻭﺩ ﺐﺑﺮﻧﺩ."}},
`
However I get this error:

```
File "run_translation.py", line 496, in compute_metrics
    result = metric.compute(predictions=decoded_preds, references=decoded_labels)
  File "/home/pouramini/miniconda3/lib/python3.7/site-packages/datasets/metric.py", line 40$
, in compute
    output = self._compute(predictions=predictions, references=references, **kwargs)
  File "/home/pouramini/.cache/huggingface/modules/datasets_modules/metrics/sacrebleu/4dba4$
29caa3766d885f0b9cde070fedb22ac3190c264a6454b8ea6703ddd466/sacrebleu.py", line 128, in _com$
ute
    use_effective_order=use_effective_order,
  File "/home/pouramini/miniconda3/lib/python3.7/site-packages/sacrebleu/compat.py", line 3$
, in corpus_bleu
    sys_stream, ref_streams, use_effective_order=use_effective_order)
  File "/home/pouramini/miniconda3/lib/python3.7/site-packages/sacrebleu/metrics/bleu.py", $
ine 286, in corpus_score
    raise EOFError("No valid references for a sentence!")
EOFError: No valid references for a sentence!

```

@patil-suraj @patrickvonplaten @lhoestq 

Comments:
Hi @puraminy 

Could you debug the `compute_metrics` function? Maybe print/log a few `decoded_labels` that are used as references. Bit hard to guess without running the script.

@patil-suraj  

I actually did, and the problem is that the predictions are in English than Persian (the target language)! I consider it a serious bug and I posted more details in:

https://github.com/huggingface/transformers/issues/12010

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Add loss reduction parameter in forward() method
Author: marekrydlewski
Pull request: no

# Add loss reduction parameter in forward() method

Add possibility to choose reduction method in loss functions used in `forward` method in most of the models (like all BERT based, `BertForMaskedLM`, `BigBirdForMaskedLM` etc.). Currently, models use losses without the possibility to pass additional parameters.

From PyTorch docs [nn.CrossEntropyLoss ](https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html):

> reduction (string, optional) – Specifies the reduction to apply to the output: 'none' | 'mean' | 'sum'. 'none': no reduction will be applied, 'mean': the weighted mean of the output is taken, 'sum': the output will be summed. Note: size_average and reduce are in the process of being deprecated, and in the meantime, specifying either of those two args will override reduction. Default: 'mean'

## Motivation

Especially in models like `BertForMaskedLM` passing a reduction like `none` instead of the default `mean` can be very handy to check individual losses for tokens.

In our project, we handle this by creating our own class inheriting from hugginface model and overwriting `forward()` method.

## Your contribution

A quick demonstration of the idea on the example of base `Bert` models: 
https://github.com/marekrydlewski/transformers/commit/83994b12085b3187f0b80fbb9a6d4a7b5e4bc8de

There are no updated docs, it's just a demonstration.

Then it can be used like:
```
model = BertForMaskedLM(...)
output = model(..., loss_reduction='none')
```


Comments:
This has been asked before, but I don't think this is on the roadmap. If you want to use a different loss reduction, you can easily overwrite the model and plug in your custom loss function. See also https://github.com/huggingface/transformers/issues/9625#issuecomment-762167788 and https://github.com/huggingface/transformers/issues/7024#issue-696684075. 

Otherwise, the library would become a bit cluttered with all kinds of custom parameters, so simplicity is favored.



This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Pickle auto models
Author: sgugger
Pull request: yes

# What does this PR do?

The auto-generated classes for the Auto models are not picklable, because they are dynamically generated (so pickle can't trace them properly). This PR changes a little bit the way to create the Auto classes in each of their modeling files like proper classes, then update them to add the right methods. As a result, the auto classes are now picklable.

Fixes #12621

Comments:
GPT2: discrepancy between `inputs_embeds` and `input_ids` when the input sentence has length = 1 
Author: danyaljj
Pull request: no

Calling GPT2 with `inputs_embeds` vs calling it with `input_ids` leads to different outcomes when the input sentence has length 1. See the script to reproduce the issue. 

## Environment info
- `transformers` version: tried 4.2.1 and 4.8.2 
- Platform: MacOS and Ubuntu 
- Python version: 3.7 
- PyTorch version (GPU?): 1.9.0
- Using GPU in script?: tried GPU and CPU 
- Using distributed or parallel set-up in script?: Nope 

### Who can help
@patrickvonplaten, @LysandreJik

## Information
Model I am using (Bert, XLNet ...): GPT2 


## To reproduce

```python 
model_name = 'gpt2'
print(" ==> Loading the models . . . ")
model = GPT2LMHeadModel.from_pretrained(model_name, output_hidden_states=True)
model.to(device)
model.eval()
tokenizer = GPT2Tokenizer.from_pretrained(model_name)

batch_size = 5
prefix_length = 1
for sentence_len in range(10, 0, -1):
    sentence_str = " ".join(["good"] * sentence_len)
    input_ids = tokenizer([sentence_str] * batch_size, return_tensors='pt')['input_ids'].to(device)
    inputs_embeds = model.transformer.wte(input_ids).squeeze()

    out1 = model(inputs_embeds=inputs_embeds)
    out2 = model(input_ids=input_ids)

    diff = torch.abs(torch.mean(out1.logits - out2.logits))
    print(f" - sentence length: {sentence_len} <-> {diff}")
```

which gives me the following: 
```
 - sentence length: 10 <-> 0.0
 - sentence length: 9 <-> 0.0
 - sentence length: 8 <-> 0.0
 - sentence length: 7 <-> 0.0
 - sentence length: 6 <-> 0.0
 - sentence length: 5 <-> 0.0
 - sentence length: 4 <-> 0.0
 - sentence length: 3 <-> 0.0
 - sentence length: 2 <-> 0.0
 - sentence length: 1 <-> 30.124570846557617
```

The last line is the surprise: when the input sentence has length == 1, `inputs_embeds` and `input_ids` lead to different outcomes. 

FYI @qkaren

Comments:
Hello! I think the issue here lies in this line: `inputs_embeds = model.transformer.wte(input_ids).squeeze()`. You're calling `squeeze` on the inputs_embeds, which are of dimension `(5, 1, 768)`, which correspond to `(batch_size, sequence_length, hidden_size)`. Unfortunately, you're squeezing out the sequence length dimension (which is always required), which the model needs in order to construct the attention mask.

Remove the `.squeeze` or specify which dimension it should squeeze and you should be fine.
Fixed docs
Author: KickItLikeShika
Pull request: yes



- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
[WIP] Patch BigBird tokenization test
Author: LysandreJik
Pull request: yes

This patches the BigBird integration test.

The core of the issue is that the `[MASK]` token is an `AddedToken` with `lstrip=True`. It, therefore, gobbles up the spaces on the left without getting a sentence piece underline.

Therefore, when decoding, the internal sentence piece tokenizer is unaware that it should add a space in front of the `[MASK]` token.

However, the original tokenizer does correctly decode with the space, so I believe there's an issue with our implementation.

@vasudevgupta7 do you know of the difference between the two implementations? Also cc @n1t0 and @SaulLu 

Do not merge this as this isn't the correct fix :)

Comments:
Hey @LysandreJik,

Even original tokenizer is not introducing space before `[MASK]`, so I think tokenizer is alright & the test is wrong instead.

```
wget https://huggingface.co/google/bigbird-roberta-base/resolve/main/spiece.model
s = spm.SentencePieceProcessor(model_file='spiece.model')
s.decode([7434, 9894, 67, 9894, 7434])
```


Great, then merging this! Thanks @vasudevgupta7
Nothing
Author: vivekvkashyap
Pull request: no

## To reproduce

Steps to reproduce the behavior:

1.
2.
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     ```python
      model = FlaxGPT2ForMultipleChoice.from_pretrained('gpt2')
      ```
  Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->


Comments:
the best way to cut the upper layers
Author: cherepanovic
Pull request: no

Salut, 

what would be the best way to cut the upper layers of transformers (from 12 cutting 5 upper layer results 7 layers model for the use)?

Best regards

Comments:
Hi, don't know which model you are using so I can't answer precisely but here is the general workflow:
1. load the relevant pretrained configuration with `config = config_class.from_pretrained('your-model-of-interest')`
2. Reduce the number of layers in the configuration with for example: `config.num_hidden_layers = 5` (here you have to check the correct attribute for your model).
3. Use the modified config to build and instantiate your model: `model = model_class.from_pretrained('your-model-of-interest', config=config)`.

Pretty easy, isn't it?

>>Pretty easy, isn't it?

indeed!


```
config = XLNetConfig.from_pretrained('xlnet-base-cased')
config.num_hidden_layers = 3
```
raised this error

`AttributeError: can't set attribute`

config.n_layer = 3 does it work

> Hi, don't know which model you are using so I can't answer precisely but here is the general workflow:
> 
>     1. load the relevant pretrained configuration with `config = config_class.from_pretrained('your-model-of-interest')`
> 
>     2. Reduce the number of layers in the configuration with for example: `config.num_hidden_layers = 5` (here you have to check the correct attribute for your model).
> 
>     3. Use the modified config to build and instantiate your model: `model = model_class.from_pretrained('your-model-of-interest', config=config)`.
> 
> 
> Pretty easy, isn't it?

I assume this gives the upper 5 layers. Is there a way to get the lower 5 layers ? 

it's kind of annoying and non-intuitive, but @cherepanovic, the reason why you're seeing this message is that there are several parameters in many of the configs that are not parameters, but properties. I suppose the authors did this to emphasize that they should not be changed after the model is initialized? Not sure. But here they are:
```
    @property
    def max_position_embeddings(self):
        return self.n_positions

    @property
    def hidden_size(self):
        return self.n_embd

    @property
    def num_attention_heads(self):
        return self.n_head

    @property
    def num_hidden_layers(self):
        return self.n_layer
```

You can change these, after initialization, by referring to the actual parameters that the property returns.

IMO it certainly isn't "pretty easy", as doubly-named parameters/properties is kinda poor practice, and it would've been easy from a coding perspective to put getters and setters in there as well.