noamaanMulla-03/transformers-issues huggingface dataset:
Using Bart with input_embeds to generate text without input_id return error
Author: Shj451148969
Pull request: no

This is my code
 ```
           generated_ids = plm.generate(input_ids=None,
                                         inputs_embeds=student_embeddings,
                                         attention_mask=node_masks,
                                         num_beams=4,
                                         max_length=config["max_seq_length"],
                                         early_stopping=True)
```

I want to generate text with inputs_embeds, but in generation_utils.py have problem.
The function with `_prepare_decoder_input_ids_for_generation` use input id, but I put the input_ids is None.

```
    def _prepare_decoder_input_ids_for_generation(
        self, input_ids: torch.LongTensor, decoder_start_token_id: int = None, bos_token_id: int = None
    ) -> torch.LongTensor:
        decoder_start_token_id = self._get_decoder_start_token_id(decoder_start_token_id, bos_token_id)
        decoder_input_ids = (
            torch.ones((input_ids.shape[0], 1), dtype=torch.long, device=input_ids.device) * decoder_start_token_id
        )
        return decoder_input_ids
```
```
            # add encoder_outputs to model_kwargs
            model_kwargs = self._prepare_encoder_decoder_kwargs_for_generation(input_ids, model_kwargs)

            # set input_ids as decoder_input_ids
            if "decoder_input_ids" in model_kwargs:
                input_ids = model_kwargs.pop("decoder_input_ids")
            else:
                input_ids = self._prepare_decoder_input_ids_for_generation(
                    input_ids, decoder_start_token_id=decoder_start_token_id, bos_token_id=bos_token_id
                )
```
So, what should I do ?

Comments:
@Shj451148969 
hello, I also face the same problem with `T5ForConditionalGeneration`.

I found that the error doesn't occur if I pass `decoder_input_ids` consisting of `pad_token_id` to the `generate` as start tokens.

https://github.com/huggingface/transformers/issues/12218


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Keras metric callback
Author: Rocketknight1
Pull request: yes

Hey, here's the unfinished Keras metric PR, co-authored with @merveenoyan!

Still to do:

1) The outputs from predict/generate are not being concatenated properly, this will be quick to fix! (Edit: Done)
2) I haven't tested prediction with generate at all. `generate()` is still very slow, but that's a separate PR :fearful: 
3) More broad testing with multiple metric functions.

Comments:
I feel like it makes sense to test on three notebooks. (could be overkill so you decide)
- Token classification
- Translation
- Masked language modeling
If you approve I’ll test for these ones. I wonder how it would look like with multiple metrics so in one of those I could test with multiple metrics (prolly token classification one)
@Rocketknight1 

@LysandreJik Absolutely yes, that's getting added!
Convert rst files
Author: sgugger
Pull request: yes

# What does this PR do?

This PR converts all remaining rst files to mdx and adapts the templates as well as the add_new_model command. The script for checking the table in the serialization page also needs a slight update.

Comments:
Merging and lookin if the docs are alright :-) 

Fantastic, thanks for taking care of the rest @sgugger!
Fix `AttributeError` from `PreTrainedTokenizerFast.decoder`
Author: aphedges
Pull request: yes

# What does this PR do?

Calling the `decoder` attribute for the T5 fast tokenizer raises an error because it tries accessing the non-existent `_tokenizer` field of `tokenizers.Tokenizer`. In `PreTrainedTokenizerFast.decoder`, `self._tokenizer._tokenizer.decoder` is executed, which contains a probably unintentional repeat of `_tokenizer`.

Before:

```python
>>> from transformers import AutoTokenizer
>>> tokenizer = AutoTokenizer.from_pretrained('t5-base')
>>> type(tokenizer)
<class 'transformers.models.t5.tokenization_t5_fast.T5TokenizerFast'>
>>> tokenizer.decoder
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "transformers/src/transformers/tokenization_utils_fast.py", line 183, in decoder
    return self._tokenizer._tokenizer.decoder
AttributeError: 'tokenizers.Tokenizer' object has no attribute '_tokenizer'
```

After:

```python
>>> from transformers import AutoTokenizer
>>> tokenizer = AutoTokenizer.from_pretrained('t5-base')
>>> type(tokenizer)
<class 'transformers.models.t5.tokenization_t5_fast.T5TokenizerFast'>
>>> tokenizer.decoder
<tokenizers.decoders.Metaspace object at 0x190c817b0>
```

I'm not sure how this hasn't been caught before. The line has not been touched since June 2020, and the `decoder` property is presumably being used somewhere. It's hard to tell if it has any usages in the repository because PyCharm finds a large number of unrelated fields named `decoder` when I search.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

tokenizers: @n1t0, @LysandreJik

Comments:
Pinging @n1t0, @LysandreJik.
Add 'with torch.no_grad()' to ALBERT integration test forward pass
Author: h-holm
Pull request: yes

# What does this PR do?

As proposed in #14642, this encapsulates the forward pass in the ALBERT integration test with "with torch.no_grad():".

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@NielsRogge 

Comments:
TrainingArguments docstring typo fix
Author: navalnica
Pull request: yes

# What does this PR do?

This PR fixes `TrainingArguments` docstring, particulary the part that describes `ignore_data_skip` parameter. 

No related issues found.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger 

Comments:
No the docstring is correct: it is when setting that argument to `True` that the data is ignored and training begins faster.

oh, now I see the logic.
still, the phrasing is a bit odd because parameter name `ignore_data_skip` implies the opposite of the first phrase in the description: "whether or not **to skip** the epochs and batches" (instead of "whether or not **to ignore the skip** the epochs and batches"). I think it could be slightly rephrased to improve readability.
thanks anyway

Feel free to amend your PR to rephrase then :-)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Where do I find the class documentation
Author: avnerbarr
Pull request: no
Labels: name: Documentation

Stupid question:

The website is great and all, but where do I find the FULL code documentation for the APIs (i.e. pydoc or something)? For instance any time a class is "highlighted" , i.e. `model` I would expect to be able to click on it and see the full set of functions/properties it provides.

Also, navigating the website is cumbersome. The left menu doesn't scroll to the section you are currently visiting, the additional nested sections (headers) for that subject should always be visible (and have a always visible "+" icon in the menu pane) to be easier to navigate to. I find myself clicking away (since I might have forgotten under what something is nested under) , going back and forth etc. because of that. 

Thanks!






Comments:
Hy @avnerbarr, the documentation is undergoing a large refactor which should be out in a couple of days, and which should hopefully resolve most of your pain points, stay tuned.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Hey @avnerbarr, do you have any feedback of the new documentation frontend?

seems much better. I'll update once I start working with it 👍
[Tests] Update speech diarization and WavLM tolerances
Author: anton-l
Pull request: yes

# What does this PR do?

Turns out the difference in `nn.GroupNorm` between torch `1.9` and `1.10` (https://github.com/pytorch/pytorch/issues/67907) is more noticeable than I initially expected. This PR updates the tolerances on some newer tests where the reference values were obtained on `1.10`. When we update the CI env to `1.10`, these can be safely reverted.


Comments:
`FAILED examples/pytorch/test_examples.py::ExamplesTests::test_run_image_classification` is unrelated, merging...
[doc] bug in docstring conversion
Author: stas00
Pull request: no

There is an issue of rendering items in a docstring (in some places)

Here is an example:
https://huggingface.co/docs/transformers/main_classes/logging#transformers.utils.logging.set_verbosity
 has a lose `<`

but I'm not sure where this one is coming from

It appears to be a messed up `<ul>` block:

```
Logging level, e.g., one of:</p>
<ul>
<li><code>transformers.logging.CRITICAL</code> or <code>transformers.logging.FATAL</code></li>
<li><code>transformers.logging.ERROR</code></li>
<li><code>transformers.logging.WARNING</code> or <code>transformers.logging.WARN</code></li>
<li><code>transformers.logging.INFO</code></li>
<li><code>transformers.logging.DEBUG</code></li>
<
							</span></span>
					</li></ul>
```

So the following chunk shouldn't be there:
```
<
							</span></span>
					</li>
```

It's a systemic issue, here are more examples:
https://huggingface.co/docs/transformers/main_classes/data_collator#transformers.DataCollatorWithPadding
and several after it.

@sgugger 

Comments:
It seems to be happening every time we have a list in a parameter description. Filing an [issue](https://github.com/huggingface/doc-builder/issues/70) on `doc-builder` for this, Mishig will have a look when he's back from vacation :-)

Closing on this side as there is nothing to do in Transformers. The generated MDX is correct.
Consistent speech model input names for the Seq2SeqTrainer generate function
Author: OllieBroadhurst
Pull request: no

# 🚀 Feature request

Could we maybe have a consistent naming convention for speech models? So far we have:
- [`input_features`](https://huggingface.co/transformers/model_doc/speech_to_text.html#speech2textforconditionalgeneration)
- [`input_values`](https://huggingface.co/transformers/model_doc/wav2vec2.html#wav2vec2forctc)
- [`input_ids`](https://huggingface.co/transformers/model_doc/speech_to_text_2.html#speech2text2forcausallm)

From what I can tell, these are mostly the same for the purposes of how the `Seq2SeqTrainer` interprets them.

## Motivation

This would prevent the need for custom `Seq2SeqTrainer` classes and would make training more modular.

## Your contribution

A change in param names would do the trick but could break a lot of code. Alternatively adding the capability to accept different key values in the `generate`  function [here](https://github.com/huggingface/transformers/blob/41436d3dfb98e0d17f018db29790b65663358edf/examples/legacy/seq2seq/seq2seq_trainer.py#L219) would work too using a (clunky) mapping such as `INPUT_MAPPING_LABELS = {"input_features": "input_ids", "input_values": "input_ids", "input_ids": "input_ids"}`.


Comments:
Yes indeed. @patrickvonplaten 

Also, the `generate` method will be extended to also work with `pixel_values` (for models that generate text given an image). The general input of `generate` could then be called `inputs` (which could be `input_ids` for text, `input_features` for speech, `pixel_values` for images, etc.).

Using a mapping is probably the way to ensure backwards compatibility.

I can tackle this next week :-)

Related: https://github.com/huggingface/transformers/issues/14421

This is resolved now with https://github.com/huggingface/transformers/pull/14802/files no?

Hey there. Looks cool! One super edge case is `SpeechEncoderDecoderModel`.

For example, if we use a `Wav2Vec2Model` encoder and a `XLMRobertaForCausalLM` decoder, we would have `['input_ids', 'attention_mask']` when using either `processor.tokenizer.model_input_names`  for Wav2Vec2 or `tokenizer.model_input_names` for XLMRoBERTa, when in fact the model requires `input_values` in the `generate` function.

@patrickvonplaten the only way I can think of a truly generalized solution without breaking changes or mappings, is using `inspect` on the `forward` function:

```python

import inspect
from transformers import SpeechEncoderDecoderModel

model = SpeechEncoderDecoderModel.from_pretrained('facebook/s2t-wav2vec2-large-en-de')
list(inspect.signature(model.forward).parameters)[0]
```

For the above example that would return `input_values` (as opposed to using `Speech2TextProcessor.tokenizer.model_input_names` which would show `input_ids`). But like I say, super edge case and happy to leave it as is.

Great catch @OllieBroadhurst! I'll include a fix in https://github.com/huggingface/transformers/pull/14856 . Hope to have it merged by tomorrow
Better logic for getting tokenizer config in AutoTokenizer
Author: sgugger
Pull request: yes

# What does this PR do?

In this PR, we make the logic in `AutoTokenizer` a little bit better by checking the tokenizer config is in the list of files of the repo when trying to get it, instead of bluntly trying to load it. This makes the function fail early with a clear error message if the repo name passed contains a typo.

Comments:
Thanks!
Is it possible to use multiprocessing for pipelines?
Author: Weilin37
Pull request: no
Labels: name: wontfix

I am trying to use multiprocessing for pipelines, but it seems that it's not working. I think it's because the pipeline already uses multiprocessing features and so you can't have multiprocessing inception. Anyone able to get it to work?

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


I have the same the problem. When you close the issue does it mean that it is fixed? 

I have also met this problem, how can I solve this? Thanks！！！！

> I have also met this problem, how can I solve this? Thanks！！！！

I switched to using `nlp.pipe`, which is the built-in function for multiprocessing, instead of doing it by hand.
Automatic Mixed Precision Support for (some) Flax Transformers
Author: luyug
Pull request: no

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

[Mixed precision](https://arxiv.org/abs/1710.03740) can reduce memory usage and improve computation efficiency. Thanks to bfloat16, it is very straightforward on TPU. However, on GPU it requires type casting for certain operations. It will be great if automatic type casting for mixed precision is implemented in (some popular) Flax models so that they can run efficiently also on GPUs.

## Motivation

Ideally, we would want JAX/Flax Transformers to run easily and efficiently on both GPU and TPU. Having mixed precision will help speed up both training and inference by big margins. However, on GPU this requires both loss/gradient scaling and auto casting implemented. Currently auto casting is not supported by JAX or Flax (although gradient scaling is. )

JAX team said AMP should be external to JAX https://github.com/google/jax/issues/5257.
Flax has a milestone, which is of low priority for the moment https://github.com/google/flax/milestone/6.

My intuition, which may be wrong, is that since we are dealing with transformers, there should be a relatively smaller set operations, such as softmax, that needs special treatments compared to the [full set of ops in pytorch](https://pytorch.org/docs/stable/amp.html#autocast-op-reference).

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Why do I use XLMRobertaTokenizer and return an error on token_type_ids?
Author: wmathor
Pull request: no

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
```python
encoded_pair = self.tokenizer(sent_ko, sent_cn,
                                      padding='max_length',  # Pad to max_length
                                      truncation=True,       # Truncate to max_length
                                      max_length=self.maxlen,  
                                      return_tensors='pt')  # Return torch.Tensor objects
token_ids = encoded_pair['input_ids'].squeeze(0)
attn_masks = encoded_pair['attention_mask'].squeeze(0)
token_type_ids = encoded_pair['token_type_ids'].squeeze(0)
```
```
File "/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils_base.py", line 234, in __getitem__
    return self.data[item]
KeyError: 'token_type_ids'
```

Comments:
I also have this question
Facing Problems with RobertaForSequenceClassification.from_pretrained()
Author: aflah02
Pull request: no

I've been trying to run the code present here on new data - https://github.com/machelreid/lewis

In one of the steps there are the following lines of codes - 

```
    classifier = (
        RobertaForSequenceClassification.from_pretrained(
            f"{args.hf_dump}/pytorch_model.bin",
            config=f"{args.hf_dump}/config.json",
            output_attentions=True,
        )
        .half()
        .cuda()
        .eval()
    )
```

They are present inside https://github.com/machelreid/lewis/blob/master/get_synthesized_data.py

I get this error - 

``Traceback (most recent call last):
  File "get_synthesized_data.py", line 56, in <module>
    output_attentions=True,
  File "/home/aflah20082/yes/envs/lewis/lib/python3.7/site-packages/transformers/modeling_utils.py", line 1272, in from_pretrained
    **kwargs,
  File "/home/aflah20082/yes/envs/lewis/lib/python3.7/site-packages/transformers/configuration_utils.py", line 501, in from_pretrained
    config_dict, kwargs = cls.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/home/aflah20082/yes/envs/lewis/lib/python3.7/site-packages/transformers/configuration_utils.py", line 554, in get_config_dict
    local_files_only=local_files_only,
  File "/home/aflah20082/yes/envs/lewis/lib/python3.7/site-packages/transformers/configuration_utils.py", line 842, in get_configuration_file
    path_or_repo, revision=revision, use_auth_token=use_auth_token, local_files_only=local_files_only
  File "/home/aflah20082/yes/envs/lewis/lib/python3.7/site-packages/transformers/file_utils.py", line 1952, in get_list_of_files
    return list_repo_files(path_or_repo, revision=revision, token=token)
  File "/home/aflah20082/yes/envs/lewis/lib/python3.7/site-packages/huggingface_hub/hf_api.py", line 603, in list_repo_files
    repo_id, revision=revision, token=token, timeout=timeout
  File "/home/aflah20082/yes/envs/lewis/lib/python3.7/site-packages/huggingface_hub/hf_api.py", line 586, in model_info
    r.raise_for_status()
  File "/home/aflah20082/yes/envs/lewis/lib/python3.7/site-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/api/models/downloads/roberta-classifier/dataset-eval/config.json``

I can't seem to figure out why does it try to go to that link since the path was to a local file


Comments:
I figured out my mistake the config file hadn't been generated 
Fixed that now
Feature/fix slow test in mluke
Author: ryokan0123
Pull request: yes

# What does this PR do?

Fix slow tests in MLuke/Luke (https://github.com/huggingface/transformers/pull/14690).

* Replace tokenizers to dummy tokenizers in `MLukeTokenizerTest/LukeTokenizerTest`.
* Add a missing entry in toctree.

## Who can review?
@sgugger @NielsRogge


Comments:
Merging this :)
BERT outperformed XLNet
Author: kutayoncuyilmaz
Pull request: no

Hi,

I am doing a tweet sentiment classification (binary) project and I want to compare the F1 scores of XLNet and BERT. I expected XLNet to outperform BERT , however, BERT outperformed XLNet other permutation language models.I know it shouldn't be necessarily the case but I can't explain the reason. I used Huggingface transformers and pretrained model. Where should I look to understand the result?

dataset link : http://help.sentiment140.com/for-students

the reason why I think XLNet would outperform BERT : https://towardsdatascience.com/what-is-xlnet-and-why-it-outperforms-bert-8d8fce710335

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.