noamaanMulla-03/transformers-issues huggingface dataset:
KeyError when using AutoTokenizer for facebook/detr-resnet-*
Author: grassjelly
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.16.0
- Platform: Ubuntu 20.04
- Python version: 3.8.12
- PyTorch version (GPU?): 1.11.0
- Tensorflow version (GPU?): NA
- Using GPU in script?: NA 
- Using distributed or parallel set-up in script?: No

### Who can help
- DETR: @NielsRogge
- Tokenizers: @SaulLu
## Information
I'm following this tutorial https://huggingface.co/docs/transformers/serialization on how to export models to ONNX. Tr
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("facebook/detr-resnet-101")ying to export one for DETR but I can't proceed as I'm stuck with this error on AutoTokenizer:

```
Traceback (most recent call last):
  File "detr_config.py", line 2, in <module>
    tokenizer = AutoTokenizer.from_pretrained("facebook/detr-resnet-101")
  File "/home/juan/miniconda3/envs/pytorch/lib/python3.8/site-packages/transformers/models/auto/tokenization_auto.py", line 530, in from_pretrained
    tokenizer_class_py, tokenizer_class_fast = TOKENIZER_MAPPING[type(config)]
  File "/home/juan/miniconda3/envs/pytorch/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 565, in __getitem__
    raise KeyError(key)
KeyError: <class 'transformers.models.detr.configuration_detr.DetrConfig'>
```
Here's the snippet of code to reproduce the error:
```
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("facebook/detr-resnet-101")
```



Comments:
Hi,

DETR is a vision model, not a text model, hence it doesn't have a tokenizer, but a so-called feature extractor (useful for preparing images for the model). You can load it using the AutoFeatureExtractor API:

```
from transformers import AutoFeatureExtractor

feature_extractor = AutoFeatureExtractor.from_pretrained("facebook/detr-resnet-101")
```

thanks @NielsRogge . That worked for me.
[modeling_utils] better explanation of ignore keys
Author: stas00
Pull request: yes

Integrating the improved explanation of ignore keys by @sgugger at https://github.com/huggingface/transformers/issues/16719#issuecomment-1096878395 with some tweaks from myself.

It's still unclear whether they should include the base model prefix or not, but we can sort it out when https://github.com/huggingface/transformers/issues/16719 gets more clarity

@sgugger 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Nystromformer Pretrained model with longer sequence length 
Author: etrigger
Pull request: no

The released Nystromformer model only deals with 512 tokens. Will we release some pretrained models with max sequence length such as 1024, 2048 and 4096 mentioned in the paper.

@novice03 @NielsRogge 

Comments:
Hello, the model for larger sequence lengths might be released in a few weeks.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hi,

Nyströmformer trained on sequence length 1024 is available now: https://huggingface.co/uw-madison/nystromformer-1024

Closing this issue.
CI: setup-dependent pip cache
Author: gante
Pull request: yes

# What does this PR do?

This PR makes two changes to the way we cache our pip dependencies in the `add-model-like.yml` GH actions workflow:
1. The name of the cache depends on the hash of `setup.py`;
2. We do not restore the cache from partial name matches.

(this pattern exists in one of our CI files, `github-torch-hub.yml` , [here](https://github.com/huggingface/transformers/blob/main/.github/workflows/github-torch-hub.yml#L30))

Together, these changes will make us start from a fresh environment whenever we change `setup.py`. Having a stale cache was causing us to have dependency problems (e.g. [old, incompatible protobuf version](https://github.com/huggingface/transformers/runs/6007067240?check_suite_focus=true)), and potentially miss dependency issues from fresh installs.

If you agree, I will also port these changes to `model-templates.yml` and `update_metdata.yml`, which have the same pattern/issue. EDIT: ported.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Cool, porting the changes to the other two files as well then 👍 

@gante: to learn more from you: how  you figured out the cause for the error you mentioned:

https://github.com/huggingface/transformers/runs/6007067240?check_suite_focus=true

If it was me, I don't even know if I could figure this out!

@ydshieh It definitely helps that I had this exact issue (stale CI caches) in my previous role :) 

To pin the error to this issue, I reran the failing CI workflow locally, from a fresh virtual env. Since it ran without issues, I had a look at the `.yml` file, and saw that it had a cache for `pip`. Then I went on to see that `pip install -e .[dev]` was doing in the failing CI file, and I noticed that it had error messages due to incompatible package versions, which I did not have locally -- because an old version was cached.
Reduce Funnel PT/TF diff
Author: ydshieh
Pull request: yes

# What does this PR do?

Same as #15684, but on PT test side.

As mentioned in #15684, this is not a real bug in model. Just a setting in the test configuration.

## Comment

The change in `modeling_tf_funnel.py` is to address **a real issue** regarding **weight initialization**, see

https://github.com/huggingface/transformers/blob/15de7a010ddcdec0532b30d1eb6c28e7b314a6a9/src/transformers/models/funnel/configuration_funnel.py#L79-L84
and
https://github.com/huggingface/transformers/blob/15de7a010ddcdec0532b30d1eb6c28e7b314a6a9/src/transformers/models/funnel/modeling_funnel.py#L812-L814

**(But, this issue is not the cause of the large diff between PT/TF)**

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add defensive check for config num_labels and id2label
Author: sgugger
Pull request: yes

# What does this PR do?

As seen in #16600, there can be some unclear errors when the user tries to pass together an inconsistent `num_labels` and `id2label`. This PR addresses that with a clear error message.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Might I suggest to add an example in the error message? Especially for regression where num_labels=1, it might not be obvious to users what the id2label map has to look like. Suggestion:

```
f"You passed along `num_labels={kwargs['num_labels']}` with an incompatible ID to label map:
{kwargs['id2label']}. The id2label map should be a dictionary of ID (int) to label (str). E.g., for
num_labels=1 (regression), it should look like this: {0: "LABEL_0"},
even if you do not have any explicitly labelled data."
```

I really don't understand why you want to pass both, as setting `num_labels=1` will do exactly that.

> I really don't understand why you want to pass both, as setting `num_labels=1` will do exactly that.

Exactly, but this comes back to the original issue that I posted. Passing num_labels=1, id2label=None causes issues, because id2label=None overwrites the generated id2label map. So the only thing that works is:

```python
config = BertConfig.from_pretrained(model_name_or_path, num_labels=1, id2label=None)
config.num_labels = num_labels
```

or 

```python
config = BertConfig.from_pretrained(model_name_or_path, num_labels=1, id2label={0: "LABEL_0"})
```

Yet it is not obvious why

```python
config = BertConfig.from_pretrained(model_name_or_path, num_labels=1, id2label=None)
```

doesn't work even though you don't strictly need labels for a regression problem, and even though id2label=None is the default argument.

And yes, I am aware that _most users_ will not encounter this issue because they will not explicitly pass id2label=None, but that does not mean that it cannot happen. And if it does it should be made obvious to the user why something goes wrong. I often write code for different use-cases, and as my issue showed, you will encounter this issue if you need to write code for different num_labels/tasks dynamically. If users are not expected to write code like that, it doesn't hurt to tell them in the error message how they should write their code instead. 

You are right though that my message seemed to imply that they _have_ to provide an id2label map. Suggestion:

f"You passed along `num_labels={kwargs['num_labels']}` with an incompatible ID to label map:
{kwargs['id2label']}. If given (not required), the id2label map should be a dictionary of ID (int)
to label (str). Note that explicitly setting id2label to None may lead to unexpected errors. 
Instead, do not pass the id2label argument at all or pass a dummy id2label with the same len 
as num_labels."

I adapted the error message slightly to insist on removing one of the incompatible kwarg.
Add test suite for flaubert tokenizer
Author: SaulLu
Pull request: no

This is tragic, the flaubert tokenizer has no tests. :crying_cat_face: 

Details to accompany this issue are coming soon, including a template to add these missing tests. This will probably be a good first issue.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

included into a more global issue: #16627
Add self training code for text classification
Author: tuvuumass
Pull request: yes

This is an implementation of the self-training algorithm (without task augmentation) for classification tasks proposed in the [EMNLP 2021](https://2021.emnlp.org/) paper: [STraTA: Self-Training with Task Augmentation for Better Few-shot Learning](https://arxiv.org/abs/2109.06270).  For the original codebase, please check out https://github.com/google-research/google-research/tree/master/STraTA. Note that this code can be used as a tool for automatic data labeling.

The pull request includes a README.md file with detailed instructions on how to set up a virtual environment and install necessary packages. It also includes a demo `run.sh` on how to perform self-training with a BERT Base model on the SciTail science entailment dataset using 8 labeled examples per class.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

> Very nice, thanks a lot for adding this new example! Just to be sure, the empty strata file is intended? I didn't get why it's there.

Good catch, @sgugger. Just removed the empty strata file. Thanks!
Fixup no_trainer examples scripts and add more tests
Author: muellerzr
Pull request: yes
Labels: name: PyTorch

name: External

name: Examples

# Fixup `no_trainer` Examples and Bolster their tests

## What does this add?

This changes the logging behavior inside the `no_trainer` scripts, slightly changes how the initial configuration is stored, and adds tests for the tracking API.

## Who is it for?

Users of `transformers` who want to try out `Accelerate` quickly

## Why is this needed?

I was made aware that the scripts were laggy when it came to how logs were sent to weights and biases when using the `no_trainer` scripts, and this was due to the step being passed in as a parameter, causing a lag in when it gets uploaded.

To follow akin to the original Accelerate scripts, these are now passed in as a `"step"` parameter to the overall dictionary logged via `accelerate.log()`

`TensorBoard` also does not like when `Enum`'s are logged, so there is a manual adjustment rightr before saving the hyperparemeters to get the enum value from the LR Scheduler type. 

Finally, as `TensorBoard` is a test requirement, I added in tests for tracking inside the no_trainer tests, as `TensorBoard` is also how we test that behavior in the CI in Accelerate proper.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Raise an error instead of a warning when model files are not loaded correctly
Author: hasansalimkanmaz
Pull request: no

# 🚀 Feature request

Currently, when I initialize a model and if my pre-trained model files aren't fully matched with my model architecture, code silently logs the event and warns the user. I think it is better to have a flag to stop the training if model weights are not loaded as expected.

## Motivation

With the current implementation, user can train a model from scratch without knowing it if he/she doesn't look at the logs carefully (Even if he/she should look at it).


## Your contribution

I am open to work for this issue. If you have any idea about how to implement it, let me know. I can start working on this issue in the coming weeks (not for now).

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Today, I have encountered a bug in our codebase due to this issue. The model loading wasn't ok for some reason and the code just logged the warnings. I would like to get an error instead of a warning. I can't monitor thousands of models that are running in production for this particular warning message. Could you help me to work on this issue?
Fix batch size in evaluation loop
Author: sgugger
Pull request: yes

# What does this PR do?

The batch size used in the evaluation loop is wrong: it's using the per device batch size, which is different from the actual batch size when using DataParallel with more than one GPU. As a result, the `test_evaluate` test is failing for 2 GPUs (see #16716).

This PR fixes that.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix nightly CI Accelerate failures
Author: muellerzr
Pull request: yes
Labels: name: Tests

name: Examples

name: bug

# Fix Nightly CI Build failures for Accelerate examples

## What does this add?

Changes the `run_examples_torch_all` CircleCI workflow to also install Accelerate from git for the time being, until the next release is performed.

The CI was failing due to accelerate being tested with unreleased fixes and improvements

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_16783). All of your documentation changes will be reflected on that endpoint.
[deepspeed / m2m_100] make deepspeed zero-3 work with layerdrop
Author: stas00
Pull request: yes
Labels: name: DeepSpeed

Same as I had to fix in `wav2vec2` it looks that this fix should eventually go to all models that use `LayerDrop`. At least at the moment Deepspeed is not capable of randomly skipping layers, so this PR uses the same now well tested workaround I used in `wav2vec2`, where all layers always run when deepspeed zero-3 is detected, but the results are ignored if it was meant to be skipped.

https://github.com/huggingface/transformers/blob/69233cf03be5fbce0492f3997e139c4d05499e27/src/transformers/models/wav2vec2/modeling_wav2vec2.py#L817-L849

Perhaps one day Deepspeed will be able to randomly skip layers, at the moment the solution is not the most efficient one. I made a [request](https://github.com/microsoft/DeepSpeed/issues/1888).

When ZeRO-3 is not used the original code path is taken.

The test exercising this code path will be merged as part of this huge additional tests set PR https://github.com/huggingface/transformers/pull/12695 (it's been long overdue).

For posterity, the error for this issue will look something like:

```
RuntimeError: tracing error at step 42: expected the next 2 parameters in the parameter fetch queue to be 
({'id': 26, 'status': 'AVAILABLE', 'numel': 1024, 'ds_numel': 1024, 'shape': (1024,), 'ds_shape': (1024,), 'requires_grad': True, 'grad_shape': None, 'persist': True, 'active_sub_modules': {24}}, {'id': 27, 'status': 'AVAILABLE', 'numel': 1024, 'ds_numel': 1024, 'shape': (1024,), 'ds_shape': (1024,), 'requires_grad': True, 'grad_shape': None, 'persist': True, 'active_sub_modules': {24}}) 
but got 
({'id': 115, 'status': 'NOT_AVAILABLE', 'numel': 0, 'ds_numel': 1024, 'shape': (0,), 'ds_shape': (1024,), 'requires_grad': True, 'grad_shape': None, 'persist': True, 'active_sub_modules': set()}, {'id': 116, 'status': 'NOT_AVAILABLE', 'numel': 0, 'ds_numel': 1048576, 'shape': (0,), 'ds_shape': (1024, 1024), 'requires_grad': True, 'grad_shape': None, 'persist': False, 'active_sub_modules': set()}).
```

Fixes: https://github.com/huggingface/transformers/issues/16688

@patil-suraj, @sgugger 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Kill async pushes when calling push_to_hub with blocking=True
Author: sgugger
Pull request: yes

# What does this PR do?

This PR fixes a bug that sometimes appear in the `Trainer` when `push_to_hub=True`: if one of the async pushes finishes after a regular non-async push, the history gets messed up and we end up with an error like this:

```
The push command with PID 1468 failed.
remote: error: cannot lock ref 'refs/heads/main': is at 07c85fd69cd46a7daee6323c5a5eefc3e6a886da but expected 1fd7f122ef725f8e340a14bc97d537812de44076      
```

To fix this, when the `Trainer` (or the user) calls `push_to_hub` with `blocking=True`, we interrupt any push in progress. The commit history will still be good since the commit don't take time.

cc @philschmid who had the error.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I think this is the right solution, but I wonder if we can't test it in a reproducible manner. Maybe we could do something like the following:

- Commit a largish file, and push that
- Reset the head without keeping the changes so that we're back on the previous commit
- Commit a small file, and push that.

Now the first commit will likely fail with the error above. Unfortunately, this is likely a very heavy test, so I'm not sure it should be part of the CI. It can stil be used to test the validity of the solution above, though!

Actually @philschmid but let's say we are the same person ;-) 

For more context tested with `roberta-large`
Question about google/bigbird-pegasus-large-pubmed
Author: tetsu9923
Pull request: no

Hi, I have a question about the BigBird seq2seq model [google/bigbird-pegasus-large-pubmed](https://huggingface.co/google/bigbird-pegasus-large-pubmed).
How was this model trained?
Was it pretrained with just word-level or sentence-level masking?
Or was it further fine-tuned by the summarization task?
Thank you.

Comments:
From the model card:

> This checkpoint is obtained after fine-tuning BigBirdPegasusForConditionalGeneration for summarization on pubmed dataset from [scientific_papers](https://huggingface.co/datasets/scientific_papers)[](https://huggingface.co/google/bigbird-pegasus-large-pubmed#bibtex-entry-and-citation-info).

Also, please use our [forum](https://discuss.huggingface.co/) for such questions. We'd like to keep Github issues for bugs/feature requests.

Thanks!
Add semantic segmentation example script, no trainer
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR adds an example script regarding fine-tuning any model supported by the `AutoModelForSemanticSegmentation` API on a semantic segmentation dataset, including regularly pushing to the hub during training as well as WandB logging.

I switched to using Accelerate as I had a bug with the Trainer. 

To do:

- [ ] fix Tensorboard logs

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_16630). All of your documentation changes will be reflected on that endpoint.

I've added all requested changes. As you can see, I'm updating the repo using commits rather than creating subfolders.

Updated the script to create subfolders like the other examples (and these aren't pushed to the hub based on the gitignore file).

Some issue with git, closing this PR in favor of a new one.
Improve image classification example
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR improves the image classification example script, by:
- replacing the `nateraw/imagefolder` hack with [`ImageFolder`](https://huggingface.co/docs/datasets/v2.0.0/en/image_process#imagefolder)
- improving the README, explaining how to easily train on custom data.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
wav2vec2_alignment
Author: arijitx
Pull request: yes


# What does this PR do?

Generate character level alignment with wav2vec2 models for audio and text pairs.
https://github.com/huggingface/transformers/issues/16570



## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. https://github.com/huggingface/transformers/issues/16570
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@patrickvonplaten 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_16781). All of your documentation changes will be reflected on that endpoint.

<del>Hey @arijitx </del>

<del>The PR looked good! :-) Feel free to open it again if you want</del>

Superseded by https://github.com/huggingface/transformers/pull/16782
Missing commas causing concatenation
Author: code-review-doctor
Pull request: no

Missing comma on results in strings being implicitly concatenated together. Probably not what was intended

This is the affected line:

https://github.com/huggingface/transformers/blob/main/tests/bert_japanese/test_tokenization_bert_japanese.py#L176
https://github.com/huggingface/transformers/blob/main/tests/bert_japanese/test_tokenization_bert_japanese.py#L249

I found this issue automatically, see other issues [here](https://codereview.doctor/huggingface/transformers)

Comments:
[trainer / deepspeed] fix hyperparameter_search
Author: stas00
Pull request: yes

This PR:
- fixes `hyperparameter_search` deepspeed config reset fix up that got out of sync with the normal code path
- adds a test so that will not happen in the future.
- adds a new group of pip deps: `deepspeed-testing`

@sgugger 

Fixes: https://github.com/huggingface/transformers/pull/11966#issuecomment-1058493821

Comments:
_The documentation is not available anymore as the PR was closed or merged._