noamaanMulla-03/transformers-issues huggingface dataset:
load_best_model_at_end
Author: ZeyiLiao
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.23.0.dev0
- Platform: Linux-4.15.0-189-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.3
- Huggingface_hub version: 0.10.0
- PyTorch version (GPU?): 1.8.1+cu111 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: <yes

### Who can help?

@sgugger ,Hi, When I finish traning t5 model with args ```--load_best_model_at_end```, I found that it still have four checkpoints dir at output_dir(I set max_epoch=4),and there is also a ```pytorch_model.bin``` at the output file.  Like they organized like this way
```
output_dir
--pytorch_model_1.bin

--ckpt1_dir
----pytorch_model_2.bin

--ckpt2_dir
----pytorch_model_3.bin

--ckpt3_dir
----pytorch_model_4.bin

--ckpt4_dir
----pytorch_model_5.bin
```

Does it means that the pytorch_model_1.bin is the best model or I can say I can AutoModel.from_pretrained(''./output_dir) would be correct?

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

None

### Expected behavior

None

Comments:
Please use the [forums](https://discuss.huggingface.co/) for such questions as we keep issues for bugs and feature requests only. As indicated by the name of the arg, the best model is the one loaded inside the Trainer. You should save it wherether you want for further use.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[wip: test doc-build]
Author: mishig25
Pull request: yes


Comments:
_The documentation is not available anymore as the PR was closed or merged._

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Generate: TF contrastive search with XLA support
Author: gante
Pull request: yes

# What does this PR do?

Adds contrastive search to TF, with XLA support.

In essence, TF's contrastive search is very similar to PT's, adapted to the structure that is present in other TF XLA generation functions (i.e. has a dedicated function to loop over, a separate function to update `model_kwargs` when in XLA mode, ...). The most notable difference is how the best candidate token (and associated model variables) are gathered -- PT relies on slicing, which TF doesn't support, so a `tf.gather` alternative is used.

The exact same integration tests (with the same input, model, and outputs) were added whenever possible. Three integration tests were not added, which will be addressed in a follow-up PR:
1. [GPT-J](https://github.com/huggingface/transformers/blob/d447c460b16626c656e4d7a9425f648fe69517b3/tests/models/gptj/test_modeling_gptj.py#L577) -- PT's test runs at half precision, for which we don't have the same TF facilities
2. [OPT](https://github.com/huggingface/transformers/blob/d447c460b16626c656e4d7a9425f648fe69517b3/tests/models/opt/test_modeling_opt.py#L495) -- OPT is not XLA compatible atm (it runs, but the position embeddings are wrong with padded structures, so we get different outputs)
3. [T5](https://github.com/huggingface/transformers/blob/d447c460b16626c656e4d7a9425f648fe69517b3/tests/models/t5/test_modeling_t5.py#L1227) -- the model used for this test do not have TF weights

Comments:
_The documentation is not available anymore as the PR was closed or merged._

(@Rocketknight1 is off, so I'm merging this to not slow down the corresponding blog post, which will contain TF examples thanks to this PR :D In any case, have a quick look when you're back, to ensure we kill any bad pattern before v4.25 gets released!)
Skip 2 tests in `VisionTextDualEncoderProcessorTest`
Author: ydshieh
Pull request: yes

# What does this PR do?

These 2 tests in `VisionTextDualEncoderProcessorTest` will be fixed when we add the new `AutoImageProcessor`.

Current error is

```bash
AttributeError: 'NoneType' object has no attribute 'from_dict
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._
docs: Replace unsupported `facebookresearch/bitsandbytes`
Author: tomaarsen
Pull request: yes

# What does this PR do?

* Replace unsupported https://github.com/facebookresearch/bitsandbytes with https://github.com/TimDettmers/bitsandbytes, which is by the same author and still being maintained and updated. For reference, the latter repository is the one mentioned in your blog post https://huggingface.co/blog/hf-bitsandbytes-integration, which was co-written by Tim Dettmers.


## Before submitting
- [x] This PR fixes a typo or improves the docs

## Who can review?

Documentation: @sgugger
@TimDettmers

---

- Tom Aarsen

Comments:
_The documentation is not available anymore as the PR was closed or merged._
"local_files_only" mode fails on Windows due to path format inconsistency
Author: DeXtmL
Pull request: no
Labels: name: bug

### System Info

When loading models using "local_files_only" mode, huggingface goes out to grab the commit hash from a resolved file name. Usually this works fine on POSIX standard OSes, but it fails on Windows due to hardcoded regex search pattern.

This issue can be fixed in one line (add one line in utils/hub.py), see this screenshot:
![hub-fix](https://user-images.githubusercontent.com/13725477/193199857-c8ba5cd3-b9c1-48dc-91f0-7e9bf128c587.jpg)


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

To reproduce, 

1. Install huggingface transformers on Windows (anaconda)
2. run the following code without "local_files_only" mode and let huggingface cache all the needed files
3. run it again with local_files_only=True

code:

```
from transformers import AutoTokenizer, AutoConfig

model_name = "google/t5-efficient-small"  # any correct model name can used to reproduce the error here. 
config = AutoConfig.from_pretrained(model_name ) 
tokenizer = AutoTokenizer.from_pretrained(model_name, config=config, local_files_only=True)  # TypeError
```



### Expected behavior

Huggingface transformers should work correctly across all supported OSes, using unified file path standard.
One quick fix is provided above, no extended test was conducted on the fix though.

Environment:
Windows 11
Huggingface transformers: 4.22.1
python 3.9

Comments:
Fixed here https://github.com/huggingface/transformers/pull/19178

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add decisiontransformer to onnx config
Author: skanjila
Pull request: yes

# What does this PR do?
Adds an onnx configuration for decision transformer


Fixes # (issue)
https://github.com/huggingface/transformers/issues/18191

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?
In progress while PR is being reviewed

## Who can review?
@ChainYo @regisss  Here's a first cut, I will be reading the documentation on testing this while the PR is being reviewed, again appreciate any help you can provide as this is my first time through this process 



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19189). All of your documentation changes will be reflected on that endpoint.

@ChainYo @regisss bumping this PR

> @ChainYo @regisss bumping this PR

It looks good to me!
Thanks for iterating on this, @skanjila.

**EDIT**: Did you do ci tests before pushing? (linting and stuff?)

@ChainYo @regisss  I reran make style and everything passed locally and also added an entry to DecisionTransformer in the test_onnx_v2.py file, please have a look now and lmk what else is missing

@skanjila Hmm too many files got reformatted, I think your version of Black is different from what we use. Could you run `pip install transformers["quality"]` and then `make style`?

@regisss done, and make style worked, please recheck

@skanjila You also need to run `make fix-copies`, I forgot this one my bad

And there are still many reformatted files, did you remove the files from the previous commit?

@regisss here is what I did
1) I reset git to an earlier commit before all my changes
2) I added back the entry for the decision_tranformer in test_onnx_v2.py
3) I reran make style after installing transformers["quality"]
4) I ran make fix-copies
5) steps 3/4 were successful

Should be good to go, please have a look , I look forward to finishing this work.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix generate_dummy_inputs for ImageGPTOnnxConfig
Author: ydshieh
Pull request: yes

# What does this PR do?

`ImageGPT` ONNX tests fail with

```bash
TypeError: __call__() takes 2 positional arguments but 3 were given
```

This is due to the way of calling `preprocessor(input_image, framework)` is not working with the changes introduced in Image Process PR #19796.

This PR updates the calling way by using keyword arguments.


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Discrepancy between PegasusTokenizer and PegasusTokenizerFast
Author: Hannibal046
Pull request: no

### System Info

- `transformers` version: 4.22.0.dev0
- Platform: Linux-5.8.0-51-generic-x86_64-with-glibc2.10
- Python version: 3.8.13
- Huggingface_hub version: 0.8.1
- PyTorch version (GPU?): 1.12.1+cu113 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>


### Who can help?

@patil-suraj

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```python
from transformers import PegasusTokenizerFast,PegasusTokenizer
toker_fast = PegasusTokenizerFast.from_pretrained("/data/pretrained_model/pegasus_large/")
toker = PegasusTokenizer.from_pretrained("/data/pretrained_model/pegasus_large/")
print(toker_fast.decode(toker_fast.encode("huggingface/transfomer"),skip_special_tokens=False)) ## huggingface/transfomer

print(toker.decode(toker.encode("huggingface/transfomer"),skip_special_tokens=False)) ## huggingface/transfomer
```

### Expected behavior

These two should be the same. I suppose this is the problem of `PegasusTokenizer`. Because EOS token is needed in Generation Task.

Comments:
I think this may be the problem of `PegasusTokenizer.decode()`. It indeed add `<\s>` to the end of the sentence, but decoder fails to keep it even when `skip_special_tokens=False`
<img width="819" alt="draft ipynb — selfmem  SSH: wict3090  2022-11-06 10-19-50" src="https://user-images.githubusercontent.com/38466901/200150974-1bde2180-8c6a-4ea0-94f8-6e36a20fceca.png">


I think this was recently fixed by #15775. At least on `google/pegasus-xsum` and the main branch of Transformers, I don't see any differences in the outputs.
Not sure if this is the model you were using since yours is local. Could you give us a repo ID on the Hub if the issue persists on your side?

after updating transformers to the latest version by
```shell
pip uninstall transformers
pip install transformers
```
on `google/pegasus-large` and `google/pegasus-xsum`, the problem still exists:
![draft ipynb — selfmem  SSH: 45a3159k71 zicp vip  2022-11-07 23-31-11](https://user-images.githubusercontent.com/38466901/200349559-03707957-2f9c-4a80-9f06-909d27b79585.png)
![draft ipynb — selfmem  SSH: 45a3159k71 zicp vip  2022-11-07 23-32-12](https://user-images.githubusercontent.com/38466901/200349758-2f9a86a2-0aa0-433d-aa14-a307c164d2d5.png)


Sorry, I didn't notice this PR has not been merged yet.
This https://github.com/huggingface/transformers/pull/15775 indeed solves this problem. Thanks !
![draft ipynb — selfmem  SSH: 45a3159k71 zicp vip  2022-11-07 23-45-40](https://user-images.githubusercontent.com/38466901/200352923-40e6ef4e-d60e-4441-98ca-ecd9aad2c061.png)
docs: Resolve many typos in the English docs
Author: tomaarsen
Pull request: yes

# What does this PR do?

* Fixes typo in `python -m transformers.onnx --help`.
* Fixes many typos in the English documentation via `codespell`.

- [x] This PR fixes a typo or improves the docs

## Who can review?

Documentation: @sgugger

- Tom Aarsen

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Use huggingface_hub.model_info() to get pipline_tag
Author: y-tag
Pull request: yes

# What does this PR do?

This PR replaces raw HTTP GET with `huggingface_hub.model_info()` to get `pipeline_tag` of model.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@Narsil @LysandreJik 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks for your PR @y-tag!
docs: Fixed variables in f-strings
Author: tomaarsen
Pull request: yes

# What does this PR do?

Fixes unknown variables in some documentation code blocks' f-strings.

- [x] This PR fixes a typo or improves the docs

## In addition...
The following code block, which can be found at https://huggingface.co/docs/transformers/custom_models#writing-a-custom-model & [custom_models.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/custom_models.mdx), uses `torch` without `import torch` being performed in any prior docstring.

```python
class ResnetModelForImageClassification(PreTrainedModel):
    config_class = ResnetConfig

    def __init__(self, config):
        super().__init__(config)
        block_layer = BLOCK_MAPPING[config.block_type]
        self.model = ResNet(
            block_layer,
            config.layers,
            num_classes=config.num_classes,
            in_chans=config.input_channels,
            cardinality=config.cardinality,
            base_width=config.base_width,
            stem_width=config.stem_width,
            stem_type=config.stem_type,
            avg_down=config.avg_down,
        )

    def forward(self, tensor, labels=None):
        logits = self.model(tensor)
        if labels is not None:
            loss = torch.nn.cross_entropy(logits, labels)
            return {"loss": loss, "logits": logits}
        return {"logits": logits}
```
If preferred, I can add `import torch` in this code block in this PR, or make a new one for it.

## Who can review?

Documentation: @sgugger

- Tom Aarsen

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Done & done!

Thanks again for your contribution!
docs: Replace awkward `timm` link with the expected one
Author: tomaarsen
Pull request: yes

# What does this PR do?

Replace https://github.com/rwightman/pytorch-image-models/tree/master/timm with https://github.com/rwightman/pytorch-image-models. 

## Reasoning
1. The URL has the hardcoded `master` branch, despite the `timm` being branch being renamed to `main` nowadays.
2. The URL points to the `timm` folder for some reason, when linking to the root, i.e. where a README is visible, is much more sensible.

## Before submitting
- [x] This PR fixes a typo or improves the docs

## Who can review?

Documentation: @sgugger

---

I just keep running into small issues here and there! More than happy to help fix them, though.

- Tom Aarsen

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[PoC] Add resources to each model's doc page
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR is a small PoC to add resources (notebooks, scripts, blogs, etc.) to each model's doc page.

Ideally (not sure if it's possible), but it'd be great if we could (partially) automate this.

The idea being that, if a new model is added to the FOR_IMAGE_CLASSIFICATION_MAPPING for instance, the doc page automatically will add a link to the image classification notebooks and scripts. Not sure if this is possible, cc @sgugger.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

One question I have: does it matter if the resources diverge from the API described in the docs?

@stevhliu, I think this is a pretty significant improvement for the model pages. Would you be down to try doing something similar for other high-profile architectures? 

If so, I would encourage the following:
- Identifty the 20 most popular model architectures, either through number of downloads or through doc pages views
- Open an issue to track the update of these 20 most popular architectures.
- Start working on some of them one at a time
- Open this up to community contributions

Would you be down to kickstart such a project? In doing so, I'm sure we'll have more visibility over what we can automate, and what we cannot automate. I'm pretty sure we can automate a portion of it but I don't think we can have high-quality resources here only using automation.

Closing this PR as it was just a PoC. Will continue the work in other PRs
Autotokenizer/ LED/BARTTokenizer won't cast to CUDA
Author: M-Chimiste
Pull request: no
Labels: name: bug

### System Info

Hello,

I'm running into the following issue when trying to run an LED model.  It appears that the tokenizer won't cast into CUDA.  I've seen this work in the past, but apparently something has gone amiss.  I'm not entirely sure why this behavior is being exhibited.


`from transformers import (
    AutoTokenizer,
    LEDForConditionalGeneration,
)
import torch
import pickle
 
device = "cuda:0" if torch.cuda.is_available() else "cpu"
primer_path = "allenai/PRIMERA-multixscience"
tokenizer = AutoTokenizer.from_pretrained(primer_path, return_tensors="pt").to(device)
model = LEDForConditionalGeneration.from_pretrained(primer_path).to(device)
model.gradient_checkpointing_enable()
PAD_TOKEN_ID = tokenizer.pad_token_id
DOCSEP_TOKEN_ID = tokenizer.convert_tokens_to_ids("<doc-sep>")`

`2022-09-30 05:10:51 : Traceback (most recent call last):
2022-09-30 05:10:51 :   File "/summarization.py", line 17, in <module>
2022-09-30 05:10:51 :     tokenizer = AutoTokenizer.from_pretrained(primer_path, return_tensors="pt").to(device)
2022-09-30 05:10:51 : AttributeError: 'LEDTokenizerFast' object has no attribute 'to'`

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Simply try to import PRIMERA and use a tokenizer with CUDA

### Expected behavior

I would expect the tokenizer to cast onto the GPU, not give an exception that to is not an attribute.

Comments:
Hello @M-Chimiste Please assign me and btw I'm new to the contribution so guide accordingly
Thank you!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add ConvNeXt Mask R-CNN
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR is an initial draft for implementing the classic Mask R-CNN framework with ConvNeXt as backbone. 

The framework is implemented in a single script, with the exception of 3 files (for now):
* assign_result.py
* losses.py
* mask_target.py

As we have a one model, one file policy, I'm reimplementing ConvNeXT leveraging Copied from statements. So `ConvNextMaskRCNNModel` is almost identical to `ConvNextModel`. This way, the backbone used for object detection stays independent from the original one. In this case for instance, extra layernorms are added after each stage.

There's a dependency on torchvision, which is used for NMS (non-maximum suppression, a postprocessing algorithm used by both the RPN head and the RoI head).

To do:

- [x] update NumPy logic to pure PyTorch (i.e. channels first everywhere) - see branch `add_convnext_maskrcnn_torch_shapes`
- [ ] update outputs of model to have channels first (no NumPy)

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Adapt has_labels test when no labels were found
Author: sgugger
Pull request: yes

# What does this PR do?

As #20105 highlights it, the new way we infer default label names for models might not work for models outside of Transformers This PR reverts to the old default for non-`PreTrainedModel` models

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20113). All of your documentation changes will be reflected on that endpoint.