noamaanMulla-03/transformers-issues huggingface dataset:
Check list of models in the main README and sort it
Author: sgugger
Pull request: yes

# What does this PR do?

In our effort to have the repo consistency check catch all potential mistakes a contributor can make when adding a new model, this PR adds a new script to make sure all added models are also put in the main README. As a bonus, it also enforces that said README is sorted in alphabetical order cause that's prettier.

Some models are not supposed to be in the main README, so there is a new list for those black sheeps. Some models have different names in the main README and in the lib, there is a map for that.

MobileBERT and RAG were not in the README and I think they should o I added them, the other absent do not have a paper from what I checked. I fixed typos (usually in casing) as well.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[trainer/deepspeed] load_best_model (reimplement re-init)
Author: stas00
Pull request: yes

This PR fixes https://github.com/huggingface/transformers/issues/17114

The `deepspeed_reinit` hack proved to not always work, as some stored args appear to be either stale or wrong (e.g. the optimizer can be a deepspeed outer optimizer which shouldn't be the case), so trying to just do a full init from scratch instead. 

And then there was an issue on the deepspeed side with model getting deepspeed hooks added multiple time which was breaking everything. Fixed in https://github.com/microsoft/DeepSpeed/pull/1947.

I spent many hours trying to reproduce the problem in the usual way via examples scripts to make a test, but alas, it just won't fail in the right places. So I ended up re-implemented `test_load_best_model` using code derived from @base-y's repro example. So super appreciating having their script.

### Blocking events

- [x] merge https://github.com/microsoft/DeepSpeed/pull/1947
- [x] update the dependency table to the new ds release after the merge 0.6.5

Fixes: https://github.com/huggingface/transformers/issues/17114

Comments:
_The documentation is not available anymore as the PR was closed or merged._

when will this pr be merged ?

We needed to wait for a new deepspeed release, which I see has happened already, so yes, we can merge this shortly.
Not able to import tensorflow OPT
Author: Leli1024
Pull request: no
Labels: name: bug

### System Info

```shell
Using the latest git repo as a source install, when I try to import TFOPTForCasualLM I will get the following error.


ImportError: cannot import name 'TFOPTForCasualLM' from 'transformers' (/anaconda/envs/azureml_py38/lib/python3.8/site-packages/transformers/__init__.py)
```
```


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
from transformers import TFOPTForCausalLM, GPT2Tokenizer

model = TFOPTForCausalLM.from_pretrained("facebook/opt-350m")
tokenizer = GPT2Tokenizer.from_pretrained("facebook/opt-350m")
```

### Expected behavior

```shell
Was expecting model class to import normally
```


Comments:
I just tried this with latest source install and it's working for me. Make sure you have TF installed to be able to import TF models.

> I just tried this with latest source install and it's working for me. Make sure you have TF installed to be able to import TF models.

I have tensorflow installed, even checked it by running a quick test

Could you share details about your script and maybe your environment? 

> Could you share details about your script and maybe your environment?

I was working on an Azure compute machine. I found the problem, apparently I needed to install tensorflow GPU. as well as the regular distro
[WIP] Add ResNets in TF
Author: sayakpaul
Pull request: yes

This PR adds the OG of the OGs when it comes to computer vision models: ResNet. There are a few rough edges we need to figure out in the cross-loading of weights. 

When I ran `RUN_SLOW=1 python -m pytest tests/models/resnet/test_modeling_resnet.py`, during the integration test, it complained about weight mismatches for all the layers. 

So, naturally, when I did a little standalone integration test locally with the TF model the same issue surfaced and the logit assertion failed. FYI, it fails for the PT model too as mentioned earlier. Here's my integration test for TF model:

```py
from PIL import Image
import numpy as np

from src.transformers.models.resnet.modeling_tf_resnet import TFResNetForImageClassification
from transformers import AutoFeatureExtractor


def prepare_img():
    image = Image.open("./tests/fixtures/tests_samples/COCO/000000039769.png")
    return image


feature_extractor = AutoFeatureExtractor.from_pretrained(
    "microsoft/resnet-50"
)
model = TFResNetForImageClassification.from_pretrained(
    "microsoft/resnet-50", from_pt=True
)


image = prepare_img()
inputs = feature_extractor(images=image, return_tensors="tf")
outputs = model(**inputs)

expected_shape = [1, 1000]
assert outputs.logits.shape == expected_shape

expected_slice = np.array([-11.1069, -9.7877, -8.3777])

np.testing.assert_allclose(outputs.logits[0, :3].numpy(), expected_slice, atol=1e-4)
```

@amyeroberts @FrancescoSaverioZuppichini please advise here. After this issue is resolved I will start working on the test cases. 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Should have checked before. 

Closing in spirit of https://github.com/huggingface/transformers/pull/17427
Training large huggingface models on Azure with CUDA? [OPT]
Author: Leli1024
Pull request: no

I am trying to finetune the 1.3b and 30b OPT variant models respectively, however each time I try to train them I will always get the following error:
```
RuntimeError: CUDA out of memory. Tried to allocate 16.00 MiB (GPU 0; 11.17 GiB total capacity; 1.95 GiB already allocated; 8.50 MiB free; 1.95 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
```

I have tried reducing the batch size to 1. However short of running the model on CPU which would take way too much time, it will crash each time. If it's relevant I am using an Azure environment which uses the Tesla K80 accelerator. Has anyone managed to train or use these models on GPU?

Comments:
Hi, for training such large models, a Tesla K80 won't suffice. You typically need # of parameters * 18 in terms of bytes of RAM when fine-tuning. So for the 30 billion parameter model, it's 30 billion * 18 = 540 GB. It's because you need 18 bytes per parameter to store not only the parameter itself, but also its gradient and optimizer states.

However there are tricks like mixed precision and frameworks like DeepSpeed to fit giant models, you can read more in our guide here: https://huggingface.co/docs/transformers/performance
Fix all offload and MP tests
Author: sgugger
Pull request: yes

# What does this PR do?

This should fix all GPU tests for model parallelism and offload. I stopped trying to make the model bigger since it was causing so many failures (unrelated to current failing tests). Then there was a tiny bug in `from_pretrained` when tied wieghts are present, as highlighted by the failure of the GPT-Neo offload tests. Adding the tie weights before computing the auto device map is necessary to know which weights are tied when auto-generated this device map.

The OPT model for testing was too tiny for the MP/offload tests (that's why there was some logic to make tiny models bigger there) so I just adjusted its size (speed is not really affected).

Finally the T5 tests starting failing after the tied weights because the decoder has to be tied with the shared layer, which require bigger models than the tiny one used for test. Here I didn't make the model bigger since the tests already take some time, so I adjusted the percentages used for the total model size in those tests. I added a new class variable for that, but happy to overwrite the tests in the T5 modeling test if you prefer it.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
install dev. version of accelerate in docker file
Author: ydshieh
Pull request: yes

# What does this PR do?

With an offline discussion with @muellerzr regarding this CI failure
```
tests/trainer/test_trainer.py::TrainerIntegrationTest::test_auto_batch_size_finder
(line 776)  ImportError:
```
I update the docker file in this PR. Will try to build the new docker image once this PR is merged.


Comments:
_The documentation is not available anymore as the PR was closed or merged._
[trainer] sharded _load_best_model
Author: stas00
Pull request: yes

Looks like a copy-in-paste issue. This code path is probably untested.

@sgugger 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thank you for explaining why the testing of this path is complicated, Sylvain.

I think I can make it partially tested by using zero3 w/o `"stage3_gather_16bit_weights_on_model_save"` which would make it fall through and at least test that condition. I will be adding these tests here https://github.com/huggingface/transformers/pull/17151
Clean imports to fix test_fetcher
Author: sgugger
Pull request: yes

# What does this PR do?

We noticed that a lot of tests are being picked from some modeling files, there are two reasons for that.

1. The circular import being `modeling_tf_utils` and `modelcard` makes `modeling_tf_utils` be a file impacted by `modelcard`. `modeling_tf_utils` triggers pretty much all tests as it's tightly linked to `modeling_utils` via `modeling_tf_pytorch_utils`.
2. ONNX also has some circular thing going on as:
    - it's imported in models that define an ONNX Config
    - and in return the module imports all models having an ONNX Config in `onnx.features`

This was discovered by developing a tool that prints the tree of depending modules for a given module/test which is incuded in this PR. To use it, in the root of the repo, just do:
```
python utils/tests_fetcher.py --print_dependencies_of src/transformers/models/bert/modeling_bert.py
```

The fix for 1 is to use the special comment that will make the `test_fetcher` ignore the circular import.

The fix for 2 is to remove all configs imports in `onnx.feaures` but rely on the config names, then import them dynamically at the creation of `FeaturesManager` (in a 100% backward compatible manner).

cc @ydshieh since you reported the slowdown.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Failure is flaky, so merging :-)
Update index.mdx
Author: BritneyMuller
Pull request: yes

# What does this PR do?

This PR updates our Expert Acceleration Program image with a new image featuring our experts.

This is similar to our [Transformers/README.md image update](https://github.com/huggingface/transformers/pull/16615) that has proven to be successful.

![support-image-update](https://user-images.githubusercontent.com/5594118/171916834-cfa8cc32-1b53-45b3-b55d-fab8042a3766.png)


Comments:
_The documentation is not available anymore as the PR was closed or merged._
[deepspeed] fix load_best_model test
Author: stas00
Pull request: yes

Fixes https://github.com/huggingface/transformers/pull/17151 to run `from_pretrained` with emulated dist env.

While it was working on my setup on CI it failed with:

```
tests/deepspeed/test_deepspeed.py:756: in test_load_best_model
    model = T5ForConditionalGeneration.from_pretrained(T5_TINY)
src/transformers/modeling_utils.py:2116: in from_pretrained
    init_contexts = [deepspeed.zero.Init(config_dict_or_path=deepspeed_config())] + init_contexts
/opt/conda/lib/python3.8/site-packages/deepspeed/runtime/zero/partition_parameters.py:693: in __init__
    self.local_device = torch.device('cuda:{}'.format(os.environ["LOCAL_RANK"]))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = environ({'NPP_VERSION': '11.3.2.139', 'NVIDIA_VISIBLE_DEVICES': 'all', 'DALI_BUILD': '2054952', 'GITHUB_WORKSPACE': '/...RRENT_TEST': 'tests/deepspeed/test_deepspeed.py::TrainerIntegrationDeepSpeed::test_load_best_model_zero3_fp16 (call)'})
key = 'LOCAL_RANK'
    def __getitem__(self, key):
        try:
            value = self._data[self.encodekey(key)]
        except KeyError:
            # raise KeyError with the original key value
>           raise KeyError(key) from None
E           KeyError: 'LOCAL_RANK'
```

the test is exactly the same, just moved  a big chunk of it into the `with mockenv_context` - no code changes

@sgugger 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Beginning word ids
Author: oorojoo
Pull request: no

tokenizer = AutoTokenizer.from_pretrained("roberta-base")
inputs = tokenizer('in an ideal situation, it works')
print(inputs.word_ids())

Returns
[None, 0, 1, 2, 3, 3, 3, 4, 5, 6, None]

Are there any methods to identify which token is the beginning position for a word 
For example: 
Returns
[None, 1, 1, 1, 1, 0, 0, 1, 1, 1, None]

where 1 is the start token for a given word and 0 is a token that is not the start token for a given word.



Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Permission denied
Author: muhammad-ahmed-ghani
Pull request: no
Labels: name: bug

### System Info

```shell
transformers==4.19.2
```


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("hkunlp/from_all_T5_large_prefix_spider_with_cell_value2")

model = AutoModel.from_pretrained("hkunlp/from_all_T5_large_prefix_spider_with_cell_value2")

### Expected behavior

```shell
File "/home/user/anaconda3/envs/test/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 1850, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/home/user/anaconda3/envs/test/lib/python3.8/site-packages/transformers/models/t5/tokenization_t5_fast.py", line 128, in __init__
    super().__init__(
  File "/home/user/anaconda3/envs/test/lib/python3.8/site-packages/transformers/tokenization_utils_fast.py", line 107, in __init__
    fast_tokenizer = TokenizerFast.from_file(fast_tokenizer_file)
Exception: Permission denied (os error 13)
```


Comments:
Hi, can you also open a Discussion on the model repo, i.e. https://huggingface.co/hkunlp/from_all_T5_large_prefix_spider_with_cell_value2 (and link this issue to/from there)?

Thanks 🙏
Dropping support for Python 3.6
Author: sgugger
Pull request: no

Python 3.6 [has been retired](https://peps.python.org/pep-0494/#lifespan) on December 21st 2021 and won't get new security releases anymore. As a result, Transformers will stop supporting Python 3.6 and have a minimum requirement of Python 3.7 in the next release (v4.19.0), around the beginning of May.

Comments:
What is the purpose of dropping support while not using latest features of newer python versions (or newer versions of other libraries, which are dropping support)?

Hey @LSinev, thank you for asking. There are some features that would have been implemented in a simpler manner had Python 3.6 been dropped earlier (https://github.com/huggingface/transformers/issues/15739, @sgugger can chime in on the init system), and we already use features that are only available in 3.7 (`dataclasses`). 

Furthermore:
- Python 3.6 is now EOL and will not be receiving new bug and security patches
- All backends of `transformers` newer versions (torch 1.11, tensorflow 2.8, jax  0.2.18) now require python 3.7

We were not using the features of newest Python versions without dropping the old ones ;-)
For instance, our inits will now be able to use [PEP 562](https://docs.python.org/3/whatsnew/3.7.html#whatsnew37-pep562) 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
pip install "sacremoses>=0.0.50" breaks on SageMaker Studio
Author: juliensimon
Pull request: no
Labels: name: bug

### System Info

```shell
This was verified today on a fresh SageMaker Studio instance running in us-west-2.

It's not a Transformer issue, but as sacremoses is a dependency, this is likely to break 'pip install transformers' on SageMaker Studio at some point.
```


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1) Open an SM Studio notebook

2) Run the following cell:
```
%%sh
pip install "sacremoses>=0.0.50"
```

The obvious workaround for now is
```
pip install "sacremoses==0.0.49"
```



### Expected behavior

```shell
sacremoses should install without error.
```


Comments:
Thanks for the issue @juliensimon, this should be fixed by https://github.com/huggingface/transformers/pull/17049. It will be in the next release which should drop early next week.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Should be fixed now!
cannot import name 'Data2VecForCTC' from 'transformers'
Author: Sorrow321
Pull request: no
Labels: name: bug

### System Info

```shell
- `transformers` version: 4.18.0
- Platform: Linux-5.13.0-40-generic-x86_64-with-glibc2.31
- Python version: 3.9.7
- Huggingface_hub version: 0.2.1
- PyTorch version (GPU?): 1.11.0 (True)
- Tensorflow version (GPU?): 2.7.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```


### Who can help?

@patrickvonplaten

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Following the code sample in https://huggingface.co/facebook/data2vec-audio-large-960h I'm trying to import the name **Data2VecForCTC**, but unsuccessful.
Possibly a typo: **Data2VecForCTC** -> **Data2VecAudioForCTC**


### Expected behavior

```shell
Correct code sample is expected.
```


Comments:
Hi,

Data2Vec is only available on the main branch for now: pip install git+https://github.com/huggingface/transformers.git.

Hi

I can't find the name **Data2VecForCTC** in https://github.com/huggingface/transformers/blob/main/src/transformers/__init__.py either.

Here it is: https://github.com/huggingface/transformers/blob/dced262409177586bb510b6b724c762fb89da0e8/src/transformers/__init__.py#L880

Note that: 

```python
from transformers import Data2VecAudioForCTC

model = Data2VecAudioForCTC.from_pretrained("...")
```

should also already work on master

Data2VecForCTC is not the same as Data2Vec**Audio**ForCTC 

Good observation!

There is no `Data2VecForCTC` ;-)

Yes, but there is on the [website](https://huggingface.co/facebook/data2vec-audio-large-960h)
![image](https://user-images.githubusercontent.com/20703486/165815507-0fb4054f-f6e2-4f48-9a2c-ca21a497ad27.png)


Soon we'll have a feature that allows you to report this directly on the model repo ;) stay tuned!

But yes I'll fix it, thanks for reporting 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Opened a PR to fix it: https://huggingface.co/facebook/data2vec-audio-large-960h/discussions/1
About model loading without parameter
Author: LiuJinzhe-Keepgoing
Pull request: no

https://github.com/huggingface/transformers/blob/6e535425feae20ca61a8b10ae5e8a7fab4d394ba/src/transformers/models/vilt/modeling_vilt.py#L1175
Hello, I want to use vilt in other fields through from_ Can Pretrain only load models without loading model parameters. Because the coco dataset is very different from my domain dataset. I don't know if there is any way.

Comments:
To initialize a model without any pretrained weights, you can just load it with a configuration:

```
from transformers import ViltConfig, ViltForImageAndTextRetrieval

config = ViltConfig()
model = ViltForImageAndTextRetrieval(config)
```
All weights will be randomly initialized.

Closing this issue, as I believe I've answered it. Feel free to re-open.
Adding the Portuguese version of the tasks/token_classification.mdx documentation
Author: jonatasgrosman
Pull request: yes

# What does this PR do?

Adding the Portuguese version of the tasks/token_classification.mdx documentation

Work on #16824

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

@omarespejel 



Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thank you @jonatasgrosman for the translation! 🤗

LGTM @sgugger.