noamaanMulla-03/transformers-issues huggingface dataset:
Hyperparameter Sweep for Selection of Best Pre-trained model
Author: ss2342
Pull request: no

### Feature request

Perhaps I have not been able to figure this out, but the way that the current hyperparameter_search function is set up is that it only allows you to pass in  hyperparameters that are in the TrainingArguments.  However, one key hyperparameter that we are not currently able to pass in  the model types themselves. If I want to pass in a list of models that I would like to try out to hyperparameter space, it is not possible to do with the current set up. For example I want to try out passing in something like this to  my hyperparameter space.

`model_type = trial.suggest_categorical(["bert-base-uncased", "roberta-base", "xlnet"])`

### Motivation

Being able to pass in  model names as part of the hp_space could be very useful especially when one is trying to determine which models might be best for their use case. Compiling a list of different models after reading literature and passing in that compiled list to a  hyperparameter sweep could be very useful. 

It could be used in this  manner: 

```
model_type = trial.suggest_categorical(["bert-base-uncased", "roberta-base", "xlnet"])
epochs = trial.suggest_categorical("epochs", EPOCHS)
    batch_size = trial.suggest_categorical("batch_size", BATCH_SIZE)
    learning_rate = trial.suggest_categorical("learning_rate", LEARNING_RATES)
    scheduler = trial.suggest_categorical("scheduler", SCHEDULERS)
    model_name = trial.suggest_categorical("model_name", MODEL_NAMES)
    
    hp_space = {
        "model_name": model_name,
        "batch_size": batch_size,
        "learning_rate": learning_rate,
        "scheduler": scheduler,
        "epochs": epochs,
    }

## Passing it to  trainer
trainer = Trainer(
            training_args,
            train_dataset=train_dataset,
            eval_dataset=val_dataset,
            data_collator=data_collator,
            tokenizer=tokenizer,
        )
trainer.hyperparameter_search(hp_space=hp_space)
```

### Your contribution

I would love to help work on this issue, if a solution does not possibly exist.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
T5tokenizer.pre_trained("t5-small") is not callable whereas AutoTokenizer worked fine
Author: mellow-d
Pull request: no


Comments:
Hi @mellow-d  👋 Having a popular project like `transformers` means we get many support and feature requests — if we want to maximize how much we help the community, the community has to help us stay productive 🙏

To that end, please share a *short* script where the issue is clearly reproducible on *any* computer. Thank you 🤗

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Download pretrained models from a new conda virtualenv with higher python version and higher transformers version
Author: sujoung
Pull request: no

### System Info

Old 
python==3.7
transformers==3.5.0

New 
python==3.9
transformers==4.22.2

### Who can help?

@LysandreJik,  @NielsRogge

Hello! Nothing really critical I think, but I stumbled upon this message so I share it here. Think this is some kind of legacy handling part, but it works fine. It is just a warning message that I wanted to share.

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Download bert-base-uncased on environment with python=3.7 & transformers==3.5
`
from transformers import AutoTokenizer, AutoModelWithLMHead
entk = AutoTokenizer.from_pretrained("distilbert-base-uncased")
enlm = AutoModelWithLMHead.from_pretrained("distilbert-base-uncased")
`

2. Create another virtualenv with python=3.9
3. install transformers==4.22.2
4. Download bert-base-uncased by using the same snippet above

### Expected behavior


The cache for model files in Transformers v4.22.0 has been updated. Migrating your old cache. This is a one-time only operation. You can interrupt this and resume the migration later on by calling `transformers.utils.move_cache()`.
Moving 114 files to the new cache system
  0%|                                                                                      | 0/114 [00:00<?, ?it/s]
There was a problem when trying to move your cache:

  File "/Users/sujoungbaeck/opt/anaconda3/envs/hubert-api-package__3.9/lib/python3.9/site-packages/transformers/utils/hub.py", line 1128, in <module>
    move_cache()

  File "/Users/sujoungbaeck/opt/anaconda3/envs/hubert-api-package__3.9/lib/python3.9/site-packages/transformers/utils/hub.py", line 1071, in move_cache
    hub_metadata[url] = get_hub_metadata(url, token=token)

  File "/Users/sujoungbaeck/opt/anaconda3/envs/hubert-api-package__3.9/lib/python3.9/site-packages/transformers/utils/hub.py", line 996, in get_hub_metadata
    huggingface_hub.file_download._raise_for_status(r)

AttributeError: module 'huggingface_hub.file_download' has no attribute '_raise_for_status'

Please file an issue at https://github.com/huggingface/transformers/issues/new/choose and copy paste this whole message and we will do our best to help.

Comments:
Thank you for the report @sujoung, looking into it.

Indeed, this will be fixed once https://github.com/huggingface/transformers/pull/19244 is in a release (the release will likely be done Monday or Tuesday)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Proof of Concept - Better Transformers
Author: younesbelkada
Pull request: yes

# What does this PR do?

A Proof of Concept of the Better Transformers integration into `transformers` - more details coming soon. Also comparing this implementation with an integration in optimum https://github.com/huggingface/optimum/pull/422 

cc @HamidShojanazeri 

https://github.com/huggingface/transformers/pull/19553

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19632). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fixing Spelling Error in Testing Documentation - Issue #20194
Author: kasmith11
Pull request: yes

# What does this PR do?
This PR fixes a spelling error in the testing documentation. Changes "checkt" to "check".

Fixes #20194

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@sgugger 

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20220). All of your documentation changes will be reflected on that endpoint.
Enable PyTorch 1.13
Author: sgugger
Pull request: yes

# What does this PR do?

This PR enables PyTorch 1.13 for Transformers so we can start adding functionality like safer loading with `torch.load`. Since there are no wheels for torch scatter, this comes at the price of uninstalling `torch-scatter`. However the PR to move away from this dep and use the PyTorch core ops seems well under way, so skipping the TAPAS tests for now until the PR is merged does not seem like a heavy price to pay (cc @NielsRogge for information).

A couple of tests are still failing, which are all torch FX tests (cc @michaelbenayoun, see failing job [here](https://app.circleci.com/pipelines/github/huggingface/transformers/51241/workflows/4a2c365b-8721-4c2c-adac-54f0fd7cf9c8/jobs/613048)). I'm skipping them and we can fix them next week in a followup PR.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

cc @sgugger , there is a PR

https://github.com/huggingface/transformers/pull/20149

but we forgot to ping you.

(Oh, I just saw you are aware of that PR)

Hi @sgugger :) - One question, you have removed `torch-scatter` dependency but at the same time you have added decorator `require_scatter` for `test_pt_tf_model_equivalence` test. Is that correct or am I just missing something?

The scatter dependency is only removed from the CPU runners on circleCI, it's not removed from the library byt his PR, that's your job ;-) . When testing in our other setups that include the `scatter` dependency, the tests will be run (until your PR is merged and the dep is removed entirely).
Slightly alter Keras dummy loss
Author: Rocketknight1
Pull request: yes

The existing `dummy_loss` returns a scalar, but this is incorrect - Keras expects loss functions to return a vector of per-sample losses. This only causes issues when the user passes `sample_weight` to `fit()` - I'll probably adjust our tests to include that to make sure we don't regress on this in future.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@gante Can you take one more look at the expanded tests?

Also, the failing test looks like flakiness with a too-low threshold to me, unrelated to this PR
Update reqs to include min gather_for_metrics Accelerate version
Author: muellerzr
Pull request: yes
Labels: name: Examples

# What does this PR do?
Update all the PyTorch examples using `accelerate` and `gather_for_metrics` to include a minimum accelerate version of 0.12.0 since this introduced `gather_for_metrics`


Related to https://github.com/huggingface/accelerate/issues/854

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add clip resources to the transformers documentation
Author: ambujpawar
Pull request: yes

# What does this PR do?


<!-- Remove if not applicable -->

Fixes #20055 (partially)


## Before submitting
- [x] This PR improves the docs of CLIP by adding common and most used resources
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. #20055 
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).

## Who can review?

@stevhliu Please can you have a look?


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20190). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20190). All of your documentation changes will be reflected on that endpoint.

Folks, please make sure resources which are added are talking about the particular model.
Allow trainer to return eval. loss for CLIP-like models
Author: ydshieh
Pull request: yes

# What does this PR do?

Allow trainer to give **evaluation** loss for CLIP-like models.

Currently, this line
https://github.com/huggingface/transformers/blob/07d8d6e2f7a920d399e5e86a82d78179cdfa6746/src/transformers/trainer.py#L3192
gives `has_labels = False` for CLIP-like models, and can't give loss value in the evaluation.

without this PR:
```bash
***** eval metrics *****
  epoch                   =        1.0
  eval_runtime            = 0:00:01.67
  eval_samples_per_second =      9.571
  eval_steps_per_second   =      4.785

```

with this PR.
```bash
***** eval metrics *****
  epoch                   =        1.0
  eval_loss               =     0.8159
  eval_runtime            = 0:00:01.66
  eval_samples_per_second =      9.598
  eval_steps_per_second   =      4.799

```

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger Hopefully the change covers everything that could happen now and in the future.
Question about layer norm in T5
Author: FrostML
Pull request: no

I notice that there is no bias and no subtraction of mean in layer norm.

I understand no bias but I'm confused about the meaning of computing variance without subtraction of mean. 

Normally, we compute variance, for example:
<img width="491" alt="image" src="https://user-images.githubusercontent.com/7160927/201863288-bcb63739-0f74-4487-bbb8-91299dad2eef.png">
But it's different here. Why is that?

Hoping some explanation. 

Comments:
Please use the [forums](https://discuss.huggingface.co/) to ask such questions as we keep issues for bugs and feature requests only.

Sure. have created this topic [question-about-layer-norm-in-t5](https://discuss.huggingface.co/t/question-about-layer-norm-in-t5/26142). I'd close this issue.
Complete doc migration
Author: mishig25
Pull request: yes

Reverts https://github.com/huggingface/transformers/pull/20125

Everything is handled on the doc-builder side now 😊

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20267). All of your documentation changes will be reflected on that endpoint.

LGTM, and thanks for the awesome work. I bag my courage to approve.


well, I forgot to approve after leaving a comment, sorry.
Make ViltModel forward method arguments (inputs_embeds and image_embeds) consistent
Author: Infrared1029
Pull request: no

### Feature request

The inputs_embeds arguemnt in Vilt expects the word embeddings (which is [cls_token_emb, token_embs, sep_token_emb]) of the input without adding the pos or token_type embs yet, while the image_embeds argument expects the processed embeddings [cls_token_emb, patch_embs] + pos_embs + token_types , which seems a bit inconsistent?  

### Motivation

I was a bit confused why the following code was not producing the same output and then read the source code an realized the issue:

```
from transformers import ViltProcessor, ViltModel
from PIL import Image
import requests

# prepare image and text
url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)
text = "hello world"

processor = ViltProcessor.from_pretrained("dandelin/vilt-b32-mlm")
model = ViltModel.from_pretrained("dandelin/vilt-b32-mlm")

inputs = processor(image, text, return_tensors="pt")
outputs_1 = model(**inputs)


# using embeds (the wrong way)
img_embs, img_mask, (patch_index, (h, w)) = model.embeddings.visual_embed(inputs['pixel_values'], inputs['pixel_mask'])
txt_embs = model.embeddings.text_embeddings(inputs['input_ids'])
outputs_2 = model(inputs_embeds=txt_embs, image_embeds=img_embs, pixel_mask=img_mask)


# seems to be the correct way
img_embs, img_mask, (patch_index, (h, w)) = model.embeddings.visual_embed(inputs['pixel_values'], inputs['pixel_mask'])
txt_embs = txt_emb = model.embeddings.text_embeddings.word_embeddings(inputs['input_ids']) # word_embeddings instead
outputs_3 = model(inputs_embeds=txt_embs, image_embeds=img_embs, pixel_mask=img_mask)


```

outputs_2 and outputs_3 are (almost) identical (they aren't exact matches but the total difference between all entries is around -0.009) while outputs_1 and outputs_2 are just different.

### Your contribution

Make both arguments either take the processed embeddings (after adding everything) or just the raw embeddings before adding anything. unless ofc, this is a deliberate decision made for reasons I don't know :o. If it is not intended, then I can try and submit a pr that fixes it.

Comments:
Hi,

Thanks for reporting. I'm afraid we can't change it for backwards compatibility reasons, as people might have already used `image_embeds` with the way it is implemented now. Cc'ing @sgugger to confirm

Indeed. The documentation can be clarified however, to make this less surprising to users.

makes sense! i guess the docs could be indeed slightly clarified 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Error while converting BigBirdPegasus tensorflow checkpoints into pytorch model using "convert_bigbird_pegasus_tf_to_pytorch.py"
Author: dehghanm
Pull request: no

Hi,

I want to covert TensorFlow checkpoints generated from training a BigbirdPegasus model for the summarization task into a PyTorch model using the prepared script for it (i.e., "convert_bigbird_pegasus_tf_to_pytorch.py") which is located in the following URL:
https://github.com/huggingface/transformers/tree/main/src/transformers/models/bigbird_pegasus/modeling_bigbird_pegasus.py

During conversion, I faced the following error:
Traceback (most recent call last):
  **File "convert_ckpt.py", line 212, in <module>
    convert_bigbird_pegasus_ckpt_to_pytorch(args.tf_ckpt_path, args.save_dir, config_update=config_update)
  File "convert_ckpt.py", line 202, in convert_bigbird_pegasus_ckpt_to_pytorch
    torch_model = convert_bigbird_pegasus(tf_weights, config_update)
  File "convert_ckpt.py", line 148, in convert_bigbird_pegasus
    raise ValueError(f"could not find new key {new_k} in state dict. (converted from {k})")
ValueError: could not find new key model.decoder.layernorm_embedding.bias.Adafactor in state dict. (converted from pegasus/decoder/LayerNorm/beta/Adafactor)**

Can anybody help me with solving this error?

Thanks in advance



Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Error while running deepseed in dreambooth
Author: aniketgore
Pull request: no

### System Info

NotImplementedError: Could not run 'xformers::efficient_attention_forward_generic' with arguments from the 'CUDA' backend.

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

NotImplementedError: Could not run 'xformers::efficient_attention_forward_generic' with arguments from the 'CUDA' backend.

### Expected behavior

NotImplementedError: Could not run 'xformers::efficient_attention_forward_generic' with arguments from the 'CUDA' backend.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
this error occurs how to fix it
Author: rajatj86
Pull request: no

torchvision\io\image.py:13: UserWarning: Failed to load image Python extension: torch\_jit_internal.py:751: UserWarning: Unable to retrieve source for @torch.jit._overload function: <function _DenseLayer.forward at 0x0000024B8516D550>. warnings.warn(f"Unable to retrieve source for @torch.jit._overload function: {func}.") torch\_jit_internal.py:751: UserWarning: Unable to retrieve source for @torch.jit._overload function: <function _DenseLayer.forward at 0x0000024B851838B0>. warnings.warn(f"Unable to retrieve source for @torch.jit._overload function: {func}.") The cache for model files in Transformers v4.22.0 has been updated. Migrating your old cache. This is a one-time only operation. You can interrupt this and resume the migration later on by calling `transformers.utils.move_cache()`. Moving 6 files to the new cache system 0%|                                                                                            | 0/6 [00:02<?, ?it/s] There was a problem when trying to move your cache: File "transformers\utils\hub.py", line 1077, in <module> File "transformers\utils\hub.py", line 1040, in move_cache File "transformers\utils\hub.py", line 997, in move_to_new_cache File "huggingface_hub\file_download.py", line 841, in _create_relative_symlink Please file an issue at https://github.com/huggingface/transformers/issues/new/choose and copy paste this whole message and we will do our best to help.

Comments:
Hey @rajatj86, it's unfortunate that this issue occurs. Unless you have important data in your Hugging Face cache, I would advise removing it.

Otherwise, please update both your `transformers` and `huggingface_hub` versions and post the message here, as it should contain more information.

Thank you!


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.