noamaanMulla-03/transformers-issues huggingface dataset:
`No such file or directory` when setting `cache_dir`
Author: rcalland
Pull request: no

### System Info

- `transformers` version: 4.26.1
- Platform: macOS-13.1-arm64-arm-64bit
- Python version: 3.10.9
- Huggingface_hub version: 0.13.1
- PyTorch version (GPU?): not installed (NA)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

**Note that `cache_model` directory exists**:

```
from transformers import CLIPProcessor, CLIPModel

processor = CLIPProcessor.from_pretrained("openai/clip-vit-base-patch32", cache_dir="cache_model")


inputs = processor(text=["a photo of a cat", "a photo of a dog"], images=None, return_tensors="np", padding=True)

print(inputs)
```

### Expected behavior

in recent versions of `transformers`, the code snippet above produces the following error:

```
FileNotFoundError: [Errno 2] No such file or directory: 'cache_model/models--openai--clip-vit-base-patch32/snapshots/e6a30b603a447e251fdaca1c3056b2a16cdfebeb/preprocessor_config.json'
```

whereas with `transformers==4.20.1`, the snippet successfully runs

Comments:
I just tried your code sample and it runs without issue on both main and 4.26.1. Are you sure `"cache_model"` is in a folder you have write access to?

hi @sgugger, you are correct, I just made a fresh virtualenv and my snippet indeed does work. However, it doesn't work in my dev environment, nor in my projects CI, so I suspect theres an issue with a dependency somewhere?

I dug into what is happening inside `cache_model` in both environments, and found a clue:

In my new virtualenv:
```
lrwxr-xr-x  1 richard  staff   136B Mar 14 14:38 preprocessor_config.json -> /Users/richard/Projects/huggingface_bug/cache_model/models--openai--clip-vit-base-patch32/blobs/5a12a1eb250987a4eee0e3e7d7338c4b22724be1
```

in my dev environment:
```
lrwxr-xr-x  1 richard  staff    96B Mar 14 15:05 preprocessor_config.json -> cache_model/models--openai--clip-vit-base-patch32/blobs/5a12a1eb250987a4eee0e3e7d7338c4b22724be1
```

you can see that the former creates a symlink to an absolute path, whereas the latter uses a relative path, which may explain why it cannot find the actual file.

Do you have any ideas what could be causing this behaviour?

Might be a different version of `huggingface_hub`?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[WIP] Adding GPT2 with Multi Query Attention
Author: lvwerra
Pull request: yes

# Adding GPT2 with Multi Query Attention

This PR adds a GPT2 architecture with Multi Query Attention (MQA). With MQA the V,K weights are shared across heads and only Qs are unique which makes it possible to run the model with very large batches.

This is the Architecture used in [BigCode's SantaCoder](https://huggingface.co/bigcode/santacoder).

There are a few things to do before we can merge the PR:
- add performance improvements suggested by @jlamypoirier
- fix tests:
    - there is an issue with `past`
    - there is an issue with loading the tokenizer (i guess missing vocab file in repo?)
    - fix the generation examples

You can run the tests with:
```bash
RUN_SLOW=1 python -m pytest -s -v ./tests/models/gpt2mqa/
```

cc @bigximik @jlamypoirier @RaymondLi0 




To review when ready I tag @ArthurZucker and @younesbelkada.

Comments:
Regarding tests `test_batch_generation` and `test_batch_generation_2heads`. If  token initialisation class is changed form `GPT2Tokenizer` to `GPT2TokenizerFast` the test passes through until generated tokens assertion. Is it  intended behaviour or the loading functionality should have  rerouted from the default class?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Closing in favour of #22575
Hardcode GELU as the intermediate activation for ESM
Author: Rocketknight1
Pull request: yes

One more issue revealed by the nucleotide transformer port! This time it's the activation function - ESM uses a hardcoded GELU, which the PyTorch port gets right, but the TF port used an intermediate block copied from BERT which reads `config.hidden_act`. This value was set to `gelu` for all of the original ESM checkpoints, so the bug was silent until we tried making some new checkpoints from scratch. 

This PR replaces `config.hidden_act` with a hardcoded `gelu`. All ESM tests (including slow / cross-tests) pass locally.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Just to understand a bit better, does this mean that the nucleotide model has a different activation set in its config used for other layers? 

Actually, no! It also always expects gelu, which matches the original ESM (both the port to HF and the original repo at Meta). The issue here is that the TF version is reading `config.hidden_act`, but that isn't even set by default - the bug slipped in because whatever way we constructed the original ESM checkpoints, that value was always set in the configs, so the issue was silent until we tried to make new configs for nucleotide transformer and they suddenly broke in TF.

@Rocketknight1 Would a possible solution to this be to update the ESM configuration to have `hidden_act` as `"gelu"` by default? If I've understood correctly, the original ESM model configs have the `hidden_act` attribute. In which case, as a user, if I updated this I'd expect it to be propagated when constructing a model from the config. 

@amyeroberts I'm not sure that's the best course here! `hidden_act` is actually a parameter on the base config that `EsmConfig` inherits from. As such, it's not included in the documentation for `EsmConfig` at all. The attribute just happened to be set (I think by the ESM team) when they created the ESM checkpoints, which masked the bug. I think the right solution is to just not read the attribute at all in the code for either framework.

Also, I spotted one minor issue with the weight tying fix I made for ESM, and I'm sneaking a fix for it into this PR. (decoder should be a layer when it's untied to make sure weight crossloading works properly, not a bare weight matrix).

No probs, I think they helped!
Flaky test for NLLB-MoE Model 
Author: amyeroberts
Pull request: no

### System Info

Environment: Circle CI test_torch image, defined [here](https://github.com/huggingface/transformers/blob/c15f937581048fa28f77794c4f6a257e14d272cb/.circleci/create_circleci_config.py#L211)

### Who can help?

@ArthurZucker 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

The test `tests/models/nllb_moe/test_modeling_nllb_moe.py::NllbMoeModelTest::test_decoder_model_past_with_large_inputs` is flaky and occasionally fails. Note: the tolerance on the test is also very low. 

Error cannot be deterministically reproduced. The test passes when I run it locally. 

An example of a failed run: https://app.circleci.com/pipelines/github/huggingface/transformers/60931/workflows/4642cd50-8d8c-4a6b-bea9-b077ff7400cf/jobs/747651

This came from this run: https://github.com/huggingface/transformers/runs/12389736536

And this unrelated PR: https://github.com/huggingface/transformers/pull/21855  

### Expected behavior

Tests pass on every run. 

Comments:
Yep, I have no idea why 😓 

Looked into this and I think the flakiness is caused by the natural variability in the sparse MoE layers. Specifically that when they calculate which experts to use in the gating logic, they’re computing probabilities imperfectly for two different sets of inputs: one with prior inputs concatenated with the past key values and one with just the past key values. 

The test usually passes cause magnitude of the difference is usually likely to be small. Notably, when the vocab size is increased this pass rate goes up (and vice versa) since the increased representational capacity can help the model make more accurate decisions about which experts to use for each input. For example, increasing the vocab size in the config from its current 99 to 999 increases the pass rate from ~80% to ~95%.

I think this flakiness is inherent in the sparse layers, but if I understand right the point of the test is to check the decoder uses the past properly, so I put up a PR that edits the test to use dense layers and moves to the rtol down to 1e-3 to be in line with the other models’ version of this check.
add pref_train_gpu_one.mdx
Author: Baelish03
Pull request: yes

See issue #17459
Good evening.
I didn't translate technical terms and I preferred to keep them in english. So I hope it's all ok.
I had some problems in previous pull requests; if it fails again, please can you say how to resolve?
Good bye.


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_22926). All of your documentation changes will be reflected on that endpoint.
Fix a minor bug in CI slack report
Author: ydshieh
Pull request: yes

# What does this PR do?

#22798 added code to show the difference between 2 CI runs. However, the previous CI run(s) may not yet produced the artifact `test_failure_tables`, and we got `KeyError: 'model_failures_report.txt'` in the last run.

This PR adds some check.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Feature to convert videomae huge and small finetuned on kinetics and ssv2 added to the videomae to pytorch converter
Author: sandstorm12
Pull request: yes

# What does this PR do?

Adds the feature to convert VideoMAE huge and small to hugging face compatible pytorch model. The following models are added to the converter:

1. VideoMAE huge fine-tuned Kinetics
2. VideoMAE small fine-tuned Kinetics
3. VideoMAE small fine-tuned SSV2

## Who can review?

@NielsRogge

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@NielsRogge Uploaded the huge-kinetics, small-kinetics, and small-ssv2 models to the HF model hub under the following names:

1. sandstorm12/videomae-huge-finetuned-kinetics
2. sandstorm12/videomae-small-finetuned-kinetics
3. sandstorm12/videomae-small-finetuned-ssv2

Let me know if anything else is needed.
Fix GPT2 position ids issues
Author: ArthurZucker
Pull request: yes

# What does this PR do?

Follow up PR of #21080

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_22155). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[trainer] bug in resume and gas>1
Author: stas00
Pull request: no

https://github.com/huggingface/transformers/pull/22098 fixed the issue with GAS>1 at the epoch boundary.

the same bug will still happens at resume boundary, since `total_batched_samples` is currently reset to 0.

So need to save `total_batched_samples` and restore from the saved value on resume.

Comments:
Actually thought more about it this morning. The gradients accumulated before the save will be lost, so even if we save the `total_batched_samples` variable, we won't be able to resume training with the same gradients (they will be 0 instead of whatever was accumulated before the checkpoint).

So I think leaving the situation as is is okay, there is a tiny bit of training lost but it shouldn't impact convergence. And we should document somewhere that we do not guarantee checkpoints will not yield the exact same model using `save_strategy="epoch"` in conjunction with gradient accumulation.

oh, I wrongly assumed that they were saved. Yes, then it makes sense. There will be no miscalculation then, just some very minor intermediary results loss. I think it's all good.



This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Return attention_mask in FeatureExtractionPipeline output
Author: anruijian
Pull request: no

### Feature request

Return `attention_mask` as one output of the FeatureExtractionPipeline so that padding token embeddings can be ignored.

### Motivation

**Who can help?**
@Narsil

When using the `FeatureExtractionPipeline` to generate sentence embeddings, the input to the pipeline processes a raw sentence with a tokenizer. The output of the pipeline is a tensor of shape `[1, seq_len, hidden_dim]`. If the input is padded, `seq_len` is equal to the `max_length` of the tokenizer or longest seq in the batch. 

However, when performing mean pooling of individual word embeddings to obtain the sentence embedding, one may want to use `attention_mask` in order to ignore the padding token embeddings (see the mean pooling example below). But, FeatureExtractionPipeline does not return `attention_mask` as part of its output.

```python
#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    sum_embeddings = torch.sum(token_embeddings * input_mask_expanded, 1)
    sum_mask = torch.clamp(input_mask_expanded.sum(1), min=1e-9)
    return sum_embeddings / sum_mask
```


### Your contribution

I can submit a pull request to the issue if it sounds good to you!

Comments:
This doesn't seem like a use-case for the pipeline though. Since you want access to the process inputs, you should just used the tokenizer and the model directly.

Your comment makes sense. As my goal aligns with the pipeline's main functionality, I think I will subclass `FeatureExtractionPipeline` and make small modifications to achieve my goal. Feel free to close the issue. Thank you!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Something confused me about Nystromformer
Author: 1649759610
Pull request: no

### System Info

Any GPU Machine with Transformer 4.26.0

### Who can help?

@ArthurZucker @younesbelkada @sgugger @novice03 

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

The parameter `segment-means-seq-len`  in Nystromformer config is set to be 64, and is equal to another parameter num_landmarks (64). [refer to code](https://github.com/huggingface/transformers/blob/main/src/transformers/models/nystromformer/configuration_nystromformer.py#L106)

But if they are equal, the Nystromformer will perform a O(n2) attention like bert,  not the nystrom-attention proposed in original paper: https://arxiv.org/abs/2102.03902.  [refer to code](https://github.com/huggingface/transformers/blob/main/src/transformers/models/nystromformer/configuration_nystromformer.py#L106)

Through experimentation and anslysis, I think the parameter `segment-means-seq-len` should be the length of the  tokenized input sequence.  It should not be set to 64， if you set to be 64，It means you wanna use O(n2) attention，not nystrom attention.

So,  there is a problem with the code, or is my understanding wrong?  Addtionally, whether the model weight [w-madison/nystromformer-5](https://github.com/huggingface/transformers/blob/main/src/transformers/models/nystromformer/configuration_nystromformer.py#L24) is trained with O(n2)？ if yes, whether the modlel weight will not run with nystrom-attention, it need to be pretrain with nystrom-attention?

### Expected behavior

The parameter `segment-means-seq-len` is set to be the real tokenized sequence length, so the nystrom-attention can be used to train or inference.

Comments:
I'll ping @novice03 here as he's an expert on Nyströmformer 

Hello @1649759610, thank you for making this post. It looks like this is indeed an issue in the code that I might have overlooked. You are correct that `segment-means-seq-len` should be set to the length of the input. If I were to fix it, I would just remove the `segment-means-seq-len` parameter and set `self.seq_len` in the model to `config.max_position_embeddings`. I think a pull request would have to be made to make these changes. I am also guessing that the tests need to be changed accordingly. 

However, regarding the checkpoints, they were trained with Nystrom attention and not O(n^2) attention. This is just an issue in the HuggingFace implementation. So, they need not be re-trained. 

@1649759610 Would you like to make a PR about this?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
🌐 [i18n-KO] Translated `pipeline_tutorial.mdx` to Korean
Author: wonhyeongseo
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Part of #20179 (issue)


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@wonhyeongseo, @jungnerd, @0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee
Team Pseudo-Lab, please review this PR.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hello Mr. @sgugger , thank you for your patience! May you please review & merge this PR?

The internal review time frame (7 days) has passed since I requested feedback from my colleagues at PseudoLab.
Flan-T5-small and T5-small have different number of layers?
Author: taidnguyen
Pull request: no

### System Info

Hi there, it appears that `google/flan-t5-small` and `t5-small` have different number of layers:
- Flan-T5 config: https://huggingface.co/google/flan-t5-small/blob/main/config.json
- T5 config: https://huggingface.co/t5-small/blob/main/config.json

I only find this inconsistency with *small*. The rest of the sizes (base/large/3b/11b) seem to match up for these two sets of models. I have not been able to find much information on this.

Is there a reason for Flan-T5-small to have more layers than its non instruction-tuned counterpart? I assume they should be equal. Thank you!

### Who can help?

CC: @sgugger 

Comments:
cc @younesbelkada 

@sgugger Ah perhaps the authors inherited the flan-t5-small checkpoint from the improved `google/t5-v1_1-small` instead of `t5-small`. Its config file is a better match.

Hi @taidnguyen,
This is absolutely correct, 
According to the `t5x` repository, flan-t5 are derived from the `t5-v1_1` family as their config files refer to the config files of the `t5-v1_1` models. This information can be found on the original repository that hosts the original flan-t5 weights: https://github.com/google-research/t5x/blob/main/docs/models.md#flan-t5-checkpoints 
Hope this helps!

@younesbelkada That helps - thank you! They documented this in their repo (as you show) but not in their paper, so definitely a surprise to me assuming that the inherited checkpoint of Flan-T5 is the original T5. I will close the issue.