noamaanMulla-03/transformers-issues huggingface dataset:
Update feature selection in to_tf_dataset
Author: amyeroberts
Pull request: yes

# What does this PR do?

Updates feature selection to ensure returned dataset structure is consistent after merging of datasets PR: https://github.com/huggingface/datasets/pull/5602. The PR makes it possible to return a TF dataset with a dict structure even if only a single feature is selected. 

Compatibility with this version of datasets was run with [this commit](https://github.com/huggingface/transformers/pull/21935/commits/b64204bc1093edd7e3666ad76354fa09405cf4ec) and had a [successful run](https://github.com/huggingface/transformers/actions/runs/4555221658/jobs/8034018899).

Note: In all the cases here, the examples were tested with and without these updates. The models would successfully train with both the new and old dataset structures. 

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
_The documentation is not available anymore as the PR was closed or merged._

cc @MKhalusova Reference on updated API for selecting features from TF datasets

Thanks for updating the docs too! Looks neat :)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
🌐 [i18n-KO] translate `create_a_model` doc to Korean
Author: gabrielwithappy
Pull request: yes

# What does this PR do?

Translated the `create_a_model.mdx` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?
Team PseudoLab, may you please review this PR?
@0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd

@sgugger, @ArthurZucker, @eunseojo May you please review this PR? 

<!-- @sgugger, @ArthurZucker, @eunseojo May you please review this PR? -->
<!-- Team PseudoLab, may you please review this PR?
@0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd -->

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Team PseudoLab, may you please review this PR?
@0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd


리뷰 해주신 내용 확인했습니다. 
확인해보고 다시 업데이트 할께요~

@sgugger, @ArthurZucker, @eunseojo May you please review this PR? 

There are still a couple of comments left to address @gabrielwithappy 

@sgugger oh, I just found! I will chech and change to draft before ready-to-review 
:-)

@sgugger, @ArthurZucker, @eunseojo 
May you please review this PR?
I checked anchors of titles in the doc and fixed all reviews 
BRs.
Generate: assisted generation with sample (take 2)
Author: gante
Pull request: yes

# What does this PR do?

I was writing the blog post about assisted generation and realized that there is a much better way to solve the `do_sample=True` case 👀 Apologies for the repeated review request, but I believe this is a significant upgrade.

In a nutshell, the existing `temperature` argument provides a natural control mechanism for assisted generation with `do_sample=True`, when controlling how flat the distribution is at the sampling step. Lower temperature = high probability tokens become more likely to be sampled = more predictable = more likely to match the candidate tokens from the assistant model = assisted generation works faster.

Compared to the [other PR](https://github.com/huggingface/transformers/pull/22862), which was closed, this approach has the following pros and cons:
- Pros:
  - No new argument;
  - Behaves exactly like `.sample`, which users already understand well. No new heuristics;
  - As fast as the other method for similar randomness levels (see numbers below).
- Cons:
  - Internally, more than one sampling step will occur per output token. If we set a seed, the output will be different than `.sample`'s for the same seed. Not a deal breaker per se, but it means subtle bugs may be tough to catch.

## Performance numbers

  I've run the [benchmark](https://github.com/gante/huggingface-demos/tree/main/experiments/faster_generation) I've been running for assisted generation, but now for several `temperature` values. The values below are for `facebook/opt-6.7b` as the main model, `facebook/opt-125m` as the assistant model, running `.generate` starting from inputs taken from the C4 test set (i.e. quite random, the dataset I tested where assisted generation struggles the most), on a RTX3090. Note that most LLMs nowadays use temperature between 0.7 and 0.9.

TL;DR -- it's slower than greedy assisted generation, as it is expected, but it will still secure solid speedups e.g. with INT8.
  
<img width="418" alt="Screenshot 2023-04-23 at 14 58 10" src="https://user-images.githubusercontent.com/12240844/233844046-953999b7-3f9b-4f97-bab6-b4e4f50943ef.png">
  

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Install `accelerete@main` in PyTorch Past CI jobs.
Author: ydshieh
Pull request: yes

# What does this PR do?

Install `accelerete@main` in PyTorch Past CI jobs.

### Context

In ##22393, we added back `deepspeed` in Past CI docker image. Later in #22859, we decide to use `accelerate@main`, but I forgot to apply the same change in Past CI docker file, as I mistakenly thought Past CI doesn't use `accelerate`.

- However, `[deepspeed-testing]` (installed in Past CI docker) includes `accelerate`, and we want it to be `accelerate@main`.
- We can't include `acclerate` it in the docker image, as for the TF Past CI, it will break something
  - there was a remark: `accelerate requires torch, and this causes import issues for TF-only testing` 



Comments:
_The documentation is not available anymore as the PR was closed or merged._
Pix2struct screen2words not working
Author: lambainsaan
Pull request: no

### System Info

- `transformers` version: 4.27.2
- Platform: macOS-13.2.1-x86_64-i386-64bit
- Python version: 3.10.10
- Huggingface_hub version: 0.13.3
- PyTorch version (GPU?): 2.0.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?[/GPU](https://file+.vscode-resource.vscode-cdn.net/GPU)?[/TPU](https://file+.vscode-resource.vscode-cdn.net/TPU)?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help?

@younesbelkada 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

When running the code,

```
from transformers import AutoProcessor, AutoModelForSeq2SeqLM

processor = AutoProcessor.from_pretrained("google/pix2struct-screen2words-large")

model = AutoModelForSeq2SeqLM.from_pretrained("google/pix2struct-screen2words-large")
```

I am getting the error

```
[transformers/models/auto/processing_auto.py:270](/codes/pix2struct-tryout/~/codes/pix2struct-tryout/.venv/lib/python3.10/site-packages/transformers/models/auto/processing_auto.py:270), in AutoProcessor.from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
    267     else:
    268         processor_class = processor_class_from_name(processor_class)
--> 270     return processor_class.from_pretrained(
    271         pretrained_model_name_or_path, trust_remote_code=trust_remote_code, **kwargs
    272     )
    274 # Last try: we use the PROCESSOR_MAPPING.
    275 if type(config) in PROCESSOR_MAPPING:

AttributeError: 'NoneType' object has no attribute 'from_pretrained'
```

### Expected behavior

The model variable must be populated with the model.

Comments:
Hi @lambainsaan, thanks for raising this issue!

Pix2Struct was merged into `main` after the 4.27.2 release. To get the most recent version of the codebase, you can install from the dev branch by running: 
`pip install git+https://github.com/huggingface/transformers`.

Note: It is not possible to load `Pix2Struct` with `AutoModelForSeq2SeqLM` API

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

I believe this issue is fixed, closing it now!
[`SAM`] Add sam doc
Author: younesbelkada
Pull request: yes

# What does this PR do?
 
As suggested by @LysandreJik offline, this PR adds a nice docstring for `SamModel` showing users how to leverage Auto API to run SAM


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Failing tests seems to be unrelated, merging!
[DOCTEST] Refactor doctest for simplicity and safety
Author: ArthurZucker
Pull request: yes

# What does this PR do?
This a a draft PR to simplify the testing of documentation, which would rely on the `doctest` API. 
The documentation tests related to any model added will also be run as part of the CI that are not slow, juste to make sure that we fix everything in one go. (and that the contributors have an easier time doing it). 
I am also learning about github workflows and CI jobs, which is a good exercice! 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_21152). All of your documentation changes will be reflected on that endpoint.
[i18n-<languageCode>] Translating docs to <languageName>
Author: FEXPLORE
Pull request: no
Labels: name: WIP

<!--
Note: Please search to see if an issue already exists for the language you are trying to translate.
-->

Hi!

Let's bring the documentation to all the <languageName>-speaking community 🌐 (currently 0 out of 267 complete)

Who would want to translate? Please follow the 🤗 [TRANSLATING guide](https://github.com/huggingface/transformers/blob/main/docs/TRANSLATING.md). Here is a list of the files ready for translation. Let us know in this issue if you'd like to translate any, and we'll add your name to the list.

Some notes:

* Please translate using an informal tone (imagine you are talking with a friend about transformers 🤗).
* Please translate in a gender-neutral way.
* Add your translations to the folder called `<languageCode>` inside the [source folder](https://github.com/huggingface/transformers/tree/main/docs/source).
* Register your translation in `<languageCode>/_toctree.yml`; please follow the order of the [English version](https://github.com/huggingface/transformers/blob/main/docs/source/en/_toctree.yml).
* Once you're finished, open a pull request and tag this issue by including #issue-number in the description, where issue-number is the number of this issue. Please ping @ArthurZucker, @sgugger for review.
* 🙋 If you'd like others to help you with the translation, you can also post in the 🤗 [forums](https://discuss.huggingface.co/).

## Get Started section

- [ ] [index.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/index.mdx) https://github.com/huggingface/transformers/pull/20180
- [ ] [quicktour.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/quicktour.mdx) (waiting for initial PR to go through)
- [ ] [installation.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/installation.mdx).

## Tutorial section
- [ ] [pipeline_tutorial.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/pipeline_tutorial.mdx)
- [ ]  [autoclass_tutorial.mdx](https://github.com/huggingface/transformers/blob/master/docs/source/autoclass_tutorial.mdx)
- [ ]  [preprocessing.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/preprocessing.mdx)
- [ ]  [training.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/training.mdx)
- [ ]  [accelerate.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/accelerate.mdx)
- [ ]  [model_sharing.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_sharing.mdx)
- [ ]  [multilingual.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/multilingual.mdx)

<!--
Keep on adding more as you go 🔥
-->


Comments:
fixed small typo in code example
Author: jvanmelckebeke
Pull request: yes

# What does this PR do?

Fixes a small typo in a code example in the single GPU inference docs. Renamed the `text` variable to `prompt` as `prompt` is used in the line below as parameter for the tokenizer.


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Documentation: @sgugger , @stevhliu and @MKhalusova

(sorry if tagging is too much for just this tiny repo)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Avoid invalid escape sequences, use raw strings
Author: TurtleOrangina
Pull request: yes

# What does this PR do?
Fixes invalid escape sequences in strings, they are illegal in python, and will throw a SyntaxError if run with "-W error", and always throw a syntax error starting with python-3.12 (planned). With python between 3.6 and 3.11 and "-W default" they produce a DeprecationWarning:
```
> python -W error -c '"(.*?)-\d{5}-of-\d{5}"'                                                                                                                                 ✔    File "<string>", line 1
    "(.*?)-\d{5}-of-\d{5}"
    ^^^^^^^^^^^^^^^^^^^^^^
SyntaxError: invalid escape sequence '\d'
```
This has been fixed in the past, e.g. https://github.com/huggingface/transformers/pull/4924 - but missing linter support and the fact that python only fails this with "-W" flag set has let the issue be re-introduced. This PR fixes those occurrences and enables ruff "W605" error, which will prevent this for the future. 

## Who can review?

Maybe
@sgugger who added ruff in the first place or (Why was "W605" disabled when switching to ruff?)
@patrickvonplaten wo recently introduced some invalid escape sequences
@LysandreJik who merged the linked fix from 2020



Comments:
_The documentation is not available anymore as the PR was closed or merged._

Failures are unrelated to this PR and due to the last release of huggingface_hub. All is fixed on main so merging :-)

Rebased on top of current main branch to pass the tests.



> Rebased on top of current main branch to pass the tests.

Does not seem to help the tests, let me know if I should do anything else for this PR

The Hub is currently having high-response times due to some abusive traffic, which is why the tests are all red. I just forgot to push the merge button yesterday, so merging this as it shouldn't have any negative impact on main.
[`DocTest`] Fix correct checkpoint
Author: younesbelkada
Pull request: yes

# What does this PR do?

Related failing test: https://github.com/huggingface/transformers/actions/runs/4793034296/jobs/8525118203
Sets the correct (and lighter) checkpoint name in the docstring 

cc @amyeroberts @ArthurZucker 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
🌐 [i18n-KO] Fixed `tasks/masked_language_modeling.mdx`
Author: HanNayeoniee
Pull request: yes

# What does this PR do?

Fixes #22838

![image](https://user-images.githubusercontent.com/33839093/234049816-7c69151b-e2d1-4018-9a3c-bf9778723f68.png)



## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
Team PseudoLab, may you please review this PR?
@0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd

May you please review this PR?
@sgugger, @ArthurZucker, @eunseojo

Comments:
_The documentation is not available anymore as the PR was closed or merged._

LGTM. If you may, please:

- **amend your commit message to be more descriptive than just `fixed`.**
  (e.g. `fix: docs: missing newline before code block`)
- (Optional) provide relevant screenshots of your fixes

> LGTM. If you may, please:
> 
> * **amend your commit message to be more descriptive than just `fixed`.**
>   (e.g. `fix: docs: missing newline before code block`)
> * (Optional) provide relevant screenshots of your fixes

Thanks for your review!
I amended the last commit message and added a screenshot in my first comment!
Hosted Files Compression
Author: jozefchutka
Pull request: no

### Feature request

I am very new to hugging face, so I am not sure this is the right place to request, if not please guide me.

I was thinking, the hosted files (i.e. models) could use compression like brotli. Considering its all static files this could be done once instead on per request. 

For example, [decoder_model_merged.onnx](https://huggingface.co/Xenova/transformers.js/blob/main/quantized/openai/whisper-tiny.en/speech2seq-lm-with-past/decoder_model_merged.onnx) has ~50MB but can be compressed ~30MB using brotli:

```
brotli decoder_model_merged.onnx -o decoder_model_merged.onnx.br -Z -f
```


### Motivation

There are many sites and online demos using huggingface cdn, fetching large models. There might be substantial reduction in traffic and waiting times if these files are compressed. Considering it follows the request headers and serves with proper response headers this will be very transparent (no code change needed) for end devs / users.

### Your contribution

I am sorry, I am no expert in this field nor I have knowledge of cdn architecture used

Comments:
Answered in https://github.com/huggingface/huggingface_hub/issues/1446#issuecomment-1521893687 on why this will unfortunately not be supported anytime soon :confused: Since we have a generic issue in `huggingface_hub`, I think we can close this one.
Place static llama variables for multigpu
Author: xloem
Pull request: yes


# What does this PR do?

When using accelerate, attention_mask and position_ids were being retransferred for every layer after the first device. This change transfers them once in advance.



## Who can review?

@ArthurZucker @pacman100



Comments:
Not really in favor of this, if the problem is with the way accelerate handles a for loop, should be solved in accelerate. cc @sgugger 

I am imagining accelerate might be able to store a cache of variables by object id so as to not repeatedly transfer the same one. When to empty such a cache is unclear to me.

_The documentation is not available anymore as the PR was closed or merged._

This is already done by Accelerate behind the scenes, so there is no need for this PR.

@sgugger Accelerate moves the weights prior to the model forward function. Since `attention_mask` and `position_ids` (unlike `hidden_states`) are never returned back from a forward function, it moves them again and again for every layer.

I'm not sure the actual time you lose for that is worth changing the code in Transformers however.

I’m on a system with iommu=soft where data transfer is very slow. I wanted to provide numbers for the speed change on my system, which was significant enough that I opened this PR, before closing it out. However, I am busy for a day or two.

Regardless it is clear that you would prefer a solution be found for accelerate than transformers. I opened this after seeing the various PP commits adding similar code, although they are addressing a more serious issue.

I’ll come back to this to add my numbers or feel free to close it out for now.

Running llama 65b with [software iommu](https://github.com/pytorch/pytorch/issues/1637#issuecomment-338268158), this change drops my inference time from 25.11 s/token to 19.45 s/token, which is 22.5% of the inference delay. Thoughts?

I discussed it more internally with other core maintainers and we decided this falls into specific-hardware optimizations that we don't accept to avoid bloating the code of the models. You can still use your changes locally and share them with others via our code in the Hub API though.

Thanks for your consideration and ideas of other approaches.