noamaanMulla-03/transformers-issues huggingface dataset:
🌐 [i18n-KO] Translated `generation_utils.md` to Korean
Author: yijun-lee
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `generation_utils.md` file of the documentation to Korean.
Thank you in advance for your review.

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [ ] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 N조 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! 차례대로 1 ~ 5조 -->
May you please review this PR?
@jungnerd, @cjfghk5697, @yijun-lee, @mreraser


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. N조 팀원들과 리뷰가 끝난 후에 아래 주석을 노출해주세요! -->
@stevhliu May you please review this PR?

Comments:

LGTM😎

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33818). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
M2M100Tokenizer vocabulary size is not equal to the m2m embedding_size for the "facebook/m2m100_418M" model.
Author: GerrySant
Pull request: no
Labels: name: Core: Tokenization

name: bug

### System Info

- `transformers` version: 4.42.3
- Platform: Linux-5.15.0-113-generic-x86_64-with-glibc2.31
- Python version: 3.12.3
- Huggingface_hub version: 0.23.4
- Safetensors version: 0.4.3
- Accelerate version: 0.32.1
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.3.1+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA A100-SXM4-80GB

### Who can help?

@ArthurZucker 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Run the code snippet below:
```python
from transformers import M2M100Config, M2M100ForConditionalGeneration, M2M100Tokenizer

model = M2M100ForConditionalGeneration.from_pretrained("facebook/m2m100_418M")
tokenizer = M2M100Tokenizer.from_pretrained("facebook/m2m100_418M")

print(len(tokenizer) == model.get_input_embeddings().weight.shape[0])
```

### Expected behavior

I would expect the result of the above code to be True, however it is False. 

Since the size of the tokenizer vocabulary and the `embedding_size` of the model are different, this causes unwanted behaviors. For example, in `examples/pytorch/translation/run_translation.py` there is this code fragment in charge of performing this same check, and in case it is not fulfilled it resizes the model embeddings.

```python
embedding_size = model.get_input_embeddings().weight.shape[0]
if len(tokenizer) > embedding_size:
    model.resize_token_embeddings(len(tokenizer))
```

Comments:
cc @ArthurZucker 

Hey! Feel free to update that step, in general there is absolutely no guarantee that the tokenizer has the same length as the model's input embeddings:
- you can have holes in your tokenizer's vocab
- the embedding can be padded for performance reasons. 
🤗 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Centralized Page for Domain Whitelisting (Hugging Face Models)
Author: AghaDurrani
Pull request: no
Labels: name: Feature request

### Feature request

Is it possible to create a dedicated page that lists all the domains where Hugging Face models, such as those stored on cdn-lfs.hf.co, are hosted? This page would serve as a reference for users working in corporate environments, where whitelisting of individual domains is required.

### Motivation

Many organizations, particularly those behind corporate networks, require that domains be whitelisted individually for security and access reasons. For users working in such environments, identifying and manually whitelisting the necessary domains for model access can be a cumbersome process. A centralized list of the domains Hugging Face uses to host and serve models would significantly streamline this process, ensuring seamless access to models across corporate networks without delays or security issues.



Comments:
cc @Wauplin more your domain! 🤗 

Ping @XciD on this one. Would be worth having an exhaustive list somewhere in the docs

Hey,

Current list is:
```
cdn-lfs-us-1.hf.co
cdn-lfs-eu-1.hf.co
cdn-lfs.hf.co
```

We could maybe create something like: https://api.github.com/meta

thanks for the list!
Phi3: fix attn for sliding window
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/32945. The reason is that Phi3 prev prepared 4D attn with sliding window while the new `updte_causal_mask` didn't take that into account. This PR fixes it and adds a test

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33586). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Done, I think this PR is ready to be reviewed/merged.

Depending on which PR is merged first, this or the linked one, I will rebase and apply necessary changes. Then I'll add the TODO comment for moving `target_length` logic to `_prepare_attention_mask`

Rebased main and updated accordingly by moving `prepare_causal_mask` to XXXModel. Also, noticed Phi3Moe was added while PR was in progress and it is same as Phi3, so I propagated changes there too

Will be merging tomorrow if no comment remain :)

Hey @zucchini-nlp, while working on https://github.com/huggingface/transformers/pull/33619 I had issues with the 4d masks and just found this PR - however, it is not only an issue for Phi3! For what I could see, the following models have the exact same issue (`AttentionMaskConverter._ignore_causal_mask_sdpa()` does not check for the `sliding_window` resulting in wrong masks, and neither do `_prepare_4d_causal_attention_mask_with_cache_position()`). 
- Mimi
- Mixtral
- PhiMoe
- Qwen2
- Qwen2Moe
- Qwen2VL
- Starcoder2

Let me know if you can fix it or if you want me to jump on it.

@Cyrilvallez oh I see, didn't know we had more models that support sliding window. I can propagate changes to other models, sure :)

Yes! I think I listed them all but you can maybe double-check so that all of them get correctly fixed 🤗

@ArthurZucker done! I had to change tests for Qwen2 models because otherwise we won't get same results for long padded input as for the base input. Applying sliding mask results in minor differences
`word_to_chars()` doesn't work as expected for Llama3.1-8b
Author: neurothew
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.45.1
- Platform: Linux-5.15.0-92-generic-x86_64-with-glibc2.35
- Python version: 3.12.4
- Huggingface_hub version: 0.25.1
- Safetensors version: 0.4.3
- Accelerate version: 0.32.1
- Accelerate config:    not found
- PyTorch version (GPU?): 2.3.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA RTX 6000 Ada Generation

### Who can help?

@ArthurZucker @itazap 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I am trying to retrieve the "word" as defined by `word_ids()` by retrieving the character span.

```Python
from transformers import AutoTokenizer
model_name = "meta-llama/Meta-Llama-3.1-8B"
this_tokenizer = AutoTokenizer.from_pretrained(model_name)

this_sent = "Hello World!"
this_encode = this_tokenizer.encode_plus(this_sent)
print(this_encode.word_to_chars(0))
```

And the output is:
```Python
CharSpan(start=0, end=0)
```

It doesn't happen with some other models such as BERT:
```Python
model_name = "bert-base-uncased"
this_tokenizer = AutoTokenizer.from_pretrained(model_name)

this_sent = "Hello World!"
this_encode = this_tokenizer.encode_plus(this_sent)
print(this_encode.word_to_chars(0))
```

With the output being:
```Python
CharSpan(start=0, end=5)
```
And the word "Hello" can be extracted via `this_sent[0:5]` easily. I wonder if it might have something to do with the tokenizer? I have tried BERT, RoBERTa, GPT-2, Qwen2.5 so far, and there were no problems.

For Llama models, I have tried llama3-8b, llama3.1-8b, llama3.2-1b and llama3.2-3b without success.


### Expected behavior

`word_to_chars()` should give the correct character span for llama models.

Comments:
Hey! This is a duplicate of #33675 and should now be fixed. I'll push a new version of tokenizers to propagate this (https://github.com/huggingface/tokenizers/pull/1640) 

https://github.com/huggingface/tokenizers/releases/tag/v0.20.1 Release is out, closing!
HfArgumentParser should parse hyphens in long-option parameters
Author: djmarti
Pull request: no
Labels: name: Feature request

### Feature request

Given a dataclass
```python
from dataclasses import dataclass

@dataclass
class Config:
    item_type: str
    item_length: int
```
I would expect `HfArgumentParser.parse_args_into_dataclasses` to correctly parse parameters like `--item-type` or `--item-length` (with a hyphen instead of an underscore). Unfortunately, it doesn't:
```python
from transformers import HfArgumentParser
parser = HfArgumentParser([Config])
parser.parse_args_into_dataclasses(args=["--item-type", "my_type", "--item-length", "23"])
```
throws an error because it is expecting `--item_type` and `--item_length`.

### Motivation

1. Consistency with `argparse`. [`argparse` converts hyphens into underscores before assignment](https://docs.python.org/dev/library/argparse.html#dest):
   >  If no long option strings were supplied, dest will be derived from the first short option string by stripping the initial - character. Any internal - characters will be converted to _ characters to make sure the string is a valid attribute name. 
2. It is standard convention to use hyphens in long-option parameters (e.g., `--file-type`, not `--file_type`).

### Your contribution

If there is no reason preventing this feature request to be accepted, yes, I could contribute to a PR.
I just want to know if there is a fundamental reason why this is not implemented yet.

Comments:
yeah feel free to open a PR! 

if i understand the problem statement correctly then in "parse_args_into_dataclasses" function in -> src/transformers/hf_argparser.py file , we should be adding 
`
args = [arg.replace("-", "_") if arg.startswith("--") else arg for arg in args]
`
this sort of conditional processing for '--'. 

Correct me if i am wrong though. 
Shall i make a PR if not done yet @djmarti  ?


Hi @GargDivanshu, I have submitted a PR. I had worked already on a fix when I saw your message.
Fix zero shot detection pipeline
Author: qubvel
Pull request: yes
Labels: name: Vision

# What does this PR do?

1. Fix `zero-shot-object-detection` pipeline for grounding dino
2. Speedup `zero-shot-object-detection` in `len(canidate_labels)` times

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #32206

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @SunMarc

Integrations:

- deepspeed: HF Trainer/Accelerate: @muellerzr
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32490). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Sync QuestionAnsweringPipeline
Author: Rocketknight1
Pull request: yes

This PR synchronizes the QuestionAnsweringPipeline with the Hub spec for the `question-answering` task. There's a significant change here: The old pipeline had special handling for `SQuAD` inputs. In fact, you could argue it was primarily a `SQuAD` benchmarking pipeline! This PR deprecates those inputs to focus on the `question` and `context` inputs instead.

Note for reviewers: You can't see it in the diff, but `topk` has already been renamed to `top_k` in the pipeline code, with a deprecation warning. As a result, only docstring changes are needed for that arg.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34039). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Specifying torch dtype in Qwen2VLForConditionalGeneration
Author: htahboub
Pull request: yes

Fixing a bug that led to the warning

```You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour```

every time this model was initialized with `attn_implementation="flash_attention_2"`, even with the dtype specified.

@amyeroberts, @qubvel

Comments:
cc @zucchini-nlp if you have bandwidth

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33953). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@zucchini-nlp Ah I see, agreed in that case. And I agree that changing the fallback from `None` to the default dtype would be a lot cleaner, so I'll make that change to `modeling_utils.py` then and revert the current proposed fix.

Hey @zucchini-nlp, since this is a pretty easy fix for the bug, I suppose we should either merge now or close this PR since the change is already included in your big PR. So, if you think your PR is about to be merged soon, please let me know so we can just close this, or if it's gonna take a while longer, then I think we'd be better off merging this as soon as possible because the warning with Qwen can be a bit confusing to people encountering it. Thanks!

Thanks 🤗
Fix data_seed unused
Author: MekkCyber
Pull request: yes

# What does this PR do?
This PR fixes the problem related to the data_seed being unused. It was removed during a refactor of how samplers used to work.

Fixes https://github.com/huggingface/transformers/issues/31818 https://github.com/huggingface/trl/issues/1810

You need to use this accelerate PR : https://github.com/huggingface/accelerate/pull/3130

For example in this code : 
```
from datasets import load_dataset
from transformers import AutoTokenizer, TrainingArguments, AutoModelForSequenceClassification, Trainer, set_seed
import torch

set_seed(125)

tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
model = AutoModelForSequenceClassification.from_pretrained("bert-base-uncased", num_labels=2)

dataset = load_dataset("glue", "mrpc", split="train")

def tokenize_function(example):
    return tokenizer(example["sentence1"], example["sentence2"], truncation=True)

dataset = dataset.map(tokenize_function, batched=True)
SEED = 126
DATA_SEED = 762
training_args = TrainingArguments(f"test-trainer_dataseed_{DATA_SEED}_seed_{SEED}", max_steps=400, logging_steps=40, seed=SEED, data_seed=DATA_SEED)
trainer = Trainer(
    model,
    training_args,
    train_dataset=dataset,
    tokenizer=tokenizer,
)

trainer.train()
```
When we set the seed using `set_seed`, if the data sampler's seed is chosen using `torch.random.initial_seed()`, it will remain deterministic even when the `data_seed` value changes. To address this, the PR checks whether a generator using the `data_seed` is already available. If it exists, we use the initial `data_seed` provided by the user. If not, we randomly select one. This ensures that even when `set_seed` is used, the randomness of the data sampler remains unaffected when different `data_seed` values are used.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33731). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

cc @MekkCyber  let's update the PR since we merged this PR : https://github.com/huggingface/accelerate/pull/3150#event-14575790028
[PEFT] Support low_cpu_mem_usage option for PEFT loading adapters
Author: BenjaminBossan
Pull request: yes

# What does this PR do?

PEFT added support for `low_cpu_mem_usage=True` when loading adapters in https://github.com/huggingface/peft/pull/1961. This feature is now available when installing PEFT v0.13.0. With this PR, this option is also supported when loading PEFT adapters directly into transformers models.

Additionally, with this PR, https://github.com/huggingface/diffusers/pull/9510 will be unblocked, which implements this option in diffusers.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33725). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@ArthurZucker could you give this a look? 

not sure what's going on with the tests, rebasing should help !

@ArthurZucker I still get `fetch_tests - Failed` from `setup_and_quality`. Any idea what could be the cause? I ran `ruff` on the touched files.

I had to force push as the rebasing you did went a bit wrong! Should be good now! 🤗 

> I had to force push as the rebasing you did went a bit wrong! Should be good now! 🤗

Oops, sorry about that. Anything that needs to be done from my side for this PR?

Nope, I was just waiting for the cis!
[TESTS] ASR pipeline
Author: ylacombe
Pull request: yes

Fix two ASR pipeline tests! 
Both didn't pass before and pass now.

Note that the other failing tests are mostly related to failing word timestamps Whisper generation, that we're working to fix with @eustlb

cc @ydshieh and @ArthurZucker 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33925). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Update Blip2 `is_pipeline_test_to_skip` method signature
Author: qubvel
Pull request: yes

# What does this PR do?

Update Blip2 `is_pipeline_test_to_skip` method signature to fix test failure.
https://app.circleci.com/pipelines/github/huggingface/transformers/107299/workflows/7171a45b-75d9-462e-a77c-ca7abd1cab78/jobs/1423992 

cc @Rocketknight1 

Comments: