noamaanMulla-03/transformers-issues huggingface dataset:
Add VideoInpaintPipeline for temporally-consistent diffusion-based video inpainting
Author: Aki-07
Pull request: no
Labels: name: Feature request

### Feature request

Introduce a new pipeline to extend existing image inpainting capabilities (StableDiffusionInpaintPipeline) to videos. The goal is to provide a native, GPU-optimized API within Diffusers that performs temporally coherent video inpainting instead of independent per-frame processing.

### Motivation

Current video inpainting approaches in the community simply loop over frames and call the image inpainting pipeline repeatedly.
This leads to:
 
- Temporal flicker and inconsistent textures between frames.
- Poor GPU utilization and high memory overhead.
- Lack of tools to maintain motion coherence or reuse diffusion latents across time.

A built-in VideoInpaintPipeline would make it possible to remove objects, restore scenes, or creatively edit videos using diffusion models while keeping motion and lighting consistent across frames.

### Your contribution

I plan to:

- Implement VideoInpaintPipeline as a subclass of DiffusionPipeline, leveraging StableDiffusionInpaintPipeline under the hood.
- Add temporal consistency mechanisms, such as latent reuse between frames and optional optical-flow–guided warping (RAFT / GMFlow).
- Optimize performance through batched FP16 inference, scheduler noise reuse, and optional torch.compile acceleration.

Provide a clean user API compatible with existing pipelines:
```
from diffusers import VideoInpaintPipeline

pipe = VideoInpaintPipeline.from_pretrained(
    "runwayml/stable-diffusion-inpainting",
    use_optical_flow=True,
    compile=True,
)

result = pipe(
    video_path="input.mp4",
    mask_path="mask.mp4",
    prompt="replace background with a snowy mountain",
    num_inference_steps=10,
)
result.video.save("output.mp4")

```
- Contribute documentation and tests demonstrating temporal coherence, performance benchmarks, and example notebooks for real-world use.

Comments:
Could be interesting! Not sure who to ping - cc @molbap @yonigozlan @NielsRogge @sayakpaul. I'm happy to review the pipeline code, but I'd like someone more familiar with video diffusers to tell me if it's worth having a pipeline for this or if it's very specific/niche

Hey @Aki-07! Please open an issue on diffusers ;)

Sure @yonigozlan, updated :)
https://github.com/huggingface/diffusers/issues/12477

Hey @Aki-07 would it be okay if you closed this issue as it is already tracked in diffusers?

Sure, Linking the diffusers issue here
[Add VideoInpaintPipeline for temporally-consistent diffusion-based video inpainting](https://github.com/huggingface/diffusers/issues/12477)
torch 2.9 don't ❤️ torchcodec 💔 
Author: ydshieh
Pull request: yes

# What does this PR do?

Let's pin torch 2.8 for now. 

See https://github.com/meta-pytorch/torchcodec/issues/912

It is causing problems on our CircleCI for now.


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41610). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix ckpt in docs
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/41640 and updates the ckpt to the correct one

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41659). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: florence2
AttributeError: BartTokenizerFast has no attribute image_token. Did you mean: 'mask_token'?
Author: conceptofmind
Pull request: no
Labels: name: bug

### System Info

Ubuntu

### Who can help?

_No response_

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
import torch
import requests
from PIL import Image
from transformers import AutoProcessor, Florence2ForConditionalGeneration


model = Florence2ForConditionalGeneration.from_pretrained(
    "microsoft/Florence-2-large",
    dtype=torch.bfloat16,
)
processor = AutoProcessor.from_pretrained("microsoft/Florence-2-large")

url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/car.jpg?download=true"
image = Image.open(requests.get(url, stream=True).raw).convert("RGB")

task_prompt = "<OD>"
inputs = processor(text=task_prompt, images=image, return_tensors="pt").to(model.device, torch.bfloat16)

generated_ids = model.generate(
    **inputs,
    max_new_tokens=1024,
    num_beams=3,
)
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=False)[0]

image_size = image.size
parsed_answer = processor.post_process_generation(generated_text, task=task_prompt, image_size=image_size)

print(parsed_answer)
```

### Expected behavior

```
    raise AttributeError(f"{self.__class__.__name__} has no attribute {key}")
AttributeError: BartTokenizerFast has no attribute image_token. Did you mean: 'mask_token'?
```

Comments:
`transformers>=4.53.3`

Seeing lots of missing weights in that checkpoint too - cc @zucchini-nlp if you can figure this one out!

Oh, for the non-remote code loading we need to use converted weights which are in https://huggingface.co/florence-community. Let me check if the docs are all updated, but it should have been already

How should something such as this be converted, then? https://huggingface.co/MiaoshouAI/Florence-2-large-PromptGen-v2.0

I imagine these changes break compatibility with numerous previously trained models. Proper conversion should be added to the documentation.

You have the conversion scripts available in model folders usually, for Florence it is in https://github.com/huggingface/transformers/blob/main/src/transformers/models/florence2/convert_florence2_original_pytorch_to_hf.py

It was used to convert the microsoft weights, so if the finetuned models has the same format as the official checkpoint you can use the above script
where is examples/rag from original paper?
Author: IgorKasianenko
Pull request: no
Labels: name: bug

### System Info

https://arxiv.org/pdf/2005.11401 mentions https://github.com/huggingface/transformers/blob/main/examples/rag but it is not there. Add redirect if possible

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Go to https://github.com/huggingface/transformers/blob/main/examples/rag

### Expected behavior

some example instead of 404

Comments:
can i get to solve this issue? would be willing to help. i am very interested in solving this.

Hi @IgorKasianenko, the RAG example (and others as well) seems to have been moved to other repository: [huggingface/transformers-research-projects/rag](https://github.com/huggingface/transformers-research-projects/tree/main/rag)

They were removed in this PR: https://github.com/huggingface/transformers/pull/36645

I’d like to fix this issue as part of Hacktoberfest 2025. Could you assign this issue to me.

is anyone working on this?


There's nothing to fix here, unfortunately! Many legacy examples were becoming difficult to maintain and triggering security warnings, and weren't really relevant to modern workflows, so we moved them to [huggingface/transformers-research-projects/](https://github.com/huggingface/transformers-research-projects/tree/main/)

Hey @IgorKasianenko okay to close this issue?
torch 2.9 ❤️ torchcodec now 🎉  (maybe) 
Author: ydshieh
Pull request: yes

Reverts huggingface/transformers#41610

We need this PR otherwise the new torchcodec 0.8  will break the pinned torch 2.8

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41682). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Ci crying becausse torchcodec
Author: ydshieh
Pull request: yes

# What does this PR do?

torchcodec 0.8 is out. With torch 2.8, work crash issues. with torch 2.9, some torchcodec issues.

Let's pin further torchcodec<0.8 for now

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41684). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
torch 2.9 still don't ❤️ torchcodec 0.8 💔
Author: ydshieh
Pull request: yes

# What does this PR do?

torchcodec 0.8 is out. With torch 2.8, work crash issues. with torch 2.9, some torchcodec issues.

Let's pin further torchcodec<0.8 for now

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41686). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Remove deprecated `use_auth_token` parameter
Author: Wauplin
Pull request: yes

`use_auth_token` parameter has been deprecated for a very long time (2 years?) and will be completely removed from `huggingface_hub` in v1.0.

I think `transformers` v5 is a perfect timing to also remove it from this library. In practice there has been a deprecation cycle for a long time (with both `token` and `use_auth_token` being accepted) so friction should be minimal.

(cc @hanouticelina)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41666). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Can someone merge this one? I feel that the only failing test is not related to my PR :smiley: 

```
FAILED examples/pytorch/test_pytorch_examples.py::ExamplesTests::test_run_semantic_segmentation - AssertionError: 0.0948333740234375 not greater than or equal to 0.1
```

(cc @ydshieh ?)

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, bark, clvp, qwen2_5_omni, wav2vec2

@Wauplin I can't merge as the CI is red: jobs have crashing worker. Could you verify if those are caused by your PR?

Arg, the CI from yesterday's commit worked perfectly fine, except for the test I mentioned. Now that I rebased from main it seems that all tests are crashing. I suspect this is not due to my PR but that it's also failing on `main`. Let's not merge for now

it's torchcodec issue, sorry.

I will built the new docker images and push
[v5] Remove deprecated `use_auth_token`
Author: cyyever
Pull request: yes

# What does this PR do?

Remove `use_auth_token` from code and docs.

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, bark, qwen2_5_omni, wav2vec2

This seems pretty core, so pinging @ArthurZucker @cyrilvallez!
Remove  require_torch_bf16_gpu
Author: cyyever
Pull request: yes

# What does this PR do?

Remove `require_torch_bf16_gpu` because it is now unused.

Comments:
mostly used in testing so cc @ydshieh

@ydshieh Shouldn't affect tests.

sorry being late. Happy to merge. Let me know if you want to push the commit .

@ydshieh Because the factoring of @SunMarc , not sure whether this PR is needed.
🚨 Remove torchscript support
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. The API is old and very badly supported. It's also superseded by dynamo


Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: bark, bart, bigbird_pegasus, bloom, conditional_detr, csm, d_fine, dab_detr

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41688). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Remove `use_auth_token`
Author: SunMarc
Pull request: yes

# What does this PR do?

This PR removes `use_auth_token` for v5

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41588). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@bot /style 

Style bot fixed some files and pushed the changes.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, bark, clvp, qwen2_5_omni, wav2vec2

@bot /style 

Style bot fixed some files and pushed the changes.
Enable faiss-cpu on Windows
Author: cyyever
Pull request: yes

# What does this PR do?

The current faiss-cpu supports Windows. Therefore the PR enables that test case.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41678). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Use | for Optional and Union typing 
Author: cyyever
Pull request: yes

# What does this PR do?
This PR replaces `Optional` and `Union` typing with `|` in some src files.
<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41675). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix Pylint warnings
Author: cyyever
Pull request: yes

# What does this PR do?

This PR fixes some Pylint warnings.

Comments:
@Rocketknight1 
Indeed raise should follow an Exception, otherwise the [documentation](https://docs.python.org/3.14/reference/simple_stmts.html#raise) says
```
If no expressions are present, raise re-raises the exception 
that is currently being handled, 
which is also known as the active exception.
```
IMO we should add the missing Exception.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: bart, blenderbot, blenderbot_small, edgetam, fsmt, led, marian, mbart, mvp, opt, pegasus, pegasus_x, plbart, qwen2_vl, qwen3_vl, rag

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41644). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@Rocketknight1 CI is now green.
Add function for reversing chat templates
Author: LakeYin
Pull request: no
Labels: name: Feature request

### Feature request

Currently `tokenizer.apply_chat_template` can be used to convert a chat in the form list of dicts into a formatted string for text generation. However, there is currently no convenient method for doing the reverse. A new function (hypothetically called `tokenizer.parse_chat_template`) could handle this.

See also:
https://stackoverflow.com/questions/79248499/how-to-reverse-the-tokenizer-apply-chat-template-method-and-handle-streaming-r
https://stackoverflow.com/questions/79248486/how-to-reverse-the-tokenizer-apply-chat-template

### Motivation

Logically if there is a function for converting a chat into a string, there should be a function that does the reverse, which would make things easier when working with `AutoModelForCausalLM` with chat models. This feature is already implemented in the `pipeline` API, so it should be straightforward to create a version that's exposed to end users. 

### Your contribution

N/A

Comments:
I think it's not possible to do so, considering we can put whatever we want inside the user prompt 

This is a feature that's currently under development! It's been quite difficult to make it work,but I'm actively working on finalizing a spec: https://github.com/huggingface/transformers/pull/39609

This is so needed
🚨 Remove torch.fx support
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. The API is old and very badly supported. It's also superseded by dynamo

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41683). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: aimv2, albert, align, altclip, arcee, audio_spectrogram_transformer, aya_vision, bamba, gpt_neo, gptj, hiera, longt5, mt5, pix2struct, pop2piano, recurrent_gemma
Remove skipped tests without parents
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

Tensorflow tests that were still present! And a function not used anymore after https://github.com/huggingface/transformers/pull/41683 and https://github.com/huggingface/transformers/pull/41688


Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: mpnet, speech_to_text, tapas

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41691). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[`Attn`] Allow dynamic causality in SDPA via Kwargs
Author: vasqu
Pull request: yes

As per title, it's
- 1. Nice for power users
- 2. Apparently some models do need to switch like CLIP (depending on their input, see https://github.com/huggingface/transformers/pull/41673#discussion_r2438734215)

This allows us to rely on the set module's attribute per default but overwrite with kwarg if given. cc @zucchini-nlp 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41692). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

https://github.com/huggingface/transformers/pull/41692#pullrequestreview-3350812991 cc @yonigozlan @molbap for vision models refactors to keep in mind!
Add optional logits slicing support to GPT2LMHeadModel
Author: abdulrehhhhman
Pull request: yes

This PR adds support for an optional `logits_to_keep` argument in GPT2LMHeadModel.
It allows users to slice logits output for memory-efficient inference or analysis.

Example:
```python
outputs = model(**inputs, logits_to_keep=2)


Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gpt2

Hi @huggingface-team 👋
This PR introduces an optional logits_to_keep argument for GPT2LMHeadModel.forward, enabling selective slicing of output logits for memory-efficient inference and analysis.
It is backward-compatible and passes the existing GPT-2 test suite locally.
Thanks for reviewing!

The two lines above already do this!

```python
slice_indices = slice(-logits_to_keep, None) if isinstance(logits_to_keep, int) else logits_to_keep
logits = self.lm_head(hidden_states[:, slice_indices, :])
```

Thanks for the clarification! That makes sense — I see it’s already handled by those lines.
I learned a lot from digging into GPT2LMHeadModel’s internals
Simplify GQA conditions in sdpa_attention.py
Author: justinchuby
Pull request: yes

Removed unnecessary checks for key being a torch.fx.Proxy in GQA conditions because fx tracing is no longer supported, and torch.export supports enable_gqa.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41699). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
🌐 [i18n-KO] Translated gemma3n.md to Korean
Author: HyunZ118
Pull request: yes

# What does this PR do?

Translated the gemma3n.md file of the documentation to Korean.
Thank you in advance for your review.

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

May you please review this PR?
@jungnerd, @yijun-lee, @Kim-Ju-won, @FacerAin, @judy-choi, @AhnJoonSung, @maximizemaxwell, @nsbg 


Comments:
@AhnJoonSung  바쁘신 와중에 리뷰 해주셔서 감사합니다 !!

@TaskerJang 시간 괜찮으실 때 확인해주시면 감사하겠습니다!

@TaskerJang 제가 번역 후 3번 이상 검토하는데, 여러 번 확인했을 때도 못찾은 오탈자를 찾아주셨네요,, 문장 표현 개선 제안주신 부분도 수정하였습니다! 꼼꼼한 리뷰 감사드립니다~~

@jungnerd @yijun-lee 리뷰 수정 완료하였습니다! 확인 및 검토 부탁드립니다!

@stevhliu 
May you please review this PR?

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_40873). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@stevhliu 
All the suggested changes have been applied.
Thank you for the detailed feedback🤗
Please take a final look when you have a moment.

build-doc

[Building docs for all languages...](https://github.com/huggingface/transformers/actions/runs/18570426689)

@stevhliu 
I’ve completed the updates. Please take a look. Thank you!

build-doc

[Building docs for all languages...](https://github.com/huggingface/transformers/actions/runs/18598954359)
pin torchcodec on CI docker image
Author: ydshieh
Pull request: yes

# What does this PR do?

As previous PR, we have issues with torch torchcodec 2.9

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: vit

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41702). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
pin torchcodec on CI docker image
Author: ydshieh
Pull request: yes

# What does this PR do?

As previous PR, we have issues with torch torchcodec 2.9

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41703). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
further improve `utils/check_bad_commit.py` (#41658)
Author: ydshieh
Pull request: yes

# What does this PR do?



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41690). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.