noamaanMulla-03/transformers-issues huggingface dataset:
Fix AutoImageProcessor.register and documentation in auto processing modules
Author: MilkClouds
Pull request: yes

# What does this PR do?

This PR fixes several issues in the auto processing modules:

1. **Fixes `AutoImageProcessor.register` bug**: Removes incorrect validation logic that was copied from tokenizers. The validation checked for `slow_image_processor_class` attribute consistency, but fast image processors don't have this attribute (unlike fast tokenizers), causing the `register` method to fail when registering custom image processors.

2. **Fixes documentation errors**: Corrects copy-paste errors in docstrings where "tokenizer" was incorrectly used instead of the appropriate processor type (feature extractor, image processor, video processor).

3. **Fixes typos**: Corrects "fine" → "find" in comments across multiple auto modules.

4. **Improves `AutoVideoProcessor` trust handling**: Adds proper upstream repository extraction from `video_processor_auto_map` when resolving trust_remote_code.

## Changes by file:

- `feature_extraction_auto.py`: Fixed typo and corrected docstring to reference feature extractors instead of tokenizers
- `image_processing_auto.py`: Removed incorrect `slow_image_processor_class` validation and fixed import in docstring example
- `processing_auto.py`: Fixed typo in comment
- `tokenization_auto.py`: Fixed typo in comment  
- `video_processing_auto.py`: Fixed docstring reference and added upstream repo handling for trust_remote_code

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@ArthurZucker @Rocketknight1 (auto modules and processing)



Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto

For reviewer's information: this PR is ready for review!

This is definitely one for @zucchini-nlp, but you might need to wait until she's back!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41864). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix Qwen3-Omni RoPE
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title, after the last PR with RoPE refactoring, Qwen3-Omni model has issues when loading the model. One of the many sub-configs doesn't call standardization on RoPE which causes issues

I also updated slow tests with correct checkpoint, right now they use Omni-2 checkpoints and thus do not test anything

cc @BakerBunker 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41778). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

run-slow: qwen3_omni_moe

This comment contains run-slow, running the specified jobs: 

models: ['models/qwen3_omni_moe']
quantizations: [] ...

run-slow: qwen3_omni_moe

This comment contains run-slow, running the specified jobs: 

models: ['models/qwen3_omni_moe']
quantizations: [] ...

run-slow: qwen3_omni_moe

This comment contains run-slow, running the specified jobs: 

models: ['models/qwen3_omni_moe']
quantizations: [] ...

run-slow: qwen3_omni_moe

This comment contains run-slow, running the specified jobs: 

models: ['models/qwen3_omni_moe']
quantizations: [] ...

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: qwen3_omni_moe

The slow tests should have triggered before merging but it was not using the correct checkpoint. So after I adjusted them, they will be failing without fix from this PR

I don't think we need a dummy test, because it fails for certain checkpoint from the hub
Annoying typo in attention error message
Author: manueldeprada
Pull request: yes

hi again ❤️  just found this annoying typo as a user

@Cyrilvallez maybe?

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42037). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix another `Argument list too long` in `pr_slow_ci_suggestion.yml`
Author: ydshieh
Pull request: yes

# What does this PR do?

Fix another `Argument list too long` in `pr_slow_ci_suggestion.yml`

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: vit

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42061). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[WIP] 🚨 clean xcodec 🧼
Author: eustlb
Pull request: yes

# What does this PR do?

Lots of stuff to clean on Xcodec integration to reach the source of truth expected quality, mainly:
1. Stuff could have been simplified a lot, the current integration is mainly a dump from the original codebase, and modular should have been used!
2. The semantic decoder is instantiated but not used since we do not support training (and same for it's proj layer), adding dead weight to the model. Either we use it and enable training, either we remove it. No big interest in supporting training for now →  let's remove
3. The two passes on the encoder only to get the ouput shape causes mem spikes, it is uncessary! 

> [!WARNING]
> This PR **is** breaking. TODO: let's see how to reduce the arm for users...

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: dac, xcodec
KeyError: 'rope_parameters_factor' when using convert_gpt_oss_weights_to_hf.py for openai/gpt-oss-20b
Author: ghost
Pull request: no
Labels: name: bug

### System Info

Transformer is on commit 64397a830188a35b5e9056036e13bb6ccfe434d8 of main branch.
Latest as of Nov 4 2025




### Who can help?

@zucchini-nlp 
PR Introducing the key in the convert file is https://github.com/huggingface/transformers/pull/39847
Can you please help with this issue. 

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Steps to reproduce the error:

1. huggingface-cli download openai/gpt-oss-20b
2. git clone https://github.com/huggingface/transformers.git 
3. cd transformers
4. pip install -e .
5. cd src/transformers/models/gpt_oss
6. python convert_gpt_oss_weights_to_hf.py --input_dir modelsopenai-gpt-oss-20b/snapshots/2e8f8052ee2aeee907f76e08c08b9fdde8677ca8/original/ --output_dir ./gpt_oss_20b_fp16


Error:
Traceback (most recent call last):
  File "transformers/src/transformers/models/gpt_oss/convert_gpt_oss_weights_to_hf.py", line 831, in <module>
    main()
  File "transformers/src/transformers/models/gpt_oss/convert_gpt_oss_weights_to_hf.py", line 815, in main
    write_model(
  File "transformers/src/transformers/models/gpt_oss/convert_gpt_oss_weights_to_hf.py", line 163, in write_model
    "factor": float(original_config.pop("rope_parameters_factor")),
KeyError: 'rope_parameters_factor'


### Expected behavior

Expected it to complete successfully which is still happening if I revert the mentioned commit.

Comments:
Fix model_input_names singleton issue causing shared state
Author: yashwantbezawada
Pull request: yes

# What does this PR do?

Fixes #42024

I found this while testing tokenizers - when you modify `model_input_names` on one tokenizer instance, it was affecting all other instances of the same tokenizer class.

The problem is that `model_input_names` is defined as a class-level list, and in the `__init__` method (line 1417), when no custom `model_input_names` is provided, it was just referencing the class attribute directly instead of making a copy.

So all instances were sharing the same list object. This is a classic Python gotcha with mutable class attributes.

The fix is simple - wrap it in `list()` to create a new list for each instance:

**Before:**
```python
self.model_input_names = kwargs.pop("model_input_names", self.model_input_names)
```

**After:**
```python
self.model_input_names = list(kwargs.pop("model_input_names", self.model_input_names))
```

This ensures each tokenizer instance gets its own independent copy of the list. Now modifications to one instance won't affect others.

The reproduction from the issue shows the problem clearly - with the fix, the second tokenizer instance will have the original list values instead of inheriting the modifications from the first instance.


Comments:
This solution LGTM but cc @arthurzucker @itazap!
Add StreamingSentimentPipeline: Real-Time Sentiment Analysis for Live Data Streams (WebSocket, Kafka, HTTP)
Author: 3015pavan
Pull request: no
Labels: name: Feature request

### Feature request

Introduce a Streaming Sentiment Pipeline in Hugging Face Transformers that supports real-time sentiment analysis over live text streams — such as Twitter/X feeds, Reddit comments, or Kafka message queues — using asynchronous data ingestion.

This will extend the existing pipeline("sentiment-analysis") API to handle continuous inputs

### Motivation

Transformers pipelines currently support static or batched text input.
However, real-world use cases — especially in:

Finance (market sentiment)

Social media analytics

Brand monitoring

Content moderation

require continuous, low-latency sentiment tracking from live data sources.

Without native support, developers must write custom asyncio or WebSocket wrappers around pipelines.
An official implementation would standardize this for the entire ecosystem.

### Your contribution


I  propose to implement this feature as follows:

1. **Implement `AutoMergeAdapters` class**
   - Create a new module: `src/transformers/adapters/auto_merge_adapters.py`
   - Extend existing PEFT logic (`merge_and_unload`) to support merging 2+ LoRA adapters.
   - Allow optional merge weights and scaling factors.

2. **Add CLI integration**
   - Extend `transformers-cli` with a new subcommand: `merge-adapters`.
   - Parameters: `--base`, `--adapters`, `--weights`, `--save`.

3. **Testing & Validation**
   - Add unit tests in `tests/adapters/test_auto_merge_adapters.py`.
   - Verify parameter alignment, weight averaging, and output consistency.

4. **Documentation**
   - Add a new guide: *Combining Multiple LoRA Adapters*.
   - Provide a sample notebook under `examples/adapters/merge_multiple_adapters.ipynb`.

5. **Model Demo**
   - Publish a demo model on the Hub showing three LoRA adapters merged for multi-domain sentiment analysis.
   - 

Comments:
hey can you please check the pr @3015pavan , leme know if you need anything 

This is a copy of your previous issue #42014

No bro it is related to streaming analysis only open the issue

On Thu, Nov 6, 2025, 6:28 PM Matt ***@***.***> wrote:

> *Rocketknight1* left a comment (huggingface/transformers#42035)
> <https://github.com/huggingface/transformers/issues/42035#issuecomment-3497148932>
>
> This is a copy of your previous issue #42014
> <https://github.com/huggingface/transformers/issues/42014>
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/42035#issuecomment-3497148932>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/BPB2JCRXES5V66DW56EBFPT33NAX3AVCNFSM6AAAAACLGYIFGWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTIOJXGE2DQOJTGI>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>
RT-Detr 2d positional embedding bug
Author: konstantinos-p
Pull request: no
Labels: name: bug

name: Vision

I believe that there is a subtle bug in how 2d positional embeddings are generated for the rt_detr model. Specifically, when generating a meshgrid and then flattening, the positional embeddings no longer match the intended image pixels. We can see this in the script below

```python
import torch 
# Create 2d grid
x = torch.tensor([0, 1, 2, 3])
y = torch.tensor([0, 1, 2])

grid_x, grid_y = torch.meshgrid(x, y, indexing='ij')

print(grid_x)
#tensor([[0, 0, 0],
#        [1, 1, 1],
#        [2, 2, 2],
#        [3, 3, 3]])
print(grid_y)
#tensor([[0, 1, 2],
#        [0, 1, 2],
#        [0, 1, 2],
#        [0, 1, 2]])

# Create image
image = torch.range(0,11).reshape((3,4)).int()
print(image)
#tensor([[ 0,  1,  2,  3],
#        [ 4,  5,  6,  7],
#        [ 8,  9, 10, 11]], dtype=torch.int32)

#Print flattened grid and image
print(grid_x.flatten())
# tensor([0, 0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 3])
print(grid_y.flatten())
# tensor([0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 1, 2])
print(image.flatten())
# tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11])
```
This means that for images that are not square the embeddings don't encode the intended spatial information.

This issue comes from the following line. I believe setting indexing='xy' would have been the correct choice. 

https://github.com/huggingface/transformers/blob/fa36c973fc5f2f473f58013097e21ed74025ea42/src/transformers/models/rt_detr/modeling_rt_detr.py#L1154

While RT-Detr was intended by the authors to work on square images, generalizing correctly to non-square images could be important for future research.

I will make a PR with a simple fix, that is consistent with the current positional embeddings but generalizes correctly to variable image sizes.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Fixed in #41380 I think!
RT-Detr docs should reflect fixed 640x640 input size
Author: konstantinos-p
Pull request: no
Labels: name: Documentation

The authors of RT-Detr mention that the model was trained on 640x640 images and was meant to be used for inference on 640x640 images. Also, the current implementation has certain quirks that make training/inferring on images of different sizes problematic. For example, the pixel masks used for batching images of varying sizes are discarded.

https://github.com/huggingface/transformers/blob/0452f28544f3626273d25f07f83c0e5f7da2d47a/src/transformers/models/rt_detr/modeling_rt_detr.py#L1645

The above are not clear in the current docs. I'll open a PR which adds a few lines in the docs to notify users about these issues.

Comments:
Hii, I like to work on this issue. Issue number #41363

Thanks @YashKerkarTech04 however as you can see I already have a PR open for this issue.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Fixed by #41363 I think!
Revert back to use GitHub context 
Author: ydshieh
Pull request: yes

# What does this PR do?

Revert back to use GitHub context 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42065). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix KeyError in _is_package_available for packages with dotted names
Author: yashwantbezawada
Pull request: yes

# What does this PR do?

Fixes #41981

I ran into this issue while testing with `optimum.quanto`. The function was crashing with `KeyError` when checking for packages with dotted names.

The problem is in `import_utils.py` line 56 - when it tries to look up the package in `PACKAGE_DISTRIBUTION_MAPPING`, packages with dots in the name (like `optimum.quanto`) aren't always there. The mapping might only have `optimum` as a key, not `optimum.quanto`.

The function already has fallback logic to handle cases where package metadata can't be found - it imports the package directly and checks for `__version__`. But the exception handler only caught `PackageNotFoundError`, not `KeyError`.

I added `KeyError` to the exception handler so the fallback works for both cases. Now when a package name isn't in the mapping, it'll use the same fallback path instead of crashing.

This is a one-line change that handles the edge case without affecting normal operation. The existing tests should still pass since the behavior for normal packages is unchanged.


Comments:
Yes, this makes sense to me! Since it should only have an impact in situations that were already causing a crash, I think it's safe to merge this.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42050). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
fix(benchmarks): prevent version conflicts and ensure proper cleanup in continuous batching
Author: Copilot
Pull request: yes

The CI was failing with `TypeError: ContinuousBatchingManager.add_requests() got an unexpected keyword argument 'streaming'` due to a stale PyPI installation being used instead of the local source. Additionally, exceptions during benchmark execution left the manager running, causing downstream OOM errors.

## Changes

- **Remove transformers from benchmark requirements**: The workflow explicitly installs from source in editable mode, but `benchmark_v2/requirements.txt` was installing from PyPI first, creating version conflicts
- **Add try-finally cleanup**: Wrap `time_generate_batch()` logic to ensure `manager.stop()` is always called, preventing GPU memory leaks when exceptions occur

```python
manager = self.model.init_continuous_batching(config)
manager.start()
try:
    # benchmark logic
    manager.add_requests(inputs, max_new_tokens=max_new_tokens, streaming=True)
    # ...
finally:
    manager.stop()  # Always cleanup, even on exception
```

<!-- START COPILOT CODING AGENT TIPS -->
---

✨ Let Copilot coding agent [set things up for you](https://github.com/huggingface/transformers/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot) — coding agent works faster and does higher quality work when set up for your repo.


Comments:
cc @ydshieh!
Revert back to use GitHub context 
Author: ydshieh
Pull request: yes

# What does this PR do?

Revert back to use GitHub context .

After the previous PR

Avoid explicit checkout in workflow (#42057), when a PR touched a lot of files, we get 

> Error: An error occurred trying to start process '/home/runner/actions-runner/cached/externals/node20/bin/node' with working directory '/home/runner/work/transformers/transformers'. Argument list too long

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42066). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix missing arg in check_docstring
Author: yonigozlan
Pull request: yes

# What does this PR do?

As the title says

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42054). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@zucchini-nlp Fully agreed I'll try to think of something better. The difficulty I found is that some models on the hub use this attribute in their image processor, even if there's no mention of it anywhere in the corresponding model files. So it's hard to determine which model should support this kwarg.

In the meantime though, this missing arg is causing a lot of annoying warning (totally my fault) so if we can merge this quickly that'd be great.
fix(benchmarks): ensure continuous batching manager cleanup on success path
Author: Copilot
Pull request: yes

Commit 47ee367 broke the cleanup logic in `time_generate_batch()` by replacing `finally:` with `except:`, causing `manager.stop()` to only execute on exceptions. When benchmarks completed successfully, the manager thread remained running indefinitely, preventing proper job termination.

## Changes

- Changed exception handling in `time_generate_batch()` from `try-except` to `try-except-finally`
- Moved `manager.stop()` to `finally` block to ensure cleanup in all code paths
- Preserved exception re-raising in `except` block for debugging

```python
# Before (broken)
try:
    # ... benchmark logic ...
    return results
except Exception as e:
    manager.stop()  # Only called on failure
    raise e

# After (fixed)
try:
    # ... benchmark logic ...
    return results
except Exception as e:
    raise e  # Preserve stack trace for debugging
finally:
    manager.stop()  # Always called
```

<!-- START COPILOT CODING AGENT TIPS -->
---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our [2 minute survey](https://gh.io/copilot-coding-agent-survey).


Comments:
fix(benchmarks): ensure CI fails when all benchmarks fail
Author: Copilot
Pull request: yes

The benchmark workflow was completing successfully even when all benchmarks failed, making CI unable to detect failures.

**Root cause:** `run_benchmarks.py` exits with code 0 regardless of whether benchmarks succeed or fail. When `runner.run_benchmarks()` returns an empty results dict (all configs failed), the script proceeds normally and exits successfully.

**Changes:**
- Add explicit exit status check after benchmark execution
- Exit with code 1 and error log when no benchmarks complete successfully  
- Exit with code 0 and success log when benchmarks complete normally

```python
# Check if we have any successful benchmark results
if len(results) == 0:
    logger.error("No benchmarks completed successfully. All benchmark configurations failed.")
    sys.exit(1)

# ... push results if needed ...

logger.info("Benchmark run completed successfully")
sys.exit(0)
```

This ensures the CI job properly fails when things go wrong, addressing the issue where the "Self-hosted runner (benchmark)" step was marked successful despite benchmark failures.

<!-- START COPILOT CODING AGENT TIPS -->
---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our [2 minute survey](https://gh.io/copilot-coding-agent-survey).


Comments:
[deepspeed tests fixes]
Author: stas00
Pull request: yes

Fixing a few deepspeed tests


Comments:
cc @sunmarc @mekkcyber for the deepspeed integration

this should be good to merge, @SunMarc 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41925). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.