[Quantization]: Refactor is_quantization_compressed for format-based detection
Author: rigen1048
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47152)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47152)
<!-- ci-dashboard-badge:end -->

# What does this PR do?
## **Problem**  
Models saved with quantization_status: "frozen" but in a quantized format (e.g. RedHatAI/Llama-3.2-1B-Instruct-FP8) do not decompress when run_compressed=False. The current check is too strict because it requires status == "compressed", causing raw FP8 values to remain in the weights.

## **Solution**  
Update is_quantization_compressed to determine compression based on the data format rather than the lifecycle status.

### *Backward Compatibility*
1. **Logical Consistency**: If format != "dense", the weights are physically compressed and must be decompressed for full-precision runs, regardless of the status string.  
2. **Robustness**: Scoping the check to quant_method == COMPRESSED_TENSORS ensures the property only targets the intended method and does not interfere with other quantization schemes.  
3. **Compatibility**: Any model where the previous status-based check was true already satisfies format != "dense". No existing working paths are changed.  
4. The check now uses the actual tensor format (instead of status) for compressed-tensors models. This correctly handles frozen models and MoE expert weights.

### **Verification**  
Fixes #47145. 
- Verified against the reproducible script in the issue. 
- Added regression test. Coauthored by @subin9.

### **Diff**
```Python
#Before:
 return self.is_quantized and (qc is not None and qc.quantization_status == QuantizationStatus.COMPRESSED)

# After:
        return (
            self.is_quantized
            and qc is not None
            and qc.quant_method == QuantizationMethod.COMPRESSED_TENSORS
            and qc.format != "dense"
        )
```
<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #47145

## Code Agent Policy

The Transformers repo is currently being overwhelmed by a large number of PRs and issue comments written by
code agents. We are currently bottlenecked by our ability to review and respond to them. As a result, 
**we ask that new users do not submit pure code agent PRs** at this time. 
You may use code agents in drafting or to help you diagnose issues. We'd also ask autonomous "OpenClaw"-like agents
not to open any PRs or issues for the moment.

PRs that appear to be fully agent-written will probably be closed without review, and we may block users who do this
repeatedly or maliciously. 

This is a rapidly-evolving situation that's causing significant shockwaves in the open-source community. As a result, 
this policy is likely to be updated regularly in the near future. For more information, please read [`CONTRIBUTING.md`](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md).

- [X ] I confirm that this is not a pure code agent PR.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://huggingface.co/docs/transformers/contributing) and the
      [Pull Request](https://huggingface.co/docs/transformers/pr_checks) checks?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. [Hugging Face Transformers Issue #47145](https://github.com/huggingface/transformers/issues/47145)
- [ ] Did you make sure to update the documentation with your changes according to the [guidelines](https://github.com/huggingface/transformers/tree/main/docs)?
- [ ] Did you write any new necessary [tests](https://huggingface.co/docs/transformers/testing)?


## Who can review?
@SunMarc

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker @Cyrilvallez @vasqu
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @Cyrilvallez (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc
- kernels: @vasqu @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad 

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.

 -->

Comments:
Thanks for jumping on this. Looks like we landed on the same diagnosis (status vs format), good to know it wasn't just me.

A couple of things that might help it land:

1. Worth adding a regression test? There's existing infra in `tests/quantization/compressed_tensors_integration/`, and the `stories42M` compressed/uncompressed pair already exercises the decompress path. I wrote a small one for the frozen-fp8 case while looking into this (checks the 448 vs dequantized weight, plus that generation isn't garbage). Happy to share if useful.

2. Did you consider fixing `is_quantization_compressed` itself (format-based) instead of the gate in `_process_model_after_weight_loading`? As-is the property still returns False for frozen checkpoints, so `post_init` keeps forcing `run_compressed=False` and emitting the "only supported for compressed models" warning, and the `run_compressed=True` (compressed execution) path stays unavailable for these models. Keying the property off `format != dense` covers both, and from a quick check it only changes the frozen case (compressed / int / pack / dense unaffected).

Either way fixes the reported bug. @SunMarc probably has a preference on where this should live.


Good call on the regression test — I'd appreciate it if you share it. On fixing is_quantization_compressed directly: I considered it, but I want to make sure compress_model in _process_model_before_weight_loading handles frozen models on meta device correctly before changing the property globally. My current fix is more surgical and guaranteed safe for the decompression path. Happy to expand it if @SunMarc prefers the property-level fix

I had a checkpoint handy so I tried the property-level change to sanity-check the meta-device concern, and it held up: `compress_model` still runs on the frozen model while the params are on meta, and both paths load fine (`run_compressed=True` stays fp8, `run_compressed=False` decompresses to ~0.68, generation correct in both). I only tested CPU with meta init, so a GPU/device_map check would still be worth doing.

For @SunMarc to weigh: with the gate-only change the property still returns False for frozen checkpoints, so `post_init` keeps forcing `run_compressed=False` and the "only supported for compressed models" warning still fires. The property version avoids that but is broader, so it's your call which trade-off you'd rather have.

On the test, happy to just push it as a commit you can cherry-pick, or paste it here if that's easier. Let's get it into your PR.


Thanks to @subin9 for the incredible support and @SunMarc for the technical guidance. I have updated is_quantization_compressed and validated the fix against the reproduction script with some slight tweaking. If the GitHub Actions pass, the fix should be ready to merge. @subin9, I would really appreciate it if you could paste the test in the comments here.

@SunMarc, I'd like to credit @subin9 as a co-author for this fix. Would you mind handling the attribution during the merge?

Thanks @rigen1048! Here's the small regression test I mentioned — it loads the frozen fp8 checkpoint with `run_compressed=False` and checks the weight is dequantized (~0.68, not the raw fp8 max of 448) and that generation isn't garbage. It pulls the 1B model so I marked it `@slow`. Fits `CompressedTensorsTest` in `tests/quantization/compressed_tensors_integration/test_compressed_tensors.py`, passes locally on CPU with the current property change:

```python
@slow
def test_frozen_fp8_dequantized_on_load(self):
    # Regression test for #47145: a frozen float-quantized checkpoint must be
    # dequantized on load (weight_scale applied) rather than left as raw fp8.
    model_id = "RedHatAI/Llama-3.2-1B-Instruct-FP8"
    model = AutoModelForCausalLM.from_pretrained(
        model_id,
        device_map="cpu",
        dtype=torch.float32,
        quantization_config=CompressedTensorsConfig(run_compressed=False),
    )
    weight = model.model.layers[0].self_attn.q_proj.weight
    # dequantized max is ~0.68; raw fp8 max would be 448
    self.assertLess(weight.abs().max().item(), 5.0)

    tokenizer = AutoTokenizer.from_pretrained(model_id)
    input_ids = tokenizer("The capital of France is", return_tensors="pt").input_ids
    output = tokenizer.decode(
        model.generate(input_ids, max_new_tokens=8, do_sample=False)[0][input_ids.shape[1]:],
        skip_special_tokens=True,
    )
    self.assertIn("Paris", output)
```

(needs `slow` added to the `transformers.testing_utils` import.) If you'd rather have something that runs on every CI without the download, I can also add a cheap config-level check asserting `is_quantization_compressed` is True for float-quantized/frozen and False for dense.


Thanks again @subin9, I’ve updated the test to match the rest of the structure in the compressed_tensors_integration tests.
I’ve also added you as a co-author on the test commit. Really appreciate all your help on this one!
@sunmarc please take a look at the latest changes when you have a moment.

Thanks again @rigen1048! One small thing on the co-author — GitHub won't link `subin9@users.noreply.github.com` to my account since it was created after 2017. Could you amend the trailer on the test commit to `101092510+subin9@users.noreply.github.com` and force-push before it merges? Appreciate it 🙏


The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47152). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@sunmarc Sorry for the noise!
I took the opportunity to polish the PR — cleaning up the git history, commit messages, and PR description for a cleaner merge and easier future reference.

The core logic remains exactly the same as last time; I just made the code more readable and developer-friendly.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: compressed_tensors_integration

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47152)
**Latest run:** [29103069421:2](https://github.com/huggingface/transformers/actions/runs/29103069421)
**Result:** `failure` | **Jobs:** 15 | **Tests:** 170,376 | **Failures:** 1 | **Duration:** 16h 3m

<!-- ci-dashboard-recap:end -->

Hi @SunMarc, this PR has been open for a week with auto-merge enabled. The only current failure is a known flaky test in BigBirdPegasus (qa_outputs.bias gradient mismatch), which is unrelated to my changes.
However, I see that the compressed_tensors_integration slow tests haven't been triggered yet. Could you please run those tests and, if they pass, proceed with the merge? Thanks!