Model quantized via sinq broken after save_pretrained and from_pretrained
Author: k355l3r-5yndr0m3
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 5.8.1
- Platform: Linux-7.0.5-arch1-1-x86_64-with-glibc2.43
- Python version: 3.12.13
- Huggingface_hub version: 1.14.0
- Safetensors version: 0.7.0
- Accelerate version: 1.13.0
- Accelerate config: 	not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.11.0+cu130 (CUDA)
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA GeForce GTX 1650

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

The following code segment fail with `KeyError: 'packing'`

If the model is not saved and loaded (via `save_pretrained` and `from_pretrained`) and is used right away after the first `from_pretrained`, the model behaves as expected.

NOTE: I have only tried this with `google/gemma-4-E4B-it` so this may be an issue unique to the model instead of sinq.
```
from transformers import AutoProcessor, AutoModelForCausalLM, SinqConfig

def quantize_model(model_id: str, save_dst: str):
    # Quantize and save resulting model
    quant_cfg = SinqConfig(
        nbits=8,
        group_size=64,
        tiling_mode='2D',
        method='sinq',
        modules_to_not_convert=["lm_head", "model.audio_tower"],
    )

    model = AutoModelForCausalLM.from_pretrained(
        model_id,
        device_map = 'cpu',
        quantization_config=quant_cfg,
    )

    model.save_pretrained(save_dst)



def test_quantized(model_id: str, save_dst: str):
    # Loading quantized model and appropriate processor 
    processor = AutoProcessor.from_pretrained(model_id)
    model = AutoModelForCausalLM.from_pretrained(save_dst, device_map='cpu')

    # Testing quantized model
    chat = []
    chat.append({ 'role': 'user', 'content': 'this is a test.' })

    model_input = processor.apply_chat_template(chat, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt").to(model.device)
    skip_length = model_input['input_ids'].shape[-1]

    new_tokens = model.generate(**model_input, max_new_tokens=2048)[0, skip_length:]
    
    message = processor.parse_response(processor.decode(new_tokens))

    print(message)


model_id = 'google/gemma-4-E4B-it'
save_dst = './gemma-4-E4B-it-sinq/'

quantize_model(model_id, save_dst)
test_quantized(model_id, save_dst)
```

### Expected behavior

Expect a chat message to be printed. Example:
`{'role': 'assistant', 'content': "Understood. I'm ready when you are! How can I help you with this test?"}`


Comments:
I would like to work on this.

I'm looking into this as well.
will share findings here.

Quick update for anyone landing here: the root cause is in `sinq`, not `transformers`. `transformers` just dispatches to `Quantizer.dequantize`, which uses bracket access on a `"packing"` key that `SINQLinear.load_state_dict` legitimately pops on reload for any `nbits=8` checkpoint (since `N*K*8//8 == N*K`, the "already unpacked" branch always fires).

I ran the code snippet provided above after the fix:
```sh
(.venv) shrirang@shrirang:~/Desktop/Learning/Transformers$ python test.py
Found gemlite installation, fast SINQ-ference for 4-bit models
Loading weights: 100%|██████████████████████████████████████████████████████████████████████████| 2076/2076 [09:56<00:00,  3.48it/s]
Found gemlite installation, fast SINQ-ference for 4-bit models
Using SINQ patch functions to download SINQ-quantized models
Use sinq flag is True
100%|██████████████████████████████████████████████████████████████████████████████████████████| 971/971 [00:00<00:00, 95672.18it/s]
100%|██████████████████████████████████████████████████████████████████████████████████████████| 593/593 [00:00<00:00, 29697.11it/s]
{'role': 'assistant', 'content': "Understood. I'm ready for your test. How can I help you?"}
```
(test.py has the codesnippet as is.)

My system details are mentioned in the issue create in the SINQ repo.


Filed upstream and proposed the one-line fix:
- Issue: huawei-csl/SINQ#25
- PR: huawei-csl/SINQ#26

Once that merges and ships in a `sinq` release, this can be closed (or resolved by bumping the `sinq` minimum version in `transformers`'s extras, whichever maintainers prefer). Happy to send that version-bump PR here if useful.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.