MedGemma worked find prior to 4.52.3 release but now errors
Author: harpreetsahota204
Pull request: no
Labels: name: bug

### System Info

Hi,

Seeing this error while using MedGemma. Used yesterday on 4.51.3, upgraded transformers today and now see this error

```
Error: Unexpected type in sourceless builder transformers.models.gemma3.configuration_gemma3.Gemma3TextConfig

from user code:
   File "/home/harpreet/miniconda3/envs/fo_develop/lib/python3.11/site-packages/transformers/models/gemma3/modeling_gemma3.py", line 1345, in forward
    outputs = self.model(
  File "/home/harpreet/miniconda3/envs/fo_develop/lib/python3.11/site-packages/transformers/utils/generic.py", line 969, in wrapper
    output = func(self, *args, **kwargs)
  File "/home/harpreet/miniconda3/envs/fo_develop/lib/python3.11/site-packages/transformers/models/gemma3/modeling_gemma3.py", line 1205, in forward
    causal_mask = self._update_causal_mask(
  File "/home/harpreet/miniconda3/envs/fo_develop/lib/python3.11/site-packages/transformers/models/gemma3/modeling_gemma3.py", line 1024, in _update_causal_mask
    if self.config.text_config._attn_implementation == "flash_attention_2":
  File "/home/harpreet/miniconda3/envs/fo_develop/lib/python3.11/site-packages/transformers/configuration_utils.py", line 211, in __getattribute__
    return super().__getattribute__(key)

Set TORCHDYNAMO_VERBOSE=1 for the internal stack trace (please do this especially if you're reporting a bug to PyTorch). For even more developer context, set TORCH_LOGS="+dynamo"
```

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Steps to reproduce.

Install transfromers 4.51.3 and follow MedGemma model card for usage.

Then, upgrade to latest transformers, and observe the error

### Expected behavior

I'd expect the model to run per the model card

Comments:
@harpreetsahota204 Gemma2 and gemma3 uses special type of cache and thus automatically tries to compile when running inference. And with "torch >= 2.7.0" there are currently graph breaks when compiling the model

As a workaround you can disable compile or downgrade torch:

`model.generate(**inputs, disable_compile=True)`

I have encountered this same issue, and would like to know if anyone found a workaround.

Hi @abdulaihalidu! 👋
I've just submitted a permanent fix for this issue in PR #38389. The problem was that the _update_causal_mask method in PaliGemma (which MedGemma inherits from) was accessing _attn_implementation during torch compilation, causing graph breaks in torch 2.7.0.

> [@harpreetsahota204](https://github.com/harpreetsahota204) Gemma2 and gemma3 uses special type of cache and thus automatically tries to compile when running inference. And with "torch >= 2.7.0" there are currently graph breaks when compiling the model
> 
> As a workaround you can disable compile or downgrade torch:
> 
> `model.generate(**inputs, disable_compile=True)`

This fixed the problem for me for now

@dscarmo, yes, that solves the issue, thanks!

I've been experiencing an issue where the model: "MEDGEMMA-4B" is taking significantly longer to respond. It was working fine when I used it last Friday, but now the response times are unusually slow. Do you have any idea what might be causing this and still not getting any response back too?

@Sabeeh789 which torch version you are using? The compilation is super slow in "torch==2.7.0" and gemma3 models always compile automatically due to the cache type it uses. I recommend to downgrade torch to "2.6.0" and do a small warm-up before checking the latency

Okay, I’ll do this. I’ve installed PyTorch 2.7.0, and last Friday the model was loading fast and giving responses with no latency. But now, although the model still loads quickly, the image analysis part is taking a very long time and gives no response. I’m not sure why this is happening.

@Sabeeh789 if the bottleneck is the model's forward pass taking long time, then it most probably has something to do with torch and compilations. Another option if you cannot downgrade torch because of other dependency issues, then the above snippet I commented will work

`model.generate(inputs, disable_compile=True)`

I was facing the same problem and the solution was to revert to torch-2.6.0 and torchvision-0.21.0

My problem wasn't solved with torch==2.6.0 due to a dependency conflict. I reinstalled torch==2.7.0 and rebooted my PC. After that, it suddenly started working. The issue was that the code was using the CPU instead of the GPU, which is why the response time was so long. Now it's working fine.



This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.