maybe a bug on phi3 model after refactor or not ?
Author: Onverra-sudo
Pull request: no
Labels: name: bug

### System Info

for your information after the refactor https://github.com/huggingface/transformers/commit/2c47618c1a282f925446506d53108dc6e82d9ef0

the omnigen node for comfui is broken. 
https://github.com/set-soft/ComfyUI_OmniGen_Nodes

I patch manualy transformer to restor the old phi3 model 

[transformers_patch_phi3old.zip](https://github.com/user-attachments/files/19998821/transformers_patch_phi3old.zip)

But it's not a good solution.

example of bug with the new phi3 model from refactor https://github.com/1038lab/ComfyUI-OmniGen/issues/37#issuecomment-2803268979 in forward function.

Can you explain to me how I can update the omnigen to module with the now Phi3 model after transformers refactor ?

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction


install comfyui https://github.com/comfyanonymous/ComfyUI?tab=readme-ov-file
install omnigen module https://github.com/set-soft/ComfyUI_OmniGen_Nodes
lanch omnigen module with 4.51.3

### Expected behavior

no error on 
  File "D:\tools\ai\pinokio\api\comfy.git\app\custom_nodes\comfyui-omnigen\OmniGen\transformer.py", line 157, in forward
    layer_outputs = decoder_layer(
  File "D:\tools\ai\pinokio\api\comfy.git\app\env\lib\site-packages\torch\nn\modules\module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "D:\tools\ai\pinokio\api\comfy.git\app\env\lib\site-packages\torch\nn\modules\module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\tools\ai\pinokio\api\comfy.git\app\env\lib\site-packages\transformers\models\phi3\modeling_phi3.py", line 295, in forward
    hidden_states, self_attn_weights = self.self_attn(
  File "D:\tools\ai\pinokio\api\comfy.git\app\env\lib\site-packages\torch\nn\modules\module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "D:\tools\ai\pinokio\api\comfy.git\app\env\lib\site-packages\torch\nn\modules\module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\tools\ai\pinokio\api\comfy.git\app\env\lib\site-packages\transformers\models\phi3\modeling_phi3.py", line 189, in forward
    cos, sin = position_embeddings
TypeError: cannot unpack non-iterable NoneType object

Comments:
@Onverra-sudo hey!

After that refactor, all models prepare cos/sin for RoPE in the base model and pass it further to each decoder layer as `position_embeddings`. So in case ComfyUI redefines the base language model, you can move RoPE layer and create cos/sin in advance like here

https://github.com/huggingface/transformers/blob/7a3e208892c06a5e278144eaf38c8599a42f53e7/src/transformers/models/phi3/modeling_phi3.py#L579-L581

Thanks but i Think it's to complacated for me (models are a new domain for me), after few research the problen is in the Omnigen model not in comfyui node.

https://github.com/VectorSpaceLab/OmniGen/tree/main/OmniGen

All comfyui node reuse the source code from this model repo.


@Onverra-sudo not sure if you're the maintainer of OmniGen repo, if not feel free to open issue in OmniGen. The necessary fix is in [these lines](https://github.com/VectorSpaceLab/OmniGen/blob/main/OmniGen/transformer.py#L61-L166), where the `forward` was overwritten by authors

It has to follow same pattern as the `forward` from transformers. Overall, I'd suggest to simply pin a lower version for transformers because we may change many stuff breaking `OmniGen` again

Yes I am not the Omnigen maintener and I open an issue on omnigen (https://github.com/VectorSpaceLab/OmniGen/issues/212), I wait a patch by omnigen teams and during this period I will used my workarround base on the previous phi3 model.

Thanks for all.

Great, closing as resolved :)

Just one last thing, I share with you my manual workarround in the following zip.

[transformers_patch_phi3old.zip](https://github.com/user-attachments/files/20001508/transformers_patch_phi3old.zip)

With that after merge with transformers 4.51.3 it's possible to reuse the old phi3 model, and to unblock some people.

from transformers import Phi3Config, Phi3Model
shall be replaced by
from transformers import Phi3ConfigOld, Phi3ModelOld