RuntimeError: Detected that you are using FX to symbolically trace a dynamo-optimized function. This is not supported at the moment.
Anybody is getting this error, trying to finetune it (in my case, using a similar approach to SapBERT): RuntimeError: Detected that you are using FX to symbolically trace a dynamo-optimized function. This is not supported at the moment.
You should be able to specify reference_compile=False in the from_pretrained("answerdotai/ModernBERT-base", reference_compile=False). In short, the reference implementation uses torch.compile for some components of the model. This is not compatible with FX, so we also give you the option to disable the compile.
- Tom Aarsen
Hey! To add to
@tomaarsen
's comment, if you are using multi-GPU with a HuggingFace trainer, this can sometimes be resolved by using accelerate to handle the multi-GPU aspect.
Thank you, Tom. Now it is working perfectly.