FacebookAI

/

xlm-roberta-large

like 279

Nice!

@LysandreJik - good to merge for you?

I think it's important to verify equivalence between the PyTorch and TensorFlow models. @Rocketknight1 , @joaogante , would you have the bandwidth to write a small script that would allow us to check the equivalence of models in PRs?

I'll add it today as a CLI (I have a personal script), and will run the checks against these weights 👍

The weights loaded successfully, but the model output was very different -- 30.353527069091797, which is much much higher than our 1e-5 TF-PT tolerance. The TF weights file is also much smaller than the PT/Flax weights (~500 MB vs ~2 GB).

Perhaps the model weights were stored in half-precision ( @hoanhtu )? In any case, I do NOT advise merging this PR.

Correct conversion here: https://huggingface.co/xlm-roberta-large/discussions/3 :)

Thanks for double-checking here @joaogante !

(closing the PR in favor of https://huggingface.co/xlm-roberta-large/discussions/3, which I will be merging )

@hoanhtu out of curiosity, do you remember how you created this checkpoint?