sentence-transformers

/

distiluse-base-multilingual-cased

like 13

@nreimers I am not sure if you get notifications when a PR is opened. This PR proposes the addition of weights compatible with a Rust port of Pytorch so that they can be used in the https://github.com/guillaume-be/rust-bert library

looks good to me!

@guillaume-be Can you please point us to more details about the conversion? (a link to a specific issue inside your repo is totally fine) Thanks!

@julien-c sure - the conversion script is available at https://github.com/guillaume-be/rust-bert/blob/master/utils/convert_model.py . This essentially loads the Pytorch checkpoint (which is a pickle file and therefore Python specific), iterates through the model parameters and serializes them as C-arrays using numpy. This can then be loaded from Rust using the https://github.com/guillaume-be/rust-bert/blob/master/src/convert-tensor.rs executable. This is adapted for transformers from the tools made available on the tch-rs crate at https://github.com/LaurentMazare/tch-rs/blob/main/examples/tensor-tools.rs

👍

Hi @guillaume-be
sorry, I don't receive the notifications yet.

Thanks so much for these PRs and the conversion scripts. :)