Can I encode sentences? #1

by ancordovag - opened

Hello. I want to encode spanish sentences. Is it suitable to do it with bertin? How do I do so?
I tried with model.encode, but did not work.

BERTIN GPT-J-6B is a decoder-only model. If you want to encode sentences or build your own sentence embedding model, you'd need BERTIN RoBERTa.

That said, it seems you might be able to use the last token's embedding as a sort of sentence embedding: https://github.com/huggingface/transformers/issues/3168. Still, a RoBERTa/BERT embedding might work better.

versae changed discussion status to closed