Can I encode sentences?

#1
by ancordovag - opened

Hello. I want to encode spanish sentences. Is it suitable to do it with bertin? How do I do so?
I tried with model.encode, but did not work.

BERTIN Project org

BERTIN GPT-J-6B is a decoder-only model. If you want to encode sentences or build your own sentence embedding model, you'd need BERTIN RoBERTa.

BERTIN Project org

That said, it seems you might be able to use the last token's embedding as a sort of sentence embedding: https://github.com/huggingface/transformers/issues/3168. Still, a RoBERTa/BERT embedding might work better.

versae changed discussion status to closed

Sign up or log in to comment