Sentence Similarity
sentence-transformers
PyTorch
Rust
Safetensors
Transformers
English
bert
feature-extraction
Inference Endpoints
text-embeddings-inference
5 papers

max token length question

#3
by ColinKhan - opened

Hi!
I've been using this model trying to perform vector search.
Recently i notice the default max sequence length of it is 128, while on the page it says max sequence length is 256.
However on the lower part of the page, it says the model was trained with 128 token length.
So i'm not sure if it's ok to increase token length to 256, will this decrease the quality of vector because hyper parameters were trained with 128 token length?
And since it's in sentence-transformer library, max_sequence_length can even be set as 512. Can I also do this for this model?

Thanks!

Sign up or log in to comment