Rotary Scaling Factor of 4 for 8k context (Do not merge)

#23
by nbroad HF staff - opened

This is a revision that updates the "rotary_scaling_factor" to 4.0 which corresponds with a sequence length of 8192 tokens.

This PR should not be merged, as it is intended only for usage in TEI by specifying the revision argument.

Here is how you can use this model:

model=nomic-ai/nomic-embed-text-v1.5
revision=refs/pr/23
volume=$PWD/data # share a volume with the Docker container to avoid downloading weights every run

docker run --gpus all -p 8080:80 -v $volume:/data --pull always ghcr.io/huggingface/text-embeddings-inference:1.2 --model-id $model --revision $revision
Ready to merge
This branch is ready to get merged automatically.

Sign up or log in to comment