Allow setting seq_len/size/dim for gated models

#121
by tomaarsen HF staff - opened
Massive Text Embedding Benchmark org
edited Jun 5

Hello!

Pull Request overview

  • Allow setting seq_len/size/dim for gated models

Details

By throwing exceptions more quickly when data can't be gathered from the model automatically, we can use the model_meta.yaml with models not marked as external to specify some of the parameters there. This fixes https://huggingface.co/nvidia/NV-Embed-v1 and https://huggingface.co/Linq-AI-Research/Linq-Embed-Mistral not having any model sizes/memory usage/max tokens:

image.png

cc @nada5 @linqresearch as this affects your models.

  • Tom Aarsen
tomaarsen changed pull request status to open
Massive Text Embedding Benchmark org

Looks great! Maybe explaining EXTERNAL_MODEL_TO_SIZE[name_without_org] * 1e6 * 4 / 1024**3 with a comment in the code could help but else feel free to merge!

Massive Text Embedding Benchmark org
tomaarsen changed pull request status to merged

Sign up or log in to comment