Add quantized ONNX weights
Hey
@bwang0911
and
@Xenova
,
unfortunately due to the recent policy changes in Jina's repos considering access/authentication (I think like 1-2 months ago), it's not possible anymore to load the quantized onnx embeddings for feature extraction in transformers.js from the jinaai repos.
It leads to:
Uncaught (in promise) Error: Unauthorized access to file: "https://huggingface.co/jinai/jina-embeddings-v2-base-de/resolve/main/tokenizer_config.json".
It's quite a pity as Jina's DE-EN embeddings are probably the best German embeddings out there and users of SemanticFinder cannot load them. I would definitely like to avoid having to fork the repos. I know that for now I could use Xenova/jina-embeddings-v2-base-de but I'd generally prefer using the weights from it's original source.
Can any of you think of a systematic solution? E.g. is it possible to allow "public" access to a subdir in a repo (like just onnx weights)?
I know that for now I could use Xenova/jina-embeddings-v2-base-de but I'd generally prefer using the weights from it's original source.
@do-me https://huggingface.co/Xenova/jina-embeddings-v2-base-de uses the exact same ONNX files, and was made for this exact purpose :)
Can any of you think of a systematic solution? E.g. is it possible to allow "public" access to a subdir in a repo (like just onnx weights)?
Unfortunately not, since requests made in a browser never attach authentication headers. One bypass would be proxying via a server, but that would kind of defeat the purpose of running in-browser.