ERROR - just fyi.

#3
by ctranslate2-4you - opened

There's an error with this model when using HuggingFaceBgeEmbeddings that doesn't occur when using "HuggingFaceEmbeddings," just FYI. The results seem horrible...I verified this because I used HuggingFaceBgeEmbeddings with the large and base version 1.5 models and didn't encounter the problem. Additionally, when I used HuggingFaceEmbeddings (not the Bge variety) with the large version 1.5 model, the problem went away entirely and the good results were back. Thanks.

Sorry...to clarify...
using HuggingFaceBgeEmbeddings with the large model, version 1.5, did not work...but HuggingFaceEmbeddings did work with it.

For all other models (original and all 1.5 versions), HuggingFaceBgeEmbeddings worked. Seems to be a problem with the large version 1.5 only. Sorry, I forgot you guys have large, base, and small...not "xl" like some other embedding models and the lowest one is "base."

This comment has been hidden
Beijing Academy of Artificial Intelligence org

Sorry for the late reply. I missed this comment.

The difference between the HuggingFaceBgeEmbeddings and HuggingFaceEmbeddings is whether instruction is added. Maybe the instruction will drop the performance in your cases. More information about the evaluation method provided will be helpful in analyzing this problem.

Sign up or log in to comment