Package: open-text-embeddings
open-text-embeddingsMany open source projects support the compatibility of thecompletionsand thechat/completionsendpoints of the OpenAI API, but do not support theembeddingsendpoint.The goal of this project is to create an OpenAI API-compatible version of theembeddingsendpoint, which serves open source sentence-transformers models and other models supported by the LangChain'sHuggingFaceEmbeddings, HuggingFaceInstructEmbeddings and HuggingFaceBgeEmbeddings class.ℹ️ Supported Text Embeddings ModelsBelow is a compilation of open-source models that are tested via theembeddingsendpoint:BAAI/bge-large-enintfloat/e5-large-v2sentence-transformers/all-MiniLM-L6-v2sentence-transformers/all-mpnet-base-v2universal-sentence-encoder-large/5(Please refer to theuniversal_sentence_encoderbranch for more details)The models mentioned above have undergone testing and verification. It is worth noting that all sentence-transformers models are expected to perform seamlessly with the endpoint.It may not be immediately apparent that utilizing theBAAI/bge-*andintfloat/e5-*series of models with theembeddingsendpoint can yield different embeddings for the sameinputvalue, depending on how it is sent to theembeddingsendpoint. Consider the following examples:Example 1:{"input":"The food was delicious and the waiter..."}Example 2:{"input":["The food was delicious and the waiter..."]}This discrepancy arises because theBAAI/bge-*andintfloat/e5-*series of models require the addition of specific prefix text to theinputvalue before creating embeddings to achieve optimal performance. In the first example, where theinputis of typestr, it is assumed that the embeddings will be used for queries. Conversely, in the second example, where theinputis of typeList[str], it is assumed that you will store the embeddings in a vector database. Adhering to these guidelines is essential to ensure the intended functionality and optimal performance of the models.🔍 DemoTry out open-text-embeddings in your browser:🖥️ Standalone FastAPI ServerTo run the embeddings endpoint locally as a standalone FastAPI server, follow these steps:Install the dependencies by executing the following commands:pipinstall--no-cache-diropen-text-embeddings[server]Download the desired model using the following command, for exampleintfloat/e5-large-v2:./download.shintfloat/e5-large-v2Run the server with the desired model using the following command which normalize embeddings is enabled by default:MODEL=intfloat/e5-large-v2python-mopen.text.embeddings.serverSet theNORMALIZE_EMBEDDINGSto0orFalseif the model doesn't support normalize embeddings, for example:MODEL=intfloat/e5-large-v2NORMALIZE_EMBEDDINGS=0python-mopen.text.embeddings.serverIf a GPU is detected in the runtime environment, the server will automatically execute using thecubamode. However, you have the flexibility to specify theDEVICEenvironment variable to choose betweencpuandcuba. Here's an example of how to run the server with your desired configuration:MODEL=intfloat/e5-large-v2DEVICE=cpupython-mopen.text.embeddings.serverThis setup allows you to seamlessly switch between CPU and GPU modes, giving you control over the server's performance based on your specific requirements.You can enabled verbose logging by setting theVERBOSEto1, for example:MODEL=intfloat/e5-large-v2VERBOSE=1python-mopen.text.embeddings.serverYou will see the following text from your console once the server has started:INFO:Startedserverprocess[19705]INFO:Waitingforapplicationstartup.
INFO:Applicationstartupcomplete.
INFO:Uvicornrunningonhttp://0.0.0.0:8000(PressCTRL+Ctoquit)☁️ AWS Lambda FunctionTo deploy the embeddings endpoint as an AWS Lambda Function using GitHub Actions, follow these steps:Forkthe repo.Add your AWS credentials (AWS_KEYandAWS_SECRET) to the repository secrets. You can do this by navigating tohttps://github.com/your-username/open-text-embeddings/settings/secrets/actions.Manually trigger theDeploy DevorRemove DevGitHub Actions to deploy or remove the AWS Lambda Function.🧪 Testing the Embeddings EndpointTo test theembeddingsendpoint, the repository includes anembeddings.ipynbnotebook with a LangChain-compatibleOpenAIEmbeddingsclass.To get started:Install the dependencies by executing the following command:pipinstall--no-cache-diropen-text-embeddingsopenaiExecute the cells in the notebook to test the embeddings endpoint.🧑‍💼 ContributingContributions are welcome! Please check out the issues on the repository, and feel free to open a pull request.
For more information, please see thecontributing guidelines.Thank you very much for the following contributions:Vokturzcontributed#2: support for CPU/GPU choice and initialization before starting the app.jayxuzcontributed#5: improved OpenAI API compatibility, better support for previous versions of Python (start from v3.7), better defaults and bug fixes.📔 LicenseThis project is licensed under the terms of the MIT license.🗒️ CitationIf you utilize this repository, please consider citing it with:@misc{open-text-embeddings,
  author = {Lim Chee Kin},
  title = {open-text-embeddings: Open Source Text Embedding Models with OpenAI API-Compatible Endpoint},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/limcheekin/open-text-embeddings}},
}