|
--- |
|
tags: |
|
- sentence-transformers |
|
- feature-extraction |
|
- sentence-similarity |
|
- transformers |
|
license: mit |
|
language: |
|
- en |
|
--- |
|
|
|
|
|
<h1 align="center">Infinity Embedding Model</h1> |
|
|
|
This is the stable default model for infinity. |
|
|
|
```bash |
|
pip install infinity_emb[all] |
|
``` |
|
|
|
More details about the infinity inference project please refer to the Github: [Infinity](https://github.com/michaelfeil/infinity). |
|
|
|
|
|
## Usage for Embedding Model via infinity in Python |
|
|
|
To deploy files with the [infinity_emb](https://github.com/michaelfeil/infinity) pip package. |
|
Recommended is `device="cuda", engine="torch"` with flash attention on gpu, and `device="cpu", engine="optimum"` for onnx inference. |
|
|
|
|
|
```python |
|
import asyncio |
|
from infinity_emb import AsyncEmbeddingEngine, EngineArgs |
|
|
|
sentences = ["Embed this is sentence via Infinity.", "Paris is in France."] |
|
engine = AsyncEmbeddingEngine.from_args( |
|
EngineArgs( |
|
model_name_or_path = "michaelfeil/bge-small-en-v1.5", |
|
device="cuda", |
|
# or device="cpu" |
|
engine="torch", |
|
# or engine="optimum" |
|
compile=True # enable torch.compile |
|
)) |
|
|
|
async def main(): |
|
async with engine: |
|
embeddings, usage = await engine.embed(sentences=sentences) |
|
asyncio.run(main()) |
|
``` |
|
|
|
## CLI interface |
|
|
|
The same args |
|
|
|
```bash |
|
pip install infinity_emb |
|
infinity_emb --model-name-or-path michaelfeil/bge-small-en-v1.5 --port 7997 |
|
``` |
|
|
|
|
|
## Contact |
|
If you have any question or suggestion related to this project, feel free to open an issue or pull request. |
|
You also can email Michael Feil (infinity at michaelfeil.eu). |
|
|
|
|
|
## Citation |
|
|
|
If you find this repository useful, please consider giving a star :star: and citation |
|
|
|
``` |
|
@software{Feil_Infinity_2023, |
|
author = {Feil, Michael}, |
|
month = oct, |
|
title = {{Infinity - To Embeddings and Beyond}}, |
|
url = {https://github.com/michaelfeil/infinity}, |
|
year = {2023} |
|
} |
|
``` |
|
|
|
## License |
|
Infinity is licensed under the [MIT License](https://github.com/michaelfeil/infinity/blob/master/LICENSE). |
|
|
|
|
