File size: 1,990 Bytes
c48371d
3e84a86
 
 
 
 
c48371d
3e84a86
 
c48371d
3e84a86
 
b6f0cde
3e84a86
c5e53d3
3e84a86
c5e53d3
 
 
3e84a86
c5e53d3
3e84a86
 
c5e53d3
3e84a86
c256d82
3e84a86
 
c5e53d3
3e84a86
 
 
 
 
 
c256d82
 
c5e53d3
 
ab7b31b
c256d82
 
3e84a86
 
 
 
 
 
 
 
c5e53d3
 
 
 
 
 
 
 
 
3e84a86
 
 
b6f0cde
3e84a86
 
 
 
 
 
 
b6f0cde
 
 
 
 
 
3e84a86
 
 
 
b6f0cde
3e84a86
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
---
tags:
- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers
license: mit
language:
- en
---


<h1 align="center">Infinity Embedding Model</h1>

This is the stable default model for infinity.

```bash
pip install infinity_emb[all]
```

More details about the infinity inference project please refer to the Github: [Infinity](https://github.com/michaelfeil/infinity).


##  Usage for Embedding Model via infinity in Python

To deploy files with the [infinity_emb](https://github.com/michaelfeil/infinity) pip package.
Recommended is `device="cuda", engine="torch"` with flash attention on gpu, and `device="cpu", engine="optimum"` for onnx inference.


```python
import asyncio
from infinity_emb import AsyncEmbeddingEngine, EngineArgs

sentences = ["Embed this is sentence via Infinity.", "Paris is in France."]
engine = AsyncEmbeddingEngine.from_args(
    EngineArgs(
        model_name_or_path = "michaelfeil/bge-small-en-v1.5",
        device="cuda",
        # or device="cpu"
        engine="torch",
        # or engine="optimum"
        compile=True # enable torch.compile
))

async def main(): 
    async with engine:
        embeddings, usage = await engine.embed(sentences=sentences)
asyncio.run(main())
```

## CLI interface

The same args

```bash
pip install infinity_emb
infinity_emb --model-name-or-path michaelfeil/bge-small-en-v1.5 --port 7997
```


## Contact
If you have any question or suggestion related to this project, feel free to open an issue or pull request.
You also can email Michael Feil (infinity at michaelfeil.eu).


## Citation

If you find this repository useful, please consider giving a star :star: and citation

```
@software{Feil_Infinity_2023,
author = {Feil, Michael},
month = oct,
title = {{Infinity - To Embeddings and Beyond}},
url = {https://github.com/michaelfeil/infinity},
year = {2023}
}
```

## License
Infinity is licensed under the [MIT License](https://github.com/michaelfeil/infinity/blob/master/LICENSE).