michaelfeil commited on
Commit
a569d04
1 Parent(s): a5beb1e

Update infinity section in readme

Browse files
Files changed (1) hide show
  1. README.md +8 -1
README.md CHANGED
@@ -2904,7 +2904,7 @@ from infinity_emb import AsyncEmbeddingEngine, EngineArgs
2904
 
2905
  sentences = ["Embed this is sentence via Infinity.", "Paris is in France."]
2906
  engine = AsyncEmbeddingEngine.from_args(
2907
- EngineArgs(model_name_or_path = "BAAI/bge-large-en-v1.5", device="cpu", engine="optimum" # or engine="torch"
2908
  ))
2909
 
2910
  async def main():
@@ -2913,6 +2913,13 @@ async def main():
2913
  asyncio.run(main())
2914
  ```
2915
 
 
 
 
 
 
 
 
2916
  ### Usage for Reranker
2917
 
2918
  Different from embedding model, reranker uses question and document as input and directly output similarity instead of embedding.
 
2904
 
2905
  sentences = ["Embed this is sentence via Infinity.", "Paris is in France."]
2906
  engine = AsyncEmbeddingEngine.from_args(
2907
+ EngineArgs(model_name_or_path = "BAAI/bge-base-en-v1.5", device="cpu", engine="optimum" # or engine="torch"
2908
  ))
2909
 
2910
  async def main():
 
2913
  asyncio.run(main())
2914
  ```
2915
 
2916
+ for a docker based deployment with infinity using torch:
2917
+ ```bash
2918
+ docker run --gpus all -v $PWD/data:/app/.cache -p "7997":"7997" \
2919
+ michaelf34/infinity:0.0.68 \
2920
+ v2 --model-id BAAI/bge-base-en-v1.5 --revision "main" --dtype float16 --batch-size 32 --engine torch --port 7997
2921
+ ```
2922
+
2923
  ### Usage for Reranker
2924
 
2925
  Different from embedding model, reranker uses question and document as input and directly output similarity instead of embedding.