Hugging Face's logo

Spaces:
vutuka
/
fast-inference-nllb
Paused

App Files Community
winstxnhdw commited on
Commit
a9f955e
•
1 Parent(s): 7a37fb5

perf: use `AVX512` for inference

Browse files
Files changed (2) hide show
  1. Dockerfile +5 -3
  2. README.md +1 -1
Dockerfile CHANGED
@@ -1,7 +1,9 @@
1
  FROM ghcr.io/winstxnhdw/nllb-api:main
2
 
3
  ENV SERVER_PORT 5000
4
- ENV BACKEND_INTERNAL_URL http://localhost:5000
5
- ENV BACKEND_URL :7860
 
 
6
 
7
- EXPOSE 7860
 
1
  FROM ghcr.io/winstxnhdw/nllb-api:main
2
 
3
  ENV SERVER_PORT 5000
4
+ ENV APP_PORT 7860
5
+ ENV OMP_NUM_THREADS 4
6
+ ENV CT2_USE_EXPERIMENTAL_PACKED_GEMM 1
7
+ ENV CT2_FORCE_CPU_ISA AVX512
8
 
9
+ EXPOSE $APP_PORT
README.md CHANGED
@@ -8,7 +8,7 @@ python_version: 3.11.4
8
  models: [winstxnhdw/nllb-200-distilled-1.3B-ct2-int8]
9
  tags: [translation, translator, nllb, ctranslate2]
10
  pinned: true
11
- base_path: /docs
12
  ---
13
 
14
  Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
8
  models: [winstxnhdw/nllb-200-distilled-1.3B-ct2-int8]
9
  tags: [translation, translator, nllb, ctranslate2]
10
  pinned: true
11
+ base_path: /api/docs
12
  ---
13
 
14
  Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference