cbensimon HF staff commited on
Commit
4edd64f
1 Parent(s): 1a7a096

SentenceTransformer GPU device

Browse files

Any reason for setting `mps` as device for the GPU model ?

Logs currently show this error :

``` python
File "/home/user/app/backend/semantic_search.py", line 45, in embed_func
return st_model_gpu.encode(query)
File "/home/user/.local/lib/python3.10/site-packages/sentence_transformers/SentenceTransformer.py", line 153, in encode
self.to(device)
File "/home/user/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1145, in to
return self._apply(convert)
File "/home/user/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 797, in _apply
module._apply(fn)
File "/home/user/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 797, in _apply
module._apply(fn)
File "/home/user/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 797, in _apply
module._apply(fn)
[Previous line repeated 1 more time]
File "/home/user/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 820, in _apply
param_applied = fn(param)
File "/home/user/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1143, in convert
return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
RuntimeError: PyTorch is not linked with support for mps devices
WARNING:backend.semantic_search:Using CPU
```

I did not test it with `cuda` so there's a slight risk that it breaks the Space

Files changed (1) hide show
  1. backend/semantic_search.py +1 -1
backend/semantic_search.py CHANGED
@@ -27,7 +27,7 @@ logger.info(f"Time taken to load LanceDB: {lancedb_loading_time:.6f} seconds")
27
  start_time = time.perf_counter()
28
 
29
  name = "sentence-transformers/paraphrase-multilingual-minilm-l12-v2"
30
- st_model_gpu = SentenceTransformer(name, device='mps')
31
  st_model_cpu = SentenceTransformer(name, device='cpu')
32
 
33
 
 
27
  start_time = time.perf_counter()
28
 
29
  name = "sentence-transformers/paraphrase-multilingual-minilm-l12-v2"
30
+ st_model_gpu = SentenceTransformer(name, device='cuda')
31
  st_model_cpu = SentenceTransformer(name, device='cpu')
32
 
33