sergeyzh commited on
Commit
85fd00f
1 Parent(s): c8c06a2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +27 -3
README.md CHANGED
@@ -78,15 +78,39 @@ print(util.dot_score(embeddings, embeddings))
78
 
79
  ## Быстродействие и размеры
80
 
 
 
81
  | Модель | CPU | GPU | size | dim | n_ctx | n_vocab |
82
  |:---------------------------------|----------:|----------:|----------:|----------:|----------:|----------:|
83
  | [intfloat/multilingual-e5-large](https://huggingface.co/intfloat/multilingual-e5-large) | 149.026 | 15.629 | 2136 | 1024 | 514 | 250002 |
84
  | **sergeyzh/LaBSE-ru-sts** |**42.835** | **8.561** | **490** | **768** | **512** | **55083** |
85
- | [sergeyzh/rubert-tiny-sts](https://huggingface.co/sergeyzh/rubert-tiny-sts) | 3.208 | 2.866 | 111 | 312 | 2048 | 83828 |
86
- | [Tochka-AI/ruRoPEBert-e5-base-512](https://huggingface.co/Tochka-AI/ruRoPEBert-e5-base-512) | 43.314 | 9.338 | 530 | 768 | 512 | 69382 |
87
  | [cointegrated/LaBSE-en-ru](https://huggingface.co/cointegrated/LaBSE-en-ru) | 42.867 | 8.549 | 490 | 768 | 512 | 55083 |
88
- | [cointegrated/rubert-tiny2](https://huggingface.co/cointegrated/rubert-tiny2) | 3.212 | 2.850 | 111 | 312 | 2048 | 83828 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
89
 
 
 
 
 
 
 
 
90
 
91
  ## Связанные ресурсы
92
  Вопросы использования модели обсуждаются в [русскоязычном чате NLP](https://t.me/natural_language_processing).
 
78
 
79
  ## Быстродействие и размеры
80
 
81
+ На бенчмарке [encodechka](https://github.com/avidale/encodechka):
82
+
83
  | Модель | CPU | GPU | size | dim | n_ctx | n_vocab |
84
  |:---------------------------------|----------:|----------:|----------:|----------:|----------:|----------:|
85
  | [intfloat/multilingual-e5-large](https://huggingface.co/intfloat/multilingual-e5-large) | 149.026 | 15.629 | 2136 | 1024 | 514 | 250002 |
86
  | **sergeyzh/LaBSE-ru-sts** |**42.835** | **8.561** | **490** | **768** | **512** | **55083** |
87
+ | [sergeyzh/rubert-tiny-sts](https://huggingface.co/sergeyzh/rubert-tiny-sts) | 3.208 | 3.379 | 111 | 312 | 2048 | 83828 |
88
+ | [Tochka-AI/ruRoPEBert-e5-base-512](https://huggingface.co/Tochka-AI/ruRoPEBert-e5-base-512) | 43.314 | 9.338 | 532 | 768 | 512 | 69382 |
89
  | [cointegrated/LaBSE-en-ru](https://huggingface.co/cointegrated/LaBSE-en-ru) | 42.867 | 8.549 | 490 | 768 | 512 | 55083 |
90
+ | [cointegrated/rubert-tiny2](https://huggingface.co/cointegrated/rubert-tiny2) | 3.212 | 3.384 | 111 | 312 | 2048 | 83828 |
91
+
92
+
93
+
94
+ При использовании батчей с `sentence_transformers`:
95
+
96
+ ```python
97
+ from sentence_transformers import SentenceTransformer
98
+
99
+ model_name = 'sergeyzh/LaBSE-ru-sts'
100
+ model = SentenceTransformer(model_name, device='cpu')
101
+ sentences = ["Тест быстродействия на CPU Ryzen 7 3800X: batch = 50"] * 50
102
+ %timeit -n 5 -r 3 model.encode(sentences)
103
+
104
+ # 882 ms ± 104 ms per loop (mean ± std. dev. of 3 runs, 5 loops each)
105
+ # 50/0.882 = 57 snt/s
106
 
107
+ model = SentenceTransformer(model_name, device='cuda')
108
+ sentences = ["Тест быстродействия на GPU RTX 3060: batch = 1500"] * 1500
109
+ %timeit -n 5 -r 3 model.encode(sentences)
110
+
111
+ # 792 ms ± 29 ms per loop (mean ± std. dev. of 3 runs, 5 loops each)
112
+ # 1500/0.792 = 1894 snt/s
113
+ ```
114
 
115
  ## Связанные ресурсы
116
  Вопросы использования модели обсуждаются в [русскоязычном чате NLP](https://t.me/natural_language_processing).