Update model card slightly

#2
by tomaarsen HF staff - opened
Files changed (1) hide show
  1. README.md +22 -9
README.md CHANGED
@@ -114,7 +114,7 @@ language:
114
  # nomic-embed-text-v2-moe: Multilingual Mixture of Experts Text Embeddings
115
 
116
  ## Model Overview
117
- nomic-embed-text-v2-moe is SoTA multilingual MoE text embedding model:
118
 
119
  - **High Performance**: SoTA Multilingual performance compared to ~300M parameter models, competitive with models 2x in size
120
  - **Multilinguality**: Supports ~100 languages and trained over 1.6B pairs
@@ -157,14 +157,15 @@ For best performance on GPU, please install
157
  pip install torch transformers einops git+https://github.com/nomic-ai/megablocks.git
158
  ```
159
 
160
- **Important**: the text prompt *must* include a *task instruction prefix*, instructing the model which task is being performed.
 
 
161
 
162
- For queries/questions, please use `search_query: ` and `search_document: ` for the corresponding document
163
 
164
- **Transformers**
165
 
166
-
167
- If using Transformers, **make sure to prepend the task instruction prefix**
168
 
169
  ```python
170
  import torch
@@ -187,11 +188,17 @@ with torch.no_grad():
187
  model_output = model(**encoded_input)
188
  embeddings = mean_pooling(model_output, encoded_input['attention_mask'])
189
  embeddings = F.normalize(embeddings, p=2, dim=1)
 
 
 
 
 
 
190
  ```
191
 
192
- **SentenceTransformers**
193
 
194
- With SentenceTransformers, you can specify the prompt_name (query or passage)
195
 
196
  ```python
197
  from sentence_transformers import SentenceTransformer
@@ -199,6 +206,12 @@ from sentence_transformers import SentenceTransformer
199
  model = SentenceTransformer("nomic-ai/nomic-embed-text-v2-moe", trust_remote_code=True)
200
  sentences = ["Hello!", "¡Hola!"]
201
  embeddings = model.encode(sentences, prompt_name="passage")
 
 
 
 
 
 
202
  ```
203
 
204
  ## Performance
@@ -221,7 +234,7 @@ nomic-embed-text-v2-moe performance on BEIR at 768 dimension and truncated to 25
221
  ## Limitations
222
  - Performance may vary across different languages
223
  - Resource requirements may be higher than traditional dense models due to MoE architecture
224
- - Must have trust_remote_code=True when loading the model
225
 
226
  ## Training Details
227
 
 
114
  # nomic-embed-text-v2-moe: Multilingual Mixture of Experts Text Embeddings
115
 
116
  ## Model Overview
117
+ `nomic-embed-text-v2-moe` is SoTA multilingual MoE text embedding model:
118
 
119
  - **High Performance**: SoTA Multilingual performance compared to ~300M parameter models, competitive with models 2x in size
120
  - **Multilinguality**: Supports ~100 languages and trained over 1.6B pairs
 
157
  pip install torch transformers einops git+https://github.com/nomic-ai/megablocks.git
158
  ```
159
 
160
+ > [!IMPORTANT]
161
+ > **Important!**
162
+ > The text prompt *must* include a *task instruction prefix*, instructing the model which task is being performed.
163
 
164
+ Please use `search_query: ` before your queries/questions, and `search_document: ` before your documents.
165
 
166
+ ### Transformers
167
 
168
+ If using Transformers, **make sure to prepend the task instruction prefix**.
 
169
 
170
  ```python
171
  import torch
 
188
  model_output = model(**encoded_input)
189
  embeddings = mean_pooling(model_output, encoded_input['attention_mask'])
190
  embeddings = F.normalize(embeddings, p=2, dim=1)
191
+ print(embeddings.shape)
192
+ # torch.Size([2, 768])
193
+
194
+ similarity = F.cosine_similarity(embeddings[0], embeddings[1])
195
+ print(similarity)
196
+ # tensor(0.9118)
197
  ```
198
 
199
+ ### SentenceTransformers
200
 
201
+ With SentenceTransformers, you can specify the `prompt_name` as either `"query"` or `"passage"`, and the task instruction will be included automatically.
202
 
203
  ```python
204
  from sentence_transformers import SentenceTransformer
 
206
  model = SentenceTransformer("nomic-ai/nomic-embed-text-v2-moe", trust_remote_code=True)
207
  sentences = ["Hello!", "¡Hola!"]
208
  embeddings = model.encode(sentences, prompt_name="passage")
209
+ print(embeddings.shape)
210
+ # (2, 768)
211
+
212
+ similarity = model.similarity(embeddings[0], embeddings[1])
213
+ print(similarity)
214
+ # tensor([[0.9118]])
215
  ```
216
 
217
  ## Performance
 
234
  ## Limitations
235
  - Performance may vary across different languages
236
  - Resource requirements may be higher than traditional dense models due to MoE architecture
237
+ - Must use `trust_remote_code=True` when loading the model to use our custom architecture implementation
238
 
239
  ## Training Details
240