Hugging Face's logo

CocoRoF
/
ModernBERT-SimCSE-multitask_v03-distill

Sentence Similarity
sentence-transformers
Safetensors
modernbert
feature-extraction
Generated from Trainer
dataset_size:449904
loss:CosineSimilarityLoss
Eval Results
Inference Endpoints
Model card Files Community
CocoRoF commited on
Commit
7f9cc7f
·
verified ·
1 Parent(s): 3e54693

CocoRoF/ModernBERT-SimCSE-multitask_v03-distill

Browse files
Files changed (7) hide show
  1. 1_Pooling/config.json +10 -0
  2. 2_Dense/config.json +1 -0
  3. 2_Dense/model.safetensors +3 -0
  4. README.md +892 -0
  5. config_sentence_transformers.json +10 -0
  6. modules.json +20 -0
  7. sentence_bert_config.json +4 -0
1_Pooling/config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "word_embedding_dimension": 768,
3
+ "pooling_mode_cls_token": false,
4
+ "pooling_mode_mean_tokens": true,
5
+ "pooling_mode_max_tokens": false,
6
+ "pooling_mode_mean_sqrt_len_tokens": false,
7
+ "pooling_mode_weightedmean_tokens": false,
8
+ "pooling_mode_lasttoken": false,
9
+ "include_prompt": true
10
+ }
2_Dense/config.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"in_features": 768, "out_features": 1024, "bias": true, "activation_function": "torch.nn.modules.activation.Tanh"}
2_Dense/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:98502da6c4dbee1502fa8ebc31ff356b5762eb792a899d4e5339d3cd3a7c0ae4
3
+ size 3149984
README.md ADDED
@@ -0,0 +1,892 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - sentence-transformers
4
+ - sentence-similarity
5
+ - feature-extraction
6
+ - generated_from_trainer
7
+ - dataset_size:449904
8
+ - loss:CosineSimilarityLoss
9
+ base_model: CocoRoF/ModernBERT-SimCSE-multitask_v03-retry
10
+ widget:
11
+ - source_sentence: 우리는 움직이는 동행 우주 정지 좌표계에 비례하여 이동하고 있습니다 ... 약 371km / s에서 별자리 leo
12
+ 쪽으로. "
13
+ sentences:
14
+ - 두 마리의 독수리가 가지에 앉는다.
15
+ - 다른 물체와는 관련이 없는 '정지'는 없다.
16
+ - 소녀는 버스의 열린 문 앞에 서 있다.
17
+ - source_sentence: 숲에는 개들이 있다.
18
+ sentences:
19
+ - 양을 보는 아이들.
20
+ - 여왕의 배우자를 "왕"이라고 부르지 않는 것은 아주 좋은 이유가 있다. 왜냐하면 그들은 왕이 아니기 때문이다.
21
+ - 개들은 숲속에 혼자 있다.
22
+ - source_sentence: '첫째, 두 가지 다른 종류의 대시가 있다는 것을 알아야 합니다 : en 대시와 em 대시.'
23
+ sentences:
24
+ - 그들은 그 물건들을 집 주변에 두고 가거나 집의 정리를 해칠 의도가 없다.
25
+ - 세미콜론은 혼자 있을 수 있는 문장에 참여하는데 사용되지만, 그들의 관계를 강조하기 위해 결합됩니다.
26
+ - 그의 남동생이 지켜보는 동안 집 앞에서 트럼펫을 연주하는 금발의 아이.
27
+ - source_sentence: 한 여성이 생선 껍질을 벗기고 있다.
28
+ sentences:
29
+ - 한 남자가 수영장으로 뛰어들었다.
30
+ - 한 여성이 프라이팬에 노란 혼합물을 부어 넣고 있다.
31
+ - 두 마리의 갈색 개가 눈 속에서 서로 놀고 있다.
32
+ - source_sentence: 버스가 바쁜 길을 따라 운전한다.
33
+ sentences:
34
+ - 우리와 같은 태양계가 은하계 밖에서 존재할 수도 있을 것입니다.
35
+ - 그 여자는 데이트하러 가는 중이다.
36
+ - 녹색 버스가 도로를 따라 내려간다.
37
+ datasets:
38
+ - x2bee/misc_sts_pairs_v2_kor_kosimcse
39
+ pipeline_tag: sentence-similarity
40
+ library_name: sentence-transformers
41
+ metrics:
42
+ - pearson_cosine
43
+ - spearman_cosine
44
+ - pearson_euclidean
45
+ - spearman_euclidean
46
+ - pearson_manhattan
47
+ - spearman_manhattan
48
+ - pearson_dot
49
+ - spearman_dot
50
+ - pearson_max
51
+ - spearman_max
52
+ model-index:
53
+ - name: SentenceTransformer based on CocoRoF/ModernBERT-SimCSE-multitask_v03-retry
54
+ results:
55
+ - task:
56
+ type: semantic-similarity
57
+ name: Semantic Similarity
58
+ dataset:
59
+ name: sts dev
60
+ type: sts_dev
61
+ metrics:
62
+ - type: pearson_cosine
63
+ value: 0.8220874775898197
64
+ name: Pearson Cosine
65
+ - type: spearman_cosine
66
+ value: 0.8282368218808581
67
+ name: Spearman Cosine
68
+ - type: pearson_euclidean
69
+ value: 0.7929031352092236
70
+ name: Pearson Euclidean
71
+ - type: spearman_euclidean
72
+ value: 0.7979913252239026
73
+ name: Spearman Euclidean
74
+ - type: pearson_manhattan
75
+ value: 0.7936882861676204
76
+ name: Pearson Manhattan
77
+ - type: spearman_manhattan
78
+ value: 0.7996541111809876
79
+ name: Spearman Manhattan
80
+ - type: pearson_dot
81
+ value: 0.7010536213435227
82
+ name: Pearson Dot
83
+ - type: spearman_dot
84
+ value: 0.6844746263331734
85
+ name: Spearman Dot
86
+ - type: pearson_max
87
+ value: 0.8220874775898197
88
+ name: Pearson Max
89
+ - type: spearman_max
90
+ value: 0.8282368218808581
91
+ name: Spearman Max
92
+ ---
93
+
94
+ # SentenceTransformer based on CocoRoF/ModernBERT-SimCSE-multitask_v03-retry
95
+
96
+ This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [CocoRoF/ModernBERT-SimCSE-multitask_v03-retry](https://huggingface.co/CocoRoF/ModernBERT-SimCSE-multitask_v03-retry) on the [misc_sts_pairs_v2_kor_kosimcse](https://huggingface.co/datasets/x2bee/misc_sts_pairs_v2_kor_kosimcse) dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
97
+
98
+ ## Model Details
99
+
100
+ ### Model Description
101
+ - **Model Type:** Sentence Transformer
102
+ - **Base model:** [CocoRoF/ModernBERT-SimCSE-multitask_v03-retry](https://huggingface.co/CocoRoF/ModernBERT-SimCSE-multitask_v03-retry) <!-- at revision 8ea8efa5d7e41826a9093b28badc01ed44d01ace -->
103
+ - **Maximum Sequence Length:** 2048 tokens
104
+ - **Output Dimensionality:** 1024 dimensions
105
+ - **Similarity Function:** Cosine Similarity
106
+ - **Training Dataset:**
107
+ - [misc_sts_pairs_v2_kor_kosimcse](https://huggingface.co/datasets/x2bee/misc_sts_pairs_v2_kor_kosimcse)
108
+ <!-- - **Language:** Unknown -->
109
+ <!-- - **License:** Unknown -->
110
+
111
+ ### Model Sources
112
+
113
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
114
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
115
+ - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
116
+
117
+ ### Full Model Architecture
118
+
119
+ ```
120
+ SentenceTransformer(
121
+ (0): Transformer({'max_seq_length': 2048, 'do_lower_case': False}) with Transformer model: ModernBertModel
122
+ (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
123
+ (2): Dense({'in_features': 768, 'out_features': 1024, 'bias': True, 'activation_function': 'torch.nn.modules.activation.Tanh'})
124
+ )
125
+ ```
126
+
127
+ ## Usage
128
+
129
+ ### Direct Usage (Sentence Transformers)
130
+
131
+ First install the Sentence Transformers library:
132
+
133
+ ```bash
134
+ pip install -U sentence-transformers
135
+ ```
136
+
137
+ Then you can load this model and run inference.
138
+ ```python
139
+ from sentence_transformers import SentenceTransformer
140
+
141
+ # Download from the 🤗 Hub
142
+ model = SentenceTransformer("CocoRoF/ModernBERT-SimCSE-multitask_v03-distill")
143
+ # Run inference
144
+ sentences = [
145
+ '버스가 바쁜 길을 따라 운전한다.',
146
+ '녹색 버스가 도로를 따라 내려간다.',
147
+ '그 여자는 데이트하러 가는 중이다.',
148
+ ]
149
+ embeddings = model.encode(sentences)
150
+ print(embeddings.shape)
151
+ # [3, 1024]
152
+
153
+ # Get the similarity scores for the embeddings
154
+ similarities = model.similarity(embeddings, embeddings)
155
+ print(similarities.shape)
156
+ # [3, 3]
157
+ ```
158
+
159
+ <!--
160
+ ### Direct Usage (Transformers)
161
+
162
+ <details><summary>Click to see the direct usage in Transformers</summary>
163
+
164
+ </details>
165
+ -->
166
+
167
+ <!--
168
+ ### Downstream Usage (Sentence Transformers)
169
+
170
+ You can finetune this model on your own dataset.
171
+
172
+ <details><summary>Click to expand</summary>
173
+
174
+ </details>
175
+ -->
176
+
177
+ <!--
178
+ ### Out-of-Scope Use
179
+
180
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
181
+ -->
182
+
183
+ ## Evaluation
184
+
185
+ ### Metrics
186
+
187
+ #### Semantic Similarity
188
+
189
+ * Dataset: `sts_dev`
190
+ * Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)
191
+
192
+ | Metric | Value |
193
+ |:-------------------|:-----------|
194
+ | pearson_cosine | 0.8221 |
195
+ | spearman_cosine | 0.8282 |
196
+ | pearson_euclidean | 0.7929 |
197
+ | spearman_euclidean | 0.798 |
198
+ | pearson_manhattan | 0.7937 |
199
+ | spearman_manhattan | 0.7997 |
200
+ | pearson_dot | 0.7011 |
201
+ | spearman_dot | 0.6845 |
202
+ | pearson_max | 0.8221 |
203
+ | **spearman_max** | **0.8282** |
204
+
205
+ <!--
206
+ ## Bias, Risks and Limitations
207
+
208
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
209
+ -->
210
+
211
+ <!--
212
+ ### Recommendations
213
+
214
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
215
+ -->
216
+
217
+ ## Training Details
218
+
219
+ ### Training Dataset
220
+
221
+ #### misc_sts_pairs_v2_kor_kosimcse
222
+
223
+ * Dataset: [misc_sts_pairs_v2_kor_kosimcse](https://huggingface.co/datasets/x2bee/misc_sts_pairs_v2_kor_kosimcse) at [e747415](https://huggingface.co/datasets/x2bee/misc_sts_pairs_v2_kor_kosimcse/tree/e747415cfe9ff51d1c1550b8a07e5014c01dea59)
224
+ * Size: 449,904 training samples
225
+ * Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
226
+ * Approximate statistics based on the first 1000 samples:
227
+ | | sentence1 | sentence2 | score |
228
+ |:--------|:---------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------|
229
+ | type | string | string | float |
230
+ | details | <ul><li>min: 6 tokens</li><li>mean: 18.3 tokens</li><li>max: 69 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 18.69 tokens</li><li>max: 66 tokens</li></ul> | <ul><li>min: 0.11</li><li>mean: 0.77</li><li>max: 1.0</li></ul> |
231
+ * Samples:
232
+ | sentence1 | sentence2 | score |
233
+ |:-------------------------------------------------|:-------------------------------------------|:--------------------------------|
234
+ | <code>주홍글씨는 언제 출판되었습니까?</code> | <code>《주홍글씨》는 몇 년에 출판되었습니까?</code> | <code>0.8638778924942017</code> |
235
+ | <code>폴란드에서 빨간색과 흰색은 무엇을 의미합니까?</code> | <code>폴란드 국기의 색상은 무엇입니까?</code> | <code>0.6773715019226074</code> |
236
+ | <code>노르만인들은 방어를 위해 모트와 베일리 성을 어떻게 사용했는가?</code> | <code>11세기에는 어떻게 모트와 베일리 성을 만들었습니까?</code> | <code>0.7460665702819824</code> |
237
+ * Loss: [<code>CosineSimilarityLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosinesimilarityloss) with these parameters:
238
+ ```json
239
+ {
240
+ "loss_fct": "torch.nn.modules.loss.MSELoss"
241
+ }
242
+ ```
243
+
244
+ ### Evaluation Dataset
245
+
246
+ #### Unnamed Dataset
247
+
248
+ * Size: 1,500 evaluation samples
249
+ * Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
250
+ * Approximate statistics based on the first 1000 samples:
251
+ | | sentence1 | sentence2 | score |
252
+ |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------|
253
+ | type | string | string | float |
254
+ | details | <ul><li>min: 7 tokens</li><li>mean: 20.38 tokens</li><li>max: 52 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 20.52 tokens</li><li>max: 54 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.42</li><li>max: 1.0</li></ul> |
255
+ * Samples:
256
+ | sentence1 | sentence2 | score |
257
+ |:-------------------------------------|:------------------------------------|:------------------|
258
+ | <code>안전모를 가진 한 남자가 춤을 추고 있다.</code> | <code>안전모를 쓴 한 남자가 춤을 추고 있다.</code> | <code>1.0</code> |
259
+ | <code>어린아이가 말을 타고 있다.</code> | <code>아이가 말을 타고 있다.</code> | <code>0.95</code> |
260
+ | <code>한 남자가 뱀에게 쥐를 먹이고 있다.</code> | <code>남자가 뱀에게 쥐를 먹이고 있다.</code> | <code>1.0</code> |
261
+ * Loss: [<code>CosineSimilarityLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosinesimilarityloss) with these parameters:
262
+ ```json
263
+ {
264
+ "loss_fct": "torch.nn.modules.loss.MSELoss"
265
+ }
266
+ ```
267
+
268
+ ### Training Hyperparameters
269
+ #### Non-Default Hyperparameters
270
+
271
+ - `overwrite_output_dir`: True
272
+ - `eval_strategy`: steps
273
+ - `gradient_accumulation_steps`: 16
274
+ - `learning_rate`: 8e-05
275
+ - `num_train_epochs`: 10.0
276
+ - `warmup_ratio`: 0.2
277
+ - `push_to_hub`: True
278
+ - `hub_model_id`: CocoRoF/ModernBERT-SimCSE-multitask_v03-distill
279
+ - `hub_strategy`: checkpoint
280
+ - `batch_sampler`: no_duplicates
281
+
282
+ #### All Hyperparameters
283
+ <details><summary>Click to expand</summary>
284
+
285
+ - `overwrite_output_dir`: True
286
+ - `do_predict`: False
287
+ - `eval_strategy`: steps
288
+ - `prediction_loss_only`: True
289
+ - `per_device_train_batch_size`: 8
290
+ - `per_device_eval_batch_size`: 8
291
+ - `per_gpu_train_batch_size`: None
292
+ - `per_gpu_eval_batch_size`: None
293
+ - `gradient_accumulation_steps`: 16
294
+ - `eval_accumulation_steps`: None
295
+ - `torch_empty_cache_steps`: None
296
+ - `learning_rate`: 8e-05
297
+ - `weight_decay`: 0.0
298
+ - `adam_beta1`: 0.9
299
+ - `adam_beta2`: 0.999
300
+ - `adam_epsilon`: 1e-08
301
+ - `max_grad_norm`: 1.0
302
+ - `num_train_epochs`: 10.0
303
+ - `max_steps`: -1
304
+ - `lr_scheduler_type`: linear
305
+ - `lr_scheduler_kwargs`: {}
306
+ - `warmup_ratio`: 0.2
307
+ - `warmup_steps`: 0
308
+ - `log_level`: passive
309
+ - `log_level_replica`: warning
310
+ - `log_on_each_node`: True
311
+ - `logging_nan_inf_filter`: True
312
+ - `save_safetensors`: True
313
+ - `save_on_each_node`: False
314
+ - `save_only_model`: False
315
+ - `restore_callback_states_from_checkpoint`: False
316
+ - `no_cuda`: False
317
+ - `use_cpu`: False
318
+ - `use_mps_device`: False
319
+ - `seed`: 42
320
+ - `data_seed`: None
321
+ - `jit_mode_eval`: False
322
+ - `use_ipex`: False
323
+ - `bf16`: False
324
+ - `fp16`: False
325
+ - `fp16_opt_level`: O1
326
+ - `half_precision_backend`: auto
327
+ - `bf16_full_eval`: False
328
+ - `fp16_full_eval`: False
329
+ - `tf32`: None
330
+ - `local_rank`: 0
331
+ - `ddp_backend`: None
332
+ - `tpu_num_cores`: None
333
+ - `tpu_metrics_debug`: False
334
+ - `debug`: []
335
+ - `dataloader_drop_last`: True
336
+ - `dataloader_num_workers`: 0
337
+ - `dataloader_prefetch_factor`: None
338
+ - `past_index`: -1
339
+ - `disable_tqdm`: False
340
+ - `remove_unused_columns`: True
341
+ - `label_names`: None
342
+ - `load_best_model_at_end`: False
343
+ - `ignore_data_skip`: False
344
+ - `fsdp`: []
345
+ - `fsdp_min_num_params`: 0
346
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
347
+ - `fsdp_transformer_layer_cls_to_wrap`: None
348
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
349
+ - `deepspeed`: None
350
+ - `label_smoothing_factor`: 0.0
351
+ - `optim`: adamw_torch
352
+ - `optim_args`: None
353
+ - `adafactor`: False
354
+ - `group_by_length`: False
355
+ - `length_column_name`: length
356
+ - `ddp_find_unused_parameters`: None
357
+ - `ddp_bucket_cap_mb`: None
358
+ - `ddp_broadcast_buffers`: False
359
+ - `dataloader_pin_memory`: True
360
+ - `dataloader_persistent_workers`: False
361
+ - `skip_memory_metrics`: True
362
+ - `use_legacy_prediction_loop`: False
363
+ - `push_to_hub`: True
364
+ - `resume_from_checkpoint`: None
365
+ - `hub_model_id`: CocoRoF/ModernBERT-SimCSE-multitask_v03-distill
366
+ - `hub_strategy`: checkpoint
367
+ - `hub_private_repo`: None
368
+ - `hub_always_push`: False
369
+ - `gradient_checkpointing`: False
370
+ - `gradient_checkpointing_kwargs`: None
371
+ - `include_inputs_for_metrics`: False
372
+ - `include_for_metrics`: []
373
+ - `eval_do_concat_batches`: True
374
+ - `fp16_backend`: auto
375
+ - `push_to_hub_model_id`: None
376
+ - `push_to_hub_organization`: None
377
+ - `mp_parameters`:
378
+ - `auto_find_batch_size`: False
379
+ - `full_determinism`: False
380
+ - `torchdynamo`: None
381
+ - `ray_scope`: last
382
+ - `ddp_timeout`: 1800
383
+ - `torch_compile`: False
384
+ - `torch_compile_backend`: None
385
+ - `torch_compile_mode`: None
386
+ - `dispatch_batches`: None
387
+ - `split_batches`: None
388
+ - `include_tokens_per_second`: False
389
+ - `include_num_input_tokens_seen`: False
390
+ - `neftune_noise_alpha`: None
391
+ - `optim_target_modules`: None
392
+ - `batch_eval_metrics`: False
393
+ - `eval_on_start`: False
394
+ - `use_liger_kernel`: False
395
+ - `eval_use_gather_object`: False
396
+ - `average_tokens_across_devices`: False
397
+ - `prompts`: None
398
+ - `batch_sampler`: no_duplicates
399
+ - `multi_dataset_batch_sampler`: proportional
400
+
401
+ </details>
402
+
403
+ ### Training Logs
404
+ <details><summary>Click to expand</summary>
405
+
406
+ | Epoch | Step | Training Loss | Validation Loss | sts_dev_spearman_max |
407
+ |:------:|:----:|:-------------:|:---------------:|:--------------------:|
408
+ | 0.0228 | 10 | 0.3524 | - | - |
409
+ | 0.0455 | 20 | 0.3496 | - | - |
410
+ | 0.0683 | 30 | 0.3515 | - | - |
411
+ | 0.0911 | 40 | 0.348 | - | - |
412
+ | 0.1138 | 50 | 0.3409 | - | - |
413
+ | 0.1366 | 60 | 0.347 | - | - |
414
+ | 0.1593 | 70 | 0.3377 | - | - |
415
+ | 0.1821 | 80 | 0.3317 | - | - |
416
+ | 0.2049 | 90 | 0.3279 | - | - |
417
+ | 0.2276 | 100 | 0.3264 | - | - |
418
+ | 0.2504 | 110 | 0.3116 | - | - |
419
+ | 0.2732 | 120 | 0.3055 | - | - |
420
+ | 0.2959 | 130 | 0.3042 | - | - |
421
+ | 0.3187 | 140 | 0.2928 | - | - |
422
+ | 0.3414 | 150 | 0.2835 | - | - |
423
+ | 0.3642 | 160 | 0.2665 | - | - |
424
+ | 0.3870 | 170 | 0.2665 | - | - |
425
+ | 0.4097 | 180 | 0.2486 | - | - |
426
+ | 0.4325 | 190 | 0.2387 | - | - |
427
+ | 0.4553 | 200 | 0.2283 | - | - |
428
+ | 0.4780 | 210 | 0.2237 | - | - |
429
+ | 0.5008 | 220 | 0.2204 | - | - |
430
+ | 0.5235 | 230 | 0.205 | - | - |
431
+ | 0.5463 | 240 | 0.2002 | - | - |
432
+ | 0.5691 | 250 | 0.1904 | 0.0330 | 0.7921 |
433
+ | 0.5918 | 260 | 0.1834 | - | - |
434
+ | 0.6146 | 270 | 0.1776 | - | - |
435
+ | 0.6374 | 280 | 0.1665 | - | - |
436
+ | 0.6601 | 290 | 0.1625 | - | - |
437
+ | 0.6829 | 300 | 0.1585 | - | - |
438
+ | 0.7056 | 310 | 0.1522 | - | - |
439
+ | 0.7284 | 320 | 0.1552 | - | - |
440
+ | 0.7512 | 330 | 0.1448 | - | - |
441
+ | 0.7739 | 340 | 0.1428 | - | - |
442
+ | 0.7967 | 350 | 0.1401 | - | - |
443
+ | 0.8195 | 360 | 0.1399 | - | - |
444
+ | 0.8422 | 370 | 0.1389 | - | - |
445
+ | 0.8650 | 380 | 0.1372 | - | - |
446
+ | 0.8878 | 390 | 0.1338 | - | - |
447
+ | 0.9105 | 400 | 0.1361 | - | - |
448
+ | 0.9333 | 410 | 0.1389 | - | - |
449
+ | 0.9560 | 420 | 0.1328 | - | - |
450
+ | 0.9788 | 430 | 0.1375 | - | - |
451
+ | 1.0 | 440 | 0.1266 | - | - |
452
+ | 1.0228 | 450 | 0.1269 | - | - |
453
+ | 1.0455 | 460 | 0.1262 | - | - |
454
+ | 1.0683 | 470 | 0.127 | - | - |
455
+ | 1.0911 | 480 | 0.1306 | - | - |
456
+ | 1.1138 | 490 | 0.1266 | - | - |
457
+ | 1.1366 | 500 | 0.1247 | 0.0405 | 0.7995 |
458
+ | 1.1593 | 510 | 0.1258 | - | - |
459
+ | 1.1821 | 520 | 0.1277 | - | - |
460
+ | 1.2049 | 530 | 0.13 | - | - |
461
+ | 1.2276 | 540 | 0.1291 | - | - |
462
+ | 1.2504 | 550 | 0.1287 | - | - |
463
+ | 1.2732 | 560 | 0.1233 | - | - |
464
+ | 1.2959 | 570 | 0.1242 | - | - |
465
+ | 1.3187 | 580 | 0.1242 | - | - |
466
+ | 1.3414 | 590 | 0.1227 | - | - |
467
+ | 1.3642 | 600 | 0.1201 | - | - |
468
+ | 1.3870 | 610 | 0.1247 | - | - |
469
+ | 1.4097 | 620 | 0.1249 | - | - |
470
+ | 1.4325 | 630 | 0.1213 | - | - |
471
+ | 1.4553 | 640 | 0.1217 | - | - |
472
+ | 1.4780 | 650 | 0.1204 | - | - |
473
+ | 1.5008 | 660 | 0.1191 | - | - |
474
+ | 1.5235 | 670 | 0.1163 | - | - |
475
+ | 1.5463 | 680 | 0.1171 | - | - |
476
+ | 1.5691 | 690 | 0.1208 | - | - |
477
+ | 1.5918 | 700 | 0.1194 | - | - |
478
+ | 1.6146 | 710 | 0.1173 | - | - |
479
+ | 1.6374 | 720 | 0.1177 | - | - |
480
+ | 1.6601 | 730 | 0.1148 | - | - |
481
+ | 1.6829 | 740 | 0.1134 | - | - |
482
+ | 1.7056 | 750 | 0.1167 | 0.0422 | 0.8092 |
483
+ | 1.7284 | 760 | 0.1145 | - | - |
484
+ | 1.7512 | 770 | 0.114 | - | - |
485
+ | 1.7739 | 780 | 0.1136 | - | - |
486
+ | 1.7967 | 790 | 0.1123 | - | - |
487
+ | 1.8195 | 800 | 0.1115 | - | - |
488
+ | 1.8422 | 810 | 0.1127 | - | - |
489
+ | 1.8650 | 820 | 0.1137 | - | - |
490
+ | 1.8878 | 830 | 0.1137 | - | - |
491
+ | 1.9105 | 840 | 0.1123 | - | - |
492
+ | 1.9333 | 850 | 0.1115 | - | - |
493
+ | 1.9560 | 860 | 0.1105 | - | - |
494
+ | 1.9788 | 870 | 0.1133 | - | - |
495
+ | 2.0 | 880 | 0.1049 | - | - |
496
+ | 2.0228 | 890 | 0.1091 | - | - |
497
+ | 2.0455 | 900 | 0.111 | - | - |
498
+ | 2.0683 | 910 | 0.1101 | - | - |
499
+ | 2.0911 | 920 | 0.1078 | - | - |
500
+ | 2.1138 | 930 | 0.1097 | - | - |
501
+ | 2.1366 | 940 | 0.108 | - | - |
502
+ | 2.1593 | 950 | 0.1077 | - | - |
503
+ | 2.1821 | 960 | 0.1087 | - | - |
504
+ | 2.2049 | 970 | 0.1058 | - | - |
505
+ | 2.2276 | 980 | 0.1071 | - | - |
506
+ | 2.2504 | 990 | 0.1058 | - | - |
507
+ | 2.2732 | 1000 | 0.1104 | 0.0434 | 0.8156 |
508
+ | 2.2959 | 1010 | 0.1036 | - | - |
509
+ | 2.3187 | 1020 | 0.1068 | - | - |
510
+ | 2.3414 | 1030 | 0.1033 | - | - |
511
+ | 2.3642 | 1040 | 0.1058 | - | - |
512
+ | 2.3870 | 1050 | 0.105 | - | - |
513
+ | 2.4097 | 1060 | 0.1052 | - | - |
514
+ | 2.4325 | 1070 | 0.1013 | - | - |
515
+ | 2.4553 | 1080 | 0.1037 | - | - |
516
+ | 2.4780 | 1090 | 0.1031 | - | - |
517
+ | 2.5008 | 1100 | 0.1057 | - | - |
518
+ | 2.5235 | 1110 | 0.1051 | - | - |
519
+ | 2.5463 | 1120 | 0.1019 | - | - |
520
+ | 2.5691 | 1130 | 0.1018 | - | - |
521
+ | 2.5918 | 1140 | 0.1007 | - | - |
522
+ | 2.6146 | 1150 | 0.1035 | - | - |
523
+ | 2.6374 | 1160 | 0.1032 | - | - |
524
+ | 2.6601 | 1170 | 0.1036 | - | - |
525
+ | 2.6829 | 1180 | 0.0971 | - | - |
526
+ | 2.7056 | 1190 | 0.1015 | - | - |
527
+ | 2.7284 | 1200 | 0.104 | - | - |
528
+ | 2.7512 | 1210 | 0.1007 | - | - |
529
+ | 2.7739 | 1220 | 0.102 | - | - |
530
+ | 2.7967 | 1230 | 0.0994 | - | - |
531
+ | 2.8195 | 1240 | 0.0972 | - | - |
532
+ | 2.8422 | 1250 | 0.0969 | 0.0437 | 0.8185 |
533
+ | 2.8650 | 1260 | 0.0968 | - | - |
534
+ | 2.8878 | 1270 | 0.1003 | - | - |
535
+ | 2.9105 | 1280 | 0.1036 | - | - |
536
+ | 2.9333 | 1290 | 0.0969 | - | - |
537
+ | 2.9560 | 1300 | 0.0965 | - | - |
538
+ | 2.9788 | 1310 | 0.0974 | - | - |
539
+ | 3.0 | 1320 | 0.0905 | - | - |
540
+ | 3.0228 | 1330 | 0.1006 | - | - |
541
+ | 3.0455 | 1340 | 0.0952 | - | - |
542
+ | 3.0683 | 1350 | 0.0971 | - | - |
543
+ | 3.0911 | 1360 | 0.0943 | - | - |
544
+ | 3.1138 | 1370 | 0.0996 | - | - |
545
+ | 3.1366 | 1380 | 0.0971 | - | - |
546
+ | 3.1593 | 1390 | 0.097 | - | - |
547
+ | 3.1821 | 1400 | 0.0937 | - | - |
548
+ | 3.2049 | 1410 | 0.0955 | - | - |
549
+ | 3.2276 | 1420 | 0.0963 | - | - |
550
+ | 3.2504 | 1430 | 0.0938 | - | - |
551
+ | 3.2732 | 1440 | 0.0986 | - | - |
552
+ | 3.2959 | 1450 | 0.0949 | - | - |
553
+ | 3.3187 | 1460 | 0.0932 | - | - |
554
+ | 3.3414 | 1470 | 0.096 | - | - |
555
+ | 3.3642 | 1480 | 0.0919 | - | - |
556
+ | 3.3870 | 1490 | 0.093 | - | - |
557
+ | 3.4097 | 1500 | 0.0925 | 0.0438 | 0.8201 |
558
+ | 3.4325 | 1510 | 0.0935 | - | - |
559
+ | 3.4553 | 1520 | 0.0928 | - | - |
560
+ | 3.4780 | 1530 | 0.0914 | - | - |
561
+ | 3.5008 | 1540 | 0.0912 | - | - |
562
+ | 3.5235 | 1550 | 0.091 | - | - |
563
+ | 3.5463 | 1560 | 0.0906 | - | - |
564
+ | 3.5691 | 1570 | 0.0936 | - | - |
565
+ | 3.5918 | 1580 | 0.0943 | - | - |
566
+ | 3.6146 | 1590 | 0.0925 | - | - |
567
+ | 3.6374 | 1600 | 0.0908 | - | - |
568
+ | 3.6601 | 1610 | 0.0933 | - | - |
569
+ | 3.6829 | 1620 | 0.0917 | - | - |
570
+ | 3.7056 | 1630 | 0.0887 | - | - |
571
+ | 3.7284 | 1640 | 0.0903 | - | - |
572
+ | 3.7512 | 1650 | 0.0934 | - | - |
573
+ | 3.7739 | 1660 | 0.0906 | - | - |
574
+ | 3.7967 | 1670 | 0.0886 | - | - |
575
+ | 3.8195 | 1680 | 0.0915 | - | - |
576
+ | 3.8422 | 1690 | 0.0924 | - | - |
577
+ | 3.8650 | 1700 | 0.094 | - | - |
578
+ | 3.8878 | 1710 | 0.0899 | - | - |
579
+ | 3.9105 | 1720 | 0.0881 | - | - |
580
+ | 3.9333 | 1730 | 0.0884 | - | - |
581
+ | 3.9560 | 1740 | 0.0894 | - | - |
582
+ | 3.9788 | 1750 | 0.0892 | 0.0441 | 0.8215 |
583
+ | 4.0 | 1760 | 0.0812 | - | - |
584
+ | 4.0228 | 1770 | 0.0878 | - | - |
585
+ | 4.0455 | 1780 | 0.0869 | - | - |
586
+ | 4.0683 | 1790 | 0.09 | - | - |
587
+ | 4.0911 | 1800 | 0.0875 | - | - |
588
+ | 4.1138 | 1810 | 0.086 | - | - |
589
+ | 4.1366 | 1820 | 0.0888 | - | - |
590
+ | 4.1593 | 1830 | 0.086 | - | - |
591
+ | 4.1821 | 1840 | 0.0869 | - | - |
592
+ | 4.2049 | 1850 | 0.0885 | - | - |
593
+ | 4.2276 | 1860 | 0.0891 | - | - |
594
+ | 4.2504 | 1870 | 0.0853 | - | - |
595
+ | 4.2732 | 1880 | 0.0849 | - | - |
596
+ | 4.2959 | 1890 | 0.0856 | - | - |
597
+ | 4.3187 | 1900 | 0.0863 | - | - |
598
+ | 4.3414 | 1910 | 0.0849 | - | - |
599
+ | 4.3642 | 1920 | 0.0855 | - | - |
600
+ | 4.3870 | 1930 | 0.0841 | - | - |
601
+ | 4.4097 | 1940 | 0.0893 | - | - |
602
+ | 4.4325 | 1950 | 0.0847 | - | - |
603
+ | 4.4553 | 1960 | 0.0866 | - | - |
604
+ | 4.4780 | 1970 | 0.0866 | - | - |
605
+ | 4.5008 | 1980 | 0.0844 | - | - |
606
+ | 4.5235 | 1990 | 0.0846 | - | - |
607
+ | 4.5463 | 2000 | 0.0847 | 0.0435 | 0.8220 |
608
+ | 4.5691 | 2010 | 0.0831 | - | - |
609
+ | 4.5918 | 2020 | 0.0843 | - | - |
610
+ | 4.6146 | 2030 | 0.086 | - | - |
611
+ | 4.6374 | 2040 | 0.0851 | - | - |
612
+ | 4.6601 | 2050 | 0.0844 | - | - |
613
+ | 4.6829 | 2060 | 0.0843 | - | - |
614
+ | 4.7056 | 2070 | 0.0854 | - | - |
615
+ | 4.7284 | 2080 | 0.0851 | - | - |
616
+ | 4.7512 | 2090 | 0.0822 | - | - |
617
+ | 4.7739 | 2100 | 0.0859 | - | - |
618
+ | 4.7967 | 2110 | 0.0844 | - | - |
619
+ | 4.8195 | 2120 | 0.0853 | - | - |
620
+ | 4.8422 | 2130 | 0.0815 | - | - |
621
+ | 4.8650 | 2140 | 0.0833 | - | - |
622
+ | 4.8878 | 2150 | 0.0817 | - | - |
623
+ | 4.9105 | 2160 | 0.0873 | - | - |
624
+ | 4.9333 | 2170 | 0.0813 | - | - |
625
+ | 4.9560 | 2180 | 0.0829 | - | - |
626
+ | 4.9788 | 2190 | 0.0812 | - | - |
627
+ | 5.0 | 2200 | 0.0776 | - | - |
628
+ | 5.0228 | 2210 | 0.083 | - | - |
629
+ | 5.0455 | 2220 | 0.0821 | - | - |
630
+ | 5.0683 | 2230 | 0.0806 | - | - |
631
+ | 5.0911 | 2240 | 0.0809 | - | - |
632
+ | 5.1138 | 2250 | 0.0814 | 0.0431 | 0.8225 |
633
+ | 5.1366 | 2260 | 0.0808 | - | - |
634
+ | 5.1593 | 2270 | 0.0791 | - | - |
635
+ | 5.1821 | 2280 | 0.0811 | - | - |
636
+ | 5.2049 | 2290 | 0.0805 | - | - |
637
+ | 5.2276 | 2300 | 0.0817 | - | - |
638
+ | 5.2504 | 2310 | 0.0772 | - | - |
639
+ | 5.2732 | 2320 | 0.0799 | - | - |
640
+ | 5.2959 | 2330 | 0.0829 | - | - |
641
+ | 5.3187 | 2340 | 0.077 | - | - |
642
+ | 5.3414 | 2350 | 0.0801 | - | - |
643
+ | 5.3642 | 2360 | 0.0812 | - | - |
644
+ | 5.3870 | 2370 | 0.0788 | - | - |
645
+ | 5.4097 | 2380 | 0.0776 | - | - |
646
+ | 5.4325 | 2390 | 0.0785 | - | - |
647
+ | 5.4553 | 2400 | 0.0771 | - | - |
648
+ | 5.4780 | 2410 | 0.0788 | - | - |
649
+ | 5.5008 | 2420 | 0.0796 | - | - |
650
+ | 5.5235 | 2430 | 0.0793 | - | - |
651
+ | 5.5463 | 2440 | 0.0813 | - | - |
652
+ | 5.5691 | 2450 | 0.0757 | - | - |
653
+ | 5.5918 | 2460 | 0.079 | - | - |
654
+ | 5.6146 | 2470 | 0.0797 | - | - |
655
+ | 5.6374 | 2480 | 0.0794 | - | - |
656
+ | 5.6601 | 2490 | 0.0808 | - | - |
657
+ | 5.6829 | 2500 | 0.0796 | 0.0424 | 0.8230 |
658
+ | 5.7056 | 2510 | 0.0802 | - | - |
659
+ | 5.7284 | 2520 | 0.0799 | - | - |
660
+ | 5.7512 | 2530 | 0.0802 | - | - |
661
+ | 5.7739 | 2540 | 0.0813 | - | - |
662
+ | 5.7967 | 2550 | 0.0772 | - | - |
663
+ | 5.8195 | 2560 | 0.0766 | - | - |
664
+ | 5.8422 | 2570 | 0.0778 | - | - |
665
+ | 5.8650 | 2580 | 0.076 | - | - |
666
+ | 5.8878 | 2590 | 0.0787 | - | - |
667
+ | 5.9105 | 2600 | 0.0794 | - | - |
668
+ | 5.9333 | 2610 | 0.076 | - | - |
669
+ | 5.9560 | 2620 | 0.0773 | - | - |
670
+ | 5.9788 | 2630 | 0.0755 | - | - |
671
+ | 6.0 | 2640 | 0.0725 | - | - |
672
+ | 6.0228 | 2650 | 0.0738 | - | - |
673
+ | 6.0455 | 2660 | 0.0762 | - | - |
674
+ | 6.0683 | 2670 | 0.0761 | - | - |
675
+ | 6.0911 | 2680 | 0.0771 | - | - |
676
+ | 6.1138 | 2690 | 0.0765 | - | - |
677
+ | 6.1366 | 2700 | 0.0755 | - | - |
678
+ | 6.1593 | 2710 | 0.0771 | - | - |
679
+ | 6.1821 | 2720 | 0.0748 | - | - |
680
+ | 6.2049 | 2730 | 0.0768 | - | - |
681
+ | 6.2276 | 2740 | 0.0766 | - | - |
682
+ | 6.2504 | 2750 | 0.0766 | 0.0422 | 0.8239 |
683
+ | 6.2732 | 2760 | 0.076 | - | - |
684
+ | 6.2959 | 2770 | 0.0753 | - | - |
685
+ | 6.3187 | 2780 | 0.0735 | - | - |
686
+ | 6.3414 | 2790 | 0.0751 | - | - |
687
+ | 6.3642 | 2800 | 0.0738 | - | - |
688
+ | 6.3870 | 2810 | 0.0749 | - | - |
689
+ | 6.4097 | 2820 | 0.0753 | - | - |
690
+ | 6.4325 | 2830 | 0.077 | - | - |
691
+ | 6.4553 | 2840 | 0.0747 | - | - |
692
+ | 6.4780 | 2850 | 0.0722 | - | - |
693
+ | 6.5008 | 2860 | 0.0736 | - | - |
694
+ | 6.5235 | 2870 | 0.073 | - | - |
695
+ | 6.5463 | 2880 | 0.0774 | - | - |
696
+ | 6.5691 | 2890 | 0.075 | - | - |
697
+ | 6.5918 | 2900 | 0.0718 | - | - |
698
+ | 6.6146 | 2910 | 0.0727 | - | - |
699
+ | 6.6374 | 2920 | 0.0735 | - | - |
700
+ | 6.6601 | 2930 | 0.0726 | - | - |
701
+ | 6.6829 | 2940 | 0.075 | - | - |
702
+ | 6.7056 | 2950 | 0.0728 | - | - |
703
+ | 6.7284 | 2960 | 0.0713 | - | - |
704
+ | 6.7512 | 2970 | 0.0722 | - | - |
705
+ | 6.7739 | 2980 | 0.0753 | - | - |
706
+ | 6.7967 | 2990 | 0.0733 | - | - |
707
+ | 6.8195 | 3000 | 0.0727 | 0.0425 | 0.8243 |
708
+ | 6.8422 | 3010 | 0.0729 | - | - |
709
+ | 6.8650 | 3020 | 0.073 | - | - |
710
+ | 6.8878 | 3030 | 0.0739 | - | - |
711
+ | 6.9105 | 3040 | 0.0717 | - | - |
712
+ | 6.9333 | 3050 | 0.0719 | - | - |
713
+ | 6.9560 | 3060 | 0.0712 | - | - |
714
+ | 6.9788 | 3070 | 0.0712 | - | - |
715
+ | 7.0 | 3080 | 0.0674 | - | - |
716
+ | 7.0228 | 3090 | 0.0729 | - | - |
717
+ | 7.0455 | 3100 | 0.0712 | - | - |
718
+ | 7.0683 | 3110 | 0.0701 | - | - |
719
+ | 7.0911 | 3120 | 0.0699 | - | - |
720
+ | 7.1138 | 3130 | 0.0675 | - | - |
721
+ | 7.1366 | 3140 | 0.0699 | - | - |
722
+ | 7.1593 | 3150 | 0.0716 | - | - |
723
+ | 7.1821 | 3160 | 0.0707 | - | - |
724
+ | 7.2049 | 3170 | 0.0717 | - | - |
725
+ | 7.2276 | 3180 | 0.0709 | - | - |
726
+ | 7.2504 | 3190 | 0.071 | - | - |
727
+ | 7.2732 | 3200 | 0.0722 | - | - |
728
+ | 7.2959 | 3210 | 0.072 | - | - |
729
+ | 7.3187 | 3220 | 0.0729 | - | - |
730
+ | 7.3414 | 3230 | 0.0678 | - | - |
731
+ | 7.3642 | 3240 | 0.0705 | - | - |
732
+ | 7.3870 | 3250 | 0.0715 | 0.0426 | 0.8256 |
733
+ | 7.4097 | 3260 | 0.0703 | - | - |
734
+ | 7.4325 | 3270 | 0.0699 | - | - |
735
+ | 7.4553 | 3280 | 0.071 | - | - |
736
+ | 7.4780 | 3290 | 0.0692 | - | - |
737
+ | 7.5008 | 3300 | 0.0693 | - | - |
738
+ | 7.5235 | 3310 | 0.0661 | - | - |
739
+ | 7.5463 | 3320 | 0.0702 | - | - |
740
+ | 7.5691 | 3330 | 0.0697 | - | - |
741
+ | 7.5918 | 3340 | 0.072 | - | - |
742
+ | 7.6146 | 3350 | 0.0693 | - | - |
743
+ | 7.6374 | 3360 | 0.0691 | - | - |
744
+ | 7.6601 | 3370 | 0.0702 | - | - |
745
+ | 7.6829 | 3380 | 0.0672 | - | - |
746
+ | 7.7056 | 3390 | 0.0698 | - | - |
747
+ | 7.7284 | 3400 | 0.0687 | - | - |
748
+ | 7.7512 | 3410 | 0.0654 | - | - |
749
+ | 7.7739 | 3420 | 0.0687 | - | - |
750
+ | 7.7967 | 3430 | 0.0679 | - | - |
751
+ | 7.8195 | 3440 | 0.0713 | - | - |
752
+ | 7.8422 | 3450 | 0.0676 | - | - |
753
+ | 7.8650 | 3460 | 0.0708 | - | - |
754
+ | 7.8878 | 3470 | 0.0666 | - | - |
755
+ | 7.9105 | 3480 | 0.0675 | - | - |
756
+ | 7.9333 | 3490 | 0.0693 | - | - |
757
+ | 7.9560 | 3500 | 0.0688 | 0.0427 | 0.8260 |
758
+ | 7.9788 | 3510 | 0.068 | - | - |
759
+ | 8.0 | 3520 | 0.063 | - | - |
760
+ | 8.0228 | 3530 | 0.0659 | - | - |
761
+ | 8.0455 | 3540 | 0.0639 | - | - |
762
+ | 8.0683 | 3550 | 0.0678 | - | - |
763
+ | 8.0911 | 3560 | 0.0689 | - | - |
764
+ | 8.1138 | 3570 | 0.0687 | - | - |
765
+ | 8.1366 | 3580 | 0.0672 | - | - |
766
+ | 8.1593 | 3590 | 0.0659 | - | - |
767
+ | 8.1821 | 3600 | 0.0658 | - | - |
768
+ | 8.2049 | 3610 | 0.0664 | - | - |
769
+ | 8.2276 | 3620 | 0.0659 | - | - |
770
+ | 8.2504 | 3630 | 0.0664 | - | - |
771
+ | 8.2732 | 3640 | 0.0652 | - | - |
772
+ | 8.2959 | 3650 | 0.0683 | - | - |
773
+ | 8.3187 | 3660 | 0.0641 | - | - |
774
+ | 8.3414 | 3670 | 0.0672 | - | - |
775
+ | 8.3642 | 3680 | 0.0655 | - | - |
776
+ | 8.3870 | 3690 | 0.0661 | - | - |
777
+ | 8.4097 | 3700 | 0.0638 | - | - |
778
+ | 8.4325 | 3710 | 0.0675 | - | - |
779
+ | 8.4553 | 3720 | 0.0648 | - | - |
780
+ | 8.4780 | 3730 | 0.067 | - | - |
781
+ | 8.5008 | 3740 | 0.0684 | - | - |
782
+ | 8.5235 | 3750 | 0.0667 | 0.0420 | 0.8268 |
783
+ | 8.5463 | 3760 | 0.0645 | - | - |
784
+ | 8.5691 | 3770 | 0.0652 | - | - |
785
+ | 8.5918 | 3780 | 0.0633 | - | - |
786
+ | 8.6146 | 3790 | 0.065 | - | - |
787
+ | 8.6374 | 3800 | 0.064 | - | - |
788
+ | 8.6601 | 3810 | 0.0677 | - | - |
789
+ | 8.6829 | 3820 | 0.0661 | - | - |
790
+ | 8.7056 | 3830 | 0.0653 | - | - |
791
+ | 8.7284 | 3840 | 0.0625 | - | - |
792
+ | 8.7512 | 3850 | 0.0651 | - | - |
793
+ | 8.7739 | 3860 | 0.0656 | - | - |
794
+ | 8.7967 | 3870 | 0.0636 | - | - |
795
+ | 8.8195 | 3880 | 0.0655 | - | - |
796
+ | 8.8422 | 3890 | 0.0647 | - | - |
797
+ | 8.8650 | 3900 | 0.0638 | - | - |
798
+ | 8.8878 | 3910 | 0.0636 | - | - |
799
+ | 8.9105 | 3920 | 0.0666 | - | - |
800
+ | 8.9333 | 3930 | 0.062 | - | - |
801
+ | 8.9560 | 3940 | 0.065 | - | - |
802
+ | 8.9788 | 3950 | 0.0643 | - | - |
803
+ | 9.0 | 3960 | 0.0594 | - | - |
804
+ | 9.0228 | 3970 | 0.0616 | - | - |
805
+ | 9.0455 | 3980 | 0.0638 | - | - |
806
+ | 9.0683 | 3990 | 0.0625 | - | - |
807
+ | 9.0911 | 4000 | 0.0665 | 0.0414 | 0.8276 |
808
+ | 9.1138 | 4010 | 0.0624 | - | - |
809
+ | 9.1366 | 4020 | 0.0621 | - | - |
810
+ | 9.1593 | 4030 | 0.0648 | - | - |
811
+ | 9.1821 | 4040 | 0.0622 | - | - |
812
+ | 9.2049 | 4050 | 0.0635 | - | - |
813
+ | 9.2276 | 4060 | 0.061 | - | - |
814
+ | 9.2504 | 4070 | 0.0602 | - | - |
815
+ | 9.2732 | 4080 | 0.0613 | - | - |
816
+ | 9.2959 | 4090 | 0.0604 | - | - |
817
+ | 9.3187 | 4100 | 0.0623 | - | - |
818
+ | 9.3414 | 4110 | 0.0641 | - | - |
819
+ | 9.3642 | 4120 | 0.0635 | - | - |
820
+ | 9.3870 | 4130 | 0.0608 | - | - |
821
+ | 9.4097 | 4140 | 0.0611 | - | - |
822
+ | 9.4325 | 4150 | 0.0607 | - | - |
823
+ | 9.4553 | 4160 | 0.0631 | - | - |
824
+ | 9.4780 | 4170 | 0.0618 | - | - |
825
+ | 9.5008 | 4180 | 0.0609 | - | - |
826
+ | 9.5235 | 4190 | 0.0613 | - | - |
827
+ | 9.5463 | 4200 | 0.0606 | - | - |
828
+ | 9.5691 | 4210 | 0.0595 | - | - |
829
+ | 9.5918 | 4220 | 0.0609 | - | - |
830
+ | 9.6146 | 4230 | 0.061 | - | - |
831
+ | 9.6374 | 4240 | 0.0616 | - | - |
832
+ | 9.6601 | 4250 | 0.0613 | 0.0418 | 0.8282 |
833
+ | 9.6829 | 4260 | 0.0623 | - | - |
834
+ | 9.7056 | 4270 | 0.0605 | - | - |
835
+ | 9.7284 | 4280 | 0.0637 | - | - |
836
+ | 9.7512 | 4290 | 0.0604 | - | - |
837
+ | 9.7739 | 4300 | 0.0606 | - | - |
838
+ | 9.7967 | 4310 | 0.0622 | - | - |
839
+ | 9.8195 | 4320 | 0.0598 | - | - |
840
+ | 9.8422 | 4330 | 0.0611 | - | - |
841
+ | 9.8650 | 4340 | 0.0604 | - | - |
842
+ | 9.8878 | 4350 | 0.0598 | - | - |
843
+ | 9.9105 | 4360 | 0.0626 | - | - |
844
+ | 9.9333 | 4370 | 0.0624 | - | - |
845
+ | 9.9560 | 4380 | 0.0617 | - | - |
846
+ | 9.9788 | 4390 | 0.0603 | - | - |
847
+
848
+ </details>
849
+
850
+ ### Framework Versions
851
+ - Python: 3.11.10
852
+ - Sentence Transformers: 3.4.1
853
+ - Transformers: 4.48.3
854
+ - PyTorch: 2.5.1+cu124
855
+ - Accelerate: 1.3.0
856
+ - Datasets: 3.3.0
857
+ - Tokenizers: 0.21.0
858
+
859
+ ## Citation
860
+
861
+ ### BibTeX
862
+
863
+ #### Sentence Transformers
864
+ ```bibtex
865
+ @inproceedings{reimers-2019-sentence-bert,
866
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
867
+ author = "Reimers, Nils and Gurevych, Iryna",
868
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
869
+ month = "11",
870
+ year = "2019",
871
+ publisher = "Association for Computational Linguistics",
872
+ url = "https://arxiv.org/abs/1908.10084",
873
+ }
874
+ ```
875
+
876
+ <!--
877
+ ## Glossary
878
+
879
+ *Clearly define terms in order to be accessible across audiences.*
880
+ -->
881
+
882
+ <!--
883
+ ## Model Card Authors
884
+
885
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
886
+ -->
887
+
888
+ <!--
889
+ ## Model Card Contact
890
+
891
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
892
+ -->
config_sentence_transformers.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "__version__": {
3
+ "sentence_transformers": "3.4.1",
4
+ "transformers": "4.48.3",
5
+ "pytorch": "2.5.1+cu124"
6
+ },
7
+ "prompts": {},
8
+ "default_prompt_name": null,
9
+ "similarity_fn_name": "cosine"
10
+ }
modules.json ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.models.Transformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_Pooling",
12
+ "type": "sentence_transformers.models.Pooling"
13
+ },
14
+ {
15
+ "idx": 2,
16
+ "name": "2",
17
+ "path": "2_Dense",
18
+ "type": "sentence_transformers.models.Dense"
19
+ }
20
+ ]
sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 2048,
3
+ "do_lower_case": false
4
+ }