pj-mathematician commited on
Commit
0e9c103
·
verified ·
1 Parent(s): 88cb515

Upload folder using huggingface_hub

Browse files
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
1_Pooling/config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "word_embedding_dimension": 3584,
3
+ "pooling_mode_cls_token": false,
4
+ "pooling_mode_mean_tokens": false,
5
+ "pooling_mode_max_tokens": false,
6
+ "pooling_mode_mean_sqrt_len_tokens": false,
7
+ "pooling_mode_weightedmean_tokens": false,
8
+ "pooling_mode_lasttoken": true,
9
+ "include_prompt": true
10
+ }
README.md CHANGED
@@ -1,3 +1,529 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - sentence-transformers
4
+ - sentence-similarity
5
+ - feature-extraction
6
+ - generated_from_trainer
7
+ - dataset_size:124788
8
+ - loss:CachedGISTEmbedLoss
9
+ base_model: Alibaba-NLP/gte-Qwen2-7B-instruct
10
+ widget:
11
+ - source_sentence: 其他机械、设备和有形货物租赁服务代表
12
+ sentences:
13
+ - 其他机械和设备租赁服务工作人员
14
+ - 电子和电信设备及零部件物流经理
15
+ - 工业主厨
16
+ - source_sentence: 公交车司机
17
+ sentences:
18
+ - 表演灯光设计师
19
+ - 乙烯基地板安装工
20
+ - 国际巴士司机
21
+ - source_sentence: online communication manager
22
+ sentences:
23
+ - trades union official
24
+ - social media manager
25
+ - budget manager
26
+ - source_sentence: Projektmanagerin
27
+ sentences:
28
+ - Projektmanager/Projektmanagerin
29
+ - Category-Manager
30
+ - Infanterist
31
+ - source_sentence: Volksvertreter
32
+ sentences:
33
+ - Parlamentarier
34
+ - Oberbürgermeister
35
+ - Konsul
36
+ pipeline_tag: sentence-similarity
37
+ library_name: sentence-transformers
38
+ ---
39
+
40
+ # Job - Job matching finetuned Alibaba-NLP/gte-Qwen2-7B-instruct
41
+
42
+ Best performing model on [TalentCLEF 2025](https://talentclef.github.io/talentclef/) Task A. Use it for job title matching
43
+
44
+ ## Model Details
45
+
46
+ ### Model Description
47
+ - **Model Type:** Sentence Transformer
48
+ - **Base model:** [Alibaba-NLP/gte-Qwen2-7B-instruct](https://huggingface.co/Alibaba-NLP/gte-Qwen2-7B-instruct) <!-- at revision a8d08b36ada9cacfe34c4d6f80957772a025daf2 -->
49
+ - **Maximum Sequence Length:** 512 tokens
50
+ - **Output Dimensionality:** 3584 dimensions
51
+ - **Similarity Function:** Cosine Similarity
52
+ - **Training Datasets:**
53
+ - full_en
54
+ - full_de
55
+ - full_es
56
+ - full_zh
57
+ - mix
58
+ <!-- - **Language:** Unknown -->
59
+ <!-- - **License:** Unknown -->
60
+
61
+ ### Model Sources
62
+
63
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
64
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
65
+ - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
66
+
67
+ ### Full Model Architecture
68
+
69
+ ```
70
+ SentenceTransformer(
71
+ (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: Qwen2Model
72
+ (1): Pooling({'word_embedding_dimension': 3584, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
73
+ (2): Normalize()
74
+ )
75
+ ```
76
+
77
+ ## Usage
78
+
79
+ ### Direct Usage (Sentence Transformers)
80
+
81
+ First install the Sentence Transformers library:
82
+
83
+ ```bash
84
+ pip install -U sentence-transformers
85
+ ```
86
+
87
+ Then you can load this model and run inference.
88
+ ```python
89
+ from sentence_transformers import SentenceTransformer
90
+
91
+ # Download from the 🤗 Hub
92
+ model = SentenceTransformer("pj-mathematician/JobGTE-7b-Lora")
93
+ # Run inference
94
+ sentences = [
95
+ 'Volksvertreter',
96
+ 'Parlamentarier',
97
+ 'Oberbürgermeister',
98
+ ]
99
+ embeddings = model.encode(sentences)
100
+ print(embeddings.shape)
101
+ # [3, 3584]
102
+
103
+ # Get the similarity scores for the embeddings
104
+ similarities = model.similarity(embeddings, embeddings)
105
+ print(similarities.shape)
106
+ # [3, 3]
107
+ ```
108
+
109
+ <!--
110
+ ### Direct Usage (Transformers)
111
+
112
+ <details><summary>Click to see the direct usage in Transformers</summary>
113
+
114
+ </details>
115
+ -->
116
+
117
+ <!--
118
+ ### Downstream Usage (Sentence Transformers)
119
+
120
+ You can finetune this model on your own dataset.
121
+
122
+ <details><summary>Click to expand</summary>
123
+
124
+ </details>
125
+ -->
126
+
127
+ <!--
128
+ ### Out-of-Scope Use
129
+
130
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
131
+ -->
132
+
133
+ <!--
134
+ ## Bias, Risks and Limitations
135
+
136
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
137
+ -->
138
+
139
+ <!--
140
+ ### Recommendations
141
+
142
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
143
+ -->
144
+
145
+ ## Training Details
146
+
147
+ ### Training Datasets
148
+ <details><summary>full_en</summary>
149
+
150
+ #### full_en
151
+
152
+ * Dataset: full_en
153
+ * Size: 28,880 training samples
154
+ * Columns: <code>anchor</code> and <code>positive</code>
155
+ * Approximate statistics based on the first 1000 samples:
156
+ | | anchor | positive |
157
+ |:--------|:-------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|
158
+ | type | string | string |
159
+ | details | <ul><li>min: 2 tokens</li><li>mean: 4.4 tokens</li><li>max: 9 tokens</li></ul> | <ul><li>min: 2 tokens</li><li>mean: 4.42 tokens</li><li>max: 10 tokens</li></ul> |
160
+ * Samples:
161
+ | anchor | positive |
162
+ |:-----------------------------------------|:-----------------------------------------|
163
+ | <code>air commodore</code> | <code>flight lieutenant</code> |
164
+ | <code>command and control officer</code> | <code>flight officer</code> |
165
+ | <code>air commodore</code> | <code>command and control officer</code> |
166
+ * Loss: [<code>CachedGISTEmbedLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cachedgistembedloss) with these parameters:
167
+ ```json
168
+ {'guide': SentenceTransformer(
169
+ (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel
170
+ (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
171
+ (2): Normalize()
172
+ ), 'temperature': 0.01, 'mini_batch_size': 64, 'margin_strategy': 'absolute', 'margin': 0.0}
173
+ ```
174
+ </details>
175
+ <details><summary>full_de</summary>
176
+
177
+ #### full_de
178
+
179
+ * Dataset: full_de
180
+ * Size: 23,023 training samples
181
+ * Columns: <code>anchor</code> and <code>positive</code>
182
+ * Approximate statistics based on the first 1000 samples:
183
+ | | anchor | positive |
184
+ |:--------|:---------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|
185
+ | type | string | string |
186
+ | details | <ul><li>min: 2 tokens</li><li>mean: 9.11 tokens</li><li>max: 33 tokens</li></ul> | <ul><li>min: 2 tokens</li><li>mean: 9.41 tokens</li><li>max: 33 tokens</li></ul> |
187
+ * Samples:
188
+ | anchor | positive |
189
+ |:----------------------------------|:-----------------------------------------------------|
190
+ | <code>Staffelkommandantin</code> | <code>Kommodore</code> |
191
+ | <code>Luftwaffenoffizierin</code> | <code>Luftwaffenoffizier/Luftwaffenoffizierin</code> |
192
+ | <code>Staffelkommandantin</code> | <code>Luftwaffenoffizierin</code> |
193
+ * Loss: [<code>CachedGISTEmbedLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cachedgistembedloss) with these parameters:
194
+ ```json
195
+ {'guide': SentenceTransformer(
196
+ (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel
197
+ (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
198
+ (2): Normalize()
199
+ ), 'temperature': 0.01, 'mini_batch_size': 64, 'margin_strategy': 'absolute', 'margin': 0.0}
200
+ ```
201
+ </details>
202
+ <details><summary>full_es</summary>
203
+
204
+ #### full_es
205
+
206
+ * Dataset: full_es
207
+ * Size: 20,724 training samples
208
+ * Columns: <code>anchor</code> and <code>positive</code>
209
+ * Approximate statistics based on the first 1000 samples:
210
+ | | anchor | positive |
211
+ |:--------|:---------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|
212
+ | type | string | string |
213
+ | details | <ul><li>min: 3 tokens</li><li>mean: 9.42 tokens</li><li>max: 35 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 9.18 tokens</li><li>max: 35 tokens</li></ul> |
214
+ * Samples:
215
+ | anchor | positive |
216
+ |:------------------------------------|:-------------------------------------------|
217
+ | <code>jefe de escuadrón</code> | <code>instructor</code> |
218
+ | <code>comandante de aeronave</code> | <code>instructor de simulador</code> |
219
+ | <code>instructor</code> | <code>oficial del Ejército del Aire</code> |
220
+ * Loss: [<code>CachedGISTEmbedLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cachedgistembedloss) with these parameters:
221
+ ```json
222
+ {'guide': SentenceTransformer(
223
+ (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel
224
+ (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
225
+ (2): Normalize()
226
+ ), 'temperature': 0.01, 'mini_batch_size': 64, 'margin_strategy': 'absolute', 'margin': 0.0}
227
+ ```
228
+ </details>
229
+ <details><summary>full_zh</summary>
230
+
231
+ #### full_zh
232
+
233
+ * Dataset: full_zh
234
+ * Size: 30,401 training samples
235
+ * Columns: <code>anchor</code> and <code>positive</code>
236
+ * Approximate statistics based on the first 1000 samples:
237
+ | | anchor | positive |
238
+ |:--------|:--------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|
239
+ | type | string | string |
240
+ | details | <ul><li>min: 3 tokens</li><li>mean: 4.7 tokens</li><li>max: 12 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 5.04 tokens</li><li>max: 19 tokens</li></ul> |
241
+ * Samples:
242
+ | anchor | positive |
243
+ |:------------------|:---------------------|
244
+ | <code>技术总监</code> | <code>技术和运营总监</code> |
245
+ | <code>技术总监</code> | <code>技术主管</code> |
246
+ | <code>技术总监</code> | <code>技术艺术总监</code> |
247
+ * Loss: [<code>CachedGISTEmbedLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cachedgistembedloss) with these parameters:
248
+ ```json
249
+ {'guide': SentenceTransformer(
250
+ (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel
251
+ (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
252
+ (2): Normalize()
253
+ ), 'temperature': 0.01, 'mini_batch_size': 64, 'margin_strategy': 'absolute', 'margin': 0.0}
254
+ ```
255
+ </details>
256
+ <details><summary>mix</summary>
257
+
258
+ #### mix
259
+
260
+ * Dataset: mix
261
+ * Size: 21,760 training samples
262
+ * Columns: <code>anchor</code> and <code>positive</code>
263
+ * Approximate statistics based on the first 1000 samples:
264
+ | | anchor | positive |
265
+ |:--------|:---------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|
266
+ | type | string | string |
267
+ | details | <ul><li>min: 1 tokens</li><li>mean: 4.98 tokens</li><li>max: 14 tokens</li></ul> | <ul><li>min: 1 tokens</li><li>mean: 7.22 tokens</li><li>max: 27 tokens</li></ul> |
268
+ * Samples:
269
+ | anchor | positive |
270
+ |:------------------------------------------|:----------------------------------------------------------------|
271
+ | <code>technical manager</code> | <code>Technischer Direktor für Bühne, Film und Fernsehen</code> |
272
+ | <code>head of technical</code> | <code>directora técnica</code> |
273
+ | <code>head of technical department</code> | <code>技术艺术总监</code> |
274
+ * Loss: [<code>CachedGISTEmbedLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cachedgistembedloss) with these parameters:
275
+ ```json
276
+ {'guide': SentenceTransformer(
277
+ (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel
278
+ (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
279
+ (2): Normalize()
280
+ ), 'temperature': 0.01, 'mini_batch_size': 64, 'margin_strategy': 'absolute', 'margin': 0.0}
281
+ ```
282
+ </details>
283
+
284
+ ### Training Hyperparameters
285
+ #### Non-Default Hyperparameters
286
+
287
+ - `per_device_train_batch_size`: 128
288
+ - `per_device_eval_batch_size`: 128
289
+ - `gradient_accumulation_steps`: 2
290
+ - `num_train_epochs`: 2
291
+ - `warmup_ratio`: 0.05
292
+ - `log_on_each_node`: False
293
+ - `fp16`: True
294
+ - `dataloader_num_workers`: 4
295
+ - `fsdp`: ['full_shard', 'auto_wrap']
296
+ - `fsdp_config`: {'transformer_layer_cls_to_wrap': ['Qwen2DecoderLayer'], 'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
297
+ - `ddp_find_unused_parameters`: True
298
+ - `gradient_checkpointing`: True
299
+ - `batch_sampler`: no_duplicates
300
+
301
+ #### All Hyperparameters
302
+ <details><summary>Click to expand</summary>
303
+
304
+ - `overwrite_output_dir`: False
305
+ - `do_predict`: False
306
+ - `eval_strategy`: no
307
+ - `prediction_loss_only`: True
308
+ - `per_device_train_batch_size`: 128
309
+ - `per_device_eval_batch_size`: 128
310
+ - `per_gpu_train_batch_size`: None
311
+ - `per_gpu_eval_batch_size`: None
312
+ - `gradient_accumulation_steps`: 2
313
+ - `eval_accumulation_steps`: None
314
+ - `torch_empty_cache_steps`: None
315
+ - `learning_rate`: 5e-05
316
+ - `weight_decay`: 0.0
317
+ - `adam_beta1`: 0.9
318
+ - `adam_beta2`: 0.999
319
+ - `adam_epsilon`: 1e-08
320
+ - `max_grad_norm`: 1.0
321
+ - `num_train_epochs`: 2
322
+ - `max_steps`: -1
323
+ - `lr_scheduler_type`: linear
324
+ - `lr_scheduler_kwargs`: {}
325
+ - `warmup_ratio`: 0.05
326
+ - `warmup_steps`: 0
327
+ - `log_level`: passive
328
+ - `log_level_replica`: warning
329
+ - `log_on_each_node`: False
330
+ - `logging_nan_inf_filter`: True
331
+ - `save_safetensors`: True
332
+ - `save_on_each_node`: False
333
+ - `save_only_model`: False
334
+ - `restore_callback_states_from_checkpoint`: False
335
+ - `no_cuda`: False
336
+ - `use_cpu`: False
337
+ - `use_mps_device`: False
338
+ - `seed`: 42
339
+ - `data_seed`: None
340
+ - `jit_mode_eval`: False
341
+ - `use_ipex`: False
342
+ - `bf16`: False
343
+ - `fp16`: True
344
+ - `fp16_opt_level`: O1
345
+ - `half_precision_backend`: auto
346
+ - `bf16_full_eval`: False
347
+ - `fp16_full_eval`: False
348
+ - `tf32`: None
349
+ - `local_rank`: 0
350
+ - `ddp_backend`: None
351
+ - `tpu_num_cores`: None
352
+ - `tpu_metrics_debug`: False
353
+ - `debug`: []
354
+ - `dataloader_drop_last`: True
355
+ - `dataloader_num_workers`: 4
356
+ - `dataloader_prefetch_factor`: None
357
+ - `past_index`: -1
358
+ - `disable_tqdm`: False
359
+ - `remove_unused_columns`: True
360
+ - `label_names`: None
361
+ - `load_best_model_at_end`: False
362
+ - `ignore_data_skip`: False
363
+ - `fsdp`: ['full_shard', 'auto_wrap']
364
+ - `fsdp_min_num_params`: 0
365
+ - `fsdp_config`: {'transformer_layer_cls_to_wrap': ['Qwen2DecoderLayer'], 'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
366
+ - `tp_size`: 0
367
+ - `fsdp_transformer_layer_cls_to_wrap`: None
368
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
369
+ - `deepspeed`: None
370
+ - `label_smoothing_factor`: 0.0
371
+ - `optim`: adamw_torch
372
+ - `optim_args`: None
373
+ - `adafactor`: False
374
+ - `group_by_length`: False
375
+ - `length_column_name`: length
376
+ - `ddp_find_unused_parameters`: True
377
+ - `ddp_bucket_cap_mb`: None
378
+ - `ddp_broadcast_buffers`: False
379
+ - `dataloader_pin_memory`: True
380
+ - `dataloader_persistent_workers`: False
381
+ - `skip_memory_metrics`: True
382
+ - `use_legacy_prediction_loop`: False
383
+ - `push_to_hub`: False
384
+ - `resume_from_checkpoint`: None
385
+ - `hub_model_id`: None
386
+ - `hub_strategy`: every_save
387
+ - `hub_private_repo`: None
388
+ - `hub_always_push`: False
389
+ - `gradient_checkpointing`: True
390
+ - `gradient_checkpointing_kwargs`: None
391
+ - `include_inputs_for_metrics`: False
392
+ - `include_for_metrics`: []
393
+ - `eval_do_concat_batches`: True
394
+ - `fp16_backend`: auto
395
+ - `push_to_hub_model_id`: None
396
+ - `push_to_hub_organization`: None
397
+ - `mp_parameters`:
398
+ - `auto_find_batch_size`: False
399
+ - `full_determinism`: False
400
+ - `torchdynamo`: None
401
+ - `ray_scope`: last
402
+ - `ddp_timeout`: 1800
403
+ - `torch_compile`: False
404
+ - `torch_compile_backend`: None
405
+ - `torch_compile_mode`: None
406
+ - `include_tokens_per_second`: False
407
+ - `include_num_input_tokens_seen`: False
408
+ - `neftune_noise_alpha`: None
409
+ - `optim_target_modules`: None
410
+ - `batch_eval_metrics`: False
411
+ - `eval_on_start`: False
412
+ - `use_liger_kernel`: False
413
+ - `eval_use_gather_object`: False
414
+ - `average_tokens_across_devices`: False
415
+ - `prompts`: None
416
+ - `batch_sampler`: no_duplicates
417
+ - `multi_dataset_batch_sampler`: proportional
418
+
419
+ </details>
420
+
421
+ ### Training Logs
422
+ | Epoch | Step | Training Loss |
423
+ |:------:|:----:|:-------------:|
424
+ | 0.0165 | 1 | 4.5178 |
425
+ | 0.0331 | 2 | 3.8803 |
426
+ | 0.0496 | 3 | 2.8882 |
427
+ | 0.0661 | 4 | 4.5362 |
428
+ | 0.0826 | 5 | 3.6406 |
429
+ | 0.0992 | 6 | 3.5285 |
430
+ | 0.1157 | 7 | 4.1398 |
431
+ | 0.1322 | 8 | 4.1543 |
432
+ | 0.1488 | 9 | 4.4487 |
433
+ | 0.1653 | 10 | 4.7408 |
434
+ | 0.1818 | 11 | 2.1874 |
435
+ | 0.1983 | 12 | 3.3176 |
436
+ | 0.2149 | 13 | 2.8286 |
437
+ | 0.2314 | 14 | 2.87 |
438
+ | 0.2479 | 15 | 2.4834 |
439
+ | 0.2645 | 16 | 2.7856 |
440
+ | 0.2810 | 17 | 3.1948 |
441
+ | 0.2975 | 18 | 2.1755 |
442
+ | 0.3140 | 19 | 1.9861 |
443
+ | 0.3306 | 20 | 2.0536 |
444
+ | 0.3471 | 21 | 2.7626 |
445
+ | 0.3636 | 22 | 1.6489 |
446
+ | 0.3802 | 23 | 2.078 |
447
+ | 0.3967 | 24 | 1.5864 |
448
+ | 0.4132 | 25 | 1.8815 |
449
+ | 0.4298 | 26 | 1.8041 |
450
+ | 0.4463 | 27 | 1.7482 |
451
+ | 0.4628 | 28 | 1.191 |
452
+ | 0.4793 | 29 | 1.4166 |
453
+ | 0.4959 | 30 | 1.3215 |
454
+ | 0.5124 | 31 | 1.2907 |
455
+ | 0.5289 | 32 | 1.1294 |
456
+ | 0.5455 | 33 | 1.1586 |
457
+ | 0.5620 | 34 | 1.551 |
458
+ | 0.5785 | 35 | 1.3628 |
459
+ | 0.5950 | 36 | 0.9899 |
460
+ | 0.6116 | 37 | 1.1846 |
461
+ | 0.6281 | 38 | 1.2721 |
462
+ | 0.6446 | 39 | 1.1261 |
463
+ | 0.6612 | 40 | 0.9535 |
464
+ | 0.6777 | 41 | 1.2086 |
465
+ | 0.6942 | 42 | 0.7472 |
466
+ | 0.7107 | 43 | 1.0324 |
467
+ | 0.7273 | 44 | 1.0397 |
468
+ | 0.7438 | 45 | 1.185 |
469
+ | 0.7603 | 46 | 1.2112 |
470
+ | 0.7769 | 47 | 0.84 |
471
+ | 0.7934 | 48 | 0.9286 |
472
+ | 0.8099 | 49 | 0.8689 |
473
+ | 0.8264 | 50 | 0.9546 |
474
+ | 0.8430 | 51 | 0.8283 |
475
+ | 0.8595 | 52 | 0.757 |
476
+ | 0.8760 | 53 | 0.9199 |
477
+ | 0.8926 | 54 | 0.7404 |
478
+ | 0.9091 | 55 | 1.0995 |
479
+ | 0.9256 | 56 | 0.8231 |
480
+ | 0.9421 | 57 | 0.6297 |
481
+ | 0.9587 | 58 | 0.9869 |
482
+ | 0.9752 | 59 | 0.9597 |
483
+ | 0.9917 | 60 | 0.7025 |
484
+ | 1.0 | 61 | 0.4866 |
485
+
486
+
487
+ ### Framework Versions
488
+ - Python: 3.11.11
489
+ - Sentence Transformers: 4.1.0
490
+ - Transformers: 4.51.3
491
+ - PyTorch: 2.6.0+cu124
492
+ - Accelerate: 1.6.0
493
+ - Datasets: 3.5.0
494
+ - Tokenizers: 0.21.1
495
+
496
+ ## Citation
497
+
498
+ ### BibTeX
499
+
500
+ #### Sentence Transformers
501
+ ```bibtex
502
+ @inproceedings{reimers-2019-sentence-bert,
503
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
504
+ author = "Reimers, Nils and Gurevych, Iryna",
505
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
506
+ month = "11",
507
+ year = "2019",
508
+ publisher = "Association for Computational Linguistics",
509
+ url = "https://arxiv.org/abs/1908.10084",
510
+ }
511
+ ```
512
+
513
+ <!--
514
+ ## Glossary
515
+
516
+ *Clearly define terms in order to be accessible across audiences.*
517
+ -->
518
+
519
+ <!--
520
+ ## Model Card Authors
521
+
522
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
523
+ -->
524
+
525
+ <!--
526
+ ## Model Card Contact
527
+
528
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
529
+ -->
adapter_config.json ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "Alibaba-NLP/gte-Qwen2-7B-instruct",
5
+ "bias": "none",
6
+ "eva_config": null,
7
+ "exclude_modules": null,
8
+ "fan_in_fan_out": false,
9
+ "inference_mode": false,
10
+ "init_lora_weights": true,
11
+ "layer_replication": null,
12
+ "layers_pattern": null,
13
+ "layers_to_transform": null,
14
+ "loftq_config": {},
15
+ "lora_alpha": 64,
16
+ "lora_bias": false,
17
+ "lora_dropout": 0.1,
18
+ "megatron_config": null,
19
+ "megatron_core": "megatron.core",
20
+ "modules_to_save": null,
21
+ "peft_type": "LORA",
22
+ "r": 32,
23
+ "rank_pattern": {},
24
+ "revision": null,
25
+ "target_modules": [
26
+ "q_proj",
27
+ "k_proj",
28
+ "v_proj",
29
+ "o_proj"
30
+ ],
31
+ "task_type": "FEATURE_EXTRACTION",
32
+ "use_dora": false,
33
+ "use_rslora": false
34
+ }
adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:de4e685349f1155baacd9be9350b5a37692af09de116bcd82e4e4ac18966c814
3
+ size 80769088
added_tokens.json ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ {
2
+ "<|endoftext|>": 151643,
3
+ "<|im_end|>": 151645,
4
+ "<|im_start|>": 151644
5
+ }
config_sentence_transformers.json ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "__version__": {
3
+ "sentence_transformers": "4.1.0",
4
+ "transformers": "4.51.3",
5
+ "pytorch": "2.6.0+cu124"
6
+ },
7
+ "prompts": {
8
+ "query": "Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery: "
9
+ },
10
+ "default_prompt_name": null,
11
+ "similarity_fn_name": "cosine"
12
+ }
merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
modules.json ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.models.Transformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_Pooling",
12
+ "type": "sentence_transformers.models.Pooling"
13
+ },
14
+ {
15
+ "idx": 2,
16
+ "name": "2",
17
+ "path": "2_Normalize",
18
+ "type": "sentence_transformers.models.Normalize"
19
+ }
20
+ ]
pytorch_model_fsdp.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aecde96d535d523cc318234a0bbed463c891d949f00a0a815ec4a400e16cd89f
3
+ size 28357611916
sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 512,
3
+ "do_lower_case": false
4
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "additional_special_tokens": [
3
+ "<|im_start|>",
4
+ "<|im_end|>"
5
+ ],
6
+ "eos_token": {
7
+ "content": "<|endoftext|>",
8
+ "lstrip": false,
9
+ "normalized": false,
10
+ "rstrip": false,
11
+ "single_word": false
12
+ },
13
+ "pad_token": {
14
+ "content": "<|endoftext|>",
15
+ "lstrip": false,
16
+ "normalized": false,
17
+ "rstrip": false,
18
+ "single_word": false
19
+ }
20
+ }
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2f79052deba517b0663d877714e117a31a4a6243cddb85fc4443c80a2fa65a20
3
+ size 11419302
tokenizer_config.json ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "added_tokens_decoder": {
4
+ "151643": {
5
+ "content": "<|endoftext|>",
6
+ "lstrip": false,
7
+ "normalized": false,
8
+ "rstrip": false,
9
+ "single_word": false,
10
+ "special": true
11
+ },
12
+ "151644": {
13
+ "content": "<|im_start|>",
14
+ "lstrip": false,
15
+ "normalized": false,
16
+ "rstrip": false,
17
+ "single_word": false,
18
+ "special": true
19
+ },
20
+ "151645": {
21
+ "content": "<|im_end|>",
22
+ "lstrip": false,
23
+ "normalized": false,
24
+ "rstrip": false,
25
+ "single_word": false,
26
+ "special": true
27
+ }
28
+ },
29
+ "additional_special_tokens": [
30
+ "<|im_start|>",
31
+ "<|im_end|>"
32
+ ],
33
+ "auto_map": {
34
+ "AutoTokenizer": [
35
+ "Alibaba-NLP/gte-Qwen2-7B-instruct--tokenization_qwen.Qwen2Tokenizer",
36
+ "Alibaba-NLP/gte-Qwen2-7B-instruct--tokenization_qwen.Qwen2TokenizerFast"
37
+ ]
38
+ },
39
+ "bos_token": null,
40
+ "chat_template": "{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
41
+ "clean_up_tokenization_spaces": false,
42
+ "eos_token": "<|endoftext|>",
43
+ "errors": "replace",
44
+ "extra_special_tokens": {},
45
+ "model_max_length": 32768,
46
+ "pad_token": "<|endoftext|>",
47
+ "split_special_tokens": false,
48
+ "tokenizer_class": "Qwen2Tokenizer",
49
+ "unk_token": null
50
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1a6d6d74471e56cacc03a06ba2fd261195e7f488024ecc146babdb59577a8524
3
+ size 5688
vocab.json ADDED
The diff for this file is too large to render. See raw diff