Chu1111 commited on
Commit
1912f64
1 Parent(s): 67c22dd

Upload folder using huggingface_hub

Browse files
README.md CHANGED
@@ -1,3 +1,302 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - mteb
4
+ - qihoo360
5
+ - 奇虎360
6
+ - RAG-reranking
7
+ model-index:
8
+ - name: 360Zhinao-1_8B-reranking
9
+ results:
10
+ - task:
11
+ type: Reranking
12
+ dataset:
13
+ type: None
14
+ name: MTEB CMedQAv1
15
+ config: default
16
+ split: test
17
+ revision: None
18
+ metrics:
19
+ - type: map
20
+ value: 86.75017961853382
21
+ - type: mrr
22
+ value: 89.15436507936508
23
+ - task:
24
+ type: Reranking
25
+ dataset:
26
+ type: None
27
+ name: MTEB CMedQAv2
28
+ config: default
29
+ split: test
30
+ revision: None
31
+ metrics:
32
+ - type: map
33
+ value: 87.91572151930174
34
+ - type: mrr
35
+ value: 89.98869047619048
36
+ - task:
37
+ type: Reranking
38
+ dataset:
39
+ type: None
40
+ name: MTEB MMarcoReranking
41
+ config: default
42
+ split: dev
43
+ revision: None
44
+ metrics:
45
+ - type: map
46
+ value: 37.28779203409935
47
+ - type: mrr
48
+ value: 36.23730158730159
49
+ - task:
50
+ type: Reranking
51
+ dataset:
52
+ type: None
53
+ name: MTEB T2Reranking
54
+ config: default
55
+ split: dev
56
+ revision: None
57
+ metrics:
58
+ - type: map
59
+ value: 68.55153559405632
60
+ - type: mrr
61
+ value: 79.62773774596725
62
+ license: apache-2.0
63
+ library_name: transformers
64
+ ---
65
+
66
+ <div align="center">
67
+ <h1>
68
+ 360智脑
69
+ </h1>
70
+ </div>
71
+ <div align="center">
72
+ <a href="https://huggingface.co/qihoo360">Hugging Face</a>&nbsp&nbsp | &nbsp&nbspn
73
+ <a href="https://www.modelscope.cn/profile/qihoo360">ModelScope</a>&nbsp&nbsp | &nbsp&nbspn
74
+ </div>
75
+ <br>
76
+ <p align="center">
77
+ Welcome to 360 Zhinao<a href="https://ai.360.com"> https://ai.360.com </a>
78
+ </p>
79
+
80
+ <br>
81
+
82
+ # MTEB Leaderboard Chinese Reranking Results
83
+ We have validated the performance of our model on the [mteb-chinese-reranking leaderboard](https://huggingface.co/spaces/mteb/leaderboard). Currently, the open-source models on this leaderboard are primarily bidirectional discriminative models (BERT-like models). The only unidirectional generative model (GPT-like model) is gte-Qwen1.5-7B-instruct, which has an average score of 66.38, ranking 25th, with less than ideal results. Our self-developed unidirectional generative model, zhinao_1-8b_reranking, achieved an average score of 70.13, currently ranking first overall and first among open-source models, opening up new possibilities for generative models to undertake discriminative tasks.
84
+
85
+ | Model | T2Reranking | MMarcoReranking | CMedQAv1 | CMedQAv2 | Avg |
86
+ |:-------------------------------|:--------:|:--------:|:--------:|:--------:|:--------:|
87
+ | **360Zhinao-1_8B-reranking** | 68.55 | 37.29 | 86.75 | 87.92 | 70.13 |
88
+ | piccolo-large-zh-v2 | 67.15 | 33.39 | 90.14 | 89.31 | 70 |
89
+ | Baichuan-text-embedding | 67.85 | 34.3 | 88.46 | 88.06 | 69.67 |
90
+ | stella-mrl-large-zh-v3.5-1792d | 66.43 | 28.85 | 89.18 | 89.33 | 68.45 |
91
+ | PEG | 69.43 | 33.55 | 86.56 | 84.09 | 68.41 |
92
+ | bge-reranker-base | 67.28 | 35.46 | 81.27 | 84.1 | 67.03 |
93
+ | bge-reranker-large | 67.6 | 37.17 | 82.14 | 84.19 | 67.78 |
94
+
95
+
96
+ # Requirements
97
+
98
+ ```bash
99
+ pip install -r requirements.txt
100
+ ```
101
+
102
+ If your GPU supports fp16 or bf16 precision, we also recommend installing [flash-attention](https://github.com/Dao-AILab/flash-attention) (**now with support for flash attention 2**) to improve your runtime efficiency and reduce memory usage. (**flash-attention is optional and not required for running this project**)
103
+
104
+ ```bash
105
+ git clone https://github.com/Dao-AILab/flash-attention
106
+ cd flash-attention && pip install .
107
+ # The installation below is optional and might be slow.
108
+ # pip install csrc/layer_norm
109
+ # No need to install the following if the flash-attn version is above 2.1.1.
110
+ # pip install csrc/rotary
111
+ ```
112
+
113
+ # Model Introduction
114
+
115
+ The zhinao_1-8b_reranking model utilizes the self-developed zhinao_1-8b_base model as its foundation. Through iterative discovery and resolution of the following technical issues, it continuously stimulates the world knowledge inherent in the large model during the pre-training phase, better bridging the gap between generative models and discriminative tasks.
116
+
117
+ ## Data Processing
118
+
119
+ The model training did not utilize world knowledge, meaning it neither continued pre-training with domain-specific data nor fine-tuned datasets outside of the four datasets on the leaderboard. It only used the four datasets within the leaderboard, carefully iterating through data perception, and targeting different datasets for data cleaning and mining to ensure that the ranking in individual tasks could reach the top three level.
120
+
121
+ ## Resolving Task Conflicts
122
+
123
+ When merging four tasks, due to different data domain distributions, answer patterns, training data volumes, convergence steps, and even sequence lengths, conflicts exist between different tasks. Deeply resolving these conflict issues is crucial to obtaining a universal model with the best comprehensive indicators across different tasks.
124
+
125
+ ## Resolving Training Instability
126
+
127
+ Unlike generative tasks that produce multiple characters, using generative models for discriminative tasks requires the model to output a continuous value. Therefore, there is an oscillation problem during the training process. Deeply analyzing and resolving training instability can result in a model with better generalization and robustness.
128
+
129
+
130
+ # Inference Script
131
+
132
+ ```python
133
+ from typing import cast, List, Union, Tuple, Dict, Optional
134
+
135
+ import numpy as np
136
+ import torch
137
+ from tqdm import tqdm
138
+ from transformers import AutoModel, AutoTokenizer, AutoModelForSequenceClassification
139
+ import transformers
140
+ from transformers.trainer_pt_utils import LabelSmoother
141
+ IGNORE_TOKEN_ID = LabelSmoother.ignore_index
142
+
143
+ def preprocess(
144
+ sources,
145
+ tokenizer: transformers.PreTrainedTokenizer,
146
+ max_len: int = 1024,
147
+ system_message: str = "",
148
+ #system_message: str = "You are a helpful assistant.",
149
+ device = None,
150
+ ) -> Dict:
151
+ roles = {"user": "<|im_start|>user", "assistant": "<|im_start|>assistant"}
152
+ answer_len = 64
153
+
154
+ im_start = tokenizer.im_start_id
155
+ im_end = tokenizer.im_end_id
156
+ nl_tokens = tokenizer('\n').input_ids
157
+ _system = tokenizer('system').input_ids + nl_tokens
158
+ _user = tokenizer('user').input_ids + nl_tokens
159
+ _assistant = tokenizer('assistant').input_ids + nl_tokens
160
+
161
+ # Apply prompt templates
162
+ input_ids, targets = [], []
163
+ for i, source in enumerate(sources):
164
+ ## system_message
165
+ input_id, target = [], []
166
+ system = [im_start] + _system + tokenizer(system_message, max_length=max_len-answer_len, truncation=True).input_ids + [im_end] + nl_tokens
167
+ input_id += system
168
+ target += [im_start] + [IGNORE_TOKEN_ID] * (len(system)-3) + [im_end] + nl_tokens
169
+ assert len(input_id) == len(target)
170
+
171
+ ## query ans
172
+ source = "\n\n".join(source)
173
+ role = "<|im_start|>user"
174
+ _input_id = tokenizer(role, max_length=max_len-answer_len, truncation=True).input_ids + nl_tokens + \
175
+ tokenizer(source, max_length=max_len-answer_len, truncation=True).input_ids + [im_end] + nl_tokens
176
+ input_id += _input_id
177
+ if role == '<|im_start|>user':
178
+ _target = [im_start] + [IGNORE_TOKEN_ID] * (len(_input_id)-3) + [im_end] + nl_tokens
179
+ elif role == '<|im_start|>assistant':
180
+ _target = [im_start] + [IGNORE_TOKEN_ID] * len(tokenizer(role, max_length=max_len-answer_len, truncation=True).input_ids) + \
181
+ _input_id[len(tokenizer(role, max_length=max_len-answer_len, truncation=True).input_ids)+1:-2] + [im_end] + nl_tokens
182
+ else:
183
+ raise NotImplementedError
184
+ target += _target
185
+
186
+ ## label use placeholder 0; It will be masked later in the modeling_zhinao.py
187
+ role = "<|im_start|>assistant"
188
+ _input_id = tokenizer(role, max_length=max_len-answer_len, truncation=True).input_ids + nl_tokens + \
189
+ tokenizer("0", max_length=max_len-answer_len, truncation=True).input_ids + [im_end] + nl_tokens
190
+ input_id += _input_id
191
+ if role == '<|im_start|>user':
192
+ _target = [im_start] + [IGNORE_TOKEN_ID] * (len(_input_id)-3) + [im_end] + nl_tokens
193
+ elif role == '<|im_start|>assistant':
194
+ _target = [im_start] + [IGNORE_TOKEN_ID] * len(tokenizer(role, max_length=max_len-answer_len, truncation=True).input_ids) + \
195
+ _input_id[len(tokenizer(role, max_length=max_len-answer_len, truncation=True).input_ids)+1:-2] + [im_end] + nl_tokens
196
+ else:
197
+ raise NotImplementedError
198
+ target += _target
199
+
200
+ assert len(input_id) == len(target)
201
+ input_id += [tokenizer.pad_token_id] * (max_len - len(input_id))
202
+ target += [IGNORE_TOKEN_ID] * (max_len - len(target))
203
+ if len(input_id) > max_len:
204
+ print("max_len_error")
205
+ print(tokenizer.decode(input_id))
206
+
207
+ input_ids.append(input_id[:max_len])
208
+ targets.append(target[:max_len])
209
+ input_ids = torch.tensor(input_ids, dtype=torch.int)
210
+ targets = torch.tensor(targets, dtype=torch.int)
211
+ #print(f"input_ids {input_ids.shape}")
212
+ #print(f"targets {targets.shape}")
213
+
214
+ return dict(
215
+ input_ids=input_ids.to(device),
216
+ labels=targets.to(device),
217
+ attention_mask=input_ids.ne(tokenizer.pad_token_id).to(device),
218
+ )
219
+
220
+ class FlagRerankerCustom:
221
+ def __init__(
222
+ self,
223
+ model_name_or_path: str = None,
224
+ use_fp16: bool = False
225
+ ) -> None:
226
+ self.tokenizer = transformers.AutoTokenizer.from_pretrained(
227
+ model_name_or_path,
228
+ model_max_length=1024,
229
+ padding_side="right",
230
+ use_fast=False,
231
+ trust_remote_code=True
232
+ )
233
+ self.tokenizer.pad_token_id = self.tokenizer.eod_id
234
+ config = transformers.AutoConfig.from_pretrained(
235
+ model_name_or_path,
236
+ trust_remote_code=True,
237
+ bf16=True,
238
+ )
239
+ config.use_cache = False
240
+ self.model = transformers.AutoModelForCausalLM.from_pretrained(
241
+ model_name_or_path,
242
+ config=config,
243
+ trust_remote_code=True,
244
+ )
245
+ self.model.linear.bfloat16()
246
+
247
+ if torch.cuda.is_available():
248
+ self.device = torch.device('cuda')
249
+ elif torch.backends.mps.is_available():
250
+ self.device = torch.device('mps')
251
+ else:
252
+ self.device = torch.device('cpu')
253
+ use_fp16 = False
254
+ if use_fp16:
255
+ self.model.half()
256
+
257
+ self.model = self.model.to(self.device)
258
+
259
+ self.model.eval()
260
+
261
+ self.num_gpus = torch.cuda.device_count()
262
+ if self.num_gpus > 1:
263
+ print(f"----------using {self.num_gpus}*GPUs----------")
264
+ self.model = torch.nn.DataParallel(self.model)
265
+
266
+ @torch.no_grad()
267
+ def compute_score(self, sentence_pairs: Union[List[Tuple[str, str]], Tuple[str, str]], batch_size: int =128,
268
+ max_length: int = 1024) -> List[float]:
269
+ if self.num_gpus > 0:
270
+ batch_size = batch_size * self.num_gpus
271
+
272
+ assert isinstance(sentence_pairs, list)
273
+ if isinstance(sentence_pairs[0], str):
274
+ sentence_pairs = [sentence_pairs]
275
+
276
+ all_scores = []
277
+ for start_index in tqdm(range(0, len(sentence_pairs), batch_size), desc="Compute Scores",
278
+ disable=len(sentence_pairs) < 128):
279
+ sentences_batch = sentence_pairs[start_index:start_index + batch_size] # [[q,ans],[q, ans]...]
280
+ inputs = preprocess(sources=sentences_batch, tokenizer=self.tokenizer,max_len=1024,device=self.device)
281
+ scores = self.model(**inputs, return_dict=True).logits.view(-1, ).float()
282
+ all_scores.extend(scores.cpu().numpy().tolist())
283
+
284
+ if len(all_scores) == 1:
285
+ return all_scores[0]
286
+ return all_scores
287
+
288
+
289
+ if __name__ == "__main__":
290
+ model_name_or_path = "360Zhinao-1_8B-reranking"
291
+ model = FlagRerankerCustom(model_name_or_path, use_fp16=False)
292
+ inputs=[["What Color Is the Sky","Blue"], ["What Color Is the Sky","Pink"],]
293
+ ret = model.compute_score(inputs)
294
+ print(ret)
295
+
296
+ ```
297
+
298
+ ## License
299
+ The source code of this repository follows the open-source license Apache 2.0.
300
+ 360​Zhinao open-source models support commercial use. If you wish to use these models or continue training them for commercial purposes, please contact us via email (g-zhinao-opensource@360.cn) to apply. For the specific license agreement, please see <<360 Zhinao Open-Source Model License>>.
301
+
302
+
config.json ADDED
File without changes
configuration_zhinao.py ADDED
File without changes
generation_config.json ADDED
File without changes
generation_utils.py ADDED
File without changes
latest ADDED
File without changes
modeling_zhinao.py ADDED
File without changes
pytorch_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:77baec385e5aab207797882f836e26018fb50f058f8c5dbc4430cedf9655be03
3
+ size 2070999040
requirements.txt ADDED
@@ -0,0 +1,213 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ accelerate==0.26.1
2
+ aiohttp==3.9.1
3
+ aiosignal==1.3.1
4
+ annotated-types==0.6.0
5
+ anyio==4.2.0
6
+ appdirs==1.4.4
7
+ argon2-cffi==23.1.0
8
+ argon2-cffi-bindings==21.2.0
9
+ arrow==1.3.0
10
+ asttokens==2.4.1
11
+ async-lru==2.0.4
12
+ async-timeout==4.0.3
13
+ attrs==23.2.0
14
+ auto-gptq==0.5.1
15
+ Babel==2.14.0
16
+ beautifulsoup4==4.12.3
17
+ beir==2.0.0
18
+ bleach==6.1.0
19
+ blessed==1.20.0
20
+ certifi==2023.11.17
21
+ cffi==1.16.0
22
+ charset-normalizer==3.3.2
23
+ click==8.1.7
24
+ cmake==3.28.1
25
+ colorama==0.4.6
26
+ coloredlogs==15.0.1
27
+ comm==0.2.1
28
+ datasets==2.14.7
29
+ debugpy==1.8.0
30
+ decorator==5.1.1
31
+ deepspeed==0.13.1
32
+ defusedxml==0.7.1
33
+ dill==0.3.7
34
+ docker-pycreds==0.4.0
35
+ einops==0.7.0
36
+ elasticsearch==7.9.1
37
+ exceptiongroup==1.2.0
38
+ executing==2.0.1
39
+ faiss-cpu==1.7.4
40
+ faiss-gpu==1.7.2
41
+ fastjsonschema==2.19.1
42
+ filelock==3.13.1
43
+ FlagEmbedding==1.1.9
44
+ flash_attn==2.3.6
45
+ fqdn==1.5.1
46
+ frozenlist==1.4.1
47
+ fsspec==2023.10.0
48
+ gekko==1.0.6
49
+ gitdb==4.0.11
50
+ GitPython==3.1.41
51
+ google==3.0.0
52
+ gpustat==1.1.1
53
+ hjson==3.1.0
54
+ huggingface-hub==0.17.3
55
+ humanfriendly==10.0
56
+ icecream==2.1.3
57
+ idna==3.6
58
+ importlib-metadata==7.0.1
59
+ ipykernel==6.29.0
60
+ ipython==8.18.1
61
+ ipywidgets==8.1.1
62
+ isoduration==20.11.0
63
+ jedi==0.19.1
64
+ Jinja2==3.1.3
65
+ joblib==1.3.2
66
+ json5==0.9.14
67
+ jsonlines==4.0.0
68
+ jsonpointer==2.4
69
+ jsonschema==4.21.1
70
+ jsonschema-specifications==2023.12.1
71
+ jupyter==1.0.0
72
+ jupyter-console==6.6.3
73
+ jupyter-events==0.9.0
74
+ jupyter-lsp==2.2.2
75
+ jupyter_client==8.6.0
76
+ jupyter_core==5.7.1
77
+ jupyter_server==2.12.5
78
+ jupyter_server_terminals==0.5.2
79
+ jupyterlab==4.0.12
80
+ jupyterlab-widgets==3.0.9
81
+ jupyterlab_pygments==0.3.0
82
+ jupyterlab_server==2.25.2
83
+ libretranslatepy==2.1.1
84
+ lightning-utilities==0.10.1
85
+ lit==18.1.2
86
+ lxml==5.1.0
87
+ markdown-it-py==3.0.0
88
+ MarkupSafe==2.1.3
89
+ matplotlib-inline==0.1.6
90
+ mdurl==0.1.2
91
+ mistune==3.0.2
92
+ mpmath==1.3.0
93
+ mteb==1.1.1
94
+ multidict==6.0.4
95
+ multiprocess==0.70.15
96
+ nbclient==0.9.0
97
+ nbconvert==7.14.2
98
+ nbformat==5.9.2
99
+ nest-asyncio==1.6.0
100
+ networkx==3.2.1
101
+ ninja==1.11.1.1
102
+ nltk==3.8.1
103
+ notebook==7.0.7
104
+ notebook_shim==0.2.3
105
+ numpy==1.26.3
106
+ nvidia-cublas-cu11==11.10.3.66
107
+ nvidia-cublas-cu12==12.1.3.1
108
+ nvidia-cuda-cupti-cu11==11.7.101
109
+ nvidia-cuda-cupti-cu12==12.1.105
110
+ nvidia-cuda-nvrtc-cu11==11.7.99
111
+ nvidia-cuda-nvrtc-cu12==12.1.105
112
+ nvidia-cuda-runtime-cu11==11.7.99
113
+ nvidia-cuda-runtime-cu12==12.1.105
114
+ nvidia-cudnn-cu11==8.5.0.96
115
+ nvidia-cudnn-cu12==8.9.2.26
116
+ nvidia-cufft-cu11==10.9.0.58
117
+ nvidia-cufft-cu12==11.0.2.54
118
+ nvidia-curand-cu11==10.2.10.91
119
+ nvidia-curand-cu12==10.3.2.106
120
+ nvidia-cusolver-cu11==11.4.0.1
121
+ nvidia-cusolver-cu12==11.4.5.107
122
+ nvidia-cusparse-cu11==11.7.4.91
123
+ nvidia-cusparse-cu12==12.1.0.106
124
+ nvidia-ml-py==12.535.133
125
+ nvidia-nccl-cu12==2.18.1
126
+ nvidia-nvjitlink-cu12==12.3.101
127
+ nvidia-nvtx-cu11==11.7.91
128
+ nvidia-nvtx-cu12==12.1.105
129
+ optimum==1.14.0
130
+ overrides==7.7.0
131
+ packaging==23.2
132
+ pandas==2.1.4
133
+ pandocfilters==1.5.1
134
+ parso==0.8.3
135
+ peft==0.6.1
136
+ pexpect==4.9.0
137
+ pillow==10.2.0
138
+ platformdirs==4.2.0
139
+ prometheus-client==0.19.0
140
+ prompt-toolkit==3.0.43
141
+ protobuf==4.25.2
142
+ psutil==5.9.7
143
+ ptyprocess==0.7.0
144
+ pure-eval==0.2.2
145
+ py-cpuinfo==9.0.0
146
+ pyarrow==14.0.2
147
+ pyarrow-hotfix==0.6
148
+ pycparser==2.21
149
+ pydantic==2.6.0
150
+ pydantic_core==2.16.1
151
+ Pygments==2.17.2
152
+ pynvml==11.5.0
153
+ python-dateutil==2.8.2
154
+ python-json-logger==2.0.7
155
+ pytrec-eval==0.5
156
+ pytz==2023.3.post1
157
+ PyYAML==6.0.1
158
+ pyzmq==25.1.2
159
+ qtconsole==5.5.1
160
+ QtPy==2.4.1
161
+ referencing==0.33.0
162
+ regex==2023.12.25
163
+ requests==2.31.0
164
+ rfc3339-validator==0.1.4
165
+ rfc3986-validator==0.1.1
166
+ rich==13.7.0
167
+ rotary-emb==0.1
168
+ rouge==1.0.1
169
+ rpds-py==0.17.1
170
+ safetensors==0.4.1
171
+ scikit-learn==1.3.2
172
+ scipy==1.11.4
173
+ Send2Trash==1.8.2
174
+ sentence-transformers==2.2.2
175
+ sentencepiece==0.1.99
176
+ sentry-sdk==1.40.0
177
+ setproctitle==1.3.3
178
+ six==1.16.0
179
+ smmap==5.0.1
180
+ sniffio==1.3.0
181
+ soupsieve==2.5
182
+ stack-data==0.6.3
183
+ sympy==1.12
184
+ terminado==0.18.0
185
+ threadpoolctl==3.2.0
186
+ tiktoken==0.5.2
187
+ tinycss2==1.2.1
188
+ tokenizers==0.14.1
189
+ tomli==2.0.1
190
+ torch==2.1.2
191
+ torchmetrics==1.3.0.post0
192
+ torchvision==0.16.2
193
+ tornado==6.4
194
+ tqdm==4.66.1
195
+ traitlets==5.14.1
196
+ transformers==4.34.0
197
+ transformers-stream-generator==0.0.4
198
+ translate==3.6.1
199
+ triton==2.1.0
200
+ types-python-dateutil==2.8.19.20240106
201
+ typing_extensions==4.9.0
202
+ tzdata==2023.4
203
+ uri-template==1.3.0
204
+ urllib3==2.1.0
205
+ wandb==0.16.2
206
+ wcwidth==0.2.13
207
+ webcolors==1.13
208
+ webencodings==0.5.1
209
+ websocket-client==1.7.0
210
+ widgetsnbextension==4.0.9
211
+ xxhash==3.4.1
212
+ yarl==1.9.4
213
+ zipp==3.17.0
rng_state_0.pth ADDED
File without changes
rng_state_1.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aa1a5768088961a39dd71fd2b1b14b3993332dc7ba835dd610bb0c5395be2c93
3
+ size 15920
rng_state_2.pth ADDED
File without changes
rng_state_3.pth ADDED
File without changes
rng_state_4.pth ADDED
File without changes
rng_state_5.pth ADDED
File without changes
rng_state_6.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8628151f541fc32f66e6527d07b52899e6b27a9d7db8a5da5b88ab8fd3b80730
3
+ size 15920
rng_state_7.pth ADDED
File without changes
special_tokens_map.json ADDED
File without changes
tokenization_zhinao.py ADDED
File without changes
tokenizer_config.json ADDED
File without changes
trainer_state.json ADDED
@@ -0,0 +1,2153 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": null,
3
+ "best_model_checkpoint": null,
4
+ "epoch": 1.8537243006403776,
5
+ "eval_steps": 500,
6
+ "global_step": 11000,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.01,
13
+ "learning_rate": 1.2605042016806723e-05,
14
+ "loss": 0.0184,
15
+ "step": 50
16
+ },
17
+ {
18
+ "epoch": 0.02,
19
+ "learning_rate": 2.5210084033613446e-05,
20
+ "loss": 0.0142,
21
+ "step": 100
22
+ },
23
+ {
24
+ "epoch": 0.03,
25
+ "learning_rate": 2.999948467631686e-05,
26
+ "loss": 0.0136,
27
+ "step": 150
28
+ },
29
+ {
30
+ "epoch": 0.03,
31
+ "learning_rate": 2.999648186693593e-05,
32
+ "loss": 0.0132,
33
+ "step": 200
34
+ },
35
+ {
36
+ "epoch": 0.04,
37
+ "learning_rate": 2.999079852637007e-05,
38
+ "loss": 0.0126,
39
+ "step": 250
40
+ },
41
+ {
42
+ "epoch": 0.05,
43
+ "learning_rate": 2.9982435670482325e-05,
44
+ "loss": 0.0123,
45
+ "step": 300
46
+ },
47
+ {
48
+ "epoch": 0.06,
49
+ "learning_rate": 2.9971394794083023e-05,
50
+ "loss": 0.0122,
51
+ "step": 350
52
+ },
53
+ {
54
+ "epoch": 0.07,
55
+ "learning_rate": 2.9957677870662595e-05,
56
+ "loss": 0.0115,
57
+ "step": 400
58
+ },
59
+ {
60
+ "epoch": 0.08,
61
+ "learning_rate": 2.994128735203883e-05,
62
+ "loss": 0.0113,
63
+ "step": 450
64
+ },
65
+ {
66
+ "epoch": 0.08,
67
+ "learning_rate": 2.9922226167918624e-05,
68
+ "loss": 0.0113,
69
+ "step": 500
70
+ },
71
+ {
72
+ "epoch": 0.08,
73
+ "eval_ap_CMedQAv1": 0.8589192417053253,
74
+ "eval_ap_CMedQAv2": 0.8619373174701326,
75
+ "eval_ap_Mmarco": 0.33024273953903277,
76
+ "eval_ap_T2Reranking": 0.6854621357989829,
77
+ "eval_avg_ap": 0.6841403586283684,
78
+ "eval_loss": 0.1156548261642456,
79
+ "eval_mrr_CMedQAv1": 0.8823821428571428,
80
+ "eval_mrr_CMedQAv2": 0.8846333333333333,
81
+ "eval_mrr_Mmarco": 0.32088492063492063,
82
+ "eval_mrr_T2Reranking": 0.7938369818701573,
83
+ "eval_ndcg@10_CMedQAv1": 0.9590048789978027,
84
+ "eval_ndcg@10_CMedQAv2": 0.982421875,
85
+ "eval_ndcg@10_Mmarco": 0.15075407922267914,
86
+ "eval_ndcg@10_T2Reranking": 0.6162768602371216,
87
+ "eval_ndcg@1_CMedQAv1": 0.949999988079071,
88
+ "eval_ndcg@1_CMedQAv2": 1.0,
89
+ "eval_ndcg@1_Mmarco": 0.10000000149011612,
90
+ "eval_ndcg@1_T2Reranking": 0.6266667246818542,
91
+ "eval_ndcg@3_CMedQAv1": 0.9617319107055664,
92
+ "eval_ndcg@3_CMedQAv2": 0.9882680177688599,
93
+ "eval_ndcg@3_Mmarco": 0.2184327393770218,
94
+ "eval_ndcg@3_T2Reranking": 0.5918937921524048,
95
+ "eval_ndcg@5_CMedQAv1": 0.9577357172966003,
96
+ "eval_ndcg@5_CMedQAv2": 0.9842175245285034,
97
+ "eval_ndcg@5_Mmarco": 0.1987150013446808,
98
+ "eval_ndcg@5_T2Reranking": 0.5972660779953003,
99
+ "eval_ndcg_CMedQAv1": 0.9578067064285278,
100
+ "eval_ndcg_CMedQAv2": 0.9620075225830078,
101
+ "eval_ndcg_Mmarco": 0.4313792586326599,
102
+ "eval_ndcg_T2Reranking": 0.8985671997070312,
103
+ "eval_runtime": 1325.9604,
104
+ "eval_samples_per_second": 301.688,
105
+ "eval_steps_per_second": 0.295,
106
+ "step": 500
107
+ },
108
+ {
109
+ "epoch": 0.09,
110
+ "learning_rate": 2.9900497725374308e-05,
111
+ "loss": 0.0112,
112
+ "step": 550
113
+ },
114
+ {
115
+ "epoch": 0.1,
116
+ "learning_rate": 2.9876105908234656e-05,
117
+ "loss": 0.0111,
118
+ "step": 600
119
+ },
120
+ {
121
+ "epoch": 0.11,
122
+ "learning_rate": 2.9849055076390685e-05,
123
+ "loss": 0.0112,
124
+ "step": 650
125
+ },
126
+ {
127
+ "epoch": 0.12,
128
+ "learning_rate": 2.981935006501634e-05,
129
+ "loss": 0.011,
130
+ "step": 700
131
+ },
132
+ {
133
+ "epoch": 0.13,
134
+ "learning_rate": 2.978699618370422e-05,
135
+ "loss": 0.0107,
136
+ "step": 750
137
+ },
138
+ {
139
+ "epoch": 0.13,
140
+ "learning_rate": 2.9751999215516562e-05,
141
+ "loss": 0.0109,
142
+ "step": 800
143
+ },
144
+ {
145
+ "epoch": 0.14,
146
+ "learning_rate": 2.971436541595152e-05,
147
+ "loss": 0.0109,
148
+ "step": 850
149
+ },
150
+ {
151
+ "epoch": 0.15,
152
+ "learning_rate": 2.967410151182503e-05,
153
+ "loss": 0.0102,
154
+ "step": 900
155
+ },
156
+ {
157
+ "epoch": 0.16,
158
+ "learning_rate": 2.963121470006846e-05,
159
+ "loss": 0.0103,
160
+ "step": 950
161
+ },
162
+ {
163
+ "epoch": 0.17,
164
+ "learning_rate": 2.9585712646442172e-05,
165
+ "loss": 0.01,
166
+ "step": 1000
167
+ },
168
+ {
169
+ "epoch": 0.17,
170
+ "eval_ap_CMedQAv1": 0.8514522230239366,
171
+ "eval_ap_CMedQAv2": 0.8667097304641765,
172
+ "eval_ap_Mmarco": 0.33328945740605653,
173
+ "eval_ap_T2Reranking": 0.681851884051082,
174
+ "eval_avg_ap": 0.6833258237363129,
175
+ "eval_loss": 0.11742711067199707,
176
+ "eval_mrr_CMedQAv1": 0.875027380952381,
177
+ "eval_mrr_CMedQAv2": 0.8912928571428572,
178
+ "eval_mrr_Mmarco": 0.3221150793650793,
179
+ "eval_mrr_T2Reranking": 0.7927472568806352,
180
+ "eval_ndcg@10_CMedQAv1": 0.9848238229751587,
181
+ "eval_ndcg@10_CMedQAv2": 1.0,
182
+ "eval_ndcg@10_Mmarco": 0.1760159730911255,
183
+ "eval_ndcg@10_T2Reranking": 0.5729199647903442,
184
+ "eval_ndcg@1_CMedQAv1": 1.0,
185
+ "eval_ndcg@1_CMedQAv2": 1.0,
186
+ "eval_ndcg@1_Mmarco": 0.20000000298023224,
187
+ "eval_ndcg@1_T2Reranking": 0.6088888645172119,
188
+ "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
189
+ "eval_ndcg@3_CMedQAv2": 0.9999998807907104,
190
+ "eval_ndcg@3_Mmarco": 0.2740204632282257,
191
+ "eval_ndcg@3_T2Reranking": 0.5606718063354492,
192
+ "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
193
+ "eval_ndcg@5_CMedQAv2": 0.9999998807907104,
194
+ "eval_ndcg@5_Mmarco": 0.2053453028202057,
195
+ "eval_ndcg@5_T2Reranking": 0.5687648057937622,
196
+ "eval_ndcg_CMedQAv1": 0.958914577960968,
197
+ "eval_ndcg_CMedQAv2": 0.9623709917068481,
198
+ "eval_ndcg_Mmarco": 0.45813989639282227,
199
+ "eval_ndcg_T2Reranking": 0.8942973017692566,
200
+ "eval_runtime": 1104.4572,
201
+ "eval_samples_per_second": 362.192,
202
+ "eval_steps_per_second": 0.354,
203
+ "step": 1000
204
+ },
205
+ {
206
+ "epoch": 0.18,
207
+ "learning_rate": 2.953760348416533e-05,
208
+ "loss": 0.0099,
209
+ "step": 1050
210
+ },
211
+ {
212
+ "epoch": 0.19,
213
+ "learning_rate": 2.9486895812462135e-05,
214
+ "loss": 0.0103,
215
+ "step": 1100
216
+ },
217
+ {
218
+ "epoch": 0.19,
219
+ "learning_rate": 2.943359869502476e-05,
220
+ "loss": 0.0099,
221
+ "step": 1150
222
+ },
223
+ {
224
+ "epoch": 0.2,
225
+ "learning_rate": 2.9377721658393268e-05,
226
+ "loss": 0.0097,
227
+ "step": 1200
228
+ },
229
+ {
230
+ "epoch": 0.21,
231
+ "learning_rate": 2.9319274690252808e-05,
232
+ "loss": 0.0099,
233
+ "step": 1250
234
+ },
235
+ {
236
+ "epoch": 0.22,
237
+ "learning_rate": 2.9258268237648375e-05,
238
+ "loss": 0.0098,
239
+ "step": 1300
240
+ },
241
+ {
242
+ "epoch": 0.23,
243
+ "learning_rate": 2.9194713205117454e-05,
244
+ "loss": 0.0099,
245
+ "step": 1350
246
+ },
247
+ {
248
+ "epoch": 0.24,
249
+ "learning_rate": 2.9128620952740903e-05,
250
+ "loss": 0.0098,
251
+ "step": 1400
252
+ },
253
+ {
254
+ "epoch": 0.24,
255
+ "learning_rate": 2.906000329411242e-05,
256
+ "loss": 0.0096,
257
+ "step": 1450
258
+ },
259
+ {
260
+ "epoch": 0.25,
261
+ "learning_rate": 2.898887249422691e-05,
262
+ "loss": 0.0097,
263
+ "step": 1500
264
+ },
265
+ {
266
+ "epoch": 0.25,
267
+ "eval_ap_CMedQAv1": 0.8415707281395,
268
+ "eval_ap_CMedQAv2": 0.8617282454306143,
269
+ "eval_ap_Mmarco": 0.369688968005382,
270
+ "eval_ap_T2Reranking": 0.6881681817864197,
271
+ "eval_avg_ap": 0.690289030840479,
272
+ "eval_loss": 0.12302990257740021,
273
+ "eval_mrr_CMedQAv1": 0.8641345238095237,
274
+ "eval_mrr_CMedQAv2": 0.8869757936507937,
275
+ "eval_mrr_Mmarco": 0.36682936507936503,
276
+ "eval_mrr_T2Reranking": 0.8009392026952963,
277
+ "eval_ndcg@10_CMedQAv1": 0.9867491722106934,
278
+ "eval_ndcg@10_CMedQAv2": 1.0,
279
+ "eval_ndcg@10_Mmarco": 0.2047528326511383,
280
+ "eval_ndcg@10_T2Reranking": 0.6122068166732788,
281
+ "eval_ndcg@1_CMedQAv1": 1.0,
282
+ "eval_ndcg@1_CMedQAv2": 1.0,
283
+ "eval_ndcg@1_Mmarco": 0.4333333373069763,
284
+ "eval_ndcg@1_T2Reranking": 0.5628505945205688,
285
+ "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
286
+ "eval_ndcg@3_CMedQAv2": 0.9999998807907104,
287
+ "eval_ndcg@3_Mmarco": 0.30372515320777893,
288
+ "eval_ndcg@3_T2Reranking": 0.5843338966369629,
289
+ "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
290
+ "eval_ndcg@5_CMedQAv2": 0.9999998807907104,
291
+ "eval_ndcg@5_Mmarco": 0.26333072781562805,
292
+ "eval_ndcg@5_T2Reranking": 0.5988883972167969,
293
+ "eval_ndcg_CMedQAv1": 0.9555119276046753,
294
+ "eval_ndcg_CMedQAv2": 0.9630300402641296,
295
+ "eval_ndcg_Mmarco": 0.4870051443576813,
296
+ "eval_ndcg_T2Reranking": 0.8979281187057495,
297
+ "eval_runtime": 1126.8763,
298
+ "eval_samples_per_second": 354.987,
299
+ "eval_steps_per_second": 0.347,
300
+ "step": 1500
301
+ },
302
+ {
303
+ "epoch": 0.26,
304
+ "learning_rate": 2.8915241267288212e-05,
305
+ "loss": 0.0095,
306
+ "step": 1550
307
+ },
308
+ {
309
+ "epoch": 0.27,
310
+ "learning_rate": 2.8839122774436504e-05,
311
+ "loss": 0.0097,
312
+ "step": 1600
313
+ },
314
+ {
315
+ "epoch": 0.28,
316
+ "learning_rate": 2.8760530621395827e-05,
317
+ "loss": 0.0096,
318
+ "step": 1650
319
+ },
320
+ {
321
+ "epoch": 0.29,
322
+ "learning_rate": 2.8679478856042137e-05,
323
+ "loss": 0.0098,
324
+ "step": 1700
325
+ },
326
+ {
327
+ "epoch": 0.29,
328
+ "learning_rate": 2.8595981965892344e-05,
329
+ "loss": 0.0093,
330
+ "step": 1750
331
+ },
332
+ {
333
+ "epoch": 0.3,
334
+ "learning_rate": 2.851005487551475e-05,
335
+ "loss": 0.0094,
336
+ "step": 1800
337
+ },
338
+ {
339
+ "epoch": 0.31,
340
+ "learning_rate": 2.8421712943861372e-05,
341
+ "loss": 0.0095,
342
+ "step": 1850
343
+ },
344
+ {
345
+ "epoch": 0.32,
346
+ "learning_rate": 2.8330971961522614e-05,
347
+ "loss": 0.0094,
348
+ "step": 1900
349
+ },
350
+ {
351
+ "epoch": 0.33,
352
+ "learning_rate": 2.823784814790481e-05,
353
+ "loss": 0.0096,
354
+ "step": 1950
355
+ },
356
+ {
357
+ "epoch": 0.34,
358
+ "learning_rate": 2.8142358148331083e-05,
359
+ "loss": 0.0095,
360
+ "step": 2000
361
+ },
362
+ {
363
+ "epoch": 0.34,
364
+ "eval_ap_CMedQAv1": 0.8458553211346398,
365
+ "eval_ap_CMedQAv2": 0.8602027011804556,
366
+ "eval_ap_Mmarco": 0.3578987317739686,
367
+ "eval_ap_T2Reranking": 0.6852344199870449,
368
+ "eval_avg_ap": 0.6872977935190273,
369
+ "eval_loss": 0.12346025556325912,
370
+ "eval_mrr_CMedQAv1": 0.8699591269841269,
371
+ "eval_mrr_CMedQAv2": 0.8808488095238096,
372
+ "eval_mrr_Mmarco": 0.3493373015873016,
373
+ "eval_mrr_T2Reranking": 0.7949749331012027,
374
+ "eval_ndcg@10_CMedQAv1": 0.9936379194259644,
375
+ "eval_ndcg@10_CMedQAv2": 1.0000001192092896,
376
+ "eval_ndcg@10_Mmarco": 0.18332302570343018,
377
+ "eval_ndcg@10_T2Reranking": 0.5819754600524902,
378
+ "eval_ndcg@1_CMedQAv1": 1.0,
379
+ "eval_ndcg@1_CMedQAv2": 1.0,
380
+ "eval_ndcg@1_Mmarco": 0.20000000298023224,
381
+ "eval_ndcg@1_T2Reranking": 0.5881534814834595,
382
+ "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
383
+ "eval_ndcg@3_CMedQAv2": 0.9999998807907104,
384
+ "eval_ndcg@3_Mmarco": 0.2092163860797882,
385
+ "eval_ndcg@3_T2Reranking": 0.5824980139732361,
386
+ "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
387
+ "eval_ndcg@5_CMedQAv2": 0.9999998807907104,
388
+ "eval_ndcg@5_Mmarco": 0.19205407798290253,
389
+ "eval_ndcg@5_T2Reranking": 0.5816112160682678,
390
+ "eval_ndcg_CMedQAv1": 0.9610760807991028,
391
+ "eval_ndcg_CMedQAv2": 0.9652814865112305,
392
+ "eval_ndcg_Mmarco": 0.45809394121170044,
393
+ "eval_ndcg_T2Reranking": 0.896248996257782,
394
+ "eval_runtime": 1147.2183,
395
+ "eval_samples_per_second": 348.692,
396
+ "eval_steps_per_second": 0.341,
397
+ "step": 2000
398
+ },
399
+ {
400
+ "epoch": 0.35,
401
+ "learning_rate": 2.8044519031066117e-05,
402
+ "loss": 0.0094,
403
+ "step": 2050
404
+ },
405
+ {
406
+ "epoch": 0.35,
407
+ "learning_rate": 2.794434828426527e-05,
408
+ "loss": 0.0095,
409
+ "step": 2100
410
+ },
411
+ {
412
+ "epoch": 0.36,
413
+ "learning_rate": 2.7841863812848724e-05,
414
+ "loss": 0.0093,
415
+ "step": 2150
416
+ },
417
+ {
418
+ "epoch": 0.37,
419
+ "learning_rate": 2.773708393530104e-05,
420
+ "loss": 0.0093,
421
+ "step": 2200
422
+ },
423
+ {
424
+ "epoch": 0.38,
425
+ "learning_rate": 2.7630027380396854e-05,
426
+ "loss": 0.0092,
427
+ "step": 2250
428
+ },
429
+ {
430
+ "epoch": 0.39,
431
+ "learning_rate": 2.7520713283853237e-05,
432
+ "loss": 0.0091,
433
+ "step": 2300
434
+ },
435
+ {
436
+ "epoch": 0.4,
437
+ "learning_rate": 2.740916118490928e-05,
438
+ "loss": 0.0091,
439
+ "step": 2350
440
+ },
441
+ {
442
+ "epoch": 0.4,
443
+ "learning_rate": 2.729539102283358e-05,
444
+ "loss": 0.009,
445
+ "step": 2400
446
+ },
447
+ {
448
+ "epoch": 0.41,
449
+ "learning_rate": 2.7179423133360214e-05,
450
+ "loss": 0.009,
451
+ "step": 2450
452
+ },
453
+ {
454
+ "epoch": 0.42,
455
+ "learning_rate": 2.7061278245053856e-05,
456
+ "loss": 0.0093,
457
+ "step": 2500
458
+ },
459
+ {
460
+ "epoch": 0.42,
461
+ "eval_ap_CMedQAv1": 0.8494758904704668,
462
+ "eval_ap_CMedQAv2": 0.8578434981555412,
463
+ "eval_ap_Mmarco": 0.38035966777131036,
464
+ "eval_ap_T2Reranking": 0.6841403863981271,
465
+ "eval_avg_ap": 0.6929548606988614,
466
+ "eval_loss": 0.11943219602108002,
467
+ "eval_mrr_CMedQAv1": 0.8727809523809524,
468
+ "eval_mrr_CMedQAv2": 0.8802436507936509,
469
+ "eval_mrr_Mmarco": 0.37201190476190477,
470
+ "eval_mrr_T2Reranking": 0.7946123631127017,
471
+ "eval_ndcg@10_CMedQAv1": 0.9861699342727661,
472
+ "eval_ndcg@10_CMedQAv2": 1.0000001192092896,
473
+ "eval_ndcg@10_Mmarco": 0.21542616188526154,
474
+ "eval_ndcg@10_T2Reranking": 0.5951088666915894,
475
+ "eval_ndcg@1_CMedQAv1": 1.0,
476
+ "eval_ndcg@1_CMedQAv2": 1.0,
477
+ "eval_ndcg@1_Mmarco": 0.25,
478
+ "eval_ndcg@1_T2Reranking": 0.5847460031509399,
479
+ "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
480
+ "eval_ndcg@3_CMedQAv2": 0.9999998807907104,
481
+ "eval_ndcg@3_Mmarco": 0.2740204632282257,
482
+ "eval_ndcg@3_T2Reranking": 0.5968767404556274,
483
+ "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
484
+ "eval_ndcg@5_CMedQAv2": 0.9999998807907104,
485
+ "eval_ndcg@5_Mmarco": 0.25857046246528625,
486
+ "eval_ndcg@5_T2Reranking": 0.6022564172744751,
487
+ "eval_ndcg_CMedQAv1": 0.9562269449234009,
488
+ "eval_ndcg_CMedQAv2": 0.9615292549133301,
489
+ "eval_ndcg_Mmarco": 0.48261213302612305,
490
+ "eval_ndcg_T2Reranking": 0.8961272239685059,
491
+ "eval_runtime": 1052.5471,
492
+ "eval_samples_per_second": 380.055,
493
+ "eval_steps_per_second": 0.371,
494
+ "step": 2500
495
+ },
496
+ {
497
+ "epoch": 0.43,
498
+ "learning_rate": 2.694097747560465e-05,
499
+ "loss": 0.0093,
500
+ "step": 2550
501
+ },
502
+ {
503
+ "epoch": 0.44,
504
+ "learning_rate": 2.6818542328053576e-05,
505
+ "loss": 0.009,
506
+ "step": 2600
507
+ },
508
+ {
509
+ "epoch": 0.45,
510
+ "learning_rate": 2.66939946869489e-05,
511
+ "loss": 0.009,
512
+ "step": 2650
513
+ },
514
+ {
515
+ "epoch": 0.46,
516
+ "learning_rate": 2.6567356814434426e-05,
517
+ "loss": 0.0089,
518
+ "step": 2700
519
+ },
520
+ {
521
+ "epoch": 0.46,
522
+ "learning_rate": 2.6438651346270292e-05,
523
+ "loss": 0.0089,
524
+ "step": 2750
525
+ },
526
+ {
527
+ "epoch": 0.47,
528
+ "learning_rate": 2.630790128778696e-05,
529
+ "loss": 0.0089,
530
+ "step": 2800
531
+ },
532
+ {
533
+ "epoch": 0.48,
534
+ "learning_rate": 2.617513000977315e-05,
535
+ "loss": 0.009,
536
+ "step": 2850
537
+ },
538
+ {
539
+ "epoch": 0.49,
540
+ "learning_rate": 2.604036124429844e-05,
541
+ "loss": 0.0088,
542
+ "step": 2900
543
+ },
544
+ {
545
+ "epoch": 0.5,
546
+ "learning_rate": 2.590361908047132e-05,
547
+ "loss": 0.0091,
548
+ "step": 2950
549
+ },
550
+ {
551
+ "epoch": 0.51,
552
+ "learning_rate": 2.5764927960133396e-05,
553
+ "loss": 0.009,
554
+ "step": 3000
555
+ },
556
+ {
557
+ "epoch": 0.51,
558
+ "eval_ap_CMedQAv1": 0.8477064604578514,
559
+ "eval_ap_CMedQAv2": 0.8582835044558619,
560
+ "eval_ap_Mmarco": 0.31718590811535546,
561
+ "eval_ap_T2Reranking": 0.6816479742452222,
562
+ "eval_avg_ap": 0.6762059618185727,
563
+ "eval_loss": 0.12098982185125351,
564
+ "eval_mrr_CMedQAv1": 0.8740146825396825,
565
+ "eval_mrr_CMedQAv2": 0.8814178571428573,
566
+ "eval_mrr_Mmarco": 0.3068174603174603,
567
+ "eval_mrr_T2Reranking": 0.791166000365391,
568
+ "eval_ndcg@10_CMedQAv1": 0.9580147862434387,
569
+ "eval_ndcg@10_CMedQAv2": 0.9866949319839478,
570
+ "eval_ndcg@10_Mmarco": 0.13374407589435577,
571
+ "eval_ndcg@10_T2Reranking": 0.5563193559646606,
572
+ "eval_ndcg@1_CMedQAv1": 1.0,
573
+ "eval_ndcg@1_CMedQAv2": 1.0,
574
+ "eval_ndcg@1_Mmarco": 0.0,
575
+ "eval_ndcg@1_T2Reranking": 0.6533333659172058,
576
+ "eval_ndcg@3_CMedQAv1": 0.976535975933075,
577
+ "eval_ndcg@3_CMedQAv2": 0.9999998807907104,
578
+ "eval_ndcg@3_Mmarco": 0.11173196882009506,
579
+ "eval_ndcg@3_T2Reranking": 0.6220604777336121,
580
+ "eval_ndcg@5_CMedQAv1": 0.9553145170211792,
581
+ "eval_ndcg@5_CMedQAv2": 0.9999998807907104,
582
+ "eval_ndcg@5_Mmarco": 0.12382900714874268,
583
+ "eval_ndcg@5_T2Reranking": 0.5894810557365417,
584
+ "eval_ndcg_CMedQAv1": 0.9517234563827515,
585
+ "eval_ndcg_CMedQAv2": 0.9606796503067017,
586
+ "eval_ndcg_Mmarco": 0.39375704526901245,
587
+ "eval_ndcg_T2Reranking": 0.896885871887207,
588
+ "eval_runtime": 1062.2445,
589
+ "eval_samples_per_second": 376.586,
590
+ "eval_steps_per_second": 0.368,
591
+ "step": 3000
592
+ },
593
+ {
594
+ "epoch": 0.51,
595
+ "learning_rate": 2.5624312673490554e-05,
596
+ "loss": 0.0089,
597
+ "step": 3050
598
+ },
599
+ {
600
+ "epoch": 0.52,
601
+ "learning_rate": 2.5481798354681882e-05,
602
+ "loss": 0.0087,
603
+ "step": 3100
604
+ },
605
+ {
606
+ "epoch": 0.53,
607
+ "learning_rate": 2.5337410477287057e-05,
608
+ "loss": 0.0091,
609
+ "step": 3150
610
+ },
611
+ {
612
+ "epoch": 0.54,
613
+ "learning_rate": 2.5191174849773132e-05,
614
+ "loss": 0.0088,
615
+ "step": 3200
616
+ },
617
+ {
618
+ "epoch": 0.55,
619
+ "learning_rate": 2.5043117610881402e-05,
620
+ "loss": 0.0091,
621
+ "step": 3250
622
+ },
623
+ {
624
+ "epoch": 0.56,
625
+ "learning_rate": 2.4893265224955276e-05,
626
+ "loss": 0.0089,
627
+ "step": 3300
628
+ },
629
+ {
630
+ "epoch": 0.56,
631
+ "learning_rate": 2.4741644477209923e-05,
632
+ "loss": 0.0088,
633
+ "step": 3350
634
+ },
635
+ {
636
+ "epoch": 0.57,
637
+ "learning_rate": 2.4588282468944582e-05,
638
+ "loss": 0.0088,
639
+ "step": 3400
640
+ },
641
+ {
642
+ "epoch": 0.58,
643
+ "learning_rate": 2.4433206612698367e-05,
644
+ "loss": 0.0089,
645
+ "step": 3450
646
+ },
647
+ {
648
+ "epoch": 0.59,
649
+ "learning_rate": 2.4276444627350437e-05,
650
+ "loss": 0.0089,
651
+ "step": 3500
652
+ },
653
+ {
654
+ "epoch": 0.59,
655
+ "eval_ap_CMedQAv1": 0.8523776953190774,
656
+ "eval_ap_CMedQAv2": 0.8611607089962926,
657
+ "eval_ap_Mmarco": 0.35086797584501545,
658
+ "eval_ap_T2Reranking": 0.6760057559100935,
659
+ "eval_avg_ap": 0.6851030340176197,
660
+ "eval_loss": 0.12173164635896683,
661
+ "eval_mrr_CMedQAv1": 0.8750813492063492,
662
+ "eval_mrr_CMedQAv2": 0.8850503968253968,
663
+ "eval_mrr_Mmarco": 0.34225793650793657,
664
+ "eval_mrr_T2Reranking": 0.788319644603497,
665
+ "eval_ndcg@10_CMedQAv1": 0.9706858396530151,
666
+ "eval_ndcg@10_CMedQAv2": 1.0000001192092896,
667
+ "eval_ndcg@10_Mmarco": 0.16307643055915833,
668
+ "eval_ndcg@10_T2Reranking": 0.5981327891349792,
669
+ "eval_ndcg@1_CMedQAv1": 1.0,
670
+ "eval_ndcg@1_CMedQAv2": 1.0,
671
+ "eval_ndcg@1_Mmarco": 0.20000000298023224,
672
+ "eval_ndcg@1_T2Reranking": 0.6888889670372009,
673
+ "eval_ndcg@3_CMedQAv1": 0.9703917503356934,
674
+ "eval_ndcg@3_CMedQAv2": 0.9999998807907104,
675
+ "eval_ndcg@3_Mmarco": 0.17653605341911316,
676
+ "eval_ndcg@3_T2Reranking": 0.6413534879684448,
677
+ "eval_ndcg@5_CMedQAv1": 0.978601336479187,
678
+ "eval_ndcg@5_CMedQAv2": 0.9999998807907104,
679
+ "eval_ndcg@5_Mmarco": 0.1764817237854004,
680
+ "eval_ndcg@5_T2Reranking": 0.6482794284820557,
681
+ "eval_ndcg_CMedQAv1": 0.9572161436080933,
682
+ "eval_ndcg_CMedQAv2": 0.9616801142692566,
683
+ "eval_ndcg_Mmarco": 0.4346458315849304,
684
+ "eval_ndcg_T2Reranking": 0.8978231549263,
685
+ "eval_runtime": 1064.5644,
686
+ "eval_samples_per_second": 375.765,
687
+ "eval_steps_per_second": 0.367,
688
+ "step": 3500
689
+ },
690
+ {
691
+ "epoch": 0.6,
692
+ "learning_rate": 2.4118024533165415e-05,
693
+ "loss": 0.0089,
694
+ "step": 3550
695
+ },
696
+ {
697
+ "epoch": 0.61,
698
+ "learning_rate": 2.3957974646784935e-05,
699
+ "loss": 0.0085,
700
+ "step": 3600
701
+ },
702
+ {
703
+ "epoch": 0.62,
704
+ "learning_rate": 2.379632357616621e-05,
705
+ "loss": 0.0087,
706
+ "step": 3650
707
+ },
708
+ {
709
+ "epoch": 0.62,
710
+ "learning_rate": 2.363310021546853e-05,
711
+ "loss": 0.0085,
712
+ "step": 3700
713
+ },
714
+ {
715
+ "epoch": 0.63,
716
+ "learning_rate": 2.3468333739888613e-05,
717
+ "loss": 0.0087,
718
+ "step": 3750
719
+ },
720
+ {
721
+ "epoch": 0.64,
722
+ "learning_rate": 2.3302053600445695e-05,
723
+ "loss": 0.0088,
724
+ "step": 3800
725
+ },
726
+ {
727
+ "epoch": 0.65,
728
+ "learning_rate": 2.313428951871735e-05,
729
+ "loss": 0.0089,
730
+ "step": 3850
731
+ },
732
+ {
733
+ "epoch": 0.66,
734
+ "learning_rate": 2.2965071481526943e-05,
735
+ "loss": 0.0084,
736
+ "step": 3900
737
+ },
738
+ {
739
+ "epoch": 0.67,
740
+ "learning_rate": 2.2794429735583658e-05,
741
+ "loss": 0.0085,
742
+ "step": 3950
743
+ },
744
+ {
745
+ "epoch": 0.67,
746
+ "learning_rate": 2.262239478207607e-05,
747
+ "loss": 0.0087,
748
+ "step": 4000
749
+ },
750
+ {
751
+ "epoch": 0.67,
752
+ "eval_ap_CMedQAv1": 0.8508535708843126,
753
+ "eval_ap_CMedQAv2": 0.8605212604516235,
754
+ "eval_ap_Mmarco": 0.3549766633801287,
755
+ "eval_ap_T2Reranking": 0.6834420037661381,
756
+ "eval_avg_ap": 0.6874483746205506,
757
+ "eval_loss": 0.12024065852165222,
758
+ "eval_mrr_CMedQAv1": 0.8741781746031745,
759
+ "eval_mrr_CMedQAv2": 0.8835730158730158,
760
+ "eval_mrr_Mmarco": 0.34038492063492065,
761
+ "eval_mrr_T2Reranking": 0.7928938834617574,
762
+ "eval_ndcg@10_CMedQAv1": 0.9930569529533386,
763
+ "eval_ndcg@10_CMedQAv2": 0.9854609370231628,
764
+ "eval_ndcg@10_Mmarco": 0.13490335643291473,
765
+ "eval_ndcg@10_T2Reranking": 0.6463578939437866,
766
+ "eval_ndcg@1_CMedQAv1": 1.0,
767
+ "eval_ndcg@1_CMedQAv2": 1.0,
768
+ "eval_ndcg@1_Mmarco": 0.10000000149011612,
769
+ "eval_ndcg@1_T2Reranking": 0.6644444465637207,
770
+ "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
771
+ "eval_ndcg@3_CMedQAv2": 0.9999998807907104,
772
+ "eval_ndcg@3_Mmarco": 0.15921637415885925,
773
+ "eval_ndcg@3_T2Reranking": 0.688578188419342,
774
+ "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
775
+ "eval_ndcg@5_CMedQAv2": 0.9934396743774414,
776
+ "eval_ndcg@5_Mmarco": 0.14279724657535553,
777
+ "eval_ndcg@5_T2Reranking": 0.6808897852897644,
778
+ "eval_ndcg_CMedQAv1": 0.9591971635818481,
779
+ "eval_ndcg_CMedQAv2": 0.9625831842422485,
780
+ "eval_ndcg_Mmarco": 0.41239920258522034,
781
+ "eval_ndcg_T2Reranking": 0.8961763381958008,
782
+ "eval_runtime": 1074.1598,
783
+ "eval_samples_per_second": 372.408,
784
+ "eval_steps_per_second": 0.364,
785
+ "step": 4000
786
+ },
787
+ {
788
+ "epoch": 0.68,
789
+ "learning_rate": 2.2448997371220256e-05,
790
+ "loss": 0.0088,
791
+ "step": 4050
792
+ },
793
+ {
794
+ "epoch": 0.69,
795
+ "learning_rate": 2.2274268496763367e-05,
796
+ "loss": 0.0085,
797
+ "step": 4100
798
+ },
799
+ {
800
+ "epoch": 0.7,
801
+ "learning_rate": 2.2098239390443697e-05,
802
+ "loss": 0.0085,
803
+ "step": 4150
804
+ },
805
+ {
806
+ "epoch": 0.71,
807
+ "learning_rate": 2.192094151640817e-05,
808
+ "loss": 0.0085,
809
+ "step": 4200
810
+ },
811
+ {
812
+ "epoch": 0.72,
813
+ "learning_rate": 2.174240656558834e-05,
814
+ "loss": 0.0084,
815
+ "step": 4250
816
+ },
817
+ {
818
+ "epoch": 0.72,
819
+ "learning_rate": 2.156266645003582e-05,
820
+ "loss": 0.0082,
821
+ "step": 4300
822
+ },
823
+ {
824
+ "epoch": 0.73,
825
+ "learning_rate": 2.1381753297218183e-05,
826
+ "loss": 0.0083,
827
+ "step": 4350
828
+ },
829
+ {
830
+ "epoch": 0.74,
831
+ "learning_rate": 2.1199699444276374e-05,
832
+ "loss": 0.0082,
833
+ "step": 4400
834
+ },
835
+ {
836
+ "epoch": 0.75,
837
+ "learning_rate": 2.1016537432244663e-05,
838
+ "loss": 0.0081,
839
+ "step": 4450
840
+ },
841
+ {
842
+ "epoch": 0.76,
843
+ "learning_rate": 2.0832300000234076e-05,
844
+ "loss": 0.0083,
845
+ "step": 4500
846
+ },
847
+ {
848
+ "epoch": 0.76,
849
+ "eval_ap_CMedQAv1": 0.8549180059460657,
850
+ "eval_ap_CMedQAv2": 0.8599012485902339,
851
+ "eval_ap_Mmarco": 0.31690584463403754,
852
+ "eval_ap_T2Reranking": 0.6831123214291062,
853
+ "eval_avg_ap": 0.6787093551498609,
854
+ "eval_loss": 0.12336914986371994,
855
+ "eval_mrr_CMedQAv1": 0.8789436507936508,
856
+ "eval_mrr_CMedQAv2": 0.8820293650793651,
857
+ "eval_mrr_Mmarco": 0.30601190476190476,
858
+ "eval_mrr_T2Reranking": 0.7921361672631138,
859
+ "eval_ndcg@10_CMedQAv1": 0.9862348437309265,
860
+ "eval_ndcg@10_CMedQAv2": 0.983578085899353,
861
+ "eval_ndcg@10_Mmarco": 0.1612614393234253,
862
+ "eval_ndcg@10_T2Reranking": 0.6136462092399597,
863
+ "eval_ndcg@1_CMedQAv1": 1.0,
864
+ "eval_ndcg@1_CMedQAv2": 1.0,
865
+ "eval_ndcg@1_Mmarco": 0.30000001192092896,
866
+ "eval_ndcg@1_T2Reranking": 0.6665303111076355,
867
+ "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
868
+ "eval_ndcg@3_CMedQAv2": 0.9999998807907104,
869
+ "eval_ndcg@3_Mmarco": 0.22346392273902893,
870
+ "eval_ndcg@3_T2Reranking": 0.6385782957077026,
871
+ "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
872
+ "eval_ndcg@5_CMedQAv2": 0.9853931665420532,
873
+ "eval_ndcg@5_Mmarco": 0.20383748412132263,
874
+ "eval_ndcg@5_T2Reranking": 0.6306164860725403,
875
+ "eval_ndcg_CMedQAv1": 0.9580343961715698,
876
+ "eval_ndcg_CMedQAv2": 0.9610058069229126,
877
+ "eval_ndcg_Mmarco": 0.4440736174583435,
878
+ "eval_ndcg_T2Reranking": 0.8990576863288879,
879
+ "eval_runtime": 1049.34,
880
+ "eval_samples_per_second": 381.217,
881
+ "eval_steps_per_second": 0.373,
882
+ "step": 4500
883
+ },
884
+ {
885
+ "epoch": 0.77,
886
+ "learning_rate": 2.0647020079580543e-05,
887
+ "loss": 0.0081,
888
+ "step": 4550
889
+ },
890
+ {
891
+ "epoch": 0.78,
892
+ "learning_rate": 2.0460730787958573e-05,
893
+ "loss": 0.0082,
894
+ "step": 4600
895
+ },
896
+ {
897
+ "epoch": 0.78,
898
+ "learning_rate": 2.0273465423461677e-05,
899
+ "loss": 0.0084,
900
+ "step": 4650
901
+ },
902
+ {
903
+ "epoch": 0.79,
904
+ "learning_rate": 2.008525745865055e-05,
905
+ "loss": 0.0081,
906
+ "step": 4700
907
+ },
908
+ {
909
+ "epoch": 0.8,
910
+ "learning_rate": 1.989614053457002e-05,
911
+ "loss": 0.0081,
912
+ "step": 4750
913
+ },
914
+ {
915
+ "epoch": 0.81,
916
+ "learning_rate": 1.970614845473596e-05,
917
+ "loss": 0.0082,
918
+ "step": 4800
919
+ },
920
+ {
921
+ "epoch": 0.82,
922
+ "learning_rate": 1.9515315179093052e-05,
923
+ "loss": 0.0081,
924
+ "step": 4850
925
+ },
926
+ {
927
+ "epoch": 0.83,
928
+ "learning_rate": 1.9323674817944713e-05,
929
+ "loss": 0.0081,
930
+ "step": 4900
931
+ },
932
+ {
933
+ "epoch": 0.83,
934
+ "learning_rate": 1.9131261625856034e-05,
935
+ "loss": 0.0082,
936
+ "step": 4950
937
+ },
938
+ {
939
+ "epoch": 0.84,
940
+ "learning_rate": 1.8938109995531015e-05,
941
+ "loss": 0.0081,
942
+ "step": 5000
943
+ },
944
+ {
945
+ "epoch": 0.84,
946
+ "eval_ap_CMedQAv1": 0.8580422117025617,
947
+ "eval_ap_CMedQAv2": 0.8637152440470349,
948
+ "eval_ap_Mmarco": 0.31369636069838835,
949
+ "eval_ap_T2Reranking": 0.690709122210063,
950
+ "eval_avg_ap": 0.6815407346645119,
951
+ "eval_loss": 0.12387344986200333,
952
+ "eval_mrr_CMedQAv1": 0.8822130952380952,
953
+ "eval_mrr_CMedQAv2": 0.887097619047619,
954
+ "eval_mrr_Mmarco": 0.30120238095238094,
955
+ "eval_mrr_T2Reranking": 0.8040100321329163,
956
+ "eval_ndcg@10_CMedQAv1": 0.9928603172302246,
957
+ "eval_ndcg@10_CMedQAv2": 0.9926636815071106,
958
+ "eval_ndcg@10_Mmarco": 0.1475120484828949,
959
+ "eval_ndcg@10_T2Reranking": 0.6204460859298706,
960
+ "eval_ndcg@1_CMedQAv1": 1.0,
961
+ "eval_ndcg@1_CMedQAv2": 1.0,
962
+ "eval_ndcg@1_Mmarco": 0.10000000149011612,
963
+ "eval_ndcg@1_T2Reranking": 0.7071110606193542,
964
+ "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
965
+ "eval_ndcg@3_CMedQAv2": 0.9999998807907104,
966
+ "eval_ndcg@3_Mmarco": 0.212288498878479,
967
+ "eval_ndcg@3_T2Reranking": 0.6500719785690308,
968
+ "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
969
+ "eval_ndcg@5_CMedQAv2": 0.9999998807907104,
970
+ "eval_ndcg@5_Mmarco": 0.19501754641532898,
971
+ "eval_ndcg@5_T2Reranking": 0.6329637169837952,
972
+ "eval_ndcg_CMedQAv1": 0.9593874216079712,
973
+ "eval_ndcg_CMedQAv2": 0.9627755284309387,
974
+ "eval_ndcg_Mmarco": 0.4310723841190338,
975
+ "eval_ndcg_T2Reranking": 0.9022238850593567,
976
+ "eval_runtime": 1077.7629,
977
+ "eval_samples_per_second": 371.163,
978
+ "eval_steps_per_second": 0.363,
979
+ "step": 5000
980
+ },
981
+ {
982
+ "epoch": 0.85,
983
+ "learning_rate": 1.8744254451665046e-05,
984
+ "loss": 0.0081,
985
+ "step": 5050
986
+ },
987
+ {
988
+ "epoch": 0.86,
989
+ "learning_rate": 1.854972964477386e-05,
990
+ "loss": 0.0079,
991
+ "step": 5100
992
+ },
993
+ {
994
+ "epoch": 0.87,
995
+ "learning_rate": 1.835457034499991e-05,
996
+ "loss": 0.0083,
997
+ "step": 5150
998
+ },
999
+ {
1000
+ "epoch": 0.88,
1001
+ "learning_rate": 1.8158811435897493e-05,
1002
+ "loss": 0.0081,
1003
+ "step": 5200
1004
+ },
1005
+ {
1006
+ "epoch": 0.88,
1007
+ "learning_rate": 1.7962487908197434e-05,
1008
+ "loss": 0.008,
1009
+ "step": 5250
1010
+ },
1011
+ {
1012
+ "epoch": 0.89,
1013
+ "learning_rate": 1.7765634853552764e-05,
1014
+ "loss": 0.0079,
1015
+ "step": 5300
1016
+ },
1017
+ {
1018
+ "epoch": 0.9,
1019
+ "learning_rate": 1.7568287458266282e-05,
1020
+ "loss": 0.0079,
1021
+ "step": 5350
1022
+ },
1023
+ {
1024
+ "epoch": 0.91,
1025
+ "learning_rate": 1.7370480997001206e-05,
1026
+ "loss": 0.0078,
1027
+ "step": 5400
1028
+ },
1029
+ {
1030
+ "epoch": 0.92,
1031
+ "learning_rate": 1.717225082647604e-05,
1032
+ "loss": 0.008,
1033
+ "step": 5450
1034
+ },
1035
+ {
1036
+ "epoch": 0.93,
1037
+ "learning_rate": 1.6973632379144785e-05,
1038
+ "loss": 0.008,
1039
+ "step": 5500
1040
+ },
1041
+ {
1042
+ "epoch": 0.93,
1043
+ "eval_ap_CMedQAv1": 0.8627846587717232,
1044
+ "eval_ap_CMedQAv2": 0.8679932179664334,
1045
+ "eval_ap_Mmarco": 0.34792586376372187,
1046
+ "eval_ap_T2Reranking": 0.6833881748363431,
1047
+ "eval_avg_ap": 0.6905229788345554,
1048
+ "eval_loss": 0.1255130171775818,
1049
+ "eval_mrr_CMedQAv1": 0.8866321428571429,
1050
+ "eval_mrr_CMedQAv2": 0.8911746031746032,
1051
+ "eval_mrr_Mmarco": 0.3351349206349207,
1052
+ "eval_mrr_T2Reranking": 0.7994619214194367,
1053
+ "eval_ndcg@10_CMedQAv1": 0.9803870916366577,
1054
+ "eval_ndcg@10_CMedQAv2": 0.992663562297821,
1055
+ "eval_ndcg@10_Mmarco": 0.18294155597686768,
1056
+ "eval_ndcg@10_T2Reranking": 0.5986461043357849,
1057
+ "eval_ndcg@1_CMedQAv1": 1.0,
1058
+ "eval_ndcg@1_CMedQAv2": 1.0,
1059
+ "eval_ndcg@1_Mmarco": 0.30000001192092896,
1060
+ "eval_ndcg@1_T2Reranking": 0.6328888535499573,
1061
+ "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
1062
+ "eval_ndcg@3_CMedQAv2": 0.9999998807907104,
1063
+ "eval_ndcg@3_Mmarco": 0.24078361690044403,
1064
+ "eval_ndcg@3_T2Reranking": 0.6359131336212158,
1065
+ "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
1066
+ "eval_ndcg@5_CMedQAv2": 0.9999998807907104,
1067
+ "eval_ndcg@5_Mmarco": 0.21561172604560852,
1068
+ "eval_ndcg@5_T2Reranking": 0.6260303258895874,
1069
+ "eval_ndcg_CMedQAv1": 0.9603596925735474,
1070
+ "eval_ndcg_CMedQAv2": 0.9649822115898132,
1071
+ "eval_ndcg_Mmarco": 0.46445542573928833,
1072
+ "eval_ndcg_T2Reranking": 0.8971077799797058,
1073
+ "eval_runtime": 1115.45,
1074
+ "eval_samples_per_second": 358.623,
1075
+ "eval_steps_per_second": 0.351,
1076
+ "step": 5500
1077
+ },
1078
+ {
1079
+ "epoch": 0.94,
1080
+ "learning_rate": 1.677466115686359e-05,
1081
+ "loss": 0.0077,
1082
+ "step": 5550
1083
+ },
1084
+ {
1085
+ "epoch": 0.94,
1086
+ "learning_rate": 1.6575372724545014e-05,
1087
+ "loss": 0.0079,
1088
+ "step": 5600
1089
+ },
1090
+ {
1091
+ "epoch": 0.95,
1092
+ "learning_rate": 1.6375802703801003e-05,
1093
+ "loss": 0.008,
1094
+ "step": 5650
1095
+ },
1096
+ {
1097
+ "epoch": 0.96,
1098
+ "learning_rate": 1.6175986766575735e-05,
1099
+ "loss": 0.0078,
1100
+ "step": 5700
1101
+ },
1102
+ {
1103
+ "epoch": 0.97,
1104
+ "learning_rate": 1.5975960628769506e-05,
1105
+ "loss": 0.0081,
1106
+ "step": 5750
1107
+ },
1108
+ {
1109
+ "epoch": 0.98,
1110
+ "learning_rate": 1.5775760043854687e-05,
1111
+ "loss": 0.0077,
1112
+ "step": 5800
1113
+ },
1114
+ {
1115
+ "epoch": 0.99,
1116
+ "learning_rate": 1.5575420796485038e-05,
1117
+ "loss": 0.008,
1118
+ "step": 5850
1119
+ },
1120
+ {
1121
+ "epoch": 0.99,
1122
+ "learning_rate": 1.5374978696099378e-05,
1123
+ "loss": 0.0078,
1124
+ "step": 5900
1125
+ },
1126
+ {
1127
+ "epoch": 1.0,
1128
+ "learning_rate": 1.5174469570520917e-05,
1129
+ "loss": 0.0074,
1130
+ "step": 5950
1131
+ },
1132
+ {
1133
+ "epoch": 1.01,
1134
+ "learning_rate": 1.4973929259553187e-05,
1135
+ "loss": 0.0063,
1136
+ "step": 6000
1137
+ },
1138
+ {
1139
+ "epoch": 1.01,
1140
+ "eval_ap_CMedQAv1": 0.8625306376678427,
1141
+ "eval_ap_CMedQAv2": 0.8664329707401012,
1142
+ "eval_ap_Mmarco": 0.34889147093898054,
1143
+ "eval_ap_T2Reranking": 0.6829168328093741,
1144
+ "eval_avg_ap": 0.6901929780390745,
1145
+ "eval_loss": 0.12305427342653275,
1146
+ "eval_mrr_CMedQAv1": 0.8871357142857142,
1147
+ "eval_mrr_CMedQAv2": 0.8890535714285714,
1148
+ "eval_mrr_Mmarco": 0.3384365079365079,
1149
+ "eval_mrr_T2Reranking": 0.7948461058989156,
1150
+ "eval_ndcg@10_CMedQAv1": 0.9933746457099915,
1151
+ "eval_ndcg@10_CMedQAv2": 0.978777289390564,
1152
+ "eval_ndcg@10_Mmarco": 0.15859688818454742,
1153
+ "eval_ndcg@10_T2Reranking": 0.5381678938865662,
1154
+ "eval_ndcg@1_CMedQAv1": 1.0,
1155
+ "eval_ndcg@1_CMedQAv2": 1.0,
1156
+ "eval_ndcg@1_Mmarco": 0.20000000298023224,
1157
+ "eval_ndcg@1_T2Reranking": 0.4707619249820709,
1158
+ "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
1159
+ "eval_ndcg@3_CMedQAv2": 0.9703917503356934,
1160
+ "eval_ndcg@3_Mmarco": 0.17039181292057037,
1161
+ "eval_ndcg@3_T2Reranking": 0.4994679093360901,
1162
+ "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
1163
+ "eval_ndcg@5_CMedQAv2": 0.978601336479187,
1164
+ "eval_ndcg@5_Mmarco": 0.18008770048618317,
1165
+ "eval_ndcg@5_T2Reranking": 0.5133092999458313,
1166
+ "eval_ndcg_CMedQAv1": 0.9596377611160278,
1167
+ "eval_ndcg_CMedQAv2": 0.9619806408882141,
1168
+ "eval_ndcg_Mmarco": 0.4373508393764496,
1169
+ "eval_ndcg_T2Reranking": 0.8974445462226868,
1170
+ "eval_runtime": 1135.765,
1171
+ "eval_samples_per_second": 352.208,
1172
+ "eval_steps_per_second": 0.344,
1173
+ "step": 6000
1174
+ },
1175
+ {
1176
+ "epoch": 1.02,
1177
+ "learning_rate": 1.4773393608573946e-05,
1178
+ "loss": 0.0064,
1179
+ "step": 6050
1180
+ },
1181
+ {
1182
+ "epoch": 1.03,
1183
+ "learning_rate": 1.4572898462127985e-05,
1184
+ "loss": 0.0066,
1185
+ "step": 6100
1186
+ },
1187
+ {
1188
+ "epoch": 1.04,
1189
+ "learning_rate": 1.437247965752017e-05,
1190
+ "loss": 0.0066,
1191
+ "step": 6150
1192
+ },
1193
+ {
1194
+ "epoch": 1.04,
1195
+ "learning_rate": 1.4172173018409708e-05,
1196
+ "loss": 0.0066,
1197
+ "step": 6200
1198
+ },
1199
+ {
1200
+ "epoch": 1.05,
1201
+ "learning_rate": 1.3972014348406904e-05,
1202
+ "loss": 0.0067,
1203
+ "step": 6250
1204
+ },
1205
+ {
1206
+ "epoch": 1.06,
1207
+ "learning_rate": 1.377203942467347e-05,
1208
+ "loss": 0.0064,
1209
+ "step": 6300
1210
+ },
1211
+ {
1212
+ "epoch": 1.07,
1213
+ "learning_rate": 1.3572283991527582e-05,
1214
+ "loss": 0.0064,
1215
+ "step": 6350
1216
+ },
1217
+ {
1218
+ "epoch": 1.08,
1219
+ "learning_rate": 1.3372783754054776e-05,
1220
+ "loss": 0.0064,
1221
+ "step": 6400
1222
+ },
1223
+ {
1224
+ "epoch": 1.09,
1225
+ "learning_rate": 1.3173574371725902e-05,
1226
+ "loss": 0.0064,
1227
+ "step": 6450
1228
+ },
1229
+ {
1230
+ "epoch": 1.1,
1231
+ "learning_rate": 1.2974691452023195e-05,
1232
+ "loss": 0.0065,
1233
+ "step": 6500
1234
+ },
1235
+ {
1236
+ "epoch": 1.1,
1237
+ "eval_ap_CMedQAv1": 0.8620322442191521,
1238
+ "eval_ap_CMedQAv2": 0.8704041872289263,
1239
+ "eval_ap_Mmarco": 0.355717135660246,
1240
+ "eval_ap_T2Reranking": 0.6809892589587305,
1241
+ "eval_avg_ap": 0.6922857065167638,
1242
+ "eval_loss": 0.12283767759799957,
1243
+ "eval_mrr_CMedQAv1": 0.8863039682539682,
1244
+ "eval_mrr_CMedQAv2": 0.8930746031746032,
1245
+ "eval_mrr_Mmarco": 0.34052380952380956,
1246
+ "eval_mrr_T2Reranking": 0.7916278438705656,
1247
+ "eval_ndcg@10_CMedQAv1": 0.989758312702179,
1248
+ "eval_ndcg@10_CMedQAv2": 0.983578085899353,
1249
+ "eval_ndcg@10_Mmarco": 0.16711477935314178,
1250
+ "eval_ndcg@10_T2Reranking": 0.5755314230918884,
1251
+ "eval_ndcg@1_CMedQAv1": 1.0,
1252
+ "eval_ndcg@1_CMedQAv2": 1.0,
1253
+ "eval_ndcg@1_Mmarco": 0.10000000149011612,
1254
+ "eval_ndcg@1_T2Reranking": 0.5387619137763977,
1255
+ "eval_ndcg@3_CMedQAv1": 0.9882680177688599,
1256
+ "eval_ndcg@3_CMedQAv2": 0.9999998807907104,
1257
+ "eval_ndcg@3_Mmarco": 0.1265360563993454,
1258
+ "eval_ndcg@3_T2Reranking": 0.566280722618103,
1259
+ "eval_ndcg@5_CMedQAv1": 0.9842175245285034,
1260
+ "eval_ndcg@5_CMedQAv2": 0.9853931665420532,
1261
+ "eval_ndcg@5_Mmarco": 0.15853983163833618,
1262
+ "eval_ndcg@5_T2Reranking": 0.5730277299880981,
1263
+ "eval_ndcg_CMedQAv1": 0.9608818292617798,
1264
+ "eval_ndcg_CMedQAv2": 0.9639202952384949,
1265
+ "eval_ndcg_Mmarco": 0.4360330104827881,
1266
+ "eval_ndcg_T2Reranking": 0.8998947143554688,
1267
+ "eval_runtime": 1071.1802,
1268
+ "eval_samples_per_second": 373.444,
1269
+ "eval_steps_per_second": 0.365,
1270
+ "step": 6500
1271
+ },
1272
+ {
1273
+ "epoch": 1.1,
1274
+ "learning_rate": 1.277617054407565e-05,
1275
+ "loss": 0.0066,
1276
+ "step": 6550
1277
+ },
1278
+ {
1279
+ "epoch": 1.11,
1280
+ "learning_rate": 1.2578047132304843e-05,
1281
+ "loss": 0.0065,
1282
+ "step": 6600
1283
+ },
1284
+ {
1285
+ "epoch": 1.12,
1286
+ "learning_rate": 1.2380356630082277e-05,
1287
+ "loss": 0.0064,
1288
+ "step": 6650
1289
+ },
1290
+ {
1291
+ "epoch": 1.13,
1292
+ "learning_rate": 1.2183134373399479e-05,
1293
+ "loss": 0.0066,
1294
+ "step": 6700
1295
+ },
1296
+ {
1297
+ "epoch": 1.14,
1298
+ "learning_rate": 1.1986415614551897e-05,
1299
+ "loss": 0.0068,
1300
+ "step": 6750
1301
+ },
1302
+ {
1303
+ "epoch": 1.15,
1304
+ "learning_rate": 1.1790235515837761e-05,
1305
+ "loss": 0.0065,
1306
+ "step": 6800
1307
+ },
1308
+ {
1309
+ "epoch": 1.15,
1310
+ "learning_rate": 1.1594629143273021e-05,
1311
+ "loss": 0.0067,
1312
+ "step": 6850
1313
+ },
1314
+ {
1315
+ "epoch": 1.16,
1316
+ "learning_rate": 1.1399631460323536e-05,
1317
+ "loss": 0.0066,
1318
+ "step": 6900
1319
+ },
1320
+ {
1321
+ "epoch": 1.17,
1322
+ "learning_rate": 1.1205277321655528e-05,
1323
+ "loss": 0.0065,
1324
+ "step": 6950
1325
+ },
1326
+ {
1327
+ "epoch": 1.18,
1328
+ "learning_rate": 1.1011601466905561e-05,
1329
+ "loss": 0.0065,
1330
+ "step": 7000
1331
+ },
1332
+ {
1333
+ "epoch": 1.18,
1334
+ "eval_ap_CMedQAv1": 0.856229674863834,
1335
+ "eval_ap_CMedQAv2": 0.8728030227219203,
1336
+ "eval_ap_Mmarco": 0.35545981895349504,
1337
+ "eval_ap_T2Reranking": 0.6844460182092421,
1338
+ "eval_avg_ap": 0.6922346336871228,
1339
+ "eval_loss": 0.12674953043460846,
1340
+ "eval_mrr_CMedQAv1": 0.8826107142857142,
1341
+ "eval_mrr_CMedQAv2": 0.8945480158730159,
1342
+ "eval_mrr_Mmarco": 0.34125,
1343
+ "eval_mrr_T2Reranking": 0.7965651900570655,
1344
+ "eval_ndcg@10_CMedQAv1": 0.9865903854370117,
1345
+ "eval_ndcg@10_CMedQAv2": 0.9811555743217468,
1346
+ "eval_ndcg@10_Mmarco": 0.1592722088098526,
1347
+ "eval_ndcg@10_T2Reranking": 0.5544707179069519,
1348
+ "eval_ndcg@1_CMedQAv1": 1.0,
1349
+ "eval_ndcg@1_CMedQAv2": 1.0,
1350
+ "eval_ndcg@1_Mmarco": 0.25,
1351
+ "eval_ndcg@1_T2Reranking": 0.5022221803665161,
1352
+ "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
1353
+ "eval_ndcg@3_CMedQAv2": 0.976535975933075,
1354
+ "eval_ndcg@3_Mmarco": 0.2086598426103592,
1355
+ "eval_ndcg@3_T2Reranking": 0.5088227987289429,
1356
+ "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
1357
+ "eval_ndcg@5_CMedQAv2": 0.9830419421195984,
1358
+ "eval_ndcg@5_Mmarco": 0.17853133380413055,
1359
+ "eval_ndcg@5_T2Reranking": 0.5329388380050659,
1360
+ "eval_ndcg_CMedQAv1": 0.9582996368408203,
1361
+ "eval_ndcg_CMedQAv2": 0.962376594543457,
1362
+ "eval_ndcg_Mmarco": 0.4456784129142761,
1363
+ "eval_ndcg_T2Reranking": 0.9012719988822937,
1364
+ "eval_runtime": 1104.857,
1365
+ "eval_samples_per_second": 362.061,
1366
+ "eval_steps_per_second": 0.354,
1367
+ "step": 7000
1368
+ },
1369
+ {
1370
+ "epoch": 1.19,
1371
+ "learning_rate": 1.0818638514470987e-05,
1372
+ "loss": 0.0066,
1373
+ "step": 7050
1374
+ },
1375
+ {
1376
+ "epoch": 1.2,
1377
+ "learning_rate": 1.0626422955322185e-05,
1378
+ "loss": 0.0066,
1379
+ "step": 7100
1380
+ },
1381
+ {
1382
+ "epoch": 1.2,
1383
+ "learning_rate": 1.0434989146837435e-05,
1384
+ "loss": 0.0065,
1385
+ "step": 7150
1386
+ },
1387
+ {
1388
+ "epoch": 1.21,
1389
+ "learning_rate": 1.0244371306661786e-05,
1390
+ "loss": 0.0066,
1391
+ "step": 7200
1392
+ },
1393
+ {
1394
+ "epoch": 1.22,
1395
+ "learning_rate": 1.0054603506590841e-05,
1396
+ "loss": 0.0065,
1397
+ "step": 7250
1398
+ },
1399
+ {
1400
+ "epoch": 1.23,
1401
+ "learning_rate": 9.865719666480642e-06,
1402
+ "loss": 0.0064,
1403
+ "step": 7300
1404
+ },
1405
+ {
1406
+ "epoch": 1.24,
1407
+ "learning_rate": 9.677753548184684e-06,
1408
+ "loss": 0.0067,
1409
+ "step": 7350
1410
+ },
1411
+ {
1412
+ "epoch": 1.25,
1413
+ "learning_rate": 9.490738749519188e-06,
1414
+ "loss": 0.0065,
1415
+ "step": 7400
1416
+ },
1417
+ {
1418
+ "epoch": 1.26,
1419
+ "learning_rate": 9.30470869825771e-06,
1420
+ "loss": 0.0063,
1421
+ "step": 7450
1422
+ },
1423
+ {
1424
+ "epoch": 1.26,
1425
+ "learning_rate": 9.119696646156103e-06,
1426
+ "loss": 0.0066,
1427
+ "step": 7500
1428
+ },
1429
+ {
1430
+ "epoch": 1.26,
1431
+ "eval_ap_CMedQAv1": 0.8604172146745557,
1432
+ "eval_ap_CMedQAv2": 0.8725561316868456,
1433
+ "eval_ap_Mmarco": 0.3400242765180377,
1434
+ "eval_ap_T2Reranking": 0.6826462319596516,
1435
+ "eval_avg_ap": 0.6889109637097726,
1436
+ "eval_loss": 0.1242843046784401,
1437
+ "eval_mrr_CMedQAv1": 0.8834670634920635,
1438
+ "eval_mrr_CMedQAv2": 0.8959575396825398,
1439
+ "eval_mrr_Mmarco": 0.3298134920634921,
1440
+ "eval_mrr_T2Reranking": 0.7943680750341211,
1441
+ "eval_ndcg@10_CMedQAv1": 0.9862348437309265,
1442
+ "eval_ndcg@10_CMedQAv2": 0.9811555743217468,
1443
+ "eval_ndcg@10_Mmarco": 0.1542401760816574,
1444
+ "eval_ndcg@10_T2Reranking": 0.6371672749519348,
1445
+ "eval_ndcg@1_CMedQAv1": 1.0,
1446
+ "eval_ndcg@1_CMedQAv2": 1.0,
1447
+ "eval_ndcg@1_Mmarco": 0.10000000149011612,
1448
+ "eval_ndcg@1_T2Reranking": 0.7219048738479614,
1449
+ "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
1450
+ "eval_ndcg@3_CMedQAv2": 0.976535975933075,
1451
+ "eval_ndcg@3_Mmarco": 0.15921637415885925,
1452
+ "eval_ndcg@3_T2Reranking": 0.6977660655975342,
1453
+ "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
1454
+ "eval_ndcg@5_CMedQAv2": 0.9830419421195984,
1455
+ "eval_ndcg@5_Mmarco": 0.1712677776813507,
1456
+ "eval_ndcg@5_T2Reranking": 0.6631223559379578,
1457
+ "eval_ndcg_CMedQAv1": 0.9619690179824829,
1458
+ "eval_ndcg_CMedQAv2": 0.965739369392395,
1459
+ "eval_ndcg_Mmarco": 0.43817299604415894,
1460
+ "eval_ndcg_T2Reranking": 0.9018659591674805,
1461
+ "eval_runtime": 1054.3338,
1462
+ "eval_samples_per_second": 379.411,
1463
+ "eval_steps_per_second": 0.371,
1464
+ "step": 7500
1465
+ },
1466
+ {
1467
+ "epoch": 1.27,
1468
+ "learning_rate": 8.935735663008975e-06,
1469
+ "loss": 0.0065,
1470
+ "step": 7550
1471
+ },
1472
+ {
1473
+ "epoch": 1.28,
1474
+ "learning_rate": 8.752858630738673e-06,
1475
+ "loss": 0.0067,
1476
+ "step": 7600
1477
+ },
1478
+ {
1479
+ "epoch": 1.29,
1480
+ "learning_rate": 8.57109823751782e-06,
1481
+ "loss": 0.0063,
1482
+ "step": 7650
1483
+ },
1484
+ {
1485
+ "epoch": 1.3,
1486
+ "learning_rate": 8.390486971926502e-06,
1487
+ "loss": 0.0065,
1488
+ "step": 7700
1489
+ },
1490
+ {
1491
+ "epoch": 1.31,
1492
+ "learning_rate": 8.211057117145137e-06,
1493
+ "loss": 0.0063,
1494
+ "step": 7750
1495
+ },
1496
+ {
1497
+ "epoch": 1.31,
1498
+ "learning_rate": 8.03284074518405e-06,
1499
+ "loss": 0.0065,
1500
+ "step": 7800
1501
+ },
1502
+ {
1503
+ "epoch": 1.32,
1504
+ "learning_rate": 7.855869711150798e-06,
1505
+ "loss": 0.0066,
1506
+ "step": 7850
1507
+ },
1508
+ {
1509
+ "epoch": 1.33,
1510
+ "learning_rate": 7.680175647556236e-06,
1511
+ "loss": 0.0065,
1512
+ "step": 7900
1513
+ },
1514
+ {
1515
+ "epoch": 1.34,
1516
+ "learning_rate": 7.505789958660412e-06,
1517
+ "loss": 0.0065,
1518
+ "step": 7950
1519
+ },
1520
+ {
1521
+ "epoch": 1.35,
1522
+ "learning_rate": 7.332743814859266e-06,
1523
+ "loss": 0.0066,
1524
+ "step": 8000
1525
+ },
1526
+ {
1527
+ "epoch": 1.35,
1528
+ "eval_ap_CMedQAv1": 0.8599415671346007,
1529
+ "eval_ap_CMedQAv2": 0.8704569674862218,
1530
+ "eval_ap_Mmarco": 0.3520724310208005,
1531
+ "eval_ap_T2Reranking": 0.6867254999404075,
1532
+ "eval_avg_ap": 0.6922991163955077,
1533
+ "eval_loss": 0.1281006932258606,
1534
+ "eval_mrr_CMedQAv1": 0.8837103174603175,
1535
+ "eval_mrr_CMedQAv2": 0.8930769841269841,
1536
+ "eval_mrr_Mmarco": 0.3405515873015873,
1537
+ "eval_mrr_T2Reranking": 0.7984008097709858,
1538
+ "eval_ndcg@10_CMedQAv1": 0.9783665537834167,
1539
+ "eval_ndcg@10_CMedQAv2": 0.9797147512435913,
1540
+ "eval_ndcg@10_Mmarco": 0.1712549477815628,
1541
+ "eval_ndcg@10_T2Reranking": 0.5799515843391418,
1542
+ "eval_ndcg@1_CMedQAv1": 1.0,
1543
+ "eval_ndcg@1_CMedQAv2": 1.0,
1544
+ "eval_ndcg@1_Mmarco": 0.10000000149011612,
1545
+ "eval_ndcg@1_T2Reranking": 0.5411046147346497,
1546
+ "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
1547
+ "eval_ndcg@3_CMedQAv2": 0.9734638929367065,
1548
+ "eval_ndcg@3_Mmarco": 0.22653606534004211,
1549
+ "eval_ndcg@3_T2Reranking": 0.568535327911377,
1550
+ "eval_ndcg@5_CMedQAv1": 0.9868794679641724,
1551
+ "eval_ndcg@5_CMedQAv2": 0.9808216094970703,
1552
+ "eval_ndcg@5_Mmarco": 0.18608656525611877,
1553
+ "eval_ndcg@5_T2Reranking": 0.5744965672492981,
1554
+ "eval_ndcg_CMedQAv1": 0.9611911773681641,
1555
+ "eval_ndcg_CMedQAv2": 0.9657419323921204,
1556
+ "eval_ndcg_Mmarco": 0.4481244683265686,
1557
+ "eval_ndcg_T2Reranking": 0.9010647535324097,
1558
+ "eval_runtime": 1049.8477,
1559
+ "eval_samples_per_second": 381.032,
1560
+ "eval_steps_per_second": 0.372,
1561
+ "step": 8000
1562
+ },
1563
+ {
1564
+ "epoch": 1.36,
1565
+ "learning_rate": 7.161068147113065e-06,
1566
+ "loss": 0.0066,
1567
+ "step": 8050
1568
+ },
1569
+ {
1570
+ "epoch": 1.37,
1571
+ "learning_rate": 6.990793641417708e-06,
1572
+ "loss": 0.0065,
1573
+ "step": 8100
1574
+ },
1575
+ {
1576
+ "epoch": 1.37,
1577
+ "learning_rate": 6.821950733319783e-06,
1578
+ "loss": 0.0064,
1579
+ "step": 8150
1580
+ },
1581
+ {
1582
+ "epoch": 1.38,
1583
+ "learning_rate": 6.654569602476402e-06,
1584
+ "loss": 0.0064,
1585
+ "step": 8200
1586
+ },
1587
+ {
1588
+ "epoch": 1.39,
1589
+ "learning_rate": 6.488680167260749e-06,
1590
+ "loss": 0.0067,
1591
+ "step": 8250
1592
+ },
1593
+ {
1594
+ "epoch": 1.4,
1595
+ "learning_rate": 6.324312079414362e-06,
1596
+ "loss": 0.0066,
1597
+ "step": 8300
1598
+ },
1599
+ {
1600
+ "epoch": 1.41,
1601
+ "learning_rate": 6.161494718747061e-06,
1602
+ "loss": 0.0067,
1603
+ "step": 8350
1604
+ },
1605
+ {
1606
+ "epoch": 1.42,
1607
+ "learning_rate": 6.000257187885497e-06,
1608
+ "loss": 0.0066,
1609
+ "step": 8400
1610
+ },
1611
+ {
1612
+ "epoch": 1.42,
1613
+ "learning_rate": 5.8406283070712074e-06,
1614
+ "loss": 0.0065,
1615
+ "step": 8450
1616
+ },
1617
+ {
1618
+ "epoch": 1.43,
1619
+ "learning_rate": 5.682636609009177e-06,
1620
+ "loss": 0.0067,
1621
+ "step": 8500
1622
+ },
1623
+ {
1624
+ "epoch": 1.43,
1625
+ "eval_ap_CMedQAv1": 0.8650759729076953,
1626
+ "eval_ap_CMedQAv2": 0.8749745804892705,
1627
+ "eval_ap_Mmarco": 0.3538804931837119,
1628
+ "eval_ap_T2Reranking": 0.6878922264348706,
1629
+ "eval_avg_ap": 0.695455818253887,
1630
+ "eval_loss": 0.12287386506795883,
1631
+ "eval_mrr_CMedQAv1": 0.8906345238095239,
1632
+ "eval_mrr_CMedQAv2": 0.8960039682539682,
1633
+ "eval_mrr_Mmarco": 0.34160714285714294,
1634
+ "eval_mrr_T2Reranking": 0.7982257713511944,
1635
+ "eval_ndcg@10_CMedQAv1": 1.0000001192092896,
1636
+ "eval_ndcg@10_CMedQAv2": 0.9829331636428833,
1637
+ "eval_ndcg@10_Mmarco": 0.19680270552635193,
1638
+ "eval_ndcg@10_T2Reranking": 0.5907678008079529,
1639
+ "eval_ndcg@1_CMedQAv1": 1.0,
1640
+ "eval_ndcg@1_CMedQAv2": 1.0,
1641
+ "eval_ndcg@1_Mmarco": 0.20000000298023224,
1642
+ "eval_ndcg@1_T2Reranking": 0.5812433958053589,
1643
+ "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
1644
+ "eval_ndcg@3_CMedQAv2": 0.9999998807907104,
1645
+ "eval_ndcg@3_Mmarco": 0.22346392273902893,
1646
+ "eval_ndcg@3_T2Reranking": 0.5841401815414429,
1647
+ "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
1648
+ "eval_ndcg@5_CMedQAv2": 0.9853931665420532,
1649
+ "eval_ndcg@5_Mmarco": 0.21695800125598907,
1650
+ "eval_ndcg@5_T2Reranking": 0.5929743051528931,
1651
+ "eval_ndcg_CMedQAv1": 0.9629640579223633,
1652
+ "eval_ndcg_CMedQAv2": 0.9660819172859192,
1653
+ "eval_ndcg_Mmarco": 0.463792085647583,
1654
+ "eval_ndcg_T2Reranking": 0.9027825593948364,
1655
+ "eval_runtime": 1128.8704,
1656
+ "eval_samples_per_second": 354.36,
1657
+ "eval_steps_per_second": 0.346,
1658
+ "step": 8500
1659
+ },
1660
+ {
1661
+ "epoch": 1.44,
1662
+ "learning_rate": 5.5263103337678074e-06,
1663
+ "loss": 0.0065,
1664
+ "step": 8550
1665
+ },
1666
+ {
1667
+ "epoch": 1.45,
1668
+ "learning_rate": 5.371677423731162e-06,
1669
+ "loss": 0.0064,
1670
+ "step": 8600
1671
+ },
1672
+ {
1673
+ "epoch": 1.46,
1674
+ "learning_rate": 5.2187655186044135e-06,
1675
+ "loss": 0.0063,
1676
+ "step": 8650
1677
+ },
1678
+ {
1679
+ "epoch": 1.47,
1680
+ "learning_rate": 5.067601950473435e-06,
1681
+ "loss": 0.0067,
1682
+ "step": 8700
1683
+ },
1684
+ {
1685
+ "epoch": 1.47,
1686
+ "learning_rate": 4.918213738919363e-06,
1687
+ "loss": 0.0064,
1688
+ "step": 8750
1689
+ },
1690
+ {
1691
+ "epoch": 1.48,
1692
+ "learning_rate": 4.770627586188978e-06,
1693
+ "loss": 0.0063,
1694
+ "step": 8800
1695
+ },
1696
+ {
1697
+ "epoch": 1.49,
1698
+ "learning_rate": 4.624869872421859e-06,
1699
+ "loss": 0.0064,
1700
+ "step": 8850
1701
+ },
1702
+ {
1703
+ "epoch": 1.5,
1704
+ "learning_rate": 4.4809666509350785e-06,
1705
+ "loss": 0.0063,
1706
+ "step": 8900
1707
+ },
1708
+ {
1709
+ "epoch": 1.51,
1710
+ "learning_rate": 4.338943643566367e-06,
1711
+ "loss": 0.0065,
1712
+ "step": 8950
1713
+ },
1714
+ {
1715
+ "epoch": 1.52,
1716
+ "learning_rate": 4.1988262360764306e-06,
1717
+ "loss": 0.0065,
1718
+ "step": 9000
1719
+ },
1720
+ {
1721
+ "epoch": 1.52,
1722
+ "eval_ap_CMedQAv1": 0.8641924375078601,
1723
+ "eval_ap_CMedQAv2": 0.8717899695221966,
1724
+ "eval_ap_Mmarco": 0.37013444296743075,
1725
+ "eval_ap_T2Reranking": 0.6848922218837102,
1726
+ "eval_avg_ap": 0.6977522679702994,
1727
+ "eval_loss": 0.12518064677715302,
1728
+ "eval_mrr_CMedQAv1": 0.8881654761904761,
1729
+ "eval_mrr_CMedQAv2": 0.8928563492063493,
1730
+ "eval_mrr_Mmarco": 0.3619960317460318,
1731
+ "eval_mrr_T2Reranking": 0.7966605006931683,
1732
+ "eval_ndcg@10_CMedQAv1": 0.9898759126663208,
1733
+ "eval_ndcg@10_CMedQAv2": 0.9804811477661133,
1734
+ "eval_ndcg@10_Mmarco": 0.21500129997730255,
1735
+ "eval_ndcg@10_T2Reranking": 0.5945191979408264,
1736
+ "eval_ndcg@1_CMedQAv1": 1.0,
1737
+ "eval_ndcg@1_CMedQAv2": 1.0,
1738
+ "eval_ndcg@1_Mmarco": 0.20000000298023224,
1739
+ "eval_ndcg@1_T2Reranking": 0.5946031808853149,
1740
+ "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
1741
+ "eval_ndcg@3_CMedQAv2": 0.976535975933075,
1742
+ "eval_ndcg@3_Mmarco": 0.2357524186372757,
1743
+ "eval_ndcg@3_T2Reranking": 0.6052175760269165,
1744
+ "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
1745
+ "eval_ndcg@5_CMedQAv2": 0.9699214696884155,
1746
+ "eval_ndcg@5_Mmarco": 0.22286656498908997,
1747
+ "eval_ndcg@5_T2Reranking": 0.6057425737380981,
1748
+ "eval_ndcg_CMedQAv1": 0.9635534286499023,
1749
+ "eval_ndcg_CMedQAv2": 0.9656587839126587,
1750
+ "eval_ndcg_Mmarco": 0.46775779128074646,
1751
+ "eval_ndcg_T2Reranking": 0.9013978838920593,
1752
+ "eval_runtime": 1073.8791,
1753
+ "eval_samples_per_second": 372.506,
1754
+ "eval_steps_per_second": 0.364,
1755
+ "step": 9000
1756
+ },
1757
+ {
1758
+ "epoch": 1.53,
1759
+ "learning_rate": 4.060639473611431e-06,
1760
+ "loss": 0.0064,
1761
+ "step": 9050
1762
+ },
1763
+ {
1764
+ "epoch": 1.53,
1765
+ "learning_rate": 3.924408056226315e-06,
1766
+ "loss": 0.0063,
1767
+ "step": 9100
1768
+ },
1769
+ {
1770
+ "epoch": 1.54,
1771
+ "learning_rate": 3.7901563344698305e-06,
1772
+ "loss": 0.0064,
1773
+ "step": 9150
1774
+ },
1775
+ {
1776
+ "epoch": 1.55,
1777
+ "learning_rate": 3.6579083050319985e-06,
1778
+ "loss": 0.0063,
1779
+ "step": 9200
1780
+ },
1781
+ {
1782
+ "epoch": 1.56,
1783
+ "learning_rate": 3.5276876064548523e-06,
1784
+ "loss": 0.0064,
1785
+ "step": 9250
1786
+ },
1787
+ {
1788
+ "epoch": 1.57,
1789
+ "learning_rate": 3.3995175149072066e-06,
1790
+ "loss": 0.0064,
1791
+ "step": 9300
1792
+ },
1793
+ {
1794
+ "epoch": 1.58,
1795
+ "learning_rate": 3.273420940024165e-06,
1796
+ "loss": 0.0064,
1797
+ "step": 9350
1798
+ },
1799
+ {
1800
+ "epoch": 1.58,
1801
+ "learning_rate": 3.149420420812157e-06,
1802
+ "loss": 0.0064,
1803
+ "step": 9400
1804
+ },
1805
+ {
1806
+ "epoch": 1.59,
1807
+ "learning_rate": 3.0275381216202334e-06,
1808
+ "loss": 0.0064,
1809
+ "step": 9450
1810
+ },
1811
+ {
1812
+ "epoch": 1.6,
1813
+ "learning_rate": 2.907795828178335e-06,
1814
+ "loss": 0.0064,
1815
+ "step": 9500
1816
+ },
1817
+ {
1818
+ "epoch": 1.6,
1819
+ "eval_ap_CMedQAv1": 0.8649280235435451,
1820
+ "eval_ap_CMedQAv2": 0.8765992249434462,
1821
+ "eval_ap_Mmarco": 0.37276029660147,
1822
+ "eval_ap_T2Reranking": 0.6854808029670674,
1823
+ "eval_avg_ap": 0.6999420870138822,
1824
+ "eval_loss": 0.12521055340766907,
1825
+ "eval_mrr_CMedQAv1": 0.8882769841269842,
1826
+ "eval_mrr_CMedQAv2": 0.8982996031746031,
1827
+ "eval_mrr_Mmarco": 0.36348015873015876,
1828
+ "eval_mrr_T2Reranking": 0.7960031998581424,
1829
+ "eval_ndcg@10_CMedQAv1": 0.9872758984565735,
1830
+ "eval_ndcg@10_CMedQAv2": 0.9811555743217468,
1831
+ "eval_ndcg@10_Mmarco": 0.2056836634874344,
1832
+ "eval_ndcg@10_T2Reranking": 0.5942399501800537,
1833
+ "eval_ndcg@1_CMedQAv1": 1.0,
1834
+ "eval_ndcg@1_CMedQAv2": 1.0,
1835
+ "eval_ndcg@1_Mmarco": 0.20000000298023224,
1836
+ "eval_ndcg@1_T2Reranking": 0.5888352394104004,
1837
+ "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
1838
+ "eval_ndcg@3_CMedQAv2": 0.976535975933075,
1839
+ "eval_ndcg@3_Mmarco": 0.2765360474586487,
1840
+ "eval_ndcg@3_T2Reranking": 0.5885987281799316,
1841
+ "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
1842
+ "eval_ndcg@5_CMedQAv2": 0.9830419421195984,
1843
+ "eval_ndcg@5_Mmarco": 0.22907359898090363,
1844
+ "eval_ndcg@5_T2Reranking": 0.5933831930160522,
1845
+ "eval_ndcg_CMedQAv1": 0.962390124797821,
1846
+ "eval_ndcg_CMedQAv2": 0.9656065106391907,
1847
+ "eval_ndcg_Mmarco": 0.470781147480011,
1848
+ "eval_ndcg_T2Reranking": 0.9014945030212402,
1849
+ "eval_runtime": 1080.5882,
1850
+ "eval_samples_per_second": 370.193,
1851
+ "eval_steps_per_second": 0.362,
1852
+ "step": 9500
1853
+ },
1854
+ {
1855
+ "epoch": 1.61,
1856
+ "learning_rate": 2.7902149437031954e-06,
1857
+ "loss": 0.0067,
1858
+ "step": 9550
1859
+ },
1860
+ {
1861
+ "epoch": 1.62,
1862
+ "learning_rate": 2.6748164850726625e-06,
1863
+ "loss": 0.0065,
1864
+ "step": 9600
1865
+ },
1866
+ {
1867
+ "epoch": 1.63,
1868
+ "learning_rate": 2.5616210790690604e-06,
1869
+ "loss": 0.0063,
1870
+ "step": 9650
1871
+ },
1872
+ {
1873
+ "epoch": 1.63,
1874
+ "learning_rate": 2.4506489586922726e-06,
1875
+ "loss": 0.0064,
1876
+ "step": 9700
1877
+ },
1878
+ {
1879
+ "epoch": 1.64,
1880
+ "learning_rate": 2.3419199595431993e-06,
1881
+ "loss": 0.0064,
1882
+ "step": 9750
1883
+ },
1884
+ {
1885
+ "epoch": 1.65,
1886
+ "learning_rate": 2.2354535162782867e-06,
1887
+ "loss": 0.0064,
1888
+ "step": 9800
1889
+ },
1890
+ {
1891
+ "epoch": 1.66,
1892
+ "learning_rate": 2.1312686591356766e-06,
1893
+ "loss": 0.0064,
1894
+ "step": 9850
1895
+ },
1896
+ {
1897
+ "epoch": 1.67,
1898
+ "learning_rate": 2.0293840105336916e-06,
1899
+ "loss": 0.0063,
1900
+ "step": 9900
1901
+ },
1902
+ {
1903
+ "epoch": 1.68,
1904
+ "learning_rate": 1.92981778174216e-06,
1905
+ "loss": 0.0064,
1906
+ "step": 9950
1907
+ },
1908
+ {
1909
+ "epoch": 1.69,
1910
+ "learning_rate": 1.8325877696272857e-06,
1911
+ "loss": 0.0063,
1912
+ "step": 10000
1913
+ },
1914
+ {
1915
+ "epoch": 1.69,
1916
+ "eval_ap_CMedQAv1": 0.8635020358013351,
1917
+ "eval_ap_CMedQAv2": 0.8796776803107693,
1918
+ "eval_ap_Mmarco": 0.36752299795574767,
1919
+ "eval_ap_T2Reranking": 0.6861142187897954,
1920
+ "eval_avg_ap": 0.6992042332144118,
1921
+ "eval_loss": 0.12555988132953644,
1922
+ "eval_mrr_CMedQAv1": 0.8863333333333334,
1923
+ "eval_mrr_CMedQAv2": 0.900195238095238,
1924
+ "eval_mrr_Mmarco": 0.3552460317460317,
1925
+ "eval_mrr_T2Reranking": 0.7958933138816348,
1926
+ "eval_ndcg@10_CMedQAv1": 0.9933746457099915,
1927
+ "eval_ndcg@10_CMedQAv2": 0.9811555743217468,
1928
+ "eval_ndcg@10_Mmarco": 0.2201658934354782,
1929
+ "eval_ndcg@10_T2Reranking": 0.5817710757255554,
1930
+ "eval_ndcg@1_CMedQAv1": 1.0,
1931
+ "eval_ndcg@1_CMedQAv2": 1.0,
1932
+ "eval_ndcg@1_Mmarco": 0.30000001192092896,
1933
+ "eval_ndcg@1_T2Reranking": 0.5484000444412231,
1934
+ "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
1935
+ "eval_ndcg@3_CMedQAv2": 0.976535975933075,
1936
+ "eval_ndcg@3_Mmarco": 0.29999998211860657,
1937
+ "eval_ndcg@3_T2Reranking": 0.5612119436264038,
1938
+ "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
1939
+ "eval_ndcg@5_CMedQAv2": 0.9830419421195984,
1940
+ "eval_ndcg@5_Mmarco": 0.22993843257427216,
1941
+ "eval_ndcg@5_T2Reranking": 0.5722955465316772,
1942
+ "eval_ndcg_CMedQAv1": 0.9632137417793274,
1943
+ "eval_ndcg_CMedQAv2": 0.9660226106643677,
1944
+ "eval_ndcg_Mmarco": 0.4788861870765686,
1945
+ "eval_ndcg_T2Reranking": 0.9014593958854675,
1946
+ "eval_runtime": 1096.8598,
1947
+ "eval_samples_per_second": 364.701,
1948
+ "eval_steps_per_second": 0.356,
1949
+ "step": 10000
1950
+ },
1951
+ {
1952
+ "epoch": 1.69,
1953
+ "learning_rate": 1.7377113534705436e-06,
1954
+ "loss": 0.0063,
1955
+ "step": 10050
1956
+ },
1957
+ {
1958
+ "epoch": 1.7,
1959
+ "learning_rate": 1.64520549186226e-06,
1960
+ "loss": 0.0063,
1961
+ "step": 10100
1962
+ },
1963
+ {
1964
+ "epoch": 1.71,
1965
+ "learning_rate": 1.555086719670345e-06,
1966
+ "loss": 0.0065,
1967
+ "step": 10150
1968
+ },
1969
+ {
1970
+ "epoch": 1.72,
1971
+ "learning_rate": 1.467371145084792e-06,
1972
+ "loss": 0.0064,
1973
+ "step": 10200
1974
+ },
1975
+ {
1976
+ "epoch": 1.73,
1977
+ "learning_rate": 1.3820744467384483e-06,
1978
+ "loss": 0.0061,
1979
+ "step": 10250
1980
+ },
1981
+ {
1982
+ "epoch": 1.74,
1983
+ "learning_rate": 1.2992118709045309e-06,
1984
+ "loss": 0.0062,
1985
+ "step": 10300
1986
+ },
1987
+ {
1988
+ "epoch": 1.74,
1989
+ "learning_rate": 1.2187982287714573e-06,
1990
+ "loss": 0.0064,
1991
+ "step": 10350
1992
+ },
1993
+ {
1994
+ "epoch": 1.75,
1995
+ "learning_rate": 1.1408478937954458e-06,
1996
+ "loss": 0.0062,
1997
+ "step": 10400
1998
+ },
1999
+ {
2000
+ "epoch": 1.76,
2001
+ "learning_rate": 1.0653747991313201e-06,
2002
+ "loss": 0.0062,
2003
+ "step": 10450
2004
+ },
2005
+ {
2006
+ "epoch": 1.77,
2007
+ "learning_rate": 9.923924351420716e-07,
2008
+ "loss": 0.006,
2009
+ "step": 10500
2010
+ },
2011
+ {
2012
+ "epoch": 1.77,
2013
+ "eval_ap_CMedQAv1": 0.8637393453944692,
2014
+ "eval_ap_CMedQAv2": 0.8789491384323506,
2015
+ "eval_ap_Mmarco": 0.368903977630621,
2016
+ "eval_ap_T2Reranking": 0.6868251658388673,
2017
+ "eval_avg_ap": 0.699604406824077,
2018
+ "eval_loss": 0.12616057693958282,
2019
+ "eval_mrr_CMedQAv1": 0.8865428571428572,
2020
+ "eval_mrr_CMedQAv2": 0.9001646825396825,
2021
+ "eval_mrr_Mmarco": 0.3571944444444445,
2022
+ "eval_mrr_T2Reranking": 0.7977927426894929,
2023
+ "eval_ndcg@10_CMedQAv1": 0.9866948127746582,
2024
+ "eval_ndcg@10_CMedQAv2": 0.9709935188293457,
2025
+ "eval_ndcg@10_Mmarco": 0.19204413890838623,
2026
+ "eval_ndcg@10_T2Reranking": 0.5846768617630005,
2027
+ "eval_ndcg@1_CMedQAv1": 1.0,
2028
+ "eval_ndcg@1_CMedQAv2": 1.0,
2029
+ "eval_ndcg@1_Mmarco": 0.10000000149011612,
2030
+ "eval_ndcg@1_T2Reranking": 0.5960000157356262,
2031
+ "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
2032
+ "eval_ndcg@3_CMedQAv2": 0.9703917503356934,
2033
+ "eval_ndcg@3_Mmarco": 0.26536059379577637,
2034
+ "eval_ndcg@3_T2Reranking": 0.6046911478042603,
2035
+ "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
2036
+ "eval_ndcg@5_CMedQAv2": 0.972041130065918,
2037
+ "eval_ndcg@5_Mmarco": 0.20490367710590363,
2038
+ "eval_ndcg@5_T2Reranking": 0.597877025604248,
2039
+ "eval_ndcg_CMedQAv1": 0.9632848501205444,
2040
+ "eval_ndcg_CMedQAv2": 0.9656885266304016,
2041
+ "eval_ndcg_Mmarco": 0.463381290435791,
2042
+ "eval_ndcg_T2Reranking": 0.902158260345459,
2043
+ "eval_runtime": 1089.917,
2044
+ "eval_samples_per_second": 367.024,
2045
+ "eval_steps_per_second": 0.359,
2046
+ "step": 10500
2047
+ },
2048
+ {
2049
+ "epoch": 1.78,
2050
+ "learning_rate": 9.21913846987511e-07,
2051
+ "loss": 0.0064,
2052
+ "step": 10550
2053
+ },
2054
+ {
2055
+ "epoch": 1.79,
2056
+ "learning_rate": 8.539516322925401e-07,
2057
+ "loss": 0.0063,
2058
+ "step": 10600
2059
+ },
2060
+ {
2061
+ "epoch": 1.79,
2062
+ "learning_rate": 7.885179388954022e-07,
2063
+ "loss": 0.0062,
2064
+ "step": 10650
2065
+ },
2066
+ {
2067
+ "epoch": 1.8,
2068
+ "learning_rate": 7.256244626763186e-07,
2069
+ "loss": 0.0063,
2070
+ "step": 10700
2071
+ },
2072
+ {
2073
+ "epoch": 1.81,
2074
+ "learning_rate": 6.652824454669315e-07,
2075
+ "loss": 0.0065,
2076
+ "step": 10750
2077
+ },
2078
+ {
2079
+ "epoch": 1.82,
2080
+ "learning_rate": 6.075026730408817e-07,
2081
+ "loss": 0.0061,
2082
+ "step": 10800
2083
+ },
2084
+ {
2085
+ "epoch": 1.83,
2086
+ "learning_rate": 5.522954731859342e-07,
2087
+ "loss": 0.0063,
2088
+ "step": 10850
2089
+ },
2090
+ {
2091
+ "epoch": 1.84,
2092
+ "learning_rate": 4.996707138579266e-07,
2093
+ "loss": 0.0063,
2094
+ "step": 10900
2095
+ },
2096
+ {
2097
+ "epoch": 1.85,
2098
+ "learning_rate": 4.4963780141694446e-07,
2099
+ "loss": 0.0062,
2100
+ "step": 10950
2101
+ },
2102
+ {
2103
+ "epoch": 1.85,
2104
+ "learning_rate": 4.022056789459921e-07,
2105
+ "loss": 0.0061,
2106
+ "step": 11000
2107
+ },
2108
+ {
2109
+ "epoch": 1.85,
2110
+ "eval_ap_CMedQAv1": 0.8634801543277222,
2111
+ "eval_ap_CMedQAv2": 0.8789994898446902,
2112
+ "eval_ap_Mmarco": 0.37314697568316435,
2113
+ "eval_ap_T2Reranking": 0.6854003707502277,
2114
+ "eval_avg_ap": 0.7002567476514512,
2115
+ "eval_loss": 0.12499513477087021,
2116
+ "eval_mrr_CMedQAv1": 0.8869642857142858,
2117
+ "eval_mrr_CMedQAv2": 0.8993952380952381,
2118
+ "eval_mrr_Mmarco": 0.3613452380952381,
2119
+ "eval_mrr_T2Reranking": 0.7958653722152367,
2120
+ "eval_ndcg@10_CMedQAv1": 0.9802283048629761,
2121
+ "eval_ndcg@10_CMedQAv2": 0.9713308215141296,
2122
+ "eval_ndcg@10_Mmarco": 0.19949547946453094,
2123
+ "eval_ndcg@10_T2Reranking": 0.5712553262710571,
2124
+ "eval_ndcg@1_CMedQAv1": 1.0,
2125
+ "eval_ndcg@1_CMedQAv2": 1.0,
2126
+ "eval_ndcg@1_Mmarco": 0.20000000298023224,
2127
+ "eval_ndcg@1_T2Reranking": 0.6044243574142456,
2128
+ "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
2129
+ "eval_ndcg@3_CMedQAv2": 0.9703917503356934,
2130
+ "eval_ndcg@3_Mmarco": 0.28268030285835266,
2131
+ "eval_ndcg@3_T2Reranking": 0.5774673223495483,
2132
+ "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
2133
+ "eval_ndcg@5_CMedQAv2": 0.978601336479187,
2134
+ "eval_ndcg@5_Mmarco": 0.23054155707359314,
2135
+ "eval_ndcg@5_T2Reranking": 0.5735260844230652,
2136
+ "eval_ndcg_CMedQAv1": 0.9631568789482117,
2137
+ "eval_ndcg_CMedQAv2": 0.9658399820327759,
2138
+ "eval_ndcg_Mmarco": 0.4707724452018738,
2139
+ "eval_ndcg_T2Reranking": 0.902129054069519,
2140
+ "eval_runtime": 1050.4339,
2141
+ "eval_samples_per_second": 380.82,
2142
+ "eval_steps_per_second": 0.372,
2143
+ "step": 11000
2144
+ }
2145
+ ],
2146
+ "logging_steps": 50,
2147
+ "max_steps": 11868,
2148
+ "num_train_epochs": 2,
2149
+ "save_steps": 500,
2150
+ "total_flos": 3.458164495011021e+16,
2151
+ "trial_name": null,
2152
+ "trial_params": null
2153
+ }
training_args.bin ADDED
File without changes
zero_to_fp32.py ADDED
File without changes