yujie11112222/pipe-run2

F2LLM-v2-0.6B finetuned on domain retrieval (embedding)

This is a sentence-transformers model finetuned from codefuse-ai/F2LLM-v2-0.6B on the json dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: codefuse-ai/F2LLM-v2-0.6B
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text
• Training Dataset:
  • json
• Language: multilingual
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'Qwen3Model'})
  (1): Pooling({'embedding_dimension': 1024, 'pooling_mode': 'lasttoken', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("yujie11112222/pipe-run2")
# Run inference
queries = [
    'Instruct: Retrieve product descriptions given the product name.\nQuery: hd original xbox',
]
documents = [
    'Component to HDMI Converter YPbPr to HDMI Converter Support 720P/ 1080P for HD TV, DVD Player, Blu-ray Player, Wii, PS2/PS3, Xbox 360, Original Xbox and More (with HDMI Cable)',
    '定制苹果电脑macbookpro16保护壳14笔记本Macbook定做air13创意pro15可爱ins保护套13.3寸Mac12 m1外壳套贴膜',
    'Riko Konoe what do you do 234 pics (Japanese Edition)',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 1024] [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.4688, 0.1289, 0.1143]], dtype=torch.bfloat16)

Evaluation

Metrics

Information Retrieval

• Dataset: eval_pipe
• Evaluated with InformationRetrievalEvaluator with these parameters:

  {
      "query_prompt": "Instruct: Given a question, retrieve passages that can help answer the question.\nQuery: ",
      "corpus_prompt": ""
  }
  

Metric	Value
cosine_accuracy@1	1.0
cosine_accuracy@5	1.0
cosine_accuracy@10	1.0
cosine_precision@1	1.0
cosine_precision@5	0.2
cosine_precision@10	0.1
cosine_precision@50	0.02
cosine_recall@1	1.0
cosine_recall@5	1.0
cosine_recall@10	1.0
cosine_recall@50	1.0
cosine_ndcg@10	1.0
cosine_mrr@10	1.0
cosine_map@100	1.0

Training Details

Training Dataset

json

• Dataset: json
• Size: 160 training samples
• Columns: anchor, positive, negative_1, negative_2, negative_3, negative_4, negative_5, negative_6, negative_7, negative_8, negative_9, negative_10, negative_11, negative_12, negative_13, negative_14, negative_15, negative_16, negative_17, negative_18, negative_19, negative_20, negative_21, negative_22, negative_23, and negative_24
• Approximate statistics based on the first 100 samples:

  	anchor	positive	negative_1	negative_2	negative_3	negative_4	negative_5	negative_6	negative_7	negative_8	negative_9	negative_10	negative_11	negative_12	negative_13	negative_14	negative_15	negative_16	negative_17	negative_18	negative_19	negative_20	negative_21	negative_22	negative_23	negative_24
  type	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string
  modality	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text
  details	min: 16 tokens mean: 29.61 tokens max: 78 tokens	min: 5 tokens mean: 64.24 tokens max: 512 tokens	min: 4 tokens mean: 69.04 tokens max: 512 tokens	min: 5 tokens mean: 97.93 tokens max: 512 tokens	min: 2 tokens mean: 79.58 tokens max: 512 tokens	min: 3 tokens mean: 90.7 tokens max: 512 tokens	min: 5 tokens mean: 76.71 tokens max: 512 tokens	min: 3 tokens mean: 73.67 tokens max: 512 tokens	min: 2 tokens mean: 73.31 tokens max: 491 tokens	min: 4 tokens mean: 85.0 tokens max: 466 tokens	min: 5 tokens mean: 84.45 tokens max: 512 tokens	min: 6 tokens mean: 75.92 tokens max: 512 tokens	min: 5 tokens mean: 62.12 tokens max: 512 tokens	min: 3 tokens mean: 80.14 tokens max: 512 tokens	min: 4 tokens mean: 93.07 tokens max: 512 tokens	min: 4 tokens mean: 83.82 tokens max: 512 tokens	min: 5 tokens mean: 75.55 tokens max: 512 tokens	min: 3 tokens mean: 75.6 tokens max: 512 tokens	min: 4 tokens mean: 81.91 tokens max: 512 tokens	min: 4 tokens mean: 80.25 tokens max: 421 tokens	min: 4 tokens mean: 60.48 tokens max: 416 tokens	min: 2 tokens mean: 66.45 tokens max: 512 tokens	min: 3 tokens mean: 84.32 tokens max: 464 tokens	min: 2 tokens mean: 85.42 tokens max: 512 tokens	min: 3 tokens mean: 74.58 tokens max: 512 tokens	min: 5 tokens mean: 80.78 tokens max: 512 tokens

• Samples:

  anchor	positive	negative_1	negative_2	negative_3	negative_4	negative_5	negative_6	negative_7	negative_8	negative_9	negative_10	negative_11	negative_12	negative_13	negative_14	negative_15	negative_16	negative_17	negative_18	negative_19	negative_20	negative_21	negative_22	negative_23	negative_24
  Instruct: Given a question, retrieve the answer. Query: 宝宝喝了刚烧开的开水！	前面您所说的情况主要是不小心喝滚烫的开水烫伤舌部的情况了，对于这个情况来说，没有太特效的治疗办法，您可以让孩子含一口凉水，但是不要吞咽，而是含在口腔当中，目的是起到止痛的作用了，假如孩子的舌部烫伤非常明显的话，则需要到烧伤科进一步就诊检查和治疗了。	你好 这个情况是说明感染了HPV病毒！并病毒数量还比较高1所以是要积极进行抗病毒治疗才行的！	你要确定你拔的是不是胡子，我感觉你说的应该是嘴唇上边的汗毛，因为拔胡子时特别疼，我建议你最好不要再拔了，因为它们长得特别快，而且当你拔完之后，再长出来就会更多。	做過甚麼檢查? 內分泌六项? B超?如果都沒問題的話,治療的手段應該還是比較容易的^-----^如果月经周期持续发延迟来潮的情况，建议你要查查内分泌六项，查清楚为什么月经推迟的原因	一般口臭的原因，是口腔局部因素和胃肠因素不好。口内有残冠残根，龋坏的牙齿容易造成食物嵌塞，细菌产酸发酵，产生臭味。胃肠消化不功能不好，导致反酸嗳气，发生口臭。到口腔科门诊进一步检查，拔除残冠残根，修补龋坏的牙齿。到消化内科进一步检查，可以选用助手化药及收敛药物，比如说胃蛋白酶，乳酶生，多酶片，鞣酸蛋白等。 线上问答内容仅为参考，如有医疗需求，请务必到正规医疗机构就诊	病情分析：意见建议：你好，传统来说，月经前七天和月经后八天为成年女性的安全期，是多数人用来避孕的一种计算方法。安全期并非绝对安全，有时候女性排卵会提前，或者延后，此时安全期的判断就会变得无效。祝你幸福！	乙肝患者能不能给孩子喂奶呢	紫色毛癣菌	我们的话平时一定要控制血压血糖，我们把血压和血糖控制在正常范围之内，这样可以减少脑梗塞的发病，一旦确诊脑梗塞的话，我们应该积极治疗，主要是改善循环治疗以及抗血小板聚集治疗。各种原因引起的休克，也是患脑梗塞主要的病因。	不建议在有胃病的时候吃红薯，当有胃病时不应该吃富含纤维的酸性或粗纤维食物，或刺激性食物，如辣椒、咖啡和浓茶。红薯含有丰富的糖分和大量的粗纤维。那么这个时候不太适合胃病患者吃，因为它含有大量的糖分，吃到胃里会引起胃酸倒流、烧心的症状。此外大量纤维素不能被消化吸收，因此会加重胃酸倒流、烧心症状、加重胃痛、胃胀的症状。	问题分析： 银屑病的治疗方法有很多，每个人的体质和病情发展不同，用药方案也不一样，盲目指导用药会延误疾病。建议及时到正规的鱼鳞病专业治疗医院进行科学全面的治疗，查出发病诱因，对因对症治疗.意见建议：要注意对患处皮肤的保湿、保暖和保护，可以涂抹保湿霜，不要受冻着凉，洗澡时不要使劲的搓洗患处，对鳞屑不要撕扯，出现瘙痒尽量不要抓挠。饮食上要注意不要暴饮暴食，不要偏食，不要吃辛辣刺激的食物	桥本甲状腺炎西医治疗：注意休息，根据浓液中细菌的种类选用敏感的抗生素，浓液以形成或保守治疗不能使感染消退时，则应手术切开引流。	要生存规律,按时歇息,确保充足深度睡眠，防止加班、疲劳，饮食清淡，多食蔬菜水果，按时、规律服药。并且千万千万不要乘坐汽车，高空作业，以免有生命危险。癫痫患者在及时救治之外，患者在生活中还需要有留意要留意始终保持深度睡眠，充足的深度睡眠方才能帮你尽快远离此病，期望上述的答案可以帮助到你，谢谢！	Vichy Laboratoires Capital Soleil 30 Broad Spectrum SPF 30 Sunscreen Luxurious Protective（Avobenzone, Homosalate, Octisalate, Octocrylene and Oxybenzone）药品说明书	颈椎椎间盘突出。可以去北京同仁医院看一下去同仁医院看咽痛原因喉科	银杏外种皮内酯；银杏叶提取物；三甲散；熟地黄	Aurotussin Maximum Strength Daytime and Nighttime Cough Relief（Dextromethorphan HBr, Guaifenesin, Dextromethorphan HBr, Doxylamine Succinate）药品说明书	你好，很高兴为您解答，你说的症状是可以发生的，是糖尿病引起的微血管病变，尤其是足部，是经常发生的并发症，建议积极治疗原发病，控制血糖。      以上是对“糖尿病能引起手脚麻木吗？”这个问题的建议，希望对您有帮助，祝您健康！	Forest Heal Hand Sanitizer（Alcohol）Antiseptic preparation用药	小米和黑米中黑米的营养价值是不叫高的。在中一方面来说黑米是一种药，食兼用的大米。食用价值高。用黑米熬制的米粥清香油亮，软糯适口，营养丰富，具有很好的滋补作用。除了熬粥以外还可以制作各种营养食品和酿酒。在熬粥时泡米的水要与米同煮，以保存营养成分。	您好、想必您是一位男性、如果是男男性的话、他会有两个生长高峰、一个是青春期、另一个是23岁左右、并且一个人能否再长高、还撒于他的遗传、后天营养您好、首先不建议您服用激素治疗、激素的副作用是很明显的、所带来的后果是长得不够高完全不能比的、如肥胖等一个人能否再长高、还要看他的骨骺是否钙化您可以先去医院照x线片、后再做决定如果没钙化、您可以加强营养、钙片的补充等、等待第二次生长	一、包皮过长易致使性功能障碍。专家引用，包皮过长的男性朋友容易障碍性生活的正常实施，容易致使性交痛、困难和敏感度减低，进而致使男性功能障碍，如早泄、早泄、射精过快和不射精等，并引发夫妻生活不和谐，减低生活质量。二、包皮过长、包茎可构成包皮嵌顿。男性朋友应当多观注自己的身心健康，拥有包皮过长、包茎现象时应当尽早治疗。	综合护理干预；IUD放置术；心理护理；暖宫贴下腹部敷贴；常规护理	如果你偏瘦的话肚子就是不太显，只要B超没有问题，不用太担心。	甲状腺危象又称甲亢危象，是甲状腺毒症急性加重的一个综合征，发生原因可能与循环中的甲状腺激素水平增高有关。多发生于较重甲亢未予治疗或治疗不充分的患者。常见诱因有感染、手术、精神刺激等，临床表现为高热、大汗、心动过速、烦躁、焦虑不安、谵妄、恶心、呕吐、腹泻，严重患者可有心衰，休克和昏迷等。甲状腺危象的临床表现:(1)体温:常骤发高热,体温达39℃以上伴大汗淋漓面部潮红。(2)心动过速心率加快达140次分以上,可伴有早搏、心房纤额以至心力衰竭肺水肿。(3)烦躁不安、谵妄、嗜睡、昏迷。(4)恶心、呕吐、腹泻,甚至黄疸。少数淡漠型甲亢或老年患者缺乏典型甲亢危象表现为低热、淡漠、嗜睡、全身衰竭、休克昏迷死亡。甲状腺危象的急救处理措施：1.尽快减少甲状腺激素释放和合成，2.迅速阻滞儿茶酚胺释放：在无心衰、哮喘和房室传导阻滞的情况下，应用肾上腺素能受体阻滞剂甚为重要，必要时应在心电监视下进行。3.肾上腺皮质激素：肾上腺皮质激素既可抑制甲状腺激素的释放，又可减少T4向T3转化，并可纠正在甲状腺危象时肾上腺皮质机能相对不全。4.对症处理：包括采用药物或物理的方法降低体温，避免使用水杨酸盐降温，因它可竞争T3、T4与甲状腺结合蛋白的结合，使游离激素增加，大量水杨酸盐也增加代谢率。对甲状腺危象病人应予吸氧，补充水、电解质、维生素等治疗，烦躁时可使用镇静剂，必要时可采用人工冬眠。在饮食上，应给予患者高热量、高蛋白、高糖饮食，加强支持疗法，保持水、电解质平衡。对有感染的患者，给予适当的抗生素治疗，同时积极去除诱因。5.清除血中过多的甲状腺激素：在以上措施无效时，有条件的医院可试用换血、血浆去除、血液透析、腹膜透析等方法，以去除过多的甲状腺激素。
  Instruct: Given a product name, retrieve the corresponding product description. Query: ドラゴンいえをかう	ドラゴン、家を買う。 6巻 (ブレイドコミックス)	L'Oréal Paris EverPure Moisture Sulfate Free Shampoo and Conditioner for Color-Treated Hair, 8.5 Ounce (Set of 2)	100個*10 ペット用リボン ペット用ヘアリボン 犬 猫用髪飾り 犬リボン ヘアアクセサリー 複数の色 着脱が簡単 軽量 ゴム素材 グルーミング 誕生日 パーティー	自动大号金属透明宽胶带封箱器小胶带切割器快递打包胶纸机4.8cm	酒坛果肉沾沾点老坛果肉果冻混合口味拖肥椰肉粒布丁网红甜品零食	Waterproof Fitted Crib Mattress Pad and Toddler Crib Mattress Protective Baby Crib Mattress Cover Sheets Protector Bedding Sets Breathable and Hypoallergenic for Boys and Girls (White, Crib 28''x52'')	老君炉藤黄健骨丸颈椎病骨片康骨刺治疗痛的药键滕黄健骨凡骨胶襄吉林腾黄健疼黄健建儙片胶囊黄藤腰黄建蕂黄腾藤健腰王健中药痹吃	专升本英语2022普通高等学校天一库课专升本英语教材考试辅导2022专升本英语必刷题2000题章节习题英语词汇40天一本通全国通用	Bag Lover 撑包神器包定型内胆防变形Birkin/Kelly/Picotin/Lindy	原神枫原万叶温迪散兵魈卡套学生校园卡饭卡地铁卡公交卡保护套	ALLMAX BCAA支链氨基酸44份 90份增肌防止肌肉流失分解健身补剂男	双面吸盘茶几玻璃固定防滑器加厚强力透明无痕贴桌面木餐柜台面垫	Reliable 1 Laboratories Simethicone 125 MG Antiflatulent Chewable Tablets 60 Count	Levi's Women's New Boyfriend Jeans, Maui Views, 27 (US 4)	橱柜抽屉垫纸锡纸厨柜防水贴纸衣柜子自粘铝箔铺纸厨房防油防潮垫	Pantene Hairspray Non-aerosol, Volume Lasting Hold, Pro-V, Body & Softness Texturizing, 8.5 fl oz, Triple Pack	Grace and Frankie Seasons 1-2	Taotigzu El estante de almacenamiento de metal extensible es para gabinetes de cocina, encimeras, cocina, alimentos y utensilios, podría ahorrar espacios, blanco (blanco, 60 * 21 * 23cm)	SD秋款上新粉色拼接假两件皮衣秋季长袖儿童女宝时尚休闲套装7617	N2 Deck Belt Replaces Cub Cadet, MTD 954-05022A. Fits 46" XT1 Series.	HOLZBRINK Pata para Escritorio, Patas rectangulares de Metal, 80x20 mm, Perfil 50x72 cm, Negro, 1 Unidad, HLT-01-C-II-9005	50ml茶餐厅主人杯单杯品茗杯陶瓷家用功夫茶具喝茶杯大号茶碗茶盏	Pegatinas y Vinilos para Decoración de Pared | Círculos Puntos | Adhesivos Decorativos Nórdico Infantil | 52uds | Varios Colores (Blanco)	贝亲婴儿果蔬奶瓶清洁剂1300ml新生宝宝奶渍奶嘴玩具洗洁精补充装	Auro Ear Water-Drying Aid, 1 oz (Pack of 3)
  Instruct: Find the product that matches the user query. Query: loreal 10b	LOreal Professionnel INOA Ammonia Free Hair Color 2.1oz (10.1/10B)	假手指套残疾人食指仿真硅胶魔术道具断触软硬逼真手指中指拇指套	火石粒黑客AA级打火石瓶装ZP火机煤油砂轮式打火机通用软火石	客厅茶桌家用 小户型实木茶桌子 长方形休闲喝茶桌椅组合简约现代	LEGO Classic Large Creative Brick Box 10698 Build Your Own Creative Toys, Kids Building Kit (790 Pieces)	Lupct Yellow Basketball Shirt Compatible with Airpods 1/2 Case Silicone, Cute Cartoon 3D Cool Air pods Design Cover, Fun Fashion Stylish Funny Cases for Kids Teens Boys Character Skin Keychain Airpod	64th Man: Starring John Cena and Anna Chlumsky	铸造世界 田宫 60741 美国F4D-1战斗机 1/72	林微因纳兰性德字帖李清照字帖女生字体漂亮练字帖成年手写古风唯美小清新练字本大学生行楷书正楷字帖墨点纳兰容若全集古诗词	lenoup Glitter Bling Bling Phone Ring Holder,Sparkle Phone Ring Artificial Diamond Stand,Rhinestone Cell Phone Finger Ring for Phones,Pad(Silver)	Dodot Pañales Bebé-Seco Talla 5 (11-16 kg), 174 Pañales con Protección Antifugas, Pack Mensual	ナイキ アカデミー フットボール ソックス 001/ブラック/(ホワイト) SX4120 25-27cm	BEINNY 4 Boxes 12 Grids Nail Art Rhinestones + 1 PC Dotting Pen + 1 PC Straight Tweezer Tool Kits Nail Art Rhinestones Studs	吹上焼酎 黒古薩摩甕仕込み(芋) [ 焼酎 25度 鹿児島県 720ml ]	适用于别克凯越1.8老君威2.0景程2.0排气歧管垫排气支管密封垫片	比斯奇泰国进口巧克威化饼干小包网红夹心饼干办公室零食自选超市	Trauma Stewardship: An Everyday Guide to Caring for Self While Caring for Others	日本原装雅马哈04-14年YZF1000 R1 R6 气缸垫 中缸垫 上下垫	Fotile/方太 CXW-200-EMD1T欧式顶吸式排抽油烟机	睡衣女秋款睡裙蕾丝吊带夏季情调内衣包臀裙高级感2024年新款	EEEkit 10 Pack LED Emergency Strobe Lights, Amber White 6 LED Strobe Warning Emergency Flashing Light Caution Construction Hazard Light Bar for Car Truck Van Off Road Vehicle ATV SUV Surface Mount	2025新款慵懒风V领纯色开衫毛衣女春秋新款灰色长袖外套宽松外搭	高低床上下床儿童床女孩公主床双层床两层实木上下铺木床母子双人	2021新茶茉莉花茶特级浓香型茶叶官方旗舰店正宗四川雀舌绿茶罐装	惠普1522N主板 惠普1522NF主板 接口板

• Loss: embedding_training_components.CachedEnhancedMultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "mini_batch_size": 4,
      "mini_batch_num_tokens": null,
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0,
      "focal_gamma": 0.0,
      "online_negative_mixing": "none",
      "negative_mixing_count": 1,
      "negative_mixing_alpha": 2.0,
      "negative_mixing_beta": 2.0,
      "contrastive_weight": 1.0,
      "contrastive_loss_mode": "f2llm_split",
      "inbatch_loss_weight": 1.0,
      "hard_loss_weight": 1.0,
      "distillation_weight": 0.0,
      "distillation_type": "none",
      "distillation_temperature": 0.05,
      "embedding_mse_space": "normalized"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• num_train_epochs: 2
• learning_rate: 2e-05
• warmup_steps: 10
• weight_decay: 0.01
• load_best_model_at_end: True
• prompts: {'anchor': '', 'positive': '', 'negative_1': '', 'negative_2': '', 'negative_3': '', 'negative_4': '', 'negative_5': ''}
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• per_device_train_batch_size: 8
• num_train_epochs: 2
• max_steps: -1
• learning_rate: 2e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 10
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.01
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1.0
• label_smoothing_factor: 0.0
• bf16: False
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 8
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: True
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• dataloader_multiprocessing_context: None
• dataloader_in_order: True
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: None
• fsdp_config: None
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• local_rank: -1
• prompts: {'anchor': '', 'positive': '', 'negative_1': '', 'negative_2': '', 'negative_3': '', 'negative_4': '', 'negative_5': ''}
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}
• warmup_ratio: None

Training Logs

Epoch	Step	Training Loss	eval_pipe_cosine_ndcg@10
-1	-1	-	0.9009
0.05	1	0.5156	-
0.1	2	0.0850	-
0.15	3	0.0004	-
0.2	4	1.0703	0.8824
0.25	5	0.4941	-
0.3	6	0.0031	-
0.35	7	0.0063	-
0.4	8	0.0287	0.8917
0.45	9	0.1216	-
0.5	10	0.0630	-
0.55	11	0.0064	-
0.6	12	0.0449	0.9124
0.65	13	0.3105	-
0.7	14	0.0181	-
0.75	15	0.2021	-
0.8	16	0.0879	0.9631
0.85	17	0.0057	-
0.9	18	0.6680	-
0.95	19	1.1875	-
1.0	20	0.4883	1.0
1.05	21	0.0233	-
1.1	22	0.0547	-
1.15	23	0.0376	-
1.2	24	0.0118	1.0
1.25	25	0.0110	-
1.3	26	0.9258	-
1.35	27	0.2832	-
1.4	28	0.0100	1.0
1.45	29	0.0015	-
1.5	30	0.0317	-
1.55	31	0.0134	-
1.6	32	0.0049	1.0
1.65	33	0.0027	-
1.7	34	0.0557	-
1.75	35	0.25	-
1.8	36	0.0022	1.0
1.85	37	0.0010	-
1.9	38	0.2676	-
1.95	39	0.0093	-
2.0	40	0.1143	1.0
-1	-1	-	1.0

• The bold row denotes the saved checkpoint.

Training Time

• Training: 13.1 minutes
• Evaluation: 1.2 minutes
• Total: 14.4 minutes

Framework Versions

• Python: 3.11.15
• Sentence Transformers: 5.7.0
• Transformers: 5.15.0
• PyTorch: 2.13.0
• Accelerate: 1.14.0
• Datasets: 5.0.1
• Tokenizers: 0.22.2

Additional Resources

• Training and Finetuning Embedding Models with Sentence Transformers: the end-to-end guide for training or finetuning Sentence Transformer models.
• Introduction to Matryoshka Embedding Models: variable-size embeddings that can be truncated with minimal quality loss.
• Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval: post-training compression of embedding vectors.
• Multimodal Embedding & Reranker Models with Sentence Transformers: use text, image, audio, and video models through the same API.
• Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers: train multimodal embedding models, with a Visual Document Retrieval walkthrough.

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

CachedEnhancedMultipleNegativesRankingLoss

@misc{gao2021scaling,
    title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup},
    author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan},
    year={2021},
    eprint={2101.06983},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}