CrossEncoder based on cross-encoder/mmarco-mMiniLMv2-L12-H384-v1

This is a Cross Encoder model finetuned from cross-encoder/mmarco-mMiniLMv2-L12-H384-v1 using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.

Model Details

Model Description

Model Type: Cross Encoder
Base model: cross-encoder/mmarco-mMiniLMv2-L12-H384-v1
Maximum Sequence Length: 128 tokens
Number of Output Labels: 1 label
Supported Modality: Text

Model Sources

Documentation: Sentence Transformers Documentation
Documentation: Cross Encoder Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Cross Encoders on Hugging Face

Full Model Architecture

CrossEncoder(
  (0): Transformer({'transformer_task': 'sequence-classification', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'logits'}}, 'module_output_name': 'scores', 'architecture': 'XLMRobertaForSequenceClassification'})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("leafxyz/arabic-ecom-cross-encoder-v4")
# Get scores for pairs of inputs
pairs = [
    ['اطعمة معلبة', 'درب شتورة حمص طحينة بالفلفل الحار - 400 غ'],
    ['عطر بيبي روج', 'زيت للأطفال \u200f- Babirose'],
    ['قرص فنش', 'سياقة كونتيسا - 40 سم'],
    ['فمّا قرفة مطحونة', 'زنجبيل مطحون الخليج - 70 غ'],
    ['كيرود', 'كيرود علبة هندسية لون ازرق - 8 قطع'],
]
scores = model.predict(pairs)
print(scores)
# [ 0.6948 -4.2344 -3.0059 -5.4609 -0.0091]

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    'اطعمة معلبة',
    [
        'درب شتورة حمص طحينة بالفلفل الحار - 400 غ',
        'زيت للأطفال \u200f- Babirose',
        'سياقة كونتيسا - 40 سم',
        'زنجبيل مطحون الخليج - 70 غ',
        'كيرود علبة هندسية لون ازرق - 8 قطع',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

Training Details

Training Dataset

Unnamed Dataset

Size: 334,256 training samples
Columns: sentence1, sentence2, and label

Approximate statistics based on the first 100 samples:

	sentence1	sentence2	label
type	string	string	float
modality	text	text
details	min: 3 tokens mean: 7.34 tokens max: 13 tokens	min: 5 tokens mean: 12.17 tokens max: 21 tokens	min: 0.0 mean: 0.68 max: 1.0

Samples:

sentence1	sentence2	label
`أواني طبخ`	`شنقال ستيل لفاف`	`0.0`
`ساعات نسائية`	`خاتم شي ان - DB032`	`0.0`
`لوشن البابايا`	`لوشن البابايا لتبييض اليدين والجسم - RDL`	`1.0`

Loss: BinaryCrossEntropyLoss with these parameters:

{
    "activation_fn": "torch.nn.modules.linear.Identity",
    "pos_weight": null
}

Evaluation Dataset

Unnamed Dataset

Size: 6,822 evaluation samples
Columns: sentence1, sentence2, and label

Approximate statistics based on the first 100 samples:

	sentence1	sentence2	label
type	string	string	float
modality	text	text
details	min: 4 tokens mean: 8.0 tokens max: 14 tokens	min: 5 tokens mean: 12.86 tokens max: 29 tokens	min: 0.0 mean: 0.59 max: 1.0

Samples:

sentence1	sentence2	label
`اطعمة معلبة`	`درب شتورة حمص طحينة بالفلفل الحار - 400 غ`	`1.0`
`عطر بيبي روج`	`زيت للأطفال ‏- Babirose`	`0.0`
`قرص فنش`	`سياقة كونتيسا - 40 سم`	`0.0`

Loss: BinaryCrossEntropyLoss with these parameters:

{
    "activation_fn": "torch.nn.modules.linear.Identity",
    "pos_weight": null
}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 32
num_train_epochs: 2
learning_rate: 2e-05
warmup_steps: 0.1
fp16: True
per_device_eval_batch_size: 32

All Hyperparameters

Click to expand

per_device_train_batch_size: 32
num_train_epochs: 2
max_steps: -1
learning_rate: 2e-05
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_steps: 0.1
optim: adamw_torch_fused
optim_args: None
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
optim_target_modules: None
gradient_accumulation_steps: 1
average_tokens_across_devices: True
max_grad_norm: 1.0
label_smoothing_factor: 0.0
bf16: False
fp16: True
bf16_full_eval: False
fp16_full_eval: False
tf32: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
use_liger_kernel: False
liger_kernel_config: None
use_cache: False
neftune_noise_alpha: None
torch_empty_cache_steps: None
auto_find_batch_size: False
log_on_each_node: True
logging_nan_inf_filter: True
include_num_input_tokens_seen: no
log_level: passive
log_level_replica: warning
disable_tqdm: False
project: huggingface
trackio_space_id: None
trackio_bucket_id: None
trackio_static_space_id: None
per_device_eval_batch_size: 32
prediction_loss_only: True
eval_on_start: False
eval_do_concat_batches: True
eval_use_gather_object: False
eval_accumulation_steps: None
include_for_metrics: []
batch_eval_metrics: False
save_only_model: False
save_on_each_node: False
enable_jit_checkpoint: False
push_to_hub: False
hub_private_repo: None
hub_model_id: None
hub_strategy: every_save
hub_always_push: False
hub_revision: None
load_best_model_at_end: False
ignore_data_skip: False
restore_callback_states_from_checkpoint: False
full_determinism: False
seed: 42
data_seed: None
use_cpu: False
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_pin_memory: True
dataloader_persistent_workers: False
dataloader_prefetch_factor: None
remove_unused_columns: True
label_names: None
train_sampling_strategy: random
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
ddp_static_graph: None
ddp_backend: None
ddp_timeout: 1800
fsdp: None
fsdp_config: None
deepspeed: None
debug: []
skip_memory_metrics: True
do_predict: False
resume_from_checkpoint: None
warmup_ratio: None
local_rank: -1
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Click to expand

Epoch	Step	Training Loss	Validation Loss
0.0096	100	1.1056	-
0.0191	200	0.7532	-
0.0287	300	0.5562	-
0.0383	400	0.5104	-
0.0479	500	0.5014	0.4591
0.0574	600	0.4661	-
0.0670	700	0.4614	-
0.0766	800	0.4586	-
0.0862	900	0.4455	-
0.0957	1000	0.4397	0.4195
0.1053	1100	0.4370	-
0.1149	1200	0.4237	-
0.1244	1300	0.4051	-
0.1340	1400	0.4173	-
0.1436	1500	0.4100	0.3880
0.1532	1600	0.4113	-
0.1627	1700	0.4137	-
0.1723	1800	0.4103	-
0.1819	1900	0.4167	-
0.1915	2000	0.4328	0.3777
0.2010	2100	0.4072	-
0.2106	2200	0.3826	-
0.2202	2300	0.3917	-
0.2298	2400	0.3978	-
0.2393	2500	0.3868	0.3674
0.2489	2600	0.3779	-
0.2585	2700	0.3852	-
0.2680	2800	0.3606	-
0.2776	2900	0.3736	-
0.2872	3000	0.3762	0.3627
0.2968	3100	0.3692	-
0.3063	3200	0.3753	-
0.3159	3300	0.3725	-
0.3255	3400	0.3934	-
0.3351	3500	0.3817	0.3552
0.3446	3600	0.3568	-
0.3542	3700	0.3808	-
0.3638	3800	0.3763	-
0.3733	3900	0.3458	-
0.3829	4000	0.3536	0.3619
0.3925	4100	0.3800	-
0.4021	4200	0.3521	-
0.4116	4300	0.3568	-
0.4212	4400	0.3533	-
0.4308	4500	0.3582	0.3429
0.4404	4600	0.3586	-
0.4499	4700	0.3471	-
0.4595	4800	0.3600	-
0.4691	4900	0.3482	-
0.4787	5000	0.3554	0.3330
0.4882	5100	0.3541	-
0.4978	5200	0.3472	-
0.5074	5300	0.3580	-
0.5169	5400	0.3601	-
0.5265	5500	0.3594	0.3341
0.5361	5600	0.3542	-
0.5457	5700	0.3409	-
0.5552	5800	0.3464	-
0.5648	5900	0.3501	-
0.5744	6000	0.3403	0.3263
0.5840	6100	0.3497	-
0.5935	6200	0.3458	-
0.6031	6300	0.3415	-
0.6127	6400	0.3442	-
0.6222	6500	0.3301	0.3318
0.6318	6600	0.3388	-
0.6414	6700	0.3389	-
0.6510	6800	0.3382	-
0.6605	6900	0.3356	-
0.6701	7000	0.3471	0.3166
0.6797	7100	0.3468	-
0.6893	7200	0.3370	-
0.6988	7300	0.3554	-
0.7084	7400	0.3381	-
0.7180	7500	0.3258	0.3166
0.7276	7600	0.3244	-
0.7371	7700	0.3218	-
0.7467	7800	0.3097	-
0.7563	7900	0.3304	-
0.7658	8000	0.3225	0.3088
0.7754	8100	0.3331	-
0.7850	8200	0.3336	-
0.7946	8300	0.3270	-
0.8041	8400	0.3221	-
0.8137	8500	0.3204	0.3058
0.8233	8600	0.3263	-
0.8329	8700	0.3294	-
0.8424	8800	0.3071	-
0.8520	8900	0.3206	-
0.8616	9000	0.3316	0.3056
0.8711	9100	0.3247	-
0.8807	9200	0.3268	-
0.8903	9300	0.3248	-
0.8999	9400	0.3104	-
0.9094	9500	0.3247	0.3058
0.9190	9600	0.3022	-
0.9286	9700	0.3212	-
0.9382	9800	0.3299	-
0.9477	9900	0.2949	-
0.9573	10000	0.3326	0.3072
0.9669	10100	0.3204	-
0.9765	10200	0.3134	-
0.9860	10300	0.3285	-
0.9956	10400	0.3210	-
1.0052	10500	0.3003	0.3092
1.0147	10600	0.3074	-
1.0243	10700	0.3215	-
1.0339	10800	0.2885	-
1.0435	10900	0.3026	-
1.0530	11000	0.3039	0.3018
1.0626	11100	0.3062	-
1.0722	11200	0.3135	-
1.0818	11300	0.2951	-
1.0913	11400	0.3010	-
1.1009	11500	0.3216	0.3035
1.1105	11600	0.2951	-
1.1200	11700	0.2906	-
1.1296	11800	0.2871	-
1.1392	11900	0.3026	-
1.1488	12000	0.2925	0.3031
1.1583	12100	0.2942	-
1.1679	12200	0.3143	-
1.1775	12300	0.2847	-
1.1871	12400	0.3014	-
1.1966	12500	0.3034	0.2938
1.2062	12600	0.2999	-
1.2158	12700	0.2999	-
1.2253	12800	0.3005	-
1.2349	12900	0.2814	-
1.2445	13000	0.2985	0.2937
1.2541	13100	0.2902	-
1.2636	13200	0.2854	-
1.2732	13300	0.2981	-
1.2828	13400	0.3013	-
1.2924	13500	0.2916	0.2944
1.3019	13600	0.2913	-
1.3115	13700	0.2827	-
1.3211	13800	0.2929	-
1.3307	13900	0.2970	-
1.3402	14000	0.3006	0.2945
1.3498	14100	0.2926	-
1.3594	14200	0.2973	-
1.3689	14300	0.3117	-
1.3785	14400	0.2893	-
1.3881	14500	0.3094	0.2870
1.3977	14600	0.2750	-
1.4072	14700	0.2939	-
1.4168	14800	0.3100	-
1.4264	14900	0.2953	-
1.4360	15000	0.2680	0.2962
1.4455	15100	0.2977	-
1.4551	15200	0.2831	-
1.4647	15300	0.3071	-
1.4742	15400	0.2907	-
1.4838	15500	0.3053	0.2918
1.4934	15600	0.2787	-
1.5030	15700	0.2826	-
1.5125	15800	0.3129	-
1.5221	15900	0.2966	-
1.5317	16000	0.2903	0.2904
1.5413	16100	0.2952	-
1.5508	16200	0.2903	-
1.5604	16300	0.2817	-
1.5700	16400	0.2660	-
1.5796	16500	0.2808	0.2916
1.5891	16600	0.2886	-
1.5987	16700	0.3088	-
1.6083	16800	0.3114	-
1.6178	16900	0.2828	-
1.6274	17000	0.2924	0.2904
1.6370	17100	0.2903	-
1.6466	17200	0.2818	-
1.6561	17300	0.2726	-
1.6657	17400	0.2804	-
1.6753	17500	0.2999	0.2824
1.6849	17600	0.2765	-
1.6944	17700	0.2908	-
1.7040	17800	0.2959	-
1.7136	17900	0.2776	-
1.7231	18000	0.3067	0.2854
1.7327	18100	0.2707	-
1.7423	18200	0.2841	-
1.7519	18300	0.2832	-
1.7614	18400	0.3070	-
1.7710	18500	0.2976	0.2817
1.7806	18600	0.2844	-
1.7902	18700	0.2952	-
1.7997	18800	0.2806	-
1.8093	18900	0.2801	-
1.8189	19000	0.2679	0.2827
1.8285	19100	0.2926	-
1.8380	19200	0.3035	-
1.8476	19300	0.2689	-
1.8572	19400	0.2837	-
1.8667	19500	0.2902	0.2825
1.8763	19600	0.2723	-
1.8859	19700	0.2683	-
1.8955	19800	0.2814	-
1.9050	19900	0.2814	-
1.9146	20000	0.2869	0.2828
1.9242	20100	0.2822	-
1.9338	20200	0.2896	-
1.9433	20300	0.2876	-
1.9529	20400	0.2960	-
1.9625	20500	0.2989	0.2819
1.9720	20600	0.2739	-
1.9816	20700	0.2920	-
1.9912	20800	0.2700	-
2.0	20892	-	0.2821

Training Time

Training: 27.5 minutes

Framework Versions

Python: 3.12.13
Sentence Transformers: 5.6.0
Transformers: 5.13.1
PyTorch: 2.11.0+cu128
Accelerate: 1.14.0
Datasets: 4.0.0
Tokenizers: 0.22.2

Additional Resources

Training and Finetuning Reranker Models with Sentence Transformers: the end-to-end guide for training or finetuning Cross Encoder (reranker) models.
Multimodal Embedding & Reranker Models with Sentence Transformers: use text, image, audio, and video reranker models through the same API.
Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers: training multimodal Cross Encoders.

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}