S13v3n-2/scoring-camembert-v5

SentenceTransformer based on almanach/camembert-base

This is a sentence-transformers model finetuned from almanach/camembert-base. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: almanach/camembert-base
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'CamembertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("S13v3n-2/scoring-camembert-v5")
# Run inference
sentences = [
    "Déployer des modèles d'IA en production avec MLOps",
    'Conduire des études de marché qualitatives et quantitatives',
    "Déployer des modèles d'IA en production avec MLOps",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.0748, 1.0000],
#         [0.0748, 1.0000, 0.0748],
#         [1.0000, 0.0748, 1.0000]])

Evaluation

Metrics

Semantic Similarity

• Dataset: eval-referentiel
• Evaluated with EmbeddingSimilarityEvaluator

Metric	Value
pearson_cosine	0.9834
spearman_cosine	0.8957

Training Details

Training Dataset

Unnamed Dataset

• Size: 4,516 training samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 9 tokens mean: 13.75 tokens max: 32 tokens	min: 9 tokens mean: 13.85 tokens max: 32 tokens	min: 0.1 mean: 0.3 max: 0.9

• Samples:

  sentence1	sentence2	score
  Être curieux et pratiquer une veille active dans son domaine	Élaborer une stratégie de communication corporate cohérente	0.4
  Posséder un esprit pratique et aimer les travaux manuels techniques	Analyser et interpréter des contrats et textes juridiques complexes	0.1
  Programmer en Python ou R pour l'analyse de données	Effectuer une veille digitale et identifier les tendances émergentes	0.1

• Loss: CosineSimilarityLoss with these parameters:

  {
      "loss_fct": "torch.nn.modules.loss.MSELoss"
  }
  

Evaluation Dataset

Unnamed Dataset

• Size: 1,130 evaluation samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 9 tokens mean: 13.83 tokens max: 32 tokens	min: 9 tokens mean: 13.76 tokens max: 32 tokens	min: 0.1 mean: 0.3 max: 0.9

• Samples:

  sentence1	sentence2	score
  S'intéresser aux arts visuels, au design et à l'esthétique	Programmer en Python, Java ou JavaScript avec bonnes pratiques	0.4
  Conduire des études de marché qualitatives et quantitatives	Élaborer et suivre les budgets prévisionnels et contrôler les écarts	0.1
  Faire preuve de rigueur et de précision dans son travail	Conduire des études de marché qualitatives et quantitatives	0.4

• Loss: CosineSimilarityLoss with these parameters:

  {
      "loss_fct": "torch.nn.modules.loss.MSELoss"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• num_train_epochs: 5
• warmup_ratio: 0.1
• fp16: True
• load_best_model_at_end: True

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss	eval-referentiel_spearman_cosine
0.2817	20	0.1849	-	-
0.5634	40	0.0464	-	-
0.7042	50	-	0.0242	0.7628
0.8451	60	0.0285	-	-
1.1268	80	0.0156	-	-
1.4085	100	0.0092	0.0067	0.8870
1.6901	120	0.0064	-	-
1.9718	140	0.0049	-	-
2.1127	150	-	0.0030	0.8930
2.2535	160	0.0031	-	-
2.5352	180	0.0027	-	-
2.8169	200	0.0039	0.0026	0.8957
3.0986	220	0.0054	-	-
3.3803	240	0.0039	-	-
3.5211	250	-	0.0023	0.8956
3.6620	260	0.0018	-	-
3.9437	280	0.0027	-	-
4.2254	300	0.002	0.0020	0.8957
4.5070	320	0.0027	-	-
4.7887	340	0.0023	-	-
4.9296	350	-	0.0019	0.8957

Framework Versions

• Python: 3.13.5
• Sentence Transformers: 5.2.0
• Transformers: 4.57.1
• PyTorch: 2.9.1+cu128
• Accelerate: 1.12.0
• Datasets: 4.7.0
• Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}