helmis/e5-small-it-profiles

SentenceTransformer based on intfloat/multilingual-e5-small

This is a sentence-transformers model finetuned from intfloat/multilingual-e5-small. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: intfloat/multilingual-e5-small
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("helmis/e5-small-it-profiles")
# Run inference
sentences = [
    'Administrateur systèmes, stockage SAN NAS, NetApp, Pure Storage',
    'Admin sys Linux, automatisation Ansible, scripts Python bash',
    'Ingénieur data, feature store, Feast, Tecton, ML platform',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Triplet

• Dataset: val-it-profiles
• Evaluated with TripletEvaluator

Metric	Value
cosine_accuracy	0.9968

Training Details

Training Dataset

Unnamed Dataset

• Size: 2,480 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 10 tokens mean: 17.53 tokens max: 27 tokens	min: 10 tokens mean: 17.59 tokens max: 27 tokens	min: 10 tokens mean: 17.45 tokens max: 27 tokens

• Samples:

  anchor	positive	negative
  Python engineer, messaging queues, Celery, RabbitMQ	Développeur Python, APIs tierces, intégrations Stripe, Twilio	Testeur, gestion défauts, Jira, rapports, métriques qualité
  Senior Flutter developer, architecture propre, feature-first	Flutter engineer, Firebase Firestore, Auth, Storage, FCM	Analyste de données, Google BigQuery, requêtes SQL complexes
  Python developer, computer vision, OpenCV, YOLO, PIL	Senior Python developer, async programming, asyncio, aiohttp	Développeur Angular, Auth0, Keycloak, OIDC, guards routes

• Loss: TripletLoss with these parameters:

  {
      "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
      "triplet_margin": 5
  }
  

Evaluation Dataset

Unnamed Dataset

• Size: 310 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 310 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 10 tokens mean: 17.57 tokens max: 26 tokens	min: 11 tokens mean: 17.59 tokens max: 26 tokens	min: 11 tokens mean: 17.39 tokens max: 26 tokens

• Samples:

  anchor	positive	negative
  Angular developer, animations Angular, CDK Overlay	Développeur Angular, PWA, service workers, push notifications	Rust developer, game development, Bevy engine
  Security engineer, application security, SAST, DAST, SCA	Cybersecurity engineer, threat intelligence, vulnerability management	Data analyst, ETL, transformation données, nettoyage, qualité
  Analyste données, intégration sources multiples, consolidation	Data analyst, présentation résultats, communication non-technique	Java engineer with Spring Security and JWT authentication

• Loss: TripletLoss with these parameters:

  {
      "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
      "triplet_margin": 5
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 64
• learning_rate: 2e-05
• num_train_epochs: 10
• warmup_ratio: 0.1
• fp16: True
• load_best_model_at_end: True

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 64
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 10
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	Validation Loss	val-it-profiles_cosine_accuracy
0.2564	20	4.9364	-	-
0.5128	40	4.8798	-	-
0.7692	60	4.6359	-	-
1.0	78	-	3.9695	0.9871
1.0256	80	4.2702	-	-
1.2821	100	4.0951	-	-
1.5385	120	4.0065	-	-
1.7949	140	3.937	-	-
2.0	156	-	3.7489	0.9935
2.0513	160	3.9045	-	-
2.3077	180	3.8515	-	-
2.5641	200	3.8601	-	-
2.8205	220	3.8696	-	-
3.0	234	-	3.7102	0.9935
3.0769	240	3.8469	-	-
3.3333	260	3.8189	-	-
3.5897	280	3.8238	-	-
3.8462	300	3.7987	-	-
4.0	312	-	3.6924	0.9935
4.1026	320	3.783	-	-
4.3590	340	3.7763	-	-
4.6154	360	3.773	-	-
4.8718	380	3.7956	-	-
5.0	390	-	3.6828	0.9935
5.1282	400	3.7761	-	-
5.3846	420	3.7795	-	-
5.6410	440	3.7608	-	-
5.8974	460	3.7445	-	-
6.0	468	-	3.6786	0.9935
6.1538	480	3.7554	-	-
6.4103	500	3.7717	-	-
6.6667	520	3.7639	-	-
6.9231	540	3.7349	-	-
7.0	546	-	3.6746	0.9935
7.1795	560	3.7367	-	-
7.4359	580	3.756	-	-
7.6923	600	3.7793	-	-
7.9487	620	3.716	-	-
8.0	624	-	3.6725	0.9968
8.2051	640	3.7199	-	-
8.4615	660	3.75	-	-
8.7179	680	3.7433	-	-
8.9744	700	3.756	-	-
9.0	702	-	3.6718	0.9968
9.2308	720	3.7158	-	-
9.4872	740	3.7473	-	-
9.7436	760	3.7552	-	-
10.0	780	3.7348	3.6713	0.9968

Framework Versions

• Python: 3.12.13
• Sentence Transformers: 3.3.1
• Transformers: 4.57.6
• PyTorch: 2.10.0+cu128
• Accelerate: 1.13.0
• Datasets: 4.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}