hasinthakapiyumal/bge-reasoner-embed-ai-patterns

SentenceTransformer based on BAAI/bge-reasoner-embed-qwen3-8b-0923

This is a sentence-transformers model finetuned from BAAI/bge-reasoner-embed-qwen3-8b-0923. It maps sentences & paragraphs to a 4096-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-reasoner-embed-qwen3-8b-0923
• Maximum Sequence Length: 768 tokens
• Output Dimensionality: 4096 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'Qwen3Model'})
  (1): Pooling({'embedding_dimension': 4096, 'pooling_mode': 'lasttoken', 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    'This code implements a nearest neighbor distance pattern by calculating the minimum Euclidean distance from query points to a set of sample points. It leverages an optimized pairwise squared Euclidean distance computation, a fundamental metric widely used in machine learning for tasks like clustering, classification, and density estimation, leveraging vectorized operations for efficiency.',
]
documents = [
    'The code implements an adaptive probabilistic modeling framework, featuring automated univariate distribution selection based on statistical fit (Kolmogorov-Smirnov test) and dynamic candidate filtering. It utilizes a Gaussian copula model for multivariate distributions, separating marginal distribution fitting from dependency modeling via a correlation matrix. This architecture supports conditional inference and sampling by transforming data into a standard normal space.',
    'The code establishes a **World Model** pattern by providing a structured, semantic representation of an autonomous driving environment, organizing map data into distinct vector and raster layers like lanes, roadblocks, and drivable areas. It further implements **Perception and Querying** patterns, offering an AI agent capabilities to perform complex spatial queries such as point-in-polygon checks, proximity searches, and nearest object distance calculations, essential for environmental understanding and navigation.',
    'This code establishes a modular AI pipeline, integrating a `VectorDB` and an `embedder` for semantic processing and retrieval-augmented capabilities. It employs a pluggable "scanner" architecture, where various AI-powered modules can be dynamically configured and instantiated, optionally leveraging the vector database and embedder for tasks like input/output analysis or moderation. This design facilitates a flexible and extensible framework for managing AI system interactions.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 4096] [3, 4096]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.5820, 0.6914, 0.5234]], dtype=torch.bfloat16)

Training Details

Training Dataset

Unnamed Dataset

• Size: 2,333 training samples
• Columns: sentence_0, sentence_1, and label
• Approximate statistics based on the first 100 samples:

  	sentence_0	sentence_1	label
  type	string	string	float
  modality	text	text
  details	min: 53 tokens mean: 88.72 tokens max: 135 tokens	min: 53 tokens mean: 87.24 tokens max: 134 tokens	min: 0.0 mean: 0.28 max: 1.0

• Samples:

  sentence_0	sentence_1	label
  The code demonstrates a pattern for building highly customizable AI agents (CustomizeAgent) that integrate external tools (e.g., MCPToolkit, ArxivToolkit) to augment their capabilities. A core AI pattern is the robust handling of structured output through various parsing modes (json, xml, title, custom), ensuring precise information extraction. Furthermore, it showcases the orchestration of these agents and tools into complex workflows (WorkFlowGraph) for multi-step task execution, representing a multi-agent system pattern.	This code implements a pattern for visualizing AI agent states and behaviors by processing SimulationHistory data, specifically DetectionsTracks of tracked_objects. It extracts key features like position, velocity, and heading, and employs a tracking mechanism to assign consistent IDs to agents across frames. This processed data dynamically renders interactive plots, enabling real-time observation of AI agent dynamics.	0.0
  This code implements a pattern for automated dataset curation and preprocessing, systematically loading and filtering classification datasets from OpenML and Kaggle based on criteria like missing values, feature count, sample size, and number of classes. It prepares this data for AI model consumption by applying capping mechanisms, handling multiclass/binary scenarios, and transforming it into PyTorch tensors with controlled shuffling or sorting, indicating its use for deep learning benchmarks or training.	This code implements a recursive aggregation pattern for hierarchical data, specifically accumulating log counts across a tree-like structure of 'spans.' In AI contexts, this pattern is crucial for observability and monitoring of complex AI pipelines or model inference traces. It enables the aggregation of operational metrics from individual components up to a higher-level view, facilitating debugging and performance analysis of multi-stage AI systems.	0.0
  This code implements a modular neural network layer construction pattern, providing configurable building blocks for fully-connected and 2D convolutional layers. It utilizes factory patterns (build_normalization, build_activation) to dynamically select and instantiate various normalization techniques and activation functions. The presence of fc_block vs fc_block2 and conv2d_block vs conv2d_block2 further demonstrates the exploration of architectural micro-patterns by varying the order of operations within these fundamental layers.	This code implements patterns for distributed deep learning, primarily focusing on synchronizing model states across multiple processes. It provides a generic all_reduce_dict function for aggregating dictionary values, which is specifically utilized by SyncNormHook to periodically average normalization layer statistics (e.g., BatchNorm's running mean/variance) during distributed training. This pattern ensures consistent global statistics for normalization layers, crucial for stable and effective training in multi-GPU environments.	0.0

• Loss: ContrastiveLoss with these parameters:

  {
      "distance_metric": "SiameseDistanceMetric.COSINE_DISTANCE",
      "margin": 0.5,
      "size_average": true
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

Training Time

• Training: 10.4 minutes

Framework Versions

• Python: 3.12.3
• Sentence Transformers: 5.7.0
• Transformers: 4.57.6
• PyTorch: 2.6.0+cu124
• Accelerate: 1.15.0
• Datasets: 5.0.1
• Tokenizers: 0.22.2

Additional Resources

• Training and Finetuning Embedding Models with Sentence Transformers: the end-to-end guide for training or finetuning Sentence Transformer models.
• Introduction to Matryoshka Embedding Models: variable-size embeddings that can be truncated with minimal quality loss.
• Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval: post-training compression of embedding vectors.
• Multimodal Embedding & Reranker Models with Sentence Transformers: use text, image, audio, and video models through the same API.
• Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers: train multimodal embedding models, with a Visual Document Retrieval walkthrough.

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

ContrastiveLoss

@inproceedings{hadsell2006dimensionality,
    author={Hadsell, R. and Chopra, S. and LeCun, Y.},
    booktitle={2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06)},
    title={Dimensionality Reduction by Learning an Invariant Mapping},
    year={2006},
    volume={2},
    number={},
    pages={1735-1742},
    doi={10.1109/CVPR.2006.100}
}