vivekkopthsd/fiqa-retriever-bge-small

SentenceTransformer based on BAAI/bge-small-en-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-small-en-v1.5. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-small-en-v1.5
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 384, 'pooling_mode': 'cls', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("vivekkopthsd/fiqa-retriever-bge-small")
# Run inference
queries = [
    'Stock grant, taxes, and the IRS',
]
documents = [
    "I went through this too. There's a safe-harbor provision. If you prepay as estimated tax payments, 110% of your previous year's tax liability, there's no penalty for underpayment of the big liquidity-event tax liability.  https://www.irs.gov/publications/p17/ch04.html That's with the feds. Your state may have different rules. You would be very wise indeed to hire an accountant to prepare your return this year. If I were you I'd ask your company's CFO or finance chief to suggest somebody.   Congratulations, by the way.",
    "The value of the asset doesn't change just because of the exchange rate change. If a thing (valued in USD) costs USD $1 and USD $1 = CAN $1 (so the thing is also valued CAN $1) today and tomorrow CAN $1 worth USD $0.5 - the thing will continue being worth USD $1. If the thing is valued in CAN $, after the exchange rate change, the thing will be worth USD $2, but will still be valued CAN $1. What you're talking about is price quotes, not value. Price quotes will very quickly reach the value, since any deviation will be used by the traders to make profits on arbitrage. And algo-traders will make it happen much quicker than you can even notice the arbitrage existence.",
    '"Things I would specifically draw your attention to: the contract typically allows for an ""option"" to purchase; it does not typically compel purchase, although this is seen the purchase price is negotiated before anything gets signed the option to buy is typically available to the renter for the period of the lease contract (ie., if it\'s a 12 month contract the renter can opt to buy at any time in that 12 months) the amount of rent paid over time that will be applied to the purchase price is negotiated up-front before anything gets signed rent is paid at a slight premium (as Joe notes, if the rent should be $1000 per month, expect to pay $1200 per month) if the renter walks away they walk away empty handed; they do not get back the premium Having said all that - it\'s a contract negotiated between renter and seller and all of this is negotiable.  See also, ehow for a good overview."',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 384] [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[ 0.4181,  0.0066, -0.0466]])

Evaluation

Metrics

Information Retrieval

• Dataset: fiqa-test
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.3858
cosine_accuracy@3	0.5324
cosine_accuracy@5	0.5818
cosine_accuracy@10	0.6667
cosine_precision@1	0.3858
cosine_precision@3	0.2382
cosine_precision@5	0.1728
cosine_precision@10	0.11
cosine_recall@1	0.1952
cosine_recall@3	0.321
cosine_recall@5	0.3716
cosine_recall@10	0.4555
cosine_ndcg@10	0.3923
cosine_mrr@10	0.4731
cosine_map@100	0.3345

Training Details

Training Dataset

Unnamed Dataset

• Size: 14,080 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 100 samples:

  	sentence_0	sentence_1
  type	string	string
  modality	text	text
  details	min: 8 tokens mean: 15.48 tokens max: 28 tokens	min: 22 tokens mean: 185.17 tokens max: 256 tokens

• Samples:

  sentence_0	sentence_1
  How come I can't sell short certain stocks? My broker says “no shares are available”	In finance, short selling (also known as shorting or going short) is the practice of selling assets, usually securities, that have been borrowed from a third party (usually a broker) with the intention of buying identical assets back at a later date to return to the lender. Remember your broker has to borrow it from somewhere, other clients or if they hold those specific stocks themselves. So if it isn't possible for them to lend you those stocks, they wouldn't. High P/E stocks would find more sellers than buyers, and if the broker has to deliver them, it would be a nightmare for him to deliver all those stocks, which he had lent you(others) back to whom he had borrowed from, as well as to people who had gone long(buy) when you went short(sell). And if every body is selling there is going to be a dearth of stocks to be borrowed from as everybody around is selling instead of buying.
  How to choose a company for an IRA?	"I use TIAA-Cref for my 403(b) and Fidelity for my solo 401(k) and IRAs. I have previously used Vanguard and have also used other discount brokers for my IRA. All of these companies will charge you nothing for an IRA, so there's really no point in comparing cost in that respect. They are all the ""cheapest"" in this respect. Each one will allow you to purchase their mutual funds and those of their partners for free. They will charge you some kind of fee to invest in mutual funds of their competitors (like $35 or something). So the real question is this: which of these institutions offers the best mutual and index funds. While they are not the worst out there, you will find that TIAA-Cref are dominated by both Vanguard and Fidelity. The latter two offer far more and larger funds and their funds will always have lower expense ratios than their TIAA-Cref equivalent. If I could take my money out of TIAA-Cref and put it in Fidelity, I'd do so right now. BTW, you may or may not want t...
  Why are currency forwards needed?	e.g. a European company has to pay 1 million USD exactly one year from now While that is theoretically possible, that is not a very common case. Mostly likely if they had to make a 1 million USD payment a year from now and they had the cash on hand they would be able to just make the payment today. A more common scenario for currency forwards is for investment hedging. Say that European company wants to buy into a mutual fund of some sort, say FUSEX. That is a USD based mutual fund. You can't buy into it directly with Euros. So if the company wants to buy into the fund they would need to convert their Euros to to USD. But now they have an extra risk parameter. They are not just exposed to the fluctuations of the fund, they are also exposed to the fluctuations of the currency market. Perhaps that fund will make a killing, but the exchange rate will tank and they will lose all their gains. By creating a forward to hedge their currency exposure risk they do not face this risk (flip side...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 256
• fp16: True
• per_device_eval_batch_size: 256
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• per_device_train_batch_size: 256
• num_train_epochs: 3
• max_steps: -1
• learning_rate: 5e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1
• label_smoothing_factor: 0.0
• bf16: False
• fp16: True
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 256
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: False
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: None
• fsdp_config: None
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	fiqa-test_cosine_ndcg@10
-1	-1	0.3923

Training Time

• Training: 1.7 minutes

Framework Versions

• Python: 3.12.3
• Sentence Transformers: 5.6.1
• Transformers: 5.10.1
• PyTorch: 2.12.0+cu130
• Accelerate: 1.13.0
• Datasets: 4.8.5
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}