bugBug04S/legal-embed-modernbert-v1

SentenceTransformer based on nomic-ai/modernbert-embed-base

This is a sentence-transformers model finetuned from nomic-ai/modernbert-embed-base. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: nomic-ai/modernbert-embed-base
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'ModernBertModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("bugBug04S/legal-embed-modernbert-v1")
# Run inference
sentences = [
    "search_query: Can I add a deadlock to the front door of my unit? I have recently bought a unit in a rough area. The front door to the apartment complex does not click shut properly so anyone from the general public can walk in. The door to my unit has one of those cheap push button door handle locks. I have spoken to the body corporate and the building manager and there are currently no plans to fix the apartment complex door. I asked about adding some more security to the door of my unit, namely a good quality (fire rated) deadlock. But they replied back that deadlocks are not permitted for fire and aesthetic reasons, even locks that are fire rated. What is the legality of prohibiting deadlocks? Are fire rated deadlocks permitted legally on fire exits? They mentioned something about fire safety laws changing in 2019 but I didn't get a full understanding. Is this documented anywhere?",
    'search_document: National Construction Code For Class 2 Buildings (apartments) the relevant volume is No 2. Door hardware on fire doors must comply with AS 1905.1 (Specification C3.4 Clause 2), that is, it has to be "fire-rated". There are plenty of fire-rated deadlocks available. Normally, a door in a path of travel to an exit cannot be key operated in the direction of travel, however, there are a number of exceptions. One is if it "serves only, or is within— (A) a sole-occupancy unit in a Class 2 building" - like yours. So there is no regulatory obstacle to this. Strata Law The door (and its hardware) is part of the common property - it belongs to the body corporate, not you. Technically, you own the paint on the inside, they own everything else. So, even if you paid for the lock, it would belong to the body-corporate and they would bear all the risk of loss and damage including if it wasn\'t installed properly. They may be unwilling to take this risk. They can legally prevent you from doing anything to their property. However, you can insist that the body-corporate repair the front door. Having "no plans" is not an excuse for failing in their legal duty to repair and maintain the common property. If they refuse, you can use the dispute resolution process.',
    'search_document: Are the reasons (mom can\'t stay because of covid-19 and the son is too young to live by himself) legally justifiable reasons? Unfortunately, not in this case. The landlord can prevail by arguing that the mom, insofar as a holder of a non-immigrant visa, knew or should have known of the risks of being denied entry in the US. In terms of the Restatement (Second) of Contracts at § 154 (a) & (b), the mom is the " Party [who] Bears the Risk of a Mistake ". Since the risks of which the mom was aware include even arbitrary ones such as the USCIS officer\'s discretion , an allegation of "supervening circumstances" as per the coronavirus crisis is very unlikely to add any merit to the mom\'s legal position. Is the 8 months compensate reasonable? Although this is admittedly subjective, at first glance it appears reasonable or perhaps even generous (the landlord is proposing the midpoint: half of the remaining 16 months) considering that the landlord is entitled to the completion of the lease. The " 8 months as compensation " is essentially the landlord\'s proposal to settle his otherwise viable claim of breach of contract . You as lease holders can always attempt to negotiate and make alternative proposals, which is what you are doing already . Here, the landlord seeks to obviate court proceedings that can be a drag for both him and especially for the mom because she would be unable even to present her arguments in court and/or quasi-judicial proceedings. A settlement would benefit the landlord also from the standpoint that he would no longer be required to prove mitigation of damages (if that is a requirement in his jurisdiction). And, unless the settlement provides otherwise, the landlord would be free to immediately re-rent the unit without having to reimburse the mom (or the person(s) whose name is in the lease). That being said, it is important for the lease holder(s) to secure in writing (specifically in the settlement document) a statement from the landlord in the sense t',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.7668, 0.1163],
#         [0.7668, 1.0000, 0.1873],
#         [0.1163, 0.1873, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

• Size: 16,068 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 100 samples:

  	anchor	positive
  type	string	string
  modality	text	text
  details	min: 38 tokens mean: 171.49 tokens max: 256 tokens	min: 42 tokens mean: 198.15 tokens max: 256 tokens

• Samples:

  anchor	positive
  search_query: Does Mojang break the law with restrictions in their EULA for Minecraft? Minecraft is a world building game that is produced/sold by Mojang AB. Minecraft may be played both offline (local data) or online (connecting to a server). Minecraft's EULA says that you may not sell items to players (real money) that give them in-game benefits outside of cosmetics. However, that is Minecraft's clients EULA. How can they enforce servers to follow that rule if the servers are not owned by Mojang and they are not using Mojang's proprietary software. Wouldn't this be like selling a television set and saying to broadcasters they must follow any rules I impose.	search_document: The rule you are alluding to with respect to a television set is called the " first sale doctrine " which basically prohibits copyright and trademark owners from limiting the ability of a buyer of a good (like a CD or authorized logo T-Shirt) protected by copyright or trademark, from limiting further sales of that good (or the manner in which the good is used by its new owner) after a first retail sale of the good with copyright or trademark protections. This doctrine was derived from an old common law rule that invalidated "restraints on alienation" of property other than intellectual property on public policy grounds, and like the "restrain on alienation" rule for tangible property, the first sale doctrine that applies to intellectual property was also (at least originally) a court created common law rule. But Minecraft isn't, conceptually, a good. It is a continuing service provided over the Internet, and firms that provide continuing services on a licensed basis, a...
  search_query: Can I post edited comedy images of Indian politician in social website I edited an image of an Indian politician. Added some comedy picture taken from a Tamil movie. And also wrote some thing funny about that politicians. Is that legal. Can I post that image to a social website.. Do I face any problem in future for posting such images. What does the law say.	search_document: Legally you face no problem. The section 66A of the Indian IT Act, which used to be previously misused for penalizing anyone who dared insult a politician, has been struck down as unconstitutional by the Supreme Court of India. But the police could still detain you for 48 hours (legally) without giving any grounds; they are required to do that, but the police are seldom held accountable. The supporters of the said politician can vandalize your home and office without fear of legal action. I am not a lawyer. Whatever is posted above is my opinion and data that I believe to be true to the best of my knowledge and resources available to me. Please contact a lawyer for professional advice.
  search_query: Are CuddleFix participants engaged in prostitution? Is CuddleFix ( http://www.cuddlefix.com ) prostitution, or not? If not, where (or at what actions) is the line drawn exactly? In brief: you pay them to come to your house and lie in bed with you and spoon. With clothes on. No kissing. Regarding jurisdiction – let's say Alabama, then the U.S. at large, and finally "the West" generally.	search_document: See http://codes.lp.findlaw.com/alcode/13A/12/3/2 . Is the cuddling or spooning a sexual act or sexual contact? A court would have to decide. Perhaps there is some binding precedent on the books; I do not know. The answer to this question could be different for different jurisdictions in Alabama, since local ordinances might exist. It will almost certainly be different for different states in the US, let alone "the west" generally, whatever that means.

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "CachedMultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 64
• num_train_epochs: 1
• learning_rate: 1e-05
• warmup_steps: 0.1
• fp16: True

All Hyperparameters

Click to expand

• per_device_train_batch_size: 64
• num_train_epochs: 1
• max_steps: -1
• learning_rate: 1e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0.1
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1.0
• label_smoothing_factor: 0.0
• bf16: False
• fp16: True
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 8
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: False
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: None
• fsdp_config: None
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.0992	25	1.8452
0.1984	50	0.9378
0.2976	75	0.8986
0.3968	100	0.7386
0.4960	125	0.7398
0.5952	150	0.7222
0.6944	175	0.7030
0.7937	200	0.5992
0.8929	225	0.6804
0.9921	250	0.6471

Training Time

• Training: 32.7 minutes

Framework Versions

• Python: 3.12.13
• Sentence Transformers: 5.6.0
• Transformers: 5.13.1
• PyTorch: 2.11.0+cu128
• Accelerate: 1.14.0
• Datasets: 4.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

CachedMultipleNegativesRankingLoss

@misc{gao2021scaling,
    title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup},
    author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan},
    year={2021},
    eprint={2101.06983},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}