sylviachangdou-prayer/stylematch-authorship-v1

SentenceTransformer based on Blablablab/multilingual-style-representation

This is a sentence-transformers model finetuned from Blablablab/multilingual-style-representation. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Blablablab/multilingual-style-representation
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'XLMRobertaModel'})
  (1): Pooling({'embedding_dimension': 1024, 'pooling_mode': 'mean', 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'first time entering this dining-room, and naturally taking no special heed at what door he entered, will, upon rising to depart, commit the strangest blunders. Such, for instance, as opening the first door that comes handy, and finding himself stealing up-stairs by the back passage. Shutting that door, he will proceed to another, and be aghast at the cellar yawning at his feet. Trying a third, he surprises the housemaid at her work. In the end, no more relying on his own unaided efforts, he procures a trusty guide in some passing person, and in good time successfully emerges. Perhaps as curious a blunder as any, was that of a certain stylish young gentleman, a great exquisite, in whose judicious eyes my daughter Anna had found especial favor. He called upon the young lady one evening, and found her alone in the dining-room at her needlework. He stayed rather late;',
    "might be placed over his grave. Though with the sentiment of the epitaph Plain Talk did not disagree, he himself being at times of a hypochondriac turn--at least, so many said--yet the language struck him as too much drawn out; so, after consultation with Old Prudence, he decided upon making use of the epitaph, yet not without verbal retrenchments. And though, when these were made, the thing still appeared wordy to him, nevertheless, thinking that, since a dead man was to be spoken about, it was but just to let him speak for himself, especially when he spoke sincerely, and when, by so doing, the more salutary lesson would be given, he had the retrenched inscription chiseled as follows upon the stone. 'HERE LIE THE REMAINS OF CHINA ASTER THE CANDLE-MAKER, WHOSE CAREER WAS AN EXAMPLE OF THE TRUTH OF SCRIPTURE, AS FOUND IN THE SOBER PHILOSOPHY OF SOLOMON THE",
    'terrible mark upon her brow. Then the strong, soothing hand of the austere maid drew her head down on to the cushion, and the wild anger died away into passionate sobbing. At last she continued: “I will tell you about last night. You are aware, perhaps, that in this house all the servants sleep in the modern wing. This central block is made up of the dwelling-rooms, with the kitchen behind and our bedroom above. My maid, Theresa, sleeps above my room. There is no one else, and no sound could alarm those who are in the farther wing. This must have been well-known to the robbers, or they would not have acted as they did. “Sir Eustace retired about half-past ten. The servants had already gone to their quarters. Only my maid was up, and she had remained in her room at the top of the house until I',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.8258, 0.8016],
#         [0.8258, 1.0000, 0.6494],
#         [0.8016, 0.6494, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

• Size: 26,976 training samples
• Columns: sentence_0, sentence_1, and sentence_2
• Approximate statistics based on the first 100 samples:

  	sentence_0	sentence_1	sentence_2
  type	string	string	string
  modality	text	text	text
  details	min: 179 tokens mean: 220.95 tokens max: 300 tokens	min: 117 tokens mean: 217.77 tokens max: 274 tokens	min: 177 tokens mean: 216.25 tokens max: 369 tokens

• Samples:

  sentence_0	sentence_1	sentence_2
  I went on: "As to life, that's another thing. And I don't know that one ought to blame you very much--though it seemed rather an excessive step. I wonder now if it isn't the ugliness rather than the pain of the struggle which . . . " She shuddered visibly: "But I do blame myself," she exclaimed with feeling. "I am ashamed." And, dropping her head, she looked in a moment the very picture of remorse and shame. "Well, you will be going away from all its horrors," I said. "And surely you are not afraid of the sea. You are a sailor's granddaughter, I understand." She sighed deeply. She remembered her grandfather only a little. He was a clean-shaven man with a ruddy complexion and long, perfectly white hair. He used to take her on his knee, and putting his face near hers, talk to her in loving whispers.	be mighty careful then about what I said to anybody. Someone might have put it into his head to hang me at some yard-arm or other." "I envy your experience all the same," said Cosmo amiably. "I suppose your conscience was clear?" "I have always been most careful not to give my conscience any license to trouble me," retorted the other with a certain curtness of tone which was not offensive; "and I have lived now for some considerable time. I am really much older than I look," he concluded, giving Cosmo such a keen glance that the young man could not I help a smile. The other went on looking at him steadily for a while, then let his eyes wander to a door in a distant part of the long room as if impatient for the coming of the dinner. Then giving it up: "A man who has	alarmed the young fellow, and he was going to make a noise; but I said: "Save your wind; if you open your mouth you are dead, sure. Tackle your instrument. Lively, now! Call Camelot." "This doth amaze me! How should such as you know aught of such matters as--" "Call Camelot! I am a desperate man. Call Camelot, or get away from the instrument and I will do it myself." "What--you?" "Yes--certainly. Stop gabbling. Call the palace." He made the call. "Now, then, call Clarence." "Clarence who?" "Never mind Clarence who. Say you want Clarence; you'll get an answer." He did so. We waited five nerve-straining minutes--ten minutes --how long it did seem!--and then came a click that was as familiar to me as a human voice; for Clarence had been my own pupil. "Now, my lad, vacate! They would have known my touch, maybe, and so your call was
  American exports, at stable prices and with more liberal government guarantees and financing--curbing tax and customs loopholes that encourage undue spending of private dollars abroad--and (through OECD, NATO and otherwise) sharing with our allies all efforts to provide for the common defense of the free world and the hopes for growth of the less developed lands. While the current deficit lasts, ways will be found to ease our dollar outlays abroad without placing the full burden on the families of men whom we have asked to serve our Flag overseas. In short, whatever is required will be done to back up all our efforts abroad, and to make certain that, in the future as in the past, the dollar is as "sound as a dollar." III. But more than our exchange of international payments is out of balance. The current Federal budget for fiscal 1961 is almost certain to show	conquer the deserts, eradicate disease, tap the ocean depths, and encourage the arts and commerce. Let both sides unite to heed in all corners of the earth the command of Isaiah. . .to "undo the heavy burdens. . . let the oppressed go free." And if a beachhead of co-operation may push back the jungle of suspicion. . . let both sides join in creating not a new balance of power. . . but a new world of law. . .where the strong are just. . . and the weak secure. . .and the peace preserved. . . . All this will not be finished in the first one hundred days. Nor will it be finished in the first one thousand days. . . nor in the life of this administration, nor even perhaps in our lifetime on this planet. But let us begin. In your hands, my fellow citizens.	Given the cooperation of the Federal Reserve System, which I so earnestly seek, I am confident that this movement can continue. I pledge the American people that I will do everything in a President's power to lower interest rates and to ease money in this country. The Federal Home Loan Bank Board tomorrow morning will announce that it will make immediately available to savings and loan associations an additional $1 billion, and will lower from 6 percent to 5 3/4 percent the interest rate charged on those loans. We shall continue on a sensible course of fiscal and budgetary policy that we believe will keep our economy growing without new inflationary spirals; that will finance responsibly the needs of our men in Vietnam and the progress of our people at home; that will support a significant improvement in our export surplus, and will press forward toward easier credit and toward
  because of the efforts of hundreds of volunteers and donations from people in all walks of life. Volunteers and financial support to provide volunteers with the assistance they need, is critical to reaching Project Gutenberg-tm’s goals and ensuring that the Project Gutenberg-tm collection will remain freely available for generations to come. In 2001, the Project Gutenberg Literary Archive Foundation was created to provide a secure and permanent future for Project Gutenberg-tm and future generations. To learn more about the Project Gutenberg Literary Archive Foundation and how your efforts and donations can help, see Sections 3 and 4 and the Foundation web page at http://www.pglaf.org. Section 3. Information about the Project Gutenberg Literary Archive Foundation The Project Gutenberg Literary Archive Foundation is a non profit 501(c)(3) educational corporation organized under the laws of the state of Mississippi and granted tax exempt status by the Internal Revenue Service. The Foundat...	is for something more vital: imagination. Imagination is as the immortal God which should assume flesh for the redemption of mortal passion (Shelley). Imagination is our only guide into the world created by words. Whether those words are signed or unsigned becomes, as soon as the imagination redeems us, a matter of no importance, because we have approximated to the state in which they were written, and there are no names down there, no personality as we understand personality, no marrying or giving in marriage. What there is down there--ah, that is another enquiry, and may the clergymen and the scientists pursue it more successfully in the future than they have in the past.	on again, with a patient tenderness that I was deeply grateful for. At first, as I lay quiet on the sofa, I found it painfully difficult, I might say impossible, to get rid of the impression of the glare of the flames, their hurry and noise, and the fierce burning smell. If I dozed for a minute, I was awakened by Miss Havisham’s cries, and by her running at me with all that height of fire above her head. This pain of the mind was much harder to strive against than any bodily pain I suffered; and Herbert, seeing that, did his utmost to hold my attention engaged. Neither of us spoke of the boat, but we both thought of it. That was made apparent by our avoidance of the subject, and by our agreeing—without agreement—to make my recovery of the use of my hands a question of so many

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 32
• num_train_epochs: 1
• per_device_eval_batch_size: 32
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• per_device_train_batch_size: 32
• num_train_epochs: 1
• max_steps: -1
• learning_rate: 5e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1
• label_smoothing_factor: 0.0
• bf16: False
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 32
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: False
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: None
• fsdp_config: None
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.5931	500	3.1777

Training Time

• Training: 44.3 minutes

Framework Versions

• Python: 3.12.13
• Sentence Transformers: 5.6.0
• Transformers: 5.12.1
• PyTorch: 2.11.0+cu128
• Accelerate: 1.14.0
• Datasets: 4.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}