Upload embedding model

f1c49cb verified 8 days ago

46.7 kB

language:
  - en
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - dense
  - generated_from_trainer
  - dataset_size:59315
  - loss:MultipleNegativesRankingLoss
base_model: sentence-transformers/all-MiniLM-L6-v2
widget:
  - source_sentence: Johnny Depp plays policeman Ichabod Crane in which 1999 film?
    sentences:
      - >-
        Mythology in France epics and fairy tales as part of deeply embedded
        spiritual allegories and mythological archetypes: Mythology in France
        The mythologies in present-day France encompass the mythology of the
        Gauls, Franks, Normans, Bretons, and other peoples living in France,
        those ancient stories about divine or heroic beings that these
        particular cultures believed to be true and that often use supernatural
        events or characters to explain the nature of the universe and humanity.
        French mythology is listed for each culture. Bretons are a subset of the
        celtics that adopted Christianity. Celtic cosmology predominates their
        mythology: Gauls were another subset of Celtic people. Celtic
      - >-
        Johnny Depp in a snuff film in exchange for money for his family. Depp
        was a fan and friend of writer Hunter S. Thompson, and played his alter
        ego Raoul Duke in "Fear and Loathing in Las Vegas" (1998), Terry
        Gilliam's film adaptation of Thompson's pseudobiographical novel of the
        same name. Depp's next venture with Burton was the period film "Sleepy
        Hollow" (1999), in which he played Ichabod Crane opposite Christina
        Ricci and Christopher Walken. For his performance, Depp took inspiration
        from Angela Lansbury, Roddy McDowall and Basil Rathbone. He stated that
        he "always thought of Ichabod as a very delicate, fragile
      - >-
        Ichabod Crane Kinderhook town school district (Ichabod Crane Central
        School District) is also named for the Irving character. It is claimed
        by many in Tarrytown that Samuel Youngs is the original from whom Irving
        drew his character of Ichabod Crane". Author Gary Denis asserts that
        while the character of Ichabod Crane is loosely based on Kinderhook
        Schoolmaster, Jesse Merwin, it may possibly include elements from Samuel
        Youngs' life. Irving's characters drive the story and are most memorable
        because of his detail in describing each. He says of Ichabod Crane (the
        main character), 'He was tall, but exceedingly lank, with narrow
        shoulders, long
  - source_sentence: >-
      What is the name of Liam Gallagher's new band, whose first album is
      'Different Gear, Still Speeding'?
    sentences:
      - >-
        Different Gear, Still Speeding Different Gear, Still Speeding Different
        Gear, Still Speeding is the debut studio album by English rock band
        Beady Eye, released on 28 February 2011. It debuted at number three in
        the UK Albums Chart selling 66,817 in the first week. As of August 2012,
        the album has sold 174,487 copies in the UK. On "Different Gear, Still
        Speeding", all members contributed to the instrumentation, much like the
        later albums of Oasis. Reviews of the album have been generally
        mixed-to-favourable. According to review aggregator site Metacritic, the
        album has an average score of 65%. Reviewing for "Rolling Stone", Stacey
        Anderson, who
      - >-
        Mickey Mouse universe was originally going to be named Mortimer. Lillian
        Disney, Walt's wife, suggested the name Mickey instead. The first
        Mortimer was created by Walt Disney and Floyd Gottfredson for the
        comics. He was Minnie Mouse's ranch-owning cattleman uncle. He first
        appeared in the serial "Mickey Mouse in Death Valley" (1930). After
        that, he appeared or was referenced in many other Mickey Mouse comic
        strip adventures in the 1930s. He has occasionally appeared in more
        modern comics. In the 1936 cartoon short "Mickey's Rival", the second
        Mortimer was introduced as Mickey's competitor for Minnie's affections.
        In the comics, this Mortimer was briefly
      - >-
        Different Gear, Still Speeding gave the album 2.5 out of 5 stars, said
        "On Different Gear, the band attempts stripped down, Stones-y rock but
        ends up with 'Be Here Now'-style guitar bluster and Liam's blithely
        boilerplate lyrics". Drowned in Sound awarded the album 4/10 saying that
        "By and large it radiates the stolid competence of a band on auto-pilot,
        with a few flashes of likeable enthusiasm." The "Independent on Sunday"
        gave it 2/5 stars. Simon Goddard, reviewing for "Q", gave the album
        four-out-of-five stars and described it as "the strongest record Liam's
        made" since "(What's the Story) Morning Glory?", while Garry Mulholland,
        in his
  - source_sentence: The bands R.E.M. and the B52s both come from which town in Georgia?
    sentences:
      - >-
        E and M signaling by the resistance of the wire, but will normally be
        less than 100m for adequate noise immunity. The group of E&M signaling
        includes the following variations: E&M defines eight wires: "4-wire E&M"
        uses a 4-wire (2-pair) transmission path for the voice signal. "2-wire
        E&M" uses a single pair for both transmit and receive voice signal. This
        is much inferior to 4-wire E&M as the 2-wire interface uses hybrid
        transformers which reduce signal quality and can introduce echo. The
        mechanisms described so far only allow circuit seizure – on-hook and
        off-hook – to be signaled. In order to allow dialing over
      - >-
        Culture of Georgia (U.S. state) a fertile field for alternative rock
        bands since the late 1970s. Notable bands from Athens include R.E.M.,
        The B-52's, Widespread Panic, Drive-By Truckers, as well as bands from
        the Elephant 6 Recording Company most notably Neutral Milk Hotel. Rhythm
        and Blues is another important musical genre in Georgia. Ray Charles was
        one of popular music's most influential performers, fusing R&B, jazz,
        and country into many popular songs. Augusta native James Brown and
        Macon native Little Richard, two important figures in R&B history,
        started performing in Georgia clubs on the chitlin' circuit, fused
        gospel with blues and boogie-woogie to lay the
      - >-
        I Just Shot John Lennon I Just Shot John Lennon "I Just Shot John
        Lennon" is a song from The Cranberries' album "To the Faithful
        Departed". It is a narrative of the events of the night of December 8,
        1980, the night that musician John Lennon was murdered by Mark David
        Chapman in front of The Dakota in New York City. It is one of many
        tributes to Lennon, and also one of many other songs to recall the
        events of the night. After the narrative, there is commentary: "What a
        sad, and sorry and sickening sight". The title of the song comes from
        the
  - source_sentence: Who wrote Three Men In A Boat?
    sentences:
      - >-
        Three Men in a Boat Three Men in a Boat Three Men in a Boat (To Say
        Nothing of the Dog), published in 1889, is a humorous account by English
        writer Jerome K. Jerome of a two-week boating holiday on the Thames from
        Kingston upon Thames to Oxford and back to Kingston. The book was
        initially intended to be a serious travel guide, with accounts of local
        history along the route, but the humorous elements took over to the
        point where the serious and somewhat sentimental passages seem a
        distraction to the comic novel. One of the most praised things about
        "Three Men in a
      - >-
        Darwin, Northern Territory line. Darwin lies in the Northern Territory,
        on the Timor Sea. The city proper occupies a low bluff overlooking
        Darwin Harbour, flanked by Frances Bay to the east and Cullen Bay to the
        west. The remainder of the city is flat and low-lying, and coastal areas
        are home to recreational reserves, extensive beaches, and excellent
        fishing. Darwin is closer to the capitals of five other countries than
        to the capital of Australia: Darwin is away from Canberra. Dili (East
        Timor) is , Port Moresby (Papua New Guinea) is , Jakarta (Indonesia) is
        , Bandar Seri Begawan (Brunei) is , and
      - >-
        Three Men in a Boat 1891, "Three Women in One Boat: A River Sketch" by
        Constance MacEwen was published. This book relates the journey of three
        young university women who set out to emulate the river trip in "Three
        Men in a Boat" in an effort to raise the spirits of one of them, who is
        about to be expelled from university. To take the place of Montmorency,
        they bring a cat called Tintoretto. Three Men in a Boat is referenced in
        the 1956 parody novel on mountaineering, "The Ascent of Rum Doodle",
        where the head porter Bing is said to spend "much of his
  - source_sentence: In the RAF, what is the rank immediately above Squadron Leader?
    sentences:
      - >-
        Tybalt Tybalt Tybalt is the main antagonist in William Shakespeare's
        play "Romeo and Juliet". He is the son of Lady Capulet's brother,
        Juliet's short-tempered first cousin, and Romeo's rival. Tybalt shares
        the same name as the character Tibert/Tybalt the "Prince of Cats" in
        "Reynard the Fox", a point of mockery in the play. Mercutio repeatedly
        calls Tybalt "Prince of Cats" (perhaps referring not only to Reynard but
        to the Italian word cazzo as well). Luigi da Porto adapted the story as
        "Giulietta e Romeo" and included it in his "Historia novellamente
        ritrovata di due Nobili Amanti" published in 1530. Da Porto
      - >-
        Squadron leader RAF used major as the equivalent rank to squadron
        leader. Royal Naval Air Service lieutenant-commanders and Royal Flying
        Corps majors on 31 March 1918 became RAF majors on 1 April 1918. On 31
        August 1919, the RAF rank of major was superseded by squadron leader
        which has remained in continuous usage ever since. Promotion to squadron
        leader is strictly on merit, and requires the individual to be appointed
        to a Career Commission, which will see them remain in the RAF until
        retirement or voluntary resignation. Before the Second World War, a
        squadron leader commanded a squadron of aircraft. Today, however,
      - >-
        Squadron leader Squadron leader Squadron leader (Sqn Ldr in the RAF ;
        SQNLDR in the RAAF and RNZAF; formerly sometimes S/L in all services) is
        a commissioned rank in the Royal Air Force and the air forces of many
        countries which have historical British influence. It is also sometimes
        used as the English translation of an equivalent rank in countries which
        have a non-English air force-specific rank structure. An air force
        squadron leader ranks above flight lieutenant and immediately below wing
        commander and it is the most junior of the senior officer ranks. The air
        force rank of squadron leader has a
datasets:
  - sentence-transformers/trivia-qa-triplet
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
  - cosine_accuracy
model-index:
  - name: SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2
    results:
      - task:
          type: triplet
          name: Triplet
        dataset:
          name: trivia qa eval
          type: trivia_qa_eval
        metrics:
          - type: cosine_accuracy
            value: 0.8339999914169312
            name: Cosine Accuracy

SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2 on the trivia-qa-triplet dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: sentence-transformers/all-MiniLM-L6-v2
Maximum Sequence Length: 256 tokens
Output Dimensionality: 384 dimensions
Similarity Function: Cosine Similarity
Training Dataset:
- trivia-qa-triplet
Language: en

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'In the RAF, what is the rank immediately above Squadron Leader?',
    'Squadron leader Squadron leader Squadron leader (Sqn Ldr in the RAF ; SQNLDR in the RAAF and RNZAF; formerly sometimes S/L in all services) is a commissioned rank in the Royal Air Force and the air forces of many countries which have historical British influence. It is also sometimes used as the English translation of an equivalent rank in countries which have a non-English air force-specific rank structure. An air force squadron leader ranks above flight lieutenant and immediately below wing commander and it is the most junior of the senior officer ranks. The air force rank of squadron leader has a',
    'Squadron leader RAF used major as the equivalent rank to squadron leader. Royal Naval Air Service lieutenant-commanders and Royal Flying Corps majors on 31 March 1918 became RAF majors on 1 April 1918. On 31 August 1919, the RAF rank of major was superseded by squadron leader which has remained in continuous usage ever since. Promotion to squadron leader is strictly on merit, and requires the individual to be appointed to a Career Commission, which will see them remain in the RAF until retirement or voluntary resignation. Before the Second World War, a squadron leader commanded a squadron of aircraft. Today, however,',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.5605, 0.4394],
#         [0.5605, 1.0000, 0.5768],
#         [0.4394, 0.5768, 1.0000]])

Evaluation

Metrics

Triplet

Dataset: trivia_qa_eval
Evaluated with TripletEvaluator

Metric	Value
cosine_accuracy	0.834

Training Details

Training Dataset

trivia-qa-triplet

Dataset: trivia-qa-triplet at bfe9460
Size: 59,315 training samples
Columns: anchor, positive, and negative

Approximate statistics based on the first 1000 samples:

	anchor	positive	negative
type	string	string	string
details	min: 8 tokens mean: 20.08 tokens max: 71 tokens	min: 114 tokens mean: 139.64 tokens max: 226 tokens	min: 113 tokens mean: 138.81 tokens max: 256 tokens

Samples:

anchor	positive	negative
`"Sirius otherwise known as the ""Dog Star"" can be found in which constellation?"`	Sirius in fiction below)—and even if such were eventually discovered, with an estimated age of 230 million years the system is too young to have fostered the development of life or a complex biosphere. Sirius AB is the alpha star of the constellation Canis Major (the great dog, sometimes styled as Orion's hunting dog), whence its cognomen "the dog star". The most commonly used proper name of this star comes through the Latin "Sirius", from the Greek Σείριος ("Seirios", "glowing" or "scorcher"). The ancient Greeks observed that the appearance of Sirius heralded the hot and dry "dog days" of summer, and feared that	Dog days "Seírios", "Scorcher"), Sothis (, "Sôthis", a transcription of Egyptian "Spdt"), and the Dog Star (, "Kúōn"). The last name reflects the way Sirius follows the constellation Orion into the night sky. Sirius is by far the brightest proper star in the night sky, which caused ancient astronomers to take note of it around the world. In Egypt, its return to the night sky became known as a precursor to the annual flooding of the Nile and was worshipped as the goddess Sopdet. In Greece, it became known as the precursor of the unpleasantly hot phase of the summer. Greek poets
`Which Scottish town is the administrative centre of the Highland Region?`	Highland Scottish All operations are now part of the Stagecoach Group. Highland Scottish Highland Scottish Omnibuses Ltd was formed as a bus operating subsidiary of the Scottish Transport Group in June 1985 from Highland Omnibuses Ltd, and operated until October 1995 when the company was split into two - Highland Bus & Coach and Highland Country Buses. The companies have since remerged and operate today as Highland Country Buses. From its head office in Seafield Road, Inverness, Highland Scottish operated over the massive geographical, but sparsely populated, area of the Highland region of north west Scotland. Highland Scottish was the largest operator	Highland Scottish in the centre of its 'Highland Country' logo. In January 1996 Highland Country Buses was bought by National Express for £1.8m. Highland Bus & Coach, being the smaller of the two operators, continued to operate with the image its predecessor adopted. The two companies continued to exist under separate ownership until August 1998 when Rapson's bought Highland Country Buses back from National Express for £4m - £2.2m more than Rapson's originally sold the company for. Highland Country Buses is now a wholly owned subsidiary of Rapson's Coaches, and covers the operating area that Highland Scottish had on privatisation. The company
`Which brand of coffee is named after a hotel in Nashville, Tennessee?`	History of Nashville, Tennessee proprietor thereof, had served a special blend of coffee at the hotel's restaurant, and after drinking a cup of this coffee, Roosevelt proclaimed it "good to the last drop!" Cheek subsequently sold the blend to General Foods and to this day, Maxwell House coffee is enjoyed by millions. In 1913 Nashville was the last of several major cities in Tennessee to adopt a commission form of government, with all members of a small commission elected at-large. Compared to single-member districts, this change resulted in further limiting the political power of any African Americans who were able to vote, as their	Renaissance Nashville Hotel Renaissance Nashville Hotel The Renaissance Nashville Hotel is a hotel in Nashville, Tennessee. The building is 385 feet high with 31 floors. The hotel is physically connected to the Nashville Convention Center and is its anchor hotel. The hotel contains 649 rooms, 24 suites, 25 meeting rooms with 31,000 sq ft of meeting space, and 2 concierge levels including a Starbucks coffee shop, 2 lounges, and a full service restaurant. One of the lounges is located in an enclosed bridge walkway, spanning above Commerce Street, which connects the hotel to a parking garage across the street. This walkway was severely

Loss: MultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false
}

Evaluation Dataset

trivia-qa-triplet

Dataset: trivia-qa-triplet at bfe9460
Size: 1,000 evaluation samples
Columns: anchor, positive, and negative

Approximate statistics based on the first 1000 samples:

	anchor	positive	negative
type	string	string	string
details	min: 8 tokens mean: 20.18 tokens max: 103 tokens	min: 111 tokens mean: 139.08 tokens max: 225 tokens	min: 113 tokens mean: 138.94 tokens max: 256 tokens

Samples:

anchor	positive	negative
`What African country is projected to pass the United States in population by the year 2055?`	African immigration to the United States entering the United States. It is estimated that the current population of African immigrants to the United States is about 2.1 million. According to the Migration Policy Institute, as of 2009 two-thirds of the African immigrants were from either East or West Africa. Countries with the most immigrants to the U.S. are Nigeria, Egypt, Ethiopia, Ghana, South Africa, Somalia, Eritrea, and Kenya. Seventy five percent (75%) of the African immigrants to the U.S. come from 12 of the 55 countries, namely Nigeria, Egypt, Ghana, Ethiopia, South Africa, Kenya, Liberia, Somalia, Morocco, Cape Verde, Sierra Leone and Sudan (including what is	African immigration to the United States African countries due to many skilled hard-working Africans leaving Africa to seek their economic fortunes in the U.S. mainly and elsewhere. One major factor that contributes to migration from Africa to the United States is labor opportunities. It has been relatively easier for African immigrants for Africans with advanced education to leave and enter international labor markets. In addition, many Africans move to the United States for advanced training. For example, doctors from different African nations would move to the U.S. in order to gain more economic opportunities compared to their home country. However, as more Africans emigrate to the
`Which is the largest species of the turtle family?`	Leatherback sea turtle Leatherback sea turtle The leatherback sea turtle ("Dermochelys coriacea"), sometimes called the lute turtle or leathery turtle or simply the luth, is the largest of all living turtles and is the fourth-heaviest modern reptile behind three crocodilians. It is the only living species in the genus Dermochelys and family Dermochelyidae. It can easily be differentiated from other modern sea turtles by its lack of a bony shell, hence the name. Instead, its carapace is covered by skin and oily flesh. "Dermochelys" is the only extant genus of the family Dermochelyidae. "Dermochelys coriacea" is the only species in genus "Dermochelys". The	Okavango mud turtle Okavango mud turtle The Okavango mud turtle (Okavango terrapin) ("Pelusios bechuanicus") is a species of turtle in the family Pelomedusidae endemic to Africa. It is found in Angola, Botswana, the Democratic Republic of the Congo, Namibia (Caprivi), Zambia, and Zimbabwe. Found in central Africa, central Angola, northeastern Namibia, northern Botswana, Zimbabwe, and Zambia The Okavango mud turtle is largest species of the genus "Pelusios". The carapace is oval and elongated, with a pronounced dome, and is evenly rounded at the edges which allows the turtle to appear as a smooth rock. The carapace is very dark, often almost black, and
`How many gallons of beer are in an English barrel?`	Barrel gallon for liquids (the corn gallon of 268.8 cubic inches for solids). In Britain, the wine gallon was replaced by the imperial gallon. The tierce later became the petrol barrel. The tun was originally 256 gallons, which explains from where the quarter, 8 bushels or 64 (wine) gallons, comes. Although it is common to refer to draught beer containers of any size as barrels, in the UK this is strictly correct only if the container holds 36 imperial gallons. The terms "keg" and "cask" refer to containers of any size, the distinction being that kegs are used for beers intended	Beer Barrel Polecats Percy Pomeroy (Eddie Laughton), and work together to flee the prison. They are ultimately captured, and sent to solitary confinement. After nearly half a century later, the graying trio are finally released as senior citizens, in which Curly quips upon leaving "You know what I'm-a gonna do? I'm gonna get myself a tall, big, beautiful bottle of beer!" Moe and Larry become irate and throw Curly back into the jail, leaving him there. The title "Beer Barrel Polecats" is a pun of the song "Beer Barrel Polka". The idea of producing and selling their own beer during Prohibition was borrowed

Loss: MultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 64
per_device_eval_batch_size: 64
learning_rate: 2e-05
weight_decay: 0.01
num_train_epochs: 4
warmup_ratio: 0.1
warmup_steps: 0.1
fp16: True
load_best_model_at_end: True

All Hyperparameters

Click to expand

do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 64
per_device_eval_batch_size: 64
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 2e-05
weight_decay: 0.01
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 4
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_ratio: 0.1
warmup_steps: 0.1
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
enable_jit_checkpoint: False
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
use_cpu: False
seed: 42
data_seed: None
bf16: False
fp16: True
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: -1
ddp_backend: None
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
group_by_length: False
length_column_name: length
project: huggingface
trackio_space_id: trackio
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_for_metrics: []
eval_do_concat_batches: True
auto_find_batch_size: False
full_determinism: False
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_num_input_tokens_seen: no
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
use_cache: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss	trivia_qa_eval_cosine_accuracy
0.0011	1	0.7126	-	-
0.0539	50	0.7164	-	-
0.1079	100	0.7126	-	-
0.1618	150	0.6888	-	-
0.2157	200	0.6802	-	-
0.2697	250	0.6422	-	-
0.3236	300	0.6562	-	-
0.3776	350	0.6356	-	-
0.4315	400	0.6532	-	-
0.4854	450	0.6106	-	-
0.5394	500	0.6104	0.5472	0.7970
0.5933	550	0.6301	-	-
0.6472	600	0.6259	-	-
0.7012	650	0.5759	-	-
0.7551	700	0.6089	-	-
0.8091	750	0.5835	-	-
0.8630	800	0.5890	-	-
0.9169	850	0.5577	-	-
0.9709	900	0.5569	-	-
1.0248	950	0.5427	-	-
1.0787	1000	0.4698	0.5046	0.8190
1.1327	1050	0.4662	-	-
1.1866	1100	0.4634	-	-
1.2406	1150	0.4597	-	-
1.2945	1200	0.4585	-	-
1.3484	1250	0.5140	-	-
1.4024	1300	0.4542	-	-
1.4563	1350	0.4579	-	-
1.5102	1400	0.4910	-	-
1.5642	1450	0.5067	-	-
1.6181	1500	0.4800	0.4875	0.8300
1.6721	1550	0.4638	-	-
1.7260	1600	0.4760	-	-
1.7799	1650	0.4699	-	-
1.8339	1700	0.4912	-	-
1.8878	1750	0.4726	-	-
1.9417	1800	0.4764	-	-
1.9957	1850	0.4802	-	-
2.0496	1900	0.3941	-	-
2.1036	1950	0.3991	-	-
2.1575	2000	0.4114	0.4734	0.838
2.2114	2050	0.3981	-	-
2.2654	2100	0.4023	-	-
2.3193	2150	0.3932	-	-
2.3732	2200	0.3887	-	-
2.4272	2250	0.3894	-	-
2.4811	2300	0.3858	-	-
2.5351	2350	0.3907	-	-
2.5890	2400	0.3934	-	-
2.6429	2450	0.3871	-	-
2.6969	2500	0.3763	0.4681	0.8310
2.7508	2550	0.3997	-	-
2.8047	2600	0.3941	-	-
2.8587	2650	0.3884	-	-
2.9126	2700	0.3771	-	-
2.9666	2750	0.4168	-	-
3.0205	2800	0.3722	-	-
3.0744	2850	0.3565	-	-
3.1284	2900	0.3499	-	-
3.1823	2950	0.3428	-	-
3.2362	3000	0.3583	0.4669	0.8320
3.2902	3050	0.3444	-	-
3.3441	3100	0.3252	-	-
3.3981	3150	0.3563	-	-
3.4520	3200	0.3465	-	-
3.5059	3250	0.3328	-	-
3.5599	3300	0.3438	-	-
3.6138	3350	0.3330	-	-
3.6677	3400	0.3567	-	-
3.7217	3450	0.3462	-	-
3.7756	3500	0.3435	0.4639	0.8340
3.8296	3550	0.3532	-	-
3.8835	3600	0.3480	-	-
3.9374	3650	0.3361	-	-
3.9914	3700	0.3628	-	-

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.12.12
Sentence Transformers: 5.2.2
Transformers: 5.0.0
PyTorch: 2.9.0+cu128
Accelerate: 1.12.0
Datasets: 4.0.0
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}