turtlecap/mdbr-leaf-mt-resume-grader

SentenceTransformer based on MongoDB/mdbr-leaf-mt

This is a sentence-transformers model finetuned from MongoDB/mdbr-leaf-mt on the hf and neuralframe datasets. It maps inputs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: MongoDB/mdbr-leaf-mt
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text
• Training Datasets:
  • hf
  • neuralframe

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 384, 'pooling_mode': 'mean', 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    'Career OverviewHighly motivated Sales Associate with extensive customer service and sales experience. Outgoing sales professional with track record of driving increased sales, improving buying experience and elevating company profile with target market. QualificationsDevelopment: Using Apache Tomcat Server, Web Services, MVC pattern and OOP. *Programming Languages: C#, Java, JavaScript, CSS3 and HTML5 Work ExperienceSoftware Development Intern,11/2016-PresentKion Group–Indianapolis,IN,Design, develop and test Android and iOS applications.Involving with all phases of mobile app development from initial concept, through design, development, testing and deployment.Work with product managers to clarify function flows and provide technical insight on optimization.Work with the entire project team to balance creative objectives with business and technical requirements.Data Center Operations Associate,01/2016-PresentBoulder County, Co–Boulder,CO,Develop internal application to process tracking data.Responsible for on-site support of, Data Operations Processes and providing Security Access Controls to Life Sys users through various system applications.Troubleshoot production issues reported by end users and demonstrate ability toward the resolution on restoring functionality and monitor network server.IT Intern,01/2013-10/2013Department Of State–City,STATE,Install, configure, and upgrade the network hardware and software.Identify and solve problems with hardware and software.Troubleshoot network connectivity issues, working with remote employees. Education and Training:Computer Sciences/Mathematics,Expected inMay-2017-University of Lincoln Nebraska-Lincoln,NEGPA:Status-Computer Sciences/Mathematics 3.36 SkillsApache, balance, clarify, hardware, concept, CSS3, HTML5, Java, JavaScript, Access, C#, MVC, network hardware, network, OOP, optimization, Processes, Programming, Tomcat, Troubleshoot, upgrade',
]
documents = [
    'Skills - Cucumber BDD + Selenium UI Automation Experience Excellent knowledge and experience in testing Financial Domain applications. Hands-on experience in Test Plan, Test Case, and Test Scenario development. Experience in creating data requests based on test cases and data mining.',
    'Title: Data EngineerLocation: Ada MI (Need Locals to MI)Contract: 6+ Months Requirement:Data EngineerAdministrator (mid-level at least)a. Data bricks, Pythonb. Focus on data administration; ETL; data management activities.c. Azure CloudData Stack; Informatica or other ETL.',
    'Vaco has a client in Allentown that is looking for a Senior Accountant. We are looking for someone with manufacturing industry accounting experience including strong month end close knowledge and good cost accounting skills. Need someone with a strong understanding of GAAP and the ability to follow internal control related processes. Solid Excel skills required, and any ERP experience would be great. Excellent opportunity to join an industry leading company in a sector of a very recession resilient industry.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 384] [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.9377, 0.9290, 0.8055]])

Training Details

Training Datasets

hf

• Dataset: hf
• Size: 6,349 training samples
• Columns: resume, job, and score
• Approximate statistics based on the first 100 samples:

  	resume	job	score
  type	string	string	float
  modality	text	text
  details	min: 256 tokens mean: 256.0 tokens max: 256 tokens	min: 43 tokens mean: 226.02 tokens max: 256 tokens	min: 0.0 mean: 0.34 max: 1.0

• Samples:

  resume	job	score
  SummarySeasoned Data Entry Operator with 5 years of experience building and maintaining electronic databases. Received and reviewed new written records, transcribed information into electronic databases, verified and validated new and previously-input data fields, prepared reports, and contributed to refinement of data entry protocols. Detail-oriented professional effecting useful, accurate databases. SkillsMeticulous Attention to DetailCertified in 10-KeyData TranscriptionAdministrative Support SpecialistAdvanced Clerical KnowledgeCollecting InformationProficient With[Database]Filing and Data ArchivingVerifying Data AccuracyWritten and Verbal Communication10-Key CertificationWorkflow ManagementData VerificationTime ManagementData ManagementService OrientedData ReviewData CompilationProject ManagementReport GenerationSelf-StarterMicrosoft OfficeMultitasking and PrioritizationSpreadsheet ManagementData EntryInformation StorageAdvanced MS Office Suite KnowledgeData ProcessingData Integri...	Position Title: Senior Construction Accountant (Req #: 116) Location: Fresno, CA Date Posted: 08082023 Pay Range: $82,000 - $118,000 Position Description: Job Summary Responsible for maintaining financial records and ensures that financial transactions are properly recorded. Verifies the accuracy of entries to ledger accounts and reconciles subsidiary ledger accounts to the general ledger. Prepares complex balance sheets, profit and loss statements, and other financial reports. Analyzes current costs, revenues, financial commitments, and obligations incurred to predict future revenues and expenses. Essential Duties and Responsibilities Essential duties and other responsibilities include but are not limited to the following:Ensures accurate and timely reporting of all financial data in accordance with GAAP.Monitors revenue and expenses while coordinating collection, consolidation, and review of all financial data.Verifies the accuracy of all job costs as well as reconciling all subsidia...	0.0
  SummaryOver 3 years of professional IT experience inData EngineeringandData Analyticsusing various languages and tools likeSQL, Python, Big/HadoopExtensive experience on Data Engineering field including Ingestion, Datalake, Datawarehouse, Reporting and Analytics.Strong knowledge and experience on Data Analysis, Data Lineage, Big Data pipelines, Data quality, Data Reconciliation, Data transformation rules, Data flow diagram including Data replication, Data integration and Data orchestration toolsKnowledge and experience on AWS services like Redshift,Redshift spectrum,S3,Glue,Athena, Lambda,cloudwatch and EMRs like HIVE, PrestoExtensively used ETL methodology for performing Data Migration, Extraction, Transformation and loading usingTalendand designed data conversions from wide variety of source systems.Experienced in Data Ingestion projects to inject data intoData lakeusing multiple sources systems using Talend Bigdata.Good technical Skills inSQL Server,ETLDevelopment usingInformatica t...	Title: Data EngineerLocation: fully remote, however, priority is given to candidates local to Boston, MA; Long Island, NY; or Golden, COWork Model: 12 month contract with extensions or conversions highly possiblePay Rate: 58-68 an hour Day to DayA mid-sized telecom company is looking for a Sr Data Engineer to join their Business Intelligence Insights team. This team is responsible for all of the data migration, data clean up, data integration, and warehousing for the company. In this role, you will primarily be responsible for data migration and clean up in existing platforms as well as newly or soon to be integrated systems. It is necessary to be well versed in MS SQL, with the ability to import data, manipulate data and put it back in, querying, and building indexes. It is also crucial to be versed with SSIS to load and validate dimensions and measures. Lastly, it will be important for this person to have exposure and understanding of Salesforce - ability to import data, query data, ...	0.0
  Professional SummaryMotivated Senior Software Engineer highly effective at developing new programs and fixing problems with existing systems in robust code for high- volume businesses. Strong decision-maker with6 yearsof experience in software engineering and application development. Effective coach and mentor and committed to leading exceptionally gifted teams. Proven history of developing useful, efficient and cost-effective projects. . SkillsRuby 2.x.x, SQLRails 4.x, 5.xHtml, Css, JavaScript, Ajax, Bootstrap, Semantic UI, React 16.8Rspec, CapybaraDocker, Azure DevOps, Bamboo, Git, BitbucketPostgreSQL, MySQL, Microsoft SQL Server,Apache2.0, NginxUnix shell script, Linux EducationUniversity Of Central MissouriWarrensburg, MO,Expected in12/2015––Master of Science:Computer Science-GPA:Jawaharlal Nehru Technological UniversityHyderabad,Expected in04/2013––Bachelor of Science:Electrical, Electronics And Communications Engineering-GPA: Work HistoryCommscope-Software EngineerRichardson,TX,0...	Life at Capgemini Capgemini supports all aspects of your well-being throughout the changing stages of your life and career. For eligible employees, we offer: Flexible work Healthcare including dental, vision, mental health, and well-being programs Financial well-being programs such as 401(k) and Employee Share Ownership Plan Paid time off and paid holidays Paid parental leave Family building benefits like adoption assistance, surrogacy, and cryopreservation Social well-being benefits like subsidized back-up childelder care and tutoring Mentoring, coaching and learning programs Employee Resource Groups Disaster Relief About Capgemini Engineering World leader in engineering and R&D services, Capgemini Engineering combines its broad industry knowledge and cutting-edge technologies in digital and software to support the convergence of the physical and digital worlds. Coupled with the capabilities of the rest of the Group, it helps clients to accelerate their journey towards Intelligent Ind...	0.5

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "CoSENTLoss",
      "matryoshka_dims": [
          384,
          256,
          192
      ],
      "matryoshka_weights": [
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

neuralframe

• Dataset: neuralframe
• Size: 7,630 training samples
• Columns: resume, job, and score
• Approximate statistics based on the first 100 samples:

  	resume	job	score
  type	string	string	float
  modality	text	text
  details	min: 79 tokens mean: 167.98 tokens max: 256 tokens	min: 38 tokens mean: 71.0 tokens max: 114 tokens	min: 0.27 mean: 0.67 max: 0.91

• Samples:

  resume	job	score
  Adept at extracting and combining data from multiple sources to come up with KPIs which have a direct impact on the bottom line of a project. Skills: Data Analysis, Data Visualization, Deep Learning, Machine Learning, Adobe Analytics, Excel, Matplotlib, Python. Education: B.tech in IT (Maulana Abul University). Experience: Analyst Support at Tata Sons. Responsibilities: Hardware & Software Installation System Monitoring IT Support Networking Expertise Mail Server Administration Security Management Troubleshooting & Repair Operations Support.	Education required: Bachelor of Science (BSc) in Computer Science & Engineering Experience required: 3 to 5 years. Responsibilities: Hardware & Software Installation System Monitoring IT Support Networking Expertise Mail Server Administration Security Management Troubleshooting & Repair Operations Support.	0.8267
  Skills: C#, ASP.NET, Visual Basic, Ms Visual Studio, Ms Office Suite (Word, Excel, Project, PowerPoint, Access, Outlook, Visio), Simulations using Witness13 and SIMIO, ERP-SAP, SAS-STAT, AutoCAD, budgeting, CNC, Continuous Improvement, data collection, Product Development and Design, documentation, ERP, grinders, Inventory Control, ISO, Languages, lathe, layout, layout design, Logistics, materials, Access, Excel, Outlook, PowerPoint, Word, new product development, payroll, philosophy, process improvement, process design, processes, Project Management, project plans, quality, quality control, research, Safety, SAP, SAS, simulation, Six Sigma, SQL, Statistical Process Control, Supply Chain, Visio. Education: Master of Science in Industrial and Manufacturing Engineering; Bachelor of Engineering in Materials and Metallurgical Engineering (University of Texas Arlington; Federal University of Technology). Experience: Industrial Engineering Intern at Company Name; Industrial Engineer at Compa...	Education required: Bachelor/Honors Experience required: 3 to 7 years. Responsibilities: Full Stack Development Front-end: ReactJS, NextJS Backend: Python, Django API Design Server-Side Logic DRF (Django REST Framework) Database Management (PostgreSQL, MySQL) Version Control (Git) AWS (ECR, RDS, ECS, ALB, EC2, etc.) Linux, Docker, CI/CD, GitLab Terraform, Shell Scripting.	0.8167
  Skills: Deep learning, Python, SQL, Keras, PyTorch, PySpark, Data Science, Data Analysis, Machine Learning, Natural Language Processing, Artificial Intelligence, Computer Vision, Pattern Recognition, NLP, OpenCV, Image Processing. Education: Dual Degree (B.tech + M.Tech)(Chemical) in Chemical (Indian Institute of Technology (IIT), Kharagpur). Experience: Data Scientist at Innoplexus. Responsibilities: Full Stack Development Front-end: ReactJS, NextJS Backend: Python, Django API Design Server-Side Logic DRF (Django REST Framework) Database Management (PostgreSQL, MySQL) Version Control (Git) AWS (ECR, RDS, ECS, ALB, EC2, etc.) Linux, Docker, CI/CD, GitLab Terraform, Shell Scripting.	Education required: Bachelor/Honors Experience required: 3 to 7 years. Responsibilities: Full Stack Development Front-end: ReactJS, NextJS Backend: Python, Django API Design Server-Side Logic DRF (Django REST Framework) Database Management (PostgreSQL, MySQL) Version Control (Git) AWS (ECR, RDS, ECS, ALB, EC2, etc.) Linux, Docker, CI/CD, GitLab Terraform, Shell Scripting.	0.85

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "CoSENTLoss",
      "matryoshka_dims": [
          384,
          256,
          192
      ],
      "matryoshka_weights": [
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Evaluation Datasets

hf

• Dataset: hf
• Size: 824 evaluation samples
• Columns: resume, job, and score
• Approximate statistics based on the first 100 samples:

  	resume	job	score
  type	string	string	float
  modality	text	text
  details	min: 256 tokens mean: 256.0 tokens max: 256 tokens	min: 55 tokens mean: 213.32 tokens max: 256 tokens	min: 0.0 mean: 0.32 max: 1.0

• Samples:

  resume	job	score
  Executive ProfileExperienced business executive with demonstrated success in managing P&L and growing new business through innovative growth strategies worldwide. Excellent ability to develop relationships and strategic alliances with organization leaders to align, support and grow business initiatives. Outstanding presentation, communication and cross-cultural team management skills. Highly results-oriented leader with an entrepreneurial attitude. Skill Highlights2010 - 2011 Shunra Software, Philadelphia, PADirector, SalesResponsibilitiesEstablished sales in high value verticals and potential acquirers.HighlightsInitiated relationship and closed business with key teams at major networking and consumer software technology firms.Identified and shepherded an OEM relationship with Cisco.2008 - 2010 dynaTrace Software, Linz, Austria/Lexington, MA (sold to Compuware)Director, West SalesResponsibilitiesEstablished direct sales in western US.HighlightsInitiated and closed opportunities with a...	Reality Labs at Meta is building products that make it easier for people to connect with the ones they love most, enjoy top-notch, wire-free VR, and push the future of computing platforms. We are a team of world-class experts developing and shipping products at the intersection of hardware, software and content.As a Machine Learning Engineer on the Reality Labs team at Meta, you can help build new, innovative hardware and software that radically redefine the way people work, play and connect. What we build today could one day be the norm. So to be here today is to truly be at the heart of change and the frontier of what's to come. We're the people helping to define the metaverse. We may not have all the answers. But together, we're getting closer. Software Engineer, Machine Learning - Reality Labs Responsibilities: Play a critical role in setting the direction and goals for a sizable team, in terms of project impact, ML system design, and ML excellenceAdapt standard machine learning me...	1.0
  SummaryTo obtain a position where my business education and administrative and marketing skills can make a significant contribution to my employer and where I can make a difference in the lives of the customers I assist. HighlightsMS OfficeCustomer ServiceMarketing tool developmentOffice Administration EducationMiami Dade CollegeHialeah,FL.Expected inMay 2010––Associate of Arts:Accounting-GPA:AccountingMiami Dade CollegeHialeah,FL.Expected inJuly 2012––Bachelors of Science:Management Supervision-GPA:GPA: 3.0Management Supervision GPA: 3.0 ExperienceVolunteers Of America - Greater New Orleans-Payroll Data Entry ClerkDenham Springs,LA,06/2014-12/2014Maintains payroll information by collecting, calculating, and entering data Processing medical delivery tickets (on-going) Maintains payroll operations by following policies and procedures; reporting needed changes.Maintains employee confidence and protects payroll operations by keeping information confidential.Contributes to team effort by a...	( Candidates without any Visa Constraints) ( Local to Pittsburgh, PA will be preferred, but willing to re-locate candidates will be considered) Project: Business Analyst lllLocation: Pittsburgh, PA ( 3 days Week) Client: BNY Mellon Job Id: 70732-1Job Duration: 4-6 Month CTH Enterprises Data Governance Under Data Quality project.Regulatory reporting and KYC Screening is part of the project.Data Quality rules working with Data Lakes.Analytics Data is in the play.Min 8-12 years of working experience in the Data Quality field. Tools: Collibra, Jira, Confluence, and Soft skills in SQL is good.The location is Pittsburgh, PA, for Three days Week in the Hybrid model.One round of Interviews.Data Analyst Business Analyst kind of role:Top 3 Skills: Data Quality Model, Working with Matrix, Data AnalysisPittsburgh, PA, is the primary location, but we will also consider Lake Mary, FL Data Governance and Data Quality with Testing experience in the field is preferred. Team Size is eight members, and l...	0.0
  SummarySolution-oriented, results-driven strategic sales executive with a proven track record in generating sales growth and successfully expanding market share while ensuring existing client loyalty. Well-traveled, proven relationship builder with extensive hands- on experience in North America (especially Northeast and Southwest the USA) and Mexico. A creative problem solver and effective business closer with broad knowledge in Sales, Business Development, Engineering and manufacturing. Strategic Planning & Execution ... Budgeting & Forecasting ... New Business Development ... Direct Sales & Sales Channels / OEM Management ... Contract Negotiation & Deal Making ... Project Management ... Operations Management HighlightsGuest servicesInventory control proceduresMerchandising expertiseLoss preventionCash register operationsProduct promotions Experience01/2013toPresentRegional Sales ManagerSnap-On Tools–Seattle,WA,Process Sensors Corporation is a leading manufacturer of state-of-the-art...	Requirement - Must have experience in FACETS.LeadExecute all or part of the business analysis on business featuresuser stories. Provide oversight when junior BAs are involvedResponsible for gathering and understanding of business needs and translating them into effective user storiesrequirements.Assist in the estimation, prioritization and implementation of cost-effective business process and system solutions.Work with Product Owner Business in prioritization and implementation of cost-effective business process and system solutions.On-boarding and mentoring BAsnew BAs into the organization.Proactively acquire BU domain knowledge and facilitate the growth of domain knowledge throughout the technical teams. Lead or support Knowledge management initiativesDevelop a strong understanding of the industry & the organizational vision, mission, strategic goals	0.0

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "CoSENTLoss",
      "matryoshka_dims": [
          384,
          256,
          192
      ],
      "matryoshka_weights": [
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

neuralframe

• Dataset: neuralframe
• Size: 952 evaluation samples
• Columns: resume, job, and score
• Approximate statistics based on the first 100 samples:

  	resume	job	score
  type	string	string	float
  modality	text	text
  details	min: 75 tokens mean: 162.31 tokens max: 256 tokens	min: 38 tokens mean: 72.29 tokens max: 114 tokens	min: 0.31 mean: 0.66 max: 0.88

• Samples:

  resume	job	score
  I am an experienced analytics specialist seeking a full-time position in the field of data science and application in business problem solving, where I can apply my knowledge and skills for continuous improvement. Skills: Data Science, Customer analytics, Data Visualization, Natural language Processing, SQL, Python, R, Tableau, Alteryx, Process Improvement. Education: B.Tech in ECE (Maulana Malviya College for Informatics). Experience: Data Managerial Intern at Orchidia Tours and Travels. Responsibilities: Technical Support Troubleshooting Collaboration Documentation System Monitoring Software Deployment Training & Mentorship Industry Trends Field Visits.	Education required: B.Sc in Computer Science & Engineering from a reputed university. Experience required: At least 1 year. Responsibilities: Technical Support Troubleshooting Collaboration Documentation System Monitoring Software Deployment Training & Mentorship Industry Trends Field Visits.	0.75
  Computer Repair Technician Skills: iFixit Pro Tech Toolkit, PassMark MemTest86, Malwarebytes, Recuva, Ultimate Boot CD (UBCD), Windows Installation USB/DVD, TeamViewer, Driver Booster, Falcon Dust-Off Compressed Gas Duster, Fluke 115 Compact True-RMS Digital Multimeter. Education: Associate of Applied Science in Electronics and Computer Technology (DeVry University). Experience: Computer Repair Technician at YZA Technologies; Computer Technician at Sherwin-Williams; Help Desk Technician at OhioHealth. Responsibilities: Administrative Support Scheduling Filing & Documentation Communication Team Support Equipment Maintenance Information Provision Inventory Management Team Collaboration OHS Policy Development Safety Advice Risk Assessment Policy Review OHS Training Safety Inspections Unsafe Act Prevention Incident Investigation Report Preparation.	Education required: Bachelor/Honors Experience required: At least 5 years. Responsibilities: Administrative Support Scheduling Filing & Documentation Communication Team Support Equipment Maintenance Information Provision Inventory Management Team Collaboration OHS Policy Development Safety Advice Risk Assessment Policy Review OHS Training Safety Inspections Unsafe Act Prevention Incident Investigation Report Preparation. Skills required: •Administration •Health Safety and Environment •Safety and Security Management.	0.85
  Data Analyst, interested in Machine Learning application development. As a data analyst, I have extensive experience with data preprocessing and pipeline building and I would like to venture further into this domain to learn more about it. Skills: Data Analyst, Data Analytics, Data Analysis, Time Series Analysis, Business Analytics, Predictive Modeling, Regression Analysis, Image Processing, Data Visualization, Linear Regression, Statistical Analysis, Requirement Analysis, SQL, Tableau, Python. Education: BSc in Information Technology (Sikkim Manipal university). Experience: Deputy Data Analyst at Sanrachna SGT University. Responsibilities: Apparel Sourcing Quality Garment Sourcing Reliable Partner Buyer/Vendor Communication.	Education required: Bachelor/Honors Experience required: 1 to 3 years. Responsibilities: Apparel Sourcing Quality Garment Sourcing Reliable Partner Buyer/Vendor Communication. Skills required: Fast typing skill IELTSInternet browsing & online work ability..	0.45

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "CoSENTLoss",
      "matryoshka_dims": [
          384,
          256,
          192
      ],
      "matryoshka_weights": [
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 128
• learning_rate: 3e-05
• warmup_steps: 0.1
• bf16: True
• per_device_eval_batch_size: 128
• dataloader_num_workers: 4
• prompts: {'resume': 'Represent this sentence for searching relevant passages: '}
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• per_device_train_batch_size: 128
• num_train_epochs: 3
• max_steps: -1
• learning_rate: 3e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0.1
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1.0
• label_smoothing_factor: 0.0
• bf16: True
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 128
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: False
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 4
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• dataloader_multiprocessing_context: None
• dataloader_in_order: True
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: None
• fsdp_config: None
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• local_rank: -1
• prompts: {'resume': 'Represent this sentence for searching relevant passages: '}
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}
• warmup_ratio: None

Training Logs

Epoch	Step	Training Loss	hf loss	neuralframe loss
0.2273	25	27.3770	-	-
0.4545	50	22.8998	-	-
0.6818	75	22.3221	-	-
0.9091	100	20.1641	-	-
1.0	110	-	6.6938	17.3900
1.1364	125	20.2929	-	-
1.3636	150	20.1870	-	-
1.5909	175	21.3885	-	-
1.8182	200	20.0407	-	-
2.0	220	-	6.5927	17.2077
2.0455	225	18.7910	-	-
2.2727	250	20.8732	-	-
2.5	275	20.5679	-	-
2.7273	300	20.0285	-	-
2.9545	325	19.4513	-	-
3.0	330	-	6.5793	17.3640

Training Time

• Training: 3.8 minutes

Framework Versions

• Python: 3.11.9
• Sentence Transformers: 6.0.0
• Transformers: 5.16.1
• PyTorch: 2.11.0+cu128
• Accelerate: 1.14.0
• Datasets: 5.0.1
• Tokenizers: 0.23.1

Additional Resources

• Training and Finetuning Embedding Models with Sentence Transformers: the end-to-end guide for training or finetuning Sentence Transformer models.
• Introduction to Matryoshka Embedding Models: variable-size embeddings that can be truncated with minimal quality loss.
• Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval: post-training compression of embedding vectors.
• Multimodal Embedding & Reranker Models with Sentence Transformers: use text, image, audio, and video models through the same API.
• Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers: train multimodal embedding models, with a Visual Document Retrieval walkthrough.

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

CoSENTLoss

@article{10531646,
    author={Huang, Xiang and Peng, Hao and Zou, Dongcheng and Liu, Zhiwei and Li, Jianxin and Liu, Kay and Wu, Jia and Su, Jianlin and Yu, Philip S.},
    journal={IEEE/ACM Transactions on Audio, Speech, and Language Processing},
    title={CoSENT: Consistent Sentence Embedding via Similarity Ranking},
    year={2024},
    doi={10.1109/TASLP.2024.3402087}
}