test-trainer

This model is a fine-tuned version of bert-base-uncased on the glue and Microsoft Research Paraphrase Corpus (MRPC) dataset.

Model description

Test-trainer is a transformers model fine tuned on the bert-base-uncased model for sequence classification. It was fine uned on the glue and MRPC datasets. it was fine tuned with one objective:

-sequece classificatrion specifically Paraphrase detection (determining if two sentences mean the same).

Intended uses & limitations

This model can be used to reduce redundancy in a dataset by identifying and removing sentences that essentially convey the same meaning. This helps make the dataset more concise while retaining the core information.

How to use:

from transformers import BertForSequenceClassification, BertTokenizer import torch

#Load the model and tokenizer model = BertForSequenceClassification.from_pretrained("bert_mrpc_model") # Change this to your actual path tokenizer = BertTokenizer.from_pretrained("bert_mrpc_model")

#Define two sentences to compare sentence1 = "The quick brown fox jumps over the lazy dog." sentence2 = "A fast, dark-colored fox leaped over a sleepy canine."

#Tokenize the input sentences inputs = tokenizer(sentence1, sentence2, return_tensors="pt", padding=True, truncation=True, max_length=128)

#Run the model with torch.no_grad(): outputs = model(**inputs)

#Get the predicted class (0 = not paraphrases, 1 = paraphrases)

predicted_class = torch.argmax(outputs.logits, dim=1).item()

#Interpret the result

if predicted_class == 1: print("The sentences are paraphrases.") else: print("The sentences are NOT paraphrases.")

Training and evaluation data

This model was trained on the Microsoft Research Paraphrase Corpus (MRPC) dataset, which is part of the GLUE benchmark. The General Language Understanding Evaluation (GLUE) benchmark is a collection of natural language understanding (NLU) tasks designed to evaluate machine learning models' ability to handle different aspects of language comprehension. It includes datasets for sentiment analysis, sentence similarity, textual entailment, and more. The Microsoft Research Paraphrase Corpus (MRPC) is one of the datasets in the GLUE benchmark. It consists of 5,801 pairs of sentences, manually labeled as paraphrases (1) or not paraphrases (0). The goal is to determine whether two sentences have the same meaning, making it a sentence-pair classification task.

Training procedure

Preprocessing Before training, the dataset was preprocessed as follows:

The GLUE MRPC dataset was loaded using datasets.load_dataset(). The BertTokenizer from bert-base-uncased was used to tokenize sentence pairs. A tokenization function was created to process both sentences in each pair, applying truncation=True to ensure inputs fit within the model’s maximum sequence length. The dataset was batch tokenized using .map(batched=True) for efficiency. A DataCollatorWithPadding was used to handle dynamic padding during training. Training The model was trained using Hugging Face’s Trainer API. Training hyperparameters were set, including a learning rate of 5e-5, batch size of 8, and 3 epochs. The model was fine-tuned on tokenized MRPC data, optimizing with the AdamW optimizer. After training, the model was evaluated to ensure it properly classifies sentence pairs as paraphrases or non-paraphrases.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3.0

Training results

The model was trained for 3 epochs, completing 1,377 training steps with a final training loss of 0.1016. The training runtime was 342.55 seconds, processing approximately 32.1 samples per second.

limitations and bias

The model is fine-tuned on MRPC, which is a relatively small dataset (5,801 sentence pairs). Performance may not generalize well to other paraphrase detection tasks. It is English-only and may not work well on low-resource languages. Bias Warning: Since MRPC is sourced from news articles, the model may reflect biases present in journalistic text.

Framework versions

• Transformers 4.48.3
• Pytorch 2.5.1+cu124
• Datasets 3.3.2
• Tokenizers 0.21.0