Update README.md

README.md

@@ -1,3 +1,112 @@
----
-license: mit
----
+---
+license: mit
+---
+FireGenEmbedder
+
+FireGenEmbedder is a fine-tuned version of the MiniLM model, specifically adapted for sequence classification tasks. The model has been fine-tuned on the Stanford Natural Language Inference (SNLI) dataset to predict the relationship between two sentences, classifying them into three categories: Entailment, Neutral, and Contradiction. It is designed for applications in legal and other domains requiring inference tasks.
+
+Model Details
+
+Base Model: sentence-transformers/all-MiniLM-L6-v2
+
+Fine-tuned Dataset: Stanford Natural Language Inference (SNLI)
+
+Labels:
+
+0: Contradiction
+
+1: Neutral
+
+2: Entailment
+
+Training Epochs: 3
+
+Batch Size: 16 (both train and eval)
+
+Precision: Mixed precision for training on GPU
+
+Model Usage
+
+You can use this model to make inferences on sentence pairs by classifying their relationship.
+
+Install Dependencies
+
+To use this model, install the following libraries:
+
+pip install transformers datasets sentence-transformers torch
+
+
+Example Code
+
+Here’s an example of how to load and use the FireGenEmbedder model for inference:
+
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+import torch
+
+# Load the tokenizer and model
+model_name = "path_to_firegenembedder_model"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForSequenceClassification.from_pretrained(model_name)
+
+# Move model to device (GPU or CPU)
+device = "cuda" if torch.cuda.is_available() else "cpu"
+model.to(device)
+
+# Prepare input
+premise = "The sky is blue."
+hypothesis = "The sky is not blue."
+
+inputs = tokenizer(premise, hypothesis, return_tensors="pt", padding=True, truncation=True, max_length=128).to(device)
+
+# Inference
+with torch.no_grad():
+    outputs = model(**inputs)
+    predictions = torch.argmax(outputs.logits, dim=-1)
+
+# Print the prediction
+labels = ["Contradiction", "Neutral", "Entailment"]
+print(f"Prediction: {labels[predictions.item()]}")
+
+
+Model Fine-Tuning Process
+
+Data: The model was fine-tuned using the Stanford Natural Language Inference (SNLI) dataset. The SNLI dataset contains labeled pairs of sentences with three classes: Entailment, Neutral, and Contradiction.
+
+Training:
+
+The model was fine-tuned for 3 epochs with a batch size of 16 on a GPU.
+
+The training used mixed precision for faster computation if a GPU was available.
+
+The model is based on the MiniLM architecture, known for being lightweight and efficient, making it suitable for real-time inference tasks.
+
+Post-Training:
+
+The model was saved and zipped for easy distribution.
+
+The tokenizer and model were saved to the directory: miniLM-legal-finetuned-SNLI.
+
+Model Evaluation
+
+The model was evaluated using the validation set from the SNLI dataset, and results can be accessed as follows:
+
+# Load the model and evaluate
+results = trainer.evaluate()
+print(results)
+
+Zipped Model
+
+You can download the model as a zip file containing both the model weights and the tokenizer:
+
+Download Model
+
+Citation
+
+If you use this model in your research or application, please cite the following:
+
+@misc{firegenembedder,
+  author = {Your Name},
+  title = {FireGenEmbedder: Fine-tuned MiniLM for Legal Inference Tasks},
+  year = {2026},
+  url = {Link to your Hugging Face model page},
+}