Hugging Face's logo

Azzedde
/
llama3.2-3b-sql-expert-1-epoch

Text Generation
Transformers
PyTorch
English
llama
text-generation-inference
unsloth
trl
sft
sql
conversational
Inference Endpoints
Model card Files Community
llama3.2-3b-sql-expert-1-epoch / README.md
Azzedde's picture
Azzedde
Update README.md
2283cbe verified
preview code
|
raw
history blame contribute delete
5.87 kB
metadata
base_model: unsloth/llama-3.2-3b-instruct-unsloth-bnb-4bit
tags:
  - text-generation-inference
  - transformers
  - unsloth
  - llama
  - trl
  - sft
  - sql
license: apache-2.0
language:
  - en
datasets:
  - b-mc2/sql-create-context

Model Card for Llama3.2-3B-SQL-Expert-1Epoch

Model Details

Model Description

Llama3.2-3B-SQL-Expert-1Epoch is a fine-tuned version of Meta’s Llama-3.1-3B, specifically optimized for generating SQL queries from natural language input. The model has been trained using Unsloth for efficient fine-tuning and inference.

  • Developed by: Azzedine (GitHub: Azzedde)
  • Funded by [optional]: N/A
  • Shared by [optional]: Azzedde
  • Model Type: Large Language Model (LLM) optimized for SQL query generation
  • Language(s) (NLP): English
  • License: Apache 2.0
  • Finetuned from model [optional]: Meta-Llama-3.1-3B-Instruct

Model Sources

  • Repository: Hugging Face
  • Paper [optional]: N/A
  • Demo [optional]: N/A

Uses

Direct Use

This model is designed for generating SQL queries based on natural language inputs and is useful for:

  • Database management and administration
  • Automated query generation
  • Data analytics pipelines
  • SQL education and training
  • Business intelligence applications

Downstream Use [optional]

  • Embedding into LLM-based database assistants
  • Automating SQL-based analytics
  • Assisting developers in writing optimized queries

Out-of-Scope Use

  • General NLP tasks unrelated to SQL query generation
  • Applications requiring strong factual accuracy outside SQL

Bias, Risks, and Limitations

  • Incorrect or suboptimal queries: The model may generate queries that are syntactically correct but do not yield the intended results.
  • Lack of query optimization: The generated queries are not always optimized for performance; users should validate execution plans.
  • English-only support: The model primarily supports English-language inputs.
  • Limited schema understanding: The model does not validate database structures and may assume incorrect relationships between tables.

Recommendations

Users should:

  • Always validate generated SQL queries before executing them.
  • Use the model as an assistant, not a replacement for SQL expertise.
  • Fine-tune the model further for domain-specific databases.

How to Get Started with the Model

Use the following code to load and use the model:

from unsloth import FastLanguageModel
from transformers import AutoTokenizer

# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("Azzedde/llama3.2-3b-sql-expert-1-epoch")
model = FastLanguageModel.from_pretrained("Azzedde/llama3.2-3b-sql-expert-1-epoch")

# Example inference
sql_prompt = """Below is a SQL database schema and a question. Generate an SQL query to answer the question.

### Schema:
{schema}

### Question:
{question}

### SQL Query:
"""
input_text = sql_prompt.format(
    schema="CREATE TABLE employees (id INT PRIMARY KEY, name VARCHAR, salary DECIMAL, department_id INT);",
    question="Find the average salary per department."
)

# Tokenize and generate query
inputs = tokenizer(input_text, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=64, use_cache=True)
print(tokenizer.decode(outputs[0]))

Training Details

Training Data

  • The model was fine-tuned on a structured SQL dataset, including a mix of publicly available SQL benchmarks and synthetically generated SQL queries.

Training Procedure

  • Preprocessing: Tokenized using standard SQL syntax formatting
  • Training Hyperparameters:
    • batch_size = 4
    • gradient_accumulation_steps = 8
    • num_train_epochs = 1
    • learning_rate = 2e-4
    • fp16 = True

Evaluation

Testing Data

  • The model was evaluated on a separate test set of SQL queries derived from real-world database schemas.

Evaluation Metrics

  • Exact Match Accuracy: Percentage of queries that exactly match ground-truth SQL
  • Execution Success Rate: Percentage of generated queries that execute without errors

Results

  • High accuracy for common SQL queries
  • Some errors in complex multi-table joins and nested queries

Environmental Impact

  • Hardware Type: Tesla T4 (Google Colab)
  • Training Duration: ~1.5 hours
  • Compute Region: N/A
  • Estimated Carbon Emissions: Minimal

Technical Specifications

Model Architecture and Objective

  • Based on Llama-3.1 3B, fine-tuned with LoRA for SQL generation.

Compute Infrastructure

  • Fine-tuned using Unsloth for efficient training and inference.

Hardware

  • GPU: Tesla T4
  • Max Reserved Memory: ~6.5 GB

Software

  • Libraries Used: unsloth, transformers, TRL, datasets

Citation [optional]

BibTeX: 

@article{llama3.2-3B-SQL-Expert,
  author    = {Azzedde},
  title     = {Llama3.2-3B-SQL-Expert: An SQL Query Generation Model},
  year      = {2025},
  url       = {https://huggingface.co/Azzedde/llama3.2-3b-sql-expert-1-epoch}
}

APA:

Azzedde. (2025). Llama3.2-3B-SQL-Expert: An SQL Query Generation Model. Retrieved from Hugging Face

More Information

For questions, reach out via Hugging Face discussions or GitHub issues.

Model Card Authors

Model Card Contact