andhiyaulhaq/natural2sql

Natural Language to SQL (NL2SQL) Llama 3.2 3B - GGUF

This repository contains a fine-tuned version of unsloth/Llama-3.2-3B-Instruct-bnb-4bit, specifically trained to translate natural language questions into accurate, executable PostgreSQL queries.

The model was fine-tuned using Unsloth for 2x faster training and is provided in the GGUF format, making it highly optimized for local inference on consumer hardware using tools like Ollama or llama.cpp.

Model Details

• Base Model: unsloth/Llama-3.2-3B-Instruct-bnb-4bit
• Task: Text-to-SQL (focusing on PostgreSQL syntax)
• Architecture: Llama 3
• Format: GGUF
• Quantization: Q4_K_M (4-bit quantization. This provides an excellent balance between memory usage, inference speed, and model quality).

How to run with Ollama

1. Download the saved_finetuned_q4_k_m.gguf file to your local machine.
2. Create a file named Modelfile in the same directory with the following content:

FROM ./saved_finetuned_q4_k_m.gguf

SYSTEM """You are SQL-Llama, a specialized Text-to-SQL assistant. 
For all inputs, you MUST output ONLY valid SQL code. Do not include markdown blocks or conversational filler."""

TEMPLATE """<|begin_of_text|><|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|><|start_header_id|>user<|end_header_id|>

{{ .Prompt }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

"""

3. Build and run the model in your terminal:

# Build the model
ollama create natural2sql -f Modelfile

# Run the model
ollama run natural2sql

Prompt Format

To get the best results, provide the database schema as Context and your inquiry as Question:

Context: CREATE TABLE sales (transaction_id INT, product_name TEXT, amount DECIMAL, sale_date DATE, region TEXT);
Question: What was the total revenue from the 'North' region for transactions occurring after January 2023?

License

The base model weights are subject to the Meta Llama 3.2 Community License. Any custom datasets or fine-tuning code associated with this project are provided under the MIT License (see LICENSE file).