---
library_name: peft
tags:
- meta-llama/Llama-2-7b
- code
- instruct
- instruct-code
- sql-create-context
- text-to-sql
- LLM
datasets:
- b-mc2/sql-create-context
base_model: meta-llama/Llama-2-7b-hf
---

We finetuned Meta-Llama-2-7B on the SQL Create Context Dataset (b-mc2/sql-create-context) for 3 epochs using [MonsterAPI](https://monsterapi.ai) no-code [LLM finetuner](https://docs.monsterapi.ai/fine-tune-a-large-language-model-llm).

This dataset is an enhanced version of WikiSQL and Spider, focused on providing natural language queries and corresponding SQL CREATE TABLE statements. The dataset contains 78,577 examples and aims to improve the model's grounding in text-to-SQL tasks. The CREATE TABLE statements are particularly useful for limiting token usage and avoiding exposure to sensitive data.

The finetuning session took 7hrs and 21 mins and costed us a total of `$15.33`.

#### Hyperparameters & Run details:
- Model Path: meta-llama/Llama-2-7b
- Dataset: b-mc2/sql-create-context
- Learning rate: 0.0003
- Number of epochs: 3
- Data split: Training: 90% / Validation: 10%
- Gradient accumulation steps: 1

Loss metrics:
![training loss](train-loss.png "Training loss")

---
license: apache-2.0
---