File size: 4,572 Bytes
a6cd9e8 61dd8ac a6cd9e8 9d6c25a a6cd9e8 61dd8ac |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 |
---
language:
- en
base_model: defog/llama-3-sqlcoder-8b
pipeline_tag: text-generation
tags:
- SQL
- Code
- Meta
---
# SandLogic Technology - Quantized llama-3-sqlcoder-8b Models
## Model Description
We have quantized the llama-3-sqlcoder-8b model into two variants:
1. Q5_KM
2. Q4_KM
These quantized models offer improved efficiency while maintaining performance.
Discover our full range of quantized language models by visiting our [SandLogic Lexicon GitHub](https://github.com/sandlogic/SandLogic-Lexicons). To learn more about our company and services, check out our website at [SandLogic](https://www.sandlogic.com).
## Original Model Information
- **Name**: [llama-3-sqlcoder-8b](https://huggingface.co/defog/llama-3-sqlcoder-8b)
- **Developer**: Defog, Inc.
- **Model Type**: Text-to-SQL generation
- **Base Model**: Meta-Llama-3-8B-Instruct
- **Parameters**: 8 billion
- **License**: CC-by-SA-4.0
## Model Capabilities
The llama-3-sqlcoder-8b model is designed for generating SQL queries to answer questions, with support for Postgres, Redshift, and Snowflake databases. It has performance on-par with the most capable generalist frontier models.
## Inference Parameters
- **Temperature**: 0 (no sampling)
- **Prompt Format**:
```<|begin_of_text|><|start_header_id|>user<|end_header_id|>
Generate a SQL query to answer this question: {user_question}
{instructions}
DDL statements:
{create_table_statements}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
The following SQL query best answers the question {user_question}:
```
## Evaluation
The model was evaluated on SQL-Eval, a PostgreSQL-based evaluation framework developed by Defog for testing and alignment of model capabilities.
## Intended Use Cases
1. **SQL Generation**: Automatically generate SQL queries based on natural language questions or instructions.
2. **Database Interaction**: Assist users in interacting with Postgres, Redshift, or Snowflake databases through text-based interfaces.
3. **Data Analysis Support**: Provide SQL-based solutions to data analysis problems described in natural language.
4. **Programming Education**: Help students learn SQL concepts and syntax by providing example queries and explanations.
## Model Variants
We offer two quantized versions of the llama-3-sqlcoder-8b model:
1. **Q5_KM**: 5-bit quantization using the KM method
2. **Q4_KM**: 4-bit quantization using the KM method
These quantized models aim to reduce model size and improve inference speed while maintaining performance as close to the original model as possible.
## Usage
```bash
pip install llama-cpp-python
```
Please refer to the llama-cpp-python [documentation](https://llama-cpp-python.readthedocs.io/en/latest/) to install with GPU support.
### Basic Text Completion
Here's an example demonstrating how to use the high-level API for basic text completion:
```bash
from llama_cpp import Llama
llm = Llama(
model_path="./model/llama-3-sqlcoder-8b.Q5_K_M.gguf",
verbose=False,
# n_gpu_layers=-1, # Uncomment to use GPU acceleration
# n_ctx=2048, # Uncomment to increase the context window
)
output = llm.create_chat_completion(
messages = [
{"role": "system", "content": "You're an AI SQL coding assistant who help in solving coding questions"},
{
"role": "user",
"content": "write an simple sql table query and code to search employee name"
}
]
)
print(output["choices"][0]['message']['content'])
```
## Download
You can download `Llama` models in `gguf` format directly from Hugging Face using the `from_pretrained` method. This feature requires the `huggingface-hub` package.
To install it, run: `pip install huggingface-hub`
```bash
from llama_cpp import Llama
llm = Llama.from_pretrained(
repo_id="SandLogicTechnologies/Llama-3-Sqlcoder-8B-GGUF",
filename="*llama-3-sqlcoder-8b.Q5_K_M.gguf",
verbose=False
)
```
By default, from_pretrained will download the model to the Hugging Face cache directory. You can manage installed model files using the huggingface-cli tool.
## License
License: [CC-by-SA-4.0] Finetuned from model: [Meta-Llama-3-8B-Instruct]
## Acknowledgements
We thank Defog, Inc. for developing the original llama-3-sqlcoder-8b model and the creators of Llama3 for their foundational work.
Special thanks to Georgi Gerganov and the entire llama.cpp development team for their outstanding contributions.
## Contact
For any inquiries or support, please contact us at support@sandlogic.com or visit our [support page](https://www.sandlogic.com). |