Phi-1 1.3B GGUF

Introduction

Phi-1, a transformative AI model from Microsoft, stands out in the realm of coding assistance and general knowledge comprehension. Developed as part of the "Textbooks Are All You Need" initiative, this model is specifically designed for Python programming, offering robust support in code completion, refactoring, and responding to programming-related queries. Equipped with a Transformer-based architecture of 1.3 billion parameters, Phi-1 has been meticulously trained on a distinct mix of "textbook quality" web data and synthetically generated content. This diverse training not only grants it proficiency in Python coding but also enables it to understand and generate text across various domains, showcasing its versatility in both technical and general knowledge contexts.

Model Specifications

Architecture: Transformer, specialized for Python code and general comprehension.
Parameters: ~1.3 billion.
Training Data: A mix of high-quality web content and synthetic textbooks/exercises.
Training Details: 4 days on 8 A100 GPUs, 7 billion tokens.
Achievements: High accuracy on coding benchmarks like HumanEval and MBPP.

Quantized Model Files

Phi-1 is available in several formats, catering to different computational needs:

phi-1-q4_0.gguf: 4-bit quantization, offering a compact size of 876 MB for efficient inference.
phi-1-q8_0.gguf: 8-bit quantization, providing robust performance with a file size of 1.5 GB.
ggml-model-f16.gguf: Standard 16-bit floating-point format, with a larger file size of 2.7 GB for enhanced precision.

These formats, ranging from 4-bit to 16-bit, accommodate various computational environments, from resource-constrained devices to high-end servers.

Features and Usage

Phi-1 excels in:

Code Generation and Assistance: Autocompletion, refactoring, and query response in Python.
General Knowledge Queries: Responds accurately to a wide range of topics.

Example Usage

# Using Phi-1 for Python code generation
./main -m stash/models/microsoft/phi-1/phi-1-q8_0.gguf --color -e -s 1337 -c 2048 -n 512 -p "Question: How to create a list of prime numbers in Python?\nAnswer:"

This command configures the model to generate Python code for a specific programming query.