Uses

v v small model for running on edge with :fire: TTFT & Throughput

Use llama.cpp to inference the model

GGUF

Model size

362M params

Architecture

llama

16-bit

Inference API

Unable to determine this model's library. Check the docs .