PicoPilot-1-Preview

PicoPilot-1-Preview is a preview release of the PicoPilot language model, packaged for use with the Hugging Face Transformers library.

This repository contains the model weights, tokenizer, configuration, and supporting files required to run the model locally with Transformers.

Status: Preview release
Framework: PyTorch + Hugging Face Transformers
Model type: Causal language model

---

Features

• Compatible with Hugging Face Transformers
• Supports conversational/chat-style prompts
• Supports token streaming during generation
• Supports CUDA acceleration
• Can be loaded onto a specific GPU
• Compatible with standard model.generate() workflows
• Designed for local inference

---

Installation

Install the required dependencies:

pip install -U "torchao>=0.17.0"

import torch
from transformers import (
    AutoTokenizer,
    AutoModelForCausalLM,
    TextStreamer,
)

model_path = "Layasaran/picopilot-1-preview"
device = "cuda"

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_path)

# Load model directly GPU
model = AutoModelForCausalLM.from_pretrained(
    model_path,
    torch_dtype=None,
    device_map="auto",
)

model.eval()

messages = [
    {"role": "system", "content": ""}
    {"role": "user", "content": ""},
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
).to("cuda")

streamer = TextStreamer(
    tokenizer,
    skip_prompt=True,
    skip_special_tokens=True,
)

_ = model.generate(
    **inputs,
    streamer=streamer,
    max_new_tokens=512,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
    use_cache=True,
)