Instructions to use tudor-iustin22/Bluebird-1.5-Base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tudor-iustin22/Bluebird-1.5-Base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="tudor-iustin22/Bluebird-1.5-Base", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("tudor-iustin22/Bluebird-1.5-Base", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tudor-iustin22/Bluebird-1.5-Base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tudor-iustin22/Bluebird-1.5-Base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tudor-iustin22/Bluebird-1.5-Base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/tudor-iustin22/Bluebird-1.5-Base

SGLang

How to use tudor-iustin22/Bluebird-1.5-Base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tudor-iustin22/Bluebird-1.5-Base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tudor-iustin22/Bluebird-1.5-Base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tudor-iustin22/Bluebird-1.5-Base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tudor-iustin22/Bluebird-1.5-Base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use tudor-iustin22/Bluebird-1.5-Base with Docker Model Runner:
```
docker model run hf.co/tudor-iustin22/Bluebird-1.5-Base
```

Project Bluebird logo

Bluebird 1.5 Base

Efficient language modelling with long-range memory

Model repository · Apache 2.0 · Project Bluebird

Bluebird Highlights

Compact base language model: 742.5M parameters, a 24-layer decoder-only transformer, and a 32,768-token supported context window.
Hybrid long-range memory: local causal attention and compressed representations of previous 512-token chunks, combined through memory-augmented layers.
Grouped-query attention: 24 query heads and six key/value heads, with local attention over 2,048 tokens.
Pretraining: 23,000,514,560 recorded tokens from initial and continued pretraining.
Training environment: PyTorch mixed precision on eight Intel Gaudi 3 AI accelerators.

Model Overview

Bluebird 1.5 Base is a compact, decoder-only causal language model in the Project Bluebird family. With 742,528,520 parameters and a 32,768-token context window, it combines local causal attention with a compressed, chunked-memory mechanism for accessing information from earlier parts of a sequence.

The model is released as a base pretrained checkpoint, intended for language modelling, text completion, continued pretraining, domain adaptation, and downstream fine-tuning. It is not an instruction-tuned chat assistant.

Model specifications

Property	Specification
Model	`tudor-iustin22/Bluebird-1.5-Base`
Project	Project Bluebird
Architecture	Decoder-only transformer with local attention and chunked long-range memory
Parameters	742,528,520 (742.5M)
Transformer layers	24
Hidden size	1,536
Query attention heads	24
Key/value attention heads	6
Vocabulary size	65,536
Maximum sequence length	32,768 tokens
Local attention window	2,048 tokens
Memory chunk size	512 tokens
Learned summaries per chunk	4
Maximum memory summaries	256
Feed-forward activation	SwiGLU
Normalisation	RMSNorm
Positional embeddings	Rotary position embeddings (RoPE)
Published checkpoint precision	FP16
Training compute precision	Mixed precision
Pretraining tokens recorded	23,000,514,560
Framework	PyTorch
Intel Gaudi runtime device	`hpu`
License	Apache License 2.0

Architecture

Bluebird 1.5 uses a hybrid local-attention and chunked-memory architecture to provide a longer context window without applying full quadratic attention over the entire sequence.

Local causal attention processes the active token window.
Grouped-query attention (GQA) uses 24 query heads and six key/value heads.
Chunked memory compresses completed 512-token segments into learned memory representations.
Memory-augmented attention incorporates earlier compressed representations into the active token computation.
Learned gating integrates local attention and longer-range memory signals.

The 24-layer network includes 16 local-attention blocks and 8 memory-augmented blocks. Memory-augmented blocks are interleaved through the network at every third layer.

The context length specifies the supported input window; effective recall and reasoning quality depend on the input, task, and fine-tuning.

Quickstart

Bluebird 1.5 uses custom model code. Inspect the repository's Python files before using trust_remote_code=True.

Install the Intel Gaudi software stack and its compatible PyTorch build using the Intel Gaudi installation instructions. Then install the remaining dependencies in that environment:

python -m pip install transformers safetensors accelerate

Generate text on Intel Gaudi 3

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "tudor-iustin22/Bluebird-1.5-Base"
device = torch.device("hpu")

tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    trust_remote_code=True,
)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
).to(device).eval()

prompt = "The future of efficient language models is"
inputs = tokenizer(prompt, return_tensors="pt").to(device)

with torch.inference_mode():
    generated = model.generate(
        **inputs,
        max_new_tokens=128,
        do_sample=False,
        use_cache=False,
        pad_token_id=tokenizer.pad_token_id,
        eos_token_id=tokenizer.eos_token_id,
    )

print(tokenizer.decode(generated[0], skip_special_tokens=True))

Base-model behaviour: The prompt is treated as text to continue. Chat templates and instruction-following behaviour require a separately fine-tuned variant.

PyTorch mixed precision on Intel Gaudi

Intel Gaudi supports mixed precision through PyTorch AMP. The following pattern uses BF16 autocast for operations supported by the Gaudi runtime, while other operations may remain at higher precision:

import torch

# Assumes `model`, `batch`, and `optimizer` have been initialized.
# Move both the model and batch to the HPU device first.
with torch.autocast(device_type="hpu", dtype=torch.bfloat16):
    outputs = model(**batch)
    loss = outputs.loss

optimizer.zero_grad(set_to_none=True)
loss.backward()
optimizer.step()

For distributed training across the eight accelerators, use the Intel Gaudi-enabled PyTorch installation with the HCCL distributed backend. Actual batch sizing, compilation settings, and optimizer configuration depend on the training objective and available sequence length.

See the Intel Gaudi PyTorch documentation for current runtime, distributed training, and mixed-precision guidance.

Training

Bluebird 1.5 Base's documented training-token count is 23,000,514,560 tokens, spanning initial and continued pretraining.

Intel Gaudi 3 compute configuration

Component	Configuration
Accelerators	8 × Intel Gaudi 3 AI accelerators
Accelerator memory	128 GB HBM per device; 1,024 GB aggregate
CPU allocation	160 vCPUs
Host memory	1,792 GiB RAM
Local instance storage	25,600 GB
Deep learning framework	PyTorch
Accelerator integration	Intel Gaudi PyTorch / HPU
Compute precision	Mixed precision
Supported mixed-precision mechanism	PyTorch automatic mixed precision (AMP), including BF16 autocast
Multi-accelerator communication	HCCL

Intel Gaudi 3 uses the PyTorch HPU device interface (torch.device("hpu")). Intel's separate general-purpose GPU products use the PyTorch xpu device interface. The correct backend for Gaudi 3 is hpu.

Training data

The model's documented corpus mixture includes the following datasets:

Dataset	Approximate mixture share
FineWeb-Edu	60%
DCLM Baseline	25%
OpenWebMath	10%
Cosmopedia v2	5%

Dataset proportions are approximate. The distribution may include material subject to its own upstream dataset terms and usage restrictions.

Intended use

Bluebird 1.5 Base is suitable for:

Causal language modelling and text continuation.
Continued pretraining and domain-specific adaptation.
Supervised fine-tuning on downstream tasks.
Exploration of compact transformer architectures.
Research into chunked memory and long-context processing.
Model-efficiency and accelerator-performance studies.

Limitations

As a compact base model, Bluebird 1.5 may produce inaccurate, outdated, repetitive, biased, or unsafe text. A 32K-token input capacity does not guarantee perfect retrieval of earlier information. The base checkpoint has not been specifically aligned for conversational instruction following.

Evaluate model performance for each intended application and implement appropriate safeguards before production deployment. Do not use unverified generations as the sole basis for high-stakes decisions.

License and attribution

Project Bluebird's original release materials are distributed under the Apache License, Version 2.0. Use, modification, redistribution, and commercial use are governed by that license, including applicable notice and attribution requirements.

This release is part of the Fabric 1.5-derived 0.7B model lineage. Copyright and license notices for pre-existing model code, weights, and other third-party material remain with their respective rights holders and must be preserved where required.

The repository should include the full Apache 2.0 license text in a separate LICENSE file and retain applicable upstream NOTICE information.

Read the Apache License, Version 2.0

Citation

@misc{bluebird15base2026,
  title        = {Bluebird 1.5 Base},
  author       = {Tudor Iustin},
  year         = {2026},
  publisher    = {Project Bluebird},
  howpublished = {Hugging Face model repository},
  url          = {https://huggingface.co/tudor-iustin22/Bluebird-1.5-Base}
}

PROJECT BLUEBIRD · Efficient language modelling. Open development.

Downloads last month: -

Safetensors

Model size

0.7B params

Tensor type

F16

Collection including tudor-iustin22/Bluebird-1.5-Base

Bluebird 1.5 Model

Collection

First-Generation Open Research Model trained on Intel Gaudi 3 • 2 items • Updated 43 minutes ago