fastino/GLiDE-27B

GLiDE 27B

Inference provider: Fastino — use GLiDE through Fastino's hosted API (POST https://api.fastino.ai/v1/systemone, model fastino/GLiDE). Get an API key · API documentation

GLiDE 27B is a 27B GLiDE decision model. It scores 72.49 on the Decision Index 0.3.1 public suite, measured with the official kit on one RTX PRO 6000, with a median latency of 112 ms per request on that GPU. Weights and compute are bf16.

GLiDE is a decision model from Fastino. Given a state (text or JSON, optionally with images) and one or more typed questions, it returns a probability for every option of every question.

Fastino API

Use GLiDE through Fastino’s hosted API for classification, routing, and scoring. Send decision requests to POST https://api.fastino.ai/v1/systemone with model fastino/GLiDE. Create an account and get an API key, or follow the API quickstart. The instructions below cover running this repository locally.

Overview

• Typed decisions: choice (one of up to 255 named options), noul (yes / no, returns P(yes)) and score (ordered levels).
• Probabilities for every option, read from a 255-way decision readout. Nothing is generated.
• One forward pass per question. All questions of a request (and of concurrent requests) are scored together in micro-batches.
• Text, JSON and images: the state can be a string, any JSON value, or a list of text and image parts.
• Long contexts: up to 40,000 tokens per question prompt by default (state, question, options and image tokens).
• Interfaces: a Python API and an HTTP server speaking the System One format (POST /v1/systemone).

Hardware and memory

Weights	bf16, 52.2 GB on disk: 51.2 GB language model, 0.92 GB vision encoder, 2.6 MB readout
GPU	one GPU with 80 GB or more
GPU memory	52.2 GB after loading; 56.4 GB peak over the 750-request latency sample below
Tested GPU	NVIDIA RTX PRO 6000 Blackwell Server Edition (96 GB)
Load time	10 s from a warm local cache; longer from network storage

Weights are copied to the GPU tensor by tensor, so host memory use during loading stays low. Image preprocessing runs on the CPU.

Install

pip install huggingface_hub
hf download fastino/GLiDE --local-dir GLiDE-27B
cd GLiDE-27B
pip install -r requirements.txt

• Tested with Python 3.12, torch==2.14.0 and transformers==5.17.0 (all pins are in requirements.txt).
• flash-linear-attention==0.5.2 is optional but recommended: it provides fast kernels for the linear-attention layers. Without it transformers falls back to a much slower PyTorch implementation, which was not part of the validation. All numbers below were measured with it installed.
• At load, transformers logs that causal_conv1d is not installed and uses its PyTorch path. This is expected: the validated setup does not use that package.
• The glide/ package lives at the repository root: run from the repository directory or add it to PYTHONPATH.

Quickstart

Python

from glide import Glide

model = Glide.from_pretrained("GLiDE-27B")   # local directory, or the Hub repo id "fastino/GLiDE"

result = model.decide(
    state={"ticket": "I was charged twice for order #4821 and the second charge is still pending.", "customer_tier": "gold"},
    questions={
        "intent": {"type": "choice", "instructions": "What does the customer want?",
                   "criteria": {"refund": "Get money back", "cancel": "Cancel the order",
                                "track": "Track a delivery", "other": "Something else"}},
        "urgent": {"type": "noul", "instructions": "Should this ticket be escalated today?"},
        "priority": {"type": "score", "instructions": "What support priority should this ticket get?",
                     "criteria": ["low: no impact on the customer", "medium: the customer is inconvenienced",
                                  "high: the customer has lost money or cannot use the service"]},
    },
)

Result on an RTX PRO 6000 (rounded):

{
  "model": "glide-27b",
  "answers": {
    "intent": {"type": "choice", "choice": "refund", "confidence": 0.9639, "probabilities": {"refund": 0.9808, "cancel": 0.0019, "track": 0.0005, "other": 0.0169}},
    "urgent": {"type": "noul", "noul": 0.852, "confidence": 0.7039},
    "priority": {"type": "score", "score": 2, "expected_level": 1.9465, "confidence": 0.9023, "probabilities": {"0": 0.0031, "1": 0.0473, "2": 0.9496}}
  },
  "usage": {"input_tokens": 316, "output_tokens": 0}
}

With images (PIL images, file paths, bytes, base64 strings or data URLs):

from PIL import Image, ImageDraw

img = Image.new("RGB", (512, 512), "white")
ImageDraw.Draw(img).ellipse((128, 128, 384, 384), fill="red")

result = model.decide(
    state="A product photo from the catalogue.",
    questions={
        "color": {"type": "choice", "instructions": "What color is the shape in the image?",
                  "criteria": {"red": "Red", "green": "Green", "blue": "Blue"}},
        "circle": {"type": "noul", "instructions": "Is the shape a circle?"},
    },
    images=[img],
)

{
  "model": "glide-27b",
  "answers": {
    "color": {"type": "choice", "choice": "red", "confidence": 0.9998, "probabilities": {"red": 0.9998, "green": 0.0001, "blue": 0.0001}},
    "circle": {"type": "noul", "noul": 0.9981, "confidence": 0.9961}
  },
  "usage": {"input_tokens": 653, "output_tokens": 0}
}

model.decide_batch([{"state": ..., "questions": ...}, ...]) scores several requests together and returns one item per request: a response dict, or a glide.RequestError for a request that cannot be served (decide raises it instead).

HTTP server

GLIDE_MODEL=/models/GLiDE-27B uvicorn glide.server:app --host 0.0.0.0 --port 8000
# or
python -m glide.server --model /models/GLiDE-27B --port 8000

The model loads at startup; GET /health answers once it is ready.

curl -s localhost:8000/v1/systemone -H 'Content-Type: application/json' -d '{
  "model": "glide-27b",
  "state": "Order #4821 was delivered to the wrong address and the customer wants it re-sent today.",
  "questions": {
    "action": {"type": "choice", "instructions": "What should support do next?",
               "criteria": {"resend": "Re-ship the order", "refund": "Refund the order", "investigate": "Open a carrier investigation"}},
    "urgent": {"type": "noul", "instructions": "Is the request time-sensitive?"}
  }
}'

With an image, the state is a list of parts:

IMG=$(base64 < photo.png | tr -d '\n')
curl -s localhost:8000/v1/systemone -H 'Content-Type: application/json' -d '{
  "model": "glide-27b",
  "state": ["Inspection photo from line 3.", {"type": "image_url", "image_url": {"url": "data:image/png;base64,'"$IMG"'"}}],
  "questions": {"damaged": {"type": "noul", "instructions": "Is there visible damage on the part?"}}
}'

Request

Field	Type	Description
model	string	Optional. Ignored; the response reports the served model name (GLIDE_MODEL_NAME).
state	string, JSON value, or list of parts	The context to decide on. A list that contains image parts is multimodal: text parts are strings or {"type": "text", "text": "..."}, image parts are {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}. Text parts are joined with newlines; images are placed before the text. Any other value is used as is (JSON values are serialized).
questions	object	{key: question}. Every question is answered independently, one forward pass each.

Question fields:

Field	Description
type	choice, noul or score.
instructions	The question. A string; objects are rendered as key: value lines.
criteria	choice: object {option_key: description}, 1 to 255 options, shown in the given order; descriptions may be strings or JSON (an empty description shows only the key). noul: optional {"true": "...", "false": "..."} descriptions (default Yes / No). score: a list of level descriptions (level i is index i), or an object keyed by numeric level (sorted ascending and re-indexed from 0).

Images must be inline (data URLs or raw base64); remote URLs are not fetched. Each image is resized to at most GLIDE_MAX_PIXELS pixels (default 1024 x 1024, i.e. at most 1,024 image tokens).

Response

For the first curl request above (RTX PRO 6000, rounded; the HTTP response also carries latency_ms):

{
  "model": "glide-27b",
  "answers": {
    "action": {"type": "choice", "choice": "investigate", "confidence": 0.6127, "probabilities": {"resend": 0.1909, "refund": 0.0056, "investigate": 0.8036}},
    "urgent": {"type": "noul", "noul": 0.9937, "confidence": 0.9874}
  },
  "usage": {"input_tokens": 172, "output_tokens": 0}
}

Answer type	Fields
choice	choice: most probable key. probabilities: {key: p} over all options, summing to 1. confidence: top minus second probability.
noul	noul: P(true). confidence: |2 P(true) - 1|.
score	score: most probable level index. expected_level: sum of index x probability. probabilities: {"0": p, "1": p, ...}. confidence: top minus second probability.

usage.input_tokens is the prompt length summed over the request's questions (image tokens included). latency_ms is the server-side time for the request (queueing plus inference).

Errors return {"error": "..."}: 422 for requests the model cannot serve (see below), 401 for a missing or wrong API key, 503 when the queue is full, 504 on timeout, 500 for an internal error.

Production notes

Process model. Run one server process per GPU (workers=1) and scale out with replicas behind a load balancer. A single worker thread owns the GPU; HTTP handler threads only queue requests and wait for results.

Batching and concurrency (environment variables):

Variable	Default	Meaning
GLIDE_MODEL	required	Model directory or Hub repo id.
GLIDE_MODEL_NAME	glide-27b	model field in responses.
GLIDE_DEVICE	cuda	Torch device, e.g. cuda:1.
GLIDE_MAX_BATCH	16	Max requests scored together in one micro-batch.
GLIDE_BATCH_WAIT_MS	5	How long the worker waits to fill a micro-batch.
GLIDE_TOKEN_BUDGET	16384	Max padded tokens per forward pass; question prompts are length-sorted and packed under it (a longer prompt runs alone).
GLIDE_MAX_LEN	40000	Max tokens per question prompt, image tokens included.
GLIDE_MAX_QUEUE	1024	Queued requests before new ones get 503.
GLIDE_REQUEST_TIMEOUT_S	600	Max wait for a result before 504.
GLIDE_THREADS	1024	HTTP handler threads (concurrent open requests).
GLIDE_API_KEY	unset	If set, require Authorization: Bearer <key>.
GLIDE_MIN_PIXELS / GLIDE_MAX_PIXELS	65536 / 1048576	Image resize bounds.

Max context. 40,000 tokens per question prompt by default. Full-attention cost grows quadratically with length, so long prompts dominate a micro-batch; set client timeouts with headroom (for example 60 s). Raising GLIDE_MAX_LEN is possible but untested.

Health. GET /health returns {"status": "ok", ...} with the queue length and batch counters once the model is loaded.

Latency (bf16, warm, one request at a time; 750 requests of mixed length with 2.3 questions per request on average):

Path	Median	Mean	p80	p90	p99
Python API, RTX PRO 6000 Blackwell (96 GB)	112 ms	356 ms	328 ms	1,093 ms	3,702 ms
HTTP server, same GPU (client wall time)	113 ms	357 ms	328 ms	1,093 ms	3,695 ms

Latency grows with prompt length and the number of questions. HTTP adds about 2 ms per request on localhost (server-side latency_ms vs client wall time). Requests with one small image (50 requests, 1.3 questions each) took 122 ms median. On this mix one request already keeps the GPU busy, so add replicas for throughput.

Cold start and the kernel cache. The linear-attention kernels are compiled and autotuned by Triton on first use of each new shape, so the first requests of a fresh process take seconds longer. Compiled kernels and autotuning results are cached on disk (TRITON_CACHE_DIR, default ~/.triton/cache) and reused by a restarted process. In production:

1. point TRITON_CACHE_DIR at persistent storage shared by replicas with the same GPU type and software versions;
2. after the first deployment (or an upgrade), replay a few hundred representative requests through one replica to fill the cache;
3. keep a replica out of rotation until /health is up and its first warm-up requests have returned.

Unsupported requests get HTTP 422 with a message:

• more than 255 options: options per choice exceeds the label set (N > 255)
• a question prompt over GLIDE_MAX_LEN: prompt exceeds the maximum context length
• an image that cannot be decoded: invalid image: ...
• an unknown question type, missing state, or malformed questions / criteria

A request with an empty questions object returns 200 with empty answers.

Determinism. There is no sampling. The same request scored alone gives bit-identical results on the same GPU type and software stack. The questions of one request always share forward passes; when concurrent requests are also micro-batched, padding changes the shape of the bf16 computation and probabilities move slightly; separate processes or machines can also differ in the last bits (different kernel autotuning). Set GLIDE_MAX_BATCH=1 when results must not depend on other traffic.

Precision. Weights and compute are bf16 with PyTorch SDPA attention (Glide.from_pretrained disables the cuDNN SDPA backend process-wide, so the memory-efficient kernel is used).

Logging. Request contents are not logged.

Performance

Decision Index 0.3.1 public suite: 72.49, measured with the official kit on one RTX PRO 6000.

Limitations

• It chooses among the options it is given; it does not generate text or explanations.
• Probabilities are model estimates; set decision thresholds on your own data.
• At most 255 options per question and 40,000 tokens per question prompt (default); larger requests are rejected with 422.
• Images: still images supplied inline only; no video and no remote URLs.
• Cost grows with the number of questions: each question is a separate forward pass over the full state.

License

GLiDE’s Fastino contributions are licensed under the Fastino Community License. Third-party components retain their original licenses; see the NOTICE in the public GLiDE repository for attribution.