SandLogicTechnologies/laya-GGUF

Laya

Laya is a multilingual, non-autoregressive System 1 decision model developed by Convai Innovations. Instead of generating free-form text, Laya takes a state such as text, email, ticket, or JSON together with typed questions and returns structured answers with probabilities.

The model is designed for fast decision-making workflows where predictable, typed outputs are more useful than generated text. It can perform classification, routing, scoring, moderation, guardrail decisions, and other schema-driven tasks in a single forward pass.

The root laya checkpoint uses ModernBERT-large as its backbone and contains approximately 421 million parameters. The model is trained using Reinforcement Learning for Calibrated Decisions (RLCD), where strictly proper scoring rules encourage the model to report calibrated probability distributions rather than simply maximizing classification accuracy.

---

Model Overview

• Model Name: Laya
• Base Model: ModernBERT-large
• Parameter Count: Approximately 421 Million Parameters
• Context Length: 512 tokens
• Modalities: Text
• Primary Language: English
• Developer: Convai Innovations
• License: Apache 2.0

Laya's decision head scores the available options for each question and produces typed answers without generating a natural-language response. New decision schemas can be supplied at inference time, allowing the model to answer different classification and routing questions without retraining the underlying model.

---

Quantization Formats

This repository provides multiple GGUF quantized versions of Laya optimized for reduced storage and memory requirements. The available formats provide different trade-offs between model footprint and numerical precision.

IQ4_NL

• Size reduction of approx. 69.94% (241.98 MB) compared to 16-bit (804.93 MB)
• 4-bit non-linear quantization that provides a substantially smaller representation of the model weights.
• Suitable for decision-routing, classification, scoring, and guardrail workflows where reduced memory usage is important.
• Provides a compact representation while retaining more weight information than the supplied 3-bit variant.
• Useful for local deployment of lightweight decision systems.

IQ4_XS

• Size reduction of approx. 71.03% (233.22 MB) compared to 16-bit (804.93 MB)
• Compact 4-bit quantization designed to further reduce the model's storage footprint.
• Provides a slightly smaller representation than IQ4_NL while maintaining a 4-bit weight format.
• Suitable for lightweight classification and routing workloads on memory-constrained systems.
• Offers an efficient option when minimizing the local model footprint is a priority.

IQ3_M

• Size reduction of approx. 72.64% (220.19 MB) compared to 16-bit (804.93 MB)
• Aggressive 3-bit quantization that provides the smallest GGUF representation among the supplied Laya variants.
• Reduces storage and memory requirements for highly resource-constrained deployments.
• Suitable for experimentation with classification, routing, moderation, and structured decision workflows.
• The lower numerical precision may introduce a greater trade-off for probability calibration and decision quality compared with higher-precision variants.

Q6_K

• Size reduction of approx. 56.68% (348.73 MB) compared to 16-bit (804.93 MB)
• Higher-precision 6-bit K-Quant representation that retains more numerical information than the supplied 3-bit and 4-bit variants.
• Provides the largest file size among the supplied quantized versions while remaining substantially smaller than the F16 model.
• Suitable for workloads where preserving a higher-precision weight representation is preferred.
• Can be considered for decision systems where probability quality and output consistency are more important than maximum compression.

---

Training Background (Original Model)

Laya is trained specifically for calibrated decision-making rather than free-form text generation. Its training approach is based on Reinforcement Learning for Calibrated Decisions (RLCD).

Reinforcement Learning for Calibrated Decisions

• The model reports probability distributions over possible decisions.
• Exploration introduces zero-mean Gaussian noise into the decision logits during training.
• Rewards are based on strictly proper scoring rules, including log score and spherical score, with ranked probability scoring used for ordinal questions.
• The reward structure encourages the model to report probabilities that accurately represent its uncertainty.
• Updates use REINFORCE with a group-mean baseline in a GRPO-style training approach.
• Multi-turn conversations use TD(λ=1.0) over prefix slices.

The root English checkpoint uses a ModernBERT-large backbone with approximately 395M backbone parameters and a decision head trained from scratch, resulting in approximately 421M total parameters. The decision head contains two transformer layers, an option-marker scorer, and an act/escalate head.

---

Key Capabilities

• Text Classification
  Classifies textual states into user-defined categories and decision labels.

• Decision Routing
  Routes tickets, emails, requests, and other inputs to appropriate departments, workflows, or processing paths.

• Typed Decisions
  Supports structured question types such as choices, scores, and yes/no-style decisions.

• Probability Estimation
  Produces probability distributions associated with its decisions rather than only returning a hard classification.

• Guardrails and Moderation
  Can be used as a lightweight decision layer for moderation, safety checks, escalation, and policy-oriented routing.

• Schema-Driven Decisions
  Allows decision criteria and answer options to be defined at request time, making the model adaptable to different classification workflows without retraining.

• Fast Inference
  Answers multiple questions within a single forward pass, making it suitable for high-throughput decision pipelines.

---

Usage Example

Using Transformers

from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="convaiinnovations/laya"
)

result = classifier(
    "We were charged twice for the same invoice. Please refund the duplicate payment."
)

print(result)

For schema-driven decisions, the official Laya runtime provides a Router interface that accepts a state and a set of typed questions:

from laya import Router

router = Router()

state = "We were billed twice for March. Please refund the duplicate payment."

questions = {
    "department": {
        "type": "choice",
        "instructions": "Which department should handle this request?",
        "criteria": {
            "billing": "invoices, payments, refunds",
            "technical": "bugs, outages, system errors",
            "other": "everything else"
        }
    },
    "urgency": {
        "type": "score",
        "instructions": "How urgent is this request?",
        "criteria": [
            "not urgent",
            "soon",
            "critical deadline or blocking issue"
        ]
    }
}

result = router.predict(state, questions)

print(result["answers"])

---

Recommended Usecases

• Customer Service Routing
  Classify incoming customer requests and route them to the appropriate department.

• Email and Ticket Triage
  Automatically categorize support emails, tickets, and operational requests.

• Moderation and Guardrails
  Build lightweight classification layers for safety, policy, and escalation decisions.

• Business Decision Systems
  Convert unstructured text into structured business decisions and scores.

• Intent Classification
  Identify the intent or category associated with incoming user requests.

• Risk and Escalation Detection
  Estimate whether an input requires escalation, intervention, or additional review.

• High-Throughput Decision Pipelines
  Use a compact non-autoregressive model for applications requiring fast structured decisions.

• Research and Experimentation
  Evaluate calibrated classification, schema-driven decisions, routing, and reinforcement learning for decision models.

The official model card reports benchmark results across intent classification, natural-language inference, typed decisions, and calibration tasks. It also notes that the base English checkpoint is intended as a fast foundation for specialization rather than as a highly accurate zero-shot typed-decision model for every domain.

---

Acknowledgments

These quantized models are based on the original work by the Convai Innovations development team.

Special thanks to:

• The Convai Innovations team for developing and releasing Laya and its calibrated decision-model framework.

• The ModernBERT development team for providing the transformer backbone used by the English Laya checkpoint.

• Georgi Gerganov and the llama.cpp open-source community for enabling efficient local inference and GGUF-based model deployment.

---

Contact

For questions, feedback, or support, please reach out at support@sandlogic.com or visit https://www.sandlogic.com/.