ConeML/coneml-810m-alpha

ConeML 810M Alpha

ConeML 810M Alpha is an 810,112,512-parameter, decoder-only language model trained from scratch and instruction-tuned for constrained English tasks. Its measured profile favors single Python functions, selected small-number arithmetic families, two-step word problems, and a narrow designated-refusal policy.

Results are specific to the documented prompt formats and evaluation task families. They are not claims of general coding, reasoning, factual knowledge, or production reliability.

• Architecture: LLaMA-style; 28 layers, d=1536, 12 heads (3 KV), tied embeddings, 8192 context, 32,768-token BPE tokenizer
• Pretraining corpus: 27.87B curated tokens
• Selected pretrained-base exposure: approximately 12.32B token positions, or 15.2 tokens per parameter
• Pretraining wall time: approximately 11 days on one NVIDIA RTX 5090; fine-tuning and evaluation are excluded
• Release model.safetensors SHA-256: e5df8f2edd773ad079ba00d383ace81b96cbb9579d64c570db505f9efc032b76
• Companion release: coneml-810m-alpha-arithmetic, from the same base, has substantially higher carry/borrow accuracy and lower function and two-step accuracy

1. Measured profile

ConeML internal certification uses held-out instances from the same task families as the training generators. It is not independent third-party certification. Code is scored by executing each generated function against held-out tests.

held-out task family	result	n
Python function writing (single function, executed)	69.0%	300
two-step word problems	79.0%	500
2-digit × 1-digit multiplication	97.4%	500
1-digit multiplication	97.8%	45
1-digit addition	98.6%	71
"Which is bigger, X or Y?"	95.0%	300
2-digit subtraction with borrow	91.0%	500
comparison word problems	72.6%	500
missing addend	71.6%	500
2–3-digit addition with carry	53.2%	500
missing factor	17.8%	500

Behavioral screens:

• Designated-refusal probe: 17/17 refusal prompts produced refusals, and 5/5 in-scope contrast prompts in matching wording were answered. This small targeted probe measures adherence to that policy, not general epistemic calibration.
• Conversation-v2: 8/8 turns passed in one scripted dialogue covering cross-turn arithmetic, story revision, comparison explanation, and one unknown-personal-fact refusal. This is a single dialogue, not a general conversation benchmark.
• Everyday-reasoning screen: 11/20 by manual adjudication; the phrase-sensitive automatic scorer recorded 6/20.

2. Peer comparison and efficiency

The pair was tested against five post-trained peers on the same reduced diagnostic screen, through each model's native instruction interface, with greedy decoding, repetition penalty 1.15, and matched short-answer budgets.

model	parameters	mixed arithmetic (n=585)	executed functions (n=100)	transitive names d1/d3/d5 (n=32 each)	designated refusals (n=17)
ConeML 810M Alpha	0.81B	421 (72.0%)	83	23/15/15	13
ConeML 810M Alpha-Arithmetic	0.81B	442 (75.6%)	35	26/17/16	11
Qwen3.5	0.8B	154 (26.3%)	93	23/22/10	0
Qwen3	0.6B	223 (38.1%)	98	18/10/9	1
Llama 3.2 Instruct	1.24B	343 (58.6%)	79	16/7/5	1
TinyLlama Chat	1.1B	113 (19.3%)	51	8/18/22	0
SmolLM2 Instruct	1.7B	380 (65.0%)	96	16/6/5	1

This is not a general leaderboard: the task families match ConeML's trained surfaces. On this screen, the ConeML pair led mixed arithmetic; ConeML 810M Alpha exceeded Llama 3.2 and TinyLlama on function writing but trailed Qwen3.5, Qwen3, and SmolLM2. All models recorded zero over-refusals on five in-scope contrasts.

On the question-form name-chain depth screen, ConeML 810M Alpha-Arithmetic recorded 26/32, 17/32, and 16/32 at depths 1, 3, and 5; ConeML 810M Alpha recorded 23/32, 15/32, and 15/32. The arithmetic variant exceeded Qwen3 and Llama 3.2 at all three shown depths and Qwen3.5 at depths 1 and 5, while Qwen3.5 led it at depth 3 and TinyLlama led the group at depth 5. Entity-chain controls were mixed and are reported in the full table. Chance is 1/(depth+1); this is an exact-selection surface test, not a claim of general reasoning.

Qwen3.5 with thinking enabled reached 451/585 (77.1%), compared with ConeML 810M Alpha-Arithmetic's 442/585 (75.6%), while using at least 17.6 times the generated-token budget per item and approximately 24 times the recorded wall time. Runtime ratios are implementation-specific.

ConeML's selected base consumed 12.32B token positions—15.2 tokens per parameter and approximately 5.99e19 training FLOPs under the 6ND convention. Pretraining took approximately 11 days on one RTX 5090. Assuming, rather than claiming to have metered, a 0.70 kW average wall draw gives 184.8 kWh; at CHF 0.14–0.30/kWh, that is approximately CHF 26–55 of marginal pretraining electricity. This excludes hardware, labor, SFT, evaluation, conversion, and emissions accounting.

The complete category table, peer wins, adverse base result, evaluated revisions, assumptions, and frozen evidence commitments are in eval/PEER_COMPARISON.md.

3. Standard benchmark limits

Zero-shot, greedy decoding with a 256-token generation cap. GSM8K uses strict final-number matching; HumanEval reports pass@1 from executed tests.

benchmark	this model	same base, no fine-tune
GSM8K (test, first 200)	12/200 (6.0%)	7/200 (3.5%)
HumanEval (pass@1, tests executed)	5/164 (3.0%)	0/164 (0.0%)

These results are weak and define an important boundary: the narrower task-family results above do not transfer to broad GSM8K or HumanEval performance.

4. Supported prompt formats

Evaluated decoding is greedy with repetition penalty 1.15, as pinned in generation_config.json.

# task format (single turn)
Question: <task>
Answer:

# dialogue format (chat_template.jinja)
User:
<message>
Assistant:

Dialogue generation ends at <|endoftext|>. Prompting outside these formats, including paraphrases, was not systematically evaluated except for the published raw-completion spot check.

5. Quantized variants

Q8_0 GGUF was evaluated separately under llama.cpp. That runtime and its stop handling differ from the bf16 harness, so the scores are not directly comparable and differences cannot be attributed solely to quantization.

held-out task family	Q8_0
Python function writing	75.0% (n=300)
two-step word problems	57.0% (n=500)
addition with carry	40.2% (n=500)
multiplication families	94.8–95.6%
which-bigger	92.7% (n=300)

Use bf16 when the bf16 profile is required.

For local GGUF inference:

llama-cli \
  -m coneml-810m-alpha-Q8_0.gguf \
  -p $'Question: What is 7 + 8?\nAnswer:' \
  -n 32 --repeat-penalty 1.15 --no-conversation --single-turn

For Ollama, download Modelfile beside the Q8_0 file, then run:

ollama create coneml-810m-alpha -f Modelfile
ollama run coneml-810m-alpha "What is 7 + 8?"

6. Loading with Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "ConeML/coneml-810m-alpha"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    dtype=torch.bfloat16,
    device_map="auto",
)

prompt = "Question: What is 7 + 8?\nAnswer:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(
    **inputs,
    max_new_tokens=32,
    do_sample=False,
    repetition_penalty=1.15,
)
answer = tokenizer.decode(
    output[0][inputs["input_ids"].shape[-1]:],
    skip_special_tokens=True,
)
print(answer)

This is a text-only causal language model. Load it with AutoModelForCausalLM or LlamaForCausalLM, not a multimodal model class.

7. Intended use and safety

This release is intended for research, constrained local inference, evaluation-harness development, and experiments whose prompts match the documented formats. Review outputs before use.

It is not intended for factual retrieval, high-stakes medical, legal, or financial decisions, autonomous action, or unsandboxed execution of generated code. Generated code may be incomplete or unsafe; inspect and test it in an isolated environment.

8. Known limitations

• Not a general code model: complete programs scored 0/10, executable SQL 1/10, and bash 1/5. The 69% result is for single functions on ConeML's task-family-matched battery.
• Addition with carry is this release's weakest internally certified arithmetic family at 53.2%; the arithmetic companion scored 96.8%.
• The model was not evaluated as a factual-QA system and refused all six real-world factual prompts in the designated-refusal probe.
• Money/decimal change-making scored 0/3 on the held-out screen; the model also showed yes-bias on feasibility questions.
• Multi-step GSM-class reasoning remained weak.
• Frameless prompting was spot-checked, not systematically certified.

9. Evidence and reproducibility

The eval/ directory publishes:

• summary.json: all aggregate bf16 results used above;
• representative-samples.json: deterministic, rule-selected examples from ConeML-owned probes;
• EVALUATION_METHODOLOGY.md: evaluation definitions and scope;
• separate Q8_0 aggregate results; and
• PRIVATE_EVIDENCE_SHA256SUMS.txt: commitments to the complete row-level evidence retained by ConeML.
• peer-comparison-summary.json, PEER_COMPARISON.md, and PEER_EVIDENCE_SHA256SUMS.txt: the peer results, interpretation, and commitments to the retained peer-generation evidence.

Complete proprietary probe rows and generations are retained privately to avoid releasing the full measurement instruments. The hashes permit later disclosure to be checked against the evidence frozen at release time. Public GSM8K and HumanEval rows are not redistributed; only aggregate results are published. SHA256SUMS.txt covers every shipped file.

10. Research license and commercial work

Released for non-commercial use under CC BY-NC 4.0. Commercial use is not granted by this release. See LICENSE.md.

For commercial evaluation, model-engineering, or licensing enquiries, contact contact@coneml.com or visit ConeML for Organizations. Commercial availability does not imply that this alpha checkpoint is suitable for an unvalidated production workflow.