DarkKnighToS223/Flare-9B-low-level-programming

Flare-9B — Low-Level Programming

Flare-9B is a continued-pretrained version of Tesslate/OmniCoder-9B, adapted toward low-level and legacy programming languages:

• x86-64 assembly
• COBOL
• C

The goal of this experiment was to improve performance on niche, low-resource programming languages while retaining the base model's general coding capability.

Flare-9B is primarily a code-completion model rather than a chat model. It works best when given a function signature, partial implementation, or program skeleton.

Training

Property	Value
Base model	Tesslate/OmniCoder-9B
Parameters	9.6B
Method	QLoRA
Quantization	4-bit NF4
LoRA rank	64
LoRA alpha	128
Trainable parameters	173M (1.81%)
Training data	DarkKnighToS223/Assm-cobol-c-c
Training file	train-cpt.jsonl
Tokens processed	~35M
Training progress	0.5 epoch
Sequence length	2048
Hardware	1× NVIDIA RTX 5090
Training time	~3.3 hours
Framework	Unsloth + PEFT

Evaluation

Methodology

The low-level evaluation uses code execution rather than text similarity.

Generated completions are:

1. inserted into the corresponding test harness;
2. compiled using GCC or GnuCOBOL;
3. executed against assertions;
4. counted as correct only when compilation and all runtime checks succeed.

Evaluation settings:

Setting	Value
Metric	pass@1
Decoding	Greedy
Sampling	Disabled
Prompting mode	Code completion
Low-level tasks	10 per language
Validation	Compilation and execution

The low-level suites are intentionally small and should be treated as diagnostic evaluations, not comprehensive measurements of language proficiency.

Low-level programming

Suite	Passed	pass@1
C low-level	10/10	100.0%
COBOL	6/10	60.0%
x86-64 Assembly	2/10	20.0%
Overall	18/30	60.0%

General coding

Benchmark	Score
HumanEval (Python)	77.5% pass@1

Flare-9B achieved strong results on the diagnostic C suite and passed six of ten executed COBOL tasks. Assembly remained the weakest evaluated area, particularly for tasks involving conditionals, loops, and memory traversal.

An interesting outcome is that COBOL produced the stronger diagnostic result even though assembly comprised most of the continued-pretraining corpus and the training dataset contained only 143 COBOL examples.

A directly comparable evaluation of the unmodified base checkpoint is required before drawing strong conclusions about improvement or regression caused by CPT.

Usage

Flare-9B was continued-pretrained primarily on raw code. For best results, use completion-style prompts containing concrete code context instead of long conversational instructions.

Good prompt formats include:

• function signatures;
• partial implementations;
• program skeletons;
• comments immediately followed by code;
• explicit architecture, ABI, or compiler constraints.

Transformers

import torch
from transformers import AutoModelForImageTextToText, AutoTokenizer

model_id = "DarkKnighToS223/Flare-9B-low-level-programming"

tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    trust_remote_code=True,
)

model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype="auto",
    trust_remote_code=True,
)

prompt = """/* x86-64 System V ABI: return a + b */
.intel_syntax noprefix
.global add_asm
add_asm:
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.inference_mode():
    outputs = model.generate(
        **inputs,
        max_new_tokens=128,
        do_sample=False,
    )

generated_tokens = outputs[0, inputs["input_ids"].shape[1]:]
completion = tokenizer.decode(generated_tokens, skip_special_tokens=True)

print(completion)

Verify that your installed Transformers version supports the model architecture. If the checkpoint configuration maps to a causal language model instead, replace AutoModelForImageTextToText with AutoModelForCausalLM.

Prompt example: C

#include <stdint.h>

/* Rotate a 32-bit unsigned integer left by r bits. */
uint32_t rotate_left(uint32_t value, unsigned int r) {

Prompt example: COBOL

       IDENTIFICATION DIVISION.
       PROGRAM-ID. SUM-TO-TEN.

       DATA DIVISION.
       WORKING-STORAGE SECTION.
       01 TOTAL-VALUE PIC 9(4) VALUE 0.
       01 LOOP-INDEX  PIC 9(2) VALUE 0.

       PROCEDURE DIVISION.

Prompt example: x86-64 Assembly

.intel_syntax noprefix
.text

# System V AMD64 ABI
# int max_asm(int a, int b)

.global max_asm
max_asm:

For assembly prompts, explicitly specify:

• Intel or AT&T syntax;
• target architecture;
• calling convention;
• expected symbol name;
• input and return types.

GGUF

GGUF quantizations are available for use with llama.cpp:

• Q8_0
• Q6_K
• Q5_K_M
• Q4_K_M
• Q2_K

Q4_K_M or higher is recommended for more reliable code generation. Q2_K substantially reduces model size but may noticeably degrade output quality, especially for syntax-sensitive languages.

Limitations

• The low-level evaluation contains only 10 tasks per language.
• The reported results are preliminary and may have high variance.
• Assembly performance is currently limited.
• The model may generate code that does not compile or that violates the requested ABI.
• Generated code may contain security vulnerabilities or undefined behavior.
• HumanEval and the low-level suites measure only a limited subset of coding ability.
• Completion-oriented prompting generally works better than chat-style prompting.
• Quantization can reduce accuracy, especially at very low bit widths.
• No claim is made that the model is suitable for production-critical systems.

Always inspect, compile, test, and review generated code before use.

Reproducibility

For fully reproducible low-level results, the following should be published alongside the model:

• benchmark prompts;
• test harnesses and assertions;
• generation configuration;
• output-extraction logic;
• compiler versions and flags;
• raw model completions;
• base-model results produced with the same evaluation pipeline.

Intended use

Flare-9B is intended for:

• research on low-resource programming languages;
• code-completion experiments;
• legacy-code exploration;
• low-level programming assistance;
• continued-pretraining and parameter-efficient fine-tuning research.

It is not intended to replace compiler diagnostics, testing, static analysis, security review, or expert verification.

License

This model is released under the Apache License 2.0, following the license of the base model.

Users are responsible for reviewing the licenses and usage conditions of the base model, training dataset, dependencies, and generated outputs.