VEG-Experiment

Experimental research release. This is an early public checkpoint for the Veyrian Execution Generator (VEG). A better version is planned and will be released later.

VEG-Experiment is not a Transformer. It is a byte-level language model using the VEG-3 resonance stack: learned byte embeddings, phase/content routing, associative reads, recurrent selective fields, and gated feed-forward blocks. The implementation is NumPy/CuPy-based and does not require PyTorch or transformers.

Model

• Parameters: 81,676,032
• Vocabulary: 256 raw bytes
• Context length: 512 bytes
• Architecture: VEG-resonance-stack/3
• Precision: FP32
• Configuration: D=832, L=10, NH=13, PP=32, RK=32, FF=2240, UR=64
• Memory mode: resonance
• Repeated execution / workspace routing: disabled in this checkpoint

Files

• model.safeveyra — native VEG checkpoint, including architecture metadata
• model.safetensors — standard safetensors export with the same FP32 tensors
• model_config.json — model dimensions and metadata
• veg/ — minimal inference/training source used by this checkpoint

Evaluation

Bits-per-byte is lower-is-better byte-prediction loss. It is not a factual, reasoning, coding, or instruction-following benchmark.

Held-out split	bpb
Capability retention validation	1.2068
Capability validation	1.2336
Capability test	1.2382
Nitron validation	1.2464
Nitron test	1.2460
Original validation	1.2838

Important limitations

This checkpoint can produce fluent-looking text, but it is not reliable for factual QA, arithmetic, reasoning, coding, or instruction following. For example, during release checks it generated incorrect answers to simple factual and arithmetic prompts.

Use it for architecture research, byte-level language-model experiments, and reproduction—not for applications that require correct answers.

Requirements

pip install -r requirements.txt

GPU inference additionally requires a CUDA-compatible CuPy build, for example cupy-cuda12x on a CUDA 12 system.

Generate text

CPU:

python -m veg.infer \
  --load model.safeveyra \
  --prompt "The capital of France is" \
  --length 200

GPU with CuPy:

VEG_GPU=1 CUPY_TF32=1 python -m veg.infer \
  --load model.safeveyra \
  --prompt "The capital of France is" \
  --length 200 \
  --temperature 0.8

The .safetensors export preserves the VEG metadata, so it can also be passed to --load.

Training provenance

This checkpoint was warm-expanded from a smaller VEG-3 resonance model, trained on a broad Nitron-derived UTF-8 corpus, then continued on a capability-focused QA/code/math/science mixture with broad-corpus replay. The training objective was next-byte prediction.

Status

This is an experimental artifact. It is released to make the VEG approach inspectable and reproducible. A stronger, better-evaluated version is planned.