BR Sovereign LLM Local Smoke Model

Status: engineering artifact only

This is a tiny, randomly initialized Llama checkpoint trained for six CPU optimizer steps on eight original synthetic Brazilian Portuguese documents. It exists only to validate the local from-scratch training, in-process interruption/resume simulation, integrity, export, and loading paths. It does not establish recovery across a process, node, or scheduler restart.

Do not use this model for language generation, evaluation, downstream tasks, or any scientific conclusion. Its loss values are path-validation evidence, not a model-quality result. It was not trained on a supercomputer.

Measured configuration

Field Value
parameters 149,696
decoder layers 2
hidden size 64
attention heads 4
key/value heads 2
context length 64
observed vocabulary 447
numeric type float32
initialization seed 1701
optimizer steps 6
interruption after step 3
execution device CPU

The default implementation used PyTorch 2.13.0, Transformers 5.15.1, and Tokenizers 0.22.2 under Python 3.13.12.

Recovery evidence

Measurement Value
initial loss 6.131303787231445
uninterrupted final loss 6.114025115966797
resumed final loss 6.114025115966797
exact recovery true
gradients finite and nonzero true
model weights changed true
Python and PyTorch RNG restored true
uninterrupted state SHA-256 c68fc0da7e6126f320bda2479d2d7de83bc3e7d09c250abe3c2ccbc1ff6b4055
resumed state SHA-256 c68fc0da7e6126f320bda2479d2d7de83bc3e7d09c250abe3c2ccbc1ff6b4055
checkpoint manifest file SHA-256 c3b64c12e3d65286586e66c6a2541b18a07729c9c8fc78038d80b560de294a88
smoke report file SHA-256 989859964a6020b69783b71a66d761f85a8b8de5a31047b08fd3387c2a8a58f4
tokenizer SHA-256 fa8ce3f9d3e665e0caaffa380e2ee0b6d0e4f71e2084f5dda478c82e3e9f8e90

The full loss trajectories and final tensor hashes matched between the uninterrupted and interrupted-then-resumed executions. The published folder contains safetensors weights, model configuration, tokenizer, and the smoke training report. Optimizer state and the internal .pt recovery state are not published.

Loading

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "guicybercode/br-sovereign-llm-smoke"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id)

Loading success does not make the generated text meaningful.

Data and license

The eight synthetic documents were drafted with AI assistance for BR Sovereign LLM, reviewed and adopted by the project owner, and released under Apache-2.0 to the extent of his rights. The smoke weights are also released under Apache-2.0. This choice does not preselect a license for future scientific weights trained on another corpus.

Contamination, memorization, and safety review

  • Training input was limited to the public eight-document fixture. Its source JSONL SHA-256 is 6896439e772f212c2848846f1f908410a606d81cd8845583024c16496a07979f; the derived Parquet SHA-256 is e2b5d4c01739e92240cfe99e6dccebce3903729d083e61e86afa37f6dd86c8a2.
  • No benchmark examples or third-party corpus documents are present, so there is no benchmark-contamination result to report for this engineering run.
  • No formal extraction or memorization evaluation was run. With eight short training documents, reproduction of fixture phrases must be assumed possible.
  • No safety evaluation was run. The model is randomly initialized and trained for only six steps; its output is not suitable for use.
  • Hosted inference is disabled in the card. Loading remains documented only to verify artifact compatibility.

The scoped approval and weight-license decision are in PUBLICATION_REVIEW.md; the machine-readable removal history is takedown-ledger.json.

Author

Guilherme Monteiro, FIAP
ORCID: https://orcid.org/0009-0008-5294-224X

Downloads last month
-
Safetensors
Model size
150k params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Dataset used to train guicybercode/br-sovereign-llm-smoke