aksern/nexi-g1

nexi g1

nexi g1 is a small causal language model developed by aksern for text continuation and language modeling experiments.

The model is based on a GPT-style decoder-only Transformer architecture and is trained as a text continuation model, not as a conversational assistant or instruction-following chatbot.

Model Details

• Model name: nexi g1
• Organization: aksern
• Architecture: GPT-style causal Transformer
• Parameters: ~30.3M
• Task: Causal language modeling / text continuation
• Training objective: Next-token prediction
• Training stages: Pretraining

Intended Use

nexi g1 is intended primarily for:

• text continuation
• experimenting with small language models
• studying causal language modeling
• evaluating small-scale pretraining approaches
• research and educational purposes

The model is not designed to be a chatbot. It does not use instruction tuning or conversational fine-tuning.

For example, given:

The history of computers began

the model attempts to continue the text based on patterns learned during pretraining rather than answer the prompt as a question.

Training Data

nexi g1 was pretrained on a mixture of datasets, including Wikipedia and additional text datasets.

The training corpus was assembled for experimentation with small generative language models. Because the corpus is a mixture of sources, the model may reproduce characteristics of those sources, including encyclopedic formatting, article structures, names, lists, and other recurring patterns.

The exact dataset composition and preprocessing configuration may vary between experiments.

Training

The model was pretrained for 3 epochs.

Training completed after approximately:

• 145,254 training steps
• ~76.5 minutes
• 30.3M parameters

Final reported losses:

Train loss: 0.565
Eval loss:  3.739

The relatively large difference between training and evaluation loss indicates substantial overfitting to the training distribution.

Limitations

nexi g1 is a relatively small language model with approximately 30.3M parameters. It should not be expected to match the capabilities of modern large language models.

Known limitations include:

• repetition and generation loops
• weak long-range coherence
• inconsistent factual knowledge
• poor instruction following
• limited semantic understanding
• occasional topic drift
• sensitivity to prompts and sampling parameters
• overfitting to patterns present in the training corpus

Because Wikipedia represents a significant portion of the training data, generated text may also exhibit Wikipedia-like structures, including headings, references, lists, and encyclopedic phrasing.

For example, the model may continue:

The United States.

with an article-like structure rather than interpreting it as a request for information.

This behavior is expected for a pretrained text continuation model.

Biases and Safety

The model inherits biases, errors, and other undesirable patterns from its training data.

Generated text should not automatically be considered factual, reliable, or representative of any person, group, or event.

The model has not been specifically aligned for safety or instruction following.

Users should evaluate generated content before using it in applications or publications.

Example

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "aksern/nexi-g1"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

prompt = "The history of computers began"

inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(
    **inputs,
    max_new_tokens=100,
    temperature=0.95,
    top_p=0.95,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Sampling

nexi g1 is a generative model, so sampling parameters can significantly affect its output.

Useful parameters include:

• temperature
• top_p
• top_k
• max_new_tokens

Higher temperatures generally produce more varied output, while lower temperatures tend to produce more predictable continuations.

Repetition may still occur at common sampling settings because of the model's small size and training characteristics.

Development

nexi g1 is the first model in the nexi family.

The g1 designation refers to the first generation-oriented model in the series. Future models may explore larger architectures, improved datasets, different training configurations, and additional training stages.

nexi g1 is primarily an experimental foundation for these future models.

Acknowledgements

Built with:

• PyTorch
• Hugging Face Transformers
• Hugging Face Hub

Developed by aksern.