MiniBananaMind-V1

MiniBananaMind-V1 is a compact LLaMA-style causal language model from BananaMind. It is trained from scratch for next-token text generation on streamed FineWeb-Edu data and is intended as a small, inspectable base model for experiments, demos, and lightweight research workflows.

This is a base language model, not an instruction-tuned assistant. It is best used for continuation-style generation and experimentation rather than factual question answering or chat.

Model Details

• Developer: BananaMind
• Model type: LLaMA-style causal language model
• Library: Transformers
• Task: Text generation
• Training data: FineWeb-Edu
• Checkpoint: MiniBananaMind-V1 uploaded training checkpoint
• License: Apache 2.0

Architecture

Setting	Value
Layers	6
Hidden size	256
Attention heads	8
KV heads	8
Intermediate size	768
Context length	512 tokens
Vocabulary size	32,000
Parameters	~21.5M
Precision	float32 checkpoint

Intended Use

MiniBananaMind-V1 is suitable for:

• Small-scale language-model experiments
• Educational demos of decoder-only generation
• Testing tokenization, generation settings, and inference pipelines
• Research prototypes where a very small causal LM is useful

It is not recommended for production assistants, safety-critical use, or tasks that require reliable factual knowledge.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo_id = "BananaMind/MiniBananaMind-V1"

tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    torch_dtype=torch.float32,
    device_map="auto",
)

prompt = "A computer is a machine that"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=64,
        do_sample=True,
        temperature=0.2,
        top_p=0.9,
        repetition_penalty=1.1,
        pad_token_id=tokenizer.pad_token_id,
        eos_token_id=tokenizer.eos_token_id,
    )

print(tokenizer.decode(output[0], skip_special_tokens=True))

Generation Notes

Because this is a small base model, output quality depends heavily on prompt style and sampling settings. A temperature of 0.2 is recommended for more stable continuations. For more varied text, increase temperature or top_p.

Limitations

• The model may hallucinate facts, names, citations, and dates.
• It has not been instruction tuned or aligned for chat behavior.
• It may reproduce biases or unsafe patterns present in web-scale training data.
• The short 512-token context length limits long-document use.
• Small model size means weaker reasoning and factual recall than larger LMs.

Training Data

MiniBananaMind-V1 was trained on streamed FineWeb-Edu text. FineWeb-Edu is a large educational-quality web corpus, so users should expect broad web-language coverage as well as the usual limitations of internet-scale data.

Training data attribution: this model was trained on FineWeb-Edu, a dataset released by Hugging Face as part of the FineWeb family.

Citation

If you use this model in a project, cite the Hugging Face repository and attribute the FineWeb-Edu training data:

@misc{minibananamindv1,
  title = {MiniBananaMind-V1},
  author = {BananaMind},
  year = {2026},
  howpublished = {\url{https://huggingface.co/BananaMind/MiniBananaMind-V1}}
}

Dataset: HuggingFaceFW/fineweb-edu