Convergent-ch/Pico-1.3-Coding-Large

Pico 1.3 Coding Large

Pico 1.3 Coding Large is a tiny English model with 6.9 million parameters (14 MB) made by Convergent. It chats like the other Pico models and writes simple Python functions and simple HTML snippets when your message contains the word codingPico.

codingPico write a function that returns the largest number in a list

def largest_number(numbers):
    """
    Returns the largest number in a list of numbers.

    :param numbers: List of integers
    :return: The largest number in the list, or None if the list is empty
    """
    if not numbers:
        return None
    return max(numbers)

Without the word it does not write code; asked "How do I get you to write code?" it answers "Say codingPico together with your request, for example: codingPico write a function that adds two numbers."

Honest result

Measured with greedy decoding on tasks that were never in the training data:

Test	Result
50 simple Python tasks of the same kind as the training data, checked by running their tests	68 % pass
20 new hand-written Python tasks with function names that never occur in the training data	10 % pass (2 of 20)
30 HTML requests: markup parses with every tag closed	100 %
30 coding requests without "codingPico": answered without a code block	100 %

• The 68 % is not real problem solving: for 48 of those 50 tasks a function with the same name (for example is_prime) exists in the training data. The model mostly reproduces patterns it has seen.
• On genuinely new tasks it usually writes code that looks right but does the wrong thing (for example counting all words instead of only long ones). 2 of 20 passed.
• "Valid HTML" only means the tags are well formed. It does not mean the page matches the request: asked for a blue button it may produce a generic styled page.
• It is not tested on HumanEval or similar benchmarks; expect about 0 % there.

Use it as a toy and a demonstration of how far 7M parameters go, not as a coding assistant.

Model

Parameters	6,917,376
Architecture	Llama-style decoder (RMSNorm, SwiGLU, RoPE, tied embeddings)
Layers / hidden / MLP	6 / 256 / 704
Attention heads	4 (head dim 64)
Context	1,024 tokens
Vocabulary	8,192 (byte-level BPE trained on code and chat text)

Usage

Ollama

Download pico-1.3-coding-large.gguf and Modelfile into one folder, then:

ollama create pico1.3-coding-large -f Modelfile
ollama run pico1.3-coding-large

Use the Modelfile; without its chat template the model produces nonsense.

transformers

from transformers import AutoTokenizer, AutoModelForCausalLM

tok = AutoTokenizer.from_pretrained("Convergent-ch/Pico-1.3-Coding-Large")
model = AutoModelForCausalLM.from_pretrained("Convergent-ch/Pico-1.3-Coding-Large")

ids = tok.apply_chat_template(
    [{"role": "user", "content": "codingPico write a function that returns the largest number in a list"}],
    add_generation_prompt=True, return_tensors="pt")
out = model.generate(ids, max_new_tokens=150, do_sample=False)
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))

Chat format: <|bos|><|user|>\n...<|end|>\n<|assistant|>\n, generation stops at <|end|>. Use temperature 0 to 0.2.

Training

Trained from scratch in one stage on one RTX 3060 Ti with MakeAI: 3,200 steps of 32 x 1,024 tokens (about 7 passes over 14.8M tokens), AdamW, learning rate 2e-3 with cosine decay, bf16, about 12 minutes. Final validation loss 1.508 (perplexity 4.5) on held-out chats; the loss is only computed on the assistant's replies.

Training data (86,250 chats):

• About 28,000 simple Python tasks from nvidia/OpenCodeInstruct (CC BY 4.0): only tasks whose reference solution passes all tests, short prompts and solutions, code block only.
• 403 Python tasks and 603 HTML snippets written locally by Qwen3 8B (an abliterated community variant, Apache 2.0): Python kept only if its own assert tests pass, HTML kept only if every tag is closed. Weighted 3x and 4x.
• In every code example the user message contains "codingPico". For 30 % of the tasks the same request also appears without the word, answered with a short hint to use it.
• 40,000 general chats (TinyDialogues, MIT, generated with GPT-4 by its authors; Everyday Conversations and Smol-SmolTalk, Apache 2.0, generated with Llama 3.1 by their authors).
• 6,000 identity chats (name, maker, size, how to use codingPico).

Limitations

• Only simple, short Python functions and HTML snippets; no classes, no larger programs, no JavaScript, no debugging.
• Often writes plausible but wrong code. Always test what it writes.
• Facts in normal chat are unreliable.
• English only, context of 1,024 tokens. No safety tuning.

License

Apache 2.0.