IRx-1

A ~2B parameter chat model built for fast, private, offline use on personal devices. Runs entirely on-device — no internet connection required — and is small enough to fit comfortably on a phone or laptop.

MLX format (this repo, for MLX runtimes/Apple Silicon/iOS) or GGUF (for llama.cpp-based runtimes — LM Studio, mobile apps via llama.rn, Ollama import). The GGUF build required fixing two silent bugs in the standard MLX→GGUF export path that produce a file which loads without error but generates garbage — see that repo's model card for details.

Strengths

• Fast and lightweight — small enough to run offline on a phone or laptop, with no server round-trip
• Private by design — nothing you type ever leaves the device
• Personalized style — fine-tuned on real conversational data, so it follows instructions and formats answers in a natural, direct way rather than generic boilerplate
• Broad everyday usefulness — Q&A, writing help, planning, explanations, brainstorming, classification, summarization
• Structured tool-use — can parse free-form natural language into structured commands for a real integrated developer tool (see below)
• Improved general-knowledge accuracy — fine-tuned on 88 spot-checked geography/science/history Q&A examples generated by a larger local teacher model; most common factual questions (capitals, chemical symbols, historical facts) now answer correctly. See Limitations for what this does and doesn't fix.
• Current-events grounding available via retrieval — the reference chat client in the GitHub repo includes an optional RAG pipeline (scripts/fetch_news.py + scripts/news_context.py): a local index of Indian RSS feeds, refreshed on a schedule, searched at answer-time and handed to the model as context. This is retrieval, not retraining — the weights here are unchanged by it, and it's opt-in (only active if you run the fetch script and use chat.py). Verified grounding a real answer in an actual current article rather than hallucinating from frozen training-time knowledge.

Limitations

• Not a frontier-scale model. At ~2B parameters, it won't match large hosted models on hard multi-step reasoning, deep technical/coding problems, or breadth of world knowledge — that gap is a function of scale, not something fine-tuning erases
• Not current-events aware on its own. The model's weights are fixed as of training — by itself it has no access to recent news, prices, or events. Fine-tuning more often on news doesn't fix this reliably (news is dense with exactly the kind of fast-changing, precise facts fine-tuning is worst at — see the next point); the fix that's actually reliable is retrieval, not retraining, which the reference repo provides as an opt-in add-on (see Strengths) — the base model file here has no built-in awareness of anything past its training data.
• Fine-tuning improves but doesn't guarantee facts. Tested directly: adding accurate, spot-checked training examples measurably improved general-knowledge accuracy (most capitals, chemical symbols, and historical facts now answer correctly), but individual facts can still be wrong — especially where the base model already had a strong competing belief from pretraining (e.g. it still answers "Sahara" for the largest desert despite a correct "Antarctica" training example, and gives an imprecise population figure for India despite an accurate one in training). This is a real, partial improvement, not a fix — treat factual claims as worth double-checking, same as any small model.
• Don't enable native tool/function-calling in chat apps. The base architecture supports a native <tool_call> function-calling format, but IRx-1's fine-tuning never trained on or reinforced it. If a host app (LM Studio, Bionic, or similar) exposes tools/functions to the model, it can misfire the wrong tool for a plain question, or spiral into an unterminated loop hallucinating a fake multi-step agentic session — especially in apps with a large always-on tool/skill system prompt, which can overwhelm a model this size regardless of the question asked. Plain chat (no tools exposed) is reliable.
• Occasional off answers at higher sampling temperatures — like any small model, it can occasionally produce a repetitive or degenerate response; regenerating usually resolves it
• Best suited to short-to-medium context — very long, multi-document context windows aren't its strong suit

How it was built

Full technical detail, including the exact scripts and the story behind each data source, is in the GitHub repo.

Example queries

"Who created you?"

I was created by Ramesh Inampudi from Hyderabad, India. He built me, IRx-1, at iramesh.com.

"What questions should I ask before signing an apartment lease?"

1. What is the total rent, including security deposit and utilities?
2. What are the rent payment terms, such as due dates and early termination fees?
3. What specific appliances and furniture are included in the lease?
4. Are there any pet fees or restrictions on pets?
5. What are the maintenance responsibilities for both the landlord and tenant?
6. What are the rules regarding subletting or assigning the lease to a new tenant? ...

"What is the chemical symbol for gold?" / "Who painted the Mona Lisa?"

Au

Leonardo da Vinci

"Explain the difference between TCP and UDP in a couple of sentences."

TCP is a connection-oriented protocol that ensures reliable delivery by establishing a handshake, managing resources, and retransmitting lost packets, while UDP is a connectionless protocol that prioritizes speed over reliability, making it ideal for real-time applications like streaming and VoIP.

"Write a short, encouraging note to leave for a roommate who is stressed about exams."

Hey, I know your brain is working overtime! Just remember to breathe, and we'll tackle these problems one at a time. You've got this, and I'm right here cheering you on.

(Real, unedited outputs. Small models vary run to run — regenerate if a particular answer misses.)

Tool-use example: xGAIR

xGAIR is an MCP server that plugs AI coding assistants into any GitHub repo. Its chat CLI matched only exact command syntax; IRx-1 serves as an optional natural-language fallback, parsing free-form input into the correct structured tool call. Real, verified outputs, repo names not seen verbatim in training:

> hook up github.com/vercel/next.js
  → xgair_connect_repo { url: "github.com/vercel/next.js", repo: "next.js" }

> run discovery on stripe/stripe-node
  → xgair_discover_repo { repoId: "stripe/stripe-node" }

> check this snippet: DROP TABLE users;
  → xgair_validate { repoId: "", codeSnippet: "DROP TABLE users;" }

Known limitation: when a repo reference is embedded mid-sentence rather than at the start of the message, the extracted repoId sometimes comes back empty even though the tool choice itself is correct. Occasionally an off-topic message gets mapped to a tool call instead of {"tool": "unknown"}. Neither is catastrophic by design — the calling integration falls back to a current-repo context when repoId is empty, and a wrongly-triggered call is a harmless read, not a destructive action.

Current-events awareness: retrieval, not retraining

The weights above never change to add current-events knowledge — fine-tuning doesn't reliably teach new facts (see Limitations), and news is the worst case for that. The GitHub repo instead ships an opt-in RAG pipeline:

Verified: asked about government news, it answered grounded in an actual retrieved article rather than hallucinating from frozen training-time memory.

Usage

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("<repo-id>")

SYSTEM_PROMPT = (
    "Respond directly with only your final answer. Do not show your reasoning, "
    "planning, drafts, or a step-by-step thinking process. "
    "Your name is IRx-1. If asked who you are, what you are, who created/made/built "
    "you, who your developer or author is, or anything about the identity or "
    "background of this model, always answer in your own words that you are IRx-1, "
    "created by Ramesh Inampudi from Hyderabad, India, and point to his website "
    "iramesh.com. Never mention any other AI company or base model name."
)

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": "How do I convert Celsius to Fahrenheit?"},
]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=False, enable_thinking=False
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=200))

Notes on getting clean output: the system prompt above matters — without it, the model can surface visible "thinking" narration instead of a direct answer. Also pass enable_thinking=False to apply_chat_template, and sample at a non-zero temperature (e.g. temp=0.7 via mlx_lm.sample_utils.make_sampler) — greedy decoding (temp=0) is prone to repetition loops on a model this size. Do not expose tools/functions to this model in host apps that support function-calling — see Limitations.

Changelog

2026-09-08

• Model cards cleaned up: no base-model linkage, no license link naming anything specific — generic Apache 2.0 declaration only
• Added a creator-identity system prompt instruction, tested across phrasings
• Added the news RAG pipeline above — retrieval, not retraining; weights untouched
• Fixed GGUF export: two silent bugs in the MLX→GGUF conversion path (a conv1d weight axis-order mismatch, an RMSNorm weight offset convention mismatch) produced a file that loaded without error but generated complete garbage. Fixed and verified; working GGUF published to a separate repo.

2026-09-07

• Added a general-knowledge fine-tuning round (88 spot-checked geography/science/history examples, distilled from a larger local teacher) — verified genuine but partial accuracy improvement, documented honestly including facts that stayed wrong (see Limitations)
• Documented the tool/function-calling limitation
• Rebalanced the xGAIR intent-parsing training data (32 → 69 examples) after finding an earlier small "general knowledge about xGAIR" set was unreliable and caused hallucination; dropped it, kept the structured intent-parsing task that the data scale actually supports
• Initial release: personal-history + general-QA + distillation training pipeline, first published model card

License

Apache 2.0. IRx-1 is a derivative fine-tuned model — full Apache 2.0 terms apply as with any work under this license.