r2d2aparicio/ncm-code-doc

NCM Code Documentation (Next-Concept Modeling)

Fine-tuned checkpoint of Stanford's Next-Concept Modeling (NCM) architecture, adapted for ultra-fast, local code documentation and technical function summarization.

• GitHub Repository: artapaher-max/ncm-code-doc
• Base Architecture: Stanford University / Hazy Research (HazyResearch/ncm)
• Primary Integration: Model Context Protocol (MCP) server for local IDE agents (Antigravity IDE, Claude Desktop, Cursor).

---

Model Architecture & Details

NCM operates in a non-autoregressive (NAR) latent space through a 9-level hierarchical Vector-Quantized Variational Autoencoder (VQ-VAE).

• Non-Autoregressive Generation: Unlike standard autoregressive LLMs (which decode one token at a time), NCM decodes conceptual representations in parallel across 9 quantization depth levels.
• Inference Latency: ~50–100 ms per block on consumer desktop GPUs (e.g., NVIDIA RTX 2060 6GB).
• Target Context Length: Bounded to $\le 48$ tokens. Optimized specifically for single-sentence docstrings, function signatures, and concise technical summaries.
• Prefix Window: Up to 512 raw tokens of source code.

---

Intended Use

This model is intended to act as a lightweight, local writing assistant offloading routine documentation tasks from large frontier LLMs (Claude 3.5 Sonnet, GPT-4, Gemini):

1. Token Cost & Latency Reduction: Avoid spending frontier model tokens on repetitive single-sentence function descriptions.
2. Local MCP Server: Run locally alongside coding assistants via stdio MCP.

---

Quickstart & Usage

1. Download Model Checkpoints

Using Python:

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="r2d2aparicio/ncm-code-doc",
    local_dir="artifacts/ncm_code_doc"
)

Files downloaded:

• generator.pt (~354 MB): NextConceptTransformer weights.
• tokenizer.pt (~299 MB): Hierarchical VQ-VAE tokenizer weights with code-adapted decoder.

2. Run with MCP Server

To use this model as a tool in Claude Desktop or Antigravity IDE, clone the official MCP server:

git clone https://github.com/artapaher-max/ncm-code-doc.git
cd ncm-code-doc
pip install -e .

And configure mcpServers:

{
  "mcpServers": {
    "ncm-fast-writer": {
      "command": "/path/to/ncm-code-doc/.venv/bin/python",
      "args": ["-u", "/path/to/ncm-code-doc/ncm_doc_server.py"],
      "env": { "PYTHONUNBUFFERED": "1" }
    }
  }
}

---

Training Data & Hardware

• Dataset: 5,000 code-docstring pairs sampled from code_search_net across Python, JavaScript, and Web repositories.
• Fine-Tuning Hardware: Single NVIDIA GeForce RTX 2060 (6 GB VRAM GDDR6).
• Memory Consumption: ~2.9 GB VRAM during training (FP16 mixed precision, gradient accumulation, batch size 2).
• Inference Memory: ~2.5 GB VRAM.

---

Citation & Acknowledgments

This model is fine-tuned from the checkpoint provided by Stanford Hazy Research:

@article{ncm2024,
  title={Next-Concept Modeling: Scalable Non-Autoregressive Generation via Hierarchical Latent Space},
  author={Hazy Research, Stanford University},
  journal={arXiv preprint},
  year={2024}
}