Credits and Acknowledgments

TURBOPASTA is built upon the excellent work of Fast Apply by Kortix AI. Our model leverages their dataset and builds on their pioneering approach to code merging and transformation. Key inspirations include:

Dataset structure and generation methodology
XML-based prompt engineering approach
Evaluation metrics and benchmarking approaches

Special thanks to:

The Kortix AI team for open-sourcing Fast Apply
Their foundational work on high-speed code transformation models
The comprehensive dataset they've made available to the community

While TURBOPASTA introduces its own innovations, the groundwork laid by Fast Apply was instrumental in making this project possible. We encourage users interested in code transformation models to also check out the original Fast Apply models:

This project is licensed under Apache-2.0, consistent with Fast Apply's open-source ethos.

Based on a dataset inspired by https://www.kortix.ai/

TURBOPASTA LoRA Adapter for Qwen2.5-3B

A LoRA adapter for unsloth/Qwen2.5-3B that merges code updates using chain-of-thought reasoning and maintains strict adherence to original code structure and formatting.

Technical Specifications

Base Model

Model: unsloth/Qwen2.5-3B
LoRA Rank: 64
Target Modules: v_proj, o_proj, down_proj, up_proj, q_proj, k_proj, gate_proj
Task: CAUSAL_LM
Dropout: 0
Alpha: 32

Input/Output Format

Input XML structure:

<instruction>You are a coding assistant that helps merge code updates, ensuring every modification is fully integrated. Merge all changes from the snippet into the code. Preserve the code's structure, order, comments, and indentation exactly.</instruction>

<fastapply>
  <code>
    {original_code}
  </code>
  <update>
    {update_snippet}
  </update>
  <finalcode>
    {merged_result}
  </finalcode>
</fastapply>

The model supports multiple <fastapply> blocks for few-shot context learning. Use your stop token as </fastapply>.

Deployment

VLLM Server Setup

export VLLM_ALLOW_RUNTIME_LORA_UPDATING=1
export VLLM_ALLOW_LONG_MAX_MODEL_LEN=1

vllm serve unsloth/qwen2.5-3b \
  --gpu-memory-utilization=1 \
  --port 6002 \
  --served-model-name="turbopasta" \
  --trust-remote-code \
  --max-model-len 8192 \
  --disable-log-requests \
  --enable-lora \
  --lora-modules lora=./dataset/output/turbopasta/lora_model \
  --max-lora-rank 64

Client Implementation

import requests

def merge_code(original_code: str, update_snippet: str, vllm_url: str = "http://localhost:6002/v1/completions") -> dict:
    xml_content = (
        '<instruction>You are a coding assistant that helps merge code updates, ensuring every modification is fully '
        'integrated. Merge all changes from the snippet into the code. Preserve the code\'s structure, order, comments, '
        'and indentation exactly.</instruction>\n'
        '<fastapply>\n'
        '  <code>\n'
        f'{original_code}\n'
        '  </code>\n'
        '  <update>\n'
        f'{update_snippet}\n'
        '  </update>'
    )
    
    response = requests.post(
        vllm_url,
        json={
            "prompt": xml_content,
            "max_tokens": 6000,
            "temperature": 0.1,
            "model": "lora",
            "stop": ["</fastapply>"]
        },
        timeout=30000
    )
    
    completion = response.json()["choices"][0]["text"]
    
    # Parse XML tags
    import re
    def extract_tag(tag: str) -> str:
        match = re.search(f'<{tag}>(.*?)</{tag}>', completion, re.DOTALL)
        return match.group(1).strip() if match else ""
        
    return {
        "merged_code": extract_tag("finalcode")
    }

Batch Processing

The model works with the included data processor for parallel processing of code updates:

from request_processor import RequestProcessor

processor = RequestProcessor(
    input_file="updates.jsonl",
    output_file="merged.jsonl",
    num_threads=24
)
processor.process_file()

Input JSONL format:

{
    "id": "update_id",
    "original_code": "...",
    "update_snippet": "...",
    "file_path": "path/to/file"
}

Output JSONL format:

{
    "id": "update_id",
    "original_code": "...",
    "update_snippet": "...",
    "merged_code": "...",
    "file_path": "path/to/file",
    "processed_at": "2024-10-24 02:52:33"
}

Implementation and Performance Considerations

Uses thread pooling for parallel processing
Atomic writes with file locking
Progress tracking with tqdm
Automatic error handling and logging
Configurable thread count for optimization
Temperature set to 0.1 for consistent merges

Error Handling

Errors are captured in the output JSONL:

{
    "error": "error message",
    "processed_at": "timestamp"
}

Monitor errors in real-time:

tail -f merged.jsonl | grep error

Model Training Details

This model was trained using Force Multiplier's autotuning pipeline with the following key characteristics:

Base Model: unsloth/Qwen2.5-3B
Training Type: Few-shot learning with chain-of-thought reasoning
Special Focus: Code structure preservation and merge accuracy
LoRA Configuration: Optimized for code understanding and generation

Limitations

Maximum context length of 8192 tokens
Best suited for single-file code changes
May require multiple passes for complex refactoring
Not recommended for binary file merges