Text Generation
Transformers
Safetensors
qwen3
conversational
text-generation-inference

🚀 PIPer: On-Device Environment Setup via Online Reinforcement Learning

Models Dataset License

Democratizing environment setup with on-device sized models that match the performance of much larger proprietary systems

🎯 Overview

Environment setup—the process of configuring systems to work with specific software projects—remains a persistent challenge in software engineering. PIPer addresses this by training specialized on-device models that can automatically generate correct Bash scripts for environment configuration.

Our approach combines:

  • 📚 Supervised Fine-Tuning (SFT) with executable scripts from larger models
  • 🎯 Reinforcement Learning with Verifiable Rewards (RLVR) using lightweight proxy LLM-reward

🏆 Key Results

Model Size EnvBench avg@5 Cost per 1M tokens
PIPer 8B 19.4 $0.60
GPT-4o - 19.4 $15.00
Qwen3-32B 32B 16.2 $2.00
Qwen3-8B 8B 2.6 $0.60

🎉 PIPer achieves 9× improvement over its base model while matching GPT-4o performance at 25x lower cost

Performance vs Cost Analysis

📦 Available Artifacts

🤖 Model Checkpoints

Model Description HuggingFace Link
🏅 PIPer (Full) Complete SFT+RL trained model PIPer-icml/PIPer-8B
🎯 PIPer (RL-only) RLVR checkpoint only PIPer-icml/PIPer-8B-RL-only
📚 PIPer (SFT-only) Supervised fine-tuning only PIPer-icml/PIPer-8B-SFT-only

📊 Datasets

Dataset Description HuggingFace Link
EnvBench Zero-shot RL Training prompts and evaluation data PIPer-icml/envbench-zeroshot-rl

📊 Evaluation Benchmarks

Benchmark Description Metric Our Result
EnvBench-Python 329 Python repositories pass@5 🏆 27/329
Repo2Run 420 Python repositories pass@5 🏆 103/420
Terminal-Bench 80 terminal tasks pass@10 4/80

🙏 Acknowledgments

  • Built on top of EnvBench evaluation framework
  • Uses VeRL for efficient RL training
  • Leverages Qwen3 model family as base architecture

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

Downloads last month
3
Safetensors
Model size
8B params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for PIPer-icml/PIPer-8B

Finetuned
(3)
this model

Datasets used to train PIPer-icml/PIPer-8B