Edit model card

πŸ¦™ Llama-3-LlamaPlanner


Model Description

LlamaPlanner is a fine-tuned version of Meta's Llama-8B model which has been specifically designed for generating high-quality plans for code generation tasks. The model was trained on CodeNet-16k, a curated dataset of competitive programming problems, and their corresponding plans generated using Llama-3-70B. By leveraging the power of Parameter Efficient Fine-Tuning (PEFT), LlamaPlanner achieves performance comparable to much larger models in generating effective plans for code generation.

Model Details

  • Base Model: Llama-8B Instruct
  • Fine-Tuning Approach: Parameter Efficient Fine-Tuning (PEFT) using Unsloth
  • Training Data: CodeNet-16k, a filtered and deduplicated dataset of 16,500 competitive programming problems and their plans generated using Llama-3-70B
  • Training Infrastructure: H100-SXM5 GPU
  • Evaluation Benchmarks: HumanEval and EvalPlus

How to Use

To use LlamaPlanner with the Hugging Face Transformers library, follow these steps:

import transformers
import torch

model_id = "verifiers-for-code/Llama-3-LlamaPlanner"

pipeline = transformers.pipeline(
    "text-generation", model=model_id, model_kwargs={"torch_dtype": torch.bfloat16}, device_map="auto"

prompt = "Generate a plan for a program that sorts an array of integers in ascending order."

Training Details

LlamaPlanner was trained using the following steps:

  1. Filtering and preprocessing the CodeNet dataset to create CodeNet-16k
  2. Generating plans for each problem using Llama-3-70B
  3. Formatting the problem description, input description, output description, and samples as input, and the generated plans as output
  4. Performing PEFT on the Llama-8B Instruct base model using Unsloth with different ranks and alpha values
  5. Training on an H100-SXM5 GPU for varying epochs

Evaluation Results

LlamaPlanner was evaluated on the HumanEval and EvalPlus benchmarks using various methods, including zero-shot, self-planning, base planner model, and fine-tuned planner model. The results demonstrated that LlamaPlanner outperforms the base Llama-3-8B model by 14% on HumanEval and 11% on EvalPlus. Additionally, plans generated by LlamaPlanner helped boost the performance of Llama-3-70B on HumanEval.


If you use LlamaPlanner in your research or applications, please cite the model using the following BibTeX entry:

  title={LlamaPlanner: A Fine-Tuned Llama-8B Model for Effective Plan Generation in Code Generation Tasks},
  author={Abhinav Chinta and Sumuk Shashidhar and Vaibhav Sahai},


LlamaPlanner is released under the Apache License 2.0.


We would like to thank Meta for releasing the Llama model family and the open-source community for their contributions to the development of large language models and their applications in code generation tasks.

Downloads last month
Model size
8.03B params
Tensor type

Datasets used to train verifiers-for-code/Llama-3-LlamaPlanner