Spectral Surgery — Code
Collection
Code generation models and evaluations for Spectral Surgery. • 5 items • Updated
Llama-3.1-8B-Instruct + Magicoder + Spectral Surgery
This repository contains the Spectral Surgery adapter obtained from Llama-3.1-8B-Instruct fine-tuned on 50K Magicoder examples.
Base Model
meta-llama/Llama-3.1-8B-Instruct
Training
- Dataset: Magicoder
- Samples: 50K
- Sequence length: 4096
- Global batch size: 32
- Learning rate: 2e-5
- LoRA rank: 16
- LoRA alpha: 32
- Source checkpoint: Epoch 1
Spectral Surgery
- Target: all LoRA modules
- Output rank: 16
- Fast HNS steps: 8
- Stable HNS steps: 2
Evaluation
Greedy decoding with Pass@1.
| Model | HumanEval | MBPP |
|---|---|---|
| Base | 54.27% (89/164) | 65.37% (168/257) |
| LoRA E1 | 56.10% (92/164) | 60.70% (156/257) |
| HNS 4+1 | 58.54% (96/164) | 65.37% (168/257) |
| HNS 8+2 | 60.98% (100/164) | 64.98% (167/257) |
Relative to the base model, HNS 8+2 improves HumanEval by 6.71 percentage points while approximately preserving MBPP performance.
Relative to the vanilla LoRA checkpoint, HNS 8+2 improves HumanEval by 4.88 percentage points and MBPP by 4.28 percentage points.
- Downloads last month
- 15
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for tianzl66/Llama-3.1-8B-Instruct-Magicoder-50K-SpectralSurgery-HNS8p2
Base model
meta-llama/Llama-3.1-8B Finetuned
meta-llama/Llama-3.1-8B-Instruct