LiamCarter/blockpruner_llama2-7b_ratio0.5_original_nonstrict

Llama-2-7B BlockPruner — original ratio0.5 checkpoint

Important — non-strict parameter budget. This checkpoint follows the original BlockPruner block-count setting. The historical ratio0.5 label does not enforce an exact 50% end-to-end parameter reduction.

重要说明：参数量未被严格约束。 这是原始 BlockPruner 按 MHA/MLP 子块数量进行剪枝的 checkpoint。历史名称 ratio0.5 不代表全模型参数量被严格减少 50%。

Checkpoint summary

This is a research checkpoint derived from meta-llama/Llama-2-7b-hf with the original BlockPruner mix search setting. Each of the 32 Transformer layers is split into one MHA residual sub-block and one MLP residual sub-block, giving 64 candidate sub-blocks. The final deletion set is stored in blockpruner_config.json and in config.json.blockpruner.

Item	Value
Historical folder label	ratio0.5
Removed sub-blocks	32 / 64 (17 MHA + 15 MLP)
Sub-block-count reduction	50.00%
Dense base parameters	6,738,415,616
Remaining checkpoint parameters	3,568,439,296
Actual total-parameter reduction	47.043348179%
Weight dtype	BF16

MHA and MLP sub-blocks have different parameter counts, so a block-count target does not imply an equal total-parameter target. This checkpoint must not be described as a strict 50%-parameter model or used as a parameter-matched 50% baseline without additional budget matching.

Loading requirement

This checkpoint is not a drop-in dense Llama checkpoint. Its weight shards physically omit tensors belonging to pruned sub-blocks, while the base config is still a standard LlamaForCausalLM config. Calling AutoModelForCausalLM.from_pretrained(...) by itself can recreate the missing sub-blocks with newly initialized weights and does not correctly restore the pruned model.

Use a BlockPruner-aware loader in this order:

1. Read blockpruner_config.json (the authoritative final deletion set).
2. Instantiate the Llama-2 structure and replace the listed MHA/MLP sub-blocks with MaskedLlamaDecoderLayer.
3. Load the sharded checkpoint into that masked structure and require an exact key match.

The original implementation is available in the BlockPruner repository. This artifact was exported with transformers==4.39.3; the inference widget is disabled because a mask-aware loader is required.

Pruning provenance

• Base model: meta-llama/Llama-2-7b-hf
• Base revision: 01c7f73d771dfac7d292323805ebc428287df4f9
• Search type: mix (MHA and MLP residual sub-blocks)
• Calibration dataset: Alpaca
• Calibration samples recorded by the search artifact name: 64
• Final deleted sub-block count: 32

The ppl_search_file value in the config records the historical search artifact name. It is not required for loading because the final deletion set is embedded directly in the checkpoint config.

Intended use and limitations

This model is intended for research on structured pruning and in-context learning. No evaluation claim is included in this repository. Users should independently validate task performance and should not compare it to strictly parameter-matched pruning methods as if the nominal ratio were an exact parameter budget.

This is an independently exported research artifact, not an official checkpoint released by the BlockPruner authors. Use is subject to the Meta Llama 2 license and the terms of the base model.

Citation

@article{zhong2024blockpruner,
  title   = {BlockPruner: Fine-grained Pruning for Large Language Models},
  author  = {Zhong, Longguang and Wan, Fanqi and Chen, Ruijun and Quan, Xiaojun and Li, Liangzhi},
  journal = {arXiv preprint arXiv:2406.10594},
  year    = {2024}
}