soyuj/llama3.2-1b-bidirectional-mntp-msmarco

Bidirectional MNTP Adapter For Llama 3.2 1B

This directory contains the LoRA adapter for a bidirectional MNTP-enabled meta-llama/Llama-3.2-1B. We have performed 10,000 pre-training steps on MS MARCO.

It is intended to be loaded in this repository: https://github.com/basnetsoyuj/llama3.2-1b-sae-bidirectional-mntp-msmarco

python bidirectional_mntp.py \
  --adapter-path soyuj/llama3.2-1b-bidirectional-mntp-msmarco \
  --text "hello world"

Files

• adapter_config.json: PEFT adapter config
• adapter_model.safetensors: LoRA adapter weights

Notes

• This adapter does not include the base Meta Llama weights.
• You must have access to meta-llama/Llama-3.2-1B to load it.
• The bidirectional model runtime used to apply this adapter is adapted from HansiZeng/scaling-retriever.
• Published adapter repo: soyuj/llama3.2-1b-bidirectional-mntp-msmarco.