Torchtune Llama3-8b unexpected key: "tok_embeddings.weight"

we7313: <p>I have successfully fine-tuned Llama-2-7b with torch tune.<br>
I then test it with:</p>
<p>!tune run generate --config trained-model.yaml <br>
prompt=“what is TBS?”<br>
I get a response that reflects my training progress.</p>
<p>Today I was looking to follow that pattern and train Llama-3-8b.<br>
The training went fine.  It outputted the Adapter.pt and meta-model.pt files.<br>
When I tried to test with the same:<br>
!tune run generate --config trained-model.yaml <br>
prompt=“what is TBS?”</p>
<p>I get the following error:<br>
Exception: Error converting the state dict. Found unexpected key: “tok_embeddings.weight”. Please make sure you’re loading a checkpoint with the right format.</p>
<p>Any ideas would be great.</p>
<p>Below is my training config:<br>
model:<br>
<em>component</em>: torchtune.models.llama3.lora_llama3_8b<br>
lora_attn_modules: [‘q_proj’, ‘v_proj’]<br>
apply_lora_to_mlp: False<br>
apply_lora_to_output: False<br>
lora_rank: 8<br>
lora_alpha: 16</p>
<h1><a name="tokenizer-1" class="anchor" href="#tokenizer-1"></a>Tokenizer</h1>
<p>tokenizer:<br>
<em>component</em>: torchtune.models.llama3.llama3_tokenizer<br>
path: /root/Pretrained_base_models/Llama-3-8B/original/tokenizer.model</p>
<p>checkpointer:<br>
<em>component</em>: torchtune.utils.FullModelMetaCheckpointer<br>
checkpoint_dir: /root/Pretrained_base_models/Llama-3-8B/original/<br>
checkpoint_files: [<br>
consolidated.00.pth<br>
]<br>
recipe_checkpoint: null<br>
output_dir: /root/Trained_models/Llama-3-8b/output<br>
model_type: LLAMA3<br>
resume_from_checkpoint: False</p>
<h1><a name="dataset-and-sampler-2" class="anchor" href="#dataset-and-sampler-2"></a>Dataset and Sampler</h1>
<p>dataset:<br>
<em>component</em>: torchtune.datasets.alpaca_cleaned_dataset<br>
train_on_input: True<br>
seed: null<br>
shuffle: True<br>
batch_size: 2</p>
<h1><a name="optimizer-and-scheduler-3" class="anchor" href="#optimizer-and-scheduler-3"></a>Optimizer and Scheduler</h1>
<p>optimizer:<br>
<em>component</em>: torch.optim.AdamW<br>
weight_decay: 0.01<br>
lr: 3e-4<br>
lr_scheduler:<br>
<em>component</em>: torchtune.modules.get_cosine_schedule_with_warmup<br>
num_warmup_steps: 0</p>
<p>loss:<br>
<em>component</em>: torch.nn.CrossEntropyLoss</p>
<h1><a name="training-4" class="anchor" href="#training-4"></a>Training</h1>
<p>epochs: 1<br>
max_steps_per_epoch: null<br>
gradient_accumulation_steps: 1<br>
compile: False</p>
<h1><a name="logging-5" class="anchor" href="#logging-5"></a>Logging</h1>
<p>output_dir: /root/Trained_models/Llama-3-8b/finetune_output<br>
metric_logger:<br>
<em>component</em>: torchtune.utils.metric_logging.DiskLogger<br>
log_dir: ${output_dir}<br>
log_every_n_steps: null</p>
<h1><a name="environment-6" class="anchor" href="#environment-6"></a>Environment</h1>
<p>device: cuda<br>
dtype: bf16<br>
enable_activation_checkpointing: True</p>
<h1><a name="profiler-disabled-7" class="anchor" href="#profiler-disabled-7"></a>Profiler (disabled)</h1>
<p>profiler:<br>
<em>component</em>: torchtune.utils.profiler<br>
enabled: False</p>

sciarrilli: <p>I was getting the same error.</p>
<p>I figured out for the run generate or run eval you need to change your config.yaml to look like this for the model arguments:</p>
<p>specifically   <strong><em>component</em>: torchtune.utils.FullModelMetaCheckpointer</strong></p>
<pre><code class="lang-auto"># Model Arguments
model:
  _component_: torchtune.models.llama3.llama3_8b

checkpointer:
  _component_: torchtune.utils.FullModelMetaCheckpointer
  checkpoint_dir: ./models/llama-3-8b-instruct/original/
  checkpoint_files: [
    meta_model_0.pt,
  ]
  recipe_checkpoint: null
  output_dir: ./models/llama-3-8b-instruct/eval/
  model_type: LLAMA3
</code></pre>