Size mismatch of embedding weights while loading

username: seanbenhur
cooked: <p>Hi I have saved my vocab file and my LSTM model when I try to load my load LSTM model and vocab dict I get a size mismatch error similar to  <a href="https://discuss.pytorch.org/t/how-to-fix-size-mismatch-for-embeddings-weight/50324">this</a><br>
Can ayone help!?</p>
<pre><code class="lang-auto">RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-41-cc9010adf021&gt; in &lt;module&gt;()
      4 #from flask_ngrok import run_with_ngrok
      5 import pickle
----&gt; 6 from predict import *
      7 import threading
      8 

1 frames
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in load_state_dict(self, state_dict, strict)
   1050         if len(error_msgs) &gt; 0:
   1051             raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
-&gt; 1052                                self.__class__.__name__, "\n\t".join(error_msgs)))
   1053         return _IncompatibleKeys(missing_keys, unexpected_keys)
   1054 

RuntimeError: Error(s) in loading state_dict for LSTM:
	size mismatch for embedding.weight: copying a param with shape torch.Size([10654, 100]) from checkpoint, the shape in current model is torch.Size([25002, 100]).
</code></pre>

username: ptrblck
cooked: <p>Did you manipulate this parameter after storing the <code>state_dict</code>?<br>
If not, could you post a minimal code snippet to reproduce this issue, so that we could debug it?</p>

username: adissa
cooked: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a></p>
<p>I’m facing with a similar problem. In my case, it’s after storing<code> state_dict.</code></p>
<p>Any solution ?</p>

username: ptrblck
cooked: <p>Feel free to answer the same questions I’ve asked in my previous post.</p>

username: adissa
cooked: <p>Yes, the parameter is manipulated after storing <code>state_dict</code>.</p>
<pre><code class="lang-auto">TXT = load_field('vocab.field')
checkpoint = torch.load(PREDICT_DOMAIN_MODEL_PATH, map_location = lambda storage, log: storage)
model = ClassifModel(TXT, checkpoint['params'])
model.load_state_dict(checkpoint['state_dict'])
</code></pre>
<p>I note that the TXT vocab was saved after the training of the model. That may be the problem.</p>

username: ptrblck
cooked: <p>Yes, this could be an issue assuming <code>TXT</code> defines some parameter shapes (e.g. based on its vocab), as it could later raise the shape mismatch (e.g. if the vocab is now smaller).</p>

username: Siddharth_S
cooked: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a><br>
Even I’m facing the same kind of issue while inferencing</p>
<p><code>RuntimeError: Error(s) in loading state_dict for LlamaForCausalLM:         size mismatch for model.embed_tokens.weight: copying a param with shape torch.Size([128258, 4096]) from checkpoint, the shape in current model is torch.Size([128256, 4096]).         size mismatch for lm_head.weight: copying a param with shape torch.Size([128258, 4096]) from checkpoint, the shape in current model is torch.Size([128256, 4096]).</code></p>
<p>I added vocabs from from peft_model model tokenizer to base model tokenizer, I’m still getting this error while loading peft_model<br>
<strong>For Reference:</strong></p>
<pre><code class="lang-auto">tokenizer1 = AutoTokenizer.from_pretrained(base_model_id)
tokenizer2 = AutoTokenizer.from_pretrained(model_id)
print(len(tokenizer1))
 
tokenizer_added = set(tokenizer2.vocab).difference(tokenizer1.vocab)
tokenizer1.add_tokens(list(tokenizer_added))
print(len(tokenizer1))
 
 

base_model =AutoModelForCausalLM.from_pretrained(base_model_id,ignore_mismatched_sizes=True)
base_model. resize_token_embeddings(len(tokenizer1))
print(len(tokenizer2))
model= AutoModelForCausalLM.from_pretrained(model_id,device_map="auto")  #getting error here
print("fined model loaded")
print(len(tokenizer1))
 
prompt = """
&lt;|system|&gt; you are a medical coding expert, who have access to medical coding guidelines. extract all the medical conditions from all the sections including clinical information/condition and its associated ICD-10 descriptions(description only without code) with anatomical locations for the given clinical text of all the sections, exclude negated conditions from the given radiology record and return only the ICD-10 descriptions(description only without code) of non-negated conditions in this json format{"description":[]}, except json do not send anything else.&lt;/s&gt;
&lt;|user|&gt;im siddharth, tested hyyertension&lt;/s&gt;
&lt;|assistant|&gt;
"""
pipe = pipeline(task = 'text-generation', model=model,tokenizer=tokenizer1)
output = pipe(prompt)
print(output[0]['generated_text'])
</code></pre>