Error while trying to run run_wwm_mlm.py using my saved model: TypeError: ‘NoneType’ object is not iterable

jungminc88: <p>I have trained a BertForSequenceClassification model, saved the model and tokenizer by</p>
<pre><code class="lang-auto">model.save_pretrained('output_mlm_cls')
tokenizer.save_pretrained('output_mlm_cls')
</code></pre>
<p>I’m trying to run <code>run_mlm_wwm.py</code> and I want it to load the saved model above:</p>
<pre><code class="lang-auto">python run_mlm_wwm.py \
    --model_name_or_path /path/to/output_mlm_cls \
    --train_file /path/to/my_data.txt \
    --do_train \
    --output_dir /output_dir
</code></pre>
<p>Then I get this error message:</p>
<blockquote>
<p>Traceback (most recent call last):<br>
File “run_mlm_wwm.py”, line 408, in <br>
main()<br>
File “run_mlm_wwm.py”, line 367, in main<br>
train_result = trainer.train(resume_from_checkpoint=checkpoint)<br>
File “/home/cl/jungmin-c/.pyenv/versions/anaconda3-5.1.0/envs/jp/lib/python3.7/site-packages/transformers/trainer.py”, line 1066, in train<br>
self._load_state_dict_in_model(state_dict)<br>
File “/home/cl/jungmin-c/.pyenv/versions/anaconda3-5.1.0/envs/jp/lib/python3.7/site-packages/transformers/trainer.py”, line 1387, in _load_state_dict_in_model<br>
if set(load_result.missing_keys) == set(self.model._keys_to_ignore_on_save):<br>
TypeError: ‘NoneType’ object is not iterable</p>
</blockquote>
<p>I could not figure out what is wrong with my approach. I would greatly appreciate your suggestion.<br>
Thank you!</p>

ptrblck: <p>Based on the error message it seems that one of the inputs to <code>set</code> is a <code>None</code> object as seen here:</p>
<pre><code class="lang-python">set(None)
&gt; TypeError: 'NoneType' object is not iterable
</code></pre>
<p>My guess would be that the <code>transformers.trainer</code> object is unable to load the desired file and returns <code>None</code> instead.<br>
CC <a class="mention" href="/u/stas">@stas</a> for viz.</p>

stas: <p>Thanks for the ping, <a class="mention" href="/u/ptrblck">@ptrblck</a>!</p>
<p>I’m not familiar with: <a href="https://github.com/huggingface/transformers/blob/master/examples/research_projects/mlm_wwm/run_mlm_wwm.py" class="inline-onebox" rel="noopener nofollow ugc">transformers/run_mlm_wwm.py at master · huggingface/transformers · GitHub</a> as it’s some experimental example I have never used and it’s not being maintained, so it’s very likely to be out of sync with the current code base. Therefore <a class="mention" href="/u/jungminc88">@jungminc88</a> please file an Issue at <a href="https://github.com/huggingface/transformers/issues" class="inline-onebox" rel="noopener nofollow ugc">Issues · huggingface/transformers · GitHub</a></p>
<p>You probably need to tag the person who wrote this example, see the original PR:</p><aside class="onebox githubpullrequest" data-onebox-src="https://github.com/huggingface/transformers/pull/9887">
  <header class="source">

      <a href="https://github.com/huggingface/transformers/pull/9887" target="_blank" rel="noopener nofollow ugc">github.com/huggingface/transformers</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Pull Request">
    <svg width="60" height="60" class="github-icon" viewbox="0 0 12 16" aria-hidden="true"><path d="M11 11.28V5c-.03-.78-.34-1.47-.94-2.06C9.46 2.35 8.78 2.03 8 2H7V0L4 3l3 3V4h1c.27.02.48.11.69.31.21.2.3.42.31.69v6.28A1.993 1.993 0 0 0 10 15a1.993 1.993 0 0 0 1-3.72zm-1 2.92c-.66 0-1.2-.55-1.2-1.2 0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2 0 .65-.55 1.2-1.2 1.2zM4 3c0-1.11-.89-2-2-2a1.993 1.993 0 0 0-1 3.72v6.56A1.993 1.993 0 0 0 2 15a1.993 1.993 0 0 0 1-3.72V4.72c.59-.34 1-.98 1-1.72zm-.8 10c0 .66-.55 1.2-1.2 1.2-.65 0-1.2-.55-1.2-1.2 0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2zM2 4.2C1.34 4.2.8 3.65.8 3c0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2 0 .65-.55 1.2-1.2 1.2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/huggingface/transformers/pull/9887" target="_blank" rel="noopener nofollow ugc">Fit chinese wwm to new datasets</a>
    </h4>

    <div class="branches">
      <code>huggingface:master</code> ← <code>wlhgtc:master</code>
    </div>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2021-01-29" data-time="10:54:46" data-timezone="UTC">10:54AM - 29 Jan 21 UTC</span>
      </div>

      <div class="user">
        <a href="https://github.com/wlhgtc" target="_blank" rel="noopener nofollow ugc">
          <img alt="wlhgtc" src="https://avatars.githubusercontent.com/u/16603773?v=4" class="onebox-avatar-inline" width="20" height="20">
          wlhgtc
        </a>
      </div>

      <div class="lines" title="6 commits changed 6 files with 249 additions and 67 deletions">
        <a href="https://github.com/huggingface/transformers/pull/9887/files" target="_blank" rel="noopener nofollow ugc">
          <span class="added">+249</span>
          <span class="removed">-67</span>
        </a>
      </div>
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">Sorry for my later update. 
I make my code(especially in chinese mlm_wwm) fit t<span class="show-more-container"><a href="https://github.com/huggingface/transformers/pull/9887" target="_blank" rel="noopener nofollow ugc" class="show-more">…</a></span><span class="excerpt hidden">he newest code.
Here are the changes:
1. add `chinese_ref` key to avoid miss ref inf.
2. fix the type bug in `data_collator.py`
3. re-add `run_chinese_ref.py` cause it could run with the newest version code (4.2.2).
4. update readme</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>