KeyError Radam_buffer

diegodalvarez: <p>Trying to copy this code down <a href="https://pytorch-forecasting.readthedocs.io/en/stable/tutorials/deepar.html" rel="noopener nofollow ugc">here</a>. I made a dedicate anaconda environment for all of the packages. It’s this piece of code that is giving me problems.</p>
<pre><code class="lang-auto"># find optimal learning rate
res = trainer.tuner.lr_find(
    net,
    train_dataloaders=train_dataloader,
    val_dataloaders=val_dataloader,
    min_lr=1e-5,
    max_lr=1e01,
    early_stop_threshold=100)
</code></pre>
<p>It keeps giving me this error</p>
<pre><code class="lang-auto">File ~\anaconda3\envs\pytorch_forecasting\lib\site-packages\pytorch_lightning\trainer\connectors\checkpoint_connector.py:234, in CheckpointConnector.restore(self, checkpoint_path)
    231 self.restore_callbacks()
    233 # restore training state
--&gt; 234 self.restore_training_state()
    235 self.resume_end()

File ~\anaconda3\envs\pytorch_forecasting\lib\site-packages\pytorch_lightning\trainer\connectors\checkpoint_connector.py:286, in CheckpointConnector.restore_training_state(self)
    283 assert self.trainer.state.fn is not None
    284 if self.trainer.state.fn == TrainerFn.FITTING:
    285     # restore optimizers and schedulers state
--&gt; 286     self.restore_optimizers_and_schedulers()

File ~\anaconda3\envs\pytorch_forecasting\lib\site-packages\pytorch_lightning\trainer\connectors\checkpoint_connector.py:382, in CheckpointConnector.restore_optimizers_and_schedulers(self)
    377     if "optimizer_states" not in self._loaded_checkpoint:
    378         raise KeyError(
    379             "Trying to restore optimizer state but checkpoint contains only the model."
    380             " This is probably due to `ModelCheckpoint.save_weights_only` being set to `True`."
    381         )
--&gt; 382     self.restore_optimizers()
    384 if "lr_schedulers" not in self._loaded_checkpoint:
    385     raise KeyError(
    386         "Trying to restore learning rate scheduler state but checkpoint contains only the model."
    387         " This is probably due to `ModelCheckpoint.save_weights_only` being set to `True`."
    388     )

File ~\anaconda3\envs\pytorch_forecasting\lib\site-packages\pytorch_lightning\trainer\connectors\checkpoint_connector.py:397, in CheckpointConnector.restore_optimizers(self)
    394     return
    396 # restore the optimizers
--&gt; 397 self.trainer.strategy.load_optimizer_state_dict(self._loaded_checkpoint)

File ~\anaconda3\envs\pytorch_forecasting\lib\site-packages\pytorch_lightning\strategies\strategy.py:368, in Strategy.load_optimizer_state_dict(self, checkpoint)
    366 optimizer_states = checkpoint["optimizer_states"]
    367 for optimizer, opt_state in zip(self.optimizers, optimizer_states):
--&gt; 368     optimizer.load_state_dict(opt_state)
    369     _optimizer_to_device(optimizer, self.root_device)

File ~\anaconda3\envs\pytorch_forecasting\lib\site-packages\torch\optim\optimizer.py:244, in Optimizer.load_state_dict(self, state_dict)
    241     return new_group
    242 param_groups = [
    243     update_group(g, ng) for g, ng in zip(groups, saved_groups)]
--&gt; 244 self.__setstate__({'state': state, 'param_groups': param_groups})

File ~\anaconda3\envs\pytorch_forecasting\lib\site-packages\pytorch_forecasting\optim.py:133, in Ranger.__setstate__(self, state)
    131 def __setstate__(self, state: dict) -&gt; None:
    132     super().__setstate__(state)
--&gt; 133     self.radam_buffer = state["radam_buffer"]
    134     self.alpha = state["alpha"]
    135     self.k = state["k"]

KeyError: 'radam_buffer'
</code></pre>

ptrblck: <p>Based on the error message it seems the <code>pytorch_forcasting</code> optimizer expects to load a <code>radam_buffer</code> from the <code>state_dict</code> while the <code>state_dict</code> you are loading does not contain this key.<br>
I don’t know what <code>find_lr</code> exactly does but do you know if it’s trying to store and load <code>state_dict</code>s internally, which might cause the error?</p>

Tadashi_Mori: <p>I’m having the same problem but with this tutorial here:<br>
<a href="https://pytorch-forecasting.readthedocs.io/en/stable/tutorials/stallion.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch-forecasting.readthedocs.io/en/stable/tutorials/stallion.html</a></p>
<p>I installed a brand new environment, installed the packages with conda</p>
<p><code>conda install pytorch-forecasting pytorch&gt;=1.7 -c pytorch -c conda-forge</code></p>
<p>and I get the exact same error when running:</p>
<pre><code class="lang-auto">res = trainer.tuner.lr_find(
    tft,
    train_dataloaders=train_dataloader,
    val_dataloaders=val_dataloader,
    max_lr=10.0,
    min_lr=1e-6,
)
</code></pre>
<p>Edit: Finally solved this problem. Just add optimizer=‘adam’ in your TFT and the problem goes away.</p>

Faiga_Alawad: <p>This fixed it for me <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>