AttributeError: 'Adam' object has no attribute 'step'

Haneen_Alahmadi: <p>Can anyone assist me in solving this error AttributeError: ‘Adam’ object has no attribute ‘step’. When Bert Model is trained with NER, this error occurs. I could not solve this problem, perhaps due to the change in libraries, since it mainly stems from the update parameters step().</p>
<p>AttributeError                            Traceback (most recent call last)<br>
 in <br>
30         torch.nn.utils.clip_grad_norm_(parameters=model.parameters(), max_norm=max_grad_norm)<br>
31         # update parameters<br>
—&gt; 32         optimizer.step()<br>
33         model.zero_grad()<br>
34     # print train loss per epoch</p>
<p>1 frames<br>
/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py in <strong>getattribute</strong>(self, name)<br>
864     “”“Overridden to support hyperparameter access.”“”<br>
865     try:<br>
 → 866       return super(OptimizerV2, self).<strong>getattribute</strong>(name)<br>
867     except AttributeError as e:<br>
868       # Needed to avoid infinite recursion with <strong>setattr</strong>.</p>
<p>AttributeError: ‘Adam’ object has no attribute ‘step’</p>
<p>The codes are below:</p>
<p>from transformers import AdamW<br>
from transformers import get_linear_schedule_with_warmup<br>
from transformers import BertTokenizer, BertForSequenceClassification<br>
from transformers import DataCollatorForTokenClassification<br>
from transformers import AutoModelForTokenClassification</p>
<p>epochs = 5<br>
max_grad_norm = 1.0</p>
<p>for _ in trange(epochs, desc=“Epoch”):<br>
# TRAIN loop<br>
model.train()<br>
tr_loss = 0<br>
nb_tr_examples, nb_tr_steps = 0, 0<br>
for step, batch in enumerate(train_dataloader):<br>
# add batch to gpu<br>
batch = tuple(t.to(device) for t in batch)<br>
b_input_ids, b_input_mask, b_labels = batch<br>
# forward pass<br>
token_classifier_output = model(b_input_ids,<br>
token_type_ids=None,<br>
attention_mask=b_input_mask,<br>
labels=b_labels)<br>
token_classifier_output.loss.backward()<br>
# track train loss<br>
tr_loss += token_classifier_output.loss.item()<br>
nb_tr_examples += b_input_ids.size(0)<br>
nb_tr_steps += 1<br>
# gradient clipping<br>
torch.nn.utils.clip_grad_norm_(parameters=model.parameters(), max_norm=max_grad_norm)<br>
# update parameters<br>
optimizer.step()<br>
model.zero_grad()<br>
# print train loss per epoch<br>
print(“Train loss: {}”.format(tr_loss/nb_tr_steps))<br>
# VALIDATION on validation set<br>
model.eval()<br>
eval_loss, eval_accuracy = 0, 0<br>
nb_eval_steps, nb_eval_examples = 0, 0<br>
predictions , true_labels = [], []<br>
for batch in valid_dataloader:<br>
batch = tuple(t.to(device) for t in batch)<br>
b_input_ids, b_input_mask, b_labels = batch</p>
<pre><code>    with torch.no_grad():
         tmp_eval_loss = model(b_input_ids,
                            token_type_ids=None,
                            attention_mask=b_input_mask,
                            labels=b_labels)
         logits = model(b_input_ids, token_type_ids=None,
                       attention_mask=b_input_mask)
    logits = logits.detach().cpu().numpy()
    label_ids = b_labels.to('cpu').numpy()
    predictions.extend([list(p) for p in np.argmax(logits, axis=2)])
    true_labels.append(label_ids)
    
    tmp_eval_accuracy = flat_accuracy(logits, label_ids)
    
    eval_loss += tmp_eval_loss.mean().item()
    eval_accuracy += tmp_eval_accuracy
    
    nb_eval_examples += b_input_ids.size(0)
    nb_eval_steps += 1
eval_loss = eval_loss/nb_eval_steps
print("Validation loss: {}".format(eval_loss))
print("Validation Accuracy: {}".format(eval_accuracy/nb_eval_steps))
pred_tags = [tags_vals[p_i] for p in predictions for p_i in p]
valid_tags = [tags_vals[l_ii] for l in true_labels for l_i in l for l_ii in l_i]
print("F1-Score: {}".format(f1_score(pred_tags, valid_tags)))
</code></pre>

J_Johnson: <p>Where have you defined the optimizer?</p>
<p>I.e.:</p>
<pre><code class="lang-auto">optimizer=Adam(model.parameters(), lr=lr, ...)

</code></pre>

Haneen_Alahmadi: <p>Thank you for your response. The optimizer is defined here:</p>
<p>FULL_FINETUNING = True<br>
if FULL_FINETUNING:<br>
param_optimizer = list(model.named_parameters())<br>
no_decay = [‘bias’, ‘gamma’, ‘beta’]<br>
optimizer_grouped_parameters = [<br>
{‘params’: [p for n, p in param_optimizer if not any(nd in n for nd in no_decay)],<br>
‘weight_decay_rate’: 0.01},<br>
{‘params’: [p for n, p in param_optimizer if any(nd in n for nd in no_decay)],<br>
‘weight_decay_rate’: 0.0}<br>
]<br>
else:<br>
param_optimizer = list(model.classifier.named_parameters())<br>
optimizer_grouped_parameters = [{“params”: [p for n, p in param_optimizer]}]<br>
optimizer = Adam(optimizer_grouped_parameters, lr=3e-5)</p>

ptrblck: <p>It seems you are trying to use a <code>Keras</code> optimizer in PyTorch, which won’t work:</p>
<pre><code class="lang-python">/usr/local/lib/python3.8/dist-packages/keras/optimizers/optimizer_v2/optimizer_v2.py in getattribute(self, name)
864 “”“Overridden to support hyperparameter access.”“”
865 try:
→ 866 return super(OptimizerV2, self).getattribute(name)
867 except AttributeError as e:
868 # Needed to avoid infinite recursion with setattr.

AttributeError: ‘Adam’ object has no attribute ‘step’
</code></pre>
<p>(see the file path)</p>
<p>Use <code>torch.optim.Adam</code> and it should work.</p>

Haneen_Alahmadi: <p>Thank you, it works.</p>