anon-iclr-submission/benchname-bug-localization huggingface dataset:
Repository: thilinarajapakse/simpletransformers
Issue: "early_stopping_metric" doesn't seem to work

I am trying multi_label and multi_class classification with early stopping.

I have insert the sklearn.metrics.f1(average = "micro) function into **kwargs with using this code:

               ` kwargs = {"f1_micro": partial(sklearn.metrics.f1_score, average = "micro")}`

The f1_micro score is being calculated on the eval_df and it gets printed while training. But the argument :

"early_stopping_metric" = "f1_micro" doesn't work. It seems that it always calculates the loss function on eval_df for the early stopping functionality.
It is my bad or there is indeed a bug?

Changed files:
simpletransformers/classification/classification_model.py

diff --git a/simpletransformers/classification/classification_model.py b/simpletransformers/classification/classification_model.py
index 772f95c..f540ff5 100755
--- a/simpletransformers/classification/classification_model.py
+++ b/simpletransformers/classification/classification_model.py
@@ -501,7 +501,7 @@ class ClassificationModel:
                             best_eval_metric = results[args.early_stopping_metric]
                             self._save_model(args.best_model_dir, optimizer, scheduler, model=model, results=results)
                         if best_eval_metric and args.early_stopping_metric_minimize:
-                            if results[args.early_stopping_metric] - best_eval_metric < args.early_stopping_delta:
+                            if best_eval_metric - results[args.early_stopping_metric] > args.early_stopping_delta:
                                 best_eval_metric = results[args.early_stopping_metric]
                                 self._save_model(
                                     args.best_model_dir, optimizer, scheduler, model=model, results=results
@@ -577,7 +577,7 @@ class ClassificationModel:
                     best_eval_metric = results[args.early_stopping_metric]
                     self._save_model(args.best_model_dir, optimizer, scheduler, model=model, results=results)
                 if best_eval_metric and args.early_stopping_metric_minimize:
-                    if results[args.early_stopping_metric] - best_eval_metric < args.early_stopping_delta:
+                    if best_eval_metric - results[args.early_stopping_metric] > args.early_stopping_delta:
                         best_eval_metric = results[args.early_stopping_metric]
                         self._save_model(args.best_model_dir, optimizer, scheduler, model=model, results=results)
                         early_stopping_counter = 0