Repository: pytorch/ignite

Issue:
FastaiLRFinder does not run more than 1 epoch. Why?
I have been trying to use the `FastaiLRFinder` to find the best learning rate for my module.

If we create the trainer with the function `create_supervised_trainer` as below:

```
trainer = create_supervised_trainer(
    model, optimizer, criterion, device, output_transform=custom_output_transform
)
```

and run it:


```
with lr_finder.attach(
            trainer,
            to_save=to_save,
            num_iter=50,  
            end_lr=1.,
            step_mode='exp') as lr_finder_training:
        lr_finder_training.run(train_loader)
```

A warning will come up say: "UserWarning: Desired num_iter 50 is unreachable with the current run setup of 15 iteration (1 epochs) from ignite.contrib.handlers.param_scheduler import (LRScheduler, PiecewiseLinear)"

My dataloader has 15 batches to iterate, which means FastaiLRFinder does not allow you to run more than 1 epoch. Why?

According to their source code, we can see [here](https://github.com/pytorch/ignite/blob/14cdc046c7cac2db8a1bf838dcd09cbbe0e9c33b/ignite/contrib/handlers/lr_finder.py#L87) that this verification limits the user to run more iterations than the maximum of epochs in their dataloader.

But why? Am I missing something important here?


Discussion:
@rafaelpadilla Thank you for this question.

I agree with you, I don't see any reason to limit the run to maximum epoch. Maybe @vfdev-5 could have more inputs about it.

The fact is if end of epoch is reached, handlers will be raised (dump, checkout, loggers, metrics, validation, etc.). The main issue is here I guess.
@rafaelpadilla you can either set epoch length to more than 50 iterations or add more epochs to run:
```python
lr_finder_training.run(train_loader, epoch_length=100)
# or 
# lr_finder_training.run(train_loader, max_epochs=10)
```


**EDIT:** I agree that the API should encounter this situation and either warn the user about that or set automatically `epoch_length` to `num_iter`...
> @rafaelpadilla you can either set epoch length to more than 50 iterations or add more epochs to run:
> 
> ```python
> lr_finder_training.run(train_loader, epoch_length=100)
> # or 
> # lr_finder_training.run(train_loader, max_epochs=10)
> ```
> 
> **EDIT:** I agree that the API should encounter this situation and either warn the user about that or set automatically `epoch_length` to `num_iter`...

@vfdev-5 
It can be solved by setting `epoch_length`. 

But I guess it makes the `num_iter` parameter passed by the `attach` method meaningless.

The API is confusing.
@KickItLikeShika you have worked on LR finder recently. Can I assign this issue to you to improve it ?
@vfdev-5 Yes that's a good one, I can work on it, thanks!

Resolving patch:
diff --git a/ignite/handlers/lr_finder.py b/ignite/handlers/lr_finder.py
--- a/ignite/handlers/lr_finder.py
+++ b/ignite/handlers/lr_finder.py
@@ -3,6 +3,7 @@
 import logging
 import tempfile
 import warnings
+from math import ceil
 from pathlib import Path
 from typing import Any, Callable, Dict, List, Mapping, Optional, Union
 
@@ -103,12 +104,10 @@ def _run(
             num_iter = trainer.state.epoch_length * trainer.state.max_epochs
         else:
             max_iter = trainer.state.epoch_length * trainer.state.max_epochs  # type: ignore[operator]
-            if num_iter > max_iter:
-                warnings.warn(
-                    f"Desired num_iter {num_iter} is unreachable with the current run setup of {max_iter} iteration "
-                    f"({trainer.state.max_epochs} epochs)",
-                    UserWarning,
-                )
+            if max_iter < num_iter:
+                max_iter = num_iter
+                trainer.state.max_iters = num_iter
+                trainer.state.max_epochs = ceil(num_iter / trainer.state.epoch_length)  # type: ignore[operator]
 
         if not trainer.has_event_handler(self._reached_num_iterations):
             trainer.add_event_handler(Events.ITERATION_COMPLETED, self._reached_num_iterations, num_iter)


Test patch:
diff --git a/tests/ignite/handlers/test_lr_finder.py b/tests/ignite/handlers/test_lr_finder.py
--- a/tests/ignite/handlers/test_lr_finder.py
+++ b/tests/ignite/handlers/test_lr_finder.py
@@ -307,7 +307,8 @@ def test_num_iter_is_not_enough(lr_finder, to_save, dummy_engine, dataloader):
         with pytest.warns(UserWarning):
             trainer_with_finder.run(dataloader)
         assert_output_sizes(lr_finder, dummy_engine)
-        assert dummy_engine.state.iteration == len(dataloader)
+        assert dummy_engine.state.iteration != len(dataloader)
+        assert dummy_engine.state.iteration == 150
 
 
 def test_detach_terminates(lr_finder, to_save, dummy_engine, dataloader, recwarn):
@@ -321,6 +322,16 @@ def test_detach_terminates(lr_finder, to_save, dummy_engine, dataloader, recwarn
     # assert len(recwarn) == 0
 
 
+def test_different_num_iters(lr_finder, to_save, dummy_engine, dataloader):
+    with lr_finder.attach(dummy_engine, to_save, num_iter=200, diverge_th=float("inf")) as trainer_with_finder:
+        trainer_with_finder.run(dataloader)
+        assert trainer_with_finder.state.iteration == 200  # num_iter
+
+    with lr_finder.attach(dummy_engine, to_save, num_iter=1000, diverge_th=float("inf")) as trainer_with_finder:
+        trainer_with_finder.run(dataloader)
+        assert trainer_with_finder.state.iteration == 1000  # num_iter
+
+
 @pytest.mark.parametrize("step_mode", ["exp", "linear"])
 def test_start_lr(lr_finder, to_save, dummy_engine, dataloader, step_mode):
     with lr_finder.attach(