lr_find doesn't return the correct suggestion if some losses are nan
🐛 Bug

lr_finder doesn't return the correct suggestion if some losses are nan. The returned loss is the one corresponding to the nan value, which is very big in my case.

To Reproduce
This depends on the dataset. Please see the code sample.
Code sample

I believe this is caused by numpy. The related code should be 
  
    
      pytorch-lightning/pytorch_lightning/trainer/lr_finder.py
    
    
         Line 325
      in
      b84b024
    
  
  
    

        
          
           min_grad = (np.gradient(np.array(loss))).argmin() 
        
    
  


example_losses = [0.90, 0.89, 0.87, 0.86, 0.85, 0.84]
print(np.gradient(example_losses).argmin())
example_losses = [0.90, 0.89, 0.87, 0.86, 0.85, 0.84, float('nan')]
print(np.gradient(example_losses).argmin())
Output:
1
5

Expected behavior

Return the correct suggested loss.
Environment

CUDA:

GPU:
available:         False
version:           10.2


Packages:

numpy:             1.18.4
pyTorch_debug:     False
pyTorch_version:   1.5.0
pytorch-lightning: 0.7.6
tensorboard:       2.2.0
tqdm:              4.45.0


System:

OS:                Linux
architecture:

64bit



processor:
python:            3.7.6
version:           #1 SMP Debian 4.19.118-2 (2020-04-29)



Additional context

NA