Repository: pytorch/audio

Issue:
pipeline_wav2letter: UnboundLocalError: local variable 'loss' referenced before assignment
## 🐛 Bug
pipeline_wav2letter training error: UnboundLocalError: local variable 'loss' referenced before assignment

## To Reproduce

Steps to reproduce the behavior:

```
PYTHONPATH=`pwd`/../.. python main.py \
    --reduce-lr-valid \
    --dataset-train train-clean-100 \
    --dataset-valid dev-clean \
    --batch-size 128 \
    --learning-rate .6 \
    --momentum .8 \
    --weight-decay .00001 \
    --clip-grad 0. \
    --gamma .99 \
    --hop-length 160 \
    --win-length 400 \
    --n-bins 13 \
    --normalize \
    --optimizer adadelta \
    --scheduler reduceonplateau \
    --epochs 30 \
    --dataset-root /home/ubuntu/Data/ \
    --dataset-folder-in-archive LibriSpeech
```
Log & error:
```
/home/ubuntu/audio/torchaudio/extension/extension.py:14: UserWarning: torchaudio C++ extension is not available.
  warnings.warn('torchaudio C++ extension is not available.')
INFO:root:Namespace(batch_size=128, checkpoint='', clip_grad=0.0, dataset_folder_in_archive='LibriSpeech', dataset_root='/home/ubuntu/Data/', da
taset_train=['train-clean-100'], dataset_valid=['dev-clean'], decoder='greedy', distributed=False, epochs=30, eps=1e-08, freq_mask=0, gamma=0.99, hop_length=160, ji
t=False, learning_rate=0.6, momentum=0.8, n_bins=13, normalize=True, optimizer='adadelta', print_freq=10, progress_bar=False, reduce_lr_valid=True, rho=0.95, schedu
ler='reduceonplateau', seed=0, start_epoch=0, time_mask=0, type='mfcc', weight_decay=1e-05, win_length=400, workers=0, world_size=8)
INFO:root:Start time: 2020-10-29 00:28:04.029632
basename train-clean-100.tar.gz
folder_in_archive basename LibriSpeech train-clean-100
basename dev-clean.tar.gz
folder_in_archive basename LibriSpeech dev-clean
INFO:root:Number of parameters: 23282529
INFO:root:Checkpoint: not found
INFO:root:Epoch: 0
/home/ubuntu/miniconda3/envs/torch1.7/lib/python3.8/site-packages/torch/functional.py:515: UserWarning: stft will require the return_complex parameter be explicitly
  specified in a future PyTorch release. Use return_complex=False  to preserve the current behavior or return_complex=True to return  a complex output. (Triggered i
nternally at  /opt/conda/conda-bld/pytorch_1603729096996/work/aten/src/ATen/native/SpectralOps.cpp:653.)
  return _VF.stft(input, n_fft, hop_length, win_length, window,  # type: ignore
/home/ubuntu/miniconda3/envs/torch1.7/lib/python3.8/site-packages/torch/functional.py:515: UserWarning: The function torch.rfft is deprecated and will be removed in
 a future PyTorch release. Use the new torch.fft module functions, instead, by importing torch.fft and calling torch.fft.fft or torch.fft.rfft. (Triggered internall
y at  /opt/conda/conda-bld/pytorch_1603729096996/work/aten/src/ATen/native/SpectralOps.cpp:590.)
  return _VF.stft(input, n_fft, hop_length, win_length, window,  # type: ignore
INFO:root:Target: the freshmen are sim    Output:
INFO:root:Target: i crosed to the star    Output:
{"name": "train", "epoch": 0, "cer over target length": 1.0, "cumulative cer": 24618.0, "total chars": 24618.0, "cer": 0.0, "cumulative cer over target length": 0.0
, "wer over target length": 1.0, "cumulative wer": 4701.0, "total words": 4701.0, "wer": 0.0, "cumulative wer over target length": 0.0, "lr": 0.6, "batch size": 128
, "n_channel": 13, "n_time": 1671, "dataset length": 128.0, "iteration": 1.0, "loss": 8.766070365905762, "cumulative loss": 8.766070365905762, "average loss": 8.766
070365905762, "iteration time": 3.761294364929199, "epoch time": 3.761294364929199}
INFO:root:Target: who are ready enough    Output:
INFO:root:Target: when we saw on our w    Output:
...
{"name": "train", "epoch": 0, "cer over target length": 1.0, "cumulative cer": 5126259.0, "total chars": 5126259.0, "cer": 0.0, "cumulative cer over target length": 0.0, "wer over target length": 1.0, "cumulative wer": 981345.0, "total words": 981345.0, "wer": 0.0, "cumulative wer over target length": 0.0, "lr": 0.6, "batch size": 128, "n_channel": 13, "n_time": 1661, "dataset length": 28288.0, "iteration": 221.0, "loss": 3.354341983795166, "cumulative loss": 841.6320466995239, "average loss": 3.808289804070244, "iteration time": 1.103724718093872, "epoch time": 241.38750743865967}
INFO:root:Target: ferko again proceded    Output:
INFO:root:Target: and then the oficial    Output:
{"name": "train", "epoch": 0, "cer over target length": 1.0, "cumulative cer": 5149600.0, "total chars": 5149600.0, "cer": 0.0, "cumulative cer over target length": 0.0, "wer over target length": 1.0, "cumulative wer": 985796.0, "total words": 985796.0, "wer": 0.0, "cumulative wer over target length": 0.0, "lr": 0.6, "batch size": 128, "n_channel": 13, "n_time": 1687, "dataset length": 28416.0, "iteration": 222.0, "loss": 3.3545937538146973, "cumulative loss": 844.9866404533386, "average loss": 3.806246128168192, "iteration time": 1.0857408046722412, "epoch time": 242.4732482433319}
Traceback (most recent call last):
  File "main.py", line 668, in <module>
    spawn_main(main, args)
  File "main.py", line 661, in spawn_main
    main(0, args)
  File "main.py", line 647, in main
    scheduler.step(loss)
UnboundLocalError: local variable 'loss' referenced before assignment
```
## Expected behavior

Normal training.

## Environment

```
Collecting environment information...
PyTorch version: 1.7.0
Is debug build: True
CUDA used to build PyTorch: 11.0
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.1 LTS (x86_64)
GCC version: (Ubuntu 8.4.0-3ubuntu2) 8.4.0
Clang version: Could not collect
CMake version: version 3.18.2

Python version: 3.8 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: 10.1.243
GPU models and configuration:
GPU 0: GeForce RTX 2080 Ti
GPU 1: GeForce RTX 2080 Ti

Nvidia driver version: 450.80.02
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.19.2
[pip3] torch==1.7.0
[pip3] torchaudio==0.7.0a0+ac17b64
[pip3] torchvision==0.8.1
[conda] blas                      1.0                         mkl
[conda] cudatoolkit               11.0.221             h6bb024c_0
[conda] magma-cuda102             2.5.2                         1    pytorch
[conda] mkl                       2020.2                      256
[conda] mkl-include               2020.2                      256
[conda] mkl-service               2.3.0            py38he904b0f_0
[conda] mkl_fft                   1.2.0            py38h23d657b_0
[conda] mkl_random                1.1.1            py38h0573a6f_0
[conda] numpy                     1.19.1           py38hbc911f0_0
[conda] numpy-base                1.19.1           py38hfa32c7d_0
[conda] pytorch                   1.7.0           py3.8_cuda11.0.221_cudnn8.0.3_0    pytorch
[conda] torchaudio                0.7.0                      py38    pytorch
[conda] torchvision               0.8.1                py38_cu110    pytorch
```




Discussion:

Resolving patch:
diff --git a/examples/pipeline_wav2letter/main.py b/examples/pipeline_wav2letter/main.py
--- a/examples/pipeline_wav2letter/main.py
+++ b/examples/pipeline_wav2letter/main.py
@@ -83,13 +83,6 @@ def parse_args():
     parser.add_argument(
         "--start-epoch", default=0, type=int, metavar="N", help="manual epoch number"
     )
-    parser.add_argument(
-        "--print-freq",
-        default=10,
-        type=int,
-        metavar="N",
-        help="print frequency in epochs",
-    )
     parser.add_argument(
         "--reduce-lr-valid",
         action="store_true",
@@ -615,37 +608,35 @@ def main(rank, args):
             not args.reduce_lr_valid,
         )
 
-        if not (epoch + 1) % args.print_freq or epoch == args.epochs - 1:
-
-            loss = evaluate(
-                model,
-                criterion,
-                loader_validation,
-                decoder,
-                language_model,
-                devices[0],
-                epoch,
-                not_main_rank,
-            )
-
-            is_best = loss < best_loss
-            best_loss = min(loss, best_loss)
-            save_checkpoint(
-                {
-                    "epoch": epoch + 1,
-                    "state_dict": model.state_dict(),
-                    "best_loss": best_loss,
-                    "optimizer": optimizer.state_dict(),
-                    "scheduler": scheduler.state_dict(),
-                },
-                is_best,
-                args.checkpoint,
-                not_main_rank,
-            )
+        loss = evaluate(
+            model,
+            criterion,
+            loader_validation,
+            decoder,
+            language_model,
+            devices[0],
+            epoch,
+            not_main_rank,
+        )
 
         if args.reduce_lr_valid and isinstance(scheduler, ReduceLROnPlateau):
             scheduler.step(loss)
 
+        is_best = loss < best_loss
+        best_loss = min(loss, best_loss)
+        save_checkpoint(
+            {
+                "epoch": epoch + 1,
+                "state_dict": model.state_dict(),
+                "best_loss": best_loss,
+                "optimizer": optimizer.state_dict(),
+                "scheduler": scheduler.state_dict(),
+            },
+            is_best,
+            args.checkpoint,
+            not_main_rank,
+        )
+
     logging.info("End time: %s", datetime.now())
 
     if args.distributed:


Test patch: