Repository: pytorch
Issue: Unprompted UserWarning
Labels: module: optimizer,triaged

### 🐛 Describe the bug

When running pytorch code for machine learning, I receive error for a function for training one epoch. The code is:

```
def train_one_epoch(model, epoch_index, tb_writer, optimizer, scheduler):
    running_loss = 0.
    last_loss = 0.

    for i, data in enumerate(training_loader):
        inputs, labels = data['image'].to(device), data['rez'].to(device)
        optimizer.zero_grad()
        try:
            outputs = model(inputs.float())
        except: continue

        loss = loss_fn(outputs, labels)
        loss.backward()

        optimizer.step()

        running_loss += loss.item()
        if i % 10 == 9:
            last_loss = running_loss / 10 
            tb_x = epoch_index * len(training_loader) + i + 1
            tb_writer.add_scalar('Loss/train', last_loss, tb_x)
            running_loss = 0.
    before_lr = optimizer.param_groups[0]["lr"]
    scheduler.step()
    after_lr = optimizer.param_groups[0]["lr"]

    target = [data['rez'] for data in training_loader]
    return last_loss, scheduler, model, before_lr, after_lr
```

Although the order of optimizer and lr_scheduler is correct I receive error:
```
UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
  warnings.warn("Detected call of `lr_scheduler.step()` before optimizer.step().
```
I believe it is a bug for it's the only part of the code that uses optimizer and r_scheduler. They are initialized like this:
```
optimizer = torch.optim.SGD(model.parameters(), lr=l, momentum=m)
scheduler = torch.optim.lr_scheduler.LinearLR(optimizer, start_factor=1.0, end_factor=0.3, total_iters=10)
```


### Versions

wget line returned 404 so I am pasting this:

```
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
_openmp_mutex             5.1                       1_gnu  
abseil-cpp                20211102.0           h27087fc_1    conda-forge
absl-py                   1.4.0              pyhd8ed1ab_0    conda-forge
aiohttp                   3.8.1           py310h5764c6d_1    conda-forge
aiosignal                 1.3.1              pyhd8ed1ab_0    conda-forge
asttokens                 2.0.5              pyhd3eb1b0_0    anaconda
async-timeout             4.0.2           py310h06a4308_0  
attrs                     23.1.0             pyh71513ae_1    conda-forge
backcall                  0.2.0              pyhd3eb1b0_0    anaconda
blas                      1.0                         mkl  
blinker                   1.6.2              pyhd8ed1ab_0    conda-forge
blosc                     1.21.3               h6a678d5_0  
boost-cpp                 1.73.0              h7f8727e_12  
bottleneck                1.3.5           py310ha9d4c09_0    anaconda
brotli                    1.0.9                h5eee18b_7  
brotli-bin                1.0.9                h5eee18b_7  
brotlipy                  0.7.0           py310h7f8727e_1002  
brunsli                   0.1                  h2531618_0    anaconda
bzip2                     1.0.8                h7b6447c_0  
c-ares                    1.19.0               h5eee18b_0  
ca-certificates           2023.01.10           h06a4308_0    anaconda
cachetools                5.3.1              pyhd8ed1ab_0    conda-forge
cairo                     1.16.0               hb05425b_5  
certifi                   2022.12.7       py310h06a4308_0    anaconda
cffi                      1.15.1          py310h5eee18b_3  
cfitsio                   3.470                h5893167_7  
charls                    2.2.0                h2531618_0    anaconda
charset-normalizer        2.0.4              pyhd3eb1b0_0  
click                     8.1.6           unix_pyh707e725_0    conda-forge
cloudpickle               2.0.0              pyhd3eb1b0_0    anaconda
contourpy                 1.0.5           py310hdb19cb5_0  
cryptography              41.0.2          py310h774aba0_0  
cuda-cudart               11.7.99                       0    nvidia
cuda-cupti                11.7.101                      0    nvidia
cuda-libraries            11.7.1                        0    nvidia
cuda-nvrtc                11.7.99                       0    nvidia
cuda-nvtx                 11.7.91                       0    nvidia
cuda-runtime              11.7.1                        0    nvidia
curl                      8.1.1                h37d81fd_2  
cycler                    0.11.0             pyhd3eb1b0_0  
cytoolz                   0.12.0          py310h5eee18b_0    anaconda
dask-core                 2022.7.0        py310h06a4308_0    anaconda
dbus                      1.13.18              hb2f20db_0  
decorator                 5.1.1              pyhd3eb1b0_0    anaconda
executing                 0.8.3              pyhd3eb1b0_0    anaconda
expat                     2.4.9                h6a678d5_0  
ffmpeg                    4.3                  hf484d3e_0    pytorch
filelock                  3.9.0           py310h06a4308_0  
fontconfig                2.14.1               h52c9d5c_1  
fonttools                 4.25.0             pyhd3eb1b0_0  
freetype                  2.12.1               h4a9f257_0  
freexl                    1.0.6                h27cfd23_0  
frozenlist                1.3.3           py310h5eee18b_0  
fsspec                    2022.11.0       py310h06a4308_0    anaconda
gdal                      3.6.2           py310h708d02d_0  
geos                      3.8.0                he6710b0_0  
geotiff                   1.7.0                h2a26cda_1  
giflib                    5.2.1                h5eee18b_3  
glib                      2.69.1               he621ea3_2  
gmp                       6.2.1                h295c915_3  
gmpy2                     2.1.2           py310heeb90bb_0  
gnutls                    3.6.15               he1e5248_0  
google-auth               2.22.0             pyh1a96a4e_0    conda-forge
google-auth-oauthlib      1.0.0              pyhd8ed1ab_1    conda-forge
grpc-cpp                  1.48.2               h5bf31a4_0  
grpcio                    1.48.2          py310h5bf31a4_0  
gst-plugins-base          1.14.1               h6a678d5_1  
gstreamer                 1.14.1               h5eee18b_1  
hdf4                      4.2.13               h3ca952b_2  
hdf5                      1.10.6               h3ffc7dd_1  
icu                       58.2                 he6710b0_3  
idna                      3.4             py310h06a4308_0  
imagecodecs               2021.8.26       py310h46e8fbd_2  
imageio                   2.19.3          py310h06a4308_0    anaconda
importlib-metadata        6.8.0              pyha770c72_0    conda-forge
intel-openmp              2023.1.0         hdb19cb5_46305  
ipython                   8.8.0           py310h06a4308_0    anaconda
jedi                      0.18.1          py310h06a4308_1    anaconda
jinja2                    3.1.2           py310h06a4308_0  
jpeg                      9e                   h5eee18b_1  
json-c                    0.16                 h5eee18b_0  
jxrlib                    1.1                  h7b6447c_2    anaconda
kealib                    1.5.0                hd940352_0  
kiwisolver                1.4.4           py310h6a678d5_0  
krb5                      1.20.1               h568e23c_1  
lame                      3.100                h7b6447c_0  
lcms2                     2.12                 h3be6417_0  
ld_impl_linux-64          2.38                 h1181459_1  
lerc                      3.0                  h295c915_0  
libaec                    1.0.4                he6710b0_1    anaconda
libboost                  1.73.0              h28710b8_12  
libbrotlicommon           1.0.9                h5eee18b_7  
libbrotlidec              1.0.9                h5eee18b_7  
libbrotlienc              1.0.9                h5eee18b_7  
libclang                  10.0.1          default_hb85057a_2  
libcublas                 11.10.3.66                    0    nvidia
libcufft                  10.7.2.124           h4fbf590_0    nvidia
libcufile                 1.7.1.12                      0    nvidia
libcurand                 10.3.3.129                    0    nvidia
libcurl                   8.1.1                h91b91d3_2  
libcusolver               11.4.0.1                      0    nvidia
libcusparse               11.7.4.91                     0    nvidia
libdeflate                1.17                 h5eee18b_0  
libedit                   3.1.20221030         h5eee18b_0  
libev                     4.33                 h7f8727e_1  
libevent                  2.1.12               h8f2d780_0  
libffi                    3.4.4                h6a678d5_0  
libgcc-ng                 11.2.0               h1234567_1  
libgdal                   3.6.2                hc0e11bb_0  
libgfortran-ng            11.2.0               h00389a5_1  
libgfortran5              11.2.0               h1234567_1  
libgomp                   11.2.0               h1234567_1  
libiconv                  1.16                 h7f8727e_2  
libidn2                   2.3.4                h5eee18b_0  
libkml                    1.3.0                h096b73e_6  
libllvm10                 10.0.1               hbcb73fb_5  
libnetcdf                 4.8.1                h8322cc2_2  
libnghttp2                1.52.0               ha637b67_1  
libnpp                    11.7.4.75                     0    nvidia
libnvjpeg                 11.8.0.2                      0    nvidia
libpng                    1.6.39               h5eee18b_0  
libpq                     12.15                h37d81fd_1  
libprotobuf               3.20.3               he621ea3_0  
libspatialite             4.3.0a              h71b31bf_21  
libssh2                   1.10.0               h37d81fd_2  
libstdcxx-ng              11.2.0               h1234567_1  
libtasn1                  4.19.0               h5eee18b_0  
libtiff                   4.5.0                h6a678d5_2  
libunistring              0.9.10               h27cfd23_0  
libuuid                   1.41.5               h5eee18b_0  
libwebp                   1.2.4                h11a3e52_1  
libwebp-base              1.2.4                h5eee18b_1  
libxcb                    1.15                 h7f8727e_0  
libxkbcommon              1.0.1                hfa300c1_0  
libxml2                   2.9.14               h74e7548_0  
libxslt                   1.1.35               h4e12654_0  
libzip                    1.8.0                h5cef20c_0  
libzopfli                 1.0.3                he6710b0_0    anaconda
locket                    1.0.0           py310h06a4308_0    anaconda
lz4-c                     1.9.4                h6a678d5_0  
markdown                  3.4.4              pyhd8ed1ab_0    conda-forge
markupsafe                2.1.1           py310h7f8727e_0  
matplotlib                3.7.1           py310h06a4308_1  
matplotlib-base           3.7.1           py310h1128e8f_1  
matplotlib-inline         0.1.6           py310h06a4308_0    anaconda
mkl                       2023.1.0         h213fc3f_46343  
mkl-service               2.4.0           py310h5eee18b_1  
mkl_fft                   1.3.6           py310h1128e8f_1  
mkl_random                1.2.2           py310h1128e8f_1  
mpc                       1.1.0                h10f8cd9_1  
mpfr                      4.0.2                hb69a4c5_1  
mpmath                    1.3.0           py310h06a4308_0  
multidict                 6.0.2           py310h5eee18b_0  
munkres                   1.1.4                      py_0  
mypy-extensions           1.0.0                    pypi_0    pypi
ncurses                   6.4                  h6a678d5_0  
nettle                    3.7.3                hbbd107a_1  
networkx                  3.1             py310h06a4308_0  
nspr                      4.35                 h6a678d5_0  
nss                       3.89.1               h6a678d5_0  
numexpr                   2.8.4           py310h85018f9_1  
numpy                     1.25.2          py310h5f9d8c6_0  
numpy-base                1.25.2          py310hb5e798b_0  
oauthlib                  3.2.2              pyhd8ed1ab_0    conda-forge
openh264                  2.1.1                h4ff587b_0  
openjpeg                  2.4.0                h3ad879b_0  
openssl                   1.1.1v               h7f8727e_0  
packaging                 23.0            py310h06a4308_0  
pandas                    1.5.2           py310h1128e8f_0    anaconda
parso                     0.8.3              pyhd3eb1b0_0    anaconda
partd                     1.2.0              pyhd3eb1b0_1    anaconda
pcre                      8.45                 h295c915_0  
pcre2                     10.37                he7ceb23_1  
pexpect                   4.8.0              pyhd3eb1b0_3    anaconda
pickleshare               0.7.5           pyhd3eb1b0_1003    anaconda
pillow                    9.4.0           py310h6a678d5_0  
pip                       23.2.1          py310h06a4308_0  
pixman                    0.40.0               h7f8727e_1  
ply                       3.11            py310h06a4308_0  
poppler                   22.12.0              h381b16e_0  
poppler-data              0.4.11               h06a4308_1  
postgresql                12.15                h16c4e8d_1  
proj                      6.2.1                h05a3930_0  
prompt-toolkit            3.0.36          py310h06a4308_0    anaconda
protobuf                  3.20.3          py310h6a678d5_0  
psutil                    5.9.5                    pypi_0    pypi
ptyprocess                0.7.0              pyhd3eb1b0_2    anaconda
pure_eval                 0.2.2              pyhd3eb1b0_0    anaconda
pyasn1                    0.4.8                      py_0    conda-forge
pyasn1-modules            0.2.7                      py_0    conda-forge
pycparser                 2.21               pyhd3eb1b0_0  
pygments                  2.11.2             pyhd3eb1b0_0    anaconda
pyjwt                     2.8.0              pyhd8ed1ab_0    conda-forge
pyopenssl                 23.2.0          py310h06a4308_0  
pyparsing                 3.0.9           py310h06a4308_0  
pyqt                      5.15.7          py310h6a678d5_1  
pyqt5-sip                 12.11.0                  pypi_0    pypi
pyre-extensions           0.0.30                   pypi_0    pypi
pysocks                   1.7.1           py310h06a4308_0  
python                    3.10.12              h7a1cb2a_0  
python-dateutil           2.8.2              pyhd3eb1b0_0  
python_abi                3.10                    2_cp310    conda-forge
pytorch                   2.0.1           py3.10_cuda11.7_cudnn8.5.0_0    pytorch
pytorch-cuda              11.7                 h778d358_5    pytorch
pytorch-mutex             1.0                        cuda    pytorch
pytz                      2022.7          py310h06a4308_0    anaconda
pyu2f                     0.1.5              pyhd8ed1ab_0    conda-forge
pywavelets                1.4.1           py310h5eee18b_0    anaconda
pyyaml                    6.0             py310h5eee18b_1    anaconda
qhull                     2020.2               hdb19cb5_2  
qt-main                   5.15.2               h327a75a_7  
qt-webengine              5.15.9               hd2b0992_4  
qtwebkit                  5.212                h4eab89a_4  
re2                       2022.04.01           h27087fc_0    conda-forge
readline                  8.2                  h5eee18b_0  
requests                  2.31.0          py310h06a4308_0  
requests-oauthlib         1.3.1              pyhd8ed1ab_0    conda-forge
rsa                       4.9                pyhd8ed1ab_0    conda-forge
scikit-image              0.19.3          py310h6a678d5_1    anaconda
scipy                     1.11.1          py310h5f9d8c6_0  
setuptools                68.0.0          py310h06a4308_0  
sip                       6.6.2           py310h6a678d5_0  
six                       1.16.0             pyhd3eb1b0_1  
snappy                    1.1.9                h295c915_0    anaconda
sqlite                    3.41.2               h5eee18b_0  
stack_data                0.2.0              pyhd3eb1b0_0    anaconda
sympy                     1.11.1          py310h06a4308_0  
tbb                       2021.8.0             hdb19cb5_0  
tensorboard               2.14.0             pyhd8ed1ab_0    conda-forge
tensorboard-data-server   0.7.0           py310h52d8a92_0  
tifffile                  2021.7.2           pyhd3eb1b0_2    anaconda
tiledb                    2.3.3                h1132f93_2  
tk                        8.6.12               h1ccaba5_0  
toml                      0.10.2             pyhd3eb1b0_0  
toolz                     0.12.0          py310h06a4308_0    anaconda
torchaudio                2.0.2               py310_cu117    pytorch
torcheval                 0.0.6                    pypi_0    pypi
torchtnt                  0.2.0                    pypi_0    pypi
torchtriton               2.0.0                     py310    pytorch
torchvision               0.15.2              py310_cu117    pytorch
tornado                   6.3.2           py310h5eee18b_0  
tqdm                      4.65.0          py310h2f386ee_0  
traitlets                 5.7.1           py310h06a4308_0    anaconda
typing-inspect            0.9.0                    pypi_0    pypi
typing_extensions         4.7.1           py310h06a4308_0  
tzdata                    2023c                h04d1e81_0  
urllib3                   1.26.16         py310h06a4308_0  
wcwidth                   0.2.5              pyhd3eb1b0_0    anaconda
werkzeug                  2.3.6              pyhd8ed1ab_0    conda-forge
wheel                     0.38.4          py310h06a4308_0  
xerces-c                  3.2.4                h94c2ce2_0  
xz                        5.4.2                h5eee18b_0  
yaml                      0.2.5                h7b6447c_0    anaconda
yarl                      1.7.2           py310h5764c6d_2    conda-forge
zfp                       0.5.5                h295c915_6    anaconda
zipp                      3.16.2             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               h5eee18b_0  
zstd                      1.5.5                hc292b87_0  
```

cc @vincentqb @jbschlosser @albanD @janeyx99 @crcrpar