Repository: pytorch
Issue: Torch 1.10.0 - RuntimeError: requires_grad_ is not supported on ScriptModules
Labels: oncall: jit

## 🐛 Bug

Hello.
I was having a MAGMA problem and updated my torch version as was recommended in [#67955](https://github.com/pytorch/pytorch/issues/67955) to 1.10.0. Now, the code that ran in 1.9. is raising this function error:

```
File "/home/luis/Desktop/cog/vqgan-clip-generator/src/vqgan_clip/engine.py", line 246, in initialize_VQGAN_CLIP
    self._perceptor = clip.load(self.conf.clip_model, jit=jit)[0].eval().requires_grad_(False).to(self._device)
  File "/home/luis/anaconda3/envs/pytorch/lib/python3.9/site-packages/torch/jit/_script.py", line 914, in fail
    raise RuntimeError(name + " is not supported on ScriptModules")
RuntimeError: requires_grad_ is not supported on ScriptModules
```

## To Reproduce

Running the VQGAN+CLIP code provided in line 243 of https://github.com/rkhamilton/vqgan-clip-generator/blob/main/src/vqgan_clip/engine.py and I get the bug from torch/jit/_script.py

## Environment

```
# packages in environment at /home/luis/anaconda3/envs/pytorch:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                      1_llvm    conda-forge
absl-py                   0.15.0                   pypi_0    pypi
addict                    2.4.0                    pypi_0    pypi
aiohttp                   3.8.0                    pypi_0    pypi
aiosignal                 1.2.0                    pypi_0    pypi
albumentations            1.0.3              pyhd8ed1ab_0    conda-forge
alsa-lib                  1.2.3                h516909a_0    conda-forge
antlr4-python3-runtime    4.8                      pypi_0    pypi
aom                       3.2.0                h9c3ff4c_2    conda-forge
appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
async-timeout             4.0.0                    pypi_0    pypi
attrs                     21.2.0                   pypi_0    pypi
audioread                 2.1.9            py39hf3d152e_0    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                        py_2    conda-forge
backports.functools_lru_cache 1.6.4              pyhd8ed1ab_0    conda-forge
basicsr                   1.3.4.7                  pypi_0    pypi
beautifulsoup4            4.10.0             pyha770c72_0    conda-forge
blosc                     1.21.0               h9c3ff4c_0    conda-forge
brotli                    1.0.9                h7f98852_6    conda-forge
brotli-bin                1.0.9                h7f98852_6    conda-forge
brotlipy                  0.7.0           py39h3811e60_1001    conda-forge
brunsli                   0.1                  h9c3ff4c_0    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.18.1               h7f98852_0    conda-forge
c-blosc2                  2.0.4                h5f21a17_1    conda-forge
ca-certificates           2021.10.8            ha878542_0    conda-forge
cachetools                4.2.4                    pypi_0    pypi
cairo                     1.16.0            h6cf1ce9_1008    conda-forge
certifi                   2021.10.8        py39hf3d152e_1    conda-forge
cffi                      1.14.6           py39he32792d_0    conda-forge
cfitsio                   4.0.0                h9a35b8e_0    conda-forge
chardet                   4.0.0            py39hf3d152e_1    conda-forge
charls                    2.2.0                h9c3ff4c_0    conda-forge
charset-normalizer        2.0.0              pyhd8ed1ab_0    conda-forge
chatterbot                1.0.4                    pypi_0    pypi
chatterbot-corpus         1.2.0                    pypi_0    pypi
click                     8.0.3                    pypi_0    pypi
cloudpickle               2.0.0              pyhd8ed1ab_0    conda-forge
cmake                     3.21.3               h8897547_0    conda-forge
colorama                  0.4.4              pyh9f0ad1d_0    conda-forge
cryptography              35.0.0           py39h95dcef6_1    conda-forge
cudatoolkit               11.2.0               h73cb219_9    conda-forge
cudnn                     8.2.1.32             h86fa8c9_0    conda-forge
cycler                    0.11.0             pyhd8ed1ab_0    conda-forge
cytoolz                   0.11.0           py39h3811e60_3    conda-forge
dask-core                 2021.10.0          pyhd8ed1ab_0    conda-forge
dataclasses               0.8                pyhc8e2a94_3    conda-forge
dbus                      1.13.6               h48d8840_2    conda-forge
debugpy                   1.4.1            py39he80948d_1    conda-forge
decorator                 4.4.2                    pypi_0    pypi
einops                    0.3.2                    pypi_0    pypi
entrypoints               0.3             pyhd8ed1ab_1003    conda-forge
expat                     2.4.1                h9c3ff4c_0    conda-forge
facexlib                  0.2.1.1                  pypi_0    pypi
ffmpeg                    4.4.0                h6987444_5    conda-forge
filelock                  3.3.2              pyhd8ed1ab_0    conda-forge
filterpy                  1.4.5                    pypi_0    pypi
fontconfig                2.13.1            hba837de_1005    conda-forge
freeglut                  3.2.1                h9c3ff4c_2    conda-forge
freetype                  2.10.4               h0708190_1    conda-forge
frozenlist                1.2.0                    pypi_0    pypi
fsspec                    2021.10.1          pyhd8ed1ab_0    conda-forge
ftfy                      6.0.3                    pypi_0    pypi
future                    0.18.2           py39hf3d152e_3    conda-forge
gdown                     4.2.0              pyhd8ed1ab_0    conda-forge
geos                      3.10.0               h9c3ff4c_0    conda-forge
gettext                   0.19.8.1          h0b5b191_1005    conda-forge
gfpgan                    0.2.3                    pypi_0    pypi
giflib                    5.2.1                h36c2ea0_2    conda-forge
glib                      2.68.4               h9c3ff4c_0    conda-forge
glib-tools                2.68.4               h9c3ff4c_0    conda-forge
gmp                       6.2.1                h58526e2_0    conda-forge
gnutls                    3.6.13               h85f3911_1    conda-forge
google-auth               2.3.3                    pypi_0    pypi
google-auth-oauthlib      0.4.6                    pypi_0    pypi
graphite2                 1.3.13            h58526e2_1001    conda-forge
grpcio                    1.41.1                   pypi_0    pypi
gst-plugins-base          1.18.5               hf529b03_0    conda-forge
gstreamer                 1.18.5               h76c114f_0    conda-forge
harfbuzz                  2.9.1                h83ec7ef_1    conda-forge
hdf5                      1.12.1          nompi_h2750804_101    conda-forge
huggingface-hub           0.0.19                   pypi_0    pypi
icu                       68.2                 h9c3ff4c_0    conda-forge
idna                      3.1                pyhd3deb0d_0    conda-forge
imagecodecs               2021.8.26        py39h571908b_2    conda-forge
imageio                   2.10.1                   pypi_0    pypi
imageio-ffmpeg            0.4.5                    pypi_0    pypi
imgaug                    0.4.0                      py_1    conda-forge
intel-openmp              2021.3.0          h06a4308_3350  
ipykernel                 6.4.2            py39hef51801_0    conda-forge
ipython                   7.29.0           py39hef51801_0    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
jasper                    1.900.1           h07fcdf6_1006    conda-forge
jbig                      2.1               h7f98852_2003    conda-forge
jedi                      0.18.0           py39hf3d152e_3    conda-forge
joblib                    1.1.0              pyhd8ed1ab_0    conda-forge
jpeg                      9d                   h36c2ea0_0    conda-forge
jupyter_client            7.0.6              pyhd8ed1ab_0    conda-forge
jupyter_core              4.9.1            py39hf3d152e_0    conda-forge
jxrlib                    1.1                  h7f98852_2    conda-forge
kiwisolver                1.3.2            py39h1a9c180_1    conda-forge
kornia                    0.6.1                    pypi_0    pypi
krb5                      1.19.2               hcc1bbae_2    conda-forge
lame                      3.100             h7f98852_1001    conda-forge
lcms2                     2.12                 hddcbb42_0    conda-forge
ld_impl_linux-64          2.36.1               hea4e1c9_2    conda-forge
lerc                      3.0                  h9c3ff4c_0    conda-forge
libaec                    1.0.6                h9c3ff4c_0    conda-forge
libblas                   3.9.0            12_linux64_mkl    conda-forge
libbrotlicommon           1.0.9                h7f98852_6    conda-forge
libbrotlidec              1.0.9                h7f98852_6    conda-forge
libbrotlienc              1.0.9                h7f98852_6    conda-forge
libcblas                  3.9.0            12_linux64_mkl    conda-forge
libclang                  11.1.0          default_ha53f305_1    conda-forge
libcurl                   7.79.1               h2574ce0_1    conda-forge
libdeflate                1.8                  h7f98852_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libevent                  2.1.10               h9b69904_4    conda-forge
libffi                    3.3                  h58526e2_2    conda-forge
libflac                   1.3.3                h9c3ff4c_1    conda-forge
libgcc-ng                 11.2.0              h1d223b6_11    conda-forge
libgfortran-ng            11.2.0              h69a702a_11    conda-forge
libgfortran5              11.2.0              h5c6108e_11    conda-forge
libglib                   2.68.4               h3e27bee_0    conda-forge
libglu                    9.0.0             he1b5a44_1001    conda-forge
libgomp                   11.2.0              h1d223b6_11    conda-forge
libiconv                  1.16                 h516909a_0    conda-forge
liblapack                 3.9.0            12_linux64_mkl    conda-forge
liblapacke                3.9.0            12_linux64_mkl    conda-forge
libllvm10                 10.0.1               he513fc3_3    conda-forge
libllvm11                 11.1.0               hf817b99_2    conda-forge
libnghttp2                1.43.0               h812cca2_1    conda-forge
libogg                    1.3.4                h7f98852_1    conda-forge
libopenblas               0.3.18          pthreads_h8fe5266_0    conda-forge
libopencv                 4.5.3            py39h24e4dd1_3    conda-forge
libopus                   1.3.1                h7f98852_1    conda-forge
libpng                    1.6.37               h21135ba_2    conda-forge
libpq                     13.3                 hd57d9b9_2    conda-forge
libprotobuf               3.18.1               h780b84a_0    conda-forge
librosa                   0.8.1              pyhd8ed1ab_0    conda-forge
libsndfile                1.0.31               h9c3ff4c_1    conda-forge
libsodium                 1.0.18               h36c2ea0_1    conda-forge
libssh2                   1.10.0               ha56f1ee_2    conda-forge
libstdcxx-ng              11.2.0              he4da1e4_11    conda-forge
libtiff                   4.3.0                h6f004c6_2    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libuv                     1.42.0               h7f98852_0    conda-forge
libvorbis                 1.3.7                h9c3ff4c_0    conda-forge
libvpx                    1.11.0               h9c3ff4c_3    conda-forge
libwebp-base              1.2.1                h7f98852_0    conda-forge
libxcb                    1.13              h7f98852_1003    conda-forge
libxkbcommon              1.0.3                he3ba5ed_0    conda-forge
libxml2                   2.9.12               h72842e0_0    conda-forge
libzlib                   1.2.11            h36c2ea0_1013    conda-forge
libzopfli                 1.0.3                h9c3ff4c_0    conda-forge
llvm-openmp               12.0.1               h4bd325d_1    conda-forge
llvmlite                  0.37.0                   pypi_0    pypi
lmdb                      1.2.1                    pypi_0    pypi
locket                    0.2.0                      py_2    conda-forge
lz4-c                     1.9.3                h9c3ff4c_1    conda-forge
magma                     2.5.4                h6103c52_2    conda-forge
markdown                  3.3.4                    pypi_0    pypi
mathparse                 0.1.2                    pypi_0    pypi
matplotlib-base           3.4.3            py39h2fa2bec_1    conda-forge
matplotlib-inline         0.1.3              pyhd8ed1ab_0    conda-forge
mkl                       2021.4.0           h06a4308_640  
mkl-include               2021.4.0           h8d4b97c_729    conda-forge
moviepy                   1.0.3                    pypi_0    pypi
multidict                 5.2.0                    pypi_0    pypi
mysql-common              8.0.27               ha770c72_1    conda-forge
mysql-libs                8.0.27               hfa10184_1    conda-forge
nccl                      2.11.4.1             hdc17891_0    conda-forge
ncurses                   6.2                  h58526e2_4    conda-forge
nest-asyncio              1.5.1              pyhd8ed1ab_0    conda-forge
nettle                    3.6                  he412f7d_0    conda-forge
networkx                  2.6.3              pyhd8ed1ab_1    conda-forge
ninja                     1.10.2               h4bd325d_1    conda-forge
nltk                      3.6.5                    pypi_0    pypi
nspr                      4.32                 h9c3ff4c_0    conda-forge
nss                       3.71                 hb5efdd6_0    conda-forge
numba                     0.54.1                   pypi_0    pypi
numpy                     1.20.3                   pypi_0    pypi
oauthlib                  3.1.1                    pypi_0    pypi
olefile                   0.46               pyh9f0ad1d_1    conda-forge
omegaconf                 2.1.1                    pypi_0    pypi
opencv                    4.5.3            py39hf3d152e_3    conda-forge
opencv-python             4.5.4.58                 pypi_0    pypi
openh264                  2.1.1                h780b84a_0    conda-forge
openjpeg                  2.4.0                hb52868f_1    conda-forge
openssl                   1.1.1l               h7f98852_0    conda-forge
packaging                 21.2                     pypi_0    pypi
pandas                    1.3.4            py39hde0f152_0    conda-forge
parso                     0.8.2              pyhd8ed1ab_0    conda-forge
partd                     1.2.0              pyhd8ed1ab_0    conda-forge
pcre                      8.45                 h9c3ff4c_0    conda-forge
pexpect                   4.8.0              pyh9f0ad1d_2    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
piexif                    1.1.3                    pypi_0    pypi
pillow                    8.4.0                    pypi_0    pypi
pint                      0.18                     pypi_0    pypi
pip                       21.3.1             pyhd8ed1ab_0    conda-forge
pixman                    0.40.0               h36c2ea0_0    conda-forge
pooch                     1.5.2              pyhd8ed1ab_0    conda-forge
proglog                   0.1.9                    pypi_0    pypi
prompt-toolkit            3.0.21             pyha770c72_0    conda-forge
protobuf                  3.19.1                   pypi_0    pypi
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
py-opencv                 4.5.3            py39hef51801_3    conda-forge
pyasn1                    0.4.8                    pypi_0    pypi
pyasn1-modules            0.2.8                    pypi_0    pypi
pycparser                 2.20               pyh9f0ad1d_2    conda-forge
pydeprecate               0.3.1                    pypi_0    pypi
pygments                  2.10.0             pyhd8ed1ab_0    conda-forge
pymongo                   3.12.1                   pypi_0    pypi
pyopenssl                 21.0.0             pyhd8ed1ab_0    conda-forge
pyparsing                 2.4.7                    pypi_0    pypi
pysocks                   1.7.1            py39hf3d152e_3    conda-forge
pysoundfile               0.10.3.post1       pyhd3deb0d_0    conda-forge
python                    3.9.0           hffdb5ce_5_cpython    conda-forge
python-dateutil           2.7.5                    pypi_0    pypi
python_abi                3.9                      2_cp39    conda-forge
pytorch                   1.10.0          cuda112py39h4e14dd4_0    conda-forge
pytorch-gpu               1.10.0          cuda112py39h0bbbad9_0    conda-forge
pytorch-lightning         1.4.9                    pypi_0    pypi
pytorch-ranger            0.1.1                    pypi_0    pypi
pytz                      2021.3             pyhd8ed1ab_0    conda-forge
pywavelets                1.1.1            py39hce5d2b2_3    conda-forge
pyyaml                    3.13                     pypi_0    pypi
pyzmq                     22.3.0           py39h37b5a0c_0    conda-forge
qt                        5.12.9               hda022c4_4    conda-forge
readline                  8.1                  h46c0cb4_0    conda-forge
realesrgan                0.2.2.5                  pypi_0    pypi
regex                     2021.11.1                pypi_0    pypi
requests                  2.26.0             pyhd8ed1ab_0    conda-forge
requests-oauthlib         1.3.0                    pypi_0    pypi
resampy                   0.2.2                      py_0    conda-forge
rhash                     1.4.1                h7f98852_0    conda-forge
rsa                       4.7.2                    pypi_0    pypi
sacremoses                0.0.46                   pypi_0    pypi
scikit-image              0.18.3                   pypi_0    pypi
scikit-learn              1.0.1            py39h7c5d8c9_1    conda-forge
scipy                     1.7.1            py39hee8e79c_0    conda-forge
setuptools                58.4.0           py39hf3d152e_1    conda-forge
shapely                   1.8.0            py39hc7dd4e9_1    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
sk-video                  1.1.10                   pypi_0    pypi
sleef                     3.5.1                h9b69904_2    conda-forge
snappy                    1.1.8                he1b5a44_3    conda-forge
soupsieve                 2.0.1                      py_1    conda-forge
sqlalchemy                1.2.19                   pypi_0    pypi
sqlite                    3.36.0               h9cd32fc_2    conda-forge
taming-transformers       0.0.1                    pypi_0    pypi
tb-nightly                2.8.0a20211101           pypi_0    pypi
tensorboard               2.7.0                    pypi_0    pypi
tensorboard-data-server   0.6.1                    pypi_0    pypi
tensorboard-plugin-wit    1.8.0                    pypi_0    pypi
threadpoolctl             3.0.0              pyh8a188c0_0    conda-forge
tifffile                  2021.11.2          pyhd8ed1ab_0    conda-forge
tk                        8.6.11               h27826a3_1    conda-forge
tokenizers                0.10.3                   pypi_0    pypi
toolz                     0.11.1                     py_0    conda-forge
torch-optimizer           0.3.0                    pypi_0    pypi
torchmetrics              0.6.0                    pypi_0    pypi
torchvision               0.10.1          py39cuda112hc5182df_0_cuda    conda-forge
tornado                   6.1              py39h3811e60_2    conda-forge
tqdm                      4.62.3             pyhd8ed1ab_0    conda-forge
traitlets                 5.1.1              pyhd8ed1ab_0    conda-forge
transformers              4.12.2                   pypi_0    pypi
typing_extensions         3.10.0.2           pyha770c72_0    conda-forge
tzdata                    2021e                he74cb21_0    conda-forge
urllib3                   1.26.7             pyhd8ed1ab_0    conda-forge
wcwidth                   0.2.5              pyh9f0ad1d_2    conda-forge
werkzeug                  2.0.2                    pypi_0    pypi
wheel                     0.37.0             pyhd8ed1ab_1    conda-forge
x264                      1!161.3030           h7f98852_1    conda-forge
x265                      3.5                  h4bd325d_1    conda-forge
xorg-fixesproto           5.0               h7f98852_1002    conda-forge
xorg-inputproto           2.3.2             h7f98852_1002    conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libice               1.0.10               h7f98852_0    conda-forge
xorg-libsm                1.2.3             hd9c2040_1000    conda-forge
xorg-libx11               1.7.2                h7f98852_0    conda-forge
xorg-libxau               1.0.9                h7f98852_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h7f98852_1    conda-forge
xorg-libxfixes            5.0.3             h7f98852_1004    conda-forge
xorg-libxi                1.7.10               h7f98852_0    conda-forge
xorg-libxrender           0.9.10            h7f98852_1003    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-xextproto            7.3.0             h7f98852_1002    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xz                        5.2.5                h516909a_1    conda-forge
yaml                      0.2.5                h516909a_0    conda-forge
yapf                      0.31.0                   pypi_0    pypi
yarl                      1.7.2                    pypi_0    pypi
zeromq                    4.3.4                h9c3ff4c_1    conda-forge
zfp                       0.5.5                h9c3ff4c_7    conda-forge
zlib                      1.2.11            h36c2ea0_1013    conda-forge
zstd                      1.5.0                ha95c52a_0    conda-forge
```

```

     active environment : pytorch
    active env location : /home/luis/anaconda3/envs/pytorch
            shell level : 2
       user config file : /home/luis/.condarc
 populated config files : /home/luis/.condarc
          conda version : 4.10.3
    conda-build version : 3.20.5
         python version : 3.8.5.final.0
       virtual packages : __cuda=11.2=0
                          __linux=5.11.0=0
                          __glibc=2.31=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /home/luis/anaconda3  (writable)
      conda av data dir : /home/luis/anaconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/luis/anaconda3/pkgs
                          /home/luis/.conda/pkgs
       envs directories : /home/luis/anaconda3/envs
                          /home/luis/.conda/envs
               platform : linux-64
             user-agent : conda/4.10.3 requests/2.24.0 CPython/3.8.5 Linux/5.11.0-40-generic ubuntu/20.04.3 glibc/2.31
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False

```

The solution provided in the previous issue doesn't seem to solve the problem. Any insights are welcome,
Thank you.