Repository: pytorch
Issue: DISABLE libtorch-2.0.0+cu117 destructor exception
Labels: oncall: jit

### 🐛 Describe the bug

I try to load a torchscript model in a class and do inference. When everything is done, the destructor doesn't work (probably the reason).
The minimum reproducible code is given below.**libtorch-2.0.0+cu117**
```
#include <iostream>
#include <torch/torch.h>
#include <torch/script.h>
#include <torch/cuda.h>


class MyDLClass
{
public:
    MyDLClass() {}
    ~MyDLClass() {}
    void init()
    {
        m_device = torch::cuda::is_available() ? torch::kCUDA : torch::kCPU;
        m_model = torch::jit::load(std::string(R"(E:\code\C++\DCDL\model\yolov5s.torchscript)"));
        m_model.to(m_device);
        m_model.eval();
    }
    void run()
    {
        auto tensor = torch::randn({ 1,3,640,640 }).to(m_device);
        auto r = m_model.forward({ tensor }).toTuple()->elements()[0].toTensor();
        std::cout << r.sizes() << std::endl;
    }


private:
    torch::DeviceType m_device = torch::kCPU;
    torch::jit::Module m_model;
};



int main()
{
    MyDLClass m;
    m.init();
    m.run();
    return 0;
}

```
At the end of the run, the IDE reports an error, but I don't have enough information to determine the cause of the error.
When I roll back to **libtorch-1.10.1+cu113** version everything works fine.
![libtorch-2 0 0+cu117-error](https://github.com/pytorch/pytorch/assets/44420952/922a7293-1415-4454-b937-74850d90758b)


### Versions

Visual Studio 2019

CUDA：
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Tue_May__3_19:00:59_Pacific_Daylight_Time_2022
Cuda compilation tools, release 11.7, V11.7.64
Build cuda_11.7.r11.7/compiler.31294372_0

Libtorch：
libtorch-2.0.0+cu117

PyTorch version: 1.12.0+cu116
Is debug build: False
CUDA used to build PyTorch: 11.6
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 10 专业版
GCC version: (x86_64-posix-seh, Built by strawberryperl.com project) 8.3.0
Clang version: Could not collect
CMake version: version 3.24.0
Libc version: N/A

Python version: 3.9.12 (main, Apr  4 2022, 05:22:27) [MSC v.1916 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.19045-SP0
Is CUDA available: True
CUDA runtime version: 11.7.64
CUDA_MODULE_LOADING set to: 
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3060 Laptop GPU
Nvidia driver version: 531.61
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:

12th Gen Intel(R) Core(TM) i7-12700H

Revision=

Versions of relevant libraries:
[pip3] numpy==1.22.4
[pip3] pytorch-lightning==1.6.5
[pip3] torch==1.12.0+cu116
[pip3] torchaudio==0.12.0+cu116
[pip3] torchfile==0.1.0
[pip3] torchmetrics==0.9.3
[pip3] torchtext==0.13.0
[pip3] torchvision==0.13.0+cu116
[conda] blas                      1.0                         mkl    http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
[conda] cudatoolkit               11.3.1               h59b6b97_2    http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
[conda] mkl                       2021.4.0           haa95532_640    http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
[conda] mkl-service               2.4.0            py39h2bbff1b_0    http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
[conda] mkl_fft                   1.3.1            py39h277e83a_0    http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
[conda] mkl_random                1.2.2            py39hf11a4ad_0    http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
[conda] numpy                     1.22.4                   pypi_0    pypi
[conda] numpy-base                1.22.3           py39hca35cd5_0    http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
[conda] pytorch-lightning         1.6.5                    pypi_0    pypi
[conda] pytorch-mutex             1.0                        cuda    http://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch
[conda] torch                     1.12.0+cu116             pypi_0    pypi
[conda] torchaudio                0.12.0+cu116             pypi_0    pypi
[conda] torchfile                 0.1.0                    pypi_0    pypi
[conda] torchmetrics              0.9.3                    pypi_0    pypi
[conda] torchtext                 0.13.0                   pypi_0    pypi
[conda] torchvision               0.13.0+cu116             pypi_0    pypi




cc @EikanWang @jgong5 @wenzhe-nrv @sanchitintel