Model->forward() crash with complex models

username: BiggestZocker
cooked: <p>I try to load a trained resnet from pytroch to libtorch. My programm crashes on the line<br>
<code>auto output_pre = model.forward(inputs);</code><br>
There is no exception thrown, it just shuts down and all logs stop just there.<br>
It crashes wether I use my trained resnet converted with torch.jit.script or torch.jit.trace, or a newly initialized and then converted resnet. I also tried other nets included in pytorch, they all didnt work. When I create a very basic net and script or trace it, it works.</p>
<p>Here is a snippet of my code:</p>
<pre><code class="lang-auto">torch::DeviceType device_type = torch::kCUDA;
        obs_log(LOG_INFO, "Loading model...");
        auto model = torch::jit::load("D:/OBS Modding/Test-Output-Files/model.pt", device_type);
        
        model.to(device_type);
        model.eval();
        obs_log(LOG_INFO, "Formatting inputs");
        std::vector&lt;torch::jit::IValue&gt; inputs;
        at::Tensor test_input = torch::ones({1, 3, 270, 480},device_type);

        inputs.push_back(test_input);
        obs_log(LOG_INFO, "Input Device %d", test_input.device());
        obs_log(LOG_INFO, "CUDA availability %d", torch::cuda::is_available());
        
        
        using namespace std;
        obs_log(LOG_INFO, "Running model...");
        try{
            auto output_pre = model.forward(inputs); // &lt;--------- crash
            obs_log(LOG_INFO, "Converting output");
            auto output = output_pre.toTensor();
            obs_log(LOG_INFO, "Saving result");
            std::vector&lt;char&gt; f4 = torch::pickle_save(output);
            string pathname4("D:/OBS Modding/Test-Output-Files/Output.pt");
            ofstream textout4(pathname4.c_str(), ios::out | ios::binary);
            textout4.write((const char*)&amp;f4[0], f4.size());
            textout4.close();
        }catch (std::exception&amp; e)
        {
            obs_log(LOG_INFO, "Exception!");
            obs_log(LOG_INFO, e.what());
        }catch (int e)
        {
            obs_log(LOG_INFO, "Exception %d", e);
        }
        obs_log(LOG_INFO, "Done running model");
</code></pre>
<p>Here is the simple net with which there is no crash when loaded:</p>
<pre><code class="lang-auto">class SimpleNet(nn.Module):

    def __init__(self):
        super(SimpleNet, self).__init__()


    def forward(self, x):
        return x * 2
</code></pre>
<p>All nets and Tensors are loaded to my Cuda GPU.<br>
Here is my log, this program runs as an OBS plugin:</p>
<pre><code class="lang-auto">17:02:38.841: CPU Name: 12th Gen Intel(R) Core(TM) i5-12600KF
17:02:38.841: CPU Speed: 3686MHz
17:02:38.841: Physical Cores: 10, Logical Cores: 16
17:02:38.841: Physical Memory: 16193MB Total, 6390MB Free
17:02:38.841: Windows Version: 10.0 Build 19045 (release: 22H2; revision: 4529; 64-bit)
17:02:38.841: Running as administrator: false
17:02:38.841: Windows 10/11 Gaming Features:
17:02:38.841: 	Game DVR: On
17:02:38.841: 	Game Mode: Probably On (no reg key set)
17:02:38.842: Sec. Software Status:
17:02:38.842: 	Microsoft Defender Antivirus: enabled (AV)
17:02:38.843: 	Windows-Firewall: enabled (FW)
17:02:38.843: Current Date/Time: 2024-07-03, 17:02:38
17:02:38.843: Browser Hardware Acceleration: true
17:02:38.843: Hide OBS windows from screen capture: false
17:02:38.843: Qt Version: 6.6.2 (runtime), 6.6.2 (compiled)
17:02:38.843: Portable mode: false
17:02:39.050: OBS 30.1.2 (64-bit, windows)
17:02:39.050: ---------------------------------
17:02:39.101: ---------------------------------
17:02:39.101: audio settings reset:
17:02:39.101: 	samples per sec: 48000
17:02:39.101: 	speakers:        2
17:02:39.101: 	max buffering:   960 milliseconds
17:02:39.101: 	buffering type:  dynamically increasing
17:02:39.103: ---------------------------------
17:02:39.103: Initializing D3D11...
17:02:39.103: Available Video Adapters: 
17:02:39.104: 	Adapter 0: NVIDIA GeForce RTX 3070 Ti
17:02:39.104: 	  Dedicated VRAM: 8406433792 (7.8 GiB)
17:02:39.104: 	  Shared VRAM:    8490250240 (7.9 GiB)
17:02:39.104: 	  PCI ID:         10de:2482
17:02:39.104: 	  HAGS Status:    Disabled (Default: No, Driver status: Supported)
17:02:39.104: 	  Driver Version: 32.0.15.5585
17:02:39.105: 	  output 0:
17:02:39.105: 	    name=MSI MAG322CR
17:02:39.105: 	    pos={0, 0}
17:02:39.105: 	    size={1920, 1080}
17:02:39.105: 	    attached=true
17:02:39.105: 	    refresh=180
17:02:39.105: 	    bits_per_color=10
17:02:39.105: 	    space=RGB_FULL_G2084_NONE_P2020
17:02:39.105: 	    primaries=[r=(0.670898, 0.313477), g=(0.276367, 0.656250), b=(0.151367, 0.060547), wp=(0.313477, 0.329102)]
17:02:39.105: 	    relative_gamut_area=[709=1.239938, P3=0.914047, 2020=0.655767]
17:02:39.105: 	    sdr_white_nits=240
17:02:39.105: 	    nit_range=[min=0.061900, max=329.172607, max_full_frame=329.172607]
17:02:39.105: 	    dpi=96 (100%)
17:02:39.105: 	    id=\\?\DISPLAY#MSI3DA5#5&amp;14345bca&amp;0&amp;UID4355#{e6f07b5f-ee97-4a90-b076-33f57bf4eaa7}
17:02:39.105: 	    alt_id=\\.\DISPLAY1
17:02:39.105: 	  output 1:
17:02:39.105: 	    name=C27F390
17:02:39.105: 	    pos={1920, 0}
17:02:39.105: 	    size={1920, 1080}
17:02:39.105: 	    attached=true
17:02:39.105: 	    refresh=60
17:02:39.105: 	    bits_per_color=8
17:02:39.105: 	    space=RGB_FULL_G22_NONE_P709
17:02:39.105: 	    primaries=[r=(0.645508, 0.336914), g=(0.328125, 0.615234), b=(0.146484, 0.055664), wp=(0.313477, 0.329102)]
17:02:39.105: 	    relative_gamut_area=[709=1.018082, P3=0.750501, 2020=0.538434]
17:02:39.105: 	    sdr_white_nits=80
17:02:39.105: 	    nit_range=[min=0.500000, max=270.000000, max_full_frame=270.000000]
17:02:39.105: 	    dpi=96 (100%)
17:02:39.105: 	    id=\\?\DISPLAY#SAM0D32#5&amp;14345bca&amp;0&amp;UID4357#{e6f07b5f-ee97-4a90-b076-33f57bf4eaa7}
17:02:39.105: 	    alt_id=\\.\DISPLAY2
17:02:39.105: Loading up D3D11 on adapter NVIDIA GeForce RTX 3070 Ti (0)
17:02:39.192: D3D11 loaded successfully, feature level used: b000
17:02:39.192: DXGI increase maximum frame latency success
17:02:39.192: D3D11 GPU priority setup failed (not admin?)
17:02:39.240: ---------------------------------
17:02:39.240: video settings reset:
17:02:39.240: 	base resolution:   1920x1080
17:02:39.240: 	output resolution: 1280x720
17:02:39.240: 	downscale filter:  Bicubic
17:02:39.240: 	fps:               30/1
17:02:39.240: 	format:            NV12
17:02:39.240: 	YUV mode:          Rec. 709/Partial
17:02:39.240: NV12 texture support enabled
17:02:39.240: P010 texture support not available
17:02:39.241: Audio monitoring device:
17:02:39.241: 	name: Standard
17:02:39.241: 	id: default
17:02:39.241: ---------------------------------
17:02:39.242: No AJA devices found, skipping loading AJA UI plugin
17:02:39.242: Failed to initialize module 'aja-output-ui.dll'
17:02:39.243: No AJA devices found, skipping loading AJA plugin
17:02:39.243: Failed to initialize module 'aja.dll'
17:02:39.243: Skipping module '../../obs-plugins/64bit/asmjit.dll', not an OBS plugin
17:02:39.243: Skipping module '../../obs-plugins/64bit/c10.dll', not an OBS plugin
17:02:39.243: Skipping module '../../obs-plugins/64bit/c10_cuda.dll', not an OBS plugin
17:02:39.243: Skipping module '../../obs-plugins/64bit/caffe2_nvrtc.dll', not an OBS plugin
17:02:39.244: Skipping module '../../obs-plugins/64bit/chrome_elf.dll', not an OBS plugin
17:02:39.246: [CoreAudio encoder]: CoreAudio AAC encoder not installed on the system or couldn't be loaded
17:02:39.246: Skipping module '../../obs-plugins/64bit/cublas64_11.dll', not an OBS plugin
17:02:39.246: Skipping module '../../obs-plugins/64bit/cublasLt64_11.dll', not an OBS plugin
17:02:39.246: Skipping module '../../obs-plugins/64bit/cudart64_110.dll', not an OBS plugin
17:02:39.246: Skipping module '../../obs-plugins/64bit/cudnn64_8.dll', not an OBS plugin
17:02:39.246: Skipping module '../../obs-plugins/64bit/cudnn_adv_infer64_8.dll', not an OBS plugin
17:02:39.246: Skipping module '../../obs-plugins/64bit/cudnn_adv_train64_8.dll', not an OBS plugin
17:02:39.246: Skipping module '../../obs-plugins/64bit/cudnn_cnn_infer64_8.dll', not an OBS plugin
17:02:39.246: Skipping module '../../obs-plugins/64bit/cudnn_cnn_train64_8.dll', not an OBS plugin
17:02:39.246: Skipping module '../../obs-plugins/64bit/cudnn_ops_infer64_8.dll', not an OBS plugin
17:02:39.247: Skipping module '../../obs-plugins/64bit/cudnn_ops_train64_8.dll', not an OBS plugin
17:02:39.247: Skipping module '../../obs-plugins/64bit/cufft64_10.dll', not an OBS plugin
17:02:39.247: Skipping module '../../obs-plugins/64bit/cufftw64_10.dll', not an OBS plugin
17:02:39.247: Skipping module '../../obs-plugins/64bit/cupti64_2022.3.0.dll', not an OBS plugin
17:02:39.247: Skipping module '../../obs-plugins/64bit/curand64_10.dll', not an OBS plugin
17:02:39.247: Skipping module '../../obs-plugins/64bit/cusolver64_11.dll', not an OBS plugin
17:02:39.247: Skipping module '../../obs-plugins/64bit/cusolverMg64_11.dll', not an OBS plugin
17:02:39.247: Skipping module '../../obs-plugins/64bit/cusparse64_11.dll', not an OBS plugin
17:02:39.247: Failed to load 'en-US' text for module: 'decklink-captions.dll'
17:02:39.247: Failed to load 'en-US' text for module: 'decklink-output-ui.dll'
17:02:39.248: A DeckLink iterator could not be created.  The DeckLink drivers may not be installed
17:02:39.248: Failed to initialize module 'decklink.dll'
17:02:39.266: [AMF] Unable to load 'amfrt64.dll', error code 126.
17:02:39.267: [AMF] AMF Test failed due to one or more errors.
17:02:39.267: Failed to initialize module 'enc-amf.dll'
17:02:39.267: Skipping module '../../obs-plugins/64bit/fbgemm.dll', not an OBS plugin
17:02:39.267: Skipping module '../../obs-plugins/64bit/fbjni.dll', not an OBS plugin
17:02:39.272: Skipping module '../../obs-plugins/64bit/libcef.dll', not an OBS plugin
17:02:39.272: Skipping module '../../obs-plugins/64bit/libEGL.dll', not an OBS plugin
17:02:39.272: Skipping module '../../obs-plugins/64bit/libGLESv2.dll', not an OBS plugin
17:02:39.272: Skipping module '../../obs-plugins/64bit/libiomp5md.dll', not an OBS plugin
17:02:39.272: Skipping module '../../obs-plugins/64bit/libiompstubs5md.dll', not an OBS plugin
17:02:39.272: Skipping module '../../obs-plugins/64bit/mkl_core.1.dll', not an OBS plugin
17:02:39.272: Skipping module '../../obs-plugins/64bit/mkl_intel_thread.1.dll', not an OBS plugin
17:02:39.619: Failed to load 'en-US' text for module: 'my-test-plugin.dll'
17:02:39.619: [my-test-plugin] Test starting
17:02:39.619: [my-test-plugin] Test finished
17:02:39.619: [my-test-plugin] plugin loaded successfully (version 0.1.0)
17:02:39.619: Skipping module '../../obs-plugins/64bit/nvrtc-builtins64_118.dll', not an OBS plugin
17:02:39.619: Skipping module '../../obs-plugins/64bit/nvrtc64_112_0.dll', not an OBS plugin
17:02:39.619: Skipping module '../../obs-plugins/64bit/nvToolsExt64_1.dll', not an OBS plugin
17:02:39.631: [obs-browser]: Version 2.23.4
17:02:39.631: [obs-browser]: CEF Version 103.0.5060.134 (runtime), 103.0.0-5060-shared-textures.2594+gc69ad37+chromium-103.0.5060.134 (compiled)
17:02:39.634: NVENC supported
17:02:39.753: [NVENC] AV1 is not supported
17:02:39.754: [noise suppress]: NVIDIA denoiser disabled, redistributable not found or could not be loaded.
17:02:39.755: Failed to get NVVideoEffects.dll version info size
17:02:39.756: [NVIDIA VIDEO FX]: FX disabled, redistributable not found or could not be loaded.
17:02:39.783: [obs-websocket] [obs_module_load] you can haz websockets (Version: 5.4.2 | RPC Version: 1)
17:02:39.783: [obs-websocket] [obs_module_load] Qt version (compile-time): 6.6.2 | Qt version (run-time): 6.6.2
17:02:39.783: [obs-websocket] [obs_module_load] Linked ASIO Version: 102900
17:02:39.788: [obs-websocket] [obs_module_load] Module loaded.
17:02:39.789: Skipping module '../../obs-plugins/64bit/pytorch_jni.dll', not an OBS plugin
17:02:39.791: Skipping module '../../obs-plugins/64bit/torch.dll', not an OBS plugin
17:02:39.792: Skipping module '../../obs-plugins/64bit/torch_cpu.dll', not an OBS plugin
17:02:39.792: Skipping module '../../obs-plugins/64bit/torch_cuda.dll', not an OBS plugin
17:02:39.792: Skipping module '../../obs-plugins/64bit/torch_global_deps.dll', not an OBS plugin
17:02:39.793: Skipping module '../../obs-plugins/64bit/uv.dll', not an OBS plugin
17:02:39.793: [vlc-video]: Couldn't find VLC installation, VLC video source disabled
17:02:39.798: Skipping module '../../obs-plugins/64bit/zlibwapi.dll', not an OBS plugin
17:02:39.798: ---------------------------------
17:02:39.798:   Loaded Modules:
17:02:39.798:     win-wasapi.dll
17:02:39.798:     win-dshow.dll
17:02:39.798:     win-capture.dll
17:02:39.798:     vlc-video.dll
17:02:39.798:     text-freetype2.dll
17:02:39.798:     rtmp-services.dll
17:02:39.798:     obs-x264.dll
17:02:39.798:     obs-websocket.dll
17:02:39.798:     obs-webrtc.dll
17:02:39.798:     obs-vst.dll
17:02:39.798:     obs-transitions.dll
17:02:39.798:     obs-text.dll
17:02:39.798:     obs-qsv11.dll
17:02:39.798:     obs-outputs.dll
17:02:39.798:     obs-filters.dll
17:02:39.798:     obs-ffmpeg.dll
17:02:39.798:     obs-browser.dll
17:02:39.798:     my-test-plugin.dll
17:02:39.798:     image-source.dll
17:02:39.798:     frontend-tools.dll
17:02:39.798:     decklink-output-ui.dll
17:02:39.798:     decklink-captions.dll
17:02:39.798:     coreaudio-encoder.dll
17:02:39.798: ---------------------------------
17:02:39.798: QWidget::setTabOrder: 'first' and 'second' must be in the same window
17:02:39.798: ==== Startup complete ===============================================
17:02:39.852: All scene data cleared
17:02:39.852: ------------------------------------------------
17:02:39.853: [win-wasapi: 'Desktop-Audio'] update settings:
17:02:39.853: 	device id: default
17:02:39.853: 	use device timing: 1
17:02:39.854: [Loaded global audio device]: 'Desktop-Audio'
17:02:39.854: [win-wasapi: 'Mikrofon-/AUX-Audio'] update settings:
17:02:39.854: 	device id: default
17:02:39.854: 	use device timing: 0
17:02:39.854: [Loaded global audio device]: 'Mikrofon-/AUX-Audio'
17:02:39.856: [duplicator-monitor-capture: 'Game'] update settings:
17:02:39.856: 	display: C27F390 (1920x1080)
17:02:39.856: 	cursor: true
17:02:39.856: 	method: DXGI
17:02:39.856: 	id: \\?\DISPLAY#SAM0D32#5&amp;14345bca&amp;0&amp;UID4357#{e6f07b5f-ee97-4a90-b076-33f57bf4eaa7}
17:02:39.856: 	alt_id: \\.\DISPLAY2
17:02:39.856: 	setting_id: \\?\DISPLAY#SAM0D32#5&amp;14345bca&amp;0&amp;UID4357#{e6f07b5f-ee97-4a90-b076-33f57bf4eaa7}
17:02:39.856: 	force SDR: false
17:02:39.857: Switched to scene 'Szene'
17:02:39.858: ------------------------------------------------
17:02:39.858: Loaded scenes:
17:02:39.858: - scene 'Szene':
17:02:39.858:     - source: 'Game' (monitor_capture)
17:02:39.858: ------------------------------------------------
17:02:39.872: WASAPI: Device 'Headphones (High Definition Audio Device)' [48000 Hz] initialized (source: Desktop-Audio)
17:02:40.875: WASAPI: Device 'Headset Microphone (Oculus Virtual Audio Device)' [48000 Hz] initialized (source: Mikrofon-/AUX-Audio)
17:02:42.386: [my-test-plugin] monitor_capture
17:02:42.386: [my-test-plugin] output ready
17:02:42.386: [my-test-plugin] output ready
17:02:42.442: [my-test-plugin] Writing tensor information:
17:02:42.443: [my-test-plugin] Loading model...
17:02:42.933: [my-test-plugin] Formatting inputs
17:02:42.952: [my-test-plugin] Input Device 1
17:02:42.952: [my-test-plugin] CUDA availability 1
17:02:42.952: [my-test-plugin] Running model...

</code></pre>