Issue with TorchScript/LibTorch Compatibility - "schemas.empty() INTERNAL ASSERT FAILED"

GT_CD: <p>Hi everyone,</p>
<p>I’m working on performing inference with a Faster R-CNN (fasterrcnn_resnet50_fpn) model in a C++ environment using LibTorch. I followed this tutorial ( <a href="https://debuggercafe.com/a-simple-pipeline-to-train-pytorch-faster-rcnn-object-detection-model/" class="inline-onebox" rel="noopener nofollow ugc">A Simple Pipeline to Train PyTorch Faster RCNN Object Detection Model</a> ) for training the model, and I’ve tried converting the model using TorchScript to load it into C++.<br>
What I’ve done:</p>
<p>I modified the script to save the model with TorchScript in my custom_utils.py file as follows:<br>
python</p>
<p>class SaveBestModel:</p>
<pre><code>Class to save the best model while training. If the current epoch's 
validation loss is less than the previous least loss, then save the
model state.

def __init__(self, best_valid_loss=float('inf')):
    self.best_valid_loss = best_valid_loss

def __call__(self, current_valid_loss, epoch, model, optimizer):
    if current_valid_loss &lt; self.best_valid_loss:
        self.best_valid_loss = current_valid_loss
        print(f"\nBest validation loss: {self.best_valid_loss}")
        print(f"\nSaving best model for epoch: {epoch+1}\n")
        script_module = torch.jit.script(model)
        torch.jit.save(script_module, 'outputs/best_model_cplusplus.pt')
</code></pre>
<p>I used this script to train the model and save the TorchScript file as best_model_cplusplus.pt.</p>
<p>I’m trying to load this model in a C++ program using the following code:</p>
<p>cpp</p>
<pre><code>if (torch::cuda::is_available()) {
    std::cout &lt;&lt; "cuda available\n";
    torch::jit::script::Module module;

    try {
        module = torch::jit::load("C:/path/to/outputs/best_model_cplusplus.pt");
        std::cout &lt;&lt; "loading success\n";
    }
    catch (const c10::Error&amp; e) {
        std::cerr &lt;&lt; "\n---------------------------------------------\n" &lt;&lt; e.what() &lt;&lt; "\n---------------------------------------------\n" &lt;&lt; " : error loading the model\n";
    }
}
</code></pre>
<p>The Problem:</p>
<p>When trying to load the model in C++, I get the following exception message error:</p>
<pre><code class="lang-auto">!schemas.empty() INTERNAL ASSERT FAILED at "C:\\actions-runner\\_work\\pytorch\\pytorch\\builder\\windows\\pytorch\\torch\\csrc\\jit\\frontend\\schema_matching.cpp":581, please report a bug to PyTorch.
Exception raised from matchSchemas at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\jit\frontend\schema_matching.cpp:581 (most recent call first):
00007FFB9DCA1D2F00007FFB9DCA1C00 c10.dll!c10::SetGlobalRank [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB9DCA465600007FFB9DCA1C00 c10.dll!c10::SetGlobalRank [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB9DCA3A3600007FFB9DCA1C00 c10.dll!c10::SetGlobalRank [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB9DCA2B8D00007FFB9DCA1C00 c10.dll!c10::SetGlobalRank [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB9DCA54DB00007FFB9DCA1C00 c10.dll!c10::SetGlobalRank [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB9DCA1F3A00007FFB9DCA1C00 c10.dll!c10::SetGlobalRank [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB9DCA3C0C00007FFB9DCA1C00 c10.dll!c10::SetGlobalRank [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB9DCA31E200007FFB9DCA1C00 c10.dll!c10::SetGlobalRank [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB9DCA488B00007FFB9DCA1C00 c10.dll!c10::SetGlobalRank [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB9DCA0CA300007FFB9DCA0C70 c10.dll!c10::Error::Error [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB9DC9F55C00007FFB9DC9F480 c10.dll!c10::detail::torchCheckFail [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB9DBEF14200007FFB9DBEF110 c10.dll!c10::detail::torchInternalAssertFail [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30F49F5000007FFB30F49EB0 torch_cpu.dll!torch::jit::matchSchemas [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30F4B4C600007FFB30F4AB60 torch_cpu.dll!torch::jit::emitBuiltinCall [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30F66A6C00007FFB30F668D0 torch_cpu.dll!torch::jit::BuiltinFunction::call [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30EF989C00007FFB30ED5F40 torch_cpu.dll!torch::jit::ScriptTypeParser::operator= [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30F2175100007FFB30ED5F40 torch_cpu.dll!torch::jit::ScriptTypeParser::operator= [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30F1CDE900007FFB30ED5F40 torch_cpu.dll!torch::jit::ScriptTypeParser::operator= [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30F03CDE00007FFB30ED5F40 torch_cpu.dll!torch::jit::ScriptTypeParser::operator= [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30F1E56200007FFB30ED5F40 torch_cpu.dll!torch::jit::ScriptTypeParser::operator= [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30F0B2F900007FFB30ED5F40 torch_cpu.dll!torch::jit::ScriptTypeParser::operator= [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30ED171400007FFB30ED0750 torch_cpu.dll!torch::jit::ScriptTypeParser::ScriptTypeParser [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30E9F42D00007FFB30E9E630 torch_cpu.dll!torch::jit::meaningfulName [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30EC1F1E00007FFB30E9E630 torch_cpu.dll!torch::jit::meaningfulName [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30EA9E8E00007FFB30E9E630 torch_cpu.dll!torch::jit::meaningfulName [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30EA0F0A00007FFB30E9E630 torch_cpu.dll!torch::jit::meaningfulName [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30DF169200007FFB30DEF3F0 torch_cpu.dll!torch::jit::toGraphFunction [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30DEF1C000007FFB30DEF120 torch_cpu.dll!torch::jit::GraphFunction::ensure_defined [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30E9B99300007FFB30E9B300 torch_cpu.dll!torch::jit::CompilationUnit::define [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB316EF6F000007FFB316EAE70 torch_cpu.dll!torch::jit::readArchiveAndTensors [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB316EDD1000007FFB316EAE70 torch_cpu.dll!torch::jit::readArchiveAndTensors [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB316F5E8200007FFB316F5AA0 torch_cpu.dll!torch::jit::ClassNamespaceValue::attr [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30F216A500007FFB30ED5F40 torch_cpu.dll!torch::jit::ScriptTypeParser::operator= [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30EF94AB00007FFB30ED5F40 torch_cpu.dll!torch::jit::ScriptTypeParser::operator= [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30F2175100007FFB30ED5F40 torch_cpu.dll!torch::jit::ScriptTypeParser::operator= [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30F1CDE900007FFB30ED5F40 torch_cpu.dll!torch::jit::ScriptTypeParser::operator= [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30F03CDE00007FFB30ED5F40 torch_cpu.dll!torch::jit::ScriptTypeParser::operator= [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30F1E56200007FFB30ED5F40 torch_cpu.dll!torch::jit::ScriptTypeParser::operator= [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30F1E2B100007FFB30ED5F40 torch_cpu.dll!torch::jit::ScriptTypeParser::operator= [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30EE00AC00007FFB30ED5F40 torch_cpu.dll!torch::jit::ScriptTypeParser::operator= [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30EC209400007FFB30E9E630 torch_cpu.dll!torch::jit::meaningfulName [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30EAA00400007FFB30E9E630 torch_cpu.dll!torch::jit::meaningfulName [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30EE62FB00007FFB30ED5F40 torch_cpu.dll!torch::jit::ScriptTypeParser::operator= [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB29F1005F00007FFB29F0D9E0 torch_cpu.dll!c10::OperatorHandle::operator!= [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30F16FF400007FFB30ED5F40 torch_cpu.dll!torch::jit::ScriptTypeParser::operator= [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30F0F4FD00007FFB30ED5F40 torch_cpu.dll!torch::jit::ScriptTypeParser::operator= [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30F0F6C300007FFB30ED5F40 torch_cpu.dll!torch::jit::ScriptTypeParser::operator= [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30F1E4EC00007FFB30ED5F40 torch_cpu.dll!torch::jit::ScriptTypeParser::operator= [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30F0B2F900007FFB30ED5F40 torch_cpu.dll!torch::jit::ScriptTypeParser::operator= [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30ED171400007FFB30ED0750 torch_cpu.dll!torch::jit::ScriptTypeParser::ScriptTypeParser [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30E9F42D00007FFB30E9E630 torch_cpu.dll!torch::jit::meaningfulName [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30EC1F1E00007FFB30E9E630 torch_cpu.dll!torch::jit::meaningfulName [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30EA9E8E00007FFB30E9E630 torch_cpu.dll!torch::jit::meaningfulName [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30EA0F0A00007FFB30E9E630 torch_cpu.dll!torch::jit::meaningfulName [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30DF169200007FFB30DEF3F0 torch_cpu.dll!torch::jit::toGraphFunction [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30DEF1C000007FFB30DEF120 torch_cpu.dll!torch::jit::GraphFunction::ensure_defined [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30E9B99300007FFB30E9B300 torch_cpu.dll!torch::jit::CompilationUnit::define [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB316EF6F000007FFB316EAE70 torch_cpu.dll!torch::jit::readArchiveAndTensors [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB316EDD1000007FFB316EAE70 torch_cpu.dll!torch::jit::readArchiveAndTensors [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB316F5E8200007FFB316F5AA0 torch_cpu.dll!torch::jit::ClassNamespaceValue::attr [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30F216A500007FFB30ED5F40 torch_cpu.dll!torch::jit::ScriptTypeParser::operator= [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30F1CDE900007FFB30ED5F40 torch_cpu.dll!torch::jit::ScriptTypeParser::operator= [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30F03CDE00007FFB30ED5F40 torch_cpu.dll!torch::jit::ScriptTypeParser::operator= [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
00007FFB30F1E56200007FFB30ED5F40 torch_cpu.dll!torch::jit::ScriptTypeParser::operator= [&lt;unknown file&gt; @ &lt;unknown line number&gt;]
</code></pre>
<p>Configuration Details:</p>
<ul>
<li>PyTorch version: 2.4.1+cu124</li>
<li>LibTorch version: 2.4.1</li>
<li>CUDA version: 12.4</li>
<li>Platform: Windows</li>
</ul>
<p>What I’ve tried:</p>
<ul>
<li>I ensured that PyTorch and LibTorch are on the same version (2.4.1).</li>
<li>I followed some troubleshooting tips related to compatibility issues, but nothing seems to work.</li>
</ul>
<p>Does anyone have any suggestions or solutions for fixing this schemas.empty() error? I’m wondering if there are any specific considerations when converting the fasterrcnn_resnet50_fpn model to TorchScript.</p>
<p>Thanks in advance for your help!</p>