LIBTORCH/C++ and Unreal Engine 4 Runtime error: Variable is optimized away and not avaible

username: Jorge_L_Zapotecatl
cooked: <p>I am trying to run my BERT model in Unreal engine with Libtorch. However, at runtime I get the following error when I run the forward function: Variable is optimized away and not avaible.</p>
<p>I use: LibTorch 1.10, Visual Studio C++ 2019 and Unreal Engine 4 version 4.26.2. Note: If I use only use LibTorch 1.10 and Visual Studio C ++ 2019 in release mode, my program run fine.</p>
<p>Please, could someone give me a suggestion on how to overcome this runtime error?</p>
<p>This is my function where ocurrs the error:</p>
<p>string predict(string question, string paragraph)<br>
{<br>
vector<a>torch::jit::IValue</a> inputs;<br>
vector input_ids;<br>
vector segment_ids;<br>
vector attention_mask;<br>
vector tokens;<br>
int start_index;<br>
int end_index;<br>
string answer;</p>
<pre><code>preprocess(question, paragraph, input_ids, segment_ids, attention_mask, tokens);

int size = input_ids.size() &gt; max_seq_length ? max_seq_length : input_ids.size();
torch::Tensor tensor_input_ids = torch::from_blob(input_ids.data(), c10::IntArrayRef{ 1, size }, torch::kCPU).clone().to(torch::kInt64);
torch::Tensor tensor_attention_mask = torch::from_blob(attention_mask.data(), c10::IntArrayRef{ 1, size }, torch::kCPU).clone().to(torch::kInt64);
torch::Tensor tensor_segment_ids = torch::from_blob(segment_ids.data(), c10::IntArrayRef{ 1, size }, torch::kCPU).clone().to(torch::kInt64);

inputs.push_back(tensor_input_ids);
inputs.push_back(tensor_attention_mask);
inputs.push_back(tensor_segment_ids);

auto outputs = bert.forward(inputs).toTuple();
torch::Tensor start_scores = bert.forward(inputs).toTuple()-&gt;elements()[0].toTensor();
torch::Tensor end_scores = bert.forward(inputs).toTuple()-&gt;elements()[1].toTensor();

start_index = torch::argmax(start_scores).item().toInt();
end_index = torch::argmax(end_scores).item().toInt();

for (int i = start_index; i &lt; end_index + 1; i++) {
	if (tokens[i].substr(0, 2) == "##") {
		answer = answer + tokens[i].substr(2, tokens[i].length() - 2);
	}
	else
		answer = answer + " " + tokens[i];
}

if (answer.empty() || answer.substr(1, 6) == "[CLS]") {
</code></pre>
<h1><a name="if-language-1" class="anchor" href="#if-language-1"></a>if LANGUAGE</h1>
<pre><code>	answer = "Unable to find the answer to your question.";
</code></pre>
<p><span class="hashtag-raw">#else</span><br>
answer = “Lo siento, no puedo responder a tu pregunta.”;<br>
<span class="hashtag-raw">#endif</span><br>
}</p>
<pre><code>return answer;
</code></pre>
<p>}</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/e/2e65bdb5d08cce4ba1173451b5ef1eb07aa3317d.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/2e65bdb5d08cce4ba1173451b5ef1eb07aa3317d" title="ForwardError"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/e/2e65bdb5d08cce4ba1173451b5ef1eb07aa3317d_2_690x276.jpeg" alt="ForwardError" data-base62-sha1="6CrWsiN8xLEoVIs7r9sRuONa8fX" width="690" height="276" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/e/2e65bdb5d08cce4ba1173451b5ef1eb07aa3317d_2_690x276.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/2/e/2e65bdb5d08cce4ba1173451b5ef1eb07aa3317d_2_1035x414.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/2/e/2e65bdb5d08cce4ba1173451b5ef1eb07aa3317d_2_1380x552.jpeg 2x" data-dominant-color="27292C"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">ForwardError</span><span class="informations">1920×769 108 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <p>I can’t decipher the image and if the error message is raised there.<br>
However, could you explain the error a bit more? Are you seeing this error only while trying to debug the application or while running the application?<br>
I would guess that some Release optimizations are used and the compiler removed unused variables or code snippets, which would yield undefined behavior.<br>
E.g. I’ve been debugging <a href="https://github.com/pytorch/pytorch/issues/66119">this issue</a> which was hit by violating the strict aliasing rule and the compiler was happy to optimize the function call away if <code>-fstrict-aliasing</code> was used.</p>

username: Jorge_L_Zapotecatl
cooked: <p>Thanks ptrblck,</p>
<ol>
<li>I run my project with the Local Windows Debugger</li>
<li>I get a runtime error in the forward function<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/c/ec0e9bc41700aa41af8f5f68de75921bc9d03fc6.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/ec0e9bc41700aa41af8f5f68de75921bc9d03fc6" title="02ForwardError"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/c/ec0e9bc41700aa41af8f5f68de75921bc9d03fc6_2_625x500.jpeg" alt="02ForwardError" data-base62-sha1="xGfYQV4zYiNt37JpUEWu82QD1RA" width="625" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/c/ec0e9bc41700aa41af8f5f68de75921bc9d03fc6_2_625x500.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/e/c/ec0e9bc41700aa41af8f5f68de75921bc9d03fc6_2_937x750.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/e/c/ec0e9bc41700aa41af8f5f68de75921bc9d03fc6_2_1250x1000.jpeg 2x" data-dominant-color="3C3C3E"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">02ForwardError</span><span class="informations">1716×1371 226 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></li>
<li>I press the button continue and I get another message</li>
<li>I press the button continue again and my program runs normally</li>
</ol>

username: Jorge_L_Zapotecatl
cooked: <p>Message of step 3:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/b/7b392b297410bbae303b501947f360b1e0a98f6e.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/7b392b297410bbae303b501947f360b1e0a98f6e" title="03ForwardError"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/b/7b392b297410bbae303b501947f360b1e0a98f6e_2_624x500.jpeg" alt="03ForwardError" data-base62-sha1="hA5d92lD6hrmWTFBOuj98g1Q2Ps" width="624" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/b/7b392b297410bbae303b501947f360b1e0a98f6e_2_624x500.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/7/b/7b392b297410bbae303b501947f360b1e0a98f6e_2_936x750.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/7/b/7b392b297410bbae303b501947f360b1e0a98f6e_2_1248x1000.jpeg 2x" data-dominant-color="3B3B3D"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">03ForwardError</span><span class="informations">1720×1377 230 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <p>In case you are concerned about the inability to display local variables: rebuild your application with debug symbols, as not all local variables are available during debugging.<br>
To disable compiler optimizations during the build and add debug symbols, rebuild PyToch with <code>DEBUG=1 python setup.py install</code>.<br>
I don’t know what the error message means.</p>

username: Jorge_L_Zapotecatl
cooked: <p>I overcome the error by placing at the top of the file a: <span class="hashtag-raw">#pragma</span> optimize (“”, off)</p>
<p>However, errors continue. Applying the Debug files of LibTorch and the DebuGame mode of Unreal Engine I realized that the error starts from loading of the model.</p>
<p>First, I ran the test by serializing a sample Libtorch model.</p>
<pre><code class="lang-auto">import torch
import torchvision

# An instance of your model.
model = torchvision.models.resnet18 ()

# An example input you would normally provide to your model's forward () method.
example = torch.rand (1, 3, 224, 224)

# Use torch.jit.trace to generate a torch.jit.ScriptModule via tracing.
traced_script_module = torch.jit.trace (model, example)

traced_script_module.save ("modelG.pt")
</code></pre>
<p>After, I load the model in the constructor of my application in unreal engine and I get an error with fopen.cpp when I try to load the model:</p>
<pre><code class="lang-auto">string name_model = "C:/Users/Jorge/PycharmProjects/CreateTestModels/modelG.pt";
torch::jit::script::Module module = torch::jit::load(name_model.c_str());
</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/0/f0b525cd48c1a84634751d17336e14a7e64d1ff8.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/f0b525cd48c1a84634751d17336e14a7e64d1ff8" title="01FopenError"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/0/f0b525cd48c1a84634751d17336e14a7e64d1ff8_2_644x499.jpeg" alt="01FopenError" data-base62-sha1="yloHoKxSfa9F5N3TqlMc5VHN4DK" width="644" height="499" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/0/f0b525cd48c1a84634751d17336e14a7e64d1ff8_2_644x499.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/f/0/f0b525cd48c1a84634751d17336e14a7e64d1ff8_2_966x748.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/f/0/f0b525cd48c1a84634751d17336e14a7e64d1ff8_2_1288x998.jpeg 2x" data-dominant-color="3D3D3E"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">01FopenError</span><span class="informations">1368×1062 117 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Please, could someone give me a suggestion on how to overcome this new runtime error?</p>

username: ptrblck
cooked: <p><code>Expression: file_name != nullptr</code> points to an error during the file loading, so you would have to check that the file name is valid and exists at the specified location.</p>

username: kostas
cooked: <p>Could that be similar to <a href="https://discuss.pytorch.org/t/model-cannot-be-loaded-with-torch-load/137502" class="inline-onebox">Model cannot be loaded with torch::jit::load</a>? The model exists but cannot be loaded at the moment in my test as well.</p>

username: Jorge_L_Zapotecatl
cooked: <p>Yes, it is similar. I have verified that the file exists and disabled the antivirus in that folder and the problem continues. I will keep trying.</p>

username: kostas
cooked: <p>Solved my issue. Check the build configuration, if you use torchlib in release, build the project also in release.</p>

username: gvecchio
cooked: <p>Hi, I’m facing the same issue and it’s not exactly clear to me how you solved it. I’m using torchlib release and building the project in Development from VS. Everything works fine except when trying to call the forward method. How have you solved it?</p>

username: Jorge_L_Zapotecatl
cooked: <p>Hi, sadly Libtorch and ue4 have several integration problems. Strictly, I have not been able to resolve the integration, but I can share my progress with you if it is helpful.</p>
<ol>
<li>
<p>I first tried to directly integrate Libtorch and ue4. I get the memory error in the forward function, however if I ignore the exception I can continue normally, this may be useful if you want to do a demo or proof of concept. For example, my demo is this: <a href="https://www.youtube.com/watch?v=RqELfDc8Rqc&amp;t=375s" class="inline-onebox" rel="noopener nofollow ugc">Question Answering: BERT(Bidirectional Encoder Representation with Transformers) with LibTorch &amp; UE4 - YouTube</a>. The link where I describe the steps is here: <a href="https://forums.unrealengine.com/t/how-to-properly-integrate-libtorch-pytorch-and-unreal-engine-4/263158" class="inline-onebox" rel="noopener nofollow ugc">How to properly integrate Libtorch (Pytorch) and Unreal Engine 4? - AI (Artificial Intelliegence) - Unreal Engine Forums</a></p>
</li>
<li>
<p>Later, I tried to do it through DLLs, but it didn’t work. However, with DLLs it is possible to catch the exception and learn a little more about the problem. The link where I describe the problem is here: <a href="https://github.com/pytorch/pytorch/issues/69425" class="inline-onebox" rel="noopener nofollow ugc">Integrate Libtorch into Unreal Engine 4: _ivalue_INTERNAL ASSERT FAILED · Issue #69425 · pytorch/pytorch · GitHub</a></p>
</li>
<li>
<p>The next step I will try is to try a more elaborate solution which is to integrate libtorch as a plugin. For example: <a href="https://github.com/NeuralVFX/basic-unreal-libtorch-plugin" class="inline-onebox" rel="noopener nofollow ugc">GitHub - NeuralVFX/basic-unreal-libtorch-plugin: A "Hello World" for running LibTorch inside Unreal Engine</a> or <a href="https://bestofcpp.com/repo/AntiAnti-SimplePyTorch" class="inline-onebox" rel="noopener nofollow ugc">UE4 Plugin to execute trained PyTorch modules | BestOfCpp</a>.</p>
</li>
</ol>
<p>In case of success, I will gladly share it here :).</p>

username: gvecchio
cooked: <p>Thank you! Sad to know this is an unresolved issue. I’m trying to integrate libtorch as a plugin yet having the same problem and can’t figure how it is solved in the two repo you mentioned. The issue is also present if trying to perform a forward over an nn Module, so I guess it’s related to how either torch or UE deals with memory…</p>