Can libtorch load model from memory?

username: dancedpipi
cooked: <p>I hava an encrypt model.pt.<br>
I want decrypt model in memory and use libtorch load it while do inference.<br>
Is there any API is avaliable?</p>

username: cyanM
cooked: <p>This is the same as loading from a file, you could use torch::jit::load to load model with std::istream</p>

username: Matej_Kompanek
cooked: <p>You could try something like this -&gt;</p>
<pre><code class="lang-auto">struct MLP : public torch::nn::Module
{
	torch::nn::Linear fc1{ nullptr }, fc2{ nullptr }, fc3{ nullptr };

	MLP()
	{
		fc1 = register_module("fc1", torch::nn::Linear(2, 10));
		fc2 = register_module("fc2", torch::nn::Linear(10, 10));
		fc3 = register_module("fc3", torch::nn::Linear(10, 1));
	}

	torch::Tensor forward(torch::Tensor x)
	{
		x = torch::leaky_relu(fc1-&gt;forward(x));
		x = torch::leaky_relu(fc2-&gt;forward(x));
		return torch::sigmoid(fc3-&gt;forward(x));
	}

	std::vector&lt;char&gt; SaveMemory()
	{
		std::ostringstream oss;
		torch::serialize::OutputArchive archive;

		this-&gt;save(archive);
		archive.save_to(oss);

		std::string s = oss.str();

		const char* ptr = s.c_str();
		size_t length = s.size();
		std::vector&lt;char&gt; retval(ptr, ptr + length);
		
		return retval;
	}
	
	void LoadMemory(std::vector&lt;char&gt;&amp; data, c10::Device device)
	{
		std::istringstream iss(std::string(data.begin(), data.end()));
		
		torch::serialize::InputArchive archive;
		archive.load_from(iss, device);
		this-&gt;load(archive);
	}
};

void main()
{
	try
	{
		auto testInput = torch::randn({ 1, 2 });

		std::vector&lt;char&gt; savedState;

		{
			auto net = std::make_shared&lt;MLP&gt;();
			auto output1 = net-&gt;forward(testInput);
			std::cout &lt;&lt; "Output before save: " &lt;&lt; output1 &lt;&lt; std::endl;
			savedState = net-&gt;SaveMemory();
		}

		{
			auto net = std::make_shared&lt;MLP&gt;();
			auto output1 = net-&gt;forward(testInput);
			std::cout &lt;&lt; "Output before load: " &lt;&lt; output1 &lt;&lt; std::endl;

			net-&gt;LoadMemory(savedState, c10::Device(c10::DeviceType::CPU));

			auto output2 = net-&gt;forward(testInput);
			std::cout &lt;&lt; "Output after load: " &lt;&lt; output2 &lt;&lt; std::endl;
		}
	}
	catch (std::runtime_error&amp; e)
	{
		std::cout &lt;&lt; e.what() &lt;&lt; std::endl;
	}
	catch (const c10::Error&amp; e)
	{
		std::cout &lt;&lt; e.msg() &lt;&lt; std::endl;
	}

	system("PAUSE");
}

</code></pre>

username: tom
cooked: <p>In a similar vein as <a class="mention" href="/u/cyanm">@cyanM</a>, passing <code>std::stringstream</code>s would be a canonical way to pass “non-files” to libtorch.</p>
<p>Best regards</p>
<p>Thomas</p>

username: Carol_Ye_Liu
cooked: <p>This is super helpful! Thank you!</p>

username: Andrew_Edmondson
cooked: <p>Thanks, <a class="mention" href="/u/matej_kompanek">@Matej_Kompanek</a>, this answer keeps on giving <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"> - saved me a big headache!</p>
<p>I think it can be simplified to remove the vector data and just save it as a std::string right? Because<br>
<code>std::istringstream </code>  takes a string, so no need to create the vector in SaveMemory, just return s directly and store everything as a std::string. Or am I misunderstanding something about this?</p>

username: Matej_Kompanek
cooked: <p><a class="mention" href="/u/andrew_edmondson">@Andrew_Edmondson</a>, haven’t tried but yeah, should work</p>