Torch::jit::load with istream

username: skaldesh
cooked: <p>Hello,<br>
I have successfully traced a model in Python and wanted to load it now in my C++ app.<br>
I did so successfully using <code>torch::jit::load(filepath, device)</code>.</p>
<p>However, I need to load the <code>.pt</code> model file now from a <code>const char* modelScript</code>. Its size is given as <code>const size_t modelScriptSize</code>.</p>
<p>For this purpose, I created a helper class that subclasses <code>std::streambuf</code> and wraps the buffer. With it, I am creating a <code>std::istream</code> and supply that to <code>torch::jit::load(is, device)</code>.</p>
<pre><code class="lang-cpp">utils::MemReader mr(modelScript, modelScriptSize);
std::istream is(&amp;mr);
mod_ = torch::jit::load(is, device_);
</code></pre>
<p>I am receiving an error:</p>
<pre><code class="lang-auto">istream reader failed: checking archive.
Exception raised from validate at /tmp/pytorch/pytorch/caffe2/serialize/istream_adapter.cc:32 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;) + 0x6c (0x7f1d1b9447ac in /usr/local/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; const&amp;) + 0xfa (0x7f1d1b910866 in /usr/local/lib/libc10.so)
frame #2: caffe2::serialize::IStreamAdapter::validate(char const*) const + 0x17b (0x7f1d1e22dbeb in /usr/local/lib/libtorch_cpu.so)
frame #3: caffe2::serialize::IStreamAdapter::read(unsigned long, void*, unsigned long, char const*) const + 0x41 (0x7f1d1e22dd21 in /usr/local/lib/libtorch_cpu.so)
frame #4: &lt;unknown function&gt; + 0x3f9c02b (0x7f1d1f91c02b in /usr/local/lib/libtorch_cpu.so)
frame #5: torch::jit::load(std::shared_ptr&lt;caffe2::serialize::ReadAdapterInterface&gt;, c10::optional&lt;c10::Device&gt;, std::unordered_map&lt;std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;, std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;, std::hash&lt;std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; &gt;, std::equal_to&lt;std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; &gt;, std::allocator&lt;std::pair&lt;std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; const, std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; &gt; &gt; &gt;&amp;) + 0x6c (0x7f1d1f918a5c in /usr/local/lib/libtorch_cpu.so)
frame #6: torch::jit::load(std::istream&amp;, c10::optional&lt;c10::Device&gt;, std::unordered_map&lt;std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;, std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;, std::hash&lt;std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; &gt;, std::equal_to&lt;std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; &gt;, std::allocator&lt;std::pair&lt;std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; const, std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; &gt; &gt; &gt;&amp;) + 0xc2 (0x7f1d1f91aaa2 in /usr/local/lib/libtorch_cpu.so)
frame #7: torch::jit::load(std::istream&amp;, c10::optional&lt;c10::Device&gt;) + 0x6a (0x7f1d1f91ab8a in /usr/local/lib/libtorch_cpu.so)
</code></pre>
<p>I verified that my stream outputs the exact same content as the <code>.pt</code> file, by writing my istream into a file byte by byte and then comparing them via <code>md5sum</code> on linux. It yielded the same hash.</p>
<p>Anybody able to spot my error here?</p>
<p>Here is my code for the <code>MemReader</code> helper class:</p>
<pre><code class="lang-cpp">class MemReader : public std::streambuf {
public:
    MemReader(const char* data, size_t size);
 
private:
    int_type underflow();
    int_type uflow();
    int_type pbackfail(int_type ch);
    std::streamsize showmanyc();
 
    const char* const begin_;
    const char* const end_;
    const char* current_;
};

MemReader::MemReader(const char* data, size_t size) : 
    begin_(data), 
    end_(data + size), 
    current_(data) 
{}
 
MemReader::int_type MemReader::underflow() {
    if (current_ == end_) {
        return traits_type::eof();
    }
    return traits_type::to_int_type(*current_);
}
 
MemReader::int_type MemReader::uflow() {
    if (current_ == end_) {
        return traits_type::eof();
    }
    return traits_type::to_int_type(*current_++);
}
 
MemReader::int_type MemReader::pbackfail(int_type ch) {
    if (current_ == begin_ || (ch != traits_type::eof() &amp;&amp; ch != current_[-1])) {
        return traits_type::eof();
    }
    return traits_type::to_int_type(*--current_);
}
 
std::streamsize MemReader::showmanyc() {
    return end_ - current_;
}
</code></pre>

username: skaldesh
cooked: <p>I replaced my MemReader now with a <code>std::strstreambuf</code>. Though they produce exactly the same output on a high level, some function may have been missing from my custom class? I don’t know, these stream classes are too much C++ magic for me.</p>