Writing Tensors to disk with c++, to be loaded by python

dshin: <p>I have a c++ process that constructs torch Tensor’s and writes their numerical values to datasets in an hdf5 file. It takes advantages of hdf5’s parallel write capabilities by using multiple threads, each of which writes to a part of the hdf5 file. After the file is written, a python process loads the hdf5 data and converts it into torch Tensor’s.</p>
<p>I am wondering if I can eliminate the hdf5 middleman here. When I google for phrases like “c++ pytorch write tensor disk”, I don’t see clear tutorials or examples of how to do this. Is this type of workflow supported? Where can I find clear documentation on this?</p>

Matej_Kompanek: <p>Save C++:</p>
<pre><code class="lang-auto">auto tensor = torch::randn({ 1, 5 });
auto pickled = torch::pickle_save(tensor);
std::ofstream fout("input.pt", std::ios::out | std::ios::binary);
fout.write(pickled.data(), pickled.size());
fout.close();
</code></pre>
<p>Load python:</p>
<pre><code class="lang-auto">tensor = torch.load('input.pt')
</code></pre>
<p>From this GitHub thread:</p><aside class="onebox githubissue" data-onebox-src="https://github.com/pytorch/pytorch/issues/20356">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/issues/20356" target="_blank" rel="noopener nofollow ugc">github.com/pytorch/pytorch</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewbox="0 0 14 16" aria-hidden="true"><path fill-rule="evenodd" d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/issues/20356" target="_blank" rel="noopener nofollow ugc">Load tensor from file in C++ fails </a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2019-05-10" data-time="09:27:51" data-timezone="UTC">09:27AM - 10 May 19 UTC</span>
      </div>

        <div class="date">
          closed <span class="discourse-local-date" data-format="ll" data-date="2019-11-23" data-time="08:06:19" data-timezone="UTC">08:06AM - 23 Nov 19 UTC</span>
        </div>

      <div class="user">
        <a href="https://github.com/jbcdnr" target="_blank" rel="noopener nofollow ugc">
          <img alt="jbcdnr" src="https://avatars.githubusercontent.com/u/6209990?v=4" class="onebox-avatar-inline" width="20" height="20">
          jbcdnr
        </a>
      </div>
    </div>

    <div class="labels">
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          high priority
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          oncall: jit
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          module: cpp
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          module: serialization
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          triaged
        </span>
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">## 🐛 Bug

I don't manage to import a tensor saved in PyTorch from C++. Any hel<span class="show-more-container"><a href="" rel="noopener" class="show-more">…</a></span><span class="excerpt hidden">p on this is welcome 😄 

## To Reproduce

Save a tensor to a file in python.

```python
&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.save(torch.tensor([1., 2., 3.]), "tensor.pt")
&gt;&gt;&gt; torch.load("tensor.pt")
tensor([1., 2., 3.])
```

Build this small C++ program

```cpp
#include &lt;torch/torch.h&gt;

int main() {
  torch::Tensor tensor;
  torch::load(tensor, "tensor.pt");
  std::cout &lt;&lt; tensor &lt;&lt; std::endl;
}
```

At runtime I get:

```
terminate called after throwing an instance of 'c10::Error'
  what():  [enforce fail at inline_container.cc:137] . PytorchStreamReader failed reading zip archive: failed finding central directory
frame #0: std::function&lt;std::string ()&gt;::operator()() const + 0x11 (0x7f1da515e691 in /___/libtorch/lib/libc10.so)
frame #1: c10::ThrowEnforceNotMet(char const*, int, char const*, std::string const&amp;, void const*) + 0x49 (0x7f1da515e4a9 in /__/libtorch/lib/libc10.so)
frame #2: caffe2::serialize::PyTorchStreamReader::valid(char const*) + 0x6b (0x7f1d9c69735b in /__/libtorch/lib/libcaffe2.so)
frame #3: caffe2::serialize::PyTorchStreamReader::init() + 0x9d (0x7f1d9c69912d in /___/libtorch/lib/libcaffe2.so)
frame #4: caffe2::serialize::PyTorchStreamReader::PyTorchStreamReader(std::unique_ptr&lt;caffe2::serialize::ReadAdapterInterface, std::default_delete&lt;caffe2::serialize::ReadAdapterInterface&gt; &gt;) + 0x3b (0x7f1d9c69ab8b in /__/libtorch/lib/libcaffe2.so)
frame #5: &lt;unknown function&gt; + 0xa9b5bf (0x7f1da5e125bf in /__/libtorch/lib/libtorch.so.1)
frame #6: torch::jit::load(std::unique_ptr&lt;caffe2::serialize::ReadAdapterInterface, std::default_delete&lt;caffe2::serialize::ReadAdapterInterface&gt; &gt;, c10::optional&lt;c10::Device&gt;, std::unordered_map&lt;std::string, std::string, std::hash&lt;std::string&gt;, std::equal_to&lt;std::string&gt;, std::allocator&lt;std::pair&lt;std::string const, std::string&gt; &gt; &gt;&amp;) + 0x9a (0x7f1da5e15d2a in /__/libtorch/lib/libtorch.so.1)
frame #7: torch::jit::load(std::string const&amp;, c10::optional&lt;c10::Device&gt;, std::unordered_map&lt;std::string, std::string, std::hash&lt;std::string&gt;, std::equal_to&lt;std::string&gt;, std::allocator&lt;std::pair&lt;std::string const, std::string&gt; &gt; &gt;&amp;) + 0x68 (0x7f1da5e15ec8 in /___/libtorch/lib/libtorch.so.1)
frame #8: torch::serialize::InputArchive::load_from(std::string const&amp;, c10::optional&lt;c10::Device&gt;) + 0x38 (0x7f1da604c058 in /___/libtorch/lib/libtorch.so.1)
frame #9: void torch::load&lt;at::Tensor, char const (&amp;) [10]&gt;(at::Tensor&amp;, char const (&amp;) [10]) + 0x86 (0x55a502f5fe97 in ./load)
frame #10: main + 0x37 (0x55a502f5fb51 in ./load)
frame #11: __libc_start_main + 0xe7 (0x7f1d9a338b97 in /lib/x86_64-linux-gnu/libc.so.6)
frame #12: _start + 0x2a (0x55a502f5f96a in ./load)

[1]    30788 abort      ./load
```
## Expected behavior

I would expect to be able to load a tensor from the PyTorch dump in the C++ program.

## Environment

PyTorch version: 1.1.0
Is debug build: No
CUDA used to build PyTorch: 9.0.176

OS: Ubuntu 18.04.2 LTS
GCC version: (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0
CMake version: version 3.10.2

Python version: 3.7
Is CUDA available: Yes
CUDA runtime version: 10.1.105
GPU models and configuration:
GPU 0: GeForce GTX TITAN X
GPU 1: GeForce GTX TITAN X
GPU 2: GeForce GTX TITAN X
GPU 3: GeForce GTX TITAN X
GPU 4: GeForce GTX TITAN X
GPU 5: GeForce GTX TITAN X
GPU 6: GeForce GTX TITAN X
GPU 7: GeForce GTX TITAN X

Nvidia driver version: 418.56
cuDNN version: Could not collect

Versions of relevant libraries:
[pip] msgpack-numpy==0.4.3.2
[pip] numpy==1.15.4
[pip] numpydoc==0.8.0
[pip] pytorch-nlp==0.4.1
[pip] torch==1.1.0
[pip] torchvision==0.2.2
[conda] blas                      1.0                         mkl
[conda] mkl                       2018.0.3                      1
[conda] mkl-service               1.1.2            py37h90e4bf4_5
[conda] mkl_fft                   1.0.6            py37h7dd41cf_0
[conda] mkl_random                1.0.1            py37h4414c95_1
[conda] pytorch                   1.1.0           py3.7_cuda9.0.176_cudnn7.5.1_0    pytorch
[conda] pytorch-nlp               0.4.1                    pypi_0    pypi
[conda] torchvision               0.2.2                      py_3    pytorch

## Additional context

I downloaded the C++ frontend from [https://download.pytorch.org/libtorch/nightly/cpu/libtorch-shared-with-deps-latest.zip](https://download.pytorch.org/libtorch/nightly/cpu/libtorch-shared-with-deps-latest.zip) yesterday.</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


dshin: <p>Thanks, that works.</p>
<p>I also found this resource which shows usage of the <code>torch::save()</code> API: <a href="https://shuye.dev/kb/AI-engineering/LibTorch/" class="inline-onebox" rel="noopener nofollow ugc">LibTorch (PyTorch C++ Frontend) | Ye Shu</a></p>