Data transfer between LibTorch C++ and Eigen

andrewssobral: <h2>Data transfer between LibTorch C++ and Eigen (Questions and Help)</h2>
<p>Hello all,<br>
I’m developing a Data Transfer Tools for C++ Linear Algebra Libraries, as you can see here:<br>
<aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://github.githubassets.com/favicon.ico" class="site-icon" width="16" height="16">
      <a href="https://github.com/andrewssobral/dtt" target="_blank" rel="nofollow noopener">GitHub</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars3.githubusercontent.com/u/483114?s=400&amp;amp;v=4" class="thumbnail onebox-avatar" width="60" height="60">

<h3><a href="https://github.com/andrewssobral/dtt" target="_blank" rel="nofollow noopener">andrewssobral/dtt</a></h3>

<p>DTT - Data Transfer Tools for C++ Linear Algebra Libraries (Eigen, Armadillo, OpenCV, ArrayFire, LibTorch). - andrewssobral/dtt</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
<br>
(considering bi-dimensional arrays or matrices)<br>
and I’m wondering if you can help me on the following code for data transfer between LibTorch and Eigen:</p>
<pre><code class="lang-auto">std::cout &lt;&lt; "Testing LibTorch to Eigen:" &lt;&lt; std::endl;
// LibTorch
torch::Device device(torch::cuda::is_available() ? torch::kCUDA : torch::kCPU);
torch::Tensor T = torch::rand({3, 3});
std::cout &lt;&lt; "LibTorch:" &lt;&lt; std::endl;
std::cout &lt;&lt; T &lt;&lt; std::endl;
// Eigen
float* data = T.data_ptr&lt;float&gt;();
Eigen::Map&lt;Eigen::MatrixXf&gt; E(data, T.size(0), T.size(1));
std::cout &lt;&lt; "EigenMat:\n" &lt;&lt; E &lt;&lt; std::endl;
// re-check after changes
E(0,0) = 0;
std::cout &lt;&lt; "EigenMat:\n" &lt;&lt; E &lt;&lt; std::endl;
std::cout &lt;&lt; "LibTorch:" &lt;&lt; std::endl;
std::cout &lt;&lt; T &lt;&lt; std::endl;
</code></pre>
<p>This is the output of the code:</p>
<pre><code class="lang-bash">--------------------------------------------------
Testing LibTorch to Eigen:

LibTorch:
 0.6232  0.5574  0.6925
 0.7996  0.9860  0.1471
 0.4431  0.5914  0.8361
[ Variable[CPUFloatType]{3,3} ]

EigenMat (after data transfer):
0.6232 0.7996 0.4431
0.5574  0.986 0.5914
0.6925 0.1471 0.8361

# Modifying EigenMat, set element at (0,0) = 0
EigenMat:
     0 0.7996 0.4431
0.5574  0.986 0.5914
0.6925 0.1471 0.8361

# Now, the LibTorch matrix was also modified (OK), but the rows and columns were switched.
LibTorch:
 0.0000  0.5574  0.6925
 0.7996  0.9860  0.1471
 0.4431  0.5914  0.8361
[ Variable[CPUFloatType]{3,3} ]
</code></pre>
<p>Do someone knows what’s happening ?<br>
There’s a better way to do that?</p>
<p>I need also to do the same for Armadillo, ArrayFire and OpenCV (cv::Mat).<br>
Thanks in advance!</p>

dambo: <p>What  great contribution.<br>
A question: I did not see any example of cv2::imread(). Is that because you had issues with ABI compatibility between OpenCV and libtorch?</p>
<p>Thanks,</p>

andrewssobral: <p>Thank you <a class="mention" href="/u/dambo">@dambo</a>,<br>
Yes, I am still working on it to add more functions and examples.<br>
Cheers,<br>
Andrews</p>

dambo: <p>I am working on this issue for several days now. I will share my docker if I am successful.</p>

dambo: <p>I shall upload the code to this repo once I am done:<br>
<aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://github.githubassets.com/favicon.ico" class="site-icon" width="32" height="32">
      <a href="https://github.com/QuantScientist/DarkTorch" target="_blank" rel="nofollow noopener">GitHub</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars1.githubusercontent.com/u/18743986?s=400&amp;amp;v=4" class="thumbnail onebox-avatar" width="102" height="102">

<h3><a href="https://github.com/QuantScientist/DarkTorch" target="_blank" rel="nofollow noopener">QuantScientist/DarkTorch</a></h3>

<p>A low-latency deep learning inference server using pytorch C++ frontend &amp; NVIDIA GPUs.  - QuantScientist/DarkTorch</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>

jmaceiras: <p>Hey,</p>
<p>I also needed to transfer data from Eigen to libtorch, so I inspected your example. For a square matrix, it seems that it is a transpose porblem, but if you try the same thing with a non-square matrix, the result is chaotic <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p>After a bit of thinking, I came out with this solution (I tested on square and non square matrix).</p>
<pre><code class="lang-auto">torch::Tensor eigenVectorToTorchTensor(VectorXd e){
    auto t = torch::rand({e.rows()});
    float* data = t.data_ptr&lt;float&gt;();

    Map&lt;VectorXf&gt; ef(data,t.size(0),1);
    ef = e.cast&lt;float&gt;();

    t.requires_grad_(true);
    return t;
}

torch::Tensor eigenMatrixToTorchTensor(MatrixXd e){
    auto t = torch::rand({e.cols(),e.rows()});
    float* data = t.data_ptr&lt;float&gt;();

    Map&lt;MatrixXf&gt; ef(data,t.size(1),t.size(0));
    ef = e.cast&lt;float&gt;();
    t.requires_grad_(true);
    return t.transpose(0,1);
}
</code></pre>
<p>NB: Here I am interested into converting an Eigen object to a torch object, but I think that you can easily get the reverse function from this code.</p>
<p>Hope it helps!<br>
A+</p>
<p>/jeremy</p>

999999999: <p>Hello all, I’d like to ask whether this issue has been resolved or not?</p>

quevivasbien: <p><a class="mention" href="/u/999999999">@999999999</a> the code from <a class="mention" href="/u/jmaceiras">@jmaceiras</a> works for me, and I think that’s a good solution. The one change I would make is to use <code>torch::empty</code> instead of <code>torch::rand</code> so you don’t have to compute random values that are immediately overwritten.</p>

kila_suelika: <p>The default memory order of Eigen is ColMajor which each column is stored sequentially. But libtorch use row major so that data stores row by row. You need to use<br>
<code>Matrix&lt;double, -1, -1, RowMajor&gt;</code><br>
to define a row major matrix.</p>