Message to admins

dambo: <p>Hello,<br>
Being one of the early adopters and advocated of PyTorch, I have the strange feeling that Libtorch is, to some extent neglected. This is evident in the level of activity and chatter in the C++ sub-forum.<br>
I find that it is very hard to get answers to the simplest questions, such as: how to work in <strong>batches</strong> during <strong>inference</strong> / and or <strong>training</strong> for <strong>ImageNet</strong> like datasets? How to utilise <strong>multiple</strong> GPU’s during training and <strong>inference</strong> of traced modules? Where can I find an example for loading  an <strong>ImageNet</strong> like dataset NOT Mnist? and much more.</p>
<p>All these are trivial in PyTorch/ Python and seem almost impossible in Libtorch.</p>
<p>References to my questions:</p><aside class="quote" data-post="1" data-topic="48125">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dambo/48/29088_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/automatic-parallelization-of-models-onto-multiple-gpus/48125">"Automatic parallelization of models onto multiple GPUs"</a> <a class="badge-category__wrapper " href="/c/c/11"><span data-category-id="11" data-drop-close="true" class="badge-category " title="Topics related to the C++ Frontend, C++ API or C++ Extensions"><span class="badge-category__name">C++</span></span></a>
  </div>
  <blockquote>
    Hello, 
I have a traced model which i use in C++. 
The documentation clearly states: “Automatic parallelization of models onto multiple GPUs like torch.nn.parallel.DataParallel”. 
However I do not see this happening on a multi-gpu device. How do i explicitly force the auto module = torch::jit::load(s_model_name, torch::kCUDA); module to be defined like it would be in pure PyTorch? 
Thanks.
  </blockquote>
</aside>

<aside class="quote quote-modified" data-post="1" data-topic="48039">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dambo/48/29088_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/batch-as-input-to-module-forward-inputs-totensor/48039">Batch as input to module-&gt;forward({inputs}).toTensor()</a> <a class="badge-category__wrapper " href="/c/c/11"><span data-category-id="11" data-drop-close="true" class="badge-category " title="Topics related to the C++ Frontend, C++ API or C++ Extensions"><span class="badge-category__name">C++</span></span></a>
  </div>
  <blockquote>
    Hello, 
I am trying to work in batches while reading frames using CV2 from a video like so. I first read 8 frames, batch them into std::vector&lt;torch::jit::IValue&gt; and then run them through module-&gt;forward({inputs}).toTensor();. However, if the batch size is larger than 1, it crashes. I guess there is some reshaping involved to add the batch dimension I am just not sure how to do it in C++. 
    int batch_size =1;
    int b_counter=0;

    std::vector&lt;torch::jit::IValue&gt; inputs; //Accumulates the…
  </blockquote>
</aside>

<aside class="quote quote-modified" data-post="3" data-topic="34221">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dambo/48/29088_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/libtorch-how-to-use-torch-datasets-for-custom-dataset/34221/3">(libtorch) How to use torch::data::datasets for custom dataset?</a> <a class="badge-category__wrapper " href="/c/c/11"><span data-category-id="11" data-drop-close="true" class="badge-category " title="Topics related to the C++ Frontend, C++ API or C++ Extensions"><span class="badge-category__name">C++</span></span></a>
  </div>
  <blockquote>
    Hello, 
Were you able to come up with a working example that mimics that of the PyTorch-based dataset for reading images? 
class GenericDataset(torch.utils.data.Dataset):
  def __init__(self, labels, root_dir, subset=False, transform=None):
    self.labels = labels
    self.root_dir = root_dir
    self.transform = transform

  def __len__(self):
    return len(self.labels)

  def __getitem__(self, idx):
    img_name = self.labels.iloc[idx, 0]  # file name
    fullname = join(self.root_dir, img_n…
  </blockquote>
</aside>

<p>Is there a formal policy that allocates more resources to PyTorch related questions rather than Libtorch? If not, what is the explanation for the lack of documentation and infrequent answers in the forums?</p>
<p>Many thanks,</p>

albanD: <p>Hi,</p>
<p>I think the simple answer is that almost all people answering questions are doing it in their free time and answer mostly the questions they are comfortable with. And I don’t think any of the very active people here actually use libtorch.<br>
At least this is the reason why I don’t usually answer such question unless it is one of the few things i touched in libtorch.</p>
<p>Maybe <a class="mention" href="/u/smth">@smth</a> can give a higher level answer on this? But the day to day answer I’m sure is the one above.</p>

dambo: <p>Sure, this makes a lot of sense, however the severe lack of documentation and examples make it very hard to progress with projects based on C++. For instance, the lack of an example for such a basic flow, to load a batch of images using an imagenet style dataloader (or without) during inference and then processing the tensors that are returned.</p>
<p>BR</p>

gchanan: <p><a class="mention" href="/u/dambo">@dambo</a>:</p>
<p>Thanks for bringing up this issue.  I think your characterization that LibTorch is neglected, at least in comparison to areas like the Python Frontend and TorchScript, is very fair.  We certainly don’t have a policy that allocates more resources to some questions over others; as <a class="mention" href="/u/alband">@albanD</a> suggests, we rely on the expertise of the community and our internal developers in order to answer questions, and that naturally (but unfortunately) skews away from LibTorch currently.</p>
<p>Let me talk about our current investment in LibTorch / C++ API and what we plan to change.  By far our largest current investment in this area is with the Variable/Tensor merge in C++ – you may have seen <a href="https://github.com/pytorch/pytorch/pull/17072" rel="nofollow noopener">https://github.com/pytorch/pytorch/pull/17072</a>, for example.  We decided to invest in this way because: 1) it was the largest complaint we heard about the C++ API, particularly from our internal power users, 2) it unblocks performance gains that will affect all of PyTorch (python, C++, JIT).  However, this has unfortunately crowded out other investments in LibTorch/ C++, as you noted.</p>
<p>Our plan to remedy this is two-fold:</p>
<ol>
<li>The Variable/Tensor merge project is a one-time cost, so we will be able to naturally shift attention to LibTorch / C++ more generally once it is complete (should be soon).</li>
<li>We are also ramping up more people on LibTorch / C++ internally, so we can provide better and broader coverage of issues.</li>
</ol>
<p>Again, thanks for bringing this up, and hopefully this gives you some insight into our process and investments in this area.</p>

mhubii: <p>Hi <a class="mention" href="/u/dambo">@dambo</a>,</p>
<p>I just answered your second question, on how to forward batches.</p>
<p>Concerning your third questoin, you can find an example of a custom dataset <a href="https://discuss.pytorch.org/t/libtorch-how-to-use-torch-datasets-for-custom-dataset/34221/2">here</a> and I will try to include an explanation for how to load images with OpenCV.</p>
<p>Cheers</p>
<p>Martin</p>

Carsten_Ditzel: <p>I believe things went a little bit down hill since Peter Goldsborough went off boat…</p>

mhubii: <p>where did he go to <a class="mention" href="/u/carsten_ditzel">@Carsten_Ditzel</a> ?</p>
<p>now <a href="https://github.com/mhubii/libtorch_custom_dataset.git" rel="nofollow noopener">here</a> is a little classifier that classifies apples and bananas with a custom data loader. I will polish the readme soon so people will know how to use it.</p>

Carsten_Ditzel: <p>I thought he turned to other things lately? At least thats what I read on his blog</p>

mhubii: <p>I honestly have no idea</p>

tom: <p>So <a class="mention" href="/u/goldsborough">@goldsborough</a> <a href="http://www.goldsborough.me/distributed-systems/2019/04/09/21-16-52-a_look_at_mesos/" rel="nofollow noopener">moved to the Stream Proceccing</a> team. And boy, I do miss his awesome hacking with virtually instant bug-fixing and feature-implementations!</p>
<p>But then, we can help the C++ lib a lot building on his (and the others) great conceptual work by fleshing out the details - e.g. bidirectional RNNs, anyone?<br>
Similarly the infrastructure for C++ docs is all there, so anyone can contribute.</p>
<p>I’m can’t fill in for even half a Peter but I guess I know a thing or two about <a href="https://lernapparat.de/fixing-your-first-pytorch-bug/" rel="nofollow noopener">hacking on PyTorch</a> and I’d be happy to (lightly) mentor one or two people willing to work on things. Do tag me on a issue/PR or send mail.</p>
<p>Best regards</p>
<p>Thomas</p>

dambo: <p>Thank you so much for this.</p>