akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
TorchScript vs torch2trt for using models in C++

ar13pit: <p>Hi,</p>
<p>I would like to serialize my PyTorch model to use it with my C++ application in production. I came across the use of TorchScript (as documented in PyTorch) and torch2trt.</p>
<p>Is there a difference in the serialization achieved by the two?</p>

tom: <p>Currently,</p>
<ul>
<li>TorchScript is more general - as in it supports more operations,</li>
<li>you probably will not beat TensorRT for those things it supports.</li>
</ul>
<p>Best regards</p>
<p>Thomas</p>

ar13pit: <p><a class="mention" href="/u/tom">@tom</a> Thanks for your reply. Could you elaborate about the generality of TorchScript? And if TensorRT is better at the things it supports, will it be a good idea to have a mixture of the two working?</p>
<p>Regards,<br>
Arpit</p>

tom: <p>Well, TorchScript can represent arbitrary PyTorch operations, including custom operators.<br>
TensorRT does not support all of them.</p>
<p>Whether mixing the two is desirable depends on your parameters. It is a bit more effort, but it would seem that torch2trt lends itself to that.</p>
Python2 TorchScript type annotation tuple vs Tuple

dashesy: <p>I have a torch script that accepts a tuple[Tensor],</p>
<pre data-code-wrap="python"><code class="lang-python">@torch.jit.script
def smt_pred(confs, child, child_sizes, threshold, obj,
             height, width, a):
    # type: (Tuple[Tensor],Tuple[Tensor],Tuple[Tensor],Tensor,Tensor,Tensor,Tensor,Tensor) -&gt; Tuple[Tensor,Tensor]

</code></pre>
<p>When I call the code in Python2 I get the error</p>
<blockquote>
<p>RuntimeError: smt_pred() Expected a value of type ‘Tuple[Tensor]’ for argument ‘confs’ but instead found type ‘tuple’.</p>
</blockquote>
<p><code>tuple</code> is same as <code>Tuple</code> and I could not find how to annotate confs to be tuple of tensors. Is it possible in Python2?</p>
<hr>
<p>Changing <code>Tuple</code> to <code>List</code> works in Python2, so this is only about <code>Tuple</code></p>

dashesy: <p>Added an issue <a href="https://github.com/pytorch/pytorch/issues/25329" rel="nofollow noopener">here</a> because <code>List</code> works (and I just have to pass <code>list(confs)</code> as a workaround)</p>

driazati: <p>Copied from the issue:</p>
<p>I’m guessing that you’re passing in a tuple that doesn’t look like <code>(torch.ones(2, 2),)</code>. For a <code>Tuple</code> the type needs to be fixed and completely specified and the elements can be different types. In TorchScript, <code>a</code>, <code>b</code>, and <code>c</code> below are all different types</p>
<pre><code class="lang-python"># type is `Tuple[int, int, int]
a = (1, 2, 3)

# type is `Tuple[str, int, int]
b = ('hi', 2, 3)

# type is `Tuple[int]`
c = (2,)
</code></pre>
<p>For lists, the element types must all be the same, which is why it can be any length but you only need to specify <code>List[Tensor]</code> or <code>List[int]</code>. You can read more about it <a href="https://pytorch.org/docs/master/jit.html#supported-type" rel="nofollow noopener">here</a>.</p>
Use pretrained model's weight with gray scale image?

Rosa: <p>Since Pytorch’s pretrained imagenet models are finetuned for RGB images, is it possible to work around them with grayscale images?</p>
<p>One possible solution is repeating grayscale image over three channels or convert them to RGB to work with existing situation.</p>
<p>Is it possible to some how take the mean of the three channels weight and tweak resnet to accept the mean weights and train using grayscale images ?<br>
Any working example would be great.<br>
Thanks in advance.</p>

pkraison: <p>Hi <a class="mention" href="/u/rosa">@Rosa</a>, models are trained with imagenet work well with rgb images but in your case you can either choose to change the first conv layer according to <a href="https://discuss.pytorch.org/t/modify-resnet-or-vgg-for-single-channel-grayscale/22762">this</a> by <a class="mention" href="/u/juanfmontesinos">@JuanFMontesinos</a>  or you can easily use the repitation likethis in numpy:<br>
rgb = np.repeat(input[…,np.newaxis],3,-1)</p>

Rosa: <p>Thanks for your answer. I have already seen this post. The solution provided here mentions that:<br>
<code>you would be training from the scratch.</code><br>
Which beats the purpose of using pretrained model, If I am not wrong.  Does this approach make sure that model using the pretrained models weights ?<br>
And the 2nd solution of making grayscale to rgb , I mentioned already. If the information is same why I would like to use 3x computational resource and time ?</p>
<p>One should be able to use only grayscale image.</p>

pkraison: <p><a class="mention" href="/u/rosa">@Rosa</a> I think the weights trained on Imagenet which is RGB, so it might be not be able to generalize well on grayscales.</p>
Can we Create Neural network(Simple one such as Multi Layer perceptron) that only contains positive weights only?

OBouldjedri: <p>I was wondering if there is a specific method to create a well performing neural network with only positive weights (I already tried clipping the weight before training or so and initializing the weights with only positive value ) but still doesn’t give a good results so is there any other method to do so ?</p>
<p>Thank You</p>

spanev: <p>Did you try the solution proposed <a href="https://discuss.pytorch.org/t/positive-weights/19701/3">here</a>?</p>
<blockquote>
<p>doesn’t give a good results</p>
</blockquote>
<p>What is your baseline to establish if the result is good? Are you trying the match the accuracy of the same model but with default relative weights?</p>

OBouldjedri: <p>Thank you for your reply , I mean by a good result lower results but acceptable because we are not using negative weight that has an impact on the back prob algorithm.</p>

OBouldjedri: <p>you mean by P.data are the weight of the neural net and clamp function will clip them to 0 (if they are negative)?</p>

spanev: <blockquote>
<p>I mean by a good result lower results but acceptable because we are not using negative weight that has an impact on the back prob algorithm.</p>
</blockquote>
<p>Sure that makes sense.</p>
<blockquote>
<p>you mean by P.data are the weight of the neural net and clamp function will clip them to 0 (if they are negative)?</p>
</blockquote>
<p>That’s right.<br>
Is it what you already have tried here:</p>
<blockquote>
<p>(I already tried clipping the weight before training</p>
</blockquote>
<p>?</p>

OBouldjedri: <p>Thank you I tried that and the results are acceptable but my real targeted question were can I have some think like model.weight.data.uniform(0.0,1.0) to init the weights in positive range and keep clamping them along the way to be sure that they don’t go beyond 0 and 1 (I tried that also and it seems to me that the results are horrible around 20% acc) is there any solution for this ?</p>
<p>or how to not get affected to much by the lack of negative weights ?</p>

spanev: <p>The only way would be having an unsigned float type for the parameters, which does not exist. Otherwise, you would need to clamp something (params or grads) at some point anyway.</p>
<p>Btw I am curious, why do you need to have only positive weights?</p>
Wasserstein GAN with non-negative weights in the critic

Subho: <p>I want to train a WGAN where the conv layers in the critic are only allowed to have non-negative weights (for a technical reason). The biases, nonetheless, can take both +/- values. There is no constraint on the generator weights. I did a toy experiment on MNIST and observed that the performance is significantly worse than a regular WGAN. Any insights are welcome. Can you suggest some architectural modifications so that the nonnegativity constraint doesn’t severely impair the model capacity?</p>
PyTorch 1.2.0 build fails on 'error:identifier "__ldg" is undefined

marcjoos-phd: <p>Dear all,</p>
<p>I am trying to build PyTorch (<code>git clone</code> of the github repo) from sources but I encounter an error at compile time:</p>
<pre><code class="lang-auto">FAILED: caffe2/CMakeFiles/torch.dir/operators/torch_generated_channelwise_conv3d_op_cudnn.cu.o 

/tmp/pytorch/pytorch/caffe2/operators/channelwise_conv3d_op_cudnn.cu(102): error: identifier "__ldg" is undefined

/tmp/pytorch/pytorch/caffe2/operators/channelwise_conv3d_op_cudnn.cu(123): error: identifier "__ldg" is undefined

/tmp/pytorch/pytorch/caffe2/operators/channelwise_conv3d_op_cudnn.cu(102): error: identifier "__ldg" is undefined
          detected during instantiation of "void caffe2::DepthwiseConv3dGPUKernelNCHW(caffe2::DepthwiseArgs, const T *, const T *, T *, int) [with T=float]" 
(380): here

/tmp/pytorch/pytorch/caffe2/operators/channelwise_conv3d_op_cudnn.cu(123): error: identifier "__ldg" is undefined
          detected during instantiation of "void caffe2::DepthwiseConv3dGPUKernelNCHW(caffe2::DepthwiseArgs, const T *, const T *, T *, int) [with T=float]" 
(380): here

/tmp/pytorch/pytorch/caffe2/operators/channelwise_conv3d_op_cudnn.cu(184): error: identifier "__ldg" is undefined
          detected during instantiation of "void caffe2::DepthwiseConv3dBackpropFilterGPUKernelNCHW(caffe2::DepthwiseArgs, const T *, const T *, T *, int) [with T=float]" 
(505): here

/tmp/pytorch/pytorch/caffe2/operators/channelwise_conv3d_op_cudnn.cu(303): error: identifier "__ldg" is undefined
          detected during instantiation of "void caffe2::DepthwiseConv3dBackpropInputGPUKernelNCHW(caffe2::DepthwiseArgs, const T *, const T *, T *, int) [with T=float]" 
(515): here

6 errors detected in the compilation of "/tmp/tmpxft_00003524_00000000-13_channelwise_conv3d_op_cudnn.compute_30.cpp1.ii".
CMake Error at torch_generated_channelwise_conv3d_op_cudnn.cu.o.Release.cmake:279 (message):
  Error generating file
  /tmp/pytorch/pytorch/build/caffe2/CMakeFiles/torch.dir/operators/./torch_generated_channelwise_conv3d_op_cudnn.cu.o
</code></pre>
<p>From what I understand, <code>__ldg</code> is available only for devices with compute capabilities &gt;= 3.5, knowing that I am compiling PyTorch against CUDA 9.2.148 on a NVidia V100 GPU (so my hardware definitely meets these requirements).</p>
<p>Any help would be appreciated.</p>

ptrblck: <p>It might be related to <a href="https://github.com/pytorch/pytorch/issues/23573">this issue</a>.<br>
From the issue:</p>
<blockquote>
<p>enable the environment variable <code>export TORCH_CUDA_ARCH_LIST=7.0</code> , and this should be fixed.</p>
</blockquote>

marcjoos-phd: <p>Thanks for your help, it solved this issue.</p>