akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Trying to implement compositional code learning

wouterM: <p>Hi all,</p>
<p>Currently I am trying to implement the following paper about compressing word embeddings using deep compositional code learning (<a href="https://arxiv.org/pdf/1711.01068.pdf" rel="nofollow noopener">https://arxiv.org/pdf/1711.01068.pdf</a>). Maybe it is a bit odd to ask specific questions regarding implementing architectures but I am stuck and not certain where to go. Currently I have this:</p>
<pre><code class="lang-auto">class Encoder(nn.Module):
    def __init__(self, args):
        super(Encoder, self).__init__()
        self.E = 300
        self.K = 2
        self.M = 256
        
        self.encode = nn.Sequential(
            nn.Linear(self.E, (self.K * self.M) // 2),
            nn.Tanh(),
            nn.Linear((self.K * self.M) // 2, self.K * self.M),
            nn.Softplus()
        )

    def forward(self, x):
        encoded = self.encode(x)
        a = encoded.reshape(-1, self.M, self.K) 
        a = self._gumbel_softmax(a) 
        
        return a
                      
    def _gumbel_softmax(self, x, eps=1e-10):
        x = torch.log(x+eps)
        g = self._sample_gumbel(x.size(0))
        x = x + Variable(g)
        x = F.softmax(x, dim=2)
        
        return x
        
    def _sample_gumbel(self, batch_size, eps=1e-10):
        u = torch.zeros(batch_size, self.M, self.K).uniform_()
        
        return -1 * torch.log(-1 * torch.log(u + eps) + eps)


class Decoder(nn.Module):
    def __init__(self, args):
        super(Decoder, self).__init__()
        self.E = 300
        self.K = 2
        self.M = 256
        self.A = nn.Parameter(torch.FloatTensor(self.M*self.K, self.E), requires_grad=True)

    def forward(self, y):
        y   = y.reshape(-1, self.M * self.K)
        reconstruction = torch.matmul(y, self.A)
        
        return reconstruction
</code></pre>
<p>The code runs, but over time the reconstruction error only increases so I was wondering if I made any mistakes in the encoder-decoder architecture.</p>
Differentiable Sorting and Indices

antoalli: <p>Hi!<br>
I have a decoder outputing a tensor (let’s call it A) of size [batch_size, num_points, 3], basically a Point-Cloud in three-dimensional euclidean space. I’d like to perform sorting on each of the generated PCs with respect to a specific axis (let’s say z-axis) before feeding the now sorted (named B) [batch_size, num_points, 3] tensor to the next network layers.<br>
The problem arise because I’ve no idea on how to differentiably sort A in one step with torch.sort(). Differently, I tried also to reduce A to a new tensor of size [batch_size, num_points, 1], keeping only z-axis values, performing sort and then using the retrieved indices again on the A tensor. By doing so I’ve noticed that while the values returned by torch.sort have requires_grad=True, the same is not true for indices. How this impact on general differentiability? Is there a better way to sort each of the generated PCs with respect to a single axis (I mean xyz here) without loosing differentiability?<br>
Thank you!</p>

googlebot: <p>You can imagine sort() as doing two operations:<br>
indices = argsort(x,dim)<br>
y = gather(x,indices,dim) <span class="hashtag">#x</span>[indices] may work too</p>
<p>argsort() defines non-trainable permutation<br>
gather() applies this permutation. It is a differentiable operation - backward pass permutes gradients back to match their original positions.<br>
Note that you can use indices to gather from other tensors, that basically performs sortByKey() operation. So what you did, sorting z and reordering bigger tensor is valid.</p>

antoalli: <p>Perfect! So, since the effect of sort() in backward is just applying the same permutation to the gradients according to the new indices, the two operations (atomic/one instruction sort and two steps sort) are perfectly equivalent. I should have checked the resulting gradient by myself!</p>
Dataloader hangs with SSH remote command

woffett: <p>I’m running on a Google Cloud Compute VM. Essentially, when I submit a training command via the gcloud CLI, the program hangs during dataloading, but when I SSH in and run the exact same command interactively, it runs perfectly fine. I’m running a pretty standard loop, using tqdm to track progress</p>
<pre><code class="lang-auto">for x, y in tqdm(valloader):
   ...
</code></pre>
<p>with the training command and some command line arguments wrapped up in a shell script, e.g.:</p>
<pre><code class="lang-auto"># val.sh
python val.py --batch-size 256 --epochs 30 --lr 2e-5 --num-workers 12
</code></pre>
<p>The exact command is something like <code>gcloud compute ssh &lt;instance-name&gt; --command "nohup val.sh &gt; train.out &amp;"</code>.<br>
The program ends up spawning a bunch of workers (that I can see with <code>ps</code>) but they all end up hanging, and in the log (train.out) I can see a bunch of logging information from before the validation loop, but no output from tqdm to indicate that dataloading has begun. However, when I run the same command after directly SSHing in, everything works fine. Any thoughts?</p>
Torch.manual_seed seems not honored in torch.randint

Carol_Ye_Liu: <p>torch.manual_seed seems not honored in  torch.randint.<br>
I ran the following program multiple times in torch 1.5.1 and got different output each time.</p>
<p>torch.manual_seed=123<br>
torch.randint(3, 5, (3,))</p>
<p>torch.manual_seed(123) call used to work before torch 1.4, but with torch 1.5.1 I got the following error. So I changed the call to be torch.manual_seed=123, not sure if that is the problem.</p>
<hr>
<p>TypeError                                 Traceback (most recent call last)<br>
 in <br>
----&gt; 1 torch.manual_seed(123)</p>
<p>TypeError: ‘int’ object is not callable</p>

albanD: <p>Hi,</p>
<p><code>torch.manual_seed</code> is a function. You should call it with the new seed you want to use <a href="https://pytorch.org/docs/stable/torch.html#torch.manual_seed">doc</a>.</p>
<p>When you do <code>torch.manual_seed=123</code> you actually override the function and change it to 123. This is why you get the error you report. Don’t do this assignment and just call the function after a new import of pytorch.</p>

Carol_Ye_Liu: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/0/90739d19d87f5b16fb34d9525d5d0a1bda2847f1.png" data-download-href="https://discuss.pytorch.org/uploads/default/90739d19d87f5b16fb34d9525d5d0a1bda2847f1" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/9/0/90739d19d87f5b16fb34d9525d5d0a1bda2847f1.png" alt="image" data-base62-sha1="kBSskkBWlGW80q3wlQ055HFFGM1" width="690" height="399" data-dominant-color="1E2121"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">951×551 18.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>As you can see, with torch 1.5.1 the function torch.manual_seed cannot take in an int somehow.</p>

albanD: <p>Can you restart the notebook? This is most likely because above you did <code>torch.manual_seed = 123</code>.<br>
Doing <code>import foo</code> multiple times in python does not import it again, it just returns the same module.</p>

Carol_Ye_Liu: <p>You are right! Many thanks!</p>
Train Faster RCNN

benihime91: <p>How can i train Faster RCNN with a different backbone other than <code>ResNet-50</code> ?<br>
Let’s say that i already have a trained <code>CNN</code> that can classify <code>cats and dogs</code> how can i make this a <code>backbone</code> for the Faster RCNN network to <code>detect cats and dogs</code>.</p>
<ul>
<li>How to construct this network ?</li>
<li>How do i train it?</li>
<li>What would a general training loop for this model look like ?</li>
</ul>
Quantized model not provide performance improvements

fel88: <p>I try to run quantization benchmark:<br>
</p><aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="16" height="16">
      <a href="https://github.com/z-a-f/quantization_benchmarks" target="_blank" rel="nofollow noopener">GitHub</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars3.githubusercontent.com/u/4216323?s=400&amp;v=4" class="thumbnail onebox-avatar" width="60" height="60">

<h3><a href="https://github.com/z-a-f/quantization_benchmarks" target="_blank" rel="nofollow noopener">z-a-f/quantization_benchmarks</a></h3>

<p>PyTorch quantization benchmarks. Contribute to z-a-f/quantization_benchmarks development by creating an account on GitHub.</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>But I didn’t see any speed up with quantized model.<br>
For example:<br>
googlenet:<br>
Train time:<br>
q: 192.940<br>
f: 192.940<br>
Test time:<br>
q: 193.114<br>
f:  193.114</p>
<p>On the third model I got:<br>
Downloading: “<a href="https://download.pytorch.org/models/mobilenet_v2-b0353104.pth" rel="nofollow noopener">https://download.pytorch.org/models/mobilenet_v2-b0353104.pth</a>” to<br>
…/.cache\torch\hub\checkpoints\mobilenet_v2-b0353104.pth<br>
100%|█████████████████████████████████████| 13.6M/13.6M [00:01&lt;00:00, 8.20MB/s]</p>
<p>File “…Anaconda3\envs\torch1.5\lib\site-packages\torchvision\mod<br>
els\quantization\utils.py”, line 22, in quantize_model<br>
raise RuntimeError("Quantized backend not supported ")<br>
RuntimeError: Quantized backend not supported</p>
<p>Why does the quantized model not provide performance improvements?<br>
How can I activate quantized backend?</p>
<p>My environment:<br>
conda            4.8.3<br>
torch              1.7.0.dev20200716<br>
torchvision     0.8.0.dev20200716<br>
Python           3.6.10</p>
<p>CPU:             i5-4670</p>

fel88: <blockquote>
<blockquote>
<blockquote>
<p>print(torch.backends.quantized.supported_engines)<br>
[‘none’, ‘fbgemm’]</p>
</blockquote>
</blockquote>
</blockquote>

Vasiliy_Kuznetsov: <aside class="quote no-group" data-username="fel88" data-post="1" data-topic="89501">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/f/6a8cbe/48.png" class="avatar"> fel88:</div>
<blockquote>
<p>Quantized backend not supported</p>
</blockquote>
</aside>
<p>one thing to try would be:</p>
<pre><code class="lang-auto">torch.backends.quantized.engine = 'fbgemm'
model.qconfig = torch.quantization.get_default_qat_qconfig('fbgemm')
</code></pre>

fel88: <p>It still doesn’t work, but<br>
<a href="https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html" class="onebox" target="_blank" rel="nofollow noopener">https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html</a><br>
works well. It is enough for my purposes.</p>
Regarding use of image in text experiment

vainaijr: <p>Why do we carry experiments with attempting at making computer do tasks like text classification without showing any image to it?<br>
Isn’t it like learning a new language, and all you would be shown is words in that new language, so you would end up in a state of not knowing the meaning of any word as all of them are in a language you do not know, same thing applies for computer also.<br>
Shouldn’t all text classification experiments also have image input, just like we learn first language by being show text and associated image?</p>
Pruning a model using PyTorch

Edd: <p>Hi,</p>
<p>So I am trying to use torch.nn.utils.prune.global_unstructured as in <a href="https://pytorch.org/docs/master/generated/torch.nn.utils.prune.global_unstructured.html" rel="nofollow noopener">https://pytorch.org/docs/master/generated/torch.nn.utils.prune.global_unstructured.html</a></p>
<p>I did it on a simple model and that worked. model.cov2 or other layers and that works. I am trying to do it on a model that’s (nested)? I get errors as:<br>
AttributeError: ‘CNN’ object has no attribute ‘conv1’<br>
and other errors. I tried everything to access this deep cov1, but I couldn’t.<br>
You can find the model code below:</p>
<p>class CNN(nn.Module):<br>
def <strong>init</strong>(self):<br>
“”“CNN Builder.”""<br>
super(CNN, self).<strong>init</strong>()</p>
<pre><code>    self.conv_layer = nn.Sequential(

        # Conv Layer block 1
        nn.Conv2d(in_channels=3, out_channels=32, kernel_size=3, padding=1),
        nn.BatchNorm2d(32),
        nn.ReLU(inplace=True),
        nn.Conv2d(in_channels=32, out_channels=64, kernel_size=3, padding=1),
        nn.ReLU(inplace=True),
        nn.MaxPool2d(kernel_size=2, stride=2),

        # Conv Layer block 2
        nn.Conv2d(in_channels=64, out_channels=128, kernel_size=3, padding=1),
        nn.BatchNorm2d(128),
        nn.ReLU(inplace=True),
        nn.Conv2d(in_channels=128, out_channels=128, kernel_size=3, padding=1),
        nn.ReLU(inplace=True),
        nn.MaxPool2d(kernel_size=2, stride=2),
        nn.Dropout2d(p=0.05),

        # Conv Layer block 3
        nn.Conv2d(in_channels=128, out_channels=256, kernel_size=3, padding=1),
        nn.BatchNorm2d(256),
        nn.ReLU(inplace=True),
        nn.Conv2d(in_channels=256, out_channels=256, kernel_size=3, padding=1),
        nn.ReLU(inplace=True),
        nn.MaxPool2d(kernel_size=2, stride=2),
    )


    self.fc_layer = nn.Sequential(
        nn.Dropout(p=0.1),
        nn.Linear(4096, 1024),
        nn.ReLU(inplace=True),
        nn.Linear(1024, 512),
        nn.ReLU(inplace=True),
        nn.Dropout(p=0.1),
        nn.Linear(512, 100)
    )


def forward(self, x):
    """Perform forward."""
    # conv layers
    x = self.conv_layer(x)
    # flatten
    x = x.view(x.size(0), -1)
    # fc layer
    x = self.fc_layer(x)
    return x
</code></pre>
<p>How can I apply pruning on this model?</p>
None type object is not iterable

saumya0303: <p>Hi team,<br>
I am creating a tensor x and passing x as a parameter to a function f, while passing x as a parameter to a function f, I am getting an error <strong>TypeError: ‘NoneType’ object is not iterable</strong><br>
index = 20<br>
x= torch.tensor([index])<br>
y = f(x)<br>
Please help me to resolve this issue.</p>

aRI0U: <p>Does the error occur inside of <code>y = f(x)</code> execution or before? The two first lines of your code work as expected for me.<br>
When I get a <strong>TypeError: ‘NoneType’ object is not iterable</strong>, it is usually because I forget to write the <code>return</code> statement at the end of a function. In fact functions that return nothing actually return <code>None</code>.</p>

saumya0303: <p>error occurred while calling function f with x as parameter , and return statement is there at the end of function .</p>

aRI0U: <p>Can you print <code>x</code> just before calling <code>y = f(x)</code> and check that its value is <code>tensor([20])</code> ? If the value of <code>x</code> is correct then I think the problem comes from function <code>f</code>. Could you provide the code of function <code>f</code> ?</p>

saumya0303: <p>yes, value of x is <code>tensor([20])</code> before calling y = f(x) .</p>
Finetuning EfficientNet in Pytorch

jenn: <p>Hi everyone,</p>
<p>I would like to use an EfficientNet for image classification. Since my inputimage has 6 instead of 3 channels, I guess I need to change some layers. Currently I define my model as follows:</p>
<p>class Classifier(nn.Module):<br>
def <strong>init</strong>(self,n_classes = 4):<br>
super(Classifier, self).<strong>init</strong>()<br>
self.effnet = EfficientNet.from_name(‘efficientnet-b4’)<br>
self.l1 = nn.Linear(2000 , 256)<br>
self.dropout = nn.Dropout(0.5)<br>
self.l2 = nn.Linear(256,n_classes) #  number of classes is 4<br>
self.relu = nn.LeakyReLU()<br>
def forward(self, input):<br>
x = self.effnet(input)<br>
x = x.view(x.size(0),-1)<br>
x = self.dropout(self.relu(self.l1(x)))<br>
x = self.l2(x)<br>
return x</p>
<p>Can someone tell me how I can change layers of the EfficientNet in pytorch?</p>

ptrblck: <p>You could print the modules via <code>print(self.effnet)</code> and look for the first conv layer in the model.<br>
Once you’ve found it, e.g. as</p>
<pre><code class="lang-python">(conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
</code></pre>
<p>you could replace it using the attribute name:</p>
<pre><code class="lang-python">self.effnet.conv1 = nn.Conv2d(6, 64, 7, 2, 3, bias=False)
</code></pre>
<p>to change the number of input channels to 6.<br>
(Note that I have used a ResNet to show the workflow, so the layer name might differ in your case.)</p>

jenn: <p>Thanks, that helped a lot!</p>
Replace tensor values at partial indexes

11152: <p>I want to make the value of a specific index equal to 0. but very very slow…</p>
<pre><code class="lang-auto">mask = [1,2,3,4]

input = torch.zeros(100,64,32,32)

for m in mask:
   for i in range(100):
      input[i][m] = 0
</code></pre>
<p>How do I optimize my code?</p>

ptrblck: <p>This code should work:</p>
<pre><code class="lang-python">x = torch.zeros(100, 64, 32, 32)
mask = torch.tensor([1,2,3,4])
x[:, mask] = 1
</code></pre>
<p>Note that I changed the assignment to a 1, otherwise you would reassign a 0 to the tensor containing all zeros. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

11152: <p>Good JOB!! Many Thanks!!!</p>

curious: <p>How to do this in the last dimension of the tensor?</p>
How to transplant Pytorch to arm development board？

xifanlover: <p>the MCU of development board：i.mx6dl.<br>
The system on the arm development board has no interface.<br>
How to transplant Pytorch to arm development board？<br>
I want my QT program to call the pytorch model（file with the suffix pt）。How can I achieve this requirement？</p>

xifanlover: <p>My QT program include the libtorch.so，then can load module.pt file。But It is failed when My QT program compile to arm executable program。Becase libtorch.so can not be compiled。what can I do？</p>
Moving from PIL to opencv in pytorch

John_J_Watson: <p>I have been using pytorch for quite a while, but now, one of the projects I am working on, requires me to use opencv and no PIL <img src="https://discuss.pytorch.org/images/emoji/apple/frowning.png?v=9" title=":frowning:" class="emoji" alt=":frowning:"></p>
<p>Typically, I do this in pytorch:</p>
<pre><code class="lang-auto">        self.data_transforms = torchvision.transforms.Compose([
            torchvision.transforms.Resize((224,224)),\
            torchvision.transforms.ToTensor(),
            torchvision.transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
        ])
</code></pre>
<p>Id like to replicate the same with openCV with minimal effort (because the above snippet is present in quite a few of my dataloader classes).</p>
<p>What is the most efficient way to go about this?</p>

ptrblck: <p><a href="https://github.com/jbohnslav/opencv_transforms">This repository</a> seems to have implemented (some) <code>torchvision.transforms</code> in OpenCV.</p>
How to load python tensor in C++?

Kenisy: <p>So I copied the code from <a href="https://github.com/pytorch/pytorch/issues/20356#issuecomment-567663701" rel="nofollow noopener">https://github.com/pytorch/pytorch/issues/20356#issuecomment-567663701</a> but I got exception: c10::Error at memory location at torch::pickle_load. Anyone know how to fix it ?<br>
My python code:</p>
<pre><code class="lang-auto">    x = torch.arange(10)
    f = io.BytesIO()
    torch.save(x, f, _use_new_zipfile_serialization=True)
    # send f wherever
    with open('test.pt', "wb") as outfile:
        # Copy the BytesIO stream to the output file
        outfile.write(f.getbuffer())
</code></pre>
<p>My C++ code:</p>
<pre><code class="lang-auto">#include &lt;iostream&gt;
#include &lt;torch/torch.h&gt;

std::vector&lt;char&gt; get_the_bytes(std::string filename) {
    std::ifstream input(filename, std::ios::binary);
    std::vector&lt;char&gt; bytes(
        (std::istreambuf_iterator&lt;char&gt;(input)),
        (std::istreambuf_iterator&lt;char&gt;()));

    input.close();
    return bytes;
}

int main()
{
    std::vector&lt;char&gt; f = get_the_bytes("path\\to\\test.pt");
    torch::IValue x = torch::pickle_load(f);
    torch::Tensor tensor = x.toTensor();
    std::cout &lt;&lt; tensor;
}

</code></pre>

glaringlee: <p><a class="mention" href="/u/kenisy">@Kenisy</a><br>
Check your libtorch version and pytorch version. It might because you have different versions.</p>
How to time C++ functions for contributing?

thinking_tower: <p>Hi everyone,</p>
<p>I’m currently tackling my first issue in PyTorch by refactoring two functions. I noticed that there were two functions that appear to do the same thing but I’m guessing that time-wise they don’t operate the same. How would one run tests for benchmarking?</p>
<p>Here’s a relevant screenshot of what I wanna achieve:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/2/32e0709bb28f1d81f94c86c6000dfaa3ba0dd664.png" data-download-href="https://discuss.pytorch.org/uploads/default/32e0709bb28f1d81f94c86c6000dfaa3ba0dd664" title="Screen Shot 2020-07-21 at 10.06.28 AM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/2/32e0709bb28f1d81f94c86c6000dfaa3ba0dd664_2_690x290.png" alt="Screen Shot 2020-07-21 at 10.06.28 AM" data-base62-sha1="7g4Juurq7cvzvhUD98QiGUO7fbm" width="690" height="290" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/2/32e0709bb28f1d81f94c86c6000dfaa3ba0dd664_2_690x290.png, https://discuss.pytorch.org/uploads/default/optimized/3X/3/2/32e0709bb28f1d81f94c86c6000dfaa3ba0dd664_2_1035x435.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/3/2/32e0709bb28f1d81f94c86c6000dfaa3ba0dd664_2_1380x580.png 2x" data-dominant-color="E3ECF5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot 2020-07-21 at 10.06.28 AM</span><span class="informations">2712×1142 372 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<a href="https://github.com/allwu/pytorch/commit/4d6973326343826b1269aeaabf50e1775d06eff6" rel="noopener nofollow ugc">Relevant commit</a></p>

glaringlee: <p><a class="mention" href="/u/thinking_tower">@thinking_tower</a><br>
The performance test part to me is a self wrote script.<br>
If you are under python. You can simply use timeit with string format to get the same output layout as above.</p>

thinking_tower: <p>Hi <a class="mention" href="/u/glaringlee">@glaringlee</a>!</p>
<p>Thanks for taking the time to answer my question!</p>
<p>I just want to check if this is valid:</p>
<ol>
<li>Time the original function.</li>
<li>Make modifications to code.</li>
<li>Rebuild (<code>python setup.py develop --cmake </code> along with some flags like <code>BUILD_CAFFE2_OPS=0</code>).</li>
<li>Test (<code>python run_test.py</code> for the specific module) to ensure correctness.</li>
<li>Time the function again (now that it’s been changed).</li>
</ol>

glaringlee: <p><a class="mention" href="/u/thinking_tower">@thinking_tower</a><br>
That make sense!.</p>
（c++）How to convert (tensor) type to int (type)

EricHuiHui: <p>I use C + + to call the model of pytorch, but the model return value is tensor type, which makes me unable to do some specific operations on the return value. What should I do?</p>
<p>at::Tensor output = module.forward(inputs).toTensor();</p>

glaringlee: <p><a class="mention" href="/u/erichuihui">@EricHuiHui</a><br>
Not sure what you want to do here, can you give more information?</p>
<p>If you are sure your tensor will have only one item, 1 dimension 1 item. Then you can use .item(). Otherwise, you might need to use TensorIterator, or convert tensor to contiguous and get its data ptr. Please provide more info.</p>
Why can not I take full advantage of the CPU cores?

khalil: <p>Problem Description:<br>
Today I want to test the CPU utilization of pytorch matrix multiplication.So I run the following benchmark:</p>
<pre><code class="lang-auto">import timeit
runtimes = []
threads = [1] + [t for t in range(2, 49, 2)]
for t in threads:
    torch.set_num_threads(t)
    r = timeit.timeit(setup = "import torch; x = torch.randn(1024, 1024); y = torch.randn(1024, 1024)", stmt="torch.mm(x, y)", number=100)
    runtimes.append(r)
</code></pre>
<p>Howerver I found a weird problem : CPU cores are very unevenly loaded.One CPU core has a very high load and the others have a very low load.Just like this</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/e/ae784f81f23010b5e031f45e14b1555a427138bc.png" data-download-href="https://discuss.pytorch.org/uploads/default/ae784f81f23010b5e031f45e14b1555a427138bc" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/a/e/ae784f81f23010b5e031f45e14b1555a427138bc.png" alt="image" data-base62-sha1="oTqQkDOmcxQfGEgb8yeCdxjEiyE" width="690" height="426" data-dominant-color="101010"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">727×449 8.31 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I compile the pytorch of version 1.4.1 with gcc/7.4.0 and I set the environment variable USE_OPENMP=1.What caused this ? Please tell me ! Thanks!</p>
Converting quantized models from PyTorch to ONNX

Joseph_Konan: <p>I am trying to export a quantized int8 PyTorch model to ONNX from the following tutorial.</p>
<p><a href="https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html" class="onebox" target="_blank" rel="nofollow noopener">https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html</a></p>
<p>However, PyTorch to ONNX conversion of quantized models is not supported. Various types of quantized models will either explicitly say their conversion is not supported or they will throw an attribute error.</p>
<p>My question is — how do we do the conversion manually? Specifically, how do we define a custom mapping of ONNX operations for PyTorch classes? I assume the logic is the same for non-quantized layers, whose conversion needed to be defined until it was built-in, but I am having trouble finding an example.</p>

jerryzh168: <p>cc <a class="mention" href="/u/supriyar">@supriyar</a> might be able to help.</p>

supriyar: <p>We currently only support conversion to ONNX for Caffe2 backend. This thread has additional context on what we currently support - <a href="https://discuss.pytorch.org/t/onnx-export-of-quantized-model/76884/8" class="inline-onebox">ONNX export of quantized model</a></p>
<p>If you would like to add custom conversion logic to onnx operators for quantized pytorch ops you can follow the code in <a href="https://github.com/pytorch/pytorch/blob/master/torch/onnx/symbolic_caffe2.py" rel="nofollow noopener">https://github.com/pytorch/pytorch/blob/master/torch/onnx/symbolic_caffe2.py</a> which adds the mapping for the Caffe2 ops in ONNX.</p>

blueskywwc: <p>＠ Joseph_Konan Hello, can you now convert the quantified model to ONNX, thank you!</p>

Joseph_Konan: <p>Thanks for the update — I’ll look into this!</p>
Export LSTM to ONNX with sequence length and dynamic batch size

zhujiangang: <p>I use LSTM to modeling text with the following code, the shape of <code>inputs</code> is [batch_size, max_seq_len, embedding_size], the shape of <code>input_lens</code> is [batch_size]. <code>rnn</code> is simply a bidirectional LSTM defined as follows:</p>
<p><code>self.rnn = nn.LSTM(self.input_dim, self.hidden_size, self.num_layers,                                bidirectional=self.bidirectional, dropout=self.dropout, batch_first=True)</code></p>
<pre><code>       `padded_seq_len = inputs.shape[1]

       # Sort by length (keep idx)
        _, idx_sort = torch.sort(input_lens, dim=0, descending=True)
        _, idx_unsort = torch.sort(idx_sort, dim=0)

        inputs = inputs.index_select(0, idx_sort)
        input_lens = list(input_lens[idx_sort])

        # Handling padding in Recurrent Networks
        inputs_packed = nn.utils.rnn.pack_padded_sequence(inputs, input_lens, batch_first=True)
        self.rnn.flatten_parameters()
        if str(self.cell).lower() == 'lstm':
            outputs, (hn, cn) = self.rnn(inputs_packed)
        elif str(self.cell).lower() == 'gru':
            outputs, hn = self.rnn(inputs_packed)

        outputs = nn.utils.rnn.pad_packed_sequence(outputs, batch_first=True, total_length=padded_seq_len)[0]

        # Un-sort by length
        outputs = outputs.index_select(0, idx_unsort)
        hn = hn.index_select(1, idx_unsort)`
</code></pre>
<p>However, when I try to export this model to ONNX with dynamic batch_size, I cannot get the parity result with the original pytorch checkpoint. Anyone has such experience?</p>
How to use binary data of different length with LSTM?

odats: <p>I have a huge list of binary strings: [010101, 011111111, 0101111100011…]. The length of each string is different: from 1 to 10000 characters. The output of the Neural Network is simple: yes/no.</p>
<p>Can you please provide an example of how to define and train simple LSTM that works with this data? LSTM should read input character by character and at each step predict yes/no.</p>

vdw: <p>There are a couple of things to unpack:</p>
<ul>
<li>
<p>Sequences of 10,000 are extremely long. Sure, LSTMs proud themselves to have long-term memory but 10k time steps seems awfully long. I’m not saying that it won’t work, it’s just something too keep in mind.</p>
</li>
<li>
<p>Since the range of you sequences varies a lot, have a look at <a href="https://torchtext.readthedocs.io/en/latest/data.html#bucketiterator" rel="nofollow noopener"><code>BucketIterator</code></a> to generate batches where all sequences within one batch have the same or at least very similar size.</p>
</li>
<li>
<p>Since batches can still have sequences of different lengths, you should have a look at <a href="https://pytorch.org/docs/master/generated/torch.nn.utils.rnn.pack_padded_sequence.html#torch.nn.utils.rnn.pack_padded_sequence" rel="nofollow noopener"><code>pack_padded_sequences</code></a>.</p>
</li>
<li>
<p>Once the batches are right setting up the network model is relatively straightforward.</p>
</li>
</ul>
Multiprocessing failed using single GPU

Lorentz.B: <p>Hi, I am new to the machine learning community. For some reasons, I try to parallelly do inference using multi-core CPU and single GPU, however I just got following runtime errors.</p>
<pre><code class="lang-auto">THCudaCheck FAIL file=c:\a\w\1\s\tmp_conda_3.6_091443\conda\conda-bld\pytorch_1544087948354\work\torch\csrc\generic\StorageSharing.cpp line=232 error=71 : operation not supported
File "C:\Users\Anaconda3\lib\site-packages\torch\multiprocessing\reductions.py", line 213, in reduce_tensor
    (device, handle, storage_size_bytes, storage_offset_bytes) = storage._share_cuda_()
RuntimeError: cuda runtime error (71) : operation not supported at c:\a\w\1\s\tmp_conda_3.6_091443\conda\conda-bld\pytorch_1544087948354\work\torch\csrc\generic\StorageSharing.cpp:232
</code></pre>
<p>The following is a simplified example which can reproduce the errors.</p>
<pre><code class="lang-auto">import torch
from torch import nn

# model used to do inference
class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.fc1 = nn.Linear(100,1)
        
    def forward(self,x):
        return self.fc1(x)
    
# class running inference
class A(object):
    def __init__(self):
        pass

    def do_something(self, model):
        # do something
        x = torch.randn(100).view(-1)
        print(model.forward(x))
    
    def run(self):
        mp = torch.multiprocessing.get_context('spawn')
        processes = []

        for i in range(2):
            p = mp.Process(target=self.do_something, args=(Model().cuda(),))
            processes.append(p)

        for p in processes:
            p.start()

if __name__ == '__main__':
    a = A()
    a.run()
</code></pre>
<p>It would be greatly appreciated if anyone can help solve this problem. By the way, my PC runs on Windows 10 with one GTX 1070 GPU.</p>

peterjc123: <p><a href="https://pytorch.org/docs/stable/notes/windows.html#cuda-ipc-operations" class="onebox" target="_blank" rel="nofollow noopener">https://pytorch.org/docs/stable/notes/windows.html#cuda-ipc-operations</a></p>

pietern: <p>In your example you could choose to instantiate your model in the sub process. Then you won’t need to share CUDA tensors between the parent and the child process.</p>

xxxxxi-gg: <p>Hi,Have you solve it? I have the same problem.</p>
How to reduce the number of feature dimension?

cat-loves-donuts: <p>Well, when I finally upgraded my final project codes, I found a very interesting thing that I tried to reduce the feature dimensions after each conv layers. I had 3 conv layers and the input image have 3 dimensions and 28X28 size. The size would not change because I used padding to keep size the same. After the first conv layer the dimension became 16. And after the last layer the feature dimensions became 128. However, I wanted to use pooling layer to reduce the feature dimensions. I want to reduce it to 16 so that I can connect another layer after it without calculating the dimensions or input channels. Because if I do not do that the dimension might increase to 1000 or more.</p>

ptrblck: <p>Do you mean the number of channels by “feature dimension”?<br>
If so, you could reduce it by using another conv layer with the desired number of output channels.</p>

681408ed5ec9ed11af98: <p>If you want to decrease the image resolution, you could add a pooling layer after the conv layer.<br>
If you want to reduce the number of feature channels, you could add a 1x1 conv layer after the conv layer without reducing the image resolution.<br>
If you want to reduce both, you can add another one conv layer.</p>

cat-loves-donuts: <p>Yeah, that is what I mean. Well, I finally found a paper about this. They were using a 1X1 conv to reduce the out channels.<br>
Anyway, thank you.</p>

cat-loves-donuts: <p>Yeah, I tried to reduce the output channel and another paper said the same methods.<br>
But anyway, thank you so much.</p>

stelios: <p>I am having the same (similar) problem here. Any ideas how to solve it?</p>
<aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">
      <a href="https://github.com/steliospnnl/ppo_grid2op" target="_blank" rel="nofollow noopener">GitHub</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars2.githubusercontent.com/u/67917557?s=400&amp;v=4" class="thumbnail onebox-avatar" width="420" height="420">

<h3><a href="https://github.com/steliospnnl/ppo_grid2op" target="_blank" rel="nofollow noopener">steliospnnl/ppo_grid2op</a></h3>

<p>PPO Implementation of Grid2Op Environments. Contribute to steliospnnl/ppo_grid2op development by creating an account on GitHub.</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>