akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Self-defined collate_fn in DataLoader changes the data structure

username: Bruce_Su
cooked: <p>I have a self defined collate function which changes the shape of Tensors in the batch (padding variable-length etc.). When I run iteration twice(or multiple epochs), it throws error. After investigation, it’s because the input to the second enumerate() was the output from the first! So it’s going through the collate function twice. How to avoid it?</p>
<pre><code class="lang-auto">train_dataset = DataLoader(dataset=train_set, collate_fn=ER_Collate,
                               batch_size=BATCH_SIZE, shuffle=True)
for i, train_ in enumerate(train_dataset):
   pass
for i, train_ in enumerate(train_dataset):
   pass
</code></pre>
Add custom weight for instance normalisation

username: Ierezell
cooked: <p>I’m reimplementing a paper and they use adaptive instance norm but with custom weights comming from another network.</p>
<p>I know that I can code the norm layer from scratch (it’s not long or hard) but I was looking for a cleaner solution.</p>
<p>I didn’t find an adaIN layer in pytorch so :</p>
<p>Can I use the InstanceNorm2D from torch.nn and just pass my weights as attribute of the class (if I cast them as Parameters ?) will the gradient and everything will flow ok ?</p>
<p>Or can I use other solutions to pass custom weights to a norm layer ?</p>
<p>Thanks a lot in advance.</p>
TorchScript: Float(1, 32) vs. Tensor

username: lobachevzky
cooked: <p>I am trying to optimize a recurrent network using TorchScript. This is the code in question:</p>
<pre><code class="lang-auto">        s = self.f(torch.cat([obs[i], r, g, b], dim=-1))
        c = torch.sigmoid(self.phi_update(torch.cat([s, h], dim=-1)))
                                                    ~~~~~ &lt;--- HERE
</code></pre>
<p>And the error is</p>
<pre><code class="lang-auto">Lists must contain only a single type, expected: Float(1, 32) but found Tensor instead
</code></pre>
<p>This is the definition of <code>self.f</code>:</p>
<pre><code class="lang-auto">        self.f = torch.jit.trace(
            nn.Linear(in_size, hidden_size), 
            example_inputs=torch.rand(1, in_size))
</code></pre>
<p>So <code>Float(1, 32)</code> appears to be the output of a traced module. Is there a way to coerce <code>Float(1, 32)</code> to Tensor or vice versa?</p>
<p>Thank you.</p>
Use of getattr in forward function

username: nullgeppetto
cooked: <p>I am trying to use <code>getattr</code> in the <code>forward</code> method an <code>nn.Module</code> class. A minimalistic indicative example of what I’m trying to do is shown below:</p>
<pre><code class="lang-auto">class Net(nn.Module):
    def __init__(self, ):
        super().__init__()
        self.layer = nn.Linear(8,  16)
    def forward(self, x):
        return getattr(self, 'layer')(x)
</code></pre>
<p>I get a <code>NotImplementedError</code> when I use <code>forward</code>:</p>
<pre><code class="lang-auto">File "/usr/lib/python3.7/site-packages/torch/nn/modules/module.py", line 85, in forward
    raise NotImplementedError
NotImplementedError
</code></pre>
<p>Any ideas on how to solve it? Or even an alternative way to “create” the name of an attribute of <code>Net</code> inside <code>forward</code>? Thank you!</p>

username: ptrblck
cooked: <p>Your code works on my machine. Make sure do don’t have any typos in the definition of <code>forward</code> and also check the indentation.</p>

username: nullgeppetto
cooked: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a>, thanks for your prompt response. The truth is that I tried to make it too simple without testing it. The above indeed works, but what I truly need is something like this:</p>
<pre><code class="lang-auto">class Net(nn.Module): 
     def __init__(self, ): 
         super().__init__()
         setattr(self, 'layer', nn.ModuleDict({'a': nn.Linear(8, 16), 'b': nn.Linear(16, 4)})) 
     def forward(self, x): 
         return getattr(self, 'layer')(x)
</code></pre>
<p>I just tried it and indeed it raises a <code>NotImplementedError</code>. It seems that the use of <code>nn.ModuleDict</code> causes the problem, since simple nn modules work fine.</p>
<p>Any ideas on how to fix it? Many thanks <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: ptrblck
cooked: <p>You are currently trying to call the <code>nn.ModuleDict</code>, which won’t work, as you would have to address one of the layers:</p>
<pre><code class="lang-python">class Net(nn.Module): 
     def __init__(self, ): 
         super().__init__()
         setattr(self, 'layer', nn.ModuleDict({'a': nn.Linear(8, 16), 'b': nn.Linear(16, 4)})) 
     def forward(self, x): 
         x = getattr(getattr(self, 'layer'), 'a')(x)
         x = getattr(getattr(self, 'layer'), 'b')(x)
         return x
</code></pre>
<p>What is your use case, if I may ask?</p>

username: nullgeppetto
cooked: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a>, many thanks for your response. It should be obvious and I should have figured it out…</p>
<p>The reason for doing so is that I want my network to have a basic backbone sub-network, and a number of branches (heads). Each head is implemented as an <code>nn.ModuleDict</code>, or even <code>nn.ModuleList</code> for convenience. During forward, an <code>id</code> for which head should be used is given (as a string). If you think that’s not a good practice, please let me know.</p>
Best way to work with spark-generated datasets?

username: mikhail.osckin
cooked: <p>What is the best way to work with spark-generated output files? Now i’m using spark for raw data vectorization, so i just convert output parquet files into tfrecords and load them with tf dataset machinery. But it’s not clear how i can do something similar with pytorch because Dataset.<strong>getitem</strong> method requires item_idx, but most of big-data related tools and formats (for example parquet) don’t do well with sequential indexing.</p>

username: Lian_Jiang
cooked: <p>Any help? There seems to be a gap between spark and pytorch. We can use spark to do preprocessing and need to convert it (e.g. in parquet format) to a format friendly to pytorch. Thanks.</p>

username: mikhail.osckin
cooked: <p>Seems like the only way to do this right now is to write your own Dataset and DataLoader.</p>

username: andompesta
cooked: <p>Hi all,</p>
<p>any update on this ??<br>
I’m very interested, but also at this link there is no useful information <a href="https://docs.databricks.com/applications/deep-learning/pytorch.html" rel="nofollow noopener">https://docs.databricks.com/applications/deep-learning/pytorch.html</a></p>
<p>Apparently there is no support to load a distributed dataframe to pytorch in a spar fashion.</p>

username: aalkhalil155
cooked: <p>Hello :</p>
<p>I am encountering the same challenges at this moment working on spark data and need to feed it in Pytorch model.</p>
<p>I am wondering if there has been a solution for this common problem after all this time?</p>
<p>thanks</p>
Torch.jit.script for torch.nn.utils.rnn.pack_padded_sequence

username: gtayxlyc
cooked: <p>I have struggled with the problem several days.<br>
Now I implemented torch.nn.utils.rnn.pack_padded_sequence with variable batch size and sequence length. The model is already trained and work fine in python. I want to embed it in my c++ project. I used torch.jit.script but it always inform me that<br>
for operator (Tensor 0, Tensor 1, Tensor 2) -&gt; Tensor:<br>
expected a value of type Tensor for argument ‘2’ but found bool<br>
I changed the batch_first to tensor.ones() and in forward prediction, it informs me that<br>
argument ‘batch_first’ (position 3) must be bool<br>
it is really ridiculous. And most examples online are trace implementation.<br>
Can somebody help me or give me some hints?<br>
Thx a lot!</p>

username: julioasoto
cooked: <p>Indeed, pack_padded_sequences does not work with TorchScript yet.</p>

username: Andras
cooked: <p>I’m so sad <img src="https://discuss.pytorch.org/images/emoji/apple/frowning.png?v=9" title=":frowning:" class="emoji" alt=":frowning:"> This is Important for me…</p>

username: julioasoto
cooked: <p>I’m thinking about opening an issue in Github, just to make sure PT devs have this under their radar. Hopefully the necessary change is easy to implement (although I have no idea on how TorchScript internals work…)</p>

username: julioasoto
cooked: <p>I have opened an issue in Github to track this: <a href="https://github.com/pytorch/pytorch/issues/21282" rel="nofollow noopener">https://github.com/pytorch/pytorch/issues/21282</a></p>
Sizes of tensors must match except in dimension 0

username: GTUAR
cooked: <p><img src="https://discuss.pytorch.org/uploads/default/original/2X/f/f2737a619ae6c3c5a953f0b707849e62fa315e21.png" alt="image" data-base62-sha1="yAOXqdA82CDUppTf73ysCRpkrIZ" width="561" height="455"><br>
<img src="https://discuss.pytorch.org/uploads/default/original/2X/1/17884c01471b5a516f14285f7277273558d4f257.png" alt="image" data-base62-sha1="3maZMOxCt2SFigjIBaUrsJu5rOT" width="369" height="100"><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/d/d5ede22ed8751c6651b11640eb0048a8c4e11be0.png" data-download-href="https://discuss.pytorch.org/uploads/default/d5ede22ed8751c6651b11640eb0048a8c4e11be0" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/2X/d/d5ede22ed8751c6651b11640eb0048a8c4e11be0.png" alt="image" data-base62-sha1="uwvmApPQJU1PTHCF3Nsi32HiQ5W" width="690" height="103" data-dominant-color="F1F1F1"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">835×125 5.93 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
I tried the method in the forum but it didn’t solve it. When I use randomresizecrop, it doesn’t go wrong. When I use tramsform.resize, I get this error. How to fix it? thank you very much<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/8/81017c08036f9c49d6ddaa818d7959aa09e54307.png" data-download-href="https://discuss.pytorch.org/uploads/default/81017c08036f9c49d6ddaa818d7959aa09e54307" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/8/81017c08036f9c49d6ddaa818d7959aa09e54307_2_690x144.png" alt="image" data-base62-sha1="ipeM03BsDvjndnuqD8q5AvzLXbp" width="690" height="144" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/8/81017c08036f9c49d6ddaa818d7959aa09e54307_2_690x144.png, https://discuss.pytorch.org/uploads/default/original/2X/8/81017c08036f9c49d6ddaa818d7959aa09e54307.png 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/8/81017c08036f9c49d6ddaa818d7959aa09e54307.png 2x" data-dominant-color="2A323B"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">925×194 27.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <p>Could you try to pass the size argument as a tuple: <code>transforms.Resize((IMG_SIZE, IMG_SIZE))</code>?</p>

username: Shandilya21
cooked: <p>I have same issue , I have also resolve by transform.Resize((Img_size,img_size))<br>
be careful it is a tuple.</p>

username: GTUAR
cooked: <p>thank a lot:grinning:</p>

username: GTUAR
cooked: <p>thank a lot:grinning:</p>
Convolution ops for every index

username: suke
cooked: <p>Hello, I face a problem, for convolution ops, in normal case, nn.conv2d can handle NxCxHxW case<br>
but I need to handle MxNxCxh’xw’ h’=w’=3   M = HxW</p>
<p>how can do this convolutions using pytorch convolution api.</p>
IndexError: too many indices for tensor

username: Shashi_Gharti
cooked: <p>IndexError: too many indices for tensor of dimension 0<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/c/cb24d65b9a0ed2d4fcf5f66b869c41442098cdc2.png" data-download-href="https://discuss.pytorch.org/uploads/default/cb24d65b9a0ed2d4fcf5f66b869c41442098cdc2" title="03-17-27-03"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/c/cb24d65b9a0ed2d4fcf5f66b869c41442098cdc2_2_690x232.png" alt="03-17-27-03" data-base62-sha1="sZ5QHThvIalGXVJj5Atux6RbdS2" width="690" height="232" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/c/cb24d65b9a0ed2d4fcf5f66b869c41442098cdc2_2_690x232.png, https://discuss.pytorch.org/uploads/default/optimized/2X/c/cb24d65b9a0ed2d4fcf5f66b869c41442098cdc2_2_1035x348.png 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/c/cb24d65b9a0ed2d4fcf5f66b869c41442098cdc2.png 2x" data-dominant-color="F4F4F7"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">03-17-27-03</span><span class="informations">1100×370 48.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
I am trying to write the code to train the model and I get this error. The code is given below:</p>
<pre><code class="lang-auto">model = Classifier()
criterion = nn.NLLLoss()
optimizer = optim.Adam(model.parameters(), lr=0.003)

epochs = 30
steps = 0

train_losses, test_losses = [], []
for e in range(epochs):
    running_loss = 0
    for images, labels in trainloader:        
        optimizer.zero_grad()        
        log_ps = model(images)
        loss = criterion(log_ps, labels)
        loss.backward()
        optimizer.step()        
        running_loss += loss.item()
</code></pre>

username: ptrblck
cooked: <p>Make sure you pass the <code>mean</code> and <code>std</code> values as 1-dimensional tensors, e.g.:</p>
<pre><code class="lang-python">mean = torch.tensor([0.5, 0.5, 0.5])
</code></pre>
<p>Based on the error, it looks like you are trying to pass them as 0-dim tensors:</p>
<pre><code class="lang-python">mean = torch.tensor(0.5)
</code></pre>

username: Shashi_Gharti
cooked: <p>Thanks <a class="mention" href="/u/ptrblck">@ptrblck</a> for the hint. I changed my transform params from</p>
<pre><code class="lang-auto">transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.5), (0.5))
])

</code></pre>
<p>to</p>
<pre><code class="lang-auto">transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.5,), (0.5,))
])

</code></pre>
<p>and it worked!</p>
Torchvision custom transforms

username: n0obcoder
cooked: <p>how do i use torchvision.transforms on images read by opencv?</p>
<p>class resize_rescale():<br>
def <strong>init</strong>(self, output_size):<br>
assert isinstance(output_size, int)<br>
self.output_size = output_size</p>
<pre><code>def __call__(self, all_3_imgs):
    print('in resize_rescale !!!')
</code></pre>
<p>anc_img, pos_img, neg_img = all_3_imgs</p>
<pre><code>    return anc_img, pos_img, neg_img
</code></pre>
<p>dataset = siamese_dataset(att_faces_dataset,  transform = transforms.Compose([resize_rescale(256),<br>
]))</p>

username: Mdhvince
cooked: <p>If you want to transform your images using <code>torchvision.transforms</code>, they should be read by using <code>PIL</code> and not opencv.<br>
However Opencv is faster, so you need to create your own functions to transform your images if you want to use opencv.</p>

username: JuanFMontesinos
cooked: <p>Hi, There exist a drop-in library <a href="https://github.com/uploadcare/pillow-simd" rel="nofollow noopener">https://github.com/uploadcare/pillow-simd</a> which speeds up Pillow 4-6 times. I would recommend to use it instead of rewriting torchvision for opencv.</p>
<p>Anyway, as comment for the OP, Compose works with any function, if during the transformation chain you cast your data to numpy you may use whatever you want.</p>

username: n0obcoder
cooked: <p>i did read images using opencv, then i had to make my own transform functions. Then i used <strong>transforms.ToTensor()</strong>  It worked pretty well for me. thanks again !!! <img src="https://discuss.pytorch.org/images/emoji/apple/smile.png?v=9" title=":smile:" class="emoji" alt=":smile:"></p>
<p>class resize_rescale():<br>
def <strong>init</strong>(self, output_size):<br>
assert isinstance(output_size, int)<br>
self.output_size = output_size</p>
<pre><code>def __call__(self, img):         
    img = cv2.resize(img, (self.output_size, self.output_size)) #(width, height)
    return img
</code></pre>
<p>dataset = siamese_dataset(att_faces_dataset,<br>
transform = transforms.Compose([resize_rescale(224), transforms.ToTensor(),<br>
transforms.Normalize(mean=[0.485, 0.456, 0.406],<br>
std=[0.229, 0.224, 0.225])]))</p>

username: Mdhvince
cooked: <p>If you want to use opencv for transformation and image augmentation, have a look at  <a href="https://github.com/jbohnslav/opencv_transforms" rel="nofollow noopener">this repo</a>.</p>
Unfold second order derivative

username: Bruno1
cooked: <p>Hi,</p>
<p>I’m working on a custom layer which uses the unfold operator.<br>
I would like to compute the second order derivatives (for some hessian vector products…). However I have the following error:</p>
<blockquote>
<p>derivative for _thnn_im2col_backward is not implemented</p>
</blockquote>
<p>Can anyone tell me how to solve this? Or are there any workaround/hacks to solve this?</p>
<p>thanks for your help</p>

username: albanD
cooked: <p>Hi,</p>
<p>The problem is that the second derivative of the unfold operation is not implemented.<br>
You would need a function that computes this second derivative or the first derivative to be implemented in a differentiable way with the autograd engine.</p>

username: Bruno1
cooked: <p>Hello,</p>
<p>Thanks a lot!</p>
<blockquote>
<p>You would need a function that computes this second derivative</p>
</blockquote>
<p>How does it work exactly ? Should I write a backward for the backward ? I didn’t find many implementations of such layers.</p>
<blockquote>
<p>the first derivative to be implemented in a differentiable way with the autograd engine.</p>
</blockquote>
<p>Do you mean that as long as the backward is implemented in a differentiable way, it will automatically handle double diff ?</p>
<p>thank you</p>

username: albanD
cooked: <p>Hi,</p>
<p>When implemented in python, a forward/backward pair is defined by a Function. You can see <a href="https://pytorch.org/docs/stable/notes/extending.html" rel="nofollow noopener">here</a> in the doc how in details.<br>
By default, if higher order derivatives are required, the autograd is applied to the backward function that is defined.<br>
You can manually defined second derivatives by using 2 Functions, where the backward of the first one is simply calling the forward of the second one.<br>
You also have a decorator <code>@oncedifferentiable</code> to notify the engine that a backward is implemented in a non differentiable manner. If you ask for second derivatives for such a function, the error you see is raised.</p>
<p>For the cpp implementations, most of them are not implemented in a differentiable manner hence the error you see.</p>

username: Bruno1
cooked: <p>Awesome, it really helps ! Thank you</p>
<p>Bruno</p>
How the function torch.nn.AdaptiveAvgPool2d(output_size) works?

username: Andybert
cooked: <p>I have tried torch.nn.AdaptiveAvgPool2d(output_size) as follow:</p>
<p>m = nn.AdaptiveAvgPool2d((4,1))<br>
a=torch.randn(1, 64, 7, 9)<br>
output = m(input)</p>
<p>then I got the following results:</p>
<p>output[0,1,0]=torch.mean(a[0,1,0:2])<br>
output[0,1,1]=torch.mean(a[0,1,1:4])<br>
output[0,1,2]=torch.mean(a[0,1,3:6])<br>
output[0,1,3]=torch.mean(a[0,1,5:7])</p>
<p>Do anyone know how the function torch.nn.AdaptiveAvgPool2d(output_size) works?  Or can someone tell me where I can find the source code of torch.nn.AdaptiveAvgPool2d(output_size)?</p>

username: albanD
cooked: <p>Hi,</p>
<p>The doc can be found <a href="https://pytorch.org/docs/stable/nn.html?highlight=adaptiveavgpool2d#torch.nn.AdaptiveAvgPool2d">here</a> and the cpu implementation <a href="https://github.com/pytorch/pytorch/blob/a08119afc27f44ff488cbf18c11659d255f378a8/aten/src/THNN/generic/SpatialAdaptiveAveragePooling.c">here</a>.</p>

username: n0obcoder
cooked: <p>how do i use a linear layer after the AdaptiveAvgPool2d layer?</p>
<p>Consider this example:</p>
<p>m = nn.AdaptiveAvgPool2d((5,7))<br>
dense = nn.Linear(5, 10)</p>
<p>input = torch.randn(1, 64, 8, 9)</p>
<p>output = m2(input) # its shape should be (1, 64, 5, 7)<br>
output2 = dense(output)  # this line throws an error.</p>
<p>Please help me with this. Thanks in advance ! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: ptrblck
cooked: <p>Answered <a href="https://discuss.pytorch.org/t/how-to-use-linear-layer-after-adaptiveavgpool2d/46951/2">here</a>.</p>
Better way to expand along non singleton dimensions?

username: whoab
cooked: <p>Is there a better way to accomplish this?</p>
<pre><code class="lang-auto">mask = torch.tensor([[1, 0], [0, 1]])
for i in range(6):
    mask = torch.cat((mask, torch.tensor([[1, 0], [0, 1]])))
</code></pre>

username: ptrblck
cooked: <p><code>repeat</code> should yield this output:</p>
<pre><code class="lang-python">x = torch.tensor([[1, 0], [0, 1]])
x = x.repeat(7, 1)
</code></pre>
ResNet50 Error with Multiple outputs

username: AmardeepGanguly
cooked: <p>I am trying to modify the resnet50 to give multiple outputs .However I am getting a Dimension Mismatch Error<br>
This is my code</p>
<pre><code class="lang-auto">class ResNet(nn.Module):
    def __init__(self):
        super(ResNet, self).__init__()
        resnet = models.resnet50(pretrained=True)
        modules = list(resnet.children())[:-1]
        self.resnet = nn.Sequential(*modules)
        self.fc=nn.Linear(2048,10)

    def forward(self, x):

        x1 = self.resnet(x)  
        x1=self.fc(x1)
        x1 = x1.view(x1.size(0), -1)  
        
        x2 = self.resnet(x) 
        x2=self.fc(x2)
        x2= x2.view(x2.size(0), -1)
       
        
        return x1,x2
</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/d/d14bceab590931edd79ca865ed111325cbcd1af4.png" data-download-href="https://discuss.pytorch.org/uploads/default/d14bceab590931edd79ca865ed111325cbcd1af4" title="Capture"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/d/d14bceab590931edd79ca865ed111325cbcd1af4_2_690x174.png" alt="Capture" data-base62-sha1="tRwcStmwcWXnRLARFs3OQZ37q0Q" width="690" height="174" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/d/d14bceab590931edd79ca865ed111325cbcd1af4_2_690x174.png, https://discuss.pytorch.org/uploads/default/original/2X/d/d14bceab590931edd79ca865ed111325cbcd1af4.png 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/d/d14bceab590931edd79ca865ed111325cbcd1af4.png 2x" data-dominant-color="292D32"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Capture</span><span class="informations">896×226 24.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <p>Would <a href="https://discuss.pytorch.org/t/modify-resnet50-to-give-multiple-outputs/46905/2">this</a> answer from the other topic work?</p>
Using part of the layers in a pre trained model

username: danielafrimi
cooked: <p>Hi all,<br>
I want to build perceptual loss for video, it means that my loss contain a pre trained net (in my work im thinking of using resnet 3D which trained for video recognition task), and i want to pass my generated video and the real video through the net, and take the output of it in some layers (i.e after layer3, layer5… for each video).<br>
i know that in the package “models” in pytorch i can load specific pre trained models, and using the features function on the input. like this for example:<br>
self.vggnet=models.vgg16(pretrained=True).cuda()<br>
for i in range(id_max):<br>
x  = self.vggnet.features<a>i</a> //z is the input of the net</p>
<p>I tried to use “features” on my 3D resnet but its not working. does anyone know how to use some layers? instead of changing the forward function of the model…</p>
<p>this is the way i load it<br>
<img src="https://discuss.pytorch.org/uploads/default/original/2X/e/e2e76bfa8800547cf983cae8fc1670d6352a2aba.png" alt="image" data-base62-sha1="wnhJ5rEjbDN0sNRk59UsklBonTs" width="539" height="128"><br>
this is where i defined the net<br>
<img src="https://discuss.pytorch.org/uploads/default/original/2X/1/121f972e1cb71f1581185dc22823f94f8bdc823e.png" alt="image" data-base62-sha1="2AkhapTBYemQQi07qB4SznXm2qa" width="670" height="102"></p>

username: ngcthuong
cooked: <p>Did you find out the solution?</p>

username: danielafrimi
cooked: <p>I changed the forward function of the pretrained model that i used. instead of returning one argument, the function return an array</p>
Joint Transforms for torchvision.datasets.VOCSegmentation

username: dwkang
cooked: <p>Hello,</p>
<p>I downloaded the latest torchvision from source and am trying to apply the same data augmentation (e.g. horizontal flip) to both the input and the target. The VOCSegmentation class has a transforms argument, but I’m not sure how to use it. Would I have to write my own transform?</p>

username: dwkang
cooked: <p>I tried writing my own function</p>
<pre><code class="lang-auto">class JointHorFlip(object):
    def __call__(self, img, target):
        if random.random() &lt; .5:
            return (F.hflip(img), F.hflip(target))
        return (img, target)

train_set = torchvision.datasets.VOCSegmentation(
            root = '../data',
            year = '2012',
            image_set = 'train',
            transforms = JointHorFlip(),
    )
</code></pre>
<p>but when I check through the images, the inputs and targets are still not consistent?</p>

username: ptrblck
cooked: <p>The code looks fine. I tried to debug it, but apparently the download site is down since yesterday: <a href="http://host.robots.ox.ac.uk/pascal/VOC/voc2012/" rel="nofollow noopener">http://host.robots.ox.ac.uk/pascal/VOC/voc2012/</a></p>
<p>CC <a class="mention" href="/u/fmassa">@fmassa</a>: did they move the dataset or is the site just down at the moment?</p>
How to write LeakyRELU in pytorch

username: smani
cooked: <p>I am almost new to PyTorch. I would like to implement LeakyRELU in Pytorch from scratch, writing it as a class with the forward method. Could someone help me with writing? Should I write the backward method too as PyTorch already supports for auto differentiation?</p>

username: albanD
cooked: <p>Hi,</p>
<p>In pytorch you don’t need to have a class, your leaky relu can be a simple python function that given an input returns the output after the leaky relu.<br>
If you want to use it in nn.Sequential or similar elements, you can wrap that function in an nn.Module where you need to define the <code>__init__</code> function if needed and the <code>forward</code> function that will correspond to applying the leaky relu to an input.</p>