akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Autoencoder result color distortion

username: haeminjung
cooked: <p>I have posted a question about the normalized image seems to be a bit “off”</p><aside class="quote quote-modified" data-post="1" data-topic="76138">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/haeminjung/48/22573_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/dataloader-loading-color-distorted-image/76138">Dataloader loading color distorted image</a> <a class="badge-category__wrapper " href="/c/vision/5"><span data-category-id="5" style="--category-badge-color: #AB9364; --category-badge-text-color: #FFFFFF;" data-drop-close="true" class="badge-category " title="Topics related to either pytorch/vision or vision research related topics"><span class="badge-category__name">vision</span></span></a>
  </div>
  <blockquote>
    Quick show and tell will be the easiest way to tell you the problem. 
# Define the dataset and the loader as usual
dataset = dset.ImageFolder(root=datapath,
                           transform=transforms.Compose([
                                transforms.Resize(image_size),
                                #transforms.CenterCrop(image_size),
                                transforms.ToTensor(),
                                transforms.Normalize((0.5414, 0.5333, 0.5338),(0.1897, 0.1911, 0.19…
  </blockquote>
</aside>

<p>Relating to the problem, the output of the image compared to the input seems rather color distorted.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/8/f88846b4b7d7b1d3182b5e90bf8cee9b0b0c2445.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/f88846b4b7d7b1d3182b5e90bf8cee9b0b0c2445" title="download (2)"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/8/f88846b4b7d7b1d3182b5e90bf8cee9b0b0c2445_2_690x319.jpeg" alt="download (2)" data-base62-sha1="zsCnKwWHP71m90xQ687EiTs61RX" width="690" height="319" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/8/f88846b4b7d7b1d3182b5e90bf8cee9b0b0c2445_2_690x319.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/f/8/f88846b4b7d7b1d3182b5e90bf8cee9b0b0c2445_2_1035x478.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/f/8/f88846b4b7d7b1d3182b5e90bf8cee9b0b0c2445_2_1380x638.jpeg 2x" data-dominant-color="8D8C8C"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">download (2)</span><span class="informations">2836×1315 1.63 MB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I just one someone to point out if this is a problem that convT faces in definition, or if it’s me that’s doing something wrong?</p>
<p>The papers that I had seen on GANs, and Autoencoders never seemed to have such problems.</p>
How to use datasets.ImageFolder() for custom datasets?

username: Ratan
cooked: <p>uppose I have to design a 3 class classifier with following things in mind:</p>
<p>I have 3 folders as datasets such as:</p>
<ul>
<li>…/data/cat/ …/data/dog/ …/data/man/</li>
</ul>
<p>Each folder has images for that particular class only.</p>
<p>We divide the images into train,test,val using the following:</p>
<p>train_data = datasets.ImageFolder(train_dir, transform=data_transforms[‘train’])</p>
<p>test_data = datasets.ImageFolder(test_dir, transform=data_transforms[‘test’])</p>
<p>My question is how will ImageFolder() divide the images into train,test from EACH class folder as given above??</p>
<p>Also,</p>
<p>img,target = netx(iter(…))</p>
<p>gives the target or label. How is this generated? like how will ImageFolder() assign target=1 when taking image from …/data/cat/ and target=1 when taking from …/data/dog/ etc</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="Ratan" data-post="1" data-topic="78247">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ratan/48/12876_2.png" class="avatar"> Ratan:</div>
<blockquote>
<p>My question is how will ImageFolder() divide the images into train,test from EACH class folder as given above??</p>
</blockquote>
</aside>
<p>It won’t divide the folders automatically. <code>ImageFolder</code> takes the root folder as an argument and will use all images from all subfolders as data samples.</p>
<p>To split the dataset, you could use <code>torch.utils.data.random_split</code>, <code>torch.utils.data.Subset</code> passing the indices, or <code>torch.utils.data.sampler.SubsetRandomSampler</code> also passing the indices.</p>
<aside class="quote no-group" data-username="Ratan" data-post="1" data-topic="78247">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ratan/48/12876_2.png" class="avatar"> Ratan:</div>
<blockquote>
<p>My question is how will ImageFolder() divide the images into train,test from EACH class folder as given above??</p>
</blockquote>
</aside>
<p><code>ImageFolder</code> will sort all subfolders and assign a class label to each of them as seen in <a href="https://github.com/pytorch/vision/blob/3e60fb29986ad74d9e4dc5715479d7bd3d8b59ba/torchvision/datasets/folder.py#L108-L144">these lines of code</a>.</p>
How to implement torch.argmax in java

username: Rokuki
cooked: <p>Here is the code that I wanna implement in java for the PyTorch mobile application, but there is seem like no related API.</p>
<pre><code class="lang-python">pred = torch.argmax(outputs[0], 1)
pred = pred.cpu().data.numpy()
predict = pred.squeeze(0)
</code></pre>
<p>java</p>
<pre><code class="lang-java">final IValue[] outputTuple = module.forward(IValue.from(inputTensor)).toTuple();
final Tensor outputTensor = outputTuple[0].toTensor();
Log.d(TAG, "onCreate: " + outputTensor.getDataAsFloatArray());
</code></pre>
<p>The FloatArray is 1-dim array, it really rack my brains, Could anyone please tell me a clue. thank you</p>

username: Rokuki
cooked: <p>Put the code in <code>forward</code> method solved this problem. <img src="https://discuss.pytorch.org/images/emoji/apple/zipper_mouth_face.png?v=9" title=":zipper_mouth_face:" class="emoji" alt=":zipper_mouth_face:"></p>
<pre><code class="lang-auto">def forward(self, x):
        ...
        pred = torch.argmax(outputs[0], 1)
        pred = pred.cpu().data
        predict = pred.squeeze(0)
        predict = predict.type(torch.FloatTensor)

        return predict
</code></pre>
<p>Run the traced_script and get a new model.pt, put it in assets, the output tensor will change and could be use directly.</p>
Slice matrix with another matrix

username: mwosinski
cooked: <p>Hi,<br>
I would like the following without using for loop:</p>
<pre><code class="lang-auto">a=torch.tensor([[1,2,3,4],[5,6,7,8]])
b=torch.tensor([[1,3],[0,2]])

for i in range(len(a)):
    c = a[i][b[i]]
    print(c)
</code></pre>
<p>a - matrix of values,<br>
b - matrix of indices</p>
<p>Any ideas?</p>

username: mwosinski
cooked: <p>All right, found!</p>
<pre><code class="lang-auto">a=torch.tensor([[1,2,3,4],[5,6,7,8]])
b=torch.tensor([[1,3],[0,2]])

c=a[torch.arange(a.size(0)).unsqueeze(1),b]
</code></pre>
Use same seed for transform function across all images in a batch

username: kshitij98
cooked: <p>I am training a model which requires variable sized inputs ((4, 4), (8, 8), (16, 16)) for data augmentation.</p>
<p>I am using the following the transform function to achieve the same</p>
<pre><code class="lang-auto">        sizes = [(4, 4), (8, 8), (16, 16)]
        
        ResizeTransformation = Transforms.RandomChoice(
            [Transforms.Resize(size) for size in sizes])

        transformList = [Transforms.RandomRotation(180, Image.BILINEAR),
                         Transforms.CenterCrop(220),
                         Transforms.RandomCrop(128),
                         Transforms.RandomHorizontalFlip(),
                         ResizeTransformation,
                         Transforms.ToTensor(),
                         Transforms.Normalize((0.1483, 0.1210, 0.1257), (0.1292, 0.1050, 0.1186))]
</code></pre>
<p>But this creates an issue when using a batch size &gt; 1, As there are cases when images in the batch are resized to different dimensions.</p>
<p>I’m thinking that If we use the same seed value while transforming all the images in a batch, we can get a single tensor of shape (N, C, H, W) as the transform function will resize to the same dimensions.</p>
<p>I have looked up on the forum that we can create a list of tensors of batch size of different sizes by using a custom collate_fn, but then aren’t we training the network on a batch_size = 1 only as we’ll have to pass all the images to the networks separately since they are of different sizes?</p>
<p>Looking for a way to create single tensor of dimensions (N, C, H, W) with variable sized inputs.</p>
<p>TIA</p>

username: ptrblck
cooked: <p>Maybe it would be easier to remove the <code>ResizeTransformation</code> from <code>transformList</code> and apply <code>F.interpolate</code> on the complete batch inside the <code>DataLoader</code> loop?<br>
This would of course not be applied from each worker in <code>Dataset.__getitem__</code>, but might not necessarily be slower.</p>
<p>Would that work for you? I’m always a bit skeptical when it comes to using the seeding mechanism to fix these random transformations, as all other “random” transformations will also be fixed (e.g. <code>RandomRotation</code>).</p>
Training CNN with my own dataset

username: PKUFlyingPig
cooked: <p>I want to train a CNN with my own dataset, and I follow the guidance to define my own data set use torchvision.utils.data.Dataset. I create a txt file which contains all the paths to all my training data, and in the <strong>getitem</strong> function it opens an image file and returns the image, but I find this way too slow, maybe because in each epoch it will open this image again and read it. So should I read all the images at a time(e.g. define a global variable trainX which shaped (trainingdata_num, feature, w ,h) )? or maybe there are some other clever way?</p>
<p>below is my current codes<br>
the variable self.data contains all the paths of the training data<br>
‘’’<br>
class MyDataset(Data.Dataset):<br>
def <strong>init</strong>(self,txt_path,img_path,transform=None,have_label=True):<br>
self.img_path=img_path<br>
self.have_label=have_label<br>
self.transform=transform<br>
data=[]<br>
with open(txt_path) as f:<br>
for line in f:<br>
if self.have_label == True:<br>
data.append((line.strip(),int(line.split("_")[0])))<br>
else :<br>
data.append(line.strip())<br>
self.data=data</p>
<p>def <strong>getitem</strong>(self,index):<br>
if self.have_label == True:<br>
name,label=self.data[index]<br>
else :<br>
name=self.data[index]<br>
img=Image.open(os.path.join(self.img_path,name)).convert(‘RGB’)<br>
img=img.resize((128,128))<br>
if self.transform is not None:<br>
img=self.transform(img)<br>
if self.have_label == True:<br>
return img,label<br>
else :<br>
return img</p>
<p>def <strong>len</strong>(self):<br>
return len(self.data)<br>
‘’’</p>

username: ptrblck
cooked: <p>If your dataset is small and fits into your RAM, you might pre-load it of course and store the images in e.g. a list.<br>
Note however, that his approach would add a startup latency, since you are loading all the files before the training can begin.<br>
The lazy loading approach is used for larger datasets, which don’t fir into your memory or if you want to avoid the startup latency.</p>
<p>Also note, that you would have to transform the tensors to <code>PIL.Image</code>s and back to tensors in case you are storing the inputs as tensors.</p>
torchtext.data.TabularDataset read too big csv, kernel died

username: tiancaigg
cooked: <p>when torchtext.data.TabularDataset read big csv, it takes all memory then the jupyter lab kernel died.<br>
I tried to splits the training dataset into 2 csv, reading the first csv succeeded, but when reading the 2nd csv, it died again.<br>
Is there any method to use TabularDataset as generator?</p>
MaskRCNN Mask prediction question

username: jackck
cooked: <p>I am studying MaskRCNN and Detectron2.    My understanding is the mask head and the bbox regression heads make independent predictions and trained in parallel with a joint loss function.</p>
<p>The RPN gives an approximate bbox which gets feed into both heads.  As far as I understand, the mask head shouldn’t have knowledge about the bbox regression head’s final predicted bbox.</p>
<p>But I notice the predicted mask doesn’t extend outside of the predicted bbox (is this even true?). Why does the predicted mask neatly fits inside the final predicted bbox?  It seems to me the predict mask should be bounded by the rough RPN bbox which could extend beyond the final bbox.</p>
<p>Please help me understand.</p>
Saving segmentation output for U-net

username: Jay_Super
cooked: <p>Hi, I’m currently training a U-net for lung nodule detection. I would like to store the segmented results to use it for classification of whether the nodule itself is cancerous. My question is, how do you usually save the segmented results? Previously in Keras, I saved the output file as an .npy files for each slice. For instance, slice001.npy, slice002.npy …<br>
I would then concatenate all of the slices for classification. But is there a smarter way?</p>
Linear relation between batch size and inference time per batch

username: markus1
cooked: <p>Hi,</p>
<p>I made the following observation which I can not follow. I am using the deeplabv3_resnet50 model from torchvision and run it in eval mode with different batch sizes. The runtime including a torch.synchronize() are almost linear with the batch size. That means that the rate of images/second is almost constant. I also tried out different settings of cudnn.benchnmark = True/False. I was hoping to see a constant inference time per batch over all batch sizes because they are running in parallel on the gpu. Am I getting anything wrong here?<br>
2 - 0.02s<br>
4 - 0.031s<br>
8 - 0.05s<br>
16 - 0.094s<br>
32 - 0.178s</p>
<p>Thanks for your help.<br>
Markus</p>
<p>torch=1.4.0<br>
CUDA Version: 10.0<br>
Tesla V100</p>

username: ptrblck
cooked: <p>If I understand the results correctly, the throughput would increase from 100 images/s to 180 images/s, which is not almost constant.</p>
<p>Generally, you should see a better throughput by increasing the batch size.<br>
However, the overall performance also depends on other potential bottlenecks in the model architecture as well as the training routine, such as data loading.</p>
<p>I would also recommend to try out the latest stable release, as some performance improvements were made. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

username: markus1
cooked: <p>You are right, it is not constant but still shouldn’t there be a larger increase in  throughput if the batch size is x16 ?<br>
I measured the time only for the forward pass through the network. So I think data loading should not be an issue.<br>
I will try the latest release.</p>

username: ptrblck
cooked: <p>It depends on the utilization of the GPU and how/if the operations are already parallelizing in other dimensions than the batch size, such as the spatial dimensions of your input tensors.<br>
If your GPU utilization is already high, you won’t see a linear speedup.</p>
<p>While training, you will of course use less iterations per epoch, which is another speedup, but unrelated to your use case.</p>
Pretty model visualization

username: vliu15
cooked: <p>I’ve recently created a tool for visualization pytorch models and released a python package today. Just thought I’d share here, feel free to check out!</p>
<p>Github: <a href="https://github.com/vliu15/visunn" rel="nofollow noopener">https://github.com/vliu15/visunn</a><br>
PyPi: <a href="https://pypi.org/project/visunn/" rel="nofollow noopener">https://pypi.org/project/visunn/</a></p>
How to build a CNN BLSTM Hybrid Model?

username: Bipasha_Sen
cooked: <p>Hi,</p>
<p>I’ve been trying to implement an end to end acoustic model for speech recognition and for that I’m using normalised raw signal as my input. I’m trying to implement the following 2 architectures:</p>
<ol>
<li>CNN + DNN</li>
<li>CNN + BLSTM.</li>
</ol>
<p>For model 1, the loss is remaining constant mostly while for model 2 the loss is increasing, sometimes decreasing and sometimes going nan. I can’t seem to understand how to get it to work since a lot of papers have used this approach.</p>
<p>I’ve tried reducing the learning rate to very very low, 0.0001, I’ve tried gradient clipping but they don’t seem to work. I’ve also tried simpler architecture with few or only one layer but that doesn’t seem to work either.</p>
<p>Model 1 is using relu activation for all the layers while model 2 is using tanh for all the layers. The input wav signal is 400 dimensional vector. There are two things I don’t understand.</p>
<ol>
<li>
<p>Do I need to use a linear layer between CNN and BLSTM layer? If yes then why? If I use relu activation for the last layer of CNN would I still need the linear layer?</p>
</li>
<li>
<p>What is a dense layer? I’ve seen this term which comes after the LSTM unit.</p>
</li>
</ol>
<p>Is there any available code snippet that I can use for the implementation of such an architecture?</p>
GAN Training Instability

username: Jicheng_Li
cooked: <p>Hi guys,</p>
<p>While I was training DCGAN, I got a loss curve as follows.</p>
<p>It seems that after a certain number of (~100) iterations, generator loss will increase tremendously.</p>
<p>Any thoughts on why this will happen?<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/f/9/f9e6d1f39d12269ce9c331b436b3ea99827eaa30.png" alt="logging" data-base62-sha1="zEJq2nXtWPJpcboPcgXG7aAq2pG" width="640" height="480"></p>

username: ebarsoum
cooked: <p>Can you provide more details? How you iterate between G and D? Dataset and its size? LR? Does the discriminator loss converge (from the plot it looks like a straight line)?</p>
<p>GAN is difficult to train, there are a lot of tips and tricks that you can try.</p>
Problem in NN, I am a beginner

username: ulhaqi12
cooked: <p>class Classifier(nn.Module):</p>
<pre><code>def __init__(self):
    super().__init__()
    
    self.layer1 = nn.Linear(20,17)
    self.layer2 = nn.Linear(17,14)
    self.layer3 = nn.Linear(14,10)
    self.layer4 = nn.Linear(10,8)
    self.layer5 = nn.Linear(8,4)

def forward(self,x):
    
    x = F.relu(self.layer1(x))
    x = F.relu(self.layer2(x))
    x = F.relu(self.layer3(x))
    x = F.relu(self.layer4(x))
    x = F.softmax(self.layer5(x))
    
    return x
</code></pre>
<ul>
<li>Here training code for NN</li>
</ul>
<p>model = Classifier()</p>
<p>optimizer = optim.SGD(model.parameters(), lr=0.005)</p>
<p>criterian = nn.CrossEntropyLoss()</p>
<p>train_data = train_data.float()<br>
train_lable = train_label.float()</p>
<p>epoch = 500<br>
for i in range(epoch):</p>
<pre><code>optimizer.zero_grad()
output = model(train_data)
loss = criterian(output,train_label)
loss.backward()
optimizer.step()

print('At ',i,'/50 epoch loss is: ',loss.item())
</code></pre>
<p>I have this code for my NN and having very bad results after learning. I think there is some problem with calculating loss. I am using softmax regression for multiclass classification. Dataset is fine and well organized as it is preprocessed and available on Kaggle.<br>
I would be great if anyone will help me here.</p>
<p>i have output as class either 0, 1, 2 or 3.<br>
what changes need to be done here?</p>

username: KFrank
cooked: <p>Hello Ikram!</p>
<aside class="quote no-group quote-modified" data-username="ulhaqi12" data-post="1" data-topic="78095" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ulhaqi12/48/23221_2.png" class="avatar"> ulhaqi12:</div>
<blockquote>
<pre data-code-wrap="python"><code class="lang-python">        x = F.softmax(self.layer5(x))
        
        return x
...
criterian = nn.CrossEntropyLoss()
</code></pre>
</blockquote>
</aside>
<p>For starters, get rid of the final <code>softmax()</code> activation and pass the<br>
result of your last linear layer directly to <code>CrossEntropyLoss</code>.</p>
<p>As mentioned (but not really emphasized) in the <a href="https://pytorch.org/docs/stable/nn.html#torch.nn.CrossEntropyLoss" rel="noopener nofollow ugc">CrossEntropyLoss</a><br>
documentation, <code>CrossEntropyLoss</code> expects raw-score <em>logits</em><br>
(rather than the probabilities that are produced by <code>softmax()</code>).</p>
<p>Best.</p>
<p>K. Frank</p>

username: ulhaqi12
cooked: <p>Thank you so much KFrank,<br>
It worked.</p>
Variable Length Reduce

username: ArEsKay3
cooked: <p>I’m looking for an efficient way to do a reduce that looks like this without using the for loop because the for loop is WAY too slow. Any ideas on how to do this variable length reduce?</p>
<p>Input: I (bool) [M,1] , Start (int) [N,1], Stop (int) [N,1] -&gt; Output: O (bool) [N,1]</p>
<pre><code class="lang-auto">for n in range(N):
    O[n] = torch.all(I[Start[n]:Stop[n]])
</code></pre>

username: ArEsKay3
cooked: <p>For now I’ve found this to work OK.</p>
<pre><code class="lang-auto">s = torch.cumsum(I, dim=0)
x = s.index_select(0,start)
y = s.index_select(0,stop)
O = ((y-x) == (stop-start))
</code></pre>
Freelancers required for deep learning

username: bluemoon
cooked: <p>For the implementation of unsupervised segmentation of my image data, I am looking for freelancers support. Please contact me (<a href="mailto:blue69moon@protonmail.com">blue69moon@protonmail.com</a>) if you are interested in freelancing with me through freelancing platform like Fiverr or Upwork. Thank you.</p>
Fastest way to load vectors on-the-fly for training

username: Kevin_Shen
cooked: <p>Suppose I’m using the Dataset and Dataloader classes to load vectors (e.g. sentence embeddings trained from another model) on-the-fly (meaning I only load e.g. batchsize=16 embeddings at a time for the batch) for training my model. I might want to do this if the number of embeddings I have is large and I can’t fit it all into memory.</p>
<p>What is the best format to save the sentence embeddings in order to speed up the on-the-fly loading and hence training? What is the best way to load them?</p>
<p>The embeddings have size of 1024 dimensions. They can be thought of as being convertible to numpy arrays or torch tensors. I think the values of the embedding are quite random since they’re the latent vector of another model.</p>

username: Hartbook
cooked: <p>I think the best way would be  to use torch.save and torch.load.</p>
<p>But as IO is extremely slow I think you should avoid loading batches 1 per 1.<br>
Even if your entire dataset cannot fit into memory, I’m sure you can fit more than only one batch.<br>
Here is how I do it in my Dataset class<br>
On my disk I have saved several files containing my training examples, each file contains 250 000 examples and it’s just a single tensor of dims (250000,firstDim,secondDim…) saved using torch.save.<br>
My Dataset loads only one of these files using torch.load, and when it is asked for a batch, I simply select the batch from my giant tensor using tensor.narrow and I clone the result into GPU memory (to ensure minimal GPU usage, only one batch is loaded into GPU). When the dataloader run out of batches to select from the giant tensor, it simply load it again from a different file.<br>
I found that this method is really efficient because I/O is minimal, and you can really tune it depending on how much RAM you want to use.</p>

username: Kevin_Shen
cooked: <p>This makes sense except you can never get fully random data shuffling for each epoch, no?</p>

username: Hartbook
cooked: <p>Yes exactly, what I do is that I shuffle the file load order between each epoch and in my case it’s good enough. If you want something closer to fully random data shuffling, you could shuffle the big tensor just after you load it from the file (following <a href="https://discuss.pytorch.org/t/shuffling-a-tensor/25422/5" class="inline-onebox">Shuffling a Tensor</a> ). I think it should be sufficient.</p>

username: Kevin_Shen
cooked: <p>Ya seems reasonable. Do you have a rough sense of the relative loading time overhead in all 3 methods?</p>
<ol>
<li>Loading batch by batch</li>
<li>Loading groups of batches</li>
<li>Loading entire dataset at once (if it hypothetically did fit in memory)</li>
</ol>
<p>I guess I’m thinking in terms of X times slowdown for 1 and 2 versus 3. Maybe just some rough guesses would be nice.</p>
Problem in converting GRU to jit

username: lzj9072
cooked: <p>I have trained  a model containing GRU. When I try to convert it to jit, I found that the outputs of the original model and jit model are different. This problem can be reconstructed by the code below. It’s very strange that other operation(like fc/conv) is ok, the output of print(model(y)-jit_model(y)) is an zero matrix whhile GRU is not.</p>
<pre><code class="lang-auto">import torch
from torch import nn

class gruModel(nn.Module):
    def __init__(self):
        super(gruModel, self).__init__()
        self.biGRU = nn.GRU(256*5, 100,  num_layers=1, bidirectional=True, batch_first=True, bias=True)
        # self.fc = nn.Linear(256*5, 200)

    def forward(self, x):
        # x = self.fc(x)
        x, _ = self.biGRU(x, torch.zeros(2, x.size(0), 100, device=x.device))
        return x

if __name__ == '__main__':
    y = torch.rand([1, 256, 1280]).cuda()
    model = gruModel()
    model = torch.nn.DataParallel(model).to(torch.device('cuda'))
    model.eval()

    traced_script = torch.jit.trace(model.module, y)
    traced_script.eval()
    traced_script.save("gru_jit.pt")
    jit_model = torch.jit.load("gru_jit.pt")

    print(model(y)-jit_model(y))
</code></pre>

username: lzj9072
cooked: <p>I have found the solution…<br>
if the model contains GRU layer, you should load the model in cpu then to the gpu.</p>
<pre><code class="lang-auto">model = torch.jit.load(model_path, map_location=torch.device("cpu"))
model = model.cuda()
</code></pre>

username: eellison
cooked: <p>Hi <a class="mention" href="/u/lzj9072">@lzj9072</a> glad you found the solution to your problem.  There’s an issue out for this problem here you can follow: <a href="https://discuss.pytorch.org/t/problem-in-converting-gru-to-jit/78154" class="inline-onebox">Problem in converting GRU to jit</a></p>
CUDA C++ stream and state

username: shaibagon
cooked: <p>I am trying to follow <a href="https://pytorch.org/tutorials/advanced/cpp_extension.html" rel="nofollow noopener">this tutorial</a> and make a simple c++ extension with CUDA backend.<br>
My CPU implementation seems to work fine.</p>
<p>I am having trouble finding examples and documentation (it seems like things are constantly changing).</p>
<p>Specifically,</p>
<ol>
<li>
<p>I see pytorch cuda functions getting <code>THCState *state</code> argument - where does this argument comes from? How can i get a <code>state</code> for my function as well?</p>
</li>
<li>
<p><code>state</code> is then converted to <code>cudaStream_t stream = THCState_getCurrentStream(state);</code> For some reason, <code>THCState_getCurrentStream</code> is no longer defined? How can I get the <code>stream</code> from my <code>state</code>?</p>
</li>
</ol>
<p>Thanks,</p>

username: igor_davidyuk
cooked: <p>On the second question:<br>
I am not an expert, but I dare to suggest using<br>
c10::cuda::getCurrentCUDAStream()<br>
c10::cuda::setCurrentCUDAStream()<br>
instead.</p>
Tensor slicing using integer tensor of ranges

username: Peter_Featherstone
cooked: <p>I have the following code:</p>
<p>input   = torch.zeros(2,4,8448,2)<br>
buf      = torch.zeros(2,4,32,512,2)<br>
blocks = torch.arange(32)<br>
starts  = blocks * 256<br>
stops  = starts + 512</p>
<p>for i in range(32):<br>
buf[…,blocks[i], 0:512,:] = input[…,starts[i]:stops[i], :]</p>
<p>Now i would love to do:<br>
buf[…,blocks,0:512,:] = input[…,starts:stops,:]<br>
but this isn’t legal in pytorch.<br>
Is there a cheat way of doing this? the for loop is a real killer</p>
1.5 New Indexing API

username: Hartbook
cooked: <p>Hello,</p>
<p>With the 1.5 version, we finally have API parity between Python and c++ and it’s awesome.<br>
I would like to know if this could mean simpler code in my use cases, I’ve never used python API and I don’t really understand the syntax.</p>
<p>Here’s how I do it for now<br>
From a 3d tensor, I extract the first and last columns of the dimension number 1</p>
<pre><code class="lang-auto">torch::cat({lstmOut.narrow(1,0,1).squeeze(1), lstmOut.narrow(1,lstmOut.size(1)-1,1).squeeze(1)},1);
</code></pre>
<p>Thank you</p>
Difference between ConcatDataset & ChainDataset

username: Chris_Fleetwood
cooked: <p>Hi there,<br>
The documentation for ConcatDataset &amp; ChainDataset isn’t exactly clear (at least to me).</p>
<p>If I wanted to have a dataloader that creates batches of:</p>
<ol>
<li>A matrix</li>
<li>A vector</li>
<li>A label</li>
</ol>
<p>Does ConcatDataset allow me to iterate over the matricies and the vector from 2 different datasets simulatenously? And the shuffle will return the corresponding matrix and vector?</p>
<p>Many Thanks</p>

username: ptrblck
cooked: <p><code>ChainDataset</code> is used for <a href="https://pytorch.org/docs/stable/data.html#torch.utils.data.IterableDataset">IterableDatasets</a>, while <code>ConcatDataset</code> is used for the map-style datasets.</p>
<aside class="quote no-group" data-username="Chris_Fleetwood" data-post="1" data-topic="78031">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/chris_fleetwood/48/12567_2.png" class="avatar"> Chris_Fleetwood:</div>
<blockquote>
<p>Does ConcatDataset allow me to iterate over the matricies and the vector from 2 different datasets simulatenously?</p>
</blockquote>
</aside>
<p>No, as <code>ConcatDataset</code> will concatenate the passed datasets and won’t yield the samples simultaneously.<br>
You could <code>zip</code> the <code>DataLoaders</code> and iterate them together:</p>
<pre data-code-wrap="python"><code class="lang-python">dataset1 = TensorDataset(torch.zeros(10, 1), torch.zeros(10, 1))
dataset2 = TensorDataset(torch.ones(10, 1), torch.ones(10, 1))

loader1 = DataLoader(dataset1, num_workers=2, batch_size=2)
loader2 = DataLoader(dataset2, num_workers=2, batch_size=2)

for (x1, y1), (x2, y2) in zip(loader1, loader2):
    print(x1, y1)
    print(x2, y2)
</code></pre>

username: Chris_Fleetwood
cooked: <p>Hi Piotr,<br>
Many thanks for the thorough reply I have zipped 2 loaders together, and your continual replies all over the forum they’re a great help to a huge number of people.</p>
Pip installation problem on ubuntu

username: Manik_Hossain
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/e/1ea9b3481369e8344d92b73a103b3db32cb10434.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/1ea9b3481369e8344d92b73a103b3db32cb10434" title="fix"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/e/1ea9b3481369e8344d92b73a103b3db32cb10434_2_690x160.jpeg" alt="fix" data-base62-sha1="4nfU8eOod0JFkB0bXhtQdTtKxko" width="690" height="160" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/e/1ea9b3481369e8344d92b73a103b3db32cb10434_2_690x160.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/1/e/1ea9b3481369e8344d92b73a103b3db32cb10434_2_1035x240.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/1/e/1ea9b3481369e8344d92b73a103b3db32cb10434.jpeg 2x" data-dominant-color="15100F"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">fix</span><span class="informations">1107×257 40.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>how to overcome this</p>

username: ptrblck
cooked: <p>Try to use the complete command as:</p>
<pre><code class="lang-python">pip install torch==1.5.0+cpu torchvision==0.6.0+cpu -f https://download.pytorch.org/whl/torch_stable.html
</code></pre>
<p>If that doesn’t help, try to update your <code>pip</code> and rerun the installation.</p>

username: Manik_Hossain
cooked: <p>its solved with<br>
pip3  install torchvision==0.6.0+cpu -f <a href="https://download.pytorch.org/whl/torch_stable.html" rel="nofollow noopener">https://download.pytorch.org/whl/torch_stable.html</a></p>
<p>btw thanks</p>
Performance in c++ of torch script model

username: AZuev
cooked: <p>Hello. I use torchscript for 1 model. Have some problems with slow performance in c++.<br>
Performance: (linux, cpu)</p>
<ul>
<li>Python: 0.26sec</li>
<li>Python torchscript first: 27 sec</li>
<li>Python torchscipt second: 0.08sec</li>
<li>c++ first: 2sec</li>
<li>c++ second: 101sec.</li>
</ul>
<p>How I can improve performance on c++?<br>
Pls say what additional information I need to show.</p>
Resnet plot saliency map

username: bioinfo-dirty-jobs
cooked: <p>I would like to plot saliency map of Resnet 50 on my data after training.<br>
I take on the layer4 the  conv2 and I would like to plot.<br>
Any suggestion on how can do? thanks in advance for any help</p>
<pre><code class="lang-auto">ct=model.layer4[2].conv2.weight.detach()
torch.Size([512, 512, 3, 3, 3])
</code></pre>

username: ptrblck
cooked: <p><a href="https://captum.ai/tutorials/CIFAR_TorchVision_Interpret">Captum</a> gives you an example of visualizing the saliency.</p>
Using a single GPU for a model trained with multiple GPUs

username: heilaw
cooked: <p>I trained a model with multiple GPUs using model parallelism. When I tested that model with a single GPU, I got the following error:</p>
<pre><code>RuntimeError: cuda runtime error (10) : invalid device ordinal at torch/csrc/cuda/Module.cpp:81
</code></pre>
<p>Does it mean that if I train a model with multiple GPUs, I have to run that model with the same number of GPUs? Is there any way to avoid that? I am using the recommended approach to save my PyTorch model. <a href="http://pytorch.org/docs/notes/serialization.html#recommend-saving-models" rel="nofollow noopener">http://pytorch.org/docs/notes/serialization.html#recommend-saving-models</a></p>

username: apaszke
cooked: <p>No you don’t have to, but when you re-instantiate the model you need to specify correct GPU indices in the DataParallel module. If you serialized the whole DataParallel part, you need to take the <code>.module</code> attribute out, and re-wrap it in <code>DataParallel</code> again.</p>

username: heilaw
cooked: <p>Thanks for you reply.</p>
<p>However, I was not using DataParallel module as I was doing model parallelism instead. I distributed different parts of my model into different GPUs using <code>torch.nn.Module.cuda</code> and copied tensors between GPUs using <code>torch.Tensor.cuda</code>. I got that runtime error when I loaded my model on a single GPU.</p>

username: smth
cooked: <p><a class="mention" href="/u/heilaw">@heilaw</a> can you give a small script to reproduce this?<br>
Also, as we  mention in the link you gave, we recommend using <code>state_dict</code> and <code>load_state_dict</code> to keep things simple.</p>

username: Exlsunshine
cooked: <p>Am I understanding you right: I can inference a model with single gpu that trained by multi gpu, and vise-versa?</p>

username: ptrblck
cooked: <p>Yes, that’s possible.<br>
If you are training a multi-GPU model, you should store the <code>model.module.state_dict()</code> as explained <a href="https://pytorch.org/tutorials/beginner/saving_loading_models.html#saving-torch-nn-dataparallel-models">here</a> to remove the <code>.module</code> attributes, which might create errors when trying to load it back on a standard model.</p>
RuntimeError: CUDA error: no kernel image is available for execution on the device

username: eranb
cooked: <p>Hi everyone,<br>
I know there is a lot of questions about “CUDA error no kernel…”, but I think I have a different issue.<br>
I’m working on the university cluster, and I open virtual-env and install all the packages include torch-Cuda 9.2.  When I try to run the program I get the error: RuntimeError: CUDA error: no kernel image is available for execution on the device.</p>
<p>Now, I try to figure out if I need to install a regular torch without Cuda like torch 1.0.0.<br>
Can someone having this problem ?</p>
<p>Eran</p>

username: ptrblck
cooked: <p>What GPUs is your cluster using?<br>
Note that the binaries ship with pre-built CUDA, cudnn, etc. for GPUs with compute capability &gt;=3.7.<br>
If the GPUs you are using have an older cc, you might need to build PyTorch from source.</p>

username: eranb
cooked: <p>Yes I know that there is a lot of clusters, my question is if I install the simple version like torch == 1.0.0<br>
it will install Cuda?<br>
Because I think I don’t need to install Cuda, there is some version of Cuda on the clusters.</p>

username: ptrblck
cooked: <p>The binaries ship with their own CUDA, cudnn etc. so your local CUDA installation won’t be used.<br>
Note that I was referring to the compute capability of your GPUs, not CUDA, in case there is a misunderstanding.</p>
Save Tensor as 1D PNG image

username: Mojtaba
cooked: <p>Hello,</p>
<p>I want to save a Tensor: torch.Size([480, 854]) as a PNG image which has one dimensional depth. I used torchvision.utils.save_image but it converts this tensor to a [3,480,854] image.<br>
Please let me know if you have any suggestions.</p>
<p>Thanks.</p>

username: ptrblck
cooked: <p><code>save_image</code> uses internally <code>PIL.Image.save</code>, with an optional <code>format</code> argument as seen <a href="https://github.com/pytorch/vision/blob/c031e2875bafe633cc163fb2bfe6c61f2f23285e/torchvision/utils.py#L128">here</a>.<br>
You could potentially pass the desired <code>format</code> using <a href="https://pillow.readthedocs.io/en/3.1.x/handbook/image-file-formats.html">these PIL format docs</a>.</p>
Loading image data from pandas to pytorch

username: akshat91
cooked: <p>0</p>
<p>I am completely new to pytorch and have previously worked on keras and fastai. Currently trying an image regression task and the challenge is I have to load the data from pandas dataframe. Data frame structure:</p>
<p>ID Path Score</p>
<p>fig1 /folder/fig1.jpg 2</p>
<p>fig2 /folder/fig2.jpg 3</p>
<p>…</p>
<p>I have previously worked on loading images to pytorch directly from folders because it was a simple classification task but kind of stuck now.</p>
<p>I already have created the transformations i need but stuck at this data loading part.</p>

username: ptrblck
cooked: <p>You could create a custom <code>Dataset</code> as described <a href="https://pytorch.org/tutorials/beginner/data_loading_tutorial.html">here</a>.<br>
Inside the <code>__init__</code> you could store the <code>DataFrame</code>, and load each row in <code>__getitem__</code> e.g. via <code>self.df.iloc[idx]</code>.<br>
Once you have the row, you could get the path and score, load and process the image, and return the data and target.</p>
<p>Let me know, if you get stuck somewhere.</p>
Torchtext TabularDataset datafields add annotation or multiple label

username: tiancaigg
cooked: <pre><code class="lang-auto">TEXT = torchtext.data.Field(tokenize = word_tokenize,  fix_length= 1000)
HYBRIDS = torchtext.data.Field()
LABEL = torchtext.data.LabelField(dtype = torch.float)

datafields = {
    "Phenotype": ("labels", LABEL), 
    "sequence": ("text", TEXT),
    "hybrids":("hybrids", HYBRIDS) 
               }
</code></pre>
<p>In torchtext, I add a hybrids column, which is the sample’s names,  in order to identify which prediction are made to the sample.<br>
but no matter I define it LabelField or Field, it doesn’t work and raise error.<br>
Is there some way to add a name column to the TabularDataset as naming of the sample, but not as label or text for training?</p>

username: ptrblck
cooked: <p>What kind of error are you seeing? Could you post the complete error message here, please?</p>
What is the best way to inference a model multiple times at a time? (using only CPU)

username: kstn_hut_hat
cooked: <p>I have 1 server with 16 cores. At the same time I can receive a maximum of 8 requests to inference the model. What method should I use to optimize between memory, cpu usage and speed? Can you give me a few suggestions?<br>
Thank you very much.<br>
P / s: My current method is to use multiprocess but it seems to be ineffective. Because the model is loaded up to 8 times at a time</p>
Find the gradient of an intermediate variable

username: Abueidda
cooked: <p>Hi All,</p>
<p>Is it possible to calculate the gradients of an intermediate variable with respect to an input?</p>
<p>The layout is as follows:<br>
intermediate = g(inp1)<br>
output = f(inp1, intermediate,…)</p>
<p>To do so, we minimize the loss: Solve MSE optimization problem loss ||output-target||</p>
<p>Then after the model is trained I am interested in finding the sensitivities of an intermediate variable:<br>
d(intermediate)/d(inp1),</p>
[PyTorch Geometric] Graph convolution in batch mode

username: nullgeppetto
cooked: <p>Let’s say that we need to define a batch of graphs, of the same structure, i.e., with the same <code>edge_index</code>, but with different feature signals and different edge attributes.</p>
<p>For instance, let’s define a simple directed graph structure with the following <code>edge_index</code>:</p>
<pre><code class="lang-auto">import torch
from torch_geometric.data import Data as gData
import torch_geometric.nn as gnn
import numpy as np

edge_index = torch.tensor(np.concatenate([np.arange(num_nodes), np.roll(np.arange(num_nodes), shift=1)]).reshape(-1, num_nodes),  dtype=torch.long)
</code></pre>
<pre><code class="lang-auto">edge_index
tensor([[0, 1, 2, 3, 4, 5, 6],
        [6, 0, 1, 2, 3, 4, 5]])
</code></pre>
<p>Now, let’s define a simple graph convolution operator, e.g., <code>GCNConv</code>, that will act on such graphs:</p>
<pre><code class="lang-auto">gconv = gnn.GCNConv(in_channels=num_node_features, out_channels=32)
</code></pre>
<p>Then, if I define a graph signal as below:</p>
<pre><code class="lang-auto">x = torch.randn((num_nodes, num_node_features), dtype=torch.float)
print(x.size())
torch.Size([7, 16])
</code></pre>
<p>and pass it through <code>gconv</code>, I have:</p>
<pre><code class="lang-auto">y = gconv(x, edge_index)
print(y.size())
torch.Size([7, 32])
</code></pre>
<p>which is fine.</p>
<p>Now, I’d like to do the same in a mini-batch manner; i.e., to define a a batch of such signals, that  along with the same <code>edge_index</code> will be passed through <code>gconv</code>. Apparently, defining signals and edge attributes as 3D tensors does not work.</p>
<p>I suspect that this could be somehow done using <code>batch</code>, mentioned <a href="https://pytorch-geometric.readthedocs.io/en/latest/notes/introduction.html#mini-batches" rel="nofollow noopener">here</a>, but I cannot find any reference on how this could be done.</p>

username: navid_mahmoudian
cooked: <p>I have the same problem. Have you found any solutions?</p>

username: nsidn98
cooked: <p>You can refer to this <a href="https://github.com/rusty1s/pytorch_geometric/issues/973" rel="nofollow noopener">issue</a> in the original repository.</p>
Can I use default backward of tensor operation when writing custom CUDA layer?

username: Hakjin_Lee
cooked: <p>Hello,<br>
I try to implement my custom layer.<br>
In this custom layer, many basic tensor operations are included such as dot product, cross product.</p>
<p>I want to exploit these basic operations implemented in pytorch.</p>
<p>#<span class="hashtag">#Question</span><br>
Is it possible to import tensor operations which are default in pytorch?<br>
If I cannot import them, where can I find these basic tensor operation codes in pytorch repository?</p>

username: ptrblck
cooked: <p>I’m not sure what your actual use case is, but you could directly use torch methods to write a new layer.<br>
As long as you are using differentiable operations, Autograd will capture the computation graph and will be able to compute the backward pass.</p>
<p>If you are breaking the computation graph for some reason, you would need to implement the backward method manually as described <a href="https://pytorch.org/docs/stable/notes/extending.html">here</a>.</p>
<p>Are you writing a custom CUDA kernel or should the layer just be executable on the GPU?</p>

username: Hakjin_Lee
cooked: <p>I already tested my new layer with tensor operation in Python. I didnt’ have to implement the backward function since the layer is composed of basic tensor operations.<br>
It works well, but too slow.</p>
<p>So, I want to boost the layer by writing it in CUDA.</p>

username: ptrblck
cooked: <p>If you want to write a custom CUDA extension, have a look at <a href="https://pytorch.org/tutorials/advanced/cpp_extension.html">this tutorial</a>.</p>
Multiple input network Training

username: haeminjung
cooked: <p>Hi all.</p>
<p>I have created a network with 2 inputs to pass to forward, which are then concatenated later to be fed into another network.</p>
<pre><code class="lang-auto">def forward(self, latent, image):
    # latent, image = input_dict["latent"], input_dict["image"]
    x1 = self.net1(latent)
    print(x1.size())
    x2 = self.net2(image)
    print(x2.size())
    x = torch.cat([x1, x2], 1)
    return self.merge(x)
</code></pre>
<p>After passing it to torchsummary, it is able to compile(?) the network with given inputs.</p>
<p><code>summary(netD, [(1024,1,1), (3,64,64)])</code></p>
<p>However, I have tried various methods to pass on 2 torch.Tensors to the network for feedfoward but they all throw errors.</p>
<p><code> dlXGenerated = netG(dlZ, LR)</code></p>
<p>Can anyone point me to the correct way of passing 2 tensors to a network?</p>

username: Chris_Fleetwood
cooked: <aside class="quote quote-modified" data-post="1" data-topic="20462">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/raul_gombru/48/20077_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/concatenate-layer-output-with-additional-input-data/20462">Concatenate layer output with additional input data</a> <a class="badge-category__wrapper " href="/c/vision/5"><span data-category-id="5" data-drop-close="true" class="badge-category " title="Topics related to either pytorch/vision or vision research related topics"><span class="badge-category__name">vision</span></span></a>
  </div>
  <blockquote>
    I want to build a CNN model that takes additional input data besides the image at a certain layer. 
To do that, I plan to use a standard CNN model, take one of its last FC layers, concatenate it with the additional input data and add FC layers processing both inputs. 
[Dibujo%20sin%20t%C3%ADtulo] 
The code I need would be something like: 
additional_data_dim = 100
output_classes = 2
model = models.__dict__['inception_v3']
del(model._modules['fc'])
# TODO: Concatenate the CNN layer with the addit…
  </blockquote>
</aside>
<p>
This question may help you.</p>

username: haeminjung
cooked: <p>Ah, thanks for the head up.</p>
<p>It looks like my problem was related to multi-gpu process.<br>
Having commented out the DataParallel comand solves the problem for now.</p>