akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Dataloader for coco

prog_asker: <p>i need to change the datasets here  to coco<br>
<a>https://github.com/facebookresearch/deit/blob/main/datasets.py</a></p>
<p>in method</p>
<blockquote>
<p>build_dataset(is_train, args):</p>
</blockquote>
<p>i tried to write this line</p>
<blockquote>
<p>elif args.data_set == ‘coco’:<br>
dataset = ??? (args.data_path, train=is_train, year=2017,<br>
category=args.inat_category, transform=transform)</p>
<pre><code>    nb_classes = 1000
</code></pre>
</blockquote>
<p>but couldn’t know what is the replacement of question marks</p>

eqy: <p>Does using an out-of-the-box version of COCO work? <a href="https://pytorch.org/vision/stable/datasets.html#coco" class="inline-onebox" rel="noopener nofollow ugc">torchvision.datasets — Torchvision master documentation</a></p>

prog_asker: <p>does data_loader need specific settings or just change name of   IMNET to COCO<br>
elif args.data_set == ‘COCO’:<br>
if not args.use_mcloader:<br>
root = os.path.join(args.data_path, ‘train’ if is_train else ‘val’)<br>
dataset = datasets.ImageFolder(root, transform=transform)<br>
else:<br>
dataset = ClassificationDataset(<br>
‘train’ if is_train else ‘val’,<br>
pipeline=transform<br>
)<br>
nb_classes = 80</p>

eqy: <p>It depends on if the datasets are expected to produce the same kind of labels/annotations (e.g., are you doing classification or object detection)? If you are just doing classification I guess you can just use an <code>ImageFolder</code> provided the organization of the dataset is classes separated by folders.</p>

prog_asker: <p>excuse me another question do you mean the result of training the model will depend on the items that i specify detection of classifications ? if yes can you give me an example please</p>

eqy: <p>Taking another look at the source code, it looks like it is intended for classification. However, I’m not sure COCO is really a classification dataset but really a detection dataset. Do you have a link to the dataset you are using or can you describe the organization of the images/folders?</p>

prog_asker: <p>I’m confusing in it … it suppose to use a model doing detection but i freeze layers to not make a classification as i need features of images not classify them … so should i use detection for coco .<br>
datasets i have contains two folders (train and val)</p>

eqy: <p>You might want to take a look at the finetuning object detection tutorial: <a href="https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html" class="inline-onebox" rel="noopener nofollow ugc">TorchVision Object Detection Finetuning Tutorial — PyTorch Tutorials 1.8.1+cu102 documentation</a> as that provides a walkthrough of the scenario you are describing</p>
How can i set num_workers?

samm: <p>i have laptop with 1 gpu , should i set num_workers= 4 as the equation i found that<br>
num_workers=4*num_GPU ?</p>

eqy: <p>It really just depends on how much CPU time it takes to load data to keep the GPU busy so ultimately it depends on your particular dataset, preprocessing steps, and the relative speed of CPU and GPU in your setup. A simple way to set <code>num_workers</code> is to keep increasing it until your GPU utilization (e.g., in <code>nvidia-smi</code>) no longer increases.</p>

samm: <p>How can I get it from Nvidia-smi ?<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/c/c/cc1df0e86c13ce15549391d5c21d5a84c5af1e56.png" alt="Screenshot from 2021-06-12 21-59-05" data-base62-sha1="t7HyfiIQlXMbcIqBibjJ9FUJIAS" width="315" height="418"><br>
do you mean that ? i set it 4</p>

eqy: <p>Yes, if the GPU-util is not increasing, you don’t need to keep increasing the number of workers. You just want enough workers to keep the GPU busy.</p>

samm: <p>you mean if GPU-util is 50% i can increase number of workers ? and if it 99 i should decrease number of worker ?</p>

eqy: <p>If it is 99 you can try decreasing the number of workers but you don’t really need to do anything (the ideal is using the fewest number of workers to get the maximum utilization time). You can also use <code>time.time()</code> to track statistics like how long an epoch takes and tune how many workers you are using based off of that.</p>

samm: <p>More thanks for your time and replying</p>
Reset weight for fc layer

Chamroukhi: <p>Hi,<br>
in cross validation i use pre-trained model, i need to reset just the weights of Fc layer (classifier layer) , Thanks <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

eqy: <p>You can simply assign the last layer manually after loading it.<br>
e.g.,</p>
<pre><code class="lang-auto">pretrained_model = [some steps that gave me the pretrained model]
pretrained_model.fc = nn.Linear(input_channels, num_classes)
</code></pre>
<p>should work for most pretrained models in <code>torchvision</code>.</p>
The design of torch.autograd.functional.jacobian

liangbright: <p>I was wondering why the first parameter of torch.autograd.functional.jacobian is not the output?<br>
I have to define a dummy function that does nothing but return the output</p>
<pre><code class="lang-auto">import torch
x1=torch.tensor([1.0, 1.0], requires_grad=True)
x2=torch.tensor([2.0, 2.0], requires_grad=True)
y1=(x1+2*x2).sum()
y2=(3*x1+4*x2).sum()

def dummy_func(*args):
    #It is impossible to write the computation process/graph here
    #because it has already been done in some other functions/sections
    #y1=(x1+2*x2).sum()
    #y2=(3*x1+4*x2).sum()
    return (y1, y2)
gg=torch.autograd.functional.jacobian(dummy_func, (x1, x2), vectorize=True)

g=torch.autograd.grad(y1, [x1, x2]) 
# fail because "Trying to backward through the graph a second time,
# but the saved intermediate results have already been freed."
</code></pre>
<p>The problem of this hack is that torch.autograd.functional.jacobian will not retain the graph from (x1, x2) to (y1, y2)</p>
Best practice for prediction using new_weights

asura: <p>Hello,</p>
<p>First, I am first initializing model parameters, and predicting some input data on it.</p>
<pre><code class="lang-auto">pred = learner(adapt_data)
loss = loss_fn(pred, adapt_labels)
grad = torch.autograd.grad(loss, learner.parameters())
new_weight = list(map(lambda p: p[1] - 0.4*p[0], zip(grad, learner.parameters()))) # theta' = theta - alpha*grads
</code></pre>
<p>Now, I want to make use of these new_weights to predict on evaluation data without updating them in the model <code>learner()</code>.</p>
<pre><code class="lang-auto">pred_val = learner(eval_data, new_weight)
</code></pre>
<p>But, this obviously won’t work as <code>learner()</code> takes only 1 argument (2, if we consider self). How should I work around this problem? One way I could think of was to make use of <code>deepcopy(learner)</code> to replicate the model and update the parameters of the new model, say <code>new_learner()</code>. But, the problem is, I can’t make use of <code>loss.backward()</code> or <code>opt.step()</code> as it will update the original model.</p>
<p>I would really appreciate if you could help</p>

asura: <p>So, I tried something like this. I am not sure if this is correct, so do feel free to correct me in case it is a wrong practice.</p>
<p>I made a deepcopy of the original model. Let’s call it <code>new_learner()</code>. Similar to what I have done above, I computed the new weights in a variable <code>new_weight</code>. Now, I am made use of this to update without making use of <code>opt.step()</code>:</p>
<pre><code class="lang-auto">with torch.no_grad():
    for i, (name, params) in enumerate(new_learner.named_parameters()):
        params.copy_(new_weight[i])
</code></pre>
<p>Now, once I evaluate this <code>new_learner</code> on eval_data, I am deleting it using <code>del new_learner</code><br>
Next, I am using the loss generated from the prediction to update the original model’s weights.</p>
<pre><code class="lang-auto">pred_val = new_learner(eval_data)
loss_val = loss_fn(pred_val, eval_labels)
del new_learner
train_loss.backward()
meta_opt.step()
</code></pre>
<p>And it’s done! Please let me know if this is ok, or should I be aware of a better practice? Thank you!</p>

asura: <p>Ok, so the problem with the above solution is that <code>train_loss.backward()</code> is not updating the parameters of the <code>learner()</code> model at all.</p>
<p>So, basically, the graph is breaking somewhere thus, <code>learner.parameters()[0].grad</code> is coming out to be <code>None</code> for every parameter.</p>
Is it able to make a convolution module without bias have bias again?

feiyangsuo: <p>After instantiating a 2D convolution with <code>conv = nn.Conv2d(8, 8, 3, bias=False)</code>, whose member <code>bias</code> should be <code>None</code>, is it able to give <code>conv</code> a legal bias again (whether with random initialization or determined values)?</p>
<p>I observed that <code>bias</code> in other default convolution modules is of the type <code>Parameter</code>, so I suspect there are extra procedures beyond simply <code>conv.bias = torch.tensor(...)</code>.</p>

feiyangsuo: <p>I also asked this question <a href="https://stackoverflow.com/questions/67948404/pytorch-is-it-able-to-make-a-convolution-module-without-bias-have-bias-again" rel="noopener nofollow ugc">on stackoverflow</a> and received an answer.</p>
Dynamic Quantization of finetuned T5 model

priyank: <p>Hi All,</p>
<p>I tried quantizing a finetuned T5 model :</p>
<pre><code class="lang-auto">model = T5ForConditionalGeneration.from_pretrained('/path/to/finetuned/model')
model = torch.quantization.quantize_dynamic(model, {torch.nn.Linear}, dtype=torch.qint8)

tokenizer = T5Tokenizer.from_pretrained('t5-base')

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)

list = ["What is you name ?",
"Where do you live ?",
"You should say hi and greet him.",
"If John has cash 80 dollars then tell him to transfer",
"Tell John to buy some vegetables",]

l = []
t = []

for s in list:

	start = time.perf_counter();

	sentence = s

	text =  "paraphrase: " + sentence + " &lt;/s&gt;"

	max_len = 256

	encoding = tokenizer.encode_plus(text,pad_to_max_length=True, return_tensors="pt")
	input_ids, attention_masks = encoding["input_ids"].to(device), encoding["attention_mask"].to(device)


	# set top_k = 50 and set top_p = 0.95 and num_return_sequences = 3
	beam_outputs = model.generate(
	    input_ids=input_ids, attention_mask=attention_masks,
	    do_sample=True,
	    max_length=256,
	    top_k=120,
	    top_p=0.98,
	    early_stopping=True,
	    num_return_sequences=1
	)
 
 
 	l.append(tokenizer.decode(beam_outputs[0],skip_special_tokens=True,clean_up_tokenization_spaces=True))
    t.append("time taken = {}".format(time.perf_counter()-start))

	
for i in l:
	print(i)

for j in t:
	print(j)
</code></pre>
<p>While getting inferences it says:</p>
<p><code>You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.</code></p>
<p>and I am getting empty string as inference. What is wrong?</p>
Bounding Boxes dimension format in an object detection format

manix: <p>I just have this little doubt over the bounding boxes dimensions that is used for Object detection problem with any of the pretrained models like FRCC or YOLO.</p>
<p>Despite which bounding box format they are following(like VOC) , i have an dataset where there are several bounding boxes for a single image. There are 2 ways to go with the problem</p>
<ol>
<li>
<p>map image to each individual bounding box and take it as an data point<br>
Example : dataset format in here<br>
In Cell 3<br>
<a href="https://www.kaggle.com/dipam7/wheat-head-classification/" class="inline-onebox" rel="noopener nofollow ugc">Wheat_head_classification | Kaggle</a><br>
2)Make a dict of all the bounding boxes of an image and feeding it to the network.<br>
Example :<br>
<a href="https://www.aicrowd.com/showcase/tutorial-with-pytorch-torchvision-and-pytorch-lightning" class="inline-onebox" rel="noopener nofollow ugc">AIcrowd | Tutorial with Pytorch, Torchvision and Pytorch Lightning ! | Posts</a><br>
But in this case how is the network able to learn from this variable length number of bounding boxes.</p>
<p>I am missing some crucial understanding here which i am unable to get around with , could someone clear my doubts. <img src="https://discuss.pytorch.org/images/emoji/apple/grin.png?v=9" title=":grin:" class="emoji" alt=":grin:"></p>
</li>
</ol>
Change my username

sh0416: <p>Sorry for the irrelevant issue.</p>
<p>Is there anyone who knows how to change my username?</p>
<p>I want to change my username to “sh0416”.</p>
<p>Thanks,</p>

albanD: <p>Sure, it should be good now.</p>

manix: <p>i would like to change my username to manix or manix29 , could you guide me to change it</p>

ptrblck: <p>Could you check your “Preferences” (click on your avatar and it should be the right tab) and then at “Account” you should be able to change the user name.<br>
Let me know, if it works. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

manix: <p>But i am unable to edit it , under Accounts user name is there but it doesn’t seems to be editable.</p>

ptrblck: <p>Could you send me a PM with the preferred user name and we can continue the discussion there.</p>
Some fundemental concept problems wanted to figure out

Florentino: <p>Hello, I am wondering how do the part code do in a training loop that I saw them frequently used.<br>
Especially <code>update</code> and <code>scale</code> do.</p>
<pre><code class="lang-auto">        optimizer.zero_grad()
        scaler.scale(loss).backward()
        scaler.step(optimizer)
        scaler.update()
</code></pre>
<p>Also, in what situation we need to use<code> torch.amp.autocast()</code> ?</p>
<p>Thank you</p>

ptrblck: <p>Automatic Mixed Precision training (via <code>torch.cuda.amp</code>) can be used to speed up your training and I think the best place to get started is to check out the <a href="https://pytorch.org/docs/stable/amp.html">docs</a>, the <a href="https://pytorch.org/tutorials/recipes/recipes/amp_recipe.html">Automatic Mixed Precision recipe</a> as well as the <a href="https://pytorch.org/docs/stable/notes/amp_examples.html#amp-examples">examples</a>.</p>

Florentino: <p>Thank you very much!</p>
Is there a way to manipulate the gradient in Adam step?

phantom90: <p>Hi There,</p>
<p>Just curious is there there a way to manually allocate a gradient for an Adam step?</p>

ptrblck: <p>You could use hooks to manipulate the gradients for each parameter (its <code>.grad</code> attribute), backward hooks, or if needed you could also create a custom <code>Adam</code> implementation by reusing the PyTorch <a href="https://github.com/pytorch/pytorch/blob/master/torch/optim/adam.py">implementation</a>.</p>
Smush multi-channel tensor into one image channel with maximums

Caipi: <p>Hello there,</p>
<p>I have programmed a multiclass segmentation model, everything works fine. My output for the tests is a tensor shaped 668x388x388 thus the output contains 668 different classes. So my goal is to get a tensor (or numpy array) out of this shaped like 1x388x388 to give it to matplotlib.imshow and thus I provided a softmax, every value is between 0 and 1. I need to pick for each pixel in the resulting array the highest one out of the 668 channels…</p>
<p>Is there a method for this?</p>

ptrblck: <p>To create the predictions containing the class index associated with the highest probability (or logit) you could use:</p>
<pre><code class="lang-python">preds = torch.argmax(output, dim=0) # assuming output has the mentioned shape [nb_classes, height, weight]
</code></pre>

Caipi: <p>Thanks, also a 3DMaxPool should be possible  : D</p>
Reinforcement learning: element 0 of tensors does not require grad and does not have a grad_fn

Abhishek_Minhas: <p>the exact error is</p>
<pre><code>Variable._execution_engine.run_backward(
		RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn 
</code></pre>
<p>its a simple fully connected nn with reinforce algorithm on a cpu. So the loss is simply the mean of q_value * log_probability of all the actions in one batch. 
I think the problem is that
 (thats not it) I collect everything as lists, convert them into numpy arrays, calculate the loss and then backpropogate. However I’m not able to get things right. The following code (reinforce part omitted) replicates the error and is exactly what I have done.</p>
<pre><code>import torch
import torch.nn as nn
import torch.nn.functional as F
import statistics
import torch.optim as optim
import numpy as np


class PGN(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1= nn.Linear(5*5,64)
        self.fc2= nn.Linear(64,64)
        self.fc3= nn.Linear(64,64)
        self.fc4= nn.Linear(64,3)
    def forward(self,x):
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = F.relu(self.fc3(x))
        x = self.fc4(x)
        # x= F.softmax(x, dim=1) 
        return x




net = PGN()
optimizer = optim.Adam(net.parameters(), lr=0.1, eps=1e-3)
for params in net.parameters():
    params.requires_grad = True

# forward pass and collection log probability
state = [torch.Tensor(np.random.rand(25)) for _ in range(10)]
batch_log_probs =[]
for ele in state:
    logit = net(ele)
    prob = F.softmax(logit, dim=0)
    prob = prob.detach().numpy()
    action = np.random.choice(len(prob), p=prob)
    log_prob = F.log_softmax(logit, dim=0)
    log_prob = log_prob[action]
    batch_log_probs.append(log_prob)
q_vals = torch.Tensor(np.random.rand(10))
loss = -q_vals*batch_log_probs
loss = loss/mean()
optimizer.zero_grad() 
loss.backward()
optimizer.step()</code></pre>

ptrblck: <p>Your code isn’t executable and raises an error at:</p>
<pre><code class="lang-python">    loss = -q_vals*batch_log_probs

TypeError: only integer tensors of a single element can be converted to an index
</code></pre>
<p>After adding:</p>
<pre><code class="lang-python">batch_log_probs = torch.stack(batch_log_probs)
</code></pre>
<p>before the problematic operation and fixing <code>loss/mean()</code> to <code>loss.mean()</code> the code runs fine and doesn’t raise the reported error.</p>

Abhishek_Minhas: <p>Thank you for your reply!!! torch.stack solved my problem!</p>
<p>do you think its faster to append tensors to a list and then convert the list to a tensor using torch.stack or should I just work with tensors only and use torch.cat</p>
<p>for others<br>
the main issue I was facing was that I had a list of tensors (batch_log_probs), each with a grad function. the grad function would disappear when I would convert the list to a tensor.</p>
<p>torch.stack() preserves the grad function</p>
DataLoader parameter "shuffle" affects model accuracy

auzuha: <p>Hello ,<br>
The parameter shuffle in DataLoader class seems to affect the model in some way.<br>
I have a saved model for a binary classification task (cats vs dogs) and changing the parameter shuffle in DataLoader affects my model heavily.<br>
I have used torch.save method to save my trained model , and I used torch.load method to load it in another python file.<br>
When shuffle is set to False in DataLoader , the model gives around 52% accuracy but the saved model had about 98% accuracy during validation tests.<br>
The model only performs how it is supposed to perform is when shuffle is set to True in the prediction python file I use.<br>
The validation set used for testing accuracy of the model while changing shuffle parameter is the same.<br>
Below is a snippet of code which calculates accuracy of the model.<br>
I would also like to note that the model is a pretrained model called resnet18 , i only trained the fc layer in order to fit it to my task.</p>
<pre><code class="lang-auto">test_loader = DataLoader(test_set , batch_size=64 , shuffle=False)

total_correct = 0
total_seen = 0
for xb,yb in test_loader:
    xb=xb.cuda()
    yb=yb.cuda()
    preds = model(xb.float())
    total_correct += ((torch.sum(preds.round() == yb.reshape(-1,1))).item())
    total_seen += yb.numel()
print(total_correct / total_seen)
</code></pre>
<p>When shuffle is set to False , like the above snippet , the accuracy i get is 0.5284231339594662<br>
When shuffle is set to True , the accuracy i get is 0.957983193277311<br>
This change is only by changing shuffle to True.<br>
I don’t understand why or how DataLoader affects my pretrained model’s accuracy , please help me out.</p>
<p>Also , somehow using model.eval() solves the above problem.</p>

ptrblck: <aside class="quote no-group" data-username="auzuha" data-post="1" data-topic="123887">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/auzuha/48/30602_2.png" class="avatar"> auzuha:</div>
<blockquote>
<p>Also , somehow using model.eval() solves the above problem.</p>
</blockquote>
</aside>
<p>This would be expected, since calling <code>model.eval()</code> would disable dropout layers (shouldn’t make a difference regarding shuffling the dataset) and would use the running stats of all batchnorm layers.<br>
If you leave the model in training mode, the batchnorm layers would use the current batch stats to normalize the inputs and would also update the running stats. Shuffling the dataset is thus making a difference, which is why you should call <code>model.eval()</code> during validation and testing.</p>
Bugs in Translation tutorial for PyTorch 1.8.1

st-vincent1: <p>Not sure if this is the best place to post!<br>
I’ve worked with this tut for a couple hours and I found the following bugs:</p>
<ol>
<li>The tutorial below contains a broken link under “View on GitHub”.</li>
<li>The <code>collate_fn=generate_batch</code> is broken as <code>torch.nn.utils.rnn.pad_sequence</code> swaps the dimensions of data, yielding batches where the 0th element is full of <code>&lt;sos&gt;</code> tokens etc. The correct way would be to use <code>batch_first=True</code> in <code>pad_sequence</code>.</li>
</ol>
<p>Hope it helps!</p>

ptrblck: <p>Thanks for reporting this!<br>
I assume you mean <a href="https://pytorch.org/tutorials/beginner/torchtext_translation_tutorial.html">this tutorial</a>.<br>
CC <a class="mention" href="/u/vincentqb">@vincentqb</a> for visibility</p>

st-vincent1: <p>Yep, this one. No worries, happy to help!</p>
Using quantized addition

julien-thierry: <p>Hi,</p>
<p>There seems to be a quantized::add operator but I can’t find how to use it</p>
<pre><code class="lang-auto">class QuantAdd(torch.nn.Module):
    def __init__(self):
        super(QuantAdd, self).__init__()
        self.quant = torch.quantization.QuantStub()
        self.dequant = torch.quantization.DeQuantStub()

    def forward(self, x, y):
        x = self.quant(x)
        y = self.quant(y)
        out = x + y
        return self.dequant(out)

model = QuantAdd()

torch.backends.quantized.engine = 'qnnpack'
model.qconfig = torch.quantization.get_default_qconfig('qnnpack')
torch.quantization.prepare(model, inplace=True)

a = torch.rand(1, 3, 4, 4)
b = torch.rand(1, 3, 4, 4)

_ = model(a, b)

torch.quantization.convert(model, inplace=True)
traced_model = torch.jit.trace(model, (a, b))
</code></pre>
<p>The above will result in the error:</p>
<pre><code class="lang-auto">NotImplementedError: Could not run 'aten::empty.memory_format' with arguments from the 'QuantizedCPU' backend.
</code></pre>
<p>I tried using <code>out.copy_(x + y)</code> on a pre-allocated tensor but still get the ‘aten::empty.memory_format’ error which I guess is related to x + y return. So I tried in-place addition (both with <code>x += y</code> and <code>x.add_(y)</code> ) and I get:</p>
<pre><code class="lang-auto">NotImplementedError: Could not run 'aten::add.out' with arguments from the 'QuantizedCPU' backend.
</code></pre>
<p>Am I missing something obvious? What’s the right way to use quantized addition in a model?</p>
<p>Thanks,</p>
<p>Julien</p>

julien-thierry: <p>Actually I think I found the way.<br>
I need to use a <code>torch.nn.quantized.FloatFunctional</code> as a functor and then swap it by a <code>torch.nn.quantized.QFunctional</code>  after model conversion but before tracing.</p>
<p>Is that the recommended way to proceed?<br>
Or is there a more straightforward alternative?</p>
How to get value from Tuple inside of GenericList?

Mikhail_Miha: <p>How to get value from Tuple inside of GenericList?</p>
<pre><code class="lang-auto">auto outputs = network.forward(inputs).toTuple()-&gt;elements()[1]; 
std::cout &lt;&lt; outputs &lt;&lt; "\n";
</code></pre>
<p>Outputs:<br>
[W faster_rcnn.py:96] Warning: RCNN always returns a (Losses, Detections) tuple in scripting (function )<br>
[{boxes:   0.0000   4.0458  12.8517  17.3160<br>
0.0000   1.8971   2.1000  16.8890<br>
0.0000   3.2075  23.5053  20.9901<br>
0.0000   7.2702  38.0136  19.9625<br>
0.0000   2.8185   6.2506  15.2231<br>
0.0000   7.7666   5.7852  16.8950<br>
0.0000   7.3058   4.0280  13.9331<br>
[ CUDAFloatType{7,4} ], labels:  1<br>
1<br>
1<br>
1<br>
1<br>
1<br>
1<br>
[ CUDALongType{7} ], scores:  0.8489<br>
0.6713<br>
0.2980<br>
0.1157<br>
0.0758<br>
0.0502<br>
0.0501</p>
<p>I need to  get values to draw rectangle, with label and score</p>
Efficient way to transfer data between modules

FruitVinegar: <p>Hello. I’d like to know efficient implementation way situation like below.</p>
<p>Model is consist of independent sub-modules, and there’s forwarding order between modules so previous module’s output will send as next module’s input.(this behavior is well-defined in model’s forward function.)<br>
I’d like to add the following functionality to model:</p>
<blockquote>
<p>Designate specific modules(by configuration file) and each designated module generates (huge) data that can be shared by all modules after designated module(in forwarding order)</p>
</blockquote>
<p>For example, assume model has [m1, m2, m3, m4, m5] in sequence.<br>
If I designate m2 and m4, and each generates data2 and data4. data2 can be shared in [m2, m3, m4, m5] and data4 can be shared [m4, m5].</p>
<p>As I mentioned above, each generated data is quite huge and shareable so I don’t want to forward this data just as part of output. (also modifying well-defined original model’s forward function is annoying job)</p>
<p>Currently, I’ve implemented like below:</p>
<ol>
<li>Assign empty buffer for each modules in model.</li>
<li>Register forward hook at designated module which send generated data to following modules.</li>
</ol>
<p>But the performance seems bad: takes 2x much more times measured same condition platform compared to original model.</p>
<p>The problem is each module’s does not know other module’s existence. I’ve thought about idea ‘melting’ all modules into one ‘unified’ model make the problem easier, but the model is complicated and have so many advantages that I cannot give up when keep module-based architecture.</p>
<p>Any ideas will be welcomed. Thanks for reading!</p>
How does the transformer output one word?

Arthur_Zakirov: <p>Hello everyone,</p>
<p>as far as I understand, the grey cell maintains the shape [batch, sequence, dim] all the way through.</p>
<p>So during infernece, how is last decoder cell supposed to give the probabilities of a single word, if the output shape is  [batch, sequence, dim] with sequence &gt; 1 ?</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/e/8e5d039948b8970e6b25395cb207febc82ba320a.png" data-download-href="https://discuss.pytorch.org/uploads/default/8e5d039948b8970e6b25395cb207febc82ba320a" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/e/8e5d039948b8970e6b25395cb207febc82ba320a_2_177x500.png" alt="image" data-base62-sha1="kjp578PYFUZSp49bxeDFvHuyyMO" width="177" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/e/8e5d039948b8970e6b25395cb207febc82ba320a_2_177x500.png, https://discuss.pytorch.org/uploads/default/original/3X/8/e/8e5d039948b8970e6b25395cb207febc82ba320a.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/8/e/8e5d039948b8970e6b25395cb207febc82ba320a.png 2x" data-dominant-color="DEDDDA"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">233×655 45.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>My hypothesis is the following, is that correct?</p>
<p>iteration 1: dec_inp = [start]              → dec_out = [y0]  append last out element to input (here: y0)<br>
iteration 2:  dec_inp = [start, y0]        →    dec_out = [y0, y1]      append last out element to input (here: y1)<br>
iteration 3:  dec_inp = [start, y0,y1]   →    dec_out = [y0, y1,y2]</p>
<p>So you do not actually predict one word, but the whole sequence that you have ‘so far’ and then append the last element of that sequence to the input of the new iteration.</p>
<p>Thanks in advance</p>

pascal_notsawo: <p>Why not stay in <a href="https://discuss.pytorch.org/t/transformer-decoder-outputs/123826">one discussion</a>?</p>

Arthur_Zakirov: <p>It’s a different question, should I move it over to the other post?</p>

pascal_notsawo: <p>Honestly I find it better, especially since the answer was given in the latter.</p>
torch::NoGradGuard behavior in docker server running container ubuntu 18.04 vs ubuntu 16.04

ydas: <p>“torch::NoGradGuard no_grad;” is used to prevent gradient calculation from building up memory during inference.<br>
I am running a large model where the increase of memory is such that the memory crash unless NoGradGuard is used. I built two docker images on the same docker host server where both images are identical (same install, same make file, same cpp files…) except for the container os, i.e. one image was built on ubuntu 16.04, the other on ubuntu 18.04. Memory runs wild in the ubuntu 18.04 container while it is under control in ubuntu 16.04 container. Analysis showed that “torch::NoGradGuard no_grad;” prevents any memory increase in 16.04 but does not has any impact in 18.04.<br>
Any explanation would be welcome?</p>

ptrblck: <p>Could you provide a minimal code snippet to reproduce this issue, please?</p>

ydas: <p>Sorry. I have not managed yet to produce simple code to reproduce the issue as it involves a lot of install manual work at present. I will keep trying.</p>
What is the point of map_location argument in torh.load function if model is still on the different device after model.load_state_dict?

AndreyBocharnikov: <p>Hello everyone.<br>
I am witnessing strange and imo wrong behavior, can somebody please explain to me why this is happening?<br>
I loaded my pretrained model via<br>
<code>checkpoint = torch.load(params.pretrained_model, map_location=torch.device(cuda:0)) model.load_state_dict(checkpoint['model_state_dict'])</code><br>
And while all Tensors in <code>checkpoint['model_state_dict']</code> have device type equals to <code>cuda:0</code>, all parameters in the model have device type equals to <code>cpu</code>. Why does it happen so that if I want to load a specific Tensor as a parameter, its value is being loaded while its device type not? Does it mean that an extra copy of each parameter is being created on a different device?</p>
How to deal with labels of different shape in Object detection? (Yolov3)

Super_Pie_Camel: <p>I am currently implementing my own train.py for yolov3 object detection.</p>
<p>However, I met some problem about the labels.</p>
<p>For yolov3 object detection, it is allowed to detect multiple object in an image.<br>
In this case, different image may have different number of detected object. It means that different image may have different number of ground truth bounding boxes.</p>
<p>Ideally, I would like to concatenate the ground truth bounding boxes of same image into a dimension.</p>
<p>Assume it is a single class problem<br>
e.g<br>
[image_index, object_index, parameter]          (parameter[tx,ty,tw,th,Po,Pc1])</p>
<p>For image 1, 5 objects are detected in ground truth so its shape is [1, 5, 6]<br>
For image 2, only 2 are detected in ground truth so its shape is [1, 2, 6]</p>
<p>In this case, the label for image 1 and image 2 cannot concatenate into one tensor as they have different shape at (dimension = 1).</p>
<p>To solve this problem, I have tried to combine them using<br>
list = []<br>
list.append(label1)<br>
list.append(label2)</p>
<p>However, the tensor must be stack together in torch.utils.data.DataLoader.<br>
Hence, I got the following Error owing to different shape of labels:<br>
RuntimeError: stack expects each tensor to be equal size, but got [1, 16] at entry 0 and [6, 16] at entry 1</p>
<p>16 = 1+4+11, 11 classes</p>
<p>All my labels are retrieved from a single .csv file. I am currently implementing the train.py using 7 images and 11 classes. All images are padded and resized to 416x416</p>
<p>Thanks for your time.</p>

ptrblck: <p>You could return the targets as e.g. a <code>dict</code> as is also done in <a href="https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html">this tutorial</a>.</p>

Super_Pie_Camel: <p>Thank you. I loaded the path of the images into a list in the datasets and load batch_size number of images per batch when iterating the data_loader.</p>

manix: <p>I have a doubt over how the variable number of bounding boxes are handled by the YOLO or RCNN, FRCNN or any other object detection model.<br>
we have different number of bounding boxes for each images, but the target seems to take all the bounding boxes for an image together as a single data, how is the model handling it ?</p>