akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Clarification for the Mask RCNN tutorial

M_S: <p>Hi,<br>
In this tutorial: <a href="https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html" rel="nofollow noopener">https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html</a></p>
<p>There is a code line that has an error:</p>
<pre><code class="lang-auto">        if self.transforms is not None:
            img, target = self.transforms(img, target)
</code></pre>
<p>According to the definition in the previous code section, <code>self.transforms</code> is a <code>Compose</code> object, so it accepts only one argument:</p>
<pre><code class="lang-auto">def get_transform(train):
    transforms = []
    transforms.append(T.ToTensor())
    if train:
        transforms.append(T.RandomHorizontalFlip(0.5))
    return T.Compose(transforms)
</code></pre>
<p>Can you please clarify what’s the correct syntax that one should use?</p>

M_S: <p>Also, I think the <code>ToTensor</code> should come after <code>RandomHorizontalFlip</code>, because <code>RandomHorizontalFlip</code> accepts PIL images.</p>

tom: <p>You had me looking there, too! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p>What happens is that the tutorial ships with its own little transforms module to bridge that gap.</p>
<p>Best regards</p>
<p>Thomas</p>

M_S: <p>Yep,thanks.I already requested to delete this thread. Will read more carefully next time</p>

tom: <p>Oh, the next person with the question might find it useful. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

M_S: <p>Thanks <a class="mention" href="/u/tom">@tom</a><br>
Do you know how to save the model after training? It’s a JIT ScriptModule, and it doesn’t support pickle.dump or torch.save . There are open GitHub issues on it. I wonder if you know of a workaround.</p>

tom: <p>It is considered best practice to save the state_dict, i.e. the parameters, rather than the full model. (You need the source anyway to re-instantiate the model.) This is what all pre-trained torchvision models do, too.<br>
The obvious alternative is to trace the model to get a JIT model and save that.</p>
<p>Best regards</p>
<p>Thomas</p>
How can I use KNN, Random Forest models in Pytorch?

pydv: <p>This may seems like an X Y problem, but initially I had huge data and I was not able to train in given resources (RAM problem). So I thought I could use batch feature of Pytorch. But I want to use Methods like KNN, Random Forest, Clustering except Deep Learning. So is it possible or can I use scikit libraries?</p>

tom: <p>Skorch aims at providing sklearn functions in a PyTorch basis.<br>
That said, if there is something you need that it does not provide, sklearn is a great library and converting Tensors to NumPy arrays is seamless as long as you don’t need gradients flowing through the converted parts.</p>
<p>Best regards</p>
<p>Thomas</p>

pydv: <p>But I think Skorch, does not provide RNN, Random Forest. It only helps in Deep Neural Network.</p>

ValentinFigue: <p>Hi I’ve implemented a first version of Random Forest in Pytorch <a href="https://github.com/ValentinFigue/Sklearn_PyTorch" rel="nofollow noopener">https://github.com/ValentinFigue/Sklearn_PyTorch</a>. This is still a work in progress but I aim to add new models such as SVM.</p>
<p>Feel free to extend it,</p>
<p>Regards,</p>
<p>Valentin</p>
Handling OOM using torch.no_grad() throws import errors

vasda: <p>I am using a torchtext dataloader and during inference, while using the following,</p>
<pre><code class="lang-auto">with torch.no_grad():
    try:
        # inference code
    except RuntimeError:
        torch.cuda.empty_cache()
        continue
</code></pre>
<p>When a OOM error is encountered and tries to continue, python throws import errors.<br>
Without using torch.no_grad(), the OOM  is handled and continues to execute without any errors.<br>
I am not able to understand this behaviour.</p>

albanD: <p>Hi</p>
<p>Could you give the full stack trace of the error please? Which import fails and where?</p>

vasda: <p>My bad, while handling the error, I called the error message as re. I’m also using the regex library re, which resulted in an import error. Please ignore.</p>
When i install nb_conda, it make mistakes,help me, please!

xdd: <p><img src="https://discuss.pytorch.org/uploads/default/original/3X/1/4/14d0d1866625eefeef8f4d1ac6f6895812636233.png" alt="2" data-base62-sha1="2Y8WkMnGOzAPav62knTh3cGziDN" width="560" height="279"><br>
I install pytorch in anaconda, and i want use it in the jupyter lab</p>

xdd: <p>anyone, help me! please, i wait here all the time</p>

paganpasta: <p>Does the pytorch-cpu version work ?</p>

xdd: <p>yes, but I can’t open in jupyter lab</p>

xdd: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/8/6852dca79985f5246b95f6525e898fec2700a2e4.png" data-download-href="https://discuss.pytorch.org/uploads/default/6852dca79985f5246b95f6525e898fec2700a2e4" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/8/6852dca79985f5246b95f6525e898fec2700a2e4_2_690x330.png" alt="image" data-base62-sha1="eSTbDTwTOgGY44dEiDQCKcevFCA" width="690" height="330" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/8/6852dca79985f5246b95f6525e898fec2700a2e4_2_690x330.png, https://discuss.pytorch.org/uploads/default/optimized/3X/6/8/6852dca79985f5246b95f6525e898fec2700a2e4_2_1035x495.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/6/8/6852dca79985f5246b95f6525e898fec2700a2e4_2_1380x660.png 2x" data-dominant-color="FFFFFF"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1920×920 5.36 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
when i open it from anaconda, i get this page .</p>
<p>after i click the link , i enter</p>

xdd: <p>but if i do it from cmd, it won’t have the choice to chage the env</p>

paganpasta: <p>Ok, I think you’ll need to provide much more information on the issue. What your env info is, how you installed pytorch and how you access it. Try installing pytorch at base if you can’t switch the environment and see what the error log is.</p>

xdd: <p>in base enviroment, i install tensorflow, so at first I don’t to put them together, my enviorment is win10+cuda10.0, now I can access it from anaconda, just like I reply you at 4 floor(I need to click the link). I think it is hard to fix, i decide to give up. thanks</p>
Einsum with sparse tensors no longer working in 1.0?

gspr: <p>Having upgraded to 1.0, all my use of <code>einsum</code> has stopped working, giving me instead the runtime error: <code>sparse tensors do not have strides</code>. What’s going on here? I can’t see from the documentation that anything has changed in the <code>einsum</code>. What am I missing?</p>

Zhen_Li: <p>I encounter the same issue…</p>

tom: <p>I think this likely is a fallout from matmul not working for sparse arguments. einsum (currently) transforms the inputs and calls <code>torch.bmm</code>.</p>
<p>Best regards</p>
<p>Thomas</p>
How to install two CUDA versions on Windows

Florin_Andrei: <p>torch 1.3 uses CUDA 10.1</p>
<p>But on my system I have CUDA 10.0 which I use for other apps (and for torch 1.2).</p>
<p>Is there a way to install multiple CUDA versions on Windows that won’t trash the system?</p>

albanD: <p>I am not very familiar with windows, <a class="mention" href="/u/peterjc123">@peterjc123</a> might have a better idea.</p>
<p>My advice for linux system would be to install cuda within a conda environment to have nice encapsulation. Can you use conda environment on your setup?</p>

Florin_Andrei: <p>I use primarily Windows for CUDA-accelerated code nowadays. Linux - only occasionally.</p>
<p>I’ve tried to manage all my Python libraries only with pip so far, and I’ll keep doing that as long as possible. But I’ll give Conda some thought.</p>

albanD: <p>I don’t use conda either usually.<br>
But it has this nice feature of being able to create a contained environment where you can install libraries without conflicting with the main system. So I use it to test a specific cuda or cudnn version temporarily.</p>

peterjc123: <p>If you are using conda, then it’s not possible to install two cudatoolkit packages in one environment. You’ll have to use virtual environment. Actually, if you use the wheels package, you’ll find that we copied all the CUDA DLLs alongside with the main library.</p>

peterjc123: <p>What’s more, different versions of CUDA won’t conflict with each other because the DLLs have different names, but cuDNN DLLs will.</p>
Help with PyTorch equivalent for a Keras code

Amrit_Das: <p>Hey there, I am trying to reproduce a Keras code in PyTorch can anyone help me out.</p>
<pre><code class="lang-auto">base_inception = InceptionV3(weights='imagenet', include_top=False, 
                             input_shape=(299, 299, 3))
                             
# Add a global spatial average pooling layer
out = base_inception.output
out = GlobalAveragePooling2D()(out)
out = Dense(512, activation='relu')(out)
out = Dense(512, activation='relu')(out)
total_classes = y_train_ohe.shape[1]
predictions = Dense(total_classes, activation='softmax')(out)

model = Model(inputs=base_inception.input, outputs=predictions)

# only if we want to freeze layers
for layer in base_inception.layers:
    layer.trainable = False
    
# Compile 
model.compile(Adam(lr=.0001), loss='categorical_crossentropy', metrics=['accuracy']) 
model.summary()
</code></pre>
<p>I tried it out but I couldn’t work out the Global Average / Average Pooling part</p>
<pre><code class="lang-auto">model_ft = models.inception_v3(pretrained=True)
for param in model_ft.parameters():
    param.requires_grad = False
#num_ftrs = model_ft.classifier[6].in_features
#model_ft.classifier[6] = nn.Linear(num_ftrs,num_classes)

model_ft.aux_logits = False
num_ftrs = model_ft.fc.in_features

class inception_v3_see_smart(nn.Module):
    def __init__(self, originalModel):
        super(inception_v3_see_smart, self).__init__()
        self.Model = originalModel
        self.adaptive_pool = nn.AvgPool2d(2)
        #self.conv1 = nn.Conv2d(1000, 2000, 3)
        
        self.dense1 = nn.Linear(512,512)
        self.dense2 = nn.Linear(512,62)
    
    def forward(self, x):
        x = self.Model(x)
        print(x.shape)
        x = self.adaptive_pool(x)
        x = F.relu(self.dense1(x))
        x = F.relu(self.dense2(x))

        return x

model_ft = inception_v3_see_smart(model_ft)

print(model_ft)

model_ft = model_ft.to(device)
criterion = nn.CrossEntropyLoss()
</code></pre>
<p>Any help is really appreciated!<br>
<a class="mention" href="/u/ptrblck">@ptrblck</a></p>

albanD: <p>Isn’t global average pooling just average pooling where the kernel is the whole image?</p>

crowsonkb: <p>Try <code>nn.AdaptiveAvgPool2d(1)</code> for your <code>self.adaptive_pool</code> layer. Also, reshape the tensor from 4D to 2D after pooling, either by <code>x = x.view(x.shape[:2])</code> or an <code>nn.Flatten()</code> layer.</p>

Amrit_Das: <p>Sure thanks for the help will look into it and let you know</p>
Split and distribute a large tensor like what it does in torch.utils.data.DistributedSampler

pytorchdope: <p>AFAIK, the simplest way to do the distributed training (multiple modes) with Pytorch is something like:</p>
<pre><code>sampler = torch.utils.data.distributed.DistributedSampler(train_data)
data_loader = torch.utils.data.DataLoader(dataset, sampler=sampler)
model = torch.nn.DataParallel(model).cuda()

for data, target in data_loader:
    out = model(data)
    ...
</code></pre>
<p>But what if I already have a large tensor <code>data</code> in hand and would like to split and distribute it and get the same output as the above snippet? Specifically,</p>
<pre><code> model = torch.nn.DataParallel(model).cuda()
 data = do_sth_fuct(data)
 out = model(data)
</code></pre>
<p>Is there a PyTorch API to do so? Otherwise, what is the best way to achieve it? Thank you in advance!</p>

pritamdamania87: <blockquote>
<p>But what if I already have a large tensor  <code>data</code>  in hand and would like to split and distribute it and get the same output as the above snippet</p>
</blockquote>
<p>It really depends on where this large tensor is stored and how it is loaded. Is this large tensor stored in memory of one of the nodes? It might be helpful if you describe your system a bit more in detail and especially how the large tensor is computed/retrieved.</p>

pytorchdope: <p>Hi, thanks for your reply!</p>
<p>The tensor is stored in one of the nodes. More spefcifically, I have, say, two nodes and each of them have 8 gpus. I have a text dataset <code>train.txt</code>. I have written a function that convert the text data to  large tensor <code>X</code>.</p>
<p>If I used <code>torch.nn.parallel.DistributedDataParallel</code> in the following way:</p>
<pre><code>model = torch.nn.parallel.DistributedDataParallel(mode.cuda())
</code></pre>
<p>Would <code>model(X)</code> do what I want, that is, split the X and distribute the pieces to 16 gpus?</p>
VGG19 for MNIST

Andrew_Ulmer: <p>Hello! I am a newbie and would like to adjust a pretrained VGG19 model to accept MNIST data. How should I do this?</p>

ptrblck: <p>One approach would be to resize your input images to <code>224x224</code> using <code>torchvision.transforms.Resize</code> and then inside the loop expand the channels to 3 using <code>data = data.expand(-1, 3, -1, -1)</code>.</p>
<p>Alternative approaches would be:</p>
<ul>
<li>changing the vgg19 architecture to take smaller input images (e.g. by removing certain layers and/or changing the input features of the first linear layer)</li>
<li>resizing the input images to <code>224x224</code> and adding a conv layer before the actual model, which results in 3 output channels</li>
</ul>
<p>Let me know, if one of these approaches would work for you.</p>
Most efficient way to rotate a 1000x3 tensor of coordinates?

Yuerno: <p>Hi! If I have a tensor that has the shape, say, [1000, 3], where each row is an x, y, z point, I was wondering what the most efficient way would be in PyTorch to apply a rotation to each of these points and get a transformed [1000, 3] tensor?</p>

dsuess: <p>In what form do you have the rotation available? Is it supposed to be a special type of rotation (e.g. only around the z axis) or any general rotation in 3D?</p>
<p>My suspicion is that as soon as you have more than a few points, the best thing is to create the 3x3 rotation matrix and perform a matrix multiplication on the tensor.</p>

Yuerno: <p>Just any general rotation. Matrix multiplication makes sense, thank you!</p>
[JIT] Scripted attributes inside module

zetyquickly: <p>Hello there,</p>
<p>I’ve faced some unexpected behaviour of <code>torch.jit.script()</code>. The issue may be reproduced with the following code.<br>
As I learned from docs ( <a href="https://pytorch.org/docs/stable/jit.html#id3" rel="nofollow noopener">https://pytorch.org/docs/stable/jit.html#id3</a> ) one can use custom TorchScript classe if it’s properly written. But…</p>
<pre><code class="lang-auto">import torch
from typing import Tuple

@torch.jit.script
class MyClass(object):
    def __init__(self, weights = (1.0, 1.0, 1.0, 1.0,)):
        # type: (Tuple[float, float, float, float])
        self.weights = weights
    def apply(self):
        # type: () -&gt; Tuple[float, float, float, float]
        return self.weights

class MyModule(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.class_field = torch.jit.export(MyClass(weights = (1.0, 1.0, 1.0, 1.0,)))
                                        
    def forward(self, x):
        self.class_field.apply()
        return x + 10

m = torch.jit.script(MyModule())
</code></pre>
<p>Produces such error:</p>
<pre><code class="lang-auto">RuntimeError: 
Module 'MyModule' has no attribute 'class_field' (This attribute exists on the Python module, but we failed to convert Python type: 'MyClass' to a TorchScript type.):
at script_test.py:20:8
    def forward(self, x):
        self.class_field.apply()
        ~~~~~~~~~~~~~~~~ &lt;--- HERE
        return x + 10
</code></pre>

zetyquickly: <p>It seems that I’ve solved the problem. Found out that attributes introduced in <code>__init__(self,...)</code> should be explicitly annotated the following way:</p>
<pre><code class="lang-auto">class MyModule(torch.nn.Module):
    class_field : MyClass
    def __init__(self):
        super().__init__()
        self.class_field = torch.jit.export(MyClass(weights = (1.0, 1.0, 1.0, 1.0,)))                      
</code></pre>
<p>Ofk, if they are custom scripted classes.</p>
<p>P.S. The proper way to initialize custom class field is still unclear to me, I mean:</p>
<pre><code class="lang-auto">self.class_field = torch.jit.export(MyClass(weights = (1.0, 1.0, 1.0, 1.0,)))
</code></pre>
<p>or just</p>
<pre><code class="lang-auto">self.class_field = MyClass(weights = (1.0, 1.0, 1.0, 1.0,))
</code></pre>
<p>Behaviour is the same</p>

driazati: <p>This is a bug, I’ve filed <a href="https://github.com/pytorch/pytorch/issues/29597" rel="nofollow noopener">https://github.com/pytorch/pytorch/issues/29597</a> since this should be something that we can do automatically.</p>
<p><code>export</code> is meant to be used as a decorator on functions that need to be compiled but are not called from <code>forward</code> or anything <code>forward</code> calls. So for your code you can just get rid of the call to <code>torch.jit.export</code>. See <a href="https://pytorch.org/docs/stable/jit.html#torch.jit.export" rel="nofollow noopener">these docs</a> for details.</p>
Detectron2 does not install due to grpcio error

Owais: <p>I am trying to install Detectron2 on a computation cluster and receive the following error regarding grpcio. Please note that I have already installed PyTorch 1.3, TorchVision and other dependencies.</p>
<p>Collecting grpcio&gt;=1.24.3 (from tensorboard-&gt;detectron2==0.1)<br>
Could not find a version that satisfies the requirement grpcio&gt;=1.24.3 (from tensorboard-&gt;detectron2==0.1) (from versions: 1.11.0, 1.12.1, 1.13.0, 1.15.0, 1.16.0, 1.16.1, 1.17.1)<br>
No matching distribution found for grpcio&gt;=1.24.3 (from tensorboard-&gt;detectron2==0.1)</p>
<p>Has any one else encountered this issue?</p>

albanD: <p>From pip <a href="https://pypi.org/project/grpcio/" rel="nofollow noopener">https://pypi.org/project/grpcio/</a>, it looks like the latest version is 1.25.<br>
So you should be able to find it. Not sure why you don’t see it as an option. Maybe upgrade pip?</p>

Owais: <p>Thanks for your response.<br>
My pip is up to date, although I re-checked it. It runs the 18.1 version.<br>
I checked the version of grpcio by trying to install  using pip, and it seems to be running grpcio 1.17.1 and this might be the problem. I’ll try updating grpcio and see where it takes me.</p>
How to quickly resume batchwise sorted tensor

IapCaL: <p>Hi,</p>
<p>I want to sort a tensor and transform on it. In the end, I need to resume the tensor to original order.<br>
The operation is batchwise. Are there any efficiency method to do that without “for loop”?</p>
<pre><code class="lang-auto">x = torch.randn(4, 10)
w = torch.randn(4, 10, 10)
x_t = torch.zeros_like(x)

values, indices = torch.sort(x)
values_t = torch.bmm(values.reshape(4, 1, 10), w) #batchwise transform

for b in range(x.size(0)):
    x_t[b, indices[b]] = values_t[b]

</code></pre>
<p>Thanks</p>

albanD: <p>Hi,</p>
<p>You can use <a href="https://pytorch.org/docs/stable/tensors.html?highlight=scatter#torch.Tensor.scatter_" rel="nofollow noopener"><code>.scatter_</code></a> to replace your for loop.</p>
When to set up autograd in nn module

Ivan_Z: <p>When we define the forward process, do we need to setup autograd for any tensors, such as hidden/cell state?<br>
For example,</p>
<pre><code>    h0 = torch.zeros(self.layer_dim, x.size(0), self.hidden_dim).requires_grad_()
    c0 = torch.zeros(self.layer_dim, x.size(0), self.hidden_dim).requires_grad_()
</code></pre>
<p>In some tutorials, I found out from the beginning to the end not even one torch is set as requires_grad.<br>
Is it because in nn.LSTM module it already handles this problem?</p>

albanD: <p>Hi,</p>
<p>This is because only the Tensors for which you want the <code>.grad</code> field to be populated needs the user to call <code>requires_grad_()</code>.<br>
When working with <code>nn</code>, this is done automatically for <code>nn.Parameter</code>s.<br>
So if you use <code>nn.Parameter</code>s, you never need to call this explicitely.<br>
Unless you want extra gradients to be computed for things that are not <code>nn.Parameter</code>s.</p>

Ivan_Z: <p>Thank you for your help!</p>
C++ program: Process finished with exit code 134 (interrupted by signal 6: SIGABRT)

wy-go: <p>When I write a simple C++ program using Clion, this error occurs: Process finished with exit code 134 (interrupted by signal 6: SIGABRT)<br>
Some days ago, I just install pytorch from source, but I’m using mac without cuda-support GPU, and I suspect there is something wrong with cuba. Thanks for help!</p>

albanD: <p>Hi,</p>
<p>Could you give more informations please about your setup and the error? Which gpu / cuda version are you using. What is the stack trace associated with the error?</p>
Encoding multivalue features

olix20: <p>I’m working on a siamese-like architecture with triplet loss where the network inputs are a mix of numerical, categorical and textual features. Some of the features like “alternate names” come with multiple values. I’d like to</p>
<ul>
<li>
<p>share my encoder weights so that the same (text) encoder used for single-value fields can be reused for multi-value fields too.</p>
</li>
<li>
<p>apply the text encoder on each value of a multivalue feature, followed by max pooling for extracting the most relevant signal out of the available values.</p>
</li>
</ul>
<p>what’s the most efficient way to achieve the above?</p>
<p>Thanks <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
From chainer to pytorch

Safa_Mathlouthi: <p>how can i change these two instructions in pytorch ?<br>
these instructions used in chainer code<br>
(knowing that vrn is a resnet+unet model )</p>
<pre><code class="lang-auto">x_train = vrn.xp.asarray(image)
y_train = vrn.xp.asarray(label)
</code></pre>
<pre><code class="lang-auto">for i in range(args.iteration):
        vrn.zero_grad()
        #vrn.cleargrads()
        image, label = load_sample(
            train_df,
            args.n_batch,
            args.input_shape,
            args.output_shape
        )
        x_train = vrn.xp.asarray(image)
        y_train = vrn.xp.asarray(label)
        logits = vrn(x_train, train=True)
        logits = [logit[slices_in] for logit in logits]
        loss = F.softmax_cross_entropy(logits[-1], y_train)
</code></pre>

albanD: <p>Hi,</p>
<p>Do you always load the same batch during training? Why does <code>load_sample</code> is not a function of i? You might want to check pytorch’s Dataset and Dataloader.</p>
<p>To replace <code>vrn.xp.asarray</code>, it depends what <code>image</code> and <code>label</code> are. And what this function was doing originally?</p>

Safa_Mathlouthi: <p>Hi,<br>
image and label are 3d data of niftii type</p>

albanD: <p>I guess the simplest would to do <code>x_train = torch.from_numpy(vrn.xp.asarray(image))</code>.<br>
But you could consider reimplementing this function directly using pytorch.</p>

Safa_Mathlouthi: <p>thank you i’ll try it</p>

Safa_Mathlouthi: <aside class="quote no-group" data-username="albanD" data-post="4" data-topic="60407">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/48/215_2.png" class="avatar"> albanD:</div>
<blockquote>
<p>torch.from_numpy(vrn.xp.asarray(image))</p>
</blockquote>
</aside>
<p>i got the same problem even i  tried <code>x_train = torch.from_numpy(vrn.xp.asarray(image))</code></p>

albanD: <p>What is the error you’re seeing?<br>
Can you share the full stack trace?</p>
Fool the classifier: loss functions driving towards uniform distribution

Gabrer: <p>For a classification task, I would need a loss function which is higher when the distribution over the classes is far from the uniform distribution, and vice-versa it would be lower when the distribution over the classes of a sample is uniform.</p>
<p>So, the aim is to force the model to generate features which make the samples evenly belonging to each class for an already trained classifier.</p>
<p>A viable way I thought is to use the <strong>KL Divergence</strong> function <code>torch.nn.functional.kl_div()</code> against the uniform distribution.</p>
<p>Is there any more straightforward way in PyTorch?</p>
Training Faster RCNN with small rectangular images

shomerj: <p>I am trying to train Faster RCNN with a MobileNet V2 backbone in PyTorch. I am running into some issues and I think it is due to my input images. My input images range from 40x40 px to 50x600 px. I am trying to predict just one class and I don’t need very tight bounding boxes, just has to include the entire object. I know that ROI pooling in Faster RCNN should help with my varied input images but maybe there are some hyperparameters that I should adjust.<br>
I did adjust the anchor boxes to this:<br>
<code>anchor_generator = AnchorGenerator(sizes=((16, 32, 64,),), aspect_ratios=((0.4, 0.5, 1.0,),))</code></p>
<p>Let me know if you have any advice or input that would help my training!</p>
Trying to understand targets in ASR with CTCLoss

andcarnivorous: <p>Hi everyone,<br>
I am trying to work with the LibriSpeech dataset to build an ASR model with LSTM layers.<br>
It is still not very clear to me how I should preprocess the data correctly. I have a list of input arrays which are the MFCC values for each whole sentence file, and a list of labels.<br>
I padded all the input MFCC vectors to have the same lenght as the longest audio file.<br>
In order to get the labels I mapped character indexes to each character in the sentence, so that</p>
<pre><code class="lang-auto">label_i = "how are you?"
vectorlabel_i = np.array([4,2,5,0,11,22,0,25,2,14])
</code></pre>
<p>I then padded the labels as well with a blank character to the longest sentence length.</p>
<p>It is still not clear to me, however, how I should move from here in order to feed inputs and labels into the model.<br>
Is each whole sentence a single input? Or should I feed batches of frames (e.g. a sentence with sequence 640 fed in batches of 32)?<br>
Is the format of the label correct? I don’t think I should use an embedder given how the output is supposed to be.</p>
<p>Thanks for reading.</p>
Classification with CNN

LizLima: <p>Hi, I have a question. I try to classification object with CNN. I read that the output(feature vector) of the model (sometimes) should be equal to the number of classes, but if I want the output to be a feature vector of 256(size) and the number of classes is 500. How compare the output with the target?</p>

ptrblck: <p>Usually you are using an output layer with the same number of outputs as your number of classes.<br>
If you need a feature vector in another shape, you could use the e.g. penultimate layer output for it (sometimes called FC7 features IIRC).</p>

LizLima: <p>Thank you for your answer <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
How can I get the gradients of the weights of layer?

Jiqing_Zhang: <pre><code class="lang-auto">outputs = net(inputs)
loss = criterion(outputs, targets)
loss.backward()
for name, value in net.named_parameters():
     print(name)
</code></pre>
<p>parts of result of print as following：</p>
<pre><code class="lang-auto">cls_net.conv1.weight
cls_net.bn1.weight
cls_net.bn1.bias
cls_net.layer1.0.conv1.weight
cls_net.layer1.0.bn1.weight
cls_net.layer1.0.bn1.bias
cls_net.layer1.0.conv2.weight
cls_net.layer1.0.bn2.weight
cls_net.layer1.0.bn2.bias
cls_net.layer1.0.conv3.weight
cls_net.layer1.0.bn3.weight
cls_net.layer1.0.bn3.bias
cls_net.layer1.0.downsample.0.weight
cls_net.layer1.0.downsample.1.weight
cls_net.layer1.0.downsample.1.bias
cls_net.layer1.1.conv1.weight
cls_net.layer1.1.bn1.weight
cls_net.layer1.1.bn1.bias
cls_net.layer1.1.conv2.weight
cls_net.layer1.1.bn2.weight
cls_net.layer1.1.bn2.bias
</code></pre>
<p>and now I  want to print grad of <em>cls_net.layer1.1.bn2.weight</em><br>
<code>print('-------&gt;grad ',net.cls_net.layer1.1.bn2.weight.grad)</code><br>
but I got an error:</p>
<pre><code class="lang-auto">Original exception was:
  File "train_autonet.py", line 109
    print('-------&gt;grad ',net.cls_net.layer1.1.bn2.weight.grad)
                                         ^
SyntaxError: invalid syntax

</code></pre>
<p>It seems a python error.</p>

Jiqing_Zhang: <p>or how can I print the gradients of weights of all layers?</p>

vainaijr: <p>do you mean something like this,</p>
<pre><code class="lang-auto">for name, param in net.named_parameters():
  print(name, param.grad)
</code></pre>

Jiqing_Zhang: <p>Thank you！It solved my problem,hah</p>
Pytorch1.3 doesn't support cuda10.0?

chuang_ma: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/5/153c6da1c3f113a78861c9b5c368a255aec61165.png" data-download-href="https://discuss.pytorch.org/uploads/default/153c6da1c3f113a78861c9b5c368a255aec61165" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/5/153c6da1c3f113a78861c9b5c368a255aec61165_2_690x280.png" alt="image" data-base62-sha1="31RuCc3u963544WpEi8y6AZvnxP" width="690" height="280" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/5/153c6da1c3f113a78861c9b5c368a255aec61165_2_690x280.png, https://discuss.pytorch.org/uploads/default/optimized/3X/1/5/153c6da1c3f113a78861c9b5c368a255aec61165_2_1035x420.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/1/5/153c6da1c3f113a78861c9b5c368a255aec61165.png 2x" data-dominant-color="2D2121"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1174×477 25.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
here in the picture, I can only see the cuda9.2 and cuda10.1, doesn’t pytorch support cuda 10.0?</p>

ptrblck: <p>You could build PyTorch from source with CUDA10.0, but the binaries were update to CUDA10.1, since we had some bugs with previous versions.</p>

dhpollack: <p><a href="https://anaconda.org/pytorch/pytorch/files" class="onebox" target="_blank" rel="nofollow noopener">https://anaconda.org/pytorch/pytorch/files</a></p>
<p>They have a build of 1.3 with CUDA 10.0 but they just don’t have it on their website.  You can install it with <code>conda install -c pytorch pytorch=1.3.0 cudatoolkit=10.0</code></p>

ptrblck: <p>Oh thanks for this information! I didn’t know they were still built and uploaded (and haven’t tried to install them <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"> ).</p>

Florin_Andrei: <p>Is there a way to install the CUDA 10.0 version with pip on Windows?</p>
Why weight parameters in model are not moved to device when done model.to(device)

Nithin_Rao: <p>Hi,<br>
I initialized a model weight and bias parameter in init to learn while training but during training these weights are not moved to device cuda on performing .to(device) while the rest of the parameters moved. Please see sample code snipped</p>
<pre><code class="lang-auto">class myModule(nn.Module):
	def __init__(self,in_dim,hid_dim,out_dim):
		super(myModule,self).__init__()
		self.encoder=nn.Sequential(
			nn.Linear(in_dim,hid_dim),
            nn.ReLU(),
            nn.Linear(hid_dim,hid_dim//2),
			nn.BatchNorm1d(hid_dim//2),
			nn.ReLU(),
			nn.Dropout(p=0.2),
            nn.Linear(hid_dim//2,hid_dim//3),
			nn.BatchNorm1d(hid_dim//3),
			nn.ReLU(),
			nn.Dropout(p=0.2),
			nn.Linear(hid_dim//3,out_dim)
			)
		# self.linear=nn.Linear(out_dim,1,bias=False)
		self.S=torch.randn((out_dim,out_dim),requires_grad=True)
		self.b=torch.randn(1,requires_grad=True)

	def forward(self,inp1,inp2):
		x=self.encoder(inp1)
		y=self.encoder(inp2)
		# out=torch.pow((out1-out2),2)
		out=torch.matmul(x,y.T).diag() - torch.matmul(torch.matmul(x,self.S),x.T).diag()-torch.matmul(torch.matmul(y,self.S),y.T).diag()+self.b
		return out

</code></pre>
<p>In the above module, I am getting error at out = torch.matmul line saying self.S and self.b are not on device cude.</p>
<p>Shouldn’t doing model.to(device) transfer all model parameters to cuda?<br>
Thanks for your help</p>

tom: <p>You need to either wrap them in <code>nn.Parameter</code> (for learnable) or use <code>self.register_buffer</code> to let the model know that they are part of its state.</p>
<p>Best regards</p>
<p>Thomas</p>

Nithin_Rao: <p>Thanks Tom, for your suggestion. updated as</p>
<pre><code class="lang-auto">		S=torch.randn((out_dim,out_dim),requires_grad=True)
		self.S = nn.Parameter(S,requires_grad=True)

		b=torch.randn(1,requires_grad=True)
		self.b = nn.Parameter(b,requires_grad=True)
</code></pre>
<p>It is working but performance is very slow on Tesla K80 (colab) not even faster than running of quadcore CPU(mac pro), any reason you can think of?</p>

tom: <p>Hard to say beyond the obvious <em>make your model larger</em>  / <em>increase batch size</em>.</p>
CUDA error (3): initialization error while using img.cuda()

Rokuki: <p>Getting this error in Flask + nginx + uwsgi , but works fine in gunicorn before.</p>
<p>Error msg:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "/home/AiServer/env/lib/python3.5/site-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/AiServer/env/lib/python3.5/site-packages/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/AiServer/env/lib/python3.5/site-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/AiServer/env/lib/python3.5/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File "/home/AiServer/env/lib/python3.5/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/AiServer/env/lib/python3.5/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "./face_recognition_main.py", line 717, in cate_reco
    img_tensor = image_loader(img_pil)
  File "./face_recognition_main.py", line 322, in image_loader
    if cuda: return image.cuda()  # assumes that you're using GPU
RuntimeError: cuda runtime error (3) : initialization error at /pytorch/aten/src/THC/THCCachingAllocator.cpp:507
</code></pre>
<p>Is it the uwsgi configuration issue? I wonder if you could help me with this.thx</p>

Rokuki: <p>Got it ， it is the uwsgi configuration issue , when i remove <strong>master = true</strong> , the error is gone</p>
<pre><code class="lang-auto">[uwsgi]
# master = true
</code></pre>

gvskalyan: <p>Can you please share your configuration file <a class="mention" href="/u/rokuki">@Rokuki</a>  since I am also facing the same issue .<br>
Tried commenting master = true and no of workers and threads set to default and OMP_NUM_THREADS=2 and 1.<br>
Thanks</p>
Where can i find nn.transformer examples?

hadaev8: <p>All examples i find in google about self made attention, but i wonna use official one.</p>

tom: <p>Try</p>
<p><a href="https://pytorch.org/tutorials/beginner/transformer_tutorial.html" class="onebox" target="_blank" rel="nofollow noopener">https://pytorch.org/tutorials/beginner/transformer_tutorial.html</a></p>
<p>Best regards</p>
<p>Thomas</p>

DonCano: <p>Note that this example uses only the encoder. It’ not the full transformer</p>

hadaev8: <p>Do i get it right and encoder return 2d array like batch_size*features. Can i get 3d array batch_size*position*features like embedding? I wonna try to feed it into simple rnn.</p>

xdwang0726: <p>The nn.TransformerEncoder returns a 3d tensor with shape [S, N, E] where S is the length of your input sequence, N is batch size, E represents feature dimensions.</p>
RuntimeError: OrderedDict mutated during iteration (while using hook)

isalirezag: <p>Im using hook in my code and after a few iterations i get this error:</p>
<pre><code class="lang-auto">File "/home/alireza/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 478, in __call__
    for hook in self._forward_hooks.values():
RuntimeError: OrderedDict mutated during iteration

</code></pre>
<p>does anyone know why?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/c/ceb570484666958d7963f5258c7cce0835206b48.png" data-download-href="https://discuss.pytorch.org/uploads/default/ceb570484666958d7963f5258c7cce0835206b48" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/c/ceb570484666958d7963f5258c7cce0835206b48_2_690x274.png" alt="image" data-base62-sha1="tuD5xmWQ8gyAMV7LwENpEMH3OyI" width="690" height="274" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/c/ceb570484666958d7963f5258c7cce0835206b48_2_690x274.png, https://discuss.pytorch.org/uploads/default/optimized/2X/c/ceb570484666958d7963f5258c7cce0835206b48_2_1035x411.png 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/c/ceb570484666958d7963f5258c7cce0835206b48.png 2x" data-dominant-color="121212"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1067×424 105 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

Steve_Hu: <p>Did you fixed the bug finally? i met the situation too…</p>
How deal with this question 'X can't be expression'?

Foolish: <p>optimizer=optim.SGD(model.parameters().lr=1e-3)<br>
^<br>
SyntaxError: keyword can’t be an expression</p>

vainaijr: <p>I think if you change</p>
<pre><code class="lang-auto">optimizer=optim.SGD(model.parameters().lr=1e-3)
</code></pre>
<p>to</p>
<pre><code class="lang-auto">optimizer=optim.SGD(model.parameters(), lr=1e-3)
</code></pre>
<p>then it will work</p>

Foolish: <p>oh，so awkward,thank you very much</p>
Runtime Error for nn.BCEWithLogitsLoss() with pos_weights

Arking1995: <p>Hi, I tend to run my binary segmentation codes with loss function nn.BCEWithLogitsLoss(). The positive and negative target is not balanced so I tend to use pos_weight. However, it returns the runtime error which I don’t know why.</p>
<p>Code:<br>
model = model.cuda()<br>
w_pos = torch.tensor([4.538])<br>
criterion = nn.BCEWithLogitsLoss(pos_weight = w_pos)<br>
optimizer = torch.optim.Adam(model.parameters(), lr=0.001)</p>
<p>…<br>
im = Variable(data[0].cuda())<br>
labels = Variable(data[1].cuda())<br>
out = model(im)<br>
out1 = torch.squeeze(out)<br>
loss = criterion(out1, labels.float())</p>
<p>Outputs:<br>
Traceback (most recent call last):<br>
File “D:/DSC/Planet_test/train_200.py”, line 317, in <br>
loss = criterion(out1, labels.float())<br>
File “C:\Users\he425\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\module.py”, line 547, in <strong>call</strong><br>
result = self.forward(*input, **kwargs)<br>
File “C:\Users\he425\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\loss.py”, line 601, in forward<br>
reduction=self.reduction)<br>
File “C:\Users\he425\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\functional.py”, line 2100, in binary_cross_entropy_with_logits<br>
return torch.binary_cross_entropy_with_logits(input, target, weight, pos_weight, reduction_enum)<br>
RuntimeError: expected device cpu and dtype Float but got device cuda:0 and dtype Float</p>
<p>Is there any solutions for this issue?</p>

ptrblck: <p>Move your <code>w_pos</code> to the same device as your data and target and it should work. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

Arking1995: <p>Hi, thanks for your friendly reply!</p>
<p>The problem is the error said it expected data from CPU, but I gave it from my GPU. You can see from my codes that my model, input image, and groundtruth labels are all in GPU by .cuda(), but w_pos is not. So I think it might be the problems from those 3 variables that are already in GPU?</p>
<p>I tried w_pos = torch.tensor([4.538]).cuda() before asking this question, but it showed the similar error. I think probably it was not related to w_pos? Any new ideas?</p>
<p>Thanks again for reply!</p>

ptrblck: <p>I tried the code with every tensor on the GPU but <code>w_pos</code> and got the same error message, thus I assumed this is the error.<br>
Could you post a code snippet to reproduce this error?</p>

Arking1995: <p>I think I solved the problem by your suggestion! Thank you very much!!!</p>
Change stride of last down sampling in resnet-50

farhad-bat: <p>I use Resnet-50 as backbone in my person re-identificator. However, I have a problem. The problem is that stride of the last down sample layer is 2 but I want to change it to one and still use pre-trained weights and during training I want to learn new weights.</p>

paganpasta: <p>So if I understood it correctly, the image is being downsampled by a factor of 2 and you want to change it 1 so as to keep the input shape preserved. I’m not sure where the last downsample layer is however, it would be before some fully connected layer. If you are planning on reusing the said fully connected layers’ weights, you’ll have to re-train the network. You won’t be able to reuse the weights from the downsample layer onwards.</p>

farhad-bat: <p>I want to use pre-trained weights before fully connected, I use resnet as a backbone, so I don’t need FC layer.</p>
Loss to penalize overestimation

Mohamed_Ragab: <p>I am working on a regression problem and I am using MSE loss. But in my case, I need to penalize late predictions more than early predictions. In other words, I need to penalize (y_pred-y_true)&gt;0 more than (y_pred-y_true)&lt;0.  I would appreciate it if someone recommends the best suitable loss function that can also be smooth like MSE?</p>

crowsonkb: <p>The sigmoid function is a soft version of the step function, so you can do:</p>
<pre><code class="lang-python">diff = y_pred - y_true
diff2 = diff**2
mask = torch.sigmoid(diff)
losses = diff2 * (1 - mask) + diff2 * mask * 2
loss = losses.mean()
</code></pre>
<p>This penalizes <code>(y_pred - y_true) &gt; 0</code> approximately twice as much as the reverse. If we set <code>diff</code> to <code>torch.linspace(-5, 5, 11)</code>, then <code>losses</code> ends up as:</p>
<pre><code class="lang-python">tensor([25.1673, 16.2878,  9.4268,  4.4768,  1.2689,  0.0000,  1.7311,  7.5232,
        17.5732, 31.7122, 49.8327])
</code></pre>

rustagiadi95: <p>Hi There,</p>
<p>For this purpose, I think that using Leaky ReLu on the loss and then back propagation will help you as it traces f(x) = x for x &gt; 0 and f(x) = negative slope * x for x &lt; 0. Here keeping negative slope between (0, 1) will help you.<br>
Be sure to check out <code>torch.nn.LeakyReLU</code>.</p>
<p>All the best.</p>

Mohamed_Ragab: <p>Thank you so much for your response.</p>

Mohamed_Ragab: <p>Ok, sure I will check LeakyRelu. Thanks a bunch</p>
Convert string/path and feed into hdf5 dataset

SU801T: <p>Hi All,</p>
<p>I’m aware that this isn’t a pytorch issue directly, but was wondering, how do I convert a string or path into something that can be fed into a hdf5 table. For example, I am returning a numpy img array, label, and path to the image, from a dataloader, where the path to the image would look like this:</p>
<pre><code class="lang-auto">('mults/train/0/5678.ndpi/40x/40x-236247-16634-80384-8704.png',)
</code></pre>
<p>I basically want to feed it into a table like this:</p>
<pre><code class="lang-auto">path = 'data_' + str(phase) + '.hdf5'
hdf5_file = h5py.File(path, mode='w')
hdf5_file.create_dataset(str(phase) + '_img_paths', (len(dataloaders_dict[phase]),))
</code></pre>
<p>Maybe I am wrong to feed into such data into a table <img src="https://discuss.pytorch.org/images/emoji/apple/man_shrugging.png?v=9" title=":man_shrugging:" class="emoji" alt=":man_shrugging:"></p>
Need help with 3d classifier

Gleb_Shishaev: <p>I can’t understand how to change this code to convert it from the classifier of the whole 3d grid to a classifier of every cell in a grid.<br>
Initial dimensionality of every grid is 16 x 12 x 10 and 2 classes for every cell</p>
<pre><code class="lang-auto">class Grids_Conv3d(nn.Module):
    def __init__(self, out_1=32, out_2=64):
        super(Grids_Conv3d, self).__init__()
        self.layer1 = nn.Sequential(
            nn.Conv3d(in_channels=1, out_channels=out_1, kernel_size=5, stride=1, padding=2),
            nn.ReLU(),
            nn.MaxPool3d(kernel_size=2, stride=2))
        self.layer2 = nn.Sequential(
            nn.Conv3d(in_channels=out_1, out_channels=out_2, kernel_size=5, stride=1, padding=2),
            nn.ReLU(),
            nn.MaxPool3d(kernel_size=2, stride=2))
        self.drop_out = nn.Dropout()
        self.fc1 = nn.Linear(1536, 1920)
        self.fc2 = nn.Linear(1920, 2) #number of classes
</code></pre>
<pre><code>def forward(self, x):
    out = self.layer1(x)
    out = self.layer2(out)
    out = out.view(out.size(0), -1)
    out = self.drop_out(out)
    out = self.fc1(out)
    out = self.fc2(out)
    return F.log_softmax(out, dim=1)
</code></pre>
<p>Could anyone help me with this?<br>
Thank you!</p>

ptrblck: <p>What does “cell” mean in this context?<br>
Based on your description, I assume you would like to index smaller “cells” in your grid and feed each one into your model.<br>
If that’s the case, you would have to make sure the output size of <code>layer2</code> stays the same, so that the <code>in_features</code> of <code>self.fc1</code> don’t need to be changed.</p>

Gleb_Shishaev: <p>Thanx for your reply.<br>
Every grid is an 3D object wich consists of cells. Every cell is a property and cells have some spatial correlation which I want to reproduce by cells classification.<br>
So I’m not sure (or understand how) that indexing of cells would help me.</p>
What should be the size of random crops?

zimmer550: <p>If I have an image of 800 by 600 then should the random crop still be 256 by 256 or is there a formula for deciding the size of random crop based on original image dimensions?</p>

JuanFMontesinos: <p>There is no formula but you should consider the size of the object in the original image.<br>
As you may imagine you need to catch enough info to be useful</p>
Where is an operation recorded?

BlueBlazin: <p>Let’s say you execute the following code:<br>
<code>z = x + y</code><br>
where x and y are Tensors. This should create a computation graph with a single “Add” node. But where in the PyTorch source code is this actually done?</p>
<p>Also, where exactly is this graph stored? We don’t initialize it ourselves. So it’s created behind the scenes?</p>

tom: <p>Provided x or y requires gradients, yes, PyTorch will do that.<br>
You can see it in <code>z.grad_fn</code> and follow it using <code>z.grad_fn.next_functions</code> (and go deeper beyond that).</p>
<p>Best regards</p>
<p>Thomas</p>
How to install caffe2 for Windows?

zimmer550: <p>I am following this tutorial to install caffe2 for WIndows <a href="http://gianni.rosagallina.com/en/posts/2018/10/04/caffe2-gpu-windows-1.html" rel="nofollow noopener">http://gianni.rosagallina.com/en/posts/2018/10/04/caffe2-gpu-windows-1.html</a></p>
<p>In Step 2, the author says we have to make changes to build_windows.bat and then he says to make changes to cmake. But are these changes supposed to be written in build_windows.bat (changes like<br>
-DTORCH_CUDA_ARCH_LIST=6.1 and so on as mentioned there)?</p>