akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Relationship between padding_idx argument of nn.Embedding and padding_value argument of pad_sequence function

coyote: <p>Hi everyone,</p>
<p>Here is my question. We’re using pad_sequence and pack_padded_sequence functions to align instances in a batch (i.e. to make them equal in length). Pad_sequence function has an argument called padding_value which is in general set to zero. Morever, there is a padding_idx function in nn.Embedding. My question is that: How should I initialize my vocabulary,embedding layer so that it can be in consistent with pad_sequence? For example, which one of the below is true ?</p>
<pre><code class="lang-auto">unique_words = ['word1','word2']
w2i = { i : unique_words[i] for i in range(0, len(unique_words) ) }
emb_layer = nn.Embedding(len(w2i),emb_size)
pad_sequence(_some_input_here_, batch_first=True, padding_value=0)
</code></pre>
<p>or:</p>
<pre><code class="lang-auto"> unique_words = ['word1','word2']
 unique_words += ['&lt;&lt;PAD&gt;&gt;']         # I am adding PAD token explicitly into my dictionary
 w2i = { i : unique_words[i] for i in range(0, len(unique_words) ) }
 emb_layer = nn.Embedding(len(w2i),emb_size,padding_idx=0) # Is this necessary ? 
 pad_sequence(_some_input_here_, batch_first=True, padding_value=0)
</code></pre>
<p>Could any of you help me on this issue ? Thanks.</p>

coyote: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Do you have any idea on this ? Thanks in advance!</p>
How do you serialize a model's state dictionary to json

Rajiv_Sambasivan: <p>I need to store the models state as json. Is there a way to get a json representation of the model’s state_dict?</p>

JuanFMontesinos: <p>I would say you can use tensor’s method to_list() to get a plain representation of a tensor as a list of lists. That way you can export the state dict as a json and recover it back.</p>
Multiple loss using DataParallel

theoseo: <p>Hi, I have multiple losses.<br>
It worked well on single gpu.<br>
However when I tried it using DataParallel, it stuck on loss.backward().</p>
<p>What should I consider when I use multiple loss on multiple gpus? Below code is a simple flow of my implementation.</p>
<pre><code class="lang-auto">criterion = nn.CrossEntropyLoss().cuda()
model = DataParallel(model).cuda()


model.train()
for i, (inputs, labels) in  enumerate(train_loader)

    labels = labels.cuda(non_blocking=True)    

    logits = model(inputs)

    loss_1 = criterion(logits[0], labels)             
    loss_2 = criterion(logits[1], labels)  

    loss = loss_1 + loss_2  
    loss.backward()


</code></pre>
<p>Thanks for your advice in advance!</p>

ptrblck: <p>Are you able to use <code>nn.DataParallel</code> with a single loss?</p>

theoseo: <p>Yes, I think I managed to do it.</p>

theoseo: <p>I found the problem I had.</p>
<pre><code class="lang-auto">os.environ['CUDA_LAUNCH_BLOCKING'] = '1'
</code></pre>
<p>This code was inserted to debug when I ran it on a single gpu. After I remove code above, my code works well.</p>
Recommended way of PyTorch model behind a high traffic REST API?

praateekmahajan: <p>As title suggests, I am wondering what’s the right way to expose a PyTorch model that thousands of requests a second?</p>
<p>Would one write a REST API in C++ and serve model using TorchScript?<br>
Or can one be lazy and serve through Python based Flask app (or something more resilient) and use PyTorch as is?</p>
<p>Or can one take the extra step and try to serve using GoLang based service and somehow use a model saved as TorchScript?  (If so, how would one leverage GPU based inference?</p>
<p>What are the pros cons of the above methods?</p>

alex.veuthey: <p>Technically, it depends on the size of the model you are using… But there’s a reason people use C++ in production: python just isn’t fast enough for that kind of traffic.</p>
<p>I’ve done a classification demo website in the past, python/pytorch-only, with Flask or Bottle or something like that, and requests took a couple of seconds to be processed (it was a ResNet-34 model, with input size close to ImageNet’s, run on a 1080Ti).</p>
<p>I’d recommend going by iteration: starting with python-only code, then switch to C++ if it’s too slow. Don’t know about GoLang, though!</p>
NVProf produces profiling info with unrecognizable names

dawnlidu: <p>Hi all,</p>
<p>I profile a model for inference with Torchscript. The profiling info shows method names like kernel 1, kernel 3. Why? How could I get profiling info with normal kernel names such conv, relu, etc.</p>
<p>Thanks.</p>
<p>Du</p>

ptrblck: <p>Are you using the nvtx markers?<br>
If so, you can pass your desired names to these ranges, which will be shown in the profiler later.</p>
How to load your own dataset？

SangYC: <p>How should I inherit the ‘Dataset’ class to load data?<br>
I don’t understand why to use ‘<strong>len</strong>’,and ‘<strong>getitem</strong>’?</p>

ptrblck: <p>The <code>__len__</code> method should return the length of your dataset.<br>
In the standard use case you would just return the number of all samples you would like to load.<br>
For more advanced use cases, you could manipulate the length of the dataset artificially to e.g. support some custom sampling strategy.</p>
<p>Your <code>Dataset</code>'s <code>__getitem__</code> will be called with indices in the rage <code>[0, len(dataset)]</code>.<br>
Using this <code>index</code> you are able to implement code to load each sample (e.g. each image file) and process it there.<br>
This workflow enables you to lazily load the data, so that you can work with large datasets, which wouldn’t fit into your system RAM.</p>
<p>Also, have a look at the <a href="https://pytorch.org/tutorials/beginner/data_loading_tutorial.html">data loading tutorial</a> for more information.</p>
Runtime error: CUDA Error: no kernel image is available for execution on this device

parth_soodan: <p>Hi, I am new to using Pytorch. I am facing the above error. My system specifications are given below:</p>
<p>OS : Ubuntu 16.04<br>
CUDA version - 10.1<br>
Device count - 1<br>
Device name - GeForce GT 720<br>
Device capability - (3,5)<br>
Pytorch version - 1.4.0</p>
<p>Can someone guide me as to how to resolve this issue?</p>
<p>Thanks in advance.</p>

ptrblck: <p>The PyTorch binaries ship with CUDA for GPUs with compute capability &gt;=3.7, while your GPU has 3.5, so you would need to build from source as shown <a href="https://github.com/pytorch/pytorch#from-source">here</a>.</p>
<p>Let us know, if you get stuck somewhere. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>
How to free up all memory pytorch is taken from gpu memory

Diyago: <p>Pytorch is taking more memory then move on to the next loop, so eventually fails out of cuda memory:</p>
<p>I have some kind of high level code, so model training and etc. are wrapped by <code>pipeline_network</code> class. My main goal is to train new model every new fold.</p>
<pre><code>for train_idx, valid_idx in cv.split(meta_train[DEPTH_COLUMN].values.reshape(-1)):

        meta_train_split, meta_valid_split = meta_train.iloc[train_idx], meta_train.iloc[valid_idx]
        
        pipeline_network = unet(config=CONFIG, suffix = 'fold' + str(fold), train_mode=True)
</code></pre>
<p>But then I move on to 2nd fold everything fails out of gpu memory:</p>
<pre><code>RuntimeError: cuda runtime error (2) : out of memory at /pytorch/torch/lib/THC/generic/THCStorage.cu:58
</code></pre>
<p>At the end of epoch I tried to manually delete that pipeline with no luck:</p>
<pre><code> def clean_object_from_memory(obj): #definition
    del obj
    gc.collect()
    torch.cuda.empty_cache()

clean_object_from_memory( clean_object_from_memory) # calling
</code></pre>
<p>Calling this didn’t help as well:</p>
<pre><code>def dump_tensors(gpu_only=True):
        torch.cuda.empty_cache()
        total_size = 0
        for obj in gc.get_objects():
            try:
                if torch.is_tensor(obj):
                    if not gpu_only or obj.is_cuda:
                        del obj
                        gc.collect()
                elif hasattr(obj, "data") and torch.is_tensor(obj.data):
                    if not gpu_only or obj.is_cuda:
                        del obj
                        gc.collect()
            except Exception as e:
                pass
</code></pre>
<p>How can reset pytorch then I move on to the next fold?</p>

cyberjoac: <p>Hi, did you find a solution?</p>
Trying to load a torch model via Dropbox

Bastulli: <p>I’m trying to find a way to download a saved pytorch model hosted on Dropbox and load it into pytorch.  I can’t save or write a file to the server that is running pytorch.  I tried to download the torch model with the code below and got a attribute error.  Any other ideas do load a model without writing to a file?</p>
<pre><code class="lang-auto">        dropbox_url_act = "https://www.dropbox.com/path_here?&amp;dl=1"
        dropbox_url_cri = "https://www.dropbox.com/path_here?&amp;dl=1"
        
        req = requests.get(dropbox_url_act)
        actor = pickle.loads(req.content)
        
        req = requests.get(dropbox_url_cri)
        critic = pickle.loads(req.content)
        
        agent.actor_local.load_state_dict(torch.load(actor))
        agent.critic_local.load_state_dict(torch.load(critic))
</code></pre>
<p>AttributeError : ‘int’ object has no attribute ‘seek’. You can only torch.load from a file that is seekable. Please pre-load the data into a buffer like io.BytesIO and try to load from it instead.</p>

Bastulli: <p>Also, when trying to use torch.as_tensor, I get the following error.</p>
<p>During the algorithm initialization, the following exception has occurred: RuntimeError : Overflow when</p>
<pre><code class="lang-auto">unpacking long
  at Initialize in main.py:line 26
 :: self.agent.actor_local.load_state_dict(torch.as_tensor(actor))
 RuntimeError : Overflow when unpacking long
</code></pre>

Bastulli: <p>I found a solution using BytesIO with torch.load()</p>

Sarah_W: <p>Hi, how exactly did you solve it? <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

Bastulli: <pre><code>    from io import BytesIO
    
    '''Self.Download is a function that allows me to download from dropbox"""
    act_b64_str = self.Download("enter_url_here")
    cri_b64_str = self.Download("enter_url_here")
    
    # String Encode to bytes
    act_b = act_b64_str.encode("UTF-8")
    
    # Decoding the Base64 bytes
    act_d = base64.b64decode(act_b)

    # String Encode to bytes
    cri_b = cri_b64_str.encode("UTF-8")
    
    # Decoding the Base64 bytes
    cri_d = base64.b64decode(cri_b)

    self.agent.actor_local.load_state_dict(torch.load(BytesIO(act_d), map_location=lambda storage, loc: storage))
    self.agent.critic_local.load_state_dict(torch.load(BytesIO(cri_d), map_location=lambda storage, loc: storage))</code></pre>

Bastulli: <p>Its also worth noting that the Pytorch model was converted to Base64 before uploaded on dropbox</p>
Can't load state_dict

mzmey37: <p>I have trained a network,<br>
saving code is next:</p>
<pre><code class="lang-auto">model_to_save = model.module if hasattr(model, 'module') else model  # Only save the model it-self
output_model_file = args.output_dir / "pytorch_model.bin"
torch.save(model_to_save.state_dict(), str(output_model_file))
</code></pre>
<p>Then I try to load it and receive error:</p>
<pre><code class="lang-auto">&gt;&gt;&gt; model.load_state_dict(torch.load('./finetuned_lm/pytorch_model.bin'))
IncompatibleKeys(missing_keys=[], unexpected_keys=[])
</code></pre>
<p>These keys are empty, but still there is an error.<br>
What can be a reason to this?</p>

ptrblck: <p>The <code>IncompatibleKeys</code> message is just to show you that there were no missing or unexpected keys, thus the loading should be fine.</p>

mzmey37: <p>Wow! Thank you! Who knows, how much time I would spend investigating this <img src="https://discuss.pytorch.org/images/emoji/apple/neutral_face.png?v=9" title=":neutral_face:" class="emoji" alt=":neutral_face:"></p>

ptrblck: <p>The message seems to be a bit confusing and will most likely be improved. We are tracking this issue <a href="https://github.com/pytorch/pytorch/issues/20128" rel="nofollow noopener">here</a>.</p>

amandayeyan: <p>Please revise the message as it really confused me for some time debugging. It looks like an error message. <img src="https://discuss.pytorch.org/images/emoji/apple/sweat_smile.png?v=9" title=":sweat_smile:" class="emoji" alt=":sweat_smile:"></p>
How to inplace resize variables that require grad

aashaka: <p>I would like to resize inplace a variable that requires grad. Using <code>t.resize_(size)</code> fails with an error <code>RuntimeError: cannot resize variables that require grad</code>. Further, the solution mentioned in <a href="https://stackoverflow.com/a/50722403/4217367" class="inline-onebox" rel="noopener nofollow ugc">python - Resize PyTorch Tensor - Stack Overflow</a> doesn’t work because since Pytorch1.1, it seems that is has been disallowed (<a href="https://discuss.pytorch.org/t/runtimeerror-set-sizes-contiguous-is-not-allowed-on-tensor-created-from-data-or-detach-in-pytorch-1-1-0/44208" class="inline-onebox">RuntimeError: set_sizes_contiguous is not allowed on Tensor created from .data or .detach(), in Pytorch 1.1.0</a>). Hence it fails with</p>
<blockquote>
<p>RuntimeError: set_sizes_contiguous is not allowed on a Tensor created from .data or .detach().<br>
If your intent is to change the metadata of a Tensor (such as sizes / strides / storage / storage_offset)<br>
without autograd tracking the change, remove the .data / .detach() call and wrap the change in a <code>with torch.no_grad():</code> block.</p>
</blockquote>
<p>However, I had already done <code>t.resize_(size)</code> with a torch.no_grad() when it failed.</p>
<p>Is there a way to inplace resize a variable that requires grad? I will be resizing the tensor during backward pass again before calculating gradients, so I don’t think there will be a problem regarding gradients.</p>
<p>Thanks!</p>

ziqipang: <p>Sorry if my suggestion does not work, but you may try function “view” in this case.</p>
<p>You may find a lot of code using view for reshaping the tensors.</p>
How does on initialize weights using a non-nn.init function?

Baka_Moon: <p>Hello everyone.</p>
<p>Let’s say I wanted to use a torch function such as</p>
<p><code>torch.range(1, X, 0.5)</code> to initialize my weights for all CNN layers, where <code>X</code> would need to be flexlibe depending on size of CNN filters. How can I do so?</p>
<p>With nn.init it was very easy; I would just find the method I want to use within init library and just feed in the layer weights as a parameter to the method.</p>
<p>For example:</p>
<pre><code class="lang-auto">def init_weights(m):
    
    if type(m) == nn.Conv2d:
        
        torch.nn.init.uniform_(m.weight) # initialize weights with uniform weights
</code></pre>
<p>It seems vanilla torch has a more extensive collection of methods to help me come up with a more customized initialization.<br>
The above <code>range</code> method  is just a contrived example, but it will help me understand how to create my own initialization method using vanilla Pytorch methods.</p>
<p>I’m sorry if this question has been asked already.</p>

G.M: <p>Use <code>torch.no_grad()</code></p>
<pre><code class="lang-auto">with torch.no_grad():
    m.weight.DO_SOMETHING_()
</code></pre>
Issues with TorchVision CMakeLists.txt

ntm0110: <p>When attempting to build torchvision (master) on Ubuntu 18.04 using cmake, I follow the instructions from the GitHub docs and get the following issue with the torchvision CMakeLists.txt when running cmake -Dtorch_DIR="/path/to/libtorch/share/cmake/Torch" -Dpybind11_DIR="/path/to/pybind11/build/mock_install/share/cmake/pybind11" …</p>
<p>CMake Error at CMakeLists.txt:49 (install):<br>
install TARGETS given no LIBRARY DESTINATION for shared library target “torchvision”.</p>

ntm0110: <p>Seems to have been fixed in recent update to master.</p>
Reading the same data with multiple trainings

Carsten_Ditzel: <p>I am running different trainings on different GPUs but all of them access the same data, i.e. png images in data directories.</p>
<p>The data in only read, not written to.</p>
<p>Is that a problem and can lead to inconsistent results?</p>
<p>I am grateful for information about that.</p>
<p>thanks in advance</p>

kevalmorabia97: <p>I am running different tranings on 3 different GPUs all accessing same data. When running only 1 training, it is much faster than running all 3 (on different GPUs). Is it because all programs are accessing the same data?<br>
Can anyone please answer this?<br>
Thanks.</p>

ptrblck: <p>If you are using <code>nn.DataParallel</code>, and the model workload is small, you might see the overhead of the communication (scatter, gather of model parameters, data etc.).<br>
We usually recommend to use DistributedDataParallel with a single process for each device.</p>

kevalmorabia97: <p>I am not using nn.DataParallel. I am just running 3 different independent programs all on different GPUs that access the same data. In this case, shall I use DistributedDataParallel with a single process for each device. If so, can you please tell me how to use that? Also, would it be better to create 3 copies of data and all 3 process their copy of data? Can this reduct the overhead?</p>
<p>P.S. - I am using AWS’s p3.8xlarge machine</p>
<p>Thanks!</p>

ptrblck: <p>I think your IO operations might create a bottleneck, if all processes read from the same disk.<br>
Copying the data onto the same drive won’t avoid the bottleneck.</p>
Unexpected behavior of torchvision.ops.nms

viniciusarruda: <p>Following the example below and looking the nms <a href="https://github.com/pytorch/vision/blob/e2a8b4185e2b668b50039c91cdcf81eb4175d765/torchvision/csrc/cpu/nms_cpu.cpp" rel="nofollow noopener">source code</a>, I expected a <code>NaN</code> error, as the intersection and union will be zero.</p>
<pre><code class="lang-auto">import torchvision  # torchvision==0.5.0+cpu
import torch        # torch==1.4.0+cpu

boxes = [[0.0, 0.0, 1.0, 1.0],
          [2.0, 1.0, 1.0, 2.0]]

boxes = torch.tensor(boxes)
scores = torch.tensor([1., 0.5])

keep = torchvision.ops.nms(boxes, scores, 0.7)
</code></pre>
<p>If this same example is used with <a href="https://github.com/rbgirshick/fast-rcnn/blob/master/lib/utils/nms.py" rel="nofollow noopener">this</a> nms implementation (removing the +1 from the source code to be equivalent to the torchvision implementation), it raises a <code>NaN</code> error as expected.</p>
<p>Am I missing something ?<br>
Thanks.</p>

viniciusarruda: <p>I opened <a href="https://github.com/pytorch/vision/issues/1870" rel="nofollow noopener">this</a> github issue and it was answered and solved.</p>
"Learning PyTorch with Examples"-Tutorial: Learning Rate

Alex07: <p>Hi,</p>
<p>I am following the official <a href="https://pytorch.org/tutorials/beginner/pytorch_with_examples.html" rel="nofollow noopener">Tutorial</a> “Learning PyTorch with Examples” and don’t understand the step from “Defining new autograd functions” to using the PyTorch nn module: From the description and the code it seems like we would be doing the exact same thing. However, we have to adjust the learning from 1e-6 to 1e-4 to keep converging with the nn model, so it appears that we are not computing the same model &amp; gradients.<br>
Can someone explain what the difference is?</p>
<p>Thanks for your help!</p>
Is this a correct implementation of focal loss

bing: <p>Hi,<br>
I am trying to implement a focal loss. Below is my approach.  I have just modified the cross - entropy loss.<br>
Is it correct?</p>
<pre><code class="lang-auto">import torch.nn.functional as F
import torch.nn as nn
class FocalLoss():
    __constants__ = ['weight','ignore_index','reduction']
    
    def __init__(self, weight = None, size_average = None, gamma = 0, alpha = 0):
        super(FocalLoss, self).__init__(weight,size_average, reduce, reduction)
        self.gamma = gamma
        self.alpha  = alpha
        self.ignore_index = ignore_index
        
    def forward(self, logits, labels):
        loss = F.cross_entropy(logits, target, weight = self.weight,reduction = self.reduction, ignore_index = self.ignore_index)
        
        loss = ((1-logit) ** self.gamma) * alpha * loss
        
        return loss
    
</code></pre>
<p><a href="https://arxiv.org/abs/1708.02002" rel="nofollow noopener">Focal Loss</a></p>
Expand vs Repeat: Semantic Difference?

Rafael_R: <p>Hi,</p>
<p>How do the below two ways of extending a batch_size x feature_size tensor differ?</p>
<pre><code class="lang-auto">visual_feature = visual_feat.expand([batch_size,batch_size,self.feature_size])
text_feature = sentence_embed.repeat(1,1,batch_size).view(batch_size,batch_size,self.feature_size)
</code></pre>

albanD: <p>Hi,</p>
<ul>
<li>
<code>expand()</code> will never allocate new memory. And so require the expanded dimension to be of size 1.</li>
<li>
<code>repeat()</code> will always allocate new memory and the repeated dimension can be of any size.</li>
</ul>

aurooj: <p>Hi, regarding the same question, will it have any impact on results if we choose expand() over repeat()?</p>
<p>I am trying to learn pairwise relation within a list of vector (say Nxd). I want it to be learnable so I expand Nxd tensor to NxNxd and concat it to itself such as every vector is concatenated to every other vector resulting in NxNx2d tensor. I could use repeat() or expand() to obtain such tensor.</p>
<p>Keeping memory limitations in my mind, I used expand() but when I learn my model, I see the scores are same in every row where I was expecting to be diverse.</p>
<p>Do you think it could be because expand() does not allocate new memory?</p>

albanD: <p>Hi,</p>
<p>I didn’t understand you description fully, but it is easy to check: just replace it with repeat and see if you have the same behavior?</p>

aurooj: <p>Hi, Thanks for your reply!</p>
<p>Let me be more clear in explaining what I am trying to do here:<br>
For an Nxd tensor, I want to learn self-attention scores for each possible pair.<br>
I construct an NxNx2d tensor by doing the following:</p>
<pre><code class="lang-auto">    def tile_concat(self, in1, in2):
        assert (in1.shape == in2.shape)
        b, n, d = in1.shape
        t1 = in1.unsqueeze(2).repeat(1, 1, n, 1)
        t2 = in2.unsqueeze(1).repeat(1, n, 1, 1)
        out = torch.cat([t1, t2], -1)
        return out
</code></pre>
<p>where in1 and in2 are two different projections of same Nxd tensor.</p>
<p>I used a fully connected layer to map NxNx2d tensor to NxNx1, followed by a row-wise softmax to get probability scores.</p>
<p>Which are then used to obtain weighted encoding of Nxd size.</p>
<p>I hope this gives a more clear picture.<br>
Learning NxN matrix gives me same scores in each row. I have replaced torch.expand() with torch.repeat() now and training my model again.</p>
<p>But I doubt that it should cause any issue like I am having. Can you point to any other problem which may cause such a behavior?<br>
Thansk in advance!</p>
Conv2d_unpack and conv2d_prepack behavior

stihl1210: <p>Hey I have a question about</p>
<p><code>torch.ops.quantized.conv2d_unpack(self._packed_params)</code></p>
<p>Why returned weight in int8 and bias in fp32? How can i convert bias to fixed point?</p>
<p>Also i wanted to understand how <code>torch.ops.quantized.conv2d_prepack</code> this function is structured and what how packed_params is created.</p>

dskhudia: <p>Bias is kept in fp32 format for eager mode quantization and dynamically quantized while computing quantized FC/Conv. It’s returned in fp32 because that’s how it’s passed in to an operator as well. The reason for keeping bias in fp32 is the unavailability of input scale until the operator has executed so we can’t quantize bias until then.</p>
<p>To convert bias to quantized format, use input_scale * weight_scale with a zero_point = 0. See this <a href="https://github.com/pytorch/FBGEMM/blob/master/include/fbgemm/OutputProcessing-inl.h#L104-L108" rel="nofollow noopener">https://github.com/pytorch/FBGEMM/blob/master/include/fbgemm/OutputProcessing-inl.h#L104-L108</a> code for converting bias with act_times_weight scale.</p>
<p>Check out the code in <a href="https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/quantized/cpu/qconv_prepack.cpp" rel="nofollow noopener">https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/quantized/cpu/qconv_prepack.cpp</a> file for prepack function. If USE_FBGEMM is true, fbgemm_conv_prepack function is called for doing prepacking.</p>
Jit does not support doesn't support binary operator

G.M: <p>I have a code that looks like this</p>
<pre><code class="lang-auto">import torch as tc
from torch import jit
@jit.script
def func(inp):
    return inp&lt;&lt;1
a = tc.tensor([3,4,5])
func(a)
</code></pre>
<p>when I run it, I got the error:</p>
<pre><code class="lang-auto">torch.jit.frontend.NotSupportedError: unsupported binary operator: LShift:
  File "example.py", line 5
@jit.script
def func(inp):
    return inp&lt;&lt;1
              ~~ &lt;--- HERE
</code></pre>
<p>However, if I change <code>inp&lt;&lt;1</code> to <code>inp.__lshift__(1)</code>, it works. This also happens in C++. Is there a reason behind this or is it just that support for binary operators are not fully implemented yet? I’m using Pytorch 1.4.0 on windows with the corresponding version of libtorch.<br>
Thanks.</p>

driazati: <p>Looks like this was just an oversight, can you <a href="https://github.com/pytorch/pytorch/issues/new/choose">file a bug</a> with this info so we can have it in our issue tracker? (We’re happy to accept any PRs to add it too!)</p>
Tensor.pin_memory allocates memory on cuda:0

Evgeny_Shalnov: <p>The following code always allocates memory on device cuda:0, but I want to use another GPU for training</p>
<p>import torch<br>
a = torch.randn([20000, 20000])<br>
a.pin_memory() # &lt;- this operation allocates memory on “cuda:0”, but I want to leave it unused.<br>
b = a.cuda(‘cuda:1’)</p>

seliad: <p>Not sure why it allocates memory,<br>
But I’d try using torch.set_device(‘cuda:1’)</p>

albanD: <p>This most likely happens because to get pinned memory, we need a cuda context. And so we initialize a cuda context on the current device when it is needed. Which would be cuda:0 here by default.<br>
Changing the device will help.</p>
<p>Also if you never want to touch cuda:0, a good practice is to use the <code>CUDA_VISIBLE_DEVICES=1</code> environment variable. This acts on the nvidia driver for the current process and hides the other GPUs. That way, you are sure that you never use them.</p>

Evgeny_Shalnov: <p>torch.set_device(‘cuda:1’) helped</p>
Restrict trainable tensor values to specific range

alwynmathew: <p>How do I restrict values in a trainable tensor to a specific range, say [0, 1] ?</p>
<pre><code class="lang-auto">z = torch.zeros(1, 1, 20, 20)
z.requires_grad = True
optimizer = optim.Adam([z], lr)
for input in train_loader:
   z = z.clamp(0, 1) # Restrict trainable tensor values to specific range
   output = model(input + z)
   loss = loss_fn(output)
   optimizer.zero_grad()
   loss.backward()
   optimizer.step()
</code></pre>
<p>When I tried to clamp the tensor <code>z</code>, it threw a runtime error as below:</p>
<p><code>RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed.</code></p>
<p>Any help would be appreciated. <a class="mention" href="/u/ptrblck">@ptrblck</a> <a class="mention" href="/u/alband">@albanD</a></p>

albanD: <p>When you clamp <code>z</code> in your forward, you should not use the same variable name as the one that was given to the optimizer.</p>
<pre><code class="lang-auto">z_clamped = z.clamp(0, 1)
output = model(input + z_clamped)
</code></pre>
<p>should work.</p>
<p>Note that the clamp operation will give 0 gradients if z is out of the range, so this might not have a very good behavior while training as as soon as you get out of the range, you will get a 0 gradient. Maybe doing <code>z_clamped = z.sigmoid()</code> will have a better behavior to avoid 0 gradients.</p>
Can Windows10 release gpu memory manually？

mrzhaojun: <p>Maybe there’s a problem with my Pytorch programs, which causes the memory leak.I want to know if there is a way under Windows10 that can let me release these gpu memory?<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/2/92ec1095a7ccb9317e9ee6b997fa0a95e7910324.png" data-download-href="https://discuss.pytorch.org/uploads/default/92ec1095a7ccb9317e9ee6b997fa0a95e7910324" title="3f84307a-fa4a-41a2-a4a6-23408eaebbfe"><img src="https://discuss.pytorch.org/uploads/default/original/3X/9/2/92ec1095a7ccb9317e9ee6b997fa0a95e7910324.png" alt="3f84307a-fa4a-41a2-a4a6-23408eaebbfe" data-base62-sha1="kXJtAaCTU8sU6V4ePOEhKRwANpi" width="690" height="300" data-dominant-color="161616"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">3f84307a-fa4a-41a2-a4a6-23408eaebbfe</span><span class="informations">1137×495 17.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

albanD: <p>Hi,</p>
<p>Given that no process is running, this is most likely the nvidia driver using some memory no?</p>
Torch.stack cannot be useful for tensors with different dimensions

111229: <p>I would like to convert type(tmp) to tensor.</p>
<pre><code class="lang-auto">tmp = [torch.tensor([1]),torch.tensor([2,3])]
type(tmp) -&gt; &lt;class 'list'&gt;
type(tmp[0]) -&gt; &lt;class 'torch.Tensor'&gt;
type(tmp[0][0]) -&gt; &lt;class 'torch.Tensor'&gt;
</code></pre>
<p>How can I realize it?</p>

albanD: <p>Hi,</p>
<p>I’m afraid this is not possible with regular Tensors.</p>
<p>You can check the experimental nested tensor <a href="https://github.com/pytorch/nestedtensor">here</a> though as it seems it will fit your need.</p>
Confused about torch.max() and gradient

jam12: <pre><code class="lang-auto">x = Variable(torch.randn(1,3),requires_grad=True)
z,_ = torch.max(x,1)
z.backward()
print(x.grad)

Variable containing:
 1  0  0
[torch.FloatTensor of size 1x3]
</code></pre>
<p>I understand the max operation is a not differentiable operation. So why can I still get the gradient here?</p>

jpeg729: <p>max simply selects the greatest value and ignores the others, so max is the identity operation for that one element. Therefore the gradient can flow backwards through it for just that one element.</p>

SimonW: <p>Also, argmax is not continuous almost everywhere. But max is continuous everywhere.</p>

fiona_H: <pre><code class="lang-auto">z,y = torch.max(x,1)
</code></pre>
<p>So that is the reason <code>y</code> doesn’t have a gradient function?</p>

SunHaozhe: <p>What do you mean by “argmax is not continuous almost everywhere. But max is continuous everywhere” ?</p>
<p>Do you mean that we can do backpropagation with max operation but not argmax operation ?</p>

SimonW: <p>To be precise, I should have said that argmax is not differentiable, but max is.</p>

pcshih: <p>Is there a method to make index having gradient function?<br>
i.e.</p>
<blockquote>
<p>import torch<br>
h = torch.randn(1,2,5, requires_grad=True); print(h)<br>
val,idx = h.max(1, keepdim=True)<br>
print(val)<br>
print(idx)<br>
print(val)</p>
</blockquote>
<p>outputs are:</p>
<blockquote>
<p>tensor([[[-0.5372, -0.4683,  0.4891, -0.1686, -0.4147],<br>
[-1.4412,  1.2837, -0.4467,  0.1731,  1.3256]]], requires_grad=True)<br>
tensor([[[-0.5372,  1.2837,  0.4891,  0.1731,  1.3256]]],<br>
grad_fn=)<br>
tensor([[[0, 1, 0, 1, 1]]])</p>
</blockquote>
<p>I want the tensor([[[0, 1, 0, 1, 1]]]) to have gradient function.</p>

SimonW: <p>It is mathematically not differentiable, so no.</p>

pcshih: <p>In this <a href="http://openaccess.thecvf.com/content_ECCV_2018/papers/Mrigank_Rochan_Video_Summarization_Using_ECCV_2018_paper.pdf" rel="noopener nofollow ugc">paper</a> section 3.3</p>
<blockquote>
<p>We first select Y frames (i.e. keyframes) based on the prediction scores from the<br>
decoder.</p>
</blockquote>
<p>The decoder output is [2,320], which means non-keyframe score and key frame score of the 320 frames. We want to find a 0/1 vector according to the decoder output but the process of [2,320] → 0/1 vector seems not differentiable…</p>
<p>How to implement this in pytorch?</p>
<p>Thank you very much.</p>

netaglazer: <p>i guess it means that we can say the same thing to the minimum operation?</p>

netaglazer: <p>and also, can you explain it a little bit more?<br>
why the fact that it is identity operation for the max elememt changes the situation?<br>
and if so, why they invented the softmax?</p>

entslscheia: <p>Actually, softmax is more like softargmax… I have to say softmax is a terrible name.</p>
Using the fold function to return many separate channels

chenjesu: <p>Can the fold function not sum over each filtered bit of image to return a reconstructed image, but return each filtered bit of image in its own separate channel?<br>
I wrote some code that does this by abusing the batch argument.  But, I still have to use a for loop.  I want it to work in my training loop so I want it to do the same thing but in parallel. (assume the ones are a placeholder for the filter)  If there is no way to do it I will just use this.</p>
<pre><code class="lang-auto">
z = torch.zeros((16,4,16))
for i in range(16):
    z[i,:,i] = torch.ones((4,)) # our "ones" filter
print(f.fold(z, (5,5,), kernel_size=2).shape)
print(f.fold(z, (5,5,), kernel_size=2))
</code></pre>
<pre><code class="lang-auto">torch.Size([16, 1, 5, 5])
tensor([[[[1., 1., 0., 0., 0.],
          [1., 1., 0., 0., 0.],
          [0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.]]],


        [[[0., 1., 1., 0., 0.],
          [0., 1., 1., 0., 0.],
          [0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.]]],


        [[[0., 0., 1., 1., 0.],
          [0., 0., 1., 1., 0.],
          [0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.]]],


        [[[0., 0., 0., 1., 1.],
          [0., 0., 0., 1., 1.],
          [0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.]]],


        [[[0., 0., 0., 0., 0.],
          [1., 1., 0., 0., 0.],
          [1., 1., 0., 0., 0.],
          [0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.]]],
...etc
</code></pre>

chenjesu: <p>ok.  i think you can do this with scatter <a href="https://discuss.pytorch.org/t/what-does-the-scatter-function-do-in-layman-terms/28037/2" class="inline-onebox">What does the scatter_ function do in layman terms?</a></p>
<p>I’ll try it in the morning, but if anyone wants to do it feel free <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

albanD: <p>Not sure exactly what you want to do here. But you can remove the for loop because in a weird way, you set the diagonal of a bunch of matrices to 1. You can do this with: <code>z.diagonal(dim1=0, dim2=-1).fill_(1)</code></p>
Is it possible to do temporary calculations and remove them from computational graph?

Yuerno: <p>Hello all! In my current project, I have pass in 2D array of values (like [4096, 3] or [32768, 3]) into a neural network which processes these values and returns just [4096] or [32768]. The problem with this, as one might imagine, is that you run out of memory super fast at higher resolutions of input (like [64x64x64, 3]).</p>
<p>Luckily, only part of the values calculated from the neural network are necessary for calculations. Most of the values returned will be negative, some will be positive, and I only need to perform calculations on the positive values. Therefore, all those negative values just end up being unnecessary calculations for backprop and take up memory. Therefore, I was wondering if it’s possible to remove the extra calculations from the computational graph so that I can still work with higher resolutions and not have to backprop unneeded data.</p>

albanD: <p>Hi,</p>
<p>Are these values in two different branch of the network or all into the same Tensors?</p>
<p>Also for memory issues, you can try the <a href="https://pytorch.org/docs/stable/checkpoint.html">checkpoint</a> module to reduce memory usage at the cost of some compute.</p>

Yuerno: <p>Hey, thanks for the reply! I will take a look at checkpointing.</p>
<p>And these values are all in the same tensor. Basically, some sections of the returned [4096], [32768], etc. tensor are positive, others are negative, and I want to ignore any gradients calculated from the negative section.</p>

albanD: <p>The autograd looks at Tensors as one elementary object. So it won’t be able to keep part of it and remove others I’m afraid.</p>
LSTM feature importance

Roaldb86: <p>I have a model trained on 16 features, seq_len of 120 and in batches of 256.</p>
<p>I would like to test the loss on the model on a testset, with random sampling from a normal distribution for one features at a time so I can measure how important each features is (important features would give a high rise in loss when its random sampled)</p>
<p>Any advice on how to manipulate one feature at a time? the shape of the input is (256, 120, 16)</p>
<pre><code class="lang-auto">h = model.init_hidden(batch_size)
    with torch.no_grad():
        for inp, labels in loader:
                       
            if(train_on_gpu):
                inp, labels = inp.cuda(), labels.cuda()
            

            outputs, h = model(inp, h)
            _, predicts = torch.max(outputs, 1)
</code></pre>

G.M: <p>Try <code>inp[ : , : , F].normal_()</code> wherer <code>F</code> is the feature u want to manipulate.</p>

anantguptadbl: <p>A sample code</p>
<pre><code class="lang-auto">for curFeature in range(X.shape[1]):     
    model1Obj.zero_grad()
    print("Size of x1 is {0}".format(X.shape))
    # Alter the feature before you calculate the loss
    mu,sigma=norm.fit(X[:,curFeature])
    Y=copy.deepcopy(X)
    Y[:,curFeature]=np.random.normal(mu, sigma, 100)
    Y=Variable(torch.from_numpy(Y))
    output=model1Obj(Y)
    loss=criterion(target,output)
    print("The loss value for feature {0} is {1}".format(curFeature,loss.item()))
</code></pre>
<p>Hope this helps</p>

Roaldb86: <p>Thanks alot</p>
<p>I went with the following. looping over features on the outside of the function in order to log evry feature’s performance</p>
<pre><code class="lang-auto">def accuracy_test(model, loader, feature): 

    batch_size = loader.batch_size
    mu = 0
    sigma = 0.01
    
    train_on_gpu = (True if torch.cuda.is_available() else False)
    print("Training on gpu :{}".format(train_on_gpu))
    
    if(train_on_gpu):
        model.cuda()
        
    model.eval()
    losses = []

    criterion = nn.CrossEntropyLoss()
    h = model.init_hidden(batch_size)
    
    with torch.no_grad():
        for inp, labels in loader:
            perbutated_inp = copy.deepcopy(inp)
            perbutated_inp[ : , : , feature] = torch.HalfTensor(np.random.normal(mu, sigma, inp.shape[1]))
            
            inp = perbutated_inp
            
            if(train_on_gpu):
                inp, labels = inp.cuda(), labels.cuda()
            
            inp = inp.reshape(inp.shape[1], inp.shape[0], inp.shape[2])
        
            outputs, h = model(inp.float(), h)       
            loss = criterion(outputs, labels.long())
            losses.append(loss.item())
                      
            
    test_loss = np.mean(losses)
    
    return test_loss

</code></pre>
Model predicting in multi thread hangs gpu

hitlk: <p>I want to serve the model in multi thread, but it hangs the gpu and there is no error log.</p>
<p>Is the model’s forwarding thread-safe ?</p>
<p>This is my envs:</p>
<blockquote>
<p>PyTorch version: 1.0.1.post2<br>
Is debug build: No<br>
CUDA used to build PyTorch: 10.0.130</p>
<p>OS: Ubuntu 16.04.5 LTS<br>
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609<br>
CMake version: version 3.5.1</p>
<p>Python version: 3.5<br>
Is CUDA available: Yes<br>
CUDA runtime version: 10.0.130<br>
GPU models and configuration: GPU 0: GeForce RTX 2080<br>
Nvidia driver version: 410.79<br>
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.4.1</p>
</blockquote>
<p>and the sample code :</p>
<pre data-code-wrap="python"><code class="lang-python">from concurrent.futures import ThreadPoolExecutor
from itertools import repeat
import numpy as np

import torch
from torchvision import models

device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')


def task(model):
  num_tasks = 100

  for i in range(num_tasks):
    input_tensor = torch.ones([16, 3, 224, 224], dtype=torch.float32)
    with torch.no_grad():
      input_tensor = input_tensor.to(device)
      model(input_tensor)


vgg16 = models.vgg16(pretrained=False)
vgg16.eval()
vgg16.to(device)
num_workers = 16
with ThreadPoolExecutor(max_workers=16) as pool:
  results = pool.map(task, repeat(vgg16, num_workers))
  print(len(list(results)))
</code></pre>
<p>and <code>nvidia-smi</code>’s output is:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/2X/0/09dec5c11361c0f8c0ca1ad9681277a0b2a269c2.png" alt="image" data-base62-sha1="1pjAgZuZpGsxRcXVyN93jTp2dvI" width="630" height="298"></p>

LeviViana: <p>I don’t think you should do that unless you have multiple GPUs. <a href="https://pytorch.org/docs/stable/notes/cuda.html#use-nn-dataparallel-instead-of-multiprocessing" rel="nofollow noopener">This</a> might be useful.</p>
<p>My guess is that you are creating as much CUDA contexts as the number of CPU threads, so (i) you allocate too much memory and (ii) the contexts don’t run concurrently.</p>

leemengwei: <p>Me too here, with pytorch1.0.0 and CUDA10 and two GTX2080ti</p>

G.M: <p>Is it because of the multi-threaded data loading to GPU? <a href="https://pytorch.org/docs/stable/data.html#working-with-collate-fn" rel="nofollow noopener">https://pytorch.org/docs/stable/data.html#working-with-collate-fn</a><br>
It seems the process of moving data to GPU should not be done asynchronous. Hope this can help <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:">.</p>
Cuda convert to tensor.cpu

eranb: <p>Hi everyone,<br>
Can someone explain to me why I need to convert Cuda tensor to tensor.cpu first, and after that to Numpy?<br>
I know how to do it, but I want to know why I should do it.</p>
<p>"TypeError: can’t convert CUDA tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first. "</p>
<p>Best,<br>
Eran</p>

S_Ellis: <p>As far as I understand, and hopefully someone will correct me if I’m wrong:</p>
<ul>
<li>standard numpy arrays reside only in cpu memory</li>
<li>therefore, a cuda tensor must first be sent to the cpu before it can be converted to numpy</li>
</ul>
<p>If your question is really ‘why doesn’t calling .numpy on a cuda tensor send it to cpu by default, without the user having to specify?’, I don’t know. Surely there is a technical reason, that hopefully someone else can elaborate. You could of course write your own function to do both in one step, if you really want to avoid writing .cpu().numpy() every time, e.g.:</p>
<pre><code class="lang-auto">def cudaToNumpy(cudaTensor):
    return cudaTensor.detach().cpu().numpy() #I add a detach in case the tensor has gradients etc
</code></pre>

eranb: <p>I mean why doesn’t calling .numpy on a Cuda tensor.<br>
I just want to know what is the technical reason, but thanks!</p>

anantguptadbl: <p>Let us visualize it</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/f/bf644c69131bb2d331946107ae0ccfe3e9e6ac6c.png" data-download-href="https://discuss.pytorch.org/uploads/default/bf644c69131bb2d331946107ae0ccfe3e9e6ac6c" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/b/f/bf644c69131bb2d331946107ae0ccfe3e9e6ac6c.png" alt="image" data-base62-sha1="rj85i6mJ2OYzdlkMGIj1djsoXWk" width="690" height="344" data-dominant-color="D1D1D3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">937×468 7.69 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>What we need to understand is that in both the two dashes the architecture is very different. What the .cpu() command does is transfers the data into the memory linked to the x86 architecture which can then be leveraged by the numpy libraries. Numpy relies heavily on C which again is linked to the architecture of normal CPUs, hence the direct GPU Tensor to Numpy does not work</p>

eranb: <p>Thank you very match</p>
Stack CNN on the output of LSTM for text classification

alabijesujoba: <p>I’m trying to perform convolution on the output of LSTM in pytorch for text classification, while implementing, I discovered I needed to clone the output of the lstm to perform convolution on it as shown below:</p>
<pre><code class="lang-auto"># lstm representation 
output, (final_hidden_state, final_cell_state) = self.lstm(embedded, (h_0, c_0))
#clone the output gotten from lstm
conInput=output.clone()
#unsqueeze the output in dim = 1
conInput = conInput.transpose_(0, 1).unsqueeze(1)
#peform convolution on the processed output of lstm
conved_0 = F.relu(self.conv_0(conInput).squeeze(3))
conved_1 = F.relu(self.conv_1(conInput).squeeze(3))
conved_2 = F.relu(self.conv_2(conInput).squeeze(3))
</code></pre>
<p>Am I doing the right thing? Is there a better approach?</p>

G.M: <ol>
<li>Does not using <code>clone()</code> produces an error? If so, I suggest removing the <code>clone</code> and instead change the <code>transpose_</code> to the non in-place version and see if that works.</li>
<li>I guess that the <code>transpose_</code> would still be needed if <code>batch_first</code> of the lstm is set to true. Depending on ur case this might be wrong.</li>
<li>If u don’t need the hidden or the cell states(the cell states are likely to be useless in most cases), there is no need to keep the variables around.</li>
</ol>

alabijesujoba: <p>Not using <code>clone()</code> produces an error actually. Oh! Ok thanks. I would fix those points you raised and revert.</p>
Using CMake to create an executable that can run independently on other machines

navot: <p>Hello,</p>
<p>I’m using LibTorch and OpenCV for a program in Cpp. The compilation and building is done on Linux using CMake. The program builds and runs as expected.</p>
<p>I want to use the executable that CMake created on another Linux machine.<br>
The problem is that I don’t want to install either LibTorch nor OpenCV on the other machine. I’d rather supply the user with a single executable if possible.</p>
<p>How can CMake create a single independent executable?<br>
If making just a single file is irrelevant, how can CMake copy all needed libraries to a single directory?</p>
<p>The current CMake file:</p>
<p>cmake_minimum_required(VERSION 2.8)<br>
project(prediction)<br>
list(APPEND CMAKE_PREFIX_PATH  “libtorch”)  # the folder where libtorch in found<br>
set(CMAKE_BUILD_TYPE Release)</p>
<p>find_package( OpenCV REQUIRED )<br>
find_package( Torch REQUIRED )</p>
<p>if(NOT Torch_FOUND)<br>
message(FATAL_ERROR “Pytorch Not Found!”)<br>
endif(NOT Torch_FOUND)</p>
<p>message(STATUS “Pytorch status:”)<br>
message(STATUS "    libraries: ${TORCH_LIBRARIES}")</p>
<p>message(STATUS “OpenCV library status:”)<br>
message(STATUS "    version: ${OpenCV_VERSION}")<br>
message(STATUS "    libraries: ${OpenCV_LIBS}")<br>
message(STATUS "    include path: ${OpenCV_INCLUDE_DIRS}")</p>
<p>file(GLOB SOURCES<br>
“<em>.h"<br>
"</em>.cpp”<br>
)</p>
<p>include_directories( ${OpenCV_INCLUDE_DIRS} )<br>
add_executable(entrypoint ${SOURCES})<br>
target_link_libraries(entrypoint ${TORCH_LIBRARIES} ${OpenCV_LIBS})<br>
set_property(TARGET entrypoint PROPERTY CXX_STANDARD 14)</p>
<p>Thanks!</p>