akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Spatial_scale in torchvision.ops.roi_pool

Dishank_Bansal: <p>Can someone explain how to use <code>spatial_scale</code> in the <code>torchvision.ops.roi_pool( *input* ,  *boxes* ,  *output_size* ,  *spatial_scale=1.0* ))</code> using some example. Let’s say my original image is <strong>800*800</strong> and subsampling ratio is <strong>16</strong>, so feature map size is <strong>50*50</strong>. Now in the <code>input</code> to<code> torchvision.ops.roi_pool</code> we give <code>feature map</code> and <code>boxes</code> are in <strong>image coordinate scale</strong>.</p>
<p>So  <code>spatial_scale</code> should be equal to <code>subsampling ratio</code>?</p>

viniciusarruda: <p>Hi, did you found any explanation about this parameter ?</p>

Dishank_Bansal: <p><code>spatial_scale = 1/subsampling_ratio</code></p>
DefaultCPUAllocator: not enough memory

slavavs: <p>I have a mistake : RuntimeError: [enforce fail at …\c10\core\CPUAllocator.cpp:72] data. DefaultCPUAllocator: not enough memory: you tried to allocate 5760000 bytes. Buy new RAM!</p>
<p>I am trying to download a very large 60gb file. Is there a way to split a file into parts?</p>
<pre><code class="lang-auto">data_load = np.load('np_data_2y.npy', mmap_mode='r')

class NetDataset(Dataset):
    def __init__(self):
         self.input_data = data_load
        self.len = len(self.input_data) 
        self.x_data = torch.from_numpy(self.input_data)
        self.x_data = self.x_data.float()
    def __getitem__(self, index):
        return self.x_data[index]
    def __len__(self):
        return self.len
dataset = NetDataset()
train_loader = DataLoader(dataset=dataset, batch_size=batch_work, shuffle=False)
</code></pre>
A spectrogram inversion package for PyTorch

yoyololicon: <p>Hello folks~</p>
<p>I made an PyTorch package that included some classic spectrogram inversion algorithms (like Griffin-Lim) to recover phase information given only the magnitude response of audio. I would like to invite everyone to take a look and use it.</p>
<p>You can find its repository here: <a href="https://github.com/yoyololicon/spectrogram-inversion" rel="nofollow noopener">https://github.com/yoyololicon/spectrogram-inversion</a><br>
The docs: <a href="https://spectrogram-inversion.readthedocs.io/" rel="nofollow noopener">https://spectrogram-inversion.readthedocs.io/</a>.<br>
Any issues and contributions are wellcome.</p>
<p>Cheers <img src="https://discuss.pytorch.org/images/emoji/apple/slightly_smiling_face.png?v=9" title=":slightly_smiling_face:" class="emoji" alt=":slightly_smiling_face:"></p>

ptrblck: <p>Thanks for sharing your code! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"><br>
May I ask the naive question, when this would be used?<br>
Could we e.g. create a spectrum using a GAN and use your inversion algos to create the waveform?</p>

yoyololicon: <p>Exactly, as long as the spectrum follow the regular fourier representation.<br>
We assume the target magnitude spectrum that used in training is obtained using <code>torch.stft</code>.</p>

vincentqb: <p>Thanks for sharing your code indeed! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"> Are there additions you’d also like to see in the implementation available in the master branch of <a href="https://github.com/pytorch/audio/pull/365" rel="nofollow noopener">torchaudio</a>?</p>
Training the lstm forVariable length and want to ensure the backpropagation

dinesh_mareedu: <p>I am using packed sequence for variable length input sequences for using dataloader and using packpadsequence for giving the input to rnn , do I need to pad packed sequence for backpropagation . How can I ensure that the backprogation is correct!!</p>
<p>train_dataloader = torch.utils.data.DataLoader(train_event_loader, batch_size=2, shuffle=False,collate_fn=my_collate, num_workers=0)</p>
<p>def my_collate(batch):<br>
# batch contains a list of tuples of structure (sequence, target)<br>
data = [item[0] for item in batch]<br>
data = pack_sequence(batch, enforce_sorted=True)<br>
targets = [item[1] for item in batch]<br>
return [data, targets]</p>
<p>packed_seq_batch = torch.nn.utils.rnn.pack_padded_sequence( data,<br>
lengths=seq_lens, batch_first=True)</p>
<p>output, (hn, cn) = lstm(packed_seq_batch.float())</p>
<p>##<span class="hashtag">#Do</span> I need to do this step for backprop or shall I skip<br>
padded_output, output_lens = torch.nn.utils.rnn.pad_packed_sequence(output, batch_first=True, total_length=5)</p>
<p>Loss()</p>
How to implement tf.scatter_nd() function of tensorflow in pytorch

11196: <p>I want to implement the function called tf.scatter_nd() in pytorch. And I do not find a suitable function in pytorch. Does someone know how to do this?<br>
Thank you!</p>

albanD: <p>Hi,</p>
<p>You can use the <a href="https://pytorch.org/docs/stable/tensors.html#torch.Tensor.scatter_"><code>scatter_()</code></a> function.</p>

11196: <p>However, scatter_() is just a tensor operation. It does not have a autograd function. So maybe it cannot be used in network training.</p>

albanD: <p>All operations are actually autograd functions. You’ll get an error if the backward function is not implemented.</p>

lkhphuc: <p>Hi, can you help me with a minimal examples of converting a tf.scatter_nd using torch.Tensor.scatter_add_.<br>
The closest I’ve found is using the fancy indexing in pytorch, <a href="https://gist.github.com/airalcorn2/c7846d6fcb58a30b25ea6d97e16fe025" rel="nofollow noopener">https://gist.github.com/airalcorn2/c7846d6fcb58a30b25ea6d97e16fe025</a>.<br>
But the fancy indexing technique does not match with tf.scatter_nd when there is duplication in the index tensor, because in tf the behavior is accumulating, while fancy indexing will just replace the value at that index.<br>
Thanks.</p>

pg2455: <p><a href="https://pytorch.org/docs/stable/tensors.html?highlight=index_add_#torch.Tensor.index_add_" rel="nofollow noopener">index_add_</a> might be of help. I spent several hours looking for scatter_nd like function in pytorch, but all the links pointed to scatter_add_. The tensorflow like scatter_nd is more similar to index_add_.</p>
How to scale a circular map to 255?

I_am_you: <p>I want to create a circular map similar to how we would create a gaussian map and scale the circular map to 255. how could I do the same?</p>

ekilic: <p>the question is not clearly understood. Do you wanna draw like this<div class="lightbox-wrapper"><a class="lightbox" href="https://www.researchgate.net/profile/Yoichi_Sato2/publication/331218459/figure/fig5/AS:744488582516736@1554511318186/Example-of-Gaussian-Heatmap.png" title="" rel="nofollow noopener"><img src="https://www.researchgate.net/profile/Yoichi_Sato2/publication/331218459/figure/fig5/AS:744488582516736@1554511318186/Example-of-Gaussian-Heatmap.png" alt width="690" height="484"><div class="meta">
<svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use xlink:href="#far-image"></use></svg><span class="filename"></span><span class="informations">721×506</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use xlink:href="#discourse-expand"></use></svg>
</div></a></div></p>

I_am_you: <p>let me rephrase my question. I want to draw a circular map of radius 10 which is scaled at 255.</p>

ekilic: <p>What do you mean about ‘scaled at 255’? Is pixel value of center of circle is 255 and pixel value of border is 0?</p>

I_am_you: <p>yes that is right pixel value of center of circle is 255 and pixel value of border is 0.</p>

ekilic: <p>Maybe this sample work for you.</p>
<pre><code class="lang-auto">import numpy as np
import matplotlib.pyplot as plt


I = np.zeros((500, 500))
radius = 100
x, y = np.ogrid[-radius: radius, -radius: radius]
cy = 200
cx = 300
index = x**2 + y**2 &lt;= radius**2
a = np.sqrt((radius - abs(y))**2 + (radius - abs(x))**2)
a = (a / a.max()) * 255
I[cy-radius:cy+radius, cx-radius:cx+radius][index] = a[index]
plt.imshow(I)
plt.show()
</code></pre>
Best way to iterate through tensors

G.M: <p>Currently, I’ve seen 2 ways of iterating through a tensor.<br>
1.</p>
<pre><code class="lang-auto">for v in TENSOR:
    pass
</code></pre>
<ol start="2">
<li>
</li>
</ol>
<pre><code class="lang-auto">a = TENSOR.unbind(0)
for v in a:
    pass
</code></pre>
<p>I have 3 questions:</p>
<ol>
<li>Which of these 2 are faster in Python?</li>
<li>Which of these 2 are faster in TouchScript(I’ve seen the custome lstm uses the second way).</li>
<li>Following 2, which of the following is faster in C++:</li>
</ol>
<p>First.</p>
<pre><code class="lang-auto">for(int i=0;i&lt;TENSOR.size(0);++i){
    auto v=TENSOR[i];
}
</code></pre>
<p>Second.</p>
<pre><code class="lang-auto">auto vec=TENSOR.unbind(0);
for(int i=0;i&lt;vec.size();++i){
    auto v=vec[i];
}
</code></pre>
<p>Thanks</p>
What the different between net.zero_grad() and optimiser.zero_grad()

isaac1644: <p>I’m sorry for asking this question. But I really don’t  know the result .</p>

tux: <p>They do the same thing, the only difference is that one of them is through the optim module while the other is through the nn module. Let’s have a look at the doc.</p>
<p>From the optim doc <a href="https://pytorch.org/docs/stable/_modules/torch/optim/optimizer.html" rel="nofollow noopener">https://pytorch.org/docs/stable/_modules/torch/optim/optimizer.html</a>, you have :</p>
<pre><code class="lang-auto">def zero_grad(self):
        r"""Clears the gradients of all optimized :class:`torch.Tensor` s."""
        for group in self.param_groups:
            for p in group['params']:
                if p.grad is not None:
                    p.grad.detach_()
                    p.grad.zero_()
</code></pre>
<p>while from the nn doc (<a href="https://pytorch.org/docs/stable/_modules/torch/nn/modules/module.html#Module.zero_grad" rel="nofollow noopener">https://pytorch.org/docs/stable/_modules/torch/nn/modules/module.html#Module.zero_grad</a>), you have :</p>
<pre><code class="lang-auto">def zero_grad(self):
        r"""Sets gradients of all model parameters to zero."""
        for p in self.parameters():
            if p.grad is not None:
                p.grad.detach_()
                p.grad.zero_()
</code></pre>
<p>The only difference is <code>for group in self.param_groups</code> which represents the neural network parameter when you initialize your optimizer.</p>
How relevant are negative examples for a Unet segmentation model?

Luis_Costa: <p>Hello, I am working with a Unet segmentation model for medical image analysis, and I would like to know how important are negative examples (with empty masks) for my model to learn that some images are 100% negatives. I am asking this because I took a bunch of negative and added to my dataset, some kind of hard negative mining, and still, I am getting a lot of false positives. Does Unet learn anything with negative examples? Is there any other way to force my model to learn these ‘negative’ features?</p>
<p>My dataset is very skewed, I am working with medical image and I want to segment little specimens in 256x256 patches. The number of negative patches (no specimen) and the number of positive patches are balanced but the number of positive positive pixels is 1-2%.<br>
I read that weighted sigmoid cross-entropy might be a good idea to attenuate this difference and get a lower rate of false positives.</p>
<p>I would like to get your opinion</p>
<p>If you could also provide relevant information about this (articles, papers, questions) I would highly appreciate.</p>

CCL: <p>I saw your issue on a Pytorch U-Net implementation (<a href="https://github.com/milesial/Pytorch-UNet/issues/126" rel="nofollow noopener">https://github.com/milesial/Pytorch-UNet/issues/126</a>), I have the exact same problem with U-Net! Atm i’m getting a fairly high dice coefficient (&gt; 0.8), but poor predicted segmentation masks. I looked through the mask predictions and negative samples are definitely skewing the accuracy evaluation &amp; learning process!</p>

Luis_Costa: <p>How balanced is your dataset?</p>

CCL: <p>Approx 3:2 biased towards negative samples. What about you? I’m thinking about trying with <code>torch.utils.data.sampler.WeightedRandomSampler</code> to counteract the dataset imbalance problem. However, I’m not sure dice coefficient is the most suitable evaluation metric (it was not specified in the paper).</p>

Luis_Costa: <p>I have a very skewed dataset, 1:99 and adding negative examples is not alleviating the false positive rate.</p>
<p>I think my problem is the balancing I am doing is too favorable to the positive class. Maybe I need to modify the weights of each weights to each class. (Actually ironically I can be increasing the false positive rate by adding more negative examples with my balancing equations).</p>

CCL: <p>I’m not too sure how I can solve my issue, but to control the false positive rate, one can actually tweak the output threshold for the FCN. To reduce false positives, perhaps raising the threshold to above 0.5 would help.</p>
Difference between torch.nn and torch.nn.functional

SangYC: <p>What is the difference between torch.nn and torch.nn.functional?Many of the functions of these two modules are overlapping。</p>

CCL: <p>Duplicate of <a href="https://discuss.pytorch.org/t/what-is-the-difference-between-torch-nn-and-torch-nn-functional/33597" class="inline-onebox">What is the difference between torch.nn and torch.nn.functional?</a></p>
<p>Torch.nn defines Python classes (to be called in forward propagation), while torch.nn.fucntional defines operations which can be directly used.</p>
Output is not correct, though the loss is very less

satinder147: <p>I am trying road segmentation. I have trained a UNet like architecture. The loss becomes very less in two epochs itself. But when I try it on some data the output is not as expected. Only some dotted lines appear whereas the output should be a black and white image.</p>

mmisiur: <p>I think you need to elaborate more on the issue (and possibly post some code) if you want someone to help you.</p>
Multiprocessing practice

sh0416: <p>I totally don’t understand what is wrong…</p>
<pre><code class="lang-python">def setup(rank, world_size, seed):
    os.environ['MASTER_ADDR'] = 'localhost'
    os.environ['MASTER_PORT'] = '12355'

    # Initialize the process group
    dist.init_process_group("gloo", rank=rank, world_size=world_size)

    # Explicitly setting seed to make sure that models created in two processes
    # start from same random weights and biases.
    set_seed(seed)

def fn(rank, world_size):
    setup(rank, world_size, 0)

    print(rank)
    cleanup()

def cleanup():
    dist.destroy_process_group()

def run(fn, world_size):
    mp.spawn(fn, args=(world_size,), nprocs=world_size, join=True)

if __name__ == '__main__':
    run(fn, 4)
</code></pre>
<p>I got broken pipe…</p>
<p>Why…?</p>
<p>I use <code>torch.multiprocess</code> for <code>mp</code> and <code>torch.distributed</code> for <code>dist</code>.</p>

sh0416: <p>I found out that init process group must be executed before any processor execute destroy process group.</p>
<p>Add barrier will resolve this issue.</p>
<p>Thanks,</p>
Convert list of numbers to Tensor

Ameen_Ali: <p>Hello <br><br>
I have a problem converting a python list of numbers to pytorch Tensor :<br>
this is my code :  <br></p>
<pre><code class="lang-auto">caption_feat = [int(x)  if x &lt; 11660  else 3 for x in caption_feat]
</code></pre>
<p>printing caption_feat gives : <code>[1, 9903, 7876, 9971, 2770, 2435, 10441, 9370, 2]</code> <br><br>
I do the converting like this : <code>tmp2 = torch.Tensor(caption_feat)</code><br>
now printing tmp2 gives : <code>tensor([1.0000e+00, 9.9030e+03, 7.8760e+03, 9.9710e+03, 2.7700e+03, 2.4350e+03,         1.0441e+04, 9.3700e+03, 2.0000e+00])</code> <br><br>
However I expected to get : <code>tensor([1. , 9903, , 9971. ......])</code><br>
Any Idea?</p>

ptrblck: <p>Could you try to use <code>torch.tensor</code> with a lowercase t?</p>

Ameen_Ali: <p>Worked <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"> <br><br>
Thanks!</p>
Problems Minimising Multi-task Loss

CCL: <p>Here, I’ve declared a custom multi-task loss in Pytorch with BCEWithLogitsLoss for the (binary) mask segmentation loss, and BCELoss for the classification loss (I use fully-connected layers and then sigmoid). I weight BCEWithLogitsLoss at 0.9 and BCELoss at 0.1, sum them, and back propagate this summed loss. I’m using the Adam optimiser for minimisation. Please note that the y-axis of the plot is wrong (should be multi-task loss instead of just BCEWithLogits).<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/c/c/ccf3b788584793ff0383a431d7ad87ec52cdbcaa.jpeg" alt="plot" data-base62-sha1="tf5z28Qu3coEcgnFdwdZnHVIBNE" width="640" height="480"><br>
However, as seen from the graph, this loss cannot be minimised well. Why do you think this is? How do I solve this problem? Thanks!</p>

ptrblck: <p>Are you using the sigmoid for both layers?<br>
Note that <code>nn.BCEWithLogitsLoss</code> does not expect probabilities, but raw logits.<br>
Could you post the training code, so that we can have a look?</p>

CCL: <p>Ah yes you are completely right! I’ll fix that!</p>
How to call pytorch convolution

111227: <p>I found there is a lot of convolution function in pytorch github.  So, I have some questions.</p>
<ol>
<li>How to call their function, I open the whole file, but it still can’t run.</li>
<li>I want to use our function and trying to put it into torch conv2D. Will it work ?</li>
</ol>

Lornatang: <p>Do not mind modifying the official file, as this is likely to lead to some puzzling bugs. It is recommended to inherit the parent function and override your method.</p>
How to build torch_glow

LakeFeiLiu: <p>when i build the torch_glow use the command "cd $PATH/glow/torch_glow &amp;&amp; python3 setup.py install“<br>
there are some errors as “c10:xxx symbol can not find”.<br>
i installed the pytorch with “pip install torch==1.4.0+cpu torchvision==0.5.0+cpu -f <a href="https://download.pytorch.org/whl/torch_stable.html" rel="nofollow noopener">https://download.pytorch.org/whl/torch_stable.html</a>”<br>
my OS is “ubuntu18.04”<br>
python3.6.9</p>
<p>could anyone help me to solve it. thanks!</p>
Meaning of 'spatial_ratio' parameter in torchvision.ops.roi_align

ChopinSharp: <p>I’m using pytorch 1.3.0 with torchvision 0.4.1. I try to do RoIAlign with torchvision.ops.roi_align and I’m not  sure what’s the meaning of the  ‘spatial_ratio’ parameter. As described in the documentation, this parameter should map the input coordinates to the box coordinates. Normally, input coordinates are in the feature space and box coordinates are in the image space. So does this mean that this parameter should be greater than one?</p>

viniciusarruda: <p>Hi, did you found any explanation about it ?</p>
Understanding model parameters

Guilherme_Martins: <p>Hello, I have the following model:</p>
<pre><code class="lang-auto">class CovNet(nn.Module):

    
    def __init__(self, imageSize):
        super(CovNet, self).__init__()
        kernel = (3,3)
        self.conv1 = nn.Conv2d(in_channels = 1, out_channels = 3, kernel_size = kernel)
        wOut = self.parametersCalculator(imageSize,3,0,1,1) #Calculate size of data after conv
        wOut = self.parametersCalculator(wOut,2,0,2,1) #Calculate size of data after maxpool
        self.fc2 = nn.Linear(3 * wOut * wOut, 120)
        self.fc3 = nn.Linear(120, 60)
        self.fc4 = nn.Linear(60, 10)

    def forward(self, x):
        x = F.max_pool2d(F.relu(self.conv1(x)), 2)
        x = x.view(-1, self.num_flat_features(x))
        x = F.relu(self.fc2(x))
        x = F.relu(self.fc3(x))
        x = self.fc4(x)
        return x

    def num_flat_features(self, x):
        print(x.size())
        size = x.size()[1:]  # all dimensions except the batch dimension
        num_features = 1
        for s in size:
            num_features *= s
        return num_features
    
    def parametersCalculator(self,W,K,P,S,D):
        return int((W+2*P-D*(K-1)-1)/S + 1)
</code></pre>
<p>And I am trying to get the parameters of this model. To this end I wrote this line:</p>
<pre><code class="lang-auto">params = list(model.parameters())
</code></pre>
<p>This variable is an array with 8 length with the following shapes:<br>
params[0].shape --&gt; (3,1,3,3)<br>
params[1].shape --&gt; (3)<br>
params[2].shape --&gt; (120,507)<br>
params[3].shape --&gt; (120)<br>
params[4].shape --&gt; (60,120)<br>
params[5].shape --&gt; (60)<br>
params[6].shape --&gt; (10, 60)<br>
params[7].shape --&gt; (10)</p>
<p>The image size is 28 by 28.</p>
<p>Can someone explain to me the shapes of the params variable, i dont seem to understand what are the shapes of params 1,3,5 and 6? Are theese the biases?</p>
<p>Thank you in advance!</p>

albanD: <p>Hi,</p>
<p>Yes these are the biases for the convolution and linear layers. You can set <code>bias=False</code> when creating them and they will disappear.</p>
How do I get RandomRotation to work on 'L' images

Ayman_Al_Jabri: <p>I’m trying to RandomRotate a greyscale images using transform. But it won’t let me!<br>
can someone please help?<br>
Note: It works fine with RGB based tensors</p>
<p>Code and error message below</p>
<pre><code class="lang-auto">&gt; img = torch.randn(1,224,224)
&gt; 
&gt; tfms = Compose([ToPILImage(),RandomRotation(20)])
&gt; 
&gt; tfms(img)
&gt; .
&gt; .
&gt; .
</code></pre>
<blockquote>
<p>TypeError: function takes exactly 1 argument (3 given)</p>
</blockquote>

crowsonkb: <p>It looks like this might be a bug in <a href="https://pytorch.org/docs/stable/_modules/torchvision/transforms/functional.html#rotate" rel="nofollow noopener"><code>torchvision.transforms.functional.rotate()</code></a>, but you can work around it by defining your <code>RandomRotation</code> transform like so: <code>RandomRotation(20, fill=(0,))</code>. The problem is that by default <code>rotate()</code> passes to PIL the three-channel fill value <code>(0, 0, 0)</code> which of course breaks on one-channel images.</p>

crowsonkb: <p>It looks like this was fixed in torchvision <code>master</code> <a href="https://github.com/pytorch/vision/pull/1760" rel="nofollow noopener">seven days ago</a>.</p>

edgarriba: <p>it’s worth visiting new <code>kornia.augmentation</code> API<br>
<a href="https://kornia.readthedocs.io/en/latest/augmentation.html" class="onebox" target="_blank" rel="nofollow noopener">https://kornia.readthedocs.io/en/latest/augmentation.html</a></p>
<p>same transforms as torchvision but for torch tensors</p>
CUDA out of memory issue

leoxiaobin: <p>hi,<br>
When I try to train a model using Pytorch 1.0.0 with 8GPUs, I get the error as below:</p>
<p>‘’’<br>
RuntimeError: CUDA out of memory. Tried to allocate 30.38 MiB (GPU 0; 15.75 GiB total capacity; 7.41 GiB already allocated; 8.94 MiB free; 40.77 MiB cached)<br>
‘’’</p>
<p>And it not always happens, sometimes it can successfully run, but sometimes it throws the CUDA out of memory error.</p>
<p>It seems there is enough memory(15.75 GiB total, only 7.41 GiB used, tried to allocate 30 MiB).</p>
<p>Do anyone know how to solve it?</p>

JuanFMontesinos: <p>Are u sharing those gpus? is the runtime error consistent with nvidia-smi?</p>

leoxiaobin: <p>No share with other task. And there is also no other job on those GPUs.</p>

colesbury: <p>Can you provide more details about your training? What type of model? What type of GPUs?</p>
<p>We’re seeing similar issues internally (also involving distributed training). I’m not sure what the problem is yet, but we’re looking into it.</p>

leoxiaobin: <p>The GPU type is V100 with 16G memory. The trained model is our internal model. For single GPU training, it can work well.</p>
<p>I have also tried it in several different clusters. For another two clusters, which have the GPUs of P100 and V100 respectively, the training can work well.</p>
<p>I have also tried to use smaller batch size, even though our GPU memory is enough for the normal batch size. But for smaller batch size, it also raised the CUDA out of memory error.</p>
<p>I have tried to use smaller input image size, and it can works well.</p>

leoxiaobin: <p>hi, colesbury, have you solved the problem?</p>

y91: <p>I am facing a similar issue:</p>
<p><code>CUDA out of memory. Tried to allocate 196.50 MiB (GPU 0; 15.75 GiB total capacity; 7.09 GiB already allocated; 20.62 MiB free; 72.48 MiB cached)</code></p>
<p>It looks like there is enough memory left, yet I get an OOM error. Do we have any update on this ?</p>
<p>Thanks!</p>

santi-pdp: <p>Hi I am facing the same issue:<br>
<code>RuntimeError: CUDA out of memory. Tried to allocate 1.86 GiB (GPU 0; 15.75 GiB total capacity; 6.25 GiB already allocated; 8.44 GiB free; 17.78 MiB cached)</code></p>
<p>Any advances on this? Thanks guys!</p>

General_Intelligence: <p>run<br>
torch.cuda.empty_cache() to clear the GPU memory then restart the kernel and or close and re-open the kernel</p>
How to solve the RuntimeError when using torch.utils.tensorboard to add a graph

zhenqin: <aside class="onebox stackexchange">
  <header class="source">
      <a href="https://stackoverflow.com/questions/60082604/how-to-solve-the-runtimeerror-when-using-torch-utils-tensorboard-to-add-a-graph" target="_blank" rel="nofollow noopener">stackoverflow.com</a>
  </header>
  <article class="onebox-body">
      <a href="https://stackoverflow.com/users/11155039/zqin" target="_blank" rel="nofollow noopener">
    <img alt="zqin" src="https://www.gravatar.com/avatar/969bbc2973d111061626641ac58759ed?s=128&amp;d=identicon&amp;r=PG&amp;f=1" class="thumbnail onebox-avatar" width="60" height="60">
  </a>
<h4>
  <a href="https://stackoverflow.com/questions/60082604/how-to-solve-the-runtimeerror-when-using-torch-utils-tensorboard-to-add-a-graph" target="_blank" rel="nofollow noopener">How to solve the RuntimeError when using torch.utils.tensorboard to add a graph</a>
</h4>

<div class="tags">
  <strong>pytorch, tensorboard</strong>
</div>

<div class="date">
  asked by
  
  <a href="https://stackoverflow.com/users/11155039/zqin" target="_blank" rel="nofollow noopener">
    zqin
  </a>
  on <a href="https://stackoverflow.com/questions/60082604/how-to-solve-the-runtimeerror-when-using-torch-utils-tensorboard-to-add-a-graph" target="_blank" rel="nofollow noopener">07:10PM - 05 Feb 20 UTC</a>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
Nn.transformer model show zero loss on train and doesnt work in interferencet

hadaev8: <p>Here is code<br>
<a href="https://colab.research.google.com/drive/1g4ZFCGegOmD-xXL-Ggu7K5LVoJeXYJ75" class="onebox" target="_blank" rel="nofollow noopener">https://colab.research.google.com/drive/1g4ZFCGegOmD-xXL-Ggu7K5LVoJeXYJ75</a></p>
<p>Im not sure what im doing wrong.<br>
And still cant find tutorial or example with native pytorch transformer.</p>

zhangguanheng66: <p><a href="https://pytorch.org/tutorials/beginner/transformer_tutorial.html" class="onebox" target="_blank" rel="nofollow noopener">https://pytorch.org/tutorials/beginner/transformer_tutorial.html</a></p>

hadaev8: <p>This example have no decoder</p>
Manually setting out_features and in_features in fully connected layers

sigma_x: <p>I learnt of this functionality. For example, we have a VGG16 model:</p>
<pre><code class="lang-auto">import torchvision.models as models
model=models.vgg16()
model._modules['classifier'][6] = 1
</code></pre>
<pre><code class="lang-auto">Sequential(
  (0): Linear(in_features=25088, out_features=4096, bias=True)
  (1): ReLU(inplace)
  (2): Dropout(p=0.5)
  (3): Linear(in_features=4096, out_features=4096, bias=True)
  (4): ReLU(inplace)
  (5): Dropout(p=0.5)
  (6): Linear(in_features=4096, out_features=1, bias=True)
)
</code></pre>
<p>It turns out you can manipulate the number of input and output features, e.g.</p>
<p>vgg._modules[‘classifier’][6].in_features=5<br>
vgg._modules[‘classifier’][6].out_features=1</p>
<pre><code class="lang-auto">  (classifier): Sequential(
    (0): Linear(in_features=25088, out_features=4096, bias=True)
    (1): ReLU(inplace)
    (2): Dropout(p=0.5)
    (3): Linear(in_features=4096, out_features=4096, bias=True)
    (4): ReLU(inplace)
    (5): Dropout(p=0.5)
    (6): Linear(in_features=5, out_features=1, bias=True)
  )
</code></pre>
<p>I don’t quite understand how this thing works. If I reduce 1000 neurons to 1, which one is that? And what happens to all the weights? I realize with in_features=5 this wouldn’t work, but with out_features=1 this would. But how?</p>

ptrblck: <p>Your code shouldn’t work, as you are trying to assign an <code>int</code> as a child module.<br>
Anyway, changing the <code>out_features</code> won’t have any effect after the layer was initialized:</p>
<pre><code class="lang-python">model = models.resnet50().eval()
x = torch.randn(1, 3, 224, 224)

out1 = model(x)

model.fc.out_features = 1

out2 = model(x)

print(out1.shape, out2.shape)
&gt; torch.Size([1, 1000]) torch.Size([1, 1000])
print((out1 == out2).all())
&gt; tensor(True)
</code></pre>

sigma_x: <p>It’s a type, I meant of course</p>
<p><code>model._modules['classifier'][6].out_features = 1</code></p>
<p>So can I change both the number of <code>out_features</code> or <code>in_features</code> and the resize the weights matrix accordingly?</p>

ptrblck: <p>You could reassign a new matix with your desired shape.<br>
Changing the <code>in_*</code> and <code>out_features</code> will not change the underlying weight parameter.</p>
About the order of detach()

kekmodel: <p>Let’s assume the following situation.</p>
<pre><code class="lang-auto">logits = model(inputs)
logits_x = logits[:batch_size]
logits_u_w, logits_u_s = logits[batch_size:].chunk(2)
del logits

e.g.1
pseudo_label = torch.softmax(logits_u_w, dim=-1).detach()
e.g.2
pseudo_label = torch.softmax(logits_u_w.detach(), dim=-1)
</code></pre>
<p>Q. I want the operation between “logits_u_w” and “softmax” not to be included in autograd. What is correct? What is the reason?</p>

tux: <p>I would say that your second equation (e.g.2) is the correct one.</p>
<p>According to the doc (<a href="https://pytorch.org/tutorials/beginner/blitz/autograd_tutorial.html#sphx-glr-beginner-blitz-autograd-tutorial-py" class="inline-onebox" rel="noopener nofollow ugc">A Gentle Introduction to torch.autograd — PyTorch Tutorials 2.1.1+cu121 documentation</a>):</p>
<blockquote>
<p>To stop a tensor from tracking history, you can call  <code>.detach()</code>  to detach it from the computation history, and to prevent future computation from being tracked.</p>
</blockquote>
<p>Hence, your tensor in e.g.1 would be tracked until the <code>softmax</code> while e.g.2 would stop tracking the gradient at <code>logits_u_w</code></p>

albanD: <p>The two will actually give you the same result if you write them this way.</p>
<p>I would recommend the 2nd one like <a class="mention" href="/u/tux">@tux</a> because in the first one, you first create the graph for the softmax then discard it when you detach. In the second one, you just never create that part of the graph.<br>
But the difference is going to be very small.</p>

tux: <p>Thank you for your answer. I find the doc a bit confusing then. From the doc:</p>
<blockquote>
<p>To stop a tensor from tracking history, you can call  <code>.detach()</code>  to detach it from the computation history, and to prevent future computation from being tracked.</p>
</blockquote>
<p><code>.detach()</code> just detach the gradient from the graph and all the futur operations will not be tracked.</p>
<p>But your answer seems to say that it does stop tracking, as the doc says, AND  delete the last operation from the graph (here the <code>softmax</code> operation). Could you please clarify ?</p>

albanD: <p>The only thing <code>.detach()</code> does is return a new Tensor without gradient history.</p>
<p>But if in one line you create the result then detach it. The result that was tracking history goes out of scope. And so this part of the history is not referenced by anything and is deleted.</p>
Getting build_android/CMakeFiles/CMakeTmp/src.cxx:1:10: fatal error: 'glog/stl_logging.h' file not found

jamilireddy: <p>I am trying to build the source code on ubuntu 18;<br>
Getting the following error:<br>
build_android/CMakeFiles/CMakeTmp/src.cxx:1:10: fatal error: ‘glog/stl_logging.h’ file not found</p>
<p>I am using the following command to build the same.<br>
./scripts/build_pytorch_android.sh</p>
<p>Could you please suggest me to get out of this.</p>
<p>Thanks &amp; Regards,<br>
Jamili.</p>

Jeff_Smith: <p>This question looks like it should be routed to the Mobile section of the forum. Cc <a class="mention" href="/u/david_reiss">@David_Reiss</a></p>

David_Reiss: <p>I think that’s a normal message that happens when the build system detects that glog is not available.  Can you create a paste of the full build output?</p>
How to customize build torchscript model to be used in another code

danmalowany-allegro: <p>I want to compile my model to be executed in the Python script running on our customers computers.<br>
Currently my resnet model size is ~100MB but it needs torch, which requires 1.5GB of space.<br>
Currently i am doing this series of commands:</p>
<pre><code class="lang-auto">model = torchvision.models.resnet50(pretrained=True)
model.eval()
example = torch.ones(1, 3, 224, 224)
traced_model = torch.jit.trace(model, example)
ops = torch.jit.export_opnames(model)
traced_model.save('traced_model.pt')
with open('model_ops.yaml', 'w') as output:
    yaml.dump(ops, output)
</code></pre>
<p>The question is how to continue from here in order to build a model i can use in another python/c script without the need to load the entire torch or libtorch packages, but only what is needed based on the model operations.</p>

richard: <p>We currently don’t support this. This sounds like a reasonable feature request that is important for deployment. Please open a feature request over at <a href="https://github.com/pytorch/pytorch">https://github.com/pytorch/pytorch</a> (or voice your support on a current issue if it exists) !</p>

Jeff_Smith: <p>This sounds like it overlaps with some of the concerns of the Mobile userbase (e.g. building with only the used ops to reduce total size). Cc <a class="mention" href="/u/david_reiss">@David_Reiss</a></p>

danmalowany-allegro: <p><a class="mention" href="/u/jeff_smith">@Jeff_Smith</a> - I agree. This is relevant for mobile, as well as for other end devices. However, considering the resources in modern smartphones, the need in other end devices (not mobile) is much more prominent. This is a major issue for deployment.</p>

danmalowany-allegro: <p><a class="mention" href="/u/richard">@richard</a> - I saw some discussions about it for mobile deployment (<a href="https://pytorch.org/mobile/ios/#custom-build" rel="nofollow noopener">https://pytorch.org/mobile/ios/#custom-build</a>). However, as i answered to <a class="mention" href="/u/jeff_smith">@Jeff_Smith</a> , the need for such a solution for deployment on any end device is even greater than for modern smartphones.</p>

David_Reiss: <p>Moved discussion to <a href="https://github.com/pytorch/pytorch/issues/32690#">https://github.com/pytorch/pytorch/issues/32690#</a></p>
Pytorch train loss osscilate after some epochs on cifar10 data

Himanshu_Sundriyal1: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/b/8bf31357bd66fc07b57eb65b9822dac788647d97.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/8bf31357bd66fc07b57eb65b9822dac788647d97" title="IMG-20200205-WA0051"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/b/8bf31357bd66fc07b57eb65b9822dac788647d97_2_281x500.jpeg" alt="IMG-20200205-WA0051" data-base62-sha1="jY39DfEf1KJgmkBXP7cyJyj3am3" width="281" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/b/8bf31357bd66fc07b57eb65b9822dac788647d97_2_281x500.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/8/b/8bf31357bd66fc07b57eb65b9822dac788647d97_2_421x750.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/8/b/8bf31357bd66fc07b57eb65b9822dac788647d97_2_562x1000.jpeg 2x" data-dominant-color="888782"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">IMG-20200205-WA0051</span><span class="informations">2592×4608 795 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I have tried reducing learning rate and also changing optimisers.<br>
Further i have used lr reduce on platue , but no help with it.<br>
I would be greatful for help.</p>

martinr: <p>I suggest to let it run some more while the trend is still going down and see what happens. Probably the model is not able to generalise well beyond that point. Perhaps you need larger model? You might also look how well balanced your classes are, perhaps add some more training data if possible (or augmentation), increase batch size etc.</p>
Tensor.apply_ funtion

xiao1228: <p>Is there another way to Applies the function <code>callable</code> to each element in the tensor instead of using apply_, because apply_ is quite slow. and only works with CPU tensors…</p>

nairbv: <p><code>torch.apply_</code> is slow, and we don’t have a great efficient way to apply an arbitrary function to a tensor, but a common workaround for simple operations can be to use a mask.</p>
<p>E.g. say you wanted to do something like <code>tensor.apply_( lambda x: x + 2 if x &gt; 5 else x )</code>, instead you could write something like <code>result = (tensor &gt; 5) * 2 + tensor</code>.</p>
<p>Another function that can sometimes help is <code>torch.where</code>.</p>

xiao1228: <p>ok. i will have a try. Thank you!</p>
How to close BatchNorm when using torchvision models

coincheung: <p>I am trying to fine tune the pretrained resnet model provided by torchvision, and I need to remove serveral batchnorm layers before implementing the fine-tune. How could I close these BN layers?</p>

ptrblck: <p>This code should deactivate the <code>nn.BatchNorm</code> layers:</p>
<pre><code class="lang-python">def deactivate_batchnorm(m):
    if isinstance(m, nn.BatchNorm2d):
        m.reset_parameters()
        m.eval()
        with torch.no_grad():
            m.weight.fill_(1.0)
            m.bias.zero_()


model = nn.Sequential(
    nn.Conv2d(3, 6, 3, 1, 1),
    nn.BatchNorm2d(6)
)

x = torch.randn(10, 3, 24, 24)
output1 = model[0](x)
output2 = model(x)
print(torch.allclose(output1, output2))
&gt; False

model.apply(deactivate_batchnorm)
output1 = model[0](x)
output2 = model(x)
torch.allclose(output1, output2)
&gt; True
</code></pre>
<p>Keep in mind, that you would need to call this function again after switching the whole model between <code>model.train()</code> and <code>model.eval()</code>.</p>
<p>Another approach would be to completely remove the batch norm layers and recreate the model, which might be quite complicated based on your model and <code>forward</code>.</p>

coincheung: <p>Thanks a lot. But could setting \beta = 0 and \gamma = 1 disable the effect of batchnorm? The input activations will still be normalized with its own mean and variance among the batch.</p>

ptrblck: <p>I’ve also reset the running estimates and set the layer to <code>eval</code>, so that these running estimates will be used and no batch statistics will be calculated.<br>
It’s not pretty as you might get errors if you forget to switch this layer to <code>eval</code>, so that’s why I also suggested the other approach of removing the layers completely.</p>

coincheung: <p>Ah I see. Thank a lot.</p>
Output of Unet is a white image

satinder147: <p>I have trained a UNet.<a href="https://github.com/milesial/Pytorch-UNet/blob/master/unet/unet_model.py" rel="nofollow noopener">Unet architecture used</a> The loss also decreased to 1e-4. But when I tried it on the train data, the output is a “WHITE” blank image. I have denormalized the output as well, but no success.</p>

ptrblck: <p>How many output classes are you using and how are you denormalizing the output?</p>

satinder147: <p>2 classes, I normalised the images to 0.5 mean and 0.5 standard deviation. For denormalizing I did this:<br>
img*0.5 +0.5</p>

ptrblck: <p>The output of your model will most likely not have the same normalization as your input.<br>
Correct me, if I misunderstood the use case, but I assume your model outputs a 2-channel activation containing the class logits for each pixel.</p>
<p>Did you “denormalize” this output using the mentioned formula?</p>
<p>Could you try to get the predictions via <code>torch.argmax(output, 1)</code> and try to plot this instead?</p>

satinder147: <p>Sir, the architecture I used outputs a single-channel image. The architecture used is:<br>
My use case is segmenting road from a image.</p>
<pre><code class="lang-auto">class Unet(nn.Module):
    '''U-Net Architecture'''
    def __init__(self,inp,out):
        super(Unet,self).__init__()
        self.c1=self.contracting_block(inp,16)
        self.c2=self.contracting_block(16,32)
        self.c3=self.contracting_block(32,64)
        self.maxpool=nn.MaxPool2d(2)
        self.upsample=nn.Upsample(scale_factor=2,mode="bilinear",align_corners=True)
        self.c4=self.contracting_block(32+64,32)
        self.c5=self.contracting_block(16+32,16)
        self.c6=nn.Conv2d(16,out,1)

    def contracting_block(self,inp,out,k=3):
        block =nn.Sequential(
            nn.Conv2d(inp,out,k,padding=1),
            nn.ReLU(inplace=True),
            nn.BatchNorm2d(out),
            nn.Conv2d(out,out,k,padding=1),
            nn.ReLU(inplace=True),
            nn.BatchNorm2d(out)
        )
        return block


    def forward(self,x):
        conv1=self.c1(x) 
        x=self.maxpool(conv1)
        conv2=self.c2(x)
        x=self.maxpool(conv2)
        conv3=self.c3(x)
        x=self.upsample(conv3)
        x=torch.cat([conv2,x],axis=1)
        x=self.c4(x)
        x=self.upsample(x)
        x=torch.cat([conv1,x],axis=1)
        x=self.c5(x)
        x=self.c6(x)
        return x

</code></pre>
<p><em><strong>The inference loop is as follows:</strong></em></p>
<pre><code class="lang-auto">   img=cv2.resize(img,(256,256))
    img=trans(img).unsqueeze(0)
    img=img.type(torch.FloatTensor)
    img=img.to(device)
    mask=net(img)

    mask=mask[0].cpu().detach().numpy()
    mask=np.transpose(mask,(1,2,0))
    mask=mask*0.5+0.5
    cv2.imshow("mask",mask)
    img=img.squeeze(0).cpu().detach().numpy()
    img=np.transpose(img,(1,2,0))
    img=img*0.5+0.5
    cv2.imshow("img",img)
    cv2.waitKey(1)

</code></pre>
<p><a class="mention" href="/u/ptrblck">@ptrblck</a> Thank you very much for helping.</p>
What is torch.distributions.Normal in C++?

cruzas: <p>Hi all,</p>
<p>I am having difficulty finding the C++ equivalent of the following statement. Does anyone know?</p>
<pre><code class="lang-auto">self.distribution = torch.distributions.Normal
</code></pre>
<p>For instance, for a linear module, we have torch::nn::Linear. What is the one for a normal distribution?</p>
<p>Many thanks.</p>
Excluding positive samples in Negative Sampling

Minseok: <p>Hi,</p>
<p>I was reading several <strong><a href="https://github.com/theeluwin/pytorch-sgns/blob/master/model.py" rel="nofollow noopener">Skipgram Negative Sampling codes such as this one in Github</a></strong> and became curious how to exclude positive samples among sampled negative ones as the code seems not properly doing it. As I understood, the original <a href="https://arxiv.org/abs/1310.4546" rel="nofollow noopener">paper</a>  says that the negative sampling is sampling a few unrelated words and minimizing their likelihood. Don’t we have to strictly exclude positive samples being picked from negative sampling? I guess we have to further check whether samples from negative sampling includes one from positive samples.</p>
<p>Please correct me if I’m wrong.</p>
[PyTorch geometric] add nodes data to graph generated from networkx graph

Kris_Kulivnyk: <p>Hello guys! I’m a newby in PyTorch geometric and there is one question I can’t solve for the moment.</p>
<p>I’ve generated a torch geometric Data module from existing networkx graph</p>
<pre><code class="lang-auto">t_gen = tg.utils.from_networkx(G_numerical)
</code></pre>
<p>Now I want to add node features (node data ) to this data module</p>
<pre><code class="lang-auto">x = torch.tensor([[2,1], [5,6], [3,7], [12,0]], dtype=torch.float)
y = torch.tensor([0, 1, 0, 1], dtype=torch.float)
</code></pre>
<p>How can I do it, please?<br>
Will appreciate any reply!</p>
KL loss for small numbers

barakb: <p>Hi I’m trying to do kl distance (optimize kl loss) between two tensors, the problem is that those tensors hold small numbers, around 1e-6. why it’s a problem because I need to normalize the teacher tensor using softmax this gives me a uniform distribution, which is not the case.<br>
How can I overcome this issue, my only thought is to multiply all the values using a high constant .<br>
My code is:</p>
<pre><code class="lang-auto">def MyKlLoss(output,target):
    shapeto = [output.shape[0]*output.shape[1]*output.shape[2],output.shape[3]*output.shape[4]]
    o = output.view(shapeto) 
    t = target.view(shapeto) ### t holds very low values (1e-5, 1e-6,..) 
    o = F.log_softmax(output.view(shapeto),dim=-1)
    t = F.softmax(target.view(shapeto),dim=-1)## t is becoming a uniform dist, all 0.1111
 
    return F.kl_div(o,t)
</code></pre>
<p>I’ll be happy is someone has any idea how to make kl loss meaningful in that case.<br>
Thanks!</p>
Tensors: replacement if condition

mlearner: <p>Hello,<br>
I’m trying to replace the value of a tensor <code>x</code> when the value in <code>x</code> and the corresponding (i.e. at the same index) value in <code>y</code> are both zeros.<br>
Being <code>r</code> the value I want to use for the substitution, I want to achieve something logically equivalent to: <code>if x_i==0 and y_i == 0 then x_i = r</code>.</p>
<p>I tried with something like <code>x[x== 0 and y == 0] = r</code> but it is not working.<br>
What is the cleanest way to do this?<br>
Thank you</p>

G.M: <p>Your solution is close. Instead of <code>and</code>, you should use <code>&amp;</code>.</p>
<pre><code class="lang-auto">a = x==0
b = y==0
x[a &amp; b] = r
</code></pre>
8 bit quantization - modulo 256 inside convolutions?

dassima: <p>Hi, I am trying to understand how the PyTorch multiplications on 8 bits work when a model is quantized. Inside each convolution, do you do modulo 256 operations, which means keeping the LSB?</p>
<p>I tried to do simple ByteTensor operations and I saw that the LSBs are kept always. But I need to know if it is the same inside a quantized model.</p>
<p>Thanks a lot.<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/b/bb1f9949fb550514f0ebd0c462a125e5147a71b5.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/bb1f9949fb550514f0ebd0c462a125e5147a71b5" title="IMG_20200131_152514"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/b/bb1f9949fb550514f0ebd0c462a125e5147a71b5_2_375x500.jpeg" alt="IMG_20200131_152514" data-base62-sha1="qHmZcfLtNmecEv58RenyeY9V9Z3" width="375" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/b/bb1f9949fb550514f0ebd0c462a125e5147a71b5_2_375x500.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/b/b/bb1f9949fb550514f0ebd0c462a125e5147a71b5_2_562x750.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/b/b/bb1f9949fb550514f0ebd0c462a125e5147a71b5_2_750x1000.jpeg 2x" data-dominant-color="918B86"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">IMG_20200131_152514</span><span class="informations">2976×3968 237 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>