akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
EAST: An Efficient and Accurate Scene Text Detector in Pytorch

amitkayal: <p>Hi Experts,</p>
<p>I was looking for pytorch implementation for <a href="https://arxiv.org/pdf/1704.03155.pdf" rel="noopener nofollow ugc">https://arxiv.org/pdf/1704.03155.pdf</a> and could not locate any. It would be great if anyone can share some guidance?</p>
<p>Thanks<br>
Amit</p>
Issue loading pytorch model

ashcher51: <p>I saved a pytorch model in colab and downloaded it so that I wouldn’t lose the weights. Now, when I went back into my notebook, I uploaded the .pt file and called <code>torch.load()</code>. When I did that, I got this error:</p>
<pre><code class="lang-auto">RuntimeError: [enforce fail at inline_container.cc:145] . PytorchStreamReader failed reading zip archive: failed finding central directory
</code></pre>
<p>I would really appreciate any help with this. Thank you.</p>

ashcher51: <p>I would really appreciate it if there was a way that I could do it without rerunning my model code - it takes a long time to train</p>

AlperEkmekci: <p>You have probably a corrupted file or you’ve saved your model as a <code>TorchScript</code> with <code>torch.jit.save</code> ?</p>
Model & Hyperparameter Optimization

AlperEkmekci: <p>After training a model consists ~ 50k dataset of segmentation images, I want to be sure my parameters and models are optimize before deploy my model.</p>
<p>My interest point is that what target should I try to improve after getting above %90 IoU acc and how can I be sure about my hyperparameters and my model is working as best as it can be ?</p>
Builtin parameter extraction using c++ API

JustinZal: <p>I am using a C++ program to analyze a CNN (YoloV5x). My goal is to extract built-in function parameters, like stride length and kernel size from the model file. I am currently using the <code>torch::jit::script::Module</code> data type and <code>torch::jit::load</code> function to load the torchscript model.</p>
<p>My problem is that I see no way of accessing attributes, even though software like “Netron” can obviously extract that. The only remotely related feature is <code>model.named_parameters()</code>, which only gives weights and biases of the model, which is not enough for my analysis. Is there any programmatic way to extract the needed features of the built-in functions using the C++ API?</p>
<p>Example:<br>
When I load the model (model.pt) in python and print it, one of the function ouptuts is the following:</p>
<pre><code class="lang-auto">(1): Conv(
      (conv): Conv2d(80, 160, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
      (act): Hardswish()
)
</code></pre>
<p>I would like to get the <code>kernel_size</code>, <code>stride</code>, and <code>padding</code> parameter in C++ API.</p>
<p>P.S.<br>
In the Python program I load model.pt file and in the C++ file I load the model.torchscript.pt file.</p>
How to define a recursive custom function using autograd.Function in a elegant way?

shixinxing: <p>I am a new user to PyTorch, after reading the PyTorch Documentation <a href="https://pytorch.org/docs/stable/notes/extending.html" rel="noopener nofollow ugc">Extending PyTorch</a>, I am thinking about writing a recursive custom function. As in the example code, when writing staticmethod part ‘forward’:</p>
<p><strong>It seems that directly calling ‘forward’ function in ‘return’ line gives an error.</strong></p>
<p>So I am wondering about <strong>How to define a recursive custom function using autograd.Function in a elegant way?</strong></p>
<p>Thanks a lot for your answer and help!</p>
Hog in faster rcnn

T33K3Y: <p>Is there a way to pass hog features to faster rcnn ?</p>
<aside class="onebox allowlistedgeneric">
  <header class="source">
      <a href="https://pytorch.org/docs/stable/_modules/torchvision/models/detection/faster_rcnn.html" target="_blank" rel="noopener nofollow ugc">pytorch.org</a>
  </header>
  <article class="onebox-body">
    <img src="" class="thumbnail" width="" height="">

<h3><a href="https://pytorch.org/docs/stable/_modules/torchvision/models/detection/faster_rcnn.html" target="_blank" rel="noopener nofollow ugc">torchvision.models.detection.faster_rcnn — PyTorch 1.7.0 documentation</a></h3>



  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p><a href="https://en.wikipedia.org/wiki/Histogram_of_oriented_gradientshttps://en.wikipedia.org/wiki/Histogram_of_oriented_gradients" class="onebox" target="_blank" rel="noopener nofollow ugc">https://en.wikipedia.org/wiki/Histogram_of_oriented_gradientshttps://en.wikipedia.org/wiki/Histogram_of_oriented_gradients</a></p>
Embryo cell division time lapse dataset

Sundar_Krishna: <p>Hi ,<br>
I dont know if this is not related to the forum. I am not an expert in Pytorch or DL. I just wanted to do a prediction whether an embryo is healthy. I need a time lapse image dataset of embryos. Where can I find the dataset? How do I arrive at a correct network model which can be used? Right now I have an idea of using Resnet (how do I arrive at a correct network model which suits my problem statement in general)! Suggestions are welcome<br>
Thank you!</p>

ptrblck: <p>I don’t know, if there is an open dataset for this particular task, but would recommend to read the latest papers in this area and see, if the authors have mentioned particular datasets they have been using. If not, you might want to contact some of them.</p>
<p>This also applies for the second question: if possible, I would try to find a good baseline and start with this model in order to try to reproduce the results. If there is no baseline, you could try to use “standard” model such as the mentioned ResNet. Generally, I would also recommend to check out some courses, which explain how to build a custom model, such as <a href="https://www.fast.ai/">FastAI</a>.</p>

Sundar_Krishna: <p>Thank you so much sir, Currently found this <a href="https://Embryo_dataset" rel="noopener nofollow ugc">https://www.kaggle.com/ditzsins/grades-embryo</a> , I think it would be out of scope for me to ask if the above dataset is a time lapse image (Apologize if it is!). Yes sir FastAI is a great idea as it makes training process and other stuff more easier . Thank you so much again and sure to update you sir!</p>
How to sort a 3d tensor by coordinates in last dimension

spadel: <p>I have a tensor with shape <code>[bn, k, 2]</code>. The last dimension are coordinates and I want each batch to be sorted independently depending on the y coordinate (<code>[:, :, 0]</code>). My approach looks something like this:</p>
<pre><code class="lang-auto">import torch
a = torch.randn(2, 5, 2)
indices = a[:, :, 0].sort()[1]
a_sorted = a[:, indices]

print(a)
print(a_sorted)
</code></pre>
<p>So far so good, but I now it sorts both batches according to both index lists, so I get 4 batches in total:</p>
<pre><code class="lang-auto">a
tensor([[[ 0.5160,  0.3257],
         [-1.2410, -0.8361],
         [ 1.3826, -1.1308],
         [ 0.0338,  0.1665],
         [-0.9375, -0.3081]],

        [[ 0.4140, -1.0962],
         [ 0.9847, -0.7231],
         [-0.0110,  0.6437],
         [-0.4914,  0.2473],
         [-0.0938, -0.0722]]])

a_sorted
tensor([[[[-1.2410, -0.8361],
          [-0.9375, -0.3081],
          [ 0.0338,  0.1665],
          [ 0.5160,  0.3257],
          [ 1.3826, -1.1308]],

         [[ 0.0338,  0.1665],
          [-0.9375, -0.3081],
          [ 1.3826, -1.1308],
          [ 0.5160,  0.3257],
          [-1.2410, -0.8361]]],


        [[[ 0.9847, -0.7231],
          [-0.0938, -0.0722],
          [-0.4914,  0.2473],
          [ 0.4140, -1.0962],
          [-0.0110,  0.6437]],

         [[-0.4914,  0.2473],
          [-0.0938, -0.0722],
          [-0.0110,  0.6437],
          [ 0.4140, -1.0962],
          [ 0.9847, -0.7231]]]])
</code></pre>
<p>As you can see, I want only the 1st and the 4th batch to be returned. How do I do that?</p>

ptrblck: <p>This should work:</p>
<pre><code class="lang-python">a = torch.randn(2, 5, 2)
indices = a[:, :, 0].sort()[1]
a_sorted = a[torch.arange(a.size(0)).unsqueeze(1), indices]

print(a)
print(a_sorted)
</code></pre>
ValueError: too many dimensions 'str'

Rick_boekhoudt: <p>I keep getting this error consistently when I run my code. Have not found anything yet to solve this issue.</p>
<p>Please click on link to see my notebook and the issue.</p>
<p>Any help is great.</p>
<p>Link:</p><aside class="onebox allowlistedgeneric">
  <header class="source">
      <img src="https://colab.research.google.com/img/favicon.ico?vrz=colab-20201216-085601-RC00_347824393" class="site-icon" width="16" height="16">
      <a href="https://colab.research.google.com/drive/1ikol2D8mmiIPKhNHbcFlTfVpuU_Gf9BZ?usp=sharing" target="_blank" rel="noopener nofollow ugc">colab.research.google.com</a>
  </header>
  <article class="onebox-body">
    <img src="https://colab.research.google.com/img/colab_favicon_256px.png" class="thumbnail onebox-avatar" width="60" height="60">

<h3><a href="https://colab.research.google.com/drive/1ikol2D8mmiIPKhNHbcFlTfVpuU_Gf9BZ?usp=sharing" target="_blank" rel="noopener nofollow ugc">Google Colaboratory</a></h3>



  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


ptrblck: <p>Could you check the <code>type</code> of these inputs to <code>torch.tensor</code>?</p>
<pre><code class="lang-python">torch.tensor(touch_seq),torch.tensor(labels),torch.tensor(lengths),torch.FloatTensor(profile_data)
</code></pre>
<p>Based on the error message and stack trace at least one of these tensor creations fails with this rather unhelpful error message.</p>

tom: <pre><code class="lang-python">torch.FloatTensor(200*["*"])
</code></pre>
Implementing Weight Constraints

Ebrahim_Feghhi: <p>Hello, I am working on constraining my weights in an RNN model. My goal is that a portion of my weights will be strictly positive, and a portion strictly negative. I looked around on the forums, and it seems that the recommended approach to doing so is the following. I execute this code after my optimizer step.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/f/cfbfbe8e0de96f8ece7e1657637613395725423c.png" data-download-href="https://discuss.pytorch.org/uploads/default/cfbfbe8e0de96f8ece7e1657637613395725423c" title="Screen Shot 2020-12-20 at 9.25.54 AM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/f/cfbfbe8e0de96f8ece7e1657637613395725423c_2_690x110.png" alt="Screen Shot 2020-12-20 at 9.25.54 AM" data-base62-sha1="tDPE6bb5z4UuRN8a47h0L3Qs7yA" width="690" height="110" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/f/cfbfbe8e0de96f8ece7e1657637613395725423c_2_690x110.png, https://discuss.pytorch.org/uploads/default/optimized/3X/c/f/cfbfbe8e0de96f8ece7e1657637613395725423c_2_1035x165.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/c/f/cfbfbe8e0de96f8ece7e1657637613395725423c.png 2x" data-dominant-color="EDEEEE"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot 2020-12-20 at 9.25.54 AM</span><span class="informations">1086×174 22.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>However, doing so leads my loss values to turn to NaN. The interesting thing is if I modify my weights using another method (e.g. the tanh function or multiplying by negative 1), then the model trains without any issues. I’m not sure how I should move forward, and any help would be appreciated.</p>

ptrblck: <p>Could you check, if the output of the <code>rnn_layer</code> yields the first NaNs or if any downstream layers create these invalid outputs due to e.g. a zero output of the RNN?</p>
Saving and loading pruned models

junukwon7: <p>I’m training a pruned yolov3 using eriklindernoren’s implement.<br>
I used <code>prune.random_unstructured</code> to prune the layers and saved them with the code <code>torch.save(model.state_dict(), opt.save_path_p)</code>.<br>
However, when I try to load them with <code>model.load_state_dict(torch.load(opt.pretrained_weights))</code>, it returns an error saying that</p>
<p><code>Missing key(s) in state_dict: "module_list.#.conv_#.weight"</code><br>
<code>Unexpected key(s) in state_dict: "module_list.#.conv_#.weight_orig", "module_list.#.conv_#.weight_mask",</code>. (# is the number)</p>
<p>I searched about it but couldn’t get how to solve it. Someone said that I can change it using weight=weight_orig * weight_mask but I can’t understand why and how to do it.</p>
<p>If anyone can help, I would really appreciate it.</p>
Equivalent to numpy's nan_to_num?

Jameson_Quinn: <p>Numpy has a nan_to_num function, which replaces nan, inf, and -inf with three arbitrary constants (usually zero, something large, and some large negative number). Is there an equivalent in pytorch? If not, how would I construct my own, without assignment operators that would break the gradient; would I have to assemble something element-by-element and use torch.cat, or is there some other solution? I am fine with the fact that all replaced elements would have a gradient of 0.</p>

Jameson_Quinn: <p>I’m currently using the following code, but it’s woefully inefficient:</p>
<pre><code class="lang-auto">
def nan_to_num(t,mynan=0.):
    if torch.all(torch.isfinite(t)):
        return t
    if len(t.size()) == 0:
        return torch.tensor(mynan)
    return torch.cat([nan_to_num(l).unsqueeze(0) for l in t],0)
</code></pre>

blade: <p>Did you find an answer for this?</p>

mruberry: <p>If you’re using PyTorch’s nightly builds (or wait until PyTorch 1.8) then we’ve added nan_to_num. See its documentation here:</p>
<aside class="onebox allowlistedgeneric">
  <header class="source">
      <a href="https://pytorch.org/docs/master/generated/torch.nan_to_num.html?highlight=nan_to_num#torch.nan_to_num" target="_blank" rel="noopener nofollow ugc">pytorch.org</a>
  </header>
  <article class="onebox-body">
    <img src="" class="thumbnail" width="" height="">

<h3><a href="https://pytorch.org/docs/master/generated/torch.nan_to_num.html?highlight=nan_to_num#torch.nan_to_num" target="_blank" rel="noopener nofollow ugc">torch.nan_to_num — PyTorch master documentation</a></h3>



  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
YOYO5, Coco dataset, Windows support?

FaultyBagnose: <p>I try to follow this tutorial using PyCharm on Windows10: <a href="https://curiousily.com/posts/object-detection-on-custom-dataset-with-yolo-v5-using-pytorch-and-python/" rel="noopener nofollow ugc">https://curiousily.com/posts/object-detection-on-custom-dataset-with-yolo-v5-using-pytorch-and-python/</a><br>
A prerequisite is to run this pip install<br>
pip install git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI<br>
This gives an error which appears to be related to Visual Studio, and it seems to be a generic Windows compatibility problem:<br>
<a href="https://github.com/cocodataset/cocoapi/issues/9" class="inline-onebox" rel="noopener nofollow ugc">Windows support for python API · Issue #9 · cocodataset/cocoapi · GitHub</a><br>
(Just noted that this was supposed to be corrected according to the latest post in that thread, but it still crashes for me.)</p>
<p>Does this mean that YOLO5 cannot be reproduced in a Windows environment?</p>
<p>ps: Here is the error message.<br>
cl : Command line error D8021 : invalid numeric argument ‘/Wno-cpp’<br>
error: command ‘C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe’ failed with exit code 2</p>

FaultyBagnose: <p>Sorry for the noise I’ve created here. Seems I only needed to do ‘pip install wheel’.</p>
ImportError: cannot import name 'AdultDataset' from 'dataset'

Samuel_Adu: <p>0</p>
<p>Please I am working on AdultDataset for a classification task</p>
<p>I found out: <code>from dataset import AdultDataset</code></p>
<p>is giving the error below:</p>
<p>ImportError: cannot import name ‘AdultDataset’ from ‘dataset’</p>
<h1>Import Relevant Libraries</h1>
<pre><code class="lang-auto">import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.preprocessing import LabelEncoder
from sklearn.model_selection import train_test_split

import torch
import torch.nn as nn

import torch.optim as optim
from torch.autograd import Variable
from torch.utils.data import DataLoader


from dataset import AdultDataset

So when I am trying to create a 3 layer Feed Forward Neural network using pytorch that takes as input the a dataset entries and classifies if the individuals gains more or less than 50K (i.e., the fnlwgt label)

starting with the code below I get error

train_dataset = AdultDataset(X_train, y_train)
test_dataset = AdultDataset(X_test, y_test)

NameError Traceback (most recent call last) in 1 # Using norm_D01 2 ----&gt; 3 train_dataset = AdultDataset(X_train, y_train) 4 test_dataset = AdultDataset(X_test, y_test) 5

NameError: name 'AdultDataset' is not defined</code></pre>

Nikronic: <p>Hi,</p>
<p>What is <code>dataset</code> here? I think it is not related to PyTorch as you <code>dataset</code> seems like a custom module where <code>AdultDataset</code> should be in it.</p>
<aside class="quote no-group" data-username="Samuel_Adu" data-post="1" data-topic="107623">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/samuel_adu/48/12389_2.png" class="avatar"> Samuel_Adu:</div>
<blockquote>
<p><code>from dataset import AdultDataset</code></p>
</blockquote>
</aside>
<p>And the error is saying that <code>AdultDataset</code> does not exist in <code>dataset</code>.</p>
<p>Bests</p>

AlphaBetaGamma96: <p>Hey,</p>
<p>As <a class="mention" href="/u/nikronic">@Nikronic</a> said above, it seems that dataset is a custom module. So, make sure that you have a file called dataset.py within the same directory as your Python script because if it’s not in the same directory Python won’t be able to find it. (Unless you specifically change your working directory, or just copy the file into the same directory as your Python script).</p>
<p>And, within the file <code>dataset.py</code> make sure there’s a class called <code>AdultDataset</code> inside that file. That’s what you’re looking for with the line <code>from dataset import AdultDataset</code></p>
<p>If this <code>dataset</code> is a file you’ve installed via <code>pip</code> or <code>conda</code> make sure that your Python environment is set to use  the correct environment!</p>
<p>Kind regards,</p>
Different loss in two machnes

Hassan_Imani: <p>My code is working on two PCs with keras.<br>
But when  am runnng the same code in same PCs with same pytorch code, in one of them its working like keras, but not in the second!</p>
<p>Cuda version in both: 10.1<br>
Pytorch version in both: 1.7.0<br>
Python and every environment variables are the same.<br>
Very weired!</p>
<p>Any help is appreciated.</p>
Error of compiling torch.utils.cpp_extension with cuBLAS

DanielWang: <p>Hi, All</p>
<p>I recently encountered a problem when building a customized PyTorch operator using  <code>torch.utils.cpp_extension</code>. In my kernel, I call <code>cuBLAS_v2</code> for GEMM operations. While it can compile without errors. When I import the operator in Python, it crashed, giving me the following message.</p>
<blockquote>
<p>test_cuda.cpython-37m-x86_64-linux-gnu.so: undefined symbol: __cudaRegisterLinkedBinary_50_tmpxft_0000403a_00000000_7_test_cuda_kernel_cpp1_ii_7487cc74</p>
</blockquote>
<p>However, when I remove all cuBLAS related code in my kernel and recompile it, it can successfully run without any error.</p>
<p>Here are the options I use in my setup file.</p>
<blockquote>
<p>from setuptools import setup<br>
from torch.utils.cpp_extension import BuildExtension, CUDAExtension</p>
<p>setup(<br>
name=‘test’,<br>
ext_modules=[<br>
CUDAExtension(<br>
name=‘test_cuda’,<br>
sources=[‘test_cuda.cpp’, ‘test_cuda_kernel.cu’],<br>
extra_compile_args={‘cxx’: [‘-O3’],<br>
‘nvcc’: [‘-dc’, ‘-lcublas’, ‘-arch=sm_86’, ‘-lcuda’, ‘-lcudart’, ‘-lcudadevrt’]}<br>
}<br>
)<br>
],<br>
cmdclass={<br>
‘build_ext’: BuildExtension<br>
})</p>
</blockquote>
<p>Thanks!</p>

googlebot: <p>I’m not certain, but moving -lcublas to last position in list may help.</p>
AttributeError: 'str' object has no attribute 'load_state_dict'

111448: <p>Any help please</p>
<pre><code class="lang-auto">def load_model(model,epoch,model_type):
  path = models_path + 'saved_'+model_type+'_epoch'+str(epoch)+'.txt'
  print("Loading model from %s" % path)
  
  model.load_state_dict(torch.load(path)['model_state_dict'])


  return model
</code></pre>
<pre><code class="lang-auto"># load model
embed_epoch = 1
model_embed = load_model(model_embed,embed_epoch,'embed')
model_embed.to(device)
</code></pre>
<pre><code class="lang-auto">
Loading model from /content/drive/MyDrive/Models/saved_embed_epoch1.txt
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
&lt;ipython-input-64-da43c88e687e&gt; in &lt;module&gt;()
      1 # load model
      2 embed_epoch = 1
----&gt; 3 model_embed = load_model(model_embed,embed_epoch,'embed')
      4 model_embed.to(device)
      5 # load model

&lt;ipython-input-63-7b862763f577&gt; in load_model(model, epoch, model_type)
      3   print("Loading model from %s" % path)
      4 
----&gt; 5   model.load_state_dict(state_dict['state_dict'])
      6 
      7 

AttributeError: 'str' object has no attribute 'load_state_dict'
</code></pre>

ahmedoumar: <p>Try this:</p>
<pre><code class="lang-auto">model.load_state_dict(torch.load(model_path))
</code></pre>

111448: <p>thanks for you but after try this  it gives me this Error</p>
<pre><code class="lang-auto">Loading model from /content/drive/MyDrive/Models/saved_embed_epoch1.txt
---------------------------------------------------------------------------
IsADirectoryError                         Traceback (most recent call last)
&lt;ipython-input-57-da43c88e687e&gt; in &lt;module&gt;()
      1 # load model
      2 embed_epoch = 1
----&gt; 3 model_embed = load_model(model_embed,embed_epoch,'embed')
      4 model_embed.to(device)
      5 # load model

3 frames
/usr/local/lib/python3.6/dist-packages/torch/serialization.py in __init__(self, name, mode)
    209 class _open_file(_opener):
    210     def __init__(self, name, mode):
--&gt; 211         super(_open_file, self).__init__(open(name, mode))
    212 
    213     def __exit__(self, *args):

IsADirectoryError: [Errno 21] Is a directory: '/content/drive/MyDrive/Models/'
</code></pre>
Compare with a float number in Pytorch returns weird results

hoangcuong2011: <p>I have a simple code that generates a random tensor (float32). Then I compare each of the values in the tensor equal to the first number in the tensor. I got weird result where all True values are returning.</p>
<blockquote>
<p>import torch<br>
torch.manual_seed(1)<br>
a = torch.rand(2, 10)<br>
tensor([[0.7576, 0.2793, 0.4031, 0.7347, 0.0293, 0.7999, 0.3971, 0.7544, 0.5695,<br>
0.4388], [0.6387, 0.5247, 0.6826, 0.3051, 0.4635, 0.4550, 0.5725, 0.4980, 0.9371,<br>
0.6556]])</p>
<p>a != 0.7576</p>
<p>tensor([[True, True, True, True, True, True, True, True, True, True], [True, True, True, True, True, True, True, True, True, True]])</p>
</blockquote>
<p>Meanwhile If I changed to something like this, I got the right returning boolean values.</p>
<blockquote>
<p>a != a[0][0]</p>
<p>tensor([[False,  True,  True,  True,  True,  True,  True,  True,  True,  True],<br>
[ True,  True,  True,  True,  True,  True,  True,  True,  True,  True]])</p>
</blockquote>
<p>I am curious why this happens? Thanks!</p>

KFrank: <p>Hi Hoang!</p>
<aside class="quote no-group quote-modified" data-username="hoangcuong2011" data-post="1" data-topic="107614" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hoangcuong2011/48/27131_2.png" class="avatar"> hoangcuong2011:</div>
<blockquote>
<blockquote>
<p>a = torch.rand(2, 10)<br>
tensor([[0.7576, 0.2793, 0.4031, 0.7347, 0.0293, 0.7999, 0.3971,<br>
…</p>
</blockquote>
</blockquote>
</aside>
<p>Here <code>a</code> is only printed out to four decimal digits.  That is, <code>a[0][0]</code><br>
is a number close to, but not actually equal to <code>0.7576</code>.  Perhaps<br>
<code>a[0][0] = 0.75758379</code>.</p>
<aside class="quote no-group">
<blockquote>
<blockquote>
<p>a != 0.7576</p>
</blockquote>
</blockquote>
</aside>
<p>Here you compare <code>a[0][0]</code> (as well as the rest of <code>a</code>) with <code>0.7576</code>.<br>
But <code>a[0][0]</code> is only approximately equal <code>0.7576</code> (equal only to four<br>
decimal digits).  Because it is not exactly equal, the inequality (<code>!=</code>)<br>
test returns <code>True</code> (as it should).</p>
<aside class="quote no-group">
<blockquote>
<blockquote>
<p>a != a[0][0]</p>
<p>tensor([[False, True, True, True, True, True, True,<br>
…</p>
</blockquote>
</blockquote>
</aside>
<p>Now you are comparing <code>a</code> to <code>a[0][0]</code> (in our hypothetical example, to<br>
<code>0.75758379</code>).  <code>a[0][0]</code> is, after all, exactly equal to <code>a[0][0]</code>, so the<br>
inequality test returns <code>False</code>, as you expect.</p>
<p>Best.</p>
<p>K. Frank</p>

hoangcuong2011: <p>Thank you Frank for your detailed explanation. Truly appreciate it!</p>
Inference model with BatchNorm2D generates different output via PyTorch and OnnxRuntime

OronG13: <p>My setup is:<br>
OS - Microsoft Windows 10 Enterprise 2016 LTSB<br>
GPU - Quadro M2000M<br>
CUDA vers - 9.0, 10.0 &amp; 11.0<br>
Visual Studio - 2017, 2019<br>
Python - 3.6.8<br>
Torch 1.1.0\1.6.0<br>
onnxruntme - 1.5.1<br>
onnxruntime-gpu - 1.4.0<br>
onnx version -  1.7.0</p>
<p>Please refer to the following link which show that torch Onnx exporter maybe has a problem with BatchNorm2D operation:<br>
<a href="https://github.com/microsoft/onnxruntime/issues/3873#issuecomment-708884722" rel="noopener nofollow ugc">BatchNorm2D Export problem</a></p>
<p>You will be able to find there all required information which describe the problem.</p>
<p>I thought that it will push forward the investigation if I will open an issue here also about the referenced Onnxruntime issue.</p>
<p>Thanks,</p>

MaximZ: <p>Hello, is there any update regarding this topic?</p>

OronG13: <p>Hello,<br>
Any kind of support will be much appreciated.</p>
<p>It is a major concern that we cannot include BatchNorm2D operator in our models due to *.onnx export problem.</p>
<p>In order to be able using NVIDIA TensorRT sdk, we shall understand what is the root cause of the wrong BatchNorm2D behavior after converting the TorchScript *.pt to *.onnx format.</p>
<p>Thank,</p>
The output tensor's shape of a scripted model is fixed by example's shape?

cfanyyx: <p>In a pytorch network model, if I use <code>nn.ConvTranspose2d</code> and do <code>traced_script_module = torch.jit.trace(model, example)</code> after constructing the model. Whatever the shape size of example tensor is, I can get the output tensor, of which the shape is same as the input image, from <code>traced_script_module</code>.<br>
But if I use <code>nn.Upsample</code> instead of <code>nn.ConvTranspose2d</code> in the model. Strange things happen. I only can get the output tensor which the shape of is the same as example tensor that I used in <code>torch.jit.trace(model, example)</code>.<br>
Can anyone explain this strange thing. Is that related to <code>weak_script_method</code> decorator or something?</p>

wanchaol: <p>Hi <a class="mention" href="/u/cfanyyx">@cfanyyx</a>, thanks for raising the issue. The problem on this is because <code>nn.Upsample</code> is a module that contains data dependent control flow (which means the traced model totally depends on the input you provide during tracing and will not generalize to the new input). To make <code>nn.Upsample</code> working in JIT, you will need to use TorchScript scripting instead of tracing. You can refer more to the doc <a href="https://pytorch.org/docs/master/jit.html" rel="nofollow noopener">here</a></p>
<p>Let me know if that’s clear or not.</p>

cfanyyx: <p>Thanks for your reply. It helps me a lot. I will have a try. Actually, I tried to use TorchScript scripting before, but I didn’t make it so I changed to use tracing instead. I will try TorchScript scripting again. And I will come back if I meet any new question.</p>

DBraun: <p>What’s the solution? It seems to not be just putting <code>@torch.jit.script</code> as a decorator on the function that calls upsample.</p>
Reproducibility when using DDP in multi node multi gpu settings

amirhf: <p>I have a script that is set to be deterministic using the following lines:</p>
<pre><code class="lang-auto">seed = 0
torch.manual_seed(seed)
np.random.seed(seed)
random.seed(seed)
</code></pre>
<p>I don’t use any non-deterministic algorithm. I use 5 machines with 8 gpus each. I use DDP with NCCL backend with one process per gpu. I always check the first loss value at the first feedforward to check the determinism. For the same script I used to get a different value when I started to run the script after that. So for the <strong>first</strong> run let’s say I get the loss value of 1.52 but any run after that gives me a different loss value that is always <strong>constant</strong> now. So for example, I get the loss value of 1.45 in run 2, 3, 4, 5, etc. I checked all my conda envs and I did not see any changes in the libraries. Is this normal? Is determinism possible with DDP? What do you think can be the cause of the first run giving a different loss value? I have read something in this forum that I might need to fix NCCL rings but I do not know how!</p>
<p>I have also checked the data stream and it hasn’t changed between runs from the beginning!</p>

osalpekar: <p>There can certainly be additional non-determinism at the NCCL level. Here is a GH thread with some discussion on this: <a href="https://github.com/pytorch/pytorch/issues/48576" class="inline-onebox" rel="noopener nofollow ugc">how to avoid the precision loss(float32) caused by the gradient accumulation of Ring Allreduce in the case of ddp · Issue #48576 · pytorch/pytorch · GitHub</a>.</p>
<p>Essentially you can try using the <code>NCCL_RINGS</code> environment variable to fix the rings. Depending on your compute setup, if the network topology changes between runs, there may be additional differences in terms of the allreduce configuration used by NCCL, and I would suggest reaching out to the NCCL team to see how it can be made completely deterministic.</p>
Why changing the backends of ddp from NCCL to GLOO introduces a significant difference in train loss (model accuracy)

WencongXiao: <p>as described in <a href="https://github.com/pytorch/pytorch/issues/49966" rel="noopener nofollow ugc">[github issue]</a></p>

osalpekar: <p>I’ll echo what Natalia mentioned on the GH issue - it’s not guaranteed that the collectives (specifically allreduce) in both NCCL and Gloo will produce exactly the same results. In fact, there may be slight variations in different runs with the NCCL backend if there are slight changes in the network, topology, ring configuration, etc.</p>
<p>Both NCCL and Gloo support several different allreduce implementations, which can further be configured extensively using environment variables. These libraries choose an implementation based on the topology and other factors, so it is completely possible that they choose different implementations/configurations which leads to small differences in the allreduce results.</p>
Computing recall on very large datasets

Miguel: <p>Hello,</p>
<p>I am training a model by means of triplet loss to generate image and point cloud descriptors. In order to evaluate its performance, I am plotting the training loss, recall@6 (i.e., given and anchor image, I select the 6 closer point clouds in the database and check if ground truth is within that retrieved data) on the training split and recall@6 on validation split.</p>
<p>Validation split is relatively small and I can compute recall after each epoch, but training split is too large to fit into GPU memory. Is there a way to compute this recall measure on the whole training split in PyTorch without having memory overflow issues? Doing the processing on CPU is too time expensive.</p>
<p>I thought that maybe obtaining the recall@6 for each batch and then computing the mean to get the recall@6 at a certain epoch would give me an approximation, but I am seeing that results are very different between following that procedure and computing the recall on the whole dataset after each epoch iteration.</p>
<p>In order to check whether the model is over fitting or not, would it be a good idea to calculate the loss on the validation split? Or that is not a good idea when using triplet loss?</p>
<p>Thanks!</p>
Defining hash function for multi-dimensional tensor

Manvi_Gupta: <p>Hi everyone!<br>
I need to define a hash function (mapping), with uniform distribution on its output.</p>
<p><strong>Signature</strong>: <em>Z</em><sup><em>d</em></sup> -&gt; <em>X</em>, where <em>X</em> = {1,2,3, … n} (n = fixed integer).<br>
Thus, a function that takes a <em>d</em> dimensional tensor as input and returns an integer value corresponding to it.</p>
<p>The condition is that the output of the Hash function should have a uniform density over the range of <em>X</em>. Is there an inbuilt PyTorch function that can help here? Or any other way to implement this?</p>
<p>Thanks in advance.</p>

googlebot: <p>It seems like pytorch lacks this, but you can do<br>
hash(pickle.dumps(tensor)) % n</p>

Manvi_Gupta: <p>Thanks, <a class="mention" href="/u/googlebot">@googlebot</a> Alex.  I actually need to do this task parallelly. Is there a PyTorch utility that can help in running such a function parallelly?</p>
<p>E.g. methods like <code>torch.sum()</code> and <code>torch.mean()</code> allow us to specify a dimension over which we can reduce. I am looking for a similar way to hash the elements of a multi-dimensional input.</p>

googlebot: <p>I’m not aware of a universal routine for that.</p>
<p>If you’re implying float tensors with known ranges and reasonable entropy, “surrogate” hash values based on standard reductions can be used, e.g.:</p>
<blockquote>
<p>hashes=torch.quantize_per_tensor(torch.randn(1000,8), 0.1, 0, torch.qint32).int_repr().sum(-1) % 10<br>
torch.bincount(hashes)</p>
<blockquote>
<p>tensor(cpu,(10,)[ 92,  93, 106, 113,  94, 101,  97,  89, 100, 115])</p>
</blockquote>
</blockquote>
Problem with multitask back propagation?

luvwinnie: <p>I have a CTC and seq2seq multitask loss it shows the below errors when doing the loss.backward(), does anyone know how to debug this?</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "mtl_train.py", line 351, in &lt;module&gt;
    train(opt)
  File "mtl_train.py", line 196, in train
    cost.backward()
  File "/DataStorage2/cheesiang_leow/artibrains/single_line/crnn_seq2seq_ocr_pytorch/venv/lib/python3.7/site-packages/torch/tensor.py", line 221, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/DataStorage2/cheesiang_leow/artibrains/single_line/crnn_seq2seq_ocr_pytorch/venv/lib/python3.7/site-packages/torch/autograd/__init__.py", line 132, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: The size of tensor a (0) must match the size of tensor b (5990) at non-singleton dimension 2
</code></pre>
How to indexing a tensor with a tensor in multi-dimension?

Zhaoyi-Yan: <pre><code class="lang-python">a = torch.tensor([[1,2,3,4],[5,6,7,8]])
idx = torch.tensor([[0,2,1],[2,3,0]])

# How to do it in batch ?
c_1 = a[0][idx[0]].view(1,-1)
c_2 = a[1][idx[1]].view(1,-1)
c = torch.cat((c_1, c_2), dim=0)
</code></pre>
<p>The desired output is:</p>
<pre><code class="lang-bash">tensor([[1, 3, 2],
        [7, 8, 5]])
</code></pre>
<p>I tried <code>a[idx]</code>, however, it goes wrong.</p>

LeviViana: <p>What is <code>idx</code> and the desired output in the snippet above ?</p>

Zhaoyi-Yan: <p>Hi, I made a typo, and just made the correction for it.</p>

Zhaoyi-Yan: <p><code>idx</code> is the indexes for selecting the elements in the tensor.</p>

LeviViana: <pre><code class="lang-python">import torch
a = torch.tensor([[1,2,3,4],[5,6,7,8]])
idx = torch.tensor([[0,2,1],[2,3,0]])
idx2 = idx + torch.arange(idx.size(0)).view(-1, 1) * a.size(1)
c = a.view(-1)[idx2]
</code></pre>
<p>It works as long as <code>a</code> and <code>idx</code> have only two dimensions and <code>idx.size(0) &lt;= a.size(0)</code>.</p>

Zhaoyi-Yan: <p>Thank you fo you help <a class="mention" href="/u/leviviana">@LeviViana</a>.  It is surely a way to solve it, however, I wonder whether pytorch can support <code>a[idx]</code> , is it possible  <a class="mention" href="/u/alband">@albanD</a>  ?</p>

albanD: <p>Hi,</p>
<p><code>gather</code> is what your want!</p>
<pre><code class="lang-auto">c = a.gather(1, idx)
</code></pre>

jia_lee: <p>a_tensor[idx] is supported, I often use this way.</p>
[SOLVED]Convert color of tensors

Petro_Key: <p>Hi, thank you always for your help.<br>
I am trying to convert an image set stored in torch.Tensor of like [number_of_images, width, height],<br>
to a set of color images of like [number_of_images, 3, width, height].<br>
I would like to use the color scheme ‘magma’ that can be used in matplotlib.<br>
Is there any good way to realize it?</p>
<p>Thank you in advance <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

ptrblck: <p>You could convert the tensors to numpy arrays and apply the colormap on them:</p>
<pre><code class="lang-python">import matplotlib as mpl
import matplotlib.pyplot as plt
from matplotlib import cm


x = np.random.randn(100, 100).clip(0, 1)

magma = cm.get_cmap('magma')
x_transformed = magma(x)
plt.imshow(x_transformed)

plt.imshow(x)
plt.imshow(x, cmap='magma')
</code></pre>
<p>Note that the colormap expects normalized inputs, so I clipped the data to <code>[0, 1]</code>.<br>
Once you have these transformed arrays, you could convert them back to tensors via <code>torch.from_numpy</code>.</p>
<p>PS: What is your use case that you want to artificially inflate your channels?</p>

Petro_Key: <p>Hi ptrblck,<br>
Thank you always for your support!<br>
It solved my problem.<br>
Thank you very much!</p>

Ren_Pang: <p>Add one note, that might not be the case when you use a <code>cmap</code> in matplotlib.<br>
If you have a manual colormap that is <code>torch.Tensor</code> with shape <code>[256, 3]</code>, you could try this cool method:</p>
<pre data-code-wrap="python3"><code class="lang-python3">color_map = #Tensor of shape(256,3)
gray_image = (gray_image * 255).long() # Tensor values between 0 and 255 and LongTensor and shape of (512,512)
output = color_map[gray_image] #Tensor of shape (512,512,3)
</code></pre>
<p>from</p><aside class="quote" data-post="2" data-topic="106662">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mazhar_shaikh/48/14501_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/pytorch-colormap-gather-operation/106662/2">Pytorch colormap gather operation</a> <a class="badge-category__wrapper " href="/c/vision/5"><span data-category-id="5" data-drop-close="true" class="badge-category " title="Topics related to either pytorch/vision or vision research related topics"><span class="badge-category__name">vision</span></span></a>
  </div>
  <blockquote>
    You can use this code 
color_map = #Tensor of shape(256,3)
gray_image = (gray_image * 255).long() # Tensor values between 0 and 255 and LongTensor and shape of (512,512)
output = color_map[gray_image] #Tensor of shape (512,512,3)

Why does this work? The gray_image tensor is used as an index in the 0th dimension of color_map. The color_map tensor is accessed with this index and the new tensor created has the shape of gray_image inserted in the 0th dimension of output.
  </blockquote>
</aside>
CUDA error an illegal memory access was encountered

luziniunai: <p>When I train a model(GPU),it works fine in the first epoch(not iteration), but during the second epoch, inputs = inputs.to(device) gets error</p>

ptrblck: <p>If you are not using the latest PyTorch release, could you update to the latest stable version and rerun the code, please?<br>
If you are still facing this error, could you post an executable code snippet to reproduce it so that we could debug it?</p>
CUDA 9.2 RNN non-determinism

zpeng: <p>In the <a href="https://pytorch.org/docs/stable/generated/torch.nn.LSTM.html#torch.nn.LSTM" rel="noopener nofollow ugc">LSTM page</a>, I see the warning:</p>
<pre><code class="lang-auto">There are known non-determinism issues for RNN functions on some versions of cuDNN and CUDA. You can enforce deterministic behavior by setting the following environment variables:

On CUDA 10.1, set environment variable CUDA_LAUNCH_BLOCKING=1. This may affect performance.

On CUDA 10.2 or later, set environment variable (note the leading colon symbol) CUBLAS_WORKSPACE_CONFIG=:16:8 or CUBLAS_WORKSPACE_CONFIG=:4096:2.

See the cuDNN 8 Release Notes for more information.
</code></pre>
<p>I use CUDA 9.2, so, for CUDA 9.2 what should I do to enforce deterministic behavior? I set the random seed as follows:</p>
<pre><code class="lang-auto">    torch.manual_seed(seed)
    np.random.seed(seed)
    random.seed(seed)
    torch.backends.cudnn.deterministic = True
    torch.backends.cudnn.benchmark = False
</code></pre>
<p>Thanks.</p>

ptrblck: <p>Are you seeing the non-determinisim also when disabling cudnn via <code>torch.backends.cudnn.enabled = False</code>? If not, which cudnn version are you using?</p>
amp.Gradscaler().step(optimizer) is taking too much time!

hassannn_nasrala: <p>So I have created a class <code>TrainEpoch</code> to train a single epoch , where I defined two functions for the forward and the backward pass as follows:</p>
<pre><code class="lang-auto">class TrainEpoch():
    ....
    ....
    ....
    def Forward(self,x,y):        
            self.pre_logits = self.model(x)
            self.pre_logits = _s(self.pre_logits,self.device) #send logits to device
            self.curr_loss = self.loss_func(self.pre_logits, y)
    def Backward(self):
            self.curr_loss.backward()
            self.optimizer.step()
            self.optimizer.zero_grad()
</code></pre>
<p>This class is working fine, however I created a subclass <code>TrainEpoch_AMP</code> from the above one to use automatic mixed precision training, and set the forward and backward pass function as follows:</p>
<pre><code class="lang-auto">import torch.cuda.amp as AMP
scaler = AMP.GradScaler()

class TrainEpochAMP(TrainEpoch):
    ....
    self.scaler = scaler
    ....
    ....
    @AMP.autocast()
    def Forward(self, x, y):
        super().Forward(x, y)
       
        
    def Backward(self):
        self.scaler.scale(self.curr_loss).backward()
        self.scaler.step(self.optimizer)
        self.scaler.update()
        self.optimizer.zero_grad()
</code></pre>
<p>when i try to use amp training , the time needed to complete one epoch becomes 22hours while it is only 30 minutes without amp…<br>
After inspection, i found that <code>self.scaler.step(self.optimizer)</code> needs ~17 seconds to execute!!!<br>
what could be the problem?</p>

ptrblck: <p>The <code>GradScaler</code> shouldn’t add this massive overhead, as it’ll check for invalid gradients and skip the <code>optimiyer.step()</code> if necessary. Could you post a minimal code snippet to reproduce this issue, so that we could debug it, please?</p>
[Libtorch] How to train using multiple gpu?

HwanUng_Lee: <p>I used libtorch to create model in c++ environment and train in a single gpu.</p>
<p>Now, I want to train using multi gpu, but I don’t know how.</p>
<p>Do you have any examples related to this?</p>

ptrblck: <p>I haven’t used the C++ dataparallel API yet, but you might want to take a look at <a href="https://github.com/pytorch/pytorch/blob/c0ff08577570c23694eb0e8b8750564847daccc2/test/cpp/api/parallel.cpp#L230-L293">this test</a>.<br>
I’m unsure about the status of DDP in libtorch, which is the recommended approach for performance reasons.</p>

lampadephoria: <p>Hello,<br>
Does torch::nn::parallel::data_parallel() support torch::jit::Module?</p>

ptrblck: <p>I think <code>data_parallel</code> should work with a scripted model, as it would only chunk the inputs and transfer them to all specified GPUs as well as copying the model to these devices, as long as the eager model also runs fine in data parallel (i.e. no device mismatches are raised due to a wrong usage of a specific device inside the model).</p>
Map values of a tensor to their indices in another tensor

Fred_Ma: <pre><code class="lang-auto">target = torch.tensor([1.25, 2.5, 3, 1.25, 3])
values = torch.tensor([1.25, 2.5, 3])
</code></pre>
<p>I would like to find a fast operation that maps every values in <code>target</code> by their indices in <code>values</code>:</p>
<pre><code class="lang-auto">result: torch.tensor([0, 1, 2, 0, 2])
</code></pre>
<p>Thanks</p>

Naruto-Sasuke: <p>I convert this problem to a "finding equal values in two expanded matrixes’.  Here is the code:</p>
<pre><code class="lang-python">target = torch.tensor([1.25, 2.5, 3, 1.25, 3])
values = torch.tensor([1.25, 2.5, 3])

t_size = target.numel()
v_size = values.numel()

t_expand = target.unsqueeze(1).expand(t_size, v_size) 
v_expand = v_size.unsqueeze(0).expand(t_size, v_size)

result = (t_expand - v_expand == 0).nonzero()[:,1]
</code></pre>
Torch.reshape(-1,784) what does the -1 mean?

v-za: <p>Hey guys,</p>
<p>if my tensor shape is [128,1,28,28]</p>
<p>and when i call the reshape method .reshape(-1,784)</p>
<p>I get a tensor with the shape [128,784] which is what i want. But I don’t know what the -1 is doing? How does it reshape it like that?</p>

googlebot: <p>it is a “wildcard”, i.e. auto-calculated <em>single</em> size</p>
Regarding volatile=True

Adamya_Shyam: <p>I have a code in which  following line is written but when I ran it, some warnings occurred quoting use no_grad.<br>
What should I do for this command? Please tell the alternate.</p>
<p>def batchrun(image_batch, model):<br>
image_batch = torch.FloatTensor(image_batch).cuda()<br>
image_batch = torch.autograd.Variable(image_batch, volatile=True)<br>
feats = model(image_batch)<br>
feats = feats.data.cpu().clone().numpy()<br>
return feats</p>

albanD: <p>Hi, you should change it to:</p>
<pre><code class="lang-python">def batchrun(image_batch, model):
  image_batch = torch.tensor(image_batch, dtype=torch.float32, device="cuda")
  with torch.no_grad():
    feats = model(image_batch)
  feats = feats.detach().cpu().clone().numpy()
  return feats
</code></pre>

Adamya_Shyam: <p>Just to be sure the code was like this, so the alternate you gave according to mine is correct?</p>
<pre><code class="lang-auto">def batchrun(image_batch, model):
  image_batch = torch.FloatTensor(image_batch).cuda()
  image_batch = torch.autograd.Variable(image_batch, volatile=True)
  feats = model(image_batch)
  feats = feats.data.cpu().clone().numpy()
  return feats
</code></pre>

albanD: <p>Yes that’s the same code as above no? So the alternative is still the same.</p>

Adamya_Shyam: <p>Thanks a lot! It is working.</p>
Looking for Help with Almost Finished Video Game AI Project

peterfarrell66: <p>Hello to the Forum!</p>
<p>I’ve been working on creating an AI to play <strong>Asteroids</strong>. The game is done (in Pygame) and I’ve successfully got a DIY neural net to play it and improve using a genetic algorithm. But the improvement is tiny (and maybe random) and tracking the improvement has gotten a bit complicated for me and I’d love some help.</p>
<p>Ideally, I’d like to get my DIY NN file to show some improvement, then switch over to using Pytorch with as many hidden layers as needed to produce much better/quicker improvement.</p>
<p>I’m able to pay for your time if that sweetens the deal at all.</p>
<p>My code is <a href="https://github.com/hackingmath/Neural-Net/tree/master/November2020" rel="noopener nofollow ugc">on Github</a> if you’d like to take a look.</p>
<p>Hope to hear from you Pytorch geniuses soon!</p>
<p>Peter Farrell<br>
Plano, TX</p>