akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Getting File not found error while running setup.py file

username: Ishwar_Joshi
cooked: <p>Hi,</p>
<p>I am trying to setup the pytorch environment in my local machine and i have followed all the steps carefully but I am still getting the error below,</p>
<p>Traceback (most recent call last):<br>
File “C:\Users\ishwa\Documents\Pytorch\pytorch_ij\setup.py”, line 1376, in <br>
main()<br>
File “C:\Users\ishwa\Documents\Pytorch\pytorch_ij\setup.py”, line 1092, in main<br>
build_deps()<br>
File “C:\Users\ishwa\Documents\Pytorch\pytorch_ij\setup.py”, line 464, in build_deps<br>
build_caffe2(<br>
File “C:\Users\ishwa\Documents\Pytorch\pytorch_ij\tools\build_pytorch_libs.py”, line 84, in build_caffe2<br>
cmake.generate(<br>
File “C:\Users\ishwa\Documents\Pytorch\pytorch_ij\tools\setup_helpers\cmake.py”, line 345, in generate<br>
self.run(args, env=my_env)<br>
File “C:\Users\ishwa\Documents\Pytorch\pytorch_ij\tools\setup_helpers\cmake.py”, line 97, in run<br>
check_call(command, cwd=self.build_dir, env=env)<br>
File “C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2032.0_x64__qbz5n2kfra8p0\Lib\subprocess.py”, line 408, in check_call<br>
retcode = call(*popenargs, **kwargs)<br>
^^^^^^^^^^^^^^^^^^^^^^^^^^<br>
File “C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2032.0_x64__qbz5n2kfra8p0\Lib\subprocess.py”, line 389, in call<br>
with Popen(*popenargs, **kwargs) as p:<br>
^^^^^^^^^^^^^^^^^^^^^^^^^^^<br>
File “C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2032.0_x64__qbz5n2kfra8p0\Lib\subprocess.py”, line 1026, in <strong>init</strong><br>
self._execute_child(args, executable, preexec_fn, close_fds,<br>
File “C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2032.0_x64__qbz5n2kfra8p0\Lib\subprocess.py”, line 1538, in _execute_child<br>
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,<br>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^<br>
FileNotFoundError: [WinError 2] The system cannot find the file specified</p>
<p>anyone knows how to deal with this?</p>

username: Ishwar_Joshi
cooked: <p>I was able to solve above error after installing the vs build tools, but now I am getting another error after running the “python setup.py develop” command the error is as follows :-</p>
<p>C:\Users\ishwa\anaconda3\condabin\pytorch\third_party\pocketfft\pocketfft_hdronly.h(156): error C2039: ‘aligned_alloc’: is not a member of ‘`global namespace’’<br>
[6472/7536] Building CXX object caffe2\CMakeFiles\torch_cpu.dir__\aten\src\ATen\native\mkldnn\ConvPrepack.cpp.obj<br>
ninja: build stopped: subcommand failed.</p>
<p>can anyone please help me with this error.</p>
Nesting Dataloaders

username: kunaltyagi
cooked: <p>I have multiple video files (of same length) from which I create NTCHW shaped tensors (alongside metadata, so TensorDataset is out).</p>
<p>It would be quite easy if I can create a dataset (per video file) with a dataloader to create batches in time and then create a dataset over all video files and a top level dataloader to being them together into the desired shape.</p>
<p>But there’s no way to manage the num_workers of the 2 nested dataloaders to spread the work around.</p>
<pre data-code-wrap="py"><code class="lang-py">Dataloader(
  Dataset(
    TemporalDataloader(
      VideoDataset(file),
      num_workers=ABC,
    ) for filename in list_of_files,
    num_workers=DEF,
  )
)
</code></pre>
<p>I see there’s a <code>multiprocessing_context</code> in the API. Is that the correct way to go ahead? What are the limitations of this approach if there’s a better approach?</p>
PyTorch on MPS doesn't use M1 Max gpu at full power - frequency is at minimal value

username: Mihail
cooked: <p>For some reason, frequency of M1 Max gpu is low - 400HZ instead of maximum possible  ~1300HZ. In <a href="https://medium.com/@koypish/mps-or-mlx-for-domestic-ai-the-answer-will-surprise-you-df4b111de8a0" rel="noopener nofollow ugc">this post</a>  details are presented.<br>
<strong>Turns out, PyTorch could be 3x times faster if it would just use a boosted GPU frequency.</strong><br>
What could be the reason of this? Maybe they just don’t use Metal API properly? In comparison, MLX runs at boosted frequency and 3x faster</p>

username: Mihail
cooked: <p>This is a repo with code to reproduce:</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://github.com/mihasK/phi2-llm-on-MLX-vs-Pytorch-MPS">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">

      <a href="https://github.com/mihasK/phi2-llm-on-MLX-vs-Pytorch-MPS" target="_blank" rel="noopener nofollow ugc">GitHub</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/344;"><img src="https://opengraph.githubassets.com/2261fec69face1cf4a30a42d2bd315542d7508527e4c571ea937b054ec7f0a95/mihasK/phi2-llm-on-MLX-vs-Pytorch-MPS" class="thumbnail" width="690" height="345"></div>

<h3><a href="https://github.com/mihasK/phi2-llm-on-MLX-vs-Pytorch-MPS" target="_blank" rel="noopener nofollow ugc">GitHub - mihasK/phi2-llm-on-MLX-vs-Pytorch-MPS</a></h3>

  <p>Contribute to mihasK/phi2-llm-on-MLX-vs-Pytorch-MPS development by creating an account on GitHub.</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>You can also just use asitop to see frequency of GPU while using PyTorch on MPS</p>
How to switch mixed-precision mode in training

username: kellenf
cooked: <p>Is it possible to dynamic change some layer’s amp_mode of one network in training process?<br>
I use such method to set amp_mode attribute for all Linear &amp;&amp; BN layers.</p>
<pre><code class="lang-auto">class MyDecorator:
    def __init__(self, func):
        self.func = func
        self.amp_mode = True

    def _to_float(self, x):
        if isinstance(x, SparseConvTensor):
            x = x.replace_feature(x.features.float())
        elif isinstance(x, torch.Tensor):
            x = x.float()
        return x
    
    def __call__(self, *args, **kwargs):
        if self.amp_mode:
            with autocast(enabled=True, dtype=torch.float16):  
                return self.func(*args, **kwargs)
        else:
            with autocast(enabled=False, dtype=torch.float32):
                args = [self._to_float(x) for x in args]
                kwargs = {k: self._to_float(v) for k, v in kwargs.items()}
                return self.func(*args, **kwargs)

def creat_forward_hook(name):
    def hook(module, input, output):
        print(f'{name}, is_amp?: {module.forward.amp_mode}, input_dtype: {input[0].dtype}, output_dtype: {output.dtype}')
    return hook

for name, module in model.named_modules():
    if isinstance(module, (nn.Linear, nn.BatchNorm1d)):
        module.forward = MyDecorator(module.forward)
        module.register_forward_hook(creat_forward_hook(name))
</code></pre>
<p>When the training begin, they are use amp training, in the training process, I change some of them which occurs nan value by setting forward.amp_mode=False. But It seems not really in FP32 comput mode? Because they will occurs nan again!<br>
I training my network in 8 A100 GPUS. What else should I do beside these setting? Can you help me? THANKS!</p>
<p><a class="mention" href="/u/ptrblck">@ptrblck</a></p>

username: ptrblck
cooked: <p>I don’t think using forward hooks will work as the output is already computed. You might want to disable amp in the forward method of your model manually instead.</p>

username: kellenf
cooked: <p>Oh, the forward_hook is only used for print the dtype of inputs and outputs, it’s a little part of my project. In fact, I wrap MyDecorator for module’s forward function.<br>
I set amp_mode to false when some layer output a number which is not finite. But It seems not really work because this layer output NAN again.<br>
So how to dynamic set AMP mode in training process? Maybe need do something else in multi-gpu training?</p>
SGD update not using loss value, while calculating the update

username: Kumar_selvakumaran
cooked: <p>I was playing around with gradients, and optimizers, as I wanted to understand it from the roots. I created a toy example with a single convolutional layer, to understand how the gradients are calculated and  updated.</p>
<p>From what I have tried, It seems like the update is using only the gradient and and not the loss value itself.<br>
like so:</p>
<p>say :<br>
w_new = new parameter<br>
w = old parameter<br>
lr = learning rate<br>
d_J_w = gradient of J wrt. w<br>
J = loss</p>
<p>The operation that occurs looks like :<br>
w_new =  w - (lr * d_J_w)</p>
<p>BUT, shouldnt it actually be<br>
w_new = w - (lr * d_J_w * <strong>J</strong>)</p>
<p>Why does this happen?</p>
<p>Below is my code for reference<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/f/d/fde5d8cff8829081fc13572a63190e4400e8748c.png" alt="image" data-base62-sha1="Ae5faXjAStkftiXYaAbFsQ0x9ik" width="678" height="486"></p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/7/8/781d6855ddb2947efc7bc0ee35d264567a5cb61a.png" alt="image" data-base62-sha1="h8AiIlWsQLQcMXdQsYSiIhVn6Gu" width="669" height="348"></p>
<p>output :<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/c/6c1adfff1abd5343a0ca8423fd2527b85f8b571e.png" data-download-href="https://discuss.pytorch.org/uploads/default/6c1adfff1abd5343a0ca8423fd2527b85f8b571e" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/6/c/6c1adfff1abd5343a0ca8423fd2527b85f8b571e.png" alt="image" data-base62-sha1="fql93ZPzjyVVBFYdRKk616H51p4" width="606" height="500" data-dominant-color="3F3F3F"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">626×516 7.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I may be missing something very trivial, sorry, but, I am pretty new to pytorch.</p>

username: astle_dsa
cooked: <p>Hello, just a suggestion, but it would be great if you could add more comments to you code, it’ll make reading it faster.<br>
So the learning process is generally like this:<br>
<strong>New Weights = Old Weights - (Learning Rate * Gradient of weight wrt Loss)</strong><br>
So no, loss is not required in the above process. Also the <em>Jacobian</em> (J) is for the <strong>w</strong> with respect to Loss, and not the other way round.</p>
Construct a custom backwards function for conv2d

username: 71M073J
cooked: <p>Hi, i’ve looked through posts here and they are either too specific for my purpose or linked to deleted git repositories</p>
<p>What I would like to do is to start out with just a normal conv2d forward and backward, so that i can play with passing and editing the gradients.<br>
I’d like to start out with a backward function as if we implemented conv2d backward ourselves, and then edit it to use approximately calculated gradients</p>
<p>For the forward pass i know that it needs to use unfold to get the sliding window, but how can i make the backward?<br>
Thanks in advance!</p>

username: astle_dsa
cooked: <p>Calculating backward propagation manually for convolutional layers is a very complex task, especially without any autograd system. You would be spending a <em>lot</em> of time doing the calculations (even for a simple network/layer).<br>
You can try to implement your own toy autograd system if you really want complete control ( although I think even PyTorch allows it ? ), which is not that complex if you understand the underlying principles.</p>

username: 71M073J
cooked: <p>what i really want is to manually do backprop steps because i am trying to do a proof of concept approximation of gradients, but that’d be MUCH easier, if i had a “normal” conv2d layer backward code to just edit<br>
therefore the practical performance isn’t key<br>
so i’d appreciate if anyone can help me there</p>

username: astle_dsa
cooked: <p>I don’t think there is exactly a <em>code</em> where the back-propagation step happens. There is no backprop code, or any specific code to do backprop for conv2d as well. There are just tensors and the Tensor class which handles the back propagation on  any operation that has been done on a tensor. I would suggest exploring the Tensor class, or read chapter 13 of <strong>Grokking Deep Learning</strong> (by Andrew .D Trask) for a better explanation.</p>
Access to model parameters

username: Martin_Rechberger
cooked: <p>Hello everyone, is there a way to read and write the model parameters of a scripted model? I need, in some way, the same function as in PyTorch with ‘model.parameters()’ .</p>

username: astle_dsa
cooked: <p>You can try model.named_parameters() ? It generates an object you will have to loop over.</p>
<pre><code class="lang-auto">for name, weights in model.named_parameters():
    print(name, weights)

</code></pre>

username: Martin_Rechberger
cooked: <p>Hi, thanks for your answer. This solution works in if I can execute the scripted model in python. I forgot to mention that the model should run on Android. In that case, the method named_parameters() is no longer available. Additionally, I would also need to find a way to write the parameters back into the model</p>

username: astle_dsa
cooked: <p>No problem. Are you using TorchScript ?</p>

username: Martin_Rechberger
cooked: <p>Yes, I use TorchScript. Here is the Python script I use to convert the models. Just a brief explanation: <code>set_param</code> writes a member variable that can be later read. However, this has the disadvantage that the model becomes almost twice as large, and I can only read the parameters and not write them. In the meantime, I have already found that the parameters are stored in the constants of the scripted model. I suspect, therefore, that there is actually no way around this. Or is there?</p>
<pre><code class="lang-auto">convert_model_name = 'my_model'
device             = 'cpu'
edit_model         = False;
write_param        = True;

# load original model
model_fn = convert_model_name+'.pt'
model = torch.load('models/orig/'+model_fn, map_location=device)

param = None
if(write_param):
    param = model.parameters()

# write model parameters
model.set_param(param)

# convert for mobile
scripted_module = torch.jit.script(model)

if(edit_model):
    # Export full jit version model (not compatible mobile interpreter), leave it here for comparison
    scripted_module.save('models/scripted/'+model_fn)
    
    with ZipFile('models/scripted/'+model_fn, 'r') as zip_orig: 
        zip_orig.extractall( path='temp')

    input = input("Hit enter")

    os.system('cd temp; zip -r '+convert_model_name+'.pt '+convert_model_name)
    
    scripted_module = torch.load('temp/'+convert_model_name+'.pt')

# Export mobile interpreter version model (compatible with mobile interpreter)
fn_label = ''
if(write_param):
    optimized_scripted_module = optimize_for_mobile(scripted_module, preserved_methods=['parameters_mobile', 'set_parameters']) #, 'set_parameters'
    fn_label = '__tv'
else:
    optimized_scripted_module = optimize_for_mobile(scripted_module)
    

optimized_scripted_module._save_for_lite_interpreter('models/lite/'+convert_model_name+fn_label+'.ptl')
</code></pre>

username: astle_dsa
cooked: <p>I don’t think you can rewrite parameters of scripted models, in any case. You can try to re-make the entire models layer by layer (if it’s not too large) and just given them you desired parameters.</p>
PyTorch LSTM Data Structure

username: atn998
cooked: <p>Hi, I currently have a dataset with multiple features, where each row is a time-series and each column is a time step. For example:</p>
<div class="md-table">
<table>
<thead>
<tr>
<th>feature1_time1</th>
<th>feature1_time2</th>
<th>feature1_time3</th>
<th>feature2_time1</th>
<th>feature2_time2</th>
<th>feature2_time3</th>
<th>target</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td>4</td>
<td>7</td>
<td>10</td>
<td>2</td>
<td>1</td>
<td>0</td>
</tr>
<tr>
<td>2</td>
<td>5</td>
<td>8</td>
<td>1</td>
<td>4</td>
<td>4</td>
<td>1</td>
</tr>
<tr>
<td>3</td>
<td>6</td>
<td>9</td>
<td>4</td>
<td>6</td>
<td>5</td>
<td>0</td>
</tr>
</tbody>
</table>
</div><p>How should I re-shape the data so that I can properly represent the sequential information when I use a pytorch LSTM?</p>
<p>Currently I’ve left it the way it is, transformed the features into tensors and wrapped it inside a variable and reshaped it using this code:</p>
<pre><code class="lang-auto">X_train_tensors = Variable(torch.Tensor(X_train), requires_grad=True)
X_test_tensors = Variable(torch.Tensor(X_test), requires_grad=True)

y_train_tensors = Variable(torch.Tensor(y_train), requires_grad=True)
y_test_tensors = Variable(torch.Tensor(y_test))
</code></pre>
<p>Final Shape Looks like:</p>
<p><code>torch.Size([num_rows, 1, num_features])</code></p>
<p>The LSTM runs fine, however, I’m worried that I’ve not captured the sequential nature of the dataset by keeping it at this orientation? Should I have made every row a time-sequence and the columns a time-series? And in that case what would the final shape look like and how could I transform that using pytorch tools?</p>

username: astle_dsa
cooked: <p>Packing them as is in a variable is not a good idea: it means that feature2_time1 came <em>after</em> feature1_time3 for example, which is simply not the case. Your model did good because LSTMs can just memorise any data you give them, whether the patterns actually exist or not.</p>
<p>Also, does feature1_time1 has multiple values (1, 2, 3, etc) ? I don’t know what the feature represents, but you can try reducing the dimension.</p>

username: atn998
cooked: <p>Hi thanks for your response! I will remove the variable, how would you suggest then that I order the data?  And no feature1_time1 doesn’t have multiple values, those are different instances. So each row represents an individual user or person. <a class="mention" href="/u/astle_dsa">@astle_dsa</a></p>

username: astle_dsa
cooked: <p>The way you can arrange is probably this: (<em>num_features, num_users, num_time steps</em>) or (<em>num_users, num_time steps, num_features</em>) and pack them into a third-order tensor. So the labels or target would be a matrix. You can try out this preprocessing and test you results.</p>
Loss function to fit unordered points/shape in 2d/3d space

username: sfx
cooked: <p>For simplicity I am going start with a toy example first.</p>
<p>Lets suppose we have a set of <code>n</code> points <code>Y</code> in the space, distributed with the shape of the letter M and you want to fit those using 4 lines. These points are not ordered in the sense that the order of the points in the vector <code>Y</code> is shuffled.</p>
<p>So I built a my function <code>f</code> that given 5 points <code>P</code>, it generates 4 lines concatenated joining those 5 points. I want to find the 5 points <code>P</code> that I can use to best describe the data. This is, the lines are close to the points.</p>
<p>So basically my though here is, given values <code>x in [0,1]</code> and my function <code>f(x, P)</code> I can sample points from this function and compute the loss against the <code>Y</code>, <code>Loss(f(x,P), Y)</code>. For example if x is <code>torch.linspace(0, 1, 100)</code> I am gong to draw 100 points from my function. I can also draw <code>n</code> points. Whatever it works.</p>
<p>If the points of the ground truth would be ordered and kind of evenly distributed I can use the <code>MSELoss</code> function and I can get the 5 points (I already tried this and it works). However if the <code>Y</code> points are unordered this loss is no longer working.<br>
My question is then, which kind of loss function I can use here to find <code>P</code>?</p>
<p>My problem is a little bit more general and I have a set of points <code>Y</code> that follow a line that curves through the 3d space so I would like to use m concatenated lines given by <code>f(P})</code>  whose distribution best matches <code>Y</code>.</p>

username: sfx
cooked: <p>Apparently the The Chamfer distance can work. This distance is computed by summing the squared distances between nearest neighbor correspondences of two point clouds.</p>
No private messaging?

username: ThePie
cooked: <p>There definitely needs to be private messaging added to any forums site…</p>

username: J_Johnson
cooked: <p>Check your messages.</p>

username: ptrblck
cooked: <p>As <a class="mention" href="/u/j_johnson">@J_Johnson</a> indicates, there are private messages. Your trust level might be too low to send messages, so spend some time in the forum before trying to message users.</p>
Rendering and searching for 3D objects

username: Serhat_Bilal
cooked: <p>Hi everyone, I want to make 2D images by taking images from various angles of 3D objects with pytorch3d, that is, by rendering them, then I want to search for similarity with artificial intelligence support by extracting features from these images, for example, when I want to find a 3D object that exists in my database or when I want to check whether it exists, I can say that I want to search for it with an image that I will give as a parameter, but I am very new to this subject, can you help me with this?</p>
Python os.fork with pytorch inside docker with user or read-only flags

username: Anton_Nenaidesh
cooked: <p>Hello. I have a Gunicorn application running in a sync mode. Firstly, I load models into memory via share_module().<br>
Then Gunicorn forks the main process, and I additionally fork Gunicorn workers to achieve parallelism on the request level.<br>
Everything works locally and in Docker. But once I enable docker read-only mode (give a RW access to /var/tmp) (this is a security requirement), the state of models (state_dict) can sometimes be corrupted during os.fork(). By corrupted I mean the state_dict coefficients are changed randomly which leads to incorrect model output.<br>
I experience this behavior with both file_descriptor and file_system sharing strategies.<br>
How can I use these Docker flags and fork processes with PyTorch safely?<br>
I increased shm_size parameter in Docker so this is not a bottleneck.<br>
I use only CPU calculations.</p>
Error in comfyUI, Onxx, workflow is very slow

username: Kevin_Ceulemans
cooked: <p>Before posting a query, check the <a href="https://discuss.pytorch.org/c/faq/30">FAQs</a> - it might already be answered!<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/6/26cefb0d0ff11d4575478173034a8f0d44a2dc45.png" data-download-href="https://discuss.pytorch.org/uploads/default/26cefb0d0ff11d4575478173034a8f0d44a2dc45" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/2/6/26cefb0d0ff11d4575478173034a8f0d44a2dc45.png" alt="image" data-base62-sha1="5xjB4nsoVtIjt4PDbcYjvcXMbxH" width="690" height="145" data-dominant-color="1B1B1B"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1917×404 32.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
can you help me out with this, i don t know what to do.</p>
Difference between `torch.linalg.eigvals` and `scipy.linalg.eigvals`

username: shc443
cooked: <p>Hello I am currently sampling data from different kernels (Gaussian, Laplacian, Matern, etc)</p>
<p>The issue is that <code>torch.distributions.multivariate_normal.MultivariateNormal</code> raises a <code>ValueError</code> saying that my kernel is not <code>PositiveDefinite()</code></p>
<p>I check my kernel using <code>torch.linalg.eigvals</code>, and some of the eigenvalues are actually negative (and complex too)</p>
<p>The weird thing is that all the eigenvalues I computed from <code>scipy.linalg.eigvals</code> are all real and positive.</p>
<p>I just read a page here saying about rounding floating numbers, but which one I should believe?</p>
<p>Thank you for your time and help in advance <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=12" title=":smiley:" class="emoji" alt=":smiley:" loading="lazy" width="20" height="20"></p>

username: KFrank
cooked: <p>Hi shc!</p>
<aside class="quote no-group" data-username="shc443" data-post="1" data-topic="194186" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/258eb7/48.png" class="avatar"> shc443:</div>
<blockquote>
<p>The weird thing is that all the eigenvalues I computed from <code>scipy.linalg.eigvals</code> are all real and positive.</p>
</blockquote>
</aside>
<p>Scipy is most likely computing the eigenvalues in double precision, while<br>
pytorch is most likely doing so in single precision.  Try performing your<br>
pytorch in double precision and see if that resolves (or at least reduces)<br>
your issue.</p>
<p>If not, please post a simple, fully-self-contained, runnable script that<br>
reproduces this discrepancy, together with the output you get when you<br>
run it.</p>
<p>Best.</p>
<p>K. Frank</p>

username: shc443
cooked: <p>Thank you so much.<br>
I didn’t know about precision.<br>
I actually solved this issue with Tikhonov regularization: adding <code>np.finfo(np.float32).eps</code> on diagonal entries <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=12" title=":smiley:" class="emoji" alt=":smiley:" loading="lazy" width="20" height="20"></p>
Sparse matrix acceleration

username: iminfine
cooked: <p>I am currently engaged in the development of a novel cross-attention mechanism for transformers. In this context, consider the shape of q and k as (b, n, l, d), where b represents the batch number, n denotes the head number, l signifies the sequence length, and d stands for the feature dimensions.</p>
<p>The code mechanism employed to compute attention involves the application of the F.relu activation function to both q and k, followed by reshaping operations. Specifically, it can be expressed as follows:</p>
<p>F.relu(q).reshape(b, n * l, d) @ F.relu(k).reshape(b, n * l, d)</p>
<p>However, the reshape operation results in the multiplication of two large matrices, incurring a substantial GPU memory cost. This presents a notable drawback in terms of computational efficiency and resource utilization.</p>
<p>The aforementioned attention function can be conceptualized as the multiplication between two sparse matrices, considering the involvement of the ReLU function.<br>
To mitigate the memory consumption issue, an alternative approach could be explored: the possibility of performing Batched Sparse Matrix - Batched Sparse Matrix multiplication.</p>
<p>I am curious about your thoughts on whether this approach seems feasible.</p>
How to use 0-3 class of number in MNIST

username: jianyikele
cooked: <p>I would like to know how to use parts 0-3 of the mnist dataset so that the output neurons of the fully connected neural network end up at 4。</p>
<p>here is my code</p>
<p>trans = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,))])<br>
train_set = datasets.MNIST(‘…/data’, train=True, transform=trans, download=True)</p>
<p>test_set = datasets.MNIST(‘…/data’, train=False, transform=trans, download=True)</p>
<p>train_loader = torch.utils.data.DataLoader(train_set, batch_size=batch_size, shuffle=True)<br>
test_loader = torch.utils.data.DataLoader(test_set, batch_size=batch_size, shuffle=True)</p>
<p>Please help me fix my code</p>

username: astle_dsa
cooked: <p>Can you clearly explain your question ? Do you want the outputs of the network to be from 0 to 4 ?</p>

username: jianyikele
cooked: <p>I want to validate 0-3 in the mnist dataset, not 0-9.</p>

username: astle_dsa
cooked: <p>You’ll first need to remove the data of other remaining digits from the dataset. After that, simply set the last or the output layer to give four outputs, each corresponding to a digit.</p>

username: jianyikele
cooked: <p>sorry，i dont know how to remove</p>

username: ptrblck
cooked: <p>Take a look at <a href="https://discuss.pytorch.org/t/swapping-labels/154011/2">this post</a> which explains you could directly manipulate the internal targets and corresponding data.</p>
Jit tracing failing (torch mangling diverging operator)

username: tgangs
cooked: <p>Hello,<br>
While trying to jit trace a model with:</p>
<pre><code class="lang-auto">torch.jit.trace(model, example_inputs, check_inputs)
</code></pre>
<p>I get:</p>
<pre><code class="lang-auto">First diverging operator:
Node diff:
- %text_encoder : __torch__.attention_encoder.AttnEncoder = prim::GetAttr[name="text_encoder"](%self.1)
+ %text_encoder : __torch__.attention_encoder.___torch_mangle_80.AttnEncoder = prim::GetAttr[name="text_encoder"](%self.1)
</code></pre>
<p>Could someone assist me in understanding the meaning of this error and guide me on the steps to further debug it?</p>
<p>I’m using torch <code>1.12.0+cu113</code>.</p>
<p>Thanks.</p>
How can I replace FFN with LSTM?

username: ashok_arora
cooked: <p>Inside the Transformer encoder, there’s a FFN which I want to replace with LSTM to check performance improvements, how should I go about it?</p>
<p>The transformer inputs are of the form torch.Size([16, 12, 170, 152]) → (batch, timestamps, sensors, embedding_dim) but LSTM only works for 3D data so what would be the proceed?</p>
<p>Should I make the 4d data to 3d for LSTM by multiplying the indices 0 and 1 giving (batch * timestamps, sensors, embedding_dim) and then back to 4d for the rest of the model or is there <code>keras.layers.TimeDistributed</code> or try something else?</p>

username: vdw
cooked: <p>Hm, the expected input for the transformer is typically 3d: <code>(batch_size, seq_len, embed_size)</code>, which is also the input and output for the FFN layer. So this would be perfectly valid input for a <code>nn.LSTM</code> with <code>batch_first=True</code>.</p>
<p>I’m not sure how you put 4d data into a transformer.</p>
<p>That being said, one of the motivation behind the transformer architecture was to get rid of recurrences to yield a better parallelization. So it’s not obvious why you want to bring them back :).</p>

username: ashok_arora
cooked: <p>in my case, the <code>seq_len</code> is a 2D tensor since its a spatio-temporal dataset, where the sensors are located at different spots and each collects data at periodic intervals so that adds another dimension making the input 4D.</p>
<p>The primary motivation was to understand the impact of the FFN layer to the overall accuracy and then to see the impact of replacing it with other neural architectures like LSTM and GRU.</p>

username: vdw
cooked: <p>So can you already train an basic transformer with this input without any changes to the FFN layer. And if so, how?</p>

username: ashok_arora
cooked: <p>Yeah, the basic transformer is working and producing results. Here’s the link to the model code: <a href="https://github.com/XDZhelheim/STAEformer/blob/main/model/STAEformer.py" rel="noopener nofollow ugc">https://github.com/XDZhelheim/STAEformer/blob/main/model/STAEformer.py</a></p>

username: vdw
cooked: <p>Ah, OK… you’re already have your custom transformer implementation. To be honest, I’m not really sure how transformer semantics transfers from 3d to 4d input data.</p>

username: ashok_arora
cooked: <p>I think for most of the spatio-temporal tasks or video stuff, the transformer would have to deal with 4d input data. Would you suggest converting 4d input to 3d for transformers?</p>
Model visualization for covariance between input variables

username: hawkiyc
cooked: <p>Hi,</p>
<p>I have built a VAE model dealing with tabular oceanographical data. My question is, what shall I do to<br>
visualize model weight and clarify the covariance relationship between input variables, as shown in the following chart, with tensorboard or weights and biases.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/e/6e62fb7ed0d4953f0cef7ffcecb1d94b439633ad.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/6e62fb7ed0d4953f0cef7ffcecb1d94b439633ad" title="截圖 2023-12-24 16.39.11"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/e/6e62fb7ed0d4953f0cef7ffcecb1d94b439633ad_2_690x455.jpeg" alt="截圖 2023-12-24 16.39.11" data-base62-sha1="fKwAC9IRwTyTfoflDynA8BewtFb" width="690" height="455" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/e/6e62fb7ed0d4953f0cef7ffcecb1d94b439633ad_2_690x455.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/6/e/6e62fb7ed0d4953f0cef7ffcecb1d94b439633ad_2_1035x682.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/6/e/6e62fb7ed0d4953f0cef7ffcecb1d94b439633ad_2_1380x910.jpeg 2x" data-dominant-color="41484D"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">截圖 2023-12-24 16.39.11</span><span class="informations">1920×1267 144 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p><a href="https://playground.tensorflow.org/?source=post_page-----9f6a1d8eb57a--------------------------------#activation=tanh&amp;batchSize=10&amp;dataset=xor&amp;regDataset=reg-plane&amp;learningRate=0.03&amp;regularizationRate=0&amp;noise=0&amp;networkShape=4,2,2&amp;seed=0.88573&amp;showTestData=true&amp;discretize=true&amp;percTrainData=50&amp;x=true&amp;y=true&amp;xTimesY=false&amp;xSquared=false&amp;ySquared=false&amp;cosX=false&amp;sinX=false&amp;cosY=false&amp;sinY=false&amp;collectStats=false&amp;problem=classification&amp;initZero=false&amp;hideText=false" rel="noopener nofollow ugc">Here is the website of the screenshot.</a></p>
<p>Please note that I don’t need it to be dynamic; I only need to visualize the final model.</p>
<p>Sincerely,</p>
How to plot seed_iv

username: user1997
cooked: <p>Does anyone know how to create a Avarage TF fplot after importing the seed_iv dataset like this:</p>
<p>dataset = SEEDIVDataset(io_path=f’./seed_iv’,<br>
root_path=‘./eeg_raw_data’,<br>
online_transform=transforms.Compose([<br>
transforms.ToTensor(),<br>
transforms.To2d()<br>
]),<br>
label_transform=transforms.Select(‘emotion’))<br>
print(dataset[0])</p>
<h1><a name="eeg-signal-torchtensor62-200-1" class="anchor" href="#eeg-signal-torchtensor62-200-1"></a>EEG signal (torch.Tensor[62, 200]),</h1>
<h1><a name="coresponding-baseline-signal-torchtensor62-200-2" class="anchor" href="#coresponding-baseline-signal-torchtensor62-200-2"></a>coresponding baseline signal (torch.Tensor[62, 200]),</h1>
<h1><a name="label-int-3" class="anchor" href="#label-int-3"></a>label (int)</h1>
Does pytorch work with python 3.12.0?

username: bradbell
cooked: <p>I am getting the following response when I try to install pytorch:</p>
<pre><code class="lang-auto">&gt; pip install torch --user
ERROR: Could not find a version that satisfies the requirement torch (from versions: none)
</code></pre>
<p>Here is my system information</p>
<pre><code class="lang-auto">&gt;uname -a
Linux fedora 6.6.6-200.fc39.x86_64 #1 SMP PREEMPT_DYNAMIC Mon Dec 11 17:29:08 UTC 2023 x86_64 GNU/Linux
&gt;python --version
Python 3.12.0
</code></pre>

username: ptrblck
cooked: <p>I don’t believe the binaries are built for Python 3.12, but you might be able to build PyTorch from source for it as described in <a href="https://github.com/pytorch/pytorch/issues/110436">this tracking issue</a>.</p>
Idiom classification based on cosine similarity

username: filic
cooked: <p>Hi! I am trying to create a model that takes an input sentence which includes a multi word expression (MWE), produce a vector representation for both the MWE and its context and then based on their cosine similarity predicts if the sentence is idiomatic or not. To encode the expression I tried to take the sum of the individual words’ representation and for the context I am taking the representation of the [CLS] token. However I am running into some mat shape multiplication errors. What am I doing wrong here? Am I not calculating the cosine correctly? In my head everything is clear, but I am not that experienced with torch and transformers <img src="https://discuss.pytorch.org/images/emoji/apple/frowning.png?v=12" title=":frowning:" class="emoji" alt=":frowning:" loading="lazy" width="20" height="20"> Any help would be greatly appreciated!</p>
<pre data-code-wrap="class"><code class="lang-class">    def __init__(self, labels):
        super(DUAL_ENCODER, self).__init__()
        self.bert_model = DistilBertModel.from_pretrained("bert-base-multilingual-cased")
        self.classifier = nn.Linear(self.bert_model.config.hidden_size, labels)
        
    def forward(self, mwe_input_ids, mwe_attention_mask, context_input_ids, context_attention_mask):
        context = self.bert_model(mwe_input_ids, mwe_attention_mask)
        mwe = self.bert_model(context_input_ids, context_attention_mask)
        #extract hidden states
        #sum words of expression excluding the [CLS] and [SEP] tokens
        mwe_hidden_state = mwe.last_hidden_state[:, 1:-1, :].sum(dim=1)
        context_hidden_state = context.last_hidden_state[:, 0, :] #take the [CLS] token as the whole sentence representation
        print('mwe', mwe_hidden_state.size(), 'context', context_hidden_state.size())
        similarity = torch.cosine_similarity(mwe_hidden_state, context_hidden_state, dim=1)
        print('similarity', similarity.unsqueeze(1).size())
        predictions = self.classifier(similarity.unsqueeze(1))

        return predictions ```

RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-37-d5608ed79afa&gt; in &lt;cell line: 3&gt;()
     14         labels = batch['labels']
     15 
---&gt; 16         outputs = model(mwe_ids, mwe_attn_mask, context_ids, context_attn_mask)
     17         loss = loss_fn(outputs.squeeze(), labels)
     18         loss.backward()

5 frames
/usr/local/lib/python3.10/dist-packages/torch/nn/modules/linear.py in forward(self, input)
    112 
    113     def forward(self, input: Tensor) -&gt; Tensor:
--&gt; 114         return F.linear(input, self.weight, self.bias)
    115 
    116     def extra_repr(self) -&gt; str:

RuntimeError: mat1 and mat2 shapes cannot be multiplied (4x1 and 768x4631)</code></pre>
Make a GRU that is interchangable with an LSTM

username: andrewwarrington
cooked: <p>Hi folks,</p>
<p>I might’ve missed the point here, but i am experimenting with using a GRU or LSTM. It is kind of annoying in the downstream applications to need different code paths for if the state is a tuple (LSTM) or a tensor (GRU) (<a href="https://discuss.pytorch.org/t/how-to-change-gru-to-lstm-in-chatbot-tutorial/30417">see this discussion</a>) .  I have started using the following class which wraps the call to the GRU to make it mimic the return signature of the LSTM (so-called GRU-mimic or GRUm).</p>
<p>I’m not sure if there is a better way of doing this.  The code is in a colab <a href="https://colab.research.google.com/drive/1odnu82cxJ5Bd1pFG2SaarK-7JSlzS5r_?usp=sharing" rel="noopener nofollow ugc">here</a>.  It passes tests suggesting that the GRU is unmodified, but matches the signature of the LSTM.</p>
<p>It does incur some extra memory storing the (unused) cell states, but it is so much easier because which entries are relevant can be decided downstream.</p>
<p>Thanks,<br>
Andy</p>
<pre><code class="lang-auto">class GRUm(torch.nn.GRU):
    """
    Define a GRU cell that mimics the signature of an LSTM (_GRUm_).

    This only requires a wrapper of the forward method to strip and then re-add a cell state. 
    """

    def forward(self, input, hxcx=None):

        if hxcx is not None:
            hx, cx = hxcx
        else:
            hx = None
            cx = None

        output, hx = super().forward(input, hx)

        if cx is None:
            cx = torch.zeros_like(hx).to(hx.device)

        return output, (hx, cx)
</code></pre>

username: vdw
cooked: <p>Although it probably doesn’t fit your use case: I often built “customizable” models where I can decide to use an LSTM or a GRU using an input parameter.</p>
<p>Here is an <a href="https://github.com/chrisvdweth/nus-cs4248x/blob/master/3-neural-nlp/src/rnn.py" rel="noopener nofollow ugc">example</a>; search for class <code>RnnTextClassifier</code>. Based on a simple string input parameter the classifier either uses an <code>nn.RNN</code>, <code>nn.LSTM</code>, or <code>nn.GRU</code>. The different outputs are he handled within the <code>forward()</code> method, most basically if the hidden state is a tensor (<code>nn.RNN</code>, <code>nn.GRU</code>) or a tuple (<code>nn.LSTM</code>).</p>
Torch.cuda.memory._dump_snapshot() not activating on second iteration of model.py

username: LUE
cooked: <p>I’m doing a research project built off of Andrej Karpathy’s nanoGPT. I’ve run into a GPU memory problem and I’m trying to use torch.cuda.memory._dump_snapshot() to find the error. “model.py” is called in a loop in “train.py” as “train.py” loads the data. The problem is that while torch.cuda.memory._dump_snapshot() works on the first iteration of the “model.py” it doesn’t seem to do anything on the iteration causing the crash, so I can’t see the problem.</p>
<p>I’ve tried only loading torch.cuda.memory._record_memory_history(max_entries=100000) after a certain number of iteration, but that seems to have no effect.</p>
<p>Any help would be greatly appreciated.<br>
Thank you,<br>
Leigh</p>

username: LUE
cooked: <p>My solution was to not call estimate_loss() and call torch.cuda.memory._record_memory_history(max_entries=100000) before every micro_step and torch.cuda.memory._record_memory_history(enabled=None) after every micro_step. This allowed me to see what was going on inside my GPU. However, I’m still not sure I could produce the memory dumps after further calls to model().</p>
HIP error while using rocm

username: BassTurtle4
cooked: <p>My code:</p>
<pre data-code-wrap="from"><code class="lang-from">
sequences = df['tweetV2'].to_list()

candidate_labels = ["positive", "neutral", "negative"]

hypothesis_template = "The sentiment of this review is {}."

predictions = []

for sequence in tqdm(sequences, desc="Classifying Sentiments"):
    result = classifier(sequence, candidate_labels, hypothesis_template=hypothesis_template)
    predictions.append(result)

hf_prediction = pd.DataFrame(predictions)
</code></pre>
<p>always outputs this:</p>
<pre><code class="lang-auto">RuntimeError: HIP error: invalid device function
Compile with `TORCH_USE_HIP_DSA` to enable device-side assertions.
</code></pre>
<p>Im not sure what is going wrong, this works fine with cpu but for some reason never works with the rocm… Stable diffusion works using rocm and i’m not sure why this won’t.</p>
<p>Any help would be appreciated.</p>
<p>(edit)<br>
I changed from using rocm 5.7 to 5.6 and got the same error when i went to 5.5 i just got</p>
<p><code>Classifying Sentiments:   0%|                        | 0/116174 [00:00&lt;?, ?it/s]Segmentation fault (core dumped)</code></p>
<p>This used to work and im not sure why it doesn’t work anymore…</p>

username: a.shkorinova
cooked: <p>Seems like the problem is connected to the GPU and PCIe configuration<br>
<a href="https://rocm.docs.amd.com/projects/radeon/en/latest/docs/limitations.html#errors-due-to-gpu-and-pcie-configuration" class="onebox" target="_blank" rel="noopener nofollow ugc">https://rocm.docs.amd.com/projects/radeon/en/latest/docs/limitations.html#errors-due-to-gpu-and-pcie-configuration</a></p>

username: BassTurtle4
cooked: <p>That is not an option with my current configuration, using torch for rocm 5.2 worked just fine when running rocm 5.7 on my machine.</p>
Tensors - merge tensors in specific order

username: eugenio_b
cooked: <p>Hi!<br>
I have a dataset wrap where patterns are organized in two contiguous torch.tensor.<br>
The problem is I have to keep the order of indices<br>
suppose dataset A with indices from 0 to 3 and dataset B with indices from 4 to 7<br>
I need idx (as list)<br>
idx_list = [3,6,2,4,7]<br>
So not only interleaved between dataset but with indices in random order and order must be kept, i need as return value<br>
tensor = [A(3), B(6), A(2), B(4), B(7)]<br>
is there an efficient way to index, because the implementation is quite simple but, as shown, involve a for loop</p>
<pre><code class="lang-auto">i_patter_A = [value for value in idx_list if value &lt; len_dataset_A] # 2.1) split list based on index (discriminate values from real or fake list)
i_patter_B = [value for value in idx_list if value &gt;= len_dataset_A] 
batch_pattern = torch.zeros(len(i_patter_A)+len(i_patter_B)
for i,curr_pattern in enumerate(idx_list):
    if curr_pattern &lt; len_dataset_A:
        batch_pattern[i] = self.dataset[self.indices[curr_pattern]]
    else:
        batch_pattern[i] = self.get_generate_data(curr_pattern)                    
return batch_pattern
</code></pre>
<p>for me it’s a schoolbook-solution but does not take advantages of tensor indexing</p>
<p>many thanks</p>
<p>Eugenio</p>
Is batching only once legitimate?

username: Viktor_Todosijevic
cooked: <p>Hi,</p>
<p>I’m dealing with data that batches really slowly (I’m dealing with graphs and sparse matrices).</p>
<p>I was wondering if it’s okay/won’t hurt performance if I batch my data once before training and just feed the model those same batches in a random order, as depicted bellow:</p>
<pre data-code-wrap="python"><code class="lang-python">    loader = DataLoader(my_data, batch_size=batch_size, shuffle=True)
    batches = [batch for batch in loader]
    for epoch in range(num_epochs):
        random.shuffle(batches)
        for batch in batches:
            batch = batch.clone()
            batch = batch.to(device)
            ...
</code></pre>
<p>Is this an okay thing to do?</p>

username: KFrank
cooked: <p>Hi Viktor!</p>
<aside class="quote no-group" data-username="Viktor_Todosijevic" data-post="1" data-topic="194167" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/viktor_todosijevic/48/49306_2.png" class="avatar"> Viktor_Todosijevic:</div>
<blockquote>
<p>I was wondering if it’s okay/won’t hurt performance if I batch my data once before training and just feed the model those same batches in a random order</p>
</blockquote>
</aside>
<p>This should be fine.  Because you select your batches randomly (even though<br>
you only do this once) and in each epoch you iterate through your batches in<br>
a different order, you should be almost as well off as if you had selected new<br>
random batches for every epoch.</p>
<p>As an aside, why is batching your data so slow?  Do you have an expensive<br>
<code>collate_fn()</code> in your pipeline?  Even if preparing individual samples is<br>
expensive, you could do that part just once and then randomly batch the<br>
already-prepared samples together for each epoch using, for example,<br>
<code>stack()</code>.  That should be altogether pretty cheap.</p>
<p>Best.</p>
<p>K. Frank</p>