akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Custom dataset raising not implemented errors on __getitem__

malice: <p>Hello!</p>
<p>I’m not trying to do anything fancy, so I’m quite confused about what’s going wrong. I’m loading time series data from a csv into a custom Dataset, but <strong>getitem</strong> is raising NotImpementedErrors. Here’s the Dataset creation code:</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
import numpy as np

# Data Loading stuff
import pandas as pd
from torch.utils.data import DataLoader, Dataset




class WeatherData(Dataset):
    def __init__(self, csv_file, window, transform = None):
        self.data = pd.read_csv(csv_file)
        self.window = window
        self.transform = transform
        
    def __getitem__(self, idx):
        if torch.is_tensor(idx):
            idx = idx.tolist()
        x, y = (self.data.iloc[idx:idx+self.window, 1:].values, self.data.iloc[idx+1:idx+self.window+1, 1:].values)
        if self.transform != None:
            x, y = self.transform(x), self.transform(y)
        return x, y
    
    def __len__(self):
        return len(self.data) - self.window - 1
    
class ToTensor(object):
    def __call__(self, sample):
        return torch.tensor(sample, dtype = torch.long)
</code></pre>
<p>I’m then running this code to test the dataset:</p>
<pre><code class="lang-auto">mydataset = WeatherData('data/weather.csv', 30, transform=ToTensor)

l = len(mydataset)
print(l)

for i, s in enumerate(mydataset):
    print(i, s)
    if i ==2:
        break
</code></pre>
<p>It gives the following output/error message:</p>
<pre><code class="lang-auto">52665

---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
Input In [41], in &lt;cell line: 12&gt;()
      9 l = len(mydataset)
     10 print(l)
---&gt; 12 for i, s in enumerate(mydataset):
     13     print(i, s)
     14     if i ==2:

File ~/.local/lib/python3.10/site-packages/torch/utils/data/dataset.py:53, in Dataset.__getitem__(self, index)
     52 def __getitem__(self, index) -&gt; T_co:
---&gt; 53     raise NotImplementedError

NotImplementedError: 
</code></pre>
<p>So as far as I can tell, the data loading is fine, but something about my __getitem__ function is broken. Thanks for any help!</p>

malice: <p>Update: I was able to get my code to work, though I don’t understand what was wrong to begin with. Before, I defined WeatherData in its own .py file, and imported it with</p>
<pre><code class="lang-auto">from weatherdataloading import WeatherData
</code></pre>
<p>Indeed, any data class I define in a separate file and try to import seems to give me the same error. However, creating the dataloading class in the main file and otherwise running without change seems to work fine.</p>
Parallel training Multiple Models on Multiple GPUs with different data

clueless_coder: <p>Hello,</p>
<p>I have a working NN that simply trains to optimize a set of variables given some input data. I have hundreds of sets of data, and so far have been training each instance sequentially using a for loop. I was wondering if there’s something similar to parfor function in Matlab, where I can train multiple separate models in parallel, each on its own GPU, given its own separate data? Each model is independent, I don’t need them to update each other or communicate in anyway to each other. I simply have one machine, multiple gpus, and want each gpu to train its own model, save it once it’s done, and start training the next model in the for loop.</p>
<p>Thanks in advance.</p>
Dynamo performance

djelovina: <p>Hi all,<br>
I am trying to get the performance breakdown in dynamo/huggingface.py on cpu. Is there any simple way to reach situation where speedups are all 1, i.e. when all dynamo optimizations are disabled?<br>
It seems that disabling fusion (can_fuse always returns false) , does not play a major role for cpu.</p>
<p>Thanks in advance!</p>
About Quantization using torch.ao.quantization

RonaldYuren: <p>I am implementing post-training static Quantization in pytorch 2.1.0.<br>
While I can’t get the quantized model.  Below are my quantization step:<br>
##########<br>
def static_quantize(weight_path, qconfig_mapping):<br>
weights, imgsz = opt.weights, opt.img_size</p>
<pre><code># load Pruned_model
# 
device = select_device(opt.device)
device = torch.device("cpu")
# Load model
pruned_model = attempt_load(weights, map_location=device)  # load FP32 model

stride = int(pruned_model.stride.max())  # model stride
imgsz = check_img_size(imgsz, s=stride)  # check img_size
pruned_model.eval()
if weight_path != '1234':  # get pruned_model's weights if path is not default "1234" 
    pruned_model.load_state_dict(torch.load(weight_path),False)   #load pruned_model.pt


#  "input"  "output":  Add  QuantStub , DeQuantStub

pruned_model = QuantStub()(pruned_model)
pruned_model = DeQuantStub()(pruned_model)


# 
pruned_model = prepare(pruned_model, qconfig_mapping)
quantized_model = convert(pruned_model, inplace=False)

# save model's weights
torch.save(pruned_model.state_dict(), "static_quantized_original.pt") 
print("Static Quantization Success!!!!")    

return quantized_model
</code></pre>
<p>############<br>
and I use the function below to check:<br>
############<br>
def check_quantization(model, output_file):<br>
num_modules_to_print = 5<br>
with open(output_file, ‘w’) as f:<br>
num_printed_modules = 0<br>
for name, module in model.named_modules():<br>
if isinstance(module, nn.Conv2d):<br>
f.write(f"Module {name} is a Conv2d layer.\n")<br>
f.write(f"Quantized: {isinstance(module, nnq.Conv2d)}\n\n")<br>
elif isinstance(module, nn.Linear):<br>
f.write(f"Module {name} is a Linear layer.\n")<br>
f.write(f"Quantized: {isinstance(module, nnq.Linear)}\n\n")<br>
# Add more checks for other quantized layers if needed</p>
<pre><code>        # Print some example layers for inspection
            if num_printed_modules &lt; num_modules_to_print:
                f.write(f"+++++++++Module {name}:\n")
                f.write(f"+++++++++{module}\n\n")
                num_printed_modules += 1
        # Add more checks for other quantized layers if needed
</code></pre>
<p>##############<br>
I always get " Module model.5.conv is a Conv2d layer.<br>
Quantized: False "<br>
Please tell me how to solve this problem, Thank you</p>
Keypoint rcnn training error

magnus11: <p>I get this strange error when I run my code for training a Keypoint RCNN FPN with a ResNet50 backbone:</p>
<pre><code class="lang-auto">Exception has occurred: IndexError
index 3 is out of bounds for dimension 0 with size 1
  File "keypointrcnn.py", line 89, in &lt;module&gt;
    outputs = model(images, targets)
IndexError: index 3 is out of bounds for dimension 0 with size 1
</code></pre>
<p>Some context: I have dataset where every image is taken of one individual which is annotated with exactly 5 bounding boxes and 8 key points. The following dimensions for the targets[‘boxes’] is [5,4], targets[‘labels’] is [5], targets[‘keypoints’] is [1,8,3].</p>
<p>Here is the code for fetching the model:</p>
<pre><code class="lang-auto">def getkeypointmodel(num_classes, num_keypoints):
    model = torchvision.models.detection.keypointrcnn_resnet50_fpn(pretrained=False,
                                                                   pretrained_backbone=True,
                                                                   num_keypoints=num_keypoints,
                                                                   num_classes = num_classes)
    return model
</code></pre>
<p>Below is the training loop for my model:</p>
<h1><a name="train-the-model-1" class="anchor" href="#train-the-model-1"></a>Train the model</h1>
<pre><code class="lang-auto">device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')
model.to(device)
num_epochs = 1
for epoch in range(num_epochs):
    for images, targets in data_loader_training:
        optimizer.zero_grad()
        images = [image.to(device) for image in images]
        targets = [{k: v.to(device) for k, v in t.items()} for t in targets]

        img = images[0].shape
        tb = targets[0]['boxes'].shape
        tl = targets[0]['labels'].shape
        tk = targets[0]['keypoints'].shape

        # Forward pass
        outputs = model(images, targets)
        loss_dict = outputs['losses']
        losses = sum(loss for loss in loss_dict.values())

        # Backward pass and optimization
        losses.backward()
        optimizer.step()

    # Update the learning rate
    lr_scheduler.step()
</code></pre>
<p>Hope some can explain this strange error message to me.</p>
<p>Thanks in advance!</p>
<p><code>&gt; Preformatted text</code></p>

rounak: <p>Actually the torchvision.models.detection.keypointrcnn_resnet50_fpn is expecting the input to be a list of 2 tensors of shape (3, 300, 400) while you are providing it two lists of some shape to it , so it is causing error as in the list named <strong>images</strong> i.e. the 0th dimension of input to model, don’t have anything at index 3.</p>
<p>so please modify the shape of input you are giving to model</p>
<p>i hope it helps you</p>

magnus11: <p>Hmmm, I think you are talking about passing it a batch of image tensors, well I am doing that, but during training it also needs to be passed a list of targets as well. So I don’t think merging the images tensors with the targets is the correct thing to do unfortunately…</p>
In working level, How do I convert pyotrch to C++

learningsteady0J0: <p>Hi guys, I’m looking for a job, so I’m practicing converting PyTorch to C++. I wonder how to convert PyTorch at a working level. I found methods like TorchScript, LibTorch, and ONNX, but I don’t know which one to use.<br>
I want to know what to practice<br>
I need to know when to use each method.</p>
<p>Thank you for reading!</p>
Wrong result of torch.nn.BCEWithLogitsLoss()

Veter: <pre><code class="lang-auto">target = torch.tensor([[1.0, 0.0]])
pred = torch.tensor([[0.85, 0.15]])  # A prediction (logit)

criterion = torch.nn.BCEWithLogitsLoss()
print(criterion(pred, target).item())
</code></pre>
<p>Shows: 0.5634111166000366</p>
<p>Should be:</p>
<pre><code class="lang-auto">tSigm = nn.Sigmoid()

t2 = torch.tensor(0.85)

print( -1*math.log(tSigm(t2).item())-0.0*math.log(1-tSigm(t2).item()))
</code></pre>
<p>0.3558650064855657</p>

ptrblck: <p>The loss calculation is not wrong according to the <a href="https://pytorch.org/docs/stable/generated/torch.nn.BCEWithLogitsLoss.html#bcewithlogitsloss">docs</a> and as seen in this example, which uses the “logits” (which looks like probabilities) for both classes in this multi-class example:</p>
<pre data-code-wrap="python"><code class="lang-python">target = torch.tensor([[1.0, 0.0]])
pred = torch.tensor([[0.85, 0.15]])  
criterion = torch.nn.BCEWithLogitsLoss(reduction="none")
print(criterion(pred, target))
# tensor([[0.3559, 0.7710]])


print((-1.0)*torch.log(torch.sigmoid(pred[0, 0])))
# tensor(0.3559)
print((-1.0)*torch.log(1-torch.sigmoid(pred[0, 1])))
# tensor(0.7710)

# mean reduction
criterion = torch.nn.BCEWithLogitsLoss()
print(criterion(pred, target))
# tensor(0.5634)

print(((-1.0)*torch.log(torch.sigmoid(pred[0, 0])) + (-1.0)*torch.log(1-torch.sigmoid(pred[0, 1]))) / 2.)
# tensor(0.5634)
</code></pre>

Veter: <p>Thank you!</p>
<p>I am used to the fact that entropy is calculated as the sum of all -p*log(p) and did not take into account that for binary entropy one probability is enough.</p>
Is there any way to use augmentations on images when number of channels is not 1, 3 or 4?

chinmaester: <p>I have a dataset that has 16 channels. I want to use augmentations like crop and resize. Only problem is that torch expects images in PIL format that only supports upto 4 channels.</p>

ptrblck: <p><code>torchvision.transform</code> accepts tensors, too, and should work on image tensors with 16 channels:</p>
<pre data-code-wrap="python"><code class="lang-python">x = torch.randn(16, 224, 224)
transform = transforms.Resize((200, 200))
out = transform(x)
print(out.shape)
# torch.Size([16, 200, 200])
</code></pre>

chinmaester: <p>Thank you so much! It worked like a charm!</p>
ADAM optimizer - initial condition for minimization

hansgiel: <p>I am new to Python and Machine Learning in general. This question was raised for the case of a Gaussian Process training but I guess it is also applicable to Neural Networks.</p>
<pre><code class="lang-auto"># Find optimal model hyperparameters
model.train()
likelihood.train()

# Use the adam optimizer
optimizer = torch.optim.Adam(model.parameters(), lr=0.1)  # Includes GaussianLikelihood parameters

# "Loss" for GPs - the marginal log likelihood
mll = gpytorch.mlls.ExactMarginalLogLikelihood(likelihood, model)

for i in range(training_iterations):
    optimizer.zero_grad()
    output = model(train_x)
    loss = -mll(output, train_y)
    loss.backward()
    print('Iter %d/%d - Loss: %.3f' % (i + 1, training_iterations, loss.item()))
    optimizer.step()
</code></pre>
<p>In a typical training process, we are trying to minimize the loss function (which in this case is the Marginal Log Likelihood). My question is, what is the initial condition for that minimization? How can I input/output it to the training loop? Apologies in advance if the question is dumb or unspecific enough, I am just trying to make sense of the training stage from a mathematical perspective</p>
How can I tell the difference between a regular tensor and a nested tensor?

Matt_Pitkin: <p>With the recently introduced <a href="https://pytorch.org/docs/stable/nested.html" rel="noopener nofollow ugc">nested tensors</a>, if I create a nested tensor:</p>
<pre data-code-wrap="python"><code class="lang-python">import torch

a = torch.randn(20, 128)
nt = torch.nested.nested_tensor([a, a], dtype=torch.float32)
</code></pre>
<p>and check its <code>type</code>:</p>
<pre><code class="lang-auto">type(nt)
torch.Tensor
</code></pre>
<p>it just appears to be a regular <code>Tensor</code> object. If in a code, I wanted to differentiate between a nested tensor and a regular tensor, how could I go about doing it as both <code>type(nt) == torch.Tensor</code> and <code>isinstance(nt, torch.Tensor)</code> will return <code>True</code>?</p>
<p>One way I thought of is to use the fact that (currently) the <code>size</code> method behaves differently, i.e., for a nested tensor it requires an argument otherwise it will raise a <code>RuntimeError</code>. So, I could do:</p>
<pre><code class="lang-auto">def is_nested_tensor(nt):
    if not isinstance(nt, torch.Tensor):
        return False

    try:
        # try calling size without an argument
        nt.size()
        return False
    except RuntimeError:
        return True

    return False
</code></pre>
<p>But is there a simpler way?</p>

ptrblck: <p>You could check the <code>is_nested</code> attribute:</p>
<pre data-code-wrap="python"><code class="lang-python">a = torch.randn(20, 128)
nt = torch.nested.nested_tensor([a, a], dtype=torch.float32)

print(a.is_nested)
# False

print(nt.is_nested)
# True
</code></pre>

Matt_Pitkin: <p>Great, thanks. That’s just what I was after!</p>
Expected Tensor but got Tuple

junjie1: <p>When using pytorch to process LSTM in C++, after putting the hidden layer and input together into the forward function, an error is reported:terminate called after throwing an instance of ‘c10::Error’<br>
what():  Expected Tensor but got Tuple<br>
Exception raised from reportToTensorTypeError at …/aten/src/ATen/core/ivalue.cpp:968 (most recent call first):<br>
frame <span class="hashtag-raw">#0:</span> c10::Error::Error(c10::SourceLocation, std::string) + 0x57 (0x7fd8c2ee1617 in /home/zjliu/demo_cuda/libtorch/lib/libc10.so)<br>
frame <span class="hashtag-raw">#1:</span> c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::string const&amp;) + 0x64 (0x7fd8c2e9c98d in /home/zjliu/demo_cuda/libtorch/lib/libc10.so)<br>
frame <span class="hashtag-raw">#2:</span> c10::IValue::reportToTensorTypeError() const + 0x58 (0x7fd8a75d6a38 in /home/zjliu/demo_cuda/libtorch/lib/libtorch_cpu.so)<br>
frame <span class="hashtag-raw">#3:</span> ./example-app() [0x40adf6]<br>
frame <span class="hashtag-raw">#4:</span> ./example-app() [0x406d11]<br>
frame <span class="hashtag-raw">#5:</span> __libc_start_main + 0xf5 (0x7fd855c8b555 in /lib64/libc.so.6)<br>
frame <span class="hashtag-raw">#6:</span> ./example-app() [0x405569]</p>
<p>This is the code:</p>
<pre><code class="lang-auto"> torch::Tensor hn1 = torch::zeros({1, 10}).to(device);
        torch::Tensor hn2 = torch::zeros({1, 5}).to(device);
        torch::Tensor hn3 = torch::zeros({1, 1}).to(device);
        torch::Tensor cn1 = torch::zeros({1, 10}).to(device);
        torch::Tensor cn2 = torch::zeros({1, 5}).to(device);
        torch::Tensor cn3 = torch::zeros({1, 1}).to(device);

   

            std::vector&lt;float&gt; input_data_values = input_data_values_matrix[i];

            torch::Tensor input_data = torch::from_blob(input_data_values.data(), {1, 1, 40}).to(torch::kFloat32).to(device);



            std::vector&lt;torch::jit::IValue&gt; inputs;


            inputs.push_back(input_data.to(device));
            inputs.push_back(hn1);
            inputs.push_back(cn1);
            inputs.push_back(hn2);
            inputs.push_back(cn2);
            inputs.push_back(hn3);
            inputs.push_back(cn3);

     
             torch::Tensor output  = module.forward(inputs).toTensor(); //error code

</code></pre>
<p>please tell me how to solve it</p>
Problem in implementing NeRF paper

rounak: <p>After understanding the <strong>NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis</strong> paper, i tried implementing it but while reading the <a href="https://github.com/yenchenlin/nerf-pytorch" rel="noopener nofollow ugc">github code</a>, i was not able to understand it and complete my code due to lack of proper documentation and explanations . Please suggest me ways how can i understand it or any video material for it.</p>
<p>Thank You</p>
Is it possible to deploy a model of C++ Frontend on to target board?

keep-beyond: <p>I have a model designed by PyTorch C++ front-end, after training and save the model parameters, can I deploy the model on to target board(e.g. NVIDIA Orin) directly by just cross-compiling the model. As I know, for a python PyTorch model, it is usually exported to onnx file and may be later export to tensorrt engine file, then using tensorrt runtime to do inference during deployment on board in C++. I just wonder if the model is already in C++, can I just use this C++ code during deployment on board with C++ without model exporting, serialization and deserialization, etc.</p>
Transformer model training

dmrangak: <p>hi ,<br>
I try to train a Transformer model. Code line<br>
<strong>loss.backward(retain_graph=True)</strong><br>
generate the runtime error.</p>
<p><strong>RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [2048, 768]], which is output 0 of AsStridedBackward0, is at version 2; expected version 1 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True)</strong></p>
<p>Thank you.</p>

ptrblck: <p>This error is most likely raised due to the usage of <code>retain_graph=True</code>. Could you explain why you are using it?<br>
If it was used as a workaround to mask another error, try to fix the original error first.</p>

dmrangak: <p>Thank you very much for your comments. I will do it.</p>
Multi GPU training with DistributedDataParallel fails after last epoch is done

YossiB: <p>I followed this tutorial to enable distributed training for my model (one machine with 2 GPU’s):<br>
<a href="https://pytorch.org/tutorials/intermediate/ddp_tutorial.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/tutorials/intermediate/ddp_tutorial.html</a></p>
<p>The high level structure of my train code is documented below :</p>
<pre><code class="lang-auto">def train(rank, config):
   # setup
   torch.distributed.init_process_group(
      backend='nccl', init_method='tcp://localhost:6666', 
      world_size=config['world_size'], rank=rank)

   # actual training code
   model = torch.nn.parallel.DistributedDataParellel(module=model, device_ids=[rank])
   dataloader = torch.utils.data.DataLoader(dataset, batch_size=4, 
      shuffle=False, sampler=torch.utils.data.distributed.DistributedSampler(dataset))
   for epoch in range(config['epochs']):
      ...
   print(f'Done - rank {rank}')

   # cleanup
   torch.distributed. destroy_process_group()

if __name__ == '__main__':
   config = load_config():
   torch.multiprocessing.spawn(
      train, args=(config,), nprocs=config['world_size'])
</code></pre>
<p>In some cases it runs successfully, but on other cases (with the exact same input arguments) it prints a long failure message after all epochs are done (I can see the Done message on all processes) :</p>
<pre><code class="lang-auto">PyThreadState_Clear: warning: thread still has a frame
*** Error in 'opt/conda/bin/python': corrupted size vs. prev_size: 0x00007f38c40008f0 ***
###### Backtrace: ######
....
###### Memory map: ######
....

Exception: process 1 terminated with signal SIGABRT
</code></pre>
Extract features by small version of Resnet18

thangcuoi: <p>I want to extract features after using a small version of ResNet-18 pretrained on ImageNet. My code is below:<br>
import torchvision.models as model<br>
model = model.resnet18(pretrained = True)<br>
model = nn.Sequential(*list(model.children())[:-1])</p>
<p>for input, target in train_loader:<br>
____with torch.no_grad():<br>
________output = model(input)<br>
________output = output.view(output.size(0),-1)</p>
<p>However, it seems do not match with the statement: “we use a smaller version of ResNet18, with three times less feature maps across all layers”<br>
Please guide me how can I use “a smaller version of Resnet18”?</p>
Deprecated 'pretrained' parameter in PyTorch torchvision models - Seeking generalized solution

HassanBinHaroon: <p>Hi PyTorch community,</p>
<p>I’ve been using the torchvision library to work with pre-trained models, and <strong>I noticed that the <code>pretrained</code> parameter is deprecated</strong> in recent versions. I am looking for a <strong>generalized solution</strong> that works for every model, as my <code>model_name</code> is dynamically configured.</p>
<p>In my code, I have been using the following line to instantiate the model:</p>
<p><strong>model = models.__ dict __ [model_name] (pretrained=True)</strong></p>
<p>I want to make sure I’m using the recommended generalized approach for loading pre-trained weights that accommodates dynamic model configurations. Could someone please guide me on the valid alternative to the deprecated <code>pretrained</code> parameter for dynamically configured model names?</p>
<p>Thank you in advance for your help!</p>

HassanBinHaroon: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> and <a class="mention" href="/u/j_johnson">@J_Johnson</a> please take a look at this question. Thanks!</p>
CUDA unhandled exception

Dirk10001: <p>Hello,<br>
I have a small net running in C++.<br>
Input and target defined as following:<br>
<code>auto input = torch::randn({ TRAIN_BATCH_SIZE, 1, IMAGE_SIZE, IMAGE_SIZE }).to(device);</code><br>
<code>auto target = torch::zeros(TRAIN_BATCH_SIZE, torch::kInt64).to(device);</code></p>
<p>Filling the data with memcpy or cudaMemcpy if using CUDA or not. With C++ the net converges. With CUDA I get an exception in <code>loss.backward()</code></p>
<pre><code class="lang-auto">// memcpy or cudaMemcpy
LoadEpoch(input, target, cvImage, cvResult, imgcount);

optimizer.zero_grad();   // zero the gradient buffers
auto output = net-&gt;forward(input);
auto loss = criterion(output, target);
loss.backward();
optimizer.step();
</code></pre>
<p>The layout of all tensors is the same using CUDA or not. Result of  <code>Forward()</code> works always and <code>NLLLoss</code>. Just <code>backward()</code> throws an exception</p>
<p>c10::Error address 0x0000005A15D6C770.</p>
<p>The net is set to device and the optimizer is built with net-&gt;parameters().</p>
<p>Many thanks for your help.</p>

Dirk10001: <p>With help from <em><a href="https://discuss.pytorch.org/t/libtorch-loss-backward-c10-error/116536" class="inline-onebox">Libtorch loss.backward(); C10 Error</a></em><br>
I got the error description ‘CUDA error: device-side assert triggered’.<br>
On <a href="https://builtin.com/software-engineering-perspectives/cuda-error-device-side-assert-triggered" class="inline-onebox" rel="noopener nofollow ugc">CUDA Error: Device-Side Assert Triggered: Solved | Built In</a><br>
it is described ‘caused by an inconsistency between the number of labels and output units or an incorrect input for a loss function’. So I checked my target tensor</p>
<blockquote>
<p>auto target = torch::zeros(TRAIN_BATCH_SIZE, torch::kInt64).to(device);</p>
</blockquote>
<p>Without cudaMemcpy there is no exception. The error was in</p>
<blockquote>
<p>cudaMemcpy(&amp;resdata[i * NET_OUTPUT_COUNT + j], &amp;cvResult[imgcount], sizeof(int64_t), cudaMemcpyHostToDevice);</p>
</blockquote>
ONNX model inference produces different results for the idential input

lihan: <p>After a Pytorch model is converted to ONNX and then loaded in onnxruntime.</p>
<p>For the idential input, it produces different results in ONNX model. (works in PyTorch)</p>
<p>Please refer to this reproducible notebook</p>
<aside class="onebox allowlistedgeneric" data-onebox-src="https://colab.research.google.com/drive/1cBd0MkQ804FXjWtOME1EB1-UiTXe1elp#scrollTo=bRLuTOjO2YQU">
  <header class="source">
      <img src="https://colab.research.google.com/img/favicon.ico?vrz=colab-20210902-060053-RC01_394507854" class="site-icon" width="128" height="128">

      <a href="https://colab.research.google.com/drive/1cBd0MkQ804FXjWtOME1EB1-UiTXe1elp#scrollTo=bRLuTOjO2YQU" target="_blank" rel="noopener nofollow ugc">colab.research.google.com</a>
  </header>

  <article class="onebox-body">
    <img src="https://colab.research.google.com/img/colab_favicon_256px.png" class="thumbnail onebox-avatar" width="260" height="260">

<h3><a href="https://colab.research.google.com/drive/1cBd0MkQ804FXjWtOME1EB1-UiTXe1elp#scrollTo=bRLuTOjO2YQU" target="_blank" rel="noopener nofollow ugc">Google Colaboratory</a></h3>



  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


lihan: <p>I worked out the issue being the use of <code>PReLU</code>, not sure about the fix yet.</p>

MrCrHaM: <p>Could you please elaborate a bit on what exactly it the problem with <code>PReLU</code>?</p>
<p>I have the same problem and my model contains <code>PReLU</code> as well.</p>
Set the range of pairwise distance and cosine similarity between 0 and 1

fairy: <p>I write a BiLSTM-Siamese Network to measure the string similarities using pairwise distance and cosine similarities with the detail as follows:</p>
<pre><code class="lang-auto">class SiameseNetwork(nn.Module):
    def __init__(self, num_layers, dropout, weight_matrix, vocabs, similarity_measure):
        super(SiameseNetwork, self).__init__()        
        self.lstm_network = BiLSTM(num_layers, weight_matrix, vocabs)
        self.fc_drop = nn.Dropout(p = dropout)
        self.similarity_measure = similarity_measure
        if self.similarity_measure == 'euclidean_distance':
            self.sm = nn.PairwiseDistance(p=2)
        else:
            self.sm = nn.functional.cosine_similarity
        
    def forward(self, input1, input2):
        output1 = self.lstm_network(input1)
        output2 = self.lstm_network(input2)
        
        out1 = self.fc_drop(output1)
        out2 = self.fc_drop(output2)
        
        x = self.sm(out1, out2)
        if self.similarity_measure == 'euclidean_distance':
            x = 1-x  # The larger the x value is, the more similar the strings are.      
        x = torch.sigmoid(x)

        return x
</code></pre>
<p>I used the <strong>torch.sigmoid</strong> to make the similarity degree between 0 and 1. However, the sigmoid makes the same string pair’s similarities, not 1. Hence, I need to know how to make the range of the similarity degree in the range 0-1 using the pairwise distance and cosine similarity. 0 if the string pairs are dissimilar and 1 if the string pairs are similar. Any help would be greatly appreciated. Thank you!</p>