akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Multi label activation function

username: Julio
cooked: <p>I currently have a trained system with a Softmax as the activation function, which returns a vector of probabilities of each class, all together suming 1 (example: [0.6. 0.1, 0.15, 0.05, 0.1])<br>
I was using this model to perform a multi class classification, but i’d like to try it with a multi label approach, and i was wondering if <strong>exists any activation function that returns a number of the probability of each class being correct, independently of the others (not suming 1 necessarily. example: [0.8, 0.2, 0.1, 0.7, 0.15])</strong></p>
<p>Many thanks <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: KFrank
cooked: <p>Hi Julio!</p>
<aside class="quote no-group" data-username="Julio" data-post="1" data-topic="83238" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/julio/48/12132_2.png" class="avatar"> Julio:</div>
<blockquote>
<p>if <strong>exists any activation function that returns a number of the probability of each class being correct, independently of the others</strong></p>
</blockquote>
</aside>
<p><a href="https://pytorch.org/docs/stable/nn.functional.html#torch.nn.functional.sigmoid" rel="noopener nofollow ugc">sigmoid()</a> should do what you want.</p>
<p>A couple of points:</p>
<p>I imagine that you have a <code>Linear</code> layer feeding your activation function.<br>
The <code>Linear</code> layer produces a set raw-score <code>logits</code>.  <code>softmax()</code> converts<br>
these to a set class probabilities that sum to one.  <code>sigmoid()</code> converts<br>
each logit individually to the (binary) probability of the corresponding<br>
class be “active” or “inactive.”</p>
<p>No (built-in) loss function can be used directly with <code>softmax()</code>.  For<br>
the single-label, multi-class  case, you should  either feed the logits to<br>
<code>CrossEntropyLoss</code>, or use <code>log_softmax()</code> as the activation function,<br>
and pass its output to the <code>NLLLoss</code> loss function.</p>
<p>For the multi-label, multi-class case, you can feed <code>sigmoid()</code> to<br>
<code>BCELoss</code>, but you will be better off, for numerical reasons feeding the<br>
logits to <code>BCEWithLogitsLoss</code>.</p>
<p>I wouldn’t just take your trained single-label model and replace<br>
<code>softmax()</code> with <code>sigmoid()</code> to make multi-label predictions.  You<br>
can start with your trained single-label model, and use it as a<br>
pre-trained starting point for further training of your multi-label<br>
model.</p>
<p>Best.</p>
<p>K. Frank</p>
Derivatives of output w.r.t inpur

username: Joe_McEwen
cooked: <p>Hi, I am wanting to computer the derivatives of the output of a network with respect to the inputs.<br>
Currently, I loop through each output and at each iteration I calculate y_pred[i,j].backward() and then grab the grad of the in put X. This seems inefficient. Is there a better way?</p>
<p>Code snippet is bellow. y_pred is the output of neural network. X is the input to the neural network.</p>
<p>for i in range(y_pred.shape[0]):<br>
for j in range(y_pred.shape[1]):</p>
<pre><code>    try: 
        X.grad.zero_()
    except AttributeError:
        pass

    y_pred[i,j].backward(retain_graph=True)
    
    print(i,j)
    print(X.grad.data)</code></pre>

username: albanD
cooked: <p>Hi,</p>
<p>If you want to get all the values, I’m afraid there is no other way to do it <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:"><br>
If you want to compute a linear combination of these values, then you can do better by using the linearity of the derivative and doing the linear combination on the outputs first then doing a single backward.</p>
<p>Note that you should never use <code>.data</code> in pytorch. You can replace it with <code>.detach()</code> if you want to break the graph. Or nothing if you just want to access the Tensor.</p>
PyTorch1.5+cu10.1 doesn't work

username: alfredcs
cooked: <p>$ conda list | egrep -i ‘torch|python|cuda’<br>
_pytorch_select           0.1                       cpu_0    anaconda<br>
cudatoolkit               10.1.243             h6bb024c_0<br>
cudnn                     7.6.5                cuda10.1_0<br>
python                    3.7.4                h265db76_1<br>
torch                     1.5.0+cu101              pypi_0    pypi<br>
torchvision               0.6.0+cu101              pypi_0    pypi<br>
(base) $ python -c ‘import torch;print(torch.version.cuda)’<br>
10.1<br>
(base) $ python -c ‘import torch;print(torch.<strong>version</strong>)’<br>
1.5.0+cu101<br>
(base)$ python -c ‘import torch;print(torch.cuda.is_available())’<br>
False<br>
(base)$ python -c ‘import torch;print(torch._C._cuda_getDriverVersion())’<br>
10000</p>

username: albanD
cooked: <p>Do you have the proper card driver? Are you able to run other CUDA programs?</p>
Prevent SequentialSampler from ordering the indices

username: JanoschMenke
cooked: <p>Hi,</p>
<p>when I am using the SequentialSampler it appears that it orders the indicies I pass. Lets say I want items <code>[3,1,2]</code> it would give me the items but in the ordering <code>[1,2,3]</code> is there a way to prevent the Sampler form doing so.</p>

username: JanoschMenke
cooked: <p>50% of all Answers are found within 5 minutes after posting the questions. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"><br>
Rather than using a Sampler one can use  <code>torch.utils.data.Subset(dataset, train_index)</code></p>
How to create dataset with multiple input/output features

username: desmond13
cooked: <p>Hi, I am new to pytorch and pandas. I come from a Matlab background.</p>
<p>I am trying to import a dataset in python from a .csv file and it should be quite straightforward. My .csv contains a bunch of columns (19), some of them are input features and some are output features.</p>
<p>What I do is the following, how can I extend this code to import multiple input and output features?</p>
<pre><code class="lang-auto"># Load the data
D = pd.read_csv("result_reorganized.csv")

x_dataset = torch.tensor(D.inputFeature1.values, dtype=torch.float)
y_dataset = torch.tensor(D.outputFeature1.values, dtype=torch.float)
</code></pre>
<p>In matlab I would do something like</p>
<pre><code class="lang-auto">[ x1_dataset, x2_dataset, x3_dataset] = [torch.tensor(D.inputFeature1.values, dtype=torch.float) , torch.tensor(D.inputFeature2.values, dtype=torch.float), torch.tensor(D.inputFeature3.values, dtype=torch.float)]
</code></pre>
<p>I can’t find so far how to do this in python/pandas.<br>
Thanks for your help,<br>
Fabrizio.</p>

username: desmond13
cooked: <p>I thought that I found my answer:</p>
<pre><code class="lang-auto">x1_dataset = torch.tensor(D.inputFeature1.values, dtype=torch.float)

x2_dataset = torch.tensor(D.inputFeature2.values, dtype=torch.float)

x3_dataset = torch.tensor(D.inputFeature3.values, dtype=torch.float)

x_dataset = [x1_dataset, x2_dataset, x3_dataset]
</code></pre>
<p>However, in this way <code>x_dataset</code> is of type list while I would like to have a type <code>tensor</code></p>
<p>Could you help me?</p>

username: OrielBanne
cooked: <p>maybe try using torch.cat      (<a href="http://pytorch.org/docs/torch.html#torch.cat" rel="nofollow noopener">http://pytorch.org/docs/torch.html#torch.cat</a>)<br>
or torch.stack</p>
<p>see: <a href="https://discuss.pytorch.org/t/how-to-concatenate-list-of-pytorch-tensors/1350" class="inline-onebox">How to concatenate list of pytorch tensors?</a></p>

username: desmond13
cooked: <p>Thanks it works!</p>
<p>My code now is the following (hopefully it will help some other newbies like me)</p>
<pre><code class="lang-auto">x1_dataset = torch.tensor(D.inputFeature1.values, dtype=torch.float)

x2_dataset = torch.tensor(D.inputFeature2.values, dtype=torch.float)

x3_dataset = torch.tensor(D.inputFeature3.values, dtype=torch.float)

x_dataset = torch.stack([x1_dataset, x2_dataset, x3_dataset])
</code></pre>
MSELoss and torch.max

username: AlexisW
cooked: <p>I have a model which has 10 outputs. I want to select the index with the maximum value and do MSEloss with my label. However, it appears that if I do <code>_, max_index = torch.max(raw_result, 1)</code>, then <code>max_index.requires_grad</code> will be <code>False</code>, and that prevents the code from <code>loss.backward()</code>. Does anyone have experience on how to do so?</p>

username: justusschock
cooked: <p>Did you try <a href="https://pytorch.org/docs/master/torch.html#torch.argmax" rel="nofollow noopener">torch.argmax</a> ?</p>

username: AlexisW
cooked: <p>yeah</p>
<pre><code class="lang-auto">x = torch.randn(10, 20)
x.requires_grad = True
output = torch.argmax(x, 1)
output.requires_grad # False
</code></pre>

username: justusschock
cooked: <p>On the other hand, it seems hard to propagate the gradient back if you look at <a href="https://discuss.pytorch.org/t/gradients-with-argmax-in-pytorch/8473/2?u=justusschock">this post</a></p>

username: justusschock
cooked: <p>As proposed <a href="https://discuss.pytorch.org/t/argmax-with-pytorch/1528/4?u=justusschock">here </a> it should be a possibility to use softmax because the hard indexing is not differentiable</p>
How to set different threads number for different modules in TorchScript

username: knitvoger
cooked: <p>I read this <a href="https://pytorch.org/docs/stable/torch.html?highlight=set_num_thread#torch.set_num_threads" rel="noopener nofollow ugc">doc</a> and found that torch.set_num_threads and torch.set_num_interop_threads can only be called before running script model.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/0/3050cbc1971738b04913b177a10be352225ade11.png" data-download-href="https://discuss.pytorch.org/uploads/default/3050cbc1971738b04913b177a10be352225ade11" title="threads"><img src="https://discuss.pytorch.org/uploads/default/original/3X/3/0/3050cbc1971738b04913b177a10be352225ade11.png" alt="threads" data-base62-sha1="6Tq1qASZRWUMF2qPYAdbErzR9lL" width="690" height="362" data-dominant-color="F4F4F6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">threads</span><span class="informations">870×457 29.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Is there a way to control number of threads more flexibly in TorchScript that for a model’s different modules, we can set difference number of interop threads and introop threads to run?</p>
<p>Thanks for any answers!</p>

username: Michael_Suo
cooked: <p>No, there’s no option today for controlling parallelism on a module level</p>

username: knitvoger
cooked: <p>Got it, thank you for your answer!</p>