akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
ValueError: some parameters appear in more than one parameter group

username: debvrat
cooked: <p>I’m using <code>torch 1.4.0</code>. Here is my error trace:</p>
<pre><code class="lang-auto">  File "G:\My Drive\Debvrat_Research\Codes\Edge Detection\MultiScale-thickness\train.py", line 99, in &lt;module&gt;
    main()

  File "G:\My Drive\Debvrat_Research\Codes\Edge Detection\MultiScale-thickness\train.py", line 89, in main
    trainer=Trainer(args,net, train_loader=train_loader)

  File "G:\My Drive\Debvrat_Research\Codes\Edge Detection\MultiScale-thickness\modules\trainer.py", line 87, in __init__
    self.optimizer = torch.optim.SGD(tuned_lrs, lr=args.lr, momentum=args.momentum, weight_decay=args.weight_decay)

  File "C:\Anaconda3\lib\site-packages\torch\optim\sgd.py", line 64, in __init__
    super(SGD, self).__init__(params, defaults)

  File "C:\Anaconda3\lib\site-packages\torch\optim\optimizer.py", line 51, in __init__
    self.add_param_group(param_group)

  File "C:\Anaconda3\lib\site-packages\torch\optim\optimizer.py", line 216, in add_param_group
    raise ValueError("some parameters appear in more than one parameter group")

ValueError: some parameters appear in more than one parameter group
</code></pre>
<p>The error is thrown here:</p>
<pre><code class="lang-auto"> if not param_set.isdisjoint(set(param_group['params'])):
            raise ValueError("some parameters appear in more than one parameter group")
</code></pre>
<p>My <code>param_set</code> is coming out to be empty ({}). I neither understand the reason behind it, nor how to resolve about this. Any pointers would be useful.</p>
<p>Thanks!</p>

username: liuzhaoo
cooked: <p>maybe you can check the Parameter definition function like：<br>
`def get_1x_lr_params(model):<br>
“”"<br>
This generator returns all the parameters for conv and two fc layers of the net.<br>
“”"<br>
b = [model.conv1, model.conv2, model.conv3a, model.conv3b, model.conv4a, model.conv4b,<br>
model.conv5a, model.conv5b, model.fc6, model.fc7]<br>
for i in range(len(b)):<br>
for k in b[i].parameters():<br>
if k.requires_grad:<br>
yield k</p>
<p>def get_10x_lr_params(model):<br>
“”"<br>
This generator returns all the parameters for the last fc layer of the net.<br>
“”"<br>
# b = [model.fc7]<br>
b = [model.fc8]<br>
for j in range(len(b)):<br>
for k in b[j].parameters():<br>
if k.requires_grad:<br>
yield k<br>
`</p>
<p>Make sure there are no duplicate layers</p>
RuntimeError: element 0 of variables does not require grad and does not have a grad_fn when trying to implement Gumbel-ST sampler

username: kroscoo
cooked: <p>I currently have the following code to implement a straight-through Gumbel softmax sampler:</p>
<pre><code class="lang-auto">def gumbel_ST_sample(logits, temp=1.0):
    noise = torch.rand(logits.size())
    eps = 1e-8
    gumbel_noise = Variable(-torch.log(-torch.log(noise + eps) + eps))

    gumbel_sample = F.softmax((logits + gumbel_noise) / temp, dim=-1)

    one_hot_sample = (gumbel_sample == gumbel_sample.max(1)[0].unsqueeze(1))

    st = (one_hot_sample.float() - gumbel_sample).detach() + gumbel_sample

    return st    
</code></pre>
<p>I’m trying to test it using the following code:</p>
<pre><code class="lang-auto">test_logits = Variable(torch.rand(4, 10))
sample = gumbel_ST_sample(test_logits)
test_embeds = Variable(torch.rand(10, 5))

out = torch.mm(sample, test_embeds).sum()

grad = torch.autograd.grad(out, test_embeds)
</code></pre>
<p>But this gives me the error in the title. Can anyone help?</p>
<p>Thanks<br>
Kris</p>

username: kroscoo
cooked: <p>Stupid error:</p>
<pre><code class="lang-auto">test_embeds = Variable(torch,randn(5. 10), requires_grad=True)
</code></pre>
<p>solves it.</p>

username: marsggbo
cooked: <p>Hi! Thanks for your sharing. I have a question, it sees that you return one-hot value after the forward of <code>gumbel_ST_sample</code>, but I wonder how to return the backward value of non-one-hot function (i.e., <code>gumbel_sample </code>)?</p>
Convert C++ vector<vector<float>> to torch::tensor

username: Ian_Kennedy
cooked: <p>I am trying to convert my c++ vector to a torch tensor. However, my code is returning incorrect conversions.</p>
<pre><code class="lang-auto">    cout &lt;&lt; history &lt;&lt; endl;
    auto options1 = torch::TensorOptions().dtype(torch::kFloat32);
    input = torch::from_blob(history.data(), {size, 1, 6}, options1).to(torch::kFloat32);

    cout &lt;&lt; input &lt;&lt; endl;
</code></pre>
<p>The above code returns the following output:</p>
<p>-9 -3 -3 -12 -0 -0 -12 -2 -3 -12 -0 -0</p>
<p>(1,.,.) =<br>
-9.8681e-32  4.5793e-41 -9.8682e-32  4.5793e-41 -9.8682e-32  4.5793e-41</p>
<p>(2,.,.) =<br>
-9.8682e-32  4.5793e-41 -9.8683e-32  4.5793e-41 -9.8683e-32  4.5793e-41</p>
<p>[ CPUFloatType{2,1,6} ]</p>
Initializing QAT with pre-trained quantization parameters

username: jkosaian
cooked: <p>Hi all,</p>
<p>I apologize if this question is covered elsewhere.</p>
<p>I would like to perform quantization-aware training, but with the model initialized according to the pre-trained, post-training-quantized quantization parameters (e.g., a torchvision quantized model with layers initialized with the same scale, zero_point, etc. as in the pre-trained quantization model that is initialized with <code>quantize=True</code>).</p>
<p>That is, I’d like the initial model used for QAT to produce the same output as a pre-trained model that has been quantized using a post-training method (e.g., static quantization).</p>
<p>Is there an easy way to achieve this? I had tried hacking manually setting some of the QAT model’s FakeQuantizer parameters, but was unable to get it working properly.</p>
<p>I appreciate any help! Please let me know if my question is unclear and I will rephrase it.</p>
<p>Thanks!</p>

username: jerryzh168
cooked: <p>I don’t think that is supported right now, instead if you have access to the original floating point model you can just do qat with that. you’ll get the model with same accuracy as the post training model if you call prepare_qat for the original model and calibrate it with the same data.</p>

username: jkosaian
cooked: <p>Got it. Thanks for the response!</p>