Expected object of scalar type Long but got scalar type Float

username: blackbirdbarber
cooked: <p>Here is the code I have:</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
import torch.nn.functional as F
from torchvision import transforms
from torch.utils.data import DataLoader, Dataset, TensorDataset

bs=1
from torchvision.models import resnet18
model = resnet18(pretrained=False)
optimizer = torch.optim.SGD(model.parameters(), lr=0.1, momentum=0.9)

input = torch.rand(bs,3, 256,256)
target = torch.rand(bs, 1000)
print(target.size())

# target = target.long()
model.train()

# before we learn
p1 = model.fc.weight
od1 = model.state_dict()


output = model(input)
# print(output.size())
loss_fn = torch.nn.CrossEntropyLoss()
# loss_fn = torch.nn.BCEWithLogitsLoss()
loss = loss_fn(output, target)
print(loss)
if optimizer is not None:
        optimizer.zero_grad()
        # backward pass + optimize
        loss.backward()
        optimizer.step()

p2 = model.fc.weight
od2 = model.state_dict()

print(torch.equal(p1,p2))
</code></pre>
<p><code>torch.nn.CrossEntropyLoss()</code> should be the main loss function (criterion) for ImageNet classification. I just tried to test the code and got this error I could not fix fast.<br>
What I am expecting at the end is that <code>print(torch.equal(p1,p2))</code> will return <code>False</code>. This means that the parameter before and after the single optimization step will be different ( the model learned something ).</p>
<p>If I set the <code>torch.nn.BCEWithLogitsLoss()</code>  it will work but the params <code>p1</code> and <code>p2</code> will be just the equal.</p>
<p>What am I doing wrong?</p>

username: KFrank
cooked: <p>Hi Barber!</p>
<aside class="quote no-group quote-modified" data-username="blackbirdbarber" data-post="1" data-topic="61291" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/b/6de8d8/48.png" class="avatar"> blackbirdbarber:</div>
<blockquote>
<p>Here is the code I have:</p>
<pre><code class="lang-auto">...
input = torch.rand(bs,3, 256,256)
target = torch.rand(bs, 1000)

# target = target.long()
...
loss_fn = torch.nn.CrossEntropyLoss()
# loss_fn = torch.nn.BCEWithLogitsLoss()
loss = loss_fn(output, target)
...
</code></pre>
<p><code>torch.nn.CrossEntropyLoss()</code> should be the main loss function<br>
(criterion) for ImageNet classification. I just tried to test the code and got this error I could not fix fast.</p>
<p>If I set the <code>torch.nn.BCEWithLogitsLoss()</code>  it will work but the params <code>p1</code> and <code>p2</code> will be just the equal.</p>
<p>What am I doing wrong?</p>
</blockquote>
</aside>
<p>I assume that by “got this error” you mean the error message<br>
in your post title, “Expected object of scalar type Long but got<br>
scalar type Float.”  This was to be expected.</p>
<p><code>CrossEntropyLoss</code> takes integer class labels for its target,<br>
specifically of type <code>LongTensor</code>.  In your code you are passing<br>
it a <code>FloatTensor</code>.</p>
<p>For random “toy” data, you probably want something like<br>
<code>target = torch.randint (nClass, (bs,))</code>, where<br>
<code>nClass</code> is the number of classes in your classification problem.</p>
<p>(I see that you commented out the line of code<br>
<code># target = target.long()</code>.  Didn’t that fix this<br>
<em>particular</em> error for you.)</p>
<p>(As a further note, <code>BCEWithLogitsLoss()</code> does take a<br>
<code>FloatTensor</code> target, so it would not give you this error.  But,<br>
yes, for a multiclass – i.e., not binary – classification problem<br>
you would want <code>CrossEntropyLoss</code>.)</p>
<p>Good luck.</p>
<p>K. Frank</p>

username: blackbirdbarber
cooked: <p>Thanks Frank, I seen now…</p>
<p>by setting</p>
<pre><code class="lang-auto">target = torch.randint (1000, (1,))
print(target.size())
</code></pre>
<p>instead</p>
<pre><code class="lang-auto">target = torch.rand(bs, 1000)
print(target.size())
</code></pre>
<p>I am telling I will have just a single target value ranging from 1000 ImageNet classes.<br>
And really this code will work now.</p>
<p>(The # target = target.long() was something I saw in some other answers on the similar title, we may ignore that).</p>
<p>Now the code looks like this:</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
import torch.nn.functional as F
from torchvision import transforms
from torch.utils.data import DataLoader, Dataset, TensorDataset

bs=1
from torchvision.models import resnet18
model = resnet18(pretrained=False)
optimizer = torch.optim.SGD(model.parameters(), lr=0.1, momentum=0.9)

input = torch.rand(bs,3, 256,256)
target = torch.randint (1000, (1,))

model.train()

# before we learn
p1 = model.fc.weight
od1 = model.state_dict()


output = model(input)
# print(output.size())
loss_fn = torch.nn.CrossEntropyLoss()
# loss_fn = torch.nn.BCEWithLogitsLoss()
loss = loss_fn(output, target)
print(loss)
if optimizer is not None:
        optimizer.zero_grad()
        # backward pass + optimize
        loss.backward()
        optimizer.step()

p2 = model.fc.weight
od2 = model.state_dict()

print(torch.equal(p1,p2))
</code></pre>
<p>or I may use <code>p1 = model.conv1.weight</code> (same for p2) but the <code>print(torch.equal(p1,p2))</code> will return <code>True</code>.</p>
<p>Can anyone tell why the parameters haven’t been updated. I expected I should learn something with the single <code>optimizer.step()</code>. In that case p1 should not be equal to p2.</p>

username: KFrank
cooked: <p>Hello Barber!</p>
<aside class="quote no-group quote-modified" data-username="blackbirdbarber" data-post="3" data-topic="61291" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/b/6de8d8/48.png" class="avatar"> blackbirdbarber:</div>
<blockquote>
<pre><code class="lang-auto"># before we learn
p1 = model.fc.weight
od1 = model.state_dict()
...
p2 = model.fc.weight
od2 = model.state_dict()

print(torch.equal(p1,p2))
</code></pre>
<p><code>print(torch.equal(p1,p2))</code> will return <code>True</code>.</p>
<p>Can anyone tell why the parameters haven’t been updated.</p>
</blockquote>
</aside>
<p><code>p1</code> is a <em>reference</em> to <code>model.fc.weight</code>.  (In python (essentially)<br>
all variables are references to objects.)  So when your optimizer<br>
updates <code>model.fc.weight</code>, <code>p1</code> also “changes” in that the object<br>
<code>p1</code> refers to has changed.</p>
<p>Try setting <code>p1 = model.fc.weight.clone()</code>.  Now <code>p1</code> will refer<br>
to a new copy of the data in <code>model.fc.weight</code> that won’t be<br>
changed when <code>model.fc.weight</code> itself is changed.</p>
<p>Or you could print out (with enough precision to see a small<br>
change) a couple of elements of <code>model.fc.weight</code> before and<br>
after the optimization step and note that they change.</p>
<p>Good luck.</p>
<p>K. Frank</p>