CrossEntropyLoss for Next-Item Prediction (itemID starts from NUM_USERS)

icmp_no_request: <p>I wanna solve user-item prediction issue. For example, dataset contains 2 users and 5 items. So,</p>
<p>userID: [0, 1]<br>
itemID: [2, 3, 4, 5, 6]</p>
<p>I embed the nodes (both users and items nodes) and get the nodes embeddings after my model. For prediction, I was wondering if I should minus num_users (2 here) in the CrossEntropyLoss?</p>
<p>The code snippet is as following:</p>
<pre><code class="lang-auto">nodes_embs = nn.Embedding(2 + 5, emb_dim)
users_embs = nodes_embs[:2, :]  
items_embs = nodes_embs[2:, :]

scores = users_embs.matmul(items_embs.t())    # [2, 5]

# the labels like 
labels = torch.tensor([5, 4])    
# user1 would choose itemID=5 for next-item
# user2 would choose itemID=4 for next-item

# I was wondering if I should minus the itemID in labels tensor?
# like [5-2, 4-2] ?
loss = cross_entropy_loss(scores, labels)
</code></pre>
<p>Or how to get cross-entropy loss for the itemID starts from num_users issue? Thanks in advance.</p>

KFrank: <p>Hi icmp!</p>
<aside class="quote no-group quote-modified" data-username="icmp_no_request" data-post="1" data-topic="155856" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/icmp_no_request/48/14795_2.png" class="avatar"> icmp_no_request:</div>
<blockquote>
<p>I was wondering if I should minus num_users (2 here) in the CrossEntropyLoss?<br>
…</p>
<pre><code class="lang-auto">scores = users_embs.matmul(items_embs.t())    # [2, 5]
...
# the labels like 
labels = torch.tensor([5, 4])    
# I was wondering if I should minus the itemID in labels tensor?
# like [5-2, 4-2] ?
loss = cross_entropy_loss(scores, labels)
</code></pre>
</blockquote>
</aside>
<p>Yes, this is correct.</p>
<p>The <code>labels</code> you pass into <code>CrossEntropyLoss</code> should be integer class<br>
labels that run from <code>0</code> to <code>nClasses - 1</code>.  So if your raw labels are in<br>
<code>(2, 3, 4, 5, 6)</code> you need to map them to <code>(0, 1, 2, 3, 4)</code> by<br>
subtracting <code>2</code>.</p>
<p>Best.</p>
<p>K. Frank</p>

icmp_no_request: <p>Hi Frank <a class="mention" href="/u/kfrank">@KFrank</a> ,</p>
<p>Thank you for your reply.</p>
<p>I have another question about updating gradients of some specific items. As described above, I have 7 nodes, 2 for users and 5 for items. In one part of my model, I only need updating items embeddings. The code is as following:</p>
<pre><code class="lang-auto">nodes_embs = nn.Embedding(2 + 5, emb_dim)
nodes_weights = nodes_embs.weight

new_items_embs = nn.Embedding(5, emb_dim)
new_items_weights = new_items_embs.weight

# If I wanna update nodes_weights[2:] with new_items_weights, I try
nodes_weights[2:, :] = new_items_weights
</code></pre>
<p>but it raises <code>RuntimeError: a view of a leaf Variable that requires grad is being used in an in-place operation.</code></p>
<p>I was wondering how to solve such errors. Thanks in advance.</p>

KFrank: <p>Hi icmp!</p>
<aside class="quote no-group" data-username="icmp_no_request" data-post="3" data-topic="155856" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/icmp_no_request/48/14795_2.png" class="avatar"> icmp_no_request:</div>
<blockquote>
<pre><code class="lang-auto"># If I wanna update nodes_weights[2:] with new_items_weights, I try
nodes_weights[2:, :] = new_items_weights
</code></pre>
<p>but it raises <code>RuntimeError: a view of a leaf Variable that requires grad is being used in an in-place operation.</code></p>
</blockquote>
</aside>
<p>I don’t understand your use case, but I will make a few comments:</p>
<p>First, is <code>nodes_weights = nodes_embs.weight</code> something that you<br>
update <em>entirely</em> manually, something you update <em>solely</em> by training with<br>
backpropagation / optimization, or do you do both?</p>
<p>If you are not using optimization, just set <code>requires_grad = False</code><br>
for the original weight tensor that you refer to as <code>nodes_embs.weight</code>.</p>
<p>(if you are <em>only</em> using optimization, you shouldn’t be manually overwriting<br>
elements of <code>nodes_embs.weight</code>.)</p>
<p>If you do want to overwrite some <code>weight</code> elements that are also being<br>
optimized, use a <code>with torch.no_grad():</code> block:</p>
<pre data-code-wrap="python"><code class="lang-python">with torch.no_grad():
    nodes_weights[2:, :] = new_items_weights
</code></pre>
<p>Best.</p>
<p>K. Frank</p>