How can I stack tensors from a mapping list?

username: paula_gomez_duran
cooked: <p>Hi,</p>
<p>I have different different u, i c variables of shape [99] which contain:</p>
<pre><code>  `In [8]: u
  Out[8]:
  tensor([820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820,
          820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820,
          820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820,
          820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820,
          820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820,
          820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820,
          820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820, 820,
          820], device='cuda:0')
  
  In [9]: i
  Out[9]:
  tensor([1280, 2561, 2562, 1029, 1019, 2058, 1550, 1807, 2319, 2067, 2324, 1304,
          2331, 1052, 1565, 1308, 2591, 1567, 1314, 1573, 1574, 1066, 2090, 1324,
          1329, 2610, 2356, 1589, 1085, 1600, 1856, 1603, 1099, 1101, 1873, 1362,
          1107, 1624, 1113, 1114, 2396, 1888, 1633, 1121, 1641, 1643, 1645, 1392,
          1906, 2164, 1910, 1403, 1405, 1920, 1921, 1410, 1160, 2187, 2448, 1173,
          2456, 2459, 1183, 1184, 1187, 2467, 1192, 1450, 1963, 2481, 2226, 2232,
          1722, 2240, 1472, 1984, 1220, 2500, 1483, 1228, 1741,  977, 2260, 2005,
          1750, 2522, 2523, 1244, 1505, 2535, 1516, 1773, 1007, 1777, 2289, 1522,
          1013, 1016, 1275], device='cuda:0')
  
  In [12]: c
  Out[12]:
  tensor([2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694,
          2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694,
          2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694,
          2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694,
          2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694,
          2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694,
          2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694,
          2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694, 2694,
          2694, 2694, 2694], device='cuda:0')`
</code></pre>
<p>In order to generate their embeddings I pass to the model the unique idx of all of them so that it computes the corresponding embeddings, following the example, I pass the variable (whose  len(unique_uic) == 101) :</p>
<pre><code>`In [32]: unique_uic  
Out[32]:
tensor([ 820,  946,  959, 1009, 1010, 1013, 1044, 1109, 1147, 1149, 1152, 1182,
        1184, 1208, 1222, 1267, 1308, 1334, 1409, 1417, 1421, 1431, 1455, 1457,
        1462, 1470, 1475, 1565, 1578, 1586, 1628, 1632, 1675, 1686, 1690, 1698,
        1723, 1733, 1737, 1745, 1775, 1802, 1817, 1835, 1854, 1860, 1864, 1868,
        1879, 1892, 1894, 1896, 1915, 1918, 1930, 1947, 1951, 1968, 1978, 1984,
        2037, 2042, 2045, 2070, 2075, 2086, 2106, 2125, 2186, 2199, 2205, 2290,
        2302, 2303, 2332, 2338, 2347, 2363, 2368, 2381, 2382, 2417, 2440, 2441,
        2449, 2464, 2466, 2471, 2500, 2512, 2519, 2548, 2549, 2557, 2568, 2578,
        2580, 2582, 2617, 2618, 2694])
`
</code></pre>
<p>Then, when I compute the embeddings I get <code>embeddings</code> variable of shape <code>torch.Size([101, 64])</code> which refers to the embeddings on the variable <code>unique_uic</code> respectively.</p>
<p>However, in order to feed them into another model, I need them to be in the following shape:</p>
<pre><code>`In [34]:self.embeddings(torch.stack((u, i, c), dim=1))
 Out[34]:
tensor([[ 820, 2568, 2694],
        [ 820, 1802, 2694],
        [ 820, 2578, 2694],
        [ 820, 2580, 2694],
        [ 820, 1044, 2694],
        [ 820, 2070, 2694],
        [ 820, 2582, 2694],
        [ 820, 1817, 2694],
        [ 820, 2075, 2694],  
                  .........      
        [820, 2042, 2694],
        [ 820, 2045, 2694],
        [ 820, 2302, 2694],
        [ 820, 2303, 2694]]) 
`
</code></pre>
<p>So, I should have the embeddings in shape [99, 3, 64] as it refers to the 99 interactions (in this example) of u-i-c.  However, when I perform the embeddings I get embeddings from 0 to 101 corresponding to the unique id’s and I do not know how to perform the mapping in an efficient way.</p>
<p>If you could give me ideas it would be very helpful.</p>
<p>Thank you very much in advance!</p>

username: ptrblck
cooked: <p>I’m not sure I understand the use case completely, so feel free to correct me <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>
<p>Based on your description you are dealing with 3 different tensors, <code>u</code>, <code>i</code>, and <code>c</code>.<br>
<code>unique_uic</code> contains the 101 unique values of all three tensors and is used in some way to create dense embedding tensors of the shape <code>[101, 64]</code>, i.e. 101 tensors (one for each input index) with a feature dimension of 64.<br>
Now you would like to concatenate the three input tensors and feed to this embedding?</p>
<p>If that’s the case, I assume you would like to treat each input combination as an “input index”, which would need a mapping. However, these three tensors could create more combinations than the number of unique values, so how would you deal with it?<br>
<code>u</code> and <code>c</code> only contain a static value, but is this always the case in your workflow?</p>

username: paula_gomez_duran
cooked: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="118146">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>ase, I assume you would like to treat each input combination as an “input index”, which would need a mapping. However, these</p>
</blockquote>
</aside>
<p>Hi, thanks a lot for answering!</p>
<p>So, the first part of your understanding is correct, <code>u</code>, <code>i</code>, and <code>c</code> are different embeddings and <code>unique_uic</code> contains the 101 unique values of all three tensors, which are used to create dense embedding tensors of the shape <code>[101, 64]</code>.</p>
<p>Lets say that <code>u</code> always go from range 0-900 , <code>i</code> go from 901 - 2579, and <code>c</code> 2580-2697. In each iteration it is fed to the network a batch of <code>[99,3]</code>, which are triplets of indexes which have interacted. For example:</p>
<pre><code class="lang-auto">        [ 820, 1802, 2694], --&gt; user 820 have interacted with item 1802 under context 2694
        [ 820, 2578, 2694],
</code></pre>
<p>If I just get the embeddings of those 101 “nodes” in shape <code>[101,64]</code>, what I would like to know is how to come back to the original triplet shapes but instead of stacking indexes, stacking their corresponding embeddings. Thus, just asking how to go from those <code>[101, 64]</code> embeddings to a shape <code>[99, 3, 64]</code> embeddings, taking into account that, in those <code>101</code> unique embeddings, there are 3 different nodes (for this example the first one is a user, the last one context and the 99 in the middle are items but it can be completely different – so the shape of <code>unique_uic</code> is completely variant).</p>
<p>Thank you so much again!</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="paula_gomez_duran" data-post="3" data-topic="118146">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/paula_gomez_duran/48/30050_2.png" class="avatar"> paula_gomez_duran:</div>
<blockquote>
<p>So, the first part of your understanding is correct, <code>u</code>, <code>i</code>, and <code>c</code> are different embeddings and <code>unique_uic</code> contains the 101 unique values of all three tensors, which are used to create dense embedding tensors of the shape <code>[101, 64]</code>.</p>
</blockquote>
</aside>
<p>To just verify: the <code>[101, 64]</code> shape is defined as <code>[number_of_unique_triplets, embedding_dim]</code>?</p>
<aside class="quote no-group" data-username="paula_gomez_duran" data-post="3" data-topic="118146">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/paula_gomez_duran/48/30050_2.png" class="avatar"> paula_gomez_duran:</div>
<blockquote>
<p>If I just get the embeddings of those 101 “nodes” in shape <code>[101,64]</code>, what I would like to know is how to come back to the original triplet shapes but instead of stacking indexes, stacking their corresponding embeddings. Thus, just asking how to go from those <code>[101, 64]</code> embeddings to a shape <code>[99, 3, 64]</code> embeddings</p>
</blockquote>
</aside>
<p>That’s a bit tricky, since your first approach would map the triplets to 101 unique indices in the range <code>[0, 100]</code>. However, now you would like to feed each input separately to the embedding layer, which would then need 2698 indices, since all inputs together contain valid values in <code>[0, 2697]</code>.</p>
<p>Could you explain the idea a bit more? Note that an embedding layer is basically a lookup table for an integer input. The idea is that “sparse” inputs will be mapped to “dense” outputs.<br>
However, you are currently trying to mix the triplets with the raw indices, which won’t work out of the box, so some mapping might be needed.</p>