Use of nn.Embedding for floating type numbers

vainaijr: <p>if I have a tensor like</p>
<pre><code class="lang-auto">torch.tensor([6., 4., 9., 8.], requires_grad=True)
</code></pre>
<p>and I want to represent each of these numbers by n number of parameters, how do I use nn.Embedding, for this?<br>
if I do</p>
<pre><code class="lang-auto">x = nn.Embedding(4, 10)
</code></pre>
<p>then x(input)<br>
then input needs to be of type LongTensor, how do I pass input as a floating tensor and embedding would represent index so, 0th row would be for 6., 1st row would be for 4. and so on?</p>

spanev: <p>Hi,</p>
<p>What would be <code>n</code> in your example, 4?<br>
Let’s call <code>M</code> the maximum value you can have in your input tensor, and <code>n</code> the embedding dimension.<br>
You would have to create your layer as:</p>
<pre><code class="lang-auto">x = nn.Embedding(M+1, n)
</code></pre>
<p>In your example, 9 seems to be the biggest value so you can do:</p>
<pre><code class="lang-auto">emb = nn.Embedding(10, 10) # M = 9 and n = 10
</code></pre>
<p>and to use it, just cast the input to long:</p>
<pre><code class="lang-auto">torch.tensor([6., 4., 9., 8.], requires_grad=True)

emb(t.long())
</code></pre>
<p>I hope I understood correctly what you are trying to do here. <img src="https://discuss.pytorch.org/images/emoji/apple/slightly_smiling_face.png?v=9" title=":slightly_smiling_face:" class="emoji" alt=":slightly_smiling_face:"></p>

vainaijr: <p>no no, I want to represent each of these numbers as vectors, if I represent each of these numbers as vector of size 10, then n would be 10.</p>
<p>so a number would be represented as</p>
<pre><code class="lang-auto">6 -&gt; [0.01, 0.2, 0.5, -0.2, 0.6, 0.2, 0.7, 0.3, 0.02, 0.5]
</code></pre>
<p>similarly for 4, 9, 8<br>
finally I want a matrix of size [4x10]<br>
representing 40 parameters, 10 for each of 6, 4, 9, 8</p>
<p>if my tensor was</p>
<pre><code class="lang-auto">torch.tensor([2., 4., 6.], requires_grad=True)
</code></pre>
<p>then I want a matrix of size [3x10]<br>
representing 30 parameters, 10 for each of 2, 4, 6</p>
<p>I think embedding is not the correct way to do this</p>

spanev: <p>Well, I believe that’s exactly what my example does.</p>
<p>Let’s check:</p>
<pre><code class="lang-auto">t = torch.tensor([2., 4., 6.], requires_grad=True)
emb = nn.Embedding(10, 10)
&gt;&gt;&gt; emb(t.long()).shape
torch.Size([3, 10]) 
</code></pre>
<p>Each number maps to a row in <code>emb</code>.</p>
<p>Or am I still missing something?<br>
What are the numbers in <code>[0.01, 0.2, 0.5, -0.2, 0.6, 0.2, 0.7, 0.3, 0.02, 0.5]</code>, learnable params, right?</p>

vainaijr: <p>yes, it is correct, sorry my mistake.</p>

spanev: <p>No worries! I’m happy that that was what you were looking for <img src="https://discuss.pytorch.org/images/emoji/apple/+1.png?v=9" title=":+1:" class="emoji" alt=":+1:"></p>

free-soellingeraj: <p>The question I have here: what is interpretations of the 10 dimensional vectors?  Why 10 dimensions?  When the input feature is categorical, the interpretation is straight forward.  In your case, the entire data universe is only 3 numbers.  Are those categoricals represented by numbers or actual measurements?</p>

bharat_runwal: <p><a class="mention" href="/u/spanev">@spanev</a> Hi , thanks for this , i have a question here,after casting t to long type we no longer be able to backpropagate all the way through t (as casting is not differentiable), so is there any way we can do that also ?  TIA.</p>