How to change [1,32] int10 tensor into [1,10] int32 tensor

Maxwell_Albert: <p>Here is a question about how to do bit-wise change to pytorch tensor.<br>
I have a tensor with size [1,32] and type int16. But only 10 bits of 16 are useful.(since values are too small).<br>
And I want to change it bit-wise to tensor with size [1,10] and type int32, which means the data or the binary value are the same.<br>
How could I do this easily?</p>

ptrblck: <p>It seems you would like to “pack” the tensor somehow and store it as the <code>int32</code> dtype.<br>
In that case, you might want to write a custom C++ extension and perform the bit manipulations there.<br>
Note that the <code>int32</code> values would of course not represent the real values since you are not only packing but also interleaving the actual 10bit data.</p>

Maxwell_Albert: <p>Thank you!<br>
Is there any tutorial or related notes about tensor extension I could refer to?</p>

Matias_Vasquez: <p>I think this should do the trick.</p>
<pre><code class="lang-python">def weird_format(tensor):
    #assert tensor.shape == torch.Size([1, 32])

    temp = tensor.numpy()
    temp = np.unpackbits(temp.view(np.uint8), axis=0)

    m_s = temp[:,1::2] # Most significant bits
    l_s = temp[:,::2]  # Least significatn bits

    temp = np.vstack((m_s, l_s))[-10:, :]

    print(temp) # To see the binary representation (y axis)

    temp = np.packbits(temp[:, ::-1]).view(np.int32)
    temp = torch.tensor(temp).unsqueeze(0)

    print(temp.T) # To see what the 'new' numbers are (taking 32 bits on the x axis)

    return temp
</code></pre>
<pre><code class="lang-python"># Test
tensor = torch.arange(32, dtype=torch.int16).unsqueeze(0)

print(weird_format(tensor).shape)
</code></pre>
<pre><code class="lang-python"># Output
# Up to down are the 10 least significant bits of the input 0-31 in binary
# Left to right are the 32 bits taken for the new numbers
[[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]
 [0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1]
 [0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1]
 [0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1]
 [0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1]]

# If you put these numbers into a binary converter you get the values from above
tensor([[          0],
        [          0],
        [          0],
        [          0],
        [          0],
        [      65535],
        [   16711935],
        [ -252645136],
        [ -858993460],
        [-1431655766]], dtype=torch.int32)

# This is the shape that you wanted
torch.Size([1, 10])
</code></pre>
<p>Hope this helps and makes sense <img src="https://discuss.pytorch.org/images/emoji/apple/smile.png?v=12" title=":smile:" class="emoji" alt=":smile:" loading="lazy" width="20" height="20"></p>

KFrank: <p>Hi Maxwell!</p>
<aside class="quote no-group" data-username="Maxwell_Albert" data-post="1" data-topic="148713" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/maxwell_albert/48/31003_2.png" class="avatar"> Maxwell_Albert:</div>
<blockquote>
<p>I want to change it bit-wise to tensor with size [1,10] and type int32,</p>
</blockquote>
</aside>
<p>For what it’s worth, you can do this with pytorch’s bit-manipulation<br>
routines, without converting to numpy.</p>
<p>Here is a pure-pytorch solution, similar in concept to Matias’s approach:</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
print (torch.__version__)

_ = torch.manual_seed (2022)

def weird_format (t10):
    tbits = (t10.unsqueeze (1).bitwise_right_shift (torch.arange (10, dtype = torch.int16).unsqueeze (1)) % 2 == 1).to (torch.int32)
    t32 = tbits @ torch.ones (1, dtype = torch.int32).bitwise_left_shift (torch.arange (32, dtype = torch.int32))
    return  t32

t10a = torch.zeros (1, 32, dtype = torch.int16)
t10a[:, 0:3] = torch.tensor ([[42, 12, 48]])
t32a = weird_format (t10a)
print ('t10a:')
print (t10a)
print ('t32a:')
print (t32a)

nBatch = 3
t10b = torch.randint (2**10 - 1, (nBatch, 32), dtype = torch.int16)
t32b = weird_format (t10b)
print ('nBatch:', nBatch)
print ('t10b:')
print (t10b)
print ('t32b:')
print (t32b)
</code></pre>
<p>And here is its output:</p>
<pre><code class="lang-plaintext">1.11.0
t10a:
tensor([[42, 12, 48,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
          0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0]],
       dtype=torch.int16)
t32a:
tensor([[0, 1, 2, 3, 4, 5, 0, 0, 0, 0]], dtype=torch.int32)
nBatch: 3
t10b:
tensor([[ 248,  511,  315,  780,  559,  775,  131,  145,   10,  366,  692,  144,
          716,  389,  346,  421,  576,  404,  797,  959,  541,   44,   81,  869,
          861,  660,  792,  368,  255,  690,  306,  799],
        [ 979,  904,  552,  171,  373,  959,   24,  923,  335,  276,  979,  972,
          827,   16,  645,  669,  911,  418,  345,  727,  947,  351,  339,   13,
           83,   81,  581,  927,  972,  668,  357,  126],
        [ 711,   19,  854,   91,   30,  498,  767,  153,  873,  933,  549,  531,
          415,   83,  659,  373,  360,  529,  410,  710,  690,  188,  766,  686,
          298,  538,  786, 1010,  675,  881,  652,  646]], dtype=torch.int16)
t32b:
tensor([[-1847811850,  -267893898, -1816218054, -1791208673,   -10597241,
          2024310295,   432099843,   839564483,  -846274002, -1482877896],
        [ 1342035385, -1988422231,   -55981264, -1197106706, -1417890063,
         -1072558020,  -680784623,   941345963,  1484201907,  1008327847],
        [  805502923, -1612941185, -1058498987,  1139085784,   779548926,
           972130144,   675914093,  -654552351,   755340068,   -19247291]],
       dtype=torch.int32)
</code></pre>
<p>If all you care about are the bits, you’re good.  But to echo <a class="mention" href="/u/ptrblck">@ptrblck</a> 's<br>
comment, if your original <code>[1, 32]</code> tensor contains meaningful numbers,<br>
your resulting <code>[1, 10]</code> tensor will no longer contain meaningful numbers<br>
(except in garbled form).</p>
<p>Best.</p>
<p>K. Frank</p>

Maxwell_Albert: <p>Thanks, it gives me inspiration</p>

Maxwell_Albert: <p>Thanks, I hope I could give two solutions .<img src="https://discuss.pytorch.org/images/emoji/apple/blush.png?v=12" title=":blush:" class="emoji" alt=":blush:" loading="lazy" width="20" height="20"></p>