Best way to convert a list to a tensor?

07hyx06: <p>let <code>a=[1,2,3]</code>, then i let <code>b=torch.Tensor(a)</code> , my pycharm’s background become yellow like that<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/9/9/99f5c2770af04a01cbba850b843b053431129a4d.png" alt="%E6%97%A0%E6%A0%87%E9%A2%98" data-base62-sha1="lXZAC5Sr3hTcz6D8dyT1k0abVnf" width="259" height="74"><br>
is there exist a elegent way to convert a list to a tensor? or is my ide’s fault?</p>

zimmer550: <p>Convert list to tensor using this</p>
<pre><code class="lang-auto">a = [1, 2, 3]
b = torch.FloatTensor(a)
</code></pre>
<p>Your method should also work but you should cast your datatype to float so you can use it in a neural net</p>

Nikronic: <p>Hi,<br>
First of all, PyCharm or most of IDEs cannot really analysis libraries like PyTorch which has C++ backend and Python frontend so it is normal to get warning or missing errors but your codes works fine.</p>
<p>But about your question:<br>
When you are on GPU, <code>torch.Tensor()</code> will convert your data type to <code>Float</code>. Actually, <code>torch.Tensor</code> and <code>torch.FloatTensor</code> both do same thing.<br>
But I think better way is using <code>torch.tensor()</code> (note the case of ‘t’ character). It converts your data to tensor but retains data type which is crucial in some methods. You may know that PyTorch and numpy are switchable to each other so if your array is int, your tensor should be int too unless you explicitly change type.</p>
<p>But on top of all these, <code>torch.tensor</code> is convention because you can define following variables:<br>
<code>device</code>, <code>dtype</code>, <code>requires_grad</code>, etc.</p>
<p>Note: using <code>torch.tensor()</code> allocates new memory to copy the data of tensor. So if you want to avoid copying, use <code>torch.as_tensor(numpy_ndarray)</code>.</p>
<p>Bests<br>
Nik</p>

07hyx06: <p>1000 thx! You reply really help me a lot!</p>

Nikronic: <p>You are welcome mate!</p>
<p>Something about <a class="mention" href="/u/zimmer550">@zimmer550</a> answer that you need to convert to float to use your tensors in NN, is a rule of thumb, so in some cases like methods available in <code>functional</code> package etc, you need <code>Long</code> data type etc. So best approach is to retain the data type as it is and change it explicitly when you to enable you debug much faster when data type inconsistency exists.</p>
<p>In PyCharm, <code>Ctrl+Q</code> (I use my config from 2017 so it may be changed.) shows the documentation within your editor and always I recommend you to read it fully. There are many notes that can save you a lot of memory or runtime computation by only using a argument or triggering a function etc.</p>

Brando_Miranda: <p>I am doing something very similar but I have a (nested) list of tensors. The simplest example I have is the following:</p>
<pre><code class="lang-auto">import torch

# trying to convert a list of tensors to a torch.tensor

x = torch.randn(3, 1)
xs = [x, x]
# xs = torch.tensor(xs)
xs = torch.as_tensor(xs)
</code></pre>
<p>but I get the following error:</p>
<pre><code class="lang-auto">x = torch.randn(3, 1)
xs = [x, x]
# xs = torch.tensor(xs)
xs = torch.as_tensor(xs)
Traceback (most recent call last):
  File "/Users/brando/anaconda3/envs/automl-meta-learning/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3343, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "&lt;ipython-input-30-f846b7bfae21&gt;", line 4, in &lt;module&gt;
    xs = torch.as_tensor(xs)
ValueError: only one element tensors can be converted to Python scalars
</code></pre>
<p>any ideas what is going on? Btw, both give the same error.</p>
<hr>
<p>I guess the following works but I am unsure what is wrong with this solution:</p>
<pre><code class="lang-auto"># %%

import torch

# trying to convert a list of tensors to a torch.tensor

x = torch.randn(3)
xs = [x.numpy(), x.numpy()]
# xs = torch.tensor(xs)
xs = torch.as_tensor(xs)

print(xs)
print(xs.size())

# %%

import torch

# trying to convert a list of tensors to a torch.tensor

x = torch.randn(3)
xs = [x.numpy(), x.numpy(), x.numpy()]
xs = [xs, xs]
# xs = torch.tensor(xs)
xs = torch.as_tensor(xs)

print(xs)
print(xs.size())
</code></pre>
<p>output:</p>
<pre><code class="lang-auto">import torch
# trying to convert a list of tensors to a torch.tensor
x = torch.randn(3)
xs = [x.numpy(), x.numpy(), x.numpy()]
xs = [xs, xs]
# xs = torch.tensor(xs)
xs = torch.as_tensor(xs)
print(xs)
print(xs.size())
tensor([[[0.3423, 1.6793, 0.0863],
         [0.3423, 1.6793, 0.0863],
         [0.3423, 1.6793, 0.0863]],
        [[0.3423, 1.6793, 0.0863],
         [0.3423, 1.6793, 0.0863],
         [0.3423, 1.6793, 0.0863]]])
torch.Size([2, 3, 3])
</code></pre>

Nikronic: <p>Hi,</p>
<p>I think <a href="https://pytorch.org/docs/stable/generated/torch.tensor.html" rel="noopener nofollow ugc">torch.tensor — PyTorch 1.7.0 documentation</a> and <a href="https://pytorch.org/docs/stable/generated/torch.as_tensor.html" rel="noopener nofollow ugc">torch.as_tensor — PyTorch 1.7.0 documentation</a> have explained the difference clearly but in summary, <code>torch.tensor</code> always copies the data but <code>torch.as_tensor</code> tries to avoid that! In both cases, they don’t accept sequence of tensors.</p>
<p>The more intuitive way is stacking in a given dimension which you can find here: <a href="https://discuss.pytorch.org/t/how-to-turn-a-list-of-tensor-to-tensor/8868">How to turn a list of tensor to tensor? - PyTorch Forums</a></p>
<p>The problem with your approach is that you convert your tensors to numpy, then you will lose grads and break computational graph but stacking preserves it.</p>
<p>Actually you have posted an answer similar that issue too! <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"> <a href="https://discuss.pytorch.org/t/how-to-turn-a-list-of-tensor-to-tensor/8868/10" class="inline-onebox">How to turn a list of tensor to tensor? - #10 by Brando_Miranda</a></p>

Brando_Miranda: <p>hahaha, yea I see I didn’t make a very solid memory/understanding when I wrote that last year (or it’s been to long since?). Perhaps I can finally sort this out in my head, though in my defence, there does seem to be a lot of discussions surrounding this topic (which isn’t helping to digest this):</p>
<ul>
<li><a href="https://discuss.pytorch.org/t/how-to-convert-array-to-tensor/28809/25" class="inline-onebox">How to convert array to tensor? - #25 by mehran2020</a></li>
<li><a href="https://discuss.pytorch.org/t/how-to-turn-a-list-of-tensor-to-tensor/8868/9" class="inline-onebox">How to turn a list of tensor to tensor? - #9 by Nithin_Vasisth</a></li>
<li><a href="https://discuss.pytorch.org/t/how-to-turn-a-list-of-tensor-to-tensor/8868/10" class="inline-onebox">How to turn a list of tensor to tensor? - #10 by Brando_Miranda</a></li>
<li><a href="https://discuss.pytorch.org/t/converting-list-to-tensor/70120/8" class="inline-onebox">Converting list to tensor - #8 by Brando_Miranda</a></li>
<li><a href="https://stackoverflow.com/questions/54307225/whats-the-difference-between-torch-stack-and-torch-cat-functions/66036075#66036075" class="inline-onebox" rel="noopener nofollow ugc">python - What's the difference between torch.stack() and torch.cat() functions? - Stack Overflow</a></li>
<li><a href="https://discuss.pytorch.org/t/how-to-make-really-empty-tensor/66850/3" class="inline-onebox">How to make really empty tensor? - #3 by 1414b35e42c77e0a57dd</a></li>
</ul>
<p>it seems that the best pytorchthoning solution comes from either knowing <code>torch.cat</code> or <code>torch.stack</code>. In my use case I generate tensors and conceptually need to nest them in lists and eventually convert that to a final tensor (e.g. of size <code>[d1, d2, d3]</code>). I think the easiest solution to my problem append things to a list and then give it to <code>torch.stack</code> to form the new tensor then append that to a new list and then convert that to a tensor by again using <code>torch.stack</code> recursively.</p>
<p>For a non recursive example I think this works…will update with a better example in a bit:</p>
<pre><code class="lang-auto"># %%

import torch

# stack vs cat

# cat "extends" a list in the given dimension e.g. adds more rows or columns

x = torch.randn(2, 3)
print(f'{x.size()}')

# add more rows (thus increasing the dimensionality of the column space to 2 -&gt; 6)
xnew_from_cat = torch.cat((x, x, x), 0)
print(f'{xnew_from_cat.size()}')

# add more columns (thus increasing the dimensionality of the row space to 3 -&gt; 9)
xnew_from_cat = torch.cat((x, x, x), 1)
print(f'{xnew_from_cat.size()}')

print()

# stack serves the same role as append in lists. i.e. it doesn't change the original
# vector space but instead adds a new index to the new tensor, so you retain the ability
# get the original tensor you added to the list by indexing in the new dimension
xnew_from_stack = torch.stack((x, x, x, x), 0)
print(f'{xnew_from_stack.size()}')

xnew_from_stack = torch.stack((x, x, x, x), 1)
print(f'{xnew_from_stack.size()}')

xnew_from_stack = torch.stack((x, x, x, x), 2)
print(f'{xnew_from_stack.size()}')

# default appends at the from
xnew_from_stack = torch.stack((x, x, x, x))
print(f'{xnew_from_stack.size()}')

print('I like to think of xnew_from_stack as a \"tensor list\" that you can pop from the front')

print()

lst = []
print(f'{x.size()}')
for i in range(10):
    x += i  # say we do something with x at iteration i
    lst.append(x)
# lstt = torch.stack([x for _ in range(10)])
lstt = torch.stack(lst)
print(lstt.size())

print()
</code></pre>
<hr>
<h1>Update: With nested list of the same size</h1>
<pre><code class="lang-auto">def tensorify(lst):
    """
    List must be nested list of tensors (with no varying lengths within a dimension).
    Nested list of nested lengths [D1, D2, ... DN] -&gt; tensor([D1, D2, ..., DN)

    :return: nested list D
    """
    # base case, if the current list is not nested anymore, make it into tensor
    if type(lst[0]) != list:
        if type(lst) == torch.Tensor:
            return lst
        elif type(lst[0]) == torch.Tensor:
            return torch.stack(lst, dim=0)
        else:  # if the elements of lst are floats or something like that
            return torch.tensor(lst)
    current_dimension_i = len(lst)
    for d_i in range(current_dimension_i):
        tensor = tensorify(lst[d_i])
        lst[d_i] = tensor
    # end of loop lst[d_i] = tensor([D_i, ... D_0])
    tensor_lst = torch.stack(lst, dim=0)
    return tensor_lst
</code></pre>
<p>here is a few unit tests (I didn’t write more tests but it worked with my real code so I trust it’s fine. Feel free to help me by adding more tests if you want):</p>
<pre><code class="lang-auto">
def test_tensorify():
    t = [1, 2, 3]
    print(tensorify(t).size())
    tt = [t, t, t]
    print(tensorify(tt))
    ttt = [tt, tt, tt]
    print(tensorify(ttt))

if __name__ == '__main__':
    test_tensorify()
    print('Done\a')
</code></pre>
<hr>
<h1>When nested list are variable length</h1>
<p>related for variable length:</p>
<ul>
<li><a href="https://stackoverflow.com/questions/55050717/converting-list-of-tensors-to-tensors-pytorch" class="inline-onebox" rel="noopener nofollow ugc">python - converting list of tensors to tensors pytorch - Stack Overflow</a></li>
<li><a href="https://discuss.pytorch.org/t/nested-list-of-variable-length-to-a-tensor/38699" class="inline-onebox">Nested list of variable length to a tensor</a></li>
</ul>
<p>I didn’t read those very carefully but I assume they must be padding somehow (probably need to calculate the largest length/dimension to padd is my guess and then do some sort of recursion like I did above).</p>

rpatrik96: <p>If someone is looking into the performance aspects of this, I’ve done a small experiment. In my case, I needed to convert a list of scalar tensors into a single tensor.</p>
<pre><code class="lang-python">import torch
torch.__version__ # 1.10.2
x = [torch.randn(1) for _ in range(10000)]
torch.cat(x).shape, torch.stack(x).shape # torch.Size([10000]), torch.Size([10000, 1])

%timeit torch.cat(x) # 1.5 ms ± 476 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%timeit torch.cat(x).reshape(-1,1) # 1.95 ms ± 534 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%timeit torch.stack(x) # 5.36 ms ± 643 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
</code></pre>
<p>My conclusion is that even if you want to have the additional dimension of <code>torch.stack</code>, using <code>torch.cat</code> and then <code>reshape</code> is better.</p>

fulltry: <p>Works, but in many other cases <code>torch.FloatTensor(a)</code> produces<br>
<code>TypeError: expected TensorOptions(dtype=float, device=cpu ...</code><br>
Therefore, often Nikronic’s answer is better: <code>torch.tensor()</code><br>
Sometimes: <code>torch.from_numpy</code></p>