Get the mean from a list of tensors

jpainam: <p>Hi!<br>
I have a list of tensors</p>
<pre><code class="lang-auto">my_list = [tensor(0.8223, device='cuda:0'), tensor(1.8351, device='cuda:0'), tensor(1.4888, device='cuda:0'),]
</code></pre>
<p>and i want their mean. I tried <code>torch.mean(my_list)</code>, but got this error</p>
<pre><code class="lang-auto">TypeError: mean(): argument 'input' (position 1) must be Tensor, not list
</code></pre>
<p>Then I  tried this <code>np.mean(my_list)</code> and got this error</p>
<pre><code class="lang-auto">in _mean ret = ret.dtype.type(ret / rcount)
AttributeError: 'torch.dtype' object has no attribute 'type'
</code></pre>
<p>This solution works for me. Is there a better way of doing this and not using <code>detach()</code> first?</p>
<pre><code class="lang-auto">new_list = [x.cpu().detach().numpy() for x in my_list]
my_mean = np.mean(new_list)
</code></pre>
<p>Also, what to do If i want the returned value to be a <code>tensor(my_mean, device='cuda:0')</code>.<br>
I tried this <code>torch.cuda.FloatTensor(my_mean)</code> but got this error</p>
<pre><code class="lang-auto">TypeError: new(): data must be a sequence (got numpy.float32)
</code></pre>
<p>Thank you.</p>

albanD: <p>Hi,<br>
If each element tensor contain a single value, you can use <code>.item()</code> on it to get this value as a python number and then you can do <code>mean(your_list)</code>.<br>
If you want to work with Tensors to keep gradients for example. You can use <code>torch.cat(your_list, 0)</code> to concatenate the list into a single tensor. Then you can call <code>.mean()</code> on the resulting tensor.</p>

smth: <pre><code class="lang-auto">mean = torch.mean(torch.stack(my_list))
</code></pre>

jpainam: <aside class="quote no-group" data-username="albanD" data-post="2" data-topic="31989">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/48/215_2.png" class="avatar"> albanD:</div>
<blockquote>
<p><code>torch.cat(your_list, 0)</code></p>
</blockquote>
</aside>
<p>Thanks. Tried <code>torch.cat(my_list, 0)</code>, but got this error</p>
<pre><code class="lang-auto">RuntimeError: zero-dimensional tensor (at position 0) cannot be concatenated
</code></pre>
<p><a class="mention" href="/u/smth">@smth</a> solution works</p>
<p>Thank you all for the quick replies</p>

ntomita: <p>The accepted solution works for 0-dim tensor or only when a global mean is required. For the sake of completeness I would add the following as a generalized solution for obtaining <em>element-wise</em> mean tensor where input list is multi-dimensional same-shape tensors.</p>
<blockquote>
<p>torch.mean(torch.stack(my_list), dim=0)</p>
</blockquote>

Mauricio_Maroto: <p><a class="mention" href="/u/ntomita">@ntomita</a> Hi, looking at your great answer, any hint on how to do something similar but with min or max?</p>

ntomita: <p>Hi <a class="mention" href="/u/mauricio_maroto">@Mauricio_Maroto</a>. Hope you already found an answer to your question. You can use torch.max and torch.min functions in the similar way as torch.mean, except you ignore indices output.</p>
<blockquote>
<p>maxvals, _ = torch.max(torch.stack(my_list), dim=0)</p>
</blockquote>

Mauricio_Maroto: <p>Hi <a class="mention" href="/u/ntomita">@ntomita</a> yes, I figured about that same answer of yours, noticing the stack pytorch logic for this kind of operation, and then considering the dim too. Thanks anyway!</p>

Yasser_Khalil: <p>Hi,</p>
<p>If each element of a tuple/list has a tensor, and I would like to find the mean of each tensor at dim=1. How can it be done without the need of a for loop?</p>
<p>Example:</p>
<pre><code class="lang-auto">sample = (tensor([[0.], [0.], [0.]], device='cuda:0'), 
                 tensor([[2., 3.], [2., 4.], [4., 3.]], device='cuda:0'),
                 tensor([[1.], [3.], [4.]], device='cuda:0'), 
                 tensor([[1., 3.], [0., 0.], [4., 2.]], device='cuda:0'))
</code></pre>
<p>sample = (tensor([[0.], [0.], [0.]], device=‘cuda:0’),<br>
tensor([[2.5.], [3.], [3.5]], device=‘cuda:0’),<br>
tensor([[1.], [3.], [4.]], device=‘cuda:0’),<br>
tensor([[2.], [0.], [3.]], device=‘cuda:0’))</p>
<p>Thanks</p>

AlphaBetaGamma96: <p>you could try,</p>
<pre><code class="lang-auto">tuple([t.mean(dim=1) for t in sample])
</code></pre>
<p>Example,</p>
<pre><code class="lang-auto">sample = (torch.randn(3,1), torch.randn(3,2), torch.randn(3,1))
&gt;&gt;&gt; sample
(tensor([[-0.6243],
        [ 0.2206],
        [-1.9922]]), tensor([[-0.7084,  2.9351],
        [-0.8864, -0.7852],
        [-0.3822,  0.1747]]), tensor([[-2.1211],
        [ 1.0442],
        [ 0.8821]]))
&gt;&gt;&gt; tuple([t.mean(dim=1) for t in sample])
(tensor([-0.6243,  0.2206, -1.9922]), tensor([ 1.1134, -0.8358, -0.1038]), tensor([-2.1211,  1.0442,  0.8821]))

</code></pre>

jose-solorzano: <p>The answers here suggest stacking the tensors, then getting the mean. Stacking allocates a new tensor. If you do something like (t1 + t2 + t3) / 3, that builds a tensor graph. It would be nice to have native support for this common operation.</p>