Finding means and stds of a bunch of torch.Tensors (that are converted from ndarray images)

Mona_Jalal: <pre><code class="lang-auto">to_tensor = transforms.ToTensor()
img = to_tensor(train_dataset[0]['image'])
img
</code></pre>
<p>Converts my images values between 0 and 1 which is expected.  It also converts img which is an ndarray to a torch.Tensor</p>
<p>Previously, without using to_tensor (which I need it now), the following code snippet worked (not sure if this is best way to find means and stds of train set, however now doesn’t work. How can I make it work?</p>
<pre><code class="lang-auto">image_arr = []

for i in range(len(train_dataset)):
    image_arr.append(to_tensor(train_dataset[i]['image']))

print(np.mean(image_arr, axis=(0, 1, 2)))
print(np.std(image_arr, axis=(0, 1, 2)))
</code></pre>
<p>The error is:</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
&lt;ipython-input-147-0e007c030629&gt; in &lt;module&gt;
      4     image_arr.append(to_tensor(train_dataset[i]['image']))
      5 
----&gt; 6 print(np.mean(image_arr, axis=(0, 1, 2)))
      7 print(np.std(image_arr, axis=(0, 1, 2)))

&lt;__array_function__ internals&gt; in mean(*args, **kwargs)

~/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py in mean(a, axis, dtype, out, keepdims)
   3333 
   3334     return _methods._mean(a, axis=axis, dtype=dtype,
-&gt; 3335                           out=out, **kwargs)
   3336 
   3337 

~/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py in _mean(a, axis, dtype, out, keepdims)
    133 
    134 def _mean(a, axis=None, dtype=None, out=None, keepdims=False):
--&gt; 135     arr = asanyarray(a)
    136 
    137     is_float16_result = False

~/anaconda3/lib/python3.7/site-packages/numpy/core/_asarray.py in asanyarray(a, dtype, order)
    136 
    137     """
--&gt; 138     return array(a, dtype, copy=False, order=order, subok=True)
    139 
    140 

ValueError: only one element tensors can be converted to Python scalars
</code></pre>

fadetoblack: <p><code>image_arr</code> seems to be a python list. Can you tell us the shape of <code>image_arr[0]</code>?</p>
<p>If the shape of each element of <code>image_arr</code>  looks like <code>[height,width,channels]</code> then the output of <code>np.mean(image_arr,axis=(0,1,2))</code> should be an array of three values.</p>

Mona_Jalal: <p>yes, I have defined image_arr myself to be a list by <code>image_arr = []</code></p>

Mona_Jalal: <aside class="quote no-group" data-username="Mona_Jalal" data-post="1" data-topic="99432">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mona_jalal/48/25715_2.png" class="avatar"> Mona_Jalal:</div>
<blockquote>
<p><code>ValueError: only one element tensors can be converted to Python scalars</code></p>
</blockquote>
</aside>
<p>did you check the error?</p>
<pre><code class="lang-auto">ValueError: only one element tensors can be converted to Python scalars
</code></pre>

Mona_Jalal: <p>Please note that both of the following solutions are wrong in the sense that they create a vector of 800 values for mean while I only expect 3 values since I only have 3 channels (RGB)</p>
<pre><code class="lang-auto">image_arr = []

for i in range(len(train_dataset)):
    image_arr.append(to_tensor(train_dataset[i]['image']))
                     
mean = torch.mean(torch.stack(image_arr, dim=0), dim=(0, 1, 2))
std = torch.std(torch.stack(image_arr, dim=0), dim=(0, 1, 2))

mean.shape
</code></pre>
<p>and</p>
<pre><code class="lang-auto">
image_arr = []
for i in range(len(train_dataset)):
       image_arr.append((to_tensor(train_dataset[i]['image'])).cpu().detach().numpy())

means = np.mean(image_arr, axis=(0, 1, 2))
stds = np.std(image_arr, axis=(0, 1, 2))
means.shape
</code></pre>

fadetoblack: <p>This should work:</p>
<pre><code class="lang-auto">image_arr = []

for i in range(len(train_dataset)):
    image_arr.append(to_tensor(train_dataset[i]['image']))

image_arr = [x.unsqueeze_(0) for x in image_arr]
image_tens = torch.cat(image_arr, dim=0)

print(image_tens.mean((0,1,2)))
</code></pre>
<p>Basically, if you have a list of tensors, each of shape <code>[height, width, channels]</code>, then you can add one more dimension to each tensor using <code>unsqueeze()</code>, to get each tensor in the list to be of shape <code>[1, height, width, channels]</code>, then you can concatenate all those tensors to get a bigger tensor of size <code>[len(image_arr), height, width, channels]</code>. Now if the whole object is a tensor, it’s easy to calculate mean.</p>

Mona_Jalal: <pre><code class="lang-auto">image_arr = []

for i in range(len(train_dataset)):
    image_arr.append(to_tensor(train_dataset[i]['image']))
                     
mean = torch.mean(torch.stack(image_arr, dim=0), dim=(0, 2, 3))
std = torch.std(torch.stack(image_arr, dim=0), dim=(0, 2, 3))

print(mean.shape)
print("mean is {} and std is {}".format(mean, std))
</code></pre>
<p>worked</p>
<p>and answer is:</p>
<pre><code class="lang-auto">torch.Size([3])
mean is tensor([0.3809, 0.3810, 0.3810]) and std is tensor([0.1127, 0.1129, 0.1130])
</code></pre>

Mona_Jalal: <p>Please check my own solution to see why 0, 1, 2 will not work and give a result of 800 dimensions</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/c/ccbe4a1e83d7cf6430a7dff3764e1d5c85949e06.png" data-download-href="https://discuss.pytorch.org/uploads/default/ccbe4a1e83d7cf6430a7dff3764e1d5c85949e06" title="Screenshot_2020-10-15_01-52-58"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/c/ccbe4a1e83d7cf6430a7dff3764e1d5c85949e06_2_690x263.png" alt="Screenshot_2020-10-15_01-52-58" data-base62-sha1="tdf62qq7WyMBuQvYtZenLqOpRwq" width="690" height="263" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/c/ccbe4a1e83d7cf6430a7dff3764e1d5c85949e06_2_690x263.png, https://discuss.pytorch.org/uploads/default/optimized/3X/c/c/ccbe4a1e83d7cf6430a7dff3764e1d5c85949e06_2_1035x394.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/c/c/ccbe4a1e83d7cf6430a7dff3764e1d5c85949e06_2_1380x526.png 2x" data-dominant-color="EBEBEB"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot_2020-10-15_01-52-58</span><span class="informations">1962×748 97.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

fadetoblack: <p>Apologies, it’s <code>image_tens</code>, not <code>image.tens</code>, edited my response.</p>
<p>Also, it depends on the ordering of the dimensions. If your image shape is <code>[channels, width, height]</code>, then you should take mean over axes 0,2,3 since 1 is channels.</p>

fadetoblack: <p>Yes, as I said, maybe your images are of shape <code>[channels, width, height]</code> and not <code>[width, height, channels]</code>. Basically, the dimension having channels should be skipped while taking the mean. If the 1st dimension has channels, we can pass (0,2,3) to the mean function. If the 3rd dimension has channels, then we use (0,1,2) inside the mean function. Hope I made sense.</p>