Changing built-in ResNet50 model to 1 channel images - how to set transforms.Normalize([...])?

cdahms: <p>I’m attempting to use the PyTorch built-in ResNet 50 model from <a href="https://pytorch.org/docs/stable/torchvision/models.html" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/torchvision/models.html</a> with single-channel (grayscale) images.</p>
<p>I figured out from various posts on this forum that I needed to change my model setup like so:</p>
<pre><code class="lang-auto"># MyResNet50

import torchvision
import torch.nn as nn

def buildResNet50Model(numClasses):
    # get the stock PyTorch ResNet50 model w/ pretrained set to True
    model = torchvision.models.resnet50(pretrained = True)

    # freeze all model parameters so we don’t backprop through them during training (except the FC layer that will be replaced)
    for param in model.parameters():
        param.requires_grad = False
    # end for
    
    # !!!!!! this line is specific to the 3 channel to one channel change, other lines in this function are the same as before !!!!!!
    # change 1st conv layer from 3 channel to 1 channel
    model.conv1 = nn.Conv2d(1, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)

    # the last (fully connected) layer per the number of classes
    # first, get/save the current number of input features to the fc layer
    numFcInputs = model.fc.in_features
    # now replace the fc layer with minor changes, and using our number of classes
    model.fc = nn.Sequential(nn.Linear(numFcInputs, 256),
                             nn.ReLU(),
                             nn.Dropout(0.2),
                             nn.Linear(256, numClasses))

    return model
# end function
</code></pre>
<p>Then when I went to train again I got this error:</p>
<blockquote>
<p>RuntimeError: output with shape [1, 224, 224] doesn’t match the broadcast shape [3, 224, 224]</p>
</blockquote>
<p>I figured out from this post <a href="https://stackoverflow.com/questions/59218671/runtimeerror-output-with-shape-1-224-224-doesnt-match-the-broadcast-shape" class="inline-onebox" rel="noopener nofollow ugc">python - RuntimeError: output with shape [1, 224, 224] doesn't match the broadcast shape [3, 224, 224] - Stack Overflow</a> that I need to change the transforms.Normalize part of the transform from 3 numbers to 1.  Based on various PyTorch examples I’m using this for a transform:</p>
<pre><code class="lang-auto">TRANSFORM = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.CenterCrop(size=224),
    transforms.ToTensor(),
    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ])
</code></pre>
<p>so it seems I need to change the <code>transforms.Normalize</code> line to:</p>
<p><code>transforms.Normalize([x], [y])</code></p>
<p>Where <code>x</code> and <code>y</code> are floating point numbers, at which point the network then trains successfully.</p>
<p>My question is, how do I determine x and y?  Here are some possibilities I could think of:</p>
<pre><code class="lang-auto"># use 1st number
transforms.Normalize([0.485], [0.229])
# use middle number
transforms.Normalize([0.456], [0.224])
# use average
transforms.Normalize([0.499], [0.226])
# use nice round numbers
transforms.Normalize([0.5], [0.25])
# use 0.5 for both
transforms.Normalize([0.5], [0.5])
</code></pre>
<p>Since the origin of the of the original six numbers <code>[0.485, 0.456, 0.406], [0.229, 0.224, 0.225]</code> is not especially clear (see <a href="https://github.com/pytorch/vision/issues/1439" class="inline-onebox" rel="noopener nofollow ugc">Origin of the means and stds used for preprocessing? · Issue #1439 · pytorch/vision · GitHub</a>) I’m not sure how to go about determining which of the above (or something entirely different) is best, other than trial and error.  Any suggestions?</p>

ptrblck: <p>You could calculate the stats from your current training dataset, as was done for the <code>ImageNet</code> data for the posted values (or just use the <code>0.5</code> “defaults”).</p>

cdahms: <p>Thanks for the update, it looks like I should follow these posts to calculate stats:</p>
<aside class="quote quote-modified" data-post="1" data-topic="34949">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kuzand/48/8268_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/computing-the-mean-and-std-of-dataset/34949">Computing the mean and std of dataset</a> 
  </div>
  <blockquote>
    Hello. So I am trying to compute the mean and the standard deviation per channel of my train dataset (three-channel images of different shapes). 
For the mean I can do it in two ways, but I get slightly different results. 
import torch
from torchvision import datasets, transforms

dataset = datasets.ImageFolder('train',
                 transform=transforms.ToTensor())

First computation: 
mean = 0.0
for img, _ in dataset:
    #mean += img.sum([1,2])/torch.numel(img[0])
    mean += img.mean([1,2…
  </blockquote>
</aside>

<aside class="quote quote-modified" data-post="1" data-topic="84890">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/h/e47774/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/efficient-way-of-calculating-mean-std-etc-of-subset-dataset/84890">Efficient way of calculating mean, std, etc of Subset Dataset</a> 
  </div>
  <blockquote>
    I have a Dataset class that loads two datasets from their respective folders (train and test). 
I would like to create a validation set from the training set. For this I am using the random_split function. 
This results in two Subset-Datasets: train_dataset and valid_dataset. For normalization I would like to calculate the mean and std (or min/max) of the training set, but it is not possible to do a simple call of mean() to the dataset, because it is a Subset. 
What I can do is calculate the mea…
  </blockquote>
</aside>