How to preprocess input for pre trained networks?

tsterin: <p>Hi all,<br>
I was wondering, when using the pretrained networks of torchvision.models module, what preprocessing should be done on the input images we give them ?<br>
For instance I remember that if you use VGG 19 layers you should substract the following means [103.939, 116.779, 123.68].<br>
Where can I find these numbers (and even better with std infos) for alexnet, resnet and squeezenet ?</p>
<p>Thank you very much</p>

smth: <p>All pretrained torchvision models have the same preprocessing, which is to normalize using the following mean/std values: <a href="https://github.com/pytorch/examples/blob/97304e232807082c2e7b54c597615dc0ad8f6173/imagenet/main.py#L197-L198" rel="nofollow noopener">https://github.com/pytorch/examples/blob/97304e232807082c2e7b54c597615dc0ad8f6173/imagenet/main.py#L197-L198</a> (input is RGB format)</p>

tsterin: <p>Thank you very much!</p>

qianguih: <p>Hi, it looks like the pixel intensities have been rescaled to [0 1] before normalization. It that right?</p>

smth: <p><a class="mention" href="/u/qianguih">@qianguih</a> yes they have to be RGB normalized to [0, 1] before further applying the normalization that I pointed out.</p>

qianguih: <p>I see. Thank you very much!</p>

ecolss: <p>This is important information, I wonder it’s not put in the doc but in the example code?</p>

mehdi-shiba: <p>Agreed. If it wasn’t for this thread, I would have missed this important Normalization step for sure. It would be nice if it could be added to the documentation.</p>

mattmacy: <p>This is pretty key information. Without doing this, and only doing mean centering and stddev normalization of the original Hunsfield units, I need to keep batch normalization enabled during test to see reasonable results from my volumetric segmentation network.</p>
<p>This should really be in bold somewhere.</p>

youkaichao: <p>can we put the <strong>mean</strong> and <strong>std</strong> inside the resnet model?just need to register a buffer.</p>

Deeply: <aside class="quote no-group quote-post-not-found" data-username="Sphinxs" data-post="11" data-topic="683">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sphinxs/48/8883_2.png" class="avatar"> Sphinxs:</div>
<blockquote>
<p>the values of the channels <code>0</code> and <code>2</code> are transformed to  <code>inf</code> , is that right ?</p>
</blockquote>
</aside>
<p>Of course not.<br>
Obviously, you have ‘division by zero’ somewhere. Try to debug the code to figure out the source of error.</p>

Deeply: <p>Shouldn’t the <a href="https://pytorch.org/docs/stable/torchvision/transforms.html#torchvision.transforms.Normalize" rel="noopener nofollow ugc">mean and std</a> be tuples?</p>
<blockquote>
<p>normalize = torchvision.transforms.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225) )</p>
</blockquote>
<p>Or, it is OK to use list sequences as shown in your case?</p>

Deeply: <p>Maybe you can remove this normalization to see if you are still having these <code>inf</code> values.<br>
If yes, this means the source of error is somewhere else than <code>Normalize</code>.<br>
If not, could you try a dummy normalization with<br>
<code>normalize = torchvision.transforms.Normalize(mean=(1, 1, 1), std=(1, 1, 1) )</code><br>
?</p>

e3312f50f3ba76f35a60: <p>If I use the pretrained model on ImageNet and fine-tune it on my own dataset, should I re-calculate the mean and std with my own dataset?</p>

jdhao: <p>Using the mean and std on ImageNet is pretty standard practice.  Since the mean and std are calculated using a million of images, the statistics is pretty stable. Also the pretrained model is trained using the mean and std in ImageNet. I do not recommend changing the mean and std to that on your small dataset.</p>

r0mer0m: <p>Thank you for your comment. I’ve been having that doubt a while ago. So basically what I infer from the comments as a summary is that the best practice is to leverage:</p>
<blockquote>
<p>Stability of our images (mean/std)<br>
Similarity of our image dataset to ImageNet</p>
</blockquote>
<p>in our specific task and dataset to check what’s the best option  to normalise.</p>

Nikronic: <p>Actually, this link points to correct lines:<br>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/pytorch/examples/blob/master/imagenet/main.py#L197-L198" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/examples/blob/master/imagenet/main.py#L197-L198" target="_blank" rel="nofollow noopener">pytorch/examples/blob/master/imagenet/main.py#L197-L198</a></h4>
<pre class="onebox"><code class="lang-py"><ol class="start lines" start="197" style="counter-reset: li-counter 196 ;">
<li>normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406],</li>
<li>                                 std=[0.229, 0.224, 0.225])</li>
</ol></code></pre>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>

MasayoMusic: <aside class="quote no-group" data-username="Nikronic" data-post="20" data-topic="683">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nikronic/48/71873_2.png" class="avatar"> Nikronic:</div>
<blockquote>
<ul>
<li>normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406],</li>
<li>std=[0.229, 0.224, 0.225])</li>
</ul>
</blockquote>
</aside>
<p>Is this applicable to all imagenet models that come with Pytorch?<br>
I remember in keras each model had it’s own specific preprocessing function,</p>

Nikronic: <p>Yes, this is the step has been used in all models of ImageNet.<br>
Here is the link to ReadMe:<br>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/pytorch/examples/blob/master/imagenet/README.md#imagenet-training-in-pytorch" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/examples/blob/master/imagenet/README.md#imagenet-training-in-pytorch" target="_blank" rel="nofollow noopener">pytorch/examples/blob/master/imagenet/README.md#imagenet-training-in-pytorch</a></h4>
<pre><code class="lang-md"># ImageNet training in PyTorch

This implements training of popular model architectures, such as ResNet, AlexNet, and VGG on the ImageNet dataset.

## Requirements

- Install PyTorch ([pytorch.org](http://pytorch.org))
- `pip install -r requirements.txt`
- Download the ImageNet dataset and move validation images to labeled subfolders
    - To do this, you can use the following script: https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/valprep.sh

## Training

To train a model, run `main.py` with the desired model architecture and the path to the ImageNet dataset:

```bash
python main.py -a resnet18 [imagenet-folder with train and val folders]
```

The default learning rate schedule starts at 0.1 and decays by a factor of 10 every 30 epochs. This is appropriate for ResNet and models with batch normalization, but too high for AlexNet and VGG. Use 0.01 as the initial learning rate for AlexNet or VGG:
</code></pre>

  This file has been truncated. <a href="https://github.com/pytorch/examples/blob/master/imagenet/README.md#imagenet-training-in-pytorch" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>
<p>All of the mentioned models use this normalization.</p>

pp18: <p><a class="mention" href="/u/smth">@smth</a> I have confusion regarding the use of same mean and std for all data. Why can’t we use per image in that way each image is independent for nomalization?</p>