Memory usage is increasing when use cpu to inference resnet101 network model

username: ChinaSilence
cooked: <pre><code class="lang-auto">2020-10-26 16:51:25.000  INFO 6976 --- [     MainThread] __init__             : Extract: /Users/admin/Downloads/1602302351103/test/1600.jpg
torch.Size([1, 3, 1024, 724])
Filename: /Users/admin/code/tezign/cbir-feature-extract/core/resnet.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
   140    506.9 MiB    506.9 MiB           1       @profile
   141                                             def forward(self, x):
   142    553.1 MiB     46.1 MiB           1           x = self.conv1(x)
   143    598.6 MiB     45.5 MiB           1           x = self.bn1(x)
   144    598.6 MiB      0.1 MiB           1           x = self.relu(x)
   145    612.9 MiB     14.3 MiB           1           x = self.maxpool(x)
   146                                         
   147    681.8 MiB     68.9 MiB           1           x = self.layer1(x)
   148    707.1 MiB     25.4 MiB           1           x = self.layer2(x)
   149    755.9 MiB     48.7 MiB           1           x = self.layer3(x)
   150    788.4 MiB     32.5 MiB           1           x = self.layer4(x)
   151    788.4 MiB      0.0 MiB           1           x = self.avgpool(x)
   152                                         
   153    788.4 MiB      0.0 MiB           1           return x


torch.Size([2048])
2020-10-26 16:51:29.000  INFO 6976 --- [     MainThread] __init__             : Extract: /Users/admin/Downloads/1602302351103/test/1701.jpg
torch.Size([1, 3, 768, 1024])
Filename: /Users/admin/code/tezign/cbir-feature-extract/core/resnet.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
   140    685.6 MiB    685.6 MiB           1       @profile
   141                                             def forward(self, x):
   142    733.6 MiB     48.0 MiB           1           x = self.conv1(x)
   143    781.6 MiB     48.0 MiB           1           x = self.bn1(x)
   144    781.6 MiB      0.0 MiB           1           x = self.relu(x)
   145    781.6 MiB      0.0 MiB           1           x = self.maxpool(x)
   146                                         
   147    854.1 MiB     72.4 MiB           1           x = self.layer1(x)
   148    940.2 MiB     86.1 MiB           1           x = self.layer2(x)
   149    985.1 MiB     45.0 MiB           1           x = self.layer3(x)
   150   1022.6 MiB     37.4 MiB           1           x = self.layer4(x)
   151   1022.6 MiB      0.0 MiB           1           x = self.avgpool(x)
   152                                         
   153   1022.6 MiB      0.0 MiB           1           return x


torch.Size([2048])
2020-10-26 16:51:32.000  INFO 6976 --- [     MainThread] __init__             : Extract: /Users/admin/Downloads/1602302351103/test/1728.jpg
torch.Size([1, 3, 1024, 576])
Filename: /Users/admin/code/tezign/cbir-feature-extract/core/resnet.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
   140    817.4 MiB    817.4 MiB           1       @profile
   141                                             def forward(self, x):
   142    853.4 MiB     36.0 MiB           1           x = self.conv1(x)
   143    889.5 MiB     36.0 MiB           1           x = self.bn1(x)
   144    889.5 MiB      0.0 MiB           1           x = self.relu(x)
   145    898.5 MiB      9.0 MiB           1           x = self.maxpool(x)
   146                                         
   147    952.8 MiB     54.3 MiB           1           x = self.layer1(x)
   148   1027.0 MiB     74.2 MiB           1           x = self.layer2(x)
   149   1034.5 MiB      7.5 MiB           1           x = self.layer3(x)
   150   1064.4 MiB     29.9 MiB           1           x = self.layer4(x)
   151   1064.4 MiB      0.0 MiB           1           x = self.avgpool(x)
   152                                         
   153   1064.4 MiB      0.0 MiB           1           return x


torch.Size([2048])
2020-10-26 16:51:35.000  INFO 6976 --- [     MainThread] __init__             : Extract: /Users/admin/Downloads/1602302351103/test/1729.jpg
torch.Size([1, 3, 1024, 576])
Filename: /Users/admin/code/tezign/cbir-feature-extract/core/resnet.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
   140    960.9 MiB    960.9 MiB           1       @profile
   141                                             def forward(self, x):
   142    996.9 MiB     36.0 MiB           1           x = self.conv1(x)
   143   1032.8 MiB     35.9 MiB           1           x = self.bn1(x)
   144   1032.8 MiB      0.0 MiB           1           x = self.relu(x)
   145   1041.8 MiB      9.0 MiB           1           x = self.maxpool(x)
   146                                         
   147   1095.6 MiB     53.7 MiB           1           x = self.layer1(x)
   148   1149.4 MiB     53.8 MiB           1           x = self.layer2(x)
   149   1149.4 MiB      0.0 MiB           1           x = self.layer3(x)
   150   1151.5 MiB      2.1 MiB           1           x = self.layer4(x)
   151   1144.8 MiB     -6.8 MiB           1           x = self.avgpool(x)
   152                                         
   153   1144.8 MiB      0.0 MiB           1           return x


torch.Size([2048])
2020-10-26 16:51:37.000  INFO 6976 --- [     MainThread] __init__             : Extract: /Users/admin/Downloads/1602302351103/test/1730.jpg
torch.Size([1, 3, 1024, 576])
Filename: /Users/admin/code/tezign/cbir-feature-extract/core/resnet.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
   140    960.4 MiB    960.4 MiB           1       @profile
   141                                             def forward(self, x):
   142    996.4 MiB     36.0 MiB           1           x = self.conv1(x)
   143   1032.3 MiB     35.9 MiB           1           x = self.bn1(x)
   144   1032.3 MiB      0.0 MiB           1           x = self.relu(x)
   145   1041.3 MiB      9.0 MiB           1           x = self.maxpool(x)
   146                                         
   147   1095.1 MiB     53.7 MiB           1           x = self.layer1(x)
   148   1148.9 MiB     53.8 MiB           1           x = self.layer2(x)
   149   1148.9 MiB      0.0 MiB           1           x = self.layer3(x)
   150   1151.0 MiB      2.1 MiB           1           x = self.layer4(x)
   151   1148.1 MiB     -2.9 MiB           1           x = self.avgpool(x)
   152                                         
   153   1148.1 MiB      0.0 MiB           1           return x


torch.Size([2048])
2020-10-26 16:51:40.000  INFO 6976 --- [     MainThread] __init__             : Extract: /Users/admin/Downloads/1602302351103/test/1873
torch.Size([1, 3, 835, 1024])
Filename: /Users/admin/code/tezign/cbir-feature-extract/core/resnet.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
   140    969.1 MiB    969.1 MiB           1       @profile
   141                                             def forward(self, x):
   142   1021.4 MiB     52.3 MiB           1           x = self.conv1(x)
   143   1073.6 MiB     52.3 MiB           1           x = self.bn1(x)
   144   1073.6 MiB      0.0 MiB           1           x = self.relu(x)
   145   1089.9 MiB     16.3 MiB           1           x = self.maxpool(x)
   146                                         
   147   1168.6 MiB     78.7 MiB           1           x = self.layer1(x)
   148   1223.2 MiB     54.6 MiB           1           x = self.layer2(x)
   149   1272.3 MiB     49.1 MiB           1           x = self.layer3(x)
   150   1280.7 MiB      8.4 MiB           1           x = self.layer4(x)
   151   1280.7 MiB      0.0 MiB           1           x = self.avgpool(x)
   152                                         
   153   1280.7 MiB      0.0 MiB           1           return x


torch.Size([2048])
2020-10-26 16:51:43.000  INFO 6976 --- [     MainThread] __init__             : Extract: /Users/admin/Downloads/1602302351103/test/1874
torch.Size([1, 3, 961, 1024])
Filename: /Users/admin/code/tezign/cbir-feature-extract/core/resnet.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
   140   1127.2 MiB   1127.2 MiB           1       @profile
   141                                             def forward(self, x):
   142   1187.4 MiB     60.2 MiB           1           x = self.conv1(x)
   143   1247.5 MiB     60.1 MiB           1           x = self.bn1(x)
   144   1247.5 MiB      0.0 MiB           1           x = self.relu(x)
   145   1262.5 MiB     15.0 MiB           1           x = self.maxpool(x)
   146                                         
   147   1413.0 MiB    150.4 MiB           1           x = self.layer1(x)
   148   1542.6 MiB    129.6 MiB           1           x = self.layer2(x)
   149   1568.0 MiB     25.5 MiB           1           x = self.layer3(x)
   150   1613.7 MiB     45.7 MiB           1           x = self.layer4(x)
   151   1613.7 MiB      0.0 MiB           1           x = self.avgpool(x)
   152                                         
   153   1613.7 MiB      0.0 MiB           1           return x


torch.Size([2048])
2020-10-26 16:51:48.000  INFO 6976 --- [     MainThread] __init__             : Extract: /Users/admin/Downloads/1602302351103/test/1875
torch.Size([1, 3, 788, 1024])
Filename: /Users/admin/code/tezign/cbir-feature-extract/core/resnet.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
   140   1424.6 MiB   1424.6 MiB           1       @profile
   141                                             def forward(self, x):
   142   1473.8 MiB     49.3 MiB           1           x = self.conv1(x)
   143   1523.1 MiB     49.3 MiB           1           x = self.bn1(x)
   144   1523.1 MiB      0.0 MiB           1           x = self.relu(x)
   145   1535.4 MiB     12.3 MiB           1           x = self.maxpool(x)
   146                                         
   147   1609.6 MiB     74.2 MiB           1           x = self.layer1(x)
   148   1661.2 MiB     51.6 MiB           1           x = self.layer2(x)
   149   1707.8 MiB     46.6 MiB           1           x = self.layer3(x)
   150   1727.7 MiB     19.8 MiB           1           x = self.layer4(x)
   151   1727.7 MiB      0.0 MiB           1           x = self.avgpool(x)
   152                                         
   153   1727.7 MiB      0.0 MiB           1           return x


torch.Size([2048])
2020-10-26 16:51:51.000  INFO 6976 --- [     MainThread] __init__             : Extract: /Users/admin/Downloads/1602302351103/test/1876
torch.Size([1, 3, 744, 1024])
Filename: /Users/admin/code/tezign/cbir-feature-extract/core/resnet.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
   140   1561.1 MiB   1561.1 MiB           1       @profile
   141                                             def forward(self, x):
   142   1607.6 MiB     46.5 MiB           1           x = self.conv1(x)
   143   1654.1 MiB     46.5 MiB           1           x = self.bn1(x)
   144   1654.1 MiB      0.0 MiB           1           x = self.relu(x)
   145   1665.7 MiB     11.6 MiB           1           x = self.maxpool(x)
   146                                         
   147   1735.8 MiB     70.1 MiB           1           x = self.layer1(x)
   148   1836.8 MiB    100.9 MiB           1           x = self.layer2(x)
   149   1869.1 MiB     32.3 MiB           1           x = self.layer3(x)
   150   1869.2 MiB      0.1 MiB           1           x = self.layer4(x)
   151   1869.2 MiB      0.0 MiB           1           x = self.avgpool(x)
   152                                         
   153   1869.2 MiB      0.0 MiB           1           return x
</code></pre>
<p>This is memory usage log, tracked with “Memory-Profile” python package<br>
I have tried  "self.net.eval() " and “with torch.no_grad()”, but It was not worked.<br>
And I find a weird phenomenon, When input varied size images, that Memory increasing is bigger than same size input.<br>
Does anyone have encountered the same problem?</p>
<p>Main environment:<br>
Mac OS<br>
Python 3.7<br>
PyTorch 1.2(also tried with PyTorch 1.5)</p>

username: BboyHanat
cooked: <p>see this:<br>
</p><aside class="onebox githubissue">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/issues/27971" target="_blank" rel="noopener nofollow ugc">github.com/pytorch/pytorch</a>
  </header>
  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewBox="0 0 14 16" aria-hidden="true"><path d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/issues/27971" target="_blank" rel="noopener nofollow ugc">High memory usage for CPU inference on variable input shapes (10x compared to pytorch 1.1)</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2019-10-15" data-time="10:23:00" data-timezone="UTC">10:23AM - 15 Oct 19 UTC</span>
      </div>

        <div class="date">
          closed <span class="discourse-local-date" data-format="ll" data-date="2020-05-16" data-time="20:01:37" data-timezone="UTC">08:01PM - 16 May 20 UTC</span>
        </div>

      <div class="user">
        <a href="https://github.com/lopuhin" target="_blank" rel="noopener nofollow ugc">
          <img alt="lopuhin" src="https://avatars2.githubusercontent.com/u/424613?v=4" class="onebox-avatar-inline" width="20" height="20">
          lopuhin
        </a>
      </div>
    </div>
  </div>
</div>

<div class="github-row">
  <p class="github-content">🐛 Bug
In pytorch 1.3, when doing inference with resnet34 on CPU with variable input shapes, much more memory is used compared...</p>
</div>

<div class="labels">
    <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">high priority</span>
    <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">module: cpu</span>
    <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">module: memory usage</span>
    <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">module: mkldnn</span>
    <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">triaged</span>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>