CPU RAM is increasing while Profiling

surya00060: <p>Hello All</p>
<p>I have been trying to profile various two-layer CNN and I find my RAM increasing linearly. I have attached my profiling code snippet. Is there something which I can do to reduce RAM access?</p>
<pre><code class="lang-auto">for i in range(len(twopair)):
    model = nn.Sequential(nn.Conv2d(input_channels, twopair[i][0], kernel, stride, padding, bias=False),
    nn.Conv2d(twopair[i][0], twopair[i][1], kernel, stride, padding, bias=False))
    time = []
    for j in range(ExpEpoch):
        x = torch.randn([batch_size, input_channels, input_dim, input_dim])
        with torch.autograd.profiler.profile() as prof:
            y = model(x)
        time.append(prof.self_cpu_time_total)
</code></pre>
<p>Thanks in advance for any replies!</p>

ptrblck: <p>I’m seeing an increased memory usage as well, which will be freed eventually.<br>
You might try to lower the threshold for the garbage collection using <a href="https://docs.python.org/3/library/gc.html?highlight=set_threshold#gc.set_threshold">gc.set_threshold</a>.</p>
<p>Anyway, I would assume it should automatically kick in once you run out of memory.</p>

surya00060: <p>Unfortunately, GC didn’t kick in automatically and the process got killed due to out of memory.</p>

ptrblck: <p>That might be another issue.<br>
Could you post the undefined variables so that we could try to reproduce it?</p>
<p>I just used some dummy setups and saw an increase in memory usage, which was then cleared after a few iterations.</p>

surya00060: <p>Sure. I have been trying to profile for a large number of filters of Convolution on CPUs and found that process gets killed on few Machines with 16, 32 G of RAM.</p>
<pre><code class="lang-auto">ExpEpoch = 30

input_dim = 224
batch_size = 1
kernel = 3
stride = 1
padding = 1
OutputLayers = [3, 6, 12, 16, 18, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 144, 160, 176, 192, 208, 224, 240, 256, 288, 320, 384, 432, 480, 512, 576, 640, 672, 720, 768, 896, 960]
twopair = []
input_channels = 3
for i in OutputLayers:
    for j in OutputLayers:
        twopair.append([i,j])

name =  "backupconv_" + platform.node() + "two.csv"
bfile = open(name,'a')
layer = 2

for i in range(len(twopair)):
    model = nn.Sequential(nn.Conv2d(input_channels, twopair[i][0], kernel, stride, padding, bias=False),
    nn.Conv2d(twopair[i][0], twopair[i][1], kernel, stride, padding, bias=False))
    time = []
    for j in range(ExpEpoch):
        x = torch.randn([batch_size, input_channels, input_dim, input_dim])
        with torch.autograd.profiler.profile() as prof:
            y = model(x)
        time.append(prof.self_cpu_time_total)
    mean_time = statistics.mean(time)
    vari_time = statistics.stdev(time)
    point = [layer, input_dim, kernel, stride, padding, input_channels, twopair[i][0], twopair[i][1],  mean_time, vari_time]
    for itr in point:
        writestring = writestring + str(itr) + ','
    writestring += '\n'
    bfile.write(writestring)
</code></pre>

Ashima_Garg: <p>Were you able to resolve the memory error? I am facing the similar problem, the process gets killed when its out of memory. Is there a solution to resolve this issue?</p>
<p>Thanks.</p>

Ashima_Garg: <p>Can the out of memory (or the process gets killed) get resolved with <code>torch.autograd.profiler.emit_nvtx():</code>?</p>

ptrblck: <p>The original issue was reporting an increase in CPU RAM. Are you seeing the same or is your GPU running out of memory?<br>
In the latter case, could you install the latest nightly binary (or build from source), as <a href="https://github.com/pytorch/pytorch/pull/39962">this fix</a> might be related to an increase in GPU memory usage.</p>