Issue 1:
Hi everyone,

I've found a nasty memory leak while using np.random.shuffle:

import numpy as np

@profile
def shuffle_leaktest():
x = np.random.randn(1000, 1000)
for ii in xrange(1000):
np.random.shuffle(x)
del x

if __name__ == '__main__':
print "numpy version: " + np.__version__
shuffle_leaktest()

$ python -m memory_profiler shuffle_leaktest.py

numpy version: 1.9.0.dev-fae89b0
Filename: shuffle_leaktest.py

Line # Mem usage Increment Line Contents
================================================
3 110.535 MiB 0.000 MiB @profile
4 def shuffle_leaktest():
5 118.195 MiB 7.660 MiB x = np.random.randn(1000, 1000)
6 337.770 MiB 219.574 MiB for ii in xrange(1000):
7 337.770 MiB 0.000 MiB np.random.shuffle(x)
8 337.770 MiB 0.000 MiB del x

Using git bisect I've tracked down the first instance of memory leakage to commit 607863d.

No leak:

numpy version: 1.9.0.dev-c7a30d5
Filename: shuffle_leaktest.py

Line # Mem usage Increment Line Contents
================================================
3 103.633 MiB 0.000 MiB @profile
4 def shuffle_leaktest():
5 111.285 MiB 7.652 MiB x = np.random.randn(1000, 1000)
6 111.305 MiB 0.020 MiB for ii in xrange(1000):
7 111.305 MiB 0.000 MiB np.random.shuffle(x)
8 103.672 MiB -7.633 MiB del x

Leak:

numpy version: 1.9.0.dev-607863d
Filename: shuffle_leaktest.py

Line # Mem usage Increment Line Contents
================================================
3 112.664 MiB 0.000 MiB @profile
4 def shuffle_leaktest():
5 120.316 MiB 7.652 MiB x = np.random.randn(1000, 1000)
6 183.031 MiB 62.715 MiB for ii in xrange(1000):
7 183.031 MiB 0.000 MiB np.random.shuffle(x)
8 175.398 MiB -7.633 MiB del x

However, this leak seems to be much less pronounced than the one I'm currently seeing in master. It seems that things suddenly got a lot worse with commit 9f8771a:

Small leak:

numpy version: 1.9.0.dev-f57c77b
Filename: shuffle_leaktest.py

Line # Mem usage Increment Line Contents
================================================
3 87.219 MiB 0.000 MiB @profile
4 def shuffle_leaktest():
5 94.871 MiB 7.652 MiB x = np.random.randn(1000, 1000)
6 157.789 MiB 62.918 MiB for ii in xrange(1000):
7 157.789 MiB 0.000 MiB np.random.shuffle(x)
8 150.156 MiB -7.633 MiB del x

Big leak:

numpy version: 1.9.0.dev-9f8771a
Filename: shuffle_leaktest.py

Line # Mem usage Increment Line Contents
================================================
3 118.914 MiB 0.000 MiB @profile
4 def shuffle_leaktest():
5 126.566 MiB 7.652 MiB x = np.random.randn(1000, 1000)
6 346.133 MiB 219.566 MiB for ii in xrange(1000):
7 346.133 MiB 0.000 MiB np.random.shuffle(x)
8 346.133 MiB 0.000 MiB del x

Issue 2:
Hi everyone,

I found a memory leak writing code using the theano module and I was able to trace it back to numpy.

I already posted on the theano mailing list about it

Sample code to reproduce the leak:

Required modules:

Numpy (commit: 0c9f285f38ea4d143c5e79badd0d36cb808242a6)

Scipy (commit: d9a8c214c4cea75f7d240957a646b562cf081f8b)

Theano (commit: 26d913091e56b3e80e7c562175b856a90dd782e4)

for "@Profile" decorator: memory_profiler and psutils

I also use OpenBlas

import theano
from theano import tensor as T
import numpy as np

k = T.iscalar("k")
A = T.vector("A")

# Symbolic description of the result
result, updates = theano.scan(fn=lambda prior_result, A: prior_result * A,
outputs_info=T.ones_like(A),
non_sequences=A,
n_steps=k)

print "Compiling function"
power = theano.function(inputs=[A,k], outputs=result, updates=updates)
print "Compiling function - done"

ip = np.arange(10000)
x = None
print "Starting loop"
@profile
def test():
for i in xrange(10000):
x = power(ip,6)
test()
print "Done."

For more logs:

https://gist.github.com/ogh/b6694c8eb7793b454dee

With the same configuration except for numpy switched out with a version from last September everything works fine without any leak.

(tried the November version since I was using it on a server where I did not have the memory leak)

The version of numpy that works without a leak: commit135443768a24ab4fbfd4fa5c8fc40f27d2e25c96

Can you give me any advice on how to narrow down the problem further?