numpy/numpy — pull request #4606 by juliantaylor (2014-04-09 18:16:27+00:00)
Title: ENH: add small memory chunk cache

Add very simplistic caching of small memory blocks (< 256 bytes)
This improves the performance of small array operations by about 15%-20%:

without cache:
In [1]: d = np.array([1,1])
In [2]: %timeit d*d*d*d
1000000 loops, best of 3: 2.1 µs per loop

with cache:
In [1]: d = np.array([1,1])
In [2]: %timeit d*d*d*d
1000000 loops, best of 3: 1.7 µs per loop

The cache is a simple list of buckets of pointers, each bucket contains
pointers to blocks of a minimum size so they can be retrieved in
constant time.

All pointers can still be deleted with free (though one should still use
the appropriate wrappers) but this implies the size of the block must be
known at the time of the free in order to place the pointer in the right
bucket. This should not be an issue for well behaved numpy users.

Technically the ABI is broken in this change as the size of the
dimension memory block has been decreased from 3 * nd to 2 * nd, the
increase to 3 was a relic from the NA branch and is not needed anymore.
It should be unlikely someone relies on this size.