mapnik-org/mapnik — Issue #655: mapnik2 uses a lot more memory than before
Author: artemp
Created: 2011-10-11 08:06:40+00:00
State: open
Labels: Core_Library

When I render the following tile with mod_tile's renderd it requires significantly more memory when I use the trunk mapnik2 code instead of the mapnik-0.7.x branch.

http://localhost/osm_tiles2/10/511/340.png
-- this meta tile covers all of central London and hence it is a pretty difficult test case.

With mapnik-0.7.x the peak virtual memory usage of the renderd process is around 1GB
{{{
renderd[28969]: DEBUG: Got command RenderPrio fd(9) xml(default), z(10), x(511), y(340)
renderd[28969]: DEBUG: Connection 0, fd 9 closed, now 0 left
renderd[28969]: DEBUG: DONE TILE default 10 504-511 336-343
renderd[28969]: DEBUG: DONE TILE default 10 504-511 336-343 in 26.177 seconds
$ grep VmPeak /proc/`pidof renderd`/status
VmPeak: 1028044 kB
}}}

With mapnik2 the peak usage is over 8GB:
{{{
renderd[29347]: DEBUG: Got command RenderPrio fd(67) xml(default), z(10), x(511), y(340)
renderd[29347]: DEBUG: Connection 0, fd 67 closed, now 0 left
renderd[29347]: DEBUG: DONE TILE default 10 504-511 336-343
renderd[29347]: DEBUG: DONE TILE default 10 504-511 336-343 in 182.653 seconds
$ grep VmPeak /proc/`pidof renderd`/status
VmPeak: 8434556 kB
}}}

The rendering time is a lot slower, probably due to the swapping caused by this extra memory usage.

I have not dug any deeper to figure out where the memory is being used. I'll probably start by turning off the new featureset caching because it looks like a likely culprit.