artemp/trac2github — Issue #596: Feature attributes being ignored when the .dbf is bigger than 2 GiB
Author: artemp
Created: 2011-10-09 20:42:31+00:00
State: open
Labels: Core_Library

Suppose a big shapefile, with a .dbf file bigger than 2 GiB. If there is a filter on the attributes of a feature, but these attributes don't reside in the lower 2 GiB of the .dbf file, the filters may not work.

This issue has been raised during the FOSS4G 2010 WMS benchmark (see http://wiki.osgeo.org/wiki/Benchmarking_2010). The "buildings" layer has a .dbf file which is 2.8 GiB in size. The rendering of buildings depend on the "singular" attribute of the feature, but it is ignored for some of the features.

As of now, the issue can be reproduced at http://demo.cleverelephant.ca/~pramsey/maps.html?zoom=15&lat=40.43703&lon=-3.66632&layers=B - Mapnik ignores singular buildings in that area, as the singular attribute of these buildings reside in the upper part of the .dbf file.

MapServer suffered from this problem also (see http://trac.osgeo.org/mapserver/ticket/3514). The idea behind the MapServer fix (using bigger integers for offsets, and a 64-bit fseek() ) could be applied to Mapnik.