mapbox/tilemill — Issue #785: Faster wgs84 rendering support in TileMill
Author: springmeyer
Created: 2011-10-12 19:03:47+00:00
State: open

Its very common for source data to be in EPSG:4326/WGS84, whether a small geojson file, a massive table in postgis, or a county shapefile. The rendering of this data needs to be ultra fast, and can be.

Technically the transformation needed for displaying this data on a spherical mercator map is trivially easy and fast, and spherical mercator is partly designed to make this so. So, ideally it would have no noticeable performance impact to store and render data layers from WGS84.

In reality however, because proj4 is used under the hood in Mapnik for on-the-fly re-projection, this transformation is not trivially fast. Also, and this is more of a problem, because of the lack of thread safety in proj4, Mapnik (for years now, originally to ensure the tile.osm.org server did not crash) has used scoped mutex locks around pj_transform.

The pj_transform threading issues have long ago been solved in yet-unreleased proj4 trunk by Frank, and so for our TileMill OSX app we statically link the latest proj4 trunk code, and even build it with `--no-mutex`, leading to much faster reprojection.

But on Ubuntu the TileMill packages likely are using proj 4.7 which leads to massively slower reprojection in Mapnik: https://github.com/mapnik/mapnik/blob/master/src/proj_transform.cpp#L155-157

So, potential solutions are:

* Try to support getting a proj 4.8 release out in the wild so that ubuntu packages can pull it in: http://trac.osgeo.org/proj/milestone/4.8.0
* Customize our tilemill.deb package to use a custom compiled proj4 trunk (has added benefit of also being able to build proj4 `--no-mutex`)
* Avoid proj4 completely and leverage custom, simplified, and very fast built in transforms in mapnik. This is already implemented in https://github.com/mapnik/mapnik/commit/ac3e43e5a4a4214c7 but not leveraged in tilemill because the proj4 string matching does not work. This is because TileMill does not use the `+init:epsg` syntax for the reasons of: 1) we don't trust the proj4 string that proj4 epsg files carry for 900913 and/or 3857, 2) we need to ensure that our spherical mercator proj4 has +wkt and +over (#330), and 3) our TileMill OSX app does not bundle the epsg files so does not support the +init syntax at all yet (#560)