openplans/OpenTripPlanner — Issue #566: Crop PBF or XML OSM file to bounding box/polygon within graphbuilder 
Author: andrewbyrd
Created: 2011-11-27 02:27:31+00:00
State: open

This ticket is a follow-up to a conversation with David Emory. Currently, OTP allows fetching and cacheing just enough OSM tiles to provide coverage for a set of transit stops, but this depends on a server that is not really intended for bulk downloading. The alternative is to use a local file containing OSM data (planet or regional extract); currently these must be pre-cropped with external tools (usually Osmosis) to avoid importing an unnecessarily huge amount of street data into the OTP graph.

For certain applications it would be very practical to keep a continent- or planet-sized protobuf OSM (PBF) file on the local machine and extract only the necessary data on the fly during graph building. This involves filtering by geographic location, as well as filtering out irrelevant ways such as buildings and administrative borders.

There are two separate issues here:
A. Derive an appropriate bounding box or polygon from GTFS and/or other information.
B. Filter a PBF to get OSM for that area, and do so efficiently even if it is planet-sized.

A. Options for deriving the bouding polygon:

- One crude bounding box. While a single bounding box would be the wrong approach for a huge regional graph (Pacific Northwest) it would be fine for Portland and probably much more efficient than the irregularly-shaped union of a bunch of buffers around transit stops.

- (Convex hull around) the union of buffers / bounding boxes around stops.

We should provide a couple of options, and decide automatically between them after examining the stops' locations.

JTS has buffer creation, "contains" testing, constructuve geometry and seemingly anything else we would need.

Judging from the source code JTS "contains" relation testing is 2-step : bounding boxes are compared before the true, more complex shapes are used. If so we might as well use more complicated bounding polygons. [http://jts-topo-suite.svn.sourceforge.net/viewvc/jts-topo-suite/trunk/jts/java/src/com/vividsolutions/jts/geom/Geometry.java?revision=475&view=markup]

B. Options for filtering. Osmosis contains code to do all of this. See:
org.openstreetmap.osmosis.areafilter
org.openstreetmap.osmosis.core.filter.common, specifically DynamicIdTracker

Option 1:
Use Osmosis as a Maven dependency. Maven has already been broken into subprojects, and the core module does contain a Maven POM and distribution repository information. However, the area filter code is located in another module, and the Osmosis core module has some database-related transitive dependencies, since the OSM filtering capabilities are not the main purpose of Osmosis. There is discussion on the Osmosis dev list that is supportive of modularizing Osmosis and making it more usable as a library.

Option 2:
Copy all relevant Osmosis code into the OTP repository. The Osmosis license allows this (the code is in the public domain), and it should be only a few class files. When and if Osmosis is more fully mavenized and modularized, we could then easily switch over to using it as a Maven dependency. This seems like a good option to me.

Option 3:
Rewrite code that does the same thing as Osmosis. In priciple, filtering OSM is not particularly complicated, but it seems like a waste of time / bad form to reinvent convenient things like Osmosis's dynamic node tracker that breaks the nodeId "address space" into blocks and switches between bitfields and lists as needed to balance memory use and speed.