WorldBank-Transport/open-transit-indicators — pull request #449 by flibbertigibbet (2014-11-10 19:58:29+00:00)
Title: Feature/optimize memory usage

Changes to reduce memory usage, primarily in indicator calculations.

* Use "slim" mode of OSM importer, so it will cache to disk, instead of caching all to memory
* Incorporates changes from @echeipesh #445 (this supersedes), and also uses arrays on database import (arrays use less memory than lists)
* Adds JVM arguments for memory profiling with JMX/VisualVM
* Inserts database records in batches to improve speed, particularly of stop times inserts
* Build transit systems for each day of representative week separately in weekly service hours calculation, so they aren't all held in memory at once

With these changes, indicators for Chicago calculate using ~3GB of memory (before, Madrid would use > 4GB; it now uses ~2.5 GB.)