MapofLife/MOL — Issue #48: Initial hash load for loader.py duplication testing takes too long
Author: gaurav
Created: 2012-01-29 20:25:55+00:00
State: open
Labels: Priority

So I'm starting to see timeouts when downloading the list of hashes from CartoDB. We need some kind of cursor, ideally at the CartoDB API level, but if not, at the SQL level (i.e. using LIMIT and OFFSET to download the hashes). I'm marking this as priority, since this is holding up the rangemap upload.