newsapps/django-boundaryservice — pull request #39 by palewire (2013-05-01 04:51:23+00:00)
Title: loadshapefiles refactor for speed

I was able to drastically increase the speed of the loadshapefile management comment by changing a few small things.

1. The ``len(layer)`` to set the ``count`` attribute of new ``BoundarySet`` objects takes a really long time, to my surprise. Just initializing the set with zero in that field, and then letting the count be calculated via SQL after everything's in the database is way faster. And since the app was doing that recalculation anyway, you don't have to add any code.

2. A modification of the process that guarantees a unique slug that doesn't require pulling every single slug out of the database and into memory with every save call. I replaced that with some SQL count queries that I suspect are faster.

The result is that time it took my laptop to load a SHP file with ~3000 features dropped from more than 8 hours to less than 3 minutes.