cleaned up code, changed to use SQL statements for speed, removed ar-import, added indexes to database to bypass as much rails as possible