california-civic-data-coalition/django-calaccess-raw-data — pull request #72 by anabranch (2014-11-19 00:02:53+00:00)
Title: PostgreSQL Support

This resolves #38.
It's been merge-able for a while but I was doing some pep8 and code clean up. On a personal note I find that the line break at 80 characters to be extremely limiting - I don't read code on calculators/small screens and it destroys the expressiveness of some variable names.

10,000 feet view:
I create an empty table and insert all CSV data into that table. I then, through `INSERT INTO` and `SELECT` statements move that data from the temporary table into its true table. In this process I end up having to deal with a lot of unique cases not limited to integer fields being "Y/N", dates formatted in strange and inconsistent ways, and null constraints that are violated by the insert. These are all detailed in the methods like the one below.
```py
def _make_XXXXXXX_case(self, _col):
....
```
I construct the insert and select statements differently as there are issues with column ordering, then attempt to insert - returning the errors with detailed reasoning. On my latest run, I experience no errors however I suspect that more will come up as data is added. In my last couple of pull requests we can see typical fields that cause these issues - I think this would be a great pull request/issue for someone new to django.

Because I have to do some rather incidentally complex manipulations of tables (altering them on the fly) there may be times when tables/columns exist when they shouldn't. A rip and reload will have to occur for this to go away.

On a final note, `postgresql_psycopg2` can complain a bit when being installed - especially inside a virtualenv. I'd love to catalog these errors so that we can throw them in the docs.

Feedback appreciated.