unitedstates/inspectors-general — pull request #47 by spulec (2014-06-05 02:38:02+00:00)
Title: First version of SEC working with everything except OTHER and FOIA

This one was a bit ugly, so I'm going to list out the hacks/todos:

- There is a bug in the HTML [here](https://www.sec.gov/about/offices/oig/inspector_general_audits_reports.shtml) if you look for `Jan. 7, 1997`. This is handled [here](https://github.com/spulec/inspectors-general/compare/unitedstates:master...spulec:sec?expand=1#diff-d11aeeb1079fd73eb765818490615ffcR54) and [here](https://github.com/spulec/inspectors-general/compare/unitedstates:master...spulec:sec?expand=1#diff-d11aeeb1079fd73eb765818490615ffcR94).
- Sometimes multiple reports are listed under the same datetime text(ex "Southeast Regional Office" [here](https://www.sec.gov/about/offices/oig/inspector_general_audits_reports.shtml)). This forces us to pass the last published_on datetime to the parsing of the next report. We might be able to solve this with some sort of smarter parsing of a report's parent node to find the closest datetime.
- `published_date_for_report`: This method is pretty ugly. It started out as a large number of nested try/excepts, but this seemed a bit cleaner.
- The reason that Other and FOIA don't currently work is that some of the documents don't list a published date (ex "Legal Service Corporation OIG System Review Report of the SEC OIG Audit Organization" [here](https://www.sec.gov/about/offices/oig/inspector_general_reppubs_other.shtml)). We can leave the datetimes blank, hardcode them, or create some way to parse them from the pdf report.

This command currently works: `./igs --only=sec --since=1994 --dry_run --topics=ISSUED_REPORTS,INVESTIGATIVE_REPORTS,INVESTIGATIVE_MEMORANDA,SEMIANNUAL_REPORTS,TESTIMONY`