unitedstates/inspectors-general — pull request #61 by spulec (2014-07-02 01:22:42+00:00)
Title: H&HS Scraper

This scraper still needs some work, but want to get the discussion started per #59. There are still a lot of pdbs and such that I'm using for debugging.

All of the uncommented topics seem to be working.

No work has been done on the archive pages yet.

Lines 305-311 are pretty hacky. Some pages just don't report when they are published. See "Implementation of the Core Medical Services Requirement in the Ryan White Program" on http://oig.hhs.gov/reports-and-publications/oei/r.asp. I did notice that the report ids seem to specify month/year combinations that are about 1-2 years before when the report is published. My guess is that this datetime is given when the report id is requested. I'm using this as a fallback for some reports. This is obviously suboptimal, but the alternative is hardcoding a couple hundred reports. Maybe add an additional field noting that this is not the real published date?