sunlightlabs/openstates — pull request #468 by jakejohns (2013-05-28 21:00:50+00:00)
Title: Scrape individual contact pages for Republican senator email

This should add some emails to the senators. It makes another request to the individual contact pages of the legislator and scrapes their email address from there.

It uses an xpath to find where the "mailto: link" should be and checks it to see if it is in fact a mailto link.

This only works for Republicans, as I have not yet found a way to determine Democrat emails. In addition, while the Republicans sometimes publish an email address here, some do not, and some link to a contact form.

As it stands, I believe this will add the availability of 9 emails.

To write this, I had to extract the basic code, and test it separately, as the methods described in the "Contributing" guide did not seem to work (eg.billy-update )