Repository: openstates/openstates-scrapers

Issue:
HI scraper failing since at least 2017-05-04
HI has been failing since 2017-05-04

Based on automated runs it appears that HI has not run successfully in 2 days (2017-05-04).


```
  09:29:34 INFO billy: Save bill upper 2017 Regular Session: SB1165
09:29:34 INFO scrapelib: GET - http://capitol.hawaii.gov/Archives/measure_indiv_Archives.aspx?billtype=SB&billnumber=1166&year=2017
09:29:36 INFO billy: Save bill upper 2017 Regular Session: SB1166
09:29:36 INFO scrapelib: GET - http://capitol.hawaii.gov/Archives/measure_indiv_Archives.aspx?billtype=SB&billnumber=1167&year=2017
09:29:39 INFO billy: Save bill upper 2017 Regular Session: SB1167
09:29:39 INFO scrapelib: GET - http://capitol.hawaii.gov/Archives/measure_indiv_Archives.aspx?billtype=SB&billnumber=1168&year=2017
09:29:42 INFO billy: Save bill upper 2017 Regular Session: SB1168
09:29:42 INFO scrapelib: GET - http://capitol.hawaii.gov/Archives/measure_indiv_Archives.aspx?billtype=SB&billnumber=1169&year=2017
09:29:44 INFO billy: Save bill upper 2017 Regular Session: SB1169
09:29:44 INFO scrapelib: GET - http://capitol.hawaii.gov/Archives/measure_indiv_Archives.aspx?billtype=SB&billnumber=1170&year=2017
09:29:48 INFO billy: Save bill upper 2017 Regular Session: SB1170
09:29:48 INFO scrapelib: GET - http://capitol.hawaii.gov/Archives/measure_indiv_Archives.aspx?billtype=SB&billnumber=1171&year=2017
09:29:51 INFO billy: Save bill upper 2017 Regular Session: SB1171
09:29:51 INFO scrapelib: GET - http://capitol.hawaii.gov/Archives/measure_indiv_Archives.aspx?billtype=SB&billnumber=1172&year=2017
09:29:55 INFO billy: Save bill upper 2017 Regular Session: SB1172
09:29:55 INFO scrapelib: GET - http://capitol.hawaii.gov/Archives/measure_indiv_Archives.aspx?billtype=SB&billnumber=1173&year=2017
09:29:57 INFO billy: Save bill upper 2017 Regular Session: SB1173
09:29:57 INFO scrapelib: GET - http://capitol.hawaii.gov/Archives/measure_indiv_Archives.aspx?billtype=SB&billnumber=1174&year=2017
09:30:00 INFO billy: Save bill upper 2017 Regular Session: SB1174
09:30:00 INFO scrapelib: GET - http://capitol.hawaii.gov/Archives/measure_indiv_Archives.aspx?billtype=SB&billnumber=1175&year=2017
09:30:03 INFO billy: Save bill upper 2017 Regular Session: SB1175
09:30:03 INFO scrapelib: GET - http://capitol.hawaii.gov/Archives/measure_indiv_Archives.aspx?billtype=SB&billnumber=1176&year=2017
09:30:07 INFO billy: Save bill upper 2017 Regular Session: SB1176
09:30:07 INFO scrapelib: GET - http://capitol.hawaii.gov/Archives/measure_indiv_Archives.aspx?billtype=SB&billnumber=1177&year=2017
09:30:10 INFO billy: Save bill upper 2017 Regular Session: SB1177
09:30:10 INFO scrapelib: GET - http://capitol.hawaii.gov/Archives/measure_indiv_Archives.aspx?billtype=SB&billnumber=1178&year=2017
09:30:12 INFO billy: Save bill upper 2017 Regular Session: SB1178
09:30:12 INFO scrapelib: GET - http://capitol.hawaii.gov/Archives/measure_indiv_Archives.aspx?billtype=SB&billnumber=1179&year=2017
09:30:13 INFO billy: Save bill upper 2017 Regular Session: SB1179
09:30:13 INFO scrapelib: GET - http://capitol.hawaii.gov/Archives/measure_indiv_Archives.aspx?billtype=SB&billnumber=1180&year=2017
09:30:15 INFO billy: Save bill upper 2017 Regular Session: SB1180
09:30:15 INFO scrapelib: GET - http://capitol.hawaii.gov/Archives/measure_indiv_Archives.aspx?billtype=SB&billnumber=1181&year=2017
09:30:18 INFO billy: Save bill upper 2017 Regular Session: SB1181
09:30:18 INFO scrapelib: GET - http://capitol.hawaii.gov/Archives/measure_indiv_Archives.aspx?billtype=SB&billnumber=1182&year=2017
09:30:20 INFO billy: Save bill upper 2017 Regular Session: SB1182
09:30:20 INFO scrapelib: GET - http://capitol.hawaii.gov/Archives/measure_indiv_Archives.aspx?billtype=SB&billnumber=1183&year=2017
Traceback (most recent call last):
  File "/usr/local/bin/billy-update", line 9, in <module>
    load_entry_point('billy==1.9.0', 'console_scripts', 'billy-update')()
  File "/opt/openstates/billy/billy/bin/update.py", line 368, in main
    run_record += _run_scraper(stype, args, metadata)
  File "/opt/openstates/billy/billy/bin/update.py", line 102, in _run_scraper
    scraper.scrape(chamber, time)
  File "/srv/openstates-web/openstates/hi/bills.py", line 247, in scrape
    self.scrape_type(session, chamber, typ)
  File "/srv/openstates-web/openstates/hi/bills.py", line 241, in scrape_type
    self.scrape_bill(session, chamber, billy_billtype, bill_url)
  File "/srv/openstates-web/openstates/hi/bills.py", line 171, in scrape_bill
    "//a[contains(@id, 'LinkButtonMeasure')]")[0].text_content()
IndexError: list index out of range
```

Visit http://bobsled.openstates.org/ for more info.



Discussion:

Resolving patch:
diff --git a/openstates/hi/bills.py b/openstates/hi/bills.py
--- a/openstates/hi/bills.py
+++ b/openstates/hi/bills.py
@@ -3,7 +3,7 @@
 import re
 
 from .utils import get_short_codes
-from urlparse import urlparse
+import urlparse
 
 from billy.scrape.bills import BillScraper, Bill
 from billy.scrape.votes import Vote
@@ -167,9 +167,8 @@ def parse_bill_versions_table(self, bill, versions):
     def scrape_bill(self, session, chamber, bill_type, url):
         bill_html = self.get(url).text
         bill_page = lxml.html.fromstring(bill_html)
-        scraped_bill_id = bill_page.xpath(
-            "//a[contains(@id, 'LinkButtonMeasure')]")[0].text_content()
-        bill_id = scraped_bill_id.split(' ')[0]
+        qs = dict(urlparse.parse_qsl(urlparse.urlparse(url).query))
+        bill_id = '{}{}'.format(qs['billtype'], qs['billnumber'])
         versions = bill_page.xpath( "//table[contains(@id, 'GridViewVersions')]" )[0]
 
         tables = bill_page.xpath("//table")


Test patch: