Repository: openstates/openstates-scrapers

Issue:
MA failing since at least 2018-06-30
MA has been failing since 2018-06-30

Based on automated runs it appears that MA has not run successfully in 2 days (2018-06-30).


```
  04:17:14 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H4037
04:17:15 INFO pupa: save bill H4037 in 190th as bill_8b38cc62-7d0f-11e8-83d1-02fc57c428b8.json
04:17:18 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H4076
04:17:19 INFO pupa: save bill H4076 in 190th as bill_8d7fa0c2-7d0f-11e8-83d1-02fc57c428b8.json
04:17:21 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H4267
04:17:21 INFO pupa: save bill H4267 in 190th as bill_8f31cc92-7d0f-11e8-83d1-02fc57c428b8.json
04:17:24 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H4266
04:17:25 INFO pupa: save bill H4266 in 190th as bill_90fb7762-7d0f-11e8-83d1-02fc57c428b8.json
04:17:27 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H4321
04:17:28 INFO pupa: save bill H4321 in 190th as bill_92c80aec-7d0f-11e8-83d1-02fc57c428b8.json
04:17:30 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H4338
04:17:31 INFO pupa: save bill H4338 in 190th as bill_949ec536-7d0f-11e8-83d1-02fc57c428b8.json
04:17:34 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H4265
04:17:34 INFO pupa: save bill H4265 in 190th as bill_96c19e2e-7d0f-11e8-83d1-02fc57c428b8.json
04:17:37 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H4364
04:17:38 INFO pupa: save bill H4364 in 190th as bill_98b5a2b6-7d0f-11e8-83d1-02fc57c428b8.json
04:17:40 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H4245
04:17:41 INFO pupa: save bill H4245 in 190th as bill_9ab469da-7d0f-11e8-83d1-02fc57c428b8.json
04:17:44 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H4229
04:17:44 INFO pupa: save bill H4229 in 190th as bill_9cbbba6c-7d0f-11e8-83d1-02fc57c428b8.json
04:17:47 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H4301
loaded Open States pupa settings...
ma (scrape, import)
  people: {}
  committees: {}
  bills: {}
Traceback (most recent call last):
  File "/opt/openstates/venv-pupa//bin/pupa", line 11, in <module>
    load_entry_point('pupa', 'console_scripts', 'pupa')()
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/__main__.py", line 68, in main
    subcommands[args.subcommand].handle(args, other)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 260, in handle
    return self.do_handle(args, other, juris)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 305, in do_handle
    report['scrape'] = self.do_scrape(juris, args, scrapers)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 173, in do_scrape
    report[scraper_name] = scraper.do_scrape(**scrape_args)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/scrape/base.py", line 111, in do_scrape
    for obj in self.scrape(**kwargs) or []:
  File "/opt/openstates/openstates/openstates/ma/bills.py", line 47, in scrape
    yield from self.scrape_chamber('lower', session)
  File "/opt/openstates/openstates/openstates/ma/bills.py", line 66, in scrape_chamber
    yield from self.scrape_bill(session, bill, chamber)
  File "/opt/openstates/openstates/openstates/ma/bills.py", line 168, in scrape_bill
    self.scrape_actions(bill, bill_url, session)
  File "/opt/openstates/openstates/openstates/ma/bills.py", line 198, in scrape_actions
    self.scrape_action_page(bill, page)
  File "/opt/openstates/openstates/openstates/ma/bills.py", line 232, in scrape_action_page
    y = int(action_name_content[1].split('YEAS')[0])
IndexError: list index out of range
```

Visit http://bobsled.openstates.org for more info.



Discussion:

Resolving patch:
diff --git a/openstates/ma/bills.py b/openstates/ma/bills.py
--- a/openstates/ma/bills.py
+++ b/openstates/ma/bills.py
@@ -40,9 +40,20 @@ def get_refiners(self, page, refinerName):
             refiner_list[label] = refiner
         return refiner_list
 
-    def scrape(self, chamber=None, session=None):
+    def scrape(self, chamber=None, session=None, bill_no=None):
         if not session:
             session = self.latest_session()
+
+        if bill_no:
+            single_bill_chamber = False
+            if 'H' in bill_no:
+                single_bill_chamber = 'lower'
+            else:
+                single_bill_chamber = 'upper'
+
+            yield from self.scrape_bill(session, bill_no, single_bill_chamber)
+            return
+
         if not chamber:
             yield from self.scrape_chamber('lower', session)
             yield from self.scrape_chamber('upper', session)
@@ -227,7 +238,7 @@ def scrape_action_page(self, bill, page):
             # House votes
             if "Supplement" in action_name:
                 actor = "lower"
-                action_name_content = re.split(r'[A-z0-9]\s?-\s+', action_name.strip())
+                action_name_content = re.split(r'[A-z0-9]\s?-\s*', action_name.strip())
                 vote_action = action_name_content[0]
                 y = int(action_name_content[1].split('YEAS')[0])
                 n = int(action_name.strip().split('YEAS to')[1].split('NAYS')[0])


Test patch: