Repository: openstates/openstates-scrapers

Issue:
MA failing since at least 2018-07-08
MA has been failing since 2018-07-08

Based on automated runs it appears that MA has not run successfully in 6 days (2018-07-08).


```
  03:59:51 INFO pupa: save bill H3814 in 190th as bill_1a10ec3e-867b-11e8-8ad9-022484cef43c.json
03:59:54 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H3865
03:59:54 INFO pupa: save bill H3865 in 190th as bill_1bddb164-867b-11e8-8ad9-022484cef43c.json
03:59:57 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H3799
03:59:57 INFO pupa: save bill H3799 in 190th as bill_1da94ca6-867b-11e8-8ad9-022484cef43c.json
04:00:03 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H3831
04:00:03 INFO pupa: save bill H3831 in 190th as bill_213aa964-867b-11e8-8ad9-022484cef43c.json
04:00:06 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H3836
04:00:07 INFO pupa: save bill H3836 in 190th as bill_237eda56-867b-11e8-8ad9-022484cef43c.json
04:00:10 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H3887
04:00:10 INFO pupa: save bill H3887 in 190th as bill_2557437c-867b-11e8-8ad9-022484cef43c.json
04:00:13 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H3863
04:00:14 INFO pupa: save bill H3863 in 190th as bill_27867c94-867b-11e8-8ad9-022484cef43c.json
04:00:16 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H3875
04:00:16 INFO pupa: save bill H3875 in 190th as bill_2922a91a-867b-11e8-8ad9-022484cef43c.json
04:00:19 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H3800
04:00:19 INFO pupa: No vote found in supplement for vote #73
04:00:19 INFO pupa: No vote found in supplement for vote #74
04:00:19 INFO scrapelib: GET - http://malegislature.gov/RollCall/190/SenateRollCall57.pdf
04:00:19 INFO scrapelib: GET - http://malegislature.gov/RollCall/190/SenateRollCall58.pdf
04:00:21 INFO pupa: No vote found in supplement for vote #82
loaded Open States pupa settings...
ma (scrape, import)
  people: {}
  committees: {}
  bills: {}
Traceback (most recent call last):
  File "/opt/openstates/venv-pupa//bin/pupa", line 11, in <module>
    load_entry_point('pupa', 'console_scripts', 'pupa')()
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/__main__.py", line 68, in main
    subcommands[args.subcommand].handle(args, other)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 260, in handle
    return self.do_handle(args, other, juris)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 305, in do_handle
    report['scrape'] = self.do_scrape(juris, args, scrapers)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 173, in do_scrape
    report[scraper_name] = scraper.do_scrape(**scrape_args)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/scrape/base.py", line 111, in do_scrape
    for obj in self.scrape(**kwargs) or []:
  File "/opt/openstates/openstates/openstates/ma/bills.py", line 58, in scrape
    yield from self.scrape_chamber('lower', session)
  File "/opt/openstates/openstates/openstates/ma/bills.py", line 77, in scrape_chamber
    yield from self.scrape_bill(session, bill, chamber)
  File "/opt/openstates/openstates/openstates/ma/bills.py", line 179, in scrape_bill
    self.scrape_actions(bill, bill_url, session)
  File "/opt/openstates/openstates/openstates/ma/bills.py", line 217, in scrape_actions
    self.scrape_action_page(bill, page)
  File "/opt/openstates/openstates/openstates/ma/bills.py", line 243, in scrape_action_page
    y = int(action_name_content[1].split('YEAS')[0])
ValueError: invalid literal for int() with base 10: '2000 passed over vet'
```

Visit http://bobsled.openstates.org for more info.



Discussion:

Resolving patch:
diff --git a/openstates/ma/bills.py b/openstates/ma/bills.py
--- a/openstates/ma/bills.py
+++ b/openstates/ma/bills.py
@@ -240,7 +240,14 @@ def scrape_action_page(self, bill, page):
                 actor = "lower"
                 action_name_content = re.split(r'[A-z0-9]\s?-\s*', action_name.strip())
                 vote_action = action_name_content[0]
-                y = int(action_name_content[1].split('YEAS')[0])
+
+                breakdown_str = action_name_content[1]
+                # Certain bills contain "Item <int>-<int>" that break the regex
+                # ex https://malegislature.gov/Bills/190/H3800
+                if action_name.strip().startswith('Item'):
+                    breakdown_str = action_name_content[2]
+
+                y = int(breakdown_str.split('YEAS')[0])
                 n = int(action_name.strip().split('YEAS to')[1].split('NAYS')[0])
 
                 # get supplement number


Test patch: