internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: openstates/openstates-scrapers

Issue:
MT: Vote names saved with a trailing "(<name>)"
The [MT bill scraper](http://github.com/openstates/openstates/blob/63ede9983ff6b8b26aeb2c29ad715f017c940746/openstates/mt/bills.py#L443) is saving vote names with a trailing "(name)".

Some examples:
"Moore, Frederick (Eric)", "Tempel, Russel (Russ)", "Kary, Douglas (Doug)"

This is causing the vote to come back with a null `leg_id`, since the name cannot be matched back to a known legislator.




Discussion:

Resolving patch:
diff --git a/openstates/mt/bills.py b/openstates/mt/bills.py
--- a/openstates/mt/bills.py
+++ b/openstates/mt/bills.py
@@ -441,14 +441,23 @@ def _parse_votes(self, url, vote, bill):
             if not text.strip(u'\xa0'):
                 continue
             v, name = filter(None, text.split(u'\xa0'))
+            # Considering Name is brackets as short name
+            regex = re.compile(".*?\((.*?)\)")
+            short_name = re.findall(regex, name)
+            if len(short_name) > 0:
+                note = 'Short Name: ' + short_name[0]
+            else:
+                note = ''
+            # Name without brackets like 'Kary, Douglas'
+            name = re.sub("[\(\[].*?[\)\]]", "", name)
             if v == 'Y':
-                vote.yes(name)
+                vote.yes(name, note=note)
             elif v == 'N':
-                vote.no(name)
+                vote.no(name, note=note)
             elif v == 'E':
-                vote.vote('excused', name)
+                vote.vote('excused', name, note=note)
             elif v == 'A':
-                vote.vote('absent', name)
+                vote.vote('absent', name, note=note)
 
         # code to deterimine value of `passed`
         passed = None


Test patch:
Repository: openstates/openstates-scrapers

Issue:
MI scraper failing since at least 2017-04-01
MI has been failing since 2017-04-01

Based on automated runs it appears that MI has not run successfully in 5 days (2017-04-01).


```
  06:00:31 INFO billy: billy-update abbr=mi
    actions=scrape,import,report
    types=bills,legislators,votes,committees,alldata,events
    sessions=2017-2018
    terms=2017-2018
06:00:31 INFO scrapelib: GET - http://www.senate.michigan.gov/senatorinfo.html
  File "/usr/local/bin/billy-update", line 9, in <module>
    load_entry_point('billy==1.9.0', 'console_scripts', 'billy-update')()
  File "/opt/openstates/billy/billy/bin/update.py", line 368, in main
Traceback (most recent call last):
    run_record += _run_scraper(stype, args, metadata)
  File "/opt/openstates/billy/billy/bin/update.py", line 102, in _run_scraper
    response = self.get(url)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 501, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/scrapelib/__init__.py", line 272, in request
    raise HTTPError(resp)
scrapelib.HTTPError: 500 while retrieving http://www.senate.michigan.gov/senatorinfo.html
  File "/srv/openstates-web/openstates/mi/legislators.py", line 77, in scrape_upper
    scraper.scrape(chamber, time)
  File "/srv/openstates-web/openstates/mi/legislators.py", line 16, in scrape
    return self.scrape_upper(chamber, term)
    doc = self.lxmlize(url)
  File "/srv/openstates-web/openstates/utils/lxmlize.py", line 19, in lxmlize
```

Visit http://bobsled.openstates.org/ for more info.



Discussion:

Resolving patch:
diff --git a/openstates/mi/legislators.py b/openstates/mi/legislators.py
--- a/openstates/mi/legislators.py
+++ b/openstates/mi/legislators.py
@@ -73,7 +73,7 @@ def scrape_lower(self, chamber, term):
             self.save_legislator(leg)
 
     def scrape_upper(self, chamber, term):
-        url = 'http://www.senate.michigan.gov/senatorinfo.html'
+        url = 'http://www.senate.michigan.gov/senatorinfo_list.html'
         doc = self.lxmlize(url)
         for row in doc.xpath('//table[not(@class="calendar")]//tr')[3:]:
             if len(row) != 7:


Test patch:
Repository: openstates/openstates-scrapers

Issue:
IL: Capture "Subject Matter" for `event`s
IL events have a Subject Matter field on the legislature's website, which we should capture, probably as the `description` value for that `event` object.

For example, look at one of the hearings linked from [here](http://www.ilga.gov/senate/committees/hearing.asp?CommitteeID=1927), like this:

<img width="864" alt="screen shot 2017-04-23 at 00 19 37" src="https://cloud.githubusercontent.com/assets/4959135/25310740/bb6f24b8-27ba-11e7-96e3-9b102819581c.png">

cc @jonrogoff, @justgosh


Discussion:
@mileswwatkins currently the `description` value for that `event` object is occupied. Should I move it to `**kwargs` and update the `description` value as `Subject Matter field`?
It may be worth noting that hearings have either bills or `Subject Matter`.  It appears to be either one or the other.
@hiteshgarg14, it looks like right now the description and participant name are the same thing on all IL `event` items. For `description` you could append the two if you'd like, formatted something like `"{}, {}".format(meeting_name, subject_matter)`.

Also, it looks like the `participant` field sometimes has extraneous text, like `Hearing Notice For Appropriations I` instead of `Appropriations I`, which could be fixed while you're in there.

Resolving patch:
diff --git a/openstates/il/events.py b/openstates/il/events.py
--- a/openstates/il/events.py
+++ b/openstates/il/events.py
@@ -4,7 +4,6 @@
 from openstates.utils import LXMLMixin
 from billy.scrape.events import Event, EventScraper
 
-import lxml.html
 import pytz
 
 urls = {
@@ -20,8 +19,8 @@ class ILEventScraper(EventScraper, LXMLMixin):
     def scrape_page(self, url, session, chamber):
         page = self.lxmlize(url)
 
-        ctty_name = page.xpath("//span[@class='heading']")[0].text_content()
-
+        ctty_name = page.xpath("//span[@class='heading']")[0].text_content().replace(
+            "Hearing Notice For ", "")
         tables = page.xpath("//table[@cellpadding='3']")
         info = tables[0]
         rows = info.xpath(".//tr")
@@ -33,7 +32,8 @@ def scrape_page(self, url, session, chamber):
             metainf[key] = value
 
         where = metainf['Location:']
-        description = ctty_name
+        subject_matter = metainf['Subject Matter:']
+        description = "{}, {}".format(ctty_name, subject_matter)
 
         datetime = metainf['Scheduled Date:']
         datetime = re.sub("\s+", " ", datetime)


Test patch:
Repository: openstates/openstates-scrapers

Issue:
CA: email issue
State: CA

(via contact)
I’d like to call your attention to an error on your email address listings for State Senators in California.  Your database lists an address in the form of senator.lastname@sen.ca.gov, whereas the correct email is senator.lastname@senator.ca.gov.
 
It does appear that Senate staffers are firstname.lastname@sen.ca.gov.
 
We’ve been using your service, and our advocates’ emails are bouncing back.




Discussion:

Resolving patch:
diff --git a/openstates/ca/legislators.py b/openstates/ca/legislators.py
--- a/openstates/ca/legislators.py
+++ b/openstates/ca/legislators.py
@@ -244,5 +244,4 @@ def _construct_email(self, chamber, full_name):
         if chamber == 'lower':
             return 'assemblymember.' +  last_name + '@assembly.ca.gov'
         else:
-            return 'senator.' +  last_name + '@sen.ca.gov'
-        
+            return 'senator.' +  last_name + '@senator.ca.gov'


Test patch:
Repository: openstates/openstates-scrapers

Issue:
FL: No bills with signed dates for 2017 session.
State: FL

The bill data for FL 2017 session shows none with signed dates, parser doesn't appear to track Approved by Governor or other executive actions.



Discussion:

Resolving patch:
diff --git a/openstates/fl/bills.py b/openstates/fl/bills.py
--- a/openstates/fl/bills.py
+++ b/openstates/fl/bills.py
@@ -155,6 +155,10 @@ def process_history(self):
                     atype.append('passage')
                 elif action.startswith('CS passed'):
                     atype.append('passage')
+                elif action == 'Approved by Governor':
+                    atype.append('executive-signature')
+                elif action == 'Vetoed by Governor':
+                    atype.append('executive-veto')
 
                 self.obj.add_action(action, date, organization=actor, chamber=chamber,
                                     classification=atype)


Test patch:
Repository: openstates/openstates-scrapers

Issue:
WV failing since at least 2017-12-07
WV has been failing since 2017-12-07

Based on automated runs it appears that WV has not run successfully in 2 days (2017-12-07).


```
  23:01:25 CRITICAL pupa: Session(s) 2018 were reported by WestVirginia.get_session_list() but were not found in WestVirginia.legislative_sessions or WestVirginia.ignored_scraped_sessions.
no pupa_settings on path, using defaults
wv (scrape, import)
  people: {}
  bills: {}
```

Visit http://bobsled.openstates.org for more info.



Discussion:

Resolving patch:
diff --git a/billy_metadata/wv.py b/billy_metadata/wv.py
--- a/billy_metadata/wv.py
+++ b/billy_metadata/wv.py
@@ -24,7 +24,7 @@
          },
         {'name': '2017-2018',
          'start_year': 2017, 'end_year': 2018,
-         'sessions': ['2017', '20171S', '20172S'],
+         'sessions': ['2017', '20171S', '20172S', '2018'],
          },
         ],
     'session_details': {
@@ -67,6 +67,10 @@
         '20172S': {'display_name': '2017 Second Special Session',
                    'type': 'special',
                    },
+        '2018': {'display_name': '2018 Regular Session',
+                 'type': 'primary',
+                 '_scraped_name': '2018'
+                 },
     },
     'feature_flags': ['subjects', 'influenceexplorer'],
     '_ignored_scraped_sessions': [
diff --git a/openstates/wv/__init__.py b/openstates/wv/__init__.py
--- a/openstates/wv/__init__.py
+++ b/openstates/wv/__init__.py
@@ -81,6 +81,14 @@ class WestVirginia(Jurisdiction):
             "classification": "special",
             "identifier": "20172S",
             "name": "2017 Second Special Session",
+        },
+        {
+            "_scraped_name": "2018",
+            "classification": "primary",
+            "identifier": "2018",
+            "name": "2018 Regular Session",
+            "start_date": "2018-01-10",
+            "end_date": "2018-03-10",
         }
     ]
     ignored_scraped_sessions = [


Test patch:
Repository: openstates/openstates-scrapers

Issue:
MD failing since at least 2017-12-08
MD has been failing since 2017-12-08

Based on automated runs it appears that MD has not run successfully in 2 days (2017-12-08).


```
  22:01:38 CRITICAL pupa: Session(s) 2018 Regular Session were reported by Maryland.get_session_list() but were not found in Maryland.legislative_sessions or Maryland.ignored_scraped_sessions.
no pupa_settings on path, using defaults
md (scrape, import)
  people: {}
  bills: {}
```

Visit http://bobsled.openstates.org for more info.



Discussion:

Resolving patch:
diff --git a/billy_metadata/md.py b/billy_metadata/md.py
--- a/billy_metadata/md.py
+++ b/billy_metadata/md.py
@@ -106,6 +106,12 @@
                     'start_date': datetime.date(2017, 1, 11),
                     'end_date': datetime.date(2017, 4, 10),
                 },
+        '2018': {'display_name': '2018 Regular Session',
+                 'type': 'primary',
+                 '_scraped_name': '2018 Regular Session',
+                 'start_date': datetime.date(2018, 1, 10),
+                 'end_date': datetime.date(2018, 4, 9),
+                 },
     },
     feature_flags=['subjects', 'events', 'influenceexplorer'],
     _ignored_scraped_sessions=['1996 Regular Session',
diff --git a/openstates/md/__init__.py b/openstates/md/__init__.py
--- a/openstates/md/__init__.py
+++ b/openstates/md/__init__.py
@@ -127,6 +127,14 @@ class Maryland(Jurisdiction):
             "identifier": "2017",
             "name": "2017 Regular Session",
             "start_date": "2017-01-11"
+        },
+        {
+            "_scraped_name": "2018 Regular Session",
+            "classification": "primary",
+            "end_date": "2018-04-09",
+            "identifier": "2018",
+            "name": "2018 Regular Session",
+            "start_date": "2018-01-10"
         }
     ]
     ignored_scraped_sessions = [


Test patch:
Repository: openstates/openstates-scrapers

Issue:
LA failing since at least 2018-01-05
LA has been failing since 2018-01-05

Based on automated runs it appears that LA has not run successfully in 2 days (2018-01-05).


```
  no pupa_settings on path, using defaults
la (scrape, import)
  bills: {}
  people: {}
00:00:35 CRITICAL pupa: Session(s) 2018 Regular Session were reported by Louisiana.get_session_list() but were not found in Louisiana.legislative_sessions or Louisiana.ignored_scraped_sessions.
```

Visit http://bobsled.openstates.org for more info.



Discussion:

Resolving patch:
diff --git a/billy_metadata/la.py b/billy_metadata/la.py
--- a/billy_metadata/la.py
+++ b/billy_metadata/la.py
@@ -45,6 +45,7 @@
                 '2017',
                 '2017 1st Extraordinary Session',
                 '2017 2nd Extraordinary Session',
+                '2018',
             ],
         },
     ],
@@ -153,6 +154,14 @@
             'type': 'special',
             'display_name': '2017, 2nd Extraordinary Session',
         },
+        '2018': {
+            'type': 'primary',
+            'start_date': datetime.date(2018, 3, 12),
+            'end_date': datetime.date(2018, 6, 4),
+            'display_name': '2018 Regular Session',
+            '_id': '18RS',
+            '_scraped_name': '2018 Regular Session',
+        },
     },
     'feature_flags': ['subjects', 'influenceexplorer', 'events'],
     '_ignored_scraped_sessions': [
diff --git a/openstates/la/__init__.py b/openstates/la/__init__.py
--- a/openstates/la/__init__.py
+++ b/openstates/la/__init__.py
@@ -135,6 +135,14 @@ class Louisiana(Jurisdiction):
             "start_date": "2017-06-17",
             "end_date": "2017-06-29",
         },
+        {
+            "_scraped_name": "2018 Regular Session",
+            "classification": "primary",
+            "identifier": "2018",
+            "name": "2018 Regular Session",
+            "start_date": "2018-03-12",
+            "end_date": "2018-06-04",
+        },
     ]
     ignored_scraped_sessions = [
         "2016 Organizational Session",
diff --git a/openstates/la/bills.py b/openstates/la/bills.py
--- a/openstates/la/bills.py
+++ b/openstates/la/bills.py
@@ -28,6 +28,7 @@ class LABillScraper(Scraper, LXMLMixin):
         '2017 1st Extraordinary Session': '171ES',
         '2017 2nd Extraordinary Session': '172ES',
         '2017': '17RS',
+        '2018': '18RS',
     }
 
     def pdf_to_lxml(self, filename, type='html'):


Test patch:
Repository: openstates/openstates-scrapers

Issue:
MS failing since at least 2017-12-31
MS has been failing since 2017-12-31

Based on automated runs it appears that MS has not run successfully in 7 days (2017-12-31).


```
  00:00:18 CRITICAL pupa: Session(s) 2018 Regular Session were reported by Mississippi.get_session_list() but were not found in Mississippi.legislative_sessions or Mississippi.ignored_scraped_sessions.
no pupa_settings on path, using defaults
ms (scrape, import)
  bills: {}
  committee: {}
  people: {}
```

Visit http://bobsled.openstates.org for more info.



Discussion:

Resolving patch:
diff --git a/billy_metadata/ms.py b/billy_metadata/ms.py
--- a/billy_metadata/ms.py
+++ b/billy_metadata/ms.py
@@ -37,7 +37,7 @@
             'name': '2016-2019',
             'start_year': 2016, 'end_year': 2019,
             'sessions': [
-                '2016', '20161E', '20162E', '2017', '20171E',
+                '2016', '20161E', '20162E', '2017', '20171E', '2018',
             ],
         },
     ],
@@ -138,7 +138,14 @@
         },
         '20171E': {
             'display_name': '2017 First Extraordinary Session',
-        },        
+        },
+        '2018': {
+            'type': 'primary',
+            'start_date': datetime.date(2018, 1, 2),
+            'end_date': datetime.date(2016, 4, 1),
+            'display_name': '2018 Regular Session',
+            '_scraped_name': '2018 Regular Session',
+        },
     },
     'feature_flags': ['subjects', 'influenceexplorer'],
     '_ignored_scraped_sessions': [
diff --git a/openstates/ms/__init__.py b/openstates/ms/__init__.py
--- a/openstates/ms/__init__.py
+++ b/openstates/ms/__init__.py
@@ -142,6 +142,14 @@ class Mississippi(Jurisdiction):
             "identifier": "20171E",
             "name": "2017 First Extraordinary Session",
         },
+        {
+            "_scraped_name": "2018 Regular Session",
+            "classification": "primary",
+            "end_date": "2018-04-01",
+            "identifier": "2018",
+            "name": "2018 Regular Session",
+            "start_date": "2018-01-02"
+        },
     ]
     ignored_scraped_sessions = [
         "2008 First Extraordinary Session",


Test patch:
Repository: openstates/openstates-scrapers

Issue:
KY failing since at least 2018-01-04
KY has been failing since 2018-01-04

Based on automated runs it appears that KY has not run successfully in 2 days (2018-01-04).


```
  00:00:39 CRITICAL pupa: Session(s) 2018 Regular Session were reported by Kentucky.get_session_list() but were not found in Kentucky.legislative_sessions or Kentucky.ignored_scraped_sessions.
no pupa_settings on path, using defaults
ky (scrape, import)
  people: {}
  bills: {}
```

Visit http://bobsled.openstates.org for more info.



Discussion:

Resolving patch:
diff --git a/billy_metadata/ky.py b/billy_metadata/ky.py
--- a/billy_metadata/ky.py
+++ b/billy_metadata/ky.py
@@ -41,6 +41,7 @@
             'end_year': 2018,
             'sessions': [
                 '2017RS',
+                '2018RS',
             ]
         },
     ],
@@ -114,6 +115,12 @@
             'display_name': '2017 Regular Session',
             '_scraped_name': '2017 Regular Session',
         },
+        '2018RS': {
+            'type': 'primary',
+            'start_date': datetime.date(2018, 1, 2),
+            'display_name': '2018 Regular Session',
+            '_scraped_name': '2018 Regular Session',
+        },
     },
     'feature_flags': ['subjects', 'events', 'influenceexplorer'],
     '_ignored_scraped_sessions': [],
diff --git a/openstates/ky/__init__.py b/openstates/ky/__init__.py
--- a/openstates/ky/__init__.py
+++ b/openstates/ky/__init__.py
@@ -103,7 +103,15 @@ class Kentucky(Jurisdiction):
             "identifier": "2017RS",
             "name": "2017 Regular Session",
             "start_date": "2017-01-03"
-        }
+        },
+        {
+            "_scraped_name": "2018 Regular Session",
+            "classification": "primary",
+            "end_date": "2018-04-13",
+            "identifier": "2018RS",
+            "name": "2018 Regular Session",
+            "start_date": "2018-01-02"
+        },
     ]
     ignored_scraped_sessions = []
 


Test patch:
Repository: openstates/openstates-scrapers

Issue:
RI failing since at least 2018-01-02
RI has been failing since 2018-01-02

Based on automated runs it appears that RI has not run successfully in 2 days (2018-01-02).


```
  no pupa_settings on path, using defaults
ri (scrape, import)
  bills: {}
  people: {}
  committees: {}
23:00:36 CRITICAL pupa: Session(s) 2018 were reported by RhodeIsland.get_session_list() but were not found in RhodeIsland.legislative_sessions or RhodeIsland.ignored_scraped_sessions.
```

Visit http://bobsled.openstates.org for more info.



Discussion:

Resolving patch:
diff --git a/billy_metadata/ri.py b/billy_metadata/ri.py
--- a/billy_metadata/ri.py
+++ b/billy_metadata/ri.py
@@ -46,7 +46,7 @@
             'name': '2017',
             'start_year': 2017,
             'end_year': 2018,
-            'sessions': ['2017'],
+            'sessions': ['2017', '2018'],
         },
     ],
     'session_details': {
@@ -89,6 +89,11 @@
             'display_name': '2017 Regular Session',
             '_scraped_name': '2017',
         },
+        '2018': {
+            'type': 'primary',
+            'display_name': '2018 Regular Session',
+            '_scraped_name': '2018',
+        },
     },
     'feature_flags': ['subjects', 'events', 'influenceexplorer'],
     '_ignored_scraped_sessions': [
diff --git a/openstates/ri/__init__.py b/openstates/ri/__init__.py
--- a/openstates/ri/__init__.py
+++ b/openstates/ri/__init__.py
@@ -70,7 +70,14 @@ class RhodeIsland(Jurisdiction):
             "name": "2017 Regular Session",
             "start_date": "2017-01-03",
             "end_date": "2017-06-30",
-        }
+        },
+        {
+            "_scraped_name": "2018",
+            "classification": "primary",
+            "identifier": "2018",
+            "name": "2018 Regular Session",
+            "start_date": "2018-01-02",
+        },
     ]
     ignored_scraped_sessions = [
         "2015",


Test patch:
Repository: openstates/openstates-scrapers

Issue:
CT failing since at least 2018-01-07
CT has been failing since 2018-01-07

Based on automated runs it appears that CT has not run successfully in 2 days (2018-01-07).


```
  23:00:35 INFO scrapelib: GET - ftp://ftp.cga.ct.gov
23:00:35 CRITICAL pupa: Session(s) 2018 were reported by Connecticut.get_session_list() but were not found in Connecticut.legislative_sessions or Connecticut.ignored_scraped_sessions.
no pupa_settings on path, using defaults
ct (scrape, import)
  events: {}
  bills: {}
  people: {}
```

Visit http://bobsled.openstates.org for more info.



Discussion:

Resolving patch:
diff --git a/billy_metadata/ct.py b/billy_metadata/ct.py
--- a/billy_metadata/ct.py
+++ b/billy_metadata/ct.py
@@ -34,7 +34,7 @@
             'name': '2017-2018',
             'start_year': 2017,
             'end_year': 2018,
-            'sessions': ['2017'],
+            'sessions': ['2017', '2018'],
         },
     ],
     'session_details': {
@@ -68,6 +68,12 @@
             'display_name': '2017 Regular Session',
             '_scraped_name': '2017',
         },
+        '2018': {
+            'display_name': '2018 Regular Session',
+            'start_date': datetime.date(2018, 1, 10),
+            'end_date': datetime.date(2018, 5, 9),
+            '_scraped_name': '2018',
+        },
     },
     'feature_flags': ['subjects', 'events', 'influenceexplorer'],
     '_ignored_scraped_sessions': [
diff --git a/openstates/ct/__init__.py b/openstates/ct/__init__.py
--- a/openstates/ct/__init__.py
+++ b/openstates/ct/__init__.py
@@ -64,6 +64,13 @@ class Connecticut(Jurisdiction):
             "start_date": "2017-01-04",
             "end_date": "2017-06-07",
         },
+        {
+            "_scraped_name": "2018",
+            "identifier": "2018",
+            "name": "2018 Regular Session",
+            "start_date": "2018-01-10",
+            "end_date": "2018-05-09",
+        },
     ]
     ignored_scraped_sessions = [
         "2010",


Test patch: