Repository: openstates/openstates-scrapers

Issue:
NE failing since at least 2018-06-28
NE has been failing since 2018-06-28

Based on automated runs it appears that NE has not run successfully in 2 days (2018-06-28).


```
  02:30:54 INFO pupa: save membership 5b4c2cb2-7b6e-11e8-9e19-02e29baaa692 membership in ~{"classification": "legislature"} as membership_5b4c30ea-7b6e-11e8-9e19-02e29baaa692.json
02:30:54 INFO pupa: save membership 5b4c2cb2-7b6e-11e8-9e19-02e29baaa692 membership in ~{"classification": "party", "name": "Nonpartisan"} as membership_5b4c3400-7b6e-11e8-9e19-02e29baaa692.json
02:30:54 INFO scrapelib: GET - http://news.legislature.ne.gov/dist44
02:30:55 INFO pupa: save person Dan Hughes as person_5bea39ca-7b6e-11e8-9e19-02e29baaa692.json
02:30:55 INFO pupa: save membership 5bea39ca-7b6e-11e8-9e19-02e29baaa692 membership in ~{"classification": "legislature"} as membership_5bea3dda-7b6e-11e8-9e19-02e29baaa692.json
02:30:55 INFO pupa: save membership 5bea39ca-7b6e-11e8-9e19-02e29baaa692 membership in ~{"classification": "party", "name": "Nonpartisan"} as membership_5bea4028-7b6e-11e8-9e19-02e29baaa692.json
02:30:55 INFO scrapelib: GET - http://news.legislature.ne.gov/dist45
02:30:56 INFO pupa: save person Sue Crawford as person_5c938c46-7b6e-11e8-9e19-02e29baaa692.json
02:30:56 INFO pupa: save membership 5c938c46-7b6e-11e8-9e19-02e29baaa692 membership in ~{"classification": "legislature"} as membership_5c93909c-7b6e-11e8-9e19-02e29baaa692.json
02:30:56 INFO pupa: save membership 5c938c46-7b6e-11e8-9e19-02e29baaa692 membership in ~{"classification": "party", "name": "Nonpartisan"} as membership_5c93939e-7b6e-11e8-9e19-02e29baaa692.json
02:30:56 INFO scrapelib: GET - http://news.legislature.ne.gov/dist46
02:30:57 INFO pupa: save person Adam Morfeld as person_5d0a16ea-7b6e-11e8-9e19-02e29baaa692.json
02:30:57 INFO pupa: save membership 5d0a16ea-7b6e-11e8-9e19-02e29baaa692 membership in ~{"classification": "legislature"} as membership_5d0a1ae6-7b6e-11e8-9e19-02e29baaa692.json
02:30:57 INFO pupa: save membership 5d0a16ea-7b6e-11e8-9e19-02e29baaa692 membership in ~{"classification": "party", "name": "Nonpartisan"} as membership_5d0a1d2a-7b6e-11e8-9e19-02e29baaa692.json
02:30:57 INFO scrapelib: GET - http://news.legislature.ne.gov/dist47
02:30:58 INFO pupa: save person Steve Erdman as person_5dc322ac-7b6e-11e8-9e19-02e29baaa692.json
02:30:58 INFO pupa: save membership 5dc322ac-7b6e-11e8-9e19-02e29baaa692 membership in ~{"classification": "legislature"} as membership_5dc32694-7b6e-11e8-9e19-02e29baaa692.json
02:30:58 INFO pupa: save membership 5dc322ac-7b6e-11e8-9e19-02e29baaa692 membership in ~{"classification": "party", "name": "Nonpartisan"} as membership_5dc32afe-7b6e-11e8-9e19-02e29baaa692.json
02:30:58 INFO scrapelib: GET - http://news.legislature.ne.gov/dist48
02:30:59 INFO pupa: save person John Stinner as person_5e89a800-7b6e-11e8-9e19-02e29baaa692.json
02:30:59 INFO pupa: save membership 5e89a800-7b6e-11e8-9e19-02e29baaa692 membership in ~{"classification": "legislature"} as membership_5e89ac4c-7b6e-11e8-9e19-02e29baaa692.json
02:30:59 INFO pupa: save membership 5e89a800-7b6e-11e8-9e19-02e29baaa692 membership in ~{"classification": "party", "name": "Nonpartisan"} as membership_5e89aed6-7b6e-11e8-9e19-02e29baaa692.json
02:30:59 INFO scrapelib: GET - http://news.legislature.ne.gov/dist49
02:31:00 INFO pupa: save person John Murante as person_5ed7988a-7b6e-11e8-9e19-02e29baaa692.json
02:31:00 INFO pupa: save membership 5ed7988a-7b6e-11e8-9e19-02e29baaa692 membership in ~{"classification": "legislature"} as membership_5ed79c90-7b6e-11e8-9e19-02e29baaa692.json
02:31:00 INFO pupa: save membership 5ed7988a-7b6e-11e8-9e19-02e29baaa692 membership in ~{"classification": "party", "name": "Nonpartisan"} as membership_5ed79ede-7b6e-11e8-9e19-02e29baaa692.json
02:31:00 INFO scrapelib: GET - http://www.nebraskalegislature.gov/committees/standing-committees.php
02:31:00 INFO scrapelib: GET - http://www.nebraskalegislature.gov/committees/select-committees.php
02:31:01 WARNING pupa: No members found in Select committees committee.
02:31:01 WARNING pupa: No members found in Special committees committee.
Traceback (most recent call last):
  File "/opt/openstates/venv-pupa//bin/pupa", line 11, in <module>
    load_entry_point('pupa', 'console_scripts', 'pupa')()
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/__main__.py", line 68, in main
    subcommands[args.subcommand].handle(args, other)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 260, in handle
    return self.do_handle(args, other, juris)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 305, in do_handle
    report['scrape'] = self.do_scrape(juris, args, scrapers)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 173, in do_scrape
    report[scraper_name] = scraper.do_scrape(**scrape_args)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/scrape/base.py", line 120, in do_scrape
    raise ScrapeError('no objects returned from {} scrape'.format(self.__class__.__name__))
pupa.exceptions.ScrapeError: no objects returned from NECommitteeScraper scrape
loaded Open States pupa settings...
ne (scrape, import)
  bills: {}
  votes: {}
  people: {}
  committees: {}
```

Visit http://bobsled.openstates.org for more info.



Discussion:

Resolving patch:
diff --git a/openstates/ne/committees.py b/openstates/ne/committees.py
--- a/openstates/ne/committees.py
+++ b/openstates/ne/committees.py
@@ -14,8 +14,9 @@ def _scrape_standing_committees(self):
 
         committee_nodes = self.get_nodes(
             page,
-            '//div[@class="main-content"]/div[@class="panel panel-leg"][1]/'
-            'div[@class="list-group"]/a[@class="list-group-item"]')
+            '//a[@class="accordion-switch"][contains(text(), "Standing Committees")]'
+            '/ancestor::div[@class="panel panel-leg"]//div[@class="list-group"]'
+            '/a[@class="list-group-item"]')
 
         for committee_node in committee_nodes:
             committee_page_url = committee_node.attrib['href']
@@ -63,7 +64,8 @@ def _scrape_select_special_committees(self):
 
         committee_nodes = self.get_nodes(
             page,
-            '//div[@class="main-content"]/div[@class="panel panel-leg"]')
+            '//a[contains(@class, "accordion-switch")]'
+            '/ancestor::div[@class="panel panel-leg"]')
 
         for committee_node in committee_nodes:
             committee_name = self.get_node(
@@ -81,8 +83,8 @@ def _scrape_select_special_committees(self):
 
             members = self.get_nodes(
                 committee_node,
-                './div[@class="list-group"]/a[@class="list-group-item"]/'
-                'text()[normalize-space()]')
+                './/a[@class="list-group-item"]'
+                '/text()[normalize-space()]')
 
             for member in members:
                 member_name = re.sub(r'\Sen\.\s+', '', member)


Test patch: