Repository: openstates/openstates-scrapers

Issue:
SD failing since at least 2018-07-25
SD has been failing since 2018-07-25

Based on automated runs it appears that SD has not run successfully in 3 days (2018-07-25).


```
  02:04:09 INFO pupa: save organization Military and Veterans Affairs as organization_f5ca291c-916a-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization State-Tribal Relations as organization_f6a0ecc2-916a-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization Commerce and Energy as organization_f776a7a4-916a-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization Rules Review as organization_f84cd64e-916a-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization Code Commission as organization_03f97344-916b-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization Committee on Appropriations as organization_0dedfe06-916b-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization Retirement Laws as organization_0dee052c-916b-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization Appropriations as organization_0dee0b3a-916b-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization Extraordinary Cost Fund for Special Education Study as organization_0dee1152-916b-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization Government Operations and Audit as organization_0eb4178a-916b-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization Government Operations & Audit as organization_0eb42428-916b-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization Executive Board as organization_0f6b257e-916b-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization Education as organization_10225e6a-916b-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization Judiciary as organization_102267c0-916b-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization Agriculture and Natural Resources as organization_10f89ab6-916b-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization Select Committee on Discipline and Expulsion as organization_10f8a2b8-916b-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization State Affairs as organization_10f8a8e4-916b-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization Rules Review as organization_10f8aeb6-916b-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization Commerce and Energy as organization_11dca134-916b-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization Transportation as organization_12862d4e-916b-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization State-Tribal Relations as organization_12863398-916b-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization Health and Human Services as organization_144de9e6-916b-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization Local Government as organization_16fcc5e0-916b-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization Taxation as organization_17e1e3f0-916b-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization Access to Mental Health Services Study as organization_188a892e-916b-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization Bonding as organization_1d142c84-916b-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization Legislative Procedure as organization_1e8e9fb8-916b-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization Ag. Land Assessment Task Force as organization_2e2df446-916b-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: save organization Code Commission as organization_3d41d196-916b-11e8-a8bf-02f93ae6a328.json
02:04:09 INFO pupa: no session specified, using 2018
02:04:09 INFO scrapelib: GET - http://www.sdlegislature.gov/Legislative_Session/Bills/default.aspx?Session=2018
02:04:10 INFO scrapelib: GET - http://www.sdlegislature.gov/Legislative_Session/Bills/default.aspx?Session=2018
loaded Open States pupa settings...
sd (scrape, import)
  people: {}
  bills: {}
Traceback (most recent call last):
  File "/opt/openstates/venv-pupa//bin/pupa", line 11, in <module>
    load_entry_point('pupa', 'console_scripts', 'pupa')()
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/__main__.py", line 68, in main
    subcommands[args.subcommand].handle(args, other)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 260, in handle
    return self.do_handle(args, other, juris)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 305, in do_handle
    report['scrape'] = self.do_scrape(juris, args, scrapers)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 173, in do_scrape
    report[scraper_name] = scraper.do_scrape(**scrape_args)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/scrape/base.py", line 120, in do_scrape
    raise ScrapeError('no objects returned from {} scrape'.format(self.__class__.__name__))
pupa.exceptions.ScrapeError: no objects returned from SDBillScraper scrape
```

Visit http://bobsled.openstates.org for more info.



Discussion:
Working on this, looks like we just need some URL updates.

Resolving patch:
diff --git a/openstates/sd/bills.py b/openstates/sd/bills.py
--- a/openstates/sd/bills.py
+++ b/openstates/sd/bills.py
@@ -14,9 +14,8 @@ def scrape(self, chambers=None, session=None):
             session = self.latest_session()
             self.info('no session specified, using %s', session)
 
-        url = 'http://www.sdlegislature.gov/Legislative_Session' \
-              '/Bills/default.aspx?Session={}'.format(session)
-
+        url = 'https://sdlegislature.gov/Legislative_Session' \
+              '/Bills/Default.aspx?Session={}'.format(session)
         chambers = [chambers] if chambers else ['upper', 'lower']
 
         for chamber in chambers:
@@ -76,10 +75,10 @@ def scrape_bill(self, chamber, session, bill_id, title, url):
             '/following-sibling::div[1]/p/a'
         )
         for link in sponsor_links:
-            if link.attrib['href'].startswith('http://www.sdlegislature.gov/Legislators/'):
+            if link.attrib['href'].startswith('https://sdlegislature.gov/Legislators/'):
                 sponsor_type = 'person'
             elif link.attrib['href'].startswith(
-                'http://www.sdlegislature.gov/Legislative_Session/Committees'
+                'https://sdlegislature.gov/Legislative_Session/Committees'
             ):
                 sponsor_type = 'organization'
             else:


Test patch: