Repository: openstates/openstates-scrapers

Issue:
DC failing since at least 2018-08-10
DC has been failing since 2018-08-10

Based on automated runs it appears that DC has not run successfully in 6 days (2018-08-10).


```
  00:02:07 INFO pupa: no session specified, using 22
00:02:08 INFO pupa: save vote_event 22 - 2018-04-10 - Final Reading CC as vote_event_5cb29e20-a048-11e8-96e1-0215f703737a.json
00:02:08 INFO pupa: save bill PR22-0745 in 22 as bill_5cb228be-a048-11e8-96e1-0215f703737a.json
00:02:09 INFO pupa: save vote_event 22 - 2018-02-06 - Final Reading CC as vote_event_5d087138-a048-11e8-96e1-0215f703737a.json
00:02:09 INFO pupa: save bill PR22-0722 in 22 as bill_5d08624c-a048-11e8-96e1-0215f703737a.json
00:02:09 INFO pupa: save vote_event 22 - 2018-02-06 - Final Reading CC as vote_event_5d4bbfec-a048-11e8-96e1-0215f703737a.json
00:02:09 INFO pupa: save bill PR22-0721 in 22 as bill_5d4bb29a-a048-11e8-96e1-0215f703737a.json
00:02:10 INFO pupa: save vote_event 22 - 2018-07-10 - Final Reading as vote_event_5d7f0b68-a048-11e8-96e1-0215f703737a.json
00:02:10 INFO pupa: save bill PR22-0960 in 22 as bill_5d7eff56-a048-11e8-96e1-0215f703737a.json
00:02:10 INFO pupa: save vote_event 22 - 2018-07-10 - Final Reading as vote_event_5db10f1e-a048-11e8-96e1-0215f703737a.json
00:02:10 INFO pupa: save bill PR22-0959 in 22 as bill_5db10208-a048-11e8-96e1-0215f703737a.json
00:02:10 INFO pupa: save vote_event 22 - 2018-07-10 - Final Reading as vote_event_5deaa21a-a048-11e8-96e1-0215f703737a.json
00:02:10 INFO pupa: save bill B22-0893 in 22 as bill_5dea8ffa-a048-11e8-96e1-0215f703737a.json
00:02:11 INFO pupa: save vote_event 22 - 2018-07-10 - Final Reading as vote_event_5e22d5e0-a048-11e8-96e1-0215f703737a.json
00:02:11 INFO pupa: save bill B22-0901 in 22 as bill_5e22c5d2-a048-11e8-96e1-0215f703737a.json
00:02:11 INFO pupa: save vote_event 22 - 2018-07-10 - Final Reading as vote_event_5e58c1b4-a048-11e8-96e1-0215f703737a.json
00:02:11 INFO pupa: save bill B22-0885 in 22 as bill_5e58aeb8-a048-11e8-96e1-0215f703737a.json
00:02:11 INFO pupa: save vote_event 22 - 2018-07-10 - Final Reading as vote_event_5e8fd488-a048-11e8-96e1-0215f703737a.json
00:02:11 INFO pupa: save bill B22-0884 in 22 as bill_5e8fc308-a048-11e8-96e1-0215f703737a.json
00:02:12 INFO pupa: save vote_event 22 - 2018-07-10 - Final Reading CC as vote_event_5ed9e532-a048-11e8-96e1-0215f703737a.json
00:02:12 INFO pupa: save bill B22-0813 in 22 as bill_5ed9d3a8-a048-11e8-96e1-0215f703737a.json
00:02:12 INFO scrapelib: POST - http://lims.dccouncil.us/_layouts/15/uploader/AdminProxy.aspx/GetPublicAdvancedSearch
00:02:12 WARNING scrapelib: sleeping for 10 seconds before retry
00:02:22 WARNING scrapelib: sleeping for 20 seconds before retry
00:02:42 WARNING scrapelib: sleeping for 40 seconds before retry
loaded Open States pupa settings...
dc (scrape, import)
  people: {}
  committees: {}
  bills: {}
Traceback (most recent call last):
  File "/opt/openstates/venv-pupa//bin/pupa", line 11, in <module>
    load_entry_point('pupa', 'console_scripts', 'pupa')()
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/__main__.py", line 68, in main
    subcommands[args.subcommand].handle(args, other)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 260, in handle
    return self.do_handle(args, other, juris)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 305, in do_handle
    report['scrape'] = self.do_scrape(juris, args, scrapers)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 173, in do_scrape
    report[scraper_name] = scraper.do_scrape(**scrape_args)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/scrape/base.py", line 111, in do_scrape
    for obj in self.scrape(**kwargs) or []:
  File "/opt/openstates/openstates/openstates/dc/bills.py", line 299, in scrape
    response = self.post(url, headers=headers, data=param_json)
  File "/opt/openstates/venv-pupa/lib/python3.6/site-packages/requests/sessions.py", line 559, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/opt/openstates/venv-pupa/lib/python3.6/site-packages/scrapelib/__init__.py", line 292, in request
    raise HTTPError(resp)
scrapelib.HTTPError: 403 while retrieving http://lims.dccouncil.us/_layouts/15/uploader/AdminProxy.aspx/GetPublicAdvancedSearch
```

Visit http://bobsled.openstates.org for more info.



Discussion:
Looks like i missed one conversion of self.post to requests.post -- i'll fix this.

Resolving patch:
diff --git a/openstates/dc/bills.py b/openstates/dc/bills.py
--- a/openstates/dc/bills.py
+++ b/openstates/dc/bills.py
@@ -296,7 +296,7 @@ def scrape(self, session=None):
             start_record += per_page
             params["request"]["iDisplayStart"] = start_record
             param_json = json.dumps(params)
-            response = self.post(url, headers=headers, data=param_json)
+            response = requests.post(url, headers=headers, data=param_json)
             response = decode_json(response.json()["d"])
             data = response["aaData"]
 


Test patch: