Repository: openstates/openstates-scrapers

Issue:
KS failing since at least 2019-03-17
KS has been failing since 2019-03-17

Based on automated runs it appears that KS has not run successfully in 2 days (2019-03-17).


```
  06:03:33 INFO pupa: save bill SCR1609 in 2019-2020 as bill_8d128b1c-4943-11e9-a429-02b38130cc66.json
06:03:33 INFO scrapelib: GET - http://www.kslegislature.org/li/b2019_20/year1/measures/sr1721/
06:03:38 INFO pupa: save bill SR1721 in 2019-2020 as bill_8fdede5e-4943-11e9-a429-02b38130cc66.json
06:03:38 WARNING pupa: unknown action code on SB22: misc_he_200 Hearing: Tuesday, January 29, 2019, 9:30 AM Room 548-S
06:03:38 WARNING pupa: unknown action code on SB22: misc_he_200 Hearing continuation: Thursday, January 31, 2019, 9:30 AM Room 548-S
06:03:38 WARNING pupa: unknown action code on SB22: misc_he_200 Hearing continuation: Wednesday, January 30, 2019, 9:30 AM Room 548-S
06:03:38 WARNING pupa: unknown action code on SB22: cow_jcow_684 Committee of the Whole - Motion to rerefer to committee failed  Yea: 12 Nay: 28
06:03:38 WARNING pupa: unknown action code on SB22: misc_he_200 Hearing: Wednesday, February 20, 2019, 3:30 PM Room 112-N
06:03:38 WARNING pupa: unknown action code on SB22: misc_he_200 Hearing: Tuesday, February 19, 2019, 3:30 PM Room 112-N
06:03:38 INFO scrapelib: GET - http://www.kslegislature.org/li/b2019_20/measures/sb22/
06:03:43 INFO pupa: Get http://www.kslegislature.org/li/b2019_20/measures/vote_view/je_20190314184842_374109/
06:03:43 INFO pupa: save vote_event 2019-2020 - 2019-03-14 - Concurred with amendments; as vote_event_95eec9bc-4943-11e9-a429-02b38130cc66.json
06:03:43 INFO pupa: Get http://www.kslegislature.org/li/b2019_20/measures/vote_view/je_20190308084534_667513/
06:03:43 INFO pupa: save vote_event 2019-2020 - 2019-03-08 - Final Action - Passed as amended; as vote_event_96029762-4943-11e9-a429-02b38130cc66.json
06:03:44 INFO pupa: Get http://www.kslegislature.org/li/b2019_20/measures/vote_view/je_20190307135353_561816/
06:03:44 INFO pupa: save vote_event 2019-2020 - 2019-03-07 - Committee of the Whole - Be passed as amended as vote_event_961789ec-4943-11e9-a429-02b38130cc66.json
06:03:44 INFO pupa: Get http://www.kslegislature.org/li/b2019_20/measures/vote_view/je_20190307130813_055560/
06:03:44 INFO pupa: save vote_event 2019-2020 - 2019-03-07 - Committee of the Whole - Amendment by Representative Hodge was rejected as vote_event_962b7e8e-4943-11e9-a429-02b38130cc66.json
06:03:44 INFO pupa: Get http://www.kslegislature.org/li/b2019_20/measures/vote_view/je_20190307123130_531081/
06:03:44 INFO pupa: save vote_event 2019-2020 - 2019-03-07 - Committee of the Whole - Amendment by Representative Hodge was rejected as vote_event_9640d568-4943-11e9-a429-02b38130cc66.json
06:03:44 INFO pupa: Get http://www.kslegislature.org/li/b2019_20/measures/vote_view/je_20190307122135_586132/
06:03:44 INFO pupa: save vote_event 2019-2020 - 2019-03-07 - Committee of the Whole - Amendment by Representative Ward was rejected as vote_event_9656684c-4943-11e9-a429-02b38130cc66.json
06:03:44 INFO pupa: Get http://www.kslegislature.org/li/b2019_20/measures/vote_view/je_20190207152923_151356/
06:03:44 INFO pupa: save vote_event 2019-2020 - 2019-02-07 - Final Action - Passed; as vote_event_966a631a-4943-11e9-a429-02b38130cc66.json
06:03:44 INFO pupa: Get http://www.kslegislature.org/li/b2019_20/measures/vote_view/je_20190206162047_699078/
loaded Open States pupa settings...
ks (scrape, import)
  bills: {}
Traceback (most recent call last):
  File "/opt/**PGUSER**/venv-pupa//bin/pupa", line 11, in <module>
    load_entry_point('pupa', 'console_scripts', 'pupa')()
  File "/opt/**PGUSER**/venv-pupa/src/pupa/pupa/cli/__main__.py", line 68, in main
    subcommands[args.subcommand].handle(args, other)
  File "/opt/**PGUSER**/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 278, in handle
    return self.do_handle(args, other, juris)
  File "/opt/**PGUSER**/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 327, in do_handle
    report['scrape'] = self.do_scrape(juris, args, scrapers)
  File "/opt/**PGUSER**/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 175, in do_scrape
    report[scraper_name] = scraper.do_scrape(**scrape_args)
  File "/opt/**PGUSER**/venv-pupa/src/pupa/pupa/scrape/base.py", line 112, in do_scrape
    for obj in self.scrape(**kwargs) or []:
  File "/opt/**PGUSER**/**PGUSER**/**PGUSER**/ks/bills.py", line 25, in scrape
    yield from self.scrape_chamber(chamber, session)
  File "/opt/**PGUSER**/**PGUSER**/**PGUSER**/ks/bills.py", line 112, in scrape_chamber
    yield from self.scrape_html(bill, session)
  File "/opt/**PGUSER**/**PGUSER**/**PGUSER**/ks/bills.py", line 164, in scrape_html
    yield from self.parse_vote(bill, link)
  File "/opt/**PGUSER**/**PGUSER**/**PGUSER**/ks/bills.py", line 199, in parse_vote
    vote_chamber = chamber_date_line_words[0]
IndexError: list index out of range
```

Visit http://bobsled.openstates.org for more info.



Discussion:

Resolving patch:
diff --git a/openstates/ks/bills.py b/openstates/ks/bills.py
--- a/openstates/ks/bills.py
+++ b/openstates/ks/bills.py
@@ -189,6 +189,11 @@ def parse_vote(self, bill, link):
             self.warning("{} fetching vote {}, skipping".format(err, link))
             return
 
+        if 'Varnish cache server' in text:
+            self.warning("Scrape rate is too high, try re-scraping with "
+                         "The --rpm set to a lower number")
+            return
+
         if 'Page Not Found' in text or 'Page Unavailable' in text:
             self.warning("missing vote, skipping")
             return
@@ -199,7 +204,6 @@ def parse_vote(self, bill, link):
         vote_chamber = chamber_date_line_words[0]
         vote_date = datetime.datetime.strptime(chamber_date_line_words[-1], '%m/%d/%Y')
         vote_status = " ".join(chamber_date_line_words[2:-2])
-
         opinions = member_doc.xpath("//div[@id='main_content']/h3[position() > 1]/text()")
         if len(opinions) > 0:
             vote_status = vote_status if vote_status.strip() else motion[0]
diff --git a/openstates/ks/ksapi.py b/openstates/ks/ksapi.py
--- a/openstates/ks/ksapi.py
+++ b/openstates/ks/ksapi.py
@@ -188,4 +188,5 @@
     'mot_susp_206': None,
     'cur_con_101': None,  # concur. failed
     'cur_om_141': 'referral-committee',
+    'misc_he_200': None,
 }


Test patch: