Repository: openstates/openstates-scrapers

Issue:
AK failing since at least 2018-04-15
AK has been failing since 2018-04-15

Based on automated runs it appears that AK has not run successfully in 2 days (2018-04-15).


```
  03:51:31 ERROR pupa: cannot resolve pseudo id to Person: ~{"name": "REINBOLD"}
03:51:34 ERROR pupa: cannot resolve pseudo id to Person: ~{"name": "KNOPP"}
03:51:34 ERROR pupa: cannot resolve pseudo id to Person: ~{"name": "SEATON"}
03:51:37 ERROR pupa: cannot resolve pseudo id to Person: ~{"name": "GIESSEL BY REQUEST"}
03:51:39 ERROR pupa: cannot resolve pseudo id to Person: ~{"name": "NEUMAN"}
03:51:39 ERROR pupa: cannot resolve pseudo id to Person: ~{"name": "COMMUNITY & REGIONAL AFFAIRS"}
03:51:44 ERROR pupa: cannot resolve pseudo id to Person: ~{"name": "WIELECHOWSKI"}
03:51:45 ERROR pupa: cannot resolve pseudo id to Person: ~{"name": "LABOR & COMMERCE                         BY REQUEST"}
03:51:47 ERROR pupa: cannot resolve pseudo id to Person: ~{"name": "KELLY BY REQUEST"}
03:51:48 ERROR pupa: cannot resolve pseudo id to Person: ~{"name": "SHOWER"}
03:51:48 ERROR pupa: cannot resolve pseudo id to Person: ~{"name": "JOHNSTON"}
03:51:52 ERROR pupa: cannot resolve pseudo id to Person: ~{"name": "JOHNSTON BY REQUEST"}
03:51:54 ERROR pupa: cannot resolve pseudo id to Person: ~{"name": "HOFFMAN"}
03:51:59 ERROR pupa: cannot resolve pseudo id to Person: ~{"name": "EGAN"}
03:52:05 ERROR pupa: cannot resolve pseudo id to Person: ~{"name": "VON IMHOF"}
03:52:08 ERROR pupa: cannot resolve pseudo id to Person: ~{"name": "HEALTH & SOCIAL SERVICES                 BY REQUEST"}
03:52:14 ERROR pupa: cannot resolve pseudo id to Person: ~{"name": "von"}
03:52:14 ERROR pupa: cannot resolve pseudo id to Person: ~{"name": "Imhof"}
loaded Open States pupa settings...
ak (scrape, import)
  people: {}
  bills: {}
import jurisdictions...
import organizations...
import people...
import posts...
import memberships...
import bills...
import events...
import vote events...
Traceback (most recent call last):
  File "/opt/openstates/venv-pupa//bin/pupa", line 11, in <module>
    load_entry_point('pupa', 'console_scripts', 'pupa')()
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/__main__.py", line 68, in main
    subcommands[args.subcommand].handle(args, other)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 260, in handle
    return self.do_handle(args, other, juris)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 307, in do_handle
    report['import'] = self.do_import(juris, args)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 217, in do_import
    report.update(vote_event_importer.import_directory(datadir))
  File "/opt/openstates/venv-pupa/src/pupa/pupa/importers/base.py", line 196, in import_directory
    return self.import_data(json_stream())
  File "/opt/openstates/venv-pupa/src/pupa/pupa/importers/base.py", line 233, in import_data
    obj_id, what = self.import_item(data)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/importers/base.py", line 272, in import_item
    raise DuplicateItemError(data, obj, related.get('sources', []))
pupa.exceptions.DuplicateItemError: attempt to import data that would conflict with data already in the import: {'motion_classification': ['passage'], 'legislative_session_id': UUID('ba4ec600-d1cc-4fcb-8486-79e8e91531ec'), 'bill_id': 'ocd-bill/5cfb2964-6d68-4c8f-812a-2694a1c6df6b', 'identifier': '', 'motion_text': 'Shall the effective date clauses be adopted?', 'start_date': '2018-04-12', 'result': 'pass', 'extras': {}, 'organization_id': 'ocd-organization/85b8d5c1-0dde-4280-8528-9ef5d9fb5b5e'} (already imported as Shall the effective date clauses be adopted? on HB 286 in Alaska 30th Legislature (2017-2018))
obj1 sources: ['http://www.legis.state.ak.us/basis/get_jrn_page.asp?session=30&bill=HB286&jrn=2631&hse=S']
obj2 sources: ['http://www.legis.state.ak.us/basis/get_jrn_page.asp?session=30&bill=HB286&jrn=2629&hse=S']
```

Visit http://bobsled.openstates.org for more info.



Discussion:
pupa_id added in latest commit
These are indeed duplicates.  

The page the scraper is reading for each vote include all prior votes on the bill; thus, the pages have votes in common. 
 
I suspect that the correct behavior is for the scraper to only read the first entry on the page, but that's clearly not what the author intended, so I'm leaving this for now hoping someone better informed will look at it.


Resolving patch:
diff --git a/openstates/ak/bills.py b/openstates/ak/bills.py
--- a/openstates/ak/bills.py
+++ b/openstates/ak/bills.py
@@ -243,9 +243,12 @@ def parse_vote(self, bill, action, act_chamber, act_date, url):
         votes_text = re_vote_text.split(votes_text)
         votes_data = zip(votes_text[1::2], votes_text[2::2])
 
+        iVoteOnPage = 0
+
         # Process each.
         for motion, text in votes_data:
 
+            iVoteOnPage += 1
             yes = no = other = 0
 
             tally = re.findall(r'\b([YNEA])[A-Z]+:\s{,3}(\d{,3})', text)
@@ -269,7 +272,8 @@ def parse_vote(self, bill, action, act_chamber, act_date, url):
             vote.set_count('yes', yes)
             vote.set_count('no', no)
             vote.set_count('other', other)
-            vote.pupa_id = url
+
+            vote.pupa_id = (url + ' ' + str(iVoteOnPage)) if iVoteOnPage > 1 else url
 
             # In lengthy documents, the "header" can be repeated in the middle
             # of content. This regex gets rid of it.


Test patch: