dobtco/openrfps-scrapers — pull request #20 by marks (2014-03-04 03:05:41+00:00)
Title: First pass at Indiana state scraping

This is my first pass and is loosely based on the IL scraper Javascript. There is more work to do but this is a start.

Scraper output:
````
mba62:openrfps-scrapers mark$ bin/openrfps run scrapers/in/rfps.js
[ 'http://www.in.gov/cgi-bin/idoa/cgi-bin/bidad.pl?spec=300-14-63265&desc=Contract+for+Services+for+Invasive+Plant+Control&method=NEGOTIATED BID&code=T',
'http://www.in.gov/cgi-bin/idoa/cgi-bin/bidad.pl?spec=RFP-14-058&desc=Centralized+Production+and+Direct+Distribution+of+License+Plates+and+Registration+Documents&method=RFP&code=7',
'http://www.in.gov/cgi-bin/idoa/cgi-bin/bidad.pl?spec=RFQ_ISF21&desc=Misc.+Furniture&method=NOTICE TO BIDDERS&code=P',
'http://www.in.gov/cgi-bin/idoa/cgi-bin/bidad.pl?spec=RFQ_ISF20&desc=End+Tables&method=NOTICE TO BIDDERS&code=P',
'http://www.in.gov/cgi-bin/idoa/cgi-bin/bidad.pl?spec=RFQ_ISF19&desc=Sofas&method=NOTICE TO BIDDERS&code=P',
'http://www.in.gov/cgi-bin/idoa/cgi-bin/bidad.pl?spec=300-14-63398&desc=Outright+Purchase+of+Airboat+and+Trailer+for+IDNR&method=NEGOTIATED BID&code=T',
'http://www.in.gov/cgi-bin/idoa/cgi-bin/bidad.pl?spec=RFI-14-73&desc=Electronic+Media+Destruction+and+Shredding+Services&method=RFI&code=K',
'http://www.in.gov/cgi-bin/idoa/cgi-bin/bidad.pl?spec=ASA-14-079&desc=QPA+for+Snack+Products+for+Pen+Products&method=NEGOTIATED BID&code=T',
'http://www.in.gov/cgi-bin/idoa/cgi-bin/bidad.pl?spec=RFP-14-75&desc=Hard+Copy+Book+Collections&method=RFP&code=B',
'http://www.in.gov/cgi-bin/idoa/cgi-bin/bidad.pl?spec=RFI-14-081&desc=Development,+Operation+and+Maintenance+of+an+Inn+and+Related+Facilities+at+Potato+Creek+State+Park&method=RFI&code=T',
'http://www.in.gov/cgi-bin/idoa/cgi-bin/bidad.pl?spec=RFP-14-074&desc=Paint+and+Paint+Supplies&method=RFP&code=W',
'http://www.in.gov/cgi-bin/idoa/cgi-bin/bidad.pl?spec=RFP-14-049&desc=Sustained+Statewide+Public+Relations+and+Marketing+Campaign&method=RFP&code=7',
'http://www.in.gov/cgi-bin/idoa/cgi-bin/bidad.pl?spec=ASA-14-076&desc=Lab+Supplies&method=NEGOTIATED BID&code=V',
'http://www.in.gov/cgi-bin/idoa/cgi-bin/bidad.pl?spec=ASA-14-089&desc=QPA+for+Cosmetic+Grade+Soap+Products+for+Pen+Products&method=NEGOTIATED BID&code=T',
'http://www.in.gov/cgi-bin/idoa/cgi-bin/bidad.pl?spec=570-14-24746&desc=Wireless+Paging+System&method=NEGOTIATED BID&code=8',
'http://www.in.gov/cgi-bin/idoa/cgi-bin/bidad.pl?spec=RFP-14-83&desc=DNA+Sample+Collection+Services&method=RFP&code=K' ]
Done scraping!
Cached results to scrapers/in/rfps.json
[
{
"html_url": "http://www.in.gov/cgi-bin/idoa/cgi-bin/bidad.pl?spec=300-14-63265&desc=Contract+for+Services+for+Invasive+Plant+Control&method=NEGOTIATED BID&code=T",
"id": "300-14-63265",
"type": "NEGOTIATED BID",
"title": "Contract for Services for Invasive Plant Control",
"responses_open_at": "3/4/2014",
"contact_name": "Deaton, Teresa"
},
{
"html_url": "http://www.in.gov/cgi-bin/idoa/cgi-bin/bidad.pl?spec=RFP-14-058&desc=Centralized+Production+and+Direct+Distribution+of+License+Plates+and+Registration+Documents&method=RFP&code=7",
"id": "RFP-14-058",
"type": "RFP",
"title": "Centralized Production and Direct Distribution of License Plates and Registration Documents",
"responses_open_at": "3/4/2014",
"contact_name": "Thiemann, Adam"
},
{
"html_url": "http://www.in.gov/cgi-bin/idoa/cgi-bin/bidad.pl?spec=RFQ_ISF21&desc=Misc.+Furniture&method=NOTICE TO BIDDERS&code=P",
"id": "RFQ_ISF21",
"type": "NOTICE TO BIDDERS",
"title": "Misc. Furniture",
"responses_open_at": "3/5/2014",
"contact_name": "Archer, Mary Beth"
}
]
13 RFPs not printed for length considerations
````

Test output:
````
mba62:openrfps-scrapers mark$ bin/openrfps test scrapers/in/rfps.js
The scraper returns at least one result: OK
item.id is returned for all items: OK
item.type is valid for all items: Not OK
item.contact_email is a proper address (or blank): OK
download URLs are valid (or blank): OK
item.id is unique for each item: OK
item.title is returned for all items: OK
NIGP codes are digits: OK
````