phillipadsmith/everycandidate.org — pull request #14 by iirving (2014-09-24 23:47:51+00:00)
Title: validate urls for  council or tdsb data files

as discussed, here is a ruby program to test the links in the council and tdsb csv files.

to use
$ruby _scripts/validate_domains.rb [switches]

where [switches] is either council or tdsb
anything else or nothing prints out a usage help message.

so $ruby _scripts/validate_domains.rb council
or $ruby _scripts/validate_domains.rb tdsb

the program with reports on each data row if it finds a problem (for up to 4 urls for concil and 3 urls for tdsb) , usually a typo causing :
a) no web site found
b) (especially for twitter/facebook/linkedin) the page wasn't found,
c) another problem it reports on is for some redirects (for example http://maryfragedakis.ca redirects to http://www.maryfragedakis.ca/ )

fixing the problem is another kettle of fish! Sometimes its an obvious typo and sometime the website or page isn't there yet. redirects should be replaced with want they redirect to.

Next, I'll look at validating email (and alt email) address for good structure and valid domains.