excludes added to crawler
Major suite 3 bugfix using urllib2.

Added crawler-excludes option.
updated readme
updated changelog
Merge branch 'bugfix/timeout' into development
bugfix urllib2.HTTPError
spoof useragent for more successful tests
link testing pull into function
bugfix use proper test case for test bad links
add cookie processor and host header
update error message for get link status
enforce request delay in suite 3
research added
rearrange Suite 2 and 3
update README for new suite order
optimize suite 3 to avoid profiling bad links
default domain filter updated based on target string
reorganize comments
added --preseed option
triage class with summary implemented