added helper methods to
1 - get website source
2 - get robot.txt
3 - get website links
added link class to store links with
1 - link address
2 - link text