Initial serving code

- Serves simple static content
- Basic crawling API setup
Serves are now done through channels with mock crawls
Refactored command line flag handling
Moved crawling code to its own package and refactored serve/crawl management logic.
Added git ignore
Crawlers now read from HTTP gets instead of stubbing data.

- Serve code also modified to not block to recieve crawl data.
- Crawlers now return content body of seed urls
Crawler now searches through anchor links.
Added documentation