- Handled a case where a server might return an erroneous content-type.
- Crawler will check the file size of a file, if available, before attempting to parse it.
- Fixed some spacing issues.