Finished parser, mostly works ok now.
Moved main body into a function in preparation for multi-process version. Have found a problem with decoding gzip files though, not sure why they are not being decoded as text properly. Needs investigation (not working at the moment)."