Added support for compressed logs and a check against duplicate
lines.