pull shard_id from conf as an int
factor out some static metadata into an object fields
move EOL to driver
more universal, modular, pluggable output formatting ; TODO: add pluggable compression options