add tweet id to the concise output
process new raw file format