lot of formatting & preprocessing