finished most stuff for preprocessing