full preprocessing