general version of preprocessing