porting over some preprocessing algorithms