Clean up some old issues with resampling and drop/dup support

This avoids some of the swinging back and forth when using resampling or skip/dup due
to error being considered multiple times.