masks bad values for fitting
finds limits for y-axis automatically