Do not conflict when switching from brush mode to scatter plot.