Ignore mouse clicks on the object we are already tracking.