Gray dot is only drawn when not on a problem

Also a minor performance fix