Major performance improvement - don't call setNeedsDisplay from setDirty