Make sure to not override insets set by the user.