Make sure we are only building options when we need to