Only update term in inverse context if not set.