The sample size should be large enough to detect a statistically significant difference between the two versions of the product or website, while also being practical in terms of time and resources. Typically, sample sizes of at least 250 conversions are recommended.