Beware studies that apply many different tests or test many different hypotheses from the same data, and claim a significant result. Often, such studies neglect the effect of multiplicity, and the chance of a false positive is much higher than the authors recognize. Garbage in, garbage out Often in science it is the hypothesis rejections that are interesting. At the first extreme cited above, every discovery is true. At the second extreme, every discovery is false.
Few scientists seek to publish negative results.
Consequently, we see primarily the rejections in tests of a population of hypotheses of which an unknown fraction really are false. Suppose one tests a large collection of hypotheses. No matter how good the test is, it cannot make a true hypothesis false, nor a false hypothesis true. Unless the test always rejects, if fed a steady diet of false hypotheses, it will fail to reject some of them. Unless the test never rejects, if fed a steady diet of true hypotheses, it will erroneously reject some of them.