Drawing on meta-analysis techniques, this comparative procedure is called validity generalization in the personnel selection and psychometric literature. Several types of measures lend themselves particularly well to validity generalization. Thus, there is significant evidence that the validation results for general cognitive ability measures are generalizable across settings. It is not necessary, therefore, to conduct a validity study for a given job at every business location in America. The validity of 'general cognitive ability' for predicting clerical performance in one setting, for example, can be inferred from the validity found in the hundreds of previous studies.
Accurate statistics require large sample sizes.
The criterion related validity of a test in a local validation study is usually inferred only if the findings reach a certain level of magnitude called 'statistical significance'. The smaller the sample of subjects, the higher the observed validity coefficient would need to be in order to infer an acceptable level of validity. You would not expect, for example, to draw accurate predictions of a national election by polling a sample of only 15 voters. Most polls interview 1,000 voters or more. Unless there are hundreds of subjects at a particular location, the data cannot be used to draw accurate conclusions in isolation.