Detecting whether candidates have met minimum course requirements as demonstrated in a mastery test. To achieve this goal, an L2 CAT must provide examinees with a sufficiently broad range of L2 content areas and skill tasks to ensure that their true proficiency is indeed measured in the test items taken. This objective may most easily be accomplished by obtaining or designing a CAT that includes the entire range of ability in its item pool. For example, in the case of a general listening proficiency CAT, items in the pool must cover low to high listening ability levels. In addition, the items need to include a variety of listening tasks, such as comprehension of the main ideas of a conversation or mini-lecture, recognition and recall of details of a conversation, identification of specific words and phrases used in a passage, and so forth.
Reliability refers to the precision and consistency of scores derived from a test instrument.
General factors influencing reliability include, for example, whether instructions for the examinees are clear and explicit, or whether the examinees are sufficiently familiar with the format of the CAT before taking it. Situational factors include those related to the testing environment, such as noise level or whether headphones are provided. Individual factors include transient factors, such as the physical and psychological health of the test takers, and stable factors, such as examinees' experience with similar tests. Validity refers to whether a test actually measures what it purports to measure. It relates to the appropriacy of the inferences made on the basis of the test scores.