of examinees who respond correctly to an item. Item difficulties can range from 0.0 to 1.00. Items
with difficulties between 0.0 and .30 have a low proportion of respondents answering correctly
and are considered hard items. Items with difficulties between .70 and 1.00 have a high proportion
of respondents answering correctly and are considered easy items. The reader should note that the
term item difficulty is a technical term and seems contradictory to the lay person’s definition of
difficulty. An item with a low item difficulty is not an item of low difficulty, but rather a very
difficult item.

Biserial correlations (rbis), the correlation between the dichotomously scored item and the
continuously distributed subtest score, were computed as measures of item discrimination. Items
with discrimination values above .80 are typically viewed as having high discriminatory power;
items with discrimination values below .20 are typically viewed as having poor discriminatory

power.

Computational formulas for these statistics differ according to whether the subtest is
analyzed as a speeded or a power subtest. For a power subtest, item difficulty is calculated using
all examinees taking the test, under the assumption that all examinees will have an opportunity to
consider every subtest item. For a speeded subtest, difficulty is calculated using only examinees
who respond to the item or a subsequent item of the subtest. Examinees who do not attempt items
are not considered in these speeded analyses.

Subtest and Composite Analysis
Means, standard deviations, skew, kurtosis, reliability and proportion correct are

presented for each subtest. For composite analyses, means, standard deviations, skew and kurtosis

14



The item difficulty distributions of subtests Form Q1 are similar to the item difficulty
distributions of Form P1. Again, item difficulties tend to range from .20 to .80. Five subtests,
Mechanical Comprehension, Electrical Maze, Table Reading, Aviation Information and General
Science have items with item difficulties below .20. Twelve subtests have at least one item with a
difficulty value above .80. Table Reading is a relatively easy subtest; half of the items have
difficulty values above .80. All sixteen subtests have a mean level of item difficulty between .40
and .60.

Item difficulties for test Form Q2 occur predominantly in the .20 to .80 range. Six
subtests, Verbal Analogies, Mechanical Comprehension, Electrical Maze, Table Reading, Aviation
Information and General Science, include items with item difficulties below .20. Eleven subtests
include items with difficulty value greater than .80. As in P] and Q2, the majority of items from
the Table Reading subtests have difficulties above .80. Fifteen subtests had mean level of item
difficulty between .40 and .60.

The subtest difficulties of P1, Q1, and Q2 show fluctuations in the frequency distributions
of the item difficulties. Consideration of items with difficulty greater than .60 gave the same
results as found in the preliminary equating data. Q2 had two or more easier items than Q1 for
Verbal Analogies, Arithmetic Reasoning, Reading Comprehension and Math Knowledge and Q1
had two or more easier items than Q2 on Block counting. There do not appear to be any
substantial or systematic differences in the mean item difficulty of a subtest across the three test
forms, The maximum difference in subtest mean item difficulty among any two of the three test
forms ranged from .002 to .034. Only four subtests, Arithmetic Reasoning, Reading

Comprehension, Scale Reading and Hidden Figures, had a largest pairwise difference above .020.

35



Table 13. Demographic Percentages of Total Sample for Operational Equating Study

Pl Ql Q
n=4697__ n=3387_n=300
Gender Male 73.4 74.8 76.1
Female 26.3 25.0 23.7
Missing 0.3 0.3 0.2
Race American 0.8 0.6 0.9
Indian
Asian 5.1 5.4 4.5
Black 12.9 13.4 12.8
Hispanic 6.7 6.1 6.6
Caucasian 74.1 74.2 747
Missing 03 0.4 0.4
Years of 12 24.0 27.4 30.0
education 13 13.6 14.5 14.7
14 12.4 114 11.1
15 . 13.6 14.4 13.0
16 273 24.2 23.4
17 5.2 43 44
18 2.2 2.5 2.2
19 0.7 0.5 0.7
20 0.3 0.1 0.1
21 0.0 0.2 0.1
Missing, 0.6 0.5 0.3
Highest High School 58.3 63.0 65.0
degree
earned Associates 9.6 8.2 7.7
Bachelors 29.8 26.5 24.7
Masters 15 15 1.6
Doctorate 0.2 0.2 0.2
Missing 0.6 0.6 0.7
Administration

The AFOQT data for the operational equating study were collected from operational
testing sessions at the Military Processing Stations (MEPS) and their outlying sites, Mobile

Examining Team Sites (METS). Examiners followed the usual testing procedures for applicants,

31



among subtests of different forms. The maximum correlation among subtests is .76, the
correlation between Arithmetic Reasoning and Data Interpretation subtests on Form Q2. The
minimum correlation is .20 and occurs between the Word Knowledge and Electrical Maze
subtests on Form P1. The subtest intercorrelations show similar patterns across the three forms.
The maximal difference between any of the three subtest correlations in the 120 triads is greater
than .10 in only two cases; in these instances the correlations are .10 and .11. Thus, there is a
high degree of similarity among the correlation matrices across the three test forms.

The analyses of the common items on the subtests for Forms Q1 and Q2 indicate that the
means are generally similar to one another and to those of Form P]. Table 2 in Appendix B
presents the common item subtest means and standard deviations for forms P1, Q1 and Q2. The
subtest means on Forms Q1 and Q2 tend to be slightly higher than the corresponding subtest
means of Form P1, however twenty-six out of thirty-two means are within one-tenth of a standard
deviation of the P1 subtest means. The discrepancies occur for one of the comparisons between
P1 and Q1 and for five of the comparisons between P1 and Q2. In general, the common item

means across test forms are approximately equivalent.

40



Item biserial statistics similarly showed a well developed test, with all subtests having
mean biserial correlations well over .50. These mean biserial correlations for Forms Q1 and Q2
subtests were similar to one another, as well as to Form P1 subtests, and had no systematic
pattern of being higher or lower in one test form. Table 3 shows the similarity of the distributions
of biserial correlations for Forms Q1 and Q2. In some selection decisions, item content concerns
took precedence over item biserial correlations in order to reduce item redundancy and improve
subtest content quality. Some of these lower mean biserial correlations and discrepancies in the
distributions for test Forms Q can be traced to substituting items with lower biserial correlations

on the basis of content concerns.

Table 2. Distribution of Item Difficulties for Forms Q1 and Q2 at Test Construction Stage

Subtest Item Difficulties Forms Q1 (Q2)
10-19 20-29 30-39 .40-.49 50-59 .60-.69 _.70-.79 _.80-.89 _.90-.99

Verbal Analogies 0 1 2 5 3 4 4 3 3
Arithmetic Reasoning 0 0 6 3 2 6 4 4 0
Reading

Comprehension 0 0 3 2 4 6 8 2 0
Data Interpretation 0 1 1 3 (2) 6 (9) 765) 7 (6) 0(1) 0
Word Knowledge 0 0 3 5 5 5 4 3 0
Math Knowledge 0 0 0 2 7 8 6 2 0
Mechanical

Comprehension 0 0 4 8 5 3 0 0 0
Electrical Maze 6 3 2 2 3 4 0 0 0
Scale Reading }
Instrument

Comprehension 0 0 4 2 6 6 2 0 0
Block Counting 10) 0 3(4) 2 3. 48) 46 20) 1
Table Reading
Aviation Information 0 5 5 5 2 2 1 0 0
Rotated Blocks 0 2 2 5 1 2 0 3 0
General Science 0 2 4 5 4 4 1 0 0
Hidden Figures 0 0 2 3 2 2 2 2 2

Note. * Reported values are for Forms Q1 and Q2 except where a parentheses surround the value for Form Q2
indicating a discrepancy.



APPENDIX D: OPERATIONAL CONVERSION TABLES FOR AFOQT FORMS QI] AND Q2

69



SLING

t £ £ 9 T L 9 4 0 1) 9 9 £ 0 0 soindiy UOpprH
0 I £ ras € € 9 L £ 0 s v 8 if T aouatog [RIED
0 € € 9 £ ¢ s € £ 0 € L t € 0 SOOT pazjoy
0 if I 6 8 I + 8 9 0 9 L s I 0 uorjewoyuy uoneIAy
SI v s g I v s s 9 0 St 9 s v 0 Surpeoy o1qeL
4 9 v s € L s 1 0 L 6 v 0 0 sununod xorg
0 0 v 1 4 Il v 0 0 s Ir v 0 0 uorsuayasduroD Juoumnnsuy
T ol 8 ST 9 tI v1 T 0 61 6 t 0 0 Surpeay o[e9g
0 0 s L 8 L t 6 t 0 9 t I I Ooze [POM OeTT
0 0 € st z 9 8 s 0 z L 8 € 0 uorsuayasduroD jeorueyooyy
0 if 9 sl € +1 I 0 0  t 0 0 0 dBpapnouy WEA,
if T L Il s 6 8 0 0 +1 9 s 0 0 S8pojMouy PIO,
0 s L rat T €1 s T 0 8 SI z 0 0 uonejdiajuy Bed
0 t 8 L 8 L 9 c 0 Ir 8 v zt 0 uolsusyaiduiop SuIpeay
0 € 9 im ¢ 8 8 if 0 tl 6 z 0 0 Sumuoseoy onouuy
€ v s 6 v I v I 0 SI L T I I sordojeuy [eqioA,
66> 08> = 09 OP 0 08- 09 =Or- = 07 66- = 08"- or’

Os< 09< OF< 0%< 09°< -00° 08"< 0@'< ysaiqng

Apmg Suyenby Areunuyorg uy sojduresqng SL Pue D.LOU “WAV 40} Id WO 40y saN|NoUyI wioI] Jo UONNANSIq “T-V a0BL

50



REFERENCES

Angoff, W. H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.),
Educational Measurement (2nd ed.). Washington, DC: American Council on Education.

Berger, F. R., Gupta, W. B., Berger, R. M., & Skinner, J. (1988). Air Force Officer
Qualifying Test (AFOQT) Form P: Test Construction (AFHRL-TR-88-30, AD-A200 678).
Brooks AFB, TX: Manpower and Personnel Division, Air Force Human Resources Laboratory.

Berger, F. R., Gupta, W. B., Berger, R. M., & Skinner, J. (1990). Air Force Officer
Qualifying Test (AFOQT) Form P: Test Manual (AFHRL-TR-89-56, AD-A221 004). Brooks
AFB, TX: Manpower and Personnel Division, Air Force Human Resources Laboratory.

Gulliksen, H. (1950). Theory of mental tests. New York: John Wiley & Sons, Inc.

Rogers, D. L., Roach, B. W., & Short, L. O. (1986). Air Force Officer Qualifying Test
Form O: Development and Standardization (AFHRL-TR-86-24, AD-A172 037). Brooks AFB,
TX: Manpower and Personnel Division, Air Force Human Resources Laboratory.

Steuck, K. W., Watson, T. W., & Skinner, J. (1988). Air Force Officer Qualifying Test
(AFOQT): Forms P pre-implementation analyses and equating (AFHRL-TR-88-6, AD-A201
100). Brooks AFB, TX: Manpower and Personnel Division, Air Force Human Resources

Laboratory.

48



54

i) v or I 0 if £1 T 0 0 q 8 s T 0 somty USppI
0 € I v 0 0 0 Li £ 0 0 4 1a 9 T BOUdTOS JEIOUD
0 6 s if 0 I L L 0 0 T L L 0 0 SYOTEA PAeIOY
0 0 eI 9 if 0 a 8 0 0 0 4 £1 s 0 uoneUOyU] UOHeIAY
6 61 6 T 4 6 0c 6 t 0 6 €1 €I ¢ I Surpesy 9192.
1 I L q 0 z 8 9 v 0 0 6 v L 0 sununog xorg
if 9 Il 4 0 L ol £ 0 0 £ I 9 0 0 uoysuoyarduroD juownNsuy
0 L & 8 4 0 a (<4 9 0 T rat v1 Ir 4 Suppeay o[eog
0 T a L 0 0 or L £ 0 0 s 9 8 T OZR [POMS
0 0 1 g I 0 or L € 0 0 £ ST 4 0 uoysusyardwiod jeorueysoyy
0 s sI s 0 £ 02 4 0 0 (4 8 6 v 4 odpoymouy WEA,
0 ST or 0 0 T SI 8 0 T T or 6 s 0 Spo] mouy Pro
0 9 ST v 0 0 8 LI 0 0 0 s cas 9 0 uonejesdrouy Beg
0 9 91 £ 0 0 6 Ir s 0 4 9 rai s 0 uworsusyorduroy surpeoy,
0 s SI s 0 0 ras €1 0 0 £ L a £ 0 Suyuoseoy oneunpiry
0 4 61 ¢ 0 I v 91 £ I 0 4 Lt s I sordoyeuy [eqio,,
66- 08" =09" OF 07 66> 08> 09° OF 07 66" 08" 09 Or- 0%"

Os'< 09'< OF < 0% < -00° Os'< 09'< Or< O%< -00" Os'<  09< Or< O%< -00° ysaiqns

SLING OLOW Vvav

Apms Sunenby Areuruyjorg uy sajduresqng $1 W@ Pur O.LOU “WAV 305 1D WHO, 105 suoNEjaLI0D jeLESIg Jo UONNQINSI ‘s-y a1qeL


99L° 919" hl’ Oh 69 0" TE OOH EEE COT «CITE LOD ~<OFD~SHL OOP GIP 00S soma WopPrH
9so' Iso" ty PHT 99" BHT 8H SBT OVE «ETE «LOE «6I'9 «(98'S «LLS OER" soua!og [eIaUaH
819 £04’ 699 9- tO = 80—sOT—“‘i‘E’OCGS— BZ SOE «ELT «SOD «—LOD.HL'S: SOE’ (GE ULE. SOOT porwjoy
sly’ 8S 09h ph T6'T OFT: GL) ETT «LO SSZ HLT COMET «OSES «OOPS OSC OCLS@TM OT’ woneuLioyuy
uonemay
1% 10 Lo" 80 0FL «ISL «969 POI? G90T 6L:O% 6S 0S 905" Surpeoy 2198 L
IIs’ £08 798" OF Se> OL I so- Il'b 90% LS 856 610 £26 ELh IIS Sur Sununog xorg
es Sel’ L6L 6B EHCGCCGH’(“iCSHCOHE «CTH BHD SED «07D HIE’ -90E" HOE —_NOISUaYardwog
quoWnyjsuy
et Ob- 6€ tr Lv Le FEL S89 LOO SBI B0'8T GIL 89 Shr 6Ir Surpeoy 91205
ogo Bes’ 919" SPOT SOT 78’ BLT VEE ELT OHGT OCITD. «SES O'S. BBZCSSZ’OosT’ ze [2911199] 3
969 68S" CIN. fy 6’ TTT TLCS HE LOT OCOL'E: «OHO GL'S. «10's 08Z" 67's orSuayardut0D
yeorueysoyy
Is8' ZOL" 18L’ fh UST BZ GCOOTT:CLET OHS OBE (OSH «LIG «(958 «FR GSE’ LzeOLtE (OS pafmouy EW
ces’ ss" 6E8’ 80’ 60 SOL) sTLT SLT 96's EOS. CES SOS 896 «HS C10'GCLLE’ COLE ZSE— OBPO[MOUTY pro,
e18" TLL" $89" Sh EO LE SBS HGH SHY «GE «956 «S96 OBO. «ELE’««69E Zh uONeaIdzO}Uy LEG
Ses" 08L' 408’ Of «oe LTS LL KOS SSH NH «166 HES EOS 6BE 09 GIE —_UOISHoyardui0g
surpeay.
Tes" pel LOB’ ST TL ELT BOOT: 66—sTO'S:sST'® 89H «OTOL BSB «TES LOE Ste LSE" Suyuoseoy
onewipiry
€e8' LL" sk) ths th ST 68S O'S, B'S UHZT ZHI 96°01 Lov’ BLb Ley soroqeuy jequo,,
10 Wd TO td 0 10 Wd 0 10 Id 9% 10 la tO 10 Ta isaigng
AMTIQRIDY sisounyy MONS UOTEIASp plepuels uRay, y9a1100 uoModoig

Apmg Sunenby Areurunjarg ut ajduresqng ¢ 1g 103 sisaiqng Jo sonsneig aandiosaq 6-V 21921,

58



tables, which were not identical to the preliminary conversion tables. Minor discrepancies in the
conversion tables were expected due to the differences in the samples used for the preliminary and
operational equatings and to the differences in external motivation for testing. The sample of
officer commissioning applicants used in the operational equating was larger, took the test to get
in the Air Force instead of experimentally, and took the test under the actual MEPS
administration conditions instead of a large group administration, and thus equatings developed
on this sample were preferable. However, it was important to determine if the introduction of the
operational tables would cause significant changes in qualification rates for officer positions.
Qualification is determined by minimum cut-off values on some or all AFOQT composites for
occupational categories such as pilot, navigator, missile, technical and non-line officers depending
on the commissioning source of AFROTC, OTS, or the Airmen Enlisted Commissioning Program
(AECP).

To examine the effects of the operational conversion tables, the various minimum cut-off
values for officer categories and commissioning sources were identified and the raw score
conversions to percentiles for both the preliminary and operational conversion tables were listed
for a range of percentiles about those minima. The two conversion tables were very close except
for the Navigator-Technical composite on Form Q2 at the tenth percentile. ROTC pilot
qualification requires a minimum percentile of 50 on the Pilot composite and a 10 on the
Navigator-Technical composite for applicants without a pilot’s license and requires a minimum
percentile of 25 on the Pilot composite with a 10 on the Navigator-Technical composite for
applicants with a pilot’s license. A distribution of applicants in the operational equating sample

with Pilot Composite scores of 50 through 59 (n=367) showed none with a Navigator-Technical

46



s v 9 0 0 v v s 4 0 s v 9 0 0 Soindiy USpPpIy,
0 9 Il £ 0 4 p ol ¢ 0 T v 6 is 0 QOUDINS [BINED
€ v s € 0 z s g € 0 £ € 9 € 0 SYOOTA payejoy
I 9 8 s 0 0 9 I € 0 1 € ols 0 uoneuLoyuy uoNeIAy
wo s 9 0 0 «6 ¢ s I 618 s 8 0 Suipeoy 21481,
£ 8 L r4 0 9 6 4 £ 0 9 8 v r4 0 sununod xorg
0 a 8 0 0 t 6 6 0 0 0 I L 4 0 uolsuoyarduroD Juownaysuy
6 61 6 € 0 or 81 6 € 0 8 v1 ST € 0 Surpeoy a[205
0 9 v 8 4 0 v s 6 T 0 £ s or 4 Sze TOMS
0 s 6 s I 0 v €1 € 0 0 € I 9 0 uorsuayasduioD jeorueyooyy
9 91 € 0 0 9 ral L 0 0 v 81 € 0 0 ospoymouyy ye
v rai is ¢ 0 9 or 9 € 0 v @ ¢ v 0 ospoyMouyy pom,
c iat 6 0 0 £ tI L I 0 S a L if 0 uonejeidiayuy Beg
¢ a 6 0 0 € 0 if 0 z Il or Z 0 uorsuoyordwioy Surpeoy
9 IT L I 0 € Il 8 € 0 s or 6 I 0 Sujuoseoy snowy
8 + I I 6 8 L I 0 L or 9 z 0 sordopeuy [eqioa,
66" 08- 09> OF 07 66- O08" 09° OF 0% 66> 08" 09° OF 07"

0s'< 09'< ~Oh< O%< -00 O8'< 09< OF< O%< -00 08'<  09< ~0r< O%< -00 ysaqqng

<a) ie) Id

Apnig Supenby Areurunparg soy sonqnowyiq way] Jo UONNGINSI *s B1qQeL

18



Item Discrimination Analysis Results

As in the Preliminary Equating section, biserial correlations are presented in a
frequency distribution with five categories. Distributions of biserial correlations are provided in
Table 16. Table 17 presents the summary statistics for the biserial correlation values for the
sample.

The items on all three test forms, P1, Q1 and Q2, show acceptable biserial correlations.
The frequency distribution of biserial correlations in Table 7 shows that the majority of the
item biserial correlations fall in the .40 to .80 range. The subtest mean biserial correlations in
Table 8 are generally between .50 and .70 with the minimum mean biserial correlation values of
-511, .490. and .523 for Forms P1, Q1, and Q2 respectively. These biserial correlations
indicate that the dichotomous item responses correlate well with the subtest score and
discriminate well among the examinees.

Comparisons of the subtest discrimination indices of P1, Q1, and Q2 show that there
are fluctuations in the frequency distributions of the biserial correlations. Investigation of
items with discrimination greater than .60 showed different results than those found in the
preliminary equating data. Q1 had five more such items than Q2 on Scale Reading and
Instrument Comprehension and three more on Work Knowledge and Table Reading. Q2 had
three more good discriminating items than Q1 on Data Interpretation. The maximum
difference in subtest mean biserial correlation values for any two of the three test forms, P1,
Ql, and Q2, ranged from .016 to .068. Data Interpretation, Mechanical Comprehension, and
Aviation Information had reasonably higher mean biserials on Q1 and Q2 than on P1 and lower
mean biserials on Block Counting than on P1. These results are somewhat similar to those |
found in the preliminary equating data. In comparing Forms Q1 and Q2, the difference in

subtest mean biserial correlations range from .001 to .046.

36



Table C-5. Forms Q1 and Q2 Preliminary Conversion Table for Quantitative Composite

Raw score Raw score
Percentile Percentile Percentile Percentile
Qi Q2 Ql Qo
0-12 1 1 44 38 33
13 1 2 45 41 34
14 2 2 46 43 34
15 2 3 47 43 38
16 3 3 48 45 41
17 3 3 49 48 43
18 3 4 50 48 43
19 4 5 51 52 45
20 5 6 52 52 48
21 6 6 53 54 52
22 8 8 54 57 52
23 8 8 55 59 54
24 9 9 56 61 57
25 10 10 57 64 59
26 ll ll 58 66 61
27 ll ll 59 69 64
28 14 14 60 7 66
29 15 15 61 75 69
30 17 15 62 16 71
31 17 7 63 78 75
32 19 17 64 80 76
33 21 19 65 85 78
34 21 21 66 86 80
35 24 21 67 90 82
36 26 24 68 91 85
37 26 24 69 92 90
38 28 26 70 94 91
39 31 26 71 95 92
40 31 28 72 97 94
4] 33 31 B 98 97
42 34 31 4 99 98
43 34 33 15 99 99

68



Table

_
SOMIADN

11
12
13
14
15
16
17
18
19
20
21

TABLE OF CONTENTS (CONCLUDED)

REFERENCES .0o.o.o..csccsccccseesessssesesseveseeneseeeseveneseeessecuesesesissessasasscseasecesensesesseesesseveee 48
APPENDIX A: Results of Analyses for AFA, ROTC, and BMTS Subsamples for
Preliminary Equating Study ..........0......ccccceccescessessesesseeseeseesecsesseeneeeesssesseetsuseeeeeeeesseesess 49
APPENDIX B: Common Item Analyses Results ...........c0cccccsssecseceeseesssessesessesceseesecsesseees 59
APPENDIX C: Preliminary Conversion Tables for AFOQT Forms Q1 and Q2 62
APPENDIX D: Operational Conversion Tables for AFOQT Forms Q] and Q2.............. 68
LIST OF TABLES
Page
Description of AFOQT Forms Q Subtests And Composition of Aptitude
Composites oo... eccecsceccecssesseessessesseesnesrssnecsessessnessecssesssessecsueessssessussavsresecsresseeeeete 3
Distribution of Item Difficulties for Forms Q1 and Q2 at Test Construction Stage .. 8
Distribution of Biserial Correlations for Forms Q1 and Q2 at Test Construction
Stage... eee ee cece ec eee ees ceneeuesesencevceesessansassesassecenesseeseeuesessuseteseeiseeseesetssesnteceenee 9
Demographic Percentages for Total, AFA, ROTC, and BMTS samples from
Preliminary Equating Study 12
Distribution of Item Difficulties for Preliminary Equating Stud: 18
Summary Statistics of Item Difficulties for Preliminary Equating Study. 19
Distribution of Biserial Correlations for Preliminary Equating Study ..... 21
Summary Statistics of Biserial Correlations for Preliminary Equating Study . 22
Descriptive Statistics of Subtests for Preliminary Equating Study. 24
Intercorrelations of Subtests for Preliminary Equating Study....... 27
Descriptive Statistics of Composites for Preliminary Equating Stud: 28
Intercorrelations of Composites for Preliminary Equating Study . 29
Demographic Percentages for Total Sample for Operational Equating Study 31
Distribution of Item Difficulties for Operational Equating Study 33
Summary Statistics of Item Difficulties for Operational Equating Study 34
Distribution of Biserial Correlations for Operational Equating Study...... 37
Summary Statistics of Biserial Correlations for Operational Equating Study. 38
Descriptive Statistics of Subtests for Operational Equating Study 41
Intercorrelations of Subtests for Operational Equating Study....... 42
Descriptive Statistics of Composites for Operational Equating Study. . 44
Intercorrelations of Composites for Operational Equating Study.....0.......0.0ccccee 45

iv



Table D-2. Forms Q1 and Q2 Operational Conversion Table for Navigator-Technical Composite

Raw Raw Raw Raw Raw
score Percentile score Percentile score Percentile score Percentile score _ Percentile
Qi Q Qi Q2 Ql Q2z Qi Q Qi Q2
0-62 1 1 1033. 12)—«O 144 38 «(36 185 73 69 226 96 «95
63 2 1 104«12)—«Oo1 145 38 «36 186 73 70 227 96) =95
64 2 1 105 13° «12 146 39 «(37 187 740 71 228 96 896
65 2 1 106 «130 «12 147 40 38 188 74 «472 229 96 96
66 2 2 107 14° 12 1448 41 «(38 189 75 73 230 96 96
67 2 2 108 «1413 149 42 «39 190 76 73 231 #97 96
68 2 2 109 615) 13 150 43 40 191 77 74 232 «97:96
69 2 2 no 615) 614 151 43) 41 192 78 74 233 «97:97
70 2 2 lloo15) (14 152. 4342 193 79 75 234 «97 (987
7 2 2 12, «1615 153. 44 (43 194 79 76 235° (97: «(97
72 2 2 113° «16-15 154 45043 195 80 77 236 97 «(97
73 2 2 114° «17—)— «15 155 46 43 196 81 78 237 98 «(987
14 3 2 15 17 16 156 47° 44 197 81 79 238 «498 ~=—(98
75 3 2 116 «18 ~ «(16 157. 4945 198 82 79 239 98 «=—(498
76 3 2 u7 18 «#17 158 5045 199 83 80 240 98 898
77 3 3 18 «18 ~=«=«17 159 51 46 200 83 «681 241 99 98
78 3 3 119° «1918 160 52 «47 201 84 «81 242, «99 «99
719 4 3 120 20) «18 161 52 48 202. 85 82 243 99 «99
80 4 3 121 21) «(18 162 53 49 203-86 = 83 244 99 99
81 4 3 122. 21) «#19 163 54 50 204 86) 83 245° 99 99
82 4 3 123 22 «20 164 55 51 205 87 84 246 #99 99
83 5 4 124 23 «20 165 56 52 206 «487 85 247 99 99
84 5 4 125 23 21 166 «5752 207 88 = 86 248 #99 99
85 5 4 126 24 21 167 58 53 208 88 86 249 99 99
86 5 5 127, 25022 168 59 54 209 88 87 250 99 99
87 6 5 128 25 «23 169 60 55 210 89 87 251 99 99
88 6 5 129 26 «23 170 61 56 211 89) 88 252 99 = 99
89 7 5 130 27) 24 171 6257 212 90 88 253 99 = 99
90 7 6 131 2825 172) 63) (58 213 90 89 254 99° 99
91 7 6 132. 29-26 173, 6359 214 91 89 255 99 99
92 8 7 133,29 (27 174 64 «(60 215 91 90 256 99 99
93 8 7 13430. 28 175 6561 216 92 90 257 99 999
94 8 7 135 30-29 17%6 65 62 217° 9391 258 99 99
95 9 8 136 «3129 177 6563 218 93 91 259 99 99
96 9 8 137, 32-30 178 66 «63 219 94 92 260 99 99
97 9 8 138 33 «30 179 67 «(64 220 94 «93 261 99 99
98 9 9 139 34 «(31 180 68 8665 221 94 «93 262 99 99
99 10 9 140 35 «(32 181 69 = 65 222 95 94 263 99 99
100 11 9 141 36 «(33 182 70 66 223 95 «(94 264 «99 99
101 Il 10 142 36 (34 183 71 67 224 95 94 265 99 99
102,12 «10 143 3735 1847268 225. 95 95

71



Table D-1. Forms Q1 and Q2 Operational Conversion Table for Pilot Composite

Raw Raw Raw Raw
score Percentile score Percentile score Percentile score Percentile
Qi Q2 Qi Q2 Qi Qe Qi 2

0-43 1 1 84 17 14 125 55 54 166 94 93
44 2 1 85 7 15 126 55 55 167 94 94
45 2 1 86 18 17 127 56 56 168 95 94
46 2 1 87 19 17 128 57 37 169 95 95
47 2 1 88 20 18 129 58 58 170 95 95
48 2 2 89 20 19 130 60 60 171 96 96
49 3 2 90 20 20 131 61 61 172 96 96
50 3 2 91 21 20 132 62 62 173 96 96
51 3 2 92 22 20 133 63 63 174 97 96
52 3 2 93 23 21 134 63 63 175 97 97
53 3 3 94 24 22 135 64 64 176 97 97
54 3 3 95 24 23 136 65 65 177 97 97
55 3 3 96 25 24 137 66 66 178 97 97
56 4 3 97 26 24 138 67 67 179 98 97
57 4 3 98 27 25 139 69 69 180 98 97
58 4 3 99 28 26 140 70 70 181 98 98
59 4 3 100 28 28 141 71 7 182 98 98
60 5 4 101 29 28 142 3 73 183 98 98
61 5 4 102 30 29 143 14 14 184 98 98
62 6 4 103 32 30 144 75 74 185 98 98
63 6 5 104 33 31 145 16 75 186 99 98 i.
64 6 5 105 34 32 146 77 16 187 99 98
65 6 6 106 35 33 147 78 77 188 99 99
66 7 6 107 36 34 148 79 78 189 99 99
67 7 6 108 37 35 149 80 79 190 99 99
68 7 6 109 38 36 150 81 80 - 191 99 99
69 8 7 110 39 37 151 82 81 192 99 99
70 8 7 M1 41 38 152 83 82 193 99 99
7 8 7 112 42 39 153 84 83 194 99 99
72 10 8 113 42 41 154 84 84 195 99 99
73 10 8 114 43 42 155 84 84 196 99 99
14 ll 9 1s 44 42 156 85 85 197 99 99
75 8] 10 116 45 43 157 86 86 198 99 99
76 12 10 117 46 44 158 86 86 199 99 99
77 12 ll 118 47 45 159 87 87 200 99 99
78 13 8 119 48 46 160 88 88 201 99 99
79 13 12 120 50 47 161 89 89 202 99 99
80 13 12 121 51 50 162 90 89 203 99 ' 99
81 14 13 122 52 51 163 91 90 204 99 99
82 15 13 123 53 52 164 92 91 205 99 99
83 16 13 124 54 33 165 93 92

70



Table D-4. Forms Q1 and Q2 Operational Conversion Table for Verbal Composite

Raw score Percentile Percentile Raw score Percentile Percentile
Qi Qe Qi Qa
0-15 1 1 46 44 41
16 2 1 47 46 44
17 3 2 48 48 46
18 3 3 49 50 48
19 4 3 50 50 50
20 5 4 51 53 353
21 6 5 52 55 55
22 7 6 53 57 57
23 8 7 54 60 60
24 9 8 55 62 62
25 10 9 56 64 64
26 i 10 57 , 67 67
27 12 aot 58 72 69
28 13 12 59 74 74
29 14 13 60 77 17
30 15 14 61 78 78
31 7 15 62 81 81
32 18 15 63 84 84
33 19 7 64 86 86
34 21 18 65 87 87
35 23 19 66 90 90
36 24 23 67 92 92
37 26 24 68 93 93
38 27 26 69 96 97
39 30 27 70 97 98
40 32 30 mt 98 99
41 33 32 72 99 99
42 36 33 73 99 99
43 38 36 4 99 99
44 40 38 75 99 99
45 41 40

73



The skew and kurtosis values for the composites are quite similar across the three test
forms. The skew values range from -.38 to -.70; kurtosis values range from -.36 to -.97. These
skew and kurtosis values indicate the composite score distributions are relatively symmetric and
tend toward normality.

The composite intercorrelation matrix for Forms P1, Q1 and Q2 is presented in Table 12.
The data for all three forms are presented in one table to facilitate comparisons of composite
correlations across forms. The composite intercorrelations are among the composite within one
form, not among composites of different forms. The maximum correlation among composites is
_96 and results from the correlation between the Pilot and Navigator-Technical composites on all
three forms. The minimum correlation is .75 and occurs between the Verbal and Pilot composites
and Verbal and Navigator-Technical composites on Form Q1. The composite intercorrelations are
almost identical across test forms; the maximum difference between any of the three composite
correlations in a triads is .03. Thus, there is a high degree of similarity among the composite
intercorrelation matrices across the three test forms.

Table 12. Intercorrelations of Composites for Preliminary Equating Study

Test Academic
Composite form Nay-tech Aptitude Verbal Quantitative
Pilot Pl 96 85 76 84
Ql 96 85 75 85
Q 96 86 17 86
Navigator-Technical Pl 90 77 .93
Qi -90 75 93
Qa 91 77 94
Academic Aptitude Pl 94 95
Ql 93 94
Q 93 95
Verbal Pl 78
Qi 16
Q 79
29



values were calculated. Intercorrelation matrices are provided for the subtests and for the
composites.

In addition to these subtest analyses using all test items, subtest statistics were calculated
using only the common items for each subtest. These analyses check on the assumption of
randomly equivalent samples.

Equating Analysis

Equating enables two forms of a test that are intended to be parallel, which are never
precisely equivalent in level and range of difficulty, to be rendered interchangeable by converting
the score units of one test to the score units of another. Statistical equating methods establish a
relationship between raw scores on two test forms so that the score on one form can be used to
express the score on the other form. In the current study, composite scores of Forms Q] and Q2
were linked to the normative group using linear and equipercentile equating to Forms P scores
(see Angoff, 1971 for further explanation of equating).

In linear equating, two raw scores are equated if their z-score values are equivalent,
resulting in a smooth straight line. In equipercentile equating, two raw scores are equated if their
percentile ranks are equivalent. Because equipercentile equating may result in irregular equating
curves, three types of polynomial smoothing (linear, quadratic and cubic) are used, resulting in
four possible equatings. The linear and equipercentile equating methods coincide when the score
distributions are the same. In choosing from among the four possible equatings, the z-score linear
equating and three polynomial smoothings, the sample descriptive statistics and size are among
the characteristics to be considered. When the means, standard deviations, skew, and kurtosis of

the two randomly equivalent equating samples are nearly identical on both tests being equated, the

15



v oT 0 0 0 at 0 0 € Tt OT 0 0 somn3tq USpPIH
0 1 8 1 0 1 6 ) 0 1 a € 0 SOUdIDS [BIOUSH
0 a 0 0 0 WF 0 0 0 6 9 0 0 SHOO]_ PAIIOY
¢ ors if 0 z ol 8 0 0 z 8 6 0 0 uoneuoyuy UoReIAY
1 oT ak z 0 sT ool Ok z 0 st out ok I 0 Surpeoy 2192.1,
I 6 or 0 0 1 a 9 if 0 v 1 € 0 0 Bununod yoorg
6 oO oT 0 0 6 ir 0 0 0 L () 0 0 worsusyorduiog quownysuy
s o st 0 0 € [rs | 0 if sl st 9 0 Surpeoy o1e9g
0 a) % 0 0 L 8 s 0 0 9 lo 0 ze [O19
I oO 9 € 0 0 ()) a) rn) 0 0 6 8 € 0 uo}suayosduro; peorueyooy,
619 0 0 0) sl 6 if 0 0 ak I 0 0 ospaymouy WEA,
z vr 8 if 0 s 1 of 0 I 9 sl € I 0 o8pa]mouy P10
s sts 0 0 € us 0 0 0 sl 8 z 0 uonejerdioyuy eye
? sl 9 0 0 z 1 9 I 0 s a1 9 I 0 uorsuayorduioy duypeoy
s 81 z 0 0 € oz 7% 0 0 8 vl 0 0 Suuosesy onoupuy
€ wm 8 0 0 z el 6 I 0 € el 6 0 0) sordojeuy [eqion,
66- 08 09> OF- 0% 66- 08- 09° OF 0% 66- 08 09> OF- 0%

08< 09< O”< 0%< -00° 08'< _09'< Ob< 0%< 00° Os< 09< Ob< 0%< _-00° ysonqng

£40) ite) Td

Apmg Sunenbg Areurunarg Joy suonEja110_D [east Jo uoNNgmNsI *L 21921

21



item difficulty values for the total sample. Item difficulties of the subsamples across test forms are
presented in Appendix A.

As is evident from Table 5, the majority of items in P1 have difficulties ranging from .20 to
80. Electrical Maze is the only subtest that includes items with difficulties below .20. Thirteen of
the subtests have at least one item with a difficulty above .80. Approximately half of the items in
the Table Reading subtest have item difficulties above .80, suggesting that Table Reading is a
relatively easy subtest. Table 6 shows that all sixteen subtests have mean item difficulties between
.40 and .60.

Form Q1 subtests have similar item difficulty characteristics as subtests in Form P1.
Again, item difficulties tend to range from .20 to .80. Two subtests, Electrical Maze and Table
Reading have items with item difficulties below .20. Thirteen subtests have at least one item with
a difficulty value above .80. Table Reading is a relatively easy subtest; half of the items have
difficulty values above .80. Fifteen subtests have a mean level of item difficulty between .40 and
.60.

Item difficulties for test Form Q2 are predominantly in the .20 to .80 range. Three
subtests, Verbal Analogies, Mechanical Comprehension and Electrical Maze, include items with
item difficulties below .20. Twelve subtests include items with difficulty value greater than .80. As
in Pl and Q2, the majority of items form the Table Reading subtests have difficulties above .80.

Fifteen subtests have mean levels of item difficulty between .40 and .60.

17



Equating Analysis Results

Four possible equatings, the z-score linear, linear smoothed equipercentile, quadratic
smoothed equipercentile and cubic smoothed equipercentile, were developed and compared for
each composite on Q1 and Q2. The lack of nearly identical moments (skew and kurtosis) for the
score distributions ruled out the z-score linear equating method and given that sample sizes were
large enough to ensure stability, the cubic smoothing equipercentile equatings were selected for
each of the five composites on each test form. Using this equipercentile equating with cubic
smoothing, preliminary conversion tables were developed and are presented in Appendix C.

OPERATIONAL EQUATING STUDY
Subjects

Subject samples for the operational equating study were actual examinees taking the
AFOQT Forms P1, Q1, and Q2 for purposes of officer selection decisions, either into ROTC or
into Air Force commissioning for those with college degrees. Their operational scores were
provided by the preliminary conversion tables. These examinees were tested over a period from
September of 1994 through June of 1995. On July 1, 1995, Forms Q1 and Q2 were pulled from
the field while new equatings were accomplished using applicant scores.

Demographic information is presented for the total sample in Table 13. Based on the most
frequent response within a demographic categorization, subjects were predominately male,
Caucasian, with twelve or sixteen years of education and a high school degree or Bachelor’s

degree as the highest educational credential earned.

30



07 TI a7 90 £0 $0 00 00 $0 £0 90 eT SuIssIA
00 00 00 00 c0 00 00 00 00 00 v0 00 97810}90q,
00 00 00 00 00 00 00 00 00 00 00 00 SIO}SEIN
St ve 1 0c 61 ve 00 00 00 sl sT £@ sr0payorg
07 Te LY 89 89 Sb 61 00 v1 er (a4 se soyeroossy pourea
v6 9% 168 L:06 £06 £16 186 0001 = 786 16 Te6 96 looyos ysty — eordop sous
$0 00 OT £0 00 ra) 00 00 00 ra) 00 v0 BuIssI
00 00 00 00 00 00 00 00 00 00 00 00 1?
00 00 00 00 00 x) 00 00 $0 00 00 10 07
00 g0 00 00 00 70 00 00 00 00 v0 10 61
00 00 $0 80 50 sO 00 00 00 v0 70 LAU) 81
0c $0 OT $0 $0 Lt ¢0 x4 $0 60 OT TI dt
ST Le 1? Or es ss vL 6L 16 ev “sg “s oT
4 L@ 1? [a74 Lee ec 6! 98h ss 8 6r eS £97 Tee SI
98 STI Ost 869 £89 €CL Sep Le Loe Llp OLr 6 6b tI
“Lt oor 61l $0 £0 70 00 00 0 Ly cy Ve €1 uoneonpa
LL seo £99 00 00 00 00 00 00 99L Sst oor ral JO SID,
00 00 OT $0 70 ¢0 ¢0 $0 $0 v0 ray) 90 Surssiy,
SOL Lge LLL v08 O's vs 088 oL8 vss es 9 Es 14:3 ueiseone)
OL 69 “s BP ov s9 09 ge v9 ss sr ra) oruedsiy
Ver “Lit Oe! 96 vs ss we ge oF 88 v9 el pr
sT 91 OT sy v9 oF €@ SP 8T ee oF 4 ueisy
OT TI oT £0 00 TI 00 s0 vl v0 v0 eT UeIpuy UROLOUY aoey
$0 00 00 80 70 (a0) s0 00 60 90 10 v0 SuIssIW
Lsi V6 1st ol S91 “st cy vL v8 oF st Ost Spur]
8:08 608 618 vis ces Urs vs6 x49 6:06 8'r8 Lbs Ls ore Jopuen
L61= 88I=U T6I=U 6E=U ZI p=U HIPS STZ=UZOZ=U_—CLTZ=U_—SCT SS OI8=! eegsu
(40) 10 Id <0) 10 Id (<0) 10 Id <0) 10 Id
gduies STN ‘ojdues SLOW oidues Vay opdues [210

Apmg Sunenby Areurunyoig wou sojdures SLING Pur DLOU ‘VAV ‘TROL 10F sodeusoI0g s1ydesow9g “p 3192,

12



Table D-3. Forms QI and Q2 Operational Conversion Table for Academic Aptitude Composite

Raw Raw Raw
score Percentile Percentile Score Percentile Percentile score Percentile Percentile
Qi Q Qi @ Qa Qu
0-28 1 1 69 18 16 110 67 62
29 2 1 70 19 7 lll 68 63
30 2 1 71 20 18 112 69 65
31 2 1 72 21 18 113 70 67
32 2 2 73 21 19 114 7 68
33 2 2 74 22 20 115 72 69
34 2 2 75 23 21 116 75 70
35 3 2 76 24 21 117 16 71
36 3 2 77 25 22 118 76 72
37 3 2 3 26 23 119 a3 75
38 3 2 79 27 24 120° 79 16
39 3 3 80 28 25 121 80 3
40 4 3 81 28 26 122 81 79
4) 4 3 82 29 27 123 82 80
42 5 3 83 31 28 124 83 81
43 5 3 84 33 28 125 84 82
44 5 4 85 34 29 126 85 83
45 5 4 86 35 31 127 86 84
46 6 5 87 36 33 128 87 85
47 6 5 88 37 34 129 88 86
48 6 Ss 89 38 35 130 89 87.
49 7 5 90 38 36 131 90 88
50 7 6 91 40 37 132 91 90
51 8 6 92 41 38 133 92 91
52 9 7 93 43 38 134 93 92
53 9 7 94 44 40 135 93 93
- 54 9 8 95 45 41 136 94 93
55 9 9 96 47. 43 137 95 94
56 10 9 97 49 44 138 95 95
57 10 9 98 50 45 139 96 95
58 i 9 99 51 47 140 96 96
59 ll 10 100 52 49 141 97 97
60 12 10 101 53 50 142 97 97
61 13 ll 102 54 SI 143 98 98
62 14 ll 103 54 52 144 98 98
63 15 12 104 57 53 145 99 99
64 16 13 105 59 34 146 99 99
65 16 4 106 61 34 147 99 99
66 16 15 107 62 57 148 99 99
67 7 16 108 63 59 149 99 99
68 18 16 109 65 61 150 99 99

72



LoL’ fhl’ 869 88-19 6H OE-  BZ- CH CSO ~«9GT. «STC «COOL OCOl STIL USL 9tL OBL” SoIndT{ UOPPIH

I9s 79S" 68S 10> OS= = 6S TZ> 0" €8Z OT EST GOEL POET GOI 90L' LEL’ 989° souatog [RIaUaD

99 6IL’ 10L 20 OT’ 6 LS = 99> = BI= GN'T_—«OG'T-«-BB'Z_sOLH'OL «SHOL S96 StL’ ZL’ ZL9" SyDOTg parejoy

99’ 08s’ OW GE OT LS SL OTETM BT LBZ OORT «HHT «BTST SEH CO'EL SOL bb SIL’ uoneunoyuy

uonenAy

“gs I@eSET TOI- SUS PhS ~Z9 SLZE ESTE EDIE 698 BEB’ IPs Surpeoy e1qeL

189 614’ 7@8L’  €€>  6T- 9t 09 Off FEE IDE SOI EFI OO'ST €%L’ PL OBL Sununoy ypo1g,

198° 618° 878" Oz 6L> Is- Ish LOE OFF OPT I8PL 96'ET SEL’ 09L' OL’ —UoIsUaYsdwog

quoumnnsuy

89 BT = HB CTI- @S> LOS BTS HE'S. sCLBTIE SOIE 8h'6~ EB 8zZ8 OLE Surpeoy e1e05

peo’ sos’ TEL) St- TO BE" HTT 09" KE OHT MLE «OSG «OLB 998 6H sL" Ze [ROIN81F

Tel’ 89" ZL’ ph OT bh OH TH SO OLE «OWE «SLE «“OEZI «BLT «SOIT L279 IS9°  T6s° _uorsuayasdwiog

. Jeorueysoyy

“es ceo” £99 IGT «(ONE «ELT LETH Ebr ETIH SBT. ET Phe «IVES STU UW C6 CLG’ STS aBparMouZ PE

199° BSL’ IL) 90° $6 OES HL LB= OTE «SOE SEE 99'RT LEST BS'6T 262’ 918 Sze aBpajaouy piom

cs" ps9" OS" BST 66’ PO EOI- 68 «LO EL'T CITE «ETE —OSOZ BEI OSI Sse zs 908° _ UONeIardiaUy BEG

Ove Isl LtL St tS STH SL LE OSE HHE «BLE «CEO PL'ST LO'ST SEB 89L’  9bL" votsuoqeudurao

UIPeay

O9L"  86L' 608 tS- Oh ST- «89> «19 OL TEE LOH (GE GB'OZ SH6I 90°07 IL8 B08 FES" Suyuoseoy

onounpny

Los’ 685" peo" 6 9E- _—BETM ETE IL BT OHT_—CIS'OI CLOT SESE SEB Os 9ze —— SaNBopeury TeqroA

[OO CC SO <0 COC GU <0 CON © CO (OC GH 20 EN CON O70 ET) Td wsaqqng
AMqeray SISOLN MAS uOHeIAap prepue)s ues yoou09 uonodorg

Apu Sunenbg Areurunyorg uy ofduresqng y.ty 40y sjsaiqng Jo sonsneig aanduosag "2-y 219421,

56



AFOOT tests have a certain proportion of their items, usually near one-half, that are taken from
the previous version, some of which were taken from the version previous to that, and so on.
Thus, a subset of the items remains consistent until they are cycled out.
Item Selection

In selecting items for use, comparability was sought between Form P1 and Forms Q in
terms of the distributions and mean estimated (or expected) item difficulty for each subtest, the
distribution of item content and style, and difficulty associated with the item’s position in the
subtest. In addition, all item biserial correlations were desired to be positive and high (above .40)
for the correct alternative and negative for incorrect alternatives. Both common items and new
items were selected to maximize these desired characteristics.
New and Common Items

New items were selected for Forms Q from the same experimental item bank that was
used to construct Forms P (Berger, Gupta, Berger, & Skinner, 1988). AFHRL had contracted
with Psychometrics, Inc., to create this item bank from which items were to be selected for two
new parallel tests, Forms P. The Forms P test development effort marked the first time two
equivalent forms of the AFOQT were to be created. After the Forms P test development effort, a
sufficient number of items remained in the pool to create Forms Q. Now however, the item bank
is depleted and cannot be used for subsequent test construction.

In addition to the new items, approximately half of the items on Forms Q subtests came
from Forms P. About half the items in Forms P were also in Form O. Thus, approximately one-
quarter of the items are common to all three forms. The same set of common items are in both

Forms Q1 and Q2.



Table 10. Intercorrelations of Subtests for Preliminary Equating Study

Subtest AR RC DI WK MK MC _EM SR_IC_ BC TR _AI RB GS_ HF
VA Pl 70.76 68 .7%6 72 57 42 60 59 52 46 57 53 69 55
Ql 64 «78 «467 «4.77 67 «57 «40 «585244 4B 4646 6G 4B
Q 65 75 67 74 67 54 40 57 52 46 43 53 48 65 52
AR Pl 67 80 63 82 61 44 75 55 56 58 58 58 68 51
Ql 67 80 62 78 64 45 72 55 53 55 51 54 69 48
Q 71 83 60 81 59 50 .78 56 58 60 53 58 68 52
RC Pl 68 78 69 51 38 59 54 46 49 54 42 67 49
Qi 68 79 68 57 39 58 50 44 45 50 46 69 45
Q2 77° 78 71 58 44 62 53 48 49 59 49 71 47
DI. Pi 60 77 56 45 71 56 56 60 55 52 61 49
Qi 62 77 64 44 73 56 56 58 54 58 67 .50
Q2 67 82 61 51 76 57 57 59 59 59 725A
WK Pl 68 48 33 52 50 42 41 57 41 66 46
Ql 63 58 35 52 50 40 41 53 44 69 42
Q2z 64 54 37 50 47 40 39 56 42 68 43
MK Pl 55° Al 71 59 55) 59) 5754 6953
Qi 63 44 72 58 52 57 52 55 73 52
Q2 55 46 71 57 54 60 58 58 70 56
MC Pl AT 51 58 48 41 63 60 67 51
Ql AT 57 64 46 (38) «62 6269S
Q 49° 51 60 47 36 63 54 69 47
EM Pl 49 52 52 (42 38 46 44 46
Ql 46 48 43 41 36 41 42 42
Qz 52. 48 «53 39 «38 450 46 AS
SR Pl 59 64 64 53 56 56 51
Ql 59 62 63 50 55 61 .50
Q 56 61 61 48 55 59 52
IC Pl 55 AT 61 56 60 .54
Qi 51 46 62 58 59 56
Q2 55 45 (59) 56 59ST
BC Pl 58 42 55) AT 5S
Ql 57 33 53) 4550
Q 57 39 52) ATSB
TR Pl 42. 44° 40 43
Qi 37 44 4442
Q 41 49° 45° (44
Al Pl 50 62 43
Qi A559 40
Qe 49 66 42
RB Pl 54 56
Ql 52 58
Q 56 59
GS Pl 54
Ql 49
Q2 53

27



Table C-1. Forms Qi and Q2 Preliminary Conversion Table for Pilot Composite

Raw Raw Raw Raw
score Percentile score Percentile score Percentile score Percentile
Qi Q@ Qa @ QQ QQ

0-41 1 1 83 15 16 124 48 51 165 94 92
42 2 1 84 16 7 125 50 52 166 94 94
43 2 1 85 17 17 126 51 53 167 95 94
44 2 2 86 17 18 127 52 54 168 96 95
45 2 2 87 7 19 128 53 55 169 96 95
46 2 2 88 18 20 129 54 56 170 97 96
47 2 2 89 19 20 130 55 57 171 97 96
48 2 2 90 20 20 131 56 58 172 97 96
49 3 2 91 20 20 132 57 60 173 97 97
50 3 3 92 20 21 133 58 61 174 98 97
51 3 3 93 21 22 134 60 62 175 98 97
52 3 3 94 22 23 135 61 63 176 98 97
53 3 3 95 23 24 136 62 63 177 98 98
54 3 3 96 24 24 137 63 64 178 99 98
55 3 3 97 24 25 138 63 65 179 99 98
56 4 4 98 25 26 139 64 66 180 99 98
57 4 4 99 26 27 140 65 67 181 99 98
58 4 4 100 27 28 141 66 69 182 99 99
59 4 4 101 27 28 142 67 70 183 99 99
60 5 5 102 28 29 143 69 7 184 99 99
61. 5 5 103 28 30 144 70 73 185 99 99»
62 6 6 104 29 31 145 B 74 186 99 99
63 6 6 105 30 32 146 74 75 187 99 99
64 6 6 106 31 33 147 75 76 188 99 99
65 6 6 107 32 34 148 76 77 189 99 99
66 7 7 108 33 35 149 77 78 190 99 99
67° «7 7 109 34 36 150 78 79 191 99 99
68 7 7 110 35 37 151 79 80 192 99 99
69 8 8 lll 36 38 152 80 81 193 99 99
70 8 8 112 37 39 153 81 82 194 99 99
71 8 8 113 38 41 154 83 83 195 99 99
72 9 9 114 39 42 155 84 84 196 99 99
73 10 10 11s 41 42 156 84 84 197 99 99
74 10 10 116 41 43 157 85 85 198 99 99
75 ll u 7 42 43 158 86 86 199 99 99
76 ll ll 118 42 44 159 86 86 200 99 99
77 12 12 119 43 45 160 88 87 201 99 99
3 12 13 120 44 46 161 89 88 202 99 99
719 13 13 121 45 47 162 90 89 203 99 99
80 13 13 122 46 48 163 91 90 204 99 99
81 13 4 123 47 50 164 93 91 205 99 99
82 4 15

64



Table C-3. Forms QI and Q2 Preliminary Conversion Table for Academic Aptitude Composite

Raw Raw . Raw
score Percentile Percentile score Percentile Percentile score Percentile Percentile
Ql Q Ql Qe Qu Qa

0-24 1 1 67 16 16 109 63 61
25 1 2 68 16 16 110 65 62
26 1 2 69 17 16 1d 67 63
27 1 2 70 18 7 112 68 65
28 1 2 7 18 18 113 69 67
29 2 2 72 19 18 114 70 68
30 2 2 3 20 18 115 7 69
31 2 2 74 21 19 116 72 70
32 2 2 75 21 20 117 16 71
33 2 3 76 22 21 118, 78 72
34 2 3 77 23 21 119 79 75
35 2 3 78 24 22 120 80 76
36 3 3 719 25 23 121 81 78
37 3 3 80 26 24 122 82 719
38 3 3 81 27 25 123 83 81
39 3 4 82 28 26 124 84 82
40 3 4 83 28 27 125 85 83
41 4 4 84 29 28 126 86 84
42 4 5 85 31 28 127 87 85
43 5 5 86 33 29 128 88 86
44 5 5 87 34 31 129 89 87.
45 5 5 88 36 33 130 90 88
46 5 5 89 37 34 131 91 89
47 5 6 90 38 35 132 92 90
48 6 6 91 38 36 133 93 91
49 6 6 92 40 37 134 93 92
50 6 7 93 41 38 135 94 93
51 7 7 94 43 38 136 95 93
52 7 7 95 44 40 137 95 95
53 8 8 96 45 41 138 96 95
54 9 9 97 47 43 139 96 96
55 9 9 98 49 44 140 97 96
56 9 9 99 50 45 141 97 97
57 9 9 100 51 47 142 98 97
58 10 10 101 52 49 143 98 98
59 10 10 102 53 50 144 99 98
60 ll 10 103 54 51 145 99 99
61 ll 11 104 54 52 146 99 99
62 12 i 105 37 53 147 99 99
63 13 12 106 59 54 148 99 99
64 14 13 107 61 57 149 99 99
65 15 4 108 62 59 150 99 99
66 16 15

66



Table C-4. Forms Q1 and Q2 Preliminary Conversion Table for Verbal Composite

Raw score Raw score
Percentile Percentile Percentile Percentile
Qi Q Qi Q2
0-15 1 1 46 41 41
16 1 2 47 44 44
17 2 3 48 46 46
18 3 3 49 48 48
19 3 4 50 50 50
20 4 5 51 53 53
21 5 5 52 55 55
22 6 6 53 37 57
23 7 7 54 60 62
24 8 8 55 62 64
25 9 9 56 67 67
26 10 10 57 69 69
27 ll 11 58 72 72
28 12 lu 59 74 4
29 13 12 60 “a 77
30 14 13 61 78 78
31 15 14 62 81 81
32 17 15 63 84 86
33 18 17 64 86 87
34 19 18 65 87 90
35 21 19 66 90 92°,
36 23 21 67 92 93°
37 24 23 68 93 96
38 26 24 69 96 97
39 27 26 70 97 98
40 30 27 7 98 99
41 32 30 72 99 99
42 33 32 B 99 99
43 36 33 74 99 99
44 38 38 75 99 99
45 40 40

67



Teg’ e18" P18 69 BB 99 SHS LT=  Eb= CODE BRE OE ITOl S96 bOI COL 999° ECL Somndty UOPPIH

v8" S6L’  66L' 9L- OL- > h8- OT LO- OS 9% LIP EL'OT P9'OI 166 6€S OMS TES” SOUDIDS [RIOUdH

LLL) OBL’ 78L’ 9-09 B= OE LO- OVE SHE BEE ELB 606 653 6S LID 065° S¥OOLg PaIeIOY

798" 8h8' 678" COT 96 6 ST wl 0S Lh OHH «CHOI «BIO Ze «ES GOS’ Or woneUoyUy

uonemy

ve- £1 6% BS* 89- Le 1B HL LOR «OF'BZ —OO'RZ SOLZ Sh’ SEL’ 6LL’ Surpeoy 1981

vis’ 718" ss8 es Ol I= LE= 89" 7D OH SOD. OOH «ORT ~ISED ESET 799° 169° 00L" Sununoy ypog,

806° 968" 006" GT'T- OL'I- OL'I- £€- 9E- 9f- OLS EMS I9'S BOI OSI ZIZE LO Seo 609 _MoIsuayasdu0 >

quowmngysuy

Is 9@=  €L= 79 TL E> OLB OTB. OBL «ETLZ «ELE OF'SZ 669 >OL' IS9° Surpesy o1v9$

tpl Wel’ sel’ LE- «Le 60'S’: SLE HOE OLE «9S'R) OHL SCIB'L COLI’ OS8E" OTS" oze [woNoaTy

408 718" 98L’ 88 OT" 08 LOCO LTH OS «OL «946 ©=—O'OT.«SS'6 «OBR COS Lb uOIsuayardu0>

yeoluryooyy

966 906 «1E6’  EL= 8 LO) BLTM EL 769 %99 169 68LI COLI SLI E22’ SIL’ 90L' — e8pojmouy We

698° 768 -L68" 89> LO LB bbs gg PSS 009 O19 9ST 86ST IL'ST £29 ES9° Zy9 —-oBpojMouy pro

76s’ £18" «1b8’ TS OL EL oS ZG T19 OLS GIS 8T9OT 86ST LEST 659° 8b9° 399° uoNRIaIdzoyUT REG

968° 0L8' 788 «(OL «89> LL 6S BH £19 TLS 68S IS9l LBST INST 699 pb9 E19" uorsuayardino;)

UIPBOYy

806° 668° 406° L6- — PO'I- LOI" ~Ph= SZ- OF 879 P79 I€9 E991 BEST HOST 819 S79 zs9 Suyuoseoy

onewypy

£es" 808" v8" ET BE BBL ED Lb ON 06H —TLLT_—GOE'LI_EB'9T 814’ OL’ SOL’ _solBojeuy jequon,

M10 id 10 Wd © 10 Id @ 10 Wd %) 10 Ta 7 10 Id qsaiqng
Agerypay SISOUN yy MOIS UONeIASp prepurys UBoyy yoouoo uoTodo1g

Apnig Sunenbg Areurujorg 403 sysaiqng Jo sonsneyg oanduosag “6 o1qeL

24



AIR FORCE OFFICER QUALIFYING TEST (AFOQT): FORMS Q DEVELOPMENT,
PRELIMINARY EQUATING AND OPERATIONAL EQUATING

INTRODUCTION

The Air Force Officer Qualifying Test (AFOQT) provides aptitude measures for the Air
Force’s officer selection system. The AFOQT is used to select individuals for Officer Training
School, to select Reserve Officer Training Corps (ROTC) cadets for the Professional Officers
Training Course and scholarships, and to select students for Undergraduate Pilot Training and
Undergraduate Navigator Training. Air Force Academy applicants are not required to take the
AFOOT prior to entry.

The forerunners of the AFOQT were the Aviation Cadet Qualifying Examination (ACQE),
a general abilities screening test that was later replaced by the Aviation Cadet Qualifying Test
(ACQT), and the Aircrew Classification Battery (ACB), used to screen and classify potential
aircrew members. A preliminary version of the AFOQT was developed by 1952 and the AFOQT
Form A replaced the previous test batteries by 1955. A more comprehensive account of the
history and development of the AFOQT testing program was authored by Rogers, Roach and
Short, 1986. Since the initial Form A, the AFOQT has been updated periodically to ensure
currency, security and predictive validity. Forms Q are the seventeenth versions of the AFOQT.

The periodic updates of the AFOQT have historically been the responsibility of the Air
Force Human Resources Laboratory (AFHRL) now the Human Resources Directorate of the Air

Force’s Armstrong Laboratory. Updating the AFOQT currently begins



A-l

A-2

A-3

A-4

A-5

A-6

A7
A-8

A-9

B-1

B-2

C-1
C-2

C-4
C-5
D-1
D-2

D-3

D-4
D-5

LIST OF TABLES

Distribution of Item Difficulties for Form P1 for AFA, ROTC, and BMTS
Subsamples in Preliminary Equating Study...
Distribution of Item Difficulties for Form Q1 for AFA, ROTC, and BMTS
Subsamples in Preliminary Equating Study............-:c0:scssesssessseeseesseseseeeeceeneenens
Distribution of Item Difficulties for Form Q2 for AFA, ROTC, and BMTS
Subsamples in Preliminary Equating Study............-..:ssessssesess ese eseseeenteenieeneesnens
Distribution of Biserial Correlations for Form P1 for AFA, ROTC, and BMTS
Subsamples in Preliminary Equating Study.
Distribution of Biserial Correlations for Form Q1 for AFA, ROTC, and BM’
Subsamples in Preliminary Equating Study
Distribution of Biserial Correlations for Form Q2 for AFA, ROTC, and BMTS
Subsamples in Preliminary Equating Study.
Descriptive Statistics of Subtests for AFA Sample in Prelimi
Descriptive Statistics of Subtests for ROTC Sample in Preliminary Equating
Study
Descriptive Statistics of Subtests for BMTS Sample in Preliminary Equating
Study
Subtest and Composite Means and Standard Deviations Comprised of Common
Items in Preliminary Equating Study ...............cscccccscscsseesseeeeeeseeseeeeeseenneeeeeeeecenecnes
Subtest and Composite Means and Standard Deviations Comprised of Common
Items in Operational Equating Study.
Forms Q1 and Q2 Preliminary Conversion Table for Pilot Composite
Forms Q1 and Q2 Preliminary Conversion Table for Navigator-Technical
Composite... csc eecescsecsessseessesseessessseesecsnesneeseesnecsesenseenesnseseessnsareensssesensenseesonsess
Forms Q1 and Q2 Preliminary Conversion Table for Academic Aptitude
Composite ........eeccecceccecsecteeseecsec tec eeeeeneeneneeeneaneeneneens
Forms Q1 and Q2 Preliminary Conversion Table for Verbal Composite
Forms Q1 and Q2 Preliminary Conversion Table for Quantitative Composit
Forms Q1 and Q2 Operational Conversion Table for Pilot Composite
Forms Q1 and Q2 Operational Conversion Table for Navigator-Technical
Composite .........ccscceecseececseessessessessecsseecsecenseneesnetieesnesseesnseenseenseeeesessesenseaceaseneeneees
Forms Q1 and Q2 Operational Conversion Table for Academic Aptitude
Composite
Forms Q1 and Q2 Operational Conversion Table for Verbal Composite
Forms Q1 and Q2 Operational Conversion Table for Quantitative Composite.

38

60

61
63

64

65
66
67
69

70
71

72
2B



Data Analysis

The data analysis procedures for both the 1993 Preliminary Equating Study and the 1995
Operational Equating Study were nearly identical. Therefore, the data analysis section will be
presented only once for the 1993 Preliminary Equating Study, but will serve for the 1995
Operational Equating Study as well. Variations on this data analysis procedure will be noted
where appropriate, however, the major difference is that analyses will be presented for the
subgroups of AFA, ROTC, and BMTS (in Appendix A) so that future equating efforts will have
the opportunity to inform its data collection from previous efforts.

The subtests and composite formation used in these analyses are as defined previously in
Table 1, with two exceptions. After Forms P operational test booklets were printed, two items on
test form P1 were determined to be problematic and were not used in subtest scoring; one item
from Aviation Information and one item from General Science were omitted. Therefore, the
number of items for these two subtests for Form P1 differs by one from those for the

corresponding subtests of Forms Q1 and Q2.

Based on item omitting rates and omit patterns, it was determined that two subtests, Scale
Reading and Table Reading, should be analyzed as speeded subtests. For these two subtests, the
speeded computational formulas for item statistics were used. The remaining subtests were
analyzed as power subtests, even though many have a slight speeded component and would
probably be correctly classified as mixed-model subtests.

Classical Item Analysis
Item level data were computed using true score theory (Gulliksen, 1950) item statistics

such as item difficulties and item discrimination. Item difficulties (p) are defined as the proportion

13



Test Content
The test content, length, subtests, composite composition and testing time of Forms P
remained the same in Forms Q. The AFOQT has 380 items comprising 16 subtests which are
combined to create five composite scores. The subtest names, the number of items in each subtest
and their categorization into the five composites are presented in Table 1. Total testing time,
including administrative procedures, is approximately 270 minutes. A more detailed description of
the subtest content can be found in the AFOQT Forms P Test Manual (Berger, Gupta, Berger, &

Skinner, 1990).

Table 1. Description of AFOQT Forms Q Subtests and Composition of Aptitude Composites

Composites
Subtest Number Testing _Pilot Nav- Acad. Verbal Quant.
of items time Tech Apt.
(minutes)
Verbal Analogies (VA) 25 8 x x x
Arithmetic Reasoning (AR) 25 29 xX xX x
Reading Comprehension (RC) 25 18 x xX
Data Interpretation @) 25 24 x x xX
Word Knowledge (WK) 25 5 x x
Math Knowledge (MK) 25 22 x x x
Mechanical Comprehension (MC) 20 22 x x
Electrical Maze (EM) 20 10 x Xx
Scale Reading (SR) 40 15 x x
Instrument Comprehension (Ic) 20 6 xX
Block Counting (BC) 20 3 x x
Table Reading (TR) 40 7 xX x
Aviation Information (AD 20 8 xX
Rotated Blocks (RB) 15 ) x
General Science (GS) 20 10 x
Hidden Figures (HF) 15 8 x
Total 380 208°

Note. * This testing time is for minutes actually spent on the test items. Total test time including administrative
activities is 270 minutes.

One particularly noteworthy feature of later AFOQT forms is their continuity, which is

maintained by the inclusion of anchor or common items in the test forms. The more recent



16 SLp OPCS 916 OESCTSCSC«COS v6 ~COLSHSSCSLCSC*C«S soindiy USpplH
€8L 907° Sty 91s" 088° S67 690 ois” 788° 79 ozs" ats SOUBIDS [RIOUSH
06s = pws ES vss 7800S les BLZ SHS CLES SOOT PaeIOY,
ois 8% BTS COS Ose 9th OS 98 ETOH CLO uoneurioyuy uoNeTAy
£6" T€@ Les” 909° 986° v6l £6L" 109° bee” 807° S6L 86S" Surpeoy o1qeL,
8%6 86 (TLIO €66 OPE OER ste se LLB BununoD yor
OL Oh ES 61g =—sTIp’ LSS" Sli ERE TUS uolsuayasduo juounnysuy
Lye vie’ 80L 8S" T16 ese" 869° 98° O16 we Ley £9s° Burpesy o[e9§
PEL 460° LLe 99F" 869° (43 6re" vr L99° ool” Bre’ Ise OZeIN [POIIOO|A
bel Sst Orr’ v6r" SPL’ we vis” 10s" 6fL" S67 osr 16¢" uorsuayordurod JeolURYoo|Ay
6l8 WES SUL (GS €68 STS’) OTL, 6S 988 Lys’ 06988" aspoymouy We
868 =—sTEZTM— SLOSS 768 PE 099" bS €6 BLE «9996S 2BpopMouy PIO
78 OL ZENS €s8 ts tS €l6 bee HHL uonejerdiaquy ee
ws" ovr’ 099° ls bls oe" 679" 09s" Crs" L6c" 709" obs” worsuayorduioD Suipeoy
198 69E 989" HL vos Ize’ TSS” 068s SEES Suruoseay nou
8r6 PST ETL 6S 196 Se OLS 946° 807069" LBS SovdojeUy [BqIOA,
xe UW uIP] UeOyy XPYy UI URIPy UvoyAy ‘XPT UIA UeIpPa, «UROL ysajQng
<0)

10

Id

Apmg Sunenby Areururrporg Joy sonNoyyIC] way] Jo sonsnejg ArewuINg “9 9192.

19



AL/HR-TP-1996-0036

UNITED STATES AIR FORCE
ARMSTRONG LABORATORY

AIR FORCE OFFICER QUALIFYING TEST
(AFOQT): FORMS Q DEVELOPMENT,
PRELIMINARY EQUATING AND OPERATIONAL
EQUATING

Theresa M. Glomb

University of Illinois at Urbana-Champaign
Department of Psychology
603 East Daniel
Champaign, IL 61820

James A. Earles

HUMAN RESOURCES DIRECTORATE
MANPOWER AND PERSONNEL RESEARCH DIVISION
7909 Lindbergh Drive
Brooks Air Force Base, TX 78235-5352

July 1997

19970906 139

Approved for public release; distribution is unlimited.

AIR FORCE MATERIEL COMMAND
ARMSTRONG LABORATORY

HUMAN RESOURCES DIRECTORATE
7909 Lindbergh Drive

Brooks Air Force Base, TX 78235-5352



Table 19. Intercorrelations of Subtests for Operational Equating Study

Subtest AR RC DI WK MK MC EM SR_ IC BC TR Al RB GS HF
VA PL 59 68 57 65 51 49 32 44 40 40 26 36 43 55 39
Ql 55) 71607246 S128 4388636 «26—— «3237-5337
Q 56 68 58 68 44 46 31 42 37 37 29 32 36 52 32
AR Pl 58.72 47 70 «54 42 66 4350383751554
Ql 57.75 48 «69 59 Al 66 AAAS AO 85S ATS AL
Q 56.76 45 71 54 42 67 43 51 43 35 46 58 41
RC Pl 59 71 48 41 28 44 «34 38 31 33 34 «5132
Qi 64.72 47 49 29° 46 «35 36) «3234305434
Q 62.70 45 48 31 43 37 36 32 37 30 56 .29
DI Pl 46 58 49 40 64 435144374748
Qi 53 64 58 40 65 46 49 44 38 47 56 44
Q 50 62 58 44 64 46 50 44 40 48 59 41
WK Pl 36 38 «200 320.27) 272133264925
Ql 36 48 21 37 31 27 23 36 28 54 28
Q 34° 46 «230 «3203126 213565423
MK Pi 44°36 «55 35 433726653
a 48 36 540 37383424356
Qu 42 35° Sl 33° 41 35 25 40 55 39
MC Pl AT 46 530 440 275056243
Ql 50 54 61 46 33 56 59 68 49
Q2z 48 48 «57 450 29535567
EM Pl 48 48 46 34 32 45 39 40
Qi 44° «47 43° (340 324440
Q 47 46 «51 (34°33 434140
SR Pl 50 60 54 36 50 43° 47
a 48 57) 563748485
Q 45° 58 54 36 46 47 44
IC Pl 50 36 52 53) 45 AG
Ql 46° 37° 57) 5550S
Qu 50 34 55 50 5044
BC Pl 52.29 53) 3648
Ql 55.27 52 3646
Q 50 30 50 38 49
TR Pl 25 33 22 36
Qi 27° 353034
Qz 27) 31.2532
Al Pl 35 46 28
Ql 38 6.48 «29
Qz 31 49 26
RB Pl 50.52
Ql 47 S4
Q2 45 50
GS Pl 38
Ql 42
Q2 39

42



expected values were obtained with a regression formula which replaced P1 item difficulties with
P1 item difficulties reduced by the difference between the mean estimated item difficulty and the
mean obtained item difficulty. These adjustments ensured that items selected for Forms Q on
these three subtests were approximate to those of Forms P in terms of item difficulty.

Indices of item discrimination, as assessed by the biserial correlation between the item and
subtest total were computed for each item. Items were selected in an effort to approximate the
distribution of item discrimination values of Form P1. Utilizing the same criteria for construction
of previous test forms, items were selected when the correct alternative had a high positive
biserial correlation (above .40) and all incorrect alternatives had negative biserial correlations.
Subject Matter

Items from eight of the sixteen subtests are able to be classified according to content
categories. The distribution of items within these content categories was to remain consistent
across test forms. These content categories were initially constructed in an attempt to match the
content of Forms P to that of Forms O. These same classification categories were used for the
development of Forms Q with one exception; the Arithmetic Reasoning subtest content categories
were modified from the original categories to provide a more empirical framework. Most subtests
were categorized on the basis of thematic concerns, others were categorized based on structural
or graphic considerations. Content classification strategies were used for the Verbal Analogies,
Arithmetic Reasoning, Reading Comprehension, Word Knowledge, Math Knowledge, Scale
Reading, Aviation Information and General Science subtests. The remaining subtests were not

content classified, generally because all items were essentially the same in content.



Item difficulty and discrimination

The goal of item selection in constructing Forms Q was to match as closely as possible the
item difficulties of Form P1, while maintaining the ability to discriminate well between differing
levels of ability. The classical item analyses statistics of item difficulty and item discrimination
were used to make item selection decisions. For selection of the common items, item statistics
from operational use of Forms O and P were used. For selection of the new items, item statistics
were obtained from an administration of experimental items from the test bank to a sample that
included airmen basic trainees, Reserve Officer Training Corps (ROTC) cadets, and cadets
attending Officer Training School (OTS).

The information about items contained in the test bank was based on the experimental test
administration to airmen and cadets. The problems of comparing these item statistics from airmen
and cadets with those of officer candidates was addressed during the Forms P test development.
Multiple regression analyses were conducted to derive weights that could be used to estimate
difficulties that would be obtained if items were administered to actual officer candidates. These
procedures are discussed in Steuck, Watson, and Skinner (1988).

In development of Forms Q, these same item difficulty estimating techniques were also
applied with some exceptions. For all but three subtests, comparisons between the estimated
difficulty values for new items selected for Forms P and actual item difficulties obtained during
operational use of Forms P produced differences of less than .10 and allowed for the use of item
difficulties for Forms P without adjustment. Differences larger than .10 were found on the Data
Interpretation, Instrument Comprehension, and Block Counting subtests, for which additional

computational adjustments were made to arrive at the expected value. These item difficulty



0 oF T 0 € orf 0 0 T or Zz T T sondij USPPIH
0 € st 7% 0 0 L rf 0 0 0 a 8 0 souatog [eIoUeH
0 9 8 I 0 0 or 6s 0 0 0 L L I 0 syoorg pareioy
0 I or 8 if if 6 or 0 0 if 8 8 € 0 uoneunoyuy UOneIAY
Wo om ¢ 0 4 rn) I 0 6 or outs € Surpesy Oe.
I WoL if 0 if 6 L € 0 0 s wt 0 BuNUNoD Pol
z or o£ I 0 9 oO + 0 0 L oro 0 0 uorsuayordurop quoumnasuy
0 or om 0 v on) on 0 z 6 cl tt 7% Burpeoy 2[29g
0 s 6 s if 0 8 L s 0 0 9 8 € € azeyl [RO1NI8[g,
0 % lg 0 I 8 8 € 0 I 9 Uo7@ 0 worsuayardurog jRoLUEYyooyAy
if () rn) 0 6 Ws 0 0 € 6 s 8 0 aBpojmouy WEN,
0 ot 6 4 0 I 6 el 7 0 0 ’ or I adpo[ Mou PIOA\
0 L ut 0 I st ok z 0 0 € 1 8 0 uonejerdsoyuy eed
z 8 €I z 0 € oO ot I 0 if 9 a 9 0 uorsuayasduroD Surpeoy
I 8 .) 0 (4 oof 0 0 s L 8 € z Suuoseoy onouNpITY
0 6 1 0 0 0 8 tz I 0 I ok I sordoeuy [2QIoA,
66- 08 09> OP OC 66- 08> 09 OF 0% 66- 08 09- OF OC

Os< 09< Or< O%< -00° 08'< 09'< Or< O%'< __-00" O8< 09< OF< O%< -00° ysanqng

SING LOW vaV

55

Apmg Sunenby Areurunorg ur safduresqng SL puke DLOU ‘WAV 10J ZO UlOY 30y suONEJaL0_D eLasIg Jo UONINGINSIq °9-Y S192.L



L8L Loe 989° £r9" &tL 80s" 199° £9" T8L 98s" £l9° tL" somndtj UOpPIH
£89" 96E° ces” Bes” seo 900" ths 8b” SLO CLT 009° vss” SOUDIDS [BIOUDH
Cth” Les” ceo 0eo" OSL’ Bss” 879" bro OPL LOS Iso” 8b9" SIDO[q PION
£16" ser Leo" 0%" 18" Sgr 909" 1e9 r6L 8se soo 06s" uoTeUNOJU] UOHRIAY
788" €te" 069° Oro" 148° S83 669° 969° oss" 097 LiL 189° Suipeoy a1qeL,
8SL° ost” L8s° 09" 88L" Bre co 79 658" Sov" Ly tLe SupunoD xorg
616" 18h £69" 69" 818° Les’ sil SOL’ £18 Tes” 089° 89° uorsusyorduiod juounnsuy
Ish" See” ses” 9S” 6EL" ore SLs’ sss’ OSL’ Lst sos” Tis’ Surpeoy 21295
199° 6te sos” €7s" 799" 8sv ces" 06r" 8Lo° Lee 8¢s” sis” aze TPO1}99Tq
ors" 88e° L6S° 18¢° 6IL" Sse £09" £65" cOL” 10€" Irs” Les” uosusyorduroD Jeorueyooyy
ses" 66S" 669" £69" vs" bes” SLO” tLe css rata £7 689" odpojmouy WRAL
T8L" 897 £19" s6s 008" STe Leo" 19" 108" Ie L89° 649" odpojMouyy plo,
TIL bse 69S" £99" IIL cor 79s" 99° $69 Ole 61s” sig” uonejaidiowuy eyeq
88L° ore” £9S" 89° LBL Lee bls” tls’ 6sL" 6cE sss” sss” uolsusyorduroD Surpeoy
L8L or" 799" 9b 08." 9th £9" Ley” 008° ley s99° 99" Suruoseoy onounpiy
8bL oIr 19s" brs” 169° 9th Iss” 19S" SLO" 90¢" ss” ots” sordopeuy [eqion,
xe uA UvIpI _UeOT xe uA UBIp, Uva xe UN UeIpaW UL, ysaIqng

(20)

TO

Id

Apnyg Bunenby jeuonesedg 10y suoneyas05 jeHostg Jo sonsneig Aeuuing “21 IQR,

38



subtests on Form P1 and the Block Counting and Aviation Information subtests on Form Q1. The
subtest intercorrelations show similar patterns across the three forms. The maximal difference
between any of the three subtest correlations in the 120 triads is greater than .10 in only four
cases; in these instances the correlation is either .10 or .11. Thus, there is a high degree of
similarity among the correlation matrices across the three test forms.

The means of the common items on the subtests for Forms Q1 and Q2 are generally
similar to one another and to those of Form P1. Table 1 in Appendix B presents the common item
subtest means and standard deviations for forms P1, Q1 and Q2. The subtest means on Forms Q1
and Q2 tend to be slightly higher than the corresponding subtest means of Form P1, however
twenty-seven out of thirty-two means are within one-tenth of a standard deviation of the P1
subtest means. The discrepancies occur for one of the comparisons between P1 and Q1 and for
four of the comparisons between P1 and Q2. In general, the common item means across test
forms are approximately equivalent.

Composite Analysis Results

The composite level statistics for Forms P1, Q1 and Q2 are reported for the total sample
in Table 11. As would be expected given the similarity in the subtest characteristics, the
composite scores are similar across test forms. Composite means for Forms Q1 and Q2 are
generally closer than means of P1 with either of the Forms Q. The composite mean scores suggest
that Forms Q1 and Q2 are slightly easier than P1, except for the Quantitative composite. Form Q2
has higher mean composite scores than Q1 on the Navigator-Technical, Academic Aptitude, and
Quantitative composites while Form Q1 has higher mean composite scores on the Pilot and
Verbal composites. However, there should be no significant differences in mean composite scores

for Forms Q after the equating.

26



Composite Analysis Results

The composite level statistics for Forms P1, Q] and Q2 are reported for the total sample
in Table 20. As would be expected given the similarity in the subtest characteristics, the
composite scores are similar across test forms. Composite means for Forms Q1 and Q2 are
generally closer than means of P1 with either Form Q1 or Q2. The composite mean scores
suggest that Forms Q] and Q2 are slightly easier than P1, except for the Quantitative composite.
Form Q2 has higher mean composite scores than Q1 on all composites, however, there should be
no significant differences in mean composite scores for Forms Q after the equating.

The skew and kurtosis values for the composites are quite similar across the three test
forms. The skew values range from -.14 to -.28; kurtosis values range from -.10 to -.80. These
skew and kurtosis values indicate the composite score distributions are relatively symmetric and
tend toward normality.

The composite intercorrelation matrix for Forms P1, QI and Q2 is presented in Table 21.
The data for all three forms are presented in one table to facilitate comparisons of composite
correlations across forms. The composite intercorrelations are among the composite within one
form, not among composites of different forms. The maximum correlation among composites is
_93 and results from the correlation between the Pilot and Navigator-Technical composites on all
three Forms. The minimum correlation is .60 and occurs between the Pilot and Verbal composites
on Form P1. The composite intercorrelations are nearly identical across test forms; the maximum
difference between any of the three composite correlations in a triads is .02. Thus, there is a high

degree of similarity among the composite intercorrelation matrices across the three test forms.

43



In general, the descriptive statistics of the subtests are similar across test forms. Subtest
mean scores generally differed by less than one unit. Exceptions to this pattern, or subtest
differences greater than one unit were observed between Forms P1 and Q1 on Scale Reading,
between Forms P1 and Q2 on Reading Comprehension, Scale Reading and Aviation Information
and between Forms Q1 and Q2 on Arithmetic Reasoning. The negligible magnitude of these
differences provide support for the parallelism of these measures.

The skew and kurtosis values for the subtests are quite similar across test forms. The
majority of the subtests are negatively skewed and none have skew values less than -1.00 or
greater than +1.00. Kurtosis values are similar across test forms with a few values around -1.00, a
value which indicates a slightly flatter score distribution. Thus, the subtest score distributions are
relatively symmetric and tend toward normality.

Kuder-Richardson 20 reliability estimates provide evidence of generally high internal
consistency and are approximately equivalent across test forms. The majority of the reliability
values are greater than .80, and the lowest estimate is .721. Reliability estimates are not
appropriate for subtests scored as speeded tests and thus are not provided for the Scale Reading
and Table Reading subtests.

The subtest intercorrelation matrix is presented for Forms P1, Q1 and Q2 in Table 10. The
data for all three forms are presented in one table to facilitate comparisons of subtest correlations
across forms. The subtest intercorrelations are among the subtests within one form, not among
subtests of different forms. The maximum correlation among subtests is .83, and represents the
correlation between Arithmetic Reasoning and Data Interpretation subtests on Form Q2. The

minimum correlation is .33 and occurs between the Word Knowledge and Electrical Maze



APPENDIX C: PRELIMINARY CONVERSION TABLES FOR AFOQT FORMS Q1 AND Q2

63



with a test development of parallel test forms that are equivalent to previous AFOQT test forms
on item specifications such as statistics and content. In addition to the test development process,
updating the AFOQT involves a provisional equating and operational equating.

The purpose of this report is to describe the construction of the AFOQT Forms Q1 and
Q2 and the subsequent equating of these forms to the previous Forms P. The first section
discusses item selection and the procedures involved in constructing Forms Q. The second section
covers the item, subtest and composite level statistics, and equating statistics of the 1993 data
collection used for the preliminary equating analyses. The third section provides this information
for the 1995 data used in the operational equating analyses. These equating analyses are integral

in linking the new forms of the AFOQT to previous forms to ensure equivalence of measurement.

DEVELOPMENT OF FORMS Q
This test development project began in January of 1991 with the objective of developing
two new AFOQT forms, Q] and Q2, that would be equivalent with previous Forms O, P1 and P2.
In maintaining continuity in the testing program, Forms Q were developed to be as similar as
possible to previous forms in terms overall test content, test length, item difficulty, item
discrimination, subject matter, and stylistic features. Based on prior analyses, the item difficulties
of the Form P1 subtests were considered to be more similar to Forms O than P2 item difficulties

and P1 was thus chosen as the target test to be replicated.



z-score linear equating is to be preferred. Linear equating uses two parameters, the mean and
standard deviation, per test form. When the z-score linear equating is not appropriate, then one of
the three smoothings of equipercentile equatings is chosen. These polynomial smoothings are
based upon two parameters for the linear smoothing, three parameters for the quadratic and four
parameters for the cubic smoothings. The cubic smoothing of the polynomial equating fits the raw
equipercentile data more closely than the quadratic, which fits more closely than the linear. When
sample sizes and the range of scores on a test are large, the parameters of the cubic equating are
stable and thus, cubic smoothed equipercentile equating should be considered.
Results and Discussion

Item Difficulty Analysis Results

For purposes of summation, item difficulty values are presented in a frequency distribution
with five categories. The categories used are arbitrary and use of alternative categories would
have changed the distribution. These categories have been used for similar purposes in reports of
previous tests and are therefore retained for this report. The nature of the categories allow item
difficulties within one category to be further apart than item difficulties between two categories.
For example, an item difficulty of .41 is in the same category as a difficulty of .59, yet a different
category than a difficulty of .39.

Because item difficulties are sample specific, distributions of item difficulties of the
subtests are provided for each of the three subsamples as well as the total sample. However, we
will focus our discussion on the distribution of item difficulties for the total sample presented in

Table 5. Table 6 provides the summary statistics (mean, median, minimum and maximum) for the

16



-
1 6 $ 0 0 z aU 0 0 0 8 $ Zz 0 SomnsTy VOPPIH n
0 v os 0 0 8 6 z 0 0 s 6 s 0 0Ud!dg [eIOUDH

0 8 9 1 0 0 or os 0 0 0 L 8 0 0 SOTA Pareiou

0 0 wk 0 0 9 or of 0 0 + lz 0 uopewoyuy uONIAy
uf Wm 2 0 cl gt 8 I 0 oo ft 6 z 0 Surpeoy o192L

% eI € 0 0 € L 8 z 0 (4 6 8 it 0 Bununoy xolg

z 6 s ’ 0 L oro 0 0 z a 0 0 worsuayorduiog joUTsUy

0 € rn? on) 0 [nn rn) I if 8 loot Burpeoy 9[29$

0 v 6 L 0 0 s wo 0 0 ’ 4 0 azeyl [eOMNoaTy,

0 I aL 0 0 s a 0 0 v a4 0 worsuayosduroD [eouEyoo,

) 6 st I ) v LI , 0 0 € or 6 € 0 adpojmouy We

if a oom ot 0 I st 8 if 0 z 9 a | aspa]MoUy PIAA

0 z st 9 z 0 6 lf I 0 v stot z uoneesdioyuy ere

0 I 6 € 0 z oO 8 ¢ 0 0 9 wt I worsuayosduioy suipesy

0 al 6 € 0 v uu Ss 0 0 6 8 L I 0 Suluoseoy Snounpy

0 ()) 2 0 0 L sl 0 0 € él 0 sordoyeuy [eqioa,

66- 08- 09= OF 0% 66° O8- 09 OF O@ 66-  08- 09> OP O@

Os'< 09'< Or< O%< 00" Os< 09%< O< O%< _-00" O8< 09< Or< 0% < -00" asoiqng

SING OLOW VvaV

Apnyg Sunenbg Areurunorg ur sajduresqng SL Pur D.LOU ‘WAV 10} [dq WO Joy suoNea10D jeWastg Jo uONNQINSIG “p-V 192L



S39 C691 8r'9 Sv'or oL9 ror sanemuend,

99°9 TL'6L sro vS'61 199 6161 eqn,
19°71 v9'9E soe 00°9€ er?r P9'SE opmindy onuapeoy
CLLT vy'79 OVLt SLT9 LOLI 60°19 Teoruyoa], 10}eS ARN
Orel 97 0S $671 0@'0S Of'el 076 Old
soqsoduiop,
Lt 09°F CLT 8b'b SLT 9b soin3iq UappIH
98T Bee 6LT SEE 18ST sve s0UdTOS [eIOUaH
Is Tee Ig l see sl 9TE SYOOT PAzIOY
ost 6Ly Lye Isp sre S8y uoneWIOsUy UONneIAY
ee Py il ere sell LVE SOIT Surpeoy gel
sol tly £0'T 6lb 901 Liv BununoD yoorg
£e7 £o'r wee Ly Cee sy uorsusyorduroD yuounysUy
LOE LU I6€ ixat 88°€ 671 Suipeoy o[e0g
LT Loe sot ve eL Tt HOE oze [ROHN
OL'T 987 OT v6T 691 Ssz worsuayarduroy jrorueyoayy
ere co's ore 65'S ore S'S ddpajmouy WRAL
997 “9 497 "99 Lt es°9 dB po|MOUYY POA
soz &CP 961 SO'r S61 96°E uonejesdioquy Beg
SLT wo wT 679 897 st9 uorsuayorduroD Surpeay
ve LOL 00'€ 189 Te £69 Sutuoseoy oneunpy
0vz 199 917 099 rad 0s'9 saidoyeuy [eqie,,
s}so1qng

uoneMoq WOnRIAN wonerAaq

prepurig urea prepurls URI prepuris ueo

[Z0) 1O Id

60

Apryg Supenby Areurunjerg ut sway] wouruiog Jo pastduoy suoneraasg prepueig pue suvoyy oysoduiog pue ysaigng “[-q 2198,



aquownatt

a
moeonnrnantan

WaMNTODCOMOSOTOSOANM

munamnoovonnonon

wr anannvonwtonore

MOANAMARKRHMtTNHAMON
a

KR Nt wonednzoNnonNnre

8

somndty UOppIH

g0UDTDg RIDIN

S400]g PowIoY,
woneUNOyU] voNeAy
Surpeay o1qeL,

unuNoD 401g
uorsusyordwiog jusumnysuy
BurIpesy s]2oS,

OZR [ROHIOO|A
uofsusyordurop jeorueyooyy
ospopmouy We,
OSPIPMOU POA,
uonejordioquy Byeq
uorsusyorduiop Surpesy
Bummoseay snowy
soldoyeuy [eqioA,

glp ow annnannoenratan
gla toma a ans MoM
SJncoococonNnoooHnTOAS
toOVRNooVAmMaraHEA

a
a

08-
Os< 09< OF< O%< -00 08< 09'<

slocococon too OoONNOAO

N
a
a
s
hed
i=}
+

08"-
; 09< OF< O%K<

e
2
.
i}
=

calotantanmoornongauaant
BR a
N

7

0¢< —-00"

slocoooocouccoNooSoS

00"

3

g

Hes
rans

Id

ysa}qng

Apnjg sunenby [euonessdg J0J sonnoyyiq wos] Jo uoNNQUIsIG “pT A921

33



Table D-5. Forms Q1 and Q2 Operational Conversion Table for Quantitative Composite

Raw score Percentile Percentile Raw score Percentile Percentile
Qi Q Qu Q
0-13 1 1 45 41 34
14 2 1 46 43 38
15 2 1 47 43 41
16 3 2 48 45 43
17 3 2 49 48 43
18 3 3 50 52 43
19 4 3 51 52 45
20 5 3 52 54 48
21 6 4 53 57 52
22 8 5 54 59 52
23 8 6 355 61 34
24 9 8 56 64 37
25 10 8 57 66 59
26 lu 9 58 69 61
27 ll 11 59 71 64
28 14 8 60 75 66
29 15 14 61 16 69
30 7 15 62 78 71
31 17 17 63 80 75
32 19 17 64 82 76
33 21 19 65 82 3
34 21 19 66 85 80 .
35 24 21 67 86 85
36 26 21 68 90 86
37 26 24 69 91 90
38 28 26 70 92 91
39 31 26 71 94 92
40 31 28 72 95 95
41 33 31 73 97 97
42 34 31 74 98 98
43 34 33 75 99 99
44 38 34

74



£06" 668° 6S" 9s” £68" Lye oss” tes” 6£6° e8e" veo" Ls" sointy UopplH
88° £60 £0s" 660" tls" 861° Och 960" 07" Lt Or” L8r" dOUdTOS JBIDUEH
6L8° Sst Lov" sis" sos" Srv Tos” Lis” 98" est or" rats SHOOT Powsoy
06L O8T” 90%" Lov" 699° 06r" 88e" 9b" SL" Ava Le 8SP UOTeULIOJU] UONeIAY
tre" Lol" STs" £09" 856° Lor" 66L' v6S" 8£6" Lov 008" 6S" Surpeoy qe L
S16" OLT £8¢° oss” £16" LLe 9b" cos" LI16" soe” to" £9¢" Sununod yorg
80L° vee ves” IIs" 88L" 6ro Les” LIS’ 869" 6ee ess’ sis” uojsuoyerduioD yuownysuy
686" 697 099° 89s" 06" 9ST bro 6s" 606° Oke Ls" Ors” Surpeoy aeog
slo” 880° ere’ Is’ Leo x4 a ore’ 6fh oro aS See’ str azey [eolqOoT a
POL 4 & 88e° lly 699° cL Ler OLY 099° €Le 86E" sly worsusyorduro) Jeorueyooy,
S68" 96E° £69" L8S° 676 86¢° £69" Ls £06" ctr sso TLS" oSpamouy We
716" ss 09" ers" 668° 990 09" ess S16" 167° S6s” 8bs" oB8po|Mouyy PIO,
rte" Sse eso" 19S" £88" 0¢z 99° sss 806° 067° 609° cos” uonejerdroyuy eyeq
SLB" cer £9" 69S" L938" 6¢E ceo ws 878" Oe soo ls uorsusyaiduoy Surpeoy
LLs’ 6LT L990" sss’ ols" Loe 98S" ses” 168" 60E° S6S° ses” Suyuoseoy snap
856" 6LT $89" £8" L96" LT slo" 78s" 96" 107 819° L9s" sordoyeuy [eq12A,
xe UI UeIpayy Ue Xe UI URIPIT_uLd xe iy UBIP, Udy ysanqng

[<0)

10

Id

Apmig Sunenby Jeuonesiody 103 sonfnoysiq Woy] Jo sonsNerg AreuUINg “sy BQeL

34



APPENDIX A: RESULTS OF ANALYSES FOR AFA, ROTC , AND BMTS SUBSAMPLES FOR
PRELIMINARY EQUATING STUDY

49

NS



998 SCOTS" el CEL gel S:COLCt:‘«COLS we 69SCSTLSCSSC«CEL Somndiy UOPPIH,
cL SSE LOD" 109° css bth BBS (GS pss STE’ = 679" «809" S0UdIDg [LIQUID
BSL 8S LOD £59" 86L' S8h P99" L 99" 88 oss BSN ZOD" SHOOTG Paweioy,
wie SOE LHL Shs" 19h 169869" 118 8% 609" ED uoneuUoguy uoNRIAY
€86° vz Ise OL" 16 O@E’ HL TL sie = 06" LT LL Surpeoy o1qeL,
eos Ie 68S Lig STE 9S" ED" $06 Ol’ GL €0L’ Bununoy yoorg
786 LBS €8L BLL LE6" 109 = 8h’ LL $96 eS’ OSL. uorsuayarduiog juoumnysuy
Lee tp 69" OS" 6€8 06" 1p 99" sis STZ 6S" 18s" Surpesy 91205
etl LZ COS ES 0L9 = 887 Izs OS” 669 = LOE OS'S azeYAl [ROMO
£98 LTE OHS 09% = tbh 88S BOS vol fhe ELS SOS" uorsuayordurog [eotueyooy,
O0OT 949 = bh8 HB" 00'T ws 8088" 4866S 8B UT o8paymouy ye
006 = 8Lz L890" sso" Leo’ £61 OPL’ 60" ol6 = soe? TL LOL OSpojmouy Ploy
198° Teh" lL £69" Tes Shh LEDS" 08L = 8SZ OP £09" uonejerdioyuy eye
os 66h SL SOL Iss OLE LS BD 978 se’ 989" 899" uojsuoyorduioy Surpeoy
188° L7S" 6 9L" gIL’ LIS GES ETL OTL ols 8h TL’ EL Sumuoseoy spouyy
96860 ZB Les BLES ED a a 2 sordoyeuy [eqio,,
XP, UI URIPYy UROYy XeL UIA ueipay, URoyy xe wy UBIpPy UROy yso1Qng
ce) ie) Id

Apmg sunenby Areurmpporg Joy suonejor0p jevosig Jo sonsnerg Areumung -g 9192,

22



1 £ v I 9 9 £ 0 0 9 s v 0 0 saandiq uoppiH
0 0 v zt t s I I I L L v 4 0 soUdTOg [eIOUaL
0 € v £ € s 9 i 0 s L if 4 0 SYDOTA PHIeIOY
0 0 £ 9 I 9 L 9 0 Ir s £ I 0 uoHeULIOyUy UONRIAY
v 8 v or vv 9 v 9 0 4 s € € 0 Buppeoy o19eL
4 9 v £ 8 8 4 4 0 £ €l v 0 0 Sununod yoo1g
i) 0 L £ T at s 0 0 8 8 v 0 0 uorsusyarduoD wWeuTUsU]
T 6 vat £ 91 91 9 z 0 174 0 9 0 0 Surpeoy 91205
0 0 9 8 0 L € 8 4 0 L v 8 I Ooze [BOO
0 0 9 v q € I v I v L 9 4 if uosuayarduioD jeoruRysoy
0 I L v or rat € 0 0 (44 € 0 0 0 aspopmouy ye
T T IL 9 L 8 L € 0 st v v 4 0 o8pajmouyy pro(y
0 4 or € v v1 L 0 0) Lt 8 0 0 0 uonejoidsoquy beg
0 4 or if 6 ol 9 0 0 st 0 0 0 0 uorsuayorduiod Surpeoy
0 9 8 £ 8 or L 0 0 61 v 4 0 0 Sujuoseoy onounpry
£ g L I 6 v I 0 st 9 4 I I sooyeuy [eqiea,
66- = 08" os = 09 OF ov 66- 09 OF 0

08"< 09"< 700° 08"< 0@'< -00° ysaqqns

Aprig Supenby Areunujorg ur sojduresqng SL Wa pue LOU ‘VAY 10F 7D UO 105 SoNMoUsIC| way] Jo uoRNAUNsIC “EY B148I,

52



els £6°ST 8's sesT Ls os’sT aaneuend
sls €L'8T 06'S 1st 96'S 17s Teqre,,
sor 99'KE 6£01 6LE£ sv Ol TWLee apnindy ormepeoy
amas 86°8S 98'bT SLLS 00'st Lys Teoquyoey, 10723 1AeN
rama £9°9¢ Tet LU9F Se il 8r'sy Wd
sopsoduiog
ct Ley el vy sol lev somsty UopPTH
cama vO’ CLT LOT OL POE SoUa!Og [eINUEH
6r'T LOE ost soe ost L67@ Sx90[_ Payeioy
cet 6Le see LLe oe7 We uONeULOsU] UOReIAY
PLT Oe tl SLT SUIT 987 86°01 ‘Burpeoy 21981,
raat 60° orl 60°F LUT 90° BununoD por
ove lor Ce sor 8tz lor uolsusyaiduro juoumnysuy
ose soll £9'€ ISI I9€ ITIL Surpeoy eTe0g
I9T we tT 68% 8s 987 ozeY [eomnooly
eo'l “v7 OT we v9T 4 worsuayasduroD [eoruRYysoy
£4 oes E34 evs 907 ws adpojmouy Wel
svt sv9 87 99 x4 8e9 adpojMouy PION
£8 ely 98'T 16€ 6LT 98€ uoneyerdioquy Beg
Set 679 v7 86'S 4 tS worsuayaidurog Surpeoy
£87 6¥'9 wT wo 87 Ww9 Suuoseoy snowy
£07 Lv9 wt sT9 LOT 109 sordopeuy JeqioA,
s1S9}QN§

woneAed roncincyed wOneIASd

prepuris urea prepuris wea prepuels ued

a) 1d Id

Apmg Sunenbg jeuonersdo ut surayy uowrws0D jo pastduiog suonetaag piepurjg pue sueayy oysodusog pue ysaiqns ‘Z-¢ Ade.

61



L

Ml.

Iv.

VI.

CONVERSION TABLES .

TABLE OF CONTENTS

INTRODUCTION ..0.0....ccccccsessessesseesessessesseeecesessveesesesscsucsncseeseanecseciesssneseeneenecnseneenenneene 1
DEVELOPMENT OF FORMS QQ o...0....cessssssessesseesseeeeesseessneenesessssueeiesnesseenensereseneneties 2
Test Content ..........cccccccccccceesecececsessssesessessesnsenenesseneusnesasnsreeenssesisasenensarentieeseasesees 3
Item Selection. 4
New and Common Items 4
Item Difficulty and Discrimination 5
Subject Matter... 6
Stylistic Feature: 7
Results and Discussion............ 7
PRELIMINARY EQUATING STUDY....0......::cccsccsceccsseseeesesseeseeseesesneeteerseneeeenenneennenees 11
Subjects
Administration 11
Data Analysis 0.0.0.0... 13
Classical Item Analysis 13
Subtest and Composite Analysis 14
Equating Analysis 15
Results and Discussion..... 16
Item Difficulty Analysis Results 16
Item Discrimination Analysis Results 20
Subtest Analysis Results .... 23
Composite Analysis Results 26
Equating Analysis Results .. 30
OPERATIONAL EQUATING STUDY \.0.......cccccscecceceeseeseseseeeeneseesesieeiesceseeseeeneseeeeeees 30
Subjects we
Administration 31
Data Analysis . 32
Results and Discussio: 32
Item Difficulty Analysis Results 32
Item Discrimination Analysis Results 36
Subtest Analysis Results ....
Composite Analysis Results
Equating Analysis Results ..
IMPLEMENTATION EFFECTS OF INSTITUTING THE OPERATIONAL

CONCLUSIONS AND RECOMMENDATIONS ............cccccecsseceesesseteseeneteesneeeneneeeee 47

iii



TL 89L~«COLL SCO CLS To 6¢- 8¢e STE OTE IS6 C88 6L6 SSO SOO £89 soanBhy UOpPrH
Isl’ OSL’ ESL’ 8S SH Ir 6h? BOE LEE (OBE S66 «LG LBB 86h CLG LOW 2OUDIOg JEIOUSH
9SL’ PLL’ OLL 69° EL UW It 6vE OPE TEE T08 GER LLL ers LOS OES syooTg pareioy
Tes 9€8" 98L’ 1010 98 $806 SH (ON «LOH «LHR LTS LSL (Oth HOW B6E" woneunroyuy
uoneiay

fs ly CS Sh> OOS GHD LOL GT'L «SBLZ OTLZ ITLE BEL IZ Oe Surpeoy a102L

108 408 «Te8 (OH OZ> —BETME> Os BHT CLE OOH bh «OPTI «TL «OLZ «79 «OES Bununod xorg,
SL8 088" ZLB" CO's ST'I- «ETT OT =60'—s 80"——s«*OES ES LTS «SOL «OB'OT ~ILOI Lz bs §«9¢s° —_NoIsUoyoudwi0g)
yuoumysuy

Le-  €€> E> OTE TE> OST OTL SPL «69 HIST GOH GTEC 799 ZH 66S" Surpeoy o1e0g

etl’ $89 «ble «SO ST «= £0" SECO «SCTE «SSE OL CEL «CEL «OTR OtSE OSE” azeyy RONNDAT
LOL OTS’ SSL’ 69- —-T6= OSs STCE’:SC‘ETH:CSHH CLO «86° LBB BSH OO Oh —suoyorduro
yeorueysoy]

£88 LL8° 888"
ses £98" OLB
sT8 818° OLL
Ovg’L78"s EES"

S8-  06- 8r- IE- 9€- 99S 19S LOS 8TLI B99T ESOT SOL £89 WI adpopmouy Wel
8L- 68" IT- 1% ST- 80'S 9S E9S PBI GEST IT'ST 909 679 LID a8popMouyy PIOM,
a9 Ip 6@- LU S@- HO HOH BHP «IGST LOST PSST Bro «£9 69" = MoNRIoIdoyUy BIE
£L- 99 O0€- 61> 9I- 16h ITS OFS SE9T SPST 6L'PL 199 LZ9 109° —_uoIsuoYorduOD

surpeoy

$98 TL8' LL 98 06 = BB> TI LO 8OTM LH'S ELS ELIS: «HOST Eb'DT LE'PT OF ©«S8S° 609" Sujuoseay,
oneunpuy

ple 18’ OL) CO- —- LETM OS E> OE OTB H_HEH =—IL'9T —6h'9T_9L'ST_ 969" S69" _ 099" __saldoyeUY TeqIOA
10 Id @ 10 td @ 10 Id 10 Wd © 10 Id 10 Id ysorgng

ANTGeTTY ‘SISOUNY MOIS UONeIADp piepueyg uRo you109 uoNodorg

Apmg Zunenbg jeuonesed¢ soy sysajqng Jo sonsnerg oanduosag ‘g] 9qeL

4l



Subtests Analysis Results

Table 18 provides the summary statistics for the subtests for the total sample. In general,
the descriptive statistics of the subtests are similar across test forms. Subtest mean scores
generally differed by less than one unit. Exceptions to this pattern, or subtest differences greater
than one unit were observed between Forms P1 and Q1 on Scale Reading, between Forms P1 and
Q2 on Reading Comprehension, Scale Reading and General Science and between Forms Q1 and
Q2 on Arithmetic Reasoning and Scale Reading. The negligible magnitude of these differences
provide support for the parallelism of these measures.

The skew and kurtosis values for the subtests are quite similar across test forms. The
majority of the subtests are negatively skewed and none have skew values less than -1.00 or
greater than +1.00. Kurtosis values are similar across test forms with a few values around -1.00, a
value which indicates a slightly flatter score distribution. Thus, the subtest score distributions are
relatively symmetric and tend toward normality.

Kuder-Richardson 20 reliability estimates provide evidence of generally high internal
consistency and are quite similar across test forms. The majority of the reliability values are
greater than .80, and the lowest estimate is .685. In general, these reliability values are lower than
those obtained in the preliminary equating study. Reliability estimates are not appropriate for
subtests scored as speeded tests and thus are not provided for the Scale Reading and Table
Reading subtests.

The subtest intercorrelation matrix is presented for Forms P1, Q1 and Q2 in Table 19. The
data for all three forms are presented in one table to facilitate comparisons of subtest correlations

across forms. Again, the subtest intercorrelations are among the subtests within one form, not

39



Table C-2. Forms Q1 and Q2 Preliminary Conversion Table for Navigator-Technical Composite

Raw Raw Raw . Raw Raw
score Percentile score Percentile score Percentile _score Percentile _score__ Percentile
QQ QQ Qi__Q2 Qi Q2 a @
0-56 1 1 9 ll ol 1440 32 «32 182 65 64 224 96 94
57 2 2 9 WM ol 141 33 33 183 66 65 225 96 95
58 2 2 100 «6120 «(12 142 34 «(34 184 67) 65 226 96 = =—95
59 2 2 101. 12)«s«12 148° 35 (35 185 68 66 227 97:95
60 2 2 102 12 12 144 36 «36 186 69 67 228 97 96
61 2 2 103. «1313 145 36 36 187 70 68 229 «97 «96
62 2 2 104 13,13 146 37 36 188 71 69 230 97 96
63 2 2 105 14) «14 147 38 (37 189 72 70 231 98 96
64 2 2 106 «14 «414 148 38) (38 199 73 71 232° 98 97
65 2 2 1077 4 #14 149 39 «38 191 74 72 233 99 (97
66 2 2 108 15 15 150 40 39 192 74 73 234 «99 (97
67 2 2 109 #15 «(15 151 41 40 193 75 73 235° 99 (97
68 2 2 no 15) (15 152. 41: 41 194 76 74 236 99 =—98
69 3 3 lll 16 16 153. 42) 42 195 77 74 237 9998
70 3 3 1122. 1616 154 4343 196 78 75 238 499 8698
ri 3 3 1130¢417° «17 155° 4343 197 79 76 239 «99 «99
72 3 3 14 °417 #1 156 43 43 198 79 77 240 99 99
73 3 3 115 18 «18 157) 4443. 199 80 78 241 99 99
14 3 3 16 18 18 158 45 44 200 81 79 242 99 99
75 4 4 117, 1818 159 46 «45 201 81 79 243 99 «99
76 4 4 118 «19 (18 160 47 46 202 82 «80 244 99 99
77 4 4 119 «1919 161 48 47 203 83 = 81 245 99 99
78 4 4 120 20 =«20 162 49 48 204 84 «81 246 99 99
79 5 5 121 20 20 163 50 49 205 85 82 247 99 = 99
80 5 5 1220 21° «(21 164 51 50 206 «86 683 248 99 99
81 5 5 123, 21 «21 165 52 50 207 86 «684 249 99 99
82 5 5 124 220«-22 166 52 «51 208 87 85 250 99 99
83 6 6 125° 230 23 167 53 52 209 87 86 251 99 99
84 6 6 126 23 (23 168 53 52 210 88 86 252, 99 = «99
85 7 7 127) 24 = 23 169 54 53 211 89 87 253 99 «99
86 7 7 128 25 (24 170 5554 212 89 «87 254 99 99
87 7 7 129 25 25 171 5655 213 90 88 255 99 99
88 7 8 130-2525 172 57) 56 214 90 88 256 99 «99
89 8 8 131 26 26 173, 58 = (57 215 91 89 257 99-99
90 8 8 132027027 174 59 58 216 92 89 258 99 99
91 8 8 133 28-28 175 60 «59 217 93 90 259 99 99
92 9 9 134 29 «29 176 61 60 218 93 91 260 99 99
93 9 9 135 29 «29 177 62)—«CO#61 219 94 «91 261 99 99
94 9 9 136 «630029 178 63 «62 220 94 92 262 99 99
95 9 9 137 30 «30 179 63) «63 221 95 93 263 99 99
96 10 10 138 31 30 180 64 63 222 95 «93 264 99 99
97 (10 (10 139 32_—31 181 6563 223, 9694 26599 99

65



Table 3. Distribution of Biserial Correlations for Forms Q1 and Q2 at Test Construction Stage

Subtest Biserial Correlations Forms Q] (Q2)

"20-29 30-39 40-49 50-59 .60-.69 __—.70-.79 __—.80-.89 _—.90-.99
Verbal Analogies 0() 0 6(2) 12(14) 6 1Q) 0 0
Arithmetic Reasoning 0 0 1() 8 (9) 13 (9) 3 (4) 0 0
Reading

Comprehension 0 0 6(7) 10 (9) 7 2 0 0
Data Interpretation 0 1 8 (6) 9 (12) 6 1(0) 0 0)
Word Knowledge 0 1 3 (1) 6(9) 11 (10) 4 0 0
Math Knowledge 0 0 3 8 (10) 6 (5) 7(6) 1 0
Mechanical .

Comprehension 0 1 4(2) 6 (5) 8 (9) 1) 0 0
Electrical Maze 0 2 3 5 (4) 8 (5) 26) 0 0
Scale Reading 0 0(2) 21(20) 13 (11) 35) 1 0 0
Instrument

Comprehension 0 0 0 6 (5) 5 6 34 0
Block Counting 0 0 4 (2) 3 (5) 5 56) 3 (2) 0
Table Reading 0 30) 4(7) 7(5) 9 9(11) 6(7) 2m
Aviation Information 0 32) 4 10(8) 2 (4) 1(2) 0
Rotated Blocks 0 2(1) 0 (2) 7 (8) 4(2) 2 0 0
General Science 0 0 7 (5) 9 (10) 4 0() 0 0
Hidden Figures 0 0 0 2 4(5) 7 2() 0

Note. * Reported values are for Forms Q1 and Q2 except where a parentheses surround the value for Form Q

indicating a discrepancy.



T € zt L t 9 9 € 0 0 9 € ¢ T 0 somndiy UopprH
0 if € oOo 9 z s oO o€ 0 9 9 L I 0 aoualog [esoUaH
0 z s L if € L + 1 0 s L z I 0 SySOT_ Parwioy
0 0 v Wm ¢ if 9 8 s 0 8 9 9 0 0 woneuLoyu] UOnRIAY
1 org s I st v 9 0 9% OL € + 0 Surpeoy 1981
€ s € 9 € ors s 0 0 9 6 s 0 0 Bupunod xorg
0 0 s wa OE 9 oF 0 0 8 L s 0 0 uorsuayarduroD journnsuy
0 6 el oft oF ol tl 6 0 0 % MO € 0 0 Bulpesy 1835
0 0 v 6 L 0 9 v L € 0 s 9 L z aze [eounoala,
0 0 + alg I % ag 0 z oOo 9 z 0 uoysusyaiduioD yeorueyooy
0 € 9 "ms rl 6 z 0 0 zw + 0 0 0 aspoymouy We
I I L a oF 8 oO 6s z 0 1 oF s 0 0 adpo|Mouy P10
0 € 6 or s or if 0 el ol ot if 0 uonejerdzaquy eed
0 z 6 6 s L WoL 0 0 6 ao 0 0 uorsuayordutog surpeoy
0 4 L 8 8 L ol L a 0 1 L s 0 0 dutuosroy snowy
% iy $ 6 z a ok 9 0) 0 lk s 0 0 sorgopeuy Teqion,
66- O08 0 or- Oe" 66 (a) 66-  08- 09- OF- 0%

Os< 09< Or< 0%< _-00" 0s'< 09< Or< 0%< -00° 08'< 09< Or< O%< -00° ysanqng

SLNG LOU Vva¥

51

Apmag dunenby Areurumjorg ut sojduesqng gL Pur ‘O.LOU ‘WAV 40J 1D WLIO, 1Oy soNMMYFIC wij] Jo uoNNgINsIG “Z-V 1G2L



18L 16L) SSL (Ot- =OdIh> OT 667 €1€ OOE WIT SBOI OSTT 96L° SSL’ 96L" somatq UspPlH

use’ sso @SL’ by $9 OL LOE SEE LLE OIL EPIL E601 BLS 68S 685° soustog |eIOUIH

Spe LIL) ESL’ @> 0 09> SIE LOT PIE 916 886 ~%6 Lz 819 99° SyDO[_ PaeIoy,

08’ «S18 9SL’  Sh-  tS> SO- dhhh OOK Bb CEM «SBE «~FEOL @rOr 0L6 61S’ Ezs’ IS" woneuLroyuy

uoneiAy

tl 0% 0 69-9 9G IDL HBO BHL «(OFZ «—LL'OZT OBST 99L' PBL’ BSL Surpeoy o1qeL

L8L’  6bL POL’ LS- «LE OBI- ohh OL SS OLE «OEE «BNE «LSEL LT I9PT LIL 794 OL Sununog yorg,

188° 898" 048" 08- = #9 ST= 09-9 GL= HOTS. OSH SLY «EDEL «LEPI SO'PT 889 EL’ TTL’ —_uorsuayordi0D

yuoumnnsuy

fT 90" 9-08 = 99> OK TTL LOS «OLD THBZ LOST ITLZ 6bL’ BOL’ EOL Surpeoy 21295

Spl’ EEL’ StL BT we OT TT aH CILE «LOE «SOE «87H «CHB ONS. BSH CER teh Ooze [RONNOOTA

“BL LLL) SSL EL= SB= LL UT SOOO LTH ZO LOOT. «SL'OT «O70 «POS «OFS ZI’ worsuayardwio

yeorueysoyy

88-098" LLB’ GI'- —S9" 18 6s OLY OTS Py6T SOL IZ6I ZL’ LOS’ P8L' = eB P|Mou We

sis 7B Sh8 OS Zh ee 69> Sl 00'S P79 ILE 96°91 0L9 %ZL’ 004 — aBPaynour{ PIO

9€8° 608 8L  SZ- 60> +9- SOS LOP Oh FELT GOLI 9PLI TOL’ 469 914’ wONeIerdsoyU] BE

ves’ 98L" 728 = so 66 Wh Ly GLLT LSLI 1891 LZL OZ’ 69° —uoIsuayaudwtoD

Suipeoy

188° 198 [48 6S 8S EB Ur LE E> ESS TES GES OSLI SOOT LOLI O%L £89 SOL Suyuoseoy

oneuny

UL) 004" ete) LES * 69> —9> TS TSE OME LSE USBI SB'RI OC BT EBL S08 ISL’ _sardojeuy Jequen

WO dw 10 Cd OO OCOTCi(C( t*‘COTCdTOTC*dA=CwCéCSC*dTC*édA“‘<i<‘i CO UOC ysaiqns
ANTIQRITY SISOMNS] MOAS uoneIAsp plepuels ues ya1109 uoodorg

Apnyg Bupenby Areurunjorg ut ajduresqns D.LOW 40J sisaiqng Jo sonsneig aanduosoq “g-y 2198.

57



PRELIMINARY EQUATING STUDY
Subjects

Subject samples for the preliminary equating study were selected on availability but also to
have a broad range of ability. For this purpose, examinees selected were from samples of the Air
Force Academy sophomore and junior class, Air Force ROTC cadets, and airmen from the Basic
Military Training School. Hereafter these samples will be referred to as AFA, ROTC, and BMTS
respectively. ROTC and BMTS examinees were tested from mid-June to mid-August in 1992.
The AFA examinees were tested during the end of the school year in 1993.

Demographic information is presented for the total sample and these three subsamples in
Table 4. Subjects were predominately male, Caucasian, high school graduates and had attained
approximately fourteen or fifteen years of education.

Administration

The AFOQT data for the equating study were collected during four and one-half hour
testing sessions during which the standardized test procedures were observed as closely as
possible. The standardized procedures for administration are provided in the AFOQT Manual For
Administration for Forms Q1 and Q2, a document issued by Air Force Personnel Center (AFPC)
that explicates standard test conditions, test material preparation, the use of proctors, and the
protocol for conducting the testing session. Testing occurred at Lackland Air Force Base for the
examinees from the ROTC and BMTS samples and at the Air Force Academy for AFA

examinees.

ll



REPORT DOCUMENTATION PAGE

Farm Approved
OMB No. 0704-0188

aren seat of ino i tiated to vrge 1 four pepe
‘comments regarding this burden estimate or any other aspect ‘for reducing this burden, to Washington
port nd Roper 115 Jetson Drs Highway Ste 1205 Argan VA 222024008, Of of Managemen and udp Peper Reuctn Prot O7O4 O18), Weshen,DC 2509.

fe coeton at alomton Sone

ing te data needed, and

incising the time
ott lect trata, coy

ig aig ata

Severs Deveterte fe fermton

1. AGENCY USE ONLY (Leave blank)

2. REPORT DATE
July 1997

3. REPORT TYPE AND DATES COVERED

Final - January 1991-August 1996

4, TITLE AND SUBTITLE

5. FUNDING NUMBERS

Air Force Officer Qualifying Test (AFOQT): Forms Q Development, Preliminary PE - 62205F

Equating and Operational Equating PR - 1123
TA - A3

6. AUTHOR(S) ‘WU - 05

Theresa M Glomb

James A Earles

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

Armstrong Laboratory
Human Resources Directorate

Manpower & Personnel Research Division

7909 Lindbergh Drive
Brooks AFB TX _78235-5352

8. PERFORMING ORGANIZATION
REPORT NUMBER

AL/HR-TP-1996-0036

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)

10, SPONSORING/MONITORING
AGENCY REPORT NUMBER

11. SUPPLEMENTARY NOTES

Armstrong Laboratory Technical Monitor: James A. Earles, (210) 536-3257

‘12a. DISTRIBUTION AVAILABILITY STATEMENT

Approved for public release; distribution is unlimited.

12b. DISTRIBUTION CODE

13. ABSTRACT (Maximum 200 words)

Air Force Officer Qualifying Test (AFOQT) Forms Q1 and Q2 were developed to be highly similar to the just previous
Forms P1 and P2 in test content and length, item difficulty and discrimination, and stylistic features. Approximately half the
items were new and half were from Forms P. Forms Q1 and Q2 were preliminarily equated on Air Force Academy students,
Reserve Officer Training Corps cadets and Officer Training School cadets. Scoring tables were developed and AFOQT
Forms P1, Q1, and Q2 were administered operationally to over 3000 cases per form. Equating tables were accomplished
using the data from these actual applicants. It was concluded that these scoring tables should be implemented to make Air

Force Officer selection decisions.

14. SUBJECT TERMS 15. NUMBER OF PAGES
Air Force Officer Qualify Test 80
AFOQT 16. PRICE CODE
Equating
17. SECURITY CLASSIFICATION 78. SECURITY CLASSIFICATION 19. SECURITY CLASSIFICATION 20. LIMITATION OF
OF REPORT OF THIS PAGE OF ABSTRACT ABSTRACT
Unclassified Unclassified Unclassified UL

Standard F Form, 298. 8 (Rev. 2-89) (EG)
Ses ory

hg Portrm ro, WASDIOR, Oct 9¢



with the exception that they were to cycle through Forms P1, Q1 and Q2 in that order to all
examinees as they came in for testing.
Data Analysis

As mentioned previously, the data analysis section for both the preliminary and operational
equating studies are similar. The main difference in the two analysis procedures and resultant
output is that the preliminary analysis was comprised of total and subsample analyses, whereas the
operational analyses involved no subgroup analyses. In addition, the second set of equating
analyses, the operational equatings, allowed for comparisons between the preliminary and
operational equatings based on the evaluation of critical selection cut areas.

Results and Discussion
Item Difficulty Analysis Results

As in the Preliminary Equating section, item difficulty values are presented in a frequency
distribution with five categories. Distributions of item difficulties are provided in Table 14. Table
15 presents the summary statistics for the item difficulty values for the sample.

Table 14 shows the majority of items in P1 have difficulties ranging from .20 to .80.
Electrical Maze and Table Reading are the only subtests that include items with difficulties below
.20. Thirteen of the subtests have at least one item with a difficulty above .80. One-half of the
items in the Table Reading subtest have item difficulties above .80, suggesting that Table Reading
is a relatively easy subtest. The mean level of item difficulty for the subtests, shown in Table 15, is

between .40 and .60 for all sixteen subtests.

32



There are fluctuations in the frequency distributions of the item difficulties on Forms P1, QI
and Q2. When easier items are defined as those with difficulties greater than .60 then Q2 had two or
more easier items than Q] in Verbal Analogies, Arithmetic Reasoning, Reading Comprehension and
Math Knowledge and Q] had two or more easier items than Q2 in Block Counting. There are no
substantial differences in the mean item difficulty of a subtest across the three test forms. The
maximum difference in subtest mean item difficulty values among any two of the three test forms
ranged from .004 to .026. Only four subtests, Arithmetic Reasoning, Reading Comprehension, Scale
reading and Hidden Figures, had a largest pairwise difference greater than .020.

Item Discrimination Analysis Results

For purposes of summation, item discrimination values are presented in a frequency
distribution with five categories. As is the case with the item difficulty distributions, the categories
used are arbitrary and use of alternative categories would alter the distribution. These categories
have been used for similar purposes in reports of previous tests and are therefore retained for this
report. The nature of the categories allow item discriminations within one category to be further
apart than item discrimination between two categories. For example, an item discrimination of .41 is
in the same category as a discrimination of .59, yet a different category than a discrimination of .39.

Because item discriminations are sample specific, distributions of item discriminations are
provided for each of the three subsamples as well as the total sample. However, we will focus our
discussion on the frequency distribution of the total sample presented in Table 7. Readers who wish
to compare item discriminations of the subsamples across test forms should refer to Appendix A.

Table 8 provides the summary statistics for the item discrimination values for the total sample.

20



NOTICES

Publication of this paper does not constitute approval or disapproval of the ideas
or findings. It is published in the interest of scientific and technical information (STINFO)

exchange.

When Government drawings, specifications, or other data are used for any
purpose ather than in connection with a definitely Government-related procurement, the
United Stases Government incurs no responsibility or any obligation whatsoever. The fact
that the Government may have formulated or in any way supplied the said drawings,
specifications, or other data, is not to be regarded by implication, or otherwise in any
manner comstrued, as licensing the holder, or any other person or corporation; or as
conveying any rights or permission to manufacture, use, or sell any patented invention that
may in any way be related thereto.

‘The Office of Public Affairs has reviewed this paper, and it is releasable to the
National Technical Information Service, where it will be available to the general public,

including foreign nationals.
or publication,
PATRICK C. KYLLONEN

JAMES A. EARLES \
Project Scientist Technical Director

GARY D. ZANK, Zp

Chief, Manpower & Personnel Research Division

This paper has been reviewed and is app

Please notify this office, AL/HRPP, 7909 Lindbergh Drive, Brooks AFB TX 78235-5352, if your
address changes, or if you no longer want to receive our technical reports. You may write or call
the STINFO office at DSN 240-3853 or commercial (210) 536-3853



08 tL St +1 0e= Pl Sb CEH CBB OCLOOH «ET LY SANEUENO

Is L9- 89° 9e* Iv> LI- sozl OMET «=—IWET «= OSLE ELE S'S Teqo,,

ss 09 ss 87" 1e> we Lev ELST «NST L996 = OHH «OBS opmindy stuopesy

oe se st : sT- Le- 998E CECE —OHBE = BRST «99ST. «= ZO'EST ~—Teotuypay-s0ye3TAeN

81> 0 or st- 61 £987 OVS ERT HT BIL SOIT —OF'ETT 1l!d

[<) ite) Td 1 Id 20) TO Id 2 TO Td oysoduoy
sIsonny MOAS uOHeIASp prepuels ues,

Apnig Sunenbg jeuoneredy 103 soyisoduioy jo SONSHE}S oANdiiosog ‘OZ 919k.

44



68 $6= L6= 09 1s= ly svsI  OC@LTSTLT.~—sOO8'0S.-—S—iCiLO' HCL OY SATE UENS

9¢ S'- OL 89> 09" $s" 90st LIST = LS'ST— €0'6P = hGH OLY Teqio,,

ss 89° lL OL 79" 8s" OPIE POE SOOE F866 2786 IPL6 epnindy omopeoy

lL 09° 69- es 6s 8e- tLe PL'Sh ENSH BT LOT PESO «= «19'ZOT_—eoruyooy-10yeS1aeN

£9 or ss Lr 7 tp Ooee PB ZE SSE BS OTT VLEET 1l'd

(20) ite) Td £20) 10 Id 20) 10 Id [40 10 Td susodwioy
SIsOuNyy MOS UOHeIASp prepuels ueoyy

Apmig Sunenbg Areurunporg Joy soysodwoy jo sonsyerg aandiosag “11 91481,

28



0 or oF T 0 0 Tt + 0 0 0 wT 0 0 soindiJ UOPPIA
0 s wo 0 0 9 v1 0 0 0 6 8 z 0 2OUDIg [eIOUEH
0 + 0 0 0 + 0 0 0 Woo 0 0 syporg pawioy
€ 6 8 0 0 z 6 6 0 0 0 ork z 0 uoneUTOsUT UONRIAY
8 nn 4 0 9 1% 6 t 0 or of 8 z 0 Surpesy 1421,
0 8 a 0 0 0 als z 0 I slob 0 0 Sununoy 401g
€ tl 0 0 s 2 0 0 i st + 0 0 worsusyordurod uownysuy
0 () nn 0 0 sl €% «2 0 0 6 wm OL 0 Supeoy oe9$
0 L 6 v 0 0 s 6 9 0 0 v al € 0 azey [ROUT
I 6 6 1 0 0 1 8 I 0 0 9 af 0 uorsuayarduiog peorueyooyy
+ 1S 0 0 I oe oF 0 0 1 oF 0 0 ospojmouy WA
0 loom ot 0 0 1 8 I 0 I ar I 0 aSpoymouy Pio
0 oO oa 0 0 L sr 0 0 0 L 2 0 uonejeidioyyy Bed
0 Ir IL € 0 0 a (Ol € 0 0 a 4 0 uorsuoyardurod Surpeoy
0 or 9 0 0 0 o os 0 0 0 & 7% 0 0 Suyuoseoy snowy
0 6 10 0 0 6 an) 0 0 9 él 0 0 sordoyeuy [eq1oA,
66-  08- 09- OF OF 66-08 09 OP OF 66" 08- 09> OF 0%

Os< 09< OF< O%< _-00° O8'< 09< Or< 0%< “00° 08< 09< OP< 0%< _-00° ysanqns

<0) TO Id

Apnyg Sunenbg jeuonessdy 10 suonejas0Z yerastg Jo uoNNIsIq “91 9192L

37



The items on all three test forms, P1, Q1 and Q2, show acceptable biserial correlations.
The frequency distribution of biserial correlations in Table 7 shows that almost all are above .40
and the majority fall in the .60 to .80 range. The subtest mean biserial correlations in Table 8 are
generally between .50 and .70 with the minimum mean biserial correlation values of .546, .516.
and .536 for Forms P1, QI, and Q2 respectively. These biserial correlations indicate that the
dichotomous item responses correlate well with the subtest score and discriminate well among the
examinees.

In comparing the subtest discrimination indices of P1, Q1, and Q2 it is evident that there
are fluctuations in the frequency distributions of the biserial correlations. When item
discriminations of greater than .60 are taken to be good discrimination then Q1 had five more
such items in Word Knowledge than Q2 and three more in Block Counting, while Q2 had three
more than Q] in Electrical Maze. The maximum difference in subtest mean biserial correlation
values for any two of the three test forms, P1, Ql, and Q2, ranged from .011 to .090. QI and Q2
had reasonably higher mean discrimination than P1 on Data Interpretation, Rotated Blocks and
Scale Reading and lower discrimination than P1 on Block Counting. In comparing Forms Q] and
Q2, a difference in the subtest mean biserial correlations range from .000 to .057.

Subtests Analysis Results

Descriptive statistics for the subtests are provided for the total sample as well as the AFA,
ROTC and BMTS subsamples. Table 9 provides the summary statistics for the subtests for the
total sample. Readers who wish to compare subtests statistics of the subsamples across test forms

should refer to Appendix A.

23



Stylistic Features

Stylistic features of Forms O, including format, appearance on a page, type size, type face,
illustrations and legend characteristics, consistency of spelling and mathematical notations, were
closely matched in Forms P. These same stylistic features were to be replicated as closely as
possible in Forms Q, with one exception. The Bodoni typeface of previous AFOOT test forms, a
rarely used font, was replaced with the Times Roman font in Forms Q for purposes of printed
copy clarity and familiarity.

Following item selection, the subtests were submitted to the monitoring agency to be
checked for accuracy, spelling errors, typographical errors, inter-item clueing, distribution of
common items, overall presentation, etc. Adjustments were made and replacement items selected
where appropriate.

Results and Discussion

The overall results indicate that Forms Q1 and Q2 were closely equivalent with Form P1
and with one another with respect to the item selection criteria. Item difficulty analyses (using
estimates of difficulty for officer candidates) suggest that Forms Q1 and Q2 were nearly identical;
mean subtest item difficulty values differed by less than .003. In addition, the distributions of item
difficulties on both Forms Q1 and Q2 were quite close for most of the subtests, as is evident in
Table 2. Comparisons between subtest mean item difficulty of Forms Q and Form P1 further
attest to the comparability of forms with respect to item difficulty, as these discrepancies were less
than .01 for ten subtests. For the Reading Comprehension, Data Interpretation, Mechanical
Comprehension and Block Counting subtests, these discrepancies in subtest mean item difficulty

were still small, between .02 and .03.



APPENDIX B. COMMON ITEM ANALYSES RESULTS

59



Table 21. Intercorrelations of Composites for Operational Equating Study

Composite Test Nav-tech Academic Verbal Quantitative
Form Aptitude

Pilot Pl -93 74 60 74
Qi -93 -75 62 -73
Q .93 .75 62 73

Navigator Technical Pl 83 62 89
Qi 84 62 88
Q 84 61 89

Academic Aptitude Pl -90 91
Qi 89 91
Q 88 91

Verbal Pl 64
Qi 64
Q2 62

Equating Analysis Results

Four possible equatings, the z-score linear, linear smoothed equipercentile, quadratic
smoothed equipercentile and cubic smoothed equipercentile, were developed and compared for
each composite on QI and Q2. As was the case in the preliminary equating study, the evaluations
of the equatings ruled out the z-score linear equating and given that sample sizes were large
enough to ensure stability, the cubic smoothing equipercentile equatings were selected for each of
the five composites on each test form. Using this equipercentile equating with cubic smoothing,
preliminary conversion tables were developed and are presented in Appendix D.

IMPLEMENTATION EFFECTS OF INSTITUTING THE OPERATIONAL

CONVERSION TABLES

The preliminary conversion tables were used during the selection and classification of

officer commissioning applicants during the data collection for the operational equating study.

The data from the operational equating study were used to develop the operational equating

45



score anywhere as low as the tenth percentile. A distribution of applicants in the operational
equating sample with Pilot composite scores of 25 through 34 (n=352) found only three cases
with a Navigator-Technical percentile less than 10 and only 8 cases with a Navigator-Technical
percentile less than 15. Therefore, the tenth percentile minimum is basically an irrelevant
minimum, so there will be no noticeable operational effect in switching from the preliminary
conversion tables to the operational conversion tables.
CONCLUSIONS AND RECOMMENDATIONS

The AFOQT Forms Q1 and Q2 operational conversion tables based on the operational
equating study should be implemented for use in making officer selection decisions. The
operational conversion tables are more acceptable than the preliminary conversion tables because
they were based on the responses of the larger, more appropriate sample used in the operational
equating study. In the operational equating study the subjects were actual applicants for officer
commissioning who were motivated to do well, thus the operational conversions tables developed

on this sample are preferable.

47



Subject matter comparability between Forms P and Q was achieved to a satisfactory
degree. Comparisons between the frequency counts of content categories within a subtest indicate
that forms Q1 and Q2 were quite similar; four of the subtests had differing numbers of items per
category, none of which were more than two items per category differences. While content
category frequency differences were more numerous in comparisons of Forms P with Forms Q
than comparing the two Forms Q, the differences overall were still moderate in size.

The stylistic features and format of Forms Q] and Q2 are equivalent and closely compare
to Forms P in most respects, despite the antecedent considerations for item selection. Forms Q
correct response options were well balanced across all possible response choices. In some
instances, rearrangement of item responses, where rearrangement was not expected to have an
impact on examinee performance (some subtests use rules for arranging item options, such as
smallest number for option a, next smallest number for option b, etc. and a change would give a
clue to the examinee) was necessary to achieve this objective.

In summary, the Forms Q test development effort seems to have achieved the objective of
creating two parallel forms that are comparable to Form P1 in terms of item difficulty, item
discrimination, subtest content and stylistic considerations. The products of these efforts, the
Forms Q] and Q2, were used in the subsequent test evaluation phase in which preliminary and
operational equating tables were developed. These first of these phases, concerned with

developing preliminary equatings is discussed next.

10