OR FILE COPY (2

Research Report 1429

o ROTC Achievement Testing Program: N School Years 1983-1985

Fumlyo T. Hunter

SELECTE DTIC

APR 8198?

Leadership and Management Technical Area

Manpower and Personnel Research Laboratory

U. S. Army

Research Institute for the Behavioral and Social Sciences

August 1986

Approved for public release; distribution unlimitod.

874

""% % I Av
U. S. ARMY RESEARCH INSTITUTEL

FOR THE BEHAVIORAL AND SOCIAL SCIENCES

A Field Operating Agency under the Jurisdiction of the

Deputy Chief of Staff for Personnel

WM. DARRYL HENDERSON

EDGAR M. JOHNSON COL, IN

Technical Director Commanding'

Technical review by

William Haythorn

Clessen Martin

Laureli Oliver

Paul vah Rijn

N OT ICES

ISTRIBU" Primary ibution of report has been made by A Please addr ss corre.

spo Uoncernni ributi orts to: %r e I tute h eh

and Social Sciences, ATTN: PERI.POT, 5001 Eisenhower Ave., Alexandria, Virginia 22333.5600'?.

FINAL DISPOSITION: This report may be destroyed when it is no longer needed. Please do not

return it to the US. Army Research Institute for the Behavioral and Social Sciences,

NOTE: The findings in this report are not to be construed tis an official Depaitrnent of the Army N

position, unless so designated by other authorized docurnents.

. ,,* ,(* *.
UNCLASSIFIED

SECURITY CLASSIFICATION OF THIS PAGE ("omn Does fIrmered)

REPORT DOCUMENTATION PAGE READ INSTRUCTIONS

RNEFORE COMPLETING FORM

I, REPORT NUMBER 2. GOVT ACCESSION NO. 3. RECIPIENT'S CATALOG NUMBER

ARI Research Report 1429 J___,______,_.__

4. TITLE (wad Subtitle) S. TYPE OF REPORT & PERIOD COVERED

ROTC ACHIEVEMENT TESTING PROGRAM: SCHOOL YEARS November 1985-June 1986

1983-1985 6. PERFORMING ORG. REPORT NUMBER

7. AUTHOR(s) 6. CONTRACT OR GRANT NUMBER(a)

Fumiyo T. Hunter

9. PERFORMING ORGANIZATION NAME AND ADDRESS 10. PROGRAM ELEMENT, PROJECT, TASK

U.S. Army Research Institute for the Behavioral AREA & WORK UNIT NUMBERS

and Social Sciences, 5001 Eisenhower Avenue, 2Q263743A792

Alexandria, VA 22333-5600 451 7700

I1, CONTROLLING OFFICE NAME AND ADDRESS 12, REPORT DATE

U.S. Army Research Institute for the Behavioral August 1986

and Social Sciences, 5001. Eisenhower Avenue, Is, NUMBEROF PAGES

Alexandria, VA 22333-5600 38

14, MONITORING A9ENCY NAME A AOOI $SS(if dII tera,4 from. Controllind Oftflce) IS, SECURITY CLASS, (.1 this report)

Unclass if ied

13m. DECL ASSIItICATION/DOWNGRADINO

SCNLOULK

I6, DISTRIBUTION STATEMENT (.a this Report)

Approved for public release; distribution unlimited.

"17. DoiTRI•UTION STATEMENT (of the ab.tacti entered In Block 20t It different from Report)

It, SUPPLEMENTARY NOTES

The ARI, portion of this research was directed by Dr. Laurel W. Oliver.

WS, KEY WORDS (Continue on favares side Ii nBe°Ceary aid Identify by block number)

-Achievement testing, -- , Diagnostic screening,

Basic skills, ROTC

Commissioning standards,

20. ASIST'MACT (mse-.r m so reverse t it iew -e *y adm Idowlty by block numb&l)

>The Office of the Deputy Chief of Staff for Reserve Officers' Training

Corps administered English, reading, and mathematics tests to Military Science

I (MS I) and MS IV cadets to develop systematic information for formulating

diagnostic and selection standards. The Army Research Institute analyzed the A

data. This report describes the score distributions for each MS class by

gender, ethnic group, and academic major. There was little difference between

average scores of males and females. However, the scores varied substantially

(continued)

DD , 1473 EDITION Of I NOV 6 IS O&SOLETE UNCLASSIFIED

SECURITY CLASSIFICATION OF THIS PAGE (Whrem Dae. BEulred)

j. %A" ý6

-~ ** r d,~,*~* %~ *.NoO
UNCLASSIFIED

SECURITY CLASSIFICATION OF THIS PAGC(Wh'hn Daia Entored)

ARI Research Report 1429

20. (Continued)

. among ethnic groups. The findings indicate that further information is needed

"before diagnostic and selection standards using these or other achievement tests

can be set.

NTIS' CRA•I'--''-

D I IC "I AB / -

DIS 1 R•t & fl

........................- .b .r Avoilia bliy Codes. . -. -

Avail Jrd/

UtJCLAS SIFIED

i ECUMITY CLASSIFICATION OF THIS F'AGE(Wh1en Date Prit.rod)

LP

•,•:E,.•¢••.,,•.,•,•;•• •,,,,.•.•, •.,•,•j•,•,.,.•1,• •,•..'.,•.)%.3,]c.V's%.•,•5,.,,,, ,,,. .,"•" v,9

,li , .r !"'-• .L;•! r,• • 'd .•..,',.,,q-•,.¥' ,• • •, v• ,f' -' .r•.l • , ,,," w• •,,,, ,•,•w • '• -• ,'il',• : , ' I,, •, ,''- i','
Research Report 1429

ROTC Achievement Testing Program:

School Years 1983-1985

Fumiyo T. Hunter

1.

I,,

Leadership and Management Technical Area

William W. Haythorn, Chief

Manpower and Personnel Research Laboratory '

Newell K. Eaton, Director

U.S. ARMY RESEARCH INSTITUTE FOR THE BEHAVIORAL AND SOCIAL SCIENCES

5001 Eisenhower Avenue, Alexandria, Virginia 22333-5600

Office, Deputy Chief of Staff for Personnel

Department of the Army

August 1986 1%

Army Project Number Manpower, Personnel, and Training

2Q263743A792

Approved for public release; dlstrlbution unlimited.

Is,

I ii,•
ARI Research Reports and Technical Reports are intended for sponsors of

R&D tasks and for other research and military agencies. Any findings ready

for implementation at the time of publication are presented in the last part

of the Brief. Upon completion of a major phase of the task, furmal recommendations for official action normally are conveyed to appropriate military

agencies by briefing or Disposition Form.

' .4

iv

&..
FOREWORD

The Leadership and Management Technical Area of the Army 1?.catrch lnstitute for the Behavioral and Social Sciences (ARI) conducts research addres'.-

ing the issues of Army leader assessment, training, and performance. since

nearly 75% of new officers commissioned each year are trained through the

Army Reserve Officers' Training Corps (ROTC), research on the ROTC training

program has great potential for enhancing officer performance.

In 1983, the Deputy Chief of Staff for ROTC initiated the Achievement

Testing Program to assess the levels of basic skills of ROTC cadets with the

objective of developing standardized diagnostic and selection procedures.

ARI was asked to perform the data analysis.

This report describes the results of the analyses in terms of the distribution of achievement test scores for Military Science I (MS 1) and MS IV

classes as a whole and for subgroups of cadets. The report also indicates

additional types of information needed for ROTC to formulate diagnostic and

selection standards as well as the organizational, social, and political

issues that must be addressed in this process.

EDGAR M. JOHNSON

Technilcal Director
ROTC ACHIEVEMENT TESTING PROGRAM: SCHOOL YEARS 1983-1985

EXECUTIVE SUMMARY

Requirement:

The 1978 report of the Review of Education and Training of Officers (RETO)

recommended a standardized screening procedure to be established for precommissioning programs. The Deputy Chief of Staff for ROTC (DCSROTC), Training and

Doctrine Command (TRADOC), initiated a project to assess basic skills of ROTC

cadets. The ODCSROTC requested the Army Research Institute (ARI) to analyze

achievement test data for School Years 1983-85 and to report findings that

might be used by ROTC to develop diagnostic and screening standards.

Procedure:

ROTC administered the Missouri College English Test, the Nelson-Denny

Reading Test, and the Stanford Achievement Test: High School Mathematics

Test to Military Science I (MS I) and MS IV cadets. Nearly the entire population oý MS IV cadets participated, while the participation rate for MS I

was a little better than 50%. However, for all classes tested, the proportional representation by region and gender closely approximated the percent.-

age breakdowns for these subgroups of cadets reported by the ROTC Headquarters

for the respective school years and MS classes.

This report is based on data from School Years (SYs) 1983-84 and 1984-85.

The distributions of test scores were examined within each MS class by gender,

racial/ethnic group, and academic major.

Findings:

Females scored higher than males in English; males scored higher than

females in mathematics; and there were no substantial gender differences in

reading scores. However, on all tests, the ethnic minority groups scored

lower than the white group; the differences were substantial for the black

and Hispanic groups. The average scores of MS I cadets were consistently

lower than MS IV scores possibly due to attrition of lower achieving students,

selection applied at conitracting between MS II and MS III, and/or greater

experience and education of MS IV cadets. Based on scores that might be used

as possible cut scores for MS IV, the analyses identified about 20% of MS I

cadets who might profit from remedial training in English and reading.

vii
The academic majors were grouped as engineering/architecture, physical

sciences, social sciences, and "other." The first three groups scored higher

than the other group in all tests. The first three groups did not differ

greatly in reading and writing skills, though the social science group scored

lower than the other two groups in mathematics.

Utilization of Findings:

The data from this project provide comparisons of average test scores

among subgroups of cadets. However, within the scope of this project, it was

not possible to examine the relationship between precommissioning measures of

academic achievement and officer job performance. In order to ensure adequate

levels of basic skills of ROTC graduates and to meet the increasing officer

production requirement, further research and considerations of organizational,

social, and political issues are needed.

ivi

A

vii,

.'•
ROTC ACHIEVEMENT TESTING PROGRAM: SCHOOL YEARS 1983-1985

CONTENTS

Page

BACKGROUND . . . . . . . . . . , . . . . . . . . . . . .- . . . . . .. 1

Objectives of the Achievement Testing Program . . 6 . . . ... .. 2

Purpose of Report . . . . ..... .. .. ... . ... . ...................... 2

PROCEDURE . . . . . . . . . . . . . .. . .......... . . . . . . . . . 2

Achievement Tests Used in the Project . . . .. ..... . 2

Test Administration Procedure . . . . .............. ..... 4

Demographic Description of Cadets

Participating in the Achievement Testing Program ........... . 5

RESULTS . . . . . . . . . . . . . ... . . ........... .. . . ........ 9

MS IV Achievement Test Results . . . ....... ................ . 10

MS I Achievement Test Results . . .&. . . . .# * ..s. . . . .*. 18

Comparison of Average Scores by Academic Majors . v . . . . . 22

SUMMARY AND CONCLUSIONS . . . . . . . . . . . . . t 24

REFERENCES . . . . . . . . . . . . . . . . . . . a a 26

APPENDIX 1: ROTC REGION JURISDICTION . . . ........... ..... 27

LIST OF TABLES

Table I. Reliabilities of achievement tests ............. 4

2. ROTC Achievement Testing Program testing schedule

and number of cadets tested ...... .................. 5

3. Demographic composition of MS I cadets participating

in the achievement testing program (Percentage of

total tested) . . . . . . . . . . . .. . . . . . .

4. Demographic composition of MS III and MS IV cadets

p;trticipating in the achievement testing program

(Percentage of total tested) . . . ........ ............... 8%

iad,

'I.
CONTV'NTS -(Continued)_

Page

LIST OF TABLES (Continued)

Table 5. Average raw and percentile scores for Missouri College

English Test, MS IV, SY 83-84 and SY 84-85 . . . .

6. Cumulative number and percentage of MS IV cadets

scoring above various scores on Missouri College

English Test by ethnic group . . . . . . . . . . . . . . . . 11

7. Average raw and percentile scores for Nelson-Denny

Reading Test, MS IV, SY 83-84 ............... 12

8. Cumulative number and percentage of MS IV cadets

scoring above various scores on Nelson-Denny

Reading Test by ethnic group. . . . . . . . . . . . . . . . 13

9. Average raw and percentile scores for Stanford

Math Test, MS III, SY 83-84 ................ 14

10. Cumulative number and percentage of MS III cadets

scoring above various scores for Stanford Mathematics

Test by ethnic group . . * . . #. . . *. . *. . . . .. e. 14

11. Correlations between Achievement Test Scores . . . . . . . 16

12. Percentage of minority-group cadets exceeding

the 10th and the 20th ROTC percentile points

for each test and combinations of tests .. .......... ... 17

13. MS I and MS IV raw score means (SY 84-85 data) ....... ... 19

14. Percentages of MS I and MS IV cadets passing a

given score on Missouri English and Nelson-Denny

Reading Tests by ethnic group (SY 84-85) .......... 20

15. Percentage of MS I cadets in score ranges close

to a possible cut score (SY 84-85) .... ............ ... 21

16. Average raw and ROTC percentile scores by

college major. . . . . . .. . . . .. . . ..................... 23

x

X 3⁄4.
ROTC Achievement Testing Program:

School Years 1983 - 1985

Background

An important aspect of precommissioning officer training is to ensure that

all newly-commissioned Army officers have attained an adequate level of basic

skills. The Army Reserve Officers' Training Corp6 (ROTC) produces nearly 75%

of new officers commissioned each year, and this production mission is

expected to increase in the coming years. The basic skills assessment arid

development effort in the ROTC program could make an important contribution to

ROTC training effectiveness as well as officer utilization.

The 1978 report of the Review of Education and Training of Officers (RETO)

recommended that a standardized method be established to screen and evaluate

officer potential of candidates for precommissioning programs. The

recommendation was followed by the Precommissioning Assessment System (PAS)

(U.S. Department of the Army, 1978) which identified five components

considered to be important to military leadership: academic, physical,

medical, psychological, and leadership potential. PAS also established

initial guidelines for assessing each of these components for the purpose of

admitting, or contracting, cadets into ROTC Advanced Course.

The only standard currently used to assess cadets' academic competence

for commissioning is a college degree and grade point average of 2.0 or above.

However, grading standards vary greatly among universities and colleges.

Thus, a need for further efforts in refining evaluation standards was

reiterated in the 1984 report of Army Science Board Summer Study (U.S.

Department of the Army, 1984).

Responding to this need, the Deputy Chief of Staff for ROTC (DCSROTC),

the Training and Doctrine Command (TRADOC), initiated a project in 1983 to

establish diagnostic and screening standards for standardized achievement

tests. The plan was to administer achievement tests during three consecutive

school years to ROTC cadets in Military Science I (MS I) and Military Science

IV (MS IV) classes. Based on the configuration of the scores obtained from

this initial phase of the project, ROTC was to formulate a standardized method

of assessing and developing basic skills. ODCSROTC requested the Army

Research Institute (ARI) to analyze the achievement tsst data for School Years

(SY) 1983--86 to provide a description of cadet performance which would be used

by ROTC in setting diagnostic and screening standards.

1y
Objectives of the Achievement Testing Program

The Achievement Testing Program was designed to serve two purposes:

1. To establish guidelines for diagnosing basic skills ieficiencies

among cadets in the ROTC Basic Course (MS I and II' ýor the

purpose of directing those with marginal or poor p,, *ormance to

receive remedial training.

2. To set criteria for proficiency in reading, writing, and

mathematics for admissions to the Advanced Course of ROTC (MS

III and IV) and for commissioning.

The immediate objective for the achievement testing in school years 1983

through 1986 was to catablish an empirical basis for formulating preliminary

standards for diagnosis, screening, and commissioning which might be adjusted

when test results from subsequent years were examined.

Purpose of Report

The purpose of this ieport is to present results from the achievement test

data analysis. It will describe test performance of ROTC cadets as a whole,

comparisons among gender and ethnic subgroups, and how many cadets passed

various score points.

Setting selection standards is a highly complex process. Besides the test

scores, many other issues must be taken into consideration such as the

relationships between basic skills and officer job requirements, the number of

new officers needed, resources needed to continue testing, and the social/

political implications of using cut scores for selection. Much of the

information needed to address these issues is currently lacking, requiring

further empirical research.

The results presented in this report will provide one of numerous pieces

of information required, namely the test performance of ROTC cadets. However,

they alone will not be sufficient to specify sound guidelines for selection or

diagnostic procedures.

Procedure

Achievement Tests Used in the Project

ROTC selected three commercially available achie.vement tests to be used

for this project. Each test provides national "norms," i.e., performance of a

large sample of students across geographical areas, as a basis for comparison.

Missouri College English Test (Callis & Johnson, 1965). This test was

designed to assess knowledge of "the mechanics and effectiveness of written

expression" (Callis & Johnson, 1965, p. 3). It is a 90-item multiple-choice

test, requiring no actual writing. The test was standardized based on data

2
from college freshmen attending universities and two- and four-year colleges,

both private and public, from all regions of the United States, during the

early 1960s.

Two issues need to be addressed in using this test: (1) The national

norm was established for college freshmen only since the test was intended as

a diagnostic tool to identify freshman students in need of remedial training

in writing. When the test is administered to more advanced college students

in higher college grades, as in the ROTC Achievement Testing Program, the

interpretation of raw scores according to the national norms may be less

accurate. Older students may be likely to score higher, due to greater

experiences and learning opportunities and/or student attrition, part of

which may be based on lack of basic skills.

(2) The original norming sample was adequate in number and regional and type-of-school representation. however, this was the only standardization performed for this test on a national sample. Since then, educational

achievement levels have shifted (EcGeever, 1983). It may, or may not, be

accurate to judge writing abilities of the 80s students (even college

freshmen) against the performance of freshmen in the early 60s. As described

in the subsequent section, these problems were partially dealt with by

computing ROTC norms based on the data collected for this project.

The Nelson-Denny Reading Test (Brown, Bennett, & Hanua, 1981). This

test provides assessment of vocabulary knowledge, reading comprehension,

and reading rate. The two subtests (Vocabulary, Comprehension) can be used

selectively to diagnose students' weak areas. Scores from both subtests

combined (Total) indicate their overall reading achievement.

The national norms for this test were developed in the late 70s, based on

scores from students in 9th through 16th grades (high school and college).

Separate norms are published for each of the 8 grades included, so that, for

example, a college senior's score can be compared to a national sample of

college seniors. Interpreting the ROTC students' performance against the

national norm is more accurate with this test than with the Missouri English

test, because the standardization was done fairly recently and sel-irate norms

for each college grade are available.

Stanford Achievement Test: High School Mathematics Test (Gardner, Merwiln,

Callis, & Madden, 1965). This test consists of three parts: a Numeric

Competence subtest, which assesses students' general numeric and arithmetic

capabilities; a Mathematics Part A subtest, which contains problems covered in

high school elementary algebra and geometry textbooks, and a Mathematics Part

B subtest, which tests knowledge of advanced algebra, trigonometry, and some

new mathematics concepts from high school textbooks. For the purpose of

diagnosis for remediation, the test writers suggest using the Numeric

Competence subtest score.

The two problems mentioned with reference to the Missouri English test

also apply to this test. The test was standardized based ou students iu high

school grades (9th through 12th); the reference group which is closest to ROTC

cadets is the college preparatory, high school seniors. In additiou, the

3

0% *1
norming data were collected in the early 60s. Thus, interpreting the

performance of ROTC cadets, especially college juniors and seniors, according

to these norms may not be very accurate.

Reliability and validity of the achievement tests. The reliability of

these tests reported in the manuals are all in the .90s range and indicate

adequate stability of scores on these tests. Table 1 presents the reliability

coefficients and methods of deriving them for each test. The content

validity, i.e., the degree to which test items cover the subject matter, was

investigated by test developers for each test and reported to be satisfactory.

However, the question of how accurately these tests may predict college

graduates' job performance, specifically in the military setting, has not been

addressed empirically.

Table I

Reliabilities of Achievement Tests*

Reliability Sample Method

Test Coefficient Grade Number Used

Missouri College .94 13th 12,580 Split-half

English Test corrected by

Spearman-Brown

Nelson-Denny .93 13th & 239 Alternate forms

Reading Test 14th correlations

(Total scores)

Stanford-Mathematics 12th 4,897 Split-half

Numeric Competence .92 corrected by

Mathematics Total .94 Spearman-Brown

* Extracted from Callis and Johnson (1965) for English, Brown et al. (1981)

for reading, and Gardner et al. (1965) for mathematics tests.

Test Administration Procedure

The ROTC-wide administration of the three achievement tests for the

purpose of developing diagnostic and screening standards was carried out

over three consecutive school years beginning with SY 83-84. During SY 83-84,

the English and the reading tests were administered in the spring semester to

MS IV classes of mostly juniors and seniors. In addition, the reading test

was given to MS I cadets, mostly freshmen. In SY 84-85, the English and the

reading tests were administered to MS I and IV classes in the spring. The

4
mathematics test was adminiatered in the Advanced Camp duriug the summers of

1984 and 1985. In the last year of the research project (SY 85-86), data for

EnSlish and reading were collected from MS I and MS IV cadets in the fall.

This report is based on data from the first two years. Table 2 presents

the data collection achedule, the military science classes included, and the

nutaber of cadets who participated in the testing.

Tests were administered by the cadre in each ROTC detachment and then

forwarded to the test publishers for scoring. The. coded data on magnetic

tapes were sent to ARI from ROTC Headquarters for analysis.

Table 2

ROTC Achievement Testing Program

Testing Schedule and Number of Cadets Tested

Tests and Class

Time of Administration MS I MS III MS IV

SY 83-84 Spring Semester

Nelson-Denny Reading 20,218 7,524

Missouri English 7,365

Summer Advanced Camp

Stanford Mathematics 8,221

SY 84-85 Spring Semester

Nelson-Denny Reading 18,930 7,306

Missouri English 16,001 7,558

Summer Advanced Camp

Stanford Mathematics 8,265

Demographic Description of Cadets Participating in the Achievement Testing

Program

Tables 3 and 4 present the total number of cadets taking each of the

achievement tests during the two school years and the percentages of subgroups

by ROTC region, gender, racial/ethnic groups, and academic major categories.

5

%k. "'.

. ,.7
Table 3 describes the MS I students who took the English test in SY ý4-85

and the reading test in Sis 83-84 and 84-85. The totrl number of cadets

included in each of these data sets represents approximately half of the

end-of-school-year MS I enrollment for the respective years. The breakdown

of cadets across ROTC regions reflect the regional distribution of ROTC

detachments; most of the cadets attend schools in Regions I, II, and LII.

(See Appendix 1 for the states included in each ROTC region.) For the test

data, the regional representation remained fairly stable over the two years

With one exception of some decrease in cadets from Region III in SY 84-85.

Cadets identified themselves into one of the following ethnic groups:

(1) Asian/Pacific Islander, (2) black - Negro, not of Hispanic origin,

(3) Hispanic, (4) white - Caucasian, not of Hispanic origin, (5) American

Indian/native Alaskan, and (6) Other. In the analyses and tables reported

below, the American Indian/native Alaskan group, representing about .5% of

every data set, was combined with the Other group. In terms of racial/ethnic

backgrounds, approximately 73% of MS I cadets were white, 21% black, 3%

Hispanic, and 1.5% Asian/Pacific Islanders. For both school year3, about 28%

of the MS I cadets were females.

The mathematics test was administered in Advanced Camp to cadets who had

completed MS III classes and were to begin MS IV in the fall. For the sake of

simplicity, the Advanced Camp cadets who took the mathematics test and the 1S

IV cadets who took the English and the reading tests will be designated "MS

IV" cadets in the remainder of this report.

The MS IV data sets included much higher proportions of the end-of-schoolyear enrollment figures, ranging from 88 to 1009 with a mean of 96%. The

largest proportion of MS IV cadets tested came from Region I followed by

Regions II, 1IIp and IV, conforming to the regional distributions reported in

the ROTC Enrollment Reports. Compared to the MS I data, the relative

representation of the MS IV cadets from Region IV was considerably greater,

while the relative representation of Region III cadets was sf'aller.

Percentages of males and females in the MS IV data remained stable over

the two school years at approximately 16% females and 84% males. However,

compared to the MS I data, the relative proportion of females was reduced to

nearly half, from about 28% to about 16%. This suggests that a large number

of female cadets might not continue in the ROTC program at the contracting

time.

The racial/ethnic backgrounds of the MS rV cadets were: 73-76% white,

16-18% black, 4-5% Hispanic, and 2% Asian-Pacific Islanders. The proportion

of the black cadets was lower, and that of the Hispanics was slightly higher,

than in MS I classes.

6
Table 3

Demographic Composition of MS I Cadets Participating

in the Achievement Testing Program (Percentage of Total Tested)

Reading data Reading data English data

Group SY 83-84 SY 84-85 SY 84-85 _N-20,218 N=18,930 Nw16,001

ROTC Region

Region I 32% 36% 36%

Region 11 29 32 29

Region III 32 25 26

Region IV 7 7 8

Gender

Female 27 28 28

Male 73 72 72

Racial/Ethnic Group

Asian 1 2 2

Black 22 21 20

Hispanic 3 4 3

White 72 73 73

Other 2 1 2

Academic Majors

Engineer/Architect 13 12 12

Physical Science 15 14 14

Social Science 12 13 12

Medical 7 7 8

Other 54 55 54

05

7

%'

,,,",,.w "
Table 4

Demographic Composition of MS III and MS IV Cadets Participating

in the Achievement Testing Program (Percentage of Total Tested)

Reading Reading English English Math Ma th

Group SY 83-84 SY 84-85 SY 83-84 SY 84-85 SY 83-84 SY 84-85

N-7,524 N-7,306 N-7,365 N-7,558 N-8,221 N-8,265

ROTC Region

Region I 43% 44% 42% 43% 39% 43%

Region II 21 25 22 24 25 24

Region 111 19 17 20 18 20 19

Region IV 16 15 16 14 16 15

Gender

Female 16 16 16 17 15 15

Male 84 84 84 83 85 85

Racial/Ethnic Group

Asian 2 2 2 2 2 2

Black 18 16 18 16 18 18

Hispanic 4 6 5 4 5 5

White 75 76 73 76 73 73

Other 1 1 2 2 2 2

Academic Majors

Engineer/ArchiLect 9 12 9 11 11 12

Physical Science 15 16 14 15 15 16

Social Science 32 30 31 28 28 27

Medical 4 4 4 5 4 4

Other 40 38 41 41 42 41

The proportions of cadets by academic majors are also indicated in Tables

3 and 4. Academic majors were grouped, guided by the grouping procedure used

by the Military Personnel Center (MILPERCEN) Officer Distribution Division as

follows: Engineering/architecture, physical science (computer science,

physical science, natuý:al science, math/statistics), social science (social

science, psychology), medical (pre-med, nursing), other (business/accounting,

management, education, letters, fine arts, and undeclared).

The engineer/architectuce group made up about 10% of MS I and IV classes.

Likewise, the percentage of cadets in physical sciences was fairly consistent,

at about 15% of all classes tested. On the other hand, the percentage of

social science majors differed; about 12% for MS I classes and about 30% for

MS IV classes. The "Other" group constituted the majority, 55% for MS I

classes and about 40% for MS IV classes. The pre-med and nursing students

8

'I.
were combined, even though their academic programs may differ subsrantially,

since together they make up less than 10% of the ROTC population (basic and

advanced courses), and many of them are commissioned through the special

appointment avenue.

To summarize the demographic composition of the cadets participating in

the achievement testing, the proportions representing the ROTC regions,

genders, and racial/ethnic groups remained fairly constant over the two school

years. Nearly the entire population of MS IV cadets participated. For MS I

testing, the participation rate was a little better than 50%. However, for

all of these classes tested, the proportional representation by region and

gender closely approximated the percentage breakdowns reported by the ROTC

Headquarters for the respective school years and MS classes. The differences

between the sample subgroup percentages and the percentages reported in ROTC

Closing Enrollment Reports ranged from 0 to 5 percent, averaging at 1.6

percent. Comparing the demographic compositions between the MS I and IV

classes, some marked differences were noted. The proportions of females and

black cadets were lower in MS IV than in MS I.

Results

Data used for analysis were: the total score for the Missouri College

English Test, the total of the Vocabulary and Comprehension subtests for the

Helson-Denny Reading Test, and the Numeric Competence subtest score for the

Stanford Achievement Test of Mathematics.

The results of the analysis are presented in terms of the actual score a

cadet achieved on each test (raw score) and the ROTC and national percentile

scores. The raw scores are not comparable across the three achievement tests;

the maximum possible scores for English, reading, and mathematics are 90, 172,

and 45, respectively. Since a given raw score (e.g., 138 on the reading test)

in itself does not denote a certain level of competence, one might refer to

the percentile scores based on national samples. A percentile score indicates

the percentage of people who scored at or below a given raw score. However,

using the published percentile scores of these tests to infer ability levels

may not be appropriate since the English and the mathematics tests were normed

on test scores of college freshmen and high school seniors, respectively, both

in the early 1960s.

The ROTC percentile scores provide current, age-appropriate norms based on

the same ROTC reference group across tests. Using the ROTC percentiles, it is

possible to compare the relative positions of various subgroups, and also to

compare individual performance across the three tests. For example, an

English raw score of 37, a reading score of 92, and a mathematics score of 25

all correspond to the 20th ROTC percentile score for MS IV cadets for the

respective tests. This means that 20 percent of MS IV cadets scored below

these raw scores on these tests. On the other hand, despite the limitations

of the published national percentile scores, they do provide some basis on

which to compare the ROTC percentile scores against the performance of

students in general. For this purpose, both percentile scores are presented

in some of the tables.

9
The score distributions were highly similar over the two school years, as

exemplified in Table 5. For the remainder of the report, tabular and written

descriptions of the results will be based mainly on the SY 83-84 data. The MIS

IV test results will be presonted first, followed by the MS I test results.

MS IV Achievement Test Results

This section describes the results of the mathematics testing in Advanced

Camp (in the summer between the junior and the senior years) and the English

and the reading testing of MS IV cadets in the spring. Analysis of these data

was related to the issue of standards for contracting or commissioning. It

was guided by the question of what scores might be used as selection criteria

which would serve to retain the maximum number of cadets who possess

sufficient basic skills to perform their duties as Army officers. The

analyses were also directed at the differential effects that various cut

scores might have on gender and ethnic subgroups.

It must be emphasized here that the results reported below are not

intended to suggest or recommend any selection standard. The particular

ranges of scores examined in the following sections were chosen only for

illustrative purposes, i.e., to -show the patterns of score distributions for

the whole group and for subgroups.

Hissouri College English Test. For this test, Lhe maximum raw score is

90, and the national norm is based on college freshmen of the early 60s.

As shown in Table 5# the average raw score for the total group was 50 for both

years. Taking the SY 83-84 data, the average for females was romewhat higher

than for males, 54 and 49, respectively. However, more substantial

differences were found among the racial/ethnic groups. The black and Hispanic

groups scored substantially lower than the white group.

Table 6 shows the number of cadets in these ethnic groups who acored at

or above given scores on the English test. The figures In columns labeled n

indicate the actual numbers of cadets scoring at or above a given score; the

percentage iu parentheses indicates the proportion of the subgroup (e.g.,

white, black) that scored at or above this score. The "total" column

summarizes the results across the ethnic groups.

A raw score of 37 on the English test corresponds to the 20th MS IV

percentile point (meaning that approximately 80% of the total MS IV group

scored at or above this score). For SY 83-84, 88% white, 47% black, 58%

Hispanic, and 75% Asian cadets achieved a score of 37 or higher. In other

words, 20% of the total MS IV cadets failed to achieve this score, but some

ethnic groups were represented disproportionately in this 20%.

64

10 QV. 4.

Im

• ., • ,,.• .' ; •. ,,• . . • . ., .. • , .. . . . .• .• , • ,. ...... .•. .,.. .- , . . ... .

... '.. .. • ,• -• -,• '. ,,,,. •. ,.. .•.. , .. . ,., . . ... • . .r .".. . " *'... • " "" . .. .. . .. .. . ... .. ... .. . .. ... " *. 'l ... .. . . .t
Table 5

Average Raw and Percentile Scores for

Missouri College Fx.gllsh Test, MS IVt SY 83-84 and SY 84-85

SY 83-84 SY 84-85

Raw Percentile Scores Raw Percentile Scores

Score SD ROTC National Score SD ROTC National

To tal 50 15 49.6 40.2 50 15 49.5 40.4

Gender

Male 49 14 48.3 38.9 50 15 48.4 39.3

Female 54 15 56.5 46.7 53 15 55.2 45.7

Ethnic Group

Asian 48 14 44.8 36.0 49 15 46.2 37.4

Black 38 13 26.5 20.6 38 13 26.0 20.4

Hispanic 41 15 33.3 26.3 43 14 35.0 27.8

White 54 13 56.8 46.2 53 13 55.6 45.5

Other 46 16 42.9 34.4 48 15 45.6 36.9

* Standard deviation of raw scores.

Table 6

Cumulative Number and Percentage of MS IV Cadets Scoring Above

Various Scores on Misouri College English Test by Ethnic Group

ercentile White Black Higpnic Asian Oder Toht

Raw I Scores

ScoreiRl / Natioa1 a() _ (2) n (2) Wn n (2) (W) n (W)

SY 83 - 84

37 20 15 4737 (88) 629 (47) 213 (58) 116 (75) 101 (70) 5796 (79)

34 15 11 4929 (92) 747 (56) 240 (66) 121 (79) 109 (76) 6146 (84)

30 10 7 5116 (95) 914 (68) 270 (74) 137 (89) 116 (81) 6553 (89)

25 5 4 5269 (98) 1092 (82) 309 (84) 147 (95) 128 (89) 6945 (94)

21 2 2 5323 (99) 1190 (89) 330 (90) 149 (97) 133 (92) 7125 (97)

To tl 5359 (100) 1336 (100) 366 (100) 154 (100) 144 (100) 7359 (100)

NOTE: The "'oI.a" is less tdun the to1l number tsoo3d due tx missing deta.

11

- .4 '
Nelson,-Denny Reading Test. The possible raw score range for this test is

from 0 to 172. However, the national norm provides a fairly accurate basis to

interpret the raw scores relative to the performance of a national sample

since the norms were developed for each college grade in early 1980s.

The means for the total ROTC group remained stable over the two years at

the raw score of 119. Table 7 shows that the average scores for males and

females were very similar for this test, although large differences were found

amon8 the ethnic group means.

Table 7

Average Raw and Percentile Scores for

Nelson-Denny Reading Test, MS IV, SY 83-84

Raw Percentile Scores

Score SD* ROTC National

To tal 119 30 49.6 43.0

Gender Ma-l-e 119 30 49.9 43.3

Female 117 32 48.6 42.4

Ethnic Group

Asian 110 30 40.7 34.3

Black 87 26 21.1 16.0

Hispanic 96 34 30.8 25.6

White 128 24 57.5 50.8

Other 119 32 49.9 43.9

* Standard deviation of raw scores.

Table 8 displays the number of cadets, by ethnic group, passing various

score points. For SY 83-84, at the ROTC 20th percentile (raw scoce of 92),

91% of white cadets passed, but 39% and 55% of the black and Hispanic cadets,

respectively, passed. The performance of ethnic minority groupa was generally

poorer, when compared to the total group, on the Nelson-Denny Reading Test

than on the Missouri English test.

12

it.
Table 8

Cumulative Number and Percentage of MS IV Cadets Scoring

Above Various Scores on Nelson-Denny Reading Test by Ethnic Group

Wh a Black Hispanic Asian Other Total

Score R= Na tionaln% n (%)M a%) n(%) n(W n W%

SY 83 -8

92 20 14 5078 (91) 509 (39) 177 (55) 103 (72) 70 (74) 5977 (79)

86 15 10 5275 (94) 623 (47) 193 (60) 111 (79) 77 (82) 6315 (84)

77 10 5 5437 (97) 784 (60) 220 (68) 119 (84) 80 (85) 6677 (89)

64 5 2 5547 (99) 1031 (79) 252 (78) 129 (91) 88 (94) 7086 (94)

54 2 1 5591 (99) 1161 (88) 277 (85) 133 (94) 91 (97) 7294 (97)

Total 5607 (100) 1312 (100) 324 (100) 141 (100) 94 (100) 7524 (100)

Stanford Mathematics Test. As Table 9 shows, the overall raw score mean

on this test was 32 (with the possible score range of 0 to 42). The national

norms on this test are not very informative since they were developed in early

60s based on performance of college-prepara tory high school seniors. Males

performed considerably better than the females on this test. For SY 83-84,

the respective ROTC percentile means were 51.3 and 39.4. Table 9 also shows

the same pattern of ethnic group differences found with the other tests:

blacks and Hispanics scored substantially lower than whites.

Table 10 indicates that about 80% of the total ROTC group scored above a

raw score of 25. However, this group included 87% of the whites, 84% ol the

Asians, 44% of the blacks and 61% of the Hispanics.

13

%p
Table 9

Average Raw and Percentile Scores for

Stanford Math Test, MS 11, SY 83-84

Percentile Scores

Raw Score SD ROTC National

Total 32.1 8.1 49.5 43.2

Gender

Male 32.6 8.0 jl.3 44.9

Female 29.0 8.5 39.4 33.2

Ethnic Group

Asian. 33.1 7.2 52.4 45.8

Black 24.7 7.6 24.6 19.0

Hispanic 28.1 8.0 35.3 29.0

White 34.2 7.0 56.8 50.4

Other 30.9 8.3 45.1 38.6

* Standard deviation of raw scores.

Table 10

Cumulative Number and Percentage of MS III Cadets Scoring Above

Various Scores for Stanford Mathematics Test by Ethnic Group

White Black Hispanic Asian Other Total

Raw I Percentile Scom

ScoreI ROM Nsticoal n(Z) n(•%) n( M(Z) a M an MZ)

SY 83 - 84

25 20 14 5231 (87) 663 (44) 271 (61) 128 (84) 112 (74) 6405 (78)

23 15 10 5456 (91) 796 (53) 307 (69) 136 (90) 122 (81) 6817 (83)

20 10 6 5712 (96) 1018 (68) 356 (81) 140 (93) 132 (87) 7358 (90)

17 5 4 5847 (98) 1228 (82) 397 (90) 145 (96) 139 (92) 7756 (94)

14 2 2 5924 (99) 1350 (90) 422 (95) 150 (99) 144 (95) 7990 (97)

Total 5979 (100) 1492 (100) 442 (100) 151 (100) 151 (100) 8215 (10D)

Note: '1-m '"Ibl" is leis dwn t om t•oi nber tested due to missing daw.

14
The gender and ethnic group scores across three tests. There were gender

differences in two of the tests; females scored higher in English and lower in

mathematics than males. The average of males' English scores was not far

below the total average, mainly because males constituted close to 85% of the

total group. However, the females' average mathematics score was considerably

below that of the males.

The ethnic minority groups, especially the blacks and the Hispanics,

scored substantially lower tlan the white group in all skill domains tested.

For all ethnic minority groups, the percentage of cadets passing the 20th

percentile point was the lowest for the reading test. For black cadets, 39%,

44%, and 47% passed the 20th ROTC percentile points of the reading,

mathematics, and English tests, respectively.

Analysis of composite scores based on three tests. The objective of the

Achievement Testing Program was to determine basic skills qualification

standards. However, the realities of officer production mission and the

extent of differential impact of selection standards on subgroups must be

considered in this process. An individual's performance on any given test can

fluctuate, so that it is not judicious to base a critical decision concerning

his or her future career on one test score. In addition, many individuals do

not have the same degree of competence in all skill areas, but rather perform

better in one area than another.

Table 11 shows the similarities of individual performance on two tests,

e.g., between English and reading scores. The higher the correlations, the

more similar the performance on two tests. For the total group, English and

reading test scores are highly correlated (r - .70), but the associations

between English and mathematics and between reading and mathematics are only

moderate (r - .55 and .51, respectively). Many of the correlations between

the verbal-tests (English and reading) and mathematics are in the .40s range

for ethnic subgroups. These results suggest that some cadets might score

above the cut score (e.g., 40th ROTC percentile) on one test but below the cut

score on another, largely due to varying expertise in different skill areas.

15

W %.
Table 11

Correlations Between Achievement Teat Scores

English English Reading

aud and and

Reading Math Math

Total .70 .55 .51

White .63 .46 .38

Black .61 .45 .40

Hispanic .68 .53 .52

Asian .78 .48 .48

Other .71 .77 .63

Note: ROTC percentile scores were used for these analyses. Math data from

SY 83 - 84, MS III; English and reading dAta from SY 84 - 85, MS IV

classes.

If several test scores are used to evaluate cadets' overall competence in

basic skills, their strengths in one area may compensate for marginal scores

in another. This effect is shown on Table 12, which displays (in the first

three columns) the percentage of minority-group cadets passing the 10th and

the 20th ROTC percentile scores for the three tests separately. The fourth

column contains the percentages of cadets passing these points based on each

cadet's average of the English and the reading ROTC percentile scores. The

last column shows the percentages based on averages of each cadet's ROTC

percentile scores on all 3 tests.

16

"• ,:'i..'''.'.i.''',.: " "•" .",'.",'': • '.'•''.•.'' ,!." "-;.''. ''•''.,i ., .," , " " %''',.'%,',,'', " ',.-'''..°''. ''..'. %''•.I
Table 12

Percentage ROTC Percentile of Minority-Group Points for Cadets Each Test Exceeding and Combinations the 10th and of the Tests20th

English English Reading

English Reading Math and and and All

Only Only Only Reading Math Math Tests

10th Percentile

Black 66% 63% 68% 69% 75% 72% 75%

Hispanic 79 63 81 83 88 83 88

Asian 89 81 93 93 98 95 96

20th Pe'rcentile

Black 46 40 44 48 50 49 50

Hispanic 59 46 61 66 77 67 77

Asian 75 67 84 75 89 86 87

Comparing the results for separate tests and those for averaged data

indicate that the number of minority cadets who would pass a given cut point

would increase if multiple test scores are considered together. For example,

if the 20th percentile score on the English test alone were used as a

selection standard, only 46% of the black cadets would pass. If composite

scores based on all tests, or English and mathematics, were used, 50% would

pass. For the black group, which constitutes about 20% of the total MS IV

population, the difference of 4% or 5% currently amounts to 50-60 more cadets

to be commissioued.

Based on general knowledge about ability testing and the empirical

illustration made above, it seems that using a composite score for a selection

procedure would have the following advantages:

1) Multiple test scores are more reliable than one to represent an individual's overall abilities.

2) Since many individuals have different areas of strengths and since

a basic level of quantitative skills would be needed for many

officer jobs, both skill areas should be assessed.

3) If the English nnd mathematics scores are included in the

composite, the number of cadets passing the cut score is lJkely to

increase.

17

• - V* 'e~j~• ; ,. • -' "•'.'• v ." ".-, .,•. , .• ?•.>,• •. - " ". .. I
4) By increasing the number of cadets passing, this approach would

reduce the differential impact of selection process across the

ethnic minority groups.

However, currently there is no sound information on how well any of the

separate tests or composite scores would predict performance in Army officer

jobs. Analyses reported above suggest that additional research needed to

clarify these questions is warranted.

MS I Achievement Test Results

The purpose for administering the English and the reading tests to MS I

cadets was to examine the score distributions for new cadets which would, in

turn, provide a basis for establishing remediation guidelines. As with the MS

IV data, it was not possible to determine what level of reading and writing

abilities a given test score represented Pince there was no other measure of

these skills with which the achievement test scores could be compared.

Given this constraint, the following steps were taken: 1) compare raw

scores of MS I and IV cadets and 2) examine the number of MS I cadets

performing near the score which may be used as a cut score for selection

(contracting and/or commissioning). Raw scores were used for comparison since

percentile scores of MS I and MS IV groups are based on different reference

groups and are not comparable. The following comparisons are based on SY

84-85 data which included English and reading tests given to both MS I and IV

classes.

18
Table 13 shows the average raw scores of HS I and MS IV classes on English

and reading tests for each class as a whole, by gender, and by ethnic group.

The MS IV averages are consistently higher than the MS £ averages.

Table 13

MS I and MS IV Raw Score Means

(SY 84 - 85 Dta)

English Reading

MS I MS IV MS I MS IV

Total 46 (15) 50 (15) 101 (32) 119 (30)

Gender

Male 45 (15) 50 (15) 102 (32) 120 (30)

Female 48 (16) 53 (15) 99 (32) 117 (32)

Ethnic Group

Asian 47 (15) 49 (15) 101 (35) 111 (32)

Black 35 (13) 37 (13) 73 (26) 80 (27)

Hispanic 37 (16) 43 (14) 76 (35) 93 (34)

Whi t 50 (14) 53 (13) 110 (28) 128 (24)

Other 43 (16) 48 (15) 95 (34) 121 (33)

Note* Score range for the English test is 0 - 90.

Score range for the reading test ),a 0 - 172.

Standard deviations are indicated in parentheses.

Table 14 displays thL percntages of MS I cadets scoring at or above a

hypothetical cut score for MS LV cadets. The raw scores of 37 and 93, for the

English and the roading tests, respectively, were the scores at which 20% of

MS IV cadets in SY 84-85 failed. This table also shows a smaller percentage

of MS I cadets passing these hypothetical cut score points on both tests,

across ethnic groups.

19

'11
Table 14

Percentages of MS I and HlS IV Cadets Passing a Given Score

"on Missouri English and Nelson-Denny Reading Tests

by Ethnic Group (SY 84 - 85)

Raw Score White Black Hispanic Asian Other Total

English

MS I 37 80% 38% 46% 71% 61% 69%

MS IV 37 87% 46% 59% 75% 72% 78%

Reading

MS I 93 71% 20% 31% 55% 50% 58%

MS IV 93 90% 40% 46% 67% 75% 79%

Note: English raw score of 37 and reading raw score of 93 are the scores that

about 80% of MS IV cadets passed in SY 84 - 85.

It is difficult to determine why the MS IV groul1 scored higher than the MS

I group, especially since the data for the two groups are not longitudinal (i.e., not collected from the same group of people at two points In time).

Part of the difference may reflect attrition of low achievers from colleges

and ROTC programs between MS I and MS IV years. The attrition may be

voluntary or due to selection applied at contracting time. The difference may

also reflect more years of education ind greater expertise on the part of the

older cadets.

If these changes are Lhe primary reasons for the score differeuces between

MS I and MS IV cadets and If ROTC were commissioning the required number of

new officers each year, then a special effort for remediation Lraining may not

be necessary. However, currently there is a need to increase ONe number of

commissionees. If remediation programs could improve the basic skills of

marginally achieving cadets, it would enlarge the poal of *-adets &vailable for

commissioning.

An approach for establishing remadiation guidelines. As mentioned

earlier, there are no data available to determine the relationship between the

achievement test scores and performance in officer jobs. In addition, since

the Achievement Testing Program data are not longitudinal, the degree to which

MS I students might increase their basic skills without special remediation

training is not known. Civau this situation, the only feasible way to develop

preliminary guidelines for early remediation might be to use scores which

might be chosen as selection criteria as reference.

20

3 ~. Ja.
As an illustrative example, suppose that a cut score is set at the raw

score at which 80% of MS IV cadets pass. (Depending on the size of cadet pool

and officer production requirements, the cut score might be higher or lower.)

Even though the overall performance of the MS I cadets was lower than that of

MS IV classes, it is assumed that by the time of selection (for contracting or

commissioning), a cadet would be expected to pass the cut score point. It is

also assumed that those who scored somewhat above the cut score at time 1 (say

in MS I class) may not pass at a later time, since any individual's

performance on a given test can fluctuate over multiple testing. Further,

those who barely make the cut score would benefit from developing the basic

skill. Oa the other hand, those who score somewhat lower than the cut score

may also increase their skills through special training and pass the test the

second time.

The question is what proportion of MS I cadets fall in the score range of

data from SY 84-85, the MS I cadets were grouped into these score categories,

using as the "cut score point" the score where 80% of MS IV cadets passed.

Table 15 shows the percentage breakdowns by test.

Table 15

Percentage of MS I Cadets in Score Ranges

Close to a Possible Cut Score (SY 84 - 85)

Raw Score Range Percent

English

44 and above 57

37 - 43 15

30 - 36 12

29 and below 17

Reading

101 and above 50

93 - 100 9

80 - 92 14

61 - 79 15

60 and below 12

Note: English raw score of 37 and reading raw score of 93 are the

scores that 80% of MS IV cadets passed in SY 84-85.

21

NN
For English, 80% of MS IV cadets passed the raw score of 37 (the HS IV

average was 50). About 15% of MS I cadets scored 37-43. About 12% of hS I

scored 30 to 36. A score of 30 is at about the 10th percentile point for MS

IV cadets and would be an extremely low score an a selection standard. Of

course, students falling in the very low score category may possibly improve

skills to pass the cut score, with good instructions and high motivation. But

it appears particularly beneficial for cadets in the middle groups (somewhat

above and below the cut point) to be strongly encouraged to seek remedial

training.

For the Nelson-Denny reading test, the hypothetical cut score, based on

80% pass for MS IV Cadets (SY 84-85), is 93. The MS IV average is 119. The

score range that might be considered for remediation training is from 80 to

100. About 23% of MS I cadets fall in this middle range. On this test, about

12% of MS I cadets scored 61 or below, which is the range where about 5% of MS

IV cadets scored. It may be difficult for most students scoring in this range

to improve their reading skills enough to pass 92.

Comparison of Average Scores by Academic Majors

Table 16 presents average raw and ROTC percentile scores for HS I and

MS IV cadets by test and by academic major groups (i.e., engineering/

architecture, physical science, social science, medical, and other). Note

that raw score averages allow comparisons between MS I and IV cadets, while

the ROTC percentiles indicate the position of a given subgroup relative to the

total MS class. The means of MS IV class were based on data from SY 83-84 and

SY 84-85 classes combined.

2I

22 4

I'
Table 16

Average Raw and ROTC Percentile Scores by College Major

Engineer/ Physical Social

Architect Science Science Medical Other

Missouri English

MS IV (10) (15) (30) (04) (41)

Raw Score 52 52 51 54 48

ROTC Percentile 53 54 50 57 46

Nelson-Denny Reading

MS IV (10) (16) (31) (04) (39)

Raw Score 121 121 122 125 114

ROTC Percentile 52 52 53 55 44

Stanford Math

Ms 111 (11) (16) (27) (14) (42)

Raw Score 37 34 30 33 31

ROTC Percentile 68 57 43 55 45

Missouri English

MS 1 (12) (14) (12) (08) (54)

Raw Score 50 48 48 49 44

ROTC Percentile 57 53 53 54 45

Nelson-Denny Reading

Ms I (13) (15) (12) (07) (54)

Raw Score 110 103 106 103 96

ROTC Percentile 58 52 54 52 45

Note: Numbers in parentheses indicate percentage of each major group in

respective MS class.

Medical category includes pre-med and nursing majors.

For both MS 1 and IV classes, average scores on the English test were

fairly close awong the engineering, physical, and social science groups; the

group averages for al1l of these group- were 50th or above ROTC percentiles.

The "Other" group, which makes up the majority, scored lower. For MS IV

cadets, the reading averages of all groups except "Other" were very close, and

substantially higher than the "Other" average. This pattern is generally

repeated for the MS I data.

23
The mathematics test was given only to MS III cadets, and no basic and

advanced course comparison was possible. Howevez, this test revealed more

striking differences among the academic major groups. Engineering/

architecture students scored, as a group, substantially higher than the

physical science majors, who in turn scored higher than the social science and

"Other" groups.

Recall that the main difference in the academic major compositions between

MS I and IV classes was that the percentage of social science majors was

greater in MS IV than in MS I. In MS IV, this group performed as well as the

engineering and physical science groups on English and reading tests.

Summary and Conclusions

The ROTC Achievement Testing Program was designed to serve two purposes:

1) to establish guidelines for diagnosing weaknesses in basic skills and

2) to establish selection standards.

There were several constraints in the research design. Only achievement

test data were collected, with no other measure of academic achievement to

verify the meaning of scores on these tests. Coupled with outdated and/or

age-inappropriate national norms, interpretation of the "goodness" of any

given score was difficult. Different year groups of cadets were involved in

testing each year so that changes from MS I to MS IV years could not be

inferred.

Some of the major findings from the Achievement Testing Program are

summarized below. However, with the lack of many other essential data, the

findings reported in this report must be considered preliminary and partial

with reference to the issues of diagnostic and selection standards.

ROTC norms enabled comparisons of achievement levels among subgroups. For

both MS I and IV classes, gender differences were relatively minor, but there

were considerable disparities among ethnic groups across all tests. Using

composite scores, derived from averaging two or more test scores, may provide

a way to assess ovezall achievement level across various academic domains, to

increase some cadets' chances of meeting the selection criteria, and to

contribute towards ROTC officer production mission.

About 20% of MS 1 cadets scored somewhat above or somewhat below a score

that was arbitrarily selected as a possible cut score for selection. If these

cadets are informed of selection standards early in their college career and

seek additional training in their weaker areas, this may have a significant

impact on the overall quality and the number of ROTC-trained commissionees.

Analyses of academic majors s&iowed that the majority of cadets major in

subjects other than engineering/architecture, physical, and social sciences

and that they, as a group, scored lower than these groups. The engineering/

architecture and the two science groups did not differ greatly in reading and

writing skills, though the social science group scored lower than the other

two groups in mathematics.

24

6 Ow U26ý,a-jU
Ensuring the basic competency of new officers is an essential mission of

all precommissioaing programs. However, the Army ROTC also tas a mission to

increase the number of commissionees during the next several years when a

decline is expected in college enrollment. For example, ROTC projects

commissioning 7,500 new officers in FY87 with the requirement of 9,200 (U.S.

Department of the Amy, 1986, in preparation). iven with a basic skills

selection standard which disqualifies the lowest 10% of the prospective

cou•issionees, the rate of production shortfall would increase from 18% to

27%.

In order to approach systematically the multiple objectives of IOTC,

additional empirical research is needed. First, there must be a clear

knowledge of which, and how much, basic skills are needed for Army officer

jobs. Theno the effectiveness of basic skills development programs needs to

be assessed. Issues to be addressed include: Which basic skills are amenable

to developmental procedures? Which method is most effective and

cost-effective? What levels of deficiencies are most likely to be improved?

What are the rate and permanence of change?

One such effort was undertaken within the military context during the 60s

and early 70s. The focus of this project was on functional literacy among the

enlisted personnel and effectiveness of an experimental reading training

program which was intigrated into the military-job-skills training. In terms

of job-related reading skills, a substantial improvement was obtained (Sticht,

1975a, 1975b).

Unfortunately, these findings are not directly applicable to college

population, curriculum, and officer job requirements. In addition, although

basic skills remediation progzams are commonly offered at most colleges and

ulLiversities, systematic information concerning theii effectiveness is scarce.

Thus, examining the effectiveness of developmental procedures seems

warranted.

Finally, decisions regarding selection standards also must be based on

officer job analysis and knowledge of predictive relationships between various

basic skills mueasurements and officer job performance.

N

25

OL.
References

Brown, J. I., Bennett, J. Me, & Hanna, G. (1981). The Nelson-Danny Reading

Test: Examiner's Manual. Chicago: The Riverside Publishing Company.

Callils, R., & Johnson, W. (1965). Missouri College English Test: Manual for

Forms A, B, and C. New York: Harcourt, Brace and World, Incorporated.

Gardner, E. F., Merwin, J. C., Callis, R., & Madden, R. (1965). Stanford

Achievement Test High School Battery Manual. New York: Harcourt, Brace and

World, Incorporated.

McGeever, J. bi. (1983). The decline of standardized test scores in the

United States from 1965 to the present (Report NO. AEL-OP-012). Washington,

DC: National Institute of Education. (ERIC Document Reproduction Service

No. ED 252 565)

Sticht, T. G. (1975a). A progras of Army functional job reading training:

Development, implementation, and delivery systems (HumRRO Report No.

humRRO-FR-WD(CA)75-7). Alexandria, VA: Human Resources Research

Organization. (AD A012272)

Sticht, T. G. (Ed.). (1975b). Reading for working: A functional literacy

anthology. Alexandria, VA: Human Resources Research Organization.

U.S. Department of the Army. (1978). Review of Education and Training for

Officers (RETO): Implementation Rlan(Vo_ 2). Washington, DC: Office of

the Deputy Chief of Staff for Operations and Plans.

U.S. Department of the Army. (1984). Leading and manning Army 21, 1984 Summer

Etud. Washington, DC: Army Science Board. (AD A150444)

U.S. Department of the Army. (1986, in preparation). ROTC Study final report

(Volume 1). Washington, DC: Reserve Officers' Training Corps Study Group.

Sqm

26

%-_
APPENDIX 1

ROTC Region Jurisdiction

First ROTC Region, Fort Bragg, NC 28307

Connecticut Maine New York Vermont

Delaware Maryland North Carolina Virginia

District of Columbia Massachusetts Pennsylvania

Florida New Hampshire Rhode Island Puerto Rico

Georgia New Jersey South Carolina Virgin Islands

Second ROTC Region, Fort Knox, KY 40121

Illinois Kentucky Missouri Tennessee

Indiana Michigan Ohio West Virginia

Wisconsin

Third ROTC Region, Fort Riley, KS 66442

Alabama Kansas Mississippi Oklahoma

Arkansas Louisiana New Mexico Texas

Fourth ROTC Region, Fort Lewis, WA 98433

Alaska Idaho Nevada Washington

Arizona Iowa North Dakota Wyoming

California Minnesota Oregon Guam

Colorado Montana South Dakota American Samoa

Hawaii Nebraska litah

27