Technical Report 598

7-

Cognitive Processing Determinants

of Item Difficulty on the Verbal

Subtests of the Armed Services

Vocational Aptitude Battery

(V)

(14 Karen Janice Mitchell

Selection and Classification Technical Area

Manpower and Personnel Research Laboratory

II I

C

DC0 31984

U. S. ArmyE

Research Institute for the Behavioral and Social Sciences

November 1983

Approved for oublic release; distribution urnlimited.

84 11 20 '179
7W ° 77'

U. S. ARMY RESEARCH INSTITUTE

%: FOR THE BEHAVIORAL AND SOCIAL SCIENCES

A Field Operating Agency under the Jurisdiction of the

Deputy Chief of Staff for Personnel

L. NEALE COSBY

EDGAR M. JOHNSON Colonel, IN

Technical Director Commander

Techniral review by Acoession For

Michael G. Rumsey NTIS GRA&I

Clessen J. Martin DTIC TAB

Paul van Rijn Unannounced 0 Justification

By __ _ _

Distribution/----

Availability Codes

!Avatl and/or

IDist Special

NOTICES

DISTRIBUTION: Primary distribution of this report has been made by ARI.

Please address correspondence concerning distribution of reports to: U.S.

Army Research Institute for the Behavioral and Social Sciences , ATTN:

PERI-POT, 5001 Eisenhower Avenue, Alexandria. Virginia 2233-5600.

FINAL DISPOSITION: This report may be destroyed when It Is no longer

needed. Please do not return It to the U.S. Army Research tnstitute for

the Behavioral and Social Sciences.

NOTE: The findings In this report are not to be construed as an official

Department of the Army position, unless so designated by other authorized

documents.

- i -- - in. *i*liB 
UNCLASSIFID .

SECURITY CLASSIFICATION OF THIS PAGE (When Data Entered)

READ NrSTRUCTIONS REPORT DOCUMENTATION PAGE BEFORE COMPLETING FORM

1. REPORT NUMBER 12. GOVT ACCESSION NO 3. RECIPIENTS CATALOG NUMBER

Technical Report 598 217 __

4. TITLE (end Subttlde) S. TYPE OF REPORT & PERIOD COVERED

Cognitive Processing Determinants of Item

Difficulty on the Verbal Subtests of the

. Armed Services Vocational Aptitude ,. PERFORMING ORG. REPORT NUMBER *' , Battery

7. AUTNOR(m) S. CONTRACT OR GRANT NUMBER(q)

* Karen J. Mitchell

S. PERFORMING ORGANIZATION NAME AND ADDRESS 10. PROGRAM ELEMENT. PROJECT. TASK

U.S. Army Research Institute for the AREA & WORK UNIT NUMBERS

Behavioral and Social Sciences, 5001 2Q263731A792

Eisenhower Ave., Alexandria, VA 22333-5600

1t. CONTROLLING OFFICE NAME AND ADDRESS 12. REPORT DATE

U.S. Army Research Institute for the November 1983

Behavioral and Social Sciences, 5001 IS. NUMBER'OFPAGES

Eisenhower Ave., Alexandria, VA 22333-5600 42 14. MONITORING AGENCY NAME & ADDRESS(QI different from Controlling Office) IS. SECURITY CLASS. (o. this report)

Unclassi fied

IS&. DECL ASSI FICATION/ DOWNGRAOING SCHEDULE

1S. DISTRIBUTION STATEMENT (of Lhs. Report)

Approved for public release; distribution unlimited

17. DISTRIBUTION STATEMENT (of he abetrec entered in Block 20, Ii differnt fbon Report)

IS. SUPPLEMENTARY NOTES

19. KEY WORDS (Continue on reveree side It neces ry and Identify by block number)

* ASVAB Information Processing

Selection Cognitive Psychology

Classification Verbal Processing

Ability Testing Latent Trait Theory

I. ATrlACr ( .oa ensI revme . It N nemey ane Idenify by block nmber)

The purpose of this research was to identify the cognitive processing operations, stores, and strategies involved in performance on

the verbal subtests of the Armed Services Vocational Aptitude

Battery (ASVAB) 8/9/10. The project was conducted in two parts.

The research began with a review of recent work in verbal information processing. A model of verbal performance was developed as a

conceptual framework for identifying and operationalizing.process- _-

O 1e* 1473 EmoOo or I NOV 6%S11 OGSOLEU D AM 72.. UNCLASSIFIED"-"

SECURITY CLASStFICATION OF ThIS PAGE (Wbon Date Entered)

i

• -. .1 . .< . .' -o - •- . .- . . . . . . . ~: *.*:. . - %,
UNCLASSIFIED

SECURITY CLASSIFICATION OF THIS PAGE(Whm Data Xaeo00

'ing components. Theoretical and operational definitions were

drafted for cognitive processes thought to underlie performance on

the ASVAB Word Knowledge (WK) and Paragraph Comprehension (PC)

subtests. The operational definitions were used to rate the WK

and PC items on two ASVAB forms.

In Part two, item response data were modeled for eight groups

Fiscal Year 1981 Army applicants and accessions using Rasch and

Linear Logistic Latent Trait (LLLT) methods. The relative effects

of the operationalized verbal processes on item difficulty were

determined using the LLLT algorithms. For the WK subtests,

variables indexing the amount of infor ation presented by an item

and the usage frequency of the target a d response words had

sizable effects. Results for the PC sub\tests were inconclusive.

There was evidence that propositional derity and inference

* construction variables were related to difficulty for the PC

* items.

The relations of cognitive processing variables to item difficulty

for the ASVAB Word Knowledge and Paragraph Comprehension subtests

*were demonstrated. These results were used in subsequent analyses

of the predictive validity of assessed ability components to

successful performance in Army jobs. The model-building procedures were proposed for general use in assessing and documenting

cognitive nrocessing contributions to performance on tests and

test-like tasks. The procedures that were developed were proposed

for use in constructing industrial/organizational assessment

instruments that tap training- and job-relevant processing

abilities.

UNCLASSIFIED

SECURITY CLASSIFICATION OF THIS IMAGC(171@f Date Entered)

. .. . . . . . . . . . . . . . . . .

°.. . . . . . . . . . . . . . . . . . . . . . . . .. . o.
.- .- . ... -- _Z .

Technical Report 598

Cognitive Processing Determinants

of Item Difficulty on the Verbal

Subtests of the Armed Services

Vocational Aptitude Battery

Karen Janice Mitchell

Submitted by Newell K. Eaton, Chief

Selection and Classification Technical Area

Approved as technically adequate and submitted for publication by

Joyce L. Shields, Director

Manpower and Personnel

Research Laboratory

U.S. ARMY RESEARCH INSTITUTE FOR THE BEHAVIORAL AND SOCIAL SCIENCES

5001 Eisenhower Avenue, Alexandria, Virginia 22333

Office, Deputy Chief of Staff for Personnel

Department of the Army

November 1983

Army Project Number Manpower and Personnel

2Q263731A792

Approved for public release; distribuion, unlifnited.

i-ll -~i

_ ' _._.e'' -.• = ._z_. ' ._.-'' * ; - ...._ -' -.", _ ."." _..".-_.- . .'_ '-. _ -.. ,- .'.'.. -'_. . . ._.'% .', 2 . , .. . ..,/
ARI Research Reports and Technical Reports are intended for sponsors of

R&D tasks and for other research and military agencies. Any findings ready

for implementation at the time of publication are presented in the last part of the Brief. Upon completion of a major phase of the task, formal recom- mendations for official action normally are conveyed to appropriate military

agencies by briefing or Disposition Form.

iv.

. .,-...... ..... ..... - . - .] 
FOREWORD

The Selection & Classification Technical Area of the Army

Research Institute for the Behavioral & Social Sciences (ARI) is

concerned with developing effective procedures for the selection of

applicants into military service and for the classification of

accessions into Army occupational specialities. The purpose of this --

research was to examine and document the construct validity of the

verbal subtests of the current Department of Defense military selection and classification battery, the Armed Services Vocational

Aptitude Battery (ASVAB) 8/9/10.

EDGAR M. JOHNSON

Technical Director

av

'...

I.:::

... I

.............................................. ...
COGNITIVE PROCESSING DETERMINANTS OF ITEM DIFFICULTY

ON THE VERBAL SUBTESTS OF THE ARMED SERVICES

VOCATIONAL APTITUDE BATTERY

.BRIE

Requirement:

To identify the cognitive processing operations, stores, and

strategies involved in performance on the verbal subtests of the

Armed Services Vocational Aptitude Battery (ASVAB) 8/9/10.

Procedure:

Cognitive dimensions thought to underlie performance on the

ASVAB Word Knowledge (WK) and Paragraph Comprehension (PC) items

were operationlized; the verbal items on two ASVAB forms were rated

on these variables. The relative effects of the cognitive dimensions on Rasch item difficulties were assessed for eight groups of

FY81 Army applicants and accessions using Linear Logistic Latent

Trait methods.

Findings:

Analyses suggested that cognitive processing variables were

related to item difficulty for FY81 applicants and accessions on the

ASVAB 8/9/10 verbal subtests. The LLLT models predicted from 17% to

30% of the variance in the item difficulty values estimated by the

Rasch models for the WK items. The results for the PC items were

inconclusive.

Utilization of Findings:

The relevance of cognitive processing variables to item difficulty for the ASVAB 8/9/10 WK and PC subtests was demonstrated.

Further examination of the predictive utility of these and similar

.; constructs to success in Army training and military jobs is needed.

" Analyses of this type will enable assessment and documentation of the

construct validity of ASVAB subtests. These methods can be used to

develop item set.s so that specified training- and job-relevant

processing abilities are tapped.

vii :

"fo°•~~~~~~~.... .° . .. -.. .......... o.. o.. . . -.. . . . .. . . . . .- o-. - . ..... . -.. .-.--. . .•....." . . '._"..'.. . . . ...... " _ , _- ,. . ...... ' ...-.-" , . ."*,- . . .-. . '. ' . .. 2 .. .- . ._.. '"". . .-. '-. . . . .. .. .-. ; ". ; '.'. - .,..•.
COGNITIVE PROCESSING DETERMINANTS OF ITEM DIFFICULTY

ON THE VERBAL SUBTESTS OF

* THE ARMED SERVICES VOCATIONAL APTITUDE BATTERY

CONTENTS

Page -.

ABSTRACT ......... ........................ .. 1

INTRODUCTION ........ ...................... .

METHOD..................... .... 2

The Armed Services Vocational Aptitude Battery . . . 3

The Word Knowledge Subtest .... ... ........ 3

The Paragraph Comprehension Subtest .... ........ 3

Part One ........... . . .. 8

The Development of a Model ;f V;rbai Performance 8

WK and PC Item Rating Scheme ... .......... .. 12

Part Two ......... ...................... .14

RESULTS ........................ 16

Development of the Word Knowledge Model ....... 16

Cognitive Component Variables ... ........... .. 21

Form Differences ....... ................. .. 22

Group Differences ....... ................. .. 22

Development of the Paragraph Comprehension Models . • 23

DISCUSSION ......... ....................... 28

Relevance of Cognitive Processing Variables to Item

Difficulty ................ ..... .28

Determinations of the Parallelism of the Operational

Forms ......... ....................... .. 28

CONCLUSION ......... ....................... .. 29

REFERENCES ......... ....................... 31

ix I -

= .°". -. -=*
LIST OF TABLES Page

Table

1. ASVAB 8/9/10 Subtests........ ... . . . 4

2. Instructions and Sample Items from the Word

Knowledge Subtest.......... .... . 6

3. Instructions and Sample Items from the Paragraph

Comprehension Subtest ...... ............. 7

4. Verbal Performance Model ....... ............ 9

5. Part Two Applicant Sample Sizes .... ...... . 15

6. Full Model Cognitive Component Parameters: Word

Knowledge, Form One ....... ............ .. 17

7. Reduced Model Cognitive Component Parameters:

Word Knowledge, Form One ....... ....... .. 18

8. Full Model Cognitive Component Parameters: Word

Knowledge, Form Two ...... .............. .. 19

9. Reduced Model Cognitive Component Parameters:

Word Knowledge, Form Two .... ............ .. 20

10. Full Model Cognitive Comonent Parameters:

Paragraph Comprehension, Form One ... ....... 24

11. Reduced Model Cognitive Component Parameters:

Paragraph Comprehension, Form One ....... 25

12. Full Model Cognitive Component Parameters:

Paragraph Comprehension, Form Two .. ....... .. 26

13. Reduced Model Cognitive Component Parameters:

Paragraph Comprehension, Form Two .. ....... .. 27

11 GI)

. ............................
COGNITIVE PROCESSING DETERMINANTS OF VERBAL

ITEM DIFFICULTY ON THE ARMED SERVICES

VOCATIONAL APTITUDE BATTERY

ABSTRACT

Research was designed to identify cognitive processing

operations and strategies involved in performance on the

verbal subtests of the Armed Services Vocational Aptitude

Battery (ASVAB). Cognitive processes thought to underlie

performance on the Word Knowledge and Paragraph Comprehension subtests were identified. Items on two ASVAB forms

were rated on these variables. The relative effects of

cognitive processes were examined for eight groups of Ar---

Applicants and Accessions using Linear Latent Logistir

Trait methods. Analyses demonstrated the relevance r

cognitive processing variables to item difficulty fo he

ASVAB verbal items.

INTRODUCTION

The purpose of this research was to identify the cogni- tive processing operations, stores, and strategies involved

in performance on the verbal subtests of the Armed Services

Vocational Aptitude Battery (ASVAB) 8/9/10. Procedures

were developed for evaluating the underlying cognitive

ability contributions to performance on item sets.

The methods were proposed for general use in assessing and

documenting cognitive processing contributions to performance on tests and test-like tasks. The procedures that

were developed were proposed for use in constructing

industrial/organizational assessment instruments that tap

specified, training- and job-relevant processing abilities.

The specific objective of this work was to clarify

the construct validity of verbal items on ASVAB 8/9/10.

These results were used in subsequent analyses of the

predictive validity of assessed ability components to

successful performance in Army jobs. The research was

conducted in two parts.

5..°

. B -

p LL ? ? ° ; I? -.. ?. °.. .°'i. 1 -.- . ,*//'i i : > ~a ~ i
METHOD

Part 1 began with a reriew of recent work in the area of

verbal information processing. The review was directed at identifying possible sources of cognitive processing operation, store,

and strategy differences on the verbal items of ASVAB and suggesting methodological alternatives for the examination of relevant

data. Part 1 activities included:

A. Conducting a review of the definition and measurement of

cognitive processing components involved in performance on verbal

tasks.

B. Developing a model of verbal performance to provide a

conceptual framework for identifying and operationalizing processing components.

C. Drafting theoretical and operational definitions for the

ability components deemed relevant to performance on the Word

Knowledge (WK) and Paragraph Comprehension (PC) items of ASVAB.

D. Coding the Word Knowledge and Paragraph Comprehension items

on two ASVAB 8/9/10 fo'rms for the presence or absence, complexity,

or frequency of execution for each cognitive ability component.

Part 1 methods were based on the premise that variation in

item difficulty would be a function of the type and level of

required processing. It was also posited that processing requirements could be accounted for by structural features of items and

that processing requirements could be indexed or quantified

accordingly.

Part 2 analyses examined the ability component contributions

to item and person variance on the verbal items of ASVAB. Part 2

activities included:

A. Modeling item difficulty for the Word Knowledge and

Paragraph Comprehension items for Fiscal Year 1981 Army Applicants

and Accessions using linear logistic latent trait techniques;

stimulus complexity data reflecting the nature or level of involvement for relevant cognitive processing operations and performance data for selected samples of examinees fed into the

models.

B. Cross validating the models across ASVAB forms and samples

of examinees.

A description of the operational Armed Services Vocational Aptitude Battery (ASVAB) follows.

.. ... ... .-.-... 2...... ..
K TU . .. . .. . .-

The Armed Services Vocational Aptitude Battery.

The Armed Services Vocational Aptitude Battery (ASVAB) was

introduced in January 1976 as the single Department of Defense

selection and classification battery. Six replacement forms were

developed and implemented October 1, 1980. The 1980 version,

ASVAB 8/9/10, was the focus of this work.

The ASVAB is administered to over one million applicants for

military service each year. The ASVAB plays two important roles

in the enlistment of military personnel. First, ASVAB scores are

used in determining an applicant's eligibility for military

service. Second, ASVAB scores are used to establish an

individual's eligibility for assignment to specific military jobs.

The ASVAB consists of eight power and two speed tests. The

ten subtests are listed in Table 1. These subtests are included

because research and experience have demonstrated that they are

valid predictors of success in various types of military jobs.

Table 1 also provides the number of items per subtest and the

testing time limits. Although all ten subtests are individually

timed, only Numerical Operations and Coding Speed are considered

speed tests. The remaining eight subtests are power tests with

ample time limits.

The means and standard deviations included in Table 1 were

computed for the subtest raw scores of a random sample of FY81

Army Applicants described in Kass, Mitchell, Grafton, and Wing

(1981). Estimates of subtest internal consistency reliabilities

for the eight power subtests for each of the six forms of ASVAB

8/9/10 were derived from Ree, Mullins, Mathews and Massey (1981).

Parallel form reliability estimates for the two speeded subtests

were obtained from Sims and Hiatt (1981) and Wilfong (1980).

The Word Knowledge Subtest

There are thirty-five items on the Word Knowledge subtest of

the operational ASVAB. The test is constructed to assess the

examinee's understanding of the meaning of words. Surface characteristic differences associated with stem and alternatives for the

vocabulary items include: word frequency, vocabulary level,

number of syllables, grammatical class, stem type, etc. Instructions and sample items from the Word Knowledge subtest appear at

Table 2.

The Paragraph Comprehension Subtest

The ASVAB 8/9/10 Paragraph Comprehension subtest has fifteen

items. The subtest is constructed to assess the examinee's

understanding of the meaning of paragraphs. Surface characteris3
0 .- Nm .4 to U I --

N, ONJ 0# evIA

LAl rz C 06 IA LA t'

4. CM M4 Nr '.0r

IN- I

E-4.4.-

i41.43~ ~

fo I

continued next page

~ *.~. . . . . . .
44.

'00i

iII

;14

--

.

I

~~. ,

,iI..

- 5-> "- 
Table 2

The Word Knowledge Subtest

This test has questions about the meaning of words. Etch -: question has an underlined word. You are to decide which of the four possible answers most nearly means the same as the

underlined word, then blacken the appropriate space on your

answer sheet.

1. S3all most nearly means

A. sturdy

B. round

C. cheap

D. little

2. The accountant discovered an error.

A. searched

B. found

C. enlarged

D. entered

3. Inform most nearly means

A. ask

B. turn

C. tell

D. ignore

4. The wind is variable today

A. shifting

B. chilling

C. steady

D. mild

5. Cease most nearly means

A. start

B. change

C. continue

D. stop

6)
Table 3

The Paragraph Comprehension Subtest

This is a test of your ability to understand what you read.

In this section you will find one or more paragraphs of

reading material followed by incomplete statements or

questions. You are to read the paragraph and select one of

four lettered choices which best completes the statement or

answers the question. When you have selected the answer,

blacken in the correct numbered letter on your answer

sheet.

1. From a building designer's standpoint, three things that

make a home livable are the client, the building site, and

the amount of money the client has to spend.

According to the passage, to make a home livable

A. the prospective piece of land makes little difference

B. it can be built on any piece of land

C. the design must fit the owner's income and site -

D. the design must fit the designer's income

2. In certain areas water is so scarce that every attempt is

made to conserve it. For instance, on one oasis in the

Sahara Desert the amount of water necessary for each date

palm tree has been carefully determined.

How much water is each tree given?

A. no water at all

B. exactly the amount required

C. water only if it is healthy

D. water on alternate days

3. The duty of the lighthouse keeper is to keep the light

burning no matter what happens, so that ships will be warned

of the presence of dangerous rocks. If a shipwreck should

occur near the lighthouse, even though he would like to aid

in the rescue of its crew and passengers, the lighthouse

keeper must

A. stay at his light

B. rush to their aid

C. turn out the light

D. quickly sound the siren

7 0
tic differences associated with the passage, question stem, and

alternatives include: total number of words, mean grade level of

words, standard deviation of vocabulary level, number of polysylL" labic words, location of response-relevant information in the

passage, type of passage (factual, fictional, first person, second

person, etc.) average length of sentence, number of sentences,

number of associated questions and type of question. The

paragraph selections on the PC subtests have from one to four

accompanying questions. The number of distinct paragraph selections for the six operational ASVAB forms ranges from six to nine.

Instructions and sample items from the Paragraph Comprehension

subtest are at Table 3.

The unit of analysis for the Part 2 computations was the

item. The two ASVAB forms that were examined in this work each

had nine independent PC items. The probability that the ratio of

number of components to items would be high for the PC subtests

was recognized. The decision to proceed with the PC analyses in

spite of this was based on the fact that the PC items look particularly non-parallel across forms. The results that follow for

the PC items were merely suggestive.

Part One

The Development of a Model of Verbal Performance

The model of verbal performance that was developed for this

work borrows heavily from the paradigms of Hunt, Carroll,

Frederiksen, Pellegrino, Kintsch, and others. The model describes

verbal performance in the context of text analysis. The Services

use the verbal subtests of the ASVAB to predict general verbal

ability; the verbal items, however, directly assess only those

skills used in text processing. The model, therefore, characterizes verbal performance with respect to the processes which

underlie text comprehension. The model does not define a general

theory of cognitive processing.

The model depicts verbal performance by five processing or

storage structures. The first structure might be thought of as a

perceptual processor, the second as an executive or control

processor, the third as the locus of lexical access and semantic- . syntactic analysis, the fourth as knowledge-based information and

information-free storage, and the fifth as a response

processor. Each structure is discussed below. The structures are

not strictly serially or hierarchically ordered. The flow of

information within the system is not necessarily sequential or

parallel. A schematic of the model is at Table 4.

The PERCEPTUAL PROCESSOR is the structure that inputs

stimulus information to the processing system. It includes the

set of operations that converts the physical stimulus to a form

that is appropriate for the task; it includes the operations that

.- 8
Table 4

Model of Verbal Performance

VISUAL DISPLAY

Perceptual Processor

A. Visual Feature Extraction

B. Perceptual Encoding C. Decoding

Stort Term Storage

A. Lexical Access

B. Semantic-Syntactic

Analysis

Executive or

Control Processor

long Term Storage

A. Knowledge-Based

Information

Structures-) B. Infonnation-free

Functicns

Response

9 0-/1".

• . % ° .. • , . . . . .. ... . - , -... . . . . .• • - - _, -_ .. -, .-.. .--..- ...... •....--;. ...- .... ..:.... .-. -. - .- .-.. ....-. ,-. -,.-..- ......-.-.- ..,.,-.. .,,., ,- .[, M:. -.- .- ... ,-. ..
match stimuli to appropriate grapheme and phoneme

representations. The perceptual processor is characterized by

three operations described by Frederiksen as: visual feature

extraction, perceptual encoding, and decoding. Visual feature

extraction is the operation by which different types of

information about the stimulus display are extracted. The

processor may select and organize information for further

processing or it may ignore or deemphasize stimuli. Perceptual

encoding is the operation by which information is input into the

system; external stimuli are translated to internal codes.

Decoding is the operation by which arbitrary physical patterns

are recognized as representations of grapheme and phoneme

concepts in the lexicon. These operations may be thought-of as

automated processes for the samples of examinees considered here.

The processor may be thought of as a short term sensory storage

or buffer component.

The EXECUTIVE OR CONTROL PROCESSOR is the structure that

controls the flow of information in the system and has access to

the various levels of memory storage. This structure (I)

determines the nature of a problem, (2) has knowledge of the

quality of one's competence with respect to a task, (3) selects

processes for solving a problem, (4) decides on a strategy for

combining these processes, (5) dec4 les how to allocate processing

resources, (6) decides how to repredent the information upon

which processes act, (7) evaluates how well cognitive proceses

are accomplishing subgoals in terms of overall goals, and (8)

makes necessary shifts in processing strategies.

This structure is analogous to Sternberg's metacomponent and to

the executive processor described by Snow, Whitely, and others.

The LEXICAL ACCESS/SEMANTIC-SYNTACTIC ANALYSIS STRUCTURE is

a short term storage or working memory structure. This structure

can be thought of as the workbench where information is held for

concentrated cognitive processing. While in working memory,

information is highly available for retrieveal. It can be held

in short term memory while it is being operated on or transformed

by cognitive processes. There are limitations on the amount of

information that can reside in working memory. One's abilities

to process such information are, therefore, limited.

In the processing of text materials, a match must be made

between letter strings input at the perceptual processing stage

and appropriate semantic referents. Analysis in working memory

is directed at attaching meaning to perceptual patterns. For

phrase and sentence units, analysis is also directed at

organizing these meaning elements into coherent text

representations. Lexical, semantic, and syntactic knowledge is

called upon in the identification of words and in phrase and

sentential analysis.

Lexical Access is defined simply as the retrieval of

. .
information about individual words from long term memory. In

lexical access, grapheme and phoneme data drive the retrieval of

semantic information.

Semantic-syntactic analysis takes place in short term memory;

it is defined by the retrieval of knowledge-based structures and

information-free functions. These structures are discussed by

Hunt (1978). In semantic-syntactic analysis, the knowledge-based

and information-free long term memory structures are accessed and,

in the case of the information-free functions, executed in short

term memory to form a semantically coherent representation of

prose. Information about individual words stored in long term

memory is retrieved and arranged to form a semantically coherent

structure. Kintsch and van Dijk (1978) have developed a prose

processing model which references the types of knowledge-based

structures and information-free functions involved in semanticsyntactic analysis.

The fourth structure is a long term storage structure. This

structure is the locus of KNOWLEDGE-BASED INFORMATION STRUCTURES

and INFORMATION-FREE FUNCTIONS. The knowledge-based information

structures represent semantic and syntactic knowledge. These

structures represent the ability to deal with words arid the

concepts they represent. They reflect experience with and

cognizance of the English language. The knowledge-based information structures are also associated with knowledge of the

world and world events. These knowledge structures are mediated

by verbal knowledge but represent information about the world

ancillary to mastery of the English language.

The information-free functions are the operations by which

information structures are transformed to equivalent structures

necessary for task performance. No semantic or syntactic

information is associated with these strategic knowledge

structures. A distinction is made, here, between information and

processes. These operators are defined by learned, stored

transformation rules. Examples of information-free operators are

the processes of comparing and inferring. These operators

perform such functions as identifying similarities and

differences among information structures, generating missing

bridging information to establish semantic coherence for a text,

or sorting information structures into categories.

The final structure is the RESPONSE OPERATOR. This is the

structure through which appropriate actions are selected and

executed. The response operator is the structure by which the

examinee makes either an observable response, such as selecting

one response from a set of multiple alternatives or makes an internal

response such as modifying schema in long term memory.

." . . . . . . -7 -7. -
WK and PC Item Rating Scheme

Operational definitions were developed for each of the

operations/structures described above. These are documented in

Mitchell (1982). The operational definitions laid the foundation "..

for the following item rating scheme. For the Word Knowledge

subtest, items were coded on five variables. They were:

A. The total word count for target string, correct response,

and incorrect response strings. This count provided an index of

the perceptual processing load or attentional resource allocation

load imposed by an item. It was hypothesized that increases in

item difficulty would accompany increases in the encoding or

attentional resource load presented by an item. S

B. Dichotomous coding for the Lexical Access/Semantic-Syntactic

Analysis variable. Context-embedded vocabulary items were assigned

a unit score. Non-embedded items received a score of zero. It was

proposed that decreases in item difficulty would accompany the

provision of contextual information for Word Knowledge items.

0

C. The Kucera and Francis word frequency score for the target

word. This value indexed the probability that stored information

was available for the target word. It was hypothesized that

decreases in item difficulty would accompany increases in word

frequency scores for the target word.

p

D. The Kucera and Francis word frequency score for the correct

response. This value indexed the probability that stored semantic

information was available for the correct response string. It was

hypothesized that decreases in item difficulty would accompany

increases in word frequency for the correct response.

E. Count of the high frequency-correct response/low frequencyincorrect response pairs for each item. The Kucera and Francis

word frequency scores were used for the correct response and

incorrect response strings to generate a count of the high-correct

response/low-foil response pairs. This value indexed the relative

complexity of judgements of semantic identity for correct S

response/target word and incorrect response/target word pairs. It

was hypothesized that decreases in item difficulty would accompany ...

increases in the number of high frequency-correct response/low

frequency-incorrect response pairs.

The variables for the PC items were:

F. The total word count for the paragraph, correct response,

and 'incorrect response strings. This count indexed the perceptual

processing or attentional resource allocation load presented by an

item. It was hypothesized that increases in item difficulty would

accompany increases in the encoding or attentional resource load

presented by an item.

12"

.
o . . .-.

G. The number of text-based inferences necessary to maintain

semantic coherence as defined by Kintsch and van Dijk (1978) for

the target paragraphs. The construction of text-based inferences,

a controlled, mechanistic information-free function, calls upon

knowledge-based information structures. A description of the

rating system appears in Turner and Greene (1978). It was

proposed that increases in item difficulty would accompany

increases in the number of text-based inferences necessary to

maintain semantic coherence.

H. A dichotomous coding for the response inference variable.

Items for which the construction of an inference was necessary to

link the correct response to information presented in the

paragraph selection were unit coded. Coding followed Turner and

Greene (1978). Again, inference construction is a controlled,

mechanistic information-free function which draws upon knowledgebased information structures. It was hypothesized that item

difficulty would be positively related to the construction of

response inferences.

I. The number of propositions per paragraph selection divided

by the number of words in the selection. Coding, again, followed

Turner and Greene (1978). This value indexed the complexity of

propositional structure for a paragraph selection. It was

hypothesized that increases in item difficulty would accompany

increases in the number of propositions suggested by each

paragraph selection.

J. The number of arguments per paragraph selection divided by

the number of words in the selection. Argument coding agreed with

Turner and Greene (1978). Like variable four, this value indexed

complexity by processing for a paragraph selection. It was

hypothesized that increases in item difficulty would accompany

increases in the number of different arguments presented by each

paragraph selection.

K. The mean of the log transformed Kucera and Francis word

frequency indices for the arguments of each selection. The value

indexed the probability that stored semantic information was

available for the arguments of a paragraph selection. It was

hypothesized that decreases in item difficulty would accompany

increases in mean word frequency scores for paragraph selections.

Obtaining ratings for the 5 variables on the 70 Word

Knowledge items and the 6 variables on the 18 Paragraph Comprehension items took approximately 80 rater hours. 24 hours were

needed to obtain a second set of ratings on variables G-J for the

18 Paragraph Comprehension items. Percentage agreement rates for

codings on Paragraph Comprehension variables G, H, I, and J were

67%, 73%, 74%, and 92%, respectively.

13 (

*i,% ,.. ,. .. .. ,'%...... . . .. _... .... . -.. * %. . . . . , °* ii.
Part Two

The relative effects of the cognitive processing variables on

the item difficulty parameters for the verbal items of the ASVAB

were estimated using the linear logistic latent trait model

proposed by Fisher (1973). The model is described by Whitely and

Schneider (1981) and Thissen (1982).

The model is a one-parameter latent trait model. Item difficulties or location parameters are estimated using: (1) item

response data for samples of examinees, and (2) an item rating matrix reflecting structural characteristics of the items. In these

analyses, the item rating matrix was the item-by-cognitive component

array described in Part One. The matrix defines a set of linear constraints on the estimation of the item location parameters.

The one-parameter latent trait model estimates item difficulty

in the following way:

xp( Iij -

%Mere C fthe abIlIty level for pe s ' J, ar4

a~ uthg difficuty for 1tin 1.

The linear logistic latent trait model consists of a set of linear

constraints on item difficulty parameters, such that,

where t is a vector of difficulties, f;, is the

item-by-cognitive component rating matrix,Qr is a vector of

component parameters, and ct is a normalization constant. The

component parameters reflect the relative effects of the cognitive

component variables on the item location parameters.

The complete linear logistic model is given:

When the $,r matrix is an identity matrix and x is set to 0, the

linear logistic model is equivalent to the one-parameter latent

trait or Rasch model. In this case, each item defines a separate

item difficulty is the only item parameter.

Part Two analyses focused on eight samples of examinees

tested on ASVAB 8/9/10. Six of the samples were FY81 applicants;

two samples were FY81 Army accessions. Samples one and two were

14

. . .... . . . .
I

Army applicants who were randomly selected without regard to

whether they were actually admitted to the Service. Sample one

was composed of 2998 applicants administered form one of the

ASVAB. Sample two had 2925 records; these applicants took form

two.

In addition to these two samples, subsamples of applicants

defined by level of verbal ability were examined for each form.

From the two samples above, the twenty percent with the lowest

Word Knowledge and Paragraph Comprehension summed score were

selected and the twenty percent with the highest Word Knowledge and

Paragraph Comprehension summed score were selected. The N's for

these low verbal and high verbal ability samples for each form are a a b

at Table 5.5

Table 5,!

Part Two Applicant Sample Sizes

A S V A B

Verbal Ability Groups

Form 1 Form 2

Heterogeneous Ability 2998 2925

Low Ability 585 610

High Ability 604 609

These four groups were subsamples of applicant samples one and two.

Two samples of FY81 Army accessions were also selected. The

samples of accessions were students enrolled in three high-density

Army job training programs. The occupational specialties were:

95B Military Police

2. 94B Food Service Specialist

3. 76C & 76P Equipment Records and Parts Specialist and

Material Control and Accounting Specialist

Accession sample one had 400 examinees; sample two had 359 records.

Rasch item difficulties and linear logistic latent trait item

locations were estimated for applicant and accession groups using

15 

, 

S7,

":---"" -- "" .-,- a' " - ." ---. ---- " "- ptet. "" ...... - -. ..*.N:._": -:-. . . . . . ::::::::::::::::: . . . .. . . : -:--: o: ~ii ].
the marginal maximum likelihood procedures outlined by Thissen

(1982).

For the linearly constrained latent trait model, component

parameters were also calculated for each form and sample. The

component estimates from the latter analyses were examined, and

cognitive component variables with sizable contributions to the .-4

estimation of item difficulty values were retained in a reanalysis

of the data. For each group, this third analysis was computed

using the linear logistic latent trait algorithms. A reduced

model was defined by the third analyses. The statistical significance of the individual cognitive component parameters was not

assessed; estimation of standard errors for the component

parameters is presently computationally intractable.

The adequacy of the cognitive component variables in

predicting difficulty parameters for the Word Knowledge and

Paragraph Comprehension items was assessed by correlating the

Rasch iCem difficulties with the linear logistic latent trait

difficulties and the reduced linear logistic latent item

locations. The Rasch item difficulties, computed with one

parameter per item, functioned as criterion scores for each

sample. The two sets of linearly constrained logistic latent

trait item locations functioned as predictor scores. The two

simple correlations indexed the strength of the relation between

the unconstrained and contrained item difficulties estimated in

each applicant and accession group.

RESULTS

Development of the Word Knowledge Models

The full and reduced models for the Word Knowledge subtests

are at Tables 6-9. The correlations between the Rasch item

difficulties and the five-variable linearly-constrained item

difficulty values for the Word Knowledge subtests ranged from .45

to .55. There were essentially no differences in the correlations

across forms or subjects.

Reduced linear logistic latent trait models were developed

for each Word Knowledge form and sample. Three sets of values

were considered in the selection of cognitive component variables

for the reduced models. They were: (1) the cognitive component

parameters estimated by the linear logistic latent trait algorithms, (2) the intercorrelations between items ratings on the

five Word Knowledge variables, and (3) the simple correlations

between the item ratings and the Rasch or unconstrained item

difficulty values.

16

-. . - .. - . . o - - • - . - - - . ° - . .. . ... . . . .!
-)

V4

-- .- . *O

,.W.-4 I- N %D - t- %oc "

4-)

1-4

'-0,

4-))

4 f- --4 '-4 \. jc

C.) -T r- Io0 

I.) CC-l ) 8 r

0. 'VO) c> j lo 01

7I Fo I I 1'ol 4 I 

0. 4.NI0 E -1 0 H

4.)

r-(~4. X " C~ 

17
0

a)0

r-4I~ - -T

F4.-

a) a C\

'-4

"4--

)

C) 4-.1 -4

4-4 s-0

W

024-

88

-. ~~~~w *. .... f. cm 
03

w-. 4 -4 t- U'\

4

41 1-1i LC

r-4-4 t6

.4.3

2 (D

4

%-440

"4

S 4.'

'2$

K 19
to

02

00

( jQ .) J.)

-H 00 Y-i

-44. 4. 

L~0 C '4 4

2. .3

'-4A C

020
Cognitive Component Variables

The cognitive component parameters for both the Total Word

Count and Lexical Access/Semantic-Syntactic Analysis variables

were sizable for all examinee groups and both forms. These

variables were entered singly in the reduced models because item

ratings for the two variables were highly correlated. The word

count for an item was positively related to whether the target

word was content embedded or non-embedded. The Total Word Count

variable was selected for the hetereogeneous and low verbal

ability applicants and for the accessions taking form one. The

Lexical Access/Semantic-Syntactic Analysis variable was included

for the high ability applicants taking form one and for all

examinees administered form two.

The sign of the cognitive component parameter for the Total

Word count variable in the form one reduced models was positive.

The cognitive component parameters for the Lexical Access/

Semantic-Syntactic Analysis variable for form two analyses were

negative. The direction of effect was consistent with

hypothesized relations for both variables. Item difficulty was

positively related to the total number of words presented by an

item. Item difficulty decreased with the provision of contextual.

information for the Word Knowledge items.

There was evidence of interacting effects for the Total Word

Count and Lexical Access/Semantic Syntactic Analysis variables

when they were entered simultaneously in the five-variable models.

• ::: A possible interpretation of the interaction is that item difficulty does not increase with heavy encoding requirements when

the additional lexical units provide context. Because the decrements in the item difficulty correlations from the five-variable

linear logistic latent trait models to the reduced models were

* minimal when the variables were entered singly, these two vari-

" . ables were not combined in the final models.

The cognitive component parameters for the Target Word

Frequency variable for form one and the Correct Response Word

Frequency variable for form two were sizable. The Correct

* Response Word Frequency component parameter for the low verbal

ability group taking form one was also large. The Target Word

Frequency variable was entered in the reduced models for all.

groups taking form one. The Correct Response Word Frequency

variable was included for the low verbal ability group taking form

one and for all examinees taking form two.

Like the Total Word Count and Lexical Access/Semantic-Syntactic Analysis variables, item ratings on these two variables were

highly correlated. Difficult target words were accompanied by -.

"

difficult defining words. The direction of effect was consistent . with hypothesized relations for these two variables in the reduced - models. Words with low frequency counts were positively related to

.21 * * **%*...o'. .: ..
item difficulty both for the target word and the correct alternative.

The cognitive component parameters for the High Frequency

Correct Response/Low Frequency Incorrect Response variable were

small for all groups. This variable was operationalized in such a

way that its values were highly related to the values assigned on

the Correct Response Word Frequency variable.

The parameter estimates for the cognitive component variables

could not be evaluated for significance. In addition, the probabilities would be meaningless in light of the model-building

procedures used. Only the relative effects of the cognitive

processing variables included in the models could be examined.

The correlations between the reduced model item difficulty

estimates and the Rasch item difficulty values ranged from .41 and

.54. The correlations were quite similar to the correlations for

the five-variable models. The models explained from 171 to 309 of

the variance in the item difficulty values estimated by the Rasch

models.

Form Differences

The reduced models for the two forms of the Word Knowledge

subtest were judged to be conceptually similar. Both sets of

models contained a variable which indexed the amount of information presented in the item and both included a variable which

indexed the difficulty of the target or response words.

The Total Word Count variable is a measure of the amount of

information that must be encoded, analyzed, and retained by the

reader. The Lexical Access/Semantic-Syntactic Analysis variable

indexes the provision of contextual information for the target

word. For the Target Word Frequency and Correct Response Word

Frequency variables, item ratings reflect the probability that

examinees have encountered and comprehended information about

individual lexical units. For both forms item ratings for the

Total Word Count and Lexical Access/Semantic Syntactic Analysis

variables and the Target Word Frequency and Correct Response Word

Frequency variables were highly correlated.

Group Differences

Although differences in estimated Rasch and linear logistic

latent trait item difficulty values were observed for examinees

differentiated by level of verbal ability, no marked differences

between the cognitive component parameters were noted for groups

of examinees on the two forms. The composition of the models and

relative magnitude of the cognitive component parameters were . -

22
- comparable for hetereogeneous, low, and high verbal ability FY81

applicants and FY81 accessions. -

Development of the Paragraph Comprehension Models

The results of analyses on the Paragraph Comprehension

subtests were difficult to interpret. Analyses were performed on

* nine items for each subtest and relations were hypothesized for

six cognitive processing variables. The ratios of the number of

parameters to the number of items for both the full and reduced

models were such that meaningful interpretation was prohibited.

The results below are presented merely for completeness. The full

and reduced models appear at Tables 10-13.

Cognitive component variables were selected in the same way

for the reduced Paragraph Comprhension models as they were for the

reduced Word Knowledge models. The cognitive component parameters

for the # Propositions/# Words, # Arguments/# Words, and Mean

Argument Frequency variables for form one were high for the

hetereogeneous ability level applicants, the high ability applicants, and the accessions. For the low ability group, the

absolute values of the Word Count, Response Inference, # Propositions/# Words, and Mean Argument Frequency component parameters

were high.

The correlations between the six-variable model item dif- ...

ficulties and the Rasch item difficulty estimates ranged from .81

to .93 for Form One. Correlations of this magnitude were presupposed by the fact that the number of parameters was only slightly

less than the number of items. For the reduced models, the

correlations ranged from .67 to .84. There were no discernable

differences between groups.

For Form Two the correlations between the linear logistic

latent trait difficulty estimates and the Rasch item difficulties

ranged from .86 to .qq. The cognitive component parameters for

the Response Inference, # Propositions/# Words, and # Arguments/#

Words variables were high for the hetereogeneous applicant group,

the low ability group, and the accession group. The Response

Inference variable did not have a sizable weight for the high

ability group.

The correlations between the linear logistic latent trait

difficulty values for the reduced models and the Rasch difficulties ranged from .81 to .96. These models represented from

66% to 921 of the variance in the item difficulties estimated by

the Rasch models. There appeared to be no group differences.

23 ..'-"

iz?..:..• ,..............-.................. . ....~~~.... ... -........-.............-..................... ::::I
V6

000

00

43

a ,;4 *,;* * . .

000

-r4

'.44

VV

0 H i W~-CJ ~ %.

0 0

(-41 I~

C 4)
N~~ -'\ ON --.r. - -

00 *o- 6-

co

4 ) 0.~

r-I- o 0 

co -le)-4% ~

t-- F-(J U\ CJ c

I0. r4 * .O .~

4)0 Lr\ C

m~~C C.))- .

fn =14-

d) a

p4.

25~
(D

)

(%H 47\ mmmJ0M 4A t

4

W -r -, C~j M %.0 C

§ . o oc -4M 0 c

ii -4 1- C J MLC O V 

moo 0 ~ CJ~ 02C~ 0q U''4CJ \ %-

4)CIA

4) c 

r4 ~ ~ ~)~ 0

.4, .d. .. . .

Go 04 9 -4

1; Z czCA L4)

26U
07

cn U)

.)

0 e - 000

4.) r-4 " \4

I. I0

In c

C)~~ 0 a CD4iI.- 

:3v~ al w4

C. i~f-4

.~ C. E (VC-.4

mwI

I0

C~%O\ 8

V 4.1~

rq c:C)

(D- W 07 i10

t4)

% .

27Yi
DISCUSSION

Relevance of Cognitive Processing Variables to Item Difficulty

Part 2 analyses suggested that cognitive processing

variables were related to item difficulty for FY81 Army

applicants and accessions on the verbal subtests of the

ASVAB. The reduced linear logistic latent trait models

predicted from 17% to 30' of the variance in the item

difficulty values estimated by the Rasch models for the

Word Knowledge items. The reduced models were predictive

of the Rasch-computed item difficulties for the Paragraph

Comprehension items.

Additional information about the relations between

cognitive processing requirements and item difficulty might

be gleaned in future research using item sets specifically

developed to represent a varied range of proficiency on the

individual processing variables. More sensitive assessment

of individual differences on these variables and of their

relevance to item difficulty might be attained with

instruments constructed with special attention to assessing

a varied range of ability levels on the processing

variables. Evaluation, here, was possible only on the

cognitive constructs represented on the operational ASVAB

and only in terms of the component variance built into the

test.

Further information about the relations between

cognitive processing variables and item difficulty might

also be gained by fitting multidimensional item response

models to these data. The present analyses fit

unidimensional item response models to data which were

multidimensional. Both theoretical and empirical

considerations attest to the multidimensionality of the

data.

Determinations of the Parrallism of the Operational Forms

There was no evidence to suggest that the two

operational forms of the ASVAB under investigation were not

parallel. There were essentially no differences in item

ratings for the five Word Knowledge and six Paragraph

Comprehension variables across the two forms. The

cognitive component examinations of the two forms did not

point clearly to differences between forms.

28 -]
CONCLUSION

The relations of cognitive processing variables to item

difficulty for the ASVAB 8/9/10 Word Knowledge and

Paragraph Comprehension subtests were demonstrated.

Assessments of the predictive utility of these and similar

constructs to success in Army training and military jobs

are needed.

It is proposed that analyses of this type allow for the

assessment and documentation of the contributions of

cognitive processing operations, stores, and strategies to

performance on industrial/organizational assessment

instruments. These techniques can inform evaluations of

the predictiveness and relevance of aptitude and ability

measures. These methods can be used to develop item sets

so that specified, training- and job-relevant cognitive

processing abilities are tapped.

299

. • ... -
REFERENCES

Carroll, J. B. Psychometric Tests as Cognitive Tests: A New

Structure Of Intellect. In L. B. Resnick (Ed.), The Nature of

Intelligence. Hillsdale, New Jersey: Lawrence Erlbium Associates, 1976.

Carroll, J. B., & Maxwell, S. Individual Differences in Ability.

In M. R. Rosenzweig & L. W. Porters (Eds.), Annual Review of

K Psychology, Volume 30. Palo Alto, California: Annual Reviews

1979.

Frederiksen, J. A. Chronometric Study of Component Skills in

Reading. Cambridge, Massachusetts: Bolt, Beranek and Newman, 1978

Frederiksen, J. A. Component Skills in Reading: Measurement of

Individual Differences Through Chronometric Analysis. In R. E.

Snow, P. A. Federico, & W. E. Montague, (Eds.), Aptitude,

Learning and Instruction, Volume 1. Hillsdale, New Jersey:

Lawrence Erlbaum Associates, 1980.

Hunt, E. What Kind of Computer is Man? Cognitive Psychology,

1971, 2, 57-98.

Hunt, E. The Memory We Must Have. In R. Schank & K. Colby (Eds.),

Computer Models of Thought and Language. San Francisco: Freeman, 1973.

Hunt, E. Quote the Raven? Nevermore! In L. Gregg (Ed.),

Knowledge and Cognition. Hillsdale, New Jersey: Lawrence

Erlbaum Associates, 1974.

Hunt, E Varieties of Cognitive Power. In L. B. Resnik (Ed.), The

Nature of Intelligence. Hillsdale, New Jersey: Lawrence Erlbau~n

Associates, 1976.

Hunt, E. Mechanics of Verbal Ability. Psychological Review, 1978,

85, 109-130.

Hunt, E. Intelligence as an Information-Processing Concept.

British Journal of Psychology, 1980, 71, 449-474.

Hunt, E., Frost, N., & Lunneborg, D. Individual Differences in

Cognition: A New Approach to Intelligence. In G. H. Bower,

(Ed.), The Psychology of Learning and Motivation, Volume 7. New

York: Academic Press, 1973.

31l6

-_-A
Hunt, E., & Lansman, M. Cognitive Theory Applied to Individual

Differences. In W. K. Estes (Ed.), Handbook of Learning and

Cognitive Processes: Introduction to Concepts and Issues,

Volume 1. Hillsdale, New Jersey: Lawrence Erlbaum Associates,

1975.

Hunt, E., Lunneborg, C., & Lewis, J. What Does it Mean to be High

Verbal? Cognitive Psychology, 1975, 7, 194-222.

Kass, R. A., Mitchell, K. J., Grafton, F. C., & Wing, H. Factor

Structure of the Armed Service's Vocational Aptitude Battery

(ASVAB) Forms 8, 9, and 10. 1981 Army Applicant Sample.

Educational and Psychological Measurement, in press.

Kintsch, W. The Representation of Meaning in Memory. Hillsdale,

New Jersey': Lawrence Erlbaum Associates, 197-.

Kintsch, W., & Keenan J. Reading Rate and Retention as a Function

of the Number of Propositions in the Base Structure of Sentences. Cognitive Psychology, 1973, 5, 3, 257-274.

Kintsch, W., Kozminsky, E., Streby, W. J., McKoon, G., & Keenan,

J. M. Comprehension & Recall of Text as a Function of Content

Variables. Journal of Verbal Learning and Verbal Behavior,

1975, 14, 19 -214.

Kintsch, W., & van Dijk, T. Toward a Model of Text Comprehension

and Production. Psychological Review, 1978, 85, 363-394.

Kucera, H., & Francis, W. N. Computational Analysis of Present-Day

American English. Providence, Rhode Island: Brown University

Press, 1967.

Mitchell, K. Cognitive Processing Determinants of Item Difficulty

on the Verbal Subtests of the Armed Services Vocational Aptitude

Battery and Their Relation to Success in Army Training. Army

Research Institute Draft Technical Report, January 1983.

Pellegrino, J. W., & Glaser, R. Cognitive Correlates and Components

in the Analysis of Individual Differences. Intelligence, 1979,

3, 187-214.

Pellegrino, J. W., & Glaser, R. Components of Inductive Reasoning.

In R. E. Snow, P. A. Gederico, & W. E. Montague (Eds.),

Aptitude, Learning and Instruction, Volume 1. Hillsdale, New

Jersey: Lawrence Erlbaum Associates, 1980.

32-.
Ree, M. J., Mullins, C. J., Mathews, J. J., & Massey, R. H. Armed

Services Vocational Aptitude Battery: Item and Factor Analyses

of Forms 8, 9, and 10 (AFHRL-TR-81-55). Brooks Air Force Base,

TX: Air Force Human Resources Laboratory, Manpower & Personnel

Division, 1981.

Sims, W. H., & Hiatt, C. M. Validation of the Armed Services

Vocational Aptitude Batter (ASVAB) Forms 6 and 7 with

Applications to ASVAB Forms 9 and 10 (CNS 1160.

Alexandria, VA: Center For Naval Analysis, 1981.

Tnissen, D. Marginal Maximum Likelihood Estimation for the OneParameter Logistic Model. Psychometrika, 1982, 47, 2, 175-186.

Whitely, S. E. Maximum Estimation of Multicomponent Latent Trait

Models for Independent Processes. Technical Report, NIE-80-3,

for the National Institute of Education 1980.

Whitely, S. E. Measuring Aptitude Processes with Muticomponent

Latent Trait Models. Journal of Educational Measurement, 1981,

18, 2, 67-84.

Whitely, S. E., & Barnes, G. M. The Implications of Processing

Event Sequences for Theories of Analogical Reasoning. Memory

and Cognition, 1979, 1, 323-331.

Whitely, S. E., & Schneider, L. M. Process Outcome Models for

Verbal Aptitude. Technical Report, NIE-80-1, for the National

Institute of Education, 1980.

Whitely, S. E., & Schneider, L. M. Information Structure for

Geometric Analogies: A Test Theory *Approach. Applied -

Psychological Measurement, 1981, 5, 3, 383-397.

Whitely, S. E., & Schneider, L. M. Information Structure oGeometric Analogies: A Test Theory Approach. Applied

Psychological Measurement, in press.

Wilfong, H. D. ASVAB: Technical Supplement to the High School

Counselor's Guide. Fort Sheridan, IL: Directorate of Testing,

U. S. Military Enlistment Processing Command, 1980.

. . . . . . . . . . -.. . . . . . .... . . - .~ ...,-:-.-...° -