*7 - 7 7 7.

* Research Note 85-46

Job Skills Education Program:

Phase I Report

Robert K. Branson

I Florida State University

* 01 and

Lois Wison

Hazeltine Corporation

and

Beatrice J. Farr

Army Research Institute

for

Instructional Technology Systems Technical Area

Zita M. Simnutis. Chief

Training Research Laboratory

Harold F. 0'NeN, Jr., Director IOTIC

U. S. Army

Research Institute for the Behavioral and Social Sciences

March 1985

Approved for public release, distribution unlimited.

"6 1 2 074
-"" - . . L"--- " .. S "naa .''r s• 4. * -. ~ J" .. r WLN4 V -- -- -. ) -.. U' ~S .r .c ... *sl

K4 F

U. S. ARMY RESEARCH INSTITUTE

FOR THE BEHAVIORAL AND SOCIAL SCIENCES

A Field Operating Agency under the Jurisdiction of the

Deputy Chief of Staff for Personnel

L. NEALE COSBY

EDGAR M. JOHNSON Colonel, IN

Technical Director Commander

Accession For

I]TTS GRA&I

QIJAL~rV

INSPECTED

IThis report. as submitted by the contractor, has been clearowl fer release to Defense Technical information Center

I(DTIC) to comply with regulatory requirements. It has been given no primary distribution other than to DTIC

Iand will be available only through DTIC or other refereonce services such as the National Technical Information

Sshould not be construed as en officia, Depattmehi of the Army position, policy, or decision. unless so designated by other official documentation.J

--. .. • [-. *,.-.-.-.

ft~~~. ' t- -ha
. .-.. 

o

UNCLASSIFIED

SECURIrY CLASSIFICATION OF THIS PAGE (lflitn Date Entered)

REPORT DOCUMENTATION PAGE READ ISTRUCTIONS

BEFORE COMPLETING FORM

1. REPORT NUMBER dJ2 GO3,ICCS.ON r AS REIhT S C ATALOG NUMBERf

ARI Research Note 85-46 A-C. -.'

4. TITLE (aid Subiftle) s. TYPE O r REPORT A PERIOD COVERED

Interim Report L

JOB SKILLS EDUCATION PROGRAM: PHASE I REPORT September 1982-September 1983

6. PERFORMING ORG. REPORT NUMBER

7. AUTHOR(e) 8. CONTRACT OR GRANT NUMBER(e)

Robert K. Branson, Florida State University; %

Lois Wilson, Hazeltine Corporation; & MDA903-82-C-0532

Beatrice J. Farr, Army Research Institute

9. PERFORMING ORGANIZATION NAMEAr ADDRESS 1o. PROGRAM ELEMENT. PROJECT. TASK AREA & WORK UNIT NUMBERS Florida State Universi ty

Center for Educational Technology

Tallahassee, FL 32306 2Q263743A794

11. CONTROLLING OFFICE NAME AND ADDRESS 12. REPORT DATE

U.S. Army Research Institute for the Behavioral March 1985

and Social Sciences 13. NUMSER OF PAGES

5001 Eisenhower Avenue, Alexandria, VA 22333-5600 75

14. MONITORING AGENCY NAME & ADDRESS(i different from Controlling Office) IS. SECURITY CLASS. (of this report)

Unclassified

ISa. DECLASSI FICATION/ DOWNGRADING SCHEDULE --

16. DISTRIBUTION STATEMENT (of thi. Report)

Approved for public release; distribution unlimited.

17. DISTRIBUTION STATEMENT (of the abstract entered In Block 20, It different from Report)

IS. SUPPLEMENTARY NOTES

Beatrice J. Farr, contracting officer's representative. This report, together

with RNs 85-22, 85-37, 85-40, 85-41, 85-42, 85-43, 85-44, 85-45, and 85-47,

forms part of a series on the Army's Job Skills Education Program (JSEP). " -

-9. KEY WORDS (Continue on revere* old& If necee sry and Identify by block number) -

'Computer based instruction

Project management,

Basic skills,

Job skills

20. A TRAcr (mt fe s- irwere a Fl ncweary d Identilyf by block number)

- The Job Skills Education Program (JSEP) is designed to provide soldiers

with the prerequisite knowledge and skills required for successfully learning

their Military Occupational Specialties (MOS). When the JSEP is put into effect, it will replace the Army's current Basic Skills Education Program (BSEP)

with a sophisticated, computer-based system. This report details the managerial and production efforts of the JSEP, Phase I.

DO JA I 473 EDITION OF NOV 6S IS OBSOLETE UNCLASSIFIED

i SECURITY CLASSIFICATION OF THIS PAGE (Wl1m1 Dtes Entered)

............ ....... . . % " . • ° . - - , ., . . , , . . o " • - ° .
* - .~ ~ ~ 11 N7. 71_ 'VhiW1. ~2. IVi -%17. U LN UtI'f - 91.k 7Wf .N.S- -r.n..~.-.

FOREWORD :-K

The Job Skills Education Program (JSEP) is a multi-phase program begun in

Fiscal Year 1982, and designed to enhance enlisted career potential by

improving soldier job performance. The sponsor, the Education Division, -.

Office of the Deputy Chief of Staff for Personnel, expects JSEP to replace

the Army's current Basic Skills Education Program when it is implemented.

The JSEP program, being developed by Florida State University (FSU) will

result in a standardized curriculum for soldiers who demonstrate deficiencies

in the knowledge and skills required to successfully learn their Military

Occupational Specialty (MOS).

In accordance with current policy, JSEP will be an on-duty program. It will

also use a computer-based management system to facilitate an open entry/open

exit approach. At present, most of the lessons being developed will be

computer delivered; however, the plan calls for using existing materials, and

incorporating materials developed as part of other ARI efforts, whenever

appropriate.

A unique aspect of JSEP is that it builds upon a very detailed front-end

analysis of MOS Baseline Skills. The analysis covered tasks performed by

soldiers in the 94 highest density MOSs, in addition to Common Tasks (the

skills that all soldiers, regardless of their MOS, need to know). Although the

Army has over 300 MOSs, the 94 covered in the analysis represent about 80Z of

all soldiers. Perhaps the most useful product developed for the analysis was

S.a taxonomy listing more than 200 prerequisite competencies.(P.C.) for these

MOSs. The competencies were derived from detailed reviews of Soldier Manuals,

and from extensive interviews with subject-matter experts at Army schools.

This effort produced a series of tests intended to diagnose deficiencies in

the P.C.s. Modified versions of these tests will be used in JSEP.

The JSEP program will include a front-end learning strategies module

designed to improve soldier skills in reading, studying, test taking, and

problem solving. The curriculum will consist of this strategies-training, plus

180 diagnostic review lessons, and 120 skill development lessons, which are

being developed for the PLATO and MicroTICCIT computer systems. The program is

being tried out at two TRADOC sites and two FORSCOM sites, prior to an Army-wide

phased implementation.

. . . . . . . . . . . . . . . . . . , "
" JSEP PHASE I REPORT

, 0

EXECUTIVE SUMMARY

* Requirement:

The solicitation requires that all activities performed during Ph,. .e I of

the Job Skills Education Program (JSEP) be documented in a Phase I report to

the Army Research Institute (ARI).

Procedure:

The Florida State University (FSU) in conjunction with the subcontractor,

Hazeltine Corporation, has documented the activities performed in Tasks 2-10

with written reports to ARI. The Task 1 requirement was an in-process review

(IPR) presented to ARI on 19 November 1982. This IPR provided a conceptual

definition of the development of JSEP.

Findings:

Specific tasks were undertaken by the FSU-Hazeltine team which produced

the groundwork and first year's effort in the development of JSEP. Important research and development efforts were completed. Development included:

o implementation and management plan

o design specifications including instructional, engineering,

and human factors

o evaluation plan and standards

o cost benefit trade off analysis

o a predictive cost and training effectiveness analysis

o standards and a plan for the civilian academic community to award a

high school diploma on completion of JSEP

Research included:

o a review of existing basic skills curricula

o a review of the job task analysis and acceptance or rejection

of a clustering schema r

o review of successful learning strategies research and applications

.-. .. ...... ..............-.........-.....- ::-..: .- .- ..-- , ..... .... .. . .... ,7 . ..* .. ._.. ... ..
7%--777 717 77 77 -7

Use of Findings:

Phase I's efforts produced the groundwork for the remaining two phases of

JSEP and for successful implementation of JSEP throughout the Army. .

yv
JSEP PHASE 1 REPORT

CONTENTS

Page

OVERVIEW .............................. * ... * .................. 1

Operational Problem ....................... ........ .. . ...... .1

Research Overview ......................... ............... .... 1

Background............... ................ .... ......... 3

PHASE I OVERVIEW................... . .............. ........... ...... ..6

Computer Based Curriculum.............................. .. . .... ..7

*Sol dier Management System................. .... ..... . . .. ... .. .. 8

-Sal dier Testing Program. ......... 0. ............ ..... ........ .. .9

Supporting Documents and Reports. ......... .. .. ............ .. . ... 9

HASk 1AS........................... ................ 1

Task 2 ............................... .. .........................12

Task 3...................... ....... ........ ..... 14

Task 4.............. .......................... ... ... 15

Task 5............................................. ..... 18

Task 6................................................... 20

Task 7...................... ....................... 2

Task 8........................... ........ ~....... ..... 22

Task 9 ............................. ... ............... 23

Task 10............................ ... ......... ... 24

PHASE I IN PROCESS REVIEWS.......................... ............ .. 26

In-Process Review 1 ........................ ............ 26--

In-Process Review 2.. ................ ..... .... .......... . . .... 27

*CONCLUSIONS FROM PHASE I. .. .. ................ ... . . .. . . ............ 27

DIRECTIONS FOR PHASE 11................. .. ...................... 28

REFE1ENCES ............................................. 29

*APPENDIX -TASK 1 BRIEFING TRANSPARENCIES .................................30

Vii
LIST OF FIGURES

IFigure 1. Percentage change in male populations.................. 4

Figure 2. Flowchart .............. . ..................... ...... 17

vii
7,, 42 T --

JSEP PHASE I REPORT

OVERVIEW

Operational Problem

It is not new that soldiers must be trained to do their jobs. They must

be trained so that each Army job is performed competently--regardless of

differences in ability and background in entering soldiers. To accept less

would cause many mission elements to fail.

Many Army jobs are increasingly dependent upon the soldier's ability to

use high technology and the ability to learn new technology as it develops.

Soldiers need more than training. They need enough education to learn

subsequent jobs, to become eligible for promotion, an-dultimately, to provide

leadership for tomorrow's Army.

Research Objective

The goal of the US Army Job Skills Education Program (JSEP) is to design,

develop, and test a job-related computer based curriculum for possible Armywide adoption. The JSEP development project is concerned with four central

elements:

1. The computer based curriculum,

2. The soldier management plan,

3. The soldier testing program, and

4. The supporting documents and reports.

The project is divided into three phases. Phase I undertook several tasks

which included lesson design, planning documents, and reports that recommend

specific courses of action. Many of the action plans spelled out in the Phase

I reports will be carried out in Phases II and III; some will be carried out in

the years following Phase III.

Phase I contains the following tasks:

Task 1. Develop a conceptual definition for design

requirements and specifications.

Task 2. Review existing basic skills curricula.

Task 3. Review job task analysis and select Military

Occupational Specialty (MOS) clustering schema.

Task 4. Develop JSEP implementation and management plan.

' ~~. . . .. . . . .. .. '.-.-",:,"..'_2.2',- J'.'. : : ,- -;", ,- f. '.. '_' ",'._'_" -_.'__..;' " .
I.--3. . -- -. '. -1.- -'--- - .w 

Task 5. Adapt or develop design specifications

including instructional specifications,

engineering requirements, and human factors

considerations.

Task 6. Develop evaluation standards.

Task 7. Develop JSEP evaluation plan.

Task 8. Develop cost benefit tradeoff analysis.

Task 9. Conduct predictive cost and training

effectiveness analysis.

Task 10. Develop standards and a plan for the

civilian academic community to award a high

school diploma based on completion of JSEP.

Task 11. Phase I Report.

Phase II requires the development of 300 hours of lessons, the soldier

management plan, and the preliminary tryout. Phase II includes the following

tasks:

Task 12. Select hardware; where appropriate develop

software; develop instructor training

program and courseware for 300 hours of

instruction.

Task 13. Conduct preliminary tryout.

Task 14. Phase II Report.

Phase III provides a continuation of the lesson development begun in Phase

II and requires execution of the studies and evaluation plans developed in

Phase I. Phase III includes the following tasks:

Task 15. Develop software and courseware for an

additional 120 hours of instruction.

Task 16. Conduct JSEP tryout at two US Army Forces

Command (FORSCOM) sites and two US Army

Training and Doctrine Command (TRADOC)

sites.

Task 17. Conduct evaluation and revision.

Task 18. Conduct cost/training effectiveness

analysis.

Task 19. Develop technology transfer plan.

Task 20. Document total process in final report.

2 . .. ~ ... . . .. .
The purpose of this report (Task 11) is to document all activities

performed during Phase I.

Background

During the late 1970s, the Army experienced difficulty in recruiting and

retaining qualified personnel. The situation became so acute that many were

questioning the effectiveness of the all-volunteer force. Congress expressed concern over the increasing numbers of recruits the Army had been accepting

from the lowest mental aptitude level, Armed Forces Qualifying Category IV,

since all-volunteer recruiting began in 1972. Individuals in this category

have reading and mathematics abilities below the ninth grade level, ranging as

low as the fourth grade level. In 1975, the Army enlisted 10% of its nonprior

service male members from this category. The rate of category IV recruits increased to 50% by 1980, more than double the 23% category IV youth population

in the United States for the same year. At the end of fiscal year (FY) 1981,

45% or over 305,000 of the Army's enlisted population had reading and

mathematics abilities below the ninth grade level, ranging as low as the fourth

grade level. Congress directed that standards for accession quality be raised,

that at least 65% of Army male accessions be high school graduates for the

fiscal year (FY) 1983, and that no more than 20% of accessions in any service --

rank in the category IV.

A FY 1982 Department of Defense Manpower Task Force Report to the

President (Weinberger, 1982) indicated a dramatic improvement in recruiting and

retention for the FY 1982. The test scores and educational levels of new

enlistees compared favorably with civilian youth. However, the report sounded

a note of caution and projects that beginning in FY 1985, the Army will

probably experience difficulty in recruiting enough accessions who meet the

congressional quality standards. The short term picture is rosy but the long

term picture is quite clouded. The Amy's basic skills problem will be

compounded in coming years because of the decreasirg availability of 17 through

19 year-old males, depicted in Figure 1. These individuals comprise the

majority of the Amy's personnel strength. In an era of a shrinking manpower

pool, the Army will be hard pressed to maintain the current congressional

standards for accession. Additionally, the high technology Army of the future

will demand a more highly skilled soldier.

The US Army, therefore, must have a strong commitment to provide soldiers,

especially new accessions, the training and education opportunities necessary

to succeed in the modern Army. The Army Continuing Education System (ACES) has

responsibility for developing and implementing a variety of education programs

designed to fulfill this commitment. At a minimum, ACES is expected to

accomplish the following four goals with respect to soldiers:

1. increase the likelihood of performing satisfactorily both in

training and on the job;

2. increase the capability of functioning effectively in the Army

community outside the immediate work setting;

3

. .,. . "
suo~letndod a~[2W UL aBue4:p a~eua3Jad -tan~

II

CD

0 U0

(~co 20

00 0

LL-J

CC z_ co

001

6n N na

I I I I -
- .. • °, 

3. increase the potential for completing high-quality, off-duty high

school, college, vocational-technical, apprenticeship or other

educational programs; and

4. increase the potential for pursuing the wide variety of educational ,,,"

programs covered by Veterans Educational Assistance Program or other " veterans benefits.

The Education Division, Office of the Deputy Chief of Staff for Personnel

(DAPE-MPE) is the proponent for ACES in the Department of the Army. Previous

policy decisions have expanded and broadened ACES educational offerings. One

part of this effort involves a revision of the Basic Skills Educational Program

(BSEP). BSEP focuses on providing enabling skills in language, literacy,

computation, and speaking. However, an additional element has been added to

the program to address learning strategies needed by service members. The

upgrading process is expected to continue over a five year period. The US Army

Research Institute (ARI) has the mission to assist in developing and evaluating

BSEP.

The shift in the instructional concept of BSEP is toward a more

functionally-oriented program. MOS requirements will be used as the basis for

determining what the content of BSEP components should be in order to maximize

benefits to individuals and to the Army. The emphasis on functionalized

curriculum is based on research that shows a higher training payoff when basic

skills instruction is tied directly to an area of application rather than

focusing on General Educational Development (GED).

The GED approach identifies adult basic skill needs and curriculum content

by placing the problem within the context of school subject "natter content and

progression through levels of complexity within curricula. In contrast,

functionally-oriented basic skills may reflect only some of the basic skills

defined in the GED approach. Functionally-oriented basic skills education is

designed to increase adults' proficiency in performing specific basic skills in

the context of job activities rather than general studies. Thus, job-oriented

basic skills are more narrowly defined to meet the specific requirements of the

job.

One of the first programs to be designed as job-oriented basic skills

training was developed for the Army in the early 1970s. It was called the

Functional Literacy (FLIT) Program. This program focused on improving Job

reading skills of soldiers entering the Army with reading test scores at or

below the sixth reading grade level. Job reading tasks were identified as

situations in which individuals use printed material in connection with

performance of a work activity. Functionally-oriented reading education was

conducted prior to or concurrent with job training. The FLIT experience, along with recent research (Sticht, 1982) indicates that among marginally literaXte

adults, a brief, concentrated job-related reading program does produce

substantial improvement in job-related reading although it may not produce

large g'ns in general literacy.

Currently BSEP II is divided into BSEP which takes place during initial

entry training and that which takes place after Advanced Individual Training

(1T). It provides 240 hours of classroom instruction in reading, writing,

speaking, listening, and computing for soldiers through grade EI-5.

5
currently geared to raise literacy skills to at least ninth grade level as

measured by the Test of Adult Basic Education (TABE). Soldiers have been

identified as being eligible for BSEP II in one or more of the following ways:

o General Technical (GT) score on the ASVAB below 90

o commander referral based on supervisor's assessment or voluntary %'%

request by the soldier; and

o Skill Qualifications Test (SQT) score below the minimum required for

MOS verification

BSEP II instruction is contracted with accredited educational institutions.

The educational institution selects, develops, and provides most instructional

material. The Army provides field manuals, technical manuals, and soldier's

manuals needed for the program. Class sizes are normally between 10 to 20

students.

JSEP will strengthen and continue this movement toward job-related skills

training. The input data for JSEP is the results of an MOS Baseline Skills

Analysis Project conducted by RCA Educational Services for the US Army Training

and Doctrine Command (TRADOC) (Contract No. DABT60-81-C-O017). The RCA

contract encompassed the following work elements: analysis of Initial Entry

Training Course Survival Skills (IETCSS), analysis of job tasks for identifying

prerequisite competencies, clustering of analysis results, development of

diagnostic tests, and development of design specifications and BSEP curricula

models. The analysis of IETCSS identified those skills required of (and

assumed to be possessed by) soldiers in order to participate in, and graduate

from, initial entry training for selected MOS.

PHASE I OVERVIEW

Phase I is composed of 11 tasks that include lesson design and the

development of planning documents that recommend specific courses of action.

The planning addresses the four central elements of the project:

1. The computer based curriculum

2. The soldier management plan

3. The soldier testing program, and

4. The supporting documents and reports

Computer Based Curriculum

Computer based instruction. In this report, the term computer based

instruction (CBI) is used to encompass all known applications of computers to

instruction, including computer assisted instruction, computer managed

instruction, and other similar terms.

6

............. -. ,**--- ----.... -.-- 

. . . . . .. . . . . . . . . . . . . . . . . . .--- -"-".. *.,* -... * * .- - - - - - - - -. ... .. *-- - . . *-.-7. .''N .

- .- * .. .. *-- ...- . . *p*~ 
. . .- . . . . - . -W a- IA wgJ. -.-..

The entire curriculum is intended to provide some 420 hours of CBI, most

* of which will be computer delivered instruction. The Task 5 Lesson

Specifications describes the rationale for deciding which units of instruction

would be computer delivered.

At the conclusion of the project, The Florida State University (FSU)-

Hazeltine team will deliver to the Army master copies of the 420 hours of .,.'." courseware.

MOS baseline skills. The data base to be used in designing the curriculum

is the TRADOC sponsored MOS baseline skills job task analysis of the 94 most

populous MOS in the Army. We received our analysis data from RCA as they

became available.

Computer systems. The solicitation required that the PLATO system, a

product of Controlata Corporation, be one of the two systems used. The FSUHazeltine team proposed the MicroTICCIT system as the second system. All CBI

instruction developed will be available on both computer systems. Each system

will be configured to meet Amy requirements under a variety of conditions

throughout FORSCOM and TRADOC, including small remote sites as well as sites

having large JSEP populations. The system configurations and a proposed

implementation and management plan is detailed in the Task 4 report.

At the conclusion of Phase III, the FSU-Hazeltine team will deliver to the

Army 44 MicroTICCIT terminals located at two FORSCOM sites and 40 PLATO

terminals located at two TRADOC sites. The courseware intended to be used on

these systems will also be made available in the appropriate form.

JSEP tests. The TRADOC-RCA Baseline Skills Analysis Project also included . development of tests designed to measure whether soldiers could demonstrate

• .competency on identified prerequisite competencies (PCs) that were derived from

job-related tasks. Thus, soldiers in various MOS would require a different mix

of instruction, based on their personal test scores and job requirements. To

provide a personalized curriculum, we modified the RCA testing program and

developed a management plan to identify the instructional requirements of each

soldier deficient in the basic skills. The mix of instruction for each

soldier, even within the same MOS, could be very different, depending on those test scores. ""

At the conclusion of the project, the FSU-Hazeltine team will deliver to . the Army tests that will provide the information necessary to manage, diagnose,

* prescribe, and evaluate soldier progress through the JSEP curriculum.

Learninj strategies. All soldiers entering the Army have completed many

*year, of instruction in civilian schools. Regardless of how many years of

* instruction they have had or whether they have graduated from a high school,

their functional basic skills competencies show unacceptably wide variations.

Because the Army has a demand for Job performance at a constant high level of

competence, it cannot send to duty those who lack the basic skills necessary to

learn and perform their jobs without risking a drop in operational readiness.

One approach to remedial instruction is to organize soldiers into classrooms

and give them an Army sponsored version of the same instruction they had from

kindergarten through high school. The JSEP concept recognizes the shortcomings

7 -.......................................
6.1

of that methodology and has looked to research-based practical approaches to

improve results.

One of the more promising lines of research in recent years has been

* training students how to learn. Although good students always seem to develop

personal means of improving their learning habits, poorer students do not. *

Consequently, in addition to the basic skills instruction, JSEP will offer

intensive direct instruction to soldiers on how to learn Army materials. These

new learner strategies should improve soldiers' abilities to survive in

training environments and to transfer these skills to their jobs.

FSU will deliver to the Army a curriculum in learner strategies that will

attempt to train soldiers on how to improve learning skills. This curriculum * - could be transferred with minimum revision to other Army training programs

where soldier learning skills require improvement.

Soldier Management System

Objective. The objective of the JSEP soldier management system is to

provide the Tllowing key elements:

o An efficient screening test that will compare soldiers'

basic skills achievements to the requirements in their

assigned MOS and identify discrepancies.

0 A JSEP instructional prescription that will provide

sufficient training on the MOS required PCs.

0 A computer based internal instructional management

system that tracks soldiers' progress and guides

them through the instruction.

o A computer based JSEP management system that tracks the

progress of all soldiers in any location and

provides essential management reports as required.

Approach. Because there are large differences in the functional basic

skills requirements among the selected MOS and because soldiers vary -considerably in their achievements when they come to the Army, it is necessary

to design a course of study for each eligible soldier based on NOS requirements

and soldier achievement. It was further decided to adopt a structured systems

analysis approach that requires a "top-down" perspective. Realizing that there

are soldiers who need very little JSEP instruction to become fully capable of

learning all the requirements in their MOS, the first tier of instruction will__

be designed to achieve breadth of MOS coverage.

Subsequently, more in-depth coverage will be developed for those 1405

having the largest candidate JSEP populations. The second tier of instruction

is intended to offer more thorough instruction to those high priority NOS with

the greatest requirements. The design goal is to provide that optimal

instruction required to make soldiers fully capable of learning their skill

level l and 2 tasks.

8

......................... %
~I

JSEP Soldier Testing Program

It is extremely difficult to assign soldiers to instructional programs ,.6**

efficiently without knowing exactly what they know when they arrive.

Accordingly, a systematic testing program was designed to provide all necessary - information for assignment. r .

Testing procedure. During their first duty assignment soldiers are

declared candidates for the instruction provided in JSEP based on a general

test battery. Identified soldiers are given selected Diagnostic Tests to

identify deficiencies that if corrected would enable them to learn skill level

1 and 2 tasks in their MOS.

Based on their performance on the Diagnostic Test, soldiers are given a

*. recommended series of lessons selected to improve their job-related basic

skills competencies. Each lesson ends with a posttest designed to insure that

the soldiers have mastered lesson objectives. When they have completed all

required lessons, a summary posttest made up from test items on the lesson

posttests is given to ensure that they have retained what they learned. When

they pass the summary posttest, they are said to have successfully completed

JSEP.

All test data are entered into the JSEP soldier management system to

facilitate soldier scheduling and generation of specified reports.

Supporting Documents and Reports

FSU developed a full range of reports to provide documentation of JSEP

Phase I activities. Reports are described below on a task by task basis.

Task 1: Develop a conceptual definition for design requirements and

specifcatons. This conceptual deTi-iTitn--as presentd A an in-proces

review (PR) on 19 November 1982 at ARI. The Army need, reflected in this

contract effort, is to provide functional basic skills instruction on jobrelated prerequisites so that soldiers in each of the 94 selected MOS Army-wide

would be able to learn their skill level 1 and 2 jobs more efficiently.

Task 2: Review existing basic skills curricula. The Task 2 report

provifed a summary of all related exitTin materials thought to be potentially

* useful in the JSEP curriculum.

* Task 3: Review job/task analysis and select MOS clustering schema. The

Task T-report -d -aM ses the analys- s an nva-dl-TU o th roducl-s-l the RCATRADOC MOS Baseline Skills Analysis Project.

• Task 4: Develop JSEP implementation and management plan. The Task 4 . report addresses three important elements of the total JSEP program:

1. The JSEP soldier management system,

2. The implementation plan, and

9
3. The logistics and maintenance system.

Task 5: Adapt or develop design specifications, including instructional

specifications, en ine--erng reu nts, and human fac ors considerations.

The Task 5 report addresses the design 'o t-h-lesso specificatns which will

result in approximately 420 hours of instruction. These lesson specifications

are described and examples from several lessons are used to Illustrate various

issues. The lesson specification documents (which are working documents and

collectively fill thirty-three 1 1/2" size three ring binders) are not included

in the Task 5 report be ause of sheer bulk. A listing of the lessons designed

and media selection decisions are contained in the report. The engineering

specifications describe the functional requirements that must be met in the

design of the planned CBI systems. Human factors considerations for the

selected systems are also described.

Task 6: Develop evaluation standards. The Task 6 report presents

standards forevauating the JSEF design and development effort under the

current contract. A preliminary tryout will be conducted during Phase II. A

full evaluation of the project is to be undertaken in Phase III. This report

describes the standards that will provide the basis for those evaluations.

Task 7: Develop JSEP evaluation plan. The Task 7 report details the ./

*; evaluation procedures to be followed diu-r-ig the preliminary tryout to be

conducted in Phase II and for the full scale tryout, to be conducted during

Phase III at two FORSCOM and two TRADOC sites. A rationale for the evaluation

plan is presented, the objectives and procedures stated, and the size and scope

of the soldier populations are described.

Task 8: Develop cost benefit trade off analysis. The Task 8 report deals

effectively wih the 'wTTis" for tTe equpment systems specified. Predictions

about what "will beu during the period of Army-wide implementation of JSEP are

necessarily less accurate, due to rapidly changing technology and costs.

Projections are not part of Task 8.

Task 9: Conduct predictive cost and training effectiveness analysis.

The Task 9 report prests theFSU_-Ha ine approach to the costbenetts of

JSEP. Through visits to Army education centers and analysis of the costs of

development, it was possible to identify those costs that are likely to have

the most impact in the overall cost model. Resources are concentrated on

measuring those costs that would be both controllable and major contributors to

the total cost of ownership.

The life-cycle systems management model is taken into account as plans are

set forth in this report to conduct the cost training effectiveness analysis in

Task-18.

Task 10: Develop standards and a plan for the civilian academic community

N to award a hih.'school diploma bas-e -6n compT o-f- s IU"reporIU".--

presents The approach recommended by FSUUHazeline F-wi~l'h work completed in

JSEP is credited toward the award of a competency-based adult education

diploma. The combination of on-duty job oriented basic skills instruction

(JSEP) and off-duty general education represent an educational experience that

we believe diploma-granting institutions will accept for a diploma.

10 . . .. . .- . . .- ..
Task 11: Phase I re port. The Task 11 report presents an integrated

synopsis detailing a T orteefforts undertaken in Phase I.

P-..
PHASE I TASKS

Task 1

Task 1: Develop a conceptual definition for design requirements and

specifications.

Requirement. The solicitation requires a conceptual definition to insure

the contractor and the Contracting Officer's Representatives (COR) are in

agreement with the aspects underlying development of the JSEP system. This

agreement is essential to successful execution of the contract. The product of

Task 1 is specified as the conceptual definition of the design requirements and

specifications to be delivered at an IPR at ARI. This IPR presents a summary

of the issues and concepts that the contractor will consider when designing the

functional specifications. Issues to be covered as part of this briefing shall

include (but are not necessarily limited to) instructional, engineering, human

factors, and courseware issues, in addition to cost tradeoffs.

Conceptual Definition. Perhaps the best way to begin is to present our -. "' concept of what the Army will have at the conclusion of the three phases of

this project followed by a discussion of the effort required to produce that

contract product. We will estimate what we believe the Army will have to do to

realize the benefits of the project. While each of these points will be

detailed more completely in each separate task report, a summary and overview

here will aid understanding of the project.

There are four central elements of the project that will result in

specific contract products:

1. computer based curriculum,

2. soldier management plan,

3. soldier testing program, and

4. supporting documents and reports.

The four elements are distributed across three phases. Phase I is mainly devoted to lesson design and developing planning documents that recommend

specific courses of action. Many of the action plans spelled out in the Phase

I reports will be carried out in Phases II and III. Some of the documents

require actions that will be carried out In the years following Phase III.

.'Phase II requires the development of 300 hours of lessons comprising the

curriculum and the JSEP soldier management plan. The preliminary tryout .and

the principal effort on the testing program will also be in Phase II.

Phase III provides a continuation of the lesson development begun in Phase

II, and requires carrying out the studies and evaluation plans developed in

Phase I.

Task 1 was completed at an IPR on 19 November 1982 at ARI (see Phase I In

Process Reviews, below).

12

. . . . ....
a,'. - . .

Task 2

Task 2: Review existing basic skills curricula.

Requirement. The solicitation requires a review of representative samples

of materials from off-the-shelf curricula, and materials from locally developed

BSEP curricula, to determine their suitability for use in JSEP.

The product of Task 2 is specified to be a listing of recommended

materials, along with justification for their inclusion and explanation of how

they will be used, to be delivered to the Contracting Officer's Representative

(COR) at an IPR.

Procedure. There were three criteria used as guidelines in the review of

the materials. The primary consideration during Task 2 was to decide whether

the existing materials matched the target population's characteristics. A

secondary consideration was whether the materials meet the objectives selected

for inclusion in the total JSEP program. Thirdly, consideration was given as

to whether or not existing materials could be adapted or revised to bring them

more in line with the objectives selected for JSEP.

The major objectives accomplished were: (1) the acquisition of basic

skills materials, (2) evaluation of the materials obtained to decide whether

the content, format, and instructional components matched the RCA taxonomy, and

(3) development of a summary chart by prerequisite competency number for use in

JSEP.

Findings. We found that the task encompassed the following activities:

1. Materials were obtained by the project staff during visits to

military installations, Adult Basic Education (ABE) centers, and

Elementary and Secondary Education Act Chapter I Programs (Public

Law 97-35). All programs contributed materials.

2. Other services and military contractors supplied samples of their

basic skills materials currently being used.

3. Evaluations were made of all the materials obtained.

4. A summary chart of the materials by PC as specified in the RCA

taxonomy was devised. This chart lists all the PCs and the available materials for each. It includes a rating of those materials.

"'Use of findings. The result of this effort is a report containing the . mater-iTssefu to the JSEP curriculum. Included in the report is a brtef

summary of the scientific services program review of basic skills material

completed by Dr. Walter Dick at FSU and Dr. Kent Gustafson at the University of

Georgia. Dr. Walter Wager of FSU reviewed the PLATO Basic Skills Materials

under separate contract with TRADOC.

Also included in the report are some general comments about the evaluation

of basic skills materials, including a discussion of areas that should be

13
considered when making revisions, and suggestions on how the materials could be

used in the JSEP program. This report was submitted to ARI in lieu of the

required IPR.

Task 3

Task 3: Review job task analysis and select MOS clustering schema.

Requirement. The solicitation requires a review of the job task analysis

and clustering schema developed by RCA to determine their suitability for JSEP.

Subject to agreement by the COR, contractor has the option of adopting or

modifying the MOS clustering schema. The product of Task 3 is specified to be

a viable plan for clustering MOS baseline skills.

Procedure. FSU conducted a thorough and detailed review of the RCA's job

task analysis. Except for minor changes, we found the work by RCA to be useful

and helpful; consequently, we accepted and incorporated the results of the job

analysis and elements of their taxonomy into the lesson design.

We analyzed the two clustering approaches suggested by RCA. These

approaches were (1) clustering by MOS as called for in the solicitation, and,

(2) clustering by PC. To make an informed and rational choice, we studied

RCA's clustering report and curriculum model, and then tried several

alternative clustering methods.-

Our analysis of RCA's proposed curriculum model focused on two main

points:

o The Army's use of the model for implementation and

management of the curriculum

0 FSU's use in lesson development

Findings. We found the MOS Baseline Skills Analysis Project conducted by RCA to e athorough and useful analysis of the tasks performed by level 1 and

2 soldiers in the 94 MOS. We agreed with RCAs conclusions that clustering

does not provide guidance for curriculum development. Based on our

understanding of the requirement, our analysis of the RCA reports, and our own

clustering efforts, we can find no evidence that there is an underlying

* structure in the raw data that can be discovered through clustering.

RCA presented a complete, sequenced, and interdependent curriculum model.

Our analysis of the data presented in the reports available to us, however, has

not revealed any empirical support for that model.

Use of findings. The review of RCA's job task analysis facilitated -our

desii-effTr-THFspecific products used were:

o Complete Extended Task Analysis Procedure Results for each MOS

o Prerequisite Competencies (PCs) for each MOS 2i

14
-. .7-77-. 7 7 .

,..

Complete Prerequisite Competency Indicator Statement for each PC

(Volumes of all statements in RCA Job Analysis of MOS which relate to

a PC.)

o MOS-PC Matrix

Based on our analysis of all RCA contract products and our understanding

of the Army requirement, we have recommended that the clustering approach be

abandoned in favor of a JSEP soldier management model that accommodates

individual soldiers. The model uses test scores to establish a soldier's

curriculum, then sequences the instruction based on soldier progress and

performance on the lessons. We believe that this approach is totally

consistent with the capabilities of JSEP and the general requirements in the

solicitation.

Task 4

Task 4: Develop JSEP implementation and management plan.

Requirement. The solicitation requires that the contractor design an

implementation and management plan to attain maximum efficiency in use of

resources, personnel, and facilities support. Space requirements and

operational procedures, including operations and maintenance personnel, should

be specified. Record keeping and reporting techniques, including documentation

of individual progress, shall be consistent and compatible with existing Army

information systems. The plan should include procedures for revising the

implementation and management plan. Equipment, maintenance, spare parts, and

logistics requirements must be specified. The contractor shall also address

issues related to interfacing with an automated system to support transcript

requests, which is under development by the Education Division, ODCSPER.

Procedure. The FSU-Hazeltine team sought to provide an initial estimate

of the hardware, maintenance, logistics support, and operations requirements

for the computer systems to be deployed. The operation of JSEP in the

education centers was conceptualized, including the soldier flow, the

recommended decision points, and the interfaces among the various elements of

the program necessary to accommodate computer based instruction.

Findings. The JSEP management plan is intended to be used to manage

soldier flow through an education center and through JSEP. Many administrative

and procedural considerations in the soldier flow must be verified before

detailing the complete management plan for each of the major system elements.

The proposed JSEP soldier flow and decision model is shown in Figure 2.

As shown in the upper left hand corner of the flow chart (Figure 2),..

soldiers enter the education centers for a variety of reasons. Reasons and

procedures vary from one Army post to another. Normally, a counselor will

refer a soldier to take the Locator Test if the soldier has not already taken

it. Once the Locator Test has been taken, the verbal and math scores will be ' sent to the commanding officer (CO) along with a list of PCs in which the

soldier is likely to be deficient, and a list of MOS related task indicator

statements that the soldier is predicted to encounter difficulty in mastering.

15
Enter JSEP

Locator Verbal IMath

JoJ

*Designated 1

MOS-relnated JI -

Designated M, r, I I1 I I nstucio

Diagnostic Test I 1.1. . . . . . . . ,:.:..

battery i--J .

LesosIntrcio rnsut io Insrucio

Strateges.

Desifgna ted rr--I m --

m-- r . . _jj I nstruction .:::

s in r Posttest

*Ifall fail \ tl 1

posttest pPPOSttest q \XPOSttest r :7

Lessons I nstruc tion Instructi on Inst-ctio

eosttest p e osttest q IPosttest r -::':

Sumimative i' ~i~ lri":

posttest in... .- ,-

units m-.-tr

Performance Test

E CompleteJSEP

Figure 2. JSEP testing and learning of prerequisite competencies.

16

• . %
i • ' 

,. 

,, 

On the basis of such information, the CO will decide whether to send the 

soldier for further diagnostic testing. 

The Diagnostic Test will be administered in an adaptive testing form to 

prevent the possibility of a soldier being assigned excessive numbers of test 

items in the case of extremely low locator Test scores. Adaptive testing is 

designed to 11 tai1or a test" to the performance level of the soldier, thereby eliminating the need for all soldiers to see all test items. The absence of an 

adaptive test could, therefore, lead to administration of more items than are 

necessary. 

Upon the completion of the diagnostic testing procedure, recommendations 

for the array of lessons the soldiers will be assigned and an estimated length of time to complete the lessons are reported to a decision authority, most 

likely the soldiers' commanding officers. If commanding officers release the 

soldiers for JSEP, they then report to the education centers for the JSEP 

orientation program and the instruction. 

The JSEP curriculum consists of the following components: 

1. Orientation to the purpose of JSEP, its structure, operating procedures, and instruction on how to operate the computer. 

2. JSEP Locator Test. 

3. JSEP Diagnostic Test. 

4. Learning strategies lesson with embedded components in the 

1 onger 1 esson s. 

5. Shorter lessons which include three elements for all PCs: 

{1) a pretest, (2) instruction pertaining to essential 

rules, operations, and c9~cepts; practice exercises and 

content to stimulate recall of prior PC learning~- and, (3) the lesson posttest. 

6. Longer more detailed lessons on selected PCs of high 

priority with post tests. 

7. Summative posttests which are criterion-referenced 11 final 

ex ami nat 1 ons1

' covering a 11 instruction received by each 

soldier. · 

8. Performance-like test consisting of a CLOZE reading test on an 

.. . appropriate Soldier's Manual. 

JSEP Process. The process through the system can be expressed in terms~its bas1c conf1gurat1on (see Figure 2): 

. . 

1. Soldiers take the JSEP Locator Test and depending on their MOS, low 

math, and verbal sr.orers are routed to a tailored Diagnostic battery. 

17 
41.

2. On the basis of Diagnostic test scores, soldiers are routed to the

short lesson pretest for selected PCs. The lesson pretest consists of 3 items.

If soldiers miss any of these items they are routed directly into the short

lesson.

3. If all three pretest items are answered correctly, they take the

lesson posttest. If the soldiers pass the posttest, they go on to the next PC.

4. If they fail the posttest after the short lesson, they are then routed

into a longer lesson that also contains a complete posttest.

5. If soldiers fail the posttest after the longer lesson, they will then be identified to the instructor for an alternative form of instruction.

6. After the soldiers have completed all of their assigned lessons by

passing all the posttests, they take a summative posttest (comprehensive

examination) consisting of perhaps 30-45 items drawn from the pool of items on

assigned lesson posttests.

7. If they pass the comprehensive examination, they are then ready to

take the JSEP performance test which is composed of job specific test items.

8. If soldiers fail the summative posttest, they may be given remedial

instruction, which may consist of re-routing through short or long lessons,

small group or individual tutorials, or lower level self-instructional

materials from other sources.

9. After completing the remedial instruction, soldiers retake a new form

of the posttest. Each soldier will have 3 opportunities to pass the

comprehensive examination.

10. Performance testing will also be included during the full scale tryout

to ascertain whether JSEP learning gains are transferred to job specific test

items.

11. Final decision on appropriate performance testing will be made during

Phase III.

Findings. At this point in the development of JSEP, the Task 4 report

will be far more heuristic than prescriptive since there are many unanswered

policy and operations questions. And perhaps just as important, the final

configuration of the equipment system has not yet been approved. The major

purpose of presenting the present report before these issues have been resolved

is to permit reasonable and responsible budget planning.

We anticipate the implementation and management plan will go through

several revisions, the first of which occurred in Phase I. In Phase 111; the .

continuation of the implementation and management plan occurs in Task 19, the

Technology Transfer Plan.

18
I,

Task 5

Task 5: Adapt or adopt design specifications, including instructional

specifications, engineering requirements, and human factors

considerations. .

Requirement. The solicitation requires that we adapt or adopt design

specifications, including instructional specifications, engineering

requirements, and human factors considerations. To do that, we used, in part,

the specified products from the TRADOC sponsored contract with RCA. A complete

list of these contract products is presented in Appendix A of the Task 5 -.-

report.

Procedure. The first step following contract award was planning the

approach to the Job Skills Education Program (JSEP) design. Dr. Laverne Cook,

the RCA Principal Investigator on the MOS Baseline Skills Analysis Project, participated in a Florida State University (FSU) sponsored meeting of all

principal project agencies to describe the RCA work. Subsequently, a team from

Army Research Institute (ARI) and FSU visited RCA for a thorough two-day

briefing on the details of each of the RCA contract products. Following this

meeting, a plan evolved to integrate available RCA results and contract

products into the JSEP design.

During the Task 1 in-process review, the project schedule was revised to

accommodate the delayed delivery of RCA products. While it was possible to

accommodate the initial delays in receiving RCA deliverables, subsequent delays

required extensive project redesign. FSU began work on the lesson

specifications and assigned the work for engineering and human factors to

Hazeltine Corporation.

To insure prompt receipt of the RCA contract products, we issued a purchase

order to RCA for duplication and mailing of each of their required analyses and

reports. It was our initial plan to use RCA products to the fullest extent

possible so that we could concentrate our resources on those aspects of JSEP

unique to our contract.

Findings. Based on the RCA Taxonomy, the initial FSU lesson

specTificatons were developed on a selected sample of lessons thought to be

representative of the total lesson population.

Some lessons, thought to be fairly typical, were carried through the

lesson development procedure called for in the lesson specification. In order

to model in a small way the procedure planned for the entire program, two

lessons went through all the design, development, and evaluation stages. These

lessons were then field tested on an installed TICCIT system at the Marine

Corps Communications Electronics School, Twentynine Palms, California. Based

on the results of this field test, the remainder of the 180 lesson

specifications were developed according to a revised procedure.

In addition to the specific instruction designed for each of the initial

RCA Prerequisite Competencies (PCs), general instruction on learner strategies

was planned to permeate the entire curriculum. These learner strategies are

intended to encourage and support JSEP soldiers in learning and managing their

19

7.:"* . . . .. ..... -:'-
Study.

A draft of the lesson specification was designed and refined, and a

working version was developed. Prototype lessons were written to evaluate thek

entire development stage. The knowledge gained from testing the prototypes led

to the development of an approved lesson specification format.

Lesson specifications for 180 PCs were developed. A prrqust

cornpetency was defined by RCA to mean a generic basic skill that soldiers must

nain~ order to learn specific tasks on their skill level l and 2 jobs. These

PCs iire MrectTy related to-job task Fer ormance.

Each lesson specification was reviewed by an experienced Army Noncommissioned officer and reserve officer at FSU to increase the Army relatedness of the lesson specifications prior to being submitted to ARI. Each

test item received from RCA was reviewed by experienced test developers to J

insure that it was consistent with the PC, as these PCs were interpreted by FSU. Indicator statements are RCA contract products which illustrate precisely *how each of te PCs is used in each of the 94 MOS analyzed. It is these

indicator statements that truly reflect the job-relatedness of the PC and are

the basis for the instruction designed to teach it.

A thorough analysis of all engineering requirements was made for the

* microcomputer system being employed during the development effort. Further

discussion of the equipment and maintenance requirements for the entire imple- * mentation of JSEP are contained in the Task 4 Report, The Implementation and

* Management Plan.

Human factors considerations for each of the prescrIlbed systems were

* analyzed.

Use of findings. The entire effort of Task 5 produced two products: the

* lesson speifications for the 180 PCs. and useful analyses of all engineering and human factors considerations. The lesson specifications will provide the

framework for the development of the short and long lessons. The information

gained from the engineering and human factors considerations will aid in the

development of the implementation and management plan.

Task 6-

Task 6: Develop evaluation standards.

Requirement. The solicitation requires three kinds of evaluation

standards.

o Evaluation standards to determine if the system meets the stated

specifications for courseware, software and hardware.

0 Specific formative evaluation standards to determine if the system is

effective as a functional JSEP.

o Job-related evaluation standards, such as summative evaluation, to

determine if the system is effective in improving performance on the

job.

20
- -................... ....

The product of Task 6 Is a detailed report describing the evaluation

standards. Approved evaluation standards will be used in the evaluation plan.

Procedure. The strategy for developing JSEP evaluation standards was

first to articulate goals, objectives, and process steps to describe the

intents of the project. The objectives and process steps were then reviewed by

the FSU-Hazeltine JSEP executive committee and revised accordingly. Following

these procedures, an outline of performance standards was initially developed

and these were again reviewed by members of the FSU-Hazeltine JSEP executive

commi ttee.

The performance standards are, in the main, stated in the form of either

process criteria or in the form of products which contain certain kinds of

information or analyses. For example, at this time we prefer to state that we

will analyze and report the pass rate of troops at a field test site while

controlling for a variety of possible intervening variables, rather than to

predict a certain outcome. We believe that process standards must be used

during the design and development phases in order to preserve the decision

freedom until empirical data are available from the tryouts. The setting of

desirable outcome standards will be possible later when the relationship

between input variables and outcomes become better known. Only at such a time

can we set a standard to address the "how-much-development-is-enough" issue.

Findings. The following eleven objectives were identified:

1. Establish project review, control, and decision-making procedures.

2. Conduct a review of literature and instructional materials related to JSEP

purpose.

3. Design a job-related, computer based prerequisite competency curriculum

(JSEP).

4. Design an instructional support system to field test the JSEP curriculum.

5. Design a management information system to direct, monitor, and report

student progress in JSEP.

6. Develop a job-related, computer based prerequisite competency curriculum.

7. Field test a job-related, computer based prerequisite competency

curriculum.

8. Compare alternative computer based delivery systems in terms of

effectiveness, efficiency, and benefit to US Army mission.

9. Evaluate the potential impact of JSEP on Army job skills, educational

programs, and soldier career goals.

10. Explore the feasibility of using JSEP for the awarding of high school

credits and credentials.

11. Prepare specifications for the dissemination of JSEP Army-wide.

21
The process steps to accomplish each objective were identified and

performance standards for each step were listed. If there is a single I

* overarching evaluation standard, it is the degree to which JSEP, as

systematically produced curriculum can effectively eliminate PC deficiencies.

Use of findings. On the basis of the review of these proposed evaluation

*.. standards-y ARI, and their subsequent refinement, the evaluation plan (Task 7) will be developed to specify the methods of inquiry, the measures to be used,

the population sample, the proposed analyses. Until that final refinement the ,...-

important question is, what should be observed about the program that will be

the most revealing for what kinds of audiences.

I Task 7

Task 7: Develop JSEP evaluation plan.

Requirement. The solicitation requires the development of the Job Skills

Education Program (JSEP) evaluation plan. This plan should set out procedures

to establish whether any improvements on developed criterion measures were

attributable to JSEP. The contractor shall develop data collection instruments

' •and a data collection scheme which will reflect evaluation issues as well as

quality control and revision.

Procedure. Preliminary drafts of the evaluation plan were developed by The Florida State University JSEP staff and underwent review by the Hazeltine

internal review committee, the Army Research Institute, and the American

Institutes for Research. Additionally, ARI, FSU, and Hazeltine personnel

reviewed elements of the evaluation plan pertaining to the preliminary tryout

scheduled at Fort Rucker. Revisions were made to the plan according to their

suggestions.

It is planned that additional refinements in the plan will be made as it

moves from concept to operation.

Findings. The evaluation objectives, procedures, and data collection

* instruments constitute the findings of this task. The evaluation plan is

divided into two major parts: the evaluation of the preliminary tryout of JSEP

at Fort Rucker in 1984, and the full scale tryout of JSEP at two TRADOC and two

FORSCOM sites in 1985.

Use of findings. The evaluation plan will be implemented during the

prelimTniy and full scale tryouts. The data will provide the information

required to assess the effectiveness of the JSEP system.

. Task 8

• . Task 8: Develop cost benefit tradeoff analysis. '-

Requirement. To develop a list of options available with computer based

,nstruction (01), their advantages and limitations with respect to JSEP, and

their costs for varying numbers of units or systems.

22

• ... . .. "

.. , .. .-.. ',. , ... .-.-.. ..,.• .. _-,,.. .. . , .. . ,. ,. . ,' .'. - - **,: * % **,*-' j :: , ':
r (. r r ~ J '. -" .

Procedure. The contractor conducted a review of the literature relevant

to CBI and consulted with educators, psychologists, instructional systems

designers, and computer scientists in academia and private industry.

Findings. The options applicable to JSEP are whether to incorporate an

audio or video capability into the instruction, whether to make hard copy

(printer/plotter) available, whether to use color, and some environmental

considerations of the classroom.

There are certain features (adequate screen resolution and partitioning

and comparable graphics, text, and student response facilities) on the systems

studied that were also evaluated in terms of JSEP instruction.

The advantages and limitations of the options and features are discussed

and, where applicable, cost estimates are provided. This report will enable

U.S. Army decision-makers to determine the configuration of JSEP.

Utilization of Findings. The cost data will be used in models of possible

configurations to compare costs of various systems.

Task 9

Task 9: Conduct predictive cost and training effectiveness analysis.

Requirement. The solicitation requires the development of a predictive

Cost and Training Effectiveness Analysis (CTEA) model specifically for JSEP.

Procedure. The contractor conducted a review of existing CTEA models

devised forFiilitary settings. Areas for discussion included: , ".i

o the classification system

o predicting cost of training configuration

o the costing process

o estimation of training effectiveness

o comparison of alternative training programs

Findings. Based on the review of existing CTEA models, and the wide range

of features in JSEP, we developed a model to accommodate JSEP's unique

requirements. The features to be addressed and their bases include:

o the self-paced nature of JSEP rendered assumptions underlying some

existing CTEA models inappropriate.

o the open entry, open access characteristics of the program requires Nopen system" methods as opposed to "closed system.

o the need to synthesize CTEA models into one which addressed JSEP

characteristics.

23

. -, -. :
Because of the need for data not now available, we plan to conduct the

predictive CTEA after the Phase II Tryout. The effort to date has produced a L

CTEA model that will address the unique applications problems identified during

the analysis and model development.

Task 10

Task 10: Develop standards and a plan for the civilian academic community to

award a high school diploma based on completion of JSEP.

Requirement. The solicitation requires the development of standards and a

plan for the civilian academic community to award a high school diploma based

on completion of JSEP.

Procedures. The FSU response called for the team to confer with state

officials, identify state high school credentials requirements, review existing

US Army high school diploma programs, analyze JSEP materials, confer with

~US regional Army officials. accreditation officials, and maintain communication with appropriate

The team has complied with all of the prescribed actions above including

visits to Forts Bliss, Carson, Lewis, Monroe, Riley, Campbell, Bragg, Polk,

Rucker, and McPherson as well as the sponsors of the program. Visits have also

been carried out to selected school districts, and state departments of

education, and to Middle States and Southern Associations of Colleges and

Schools' Commissions on Higher Education.

The purpose of the Task 10 report is to identify considerations or issues

to be answered before the Task 10 plan can be implemented during Task 19 in

Phase III. Following the identification of issues, a list of concerns will be

given. Then, a plan will be provided in order to comply with the product

delivery requirements.

During the third year of the Contract, the FSU team will formulate the

Basic Skills Profile that will then be circulated to federal, regional, state,

and local education officials directly or indirectly associated with the Army

installations and the twenty-two (22) states identified as having the highest

military populations. In order to get credit for JSEP work, the procedure to

be followed for soldiers in each of these states will be:

1. Complete the JSEP Locator Test.

-2. Receive results which include identification of (a) deficiencies 4

related to MOS against which the JSEP diagnostic test would be.-

administered, and (b) the non-MOS related deficiencies which would

become the basis for the Education Services Officer to prescribe offduty study.

3. The student would follow the learning strategies of JSEP as outlined

in the Implementation and Management Plan.

4. Upon completion of the JSEP process and demonstration of competencies *

24
for MOS related areas, covered during duty hours (and satisfactory

completion of off-duty work), certification would be made by the

Education Services Officer for recognition at the school district or

State Department of Education level.

5. School district or state awarded high school diploma will be awarded

on the basis of the Basic Skills Profile or state prescribed

competency test associated with step 4 above.

Findings. We have found that transfer of credit between accredited

postsecondary institutions may not be the same as acceptance and credit

recognition between secondary (high schools) and postsecondary institutions.

Therefore, it is important to distinguish between a Competency-Based Adult

Education (CBAE) high school credential, a high school vocational education

credential, or an academic high school credential.

CBAE programs offered by state certified or accredited high schools have

as their purpose and focus adult educational literacy. All of the twenty-two

(22) states identified in Task 10 report that establishing the actual

requirements for the CBAE high school credential is the authority of each local

school district. The desired outcome, however, is demonstrated competency in

the basic skills. Telephone interviews with the officials at the twenty-two

(22) states revealed a strong consensus that the grade level proficiency of

CBAE high school graduates approximates eighth grade level. Therefore, the

literacy skill achievement level of JSEP should meet or exceed that of the

typical school district CBAE high school graduate.

Those two-year and four-year postsecondary institutions having an opendoor admissions policy will find the CBAE high school diploma acceptable for

admissions whether It is a local school district program or the JSEP related

credential. Selective admissions postsecondary institutions will not honor the

CBAE diploma for admissions purposes.

Some states have unique diploma requirements. The diploma unique courses

range from such courses as "Texas History" in Texas to a course on "Communism

and Americanism" in Florida, where the military training itself is acceptable

to Florida to meet that course requirement.

Present policies of the US Army make it possible for each installation to

provide an opportunity to a soldier to receive a high school diploma through

the combined on-duty and off-duty studies program. The high school diploma

programs are offered through a contract with a secondary or post-secondary

institution and this policy will be as viable and appropriate with JSEP as it

has been to date.

The specific requirements of each state are presently being accommodated

through the obligation of the contracting high school program institution at

each post to have state authorization for the high school credential.

Additional content will be necessary through off-duty work if the goal is

blanket award of high school diplomas for all personnel completing the JSEP

programs. It is apparent that great variation in the high school diploma

programs now exists. At Fort Bliss, the program is truly a CBAE program and

therefore consistent with JSEP.

25

.z--,' -.., ._'---' ,' .' .- '".' ........................................ c"... -, "-.- " - -, .a," ," ',, .. *. *. . *** .* .* :.._:--''. ._.
At Fort Carson, the actual program is an academic high school curriculum.

In that case, present Army personnel as well as those in the future would find

it obligatory to complete the required Carnegie units of academic subject

matter regardless of completing the JSEP or their scores on the Test of Adult

Basic Education (TABE).

At Fort Lewis. the contract is with a postsecondary institution that also

has authority to award the high school diploma. An analysis of their program

of studies reveals a general education curriculum and officials of that

* contracting agency indicated credit would be given for completion of JSEP.

Use of findings. There are important assumptions basic to the proposed

plan.7hTirisi hat the present Army educational services policies will

continue whereby a contract for off-duty course work will be made with an

institution awarding a high school diploma.

With few exceptions, the state officials interviewed saw the JSEP program

as offering an identifiable basic skills program that could be used as a model

or prototype for CBAE diploma programs.

The Locator Test has been designed to identify mastery and deficiencies of

verbal and quantitative competencies. Since the instrument must deal with the

total spectrum of military occupations, it is comprehensive. Therefore, a

person taking the Locator Test will have demonstrated verbal and quantitative

mastery or deficiency for a broad range of military occupations. The on-duty

participation in JSEP will be limited to the MOS of that individual. Any

remaining deficiencies will need to be considered as part of the off-duty

* requirements of that individual for meeting the high school diploma

requirement.

We believe that this plan will result in the beginning of a dialogue

between representatives of the Army and local officials to adjust the needs of

the program to the realistic possibilities n each of the states. When the

* draft curriculum is ready, it can be circulated in suitable form to appropriate

state agencies. This mechanism will begin the dialogue.

Nothing in our conversations with any of the state officials indicated

*that they were opposed to or were unwilling to consider the approach we

outlined to them.

PHASE I IN PROCESS REVIEWS

*Two-rpRs were held during Phase I.

* IPR I

The first IPR was held on November 19, 1982, at ARI. This IPR was

required as the product of Task 1. The purpose was to present a summary of

issues and concepts that the FSU-Hazeltine team would consider in designing the

project.

26
4-u ~ .'wrrrrw r''7 ' ~ -~ .* ~ .... • :,-.

The principal investigator briefed the COR and other Army unit

representatives on the FSU concept of what the Army will have at the conclusion

of the three phases of the project. Appendix A is a booklet containing

photocopies of the transparencies used in the briefing. The discussions that

followed the briefing helped clarify various points and assisted the FSUHazeltine team in understanding precisely what was expected under the contract.

IPR 2

The second IPR was held on April 28, 1983. This IPR was a combination of

the Task 5 IPR and the Phase I IPR. The purpose was to insure that FSU and the

government were in agreement on the planned functional specifications and to summarize accomplishments of Phase I.

Dr. Harry O'Neil gave a brief history of BSEP. Dr. Beatrice Farr (COR)

described excerpts from the results from needs assessment performed by the

American Institutes for Research (AIR). Dr. Branson (Principal Investigator)

reported on the progress of the Phase I effort, identified issues requiring

clarification and guidance, and recommended proceeding with Phase II. Dr.

Laverne Cook described the RCA effort and the new schedule for deliverables.

A number of key issues regarding development and evaluation were raised by

Gary Beanblossom, Ed Neff, and Les Orech of Education Division, ODCSPER, Key

among these issues were:

1. When soldiers should be referred for JSEP and by whom. :-

2. How we will know the degree to which JSEP is effective in improving

job performance.

3. How to make the program serve an Army-wide audience.

It was agreed that AIR would design an evaluation study that would

establish the contribution of JSEP to Job performance on the first duty

assignment. Another major issue centered around the question of how to provide

or supply copies of existing off-the-shelf materials. It these are to be a

part of the specified JSEP curriculum, they should either be available or

should be put "on line" in the computer to assure a future supply to users.

It appeared to be the consensus of the group that Phase I was proceeding

* according to plan and that no significant issues with regard to performance on

Phase I remained unanswered.

CONCLUSIONS FROM PHASE I

FSU and Hazeltine Corporation have provided ARI and associated agencies

with all contract deliverables required in Phase I of JSEP. Phase I of JSEP "i

has produced the groundwork for the remaining two phases, and for the successful implementation of the JSEP program throughout the Army.

27

.. . . . .. . .. . . .. . . . . . . . . . . . . . . . . .
DIRECTIONS FOR PHASE II

Phase II is a logical progression from Phase I. All the preliminary planning, analysis, and thorough preparation required before the full scale production 

Phase 

of JSEP courseware in Phase II has been successfully accomplished in I. Phase II calls for the development of 300 hours of instruction from the specifications designed in Phase I. Also, Phase II calls for a preliminary tryout using a subset of curriculum materials.

The specific tasks in Phase II are:

Task 12. Select hardware, where appropriate develop software, develop instructor training program and courseware for 300 hours of instruction.

Task 13. Conduct preliminary tryout

Task 14. Phase II Report

production 

Phase II will lead into Phase III which calls for a continuation of the of courseware (120 hours), a full scale tryout at two FORSCOM and . two TRADOC sites, a cost and training effectiveness analysis, the development of a technology transfer plan, and a JSEP Final Report.

28

S-?L2-

......................
REFERENCES

* Department of Commerce Bureau of the Census (1977), as cited in Conahan, F.

(1983). Reprtto the secretary of the Army: Poor decison and

management per Army's7basic skills education program (GAO/FPCD-83-

19.W asW~hington, UU VU.S. Gner F~counting office.~

Sticht, T. G. & Hooke, L. R. (1982). Instructional system's design for V

the Army's on-duty education program. Research Product 82-4.

Alexandria, VA: Human Resources Research Organization.

Weinberger, C. (1982). Militar manpower task force: A repor to the

*p reident on the st u a R o pe ts-6Tthe -a-v-1u .nteer fo rce (GuPO

Publication No. '1983-395-730/1) Washington, DC: U.S. Go-vernment

Printing Office.

292
-7. 777

APPENDIX

TASK 1 BRIEFING TRANSPARENCIES

30
d.,

**~~~~~~~~A~~ 

%*. ' * * ~ .4~ ~ .~

4-J -

..,. ... ... ... ... .... ... ... .

~* ~*3 mm I.

ULU

U.

'A)~ mama 

'.n. 

-- Av-L.'

311

0) '. 

.1
* .r.

0) 4-

le~

0 .mJ

w0£.0

SOMME ~ I '...'1

- * '4'IN*

* - I .V Ine'

U 9 

- ;N
* -31

0 c

* -~;i 14 ~ ;

TAW

LM-'

u I 

cu~

~c

V4. ir

L6~

cc~..

0i1"3

(.11 A'i* -
4-1

4M4)

CL L)

>-4 4*

CD, 0

4-f2

f-i-a 4ca

(Dlm 0 .~,t

- U) ~ - WUWf

- -... -- -- -
-~~~~~I 7-,*~~ - *

J ' lp 'A j * 42.- ~ '- ~ ~ *.

a ~ ~ ~ ~ a '

-F -W

-s C

0 4pG

C 0 0

w w

C CA

0 0I

0 ~ O *'>i * Fa~f;.

44-

'4~~ ~ CA ~ r C4~ 

4~~~c -m. - a
NAI.

11CD a.

000

CU

-I.77

0M) - 4

0. -a-

_ 0 - I

-$Jl .J* X~m *~

wI

w 'I'36

Cm)~~?P Q(-)'Im' in 

~m'Linaa 4 m'a

.~~. .
V' -7

.1s~

A,

01) C2

kI CL F

CL

CLr CL -~ **

0k 3*

A0. M17 CD

mCO0)

le ~ F.

~cc

0137
.. 'a,

A 1LL -

Jr, -,4, 'v

44,- Zi

ILI

ciLi

0

CL#

-,~~a C'-_ M~LL38 :~~-- '
7777777-7- - - N 

Al M~ 441

I p aa * ddl WI

~%

I;v'

0 0 c

k. I

AA Ap~i

114- Nw~I W.- ARS
A 'r

-~Zo

-' 4 -'

m~ 0 

- 4-0

4-0 4- En~

Inc 0 w cn

Gtm cco Q0 4-

00 4-0

~oq EE.

0) ) 0

40*~h V

iet

- ~.'*-L I
..... ..... IV

0- 01

U)o 4- t

4-00 k.0 CU 4-0.

Q *vn u~)O. u~n

En 4w cn

>I6.

1

41'.

a,.:ii.'
-- ~~~~~~ .. - - - - C -

CL.T~.'r-......

* **'D

* r %-

Q) 4)CLC

w 4Z

( C

4-0~ Ac 0 

LLh C/) a +j

40 0 440

(D 0C 4- *i

0j 0 .D"

(AL( a42 c 4

C -ZA
Crr

46 

W

CL 

m 4-0 I 

~ *iII

*v 

-

-

5 C 

1* 

,43

7-2-
V1.

o C

0IL.

C Cit

C~~ .) M 

ECE 0 0 0. i

4. !07.

- ~ I. .- - -
__~VII- 31 *, 1 ' *A *

VIIt

1) ,,

0 0 1

44,

4-*45 0) .~Al
~ ~ ~~ .

..___ _ .. ... r~

4-0-

4-0 m 4-

07 

-

44-

, Ir
'IN.

MO.M

Q -%-

j~ Ca 0a

0- - J

a o) *

CL 1 0.

cc. it

~0 47
T~ T- V'U L -

tv,.

-a. ."~'05

T1

CC

C C

4-

> 0 4.24-

U.U

0 0 Eh.E 0 iu0 

C ~~k.- lag .IV>. nC

-rep)'4-

CU CUW En' Z".- 0W - L.. -.. C E
- C- . - ~ L W -t-v'.it

* * ,, . ~ .f. '.4ark,

A7 s. r3!;, : : -- ~~~%& ~ ~ 4~ 4 ~ ~ ~ sr

X..

4-04

U. .D

0 C - i -p.

Ij WOC '*~ '~

cc - ' as

a ~ a.2_ "-0 0~ rL 0 II C 5 

4ANW

__ 0O . 4- 

-~~ ~~ CCA' 0 - .- .

Lim - 4- 0VI AMT Pi 0M 
-- yx...- F -. 

.41,

jr.o

4-'.

CD~

0 0

o 4-0

0) 4--

0)-

C 4 .. ~ 1.4.0

050

XV)E _ 0)0) )
-. 4~41-- -. . . . . ' . 440167.. . .

vl~4: JF

-- 0--

LM .s..1

4- 00L.

mom .. 4'

Cn C

C) -- 41*-

r .m w- 'W 4

0 0 0 4-

I~~~~ )t~4.I 0 C..C 

0 04'4-~

E D E 4aO

'> > f..v>

jiF"

51 4 .4
-in3

Va A

In4-0

4-' 4- 04-00 I

0 M r

cu m5 CL

0))

C C

+j -~0) ,.

CflO~ Ca.0LC, .7 m

0)) 4 0 -0 4-d

D CLl 0 -

4.C 4-'

S 4-w,.. ~ 0 In~nO0 0 a.~CL'4IO...

0). a, ICn

522
+.3

'K. jw VJ

* 0D

* -V. '

.1 **L *',own-%%

m4 4-A

cc~ :'

4-' (

0 0

CL)

0 d

a)~C zA.

+c a

a)c a

a) d)

40 CL kNC'

mCCL 0 a

j CL

C 3 .............. u

Aw A 4

, ~~t A1 ~
V Z 4 - - "I

*1 -~ ~.~*..AIM-

-V. ~ 1** ~4 -'

how.

-Ql

GOOMP

AL. CL C

SoS

C L

4-t*4

(D U. 4-' 4-' 4-0 *IV

cc ~

U. OA')as,; 0. 0 a) C)>" ~~"IN

4-'54

cv, - ~ .~ 

7;- IW-T~.

9%i4.-
4.4i,

* ~Ag

4- J

0 0

_ C, -J

AA ..

CL) *

_ Iv~m ..

3 U, cc

'p ~r*,..i

TLO

Al

55
~ ~ 2' *1%XWn...14~' v l 7-. 7

N--

4-'4-J

4r

4-4-0

CC ccaO r

m-0 - a

__ 4-', 4-J, 

4-'0 c

C C

(El)~~ a)I aa

LLm

Urn

561~A

Is aa .
4-J.

cr-w

..

-4-J

-0 4-'

-~ '4-00

- Cv In 57
.4K

,j l .4' , ,

I IC -

4-d 4- IK

4.

0C wi

4-0 

.O

00

4-0~

ao a' u.

58~

MIA--~ ME1t 01 

a~ -%
4,4 Wjj!"r

~~Lai4~ 4 Th *< ~ - ...-. ~: ~ 4. .* ~,. cc ~ ~ -.

z 0

w * L

LLL

-w - - - -

ccc wC

0 WD W 0

E E

-t CL w Z. o

M M

- )wC 0) - .~.4- cc

> L

010

.~59

I J 16
-. - -- . -v-- it!- - - ,.*

r% ) 0) N

0N 0r cJcO T-F 0 4--1

0- 00 C -0

:--i

ri--

7a .

0

Aa C. O

0))

L U

IIc -

.5 0.. ~ ecc

60o
0 0 v

CNCV Cco000

~ 0 0 00 IY I t-coI

DooI 1111

0. 0

U.1

0-0

cm 00

ji' 0E LL

A. t L.

6A0- YIN,
1, WO - ;,iL AA r' . !t 'S

II

t~A.

C)c coD coo'-

(OC

0 0 00 00i

4-0- o.C ~ c

4W. 70a 0.

CL.

). >

LILa
1 7. , 1- 4-

LOC 00N N

04 CN CN CY) -

000 000

0 00 0

CDI

co w

40)

- ~C C ~:*~-

4-- L.4-

ED)G 1- x

4L C

X C0)

LL)) L6 *

0.3

~ in0~Awl
** -1 77 .7K 7K

4 A ;~/-..

~'.'__ __ __~~ " *~*2 O W L, ~-;

00 00

4~~~( ~ V ~( 04 

I...

(00

0C) Law; ~

&w..

II .s.,** >~CI~c.d '
7,77 -7

RCA ETS

TASK ANALYSIS LOCATOR & DIAGNOSTIC TESTS

f 4r

FSU . L,

420 HOURS

FBSEP INSTRUCTION

2SO Trot 2 ie

, 'I

65

4.4
Nnr

U1dV cm

3303

* AON U

d3S C

CY,

0

Nnr LL

coo

UdV

ILU

AONinr an

UVMI '0

SNvr 

83:71 it______ m

. N C#) q. ' CD P% co C 0 C M CV) Ln tA f- to Ch~

66 .

*SA 1W. AT W- 3
..* :7 .s 17 - . I

, -> -

: 'V

CL (n

LU a,

4- 0LO LO ~

-D C.. '- 5Irv' 

Um -Lm -l

Al..

-~ 0) 67
vu.__ ms .

"C'

Ja >

E~ 4 .. 4- 4-0 M.

co~.

4, tTJp~ 4*r

-4 LL1 r c

4-0-

sr3 IA ~ C6 "4d

E o 68 00 - .q. 

4-Pi"