Research Note 80-34

EVALUATION AND DESIGN OF A PROTOTYPE SYSTEM FOR

MANAGEMENT OF SUPERVISED ON-THE-JOB EXTENSION

TRAINING PROGRAM (SOJET)

(D

QC

ft

C. Dennis Fink, Terrence J. Masterson and

W. Daniel Dantzler

Kinton Incorporated

TRAINING TECHNICAL AREA

o

DTIC

ELECTEI

MAN ;: 1983 ^

U. S. Army

Research Institute for the Behavioral and Social Sciences

February 1980

ADD'OV.'O tor ouDiic e'eaje; distribution unlimited.

83 G3 21 121
Unclassified

SECURITY CLASSIFICATION OF THIS PAGE (Whan Dmt» Entered)

REPORT DOCUMENTATION PAGE

1. REPORT NUMBER

Research Note 80-34

2. GOVT ACCESSION NO

4. TITLE (end Subtitle)

EVALUATION AND DESIGN OF A PROTOTYPE SYSTEM FOR

MANAGEMENT OF SUPERVISED 0N-THE-J0B EXTENSION

TRAINING PROGRAM ( SOJET)

7. AUTHORf«;

C. Dennis Fink, Terrence J. Masterson and

W. Daniel Dantzler

t. PERFORMING ORGANIZATION NAME AND ADDRESS

Kinton, Incorporated

1500 North Beauregard Street

Alwxandria, VA 22311

11. CONTROLLING OFFICE NAME AND ADDRESS

US Army Research Institute for the Behavioral

and Social Sciences, 5001 Eisenhower Avenue,

Alexandria. VA 22333

14. MONITORING AGENCY NAME ft ADDRESS??/ different from Controttlng Office)

16. DISTRIBUTION STATEMENT (of thle Report)

READ INSTRUCTIONS

BEFORE COMPLETING FORM

3. RECIPIENT'S CATALOG NUMBER

5. TYPE OF REPORT ft PERIOD COVERED

Final Report

15 Sep 1978 - 15 Feb 1980

6. PERFORMING ORG. REPORT NUMBER

B. CONTRACT OR GRANT NUMBERf»)

MDA903-78-C-2038

10. PROGRAM ELEMENT, PROJECT, TASK

AREA ft WORK UNIT NUM8ERS

2Q162722A791

12. REPORT DATE

February 1980

IS. NUMBER OF PAGES

15. SECURITY CLASS, (of thle report)

Unclassified

15«. DECLASSIFICATION/DOWNGRAOING

SCHEDULE

[

I

I

Approved for public release; distribution unlimited

17. DISTRIBUTION STATEMENT (of the ebetrmct entered in Block 20, it different hem Report)

It. SUPPLEMENTARY NOTES

19. KEY WORDS (Continue on reveree tide it neceeeery end identity by block number)

Correspondence Course Pretests

Course Supervisor SOJET

0n-the-Job Training Student Feedback

One-Shot Mailing Supervised On-the-Job Extension Training

Personnel Turbulence Training Program Management

10. ABSTRACT (Camtmuo em terete* etm H rmceeeory ewd Identity my block number)

— ^ This report describes the formative evaluation of a new system for

centrally administering nonresident training through the Army Correspondence

Course Program. This system is designed to support first line supervisors

in conducting on-the-job training (OJT) for tasks performed under their

supervision. The major objective of this evaluation was to refine administrative procedures and to streamline manually administered portions of

the system for eventual incorporation into a computerized Information

management system.Criteria used during the formative evaluation of this

DD I Jnetn M73 EDITION OF I MOV CS IS OBSOLETE Unclassified

SECURITY CLASSIFICATION OF THIS PAGE (When Die Entered)
unclassnlea

SECURITY CL IFICATION OF THIS PAOEQWii Dmt» Knffd)

o>

vV

Item 20 (Cont'd)

^system were field acceptance, administrative feasibility and costs« The

training materials administered under this new system were specially

designed to support duty position OJT for combat arms operations and

intelligence sergeants and assistants.

r^

Major features of the new system are: (1) each student is required to

have a supervisor registered in the system; (2) pre and post-tests for each

training package requested are mailed directly to the supervisor, lesson

materials are mailed directly to the student; (3) all training packages

requested are mailed in a one-shot mailing to insure that supervisor and

student have full flexibility in sequencing training to meet local priorities;

and, (4) supervisor functions as the OJT trainer by counseling the student

on task training needs, by administering pre and post-tests for each

training package, reviewing test results with the student and forwarding

results for Inclusion in the student's centrally maintained record file.

Student and supervisor responses obtained during questionnaire and

interview surveys were highly supportive of the major features of this

system. The separate mailing of test packages to the supervisor and lesson

materials to the student makes the new system cost approximately 32% more

per student per year than courses which do not directly involve the supervisor.

All other costs for the two systems are essentially the same. Supervisor

and student responses were in favor of retaining the separate mailing.

Data indicate that supervisors were performinlng the OJT trainer functions

and felt they could handle the limited amount of time required. Direct

comparison of student performance under the new system with student performance under existing correspondence course methods not directly Involving

the supervisor was not possible at this time. The existing management

information system does not permit identification and retrieval of student

record data required for these comparisons. Such comparisons will be

possible when the Army's new management information system is installed.

It was concluded that, as a result of the revisions and streamlining

accomplished during this effort, the revised system is ready to be incorporated into the development of the Army Correspondence Course Program's new

management Information system. Supplementary materials can be found in

RN 80-35 and RN 80-36.

A

ii

Unclassified

SECURITY CLASSIFICATION OF THtS PAGtfWhft Dmtm Emmr*4)
U. S. ARMY RESEARCH INSTITUTE

FOR THE BEHAVIORAL AND SOCIAL SCIENCES

A Field Operating Agency under the Jurisdiction of the

Deputy Chief of Staff for Personnel

JOSEPH ZEIDNER

Technical Director

FRANKLIN A. HART

Colonel, US Army

Commander

Research accomplished

for the Department of the Army

Kinton Incorporated

NOTICES

Accession For

NTIS CSA4I

DTIC TAB

UiiQnnci-3'>c""*d

Juctif ie.-vi. i -i...

¥

By

distribution/

Avail^bilitv Corlos

Avail and/or

L^t-'cial

DISTRIBUTION Pr«m»ty distribution of thu ftperf KM baan mad» by ARI. »taaa» addra» corratpondanca

concerning distribution of rtporit to: U. S Army Rataarch Imtuuta for tha S»havioral and Social ScwncM.

ATTN: PERI TP, 5001 Eitanhomar Avanua. AlaMandria. Virginia 22333.

FINAL DISPOSITION Th« rapon moy be- datttoyad «<h#n it •• no lonaor naadad. Plot«» do not «turn ,t to

tha u. S. Army Rataarch ln*tnut* for tha tahavioral and Social SctanoM.

NOTE Trta tindinot m mn raport it* not to Oa conatruad at an official Daportmani of tha Army petition.

union to datianatad by othar authorised documents.
b>

ACKNOWLEDGEMENT

iV

I

The authors wish to express their appreciation for the

qenerous support received from the U. S. Army Institute

for Professional Development. Also, we wish to acknowledge

the extensive assistance provided by Dr. Richard P. Kern,

Contract Office Representative for the project.

ta

i

iii
EXECUTIVE SUMMARY

OBJECTIVES

The Supervised On-the-Job Extension Training (SOJET) program is a

new approach to delivering duty position training to a unit in a nonresident mode. The program's delivery system contains features that

differ considerably from those used with traditional courses. At present the Army use^the program to administer 16 courses designed for

operations and intelligence enlisted personnel in combat battalions.

A field test of the SOJET program was started in the spring of

1978. At that time the program's delivery system had not been

debugged, and portions of the program's administrative procedures were

not compatible with those used by the Institute for Professional

Development (IPD), the agency responsible for administering the Army's

Correspondence Course Program.

The purpose of the present research was threefold: (a) to conduct a formative evaluation of the prototype SOJET program; (b) to

redesign the program's centralized administrative procedures to make

them compatible with those used by IPD; and (c) to complete the design

of the SOJET program by developing a feedback system, primarily for

course developers. In addition, the research project intended to provide baseline data for use as bench marks when evaluating future

extension raining or correspondence programs.

METHODOLOGY

Interviews with and surveys of actual and potential students and

course supervisors, in both active Army and Army Reserve/National Guard,

were used to assess field acceptance of the program and to identify

program features in need of change. Training records were analyzed to

identify the* characteristics of persons who enrolled in the program

and the rate at which they progressed through their program of study.

The administrative feasibility of the program was assessed

through discussions with personnel from the Institute for Professional

Development (IPD). These discussions provided a means of identifying

features of the SOJET central mangagement procedures which administrators considered troublesome and/or time-consuming. Solutions to these

problems were developed and implemented during the project when

possible.

Interviews with course developers at each of the Army's Combat

Arms schools provided information which was used to design a feedback
System for course developers. During in-depth interviews with IPD

personnel information was obtained on the characteristics of a new

computer-based management system soon to be implemented at IPD, SOJET

procedures then were revised to make them compatible with planned-for

computer support capabilities*

Program costs were determined by analyzing administrative actions

to identify the labor and other costs associated with each action.

FINDINGS AND CONCLUSIONS

(1) The SOJET program requires that a course supervisor register

along with a student. This course supervisor helps the student plan

his course of study, administers and scores tests, reviews test findings with the student, and reports test results to IPD. Both actual

and potential supervisors and students reported their acceptance of

the course supervisor concept. Whether the requirement to have a

registered course supervisor resulted in more effective training,

faster student progress through SOJET courses, and/or higher course

completion rates could not be determined. It was concluded that in

the absence of conclusive data, the decision to continue or to discontinue the registered supervisor requirement would have to be based

on policy.

(2) In the SOJET program subcou^se tests are sent directly to the

course supervisor. Course supervisors reported that this practice

seldom caused security or storage problems. However, it does increase

mailing and labor costs by a small amount. It was concluded that the

approach is an acceptable one for delivering test material and administrative instruction?, co field personnel. Furthermore, it emphasizes to the supervisor that he is responsible for the students training.

(3) The SOJET program employs a number of procedures designed to

minimize the time required of supervisors to conduct OJT. For each

SOJET course a Student Training Plan was provided. These plans were

designed to help course supervisors and students identify student work

and training requirements. Comments from survey respondents, plus an

analysis of Student Training Plan data, indicated that the plans often

were not being used as intended—to discriminate between the level of

skill required on the job and present skill level of a potential or

actual job encumbent. Therefore, it was suggested that the use of

STPs be discontinued.

Both students and supervisors reported that the instructional

material was of good quality; course supervisors reported that participations in the SOJET program reduced the time required of them to

conduct OJT.

vi
Procedures for returning test scores to IPD so that training

records could be centrally maintained were found to be time consuming

and cumbersome. Revised, simplified test reporting procedures were

developed and implemented.

(4) Unlike traditional correspondence courses, all SOJET subcourse

lesson materials requested by a student are mailed to him at one time.

The corresponding subcourse test materials are sent to the course

supervisor at the same time. This approach has been termed "one-shot"

mailing; its purpose is to allow students and supervisors to sequence

their program of study in accordance with unit requirements.

According to survey data, this approach was acceptable to

enrolled students and registered supervisors. However, only a third

of the students and supervisors reported taking advantage of the

sequencing opportunities afforded by this approach* An analysis of

training records showed that many students studied subcourse material

in the order in which the subcourses were numbered.

A cost analysis revealed that the "one-shot" mailing is somewhat

more expensive than the approach used to mail out material for traditional correspondence courses. Offsetting this finding were survey

reports that SOJET material often is used as reference aids or to prepare for Skill Qualification Tests by persons not enrolled in the

program. It was suggested that this last finding is a benefit that

may more than offset the higher mailing costs of the SOJET approach.

It was concluded, therefore, that the "one-shot" mailing should continue to be used with SOJET courses.

(5) The testing procedures in the SOJET program are considerably

different from those used with many correspondence courses. Subcourse

tests dra performance-oriented, and arc administered and scored by

the course supervisor. The supervisor uses a test scoring guide that

provides detailed instructions on how to score each test. Also, each

subcourse contains a pretest that if passed, allows the student to

receive credit for the subcourse without studying the lesson material.

Most course supervisors commented favorably about the scoring

guides, although they noted that some guides contain errors in need of

correction* Many supervisors did not report pretests results to IPD

unless the student passed all requirements on the pretest. Also,

supervisors complained about the forms and procedures used to report

test results to IPD. In response to these criticisms, the forms and

procedures ware revised; the requirement to submit pretest results

separate from posttest scores was discontinued. The new procedures

are easier to follow and use the same test reporting form used for

correspondence courses administered by IPD.

vii
(6) Course supervisors are responsible for giving students

immediate feedback about test results and for providing guidance

regarding what material the student should restucjy if he or she failed

portions of a test» Host supervisors and students reported that this

practice was being followed.

(7) The original SOJET program was incomplete in that procedures

for providing feedback to course developers and to program managers

had not been developed at the time the program was implemented. As

part of this effort procedures for additional feedback reports were

prepared. When implemented these procedures will generate reports

dealing with (a) student profiles, (b) subcourse cost effectiveness,

and (c) rate of subcourse utilization and test submission. These are

in addition to an item analysis report now prepared by IPO. Such

reports will provide valuable information to course developers.

(8) Considerable emphasis was placed on Identifying the probable

impact of personnel turbulence upon student progress in SOJET program

courses. Interviews with field personnel suggested that the turnover

rate within battalion S2 and S3 sections Is exceptionally high. A

telephone survey of active Arny units within CONUS indicated that most

SOJET program students and supervisors can expect to be reassigned

after six to nine months on the job. Analysis of training records

revealed that active Arny students had a lower program completion r&te

than did students from National Guard (KG) or U. S. Amy Reserve

(USAR) units. However, in terms of percentage of program completed,

active Ariny students completed a much higher percentage of their

enrolled-for program than did NG/USAR students.

On the basis of these findings, It was concluded that personnel turbulence did have a negative impact on the progress of active Amy stude-,:s and on course completion rates. This impact probably was intensified by the nature of SOJET course material, which 1s duty

position-oriented* When a person is assigned to a new duty position

there is little incentive to continue in the SOJET program. On the

other hand, it was noted that the task-oriented, duty positionoriented nature of SOJET lesson material increases the likelihood that

students will benefit from study of selected subcourscs even though ttu»y

may never complete ^r\ entire SOJET course.

(9) A cost comparison of the SOJET program with th« procedures used

to centrally administer correspondence courses revealed that the SOJET

program costs 327« more to administer. A considerable portion of the

increased cost Is due to the need to pull tests and test scoring guides

for each SOJET subcourse to a course supervisor. It was concluded that

the bulk of these higher costs would be incurred by Any f rogr.ru that

depended on field personnel to administer at\d score tests.

viii
I

(10) It was estimated that the extra cost of using SOJET procedures

(as opposed to traditional procedures) to manage the 16 SOJET program

courses for one year would be $6,288, assuming an annual enrollment of

600 students. In view of this estimate, plus the finding that there

is a perceived need for SOJET instructional material, it was concluded

that ACCP should continue to offer all SOJET courses, and that the

courses should be administered by SOJET procedures (revised) at least

through calendar year 1980. Also, it was concluded that cost considerations should not deter application of the SOJET program to a

select group of courses where the administration and scoring of performance tests by a course supervisor is appropriate. It was noted

also that the task-oriented, self-contained nature of SOJET lesson

material makes that material especially applicable for high personnel

turbulence duty positions—a student can benefit from study of one or

a few subcourses even though he may not have an opportunity to

complete an entire course.

fi

k

IX
EVALUATION AND DESIGN OF A PROTOTYPE SYSTEM FOR MANAGEMENT OF

SUPERVISED ON-THE-JOB EXTENSION TRAINING PROGRAM (SOJET)

E

CONTENTS

Page

m I. OBJECTIVES • 1

II. THE SOJET PROGRAM 2

A. PROGRAM OBJECTIVE AND OPERATIONAL APPROACH 2

B. PROGRAM CONTENT AND ORGANIZATION 4

C. PROGRAM IMPLEMENTATION 7

III. EVALUATION AND REVISION METHODOLOGY 9

A. FIELD ACCEPTANCE 9

B. STUDENT PROGRESS 12

C. ADMINISTRATIVE FEASIBILITY/PROBLEMS 13

D. COST OF SOJET PROGRAM 13

E. REVISION OF SOJET DELIVERY SYSTEM 13

F. FEEDBACK SYSTEM DESIGN 14

G. DESIGN OF ADMINISTRATIVE SYSTEM COMPATIBLE WITH

INSTRUCTIONAL MANAGEMENT INFORMATION SYSTEM (IMIS) 14

IV. DESCRIPTIVE FINDINGS 17

A. PROGRAM PUBLICITY AND ENROLLMENT 17

B. TARGET AUDIENCE VERSUS STUDENT PROFILE 17

C. STUDENT PROGRESS 23

f D. IMPACT OF PERSONNEL TURBULENCE 29

E. COST COMPARISON OF SOOET AND IPD CENTRAL MANAGEMENT

PROCEDURES 35

V. EVALUATION OF FEATURES OF THE SOJET PROGRAM DELIVERY

PROCEDURES 41

*m

A. ACCEPTANCE OF REQUIREMENT FOR A REGISTERED SUPERVISOR. 41

B. ACCEPTANCE OF ENROLLMENT PROCEDURES 45

C. ACCEPTANCE ANO ADHERENCE TO TESTING PROCEDURES 45

D. REDUCTION IN OJT TIME DEMANDS ON SUPERVISOR 53

E. LOCAL SEQUENCING OF TRAINING 63

xi
Page

Jj VI. FEEDBACK TO COURSE DEVERL0PER5 AND PROGRAM MANAGERS 69

A. RELEVANCE OF TRAINING TO LOCAL REQUIREMENTS 69

B. EXTENT TO WHICH TRAIN IP'S REACHES TARGET AUDIENCE 72

C. COST EFFECTIVENESS OF INDIVIDUAL SUBCOURSES 75

D. SUBCOURSE QUALITY CONTROL 81

E. AMOUNT OF PERSONNEL TURBULENCE 83

VII. APPLICATION OF SOJET PROCEDURES TO CORRESPONDENCE COURSES.. 87

A. TYPE OF COURSE TO WHICH THE SOJET PROCEDURES APPLY.... 87

B. COST OF SOJET PROGRAM DELIVERY SYSTEM 88

[ff C. IMPACT OF PERSONNEL TURBULENCE 88

VIII.SUMMARY AND CONCLUSTONS 91

A. COMPLETION OF SOJET SYSTEM LESIGN 91

B. EVALUATION OF SOJET DELIVERY SYSTEM 91

C STUDENT PROGRESS AND PERSONNEL TURBULENCE 93

D. COST OF SOJET PROGRAM 94

E. CONTINUANCE OF SOJET PROGRAM 94

APPENDIX A 97

xii
TABLES AND FIGURES

Page

TABLES

1 SUBCOURSES AND STUDENT CREDIT HOURS FOR SOJET

OPERATIONS/INTELLIGENCE COURSES 6

2 SUMMARY OF PROGRAM EVALUATION, REVISION AND

DESIGN ACTIVITIES 10

3 SOURCES OF INTERVIEW AND SURVEY DATA 11

4 DISTRIBUTION OF ENROLLMENTS IN TERMS OF CURRENT

DUTY POSITION 20

5 DISTRIBUTION OF ENROLLMENTS IN TERMS OF CURRENT

SKILL LEVEL 21

6 DISTRIBUTION OF ENROLLMENTS IN TERMS OF CURRENT MOS 22

7 DISTRIBUTION OF ENROLLMENTS IN TERMS OF MONTHS IN

CURRENT DUTY POSITION, AND MONTHS IN ALL OPERATIONS

OR INTELLIGENCE DUTY ASSIGNMENTS 24

8 TRAINING STATUS OF ALL STUDENTS AS C7 1 SEPTEMBER 1979.... 25

9 PERCENT OF STUDENTS ACTIVE DURING VARIOUS TIME

PERIODS FOLLOWING ENROLLMENT , 26

10 CUMULATIVE AVERAGE NUMBER OF NEW SUBCOMRSE TESTS SUBMITTED

FOLLOWING VARIOUS TIME PERIODS AFTER ENROLLMENT 28

11 PERCENT OF PROGRAM COMPLETED AFTER VARIOUS TIME INTERVALS

FOLLOWING ENROLLMENT 30

12 STATUS OF ACTIVE ARMY STUDENTS AND COURSE SUPERVISORS

(CONUS) AS OF AUGUST 1979 32

13 STATUS OF STUDENT-SUPERVISOR PAIRS AS OF AUGUST 1979 32

14 PERSONNEL TURBULENCE OF STUDENT-SUPERVISOR PAIRS AS

RELATED TO MONTHS SINCE ENROLLMENT 34

15 SUPERVISOR REPLIES TO QUESTIONS ABOUT PERSONNEL TURNOVER.. 34

16 TIME REQUIRED TO PERFORM ENROLLMENT TASK IN ACCORDANCE

WITH IPD PROCEDURES, ORIGINAL SOJET PROCEDURES AND

REVISED SOJET PROCEDURES (MINUTES) 36

xni

m -'-»- -- -»
TABLES Page

17 COMPARISON OF LABOR REQUIREMENTS AND PRINTING AND MAILING

COSTS REQUIRED TO ADMINISTER ENTIRE ACCP ACCORDING TO

IPO, ORIGINAL SOJET AND REVISED SOJET PROCEDURES 37

18 ESTIMATED ANNUAL MANHOURS REQUIRED FOR SELECTED ACCP

ACTIVITIES BASED ON IPD, ORIGINAL SOJET, AND REVISED

SOJET PROCEDURES (HOURS) 39

19 SUMMARY OF INTERVIEW AND SURVEY DATA RELATED TO

ACCEPTANCE OF THE ROLE OF COURSE SUPERVISOR 42

20 STUDENT REPLIES TO QUESTIONS RELATED TO TESTING

PROCEDURES 43

21 SUPERVISOR REPLIES TO QUESTIONS RELATED TO TESTING

PROCEDURES 44

22 AVERAGE TIME TO CONDUCT COURSE SUPERVISOR ACTIVITIES

(MINUTES) 46

23 SUMMARY OF CRITICAL COMMENTS MADE BY ENROLLED STUDENTS

AND REGISTERED SUPERVISORS 47

24 SUMMARY OF SUPERVISOR RESPONSES TO QUESTIONS RELATING

Tu SUBCOURSE TEST STORAGE AND SECURITY 49

25 NUMBER AND PERCENT OF STUDENTS FOR WHOM VARIOUS

COMBINATIONS OF PASSING AND FAILING PRETESTS AND

POSTTESTS WERE SUBMITTED 49

26 PERCENT OF STUDENTS FOR WHOM VARIOUS COMBINATIONS OF

PASSING AND FAILING PRETESTS AND POSTTESTS WERE

SUBMITTED 52

27 SUMMARY OF STUDENT REPLIES TO QUESTIONS ABOUT

SOJET LESSON MATERIAL 55

28 SUMMARY OF SUPERVISOR REPLIES TO QUESTIONS RELATED

TO SUBCOURSE EFFECTIVENESS AND JOB RELEVANCE 56

29 AVERAGE RANKS ASSIGNED TO SKILL REQUIRED AND PRESENT

SKILL LEVELS (COLUMNS 2 AND 3) OF STUDENT TRAINING PLAN... 59

30 RANGE OF RANKS USED TO COMPLETE SKILL REQUIRED AND

PRESENT SKILL LEVELS (COLUMNS 2 AND 3) OF STUDENT

TRAINING PLAN (N * 367) 61

xiv
TABLES Page

31 STUDENT AND SUPERVISOR PREFERENCE FOR MAILING APPROACH.... 61

32 MAJOR FACTORS USED TO DETERMINE ORDER IN WHICH

SUBCOURSES WERE STUDIED 64

33 ORDER IN WHICH SOJET SUBCOURSE TESTS WERE SUBMITTED

COMPARED WITH ORDER W WHICH SUBCOURSES WERE LISTED

ON COURSE CATALOG 66

34 UTILIZATION OF SOJET MATERIAL AT THE UNIT LEVEL 67

35 NUMBER OF SUBCOURSES ISSUED THROUGH APRIL 1979 AND

PERCENT OF TESTS SUBMITTED THROUGH JUNE 1979 73

36 RANK OF SOJET PROGRAM COURSE SUPERVISORS 76

37 DISTRIBUTION OF ACTIVE ARMY SUPERVISORS BY DUTY

POSITION AND COURSE REGISTERED IN 77

FIGURES

FIGURES Page

1 SOJET PROGRAM MANAGEMENT SYSTEM OBJECTIVES AND

OPERATIONAL APPROACH 3

2 ORGANIZATION OF SOJET COURSE MATERIAL 5

3 CUMULATIVE NUMBER OF ACCEPTED ENROLLMENT 18

4 ILLUSTRATIVE PAGE FROM STUDENT TRAINING PLAN FOR

OPERATIONS SERGEANT/ASSISTANT OPERATIONS SERGEANT

COURSE 58

5 ILLUSTRATION OF STUDENT SUBCOURSE CRITIQUE FORM 70

6 ILLUSTRATION OF SUPERVISOR SUBCOURSE CRITIQUE FORM 71

7 ILLUSTRATION OF SUBCOURSE ISSUE AND TEST SUBMISSION

REPORT 74

8 SOJET PROGRAM STUDENT ENROLLMENT APPLICATION FORM 78

8a SOJET PROGRAM STUDENT ENROLLMENT APPLICATION FORM

(2nd SIDE) 79

xv
FIGURES Page

9 PROPOSED PROFILE REPORT OF PERSONS FAILING A SUBCOURSE

PRETEST REQUIREMENT 80

10 ILLUSTRATION OF ITEM (REQUIREMENTS) ANALYSIS REPORT FOR A

4-REQUIREMENT SOJET SUBCOURSE TEST 82

11 SUGGESTED FORM FOR OBTAINING INFORMATION ABOUT STUDENT

AND SUPERVISOR STATUS AND MAILING ADDRESS CHANGES 84

12 ILLUSTRATIVE TRAINING ENROLLMENT AND STATUS REPORT 85

xvi
EVALUATION AND DESIGN OF A PROTOTYPE SYSTEM FOR MANAGEMENT OF

SUPERVISED ON-THE-JOB EXTENSION TRAINING PROGRAM (SOÜET)

I. OBJECTIVES

During the spring of 1978 the U.S. Army began an extensive field

test of a new way of delivering duty-position training to a unit in a

non-resident mode. This program was known as the Supervised On-theJob Extension Training (SOJET) program. The delivery system for the

program contained features that differed considerably from those used

with traditional correspondence courses.1There was a need, therefore to evaluate the cost effectiveness and the acceptability of those

features and to revise them as appropriate.

When the SOJET program was initially implemented, certain portions had to be administered on a "management by exception" basis,

using specially developed procedures. There was a need to redesign

these procedures to make them compatible with the new information

management system to be adopted by the Institute for Professional

Development (IPD), the agency responsible for administering the Army

Correspondence Course Program. In addition, a system for.providing

feedback to course developers and managers had not been designed.

The present study was undertaken to evaluate the delivery system

for the SOJET program and to complete the design of that system. The

technical objectives of the study were to: (a) conduct a formative evaluation of the prototype extension training program, and (b) complete the

design of an extension training management system for the centralized

administration and delivery of performance-oriented extension training.

^Throughout this report "traditional correspondence course" refers to

courses which are MOS rather than duty-position oriented, and do not

require the support of a registered course supervisor. Also, most

"traditional" correspondence courses employ paper and pencil knowledge tests which are centrally scored, require students to study

lesson material in a set sequence, and send to the student only a few

subcourse lesson packets at any one time.
II. THE SOJET PROGRAM

A. PROGRAM OBJECTIVES AND OPERATIONAL APPROACH

The SOJET training delivery system is based on five system objectives, which in turn are based on Army training policy and/or training

technology practices that are considered to promote effective training.

These five system objectives are listed in Figure 1, along with the

one or more operational approaches taken to implement each objective.

A basic tenet of the SOJET program is that the supervisor must be

recognized as having responsibility for the training of those under

his/her supervision. All Army supervisors have this responsibility

but in many instances it is not actively exercised. The SOJET program

required supervisors to take an active role in the training of their

personnel. For example, in the traditional correspondence course

program, the student is responsible for enrolling in the course and

pursuing his study. In the SOJET program, the course supervisor is

responsible for helping the student identify his training requirements

and for administering and scoring tests. To emphasize the nature of

this responsibility the supervisor is required to register by name

along with the student. Also, subcourse tests are sent directly to

the course supervisor.

In the absence of duty-oriented lesson material supervisory personnel are required to provide on-the-job training as best they can.

Often the supervisor must devise a course of study, actually serve as

a tutor, and maintain student training records. Those supervisors who

conscientiously fulfill their training responsibilities often spend

considerable time in conducting on-the-job training. The SOJET

program was designed in recognition of the need to reduce these time

demands. To accomplish this the SOJET program provides (a) a tool, a

student training plan, which can be used to identify job requirements

and student training requirements; (b) high quality, self-study lesson

material keyed to tasks actually performed on the job; (c) procedures

for maintaining training records centrally.

Within units some mechanism is needed whereby persons can study

or refamiliarize themselves with a task before they must perform it.

In the SOJET program, this unit need is met by providing a "one-shot"

mailing of all subcourses requested by the student. The supervisor

and the student are then free to study the course material in the

sequence which best meets their unit needs. To accomplish this the

subcourses had to be constructed so that they were self-contained

training packages requiring no prerequisite training.
I

at0

V» •»>

at

UJ

A. 3

V»

X

o

* 1 u z o < z

V»1

0

<

> 1 * 1

Qt at Ul

1

a.1

0.

3

OL

VI v»

1

<

1» I z

o 1 Ul H>

1

—i

o >-

1

<

3 -J

H>1

Z

VI

Ul1

0

VI <

ÜC1

m

o1

h» Ul < <

t1 z 1

fit

Z

Ul1

Ul v»

1

0- >• *»

1

0

« *-

1

*"1

z

UlI

*"1

o Ul

I *- VI 1 < at

at

3

vt

ou

1

5

m1

Ul

^

M VI

1

••

*** ! X I M

V* V»

LU

<

-

> at

z

o

Ul H- v% s u

Ul

>

Ul

*

Ul

a.

o

A

3

<

o ! M

z<

s 1 **i X

s

Ul

«- 1 Ul N

l/l

z at

1

>-

o

uif

l/l

Z I o 1 u < 1 Ul Uf 1 I * »» 1

v>

Ul 3O

ui 3ou«3

vt OZ3<0xzo

zo

vtoz<

u.O

is

z *» £ z

§2z z < -

ato

oz

Vt_ VI

>=°

at2 ui

3 tii^

vtU<

>ZW

> «vO

O3 -*

-2o

< vt »-

<*°

* i 1 ** o > 1 Ul

_ii

Ul

1 ft* 3

VI

Ul 1 M ••

u.1

1 >-

*1

1*z[

f Ul

Ul VI Q ui

°

s> <->

1 *"

X°

i u>

*

9 i- ui

_ !!

11 Ul **

QS

i *-

"z

3

<

1 Ul

o^

1

uz<

1*

Ul1

z* 1

5So ;;

1 u. °

Wu

;o

J;

1 ** 1 ui

°*Ü

1 **

Ul Ul «J Ü ft* -

1 ui 1 M

Oo°

I*

« at «_

3

2S1

zz2

0

1

U

1 **3

<< ** ! »- t- ui h

1" vt vi at • •

Ul Uz<*

Ü«

*-o

** li. O at *o 2 < !* — • ö Z ! < 3 T " O vt

Ul *

z

<° — M

Ul \J 2 <

si

atm 1 a. u.

VI Z

Ul 3o<u0Oziuz<>

uz

ui 53<£ vi O vi

at at < 3

•-o

isX "

<-» -J « i S >

o1

OO at*

z „.o

H

oVw «Jui zo

Ul uz

<* vi >" •" S £

O ui b2 = 5 2 > x > 5 S 2 O^ U ui O

UI v» u.

u. at0O

:s2

R

s

S5

5

IM

8W

ou

«s a.<

3^2 V» 5 H-

•••

<m

uiO

SI

UJ2*

*-m<

£>£

vt •" O uj

<

X vt at > « O

ojo

atO at

a.u a.

U£

OL

Q. <So2

S. o

•oc

10>

Q> 2oI

V) c

C7i cE

Ui

0) u3

»•
Training technology research has demonstrated conclusively that

learning is promoted by providing immediate feedback to the student.

In the traditional correspondence course a student takes an examination, the examination is centrally scored, and some four to six weeks

later the results are forwarded to the student. In the SOJET program

the supervisor administers performance tests and then immediately scores

and reviews the test results with the student. The supervisor explains

the nature of any test errors and directs the student to restudy those

portions of the lesson material dealing with the requirements he did

not pass on the test. To accomplish this system objective, the subcourse performance test is designed so that it can be scored by supervisors. In addition, a standardized scoring guide was developed for

each subcourse. This guide identifies correct re-sponses, prescribes

minimum performance standards, and provides directions for restudy.

The administration of a correspondence course program is a complicated process requiring a considerable amount of feedback. The

Institute for Professional Development (IPD) prepares a number of

management reports of interest to program managers. However, with the

exception of an item analysis report, IPD does not produce reports

that are of interest to course developers. As shown in Figure 1, some

of the reports that would be useful to both course developers and

program managers would deal with such topics as (a) the relevance of

training to local job requirements, (b) the extent to which training

is reaching the target audience, (c) the cost effectiveness of individual subcourses, (d) subcourse quality control, and (e) the impact of

personnel turbulence on training delivery.

B. PROGRAM CONTENT AND ORGANIZATION

The sixteen SOJET courses developed to date are directed to

operations (S2) and intelligence (S3) enlisted personnel duty positions, within combat arms battalions. These positions were selected

as the vehicle for the program because many persons assigned to battalion S2 or S3 positions have not had the opportunity to take formal

instructions in operations or intelligence-related tasks.

The typical organization of a SOJET course is shown in Figure 2.

Some subcourses cover tasks that are common in all types of battalions.

Also, there are a number of special subcourses unique to one of the

four combat arms branches. As a further illustration, the operations

sergeant/assistant operations sergeant lesson material is packaged

into four courses (Y01-Y04), a separate course for each of the four

combat arms branches(Table 1). Fourteen core subcourses are common to

each of the four combat arms branches. In addition, there are two or

more special subcourses for each branch. Thus, a person enrolling for

the armor operations sergeant course (Y01) enrolls for a total of 19

subcourses.
;, . SUPERVISOR

5UBCOÜRSE

TEST

PACKETS

BRANCH

TASKS

—i

Ml

Z

Ml a

m0

«A

Ml <

zz

3

u

<

<

^

0

m i-

«A

3

V»

a.

m

:

.. .__J

at MI > 0

2*

2

S

£»<2-

5*

Ml

*--*X

5S552

""•22 •"

<Q<

o<

AC Ul O *A «A

z

«*«

-

0 w>

>*

D0u

>-

«A

UJ

UJ 3E

<

&

m

<

O »-

3

3

&

u

IA V*

i

P

0)

«4-» £

0)

v»

Ul oco

•r4-»NCm

CNJ

0»3

C7i

L.
TABLE 1

SUBCOURSES AND STUDENT CREDIT HOURS FOR

SOJET OPERATIONS/INTELLIGENCE COURSES

COURSE AIR DEFENSE

ARTILERY

ARMOR FIELD

ARTILLERY

INFANTRY

OPERATIONS SERGEANT/

ASSISTANT OPERATIONS

SERGEANT

Course Number (Student

Lesson Packets) Y04 Y01 Y03 Y02

Number of Core Subcourses (for all

Combat Arms) 14 14 14 14

Number of Special

Subcourses (specific

to a Branch) 5 5 4 2

Total Number of

Subcourses 19 19 18 16

Total Student Credit

Hours 53 48 49 51

INTELLIGENCE SEßSEANT/

ASSISTANT INTELLIGENCE

SERGEANT

Course Number (Student

Lesson Packets) Y08 Y05 Y07 Y06

Number of Core Subcourses (for all

Combat Arms) 11 11 11 11

Number of Specific

Subcourses (specific

to a Branch) 5 4 5 3

Tote! Number of Subcourses 16 IS 16 14

Total Student Credit

Hours 39 32 35 34

OPERATIONS ASSISTANT/

SPECIALIST

Course Number (Student

Lesson Packets) Y12 Y09 Yll Y10

Number of Core Subcourses (for all

Combat Arms) 4 4 4 4

Number of Special Subcourses (specific

to a Branch) 6 0 0 1

Total Number of Subcourses 10 4 4 5

Total Student Credit

Hours 44 19 19 25

INTELLIGENCE ASSISTANT/

SPECIALIST i

Course Number (Student

Lesson Packets) Y16 Y13 Y1S YI4

Number of Core Subcourses (for all

Combat Arms) T 7 7 7

Number of Special

Subcourses (specific

to a Branch) 6 0 0 1

Total Number of Subcourses 13 7 7 8

Total Student Credit

Hours 43 23 23 25
As shown in Figure 2, subcourse material is organized into two

packets. The student packet contains lesson material and practical

exercises. The supervisor's test packet contains pretest material,

posttest material, scoring instructions, and instructions and forms

for recording test results and forwarding them to the Institute of

Professional Development.

C. PROGRAM IMPLEMENT! 1 UN

The SOJET program was implemented on an experimental basis during

the spring of 1978. To hasten the process of implementation and to

assure that suitable numbers of students would be available during the

program evaluation pha^.e of the implementation, an extensive publicity

campaign was conducted beginning early in 1978. The availability of

the SOJET program was announced in Combat Arms School publications and

a "GREEN publicity package" was sent to all battalions in the active

Army, Army Reserves, and National Guard. Beginning in May 1978,

program enrollment applications were accepted.

Persons responding to this publicity were sent a "Supervisor's

Guide" which contained information about the program, and instructions

and forms for enrolling. A revised version of this document, now

called a "Course Guide," is contained in Annex A.
i

III. EVALUATION AND REVISION METHODOLOGY

The evaluation, revision and design activities engaged in

during this study are shown in Table 2. These activities consisted

of:

o Determination of field acceptance of the SOJET program

o Evaluation of the administrative feasibility of the SOJET

program delivery system

o Evaluation of student progress

o Determination of SOJET program costs

o Development of revisions to program delivery procedures

o Development of improved feedback procedures

Each of these activities is described in more detail on the

following pages.

A. FIELD ACCEPTANCE

Information on field acceptance of the program was obtained from onsite interviews at Army posts and from three surveys conducted by mail.

Field Visits. From November 1978 through March 1979, on-site

interviews were held at five installations—Forts Campbell, Hood,

Riley (two visits), Sill, and Stewart. During these visits three

groups of persons were interviewed: enrolled students and registered

supervisors, potential students and supervisors who had requested

information about the program, and potential students and supervisors

who were unfamiliar with the program.

The purpose of these interviews v/*s ts determine receptivity to

the program by potential students and supervisors, the impact of

course publicity on enrollment, and the acceptance of various features

of the program's delivery system. The structured interview used

during these visits is contained in Annex A. The categories and numbers of persons interviewed are shown in Table 3, and the interview

data are summarized in Annex A.

Survey of Enrolled Students and Registered Supervisors in

National Guard and Reserve Units (1st Survey). As a supplement to
I

CO

COo• »*

UO»_•

<SJ

>

Ul

UJ oc

-JS

•

»—

o»•^

H- <

13

5?>

ut

s•# » OS 4-> > r>» o\ u

•r- 01

4-* J= 4-» ••-) £ u en v» o U O» f* rH U Ol 9» OS

«o r- cxoi f- os os r-» r- Oi

*te ac o» 4J r>» o* r** r*» o i- 3 3«fc 4J 4-> 4-> i 3 < o op a 4-» 4-» Q. Q. 4J» ^ »» ~3 JE a a» CL a. 4) 4) Q.

V» oo os OS

I Wirtfl» tfl 0)4)

WW 4) **•* r-% r* i r*. 3 x: co i to co 1 1 CO o >» O 4-» 1 >» II >» >» 1

4-» >CjQ Olf- L.

CC r- >»>> P- *-/

c

«0 oO <0 4133 JEQ <03 ri JO

33 40

oooooooo oo

OOO

"»^ V»

~2£I £?£

sc

«CO« «A ooc L.

O

C 4> Ml 1- tAU *- 4»

3 •* Ml

rmy Person

oiled stud

ed supervi

1 students

upei visors

nd USAR/NG

/superviso

test data

onnel

Potential

& supervi

onnel & re

I personne

ey respond

O -r- ° 2£

•0C 4)

41 5.

Ml •- XJ

3 r4» 41 3 Ml 4) u C 4-» C u C Ml Ml XJ C ! 3 O 1. 4» O o < L. k. <0 Ml «0 Ml w 4 0 M OC > C 4> *— +J 4-> L. W L <i>

Ml 4) XJ U. Ml

CO

C 5.41 4)L

Q> V4JP «<CC 4» 1—C 4»3

«0

> iAc ut< 4* 4* a. «QUO. •* MI &^

= t!& +J Acti USAR regi Pote cour All stud Stud IPO Actu stud IPO IPO All

4) O-*-

*0

O> L» OlQ

o

oooooooo oo

ooo

.-* Ok

• «Q

coc MI Ui

3 •#- #-

2CC Ml Ml

O •- «J Ml >| O * Ml Ml 4J» #— oS£

«—4-» X» X» t- X»O Ml*

«ACL.)— Ml L.UOC

A.—

c •«I.- Ol o Mi~-4CNjn >» o u m O C

•#- 4-> (*- 4) **-

U O» .. L. ^*

4-»

Vis1

ys: o

0

0

sis o

cords

views

surv

sis ork re

intinilinglews

data

«*-C X»

U (A

4» MI ••um

4» T3 — o — JC o *-»

> Ml 4-»U>

FieldSurve

Analy

Re

Inter

All 3

Analy

a. wo

b. pr

c. ma

nterv

urvey

°*

L. >» L.1 fc.

4) P—O 4^ 4)

m 4->

« flLMW

4-»

CC 4* 4»C

<o

~ co~< k.w^

Q

oooooo

ooOOO

**

41t h- t-

**- 41 TS X» UJ Ul

°S2* *.33

C <© CO *-C~ uO *» cO

•

MlC XL

«*-1 4» •*- «-»

nt o

Feededur

Adm

e

O *>O 4) <-.. *»

wi

•*» O. *-> >» L. •- Ml U-

>% *J» 4»

oc —• *-» o»oO OMI

*0UO 4-» •—OO 4l

CU Ml Ml—*—U MlC >»C

4>>

«••

•*»> -~< 6

-*-lA«^t»-~a.-~S

IA|I«W£ M4) ••-O t.41 *U3 Is

2^

&.XI XL3 V»*-K XL «A>4l

O> CL C4J

4-*

evel

mpro

ack

csigompa

HIS*

U

Dete

Fiel

Anal

Prog

Eval

mini

FcasSOJE

AnalProg

Revi

Deli

Proc

<

Q— -QQo -*

—•_» M«

M

~— .-— >. >>

»

«A

>*

COCoo

<*- c

«occo2

WO

10
IB

P»

ro

-JS

oo

LÜO

cca

.

«•— >> ^s< 0> 01 «-*> O iSt c > en z => c u*> r*

1

1

1

i- r^O vo in

3 V)

t/> o» w. 3 01

*0< 0.

u ^-*

CO

pM

01 >»Q) > E C

•pCc

o o>

111

*J<o CM CM

O «A < 1-

01

a.

^_^

o>cE

>> rs.oo <o

01 trtC -r- i.

>.c 'f +> o>

L. ro *J *oo

33 «/> «/»Eu

00^ 1- 4»C ft.

1

i vo

1I

1 013O *r

o• a. crv* *J

C -Q 0)C3

CSJ 0) QC —•O

fc.5

>»—

O «-^O 01

> a* ac

t- r^ ^<C -Q

3 <J3 o0O

00. ZD yi

S

CM

ro

1

1

1

C L.

*J «o 0)

«A r> *v

4-»

•*»^ >^

t/*5*•-»»*• »»g ^_

•«- oi cr» cv Co*

>. •— i»^ r*.<C

>c

"O t-•• 01o

m

— t*> t-> t/l ro VO l/>

s

CO [ o *-» o *g -«- u — C Z X *» o

«-4 ro *T j u. — w u a. < co

Cl-«- «/»

VIC *J *J *-» — <TJ j c C *J 6 C

V» t» 0»

Oi

01 01 TJ £

-o 01

o

3

3 **

3WW *-» <A

3

W

U cr

C 0» TO tfl

U

O lA t?

o

O— >*u 0)

o

l/l 0« «Auou

_ •— «/»

-o I- -*-

E**o *0 •0 ••-

«o 01 0»> *rt

*O T3w

C tCo wi

•*—

—>

*J MB

Ju *J •J tm *HB */» a» o a» ~ -— c C 0» O o — CL iflO*J'- 3> Ol o» a. u cn 3 U L> 3 O O W +J 3 c Cl »rt oi a,-Q i. t. o O t*

UJ ex CL

c Cw

n
the field visits, questionnaires were developed to obtain data from

students and course supervisors who were members of the National Guard

or U.S. Army Reserve. These questionnaires (Annex A) were sent to all

NG/USAR personnel who were in the SOJET program as of 1 January 1979—

137 enrolled students and 67 registered supervisors. The rates of

return from enrolled students and registered supervisors were 28% and

47%, respectively. The data obtained are presented in Annex A.

Survey of Persons Who-Had Requested Information on the Program

but Had Not Enrolled/Registered (2nd Survey). Before and during field

tests on the SOJET OPS/INTEL program, the program was publicized in

various ways. As a consequence many persons requested information

about the program, and were sent a Supervisor's Guide packet. If

these persons did not enroll as students or register as course supervisors within two months, they were sent a questionnaire (Annex A).

The purpose of the questionnaire was to (a) obtain reader reaction to

the program as described in the Program Supervisor's Guide, and (b)

obtain information about enrollment intentions. This questionnaire

was sent to 95 persons and the rate of return was 49%. The data

obtained are presented in Annex A.

Survey of Enrolled Students and Registered Supervisors (3rd Survey).

During August 1979, questionnaires were mailed to all active Army

enrolled students and registered supervisors and a large sample (about

two thirds) of the enrolled students and registered supervisors who

were members of the National Guard or Army Reserve. The general purpose of these questionnaires (Annex A) was to collect a variety of

data bearing on acceptance of the program's delivery system, course

content, and test scoring procedures. Questionnaires were sent to 153

active krmy students, 103 active Army supervisors, 300 NG/USAR students, and 200 NG/USAR supervisors. The rates of return for these

four ground were 13, 28, 22, and 29 percent, respectively. The data

obtained are discussed later in this report.

B. STUDENT PROGRESS

The extent to which students actively participate in a training

program can be used as a measure of field acceptance of the program.

Special procedures had been developed to manage the central administration of the SOJET program during its initial implementation.

This was necessary because much of the data collected as part of the

SOJET program could not be processed using IPO data files and computer

programs. These procedures included the collection and storage of

training data; training records were maintained on special OPS/INTEL

program files. These records were analyzed to develop data on such

matters as rate of student progress, student drop-out rates, course

completion rates, and the ratio of enrollment to actual starts in the

12
program. The findings from this analysis are presented in Section IV

of this report.

C. ADMINISTRATIVE FEASIBILITY/PROBLEMS

One of the purposes of this study was to evaluate the SOJET

program delivery system, to identify problems with it, and when

feasible, to immediately implement solutions to these problems.

Problems experienced or anticipated by unit personnel, especially

registered or potential course supervisors, were identified during the

field visits and during the first two surveys.

A second set of problems investigated were those experienced by

persons responsible for the central management of the SOJET program.

Appropriate staff members of the Institute for Professional Development were interviewed, and through extensive discussions, a number of

administrative problems were identified.

Solutions to some of these problems were implemented in the

summer of 1979. Solutions to other problems were incorporated in procedures in the SOJET Program Implementation Handbook (Appendix B).

D. COST OF SOJET PROGRAM

To determine the cost of the SOJET program, costs for administering the entire SOJET program had to be estimated. These estimates

were constructed by first obtaining cost data for the administrative

elements comprising the delivery system for the typical correspondence

program. Most of these estimates had already been developed by IPD

and were described on recently prepared "Schedule Xs." These schedules show the work elements of a major operation and provide detailed

cost estimates for each element and operation. The schedules are used

for budget planning.

Each administrative work element for a traditional correspondence

course was then reviewed and cost, personnel, or time requirements

were adjusted to reflect SOJET program procedures as described in the

SOJET Program Implementation Handbook (Annex B). IPD personnel

familiar with current SOJET administrative procedures cooperated in

this casting exercise.

E. REVISION OF SOJET DELIVERY SYSTEM

Throughout the first nine months of this project, there was a

continuing effort to identify administrative problems that could be

13
revised immediately. Discussions with IPD personnel and an analysis

of interview and questionnaire data resulted in identification of a

number of problem areas. These areas were reviewed with IPD and ARI

personnel to specify the problems for which solutions could be developed and implemented without delay. Most of these revisions involved

simplifying procedures or program information and enrollment material.

These revisions were made by the contractor, reviewed by IPD and ARI,

and then implemented during August 1979.

F. FEEDBACK SYSTEM DESIGN

The present management system for Army correspondence courses

produces a number of monthly reports of use to program managers. However, with one exception, a subcourse test item analysis report, IPD

prepares no management reports for the specific use of course developers. One of the primary purposes of this project was to design an

improved feedback system for program managers and especially for SOJET

course developers. This was accomplished as follows. First, a determination was made of the decisions that must be made by program managers and by course developers. These decisions were identified

through discussions with IPD personnel and telephonic discussions with

course develoers at the combat arms schools. Contractor personnel

then reviewed each decision area and identified the major informational inputs required to make a decision. Thirdly, the Contractor

examined IPD management reports and data files, and identified the

information IPD currently can provide in support of the major decisions which must be made by program managers and by course developers.

During this activity enrollment forms and training records were

analyzed to determine the data captured by such forms and records. A

comparison of information requirements with current information

collecting/generating capabilities resulted in the identification of a

number of information gaps—information that should be collected or

generated in order to provide course developers with a sound basis for

decision making. As a final step, proposed changes in enrollment

forms were developed, and a number of new types of feedback reports to

course developers were proposed. These are discussed in Section VI of

this report and in ANNEX B to the report.

G. DESIGN OF ADMINISTRATIVE SYSTEM COMPATIBLE WITH INSTRUCTIONAL

MANAGEMENT INFORMATION SYSTEM (IMIS)

Correspondence courses currently are managed by the U.S. Army

Training and Doctrine Command (TRADOC) educational system for nonresident instruction (TREDS-NRI). This system is somewhat obsolete

and does not have the capability to handle anticipated correspondence

course requirements in the 1980 time frame. Consequently, the system

14

-' -" -*
is being revised. The new system, currently known as the Instructional Management Information System (IMIS), will be available within

two years. This new system will be capable of storing considerably

more data and will have an expanded data processing capability IPD is

scheduled to acquire its own computer facility in support of this new

system.

Currently, certain portions of the SOJET program are being

managed "by exception." That is, special computer-based files are

used to store and process certain types of SOJET program data which

cannot now be processed by the TREDS-NRI system. One purpose of this

project was to eliminate the need for those special programs and files;

the goal was to design an administrative system that can be handled by

the new IMIS. Since IMIS will not be operational for about two years,

its capabilities and the programs that will be used with it had to be

estimated for the purposes of this project. The final design of the

SOJET administrative system was adjusted so that it was compatible

with the estimated IMIS capabilities. To accomplish this, the functional requirements of the SOJET administrative system were identified, then reviewed with persons responsible for development of

IMIS. The final version of the SOJET administrative system thus was

designed so that it can be handled by IPD's future educational management information system--IMIS.

15
IV. DESCRIPTIVE FINDINGS

A. PROGRAM PUBLICITY AND ENROLLMENT

Implementation of the SOJET program began early in 1978. To

announce the availability of the program, a short description was

mailed to all Army, National Guard, and U.S. Amy Reserve battalions.

Also, articles announcing that the program was available were published

in school newspapers. Application for enrollment into the program

began to be accepted during May 1978.

As of the end of October 1979, 728 students had been enrolled, 498

of whom were from National Guard or Reserve units. As shown in Figure

3, the rate of enrollment for active Army personnel was very slow

until Janury 1979.

On-site interviews conducted at state-side Army installations during

the fall and early winter of 1978 indicated that the publicity had not

been effective. Over half of those persons eligible for enrollment

were not aware of the program (Table B-8, Annex A). As the result of

this finding and the low enrollment rate of active Army personnel, an

intensive publicity campaign was begun in December 1978. Messages

were sent to all battalion commanders announcing the availability of

the program; a description of the program was mailed to the commanding

officer of all battalion intelligence (S2) and operations (S3)

sections; additional articles about the program were prepared for

School papers; and overseas radio spots were prepared.

As the result of this campaign, enrollment of active Army personnel began to increase and currently is rising at the rate of about

25 students per month. However, even after this additional publicity,

over 60% of the persons interviewed during February-March 1979 still

were not aware of the SOJET OPS/INTEL program (Table B-8).

The somewhat unusual publicity campaign was undertaken to

encourage enrollment in the program as rapidly as possible, so that

numbers of students would be adequate for a beginning on evaluation of

the program. In the intervening months, availability of the SOJET

operations and intelligence courses has been announced in appropriate

School catalogs and the courses have been referenced in appropriate

Soldier's Manuals. These catalogs and Soldier's Manual listings should

be sufficient to maintain the rate of enrollment at its current level.

B. TARGET AUDIENCE VERSUS STUDENT PROFILE

The SOJET program is u..-sual in that there are no restrictions on

who can enroll in the program. However, the operations and Intel 11-

17
»Ä

k\-.

i

si

o

u

c

1

a

u

o

u

0)

>

CT»

sjusprns P»U0JU3

18
gence courses were designed primarily for enlisted personnel assigned

to combat battalions. The duty-oriented nature of the course material

makes the courses most appropriate for persons now in operations or

intelligence duty positions or aspiring to such positions.

The characteristics of persons enrolled in the SOJET program are

shown in Figure 3 and in Tables 4 through 7.1Most SOJET program

enrollees (78%) were members of the National Guard or of Reserve

Units. However, from March- through October 1979 almost equal numbers

of active Army and National Guard/Reserve students enrolled in the

Program.

Eighty percent of the enrollees held a current operations or intelligence duty assignment (Table 4). Moreover, persons currently

assigned to operations (S3) duty positions tended to enroll in one of

the eight operations courses while persons currently assigned to intelligence (S2) duty positions tended to enroll in one of the eight

intelligence courses (Table 4). Also, persons currently assigned to a

sergeant or assistant sergeant duty position usually enrolled for one

of the eight senior courses while persons currently assigned to

assistant or specialist positions enrolled in one of the eight junior

courses (Table 4). Most of the persons currently assigned to nonoperations or intelligence positions enrolled in one of the operations

sergeant courses.

Seventh-three percent of all enrollees were NCOs at the 3, 4 or 5

skill level (Table 5). This seems to reflect the fact that for a

number of MOSs persons have to obtain a fairly senior rank before

there is a requirement for them to learn S2 or S3-related tasks.

Eighty-seven percent of the students held a combat MOS (Table 6).

The largest group of enrollees (30%) held an infantry MOS while an

additional 22% held a field artillery MOS. Most students who held a

non-combat MOS held an MOS in the intelligence area.

^Detailed versions of these tables are located in Appendix A.

19
TABLE 4

DISTRIBUTION OF ENROLLMENTS IN TERMS OF CURRENT DUTY POSITION*

CURRENT DUTY POSITION

Course Component

OPS SGT/

ASST OPS

SGT

INTEL SGT/

ASST INTEL

SGT

OPS

ASST/

SPEC

INTEL

ASST/

SPEC

NON OPS/

INTEL

POSITIONS

Total

OPS SGT

(Y01-Y04)

AA

NG/USAR

54

148

2

11 4

- 36

46

92

209

INTEL SGT

(Y05-Y08)

AA

NG/USAR

1

4

24

82 6

1

1

9

0

35

99

OPS SPEC

(Y09-Y12)

AA

NG/USAR

1

2

1 23

35

3

8

3

9

31

54

INTEL SPEC

(Y13-Y16)

AA

NG/USAR - 4

1

3

7

19

5

3

13

29

Selected

SubCoursesb

(Y01-Y16)

AA

NG/USAR

12

20

8

11

2

1

1

9

4

4

27

45

Total AA

NG/USAR

68

174

35

108

26

49

12

37

57

68

198

436

Enrollment as of 1 September 1979

^Persons who enrolled in less than a full course

20
TABLE 5

DISTRIBUTION OF ENROLLMENTS IN TERMS OF CURRENT SKILL LEVEL3

Course Component 1

Current

2

Skill

3

Level

4 5 Unknown Total

OPS SGT

(Y01-Y04)

AA

NG/USAR

3

3

11

7

19

34

50

70

7

88

2

7

92

209

INTEL SGT

(Y05-Y08)

AA

NG/USAR

1 5

3

15

23

9

32

2

39

3

2

35

99

OPS SPEC

(Y09-Y12)

AA

NG/USAR

25

23

5

12

1

11 2 1 5

31

54

INTEL SPEC

(Y13-Y16)

AA

NG/USAR

' 10

6

1

14

2

8 1 -

- 13

29

Selected

SubCourses*!

AA

NG/USAR

6

3

6

5

4

3

3

13

5

20

! 3

j 1

27

45

Total AA

NG/USAR

' 45

| 35

28

41

41

79

62

118

14

148

! 8

! 15

198

436

Enrollment as of 1 September 1979

^Persons who enrolled in less than a full course

21
TABLE 6

DISTRIBUTION OF ENROLLMENTS IN TERMS OF CURRENT MOSa

Course Component

(

AR

:ombat M0Sc

INF FA ADA ENG

Non-Combat MOSc

INTEL ORD OTHEF UNK Total

OPS SGT

(Y01-Y04)

AA

NG/USAR

9

46

31

71

36

42

10

5

3

7 7 11

1

15

' 2

5

92

209

INTEL SGT

(Y05-Y08)

AA

NG/USAR

3

38

15

22

4

22

5

1

6 2

12 2 1 1

35

99

OPS SPEC

(Y09-Y12)

AA

NG/USAR 14

10

20

2

6

10

3 1

4

2

2

2

3

2 4

31

54

INTEL SPEC

(Y13-Y16)

AA

NG/USAR 2 4

2

4

4

1

- 6

16

-

2

1 13

29

Selected

SubCourses^

AA

NG/USAR 6

13

7

3

19

8

2 1

1

1 2

2

7

- 27

45

Total AA

NG/USAR

12

106

69

124

47

93

37

12

| 9

9

13

38

2

17

6

27

3

6

198

436

Enrollment as of 1 September 1979

^Persons who enrolled in less than a full course

CAR » Armor, INF « Infantry, FA « Field Artillery, ADA • Air Defense Artillery,

ENG Engineering, INTEL * Intelligence, and ORD * Ordnance

22
As expected, many students had been in their current duty position for a short period of time. Thirty-nine percent and twenty-nine

percent of active Army and NG/USAR students, respectively, had held

their present duty position for six moths or less (Table 7). On the

other hand, 78% and 82%, respectively, of active Army and NG/USAR

students had a total of seven or more months in all operations or

intelligence assignments. This finding suggests that most students

were fairly familiar with operations or intelligence tasks.

The foregoing profile data suggest that the SOJET program was

reaching an appropriate target audience. The typical SOJET student

was: a senior NCO, enrolled in a course appropriate to his duty position, held a combat MOS, and had less than 12 months of experience in

his present duty position. In recent months about 50% of the students

were members of the active Army.

C. STUDENT PROGRESS

As of 1 September 1979 six hundred and thirty-four (634) students

had been accepted into the SOJET program. The training status of

these students as of 21 September 1979 is shown in Table 8. In addition to showing the training status of Active Army (AA) and National

Guard/USAR students, this table shows the training status of students

who enrolled at different time intervals. These groups will be called

"classes". Class 1 consists of students who enrolled during May-June

1978. Class 2 students enrolled during July-August 1978. Students

who enrolled during July-August 1979 are referred to as "Class 8".

Percentages calculated on the basis of total enrollment show that

program completion rates were low for both active Army (6%) and

NG/USAR (10%) students. Program completion rates did improve somewhat

for those classes of students which have been in the program a longer

period of time. Table 8 shows also that many students had never submitted a test ever though they had been enrolled in the program for a

considerable length of time. It should be noted also that participation by National Guard and USAR students was greater than for

active Army students both in terms of the actual number of persons who

enrolled ir the program (437 versus 198) and the percent of students

who were active to some degree in the program (49 % versus 30%).

Table 9 shows the percent of students active during various time

periods following enrollment. For this table, "active" is defined as

the submission of a test for a new subcourse; the test may be a

passing or failing one. The numbers displayed in Table 9 indicate

that a lower percent of active Army students submitted subcourse tests

than did NG/USAR students.

As noted earlier, most students enrolled in either the OPS SGT or

the INTEL SGT courses. For these two sets of courses Table 9 shows

23
TABLE 7

DISTRIBUTION OF ENROLLMENTS IN TERMS OF MONTHS IN CURRENT

DUTY POSITION, AND MONTHS IN ALL OPERATIONS OR INTELLIGENCE

DUTY ASSIGNMENTS

Duty Time

in Months

Months irr

AA

(N » 198)

Current Position

NG/USAR

(N « 436)

Months in All OPS/INTEL

Duty Assignments

AA NG/USAR

(N - 198) (N - 436)

1 - 6 395 29X 22$ 18X

7 - 12 28$ 24X 22% 17X

13 - 18 16X 12X 13% 10%

19 - 24 9% 9X 7X 8%

25 - 36 4X 9X 12X 111

37 - 48 - 4X 4X 8%

49+ - 7% 12X 21X

Unknown 4* 6X 8% 7X

24
I* -4it

o>

rs»

o>

t—i

1/

Ui

CD3

D.

UI

to

CO fcO

IUl s n

VO

o

^L *>8

iS

*

cc<

CO

**•

r-t CO to

S

1— oi o => ** *i

£

H- CO vo f-< CO

8 S 5

CM *** ! O

SK

-s. t*

**

CM<

C-J

i

i r«*

8

±Z oo

**

Ö

8 I § CM

«o*

i CM p>.

I

as **

*»s. r* sr

i

8

&to

1

*§

m ro vo

o

«J.S r-»

E

«

»t ** ' r~" 5

o*

i **

^ 1 ! s

*r CM *•* i Ü c^

as o%

8

,

8

g 1

<-> vo 2? vo f-4 1 c t </> vo

CO

m^

K

i

88 |i I l/l §

CM

If»

CM VO

j

JC

<"»

«

c: JC *t ¥*

8

Ö) 1 €

*s*

o VO f-4 CO

I

co5 r- ««r If»

n

*

1 co en

i3

CM

g

1 ro

8 1 O CM CM VO 5 SS

*•

0*£ VO3

K

1gVO «r8

»-« CO *t • "3

o*" tt »*

H

§

COl ro CO

^8

•~« ~* r>.

J

,„

t

^~

^

CM :•; vo to

i *i

S !

r^< KP »~« vn CO |j 1 £! m

•-« ^*

»-• *«

8

**

3

r* sr

t vo

CM CM •~i vo

frt ja

M

—•<

VO •»

i

"* %* CM O CM 1!

i c-» CM

«M ?*

*•« fc^ v\ >^ *t i 3

*n r* r> ro '•> 1 •-* ^• 4 f* |

l?

i-Z

V.

k

;

•»

5

* 1

f3 ^3 N? ''•*

S •-* <M M* vg | 1 u> »-4

»f» .> fc* ** ! »-* if» »f» s 5*»

«e

t

i OJ fs

* 1

^-i „"* 4^

j

X> 11

t. >- -Ü

»»

O ;'** 'i •^U

aI

c i* ij i» *3 *3 -3

3o *-» 4~» i •*- :i it u C "*. C V- *fl *»

I

.'!c** in •— j"> *— «» •*- :J* C» -#- ij

il •— *> *A i,". r*

9

r?

Jo

i :. 1

*• **

'J •7 ^> 1—

M

c v. -U1

1 i. co •—

c n. *> r„

Of . n L> 1 - C* o o .: -*> |

°^f ovaUabJc to DTIC rf^

^it
tus

in§

UJ

UJO

3

5

C3 8o

u. o

Ui o

UJ

CL

f-lI

CI

c

ÜJCM

f—I

fcj,

«ceo > i

CJ

8

.8

%* *•

KS

ÖS

CM

1« V*

i .a OJQ I X» I JD I -J

.8

**• VI o*— Ö VI (/» o < O

>- VI

-I»— .1 U-^

UJ 1*1

^O

r- VI 1- o«-*

w«O ux »->>.

*l

^ t- »*•

1 &. *^c*c% 3 u% 3 V.» wuiO O CV OO o. <\. >•

t>

O*>>

C* V» »~+

©ü

.8

.8

to

82

HM

3OU

.o3

V»o

u»

—3

o"u

v»Oo

c— <y

O 4» V»

i.3

oco

«

C

*- 0>

L.£

O

&

D>C 01

•r»

r»

V>

o

v»5

£• 0)

+>V

vt «•-

01

>

4-»

oU

* ex

«*-

o

o *J c *»» o r~

•«>-• «9§ «,»c o

.r«

o

=3 *»

V* «* S c

0»

?

u3

VI *>

«TJ VI A

V-

=N

o

*»

V» t- > o

»•-* £1

«-» «^ u *§ < i.*

*9 .n

26
that the highest percent of students were active during the first two

months following enrollment. This suggests that most students, if

they are going to participate at all in the program, begin their participation soon after receiving their lesson material. Table 9 shows

also that after 6 months participation in the program drops to eight

percent for AA students; for NG/USAR students this low level of participation is not reached until 12 months after enrollment. To anticipate the next section of this report, this last finding seems to be

due to personnel turbulence within the active Army.

Table 10 shows the cumulative rate at which new subcourse tests

were submitted following various periods after enrollment. This table

is based on students who submitted at least one test, passing or

failing. The data reveal that active Army students submitted many

more tests than did NG/USAR students during the early part of their

participation in the SOJET program. At the end of six months the

average active Army student had submitted tests over approximately

12 different subcourses as compared with six subcourses for NG/USAR

students. For operations and intelligence assistant/specialist courses this relationship held to a much lesser degree. For those students enrolled in selected courses only, the relationship was

reversed—more tests were submitted by NG/USAR students than by active

Army students.

The data contained in Tables 9 and 10 seemed to reflect, in

general, the opportunities for self-study which exist within the

active Army and within the National Guard/Reserves. Within the active

Army a student can look forward to reassignment to a new duty position

within a few months. Therefore, if he is to participate in a program

he must begin his participation as soon as possible and proceed as

rapidly as he can through his program of study. On the other hand,

members of national guard and reserve units are most apt to participate in training programs as part of their schedule of monthly

training activities. The data in Table 10 suggests also that students

enrolled in the SOJET Proyram should be given longer than 12 months to

complete their program. This appears to be especially true for

national guard and reserve students.

The SOJET program places no restrictions on the number of subcourses a student can enroll for. A few students enroll for only one

or two subcourses while a few others enroll for 30 or more subcourses.

About 25% of the students will enroll for one of the 16 full courses

plus a few additional subcourses of interest to them. For some students, therefore, the submission of tests over four different subcourses might mean that they have completed their total program of

instruction. For other students a submission of this number of tests

may mean that they have completed only 10 to 20% of their program.

27
9i

ÜJ

I

00

=>o

—< s>

CJ3 z

>—1 3O

-J

-Jouo

UJ t•—( Ifl s HCO z 3 LÜ

00

-J

00 —I

ro

oo C£

o LÜ

z

rH

H LÜ

LU LU 0£

_J 00 ÜJ

CO cz: r- < => H o ft o

CO 00 3 Q

00

O

2 a:

LÜ Ul Z O.

fe

oc

Ul

CO

as:

UJ

cc

Ul

UJo

CM 00 r** 00

CT> CO

CMO

vo CO uo

O vo »-( • « • * • • 1 CO C7> CO ** *r oo in rH

1-H

CM 00 r^ oo

Ch 00

CMO

^* COO

O VO

4-> rH

••%•

••

C1 CO Ol co *a- ^- 00

ft>co rH

E-H

r—

p—o

s-

-—

c r^ oo CM r^

Ul CM £^ cr. ooo r*»

rH CO yQ

•

•• ! s- i • co «a- ^ r^

CJrH co oo

+4 rH rH

W

<+- <

t/>

JC CM

O r»% co CM*- i 4JO CO CT> vr> vo O *f C rH • • • • • • O 1 CO h* C0<" «d- vo S O 1^ S

•r»

CM in r- oo

P>^ CO

o vn co

O 00

«TJ CO

••

••

••

>1 CM r** co*a- co

m

U r^. r-i

ft) PC

l-H

r* rH r^ ^> *• rH

0) VOm CM a> CM t-* r**

E1

••••••

•r- U0 rH VO CO CO CM

^

h- w-^

as CM en co

^ in ** co r*». oo m ^ ^o | • • • • • •

CO CO CM CM CO

CO 1-* Q\ CM co in

CM CO

o CO CO .-• vo

1

•••••t

,H U0 CM CM ^4 fH •—«

4-»C

ft) ft3

3

c 00 oo oo ag

s^ 2^ S^

V\3Oo

5

z

z

o

•o1 I ^ 01 -O

h- r3

4-i1 00 r- 00

C7>rH 00 IA < ft)

O

00oux

>- tf>

c o>

_l*-» -JJ Ul 4->

Ul Ul

3 VO

u

r- to hOH tO 0» <A

0»

Z 0»

zo>

C— 0)

V» •—• i/> >-*

1

O QJ (A

L.

^ v.—„ ^»O VT» tO OO

U

3 00

3 00 00 Ul

o

U3

o o.oo CL Q. >- ft»

co

<->

oo >-

O 00w a. •*-

U

+J

ft) c

flo

C7> c

r»

U)

Q.

ft) Co

IA

ftl

ft) 3

40O

toco

V)

L.

ft)

Q. Co

•oft)

to

03

CO

TJ

28
Table 11 shows the percent of program completed following various

months after enrollment. Each percent figure contained in Table 11 is

based on a different number of students, namely the number of active

students still remaining in the program. Therefore, those percentages

listed under intervals 13-14 and 15-16 (classes 1 and 2, respectively)

are based on a very small number of students. The percentages

displayed in Table 11 reflect the findings already discussed with

respect to Table 10. After six months all participating active AA

students had completed 57%-of their program versus 34% for NG/USAR.

Collectively the data displayed in Tables 7 through 11 indicate

that based on alj_ enrol lees: (1) only about 50% can be expected to

actively participate in their program of instruction. The remainder

will become "no-starts"; and (2) the percent of persons completing

their entire program will be quite low, on the order of 10%.

Based on students who are active at some point in the program,

student progress data suggests that (1) most students are active early

in their program (Table 9); (2) most new subcourse tests will be

submitted during the first six months of the program (Table 10);

(3) the number of different tests submitted will be much higher for AA

than for NG/USAR students (Table 10), and (4) of those students who

participate at all in a program most of them will complete a high percentage of their instructional program (Table 11)." It seems reasonable to conclude that these people will derive considerable benefit

from participating in the program.

D. IMPACT OF PERSONNEL TURBULENCE

In the S0JET program the requirement that the course supervisor

administer and score subcourse examinations inevitably sets the stage

for disruptions in student progress whenever the course supervisor is

unavailable. Temporary unavailability may be caused by the supervisor

being in the field, on TDY, or on leave. When the supervisor is

reassigned to another job, undergoes a permanent change of station, or

is discharged, the student will have to obtain a new course supervisor. Likewise, when the student is reassigned or sent to a new

post, his new duties are very likely to conflict with, and may not

even be relevant to, the S0JET course in which he is enrolled. Thus,

he would have little incentive for continuing in the program; at a

minimum he would have to arrange for a new course supervisor.

According to IPD estimates, personnel turnover within the active

Army is about 25% per quarter. Based on information acquired from

active Army personnel during the field visits conducted as part of

this study, it is estimated that the annual turnover rate for enlisted

and for officer personnel is at least 50% and 100%, respectively. In

29
3

r."•

taB

<0Oz

LU

CD2o

oo

3!

LU

s:

00

=3

-IO

2i

UJ 8E

vo

rH ****

LO| C0*fr CO vo

t -a _Q _Q ss

CO vo ! ^

4J fH

co oo

.

****

ct

1 -o J3 J3 r^

o

cuco vo

m

m vo

E»-H r-

»"" O

L. C

LÜ CSJ

f-t **** .8 -O*» ****

L.I VO CO flQ »H «a0> rH 00 «fr CO co 00 ^> 1 +J 1-»

<+- <

«/)

jr

4-»0 **** *t ** **%* C rH r^ f*. 1 CSJ .o vo CO vo O 1 00 CO CSJ uo f>» CO

JE en c

•r-

«00 83? -o en ss CO vo > 1 vo co CSJ csj*- vo fo

4Jc ! r-«

0> VO

LO en cn csi

•r- to voco U*> cu rH CO

m co

«e £8 *f CO

CO CSJ

rH r*» to

R

CO

* CSJ

^ CSJ «^ CSJ

CM1 ^- rH VO rH r-* co csj

^

rH CSJ rH CSJ rH m>^ CSJ rH

4-»C

4»

%3

33

C oo 00 oo OO oa S£ §^ S^ §^ EO aZ g g

CDz

o

^

"O1

a> -o

h-

—3

4-»1 oo r- 00

0»rH oo to

O IA

ooo i- -o +•>

>-

M

c a>C

01

LU UJ

3 VOu "O

r- «O HOH 40 OJ 10

3

4>

za ru>

C— Of 4->

%A <-* (A *-•

i

O 0) to

to oo too I» *>* u*-* xuo> 3 OO 300 00 LU O U 3 p» O O. OO a. o- >- tico r* o o o >- O 00«— CL •- U <

«p

(A

CÜ

-M

<Uco

•p

to

OJ

«/>

**a OJ 5l >

t- a»o

o -oo

4-> *-» c «— ••- r— CO

•r- +J r^c U (A 0)

OJ TJ

0J XI3

r— *J

L3 IA

OO

V»-U <+- o

4-»O

C .C L. 0 * QJ U JQ

oicS

a. «2

01 «o

u 30
view of such estimates, there is a high probability that many of the

potential "no-starts" and "drop-outs" reflected in the data presented in

Table 8 are due to personnel turnover.

As a prelude to the third survey of this study, an effort was

made to contact by telephone all active Army course supervisors

assigned to a CONUS installation. From these telephone contacts, specific information was obtained on the present location of 104 of 111

enrolled students and 61 of 65 registered supervisors. This information is shown in Table 12.

Of the 104 students, 73 still were assigned to the unit they were

in when they first enrolled in the SOJET program; however, 8 of the 73

were in the field, on TDY, or on leave. The remaining 31 persons were

no longer in their original unit; most of them having been reassigned

to another installation. Thus, it would appear that reassignment

accounted for lack of participation by about 30% of the students.

Of the 61 course supervisors, 26 (43%) were no longer available

to carry out their responsibilities as course supervisors; 12 had been

reassigned to another job, 11 had undergone a permanent change of station, and 3 had been discharged.

In the SOJET program unavailability of either a student or a

supervisor can disrupt student progress. From the telephone survey

described, the status of 111 student-supervisor pairs was determined.

These data are displayed in Table 13. Positive information was

obtained on 100 student-supervisor pairs; for the other 11 pairs, the

status of one or both parties was unknown. In those 100 pairs for

which information was available, there were 47 pairs in which both the

student and the supervisor were still assigned to the same unit they

were in when they first enrolled/registered in the program. Although

some of these people were on TDY or on leave, they were judged as

being available to continue in the SOJET program. For the remaining

53 pairs, either the supervisor, the student, or both were away

attending school, had been assigned to another job, had been assigned

to another post, or had left the service. On the basis of the

foregoing information, it is estimated that at least 50% of the students who enroll in the SOJET program and who are members of the

active Army will have their program disrupted.

When a student is reassigned to a new job or undergoes a permanent change of station, he is temporarily disenrolled from the SOJET

program. Once he is established in his new job or station, he has the

option of re-enrolling in the program and continuing his course of

study. This option is seldom exercised, probably because personnel

often are temporarily assigned to in the S2 and S3 sections of a battalion until they are needed elsewhere; when they are reassigned, they

31
TABLE 12

STATUS OF ACTIVE ARMY STUDENTS AND COURSE

SUPERVISORS (CONUS) AS OF AUGUST 1979

Status

Enrolled

Students

(N = 111)

Registered

Supervisors

(N = 65)

In Unit 65 25

In Field 5 4

On Leave 1 3

TDY 2 3

Reassigned 6 12

PCS 24 11

Discharged 1 3

Unknown 7 4

TABLE 13

STATUS OF STUDENT-SUPERVISOR PAIRS

AS OF AUGUST 1979«

(N « 111)

Students

Supervisors Available Not Available Unknown

Available 47 17 1

Not Available 22 14 -

Unknown 4 — 6

Total 73 31 7

aStatus as of 20 August 1979

32
have little reason to continue their S2 or S3 course of instruction

unless they have an MOS in which operations or intelligence tasks are

covered in their Soldier's Manual.

The data presented in Table 13 suggest that there is a relationship between the number of months since enrollment and turnover

rates. Table 14 contains data bearing on this relationship.

Examination shows that for the 47 pairs where both the supervisor and

student were available, it' had been an average of six months since

enrollment in the SOJET program. For the 22 pairs in which the supervisor was gone but the student was still in the unit, the students had

enrolled on an average of seven months earlier. For 17 additional

student-supervisor pairs, the student was gone but the supervisor was

present; those pairs had enrolled in the program approximately eight

months earlier. Finally, for those 14 pairs where both the student

and supervisor were gone from the unit, those pairs had enrolled an

average of 10 months prior to the telephone survey. The foregoing

data substantiate the obvious—the longer the period since enrollment

into the SOJET program, the higher the probability that both the student and supervisor will no longer be assigned to the unit they were

in when they first enrol led/registered into the program.

Another set of data bearing on personnel turbulence is displayed

in Table 15. Three questions on personnel turnover from the supervisor questionnaire used for the third survey, are listed in Table 15,

along with the percentage of persons who provided various responses

to each question. In 58% or more of the cases, the supervisors stated

that personnel turnover did not prevent students from enrolling in the

program, prevent students from completing the program, or make supervisors reluctant to register in a program. This data supports the

telephone survey findings—personnel turbulence may prevent 30-40%

of enrolled students from completing this program. It also may prevent considerable numbers of potential students from enrolling in the

program.

The impact of personnel turbulence on test submission rates also

can be inferred from the data presented in Tables 9 and 10. Table 10

shows that the rate of test submission for active Army personnel was

more rapid than that for NG/USAR personnel through the first four

months of their enrollment. After six months, very few active Army

students continued to submit tests while many NG/USAR students continued to do so. Based on the foregoing findings it seems highly probable that personnel turbulence had an adverse effect on student

progress, especially within the active Army. Removing the requirement for a registered course supervisor might decrease this effect

somewhat.

33

fcl !!
TABLE 14

PERSONNEL TURBULENCE OF STUDENT-SUPERVISOR PAIRS

AS RELATED TO MONTHS SINCE ENROLLMENT

Status of Student-Supervisor

Pairs N

Average Number of Months

Since Enrollment

Both student and supervisor present 47 6 months

Supervisor gone, student present 22 7 months

Student gone, supervisor present 17 8 months

Both student and supervisor gone 14 10 months

TABLE 15

SUPERVISOR REPLIES TO QUESTIONS ABOUT

PERSONNEL TURNOVER«

Component N

Response Options

Question Yes Sometimes No

Has personnel turnover:

a. Prevented potential students

from enrolling in the program?

AA

NG/USAR

29

53

4%

13%

35% 61%

27% 60%

b. Prevented stgudents from

completing program?

AA

NG/USAR

29

53

8%

9%

33% 58%

22% 69%

c. Made you or others reluctant

to register as a course

supervisor?

AA

NG/USAR

29

53

21%

11%

21% 58%

13% 76%

aThird Survey Data

34
E. COST COMPARISON OF SOJET AND IPD CENTRAL MANAGEMENT PROCEDURES

During this effort the absolute and comparative costs of administering a correspondence course by current IPD and by SOJET original

and revised procedures were calculated. As a first step, the time

needed to perform the tasks required of each IPD staff member was

determined. A recent "Schedule X" (Workload Estimate Chart) prepared

by IPD provided the task listings and time estimates for IPD procedures. These procedures were reviewed with IPD personnel and the time

estimates for each task raised or lowered to reflect SOJET program

procedures. Whenever possible, time estimates for tasks were based on

actual work counts and/or on personnel observation. All time and nonlabor cost estimates were derived so that they could be converted to

"cost per enrolled student per year."

The entries in Table 16 illustrate how time requirements were

calculated. IPD enrollment clerks are responsible for processing

enrollment applications, a task that can be divided into five elements. Listed under IPD procedures (Table 16) are the estimates of

the time required to perform each task element of the enrollment task.

In the SOJET program, the original procedures involved processing two

more enrollment forms than did IPD procedures, and one of these forms,

the Student Training Plan, was time consuming to edit and code. On

the other hand, the SOJET program has no eligibility requirements or

equivalent credits, eliminating two steps in the enrollment process

that must be followed under IPD procedures.

Table 16 shows that, following IPD procedures, it takes 8.25

minutes to process one enrollment application. The process requires

9.25 minutes per student when the original SOJET enrollment procedures are used. Using revised SOJET procedures, it takes about 7.5

minutes to enroll one student-supervisor pair.

To prepare estimates of annual manpower requirements, IPD converts "minutes to perform a task" into hours per year required to

handle a full student complement. From July 1978 through June 1979,

IPD processed 202,009 enrollment forms (DA Form 145). Thus, using IPD

enrollment procedures an estimated 27,776 hours (202,009 x 8.25 minutes T 60) were required in that year to process student enrollments.

Using procedures similar to those just described, IPD has calculated the hours required by each of its organizational elements to

support 202,009 students (Table 17). The estimates were adjusted to

reflect original and revised SOJET procedures. Of particular interest

is the estimate that, for IPD and for revised SOJET procedures, the

time required to process and to support one student is approximately

1.3 and 1.75 hours, respectively. Thus, revised SOJET procedures

require almost 36% more labor than do current IPD procedures.

35
TABLE 16

TIME REQUIRED TO PERFORM ENROLLMENT TASK IN ACCORDANCE WITH IPD

PROCEDURES, ORIGINAL SOJET PROCEDURES

AND REVISED SOJET PROCEDURES

(MINUTES)

Central Management Procedures

TASK

IPD*

Standard

SOJET

(Original)

SOJET

(Revised)

Student Supervisor Student Supervis

Process Incoming Enrollment Applications

a. Review forms for

correctness/

completeness 0.25 0.50 0.25 0.25 0.25

b. Compare request with

catalog entry to

determine eligibility/

availability 3.00

c. Check for and certify

equivalent credits

for student 1.50 •a as m m

d. Code information for

computer input 2.00 5.00 2.00 3.00 1.00

e. Make up and mail

welcome letter 1.50 1.50 1.50 1.50

Total Minutes 8.25 4.25 7.50

aTask elements and time requirements under IPD procedures based on data

developed by Institute for Professional Development.

36
TABLE 17

COMPARISON OF LABOR REQUIREMENTS AND PRINTING AND MAILING COSTS

REQUIRED TO ADMINISTER ENTIRE ACCP ACCORDING TO IPD,

ORIGINAL SOJET AND REVISED SOJET PROCEDURES

Central Management Procedures

Cost Categories IPD

Standard

SOJET

(Original)

SOJET

(Revised)

Labor Costs

Student Services Division

a. Office of Division Chief

b. TREDS-NRI Section

c. Processing Branch

d. Administrative Branch

2,788 hrs*

6,800

153,637

12,444

2,788 hrs.

6,816

356,171

18,300

2,788 hrs.

6,834

161,371

14,292

Production and Distribution Division 83,631 167,262 167,262

Total Man-hour

Requirements 259,300 551,377 352,547

Man-hours Required per

Student (N-202,009

Students) 1.284 2.729 1.745

Salary Required to Process One Student At

/Werace of $6.00/Hr. $ 7.70 $16.33 $10.47

Non-Labcr Costs

Print frig: Cost to support

one student0

Mailing: Cost to support

one student*»^

$18.12

7.38

$20.92

1G.48

$18.60

14.64

Estimated tost to Process

and Support One Student $33.20 553.73 $43.68

dAssume that IPD and SOJET students are each sent an average of 12 subcourses.

bSubcourscs sent by book rate; other material s«nt by First Class *nil.

Mailing costs based on non-discounted rdte»s.

37
SOJET procedures are more labor-intensive than those used by IPD

in part because a supervisor must be registered along with each

student. This doubles the paperwork required to enroll one student.

Also, SOJET procedures require that subcourse material be sent to both

the student and the course supervisor, a requirement that contributes

to the doubling of mailing costs and the labor associated with these

activities. An examination of Table 18 will clarify these points.

Table 18 lists the tasks on which IPO and SOJET labor requirements differ by at least 10%. For each listed task, the annual hours

required to process 202,009 students were determined. Points to note

are:

o For 11 of 16 listed activities, manpower requirements are

higher for SOJET procedures.

o Mailing out a SOJET course guide (original procedures) Is

the most time-consuming requirement of the SOJET program.

Abojt 10 minutes per guide are required. Under the revised procedures the course guide would be discontinued.

o Mailing test material separately to supervisors doubles the

amount of labor required to distribute subcourse material.

o The requirement to register the supervisor greatly increases

the time needed to process source documents, maintain

quality control over source documents, and complete work

unit logs.

The figures in Table 17 demonstrate that the central management

of correspondence courses, even though computer supported, is a highly

labor-intensive operation because of the number of times different

pieces of paper must be handled. For example, a one-minute increase in

the time required to process 200,000 enrollment applications would

increase IPO's workload by 3,333 hours. Obviously, doubling the

number of forms in a student application packet would increase IPO's

annual workload by thousands of hours.
TABLE 18

i

i

ESTIMATED ANNUAL KANHOURS REQUIRED FOR SELECTED ACCP ACTIVITIES

BASED ON IPD, ORIGINAL SOJET, AND REVISED SOJET PROCEDURES (HOURS)

I

Central Management Procedures

Organizational Element and Work IPD SOJET SOJET

Activity Standard (Original) (Revised)

Student Services Division

a. Office of Division Chief - - -

b. TREDS-NRI Section - - -

c. Processing Branch

(1) Branch Chief - - -

(2) Team Chief

(a) Breakdown work for team daily 313 939 626

(b) Decision making/interpreting

policy 247 210 210

(c) Maintain records 65 78 76

(3) Sr. Training Technician

(a) Correct edit listing

(b) Quality control of source

173 47 31

documents 4,230 11,421 7,614

(c) Determine waiverability

(d) Review Form Letter 15

520 1,040 1,040

867 3,466 216

(4) Training Technician

fa) Process source documents

(b) Completion of work unit log

4,506 12,797 8,111

2,050 9,082 6,888

(c) Process supervisor changes - 3,872 3,872

(5) Enrollment Clerk

(a) Process Enrollment forms 27,776 31,143 25,251

(b) Return erroneous applications 2,525 7,575 3,367

(c) Mjil SOJET Course Guide - 30,411 -

(d) Complete work unit log 2,050 1,304 1,304

d. Administration Branch

(1) All personnel

(a) Opening/processing mail 12.444 18,300 14,292

Production and Distribution Division

a. All personnnel

(0 All required activities^_ 83^G31_ 167^?^ 167,262

30
The increased cost of the SOJET program can be attributed primarily to the need for subcourse test scoring guides. This is a

document not normally employed with correspondence courses. In addition to preparation costs, scoring guides nnjst be printed, stored,

retrieved and mailed. Any training program that employs test scoring

guides usually will incur costs above those for courses where tests

are centrally scored by machine.

40
V. EVALUATION OF FEATURES OF THE SOJET

PROGRAM DELIVERY PROCEDURES

A. ACCEPTANCE OF REQUIREMENT FOR A REGISTERED SUPERVISOR

When a student enrolls in a SOJET course, a supervisor must

register along with him as a course supervisor. This course supervisor is expected to help the student identify his training requirements, take responsibility for storing and maintaining security of

test material, administer and score subcourse tests, provide immediate

test feedback to the student, and assure that test results are forwarded to the Institute for Professional Development.

Survey and interview data relating to the registered supervisor

requirement are summarized in Table 19. A majority of students (60%)

accepted the concept of having a course supervisor who is involved

with on-the-job extension training. Moreover, potential students

(89%) and supervisors (98%) expressed a willingness to accept the concept of course supervisor. Registered supervisors never were specifically queried about their willingness to register as a course instructor.

They did have an opportunity to comment on this requirement and no

complaints were received about it.

One of the potential disadvantages of the SOJET program is that

the administration of a test requires the availability of two persons,

a student and a course supervisor. In reply to a query about this

potential problem, 85% of active Arn^y students reported that their

supervisor usually or always was available: 82% of the students in

National Guard and Reserve units provided a similar report (Table 20).

However, 28% and 22% of active Anqy and NG/USAR supervisors, respectively, reported that it usually or always was difficult to find a

time when both student and supervisor were available for testing or

for review of test scores (Table 21, Question 6j).

The SOJET program is based in part on the assumption that

training benefits from the program will be increased by involving

first-line supervisors in the training. Most supervisor and student

respondents favored the active participation of a registered course

supervisor. However, as noted already, the requirement to have a

registered supervisor: (1) increases administrative costs (p. 38 )

and may make continued student participation difficult due to personnel (Supervisor) turbulence (p. 33 ). If so, this rests on the

requirement for a supervisor to administer and score tests as opposed

to the requirement to register as a course supervisor. The presence

of a registered supervisor may have increased the extent to which students participated in the SOJET program, but there are no data to substantiate this.

41

i* »ill ! !
TABLE 19

SUMMARY OF INTERVIEW AND SURVEY DATA

RELATED TO ACCEPTANCE OF THE ROLE

OF COURSE SUPERVISOR

*

SOURCE

(ANNEX A)

0 98% of potential supervisors were willing to

register as a course supervisor Table B-10

0 88% of potential supervisors were willing to

administer and score pre-and posttests Table B-9

0 89% of potential students were willing to

enroll along with a course supervisor Table B-ll

0 72% of persons requesting information about the

program reacted favorably to a description

of the program Table D-3

0 60% of enrolled students expressed a preference

for having a course supervisor; another

18% said they "did not care one way or the

other1' Table E-l

0 60% of the persons Interviewed during field

trips preferred supervisor involvement

with the SOJET program Table B-12

Table B-16

42
TABLE 20

STUDENT REPLIES TO QUESTIONS RELATED TO TESTING PROCEDURES3

Question Number Component N Response Options

and Question Never Sometimes Usually Always m

10. Does your super- AA ' i 17 — _ — 100% m

visor administer

and score your NG/USAR 60 8% 5% 13% 70% 3%

pretests?

11« Does your super- AA 17 - . - 94% 6%

visor administer

and score your NG/USAR 65 5% 3% 7% 82% 2%

posttests?

12. Is your super- AA 20 . 10% 10% 75% 5%

visor readily

available to NG/USAR 65 5% 12% 37% 45% 2%

administer and

score your tests?

13. Does your super- AA 17 • - 12% 88% -

visor provide

feedback to you NG/USAR 60 5% 13% 27% 52% 2%

within one or two

days after you

take a test?

14. Do you get to AA 16 69« 13% 6% 6% 6%

look at your

lesson material NG/USAR 58 i 53% 21% 12% 9% 5%

when taking a

pretest?

IS. Do you get to AA 17 65% 12% 12% - 12%

use your lesson

material when NG/USAR 58 57% 14% 17% 9% 3%

taking a posttest?

aThird Survey Data

43

fa**«
TABLE 21

SUPERVISOR REPLIES TO QUESTIONS RELATED TO TESTING PROCEDURES«

Question Number

Component N

Response Options

and Question Never Sometimes Usually Always

6a. Do you personally

administer and

score pretest?

AA -

NG/USAR

17

41

4%

2%

4%

6%

4%

14%

87%

78%

6b. Do you personally

administer and

score all posttests?

AA

NG/USAR

17

37 - 6%

4%

4%

96%

89%

6c. Are the test

scoring guides

easy to use?

AA

NG/USAR

17

41 61

22%

10%

48%

45%

30%

39%

6d. Are the test

scoring guides

accurate?

AA

NG/USAR

17

39

- 22%

10%

43%

56%

35%

33%

6e. Do you study

lesson or test

material before

giving a test?

AA

NG/USAR

17

41

26%

10%

35%

50%

26%

20%

13%

20%

6f. Do you allow

students to take

tests as "open

book" exams?

AA

NG/USAR

13

41

48%

55X

38%

18%

5%

18%

9%

8%

6g. Do you provide

feedback to

students within

1-2 days after

tests?

AA

NG/USAR

15

39 21

5%

4%

18%

23%

77%

71%

6h. Do you have any

problems finding

a place to store

tests?

AA

NG/USAR

19

39

88«

75%

8%

19% 4%

4%

2%

61. Do you have any

problems maintaining test

security?

AA

NG/USAR

15

45

82%

96%

18%

4% - -

6j. Is 1s difficult

to find a time

when both you

and the student

are free to take

and to review a

test?

AA

NG/USAR

15

45

1 18%

20%

54%

59%

23%

14%

5%

8%

•Third Survey Oata

44
A SOJET-like program must have some means of sending test

material to a responsible person in the field. In the SOJET program

this is accomplished by sending the material to a registered supervisor. A decision to continue this requirement or to consider other

options must be based on policy.

Although responding favorably to the concept of a registered

course supervisor, a number of students and supervisors expressed

reservations about some of the specific activities and procedures

required of course supervisors. These requirements are reviewed on

the following pages.

B. ACCEPTANCE OF ENROLLMENT PROCEDURES

When the SOJET program first was implemented, three enrollment/

registration forms, and at least one Student Training Plan had to be

completed—a process which, on the average, required 34 minutes of

supervisor time (Table 22). A number of survey respondents complained

about the time-consuming and complicated nature of the enrollment/

registration process (Table 23). Also, IPD personnel reported that

many enrollment applications had to be returned because the Student

Training Plan or one or more of the forms were filled out incorrectly.

In addition, analysis of the data recorded on Student Training Plans

led to the conclusion that these plans were not being filled out as

intended (see pp.57-61). For these reasons it was concluded that

enrollment procedures were inadequate and unacceptable, and in need of

immediate revision.

Currently the SOJET program uses a revised set of enrollment procedures and forms. The number of enrollment forms was reduced from

three to two and the requirement to fill out and return a Student

Training Plan was eliminated. Also, enrollment instructions were rewritten to make them easier to follow. As a result of these actions

the number of enrollment applications rejected in recent months has

decreased substantially.

C. ACCEPTANCE AND ADHERENCE TO TESTING PROCEDURES

The SOJET program employs a variety of testing features which, as

a total package, ^re quite unique. Data related to the acceptance of

and adherence to these features are reviewed on the following pages.

Storage and Security of Tests

In the SOJET program all lesson materials requested by the student are mailed to him immediately following enrollment. This is

45
TABLE 22

AVERAGE TIME TO CONDUCT COURSE SUPERVISOR ACTIVITIES

(MINUTES)*

ACTIVITY ACTIVE ARMY

SUPERVISORS

NG/USAR

SUPERVISORS

Officers Enlisted Officers Enlisted

Enroll one student 34 48 29 26

Give one test 30 28 39 19

Score one test 17 10 16 13

Review results of one

test with student 18 22 19 31

Send results of one

test to IPD 13 6 11 9

Total 112 114 114 98

dTh1rd Survey Data, Question 2, Supervisor Questionnaire

46

• - '- •- i

-. ^ . . a.
TABLE 23

SUMMARY OF CRITICAL COMMENTS MADE BY

ENROLLED STUDENTS AND REGISTERED SUPERVISORS*

Comments

1st Survey I! 3rd Survey

(NG/USAR Respondents! (AA & N6/USAR Respondents)

Supervisor

(N«32)

Student

(N-38)

Supervisor

(N«86)

Student

(N-85)

Enrollment procedures too

complicated and time

consuming

Scoring/admi ni strative

procedures too time

consuming

Errors 1n subcourse material,

tests, and scoring guides

Material poorly written/

material and/or test

questions confusing

Way needed to Identify which

material goes with whict

student

Master 11st needed for

material sent to student

and supervisor; cannot

tell if anything 1s missing

Takes too long to get material

after enrolling |

More time needed to complete

stucty |

Material not always relevant

Supervisor is not needed

Need "800" hotline to IPD

Miscellaneous

4

1

8

4

11 16

2

4

2

1

14

5

4

3

3

6

aNumber of persons who made same or similar comment. Replies of

active Army and N6/USAR students and supervisors have been combined«

47

^te*Ü^M^^
called Mone shot" mailing. Similarly, all corresponding test

materials are mailed to the registered course supervisor, who is

responsible for storing and maintaining security of the test material.

When the SOJET program began there was some concern as to whether

supervisors had access to adequate storage facilities. Interviews

with actual and potential supervisors and students indicated that all

supervisory personnel had access to a secure storage area. During the

third survey 81t of course supervisors reported never having

storage problems. An additional 14% reported that they sometimes

have such problems (Table 24). Reports of storage problems were most

apt to come from National Guard or Reserve personnel. This reflects

the finding that some National Guard personnel became the course

supervisor for as many as 12-15 students.

Eighty-nine percent of the registered supervisors who responded

to the third survey reported having no problems maintaining test

security. Security problems were more apt to be reported by supervisors in the active Arn\y (18%).

The foregoing findings indicate that the "one-shot" mailing of

lesson and test material is an acceptable practice—it causes few if

any storage and security problemsComplaints not related to storage or security were registered

about the "one-shot" mailing approach (Table 23). When a person

registers as a supervisor for two or more persons, he receives test

packets for each student. When the program was first implemented,

there was no easy way for the supervisor to match test material with a

student. In response to complaints about this problem the procedure

was changed. Currently, the address used to mail out test material to

the supervisor also lists the name of the student to whom the material

applies.

Tests Administered and Scored by Supervisor

The SOJET program course supervisor is responsible for administering and scoring both pretests and posttests. Detailed scoring

guides are provided for each test. The first survey, dealing with

National Guard and Reserve supervisors, Indicated that test administration and scoring procedures were being followed rather closely.

The findings of the third survey for enrolled students and registered

supervisors are presented in Tables 20 and 21.

Both enrolled students (Table 20, Questions 10 and 11) and

registered supervisors (Table 21), Questions 6a and 6b) reported that

most course supervisors do personally administer both pre and post48

ii i i
TABLE 24

SUMMARY OF SUPERVISOR RESPONSES TO QUESTIONS RELATING

TO SUBCOURSE TEST STORAGE AND SECURITY*

Question Component

6a« Do you have any

problems finding

a place to store

tests?

6b. Do you have any

problems maintaining test

security?

AA

NG/USAR

AA

NG/USAR

N

Response Options

24

48

22

51

Never Sometimes Usually Always

m 8% - 8%

75% 19% 4% 2%

82%

96%

18%

4%

«Third Survey Data

TABLE 25

NUMBER AND PERCENT OF STUDENTS FOR WHOM VARIOUS

COMBINATIONS OF PASSING AND FAILING PRETESTS AND

POSTTESTS WERE SUBMITTED

Passing or Failing

Status of Tests

Submitted

Pretest

(N - 273)«

Test Form

Posttest

(N 273)«

Passing tests only 105 (39%) 176 (64%)

Mix of passing and

failing tests 84 (31%) 14 ( 4%)

Failing tests only 20 ( 7%) mm

Did not submit this

test form 64 (23%) 83X (31X)

«Number of persons for whom either or both a pretest and posttest was submitted.

49

*Mta
tests. However, supervisors of National Guard and Reserve units were

less apt to be personally involved in test administration (82% for

posttests) than were active Am\y supervisors (94% involvement). The

first survey of NG/USAR supervisors revealed that the training NCO of

those units sometimes administered the tests; also, on occasion, students were allowed to take the tests at home, although scoring was

always done by a supervisor or training NCO. These findings indicate

that course supervisors did accept responsbility for the administration and scoring of tests. ,

Test Scoring Guides

The test scoring guides provided to course supervisors were meant

to be usable by non-subject matter experts. Most supervisors (about

80%) did report that these guides usually were easy to use. They

pointed out, however, that some guides contained errors (Table 21,

Question 6d) and sometimes were difficult to use; 7% of the NG/USAR

supervisors felt that the guides were never easy to use. Most of the

difficulties with the scoring guides seem attributable to the

collation errors which some of them contained. This problem is

expected to be removed when the guides are revised and corrected.

Generally, the data indicated that the use of test scoring guides is

an acceptable way to provide test scoring guidance to non-subject

matter experts.

Feedback to Students

SOJET program course supervisors were requested to provide immediate feedback to the student, an action that cannot be accomplished

using the traditional delivery sytstem for correspondence courses.

Almost all supervisors complied with this request. Over 95% of the

supervisors reported that they usually or always provided feedback to

students within one or two days. Active Army students confirmed this

(Table 20, Question 13). However, almost 18% of the NG/USAR students

reported that their supervisor seldom or never provided Immediate

feedback. This tended to be confirmed by the replies of their supervisors as shown in Table 21, Question 6g. These findings can be

accounted for by the difference in availability between active Army

and National Guard/Reserve supervisors. In the active Army the supervisor and student may be in daily contact, while in National Guard

and Reserve units this contact may be on a monthly basis. Indeed, some

NG/USAR students commented that test feedback usually was given them

during the next scheduled duty period.

50
Use of Subcourse Material

It seems reasonable to presume that many tasks taught 1n the

SOJET courses will be performed on the job using appropriate subcourse

lesson material as job aids; 1f this presumption 1s correct, the

ability to pass a subcourse test using SOJET subcourse lesson material

probably 1s an accurate Indication that job requirements can be met.

As shown in Table 20, 25% of active Am\y students and 42% of

NG/USAR students reported that they were allowed to take the pretests

as an "open book" test, at least on occasion. Approximately the same

percentage of students said they took posttests as "open book" tests.

As shown in Table 21, an even higher percentage of supervisors

reported the practice of "open book" testing (Question 6f).

The foregoing findings may account, 1n part, for the high percentage of persons who passed pretests or who passed posttests on the

first attempt. The SOJET subcourse material 1s explicit enough so

that there is little reason for persons to fail on an "open book"

test. Furthermore, 1t should be noted that "open book" testing is not

specifically prohibited in the SOJET program, nor in most other

correspondence courses, for that matter. It should be remembered also

that many students had been assigned to operation cr Intelligence

positions for many months (Table 7, page 24). If a decision is made

that students should not consult lesson material during testing, a

specific statement to that effect should be prominently located in

each SOJET subcourse test scoring guide.

Test Reporting Requirements

As of late September 1979, 273 of the 634 students enrolled as of

1 September 1979 had submitted tests for one or more subcourses.

Table 25 shows the number and percent of students for trftom passing and

failing pretests and posttests were submitted. The data indicate

that the majority of students (77%) who submitted any kind of test

submitted a pretest. A further analysis of the data contained in

Table 25 revealed that 61% of the students submitted passing tests

only; 18% submitted only passing pretests; 23% submitted only passing

posttests, and 20% submitted a mix of passing pretests and posttests

(Table 26).

During this study It was not possible to estimate the percent of

persons who did not submit a failing pretest. At a minimum, 64 students never submitted a pretest; they submitted only posttests, almost

all of which were scored as "passing". It seems probable that some if

not most of these students failed the pretest on one or more subcourses but ne^r reported these failures. It was concluded, therefore,

that the requirement to report falling pretest scores was to some

degree unacceptable.

51
ä

I

TABLE 26

PERCENT OF STUDENTS FOR WHOM VARIOUS

COMBINATIONS OF PASSING AND FAILING PRETESTS

AND POSTTESTS WERE SUBMITTED

s

Results of Tests Submitted

Form of Test

Submitted

All Passing

Tests

Mix of Passing

and

Failing Tests

All Failinc

Tests

Total

All Pretests

All Posttests

Mix of Preand Posttests

18%

23%

20%

9%

26%

4% 31%

23%

46%

Total 61% 35% 4%

•

52
Both the field trips to CONUS units and the Initial survey of

NG/USAR students and supervisors produced Information suggesting that

the test recording procedures were in need of change. The original

procedures reportedly were too cumbersome; the requirement to submit

failing pretest scores was not being followed in some instances; and

IPD procedures for processing SOJET subcourse test results were time

consuming. To correct these deficiencies a new set of scoring procedures has been developed and implemented on a test basis. The details

of these procedures are described in the SOJET Program Implementation

Handbook, Annex B to this report.

In general, the new procedures make use of the optical scan test

recording sheet currently employed with other correspondence courses.

The requirement to submit pretest scores separately has been dropped.

Instead, the SOJET course supervisor now records pre and posttest results on a single optical scan sheet. When the student has passed all

subcourse requirements, this scan sheet is forwarded to IPD for

further processing.

As a result of these new procedures, more supervisors should be

willing to report pretest results, and the time required by IPD personnel to process the test results will be reduced to that required

for the typical correspondence course.

D. REDUCTION IN OJT TIME DEMANDS ON SUPERVISOR

Time Requirements of Course Supervisors

The SOJET program delivery system is based on the assumption that

time spent by a course supervisor to supervise students will be more

than offset by a reduction in the time they otherwise would have spent

conducting on-the-job training. The validity of this assumption is

difficult to demonstrate because ^try few, if any, NCOs maintain a

record of the time they expend conducting OJT.

The average time spent by supervisors on the various activities

required of them as SOJET course supervisors is shown in Table 22.

Time required to enroll one student is about 35 minutes. Thereafter,

it takes about 75 minutes to administer and score the tests for one

subcourse, to review test findings with the student, and to forward

the findings to IPD. According to these figures, a student who

enrolls in a complete operations sergeant course consisting of 19 subcourses would consume a total of 24 hours of course supervisor time,

assuming that the student completed his program of instruction.

Each SOJET subcourse covers a specific task, some of which are

more complicated than others to teach. Assuming that these tasks were

53
•

learned over a six-month period 1n the SOJET program, the time required per week by a course supervisor would be approximately one

hour« This probably is a lesser amount of time than that typically

devoted to OJT. Eighty-one percent of NG/USAR and 62% of active Amy

supervisors reported that SOJET lesson material usually or always

reduced the time required for them to conduct OJT (see discussion of

lesson material, and Table 28).

During this study some'course supervisors reported that they had

or anticipated having difficulty finding the time to carry out their

program responsibilities. The effort which a course supervisor must

put into the program is fairly obvious, while the benefits to be

gained from the program seem less obvious to supervisors. This probably accounts for their occasional reluctance to participate In the

program.

Provision of High Quality Self-Study Material

One goal of the SOJET program was to develop high quality, selfinstructional material. This research did not attempt to directly evaluate that material, but the various surveys did solicit information

and opinions about lesson material quality.

Survey data (third survey) related to the quality and relevance of

subcourse material are summarized in Table 27. Most students reported

that:

o The lesson material was easy to understand.

o The lesson material adequately prepared one to take a

posttest.

o The lesson material usually covered tasks performed on the

job.

o The procedures described in the lesson material usually

were similar to those used on the job.

o The lesson material usually or always helps one perform

better on the job.

Course supervisors provided similar data, as shewn in Table 28.

In general, supervisors reported that:

o The SOJET lessons were relevant to job requirements.

o Student job proficiency noticeably improved after

students studied SOJET lesson material.

54
TABLE 27

SUMMARY OF STUDENT REPLIES TO QUESTIONS ABOUT

SOJET LESSON MATERIAL3

Question Number Component N Response Opt ions

and Question Never Sometimes Usually Always Unk/NA

17. Do you think that AA 20 - 15% 60% 25% -

SOJET material is

easy to under- NG/USAR 64 n 9% 58% 29% 2%

stand?

18. Does the study AA 20 - 15% 40% 30% 15%

of lesson

material adequately prepare NG/USAR 63 - 6% 51% 35% 8*

you to take a

posttest?

19. Does the lesson AA 20 5% 20% 40% 30% 5%

material cover

tasks which you NG/USAR 64 25% 50% 22% 3% -

perform on the

job?

20. Are the proce- AA 20 5% 20% 40% 25% 1«

dures described

in the lesson NG/USAR 64 - 22% 59% 16% 35

material similar

to those you

use on the job?

21. Does study of AA 20 . 30% 20% 40% 1«

SCJET material

help you per- NG/USAR 64 - 9% 50% 39% 2X

form better on

the job?

aThird Survey Data

55

* * '-
TABLE 28

SUMMARY OF SUPERVISOR REPLIES TO QUESTIONS RELATED

TO SUBCOURSE EFFECTIVENESS AND JOB RELEVANCE3

Question Component N Response Options

No Sometimes Usually Always Other

(Yes)

12. In your judgment AA 25 m 8$ 60$ 32$ m

are SOJET lessons

relevant to job NG/USAR 57 - 7$ 54$ 32$ 7%

requirements:

13. In your judgment AA 24 4$ 13$ 54$ 21$ &i

does the job proficiency of per- NG/USAR 48 - 15$ 27$ 50$ 8%

sonnel noticeably

improve as the

result of studying SOJET lesson

material?

14. Does the use of AA 23 13$ 22$ 35$ 26$ «

SOJET lesson

material reduce NG/USAR 49 2$ 16$ 16$ 61$ 4%

the time required to train

personnel OJT?

aThird Survey Data

56
o SOJET lesson material reduced the time required to conduct

on-the-job training.

According to the foregoing replies, both students and supervisors

were of the opinion that SOJET lesson material was relevant and of

high quality. However, active Army students and supervisors tended to

respond less positively than did NG/USAR personnel.

It should be noted that some of the SOJET lesson material did

contain errors and some of it was obsolete (Table 23). Persons interviewed in the field and respondence to all three surveys noted that

this was the case. A number of students and supervisors reported specific instances of error or obsolescence to IPO, which in turn forwarded

this information to the appropriate course developers.

Identification of Job Requirements and Training Needs

A supervisor has the responsibility for identifying job requirements, for identifying those specific requirements that job incumbents

cannot meet, and for devising means to correct incumbent deficiencies.

As an aid to the carrying out of these responsibilities, the potential

SOJET program student and supervisor were provided with a set of four

Student Training Plans (STP), a sample of which is shown in Figure 4.

A separate STP was provided for each of the four basic SOJET courses.

The STP for a particular course listed the subcourses contained

in that course and the lessons or tasks covered by each subcourse. In

effect, each STP provided an inventory of the important tasks that are

performed by persons in a particular operations or intelligence duty

position at the battalion level. The course supervisor was instructed

to review all the STPs and to select the STP most relevant to the

potential student. The supervisor then reviewed the tasks listed in

the STP and indicated in column 1 (see Figure 4) those tasks for which

the potential student was responsible. In column 2 the supervisor

indicated the degree of skill required to perform each of the tasks

checked in column 1. Following this, the supervisor and student

reviewed the list together and indicated in column 3 the present skill

of the potential student. If the present skill level for any particular task was less than that required for the job, the supervisor

checked, in column 4 of the STP, that the student wished to order that

subcourse. The STP was returned to IPD along with other enrollment

forms; the STP served to verify the subcourses enrolled in by the

student.

Table 29 shows the average estimated skill requirements (column

2) and the average estimated current skill (column 3) for 367 students. The data have been subdivided in terms of the skill level of

57
STUDENT NAME:

£ OPERATIONS SERGEANT/ASSISTANT OPERATIONS SERGEANT - TASKS FOR ALL BRANCHES OF THE COMBAT ARKS _

Cd. 1 Col. 2 Col. 3

1

I

£

I

Col. 4

Subcourao

Number

SubcetsWUsson TMo

Duty Petition: OPS SCT/ASST OPS SCT

- Combat Amg

Totti

Reouiftd

mm

r or«

SktH

Roquirtd

/»anvnMMf

Pro*«*

Skill

AMsosr cseiaff/

Stimmt ättinä

ear*Y or *

OS0001

OS0002

0SOO03

OS000*

0S0005

OS0006_

0SOO07

0S0O08

OS0009

OS0010

osoon

MATMTATN P^ftTCATIOHS«— emJ"~l

#1. Us« Chargeout Cord.

12. Determine Publications for Library.

13. Request Publications and Update Kastar List.

#4. Supcrvist Posting of Changas to Publications.

SUPERVISE PREPARATION OF CORRESPONDENCE

D

D

D

G

G

G

G

O

0

G

D

G

B».4~~I

#1. Proof Military Lattara.

#2. Proof Indorsements.

#3. Proof Disposition Forms.

#4. Proof Non-Military Ltttars and Memorandums.

SUPERVISE MATNTTNANrr OF ARMY FUNCTIONAL FTLF.S

D

G

G

G

G

D

G

D

G

G

O

G

MiNA« TsiTMTMft BESnirorPS Bm.Pl

11. Coordinate Training Schadulas.

#2. Prapart Training Aaao Forecaat.

EVALUATE COMMICT GF Tl 4TMTHC. ..,„ _,..

D

G

G

D

G

G

-.»mjn

#1. Prapart for Observation of Training.

#2. Obsarvt Conduct of Training.

#3, Evaluate Training.

PREPARE IINTT KAMMF«? PFPART...,. „ ..

G

G

G

G

G

G

G

G

G

PLAN AND COORDINATE CEREMONIES BmJH

#1. Determine Support Requirements.

#2. Deteralne Sequence of Events.

13. Prepare Letter zi Instruction (L0I).

SUPERVISE PREPARATION OF BRIEFING CHARTS

G

G

G

G

G

G

Q

G

O

»RFPARf IßADTNC CARDS F«s CRftiran/AiR MOVEMENT BmJ"1

PRFPAFE FOR GROUND MOVEMENT OPFRATtnu? _ m/H

#1. Organiie the Steps to be Performed.

12. Develop Vehlclt Loed and Road Movement Plana.

13. Prepare Road Movement Strip Map.

14. Assenble Road Movement Operation« Order

(OPORD).

#5. Assemble Information for FRAG Order.

PREPARE FOR AIR nrPiiWHewT ...

D

G

G

i D

G

J G

G

G

Ü

a

G

G

G

G

G

P»"LJ

Bm.n

11. Determine Aviation Requirements to Support

Movement.

12. Prepere Loading Plans for an Air Movement.

13. Assemble Air Movement Operation« Order

(OPORD).

#A. Prepare Briefing on Movement.

G

O

G

G

a

a

a

a

D

D

G

G

^1—J

Figure 4. Illustrative Page from Student Traininn Plan for Operations

Sergeant/Assistant Operations Sergeant Course

i

58
TABLE 29

AVERAGE RANKS ASSIGNED TO SKILL REQUIRED AND PRESENT SKILL LEVELS

(COLUMNS 2 and 3) OF STUDENT TRAINING PLAN

Student

MOS Skill

Level

Operations

Sergeant

Course

Intelligence

- Sergeant

Course

Assistant/

Specialist

Courses

All

Courses

Combined

Col 2« Col 3D Col 2 Col 3 Col 2 Col 3 Col 2 Col 3

50 3.65 2.19 3.53 2.02 3.67 2.25 3.59 2.16

40 3.48 1.85 3.57 2.12 3.53 1.53 3.50 1.8c

30 3.08 1.65 3.58 1.98 3.58 1.68 3.32 1.77

20 3.61 1.79 4.00 1.00 3.43 1.43 3.51 1.5E

10 1 3.28 1.13 - - - - 3.28 1.13

Average for

all levels 3.53 1.19 3.50 2.06 3.48 1.53 3.52 1.97

aCol 2 rankings = skill required of job holder

bCol 3 rankings = current skill of job incumbent

59

tfmmk* ä i
the student and in terms of the course selected. Those persons processed under the Operations Sergeant course include those who enrolled

only for the basic course plus those who enrolled for the course plus

additional subcourses. The same is true for STPs processed under the

Intelligence Sergeant and the Assistant/Specialist courses. Because

of a low N, the two Assistant/Specialist courses were combined. The

points to note with respect to Table 29 are:

a. For all courses estimated skill requirements (column 2)

were considerably higher than current skill level

(Column 3).

b. Persons at higher grade levels tended to report a need for

higher skill requirements and also tended to report a

higher level of current skill.

The foregoing findings are what would be expected. Persons at

higher grade levels usually have higher skills and their job usually

requires them to possess a higher skill level, in part because they

have to be able to teach a variety of tasks to subordinates.

The discriminate validity of the STP data can be judged in part

in terms of the pattern of responses provided by the students and

supervisors. If large numbers of persons assign the same rank to all

tasks or return the STPs without ranking the tasks, that is an indication that the respondents are not carefully discriminating between

task importance, skill required to peform the task, and the present

skill level of the potential student. Table 30 shown the distribution

of ranks provided in column 2 of the STP (estimated skill required).

In completing this column the range of ranks used may vary from 1 rank

(assign all tasks a rank of 4, for example) to 4 ranks. As shown in

Table 30, 41% of the respondents used only one rank when estimating

skill requirements. Further analysis showed that most of these

persons assigned a rank of 4 to all tasks. An additional 31% of the

respondents used two adjacent ranks when completing column 2 (i.e., 1

and 2, 2 and 3, etc.). Most of these respondents used a rank of 4

with the occasional use of rank 3.

Students and supervisors used a wider range of ranks when

reporting the present skill level of students (Table 30, column 3).

Only 24% of the respondents assigned the same rank to all tasks while

44% used a range of either 3 or 4 ranks.

The foregoing findings suggest that a substantial minority of

supervisors and students do not complete student training plans carefully. In particular, it appears that many räters were not

differentially rating the tasks. Moreover, approximately one-half of

the supervisors either left column 2 of the STP blank or used only a

60
TABLE 30

RANGE OF RANKS USED TO COMPLETE SKILL REQUIRED AND PRESENT SKILL LEVELS

(COLUMNS 2 AND 3) OF STUDENT TRAINING PLAN

(N = 367)

Student

Range of Ranks Used

Complete Col 2

a

to Range of Ranks Used to

Complete Col 3

a

MOS Skill

Blank 1 2 3 4 Blank 1 2 3 4

50 14 71 56 32 10 12 30 46 55 50

40 5 37 23 12 1 4 19 21 21 13

30 7 22 18 8 2 6 20 13 13 5

20 4 17 16 6 1 4 15 9 10 6

10 - 3 1 1 - - 2 2 1 -

Percent 8% 41% 31% 16% 4% 7% 24% 25% 27% 17%

aNumber of persons using this range of ranks

TABLE 31

STUDENT AND SUPERVISOR PREFERENCE FOR MAILING APPROACH

Mailing Options Component Student

Preference

Supervisor

Preference

One-shot mailing AA

NG/USAR

75%

76%

78%

91%

Send lessons 1 to

3 at a time

AA

NG/USAR

25%

22%

19%

9%

Other AA

NG/USAR 2%

3%

61
Single rank when estimating skill requirements. Approximately onethird of the respondents either left column 3 of the STP blank (7%) or

used only one rank when completing that column. On the survey

questionnaires, some students and supervisors did report that the STPs

had been useful; others, however, said that they were not useful.

. A Student Training Plan has many potential uses. It can provide

both the supervisor and the student with an overview of the tasks performed by operations and intelligence personnel. It can provide a

means for systematically identifying duty position requirements, estimating the skill level required for performing tasks acceptably, and

assessing the capability of current job incumbents. The disadvantage

of an STP is that it is time consuming to complete, and thus the

possibility exists that persons will complete it in a perfunctory and

invalid fashion. In view of the uncertain usefulness of the Student

Training Plan, and the need to reduce the time required to enroll in

the program, it was decided to discontinue the requirement to submit a

STP as part of the enrollment application. Instead, the STPs now are

provided strictly as an aid to identifying training requirements.

They are included in the enrollment package, with recommendations on

how to use them, but supervisors are instructed not to return them to

IPD.

In the final version of the SOJET program it is suggested that

the use of STPs be discontinued. By mid-1980 all combat arms school

catalogs should contain a listing of SOJET subcourses. Training requirements can be determined by a review of these listings. Also,

eliminating STPs will reduce the printing and mailing costs associated

with the SOJET program.

Training Records Centrally Maintained

For most OJT programs, training records, if they are kept at all,

are maintained at the unit level. The SOJET program maintains records

at a central location, utilizing specially designed procedures. The

goal of this central record keeping capability is to relieve supervisors of the chore of maintaining OJT records. Also, central records

provide a means for students to receive training credits and promotion

consideration.

Field acceptance of central record keeping procedures was not

directly assessed in this study. However, no direct complaints were

received about the centralized process. As noted already, complaints

related to test scoring and recording procedures were received. The

nature of these complaints and the measures devised to provide solutions to them were discussed under test administrative and scoring

procedures.

62
The standard procedures developed by IPD to handle student/

supervisor queries from the field are designed to provide student

representatives at IPD with easy access to student training records«

These procedures were used to answer questions from SOJET students/

supervisors and, according to IPD personnel, worked satisfactorily

with the SOJET program.

E. LOCAL SEQUENCING OF TRAINING

The "One-Shot" Mailing Approach

In the SOJET program all subcourses requested by a student are

sent to him immediately following enrollment. Similarly, all corresponding test material is sent to the course supervisor. This

procedure is called "one-shot" mailing. The one-shot mailing approach

is designed to allow students to study subcourses in an order

befitting their needs and interests and those of their unit.

During the field visits and the first two study surveys, considerable support was expressed for the one-shot mailing approach.

Findings related to this procedure and obtained during the third survey are contained in Table 31. Both students and course supervisors

expresssed a strong preference for this approach over that used with

traditional correspondence courses. A somewhat stronger preference

was reported by National Guard and Reserve personnel.

During the third survey students and supervisors were asked to

describe how they decided the order in which subcourses were studied.

Replies to this question are shown in Table 32. Thirty-one percent

and 58% of active Arn\y and NG/USAR students, respectively, gave

replies which suggested that they did take advantage of the scheduling

opportunities provided by the one-shot mailing approach. These

replies included: studied on basis of unit need, order determined by

supervisor, order based on student interests, and first studied

material least knowledgeable in. The remaining students said either

that they studied subcourses in numerical sequence or in a chance

order, or that they studied on the basis of factors not relevant to

student or unit needs/interests. As seen in Table 32, supervisors

gave a similar set of replies.

The extent to which students locally sequenced their training

should be reflected in the order in which subcourse tests are submitted to IPD. This order can be determined by an analysis of

student training records. The order is sometimes distorted because

two or more tests may be submitted concurrently, with no indication as

to which one was taken first.

63
TABLE 32

MAJOR FACTORS USED TO DETERMINE ORDER

IN WHICH SUBCOURSES WERE STUDIED

Factors Component

Students

(N»85)

Supervisors

(N=86)

1. Order of numbered sequence

of subcourses

AA

NG/USAR

2a

16

3

16

2. On basis of unit need

and/or training schedule

AA

NG/USAR

4

1 11

11

10

3* On basis of student

needs or interests

AA

NG/USAR 14

3

16

4. Determined by supervisor AA

NG/USAR

1

6

-

5. Studied material in which

least knowledgeable

AA

NG/USAR 4

2

5

6. Studied easiest material

Or shortest lessons first

AA

NG/USAR

5

3

2

2

7. Order selected at random AA

NG/USAR

2

7 4

8. Studied in order received

in mail

AA

NG/USAR

1

3 -

9. Miscellaneous AA

NG/USAR - 4

10.No response AA

NG/USAR

5

1

8

aNumber of persons providing same or similar comments

64
An analysis of t .e order in which the results of SOJET subcourse

tests appeared on training records is shown in Table 33. The students

represented in this Table are those who enrolled for at least a complete

course.

The medians shown in Table 33 were obtained as follows. First,

the subcourse number of the first test submitted was determined.

Secondly, the ordinal position of each of these subcourses on an

appropriate student training plan was determined. For example, a student enrolled in an Intelligence Course might have submitted first a

test on a subcourse 0S0015. This is the 4th listed subcourse on the

STP for intelligence sergeant courses. As a further illustration,

examination of five students enrolled in one of the OPS SGT courses

might shown that they first submitted tests on subcourses OS0001,

0S0003, 0S0001, 0S0005 and 0S0013. Respectively, these are the 1st,

3rd, 1st, 5th and 13th listed subcourses on the OPS SGT STP. In this

illustration, the median listed order of first-submitted tests is 3.

Using similar procedures a median can be established for tests submitted 2nd, 3rd, and so on.

The data in Table 33 indicate that there is a mild relationship

between the order in which a test is submitted and the order in which

it is listed on a student training plan. This relationship is fairly

strong for OPS SGT courses but is fairly weak for INTEL SGT courses.

In general the data support the survey findings--at least 33% of the

students tended to study the subcourses in an order which reflected

their training needs and/or the needs of their unit.

Secondary Advantages and Cost. The one-shot mailing approach

provides a student witn a complete set of course material that he or

others in his unit may use in a variety of ways. During the third

survey, both students and supervisors were asked to indicate the

various ways in which SOJET material was utilized in their unit. The

replies are summarized in Table 34.

Fifty percent or more of all respondents reported that the lesson

material was used for reference purposes and as a aid for preparing

for SQTs. Most supervisors reported that the material also was used

to train persons not enrolled in the program. It seems important to

note that active Army supervisors reported more extensive utilization

of the material than did National Guard and Reserve supervisors; probably this is because many National Guard and Reserve students kept

their material at home. Seventy-eight percent of active Army supervisors said the material was used for reference purposes, 69% reports

its use for training others, and 56% reported its use *« a« uiü Tor

preparing for SQTs. Also, 35% of active A y supervisors indicated

that the material was used to inform supervisors about how to perform

65

Ma4i
CO 00

OO 00

or

LU3

WER

UBCO

8

<!

00 00 h00 X 5

LÜO

h- •«! LU u§

00

OS

00 00

co ocxo

3 *-'

o

LUO

JUK

g

TAB

SUB

OROE

Q

LU

00

LU

X ooc

UUIU ir*s»-• o

oc

LUo

OCo

LO CM CO

o

o

•••

•

1 X»1

1

*-» CM

o r~t rH

1

o

w1

CO 1"^

••••

1 U311

a>

o

CM

en

vo

CO

LO

o

PHo

CO

•••

•

1 XJ J31

co

0> CO c* OS

*!■■

t/>m r** CM LO CM •^ r* • • • •

1 x> ,fi

1

J3

CO r«s CO r^

00

4-> CO

o

o r-*

XA vo•••

•

1 X» JO Xi

m I*-* r».

o vo

K- f-4

0)

w»u iH r».

o VO

3 to

••••

1 JQ -Q .a 1 o VO vo CO LO u

-a3

00

LO to

o CO LO

V *-

•••

•

1

• -O X)

o in vo vo

* ro

u

03

"OL LO CM

oo LO

o co

•

••

• *0• -Q .o

* LO LO vo CSJ o CSJ CO o CM LO LO

<SJ

••••n•••

** ** *> c* CO LO

*

«D

r*. f»

o CO CO LO co

«-4•••

• ft••

•

CM CM LO CSJ CM CO *»«

4-»cc

5

g

55

o

s

00

!35

5

00

§

00

o

£

£

£

g

C7> o

«o

c

*J «o 3

0)1 IA&. a 3 |V— o jQ u <o

4-»

§

v»

ft>

o» TP c »# • >

4-»

«A

e

£ "6

«AO

«A

i-3O

so

I

LOo

CM©

VO

CO

X3c

T3

t«

a;

0)oo

66
TABLE 34

UTILIZATION OF SOJET MATERIAL AT THE UNIT LEVEL*

Ways SOJET Lesson

Material Has Been

Useful

Component Student

Replies

Supervisor

Replies

As reference

material

AA

NG/USAR

82%

83%

80%

61%

As training

material/job

aids for persons

not enrolled in

AA

NG/USAR

29%

64%

72%

52%

program

To inform supervisors how certain tasks should

be performed

AA

NG/USAR

29%

42%

40%

35%

As an aid in preparing for ""

AA

NG/USAK

59%

54%

60%

50%

Other AA

NG/USAR

6%

12%

8%

6%

«Third V

Supervise».

cudent Ns » 17 (AA) and 59 (NG/USAR),

(AA) and 54 (NG/USAR).

67
certain tasks. It appears that the SOJET material has been used in a

variety of beneficiJ ways by persons other than the enrolled

students. In part this is a side benefit of the one-shot mailing

approach.

In terms of costs the one-shot mailing approach is slightly more

costly than that used with traditional correspondence courses. In the

traditional course, three lessons are first sent to the student;

following submission of at least one test, three additional subcourses

are mailed out. The average student in the traditional course

receives three mailings each containing three subcourses. In the

SOJET program all subcourses are mailed to the student in individual

.packages, and the average student receives 12 subcourses. A similar

number of packages containing subcourse test material must be sent to

SOJET course supervisors. Because of these differences mailing costs

are higher for the SOJET program than for traditional programs.

However, in terms of mailing costs it matters little whether the subcourse test packets are mailed individually or in packages of three,

or whether they are sent to supervisors or included with student

lesson material. Mailing costs are related primarily to the weight of

the material being mailed.

The benefits received from the one-shot mailing approach are difficult to quantify at this time. At least 33% of the students appear

to take advantage of the one-shot mailing approach, since they do

sequence their course of study in accordance with their needs and

interests. Also, the majority of students and supervisors reported

that they utilize SOJET material in a variety of ways. In particular,

active Army personnel reported that the material is extensively used

as reference material, as an aid to training others in the unit, and

as an aid to preparing for SQTs. For certain duty positions and

tasks, it would seem important to provide material that can be used

for informal study and reference purposes even though it never is a

part of a correspondence course. The one-shot mailing approach does

serve as a means for providing such material at the unit level.
VI. FEEDBACK TO COURSE DEVELOPERS AND PROGRAM MANAGERS

The successful management of correspondence programs requires the

collection of a variety of data that can be used by program managers

to administer the program and by course developers to make appropriate

revisions to lesson material, course prerequisites, and so on. In

this section some of the reports prepared by IPD will be reviewed and

the need for additional reports designed especially for course developers will be discussed and examples provided.

A. RELEVANCE OF TRAINING TO LOCAL REQUIREMENTS

Currently there are no formal procedures for collecting data

bearing on the relevance of various correspondence courses to local

job requirements. In the SOJET program the Student Training Plan was

designed so that the supervisor would provide information about the

relevance of the various tasks covered in one or more of the SOJET

courses. As already noted, complaints were received about the time

required to complete the Student Training Plan, and an analysis of the

data provided by the STP suggested that much of the data might be

invalid. For these reasons the formal requirement to submit a Student

Training Plan was deleted from SOJET procedures.

Replies obtained during the third survey indicated that most students and supervisors felt that the SOJET material was job relevant

(Table 28). As a means for continuing to collect such information, it

is suggested that at the end of each SOJET subcourse lessun packet a

form be provided that students and course supervisors can use to

report on various aspects of lesson or test material. Such a form is

employed with most correspondence courses. The typical form is

designed by the proponent school, usually is quite general in nature,

and is used by the student only when he has a critical comment to make

about the lesson or test material.

It is suggested that such a form would be more useful if it were

slightly structured. Students would be asked to return the form when

they had a critical comment to make about one or more of the topics

listed on the form. An example of such a form is shown in Figure 5.

Item le on that form asked the student to indicate whether the

material covered in a particular subcourse was relevant to his duty

requirements. If the student returned the form for any reason, he

would answer this and the other questions listed under Item 1 of the

form.

In the SOJET program there needs to be a form that course supervisors can use to comment about course and test material. Figure 6

69
Student Subcourse Critique Form

Subcourse Number

Instructions; Return this form if you have critical comments to make.

T. Record number of subcourse you are commenting about.

b. In Item 1 below check all comments that apply to subcourse.

c. In Itam 2 below explain nature of comment & provide solution

if you have one.

d. Use additional sheets of paper for comments if needed.

e. Fold form and staple twice and mail.

Item 1

a.

c.

d

-

e.

f. "

Check (%/) all comments that apply to listed subcourse

Lesson material incorrect or obsolete.

Lesson material difficult to understand.

Lesson material and test questions do not agree.

Answer(s) to some test questions are incorrect.

__ Lesson material not relevant to job duties.

Other:

Other:

Item 2: Provide more detailed description of comment and suggest a

solution if you have one.

Figure 5. Illustration of Student Subcourse Critique Form

70
k

Supervisor Subcourse Critique Form

Subcourse No.

Instructions: Return this form if you have critical comments to make,

a. Record number of subcourse you are commenting about.

In Item 1 below check all comments that apply to subcourse.

In Item 2 below explain nature of comment & provide solution

if you have one.

Use additional sheets of paper for comments if needed.

Fold form and staple twice and mail.

b.

c.

d.

e.

Item 1

a.

b.

c.

d.

e.

f.

Item 2:

Check (/) all comments that apply to listed subcourse

_ Test scoring guide is difficult to use.

__ Answers provided in test scoring guide are incorrect.

__ Lesson material incorrect/obsolete.

__ Lesson material difficult to understand.

_ Lesson material not relevant to job duties.

Other:

Other:

Provide a more detailed description of comments and suggest a

solution if you have one.

Figure 6. Illustration of Supervisor Subcourse Critique Form

71
contains an illustration of a Supervisor's Subcourse Critique Form for

this purpose. It is suggested that this form be located at the end of

each subcourse pretest scoring guide. As illustrated, the form would

contain a short list of questions to be answered by a check, followed

by a more detailed explanation of the complaint being registered.

An indirect way of determining the job relevance of subcourses is

to determine the rate at which the subcourses are requested. IPD produces a monthly report showing the demand rate for all subcourses for

the past 12 months. By scanning this report it becomes obvious which

subcourses are in high and in low demand. Presumably a high demand

for a subcourse would be an index of a perceived relevance at least of

the subcourse title. Table 36 shows the number of SOJET subcourses

which were issued from May 1978 through April 1979. Subcourses 0S0001

and 0S0027 are examples of high and low demand subcourses,

respectively.

Table 35 also shows the percent of students who submitted a test,

pre or posttest, passing or failing, for each SOJET subcourse. As an

illustration, during the period from May 1978 through April 1979, 340

sets of subcourse OS0001 lesson/test material were issued. During the

period from May 1978 through June 1979 some kind of test on 0S0001

material had been received from 27% of the stuidents to whom the

material had been issued. This information can be used by course

developers to identify those subcourses for which tests are seldom

submitted. They then can examine such subcourses to determine if

their tests are too difficult or if their content 1s obsolete or no

longer job relevant.

Figure 7 contains an illustration of the proposed report for providing subcourse issue information and subcourse test submission

rates to course developers. This form shows for each subcourse the

number of subcourses issued per month, the number of passing subcourse

tests received per month, and the total number of subcourses issued

and tests received during the past 12 months. This report is somewhat

similar to the subcourse utilization report now prepared by IPD.

Hosvever, additional programming would be required to develop the test

submission data.

B. EXTENT TO WHICH TRAINING REACHES TARCET AUDIENCE

Currently IPD has few procedures for determining the degree to

which training reaches the appropriate audience. One report s.':ows the

type of student—defined as Active Army, National Guard, Navy,

Civilian, and so on--who enrolls in various programs. The absence of

more detailed student profile information is in part i\uQ to the lack

of space for capturing additional data on DA Form 145, the form used

to enroll in a correspondence course.

72
g

NUMBER OF

AND PERCENT

TABLE 35

SUBCOURSES, ISSUED THROUGH APRIL 1979

OF TESTS SUBMITTED THROUGH JUNE 1979

Subcourse

No. Issued Tests!

Subcourse

No. Issued Tests

Subcourse

No. Issued Tests

OSÖÖ01 340 274 0S0023 115 15% 010865 45 7*

2 365 23* 24 110 6% 66 44 14%

3 367 22» 25 113 11% 67 33 9%

4 280 27% 26 123 8% 68 31 10%

5 284 19« i 27 57 7% 69 38 5%

6 258 16« 28 60 5% 70 47 19%

7 272 17* 29 84 12% 0N0711

8

9

282

275

18*

16%

30

0A6001

88

62

6%

47%

12

13

Recent 1 ssue

10 286 12% 02 117 19% 15

11 255 11« 03 59 37% 0R1490 69 9%

12 282 20% 04 112 17% 91 46 13%

13 282 15« 05 112 22% 92 46 20%

14 278 18% | 06 111 18% 93 45 16%

15 130 23% 07 53 47% 94 49 16%

16 131 16% 08 50 30% 95 78 12%

17 126 19% 09 53 27% 96 83 12%

18 128 18% 010860 28 0% 97 83 7%

19 150 14% 61 37 0% 98 87 11%

20 141 24% 62 41 0%

21 146 20% 63 41 0%

22 146 18% 64 52 4%

Percent of students submitting any kind of test, pre or posttest, passing

or failing

73
s;

a!

1—O LÜ

1- h- < VO vo ^o o* . >- O *r *d-

_J

ao

<h- .

LU

>•

f

O vo vo

ÜJO> rH r-l oz

h- , 1 8

CL

11

LÜ

00

C5 CSJ CM 5

-J CM CM S

*?z CSJ CM i-H 3^

>- CVJ CM

m

g

OC Uf>

^ to CO

k

00 CM 00

i

CO 00

1 CO

i

UJ

U. 5

o

1

2

i

r>

u

<A «/» IA (A

< 4JS

4-»

IA

Of

4->

(A

0)

+1

(A

0)

Xo

tK h- hs 2

"8

c

s

cnc

s

c

g23 IA3 IA9 Ul

3 IA

S «/> IA «A «A (A (A V) IA </» <0 «A <0 «A «0 IA to

«—• a. *•+ CL»* o. ** Q.

o

o

i

6

o

oo

O

O

>• *E

z

z

UJ »-# CM en Q.

to en C* en en

oc *•

^

igs

PH

or O

(-4 s

p

8. co

•Iu>

V) 3

CO

4-»

a» i

a>3

(A

IA

01

IA

S»3Oa

x>3

00

4-»

4J

IA3

atu3

en
In addition to the DA 145 Form used to enroll students, the

SOJET program used a special form to register course supervisors. The

original version of this later form collected a variety of profile

information about both students and supervisors. Some of the supervisor profile information, obtained from the supervisor registration

form, is shown in Tables 36 and 37; it is of interest but is not crucial to the successful revision of course material. The currently

used version of the supervisor registration form collects very little

profile information.

The amount of student profile information that can be collected

on future SOJET student enrollment forms will be determined primarily

by the extent to which additional data can be r,corded on a revised

version of DA Form 145. In turn, this will depend on the deletion of

certain data from the present form (Figure 8). Analysis of section 4

of DA Form 145 has shov/n that six additional columns of data could be

captured on that form. In addition, data pertinent to "RYE Date-D^y"

and "EYE Date-Day" could be deleted because this information is not

needed. This would free space on the present form for collecting

other types of data, specifically student MOS and skill level, number

of months in current duty position, and total number of months in

related duty positions. For more information about revisions to this

form see Annex B.

To make use of this new information, programs should be developed

so that reports could be prepared to display student profiJe information in response to a variety of questions. These questions should

include: (a) what are the characteristics of students enrolled in a

particular course; (b) what are the characteristics of persons who

pass or fail particular subcourse tests; (c) what type of person

enrolls but never submits a test; and so on. In response to these and

similar questions, a report similar to that shown in Figure 9 would be

produced.

C. COST EFFECTIVENESS OF INDIVIDUAL SUBCOURSES

Currently IPD produces a number of monthly reports that, in one

way or another, can be used to judge the cost effectiveness of subcourses. One report shows sub-course demand rä:e over a 12-month

period. It is used primarily to determine reorder points for subcourses. However, it also can be used to determine those subcourses for

which there is little demand—infonnation that would be of use to

course developers.

As noted previously, a type of report that would be useful to

course developers is one showing the ratio of tests submitted to subcourses ordered. Table 35 (page 73) contains this type of information

75
V», ! 1 ai

CO

eno

00

--« >

c:

UJ

o.

=>

to

UJ

to

a:

rj

tO

o

CO

o§

C.I Crf < o

^

o

ex

u.

<

l-ocooo

c

C£

L. O

l/>

•r—>

<u3

CO

ftCO

too

to

<J

tj

UJ

to

>u

or»o

C\J i uo

rH IT)

rHI

COrH

CNJ CO

NO

CNJVO

<frl>.

co on

coco

ON

coco < *JCO I

Oü>.

tO tf>I

tO 30

n. o>- O CJ-*

UO 0>

co a>

rH**

co to

co en

1-1 CNJ

i-i to

CNJ CO

tocn oo to

CT> CO I 1 CVJ 1 CO i i 1 1 1 1 r-tO 1 1 t 1 i i 1 1 1 1 Hin 1 1 1 rH i i 1-1 CNJ 1 1 CNJ CO

CNJ

*3- CO rH U0 rH*fr

1 rH moo

CO CO

rH

COsf

in

** tO

rH

rHO f^CNJ i-4 CNJ coco

«a- uo

r--. CNJ

1 CNJ «fr

1 COrH

i to coco

rH CNJ

^i CNJ1 CNJ t-l

1I rH

1 CNJ tO

r-4I

11

1111

11 ON CO

f-i

CNJ CO •-,

.I CNJ CO CNJ —1

11 to to

r-i CNJ

Xt U* ro ro M r-«CNJ —• to

O CNJ

1

1* *•

CNJ1 c«. co r-<

1 r-«

I CNJ

1

•-H

—«1

II11

1111 ^- rH

«< to *£ ui

«2o

<

«£ to < 3D

to

-- to «i too

0£<

""l to

2^

o co

W«flO

CJ•

_J V»1

UJ *• li)

1-3o

n:o >•

h- CM

tO »A *-«

toa >-

LJ t. en

t/-» ui :Jo

ft. a.o>

O to t_>—

UJ to

ft. t/1H

tO sU >-

JNW1

UJ 1- v. f>

^r too >-

-•-£ CJ>—-*

4-»uo

to

CO

«/»O

CJ >-

tl1

J- .-,

.30

O >- o

ou

OJ

'S

to

SubCourses

(Y09-Y12)

«DP o

76
TABLE 37

DISTRIBUTION OF ACTIVE ARMY SUPERVISORS BY DUTY POSITION

AND COURSE REGISTERED IN«

Courses

Supervisor

Duty

Position

Operations

Sergeant

Intel11gence Operat1ons

Sergeant Assistant

Specialist

Intelligence

Assistant/

Specialist Totals

Unit

Commander 1 . m MB

OPS Off1-

cer/Asst

OPS

Officer 24 i 25

OPS SGT/

ASST Ops

Sgt 17 1 5 m 23

INTEL

Officer/

Asst

INTEL

Officer 2 16 4 22

INTEL

Sgt/Asst

INTEL

SGT 6 3 9

Other 22 7 1 30

*If a supervisor registered 1n both a senior and junior course» he was

assigned to the senior source«

77
STUDENT ENROLLMENT APPLICATION

Supervised On-The-Job Extension Training (SOJET)

ARMY CORRESPONDENCE COURSE PROGRAM

For UM of thit for», tot SOJET Count Culdt. The proponent Mency it TRADOC.

DATE

DATA RETIRED RY THE PRIVACY ACT

AUTHORITY: 10 USC «Oil ( ) and <0>

PRINCIPAL PURPOSE: To oblate information nttemry by Army »rhoob I« aOmwirter nv4tM »articipalio* to IK» Army corttaponc'inot

eowrat program.

ROUTINE USES:

DISCLOSURE.

UM« by Army aebooli to obtain bade data net«*« to «etermin« ebfibUity for inroImtM. proetai application*.

Uin t(w«Mi rtcotta. u| perform all etbtt eOmmiatratlvt fvnttioo* inherent tu «woenl eOminJetretio«.

Mandatory. Ftlhiro to provide thto ^formation coold rooah to two applicant not httaj able to participate In the program,

SUBMIT ONI COtV. *C£ IS'STtUCTiONS Oh HgVfAsF

1. TKRU:(Unit to which aligned)

SEQ TRANS

NO CODE

SEQ TRANS

NO CODE

TITLE OF APPROVING OFFICIAL

' ' i i i I ' I I I I I I I I I .1

V 49

UNIT ADDRESS LINE 1 WIT DESIGNATION (Hay not bo Ult blank)

I i I i i I ' ' i i I I I I 1 i I I I I

«I «0

UNIT ADDRESS LINE 2 P. 0. SOX OR STREET (Hoy bo loft blank)

«f

I ' I I i I I ' i I I I I I I I I I I

m

IS

UNIT ADDRESS LINE 5 CITY, POST OR APO STATE

if

I i I I I I I IIJ ' I

7$ i4 9$ J» «7

ZIP CODE

M

2. FROM: (Kailint «ddreti to which tubcourttt art to bo ««fit)

LAST NAHE £ FIRST NAHE - HIDDLE INITIAL

SEQ TRANS

NO CODE

m

i i l I i I-I I I

J«

li' '

STUDENT ADDRESS LINE I UNIT DESIGNATION OR P.O. SOX OR STREET (Hoy wot bo loft blank

i i i i i i i i ri i i i i i i i i i

77

COURSE SUPERVISOR'S LAST NAHE - PIR1T MA« - HIMLf lKTTlAL

SEQ TRANS

SO CODE

„ii i i » ii

STUDENT ADDRESS LINE 2 CITY. POST OR APO HATE ZIP CODE

I I I I I I I I I 1 1 I S ü a if 3E

). I REQUEST DtROLUSTT Ml

o. Chock boro If enrolloont it for o ceaplete court« (chock only one court« tttlo end one brooch)

_ Operttteoc SCT ^m Oporotlooo Aaat/Spoc mmm Aroer __ Field Arc U lory

_ tntclliieece SCT lnt«lU|onc« Aaat/Spoc mmmm Infantry mmmm Air Dtfoooo Artillery

b. __ Chock boro if «areUoont to for eelected eobcewrtee only. Litt tbooo tvecewreee in lie» S of lb«

Soporvloof' a Rogictroiioo Pom.

c. __ Chock hor« if yen oro currently or hove btoo prevteooly enrolled ot a fOJCT atudoot.

/ill ft AU *m«J *A WHAMOCO ttoem TSTSiB tTSHTTn /o* K3ggl HI SETT

SCHOOL RECORD

qö?'«i;

STUDENT'S St*

SEO th.\\> T«

t I I I 1 I

COURSE

tUbCOURSE WUHJHR

SUbCOURSE

SEO CnOf

3*095

KUNER1C

CRAM?

Ess c«aup «VHSU

VAR OR |D

T5 ir IJ to to JI »I Jt JI^ J« i«

PENS CO»» Mil S/C ,„ _

*• Cu•' «

CD

MM

rm

»d M

rril

*» •' ED«* "

RYE turr tTL5oTt....

CWft,T "ouu *«*R*um

21 «I «« tt t« <t Po P3 Pf M :« ;j ;« j » 3

ATSC TttT POMt US. JOLT !»»• i;o»jt ».-i»r»nJ*>.11»-* AwtnvriirJl

Fiqure 8. SOJET Program Student Enrollment Application Form

78
is. STUDENT DATA«

STUDENT $$H

I l I i i i i I

1 9

NAT'L CUARD/USAR

RANK (IP APPLICASLE)

21 23

TOTAL MONTH OP

ACTIVE DUTT (AMT)

rm« 27

CARD

RT NO. DAT HI

10

BB

21 12

is

13 14

PRIMARY MOS

ENROLLMENT DATE

TEAR

ACTIVE

DUn RANK

IS II 1?

rm

II 20

TOTAL MONTHS XNi

CDRRINT NOS CURRENT SKILL LEVEL i

21

i i i i

32

i rm

34 3«

rm

31 40

TOTAL MCNTNS IN*

PRESENT PRESENT DOTT

DUTY POSITION (CURRENT WIT)

m

41

m

42 43

TTPE OP

EKROLLMOrr

TOTAL MONTHS IN

OPS/INTEL DUTT

(ALL ASSXCNMENTS)

J_L

JL

4$ 4?

(1) Oaaratiaaa SOT

Praaaai ««jr .(j) *,,,, Oaaratiaaa SCT

(Caack aaa): OJ 1"«»U«« SCT

(4) Aatl. laullljaaca SCT

codst cowm

ERANCN DUTY POSITION

tu

41 41

(S) Oaaratiaaa Aaat/Saac

(4) Iatalll|aaca Aatt/Saac

(?) __ ©that (flMM fttMl)i

. 1 FXALllE 1 MUST COMPLETE THIS INSTRUCTION WITHIN TNI TIME LIMITS ESTABLISHED IT THE SCHOOL COMMANDANT

AND I INTEND TO MEET THE REQUIREMENTS

<PHMMT WOt l**l««Mf St« Umil lGm*l f*r»«toM tfAfßUmmtt mm » Mi

7. COURlt SUPERVISOR:

CRADt. SSN.

SIGNATURE OAR.

4. WIT COMIANDERi

NAME « RANK.

SICNATURE. DATE.

UfcrMiMM »*vt*lMt*j t* aatallaaat awaltfte*et**a,

iM tOJtT Cwru fetfe.

MtotulM ml aaallaattaa« M* IMMI a**lla»l« «r« «aatalaat U

ITtat«. |M««> St». St» m*mm «M»W. rwnitiMiMlH«

9*m CUM CNtW • C. C«»»Mi

•MM* • O. »tMM *>MMNM • V,

fwii» «Mt—M • f.

6HM*0.

• I. CMw • O

(MM C«i«. t»M* aa* •! MM HtfwlN. to« •« — . It HA Hal.

M.

INSTAUCTIO** TO AP#%ttANTS

Campltt* by lafialy Week «tiaüaf anly i« IMI Dial art MM thfJtA If tttMi—:al ****• w ra«j«toa«\ attack tMattU aaaaU.

DO WOT flM ia aHa4H ana» Ar*«*/«to*at «dwell —laia Hath aaarka stay a* «aa4 U aaypvaca 4aU la* «at ia ««Ataaaaatf

•y.t.m*. aatar eaiy a«* catractat f*t haihawk, a*.. |QjT 14 I 3| 1 \l | 1 1*\ i\ \%\% |01

ITEM}. Oa a>t Am Ha« MM attn •! »ataa*«« aflMtol; to« «*» •*•

ta« »**« "taawft" H <a aMTHag «aw. ttoa » atoaa a*t*«aa MM

«atatitw ••»•at«. |4)T|Mt iPlt|VlO|P| iCiOl

hw «M Eta Cat« a*t to ton atoafe if «at »««MM Maa I IMMII «a

A»O MMkti. to»* *• iMfcaw »iimmit; IM »aataato. Knpaai

«VA. «•• *»f*»i*v.

ITEM 9. *»*a • atosfc tot«—» m» M a>a«a »tinO. »*»M t

•WM laa *«« tu- **r ** »•*» «teak M «a*t attaa« IHM 4 aaaaMai

MI AM» mnaaw. «MM » » iMaim ••*»**»«*•. to* timii,

VM«*<« « »A. *•• VMk « MV.

ITtM S. to«M«ai Mil/ »ft* ««MM MM) MM ktMMtl.

HI • HAIAtlt OO

9» HA*AUS OPP/WO

A* • RA'AVt RaJ

«a • MOUt OO

a». tftAft OO

a*. HBV OO

•t. VtAH OPP/WO

at • »out OPP/wo

at • MtAH tat

It - MOOt Hal

it-wocc/aoTco*

IS-POM MIL

u-utctv

14 • ton ctv

it-utAt

>t.

It«

ItUtMC

ta-CADtv I

»••lUl.lfCwMi

Haaa. HA «awaa*ataaai«at Mtaaat aanaaaal M«a«aata

wa*a Mttajwn wa«Oja«*a ami t» «IIMIII iiwrnaiai

MM«N a» MM« IMXI taaxar. IMM ajaaa Maatk tot aaaaaaai. Pf

ttO. MM. MAJ.

HVH.OaM. MtAH taaaaaaai aw aa tataaaat a«a>a tat» «aS

«MI aw nMjmm toM •! MMM wtoMM t«M. tot «a

t tow • Ott*. M IM al «MI «M* M« taw ««M4H

ITO« I. *. I 4 « •!• aalt wtUaM«ry.

Figure 8a. SOJET Program Student Enrollment Application(2nd side)

79
JBI i«

*fc° >» o «NJ 3

a* 4->

a. v> ti

a

>» a»

p

H-V (A

c£

C

U

£

3o

1** o

L. ,_

0) r—

CSJ o

|3 00

•

Z

o 4-* z c

4)

c L. o i-

•#•

3

C -M

O

O'»-

•»- v> *rf*#-

V>3

a»a

3 0)

cxoc

«-H

X

© a. co

•F» w-4

U (A •*; o u c a <o c :> ec! o

(A5

4»K

-J U»

*

•-41

CO

<

ro

c

»

o i-* •#»

•

*J a. «i ! 8S

yj oo

flM VI

«A VI

i. «o

4*—

au

O 00 00 —* to ro

*J- «NJfO COCMfO s

oo

in

«*EO

O©OQOOO

CO >- CO isj CO CD 00

ifl fO -4 Q\ w4 *-* -H

inEO

u.•

< *•

ui

<voNo

IE- L O

lllIlll

o en c* *?*oo

yj ^i 111 ^j |jj ^jj jjj

C

VI co

VI

<co

vi>

41

VI oI8e

4»

LLt N

O»

o

41—

3 4/>

OO * ..

**o

c

4«

"23 «A

4-» U.

v» a»u

'SC

«ft-

&°

VI

41 >»

4->u

wol

O *-« <"0 *-* *-* r-» CM

fihOO^Nnoo X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

=%?

o

**-c

41 o

CO

coou

41

3

oc

VI

41

4-»

a.

4) 35c

V) co

4)

a.a

4)

OS

4»

ft. 1V»scw3

80
for SOJET subcourses. For example, for subcourse 0A6007, 53 subcourses were issued during the previous 12 months and the rate of test

return was 47%. For the SOJET program this 1s a very high rate of

test return. On the other hand, subcourse 0S0030 was Issued to 88

persons and the rate of test return was only 6%.

Of course, the reasons for high demand for, or low rate of test

return for, any particular subcourse would have to be determined by an

analysis of written and Interview comments provided by the students.

In any event, the data 1n Table 35 do suggest that certain subcourses

are cost effective while other are not. It is proposed that the

actual report containing the foregoing information be formatted as

shown in Figure 7 (page 74). The course developer would have to

determine the percent of test returns, but this should be simple to

accomplish.

D. SUBCOURSE QUALITY CONTROL

Two types of feedback information are of special interest to

course developers. One type identifies deficiencies in lesson and

test material, and the other analyzes test results. The typical

correspondence course uses a student feedback form to obtain information about lesson and test material deficiencies. This information

is forwarded to course developers by IPO personnel.

As already noted (Table 20), most complaints about the SOJET

program concerned the mismatch between lesson an*' test material. These

complaints seemed due to editing and collation deficiencies, and

they should be corrected by appropriate School personnel prior to

reprinting of the material. It is suggested that the revised sub«

course material contain a student subcourse critique form as shown in

Figure 5, and that the scoring guide for each subcourse pretest contain a supervisor subcourse critique form as snown in Figure 6.

IPO employs an item analysis report to present information about

passing and failing rates for subcourse test items. Because of differences in test recording and reporting procedures, that report

could not be prepared for SOJET courses. However, as described

earlier, the SOJET test recording procedures have been modified. Now

the standard test recording form used for other correspondence courses

can be utilized to report test results for SOJET subcourses. Hence it

will be possible, in the future, to produce item analysis reports for

SOJET courses. Figure 10 shows how such a report will look. In the

SOJET program, test scoring is on a GO/NO GO basis. Therefore, the

item analysis report will show the number of students who failed test

requirements on the pretest, the first posttest, or the second posttest. Additional information about the revised test recording proce81
4-> 4-»

c u

o a» o

u u o

0» o

oooo

»0

4J

o

CM

o ooo o

in m in m ir>

«Ml CD OOOO

•*- j in in tn in

CO

o oooo

UJJ CM

Oi *-i oooo

CM

oi o oooo

0Ö| o oooo

<| o oooo

CO

o

oo ooo

oo ooo

OO OQQ in in in tn in

ooooo

ooooo

ooooo

ooooo

oo ooo

toooco^-o

r-» in •* o\

ooooo in mm in tn

ooooo

ooooo in in in in in

oo

OCOO VD

ONOOlO

ooooo

mm in in in

tnmmcoo ooooo ^^^^o «-i CM CM in in in in in CMCM CNJ CM in

ooooo ooooo oo ooo

ooooo ooooo oo ooo

OOOOO OOOOO O r**0 —• O

OOOOO OOOOO vo ONO in o

CMi-*CMCM

inifif rsO OOOOO OOOOO

CO CM CM •**•

a

•M v)

U c

0) o

V- a

u IA

o 0)

O Of

u

O t

o

e ac

o

v» a

«I Of

A3 *

C

O

ooooo S58S2 «~*cMO^in w)NcoCftO »«(siM^in

»-«r-«f>-«tt-*»-* HHHf4(\j CM CM CM CM CM

a»

lAL

3 •

_t

o o o

8

3 «•> «/> o ~« CM

O 4-» lA

lA

c *-» c

g

IA

4->

o

c

C «*-»

o •" «A SI

«A 3

CT & <+»

8.2 4»»

o

4-» (A 4J

*A *- "O u c

01 o 01 '0- Q

4-» (A w U

01 • IA JL

t- o •« c 0»

Q.Z aoa

fx* eo

o

•-* c

IA c

<v o

t *>

aw o +>

—Ö * u

or & St v»

C*J c i. oi

irt O f *-»

L «A 3 W

pi— u or «A sr&ea

«AC

WO *- *0 -D

a. o o» c

** V» O

OC • IA o

.CO» OOO» »Sac aw>

Of*

3

82
D

dures is contained in the SOJET Program Implementation Handbook,

Annex B.

E. AMOUNT OF PERSONNEL TURBULENCE

The original version of the SOJET program supervisor's guide contained a form to be used to notify IPD when either the supervisor or

the student was transferred, or for some reason was no longer able to

continue in the course work-or to continue in the role of a supervisor. The information obtained on that form could have been used to

develop data bearing on personnel turbulence. However, the data never

were used in this manner. It is doubtful whether large numbers of

students were even aware of the existence of this form, and many

supervisors probably forgot about it.

It is suggested that a revised version of this form, as shown in

Figure 11, be used to collect information about the status of supervisors and students. In addition to alerting IPD to the need to help

a student obtain a new supervisor, the form can be used by a student

to request withdrawal from further course work, to notify IPD of a

change in address, to notify IPD about changes in supervisor status

and to notify IPD of other changes as described on the form.

To increase the probability that this form will be used, it is

suggested that it be bound in each subcourse lesson packet immediately in front of the student subcourse critique form. Also, it is

suggested that it be bound in each subcourse pretest scoring guide

immediately in front of the supervisor subcourse critique form.

It is suggested that IPD maintain a file of returned critique

forms by course number and forward them on a quarterly basis to

whichever agency has overall responsibility for the SOJET program.

That agency, if it wishes, can use data on the form to determine personnel turbulence rates for students and supervisors.

On a monthly basis IPD prepares a report describing the number

of students currently enrolled in the Am\y correspondence course

program, the number of enrollments during the prior month, the number

of withdrawals, and so on. For the SOJET program it 1s recommended

that a revised version of this report be maintained on a monthly

basis. The suggested format for the report is shown 1n Figure 12. This

report would show for the preceding 12 months the number of students

enrolled by month, the number of subcourse completions per month, the

number of inactive students, the number of terminations per month, and

the number of persons who requested withdrawal from the program per

month. This information should be of use both to program managers and

to course developers, since it does provide a overview of enrollment

and completion trends, and an overall picture of the cost of the

program.

83
NOTIFICATION OF CHANGE OF STATUS OR MAILING ADDRESS

Please return this form to Institute for Professional

Development whenever one of the conditions listed below

occurs.

1. SUPERVISOR STATUS (Check all that apply)

1. Supervisor will no longer be able to perform supervisory

duties*

a. Student needs new supervisor«

b. New supervisor has been located. Supervisor Reglstra«

tlon form 1s, 1s, 1s not attached.

2. Supervisor's mailing address has changed but he still will

function as supervisor. New address 1s Indicated below.

2. STUDENT STATUS (Check all that apply)

1. Student requests to withdraw from further course work.

Reason 1s Indicated below.

2. Student 1s changing unit but wishes to continue course

work. New unit address 1s Indicated below.

3. Student's mailing address has changed but student remains

1n same assignment. New address Indicated below.

4. Other change. Describe below

3. Effective date of change

Month TJiy"

4. Student's

Name

"leäT

SSN

5. Supervisor's

Name SSN

Test Form 9 (Revised) Oct 79

FIGURE 11. SUGGESTED FORM FOR OBTAINING INFORMATION ABOUT STUDENT

AND SUPERVISOR STATUS AND MAILING ADDRESS CHANGES

84
9

r.;

o

a. ä

in

2O§I

ID

iI

00

Lü%

IAC

01

IM§

0) «A^

T> V»C 01

4» U-O6O>

•—9 0» <»•»• •»- IA

•—O 4-» ft. 4-» ** 4-»

8 J85g

1

* Sg

C 3D.C Q. CO uj ur» g Q. 1 M a

CM

IAZ

2

ss

i2

oi^

0)

«Aft.

IA3

>>5

£

10u 2^

•o 0) 8 0» IA

r—

3

408

5

au </>

5S

73e

t/> 40

fe££ 4J

3

C

4-» f- 4-» C ••- c

1

g£

*

*—

3

4-»

£

£S

IA

£

e M- 0) >» O o> ^. e

0» t- 0) •^ 8 §

IA

c

3 •^

•^ IA 40 40

IA3

U t4-1 » ü

4-

5i

>»

o>>

10

#• 4->

«

e

m3

£

3* IA3

4->

IA3

4-» 40

m

1 fe

§

IA

»-*

3 «*- 40

IA

IA

£

•

CM

not

tion

240

£

ss V.

3

er.

5

1

£ u.

2a

IA

*

f

_

i IA *3 40 c ••- u O O

IA f2 t- <t- *J

S. 'S

B

<«- <4- l*. o o O • • • i s &

r- CMm

85
VII. APPLICATION OF SOJET PROCEDURES TO CORRESPONDENCE COURSES

Throughout this research, features of the SOJET delivery system

were evaluated in terms of field acceptance, administrative feasibility and costs. Modifications to the program were made, as necessary

to debug the program(Annex B). Additional modifications have been

proposed for future implementation. A point now has been reached when

others must decide whether the SOJET program is to continue and, if

so, in what form. The observations that follow are offered so that

they may be of assistance to those who must decide about the future of

the SOJET program.

A. TYPE OF COURSE TO WHICH THE SOJET PROCEDURES APPLY

The SOJET program was intended to provide training support to

supervisors conducting duty position OJT. Therefore, SOJET procedures are most suited to courses that have the following

characteristics:

a. The instructional material is task-oriented.

b. The instructional material is duty position-oriented.

c. The lesson material covers tasks performed by specific job

encumbents.

d. The subcourses can be studied in any order.

c. The assessment of student performance can best be

accomplished by the use of performance tests.

f. Student performance can bo assessed using specially

prepared test scoring guides.

There seem to be no advantages to applying SOJET procedures to

courses that teach knowledges as opposed to skills. Traditional

correspondence course procedures probably are most cost effective for

presenting such courses. On the other hand, there are many job positions that require nnstcry of a variety of clerical or administrative

skills. For such positions it may be advantageous to develop SOJETlike courses, especially if special equipment or facilities are not

required in support of the courses.

87
B. COST OF SOJET PROGRAM DELIVERY SYSTEM

It has been estimated that the revised SOJET program delivery

system is approximately 32% more costly than those procedures used to

administer traditional correspondence courses (Table 17). As noted

already, most of the increased cost can be attributed to the need to

print, store, retrieve and mail subcourse test scoring guides. Any

delivery system that employs such guides will incur similar costs.

When the cost of the delivery system for the SOJET program and for

traditional correspondence courses are equated for printing and

mailing costs ($43.68 vs. $40.91} there is only a seven percent cost

difference between the two approaches. This difference is almost

entirely due to the added labor required to store and retrieve separately packaged lesson and test material (SOJET program).

The benefits which might be expected to offset the added SOJET

program costs could not be precisely identified, but they seem to include«

1) The "one-shot" mailing approach results in the distribution

of lesson material that can be used as reference material

by persons not enrolled or registered in the program

(Table 34).

2) Students are able to study subcourse material 1n an order

that meet their training needs and those of their unit

(Table 32).

3) A reduction 1n the time required to conduct on-the-job

training, as reported by course supervisors (Table 28).

C. IMPACT OF PERSONNEL TURBULENCE

Within active An;y units personnel turbulence is fairly high, and

this can have a negative impact on student progress 1n most correspondence courses. The impact is apt to be especially severe for duty

position courses which require Involvement of the duty position supervisor. In addition to being task-oriented and duty position-oriented,

SOJET lesson material was packaged into 16 courses, the four nost

popular of which contained 16 to 19 subcourses (Table 1). It is

highly likely that at least 50* of active Army students enrolled in

either of these four course would be reassigned before they had the

opportunity to complete their study program (Table 14, page 34). Once

reassigned, they would have little incentive for continuity their

study unless their new job still was within an intelligence (S?) or

operations (S3) section.

Personnel turbulence probably contributed to the high VJ start"

rate for active Army students. However, those students who submitted

one or more tests, especially active Army students, completed a high

88
percent of their enrolled-for program of study (Table 11). This last

finding suggests the SOJET-like courses may be appropriate for high

turbulence duty positions. The task-oriented, self-contained nature

3£ of SOJET subcourses allows a student to benefit from study of portions

%S of a course even though he may never have an opportunity to complete

>; an entire course.

lg SOJET program procedures mandate that each student have a

registered course supervisor. What indirect evidence there is

suggests that the loss of a course supervisor does not greatly affect

student progress. The SOJET system was designed so that students

could obtain a new supervisor easily. Forms were provided for

accomplishing this, and during this study 20 formal requests were made

to change course supervisors. In addition to these formal requests, a

student could find a person to act for his original course supervisor

S; without reporting this to IPD. Co» "rsatlons with active Am\y students and supervisors suggested th this did occur on occasion.

*

89
VIII. SUMMARY AND CONCLUSIONS

A. COMPLETION OF SOJET SYSTEM DESIGN

During this study the assigned objectives were accomplished.

Deficiencies In the original SOJET delivery system were Identified and

corrective procedures were developed. Some of these procedures have

been Implemented. In addition, -the design of the SOJET program's

delivery system was completed. This Included: (a) the development of

central management procedures which could be Implemented without using

special data files and management procedures; and (b) the development

of procedures for preparing a variety of feedback reports for use by

course developers. The description of the revised SOJET program dellvery system (Annex B) can, with minimum programming assistance, be

Implemented using management and data processing procedures soon to be

adopted by IPD.

B. EVALUATION OF SOJET DELIVERY SYSTEM

Acceptance of Role of Course Supervisor

In the process of refining and completing the design of the SOJET

program delivery system, the management system objectives Incorporated

Into the program and the approaches taken to achieve these objectives

were evaluated.

It was found that both students and supervisors were willing to

have course supervisors participate In extension training. More

specifically, course supervisors were willing to register by name;

help students enroll and plan their program of study; assume responsibility for test security; and assume responsibility for administrating and scoring tests, reviewing test results with students,

and reporting the results to IPD. However, supervisors expressed concern about the time required to accomplish these actions, and tended

to complain about the complexity of the original SOJET enrollment and

test score reporting procedures. On the basis of these findings,

revised and simplified procedures for enrollment and for test

reporting were developed and Implemented.

The requirement to have a registered course supervisor Increases

program administrative costs. On the other hand, course supervisors

help students Identify training requirements, provide a source of aid

to students and provide feedback to students on test results. Also,

they may, on occasion, motivate students to continue In their study

program. The beneficial ejects of these and ether supervisor actlvl91
ties could not be Identified during this research. Future versions of

the SOJET program can continue to require a course supervisor. As an

alternative, test material can be sent to a student who then passes It

on to a person who agrees to safeguard the material and to administer

and score tests (the Supervised On-the-Job Training—S0JT--approach).

Which option eventually is selected is a matter for the Army to decide.

Impact on OUT Time Requirements

It was anticipated that the SOJET program would provide training

support to the supervisor and that this support would lessen the time

required of the supervisor to conduct on-the-job training. Survey

data from course supervisors indicated that this was the case. Most

course supervisors did report that the lesson material was of good

quality and was job relevant, and that requirements for OJT were

reduced as the result of participating in the SOJET program. However,

certain features of the program did not lead to an expected reduction

in the demands on supervisor time. In particular, the Student

Training Plan, which was intended to help supervisors and students

identify training requirements, was completed in a cursory manner by

many supervisors. Because of this, and in the* Interest of reducing

the time needed to enroll in the program, the requirement to complete

and return a Student Training Plan as part of the enrollment application was discontinued. STPs still are provided in the enrollment

packet because a minority of applicants apparently find them quite

useful. It 1s suggested, however, that STPs no longer be distributed

once the list of SOJET subcourses has been published in all combat

arms school catalogs.

Impact of "One-Shot" Mailing Approach

The "one-shot" mailing approach used with the SOJET program provides students ^nd supervisors with a means for sequencing training to

fit local requirements. Survey data and test submission sequence data

both showed that about one-third of the students planned their

sequence of subcourse study on the basis of their own training need or

unit requirements. The remaining students studied subcourses in the

order in which they were numbered, or 1n accordance with other considerations irrelevant to unit needs. The "one-shot" mailing approach

also provided units with a complete set of task-oriented lesson

material. Many students and supervisors reported that their lesson

material was used by others as reference aids or for preparing for

Skill Qualification Tests. Because of mailing costs the "one-shot"

mailing approach is somewhat more expensive than that followed with

other correspondence courses. However, its continued use is

suggested.

92
Adherence to Testing Procedures

Test Scoring Guides. One objective of the SOJET program was to

provide students immediate feedback about test results. To make this

possible» test scoring guides usable by non-subject matter experts

were developed. Based on survey responses it appeared that most supervisors had little difficulty using these guides. However, a number

of supervisors noted that the scoring guides contained printing and

collation errors. This sometimes made it difficult to match the guide

material with test questions. All these problems should be corrected

before the next printing of SOJET course test and scoring guide

material.

Student Feedback. The course supervisors were requested to

score the tests and review test results with students as soon as

possible following test administration. Kost students and supervisors

reported that this requirement was followed» especially within active

Aruy units. This is more difficult to accomplish without time lags 1n

National Guard or Reserve units because such units meet only periodically.

Test Reporting Requirements. Most features of the SOJET program

dealing with the collection and reporting of feedback data for course

developers were designed during the conduct of this study. Therefore,

the usefulness of these data were not evaluated« One feature that was

assessed was the requirement to report failing as well as passing preand posttest scores to IPO. Analysis of training records suggested

that supervisors wen* reluctant to report failing scores, especially

pretest failures. In view of tins finding, and the reported need to

simplify the test reporting process, the requirement to report pretest

scores as soon us the pretests were taken was discontinued. Now,

revised SOJCT procedures for recording and reporting test scores

require that both pre- ami posttest results be recorded after the student has successfully passed all subcoursc test requirements. Also,

test results now are reported on the standard test reporting form used

with other correspondence courses instead of on a special for»

designed for the SOJCT program. These revised requirements and procedures are much less ttiwe consuming for both course supervisors and IPO

personnel.

C. STUOCHT PROGRESS AflO PERSONNEL TURBULENCE

Analysis of student training records revealed that 70fc and 51% of

AA and NG/USAR students, respectively, had never submitted a single

subcourse test (Table 8). Overall, the course completion rate for all

students who have been enrolled for at least four months (classes 1

through 6) was approximately 11 percent (calculated fron Table 6

93
data). On the other hand» analysis of student progress in terms of

the number of subcourses completed (Table 10) revealed that of those

students who submitted one or more tests, active Amy students had

completed 12 subcourses (60X of their training program) by the end of

six months following enrollment. The rate of progress tended to be

considerably less for NC/USAR students.

Data on personnel turbulence (Table 14) suggested that active

Amy personnel can anticipate reassignment after six months. For many

students their net/ duties will not be related to operations or

intelligence, and therefore they probably will discontinue their SOJET

program.

The foregoing data suggest that personnel turbulence had a negative ii^act on student progress, especially If student progress Is

defined in terms of course completion percentages. When student

progress is defined in terms of subcourse completion percentages it

appears that personnel turbulence had less of a negative impact.

Probably this was because the task-oriented, self-contained nature of

each SOJCT subcourse made It possible to benefit from the stuefy of one

or a few subcourses even though dt\ entire course was not completed.

I). COST OF SOJET PROGRAM

A cost comparison of SOJCT and traditional procedures for the

delivery and central i*ana<jei«;ent of correspondence courses and/or

extension training indicated that SOJCT procedures (revised) wsre 321

more costly (Table 17). It was determined that a substantial portion

of the increased cost of the SOJCT program could be attributed to the

need for performance touts and test scoring guides, materiel required

b> ary j.i-ocj.-am that depend; on field jrersonn»?! to administer aid score

performance tests. When these costs drQ equalized for SOJCT ami for

traditional correspondence programs it was found that SOJCT procedures

were only 7% wore costly than those e-vloyed to delivery traditional

correspondence programs (page 88). It was concluded that the extra

costs associated uith the SOJCT procedures ne^H not be a deterrent to

applying them to selected courses, especially thos<* that arc designed

to be tasV- and duty posit ion-oriented.

C. CKlTIHUMCi: CF S0JI1 PRüGftAft

Any decision to discontinue the SOJCT JTO^4 * utst tAe into

account two important factors. First, there is z nj?cd for instructional material for battalion S2 and S3 NCO duty positions. These

positions were selected as the vehicle for the SOJCT progra* in on'er

to fill M instructional void. The need for instruction*! tutorial

94
for these duty positions was substantiated during interviews with

active Army personnel. Secondly, the extra cost of using SOJET procedures (revised) to manage the 16 SOJET program courses was calculated

as $6,288 ($10.48x600 ) for an annual enrollment of 600 students (see

Table 17). Based on these two findings, it was concluded that the

SOJET courses should continue to be offered by the ACCP and should

continue to be administered using SOJET procedures for at least

through calendar year 1980. This would provide time to integrate

SOJET procedures (final) with those under adoption as part of IPO's

IMIS system. Also, it would provide time to collect additional

enrollment and training data (test submission data) which could be

used to evaluate the long-term acceptance and effectiveness of the

program. If this last suggestion is adopted, It 1$ proposed that, for

comparative purposes, similar types of enrollment and student progress

data be collected for two or three correspondence courses which use

more traditional delivery procedures. Also, data should be collected

on courses administered using IPO's Supervised On-the-Job Training

procedures.

95
IAPPENDIX A

»
TABLE A-l

DISTRIBUTION OF ENROLLMENTS IN TERMS OF CURRENT DUTY POSITION*

(Senior Sergeant Courses)

Current Duty Position

Course Component

OPS SGT/

ASST OPS

SGT

INTEL SGT/

ASST INTEL

SGT

OPS

ASST/

SPEC

INTEL

ASST/

SPEC

Non OPS/

INTEL

POSITIONS

Total

OPS SGT

AR (YOl)c AA

NG/USAR

21

49

1

4 2

- 6

23

28

78

INF (Y02) AA

NG/USAR

18

59

1

3 1

- 5

13

24

76

FA.(Y03) AA

NG/USAR

11

36 4

- - 22

10

33

50

ADA (Y04) AA

NG/USAR

4

4

-

1

- 3 7

5

INTEL SGT

AR (Y05) AA

NG/USAR

1

1

9

25 5

• 2

2

13

33

INF (Y06) AA

NG/USAR 2

5

35

•

i

4

3

9

41

FA (Y04) AA

NG/USAR 1

8

21 I

• 3

1

11

24

ADA (Y08) | AA

NG/USAR

- 2

1

m

m

• - 2

1

Selected

SubCourses1*

(Y01-Y08)

AA

NG/USAR

12

19

7

11

•

• 2

2

4

21

36

Total AA

NG/USAR

67

171

33

104

0

10

1

3

47

56

148

344

Enrollment as of 1 September 1979

^Persons who enrolled in less than a full course

c

Ar Armor; INF Infantry; FA Field Artillery; ADA Air Defense Artillery

99
TABLE Ä-l (Cont'd)

DISTRIBUTION OF ENROLLMENTS IN TERMS OF CURRENT DUTY POSITION*

(Junior Sergeant/Specialist Courses)

Current Duty Position

Course Component

OPS SGT/

ASST OPS

SGT

INTEL SGT/

ASST INTEL

SGT

OPS

ASST/

SPEC

INTEL

ASST/

SPEC

Non OPS/

INTEL

POSITIONS

Total

OPS SGT

AR (Y09)C AA

NG/USAR 1

- 7

13

3

8

3

7

13

29

INF (YIO) AA

NG/USAR 1

- 8

14 .

- 8

IE

FA (Yll) AA

NG/USAR

1

mm

- 3

3

-

2

4

5

ADA (Y12) AA

NG/USAR

- i 5

5

-

a»

6

5

INTEL SGT

AR (Y13) AA

NG/USAR

-

z

1

3

7

14

5 13

19

INF (Y14) AA

NG/USAR -

-

p

3 1 4

FA (Y15) AA

NG/USAR - 1 • 1

•

1 3

ADA (Y16) AA

NG/USAR

-

1

-

1 1

*

3

Selected

SubCourses»

(Y01-Y08)

AA

NG/USAR 1

1 2

1

1

7

2 6

9

Total AA

NG/USAR

1

3

2

4

26

39

11

34

10

12

50

92

^Enrollment as of 1 September 1979

"Persons who enrolled in less than a full course

cAr « Armor; INF « Infantry; FA- Field Artillery; ADA - Air Defense Artillery

100
TABLE A-2

DISTRIBUTION OF ENROLLMENTS IN TERMS OF CURRENT SKILL LEVEL3

(Senior Sergeant Courses)

Current Skill Level

Course Component 1 2 3 4 5 Unknown Total

OPS SGT

<•

AR (YOl) AA

NG/USAR

2

3

2

2

5 15 3

15 32 21

1

5

28

78

INF (Y02) AA

NG/USAR -

7

5

4 12 1

12 21 36 2

24

76

FA (Y03) AA

NG/USAR

- 1 9 19 3

6 15 29

1 33

50

ADA (Y04) AA

NG/USAR

1 1 1 4

1 2 2

- 7

5

INTEL SGT

AR (Y05) AA

NG/USAR

1 - 7 2

9 12 10 2

13

33

INF (Y06) AA

NG/USAR

- 1

3

5 1 2

11 13 14

- 9

41

FA (Y07) AA

NG/USAR •

2 3 6

3 7 14

m 11

24

ADA (Y08) AA

NG/USAR

- 1

1

1 2

1

Selected

SubCoursesb

(Y01-Y08)

AA

NG/USAR

1

1

5

1

4 3 5

2 11 20

3

1

21

36

Total AA

NG/USAR

S

i 4

20

11

38 62 14

59 113 147

9

10

148

344

Persons who enrolled in less than a full course

101
TABLE A-2 (Cont'd)

DISTRIBUTION OF ENROLLMENTS IN TERMS OF CURRENT SKILL LEVEL

(Junior Sergeant/Specialist Courses)

Current Skill Level

Course Component 1 . 2 3 4 5 Unknown Total

OPS SGT

AR (Y09) AA

NG/USAR

9

12

3

7

1

4 1 5

13

29

INF (YIO) AA

NG/USAR

7

8

1

1 5 1 -

8

15

FA (Yll) AA

NG/USAR

3

3

1

1 1 • •

4

5

ADA (Y12) AA

NG/USAR

6

3 1 1

1 6

5

INTEL SGT

AR (Y13) AA

NG/USAR

10

3

1

8

2

7 1 «

13

19

INF (Y14) AA

NG/USAR 1 2 1

• a»

. 4

FA (Y15) AA

NG/USAR 1 2

-

•

-

3

ADA (Y16) AA

NG/USAR 1 2

- - -

3

Selected

SubCourses5

(Y09-Y16)

AA

NG/USAR

5

2

1

4 1 2

- 6

9

Total AA

NG/USAR

40

31

7

30

3

20 5 1 s

50

92

102
TABLE A-3

DISTRIBUTION OF ENROLLMENTS IN TERMS OF CURRENT MOS*

(Senior Sergeant Courses)

f

Combat MOS Non-Combat MOS

Course Component AR INF FA ADA ENG INTEL ORD OTHER UNK Total

OPS S6T

AR (YOl) AA

NG/USAR

8

45

10 6

6 6

2 1

4 4 4 2

1

5

28

78

INF (Y02) AA

NG/USAR

1

1

21 -

56 - •

1

3 3 5

1

8 •

24

76

FA.(Y03) AA

NG/USAR -

- 30

9 36

1 1 -

- 5

1 33

50

ADA (Y04) AA

NG/USAR

- - - 7

5

- - - - - 7

5

INTEL SGT

AR (Y05) AA

NG/USAR

3

18

3 2

3 2

2 2

m

1

8

-

1

- 13

33

INF (Y06) AA

NG/USAR 20

5 1

19 -

1 2

1 1 m m

9

41

FA (Y07) AA

NG USAR

- 7 1

- 20

1 2

3 1 •

- 11

24

ADA (Y08) AA

NG/USAR

- - - 1

1

m

m

1 - - - 2

1

Selected

SubCourses8

AA

NG/USAR 1

12 2

4 19

6

2 1 1 2

1

6

- 21

36

(Y01-Y08)

Total

(Y01-Y08)

AA

NG/USAR

12

85

58 42

97 83

21

8

9

8

2

20 15

2

22 «

148

344

^Persons who enrolled 1n less than a full course

b

Eng Engineering; Intel Intelligence; Ord Ordnance

103
TABLE A-3 (Cont'd)

DISTRIBUTION OF ENROLLMENTS IN TERMS OF CURRENT MOS

(Junior Sergeant/Specialist Courses)

Combat MOS Non-Combat MOS

Course Component AR INF FA ADA ENG INTEL ORD OTHER UNK Total

OPS SGT

AR (Y09) AA

NG/USAR 11

2

6 3

5

1 1

3

2

2 1

1 4

13

29

INF (Y10) AA

NG/USAR

- 8

14

- - - -

1

-

.

8

15

FA (Yll) AA

NG/USAR

- - 2

3

1

- - 1

1

1 -

4

5

ADA (Y12) AA

NG/USAR 3 m

- 4

2

- 1 m 1 - 6

5

INTEL ASST

AR (Y13) AA

NG/USAR 2 2

2

3

4 - 6

11

"

1

1 13

19

INF (Y14) AA

NG/USAR

-

1

<m

•> •

-

3 - - - 4

FA (Y15) AA

NG USAR m • 1 m • 1

m

1 .

0

3

ADA (Y16) AA

NG/USAR

\ -

1

-

1

- m

1

m m - 0

3

Selected

SubCourses

AA

NG/USAR 5

1

3

1 2

-

1 - 1

1 -

6

9

(Y09-Y16)

Total

(Y09-Y16)

AA

NG/USAR

0

21

11

27

5

10

16

4 1

11

18

2

2

4

S

1 50

92

104
oo

r-Zs

00 I/} s rUl

>- §

00

r-Z

3 LJ

OS >•

Of h- 3 © 2 LÜ

»-«

O

00 Ul £z -J s Ul

h-

&

z

r*

i

g

e

00

^rot—t

*—« r-<

00 as

r- Ul z %

-J -J 4 5! 2

»—•

r00

c ! 3

CON

1N

N CM «vo COS

IT) 00 in

N CO

1 cn vo

« o\

N in

11 CM

c

f-i -* 1 c I =3+ 1 00 in « coco CO vo 1 1 r-i<*

1 CO

1O

1 CO

1 CM

iN CMrH

en | «a*

rH CSJ CM CM rH ^-i CM f^* CM 1 00 1 « 1 CO « ^H 1 VO i h* 1 CM 1 CM 1 1 1 CO 1 * ^ N 1 «4> «CO 1 rH <r^ N ^

CO

vo en a»

« CO

1 vo ON COCO

m co1^ rH CO «o> CMrH

CO

«« rH CO ^-* r^ wt r*

^ 9^ rH rH

V)1 1-« rH rH 1 JCUO

-P CM c

c8 oo oo a* CO CO CON CO 1** OH CON cooN CM *t

1 <^ o>N GO j .1- 1 *-H r4 r^ f* 1-* w^

C*

I-

•*

h00

>,^-l rv ro

N a> en vo VO

N en Ol CO

^ m*r

1 rH

m o\ *N VO CM co

o

+»

3CO

O-H

«Hi^ w* rH CMrH «-HCM CM w+ rH rH w^ **

CM

rH en esj CO vo OH S3 VOC0 CO*- ^•4 w4 vo r** CM in CM« 88 (VIN 1 CM CM t-t w-t « CM rH CM rHCM CO CM * 1-» CM CM CM CM CMrH

vo1

1^

mvn

O

CO CM

CM rH

CSJ rH 83 =28 38

m vo

CO CM S3 C0Q\

CM »8 HO« CM

c* en

CO CM as

z g§ CM a»o>o

CM

men

CO ©S

vn o>

COO* co5 coin 28 co a\

•HCM

N in

CM* $3S §8

4JCo 1 u

si si si si si si

4 s| s| si si «1

$

o

s

g

g

gi

%

S

g8

S

*«»* •N. **s •V. v* *s»

4-» 00 v» 4-» OO VI 4-» OO VI O0 VI 4-» OO VI 4J OO VI iu

°-5

C o.

ec A-

£ +» a.

e

c o.

cC

ac

oo•r»

r- 4-»

oo

r- 4-»

oo

r- 4->

C

oo

#- 4J

E

oo

r- +J

oo

#- 4-»

4-»c

CV

l

3

^ ••—

3 pa •wm3 r* **

U r» *#•

3

p> <pB

3

p *r"

O <0 «A

O «0 VI

U <0 VI3 <0 VI

G <o VI

u

« VI

Ü ••« c o c o e o o e o c o c o 1 •- c o C CL c o CCL c o CO. o CO. c o CO. c o CO. a C «r. 1 * • •» > • Ul • •»- • Ul « T- • Ul fm • Ul • *r- • Ul • •»" * UJ c 3 VI VI VI h- Vi v» VI r- VI VI Vlr- V) VI Vlr- VI VI VlrXO.

Vlr- 01

O

*£ 55 ££ OZ 12.

f= ££ 55 ££ *5 25 VIt

rat

r- N» O0 <#-"•»

1-

a rOO

Ö^

VO

VI t^t

VO

£

2

a «J S3

. •J

• tJ,L

>

f1

Ulo uio 61

3 «H

g

3 oo r- OO Ul r- Ul— 00 30 *JO

3

o «L

Z a. a. ZCW

0 o>-

o >-z

U

O

M

O 00 r-. tO 00 U*-* r-*-

«