TABLE 17, Average Ratings of Organizational Unit Efficiency, Morale,

and Employee Satisfaction Before and After Attending MIS

Average Average Percent Reporting
Rating’ Rating
Factor Group Before After No
MIs MTS. Higher Change Lower
Efficiency Spvr-Grad 3.5 32 69 &
Superior 3.5 25 70 5
Employees 3.3 25 60 15
Spvr-No MIS 3.8 -- -- --
Superior 3.6 -- -- --
Employees 3.5 -- -- --
Mngr-Grad 3.6 32 62 6
Superior 3.5 30 59 ll
Employees 3.5 26 55 19
Mngr-No MS. 3.8 “- -- --
Superior 3.7 -- -- --
Employees 3.6 -- -- --
Employee Spvr-Grad 3.2 30 58 12
Satisfaction Superior 3,2 28 64 8
Employees 3.0 25 50 25
Spvr-No MIS 3.4 -- -- --
Superior 3.5 “+ -- --
Employees 3.0 ve -- -- --
Mngr-Grad 3.3 3.5 26 65 9
Superior 3.3 3.5 30 57 13
Employees 3.4 3.4 23 58 19
Mngr-No MIS 3.4 a -- -- =
Superior 3.5 --- - -- --
Emp Loyees 3.0 --- -- - --
Morale Spvr-Grad 3.1 3.4 30 58 12
Superior 3.3 3.5 26 66 8
Employees 2.9 2.8 19 56 25
Spvr-No MIS 3.3 ° - -- --
Superior 3.4 - -- --
Employees 2.8 --- -- -- --
Mngr-Grad 3.3 30 59 il
Superior 3.3. 29 S7 14
Emp loyees 3.4 24 57 19
Mngr-No MIS 3.4 -- -- --
Superior 3.4 -- -- --
Employees 3.1 -- -- -- --

a
Rating scale values: 1 = minimal, 2 = acceptable, 3 = good, 4 = very yood

16


Not at all
Rarely

Find the paperwork associated with
recognition and awards confusing.

Occasionally
Almost always

Usually

Understand appropriate justifica-
tion for awards.

d
Specific and measurable objectives
are established for my unit.

Assess myself and unit by comparing
results with objectives.

Work problems are systematically
defined.

Information to determine problem
solutions is systematically
collected.

Several alternative courses of
action are considered before
choosing a means for solving
work problems.

Budget estimates are prepared in
an accurate and timely fashion.

Gay net cin i

The last six items were presented to managers only.

39



Not at all

Employee Motivations

Slightly

Somewhat

Moderately

Quite

Very

Extremely

Managerial Style

Communication

Management of Conflict

Dynamics of Change

Management by Objectives

Understanding Principles

of MBO

Implementation of MBO

Managerial Decision Making

Problem Identification/
Analysis

The Decision-Making
Process

Resource Planning

Budget Process

Logistics Planning

Position Management
and Staffing

85



views; specifically, that the MTS program was
to some degree useful, important, and valuable.
They liked much of the course content, the in-
structors, the campus atmosphere, and group
interactions. They did not care for some parts
of the course content, the testing and evaluation
included in the school, and the geographical lo-
cation of the school (Lawton). They considered
MTS of some importance as a developmental
factor for managers and supervisors but some-
what less important than personal or other or-
ganizational factors, such as their immediate
supervisors and agency policy. The content area
judged most useful by the graduates was Human
Relations/Human Factors; as a group, graduates
did not feel that any of the course content areas
were less than “moderately useful.” After com-
pleting the MTS course, graduates felt MTS had
helped them to be better prepared for their su-
pervisory/managerial positions. They desired
more help in the areas of labor-management re-
lations and performance evaluation. Finally, the
general tenor of the responses suggests that the
graduates can best be described as supporting the
MTS concept while at the same time indicating
areas in which the program could be improved.

What does all this mean? Certainly it is clear
that MTS graduates tend to believe that MTS
is a good and useful program, particularly for
new supervisors and managers. Recognizing
that the school is not perfect, they believe the
programs should be improved but not abolished.

At this point, it should be emphasized that
this part of the survey is not, nor was it designed
to be, a measure of supervisory/managerial per-
formance. The focus has been entirely on the
evaluative judgments of those who have partici-
pated in the program; the concern up to this
point has been with the essentially internal cri-
teria of determining the degree to which grad-
uates of the Supervisory and Managerial Courses
felt the training experience at MTS was useful
to them on the job.

III. Supervisory and Managerial Perform-
ance as a Function of MTS Training.

It is now appropriate to consider to what ex-
tent the MTS program, given that it was seen
as valuable by its graduates, resulted in im-
proved supervisory/managerial performance. To
accomplish this assessment, supervisory and man-

agerial graduates of MTS, their employees, and
their immediate superiors were asked to indicate
the extent to which these graduates engaged in
desirable supervisory/managerial behavior prior
to, and following, MTS training. The emphasis
was on specific and explicitly defined types of
behavior desirable in supervisors and managers
on the job. As a control to further clarify the
contribution of MTS in furthering appropriate
supervisory/managerial behavior, a similar as-
sessment was made of the performance of super-
visors and managers who had not yet attended
MTS.

A. Procedure. The portion of the survey con-
cerned with job performance was composed of
questionnaires (see Appendix, ITI.C. and VI.C.)
designed to query the respondents about the spe-
cific types of behavior expected of supervisors
and managers. The types of behavior sampled
were taken from the topics covered within the
MTS courses and were supplemented by consul-
tations with management personnel concerned
with supervisory/managerial behavior. The
items for employees and superiors of the super-
visors and managers surveyed are parallel to
those in the supervisory and managerial ques-
tionnaires, with wording changes as appropriate
for the particular group in question.

The administration of the questionnaire has
already been described (see paragraph II.A.).
However, in addition to graduates of the Super-
visory and Managerial Courses, several addi-
tional groups of FAA personnel were asked to
participate in this portion of the survey. These
various groups and the number of participants
in each are presented in Table 11.

It should also be noted that for the items that
concerned before- and after-training assessments,
the orders were counterbalanced so that one-
half of the respondents answered in terms of
post-course conditions first and vice versa.

The employees asked to participate in the sur-
vey were chosen at random from the persons

TABLE 11, MS Survey Groupe and Number of Participants

Group MES Graduates No MIS Training

Supervisors 616 226
Superiors to Supervisors 403 166
Employees of Supervisors 605 210

Managers 128 7
Superfors to Managers 106 59
Employees of Managers ul n



Instructions
Supervisor /Manager

There are three sets of questionnaires in this packet. The first
envelope (longest code number) contains the questionnaires for you
to complete. As noted by your survey team member, the other two
envelopes are for your immediate superior and for one of your
subordinates.

Do not put your name on your set of questionnaires or on any of
the envelopes.

The survey, both long (two hours) and short (30-40 minutes) forms,
consists primarily of rating scales and several open-ended questions.
Work quickly throughout the survey--use your first impression to
answer questions. Do not spend a long time on any one question.
Answer according to your first impression and proceed to the next
question.

When answering rating scale questions, place an X over the mark that
corresponds to your rating. Be sure to read each scale carefully
before rating the items, If you make an error in marking your answer,
cross it out and mark your desired choice.

For the open-ended questions, please be brief and specific. If you
cannot think of as many answers as are requested, move on to the
other items and come back later to finish if you have time.

When you are finished, place your set of questions in the envelope,
seal it and return it to the survey team member.

If you have any questions, please ask any survey team member for
information or assistance.

Remember work quickly, but carefully.

Thank you for participating.

20


VII

Are there any other comments that you would like to add about the MIS program?

40



AN EVALUATION OF THE EFFECTIVENESS OF THE
FAA MANAGEMENT TRAINING SCHOOL

I. Introduction.

At the request of the Office of the Secretary
of Transportation, an assessment of the effective-
ness of the FAA Management Training School
(MTS) was undertaken during 1973. This re-
port is concerned with the two major aspects of
that assessment: First, an evaluation of MTS by
graduates of the program; and second, an assess-
ment of the impact of the school on the behavior
of its graduates, as seen by the graduates them-
selves, their employees, and their immediate
supervisors. The report is organized around
these two major facets of the evaluation.

A. Certain Issues in Management Training
Fesearch. A look at the research conducted on
the effectiveness of management training pro-
grams suggests a number of problems that must
be confronted in conducting such studies. In the
most extensive published consideration of mana-
gerial training, Campbell, Dunnette, Lawler, and
Weick* indicate that the two major issues in re-
search on the effectiveness of such training are
criteria for change and controls for training
effects.

According to Martin,’ criteria employed in
studies of training effectiveness fall into two
categories. One, designated “internal criteria,”
is concerned with demonstrating some change
relevant to the training itself, such as attitude
change, evaluations of the “success” of the train-
ing, and improvement in before/after test scores.
The second category concerns changes occurring
in job behavior and is designated “external cri-
teria.” Such criteria include objective measures
of unit performance, ratings of job performance
by superiors or subordinates, and employee turn-
over. Most studies reported by the Campbell
et al. review employed internal criteria; only
about 30 percent of the studies measured effec-
tiveness through the preferable use of external
criteria.

The second issue of importance is control for
training effects. Controls are generally of two

types, either comparison of trained and untrained
groups or before- and after-training comparisons
within a trained group. Campbell e¢ al. point
out that without such control the link between
the training activity and the criteria, be these
internal or external, must remain extremely
ambiguous.

B. Approach of the Present Evaluation. The
present study generally is organized around the
concepts of internal and external criteria. As
noted by Campbell e¢ al.,1 both types of criteria
are necessary in determining the effectiveness of
management training: The first is aimed at
evaluating what is learned; the second, at what
is transferred to the work setting. The first
major section of this report, which is an evalua-
tion of the MTS training by graduates of the
program, is based primarily on internal criteria.
The second major section of this study, which is
an assessment of the effects of the MTS training
on the performance of the graduates as judged
by the graduates themselves, their immediate
superiors, and their employees, is primarily con-
cerned with external criteria. In addition, this
portion of the study addresses the issue of con-
trol for training effects by considering the super-
visory/managerial performance of those FAA
supervisors and managers who had not yet at-
tended MTS.

II. An Evaluation of the MTS Program by
Supervisory and Managerial Course
Graduates.

The focus of this section is on the evaluation
of the MTS program by those who have grad-
uated from it. Considered herein will be their
attitudes about their MTS experience, the judged
usefulness of the course content, and appraisals
of the overall utility of the program.

A. Procedure. A structured questionnaire was
used to systematically gather data about the
effectiveness of MTS. The questionnaire was
designed (a) to thoroughly cover the scope of



c
v

(Managers)

A. Please rate the following factors as to how important they have actually been in your
development as a supervisor/manager.

Somewhat
Slightly
Neither

Slightly
Somewhat

>
fa
s Unimportant

Quite

Important

Very
Qui

FAA policies

|
|
|
|
|
|

Personal
efforts

Organizational
climate ee LLL
Personal

qualifications

MIS

Inmediate
supervisors

Other (specify)

Please rank-order the following general subject matter areas which were taught at
MIS in terms of their usefulness to you in your present managerial position.
Place the number "1" by the most useful, the number "2" by the next most useful
and so on (the areas are listed in alphabetical order).

Creating a Productive Work Environment (Career Planning, Employee Development,
LMR, Counseling, Training)

FAA Management Environment (Role of FAA Manager, FAA Mission, Organization,
Line/Staff)

Human Factors of Management (Human Relations, Understanding Employee Behavior,
Understanding Group Behavior, Employee Motivations, Communication, Managerial
Style, Management of Conflict, Dynamics of Change)

Management by Objectives (Understanding and Implementation)
Managerial Decision Making (Problem Analysis, Decision-Making Process)

Resource Planning (Budget, Logistics, Position Management and Staffing)

© Items in this section were presented graduates of MIS only, Both the sections for
managers and supervisors are presented.

33



efficiency, morale, and employee satisfaction
(Appendix, III.B. and VI.B.), they usually re-
ported them to be from “good” to “very good,”
both before and after their MTS training; how-
ever, after MTS attendance, the ratings tended
more toward “very good.” About 30 percent of
the graduates reported improvements in their
units on these factors, about 60 percent reported
unchanged ratings, and only about 10 percent
felt that some decrease had occurred across the
interval. These results are perhaps about what
would be expected, given that MTS has usually
been seen as a beneficial experience by these
graduates. Supervisors and managers who had
not yet attended MTS (Table 17) tended to give
somewhat higher ratings when compared to the
“before MTS” ratings of graduates. The rat-
ings for both groups were at about the same
level when the “after MTS” ratings of graduates
were used in the comparison.

The same trends appeared for the superiors
and employees of the groups of supervisors and
managers surveyed. Typically, the ratings were
in the “good” to “very good” range, with the
superiors giving higher ratings than did em-
ployees on these three factors. The only average
ratings that fell below the “good” level were
those made by employees of supervisors, both
graduates and nonattendees, on the “morale”
scale; however, even in these instances, the means
were very close to the “good” level and certainly
well above the “acceptable” level.

In terms of change, both the MTS graduates
and their superiors indicated some degree of im-
proved unit functioning. For employees, this
was true only on the “efficiency” factor, Su-
periors, like the graduates, tended to report
improvement in the efficiency, morale, and em-
ployee satisfaction of their subordinate units
approximately 25 to 30 percent of the time. For
employees, these values ranged between 20 and
25 percent. The percentages of superiors and
graduates who reported decrements in efficiency
were very similar, generally close to the 10 per-
cent value. Employees indicated a decrease in
efficiency in approximately 20 percent of the
ratings.

IV. Conclusions.

The evidence presented in this study indicates
that some constructive changes occurred in grad-

17

uates as a function of supervisory/managerial
training at MTS. These findings are consistent
with the trend of the research literature on the
evaluation of such training. As Miner has
noted, the consistency of results is more than a
trivial indication that such training programs
can lead to certain changes in supervisors and
managers, particularly in their human relations
attitudes and problem-solving skills, In addi-
tion, the findings from this study indicate that
attendance at MTS is associated with some in-
crease in desirable on-the-job activities of grad-
uates, as seen by the graduates themselves, their
immediate superiors, and their employees.

It is of particular interest that while those
who attended MTS entered training with levels
of supervisory/managerial skills judged to be
somewhat below those of the supervisors and
managers who had not attended MTS, the grad-
uates returned to perform at a somewhat higher
level than the supervisors and managers without
MTS experience. Thus, even though the non-
attendees tended to have more on-the-job super-
visory managerial experience and to have had
more training opportunities than those who had
attended MTS, their levels of performance were
equalled or somewhat surpassed, on the average,
by the MTS graduates. In other words, it ap-
pears that the MTS training program was suc-
cessful in upgrading performance in supervisors
and managers up to and somewhat béyond the
level that would be gained from experience.

It should be noted that the ratings by all three
levels of personnel (supervisors/managers, su-
periors, and employees) indicate that FAA
supervisors and managers perform in an appro-
priate fashion most of the time, whether or not
they have attended MTS. Thus, it appears that
MTS serves to somewhat sharpen the effective-
ness of its graduates rather than radically alter
their on-the-job behavior. The behavioral areas
most distinctly influenced by this training are
employee counseling, performance evaluation,
and planning for training and development of
employees. There were no areas in which less-
ened effectiveness resulted from MTS attendance.

In general terms, supervisors and managers
and their superiors appear to believe in the
value, importance, and usefulness of the MTS
program. This was true whether or not the
supervisor or manager had attended MTS, al-



Il

A. Please rate yourself according to the degree to which each of the 3 following items
describes you in your current FAA position.

Not at all
Rarely
Occasionally
Usually
Almost always

Works as an employee (perform
technical job tasks)

Work as a supervisor (accomplish
specific tasks through others)

Work as a manager (decision and
policy maker)

B. On the following items, please describe the current FAA managerial "climate" as it
relates to your current position as a supervisor/manager.

Not at all
Occasionally
Almost always

Rarely
| Usually

He lpful

|
|
|
|

Tends to protect itself
against criticism

Encourages improvement in
supervisor/managerial skills

Stresses productivity

Encourages acceptance of
responsibility

Stresses quality — — — — —_—

Encourages new approaches to
supervision/management

Stresses aviation system
safety

Considers you a part of
management



APPENDIX

This appendix contains the questionnaires used in this evaluation of
MTS training. Since the forms presented to MTS graduates, other FAA
supervisors and managers, superiors, and employees are largely redundant,
the only entire questionnaire presented is the one used for the managerial
graduates of MTS. Items presented only to managers are noted; otherwise,
all items in this questionnaire were presented to supervisors as well. Also
noted are those sections deleted for supervisors and managers who had not
yet attended MTS. Questionnaires supplied immediate superiors and em-
ployees parallel these forms, with wording changes as appropriate.

19



Please rate the following specific subjects in terms of their usefulness to you in
your present managerial position.

Not at all
Slightly
Somewhat
Moderately
Quite
Very
Extremely

Creating a Productive

Work Environment
Career Planning
Training
Employee Development
Social/Political
Forces Which
Influence
Management
LYR
Employee Counseling
FAA Management Environment
Role of Management
FAA Mission
FAA Organization
FAA Policies
Line/Staff
Human Factors of Management

Human Relations

Understanding Employees
Behavior

Understanding Group
Behavior

384


TABLE 15, Average Ratings” by Managers and Superiors on Six

Behavior Ttems Concerning Only Managerial Performance

Spve tinge Superior

Before After Before After

26, objectives
established for
onie

Mngr-Grad 38 42
Magr-No MIS 420

7
ts

Mngr-Grad
Nnge-No MTS

27. Assess results in
terms of objec~
tives

a2

Nogr-Grad
Nnge-tio MIs

28, Systematically
defines vork
problems

3 43
1 a

Nnge-Crad
Mnge-No. MTS

29. Systematically
collects informa
tion on problems

0 43
2 a

30. Alternatives are
considered in
solving problens

Noge-Grad
Nngr-No. MES

45

2
3

31, Budgets are accu-

Ninge-Grad Aa ht 43
rate and tinely a3

2
Mnge-No MIS 412 oe

* nating scale values: 1 = occurs not at all, 2 = occurs rarely, 3 = oceure
occasionally, 4 = occurs usually, 5 = occurs almost
alvays

in terms of the respondent’s sex, location (e.g.,
FAA Headquarters, regional office, field facility),
or FAA service (AT, AF, FS, etc.) revealed
few statistically significant effects. However, no
pattern was apparent in these trends nor were
there any consistencies that would merit pre-
sentation and discussion.

2. Comparing attitudes toward MTS.

a. General evaluation. Most supervisors and
managers approached MTS with fairly high
expectations (Table 16 and Appendix, IIIA.
and VI.A.). They came away from MTS with
evaluations of the course that were essentially
consistent with those high expectations; in fact,
for many, the experience seems to have been
slightly better than they had expected. Between
30 and 40 percent of the graduates evaluated the
school more positively after they finished the
course than before they attended, 35 to 40 per-
cent showed no change in their evaluations, and
20 to 25 percent felt the school was less beneficial
than they had expected. However, it should be
emphasized that in most cases in which the school
was not what had been expected, the decrease in
rating was usually only one step on the rating
scale (for example, from “very useful” to “quite
useful”).

Neither facility location nor sex of the grad-
uates had an effect on their overall ratings of
MTS. There was, however, some tendency for
graduates from the Airports service, both super-

visory and managerial, to consider the school
less valuable or important than did participants
from other major FAA services (AT, AF, FS).
Thus, although the supervisors and managers in
Airports still tended to give an overall evalua-
tion that was to the positive side of the evalua-
tive scales, their ratings of MTS were somewhat
low in comparison to the other groups.

In a comparison of the ratings of supervisory/
managerial graduates to those of supervisors and
managers who had not yet attended MTS, it can
be seen that, on all the scales, the graduates re-
ported more positive feelings about MTS before
attending than did those who had not yet been
trained at MTS. In addition, the average rat-
ings made by graduates on each of the five scales
after attending MTS were higher than the cor-
responding ratings made before attending MTS.

The ratings of MTS by the superiors of the
supervisors and managers were generally posi-
tive and were similar for the graduates of MTS
and those without MTS training. In other
words, superiors expected the course to be bene-
ficial to their subordinates. Therefore, the dif-
ferences in attitudes toward the program between
graduates and those yet to attend MTS appear
to be a function of the attitudes of the indi-
viduals themselves to this type of training ex-
perience rather than a reflection of the under-
lying attitudes of their superiors.

It is of interest that before managers and
supervisors attend MTS, superiors of managers
tend to give slightly higher ratings to MTS
than do superiors of supervisors. However,
following MTS graduation by managers and
supervisors, all superiors evaluated the program
at about an equal level. This trend was also
observed in the supervisory and managerial
graduates themselves. Such a finding may re-
flect a greater belief in the general efficacy of
such training by managerial personnel (many
or most of whom have probably had some sort
of managerial training experience) in compari-
son to supervisory personnel and their superiors,
along with a somewhat less positive reaction to
the specific training experience, again perhaps
because of redundancy with other training ex-
periences for managers.

b. Effect on the organizational unit. When

graduates of MTS were asked to rate conditions
in their units along the three dimensions of


TABLE 7, Usefulness Ratings of Specific Supervisory Course

Content Areas by MIS Supervisory Course Graduates

Specific Course Content Area

Average
Rating®

Human Relations
Understanding Human Behavior
Understanding Group Dynamics
Understanding Employee Motivation
Rules of Communication
Training to Listen

Development
Performance Improvement
Performance Evaluation
Recognition and Awards
Training
Employee Counseling

Behavior
Adverse Actions
Conduct and Discipline

Management Theory
Management Identification
Management Style
Management Climate

LYR
History of Unionism
Union Recognition
Employee and Union Rights
Unfair Labor Practices
Executive Orders

FAA Organization
Mission
Organizational Structure
Line/Staff

Position Management
Position Classification
Pay/Time Administration
Absence/Leave

Staffing
Recruitment
Placement
Promotion
RIF Procedures
EEO
Reemployment and Return Rights

Safety
Safety Responsibility
Injury Compensation

auua

dowel

BAe

Cuba

a
Rating scale values:

nnn

not at all useful, 2 = slightly useful,
somewhat useful, 4 = moderately useful,
quite useful, 6
extremely useful

very useful,


Please rate the following specific subjects in terms of their usefulness to you in
your present supervisory position.

Not at all
Slightly
Somewhat
Moderately
Quite
Very
Extremely

Behavior
Adverse Actions
Conduct and Discipline
Performance Improvement
Performance Evaluation
Recognition and Awards
Training

FAA Organization
Mission
Organizational Structure
Line/Staff

Human Relations

Understanding Human
Behavior

Understanding Group
Dynamics

Understanding Employee
Motivation

Rules of Communication
Training to Listen

Employee Counseling

31



C. Please rate yourself on the following items as you were before you attended MIS. If
you were not a manager before you went to MIS, omit these items and go on to IV
(page 9). However, be sure to answer these items if you were a manager before
you went to MTS, even if you were in charge of a different unit than you are now.

3 :
2 oo» & 8 ¢
3 a a a @
@ 3 3 Q
2 u ° 3 E
° I g o 4
Zz C3 3 =) <
Employees felt free to discuss work
problems or other job-related
matters with me.
Discussed work problems or other
job-related matters with my
superior(s).
Offered unsolicited suggestions and
expressed my own opinions to my
superior(s).
Made my position on work-related
questions clear to superior(s).
Understood my superior's position
on work-related questions.
Comfortably counseled employees on
their job performance.
Indicated areas of needed improvement
to employees.
Appropriately judged the quality of
the performance of my employees.
On-the-job conduct of my employees
was appropriate. —_— _
My employees were motivated to work
hard. —_ — — — —
My employees were committed to doing
a "first-class" job. — —

26


apparent that both supervisors and managers
made more statements about what they liked best
than about what they liked least at MTS. This
measure of the overall evaluation of MTS indi-
cates that the program is positively regarded.
It is important because it is an indirect measure
of attitudes toward the school and thus is less
directly influenced by any motivations to make
the program “look good.”

TABLE 3. Summary of Statenente Made by MIS Graduates Concerning Best

Liked and Least Liked Aspects of the MIS Program

Liked Best About MIS —_Ltked Least About MIS

Subject Category

Supervivors Managers Supervisors Managers

1. Course Content vet 36. aha 29.01
TL, Instructors ws rd ae eat
TIT, Locatton/Acconmodations 12.3%, 13.4% 1.3% 1
WV, Course Approach 10.0%, 10.6% 12.3% 19,
Y. Group ateractions 9.0. 10.1% om

VE. Parttetpante 1.00 78% Ln 26

VIL. Results smn 4m om

VIIT, General Evaluation am un 0.8
TK, Course Requirements om 0.8% 25.61 3307
X. Miscellaneous ost. pes 19 Le

100.0% 00.0% 10,0" 199.0

Number of Statements 1,661 308 1,306 310

* rercentages refer to proportion of "Itked best" or "Liked Teast” statements

related to each subject category for supervisors or eanagers

The statements made by each supervisor and
manager were read by two raters and coded into
10 different categories. These categories were
derived by examining the statements to deter-
mine logical groupings. Each statement was
assigned to only one category. It should be
emphasized that the assignment of statements to
categories was completely independent for the
liked-best and liked-least portions of the ques-
tionnaire; in other words, it is possible to have a
large number of statements about what grad-
uates liked best at MTS in a category that also
contains a large number of liked-least responses.

From an examination of the tabulations for
each part (liked best and liked least) of the
questionnaire (Table 3), it is readily apparent
that supervisors and managers closely agree in
their general attitudes. Clearly, the category
with the most statements about what is liked
best about MTS is Course Content. Many of the
statements assigned to this category were gen-
eral; however, specific areas often mentioned in
a positive light are Labor-Management Relations

(LMR), Human Relations, Workshops and Role
Play, Management Theory, Equal Employment
Opportunity (EEO) (supervisors only), and
Management by Objectives (MBO) (managers
only).

The subject category mentioned next most
often was /nstructors. Statements in this cate-
gory usually referred to what were seen as the
good qualifications and preparation of the in-
structors, their beneficial teaching style and
teaching competence, and their helpful attitudes.

Many graduates responded favorably to the
school accommodations. The particular empha-
sis was on the quality of the class facilities and
the academic environment away from the de-
mands of the normal work setting. Related to
these statements were the frequent favorable
comments in the Course Approach category
concerning the “professional and training at-
mosphere” of the school, the attempts to encour-
age student involvement, and the general
philosophical orientation of the program.

Anecdotal evidence has suggested that one of
the features of MTS that students like best is
the chance to associate and interact with FAA
personnel from other locations and_ services.
This belief is confirmed by the tabulation of
liked-best statements; comments pertaining to
Group Interactions, including both class and in-
formal discussions, were frequently mentioned,
as was the chance to associate with persons from
a wide variety of FAA areas (Participants).

The graduates of MTS indicated a number of
features in the program that they regarded nega-
tively. Again, the most frequently mentioned
subject was Course Content. As noted above,
the LMR instruction was one of the best liked
parts of the program; it was also a focus for
specific dislikes about the content of the course.
Many graduates felt that too much time had
been spent on EEO and not enough time had
been allotted to Performance Evaluations. There
were also several statements that the workshops
were too long and were not well organized.

One feature seen almost entirely in a negative
manner was Course Requirements. Particular
dislike was expressed for the formal testing con-
ducted at the school. The evaluations by in-
structors and work assignments after class hours
were also given critical attention by many re-
spondents. Among the most frequent statements



the MTS courses, (b) to develop a complete pic-
ture of the positive and negative attributes of
the program, (c) to emphasize utility of course
content in actual supervisory/managerial set-
tings, and (d) to give graduates the opportunity
to provide specific feedback in a manner that
would indicate directions for constructive course
development. The questionnaire used in this
part of the survey is presented in sections IIT.A.,
TILB., IV., V., VLA., VI.B., and VII. of the
Appendix.

The specific content of the questionnaire was
developed from an analysis of the various con-
tent areas of the MTS course and incorporated,
in addition, several general evaluative dimen-
sions.

The survey questionnaires were administered
by teams of FAA supervisory/managerial per-
sonnel who visited offices and facilities in and
around 17 major metropolitan areas. Each
team member was from either FAA Headquar-
ters or the Academy, but none had any direct
connection with the MTS program. The teams,
composed of three persons each, administered
questionnaires to a total of 616 Supervisory
Course and 128 Managerial Course graduates
located at a total of 254 field offices and facilities,
five regional offices, the National Aviation Facili-
ties Experimental Center (NAFEC), the Aero-
nautical Center, and FAA Headquarters. This
sample represents approximately 10 percent of
the MTS graduates as of July 1973. The gen-
eral age and experience characteristics of these
respondents, all of whom were volunteers, are
shown in Table 1. Table 2 shows the distribu-
tion of respondents by sex, GS or WB grade
level, FAA service, and facility location. It is
readily apparent that the respondents were pre-
dominantly males at the upper grade levels, were
proportionally distributed across FAA services

TABLE 1. Age and Experience of MIS Graduate Participants”
Supervisory Managerial
Vartable Seer __Manaerieh
Average Range Average Range
Age 46.2 28-66 48,2
FAA Experience 16.8 0.6.36 20,0
Total Supervisory/Managerial
Experience Ore 12
Time tn Present Position 42 oT 34
Tine Since MIS Attendance Lz oes 13

ALL values are tn yearly units,

in terms of total personnel in the services, and
represented the spectrum of types of facilities.

TABLE 2. Distribution of Participants on Background

Varlables Considered in Survey

Variable Supervisory Managerial®
Sex
Male 387 126
Female 29 2
Grade Level
GS 5-11 38 o
2 “67 1
B 12 4
“ 209 “5
‘6 EY 40
Ws ALL Levels 6 o
TAA Services
Air Teaf tic 258 42
Alrvay Facilities Bs 34
Flight Standards 92 n
Other (e.g., Airpores, 128 38
Logtseies, Research)
Location
Washington Headquarters 2 7
Regional Offices a 2%
NAFEC % 10
Acronautical Center 89 9
Tower PTRACON/RAPCON 90 2
ance 99 10
AF Sectors 8 rts
FS Stations 4 3
Other 5 28
Motals for the variables do not match because of occasional missing date

on dems

B. Results. The results presented below are
summaries of the analyses conducted on each of
the sections of the evaluation questionnaire.
In general, chi square procedures were used to
determine the significance of differences of dis-
tributions of responses on the variables consid-
ered in the survey. Those differences discussed
in the text are based on chi square analyses sig-
nificant at the .05 level or better; all nonsignifi-
cant trends are omitted from the discussion.

The first section focuses on general evaluative
attitudes toward MTS; the second, on the utility
of the various areas covered by the Supervisory
and Managerial Courses. The remaining sec-
tions examine the areas in which MTS was help-
ful to the graduate, those in which the program.
was not of benefit, and the judged effect of MTS
on the units being supervised by MTS graduates.

1, Attitudes toward MTS.

a. Likes-dislikes questionnaire. The responses
to the open-ended questions concerning what was
liked best and least about MTS (Appendix, IV.)
gave considerable insight into the evaluation of
MTS by its graduates. From an examination of
the number of statements made about the best
and least liked aspects of MTS (Table 8), it is


TABLE 16. Average Ratings of MIS on Five Evaluative Dimensions

Average Average

. Ratin; Ratin Percent Reporting
Dimension Group Before Afters
Mrs Mrs Higher No Change —_Lower
Useless /Useful
Spvr-Grad 7.5 747 36
Superior 7.4 7.3 51
Spvr-No MIS 6.9 —_ i
Superior 7.5 wee -
Mngr-Grad 7.5 33 42
Superior 7.6 22 52
Mngr-No MIS 7.0 --- ---
Superior 27 = —_
Unimportant /Important
Spvr-Grad 7.3
Superior 7.4
Spvr-No MIS 6.9
Superior 25
Mngr-Grad 7.3 4 20
Superior 14 42 28
Mngr-No MTS 7.3 a aoe
Superior 7.8 --- oe
Bad /Good
Spvr-Grad 1 42 35
Superior 15 18 60
Spvr-No MIS 7.0 --- a+
Superior 75 --- a
Mngr-Grad 7.4 40
Superior 7.6 53
Mngr-No Mrs 7.4 --e
Superior 27 —
Worthless/Valuable
Spyr-Grad 14 36
Superior 14 26
Spve-No MES 7.0 wee
Superior 17 ae
Nngr-Grad 7.6 48
Superior 7.4 49
Mngr-No MIS 7.2 ---
Superior 7.8 ane

Undesirable/Desirable

Spvr-Grad 7.4
Superior 7.5

Spvr-No MIS 6.4
Superior 7.2

Mngr-Grad 7.4 42
Superior 17 35

Mngr-No Mrs 6.9 oo.
Superior 7.6 -

* Rating scale values: 1 = very useless (unimportant, bad, worthless, undesir-

able), 2 = quite useless, 3 = somewhat useless, 4 =
slightly useless, 5 = neither useless nor useful, 6 =
slightly useful, 7 = somewhat useful, 8 = quite useful,
9 = very useful

15



hess. Averaging more than a full rank lower
was Employee Development (for which 72.3
percent of the rankings were first, second, third,
or fourth), followed by Behavior (57.0 percent
of the respondents ranked this subject second,
third, or fourth). Ranked close to Behavior was
Management Theory. The rankings were more
diverse for this content arca; nonetheless, the
second or third rankings included 36.0 percent
of the responses. The LMR area was next in
rank, and, in this case, the rankings were quite
diverse.

TABLE 5. General Supervisory Course Content Areas Ranked

in Terms of Usefulness by MIS Supervisory Graduates

Course Content Area Average Rank

Human Relations 2.18
Employee Development 3.56
Behavior 4.03
Management Theory 4.22
Labor Management Relations 5.16
FAA Organization 5.70
Position Management 6.13
Staffing 6.47
Safety 7.47

In the lower ranks were the areas of FAA
Organization, Position Management, Staffing,
and Safety. There was no trend in the Organi-
zation rankings. The Position Management area
was usually given a fifth, sixth, or seventh rank-
ing (55.7 percent); Staffing rankings were
usually from sixth to ninth (71.6 percent). At
the bottom, and averaging more than a full rank
lower in usefulness on the job than any other
area, was Safety. A total of 76.8 percent of the
rankings for this subject were from seventh to
ninth,

The rankings for the content areas in the
Managerial Course show that, like supervisors,
the managers consider the “people oriented” area
(Human Factors) most useful (Table 6). This
area was ranked most useful by 52.6 percent of
the respondents. Next, averaging nearly a full
rank lower, was MBO. Creating a Productive
Work Environment and Managerial Decision
Making were relatively close to each other and

to MBO, but there was no clear preference as to
which ranking should be assigned to these fac-
tors; in other words, these areas were assigned
a considerable number of each of the possible
rankings.

TABLE 6. General Managerial Course Content Areas Ranked

in Terms of Usefulness by MIS Managerial Graduates

Course Content Area ‘Average Rank
Human Factors 1,93
Management by Objectives 2.83
Creating a Productive Work Environment 3.13
Managerial Decision Making 3,36
FAA Management Environment 4,79
Resource Planning 4.91

There was very clear agreement among mana-
gers that the areas of FAA Management En-
vironment and Resource Planning ranked lowest.
For both of these subject areas, 70.1 percent of
the respondents assigned rankings of fifth or
sixth,

b. Specific supervisory course content. Each
of the specific subjects taught at MTS was eval-
uated in terms of its usefulness in the super-
visory work setting (Appendix, V.C.). A
7-point scale with end points of “not at all use-
ful” (1) to “extremely useful” (7) was employed.
The results are reported below in terms of the
general content areas as ranked in the previous
sections.

(1) Human Relations. Each of the specific
content areas (topics) covered by Human Rela-
tions was judged, on the average, “very useful”
for supervisors (Table 7). The lowest rated
portion of this general area, Understanding
Group Dynamics, was rated higher than all but
one (Employee Counseling) of the specific areas
under other general] headings.

(2) Development. This group of topics was
also highly rated in terms of usefulness, par-
ticularly Employee Counseling and Performance
Evaluation. The lowest rated of these areas,
Recognition and Awards, was still seen as “quite
useful” on the average and was more highly
rated than any topic in the remaining general
content areas. There were some differences be-



Information Sheet

Supervisor /Manager
Age, M. F GS or Wage Grade level (specify which)
FAA Program: AT. AF. FS. Airports Other.

Position title (supervisory ATCS, management analyst, supervisory electronics technician,
ete.):

Organizational title (team supervisor, assistant chief, branch chief, etc.):

Present location (check one):

Washington headquarters- - - - - - = - - eee ee ee ee ee eee
Regional offices - - ----------- eee ee ee eee eee
NAFEC- - - - - - - ere rr re re ee ctr ter eee
Aerocenter ------ 7-77-77 2ee0-0-7 wee eee ee eee
Field office/facility (specify office/facility type) --------

Number of employees supervised- - - - - --- - -

Number of supervisors directly supervised - - - -

Total FAA experience (years and months) - - - - - - - - - eee
FAA supervisory/managerial experience (years and months)- - - - - - - - -
Time in present position (years and months) - -----*---- 7-7-7 >

MIS attendance

None,

Supervisory course (mo/year) Refresher (mo/year)

Managerial course (mo/year) Refresher (mo/year)

Executive School attendance

Executive School (mo/year)



Technical Report Documentation Page

T. Report No. 2. Goverament Accession No. 3. Recipient's Catalog No.
FAA-AM-75-9
4. Title and Subniie 5. Report Date

AN EVALUATION OF THE EFFECTIVENESS OF THE FAA
MANAGEMENT TRAINING SCHOOL

6. Performing Organization Code

al

Performing Organization Report No.

7. Author's) Roger C, Smith, Ph.D., Barbara Rana, B.S.,
and Deborah K. Taylor, B.S.

9. Performing Orgenization Name and Address 10. Work Unit No. (TRAIS)
FAA Civil Aeromedical Institute
P.O. Box 25082 11, Contract or Grant No.

Oklahoma City, Oklahoma 73125

13. Type of Report and Period Covered

12. Sponsoring Agency Name and Address

Office of Aviation Medicine OAM Report
Federal Aviation Administration
800 Independence Avenue, S.W. TE. Sponsoring Agency Code

Washington, D.C. 20591

15. Supplementary Notes

This research was conducted under Tasks AM-B-72-PSY-39, AM-B-73-PSY-39,
AM-C~74-PSY-39, and AM-C-75-PSY-39,

16. Abstract
This report covers the two aspects of an agency-wide assessment of the FAA Manage-
ment Training School (MIS) program. The first part of the report considers the
evaluation of MIS by approximately 10 percent of the Supervisory and Managerial
Training Course graduates. Detailed questionnaires for evaluating the MIS experi-
ence in terms of the usefulness and the quality of specific and general aspects of
the course were presented to graduates at various FAA field offices and facilities
across the Nation. It was found that supervisors and managers held similar views
of MIS: Most felt that the program had been useful in helping them to meet the
demands of their supervisory/managerial positions. The responses of the graduates
indicated areas in which improvement could be effected within the MIS framework.
The second part of the report presents the findings concerning the impact of MIS
on the behavior of course graduates. Questionnaires concerning specific types of
behavior desired of supervisors and managers were completed by graduates of MIS,
their immediate supervisors, and their employees and also by supervisors and
managers not yet trained at MIS, It was found that MIS training resulted in an
increase in desirable on-the-job activities of MIS graduates, as judged by the
graduates themselves, their supervisors, and their employees. Those who had
attended MIS entered training with supervisory/managerial skills somewhat below
the level of those of the individuals who had not yet attended (i.e., supervisors
and managers given lower priority for attendance because of their greater experience
and training), and they returned to perform at a somewhat higher level than those
without the MIS experience.

17. Key Words 18. Distribution Statement
Management Document is available to the public
Personnel through the National Technical Information
Training Service, Springfield, Virginia 22151

19. Security Classif. (of this report) 20. Security Classif. (of this page) Zi. No. of Pages | 22, Price

$3.00 PC
Unclassified Unclassified $ .95 MF

Form DOT F 1700.7 (8-72) Reproduction of completed page outhorized



TABLE 8, Usefulness Rating of Specific Managerial Course

Content Areas by MIS Managerial Course Graduates

Specttte Course Content Area averane

Human Factors of Nanagenent
Human Relations 6.0
Underacanding Enployee Behavior ea
Underatanding Group Behavior 60
Employee Motivations 6.0
Managerial Style 5.5
Communication 6.2
Management of Conflict le
Dynantes of Change 33

Management by Objectives
Understanding Principles of MBO
Implementation of

Creating a Productive Work Environment
Career Planning
Training
Employee Development
Social/Political Forces Which Influence Management
DR

yhiyyr

Employee Counseling

Managerial Decision Making
Problem Identification/Analysis
‘The Decision-Making Process

FAA Management Environment
Role of Management
FAA Mission
FAA Organization
FAA Policies
Line/Staft

BorLS

Resource Planning
Budget Process
Logistics Planning
Position Management and Staffing

yrs
ob

* Rating scale values: 1 = not at all useful, 2 = slightly useful,
3 = somevhat useful, 4 = moderately useful,
5 = quite useful, 6'= very useful,
7 = extremely useful

Principles of MBO and Implementation of
MBO, were rated as essentially of equal useful-
ness.

(8) Creating a Productive Work Environ-
ment. The usefulness ratings of-the topics under
this course content area varied considerably.

Employee Counseling, Employee Development,
and Training averaged at and above the “quite
useful” level; the remaining areas averaged be-
tween “quite useful” and “moderately useful.”
Managers from the AT program tended to rate
LMR higher than did most other managers.

(4) Managerial Decision Making. The two
specific topics concerned with decision making
were seen as “quite useful” to “very useful,” with
almost identical ratings. No important differ-
ences were noted between managers of the various
services.

(5) FAA Management Environment. The
topics in this area were generally rated in the
“moderately useful” to “quite useful” range.

Again, no differences between services were
noted.

(6) Resource Planning. The three topics
considered in this portion of the course were
rated at the “quite useful” to “moderately use-
ful” level. As with most other areas, there were
no major differences in ratings given by mana-
gers from differing circumstances and back-
grounds,

8. General post-course evaluation.

a. Benefits and shortcomings of MTS. The
participants in this part of the survey were
asked to indicate the specific ways in which MTS
had helped them in their day-to-day job func-
tioning and also to indicate those problem areas
in which the MTS experience had not been help-
ful. These data were analyzed by the same pro-
cedure used to evaluate the “liked best/liked
least” portion of the questionnaire (Appendix,
IV.).

As in other parts of the survey, the super-
visory and managerial graduates of MTS showed
considerable agreement in listing that which they
regarded as beneficial and that which they re-
garded as having been helped little by MTS
(Table 9). For both groups, the school was
most helpful, as judged by the number of com-
ments, in providing the graduates with a better
understanding of the functions of supervisors/
managers and with increased skills to meet the
requirements of their work. Two course content
areas that were mentioned quite often were
Human Relations and FAA Organization. That
Human Relations should be so cited is not sur-
prising; however, the frequent mention of FAA
Organization as a particularly helpful subject
area is unexpected in view of the relatively low
rating assigned to this topic in the rankings of
course usefulness. However, this apparent dis-
crepancy may be due to the differences in the
questions asked in these two questionnaire sec-
tions. Thus, while the instruction on FAA
Organization may not have been as useful, on a
continuing basis, as some other content areas, it
may have proved to be particularly helpful in
solving specific supervisory problems and for
this reason received its high frequency of cita-
tion as a “helpful” area. Another course content
area also mentioned quite often as helpful was
LMR; however, an essentially equal number of
respondents indicated their LMR training had


tween these ratings in terms of FAA services.
Employee Counseling was rated higher by AT
graduates than by AF and FS graduates, who
in turn rated this subject more useful than did
those in the Airports service. The Airports
graduates, as well as those from other relatively
small activities, also ranked the Training seg-
ment lower than did AT, AF, and FS graduates.

(8) Behavior. Of the two areas included in
the Behavior segment of the course, Adverse
Actions was rated lower—lower, in fact, than
any other topic except one. Even though low
rated, this topic was still judged “moderately
useful” on the average. For both topics under
this heading, the participants from FAA Head-
quarters assigned relatively low usefulness rat-
ings in comparison to those assigned by super-
visors from most other facilities. This suggests
that these matters are of relatively little conse-
quence in that one location instead of indicating
a particular course shortcoming.

(4) Management Theory. Each of the specific
subjects under this general content area was
considered “quite useful” by the respondents.
No substantial differences between ratings for
various groups were noted, with the exception
that FAA Headquarters personnel rated Man-
agement Identification somewhat lower than did
graduates from other locations.

(5) LMR. This course content area showed
considerable variability in ratings of the specific
subjects included therein. The History of
Unionism received the lowest average usefulness
rating of any subject area in the MTS program.
Employee and Union Rights, Unfair Labor
Practices, and Executive Orders were rated at
about the “quite useful” level and were highest
for the group.

There were substantial differences between
FAA services in terms of the ratings of useful-
ness of LMR topics. Typically, AT and AF
supervisors rated each of the five specific content
areas as more useful than did Airports or FS
supervisors. There was also a considerable
tendency for personnel in FAA Headquarters
and the regional offices to rate these areas as less
useful than did those assigned to field offices and
facilities. These are entirely reasonable findings,
as supervisors who found these subjects of least
utility were those who have the least contact
with labor affairs.

(6) FAA Organization.. The three topics
under this area were typically rated “moderately
useful” to “quite useful” by supervisors. The
only difference between groups was the tendency
for supervisors in the Airports service to rate
Organizational Structure and Line/Staff higher
than did graduates from other FAA services.

(7) Position Management. The ratings in
this area were comparable to those for FAA
Organization and were typically in the “moder-
ately useful” to “quite useful” range. There
were some differences between groups on the
topic of Position Classification; FS and Airports
supervisors rated this subject somewhat higher
in terms of job usefulness than did other super-
visors and in particular gave a very large num-
ber of “extremely useful” rankings.

(8) Staffing. There was more variability in
the usefulness ratings for the various topics in
this course content area than for any other. The
instruction in both EEO and Promotion was
generally seen as “quite useful.” On the other
hand, RIF Procedures and Reemployment and
Return Rights averaged somewhat below the
“moderately useful” level.

(9) Safety. In this content area, both topics
were rated in the “moderately useful” range but
were seen as particularly valuable by AF super-
visors. Again, this seems a reasonable finding,
since the AF working situation will typically
present more safety hazards to employees than
will the working situations at other types of
facilities.

ce, Specific managerial course content. Like
those for supervisors, each of the subjects taught
to managers at MTS was also evaluated on the
7-point scale described above.

(1) Human Factors of Management.. The
ratings for this area indicated that all but three
specific topics were evaluatéd in the “very use-
ful” range of application to the manager’s
setting (Table 8). The three exceptions, Man-
agement of Conflict, Managerial Style, and
Dynamics of Change, each averaged at about
the midpoint between “quite useful” and “very
useful.” There were no differences of substance
between managers from the various services on
these items.

(2) Management by Objectives. The average
rating for this content area was “quite useful.”
The two topics under this area, Understanding



Vv

(Supervisors)

A. Please rate the following factors as to how important they have
your development as a supervisor/manager.

Important
FAA policies _

Personal
efforts

Organizational
climate _

Personal
qualifications

Mrs _

Immediate
supervisors

Other (specify)

Quite

Somewhat

Slightly

Neither

|
|
|

Slightly

Somewhat

Quite

actually been in

p>
a

o
>

Unimportant

B. Please rank-order the following general subject matter areas which were taught at
MIS in terms of their usefulness to you in your present supervisory position.
Place the number "1" by the most useful, the number "2" by the next most useful
and so on (the areas are listed in alphabetical order).

Behavior (Adverse Actions, Conduct and Discipline)

Development (PER, PIP, Training, Recognition and Awards)

FAA Organization (Mission, Organization, Policies, Line/Staff)

Human Relations (Human Behavior, Small Group Dynamics, Motivation, Communica-

tion, Employee Counseling)

LMR (Labor Movement, Union Recognition, ULP's, Executive Orders)

Management Theory (Management Identification, Style, Climate, Leadership)

Position Management (Position Classification, Pay/Time Administration,

Absence/Leave)

Safety (Safety Responsibilities, Injury Compensation)

Staffing (Recruitment, Placement, Promotion, RIF, EEO)

30


Iv
Please indicate what you liked best and liked least about MIS, List your comments in ranke

order of importance. Be BRIEF and LEGIBLE,
A. Cite three specific aspects of MIS which you liked best.

(1),

(2).

G).

B, Cite three specific aspects of MIS which you liked least.

(1),

(2).

Please indicate the ways in which MIS was effective and ineffective in helping you develop

as a manager.
A. Cite three specific ways in which MIS training has helped you as a manager.

Q)

(2).

(3).

B. Cite three specific ways in which MIS training was not effective in helping you as a
manager; that is, in what areas did you need more training than was presented and what
areas of needed training were absent from the program?

(1),

(2).

(G),

29



included in this category were indications that
the course was too long.

While MTS graduates usually liked the cam-
pus atmosphere and modern facilities at MTS,
they were less complimentary about the actual
location of the school (Lawton, Oklahoma).
Under the Location/Accommodations category,
many supervisors and managers indicated a dis-
like for Lawton, the inconvenience of traveling
to Lawton, the confining aspects of dormitory
living, and group washrooms.

Another relatively common dislike of these
graduates was what they viewed as a clear slant
toward air traffic control in the presentations.
Several individuals indicated that the situations
and problems discussed seemed to be set up for
air traffic needs and were of little relevance to
their own area or service (Course Approach
category).

As previously noted, the instructors (Jnstruc-
tors category) were often mentioned in discuss-
ing the features of MTS that were liked best.
However, there were some graduates who felt
that the instructors were lacking in those quali-
ties that most others had identified as beneficial;
ie. qualifications, teaching style, and attitude
toward students,

b. MTS as a factor in the development of
supervisors and managers. The responses of
supervisors and managers to the question as to
the importance of various factors in supervisory/
managerial development were similar (Appendix,
V.A.). Far and away, they gave Personal Ef-
forts and Personal Qualifications the highest
ratings, an indication that they feel their own
attributes are the most important contributors
to their. development as supervisors or managers
(Table 4). Rated at a somewhat lower, but still
high, level of importance were the organizational
variables of supervision, policies, climate, and
MTS. It is of interest that supervisors rated
MTS relatively higher than did managers. This
finding probably reflects the differences in train-
ing experience between the two groups. Mana-
gers typically have had more advanced educa-
tional experiences, previous supervisory and
managerial training, and other developmental
experiences than have supervisors—experiences
that might tend to diminish the perceived im-
portance of MTS. Supporting this fact is the
observation that those who had attended the

TABLE 6, Factors Concributing to Supervisory/Managerial Development

Supervisors Managers
Factor Percentage” Percentage
RICESEE —poreane RIIME —teporeane
Personal Efforts 92 ons 8409) 976
Personal Qualiftentions 1 96.5 190.2) 976
Tamedtate Supervisors cy on. 170.6) 92.0
PAA Policies 1 ons 130.) 904
organteationat Climate 23 os 1002) Hb
vs oy oe 6.8 (6.9) 85.6
“Racing scale values: 9 = very important, 8 = quite important, 7 = somevhat

important, 6 = slightly important, 5 = neither teportant
nor unimportant, £0 1 = very unimportant

P the percentage of participants vho indfcated that the factor had at least some
degree of taportance,

© the values in parentheses refer to mean ratings for 38 managers vho hed
previously attended Executive School,

FAA Executive School rated the MTS program
substantially less important than did those man-
agers who had not had such an experience; it
should be noted, however, that the ratings of the
former group indicated that the MTS experience
has some importance for managers.

There were two population variables that were
related to differences in the perception of the
importance of MTS for supervisory/managerial
development. Managerial graduates from the
Aeronautical Center gave proportionally more
ratings indicating the MTS program was unim-
portant than did managerial graduates from
other locations, even though the average rating
for the Aeronautical Center group was not.
greatly different from the average rating for the
overall managerial group. (Note: As stated
previously, reported differences are statistically
significant at the .05 level or better.) Finally,
the small number of managers from the Airports
service showed a considerable tendency to rate
MTS as unimportant. Airway Facilities (AF)
and Flight Standards (FS) managers, while at-
taching some importance to MTS, did not rate
the program as important as did Air Traffic
(AT) graduates or graduates from other FAA
activities,

2. Course content evaluation.

a. General course content. When supervisors
were asked to rank the general course content
areas in terms of the usefulness of these subjects
in their present position (Appendix, V.B.), the
highest ranked area was Human Relations (Table
5). A total of 72.2 percent of the respondents
ranked this area either first or second in useful-


though those who had yet to attend did not rate
the program quite as high as did those who had
completed the training.

The ratings of efficiency, morale, and employee
satisfaction within units were also relatively
high and varied little as a function of MTS
training, although when changes did occur, they
more often reflected improvement in conditions
within the unit. As such, these results are not
unexpected; attendance at MTS is only one of a
great many factors, some or most of which are
outside the supervisor’s control, that determine
morale, efficiency, and worker satisfaction within
the work unit.

One question that should be considered is, Are
the findings valid? In any survey such as this,
it is difficult to prove that the evaluations and
ratings of performance made by the respondents
are accurate representations of their true feel-
ings. There are, however, several features of the
data that indicate the survey is probably a good
index of the current assessment of both the eval-
uation of MTS by its graduates and the evalua-
tion of on-the-job performance by all types of
respondents. It was found that a good distribu-
tion of responses across possible ratings for each
scale was obtained; that is, the respondents did
not just nonchalantly mark everything in one

~ column on the various ratings and performance
scales. Second, most respondents wrote more

than minimal statements to the open-ended por-
tions of the questionnaire, thus indicating some
investment in conscientiously completing the
evaluation task. Most important, some ratings
were influenced by the variables of facility loca-
tion and FAA service in a manner that, in each
case, was appropriate to the issue being consid-
ered; eg., the relatively higher “usefulness”
ratings for the Safety Course content area by
AF supervisors. Taken together, this all sug-
gests that the respondents, on the whole, tried
to present an accurate picture of their evalua-
tions of MTS and of supervisory/managerial
performance within the agency; there is nothing
in the data to suggest otherwise. Therefore, it
seems reasonable to conclude that these findings
are probably a legitimate estimate of the feelings
of FAA personnel concerning the worth and
behavioral impact of the MTS program.

In sum, these findings support the conclusion
that MTS has some beneficial effects on the per-
formance of graduates of the program and that
these effects tend to be greater for supervisors
than for managers. It appears that the net
effect of the program is to encourage some im-
provements in what is already seen by personnel
at all three work levels (supervisory/managerial,
superiors, employees) as generally adequate
supervisory/managerial functioning.

REFERENCES

1 Cantpbell, J. P., M. D. Dunnette, E. E. Lawler III,
and K. E. Weick, Jr.: Managerial Behavior, Per-
formance, and Effectiveness, New York, McGraw
Hill, 1970.

18

2. Martin, H. O.: The Assessment of Training, PER-
SONNEL MANAGEMENT, 39:88-98, 1957.

3. Miner, J, B.: Studies in Management Bducation,
New York, Springer, 1965.


C. Please rate yourself on the following items as you are now after having attended MIS.

> ES
« 2 3
a a
3 € ‘3
9 >
e > ] a o
* 2 3 | 23
2 M4 8 3 H
9 3 2 & 4
z 4 ° > <
Employees freely discuss work problems
or other job-related matters with me.
Discuss work problems or other job-
related matters with my superior(s).
Offer unsolicited suggestions and
express my own opinions to my
superior(s). — — —_— — —_—
Make my position on work-related
questions clear to superior(s).
Understand my superior's position on
work-related questions.
Comfortably counsel employees on
their job performance.
Indicate areas of needed improvement
to employees. — —
Appropriately judge the quality of the
performance of my employees. — —_—
On-the-job conduct of my employees is
appropriate. — — — — —
My employees are motivated to work
hard. —_ —_— — —_ —
My employees are committed to doing a
"first-class" job. — — — — —
My dealings with employees are fair
and objective. — — — — —
Equal treatment is given to all
employees and associates, includ-
ing minorities and women, — —_— —

37



After MIS or now

Equal employment opportunity
programs are supported.

Discrimination against employees
because of race, ethnic background,

or sex is avoided.

Top FAA management policies, plans,
and goals are understood.

Understand major programs of the
agency other than my own.

Employee training and development
are considered one of my major
responsibilities.

Definite plans for training and
on-the-job development of
employees are established.

Deal with employee grievances
fairly and appropriately.

Resolve employee complaints to
the mutual satisfaction of the
parties involved.

Accomplish administrative and paper-
work requirements such as budget-
ing, personnel actions, reports,
etc. efficiently and effectively.

Ways of recognizing the special
achievements or excellent work
of employees are sought out and
used,

38

Not at all

Rarely

Occasionally

Usually

Almost always



C. Please indicate the degree of satisfaction you have experienced in the FAA on the
following items.

Very Satisfied
Satisfied
Indifferent
Dissatisfied
Very Dissatisfied

Working for FAA

Being an FAA supervisor /manager

Your current supervisory/
managerial position

The FAA as an organization today

Your satisfaction with working
for the FAA three years ago
(if you were then employed
by the FAA). Specify the
type of position you heid
at that time,





working directly under the supervision of each
of the supervisors and managers who responded
to the questionnaire; however, employee partici-
pation was entirely voluntary. If an employee
declined when asked to be part of the survey,
another was asked to complete the questionnaire.

As with employees, the superior of each of the
supervisory/managerial respondents was invited
to rate the supervisory/managerial performance
of his subordinate supervisor or manager.

The age and experience characteristics of the
various groups of participants are shown in
Table 12. It is of interest that those supervisors
and managers who had not yet attended MTS
were somewhat older, had more time in their
present positions, and, at least in the case of
supervisors, had more supervisory experience

than did MTS graduates but tended to have
somewhat less total FAA experience. These
differences may have been due to the priority
for MTS training given to new and untrained
supervisors and managers. In addition, there
may have been some tendency for those who had
not yet attended MTS to be a little more reluct-
ant to participate in this type of training, per-
haps because of some longer period of experience
and less-felt need for learning opportunities in
this area. The data on employees and superiors
are about as expected; employees tended to be
younger and superiors, slightly older than the
supervisors and managers they were rating.
Both groups typically had substantial FAA ex-
perience and were well acquainted with the per-
sons they rated.

TABLE 12, Average Age and Experience of Survey Participants
Supervisors Managers
Variable No MIS No MIS Employees  Superiors
Graduates Training Graduates Training

Age 46,27 49.0 48.2 50.7 41.7 49.3
FAA Experience 16.8 16.6 20.0 18,2 11.9 20.1
Supervisory /Managerial

Experience 75 9.1 12.4 11. anor
Time in Present Position 4.2 4.6 344 4,2 44 3.7
Time Worked for

Supervisor/Manager one ones 3.0 wee
Time Supervisor/Manager

Has Worked For wees wee 3.1

@ All values are in yearly units.

As in the previous section of the report, all
differences discussed below are statistically sig-
nificant at the .05 level or better according to
chi square analyses of the data.

B. Results.

1. Supervisory/managerial behavior.

a. General trends. The overall comparisons
shown in Table 13 of ratings of supervisory/
managerial behavior for MTS graduates before
and after MTS training indicate that the grad-
uates themselves, their superiors, and their em-
ployees tend, as groups, to report an increase in

11

the occurrence of desirable behavior after at-
tendance at MTS.

The item ratings were usually close to, or just
above, point 4 on the 5-point rating scale, which
indicates that the particular behavior “usually”
occurred. This was true both before and after
MTS training, and the ratings tended to be
slightly higher when the graduates rated them-
selves than when they were rated by their su-
periors and employees. The behavior ratings
for graduates of MTS averaged one- to two-
tenths of a scale point lower before attending
MTS than the mean ratings for supervisors and



ma,

‘Average hattoas" by Group for Teena Comon to Sopervisory/Managertal Behavior Surveys

Spve/iege Superior

. == ame sowie
Before After Defore After Betore Ateer Betore Afar Before After Before After
CRE” Eee OSE we taro pe mt
tecuse work Spve-Mo MIS 4.9 _ 4 ye-Mo MES ¥
oe a it
mein, ts wees 2
mite ut mth 8
augue, fm s Satin pc “
a 8 se rie a
ns
meson, o mis are
mei hs 8 title “on
. seems tek Gh masons, en ware
Wetted mat ata momo
mis
men ta meee “os
Mme Gono mit, woos
. wea, ett 1 Salsa oon
ite Go sama" iota “ous
mi, me ae
mate Soto iit, Bos
+ wie ig Bee, bh ae ;
a a wos rote 3
mi 2 oO woo re ae
. es, Ee mmm Soa
a a a a ee cr
me ee ess “
ve tam tome sera, 32423 be saree pa o
sum
min bee he ae s res
mi Sooo i ogo rit, :
on ae ee u ri “ “
Fem
Ee ea ees mM o
Mie oO aM ottok mite “ “
oman cote quae Gg teat 2 amin pa G a
near Gir oo oM oY sprue eek, “ “
nea,
mee oe mee a 2
. pes, Say em seme tpt us
oo wou meee
Mite Soo woo mite : ‘
, isthe 2 EEE de tego a _
a a as a ite ca
win, MoO oe 8 oe m
wi, Gof og oP oe mit, u
Ome mtone some et > termine, spe oo
ements Th, Se 2x
ok
"Rating scale valves:

© teens omteees for employe

re reported; Lees, higher

work problems (item 2) than the supervisors
and managers rated themselves. Also remark-
ably consistent was the tendency of employees to
rate their supervisors and managers higher with
respect to developing an understanding of FAA
plans and goals and programs than the super-
visors. and managers rated themselves (items 16
and 17).

There were relatively few items for which
there were significant differences between MTS
graduates and those who had not yet attended
MTS. However, when differences did occur, the

1 + occurs not at all, 2» occure rarely, 3 occurs occaslonally, & = occurs nually, $ © occure almost alvaye

ore mean Leas occurrence of confuston So avarde.paperverk,

outcome was always in favor of the MTS grad-
uates. This was particularly true with respect
to the development and training area (items 18
and 19), as the ratings for these items were
higher for the graduates and their superiors and
employees on each comparison with the corre-
sponding group for those without MTS training.
Other differences were found in relation to the
awards program, equality of treatment, and
openness to discussion.

c. Sew, location, and program effects. Anal-
ysis of responses covering supervisory behavior

18



ACKNOWLEDGMENT

Any project of this extent requires the support of a great many people.
The participants in the survey gave generously of their time to provide the
data for this survey, and their efforts are deeply appreciated. In each of the
regions visited, the directors and their training staffs coordinated the visits
of the survey teams and smoothed the way through the administrative com-
plexities associated with this type of project. Mr. Howard Richardson,
ATR-1; Mr. Aaron Rothman, ATR-200 (now APT-320) ; and Mr. George
Hendon, then of AMS-510, provided counsel, cooperation, and support,
through all phases of the survey. It was, however, the intensive efforts of
the members of various FAA offices who volunteered to serve on the survey
teams that ultimately assured the success of the data collection, The mem-
bers of the team were: (from FAA Headquarters) Mr. George Selberg and
Mr. James Strnad of APT-320, Mr. Doug French of ATR-100 and now at
Washington ARTCC, and Mrs. Joy Herndon of AMS-510; (from the Acad-
emy) Mr. Jack Hardy of AAC-931 and now at Lambert-St. Louis Inter-
national Airport in St. Louis, Mr. Richard Kaser of AAC-932, Mr. J. P.
Bushnell of AAC-951 and now of AEU-200 in Brussels, and the late Mr.
Charles “Chuck” Stacy, former chief of Flight Standards Branch (AAC-
950). The authors deeply appreciate the contribution of all these individuals
to the successful completion of the project.



VI

Answer the items in the next three sections according to how you feel now after
having attended MTS.

A. I found MIS to be:

2 > > ra
3 a 4 a ‘3
° $ 2 =e 2 ¢ 2

> ro 2 c) 3S cy z 2 >

4 a A a -” a 2

Gy 3 a o a 3 a

> o a a Zz a a o >
useless useful
desirable undesirable
good —- -—-— SS oo sida
dull — — interesting
important _ —_— unimportant
worthless valuable

B. Please rate the unit you are now supervising on the following scales.

a
2 8 €
a 3S 8 &
! 2 > ry
s 8 8 Gy EA
= < o > w
Efficiency of the
work unit
Employee

satisfaction

Unit morale

36


Please indicate your likes and dislikes about supervisory/managerial work in the FAA, List
your comments in rank-order, the most important first and so on, Please make your comments
BRIEF and LEGIBLE. Answer first for FAA supervisory/managerial work at this office/facility,

then for such work in general within the FAA,
did This Office/Facility ----------*+-e+--+---7-

B.

B.

Cite 3 specific aspects of FAA supervisory/managerial work at this office/facility which
you like best.

()),

(2)

(3),

cite 3 specific aspects of FAA Supervisory/managerial work at this office/tacility which
you like least.

Q).

(2)

woe e eee eee ee eee In General- -------+---5-5-5-505550085
Cite 3 specific aspects of FAA supervisory/managerial work in general which you like best.

(D.

(2)

(G3)

cite 3 specific aspects of FAA supervisory/managerial work in general which you like least

(1).

(2).

(3).

22




LR
History of Unionism
Union Recognition
Employee and Union Rights
Unfair Labor Practices
Executive Orders

Management Theory

Management Identification
(Employee vs. manage-
ment orientation)

Management Style

Position Management

Position Classification

Pay/Time Administration
(overtime, compensatory
time, etc.)

Absence/Leave

Safety
Safety Responsibility
Injury Compensation
Staffing
Recruitment
Placement
Promotion
RIF Procedures

EEO

Reemployment and Return
Rights

Not at all

Slightly

Somewhat

Moderately

Quite

Very

Extremely

82


a
III

Answer the items in the next three sections according to how you felt before you
attended MIS.

A. I expected MIS to be:

2 > > 2
a a ue a cs
£ » a 2 <
g 3 < £ < 3 e
> 2 9 ‘e ra cy g 4 >
4 oe] a a a E oat ro
a 3 6 a ov a 6 3 Cy]
> o a a Zz a a & s
useless useful
important unimportant
good —_— —_— oo bad
dull interesting
worthless valuable
desirable undesirable

B, If you supervised the unit in which you are presently located prior to attending MIS,
please rate the unit on the following scales, If you did not supervise this unit
prior to attending MIS, omit these items and go on to III C (page 6).

Minimal
Acceptable
Good
| Very Good
| Excellent

Efficiency of work unit

|
|
|

Employee satisfaction

Unit morale

a
The items in this section were omitted from the questionnaires provided supervisors

and managers who had not attended MIS.

25



not been particularly effective in enabling them
to deal with labor problems, Clearly on the
“not helpful” side was the course content area
of Employee Development, the main issue being
that many graduates felt that too little attention
had been given to this subject. It was also ap-
parent that many supervisors and managers felt
the need for more training on general adminis-
trative procedures and practices,

TABLE 9, Summary of Statements Made by MIS Graduates Concerning

Netptut and Ineffective Aspects of MS

MES Helped MS Inettective
category
Supervisors Managers Supervisors Managers
1, Tacreaned Understanding A
Fron Course Concent 70.0% ona 96.5% a0,
1, Personal Growth rr 15.25, om 2st
LIZ, Working Throvgh Supervisory
Probleme Uich others on, S08 0.38 aoe
IV, Increased Aviaren
Supervisory/Managerial Role 5.9 ot 28 2.0
V. General MS Approach sneee 0.38 2k ot
VEL Applicability to Spect tte
FAA Progeams ween vee 3.6% aon
VII. Appropriateness to
‘Teatning of Supervtaors/
Managers oe oe hat Lot
100.0% 00.01 100,00 100,01
umber of Statements 1453 296 ors 208

fel in each of the even ated
At ts therefore possible for the
Tithe helped and ineffective headings.

* reccentages refer to proportion of
Categories, Note that the columne
toe category to have high percent

Of the other general aspects of MTS men-
tioned by the respondents to this portion of the
questionnaire, only one—Personal Growth—con-
tained a particularly notable percentage of re-
sponses, and these nearly always described some
particular self-development benefit of the pro-
gram. Many graduates (about 160 of the more
than 700 surveyed) felt that the course had
helped them to understand more about them-
selves. Others mentioned that their ability to
understand others and to cope with personal,
interpersonal, and professional problems had
increased.

b, Additional comments by graduates. As a
final area of concern, all respondents were pro-
vided an additional questionnaire that simply
asked them to add any comments they would
care to make about any aspect of MTS (Appen-
dix, VII.). Of 616 supervisors participating in
the survey, 277 (45 percent) made 509 statements
in this section; this response indicates a rather
high degree of involvement in the survey, since
this additional task was presented at the end of

a long and detailed questionnaire. Of 128 man-
agers, 48 (37 percent) responded; they contrib-
uted a total of 61 statements.

The statements focused on six general topics
(Table 10). The most frequent statement was
that the MTS should be continued. These state-
ments outnumbered suggestions to discontinue
the program by 18 to 1. There were also many
comments about specific course content areas that
should be modified in some manner (e.g., too
Jong, too short, change focus, etc.). A number
of responses suggested that high-level personnel
should attend MTS before subordinate managers
and supervisors; however, a number of other
respondents indicated a belief that the course is
really not appropriate for those with consider-
able experience. They believed that the course
should be required only for those who are enter-
ing supervisory or managerial positions. The
responses to the conduct of the refresher course
are interesting in that many of the respondents
look with favor on the notion of refresher train-
ing. The other types of comments (Table 10)
are self-explanatory and relate to matters pre-
viously discussed.

TABLE 10, Summary of Statenents Made by MIS Supervisory and

Managerial Graduates to Open-Ended Comments Question

Number of Statements
Topic of Statewent

Supervisors Managers
1, Continuation of HIS
Continue 1s W
Discontinue 8 2
IL. Recommendations for Changes in
Various Content Areas cy 5
THT, Atcendance at MS
Higher Level Perspnnel Should Go Firet 2 9
Good for Newly Appointed or Potential
Supervisors 36 3
other . 8 2
IV, Staff/Factlities/Location
Staff Caliber
28 3
Inadequate uw 1
Facilities
Excel lene n 1
Need Inprovenent 4 “
Location
Keep in Lawton 6 1
Move from Lawton 7 -
V. Refresher Course
Valuable Part of Program 36 3
Discontinue or Hove to Regions 8 =
VE, Length
Not Long Enough we 1
Too Long 25 2
VIZ. Miscellaneous 28 u

C. Conclusions. In this aspect of the MTS
evaluation, it was found that supervisors and
managers tended to share the same general



TABLE 13, Average Ratings of the Frequency of Desirable
Supervisory/Managertal Behavior
Average Average >
Rating® Rat ing Munber of Items Shoving
Group Before After =
ms Ms Increase Wo Change Decrease
MS Graduates
Supervisors 4.1 43 25 o °
Superiors 39 42 25 ° °
Eeployees 319 aa 22 ° °
Managers 42 4 26 5 °
‘Superiore 4.0 4.2 ese 0 °
Bap loyees 42 43 Es} 4 1
Not MIS Trained
Supervisors 4.2 -
Supertors 4.0 2 : :
Employees aio = :
Manpgers 42 - -
Superti ad = :
Employes 42 = = i

* Rating scale values: 1 = occurs not at all, 2 = occurs rarely, 3 = occurs
occasionally, 4 = occurs usually, 5 = occurs alnost

always

‘and chele superiors ansvered 25 behavioral items; employees of
answered 22 items. Managers and their superiors answered 31
1¢ employees of managers answered 28 itens,

managers not yet trained at MTS, while the
ratings for graduates after MTS attendance
tended to be higher than those for the nonat-
tendees by approximately the same amount.

Table 14 provides detailed data on each of the
items concerning behavior common to both super-
visors and managers. The types of supervisory/
managerial behavior that were consistently
associated with the highest ratings, in many
cases close to the “almost always” level, were
those that dealt with the fair and equal treat-
ment of employees. These items (12, 13, 14, and
15 in Table 14) received the highest ratings by
all the groups surveyed: both graduates and
nonattendees, superiors and employees. There
was also a very high degree of consistency in
the behavior area rated lowest; supervisors and
managers—both graduates of MTS and those
who had not yet attended—and their superiors
indicated that the effective administration of the
paperwork associated with the awards program
was a problem (item 24). The employees were
also highly consistent in giving their lowest rat-
ings to the item concerning the frequency with
which their supervisors provided them with in-
formation about needed areas of improvement
in their job performance (item 7).

In terms of the amount of change in super-
visory behavior, the graduates of MTS, their
superiors, and their employees tended to agree
that there was an increase in the occurrence of

12

performance counseling by supervisors (item 6)
following training at MTS. Supervisors and
their superiors also agreed that there tended to
be an increase in the occurrence of desirable be-
havior related to the development and training
of employees (items 18 and 19). As a group,
the supervisors themselves also tended to feel
that they were able to indicate areas of improve-
ment to their employees more often than they
did before attending MTS (item 7). Moreover,
their superiors indicated that an improvement
in administration of the awards program had
occurred (items 23 and 25). The employees in-
dicated that the greatest changes had occurred
in opportunities to freely discuss work problems
with their supervisors (item 81) and in their
understanding of their supervisors’ positions on
work-related matters (item 39).

For managers, the areas of greatest change
were somewhat different from those for super-
visors (Table 15). Managerial graduates of
MTS and their superiors agreed that application
of “management by objectives” (item 26), assess-
ment of performance in terms of objectives (item
27), and systematic definition of worb problems
(item 28) occurred more often after MTS at-
tendance. The group of managerial graduates
also reported that there was an increase in the
frequency with which they found ways of recog-
nizing excellence in their employees (item 23).
The superiors of the managers also indicated
that MTS had had an effect on the ability of
their subordinate managers to appropriately
judge the quality of the work of their (the
managers’) employees (item 8). According to
the employees of managerial MTS graduates,
the development of an understanding of other
FAA programs (item 17) improved most.

b. Comparisons between groups. In compar-
ing the various groups (MTS graduates, su-
periors, employees, etc.) on the specific items in
Table 14, several consistent trends in ratings
were noted. First, most of the individual items
for supervisors were rated higher by the super-
visors than by their superiors or employees.
This was also true for managers, except that the
number of items for which significant differences
were noted were fewer for managers than for
supervisors. However, superiors consistently
rated their subordinate supervisors and managers
higher on the item concerning freedom to discuss