### NOTICE: This publication is available digitally on the AFDPO WWW site at: http://www.e-publishing.af.mil. BY ORDER OF THE SECRETARY OF THE AIR FORCE AIR FORCE MANUAL 36-2236
12 NOVEMBER 2003

Personnel

GUIDEBOOK FOR AIR FORCE INSTRUCTORS

OPR: HQ AETC/ED (HQ AU/CFS, Dr M. Jenkins) Certified by: HQ AETC/ED (Lt Gen D. Lamontagne) Supersedes AFMAN 36-2236, 15 September 1994 Pages: 461 Distribution: F

This training manual presents basic teaching principles and their application in Air Force teacher-learning situations. It implements both AFPD 36-22, Military Training , and AFPD 36-23, Military Education . The text addresses how people learn and how they comm unicate. It discusses various teaching methods and techniques, and ways to evaluate learning and the reasons for such evaluation. The manual is for instructors engaged in on-the-job training (OJT) and informal instruction as well as those assigned to Air Force formal schools. Send comments and suggested improvements on AF Form 847, Recommendation for Change of Publication , through channels, to HQ AU/CFS, 60 Shumacher Street, Maxwell AFB AL 36112-5337. The use of a name or trademark of an y specific manufacturer, commercial product, commodity, or service in this publication does not imply endorsement by the Air Force. Maintain and dispose of records created as a result of processes prescrib ed in this publication in accordance with AFMAN 37-139, Records Disposition Schedul e.

S UMMARY OF REVISIONS This document is substantially revised and must be completely reviewed.
This revision adds a chapter on the experiential met hod of instruction and updates the examples for each teaching method.
Chapter 1—THE AIR FORCE INSTRUCTOR 20
1.1. Introduction. .............................................................................................................. . 20 1.2. Curriculum Planning. ................................................................................................. 20 Figure 1.1. Air Force Model for Instructional Systems Development. ....................................... 21
1.3. Instructor Skills Useful In Other Duties. ................................................................... 21 1.4. The Air Force Approach To Academic Instruction. .................................................. 21
1.5. Overview of Manual. ................................................................................................. 222 AFMAN36-2236 12 NOVEMBER 2003

1.6. Summary. ................................................................................................................... 23
Chapter 2—LEARNING THEORY 24
2.1. Introduction. .............................................................................................................. . 24
2.2. A Historical View. ..................................................................................................... 24 2.3. Combining the Approaches. ...................................................................................... 25 Figure 2.1. Levels of Knowle dge and Understanding. ................................................................ 26 Figure 2.2. Levels of Attitude and Values. ................................................................................. 27 Figure 2.3. Levels of Skill. ................................................................................................... ....... 28 2.4. Summary. ................................................................................................................... 28
Chapter 3—WRITING STUDENT-CENTE RED OBJECTIVES AND TESTS 29
3.1. Introduction. .............................................................................................................. . 29 3.2. Planning For Student-Centered Outcomes. ............................................................... 29 Table 3.1. Cognitive Taxonomy. ................................................................................................ 3 0 Table 3.2. Types of Learning. .................................................................................................. .. 31 Figure 3.1. Air Force Proficiency Code. ..................................................................................... 32 Figure 3.2. Comparison of Taxonomies of Learning Outcomes. ................................................ 33 3.3. Writing Lesson Objectives: ....................................................................................... 33 Figure 3.3. Sample Test Item. .................................................................................................. ... 36 3.4. Summary: ................................................................................................................... 37
Chapter 4—THE LEVEL-OF-LEARNING LESSON PLANNING PROCESS 39

Section 4A Overview 39
4.1. Introduction. .............................................................................................................. . 39 4.2. Bloom's Cognitive Taxonomy. .................................................................................. 39 Figure 4.1. Key Words for Knowledge, Compre hension, and Application Levels. ................... 40 Section 4B Learning Levels 40 4.3. Knowledge. ................................................................................................................ 4 0 4.4. Comprehension: ......................................................................................................... 40 Figure 4.2. Relationship Between Factor y Production and Unemployment. .............................. 41 4.5. Application. ............................................................................................................... . 42AFMAN36-2236 12 NOVEMBER 2003 3

Section 4C Level-of-Learning Objectives 43
4.6. Overview. .................................................................................................................. . 43
4.7. Student Centeredness. ................................................................................................ 43
4.8. LOL. ....................................................................................................................... .... 43 4.9. Specific Subject. ........................................................................................................ 44 4.10. Example of an LOL Objective: .................................................................................. 44 Figure 4.3. Example Level-of-L earning Objectives. ................................................................... 44 Section 4D Samples of Behavior 45
4.11. Overview. ................................................................................................................. .. 45 4.12. Defining a Sample of Behavior: ................................................................................ 45 Figure 4.4. Behavioral Verbs. .................................................................................................. ... 46 Figure 4.5. Recall of Information. ............................................................................................. .. 46 Figure 4.6. Understanding Information. ...................................................................................... 47 Figure 4.7. Applying Information. .............................................................................................. 47 4.13. Considerations in Writing Samples of Behavior: ...................................................... 47 4.14. Determining What To Sample. .................................................................................. 48 Figure 4.8. Air Force Curriculum. .............................................................................................. . 49 4.15. Summary. .................................................................................................................. . 49
Chapter 5—WRITING CRITERION OBJECTIVES 51

Section 5A Introduction 51
5.1. Overview. .................................................................................................................. . 51 5.2. Relationship Between Samp les of Behavior and Criterion Objectives. .................... 51 Figure 5.1. A Comparison of Samples of Be havior and Criterion Objectives. ........................... 52 Section 5B Constructing Criterion Objectives 52 5.3. Importance of Construction. ...................................................................................... 52
5.4. Essential Elements. .................................................................................................... 53 Figure 5.2. Essential Elements of a Criterion Objective. ............................................................ 53
5.5. Formats. ................................................................................................................... .. 53 Figure 5.3. Formats for Criterion Objectives. ............................................................................. 54 5.6. Mastery Learning. ...................................................................................................... 554 AFMAN36-2236 12 NOVEMBER 2003

5.7. Content Communicated by the Criterion Objective. ................................................. 55 Figure 5.4. Example 1 of a Criterion Objective. ......................................................................... 56 Figure 5.5. Example 2 of a Criterion Objective. ......................................................................... 56 Figure 5.6. Example 3 of a Criterion Objective. ......................................................................... 57 Figure 5.7. Example 4 of a Criterion Objective. ......................................................................... 58 Figure 5.8. Example 5 of a Criterion Objective. ......................................................................... 59 Figure 5.9. Example 6 of a Criterion Objective. ......................................................................... 60 Figure 5.10. Example 7 of a Criterion Objective. ......................................................................... 60 Figure 5.11. Example 8 of a Criterion Objective. ......................................................................... 61
5.8. Performance. .............................................................................................................. 6 3
5.9. The Action Verb. ....................................................................................................... 63 Figure 5.12. Verb Descriptors of Student Pe rformance (Criterion Objectives). ........................... 64 5.10. Object of the Action: .................................................................................................. 64
5.11. Common Errors. ......................................................................................................... 65 5.12. Summary. .................................................................................................................. . 65

Section 5C Standards 65
5.13. Overview. ................................................................................................................. .. 65 5.14. Types of Standards. .................................................................................................... 66
5.15. Common Errors. ......................................................................................................... 68 Section 5D Planning Tools 69 5.16. Task Steps As Planning Tools: .................................................................................. 69 Figure 5.13. Task Analysis of a Criterion Objective. .................................................................... 70 Figure 5.14. Sample Instruction Designs For a Task Analysis of a Criterion Objective. ............. 71 5.17. Summary. .................................................................................................................. . 72 Figure 5.15. Criterion Objective Writer's Checklist. ..................................................................... 72
Chapter 6—DEVELOPING THE LESSON PLAN 74 6.1. The Importance of a Lesson Plan: ............................................................................. 74 6.2. Determining the Objective. ........................................................................................ 74 6.3. Researching the Topic. .............................................................................................. 74 6.4. Selecting Instructional Methods. ............................................................................... 75AFMAN36-2236 12 NOVEMBER 2003 5

6.5. Lesson Planning Format. ........................................................................................... 76 Figure 6.1. Recommended Content and Co mponents of a Lesson Plan. .................................... 77 6.6. Organizing the Lesson. .............................................................................................. 77 Figure 6.2. Outline of a Lesson on Nonverbal Communication. ................................................ 81 6.7. The Strategy Statement: ............................................................................................. 82 Figure 6.3. Strategy Statement—Example 1. .............................................................................. 84 Figure 6.4. Strategy Statement—Example 2. .............................................................................. 85 Figure 6.5. Strategy Statement—Example 3. .............................................................................. 86 6.8. Choosing Support Material: ....................................................................................... 88 Figure 6.6. Choosing Appropriate Examples. ............................................................................. 90 6.9. Beginning and Ending the Lesson. ............................................................................ 92 6.10. Preparing The Final Outline: ..................................................................................... 94 Figure 6.7. Sample Keyword Outline – Nonverbal Communication. ......................................... 95 6.11. Summary. .................................................................................................................. . 96
Chapter 7—DEVELOPING KNOW LEDGE-LEVEL LESSONS 97
7.1. Introduction. .............................................................................................................. . 97 7.2. Appropriateness of Classroom Instru ction at the Knowledge Level. ........................ 97 7.3. Planning Knowledge-Level Lessons. ........................................................................ 98 7.4. Developing the Teaching Outline. ............................................................................. 98 Figure 7.1. Teaching Outline—Example 1. ................................................................................ 99 Figure 7.2. Teaching Outline—Example 2. ................................................................................ 101 Figure 7.3. Teaching Outline—Example 3. ................................................................................ 102 7.5. Support Material: ....................................................................................................... 102 7.6. Teaching the Lesson. ................................................................................................. 103 7.7. Summary. ................................................................................................................... 104
Chapter 8—DEVELOPING COMPREHENSION-LEVEL CONCEPT AND PRINCIPLE LESSONS 105
8.1. Introduction. .............................................................................................................. . 105 8.2. The Teaching of Concepts. ........................................................................................ 105 8.3. Organizing a Concept Teaching Plan. ....................................................................... 106 8.4. Testing Concept Learning. ......................................................................................... 1076 AFMAN36-2236 12 NOVEMBER 2003

8.5. Sample Concept Teaching Plans (Part I): .................................................................. 107 Figure 8.1. Sample Part I Lesson Plan. ....................................................................................... 1 08 Figure 8.2. Sample Lesson Plan Cover Sheet. ............................................................................ 109 8.6. The Teaching of Principles. ....................................................................................... 110 8.7. Logical Lesson Planning. ........................................................................................... 110 8.8. Testing Principle Learning: ....................................................................................... 112 8.9. Sample Plans for Teaching Principles (Part I—Cover Sheet). .................................. 112 Figure 8.3. Topical Pattern Sample. ............................................................................................ 113 Figure 8.4. Sample Multi-Objective Plan. ................................................................................... 114 Figure 8.5. Sample Problem-Solution Organizational Pattern. ................................................... 115 Figure 8.6. Sample Cause-Effect Organizational Pattern. .......................................................... 116 Figure 8.7. Sample Pro-Con Plus On e Organizational Pattern. .................................................. 117 8.10. Sampling Achievement of the Objective. .................................................................. 117 Figure 8.8. Sample 1—Topical Pattern. ...................................................................................... 118 Figure 8.9. Sample 2—Parallel Principles. ................................................................................. 119 Figure 8.10. Sample 3—Problem-Solution Pattern. ...................................................................... 120 Figure 8.11. Sample 4—Cause-Effect Pattern. ............................................................................. 120 Figure 8.12. Sample 5—Pro-Con Plus One Pattern. ..................................................................... 121 8.11. The Comprehension-Level Summary. ....................................................................... 121 Figure 8.13. Comprehension-Level Summary. ............................................................................. 122 8.12. Summary. .................................................................................................................. . 122
Chapter 9—DEVELOPING HIGHER-LEVEL COGNITIVE LESSONS 124
9.1. Introduction. .............................................................................................................. . 124 9.2. Higher Levels of Cognitive Learning: ....................................................................... 124 Figure 9.1. Instructional Objectives at the Application Level. ................................................... 125 Figure 9.2. Instructional Objectives at the Analysis Level. ........................................................ 126 Figure 9.3. Instructional Objectives at the Synthesis Level. ....................................................... 126 Figure 9.4. Instructional Objectives at the Evaluation Level. ..................................................... 127 9.3. Principles of Teaching and Learning at the Higher Levels. ...................................... 127 9.4. Lesson Planning. ........................................................................................................ 129AFMAN36-2236 12 NOVEMBER 2003 7

9.5. Summary. ................................................................................................................... 131
Chapter 10—WRITING AND MEASURING AFFECTIVE OBJECTIVES 132
10.1. Introduction. ............................................................................................................. .. 132 10.2. A Structure For Attitude Development—The Affective Taxonomy. ........................ 132 Figure 10.1. Krathwohl's Affective Taxonomy. ............................................................................ 134 10.3. Lesson Planning for Affective Development. ............................................................ 134 Table 10.1. Cognitive and Affect ive Consequences. ................................................................... 135 10.4. Relationship of Affective Domain to Other Domains. .............................................. 136 10.5. Affective Objectives Can be Specified and Measured. ............................................. 136 10.6. Techniques for Specifying and Measuring Affective Objectives. ............................. 136 Figure 10.2. Lesson Objective and Purpose Statement. ................................................................ 137 Figure 10.3. Lesson Objective and Outline of Main Teaching Points. ......................................... 137 10.7. Similarity of Cognitive and A ffective Learning Outcomes. ...................................... 138 10.8. Interpreting and Modifying Criterion Objectives. ..................................................... 138 Figure 10.4. Suggested Changes to Affective Objectives. ............................................................ 139 10.9. Stating Purely Affective Objectives. ......................................................................... 139 Figure 10.5. Main Point Sample. ................................................................................................ ... 140 Figure 10.6. Samples of Behavior. .............................................................................................. .. 140 Figure 10.7. Sample Lesson Plan. ............................................................................................... .. 141 10.10. Measuring Changes In Student Attitudes: ................................................................. 141 10.11. Using Published Attitude Scales. ............................................................................... 141 10.12. Making Our Own Attitude Assessment Scale. .......................................................... 142 10.13. Pretest-Posttest Design. ............................................................................................. 142 Figure 10.8. Attitude Test—Pretest-Posttest Design. ................................................................... 142 Figure 10.9. Multiple-Choice Format. .......................................................................................... 143 10.14. Measuring Attitude Changes Following the Experience. .......................................... 144 Figure 10.10. End-of-Course Critique—General. ........................................................................... 144 Figure 10.11. End-of-Course Critique—Specific. .......................................................................... 145 Figure 10.12. Sample School Criterion Objective. ......................................................................... 145 Figure 10.13. Either-Or Format. ................................................................................................ ...... 1468 AFMAN36-2236 12 NOVEMBER 2003

Figure 10.14. Open-Ended Question Format. ................................................................................. 146 10.15. Dealing With the Higher Affective Levels. ............................................................... 146
10.16. Which Affective Measure Should Be Used? ............................................................. 147 10.17. Summary. ................................................................................................................. .. 147
Chapter 11—USING QUESTIONS FOR LEARNING 148
11.1. Introduction. ............................................................................................................. .. 148 11.2. The Starting Point. ..................................................................................................... 14 8 11.3. Determine Purpose. .................................................................................................... 149 11.4. Questions Categorized By Direction. ........................................................................ 150 11.5. Effective Questioning Techniques. ............................................................................ 151 11.6. Summary. .................................................................................................................. . 153
Chapter 12—SURVEY OF TEACHING METHODS 154
12.1. Introduction. ............................................................................................................. .. 154
12.2. Presentational Methods. ............................................................................................. 154
12.3. Student Verbal Interaction Methods. ......................................................................... 156 12.4. Application Methods. ................................................................................................. 157 12.5. Summary. .................................................................................................................. . 158
Chapter 13—THE TEACHING LECTURE METHOD 159
13.1. Introduction. ............................................................................................................. .. 159 13.2. Types of Lectures. ...................................................................................................... 15 9 13.3. Other Types of Oral Presentations. ............................................................................ 159 13.4. Advantages and Disadvantages of the Lecture Method. ............................................ 160 13.5. Preparing the Lecture. ................................................................................................ 161 13.6. Presenting the Lecture. .............................................................................................. 164 Figure 13.1. Guidelines for Manuscript Lessons. ......................................................................... 165 13.7. Summary: .................................................................................................................. . 171
Chapter 14—THE GUIDED DISCUSSION METHOD 172
14.1. Introduction. ............................................................................................................. .. 172 14.2. Selection and Planning Factors. ................................................................................. 172 14.3. Organizing the Guided Discussion. ........................................................................... 174 14.4. Conducting a Guided Discussion. .............................................................................. 176AFMAN36-2236 12 NOVEMBER 2003 9

14.5. Post-Discussion Actions. ........................................................................................... 179 14.6. Summary. .................................................................................................................. . 179
Chapter 15—THE CASE STUDY METHOD 181
15.1. Introduction. ............................................................................................................. .. 181 15.2. Using Case Studies. ................................................................................................... 181 15.3. Teaching Value and Limitations: ............................................................................... 181 15.4. Typical Case Applications. ........................................................................................ 182 15.5. Types of Cases. .......................................................................................................... 1 83
15.6. Mode of Presentation: ................................................................................................ 184 15.7. Case Methodology. .................................................................................................... 184 Figure 15.1. Lesson Plan Process. .............................................................................................. ... 187 15.8. Summary. .................................................................................................................. . 189
Chapter 16—THE TEACHING INTERVIEW METHOD 191
16.1. Introduction. ............................................................................................................. .. 191 16.2. Background: ............................................................................................................... 191 16.3. Uses of the Teaching Interview. ................................................................................ 191 16.4. Initial Planning Factors. ............................................................................................. 192
16.5. Coordination. ............................................................................................................. 193
16.6. The Interview Lesson: ............................................................................................... 194 16.7. Introducing the Interview Lesson. ............................................................................. 194 16.8. Conducting the Interview. .......................................................................................... 194 16.9. Ending the Interview. ................................................................................................. 196 16.10. Sample Lesson Plan for a Teaching Interview. ......................................................... 196 16.11. Summary. ................................................................................................................. .. 196
Chapter 17—THE DEMONSTRATIO N-PERFORMANCE METHOD 197
17.1. Introduction: ............................................................................................................. .. 197 17.2. Planning and Developing a D-P Lesson. ................................................................... 197 17.3. Factors to Consider When Using the D-P Method: ................................................... 199 17.4. The Sample D-P Lesson Plan: ................................................................................... 200 17.5. Summary. .................................................................................................................. . 20010 AFMAN36-2236 12 NOVEMBER 2003

Chapter 18—USING GAMES FOR LEARNING 201
18.1. Introduction. ............................................................................................................. .. 201 18.2. Benefits of Using Games. .......................................................................................... 201 18.3. Classifying Games. .................................................................................................... 201
18.4. Various Games for Classroom Use. ........................................................................... 202 18.5. Designing a Game. ..................................................................................................... 203 18.6. Writing Game Questions. .......................................................................................... 203 18.7. Writing a Game Lesson Plan. .................................................................................... 203 18.8. Administering the Game. ........................................................................................... 204 18.9. Cautions on Using Games. ......................................................................................... 204 18.10. General Considerations When Using Games. ............................................................ 205
18.11. Dr. Sivasailam Th iagarajan's Considerations. ........................................................... 205 Figure 18.1. Dr. Thiagarajan's Experiential Checklist. ................................................................. 206 18.12. Summary. ................................................................................................................. .. 206
Chapter 19—SELECTING TEACHING METHODS 207
19.1. Introduction. ............................................................................................................. .. 207 19.2. Assumptions. .............................................................................................................. 208 Table 19.1. Teaching Methods Grid or Decision Table. .............................................................. 209
19.3. Definition of Terms. .................................................................................................. 210 19.4. Recommendations for Teaching Methods. ................................................................ 212
19.5. Lecture: .................................................................................................................. .... 212
19.6. Indirect Discourse: ..................................................................................................... 21 3
19.7. Demonstration-Performance: ..................................................................................... 214 19.8. Reading: .................................................................................................................. ... 215
19.9. Self-Paced Methods: .................................................................................................. 216 19.10. Questioning: ............................................................................................................. .. 218
19.11. Nondirected Discussion: ............................................................................................ 219
19.12. Guided Discussion: .................................................................................................... 220 19.13. Practical Exercises—Individual Projects: .................................................................. 221 19.14. Practical Exercises—Field Trips: .............................................................................. 222 19.15. Practical Exercises—Experiential. ............................................................................. 223
19.16. Case Studies: ............................................................................................................ .. 224AFMAN36-2236 12 NOVEMBER 2003 11

19.17. Summary. ................................................................................................................. .. 226
Chapter 20—INSTRUC TIONAL MEDIA 227
20.1. Introduction. ............................................................................................................. .. 227
20.2. Instructional Aids: ...................................................................................................... 2 27 20.3. Selecting Instructional Aids. ...................................................................................... 227 20.4. Types of Instructional Aids. ....................................................................................... 228 20.5. Proofreading. ............................................................................................................. . 230 20.6. Distance Learning. ..................................................................................................... 230 20.7. Summary. .................................................................................................................. . 230
Chapter 21—INTRODUCTIO N TO EVALUATION 231
21.1. Introduction. ............................................................................................................. .. 231
21.2. Nature of Evaluation. ................................................................................................. 231
21.3. How We Evaluate. ..................................................................................................... 232
21.4. Characteristics of Evaluation. .................................................................................... 232 21.5. Determining Means of Measurement. ....................................................................... 236
21.6. Reasons for Evaluation. ............................................................................................. 236 21.7. Nature of Required Behavior: .................................................................................... 236 21.8. Availability of Time and Facilities. ........................................................................... 237 21.9. Availability of Qua lified Faculty: .............................................................................. 237 21.10. Summary. ................................................................................................................. .. 237
Chapter 22—CONSTRUCTING AND ADMINISTERING CLASSROOM TESTS 239
22.1. Introduction. ............................................................................................................. .. 239 22.2. Suggestions for Preparing Any Test Item. ................................................................. 239
22.3. Selection Test Items. .................................................................................................. 239 Table 22.1. Comparison Between the Two Major Types of Test Questions. .............................. 240 22.4. Multiple-Choice Items. .............................................................................................. 241 Figure 22.1. Stems. ............................................................................................................ ............ 241 Figure 22.2. Alternatives. ..................................................................................................... ......... 242 Figure 22.3. Multiple-Choice Alternatives. .................................................................................. 24 2 Figure 22.4. Stem Written As a Question. .................................................................................... 24 3 Figure 22.5. Stem Written As an Incomplete Statement. .............................................................. 24312 AFMAN36-2236 12 NOVEMBER 2003

Figure 22.6. Multiple Response. ................................................................................................ ... 243 Figure 22.7. Definition. ....................................................................................................... .......... 244 Figure 22.8. Stem Supplemented by an Illustration. ..................................................................... 244 Figure 22.9. The Negative Variety. ............................................................................................. .. 245 Figure 22.10. Association. ..................................................................................................... .......... 245 Figure 22.11. Problem-Situation. ............................................................................................... ..... 245 Figure 22.12. Example of a Problem-Situation Format. ................................................................. 246 Figure 22.13. Checklist for Writing Multiple-Choice Items. .......................................................... 246 Figure 22.14. Multiple-Choice Test Item. ....................................................................................... 247 Figure 22.15. Corrected Item. .................................................................................................. ....... 248 22.5. Matching Test Items. ................................................................................................. 248 Figure 22.16. Example of Equal Columns. ..................................................................................... 24 9 Figure 22.17. Example of Unequal Columns. ................................................................................. 250 Figure 22.18. Example of Inference Matching. .............................................................................. 250 Figure 22.19. Example of Cause-and-Effect Matching. .................................................................. 251 Figure 22.20. Example of Coded Alternativ es for Scientific Reasoning. ....................................... 251 Figure 22.21. Sample Matching Test Item. ..................................................................................... 2 52 Figure 22.22. Checklist for Writing Matching Test Items. ............................................................. 252 Figure 22.23. Questions to Ask. ................................................................................................ ...... 253 Figure 22.24. Corrected Sample Matching Test. ............................................................................ 253
22.6. True-False Items. ....................................................................................................... 25 4 Figure 22.25. Declarative Statement. ........................................................................................... ... 254 Figure 22.26. Cluster. ......................................................................................................... ............. 255 Figure 22.27. Statement with Illustration. ..................................................................................... .. 255 Figure 22.28. Double True-False. ............................................................................................... .... 255 Figure 22.29. Checklist for Writing True-False Items. ................................................................... 256 Figure 22.30. Sample True-False Item. .......................................................................................... . 256 Figure 22.31. Corrected Item. .................................................................................................. ....... 257 22.7. Supply Test Items. ..................................................................................................... 257 22.8. Completion Items. ...................................................................................................... 257AFMAN36-2236 12 NOVEMBER 2003 13

Figure 22.32. Example 1 – Incomplete Statement. ......................................................................... 258 Figure 22.33. Example 2 – Sentence or Paragraph Completion. .................................................... 259 22.9. Checklist for Writing Completion Items. .................................................................. 259 Figure 22.34. Checklist for Writing Completion Items. ................................................................. 259 Figure 22.35. Sample Item. ..................................................................................................... ........ 259 Figure 22.36. Corrected Item. .................................................................................................. ....... 260
22.10. Short-Answer Items. .................................................................................................. 260 Figure 22.37. Direct Question Example. ......................................................................................... 261 Figure 22.38. Association. ..................................................................................................... .......... 261 Figure 22.39. Short Explanation Example. ..................................................................................... 2 61 Figure 22.40. List Example. .................................................................................................... ........ 262 Figure 22.41. Picture or Diagram Example. .................................................................................... 2 62 22.11. Essay Items. ............................................................................................................. .. 262 Figure 22.42. Guided Essay Item. ............................................................................................... .... 265 22.12. Checklist for Writing Essay or Short Answer Questions. ......................................... 265 Figure 22.43. Checklist for Writing Short-Answer and Essay Items. ............................................. 266 Figure 22.44. Short Answer or Essay Item. .................................................................................... 2 66 Figure 22.45. Corrected Essay Test Item. ....................................................................................... 267 22.13. Assembling the Test. .................................................................................................. 267 22.14. Administering the Test. .............................................................................................. 267 22.15. Scoring Supply Items. ................................................................................................ 268 22.16. Scoring Essay Test Items. .......................................................................................... 268 Figure 22.46. Sample Essay Test Question. .................................................................................... 2 70 Figure 22.47. Sample Scoring Key for Essay Test Items. ............................................................... 270 22.17. Summary. ................................................................................................................. .. 271
Chapter 23—MEASURING LEARNING OUTCOMES 272
23.1. Introduction. ............................................................................................................. .. 272
23.2. Overview and Review. ............................................................................................... 272 Figure 23.1. Sample Knowledge-Level Test Question. ................................................................ 273 Figure 23.2. Sample Comprehension-Level Test Question. ......................................................... 27414 AFMAN36-2236 12 NOVEMBER 2003

Figure 23.3. Sample Test Questions at the Comprehension Level. .............................................. 275 23.3. Test-Writing Process. ................................................................................................. 276 23.4. Summary. .................................................................................................................. . 277
Chapter 24—EVALUATION BY RATING 278
24.1. Introduction. ............................................................................................................. .. 278 Figure 24.1. Rating Scale. ..................................................................................................... ........ 278 24.2. Rating Methods. ......................................................................................................... 27 9 Figure 24.2. Descriptive Scale for Rating Student Performance. ................................................. 279 Figure 24.3. Examples of Typical Forms of the Graphic Scale. ................................................... 281 Figure 24.4. Checklist for Rating Profic iency in Instrument Flying. ............................................ 282 Figure 24.5. Product Scale. .................................................................................................... ....... 283 Figure 24.6. Single Block of Forced-Choice Instrument. ............................................................. 284 24.3. Constructing the Rating Device. ................................................................................ 285 Figure 24.7. Example of a Rating System for Two Duty Assignments. ....................................... 287 24.4. Common Rating Errors. ............................................................................................. 287 24.5. Classifying Rating Errors. .......................................................................................... 287 Figure 24.8. Error of Ce ntral Tendency. ....................................................................................... 288 Figure 24.9. Error of Standards. ............................................................................................... ..... 289 Figure 24.10. Halo or Horns Error. ............................................................................................. .... 290 Figure 24.11. Logical Error Ratings. ........................................................................................... .... 291 24.6. Improving Ratings. .................................................................................................... 291 24.7. Summary. .................................................................................................................. . 291
Chapter 25—CRITERION-REFERENCED EVALUATION 292
25.1. Introduction. ............................................................................................................. .. 292 25.2. A CRT Reporting Format. ......................................................................................... 292 Figure 25.1. Simple Reporting Format for CRT Test Data. .......................................................... 293 Figure 25.2. Detailed Reporting Fo rmat for CRT Data. ............................................................... 294 Figure 25.3. Summary Table of Mastery or Nonm astery by Curriculum Objectives. .................. 294 25.3. CRT Item Analysis. ................................................................................................... 296 Figure 25.4. Ease Index Formula. ............................................................................................... .. 297AFMAN36-2236 12 NOVEMBER 2003 15

Figure 25.5. Percent of Correct Responses to Test Items. ............................................................ 298 25.4. Group CRT Data as a Management Tool. ................................................................. 299
25.5. Characteristics of CRT Evaluation. ........................................................................... 299 Figure 25.6. Three Usable Approaches to Determine Validity. .................................................... 300 Figure 25.7. Possible Problems Causing Lack of Differentiation. ................................................ 301 25.6. Summary. .................................................................................................................. . 301
Chapter 26—NORM-REFE RENCED ANALYSIS 303

Section 26A Overview 303
26.1. Introduction. ............................................................................................................. .. 303 Section 26B Grading Methods 303
26.2. Overview. ................................................................................................................. .. 303 Figure 26.1. Tabulation of a Fre quency Distribution. ................................................................... 304
26.3. Rank-Order Grades. ................................................................................................... 304 Figure 26.2. Computation of Rank-Order Grades. ........................................................................ 305
26.4. Percentile-Rank Grades. ............................................................................................ 306 Table 26.1. Computation of Percentile-Rank Grade. ................................................................... 307 26.5. Percentile-Group Grades. ........................................................................................... 308 Table 26.2. Computation of Several Perc entile-Group Grades with Different Percentage Groupings. ................................................................................................................ 30 9
26.6. Arithmetic Mean. ....................................................................................................... 310
26.7. Standard Deviation. ................................................................................................... 310 Table 26.3. Computation of Mean, Standard Devi ation, Standard Scores, and T-Scores. .......... 311
26.8. Standard Score. .......................................................................................................... 3 12
26.9. T-Scores. ................................................................................................................. ... 312 26.10. Grading Methods Compared. ..................................................................................... 313 Figure 26.3. Raw Scores Made by Two Students. ........................................................................ 313 Figure 26.4. Rank-Order Scores Made by Same Two Students. ................................................... 314 Figure 26.5. Percentile-Rank Scores Made by Same Two Students. ............................................ 314 Figure 26.6. T-Score Grades Made by Same Two Students. ........................................................ 315 Figure 26.7. Comparison of Grading Methods. ............................................................................ 31616 AFMAN36-2236 12 NOVEMBER 2003

Figure 26.8. Weighted Scores Made by Same Two Students. ...................................................... 317

Section 26C Effectiveness of Individual Test Items 317
26.11. Ease Index of Test Items. ........................................................................................... 317 26.12. Simple Formula for Ease Index. ................................................................................ 317 Figure 26.9. Distribution of Responses to Test Item. .................................................................... 318 26.13. Interpreting the Ease Index: ....................................................................................... 318 26.14. Differentiation Index (D I) of Test Items. ................................................................... 319 26.15. Computing Differentiation Index. .............................................................................. 319 26.16. Interpreting the Differentiation Index. ....................................................................... 320 26.17. Analyzing All Responses To a Test Item. ................................................................. 320 Figure 26.10. Analysis of All Responses To a Test Item. ............................................................... 321 26.18. Summary. ................................................................................................................. .. 322
Chapter 27—USING FEEDBACK IN THE CLASSROOM 323
27.1. Introduction: ............................................................................................................. .. 323 27.2. Definition and Theory. ............................................................................................... 323 27.3. Giving Feedback to Students. .................................................................................... 324
27.4. Characteristics of Effective Feedback: ...................................................................... 324 27.5. Methods of Giving Feedback. .................................................................................... 326 Figure 27.1. Ground Rules To Aid the Instructor in Giving Feedback. ........................................ 328 27.6. Summary. .................................................................................................................. . 328
Chapter 28—STUDENT DIFFERENCES 329
28.1. Introduction. ............................................................................................................. .. 329
28.2. Match the Method to the Student. .............................................................................. 329 28.3. Match Students to the Planned Learning Experience. ............................................... 329 28.4. Combining the Two Approaches. .............................................................................. 330 28.5. Group Differences. ..................................................................................................... 330
28.6. Individual Differences. .............................................................................................. 332 Figure 28.1. Techniques for Motivating Students. ........................................................................ 334 Figure 28.2. Benefits Associated With Grading System. .............................................................. 336 Figure 28.3. Elements Found in Learning Contracts. ................................................................... 338 Figure 28.4. Personalized Instruction. ......................................................................................... .. 340AFMAN36-2236 12 NOVEMBER 2003 17

28.7. Summary. .................................................................................................................. . 341
Chapter 29—THE DYNAMICS OF SMALL LEARNING GROUPS 342

Section 29A Overview 342
29.1. Introduction. ............................................................................................................. .. 342 29.2. Small Learning Group Defined. ................................................................................. 342 Section 29B Small-Group Learning Influences 342
29.3. Benefits of Sma ll-Group Learning. ........................................................................... 342
29.4. Norms. .................................................................................................................... .... 343
29.5. Cohesiveness. ............................................................................................................. 343 29.6. Increasing Cohesiveness. ........................................................................................... 344
29.7. Consensus. ................................................................................................................ . 344

Section 29C Roles of Class Me mbers and Instructors 345
29.8. Class Member Roles. ................................................................................................. 345 29.9. Dealing with Nonproductive Roles. ........................................................................... 346 29.10. Dealing with Nonproductive Groups. ........................................................................ 347
29.11. Student Needs. ........................................................................................................... 347
29.12. The Instructor As Leader. .......................................................................................... 348 Figure 29.1. Instructor Task and Maintenance Roles. ................................................................... 348 29.13. Summary. ................................................................................................................. .. 349
Chapter 30—THE INSTRUCTOR AS A HELPER 350

Section 30A Overview 350
30.1. Introduction. ............................................................................................................. .. 350 30.2. Counseling Defined. .................................................................................................. 350 30.3. Counseling as a Process. ............................................................................................ 350 30.4. The Helping Relationship. ......................................................................................... 351 30.5. Improvement or Change in Student Behavior. .......................................................... 351 30.6. Student Requests for Assistance. ............................................................................... 351 Section 30B The Helping Relationship 351 30.7. Importance of the Helping Relationship. ................................................................... 351 30.8. The Counseling Environment. ................................................................................... 35118 AFMAN36-2236 12 NOVEMBER 2003

30.9. Stages of a Helping Interview. ................................................................................... 352
30.10. Instructor Attitudes. ................................................................................................... 3 53 30.11. Interviewing Skills. .................................................................................................... 3 54 Section 30C Counseling Approaches 355
30.12. Overview. ................................................................................................................ ... 355 30.13. Cognitive Approach. .................................................................................................. 356 30.14. Behavioral Approach. ................................................................................................ 357 30.15. Affective Approach. ................................................................................................... 357

Section 30D Referrals and Communications 358
30.16. Referral. ................................................................................................................ ..... 358 30.17. Communications Between Instructors and Students in a Counseling Environment: 359 30.18. Summary. ................................................................................................................. .. 359
Chapter 31—SELF-CONCEPT 361
31.1. Introduction. ............................................................................................................. .. 361 31.2. Description of the Self-Concept. ............................................................................... 361
31.3. Formation of the Acad emic Self-Concept. ................................................................ 362 31.4. Self-Concept Defense Mechanisms. .......................................................................... 364 31.5. Enhancing the Self-Concept: ..................................................................................... 365 31.6. Suggestions for Improving Self-Concepts: ................................................................ 365 31.7. Summary. .................................................................................................................. . 367AFMAN36-2236 12 NOVEMBER 2003 19

Attachment 1—GLOSSARY OF REFERENCES AND SUPPORTING INFORMATION 368

Attachment 2—EXAMPLE LESSON PLAN FOR FORMAL LECTURE METHOD 389

Attachment 3—EXAMPLE LESSON PLAN FOR INFORMAL LECTURE METHOD 396

Attachment 4—EXAMPLE LESSON PLAN FOR GUIDED DISCUSSION METHOD PROBLEM-SOLUTION PATTERN 404

Attachment 5—EXAMPLES OF CASE METHOD OBJECTIVES WITH CASE ELEMENTS 412

Attachment 6—EXAMPLE LESSON PLAN FOR CASE STUDY METHOD PROBLEM-SOLUTION PATTERN 414

Attachment 7—EXAMPLE LESSON PLAN FOR CASE STUDY METHOD TOPICAL PATTERN 425

Attachment 8—EXAMPLE LESSON PLAN FOR TEACHING INTERVIEW 434

Attachment 9—EXAMPLE LESSON PLAN FOR DEMONSTRATION-PERFORMANCE METHOD 442

Attachment 10—EXAMPLE LESSON PLAN FOR METHOD OF INSTRUCTION LEARNING GAME 449

Attachment 11—FEEDBACK FOR STUDENT SPEAKING 458

Attachment 12—GUIDELINES FOR PROVIDING FEEDBACK ON STUDENT WRITING 46020 AFMAN36-2236 12 NOVEMBER 2003

Chapter 1

THE AIR FORCE INSTRUCTOR

1.1. Introduction. This manual provides the practical information needed to teach adult students. While it applies to most adult education situations, th e emphasis is on a military setting. The manual was written and tested at the Air Force's Academic Instruct or School at Maxwell Air Force Base, Alabama. 1.1.1. As a new instructor, you will find the manual useful because it summarizes the best of what experts in education have written. If you have been to a library for material on teaching recently, you were probably bewildered by the array of t opics and the number of specialized texts.
1.1.2. This manual concentrates on the "academic" cl assroom. While it is hard to clearly distinguish between technical and academic instruction, you might find the follo wing comparison useful. Technical instruction normally includes a much higher pr oportion of the full range of theory and skills the graduate will need. When technical students are teste d, the instructor can be sure of what they can do; the instructor has "proof" they have mastered the course material. In academic instruction, the course usually has a more general range of possible skills , objectives, and content. Unlike technical instruction, the course often has no "cap," and students are encouraged to go beyond the minimum stated objectives. Instead of having technical instruction's "proof" of learning, the academic classroom often has only samples of learning as evidence. Instructor s must often be satisfied with a sampling process when designing individual class periods and evaluati ng student achievement. The process of sampling is further discussed in Chapter 3 and Chapter 4 (planning lessons) and Chapter 21 through Chapter 25 (evaluation).
NOTE: Do not expect to learn everything you need to know about teaching solely from a manual. Extensive as this manual may appear, it cannot provide you with the final stages of the process—teaching and being judged on your teaching skills. Even experience d instructors can profit from constructive feedback—from other teachers and students—on their lesson preparation and presentation.
1.2. Curriculum Planning. The official Air Force process for curriculum planning is Instructional System Development (ISD). ISD is a deliberate and orderl y but flexible process for planning, developing, and managing high quality instructional pr ograms. It is a sound approach which has been successful in a variety of civilian and military contexts. Figure 1.1. is a model of the functions and phases of ISD. The five functions of ISD are: management, support, adminis tration, delivery, and eval uation. The four phases in the ISD process are: analysis, design, development, and implementation. All of the functions and phases occur within a quality environment that has both external and internal constraints imposed upon it.AFMAN36-2236 12 NOVEMBER 2003 21

Figure 1.1. Air Force Model for Instructional Systems Development.

1.2.1. These functions and phases are described in detail in AFMAN 36-2234, Instructional Systems Development , and AFH 36-2235, Information for Designers of Instructional Systems, Volumes 1 through 11. This does not duplicate these publications but complements them by spelling out the instructor's tasks and respons ibilities related to cl assroom work. Use AFMAN 36-2234 and AFH 36-2235 when planning curriculum. 1.2.2. The phases in the ISD process are used as an organizing theme in this manual. Understanding the four phases will help you see how your efforts fit into the total curriculum or "instructional system" for your school or course. When planning i ndividual lessons, be aware of how the school determined the objectives and join in the process of validating them. When evaluation results are in, studythem for possible use in modifying the course. Not all instructors have the same degree of involvement in curriculum planning. Nor is it intended that reviewing this manual would qualify you as an ISD specialist. Courses for specialists in ISD more fully explain the principles and techniques presented in AFMAN 36-2234 and AFH 36-2235.

1.3. Instructor Skills Useful In Other Duties. Military personnel often comment on the value of instructor training for more general duties as an NCO oran officer. Air Force supervisors, like instructors, must give briefings, communicate in groups, counsel subordinates, and evaluate performance. Students or graduates of the resident course often comment on the value of areas such as counseling and human relations in assignments other than instructing. While you may find material in this manual helps develop such skills, it is included primarily because it is necessary for the successful instructor.
1.4. The Air Force Approach To Academic Instruction. A major theme of this manual is the focus on the students, rather than the instructors or the subj ect matter. This focus requires student-centered objectives and conscious attention to how the students react to the instruction received. For efficient instruction, students need feedback that reinforces learni ng while identifying and correcting errors. Students22 AFMAN36-2236 12 NOVEMBER 2003

need the opportunity to try what has been taught. Too often, instruction is limited to the delivery of information, either through reading assignments, lectures, films, or type 0 and type 1 computer-based training. Academic instruction should allow adult learners to practice what has been taught, receive feedback on their performance, and incorporate improve ment as they move on to new material.
1.4.1. Evidence of Successful Teaching. In the approach presented in this manual, the only acceptable evidence that successful teaching has taken place comes from indications of change in student behavior. As important as good texts, lectures, and audiovisual materials are, they are only part of the process. At least as important is feedback to the student in the form of encouragement, comments on written papers, correction of mistake s, and similar teaching activities. Instructors must plan opportunities for observing student work so they can provi de timely feedback. Feedback in technical courses is common but is often overlooked in academic classr ooms. In addition to feedback from the instructor, a well-designed course can include features such as self-tests and evaluation checklists that allow "self-correction" by the students.
1.4.2. Creativity in the Classroom. Creativity is the imaginative combining of known elements into something new and useful. One of the charges often made against military instruction is that it is too traditional and lacks creativity; dry-as-dust presenta tions are more like briefings than real teaching lectures. Instructors who are stiff and expressionless fail to commun icate fully, forcing students to memorize and recite low-level information. Hopefully, these type of military classes are largely things of the past. However, we must guard against becoming complacent. Modern technology may turn our well-intended teaching efforts into briefings or "talking heads" presentations. As such, we must constantly be aware of the barriers to our creativity.
1.4.3. Barriers to Creativity. A barrier is something that inhibits our ability to be free, natural, and spontaneous, thereby decreasing the opportunity for creative ideas. Recognizing and understanding various barriers is the first step in overcoming them. 1.4.3.1. Experts have identified four broad categories of barriers to creativity—inertia, fear, prejudice, and habit. Inertia causes many of us to continue doing what we are now doing unless we are challenged. Instructors who are a fraid to be different may avoid attracting attention by leaving things as they are. Instructors who are prejudice d reject new approaches and methods because they automatically prejudge anything new without trying it . Habit is a very real problem for Air Force instructors. Courses may be taught the same wa y time after time—not beca use the course is a good one, but because "that's how it's always been done." 1.4.3.2. Air Force instructors, then, should be creative instructors, who know when to be guided by time-tested methods and when to strike out boldly in new directions.

1.5. Overview of Manual. This manual should be read from chapter to chapter in order. This allows the new instructor to benefit most from what this manual has to offer. Many of the chapters build on information presented earlier. You may find it difficult to unde rstand a particular chapter if read out of sequence; but if you know what you are looking for, you can go directly to the appropriate chapter. 1.5.1. Lesson Planning chapters ( Chapter 2 through Chapter 6 , and Chapter 10 ) introduce the theory and practice of writing student-c entered objectives for academic instruction. The system of lesson planning combines both behavioral and cognitive learning theories. Because student behavior in the academic setting represents more than what simply meets the eye, it is important to plan for changesAFMAN36-2236 12 NOVEMBER 2003 23

in the level of student learning. The techniques pres ented will prepare you to plan for lessons designed to develop skills, knowle dge, and/or attitudes. 1.5.2. Lesson Development chapters ( Chapter 7 through Chapter 9 ) develop, organize, and support academic lessons. We must plan carefully to achieve different levels of knowledge, skills, and attitudes. Several approaches and patterns of lesson development prom ote student learning. Regardless of how well we write learning objectives, poor lesson development can hinder or even prevent desired learning outcomes.
1.5.3. While the academic instructor has a wide variety of methods available, Instructional Media and Methods chapters ( Chapter 11 through Chapter 20 ) focus on several of the most valuable—TeachingLecture, Guided Discussion, Case Study, Teaching Interview, Demonstration-Performance, and Experiential Method of Instruction. Chapter 19 , Selecting Teaching Methods, presents other methods in less detail. In addition, the questioning chapter fo cuses on the skills of classroom questioning which are fundamental to nearly all methods of instruction. 1.5.4. The Evaluation chapters ( Chapter 21 through Chapter 27 ) introduce the two major approaches to evaluation— Norm Referenced (comparing individual student performance to that of the group) and Criterion Referenced (comparing individual student performance to a predetermined objective or standard). These chapters introduce several ways to measure student performance and discuss the human element involved in provi ding meaningful feedback to students.
1.5.5. Chapter 28 through Chapter 31 deals with the learner rather than the learning process. No instructional system can succeed unless student need s are taken into account. We must address these needs whether we are planning for group or individua lized learning. Instructors have a responsibility to establish a helping relationship with students.
1.6. Summary. This manual is a fresh rethinking of a co mplex and not completely understood subject— how to teach in the academic classroom so Air Force people will be more effective on the job. No book on this subject can have all the answers. But those who study this manual carefully will know the standard terminology and be able to ask and answer many of th e right questions. With the help of this manual, instead of improvising in the classroom, we can approach our teaching j ob more systematically.24 AFMAN36-2236 12 NOVEMBER 2003

Chapter 2

LEARNING THEORY

2.1. Introduction. Learning means many things to many people. To some, learning is simply a change in behavior as a result of experience; to others, it is something more difficult to see and measure which involves changing thinking processes. We learn to name objects as children and solve complex problems as adults. We learn attitudes and values. We can evenlearn to learn by improving study skills. But, how do we explain learning? Can any one theory of learning explain how we have learned all of what we can do, what we know, and why our attitudes and values are wh at they are? More importantly, what do we need to know about learning theory to be effective Air Force instructors?

2.2. A Historical View. Many theories try to explain how we learn, but psychologists and educators do not totally agree on any one of them. Most agree, ho wever, that learning is best explained by one or a combination of two theories: behaviorism or cognitive theory.
2.2.1. Behaviorism. We know that all animals can learn. Many psychologists and educators believe that all animals, including humans, learn in about th e same way. Behaviorists believe that we learn by having our behavior reinforced, and th at our behavior can be shaped or controlled by external stimuli.In the classroom, the external stimuli are the instructors. If what we do as students is positively reinforced, we will learn to do it better and more often. If we get no reinforcement, or are punished for something we do, we tend to stop doing it. Therefore, in general, instructors can shape or modify a student's behavior by rewarding (encouraging the behavior) or punishing (inhibiting the behavior). Ithas been shown that positive reinforcement is a very strong tool for increasing a behavior. This suggests that, as instructors, we should reward the behavior students exhibit that we wish to see them continue to perform.
2.2.1.1. The technical features of behaviorism are fa r more complex than this simple explanation. Those of us who need understanding of behaviorism and its principles, especially authors of programmed instruction and other sel f-paced learning materials, should read the works of B. F. Skinner and Holland. As classroom instructors, though, we need to realize the importance of controlling learning experiences by manipula ting the classroom environment (stimuli) which gives our students a chance to behave or perform(respond) in the way we desire and can reward (reinforce). 2.2.1.2. Behaviorism can certainly be used to expl ain the way our students learn much of what we teach. We need to be aware of the importance of stimulus, response, and reinforcement as they affect our classrooms. Whether or not we view our selves as behaviorists, these are important concepts to consider as we plan, de liver, and evaluate instruction.
2.2.2. Cognitive Theory. Much of psychological thinking and experimentation today falls into the general category of cogni tive psychology. Unlike the behavioris ts, the cognitive psychologists are very concerned about what is going on inside the lear ner. Learning is not just a change in behavior; it is a change in the way a student thinks, performs, and/or feels. Learning can be measured in terms of behavior. But the behavior only represents learning; it is not equal to the learning in this theory. 2.2.2.1. Motivation, generalization, insight, an d discovery are significant concepts to cognitive theorists. Instructors have the responsibility to set up an environment that motivates students. Fur-AFMAN36-2236 12 NOVEMBER 2003 25

thermore, we must plan learning experiences thatallow students to go beyond simple recall and cause students to gain an insight of what they study.2.2.2.2. Like the behaviorists, the cognitive psychologists acknowledge the importance of reinforcing behavior and measuring changes. We need to measure behavior because it is the only way we can get a clue about what the student thinks , feels, and/or can do. While students may be able to do much more with what we teach than we are able to test, we still have to measure what we can. And, what we measure is usually limited to the kinds of behavior we can capture with a paper-and-penc il test or a performance exam. There will be errors as we try to measure these behavioral indicators, but since understanding and feelings cannot be measured directly we have little choice. Since we want to quantify learning, we have to measure and interpret student behavior.
2.3. Combining the Approaches. Both the behaviorist and cognitive approaches are useful learning theories. We can see from the two brief descriptions that each theory has contributed to the way we plan, deliver, and evaluate Air Force instruction. Perhaps the best approach to planning and managing instruction is to include features of each major theory. Retaining the notion of cognitive learning while measuring behavioral outcomes seems to be the most work able approach. We can plan for low or high order cognitive outcomes and determine if these outcomes ha ve been achieved by measuring and interpreting behavior. We often say that students really understand something because they can do this or that. Therefore, we can plan for cognitive learning, but we must use behavioral evidence to measure the learning. 2.3.1. Several approaches for planning instruction ex ist that combine cognitive concepts and behavioral evidence of learning. This manual presents anapproach for writing and measuring cognitive levels of learning that stresses the need to specify and measure behavior. We will use the cognitive taxonomy of Dr Benjamin Bloom as a frame of reference to plan instruction and to give us a better understanding of the range of possible cognitive learning outcomes. By using this taxonomy, or classification of learning outcomes, we will carefully specify behaviors that will give us reasonable evidence of learning at the various levels of knowledge and understanding. Figure 2.1. contains the basic description of the cognitive domain according to Bloom. The approach to using this taxonomy will be explained in considerable detail throughout this manual.26 AFMAN36-2236 12 NOVEMBER 2003

Figure 2.1. Levels of Knowledge and Understanding.

2.3.2. A similar scheme for specifying attitudinal objectives has been developed by Dr. David R. Krathwohl ( Figure 2.2. ). Like the Bloom taxonomy, Krathwohl attempts to arrange attitudinal objectives in an order of difficulty. We can then attach be havioral evidence to the various levels of this taxonomy for purposes of measurement. Chapter 10 provides ways in which this taxonomy may be useful for planning instructional objectives that deal with attitudes.AFMAN36-2236 12 NOVEMBER 2003 27

Figure 2.2. Levels of Attitude and Values.

2.3.3. There are several taxonomies that deal wi th cognitive, affective, and psychomotor learning. However, this manual will use the Cognitive Taxonomy by Bloom and the Affective Taxonomy by Krathwohl. It will not deal with psychomotor skills to any significant degree ( Figure 2.3. ).28 AFMAN36-2236 12 NOVEMBER 2003

Figure 2.3. Levels of Skill.

2.4. Summary. While we can plan for different levels of understanding and attitude development, we must measure student behavior. The behaviorists have taught us the value of several basic learning concepts including stimulus, response, and reinforcement. We need to plan instruction that capitalizes on these sound concepts. 2.4.1. In addition, we must remember that behavior represents a sample of learning in many cases. It is possible to change viewpoints, increase understanding, or affect the way a student feels about something. These changes in the student cannot be obser ved and must be inferred from observed behavior. As such, we must determine learning outcomes that provide evidence that we are doing our jobs as instructors. We have no choice then, but to measure behavioral indicators, which we will accept as evidence of learning. 2.4.2. The system presented here combines seve ral essential elements of behavioral and cognitive learning theory. It is not simple, but neither is l earning; however, it is workable. Years of experience with this and other similar systems indicate that Air Force instructors can use it to plan, prepare, deliver, and evaluate effective instruction.AFMAN36-2236 12 NOVEMBER 2003 29

Chapter 3

WRITING STUDENT-CENTERED OBJECTIVES AND TESTS

3.1. Introduction. When we are deciding what to teach and how we're going to measure our success in the teaching environment, it isn't enough to simply determine who our students are, what they will need to know, or how we will present the material to them. When our planning stops there, we've failed to consider the most important element of the lesson-pla nning process—what our students will be able to do once they have received and processed the information we present. 3.1.1. It doesn't necessarily follow that what we pr esent in class is what the student will learn. This belief has driven our public educational system for years, in violation of our most basic intuitions. When we break down the teaching process into its basi c elements we realize that, just as in any form of human communication, there is always the possi bility of transmitting false messages or receiving misunderstood symbols in the teaching-learning rela tionship. Without effectiv e and meaningful feedback to both the teacher and the student, the re sultant problem will go unde tected and uncorrected. 3.1.2. For this reason, we advocate a change from the traditional model that places all the responsibility for learning on the student. According to this olde r model, if the student doesn't get the message, it is the student's fault. To underscore this, we have even listed the teacher's activities, responsibilities, and objectives in the lesson plan implying that if the teacher does what is required, then the student will be "educated." Notice that when we design inst ruction completely in terms of teacher behaviors, there is little room for checking feedback the students can give us by way of behavioral change or improved abilities after the instruction has taken place. 3.1.3. Our current model attacks the assumption that, inorder to plan effective lessons, it is enough to determine what the teacher must do in the classroom . Education is a shared process. Both the student and the teacher have certain responsibilities and expe ctations. But we also stress the primacy of the teacher's responsibility to plan lessons that bri dge the gap between teacher and student responsibilities. And we propose that this gap be bridged by writing objectives that focus on the abilities we want the student to display after having received the in struction. With student-centered objectives, teachers are better able to plan teaching activities designe d to efficiently impart and display the knowledge we want the students to learn.
3.2. Planning For Student-Centered Outcomes. Since the 1950's, there has been a movement within military and business establishments to promote stude nt-centered instruction through the use of appropriate objectives. As a result, student-centered objectives that describe learning in terms of student outcomes versus instructor inputs have been used for four ge nerations of Air Force personnel. Contributing to the widespread acceptance of this new m odel are publications such as AFMAN 36-2234, Instructional Systems Development , AFH 36-2235, Volumes 1 through 11, and this manual. All of these have capitalized on the experience and principles gleaned from civilian texts used in public and private industries for years. 3.2.1. With the continued push for accountability in expenditures, the trend toward student-centered instruction is sure to continue. We have found that th is approach is "results driven," goal oriented, and client centered. It is so much more effective in meeting our educational goals that it is the Air Force standard. Moreover, since student learning is define d in terms of objectives, our measurement of success in instructing is now based on comparing stud ent performance to the objectives rather than on30 AFMAN36-2236 12 NOVEMBER 2003

comparing students to each other in a given class. Hence, we have a more accurate, objective, and stable foundation on which to make academic judgments about our students and predictions of their future success. 3.2.2. As stated earlier, learning may be defined as a change in behavior based on instruction. If this is true, our students should perform differently after receiving instruction. Moreover, if we have used student-centered objectives, that behavior should be what we predicted it would be. Table 3.1. and Table 3.2. are examples of educational taxonomies used when writing objectives. These educational taxonomies help instructors classi fy student learning outcomes. Figure 3.1. , on the other hand, helps instructors classify student learning outcomes in the training field. Figure 3.2. compares the various learning outcomes as presented in Table 3.1. , Table 3.2. , and Figure 3.1. We will be able to demonstrate our students' success in learning by having th em complete an evaluation that demonstrates the same physical or mental skill described in the obj ective. But how do we write lesson objectives? Paragraph 3.3. provides information on writing lesson objectives.
Table 3.1. Cognitive Taxonomy. (note)

NOTE: Adapted from "Taxonomy of Educational Object ives Handbook I: Cognitive Domain," ed. B. S. Bloom, et al, New York; David McKay, 1956. L I N E A B

Level of Learning Definition 1 Knowledge Recall previously learned material (facts, theories, etc.) in essentially the same form as taught. 2 Comprehension See relationships, concep ts, and abstractions beyond the simple remembering of material. Typically involves translating, interpreting, and estimating future trends. 3 Application Use learned intellectual material in new and realistic situations, including the application of rules, methods, concepts, principles, laws, and theories. 4 Analysis Break down material into its co mponent parts so that the organizational structure may be understood, including the identification of the parts, analysis of the relationships between parts, and recognition of the organizational principles involved.
5 Synthesis Put parts together to form new patterns or structures, such as a unique communication (a theme or speech), a plan of operations (a research proposal), or a set of abstract relations (schemes for classifying information). 6 Evaluation Judge the value of material for a given purpose. Learning in this area is the highest in the cognitive hierarchy because it involves elements of all the other categories, plus conscious value judgments based on clearly defined criteria.AFMAN36-2236 12 NOVEMBER 2003 31

Table 3.2. Types of Learning. L A B
I N E Learning Categories Proof of Learning 1 Forming Associations Associate, name, or re spond to a specific input (stimulus). The person associates the response with a speci fic input only. The response may be vocal, subvocal (say it to yourself), written, or motor. 2 Forming Chains Recall sequences of actions or procedures that must be recalled in a specific order. In a chain, the response to one input becomes the input to the next response. This may involve chains of verbal responses or chains of motor responses. 3 Making Discriminations Make different responses to the various members of a particular class; it means being able to distinguish among input information sources and/or types and then respond appropriately to each. 4 Making Classification Respond in a single way to all members of a particular class of observable or abstract events. This involves recognizing the essential similarity among a class of objects, pe ople, events, or abstractions and recognizing the differences separating those objects, people, events, or abstractions that are not members of the class.
5 Using Rules Apply rules to a given situa tion or condition by responding to a class of inputs with a class of actions. A rule states the particular relationship between two or more simpler concepts. It is helpful to think of rules as "if-then" statements. 6 Problem Solving Compare previously lear ned rules to create a higher order rule.32 AFMAN36-2236 12 NOVEMBER 2003

Figure 3.1. Air Force Proficiency Code.

PROFICIENCY CODE KEY

SCALE VALUE DEFINITION: The Individual 1 Can do simple parts of the task. Ne eds to be told or shown how to do most of the task. (EXTREMELY LIMITED) 2 Can do most parts of the task. Needs help only on hardest part. TASK (PARTIALLY PROFICIENT) PERFORMANCE LEVELS 3 Can do all parts of the task. Needs only a spot check of completed work. (COMPETENT) 4 Can do the complete task quickly and accurately. Can tell or show how to do the task. (HIGHLY PROFICIENT) a Can name parts, tools, and simple facts about the task. (NOMENCLATURE) b Can determine step-by-step procedures for doing the task. *TASK (PROCEDURES) KNOWLEDGE LEVELS c Can identify why and when the task must be done and why each step is needed. (OPERATING PRINCIPLES) d Can predict, isolate, and resolv e problems about the task. (COMPLETE THEORY) A Can identify basic facts and terms about the subject. (FACTS) B Can identify relationship of basic facts and state general principles **SUBJECT about the subject. (PRINCIPLES) KNOWLEDGE LEVELS C Can analyze facts and principles and draw conclusions about the subject. (ANALYSIS) D Can evaluate conditions and make proper decisions about the subject. (EVALUATION)

EXPLANATIONS * A task knowledge scale value may be used alone or with a task performance scale value to define a level of knowledge for a specific task. (Examples: b and 1b) ** A subject knowledge scale value is used alone to define a level of knowledge for a subject not directly related to any specific task, or for a subject common to several tasks. ^ This mark is used alone instead of a scale value to show that no proficienc y training is provided in the course. X This mark is used alone in course columns to show that training is required but not given due to limitations in resources.AFMAN36-2236 12 NOVEMBER 2003 33

Figure 3.2. Comparison of Taxon omies of Learning Outcomes.

3.3. Writing Lesson Objectives:
3.3.1. Determine Level-of-Learning Objective. It is usually helpful to plan learning systems with a general-to-specific strategy; that is, by starting wi th general objectives and ending with very precise performances. By writing general, but carefully de veloped, nonbehavioral objectives as the first step in planning (see Table 3.1. ), we are better able to describe the general type of behavior we're looking for from our students. An example of a nonbehavior al objective at the comprehension level would be, "The objective of this lesson is for each student to comprehend the Air Force vision statement." Notice that the verb "comprehend" has two important char acteristics. It is nonbehavioral because any student can comprehend something without doing anything that displays the comprehension and it identifies a particular level of learning—comprehension. 3.3.1.1. At this point in the process, we don't want to use behavioral verbs because behavioral verbs alone cannot tell us the level of learning they describe. It is easy to misinterpret behavioral verbs in isolation from their specified levels of learning because actions are often performed for unstated reasons or in response to unrecognizable s timuli. The level of learning identifies the character of the behavior for the teacher and stude nt and prevents misinterpretation. What nonbehavAir Force Air Force Cognitive Types of Learning Prof iciency Code Proficiency Code Types Taxonomy (AFH 36-2235, Subject Task

of (AFMAN 36-2236, Volume 10, Knowledge Knowledge Learning Table 3.1. ) Table 3.2. ) ( Figure 3.1. ) ( Figure 3.1. ) Remembering Knowledge Forming Facts Nomenclature Associations Procedures

Forming Chains

Making Discriminations Reasoning Comprehension Making Classifications Principles Operating Principles Problem Application Using Rules Analysis Complete Solving Theory Analysis Problem Solving Evaluation

Synthesis

Evaluation34 AFMAN36-2236 12 NOVEMBER 2003

ioral level-of-learning objectives do, then, is descri be the nature and context of the behavior. It is very important to decide the level of learning befo re we attempt to describe its evidence by precise behavior. 3.3.1.2. For example, requiring a student to give a definition would be an ambiguous task without specifying whether we would accept a verbatim reca ll of a definition (knowledge level) or require a definition in his or her own words (comprehension level). In each case, the behavior is "to define," but recall is a simpler task than understa nding which requires, at the very least, an ability to translate something into your own words. Al ternatively, we might ev en expect students to derive a definition as a result of problem solvi ng at the end of a course—an even higher level of learning. When we state the general level of learni ng at the first step in the writing process in the design phase, it makes each of the following ac tivities that lead to th e anticipated performance more exact.
3.3.2. List Indicators (Samples) of Behavior. After we have carefully communicated the level of learning we want our students to reach, we must deci de which student behaviors we will accept as evidence of learning. In physical tasks or simple cognitive learning, the decision is quite simple; there may be a one-to-one correlation betw een the behavior required and the real-world activity. For example, in order to measure whether a student knows th e spelling of the word "university," we would ask the student to spell the word. Or to test one's driv ing skill, the teacher would have the student drive a

car. 3.3.2.1. In higher level cognitive or affective le arning, however, the relationship between student behavior and the level of learning is not always that clear. We must decide what behaviors are associated with that thinking skill or attitude we want to measure. We know we cannot directly measure understanding or attitudes, so we must de cide which behaviors give us the best evidence of learning. Although there is always the danger thatwe may err in translating learning into behavior, it is a task we must accomplish because the value of measuring learning by its outcomes far outweighs the risk of error. 3.3.2.2. We may accept many or just a few student behaviors as evidence of learning. The important thing for us to ensure is that any performance must be comprehensive enough to give reasonable assurance that what we accept as evidence gives a true representation of that knowledge. The sample performance should provide us a measurable , observable, reliable, and verifiable behavior that directly relates to the level of learning identified in the lesson objective. 3.3.2.2.1. The behavior should be significant enough to prove to the instructor that the student has reached the objective. But what is "signifi cant?" To answer that question, one must consider the complexity of the lesson content. A simple lesson may ha ve multiple samples of behavior in which each behavior encompasse s the entire lesson content or objective. 3.3.2.2.2. As an example, a knowledge-level lesson designed to teach a list of eight leadership terms may have performance sample s that require the student to name, list, write, identify, etc., the eight leadership terms. Each of these behaviors is measurable, observable, reliable, and verifiable, and encompasses the entire content or objective of the lesson, such as to know eight specific leadership terms. If, on the other hand, the objective encompasses multiple major topics or areas, you may want the student to exhib it performance behaviors associated with each topic or area. Chapter 4 and Chapter 10 discuss the concept of sampling cognitive and affective objectives in more detail.AFMAN36-2236 12 NOVEMBER 2003 35

3.3.2.2.3. Consider a comprehension-level lesson on student evaluation that includes areas on types of evaluation, constructing evaluation in struments, guidelines for using evaluation instruments, and common errors associated with using evaluation instruments. In this lesson, you may want to write performance expectations for each of the major topics or areas. When taken together, the student performances in each of the areas will indicate whether or not the student has met the overall lesson objective; that is, to comprehend performance evaluation. Areas needing additional instruction are identified when the student is unable to perform the expected behavior in a given area or areas. If the student is una ble to perform the behaviors for one or more of the areas that make up an overa ll lesson objective, then the overall objective is not being met. The problem areas indicate where additional instruction may be needed. 3.3.2.2.4. To help clarify what kind and how many samples of behavior are needed, let's compare them to tiles in a mosaic. To accurately portray the image you desire, you need the correct number, size, and color of tiles. Tiles that are too large will not give the detail you desire and if they are too small, you may be forever trying to put them all together. Also, the correct color is required or aspects of the picture will be misrepresented. The same can be said for samples of behavior. If they are too large and general, they do not pinpoint the behaviors you expect from your students. If they are too small, yo u will be forever writing and measuring every imaginable behavior related to the lesson. A sampling is justthat, a representative sample that proves to you that the stude nt has met your objectives.
3.3.2.2.5. There needs to be a balance between the two extremes. Ultimately, you must decide how many and what kinds of behavior you require to convince you the students are where you intended them to be at the end of the instruc tion. The more complex the lesson, the more samples you may require. Remember, however, the le vel-of-learning objective is the ultimate destination, while the samples of behavior are the indi cators that tell you if the students are getting there the way you intended.
3.3.3. Develop Criterion Tests. The lesson-planning process conclude s with the construction of test items and tests to measure learning. Other chapters discuss this concept in detail. At this point, we construct test items from sample s of behavior to measure our objectives. These test items are one method we use to gather evidence that students have learned what we intended them to learn. Test items are written prior to instruction because we will use these requirements to determine what needs to be taught. For example, if our te st item is to identify the phases ofthe Air Force's ISD Model, determining this test item ahead of time gives us a pretty good idea of whatneeds to be covered in the lessons we will plan later. The support material in the lesson should enable the student to perform the samples of behavior (SOB).
3.3.3.1. Writing a Test Item:3.3.3.1.1. The practice of measuring the achievement of stated objectives—known as criterion-referenced testing—is a rigorous process. Be cause of this rigor, criterion-referenced testing has been accepted only reluctantly and slowly in some sectors of the Air Force even though student-centered objectives are widely use d. Let us take an example of a simple concept—leadership—and demonstrate the results of each of the steps for writing a test item ( Figure 3.3. ).36 AFMAN36-2236 12 NOVEMBER 2003

Figure 3.3. Sample Test Item.

3.3.3.1.2. The construction of many instructional objectives is much more difficult than the example we've provided here. In some instances, there may be several samples of learned behavior rather than just one. Objectives, whethe r written or assumed, can be much more elaborate and may require several paragraphs to de scribe the requirement. Additionally, there are many more alternative and complex ways to test learning outcomes than using simple test items. But regardless of how complex the subj ect matter or how high the level of learning might be, this process for developing student- centered objectives and test items works. The results of this systematic thought process will be consistent with sound educational practice and will satisfy the critical elements of the design phase in ISD.
3.3.4. Criterion-Referenced Objectives. As mentioned earlier, the focu s of this manual is on the academic classroom using level-of-learning objective s and samples of behavior. However, instructors must also be aware of the procedures for writing te chnical instruction objectives. These tend to be best expressed as criterion-referenced objectives (CRO) that consist of the performance expected of the students, the conditions under which they will acco mplish it, and the standards they will be held accountable to. (See Chapter 5 for more details on writing CROs.) Samples of behavior in the academic learning environment are very similar to the performance aspect of a CRO. The difference is they do not specifically state the conditions or standards. A CRO usually focuses on physical skills and abilities instead of cognitive processes.3.3.4.1. An example of what a CRO might look like is, "Upon completion of instruction, each student will be given a muzzle-loading musket with rammer, three percussion caps, and three cartridges which consist of paper, musket-ball, and black power (conditions). They will load and fire the musket (performance) 3 times within a 1- minute period. The sequence for loading and firing must be without instructor assistance and in accordance with the nine steps outlined in the 1862 Step 1. LEVEL-OF-LEARNING OBJECTIVE: The objective of this lesson is for each student to know the meaning of leadership.

Step 2. SAMPLE OF BEHAVIOR: Each student will define leadership.

Step 3. TEST ITEM: The AFROTC handbook defines leadership as __________.

a. The willingness to exercise management control over subordinates.

b. The process of inspiring effective indivi dual effort in a group environment toward achieving an objective.

c. Planning, organizing, staffing, directing, and c ontrolling the capital, material, and human resources of an organization.AFMAN36-2236 12 NOVEMBER 2003 37

version of Hardees Manual of Arms (standards)." Ju st as with the SOB, you can get too detailed in your description of the performance, conditions, and standards. Some may feel it is understood that the test will be after the instruction and would leave that statement out. Also, it can be assumed that the performance willbe outside. Others may assume that the student would realize what a cartridge consists of and therefore not detail its contents. Again, you must decide how much is enough. You do not want to be too vague with the CRO, but you can also get bogged down into minute details when dealing with comple x CROs. Try to strike th e appropriate balance. 3.3.4.2. CROs, like level-of-learning objectives and the associated samples of behavior, should be provided to the students prior to instruction. They serve as valuable tools to the students for focusing efforts and telling them what is expected in terms of performance, conditions, and standards. The actual performance of the students is the te st you will use to determine if they have met your objectives. Tell your students what you are going to teach and what you expect from them—then do exactly that.
3.3.5. Organization of Objectives. So far, our discussion has centered on objectives for a single lesson or period of instruction. However, there is usually a hierarchy of objectives associated with instruction that consists of two or more levels. Top-level outcomes de scribe the end result of the learning experience and may be stated as goals and/orobjectives while developmental objectives are a series of enabling outcomes that increase the pr obability of achieving a complex top-level outcome (AFH 36-2235, Volume 10).3.3.5.1. When organizing objectives, you may start with general statements and work toward specific lesson objectives to produce this hierarchy. This allows the students to see how the components of the instruction fit together in a developmental format, what they can expect to achieve overall, and what the enabling outcomes are for each portion of the instruction. 3.3.5.2. One way to label the portions of instruction is by: units or periods (individual lessons); phases or blocks (multiple units that make up a given set of re lated lessons); areas (blocks of instruction that make up a major portion of the curriculum such as communication skills); and course (the overall missi on of the instruction). 3.3.5.3. Each of these levels should have objective s and measurable behaviors, whether stated as CROs or levels of learning, withrelated samples of behavior.
3.4. Summary: 3.4.1. You are probably in the midst of a difficult transition from teacher-centered to student-centered instruction because most of your educational experience during early schooling has reflected teacher-centered instruction. In spite of this tradition, the Air Force has come a long way in specifying and measuring student-centered learning outcomes. Air Force instructors and other progressive educators have worked to develop appropriate techni ques to fully implement the concept of student-centered instruction. While many techniques need further development, a helpful program for student-centered instructional planning consists of determining level-of-learning objectives, listing indicators (samples) of behavior, and developing crit erion-referenced tests. For technical instruction, you may want to develop CROs that consist of a stude nt's performance, the conditions they will do it under, and the standards you will hold them accountable to. 3.4.2. The lesson planning, methodology, and evaluation chapters of this manual incorporate these procedures. While there are many approaches to st udent-centered instructional planning, these strate-38 AFMAN36-2236 12 NOVEMBER 2003

gies will work. The process of going from general level-of-learning objectives to precise criterion test items will be explained more fully in the remain ing chapters. This chapter has provided an overview of the system for writing student-centered objectives and tests. With this information, you're in the right frame of mind for efficiently improving your teaching skills.AFMAN36-2236 12 NOVEMBER 2003 39

Chapter 4

THE LEVEL-OF-LEARNING LESSON PLANNING PROCESS

Section 4A—Overview

4.1. Introduction. In Chapter 3 we introduced a process for devel oping objectives and criterion-referenced tests. In our examination of this process, we briefly introduced the academic concepts of educational taxonomies, level-of-learning objectives, and sa mples of behavior. However, instructors must also be aware of the procedures for writing technical inst ructional objectives that are called criterion-referenced objectives. This chapter will examine, in mo re detail, level-of-learning objectives and samples of behavior in Bloom's cognitive taxonomy. Chapter 10 will then cover this process for the affective taxonomy.
4.2. Bloom's Cognitive Taxonomy. Before we can begin writing leve l-of-learning objectives, we need an understanding of the types of learning outcom es represented by each level of the taxonomy. 4.2.1. This taxonomy is simply a means of rank or dering learning within the cognitive domain. We must pass through each of the rank orders or levels as we move to the more complex behaviors. We must have some knowledge in a subject area before we can comprehend concepts or principles in the same subject area. We must have comprehension of principles before we can applythese principles to new situations, and so on up the ladder of the taxonomy. Each of the levels in this rank order forms the basis for writing level-of-learning objectives, as we ll as sequencing lessons within a block of instruction. Next, we will begin with the knowledge leve l of learning and work our way through evaluation.
4.2.1.1. Knowledge. Knowledge is the recall or recognition of previously learned material (facts, theories, etc.) essentially in the same form as taught.
4.2.1.2. Comprehension. Comprehension is defined as seeing relationships, concepts, and abstractions beyond the simple remembering of material. This typically involves translating, interpreting, and estimating future trends.
4.2.1.3. Application. Application is the ability to use learne d material in new and concrete situations, including the application of rules, met hods, concepts, principles, laws, and theories.
4.2.1.4. Analysis. Analysis is the ability to break down material into its component parts so that the organizational structure may be understood, including the identification of the parts, analysis of the relationships between parts, and recogni tion of the organizational principles involved.
4.2.1.5. Synthesis. Synthesis is the ability to put parts togeth er to form new patterns or structures, such as a unique communication (a theme or speech), a plan of operations (a research proposal), or a set of abstract relations (schem es for classifying information).
4.2.1.6. Evaluation. Evaluation is defined as the ability to j udge the value of material for a given purpose. Learning in this area is the highest inthe cognitive hierarchy because it involves elements of all the other categories, plus conscious value judgments based on clearly defined criteria. 4.2.2. Student performance at the analysis, synthesis, and evaluation levels (when taken together) represents complex problem solving. However, a review of Air Force schools indicates that more than 90 percent of the material taught is at the lower three levels of the taxonomy. This fact is not surprising, and with a few exceptions, such as our senior PME schools, the situation is as it should be.40 AFMAN36-2236 12 NOVEMBER 2003

4.2.2.1. In most cases, the higher levels are more effectively reached through on-the-job experience; therefore, we will concentrate our attention on the lower three levels. (For those instructors teaching at the three higher levels, we recomme nd the discussion in the Taxonomy of Educational Objectives Handbook I: Cognitive Domain, ed. B. S. Bloom, et al, New York: David McKay, 1956).
4.2.2.2. As mentioned in Chapter 3 , we cannot directly observe such general behaviors as knowledge, comprehension, or applicati on. In order to help us write our level-of-learning objectives, let us look at some key words or phrases which would reflect knowledge, comprehension, and application ( Figure 4.1. ).
Figure 4.1. Key Words for Knowledge, Co mprehension, and A pplication Levels.

Section 4B—Learning Levels
4.3. Knowledge. At the knowledge level of the taxonomy, we ar e asking that students recall or remember information in essentially the same form in which it was given to them. The source of the information has no bearing on what we expect the student to be able to do to meet our knowledge-level objective. The students may have received the information from lectures , readings, computer-based instruction, video or audio tapes, or other means. In any case, they simp ly memorize and store information, which they then give back to us, essentially verbatim, when evaluated.
4.4. Comprehension: 4.4.1. At the comprehension level of the taxonomy, we go beyond simple recall and attempt to attach a meaning to what is communicated. In doing so, we try to establish relationships between pieces of information in order to form rules, laws, concepts , principles, and generalizations. Within the comprehension level, we can establish what we might call a mini-taxonomy. Thus, after reaching the knowledge level, students pass through all three levels of comprehension (translation, interpretation, and extrapolation) before they reach the application level.
4.4.1.1. Translation. First, we would expect a student to translate material from one form to another. As an example, we might ask students to tell us in their own words what something means, ask them to paraphrase a lengthy reading, or ask them to look at a graph or chart and tell us Knowledge Comprehension Application

Recall Translate Solve Remember Interpret Use Recognize Extrapolate Modify Memorize Implications Compute Store Consequences Construct

EffectsAFMAN36-2236 12 NOVEMBER 2003 41

verbally, or in writing, what the various parts mean . Or, in a context that we are all familiar with, we might ask to have a passage written in a foreign language translated into English. We can see that we have gone a step beyond simply recalli ng information; we are now asking the students to give a meaning to what they have learned.
4.4.1.2. Interpretation. At the second level of comprehension, we are primarily interested in students' abilities to see relationshi ps between various aspects of a communication. In order to interpret, they must first translate what the individua l parts mean and then see the relationship between these parts. We can see the evidence for inte rpretation when students perform such activities as making inferences, generalizations, and summations.
4.4.1.3. Extrapolation. At the third level of comprehension, we want our students to be able to answer the question, "What would happen if?" Thatis, we want them to go beyond the literal message contained in the communication and make predictions or extensions of consequences or trends. These predictions or extensions are based on what is given rather than on information brought from other experiences. We also expect the students to be able to tell us, with some reasonable degree of accuracy, the probability of their prediction—not statistical probability necessarily, but general probability such as "it would ve ry likely happen" or "it's only slightly probable." The following illustrates the three levels:
4.4.1.3.1. Let us assume that we want our students to comprehend the relationship between factory production and unemployment as illustrated in these two graphs ( Figure 4.2. ). First, we would want to be sure the students could translatethe information contained in the graphs. For instance, we might ask students to tell us what Point A on the factory production graph and what Point B on the unemployment graph represent. If they can successfully read the graph, they have demonstrated the ability to translate from graphic to verbal or written form.
Figure 4.2. Relationship Between Factory Production and Unemployment.

4.4.1.3.2. Our next step, interpretation, would be to ask students to identify the relationship, at least as illustrated by the two graphs, between factory production and unemployment. Now, simply knowing the meaning of Point A and Point B is no longer enough; the students must now see each graph as a whole (that is, f actory production has increased, unemployment has decreased), and then make a generalization c oncerning their relationship (such as, "as unem-42 AFMAN36-2236 12 NOVEMBER 2003

ployment decreases, factory production goes up" or "there is an inverse relationship between unemployment and factory production"). 4.4.1.3.3. Finally, we could evaluate our students' ability to function at the extrapolation level. Suppose we asked the students the following questi on: "Based on the trends illustrated in the two graphs, predict the level of factory production at the end of the first quarter of 2000." We have asked the students to go beyond the litera l communication (the two graphs) and answer the question, "What would happen if?" If they ca n extrapolate, students are able to function at the highest level of comprehension. 4.4.2. Before we leave comprehension, we should keep in mind two general concepts when writing objectives at this level: 4.4.2.1. First, we can think of all levels of the taxonomy above knowledge as increasingly complex levels of understanding. Students can be asked to predict at any of these levels. The predictions become more complex at the higher leve ls and students will be required to bring in abstractions from other experiences, such as a gene ral principle or rule of procedure. Again, referring to our graphs, at some higher level of th e taxonomy students could br ing in many other factors in addition to unemployment, such as interest rates, sales predictions, energy sources, to make a prediction about factory production. However, at the extrapolation level of comprehension, the prediction must be based solely on the informati on the students are given in the literal communication (that is, the two graphs). 4.4.2.2. Second, if our course is de signed to bring students to the application level of learning, we must ensure in our comprehensi on level objectives that we br ing students through the extrapolation level of comprehension.
4.5. Application. At the application level of the taxonomy, we ask our students to take the concepts and principles they formed at the comprehension level and put them to use in new situations. For example, suppose we just completed a block of instruction inwhich our students learned to comprehend several concepts of management and one of the concepts they learned was delegation of authority. We want to evaluate their application of the concept. We might give them a scenario of an Air Force organization in which the concept of delegation of authority was violated, causing problems in the organization. We thenask them to read the scenario and give us their solutions to the problem, but we don't tell them which concept was violated. Our criteria for evaluation is for the students to solve the problem by applying the delegation of authority concept to the organization described in the scenario. 4.5.1. Many instructors have difficulty in differentiating between application and the higher levels of comprehension and, indeed, the difference can someti mes be confusing. However, if we keep the following ideas in mind, distinguishing between the two should not be difficult: 4.5.1.1. At the comprehension level, students can demonstrate the use of a principle or concept when the principle or concept is identified. At th e application level, when given a problem situation, students must identify the concepts and pr inciples involved by themselves and then demonstrate use of the principle or concept in solvin g the problem. Thus, at the comprehension level, students demonstrate use of an identified concept or principle. At the application level, students use the concept or principle when confronted with a problem situation new to them. 4.5.1.2. For example, in the "delegation of authority" situation, if we tell students that the scenario involves the misuse of delegation of authority and then tell them to solv e the problem, they areAFMAN36-2236 12 NOVEMBER 2003 43

working at the comprehension level. But, by not telling them which concept is being violated and by making them come to this determination by themselves, we ensure they are working at the application level. 4.5.2. As we have suggested, the application level of the taxonomy is the point where students first encounter the problem-solving process. Application is , or should be, the level of learning most Air Force schools attempt to reach. Just because students comprehend concepts and principles, we have no guarantee they can applythese concepts and principles in new situations. The opportunity for practice of application level-of-learning activities must be provi ded for our students before we can expect them to function at this level. 4.5.3. Before we leave the application level, let us be sure we are clear on an area that often presents problems to both students and instructors. The word application, used at the cognitive level of learning, is not synonymous with the meaning of the wo rd as we use it in our everyday speech. If we put this in terms of educational objectives, the term "apply" is not necessarily synonymous with the term "be able to." We find the confusion most pr evalent in dealing with skill development. 4.5.3.1. Suppose we use the objective: "Be able to solve an algebraic equation for one unknown." If we think back to our high school or college al gebra courses, most of us can remember how we learned to meet this type of objective. We simply memorized the rules that applied and used these rules to solve the equation. Seldom was there any effort on our part or the teacher's, to have us comprehend the meaning behind the rule. 4.5.3.2. Thus, we could solve the problem, but we were not really at the application level of learning. We often saw the results of this type of le arning when we were asked to solve word problems in algebra or other types of mathematics where we needed a comprehension of the concepts and principles involved to solve the problem. 4.5.4. There is a clear implication here for instruct ors and curriculum developers. If our students are expected to problem-solve or troubleshoot on the jo b, then they must be at the application level oflearning. And, to get there, they must have satisfactorily passed through the lower levels of the taxonomy.
Section 4C—Level-of-Learning Objectives

4.6. Overview. A level-of-learning (LOL) obj ective, as introduced in Chapter 3 , contains three elements—student centeredness, level of learning, and speci fic subject. Let us now ta ke a closer look at the function of each element.

4.7. Student Centeredness. Because the purpose of any educational objective is to express the outcome of the lesson in terms of learner behavior, this part of the LOL objective ensures that our focus is on the word "student." We might want to write this portion of the objective as: "The objective of this lesson is for each student to... ." However, because this part of th e objective is a given, we can infer that all our objectives have this as an opening statement, whether it is actually written or not.

4.8. LOL. Every objective should state the level of the ta xonomy we expect our students to reach by the end of a period or block of instruction. By using an ISD or systems approach to curriculum development, this decision may have already been made in the analysis phase.44 AFMAN36-2236 12 NOVEMBER 2003

4.9. Specific Subject. LOL objectives may be written to varying degrees of specificity. For example, if we are writing a LOL objective for an entire block ofinstruction that could include many hours, our subject might be quite general. Norma lly, as we write objectives toward smaller and smaller units of instruction, the objectives become more specific.
4.10. Example of an LOL Objective: 4.10.1. Let us examine the three elements of the LOL objective as we relate them to an example from a block of instruction in educational evaluation. We will assume for this example that we plan to devote 30 hours to achieve our block objective.
4.10.1.1. Student Centeredness. Note that we chose to not include the statement "The objective of this lesson is for each student to know (compreh end, apply, etc.) ...", but we see that this statement is implied in each of the objectives in Figure 4.3.
4.10.1.2. LOL. Each objective contains a specific LO L (know, comprehend, or apply), but only one level is contained in each objective. For this reason, we should always state the highest level we want our students to reach in the period or block of instruction.
4.10.1.3. Specific Subject. How specific to make the subjec t of the lesson normally depends on two variables—the scope of the content covered by the objective and the LOL.
4.10.2. In Figure 4.3. , the block objective (at the application level) is quite general because of its broad scope. We would not expect our students to meet this objective until after many hours of instruction. At the other end of the scale (the knowledge level), both Period 1 and Period 2 objectivesare very specific. Our basic guideline here is to make objectives as specific as possible: the more specific the objective, the easier it will be to plan lessons and assess achievement of the objective.
Figure 4.3. Example Level-of-Learning Objectives.

Block Objective—Apply the principles of test cons truction, performance rati ng, and test analysis to assess student achievement of objectives.

Period 1 (2 hours) Know the mechanics for wri ting selection (multiple-c hoice, true-false, and matching) and supply (essay, short answ er, and completion) test questions.

Period 2 (2 hours) Know the procedures for pla nning, constructing, and grad ing selection and supply tests.

Period 3 (2 hours) Comprehend that testing is a us eful tool in assessing achievement of student learning.

Period 4 (4 hours) Apply principles and techniques of classroom test construction to measure student achievement.AFMAN36-2236 12 NOVEMBER 2003 45

4.10.3. Note that the block objective includes three majo r elements of content: principles of test construction, performance rating, and test analysis. Up to this point we have dealt with only the first element of our block objective (that is, principles of test construction). We w ould then proceed to write period objectives for performance rating and test analysis. We will not continue this procedure in this example, but the same logical progression would be used—starting our students at the knowledge level and progressing through application.
Section 4D—Samples of Behavior

4.11. Overview. We have discussed our inability to directly observe students knowing, comprehending, or applying. However, both instructors and student s need to know how well the student met the LOL objectives. Thus, we need a vehicle that will give us a base on which to plan our evaluation of achieving LOL objectives. This vehicle is called a sample of behavior.
4.12. Defining a Sample of Behavior: 4.12.1. We can define a sample of behavior as a st atement that specifies one of several measurable, observable, reliable, and verifiable behaviors that students should be able to demonstrate at the end of a period or block of instruction and which gives us significant evidence they have achieved our objectives. These samples eventually become the basis fo r our evaluation, most often in the form of test questions. The actual process of converting thes e samples to test questions is covered in Chapter 22

. 4.12.2. The word evidence is the key here. There may be a great variety of behaviors we would expect students to demonstrate to us as evidence that, fo r example, they comprehend a concept or principle. In fact, there would be far more ways than we could ever have time to observe and evaluate. For this reason, we list a reasonable number (a sampling) of be haviors that are as equivalent in nature as we can possibly make them. We conclude that if stude nts can demonstrate these behaviors, they will be able to demonstrate others. This sampling proce dure is the same as any other sampling procedure— the larger the sample, the greater confidence we can put in the evidence obtained.
4.12.3. Certain behavioral verbs lend themselves for use at each level of the taxonomy. A partial list is in Figure 4.4.46 AFMAN36-2236 12 NOVEMBER 2003

Figure 4.4. Behavioral Verbs.

4.12.4. When we use any of these verbs in a sample of behavior, we cannot guarantee that the sample is written at any particular leve l of learning. The activity following the behavioral verb determines the level at which the student will respond to the sample . The whole sample of be havior must be looked at in context. But, as a general rule, we use these type of verbs at each level. 4.12.5. As a corollary, the same verb may be used at several different levels of the taxonomy. For example, consider the word "identify." In Figure 4.4. , this word is shown at the knowledge level because it is primarily associated w ith this level. Now, let's look at how this word could be used at three different levels of the taxonomy.
4.12.5.1. In Figure 4.5. , our sample is only calling for recall of specific information.
Figure 4.5. Recall of Information.

4.12.5.2. In Figure 4.6. , we are giving the students three examples of student learning outcomes they have not seen before. Their job then is to see the relationship between what they have learned about criterion objectives and the three statements. Level of Learning Samp le Behavioral Verbs

Knowledge List, name, match, describe, define, state, outline, identify, select, explain, give an example

Comprehension Explain, compare, contrast, differentiate, predict, summarize, generalize, paraphrase, distinguish, solve, compute, identify, give an example

Application Solve, compute, prepare, use, develop, construct, modify, conduct, identify, teach

Knowledge

Objective: Know the five phases in the ISD process.

Sample of Behavior: Name the five phases in the ISD process.AFMAN36-2236 12 NOVEMBER 2003 47

Figure 4.6. Understanding Information.

4.12.5.3. In Figure 4.7. , we have asked the student to solve a problem (identify the solution) by applying a previously learned concept to a new situation.
Figure 4.7. Applying Information.

4.12.6. Behavioral verbs by themselves do not give us assurance of a specific level of learning; they must be looked at in context with the total behavioral statement.
4.13. Considerations in Writing Samples of Behavior:
4.13.1. Kind of Verb. First, the verb must reflect an obser vable behavior. Second, there should be a good variety of verbs. Thus, if our objective was atthe comprehension level and we decided to write five samples, we would not want all of our samples to specify "e xplain." Instead, we might have one sample using "explain," and others with "differentiate," "compare," "paraphrase," "predict," and so forth. These samples would give us a wide variety of behaviors on which to base our judgment that the student comprehends our objective.
4.13.2. Level of Learning. The sample must reflect the same level of learning as specified in our objective. If the sample is written at a higher or lower level, it will invalidate our effort to assess achievement of the objective at the specified level. Comprehension

Objective: Comprehend that criterion lesson planning focu ses instruction on student achievement of limited precise learning outcomes.

Sample of Behavior: Given three logical examples of student learning outcomes, identify the one best suited for a criterion objective.

Application

Objective: Apply the management concept of "delegation of authority" to a problem situation.

Sample of Behavior: Given a scenario in which a problem has arisen due to the misuse of delegation of authority, identify the best solution to the problem.48 AFMAN36-2236 12 NOVEMBER 2003

4.13.3. Evidence of Achieving Objective. The sample should give us significant evidence that our objective has been achieved. Since we may be limitedin the number of samples we can test, we cannot afford to write samples dealing w ith trivial or peripheral information.
4.13.4. Clarity. The samples must be clear, simple, and conc rete. Clarity will help us, as instructors, devise our evaluation instrument. More important ly, if we give the sample to students—and we should—they will know what they are responsible for.
4.13.5. Number of Samples. There is no magic number of samples. Instructors or curriculum developers must decide how many are need ed to determine that the students have met the objectives. At the knowledge level, we are dealing with the recall of a good deal of factual information. So, we will probably require many samples to cover a reasonable amount of information. As we move to the higher levels of learning, we will need fewer—but usually more complex—samples.
4.13.6. Time Available for Evaluation. Ideally we would want to test our students on every sample of behavior we write. Realisti cally, though, testing all samples is not always possible because of time constraints. Two basic alternatives to this proble m are to write only the number of samples we will have time to test or write the number of samples we think will adequately evaluate the objective, but actually test only a portion of the samples. 4.13.6.1. Assuming that students will be given the samp les as a part of the course overview, the second alternative is preferred. The primary drawba ck with the first alternative is that many students will key in on the small number of samples and disregard the rest of the material. By using the second alternative, students can identify all ofthe key ideas we want them to get from the lesson but will not know which of the samples we will test. Thus, they are forced to study all the key material we have indicated in the samples. 4.13.6.2. The primary deficiency in this system is that we get no feedback on student learning from those samples we do not test. However, if we teach a course several times a year, we can use alternate tests and, perhaps ove r a period of several classe s, test all the samples.
4.14. Determining What To Sample. What do we sample? The answer might appear obvious—our objectives. That is true, but the next question that must logically follow would be, "Which objectives?" The answer to this question must be based on two va riables: the structure of our evaluation program, and the use to be made of the evaluation results derived from the samples. Let us examine each of the variables to see how we can determine which objectives should be sampled.
4.14.1. The Structure of Our Evaluation Program. The curriculum for an Air Force school might be designed in the hierarchy shown in Figure 4.8.AFMAN36-2236 12 NOVEMBER 2003 49

Figure 4.8. Air Force Curriculum.

4.14.1.1. Although we would probably never write sa mples from our mission statement because it would simply be too broad to work with, any of the other levels of a course could be sampled. The key to what should be sampled is based on our evaluation program. If we have a program where tests are given after very large blocks of instruction (most often seen in longer PME courses) and in the form of a very comprehens ive essay example, writing samples for the course or area objectives might be all that is necessary. 4.14.1.2. On the other hand, in shorter courses or where testing is conducted relatively often on shorter blocks of instruction it would be more fe asible to write samples on block or period objectives.
4.14.2. The Use to Be Made of Evaluation Results. If the primary purpose of all our testing is summative in nature (that is, used to determine student s' grades or whether they pass or fail), then the guidelines in paragraph 4.14.1. are all we need. However, many schools use formative (often referred to as diagnostic or enroute) evaluation; that is, so me tests are given only to give students and instructors feedback on student progress—not for grades or pass-fail. In this situation, we might want to write samples for every period or block of instruction even though summative evaluation might be based only on samples from area or c ourse objectives. If your school or c ourse falls in this latter category, you should never use samples designated only for formative evaluation in an evaluation mode that is summative in nature. When this mistake is made, we usually end up tes ting peripheral or trivial information instead of our primary objectives.
4.15. Summary. The level-of-learning objective is our fi rst step in describing learning outcomes.4.15.1. Knowledge-level objectives state outcomes where recall of specific information is our primary concern. Comprehension-level objectives state outcomes that require students to give meaning to what they have learned, establish relationships, and extrapolate beyond the literal message of a communication. Application-level objectives and above require the student to put previously learned concepts and principles into use in progressively complex situations. Mission statement

Course objectives

-- Area or phase objectives

--- Block objectives

---- Period objectives50 AFMAN36-2236 12 NOVEMBER 2003

4.15.2. Samples of behavior allow us to evaluate student achiev ement of level-of-learning objectives. Samples are statements that specify a measurable, observable, reliable, and verifiable behavior that students should be able to demonstrate at the end ofa period or block of instruction. They give evidence that the students have achieved our objectives. Samples are always written at the same level as the objective.AFMAN36-2236 12 NOVEMBER 2003 51

Chapter 5

WRITING CRITERION OBJECTIVES

Section 5A—Introduction

5.1. Overview. A criterion objective is a word picture of what we expect students to do, the conditions they should be able to do it under, and the standards we will hold them accountable for once they have received our instruction. This chapter deals with criterion objectives as separate planning tools by illustrating their content and formats, describing many of the errors made in writing them, and providing some insight into ways they can be used to sequence instruction. 5.1.1. The criterion objective may range in length fro m a sentence or two to many paragraphs. A simple criterion objective might read, "Without the use of references, correctly name all three parts of acriterion objective." More complex criterion objectives are illustrated later in this chapter. The criterion objective may be written and given to students orit may exist only in the minds of instructors as they prepare tests to measure learning. In either ev ent, the most efficient way to describe what students should learn is to describe carefully what they will be able to do if instruction is successful. The assumption made in this chapter is that the objecti ves will be written and give n to students and others. 5.1.2. When we stop to think about it, the idea of expressing objectives or expectations in measurable terms is already familiar to us. We are used to reading the raised lettering on an automobile tire that tells us what the inflated pressure should be whenthe tire is cool. Or, thr ough Saturday afternoon television, we know that a world class sprinter, timed electronically, can run the 60-yard dash in 6.1 seconds indoors. In both cases, we describe the performance of the subject in measurable terms. We can tell what the subject does on a test and we also know about the environment and the quality or quantity of the performance.5.1.3. In our Air Force experience, as well, we can understand the requirement for an Air Force manual to be written at a specific grade level (as measur ed in a standardized way). We ask airmen in basic training to name each major air command or list, in order, all enlisted and officer ranks without error. In a management course, we might expect the succe ssful student to propose a solution to a particular type of personnel problem. In each of these instances, we have the beginnings of a fairly precise and quite easily understood instructional objective. 5.1.4. With very little additional information, two individuals, each competent in a particular field, could agree whether a student had achieved our obj ective. Once we agree on the information needed to express an objective, we can easily communicate the objective and measure it. Concern about clear communication and consistent evaluation is the primary reason for using criterion objectives in instructional planning.
5.2. Relationship Between Samples of Behavior and Criterion Objectives. Samples of behavior (SOB) are observable and measurable behavioral indi cators that help us determine if the student has achieved our general LOL objective. Because we can not "see" comprehend, we use samplings of behavioral indicators to measure our instructional success. LOL objectives and SOBs are typically used in educational settings where the student is partic ipating in cognitive and affective learning. 5.2.1. Often our objective is to change what a student thinks or feels. However, we must use the resultant change in behavior to measure the degree of learning.52 AFMAN36-2236 12 NOVEMBER 2003

5.2.2. The criterion referenced objective (CRO), on the other hand, is found extensively in training environments. In this environment, students are us ually learning a task they must perform. The CRO, when written as a performance, condition, and sta ndard, is inherently obs ervable and measurable. Both the SOB and the CRO are attempting to define the behavior we expect from the student once the instruction is complete. The SOB is taking a cogn itive and/or affective outcome and making it observable and measurable while the CRO is defining th e expectations of the student while performing a specific task under specified conditions. 5.2.3. The difference between an SOB and a criterion obj ective is that the CRO is more specific and detailed as to standards and condi tions. However, a comprehensively written SOB may contain all the elements of a criterion objective. A simple criterion objective may read like an SOB if either or both of the conditions or standards are assumed. Rather th an split hairs about where the SOB leaves off and the criterion objective begins, remember that the SOB generally contains only a statement of performance. The criterion objective generally goes into more detail by adding st andards and conditions to the performance statements. See Figure 5.1. (as well as those later in this chapter) for a comparison of SOBs and criterion objectives.
Figure 5.1. A Comparison of Samples of Behavior and Criterion Objectives.

Section 5B—Constructing Criterion Objectives
5.3. Importance of Construction. The content of a criterion objective is far more important than the format used to communicate it, but the format can a nd does affect what we read and understand. Therefore, agreeing on a basic format and elements is an important step. By agreeing on and sticking to formats, which include all of the essential elements of inform ation needed to describe learning outcomes, we can be sure that we will not leave out any significant information in our instructional objectives.

Illustrative Samples of Behavior In terpreted as a Criterion Objective

1. define...(insert term) Without the use of references, define...according to AFM xx-xxx. (knowledge)

2. give an example of...(insert concept) Given th e concept of...as develope d in class, give new examples of (insert concept) c onsistent with its attributes. (comprehension)

3. write a position paper on... (insert subject) Using the library, local experts, and a topic assigned from the area of..., write a position paper which meets the content and format standards provided in cl ass and in the assigned text. (application)AFMAN36-2236 12 NOVEMBER 2003 53

5.4. Essential Elements. Both Air Force and civilian educators agree there are certain items of information that must be included in a well-stated criterio n objective. While many authors have different labels for these items of information, there is general agre ement that criterion objectives should consist of the elements shown in Figure 5.2.
Figure 5.2. Essential Elements of a Criterion Objective.

5.5. Formats. The precision and exact language of a criterion objective varies from instructor to instructor. The final product from any instructor, however, should be a clear statement that communicates the intended outcomes of learning. A common format fo r writing objectives makes communicating easier. Both the readers of the objective and the writer know inadvance the type of information that will be contained in the objective. Writing a criterion objective, though, is more than an exercise to satisfy an official requirement. An objective that meets the format re quirement but does not communicate fully is worse than no objective at all. An instructor can send students on a wild goose chase by giving them a criterion objective that does not honestly reveal what will be taught and tested. This practice will result in two curriculums—one that exists on paper and the real one used in the classroom. 5.5.1. The order in which the elements of a criterion objective are written does not matter, although the common sequence is conditions, performance or behavior, and standards. While generally written as a sentence or short paragraph ( Figure 5.3. , Format 1), criterion objectives may also be written in columns ( Figure 5.3. , Format 2). The sentence or paragraph format lends itself to easy reading and efficient use of space in catalogs or lesson plans. The column format, which may be layered vertically or side by side ( Figure 5.3. , Format 2), avoids questions of gr ammar and punctuation and lends itself to worksheets used to construct objectives. CONDITIONS A description of the testing envi ronment including those problems, materials, and supplies that will be given (include d) or specifically excluded from a measurement situation.

PERFORMANCE The observable student behavior (or the product of that behavior) acceptable to the instructor as proof that learning has occurred.

STANDARDS The qualitative and quantitative criteria against which student performance or the product of that perfor mance will be measured to determine successful learning.54 AFMAN36-2236 12 NOVEMBER 2003

Figure 5.3. Formats for Criterion Objectives.

5.5.2. For some sophisticated instructional systems, the column format may be preferable. It is easier to analyze with the objective broken out by elements into columns. We can tabulate the supplies and materials needed for instruction by analyzing the conditions column. If we examine the performance column on several plans, we may find that different instructors are having students do the same things. Then it would be more efficient to have a common unit of instruction to save duplicated effort. Analysis of the standards column could reveal questi onable or ambiguous standards. Both formats for writing criterion objectives require the same elements of information, but the column format may make data gathering and analysis easier. Format 1

Criterion Objective: (Conditions) Given a rifle with 10 bullets a nd a target 50 yards away, (Performance) shoot the target (Standards) so that 9 out of 10 bullets hit the bull's eye.

Format 2

Criterion Objective:

Performance Conditions Standards

Prepare a written critique of the Given: 1. All deviations from the proposed course manuals and pamphlets noted. 1. A proposal for a new course with the following supporting 2. At least one plausible documents: suggestion for improvement —System requirements offered for each deviation —Lesson plans noted. —Criterion objectives —Assigned readings 3. Written critique free from —Examination errors in spelling, grammar, and punctuation. 2. AFMAN 36-2236

3. AFMAN 36-2234

4. AFH 36-2235, Volume 10AFMAN36-2236 12 NOVEMBER 2003 55

5.6. Mastery Learning. Students who achieve the criterion objectiv e are often said to have "mastered" the objective. The term "mastery learning" has become strongly associated with criterion objectives and tests that measure the achievement of criterion objectiv es. From the point of view of criterion objectives, however, mastery does not mean that a student must pe rform a task or apply a principle with 100 percent accuracy. Mastery means that the student can demonstrate achievement of the criterion objective at the level of proficiency stated. The prof iciency level may well be 100 percent accuracy, but it may also be any other percentage or standard appropriate for the objective. The decision about whether a student has achieved an objective is essentially a "yes or no" decision. Regardless of the er ror rate permitted by the objective, if students can perform unde r a given set of conditions to the standards stated in the objective, we say they have mastered the objective. See Chapter 25 for further discussion of criterion-referenced measurement.
5.7. Content Communicated by the Criterion Objective. Criterion objectives should be as simple and straightforward as possible. They are designed and written to communicate. Because good communication is the ultimate goal, no absolute set of rules for constructing a criterion objective can be provided. The suggestions in this chapter may be of assistance in writing better objectives; however, they cannot be used to defend objectives that satisfy the format but do not communicate well. 5.7.1. No criterion objective can express all the possible conditions, performances, or standards for measuring student achievement. However, the criteri on objective is one of the best devices for communicating between two or more reasonable people. The content of a criterion objective should be constantly reviewed to make sure that format a nd jargon do not get in the way of full and accuratecommunication. Even though our criterion objective may seem clear to us, the objective is unsatisfactory if our students find it unclear or incomplete. 5.7.2. With the experience gained in writing criterion objectives for both academic and technical instruction, we can make some generalizations and establish some ground rules. We should keep the following points in mind as we write statements of conditions, performance, and standards:
5.7.2.1. Conditions. A well-written condition statement will tell students what to expect when they are evaluated. What will they encounter, for instance, when they sit down to solve a problem or answer a test item? Will they have a simulate d management problem to solve or just a word listfrom the text? If students are to write or speak on a topic, who will select the topic and what outside help may they seek? This kind of information gives students a context for their evaluation. If we describe the conditions well, our students canspend less time second-guessing about how they will be tested and more time learning the material. 5.7.2.1.1. If we remember that we are trying to dr aw a word picture, condition statements for physical tasks or performance items are relatively simple. Figure 5.4. through Figure 5.7. provide examples of criterion objectives that requi re overt physical performance on the part of the learner. The condition statements should be easy to identify even without the highlighting. The description contained in the criterion objective pr ovides an adequate word picture of the conditions under which the learner will be tested. Generally speaking, criterion objectives that call for overt student behavior give a more complete accounting of equipment, materials, and other physical factors in the environment than do obj ectives requiring responses to paper and pencil items.56 AFMAN36-2236 12 NOVEMBER 2003

Figure 5.4. Example 1 of a Criterion Objective.

Figure 5.5. Example 2 of a Criterion Objective.

Criterion Objective: Given a simulated patient requiring an ankle wrap and without assistance or references, WRAP THE ANKLE WITH AN ELASTIC BANDAGE correctly according to checklist EB-3.

Analysis: We are asking students to copy a simple task (wrapping the ankle). Following a memorized procedure, they will reproduce a method for such wrappings on a simulated patient. This task is much lik e folding a flag, replacing a spark plug, or arranging ribbons and devices worn by members of the armed forces. These tasks can be performed from memory w ith little understanding required of the performer. Students need no special skills to perform the task.

Criterion Objective: THE STUDENT WILL LOCATE AND REPAIR MALFUNCTIONS IN A PGN ELECTRICAL CIRCUIT BOARD. Th ese malfunctions will be limited to shorts and incomplete circuits planted by the instructor at those points most often affected by these malfunctions . Repair manual PGN-1, schematic drawings, screwdriver, pliers, standard electrical meters, and supplies as necessary may be used. Seventy-five percent of all planted malfunctions must be identified and repaired, with a 10-minute time limit for each

.

Analysis: Although two verbs are used in the performance statement, the requirement to "locate and repair" can be dealt withas a single objective. It may be inconvenient to separate closely associat ed tasks into separate objectives. If the need exists; however, each action verb could be used as the basis for a separate objective. Each of the three essentia l elements—performance, conditions, and standards—is stated as a separate sent ence. These sentences could be arranged in any order. The general condition statement describing the malfunctions is satisfactory, although more detail could be added if it served a useful function.AFMAN36-2236 12 NOVEMBER 2003 57

Figure 5.6. Example 3 of a Criterion Objective.

Criterion Objective: Students will be given the following purchasing information: name and description of item, stock number (if available), quantity and unit cost, whether the request is recurring or nonrecurring, a reason to purchase, and a date by which the item is needed. THEY WILL USE THIS INFORMATION TO PREPARE A TYPEWRITTEN AIR FORCE FORM 9, REQUEST FOR PURCHASE. The form must be according to AFMAN 23-110, Volume 2, USAF Supply Manual, have no errors in typing, spe lling, or format, be free from math errors and have the correct organization number and fund classification code

.

Analysis: This condition statement is quite specific because it lists each of the bits of purchasing information. An acceptable substitute would also be given the necessary information. Although the standards indicate that both the typing and math are to be error free, we could assume this requirement if it was omitted. There is no time standard given, so we can assume that any problem used in this exercise will be field-tested and a reasonable time allowed based on actual experience.

Legend: Conditions, PERFORMANCE, Standards58 AFMAN36-2236 12 NOVEMBER 2003

Figure 5.7. Example 4 of a Criterion Objective.

Criterion Objective: A physician's order to start intravenous fluids. Conditions: A cooperative, consci ous adult patient. A routine situation in which extraordinary measures are not required. Equipment and supplies, including

IV fluid (as directed) in bag or bottle Tape Tubing Arm board Alcohol swabs Tourniquet Needle IV pole

PERFORMANCE: START INTRAVENOUS FLUIDS IN ARM OR HAND.

Standards: Compliance with all hospital procedures and Air Force instructions (including presentation of self and proper identification of patient against orders). Acceptable nursing practice for procedure (cleansing of puncture area, site and application of tourniquet, and blood return). No swelling above puncture. Flow regulated to rate directed by physician's orders. Patient discomfort not to be excessive for procedure considering condition of skin and visibility and condition of veins. Task to be performed without assistance.

Analysis: This is a very significant task and a lengthy objective. However, there are several bits of information that limit the scope of the task. Notice that the intravenous fluids are to be started in the arm or hand. Other locations for administering these fluids such as the f oot or jugular vein are not included. In addition, the conditions rule out children, comatose patients, and hostile adults. The listing of equipment and supplies is quite complete, but they could have been assumed with a statement such as "given all necessary equipment and supplies." Several of the standards re quire judgment, but no useful purpose would be served by making them more precise (e.g., how much discomfort is excessive?) Notice that the first standard deals with a large amount of information by referring to two external sources for information.

Legend: Conditions, PERFORMANCE, StandardsAFMAN36-2236 12 NOVEMBER 2003 59

5.7.2.1.2. Condition statements for paper and pencil test items, ( Figure 5.8. through Figure 5.11. are written with the same techniques as t hose for performance testing. However, many instructors find them more difficult to write. Inpaper and pencil testing, it is usually more difficult to actually observe each of the elements (conditions, performance, and standards). Often we see only the product of the performance, such as a research paper, rather than actually observing the writing of the paper. But the elemen ts are still present, and the word picture for the paper and pencil test requi res no less precision than with other types of performances. 5.7.2.1.3. The condition statements in the paper and pencil testing, particularly Figure 5.9. , Figure 5.10. , and Figure 5.11. , focus more on the problem presented to the student than they do to the physical environment. In addition, by carefully manipulating the conditions of testing, we can produce criterion objectives that measure a wide range of proficiency.
Figure 5.8. Example 5 of a Criterion Objective.

Criterion Objective: After viewing a videotape that recreat es a discussion between persons of different racial or ethnic backgrounds and contains a number of instances of spoken and nonverbal communication labeled by the base human relations council as racially or ethnically degrading, IDENTIFY at least two-thirds of THOSE INSTANCES.

Analysis: This criterion objective again calls for students to exercise their understanding of a situation to see how well they agree with a panel of experts. Notice that the objective assumes those students will be doing the task on their own and within a reasonable, but unstated, period of time. This approach is correct as long as there is no apparent need to spell out supervision constraints and time standards.

Legend: Conditions, PERFORMANCE, Standards60 AFMAN36-2236 12 NOVEMBER 2003

Figure 5.9. Example 6 of a Criterion Objective.

Figure 5.10. Example 7 of a Criterion Objective.

Criterion Objective: Students will be given five scenario s with descriptions of misconduct punishable under the UCMJ and three articles from the UCMJ that may have been violated. Without assistance, notes, or references, SELECT THE ARTICLE THAT HAS BEEN VIOLATED IN EACH CASE. Satisfactory performance will be no less than 4 of 5 UC MJ articles selected correctly within 20 minutes per scenario

.

Analysis: This criterion objective describes a fair ly high-level task. Much is assumed about the difficulty and complexity of the scenarios, but it is probably not reasonable to attempt much more descriptive information here. Whatever the degree of difficulty, both the instruction and the testing should be essentially the same. "Correctly" in the standard make s two assumptions: that one or more experts in the field have determined what is correct, and that the student must come up with an answer consistent wi th that of the experts. Although very simple examples could be identified at the comprehension level, violations of the UCMJ are generally so complicated it is more likely they would require understanding at the application level.

Legend: Conditions, PERFORMANCE, StandardsAFMAN36-2236 12 NOVEMBER 2003 61

Figure 5.11. Example 8 of a Criterion Objective.

5.7.2.2. Common Errors. As we write condition statements for criterion objectives, we may find ourselves making errors common to many writers. So me of these errors are predictable and can be anticipated.
5.7.2.2.1. Mixing the Outcomes of Learning With the Process of Learning. Statements such as "by the end of this course" or "aftera lecture by the base commander" are generally not appropriate in a criterion objec tive. Remember, we are describing the test—not giving a course outline. If we include informati on like this, we are saying that part of the testing situation includes attending the course or listening to a lecture by the base commander—which is probably not the case.
5.7.2.2.2. Omitting Significant Items of Information. Significance obviously involves a value judgment. Something significant to one instructor about a testing situation may not be significant to another. Here again, the real measure is what will affect the performance of our students. We may assume our students understand that they may refer to their notes, look up words in a dictionary, or ask questions. But do they know this information far enough ahead of time to prepare for their exams adequately? Theywill, only if we have included all the significant items of information in our condition statement.
5.7.2.2.3. Including Information of Little or No Significance. Some condition statements look like a list of items purchased at a shoppi ng mall. Everything imaginable is listed. Quite often, the weekend shopping inventory is lengthy, including insignificant items while excludCriterion Objective: Using a personally selected topic and wit hout restrictions on reference materials or lesson planning formats, PREPARE A DEMONSTRATION-PERFORMANCE LESSON PLAN c ontaining: A criterion objective with all required elements, A valid criterion-referenced test that determines mastery of the criterion objective, and A teaching plan generally consistent w ith accepted principles of learning and instructional design.

Analysis: This criterion objective is broad inscope and complex. Many pages of information are dealt with in the standa rds. It is probably unreasonable to put much more information into the objective. The objective calls for several qualified value judgments ("generally consistent with" and "validly determines mastery"), but such judgments can be expected in an objective at this level. In this case, both judgments require the instru ctor to be an expert, and neither can be quantified easily.

Legend: Conditions, PERFORMANCE, Standards62 AFMAN36-2236 12 NOVEMBER 2003

ing significant items. One example of this erro r involved a course designed to teach mid-level supervisors how to manage effectively. The pr imary methods were case study and guided discussion. The criterion objectives were so poorly written they destroyed the credibility of the instructor. While completely overlooking the significant conditions for evaluation, such as simulated management problems in case form, th e writer was very careful to mention in the criterion objective that the students would be given paper and ballpoint pen. We can hardly argue against paper and ballpoint pen, but that bit of information could certainly be assumed without much effect on the objective. The info rmation left out was significant and did have an effect on the students. Other routine environmental factors, such as light, air, tables, and chairs need not be listed unless there is some concer n that they might not normally be available.
5.7.2.2.4. Talking Around the Genuine Conditions. Some writers of objectives seem to have trouble telling their students what to expect in terms of the conditions for testing. Often it is not a lack of writing skill or subject expertise; these instructors still have difficulty accepting the idea that they shouldbe as open with their students as suggested here. For years, many instructors have succeeded in keeping their students in constant suspense about what they were required to learn, and instructors who have really developed the skill of deception have been able to transfer them to writing criter ion objectives. Often such expressions as "given necessary supplies and materials" or "given any management situation" are used to hide a more understandable statement. Does "given the materials," mean they will all be pre-selected and laid out for the students, or do they have to make some decisions about what materials or supplies are appropriate? Does "any management situation" really me an "any," or is there actually a very limited set of situations that could be described more precisely?
5.7.2.2.5. Describing the Mechanics Rather Than Conditions. Particularly in an academic setting, it is often easier to describe the me chanics of test administration than conditions affecting student performance. A condition stat ement like "given a test with 30 items about..." misses the whole idea of the criterion objective. Such a statement is probably inappropriate unless the test is measuring a single criteri on objective that requires 30 common items. Those who fall victim to this attractive shortcut te nd to write worthless condition statements. The temptation is to view paper-and-pencil tests containi ng several items as a single learner performance. It is possible that a written test of many items may measure a single criterion objective, but this is not usually the case. 5.7.3. Most paper-and-pencil tests measure more thanone criterion objective: Test items 1 through 5 on a particular exam may measure criterion objective #1; items 6, 8, 10, 12, and 14 may measure objective #2; and so on. A paper-and-pencil test m easuring several objectives generally does not need another overall criterion objective. Chapter 25 discusses one exception where the instructor has grouped individual criterion objectives together in a package and established an overall passing criterion. When asked what objectives a test is measuring, we should be able to point to the criterion objectives being measured by the various items within th e test. In our operating instructions, we may wish to specify the minimum number of objectives required to pass the course rather than some overall test score or percentage for the exam. 5.7.4. The word picture created by describing the c onditions should be clear and complete. Our students, another instructor, or any other reasonable person should be able to understand and recreate, if necessary, the testing environment in whic h we intend to measure student learning.AFMAN36-2236 12 NOVEMBER 2003 63

5.8. Performance. The performance portion of the criterion ob jective tells us what the student will be doing, or will have done, to prove learning. The descri ption of behavior is most easily communicated by using an action verb and a clear description of the object of that action.

5.9. The Action Verb. There is no mysterious list of "magic" or forbidden words to learn. Generally speaking, active verbs that describe observable, measur able student behavior are best. The criterion objectives illustrated in this chapter contain several acce ptable verbs for describi ng student behavior. As you can see from these examples of performance testi ng and paper-and-pencil testing, the verb gives a good indication of what the student will do. But verbs thatdescribe an internal state of mind (know, comprehend, etc.), while acceptable to indicate the level of learning, do not give us the information we need to write a criterion objective. Figure 5.12. shows a sampling of verbs generally felt to be acceptable and unacceptable for writing criterion objectives. 5.9.1. Some descriptions of student behavior do not fall clearly into e ither performance testing or pencil-and-paper testing. We may have to refine the performance description further to communicate our real intent. This ambiguity is rarely an issue for ov ert physical tasks, but it often enters the picture when the behavior is incidental, rather than critical , to the objective, which is the case in most pencil-and-paper testing. For instance, does the verb "write" in a criterion objective communicate a critical physical act, or is it incidental to the act of "editing," "composing," or "solving?" The listings in Figure 5.12. illustrate several verbs that may fall into either the critical or noncritical categories.64 AFMAN36-2236 12 NOVEMBER 2003

Figure 5.12. Verb Descriptors of Student Performance (Criterion Objectives).

5.9.2. When we write the performance information, it may help to visualize the actual measurement situation. What will we actually see? What will real ly be evaluated? Will we watch the student actually perform the task or will we collect some product produced by the student—written paper, multiple-choice exam, or a properly folded flag?
5.10. Object of the Action:5.10.1. An observable, measurable action verb gives half of the required information in the performance description; the object of the action verb is the other half of the required information. The object of the performance statement is not usually emphasized as much as the action verb, but it is very important. Note that in the sample criterion objectives ( Figure 5.4. through Figure 5.11. ) the object of the performance is clearl y stated and included as a part of the performance statement. Acceptable Verbs (observable and measurable ) Unacceptable Verbs (going on inside the learner) shoot define know describe comprehend splice apply fasten analyze turn synthesize edit evaluate assemble understand drive appreciate remove be familiar load speak

May be acceptable depending on context (affected by writer preferences, rules of the school, need for extreme precision, etc.)

Acceptable Verbs Better if Specific Behavior is Critical

list list in writing solve write solution critique verbally critique summarize write a summary identify identify by circling

nameAFMAN36-2236 12 NOVEMBER 2003 65

5.10.2. Objectives frequently indicate that a student will be able to write, re pair, or critique a "thing." Often the "thing" in the performance statement is taken for granted and gi ven little attention. The object of the action of the verb should be preciselystated. Here again, reasonableness can be a problem. Is the military letter, the object of the verb "write," the real target of performance, or is the target really "any written communication" with the military letter just a convenient device for testing? In addition, does the object named have a generally accept ed definition, or is a definition or description necessary? Do our readers share the jargon or technical language used in the objective?

5.11. Common Errors. Of the three criterion objective elements, the performance description should be the least difficult to write. There are, however, a fe w common errors we can anticipate and avoid. Some of these errors are:
5.11.1. Substituting Levels of Learning for Action Verbs. Writing objectives for specific levels of learning was introduced in Chapter 2 and Chapter 3

. Chapter 4 explains the approach to planning indepth level-of-learning objectives. These levels, or categories of learning, serve a valuable function by grouping equivalent student behaviors for sampli ng. They are not meant to be substituted—nor can they be—for the action verb in a criterion objectiv e. Any attempt to replace the criterion objective action verbs with verbs from the levels of learning (know, comprehend, etc.) will produce an unsatisfactory criterion objective.
5.11.2. Using Vague or General Action Verbs Wh en Specific Verbs Communicate Better. It may seem that choosing between good (specific) and poor (vague and general) verb s is quite simple. However, some verbs may seem to be better than they are at first glance. Verbs such as demonstrate, operate, or prepare may leave important unanswered questions: to demonstrate—by doing what? to operate—which operations? to prepare— which tasks of preparation? It is possible to use such verbs in well-written objectives; however, they have to be seen in context and evaluated on their ability to communicate clearly.
5.11.3. Overly Generalized or Poorly De fined Objects of Performance. A verb may clearly describe the desired learner behavior, but the object of that behavior may be poorly stated. Often, such terms as "any" and "all" in a performance statement do not mean "any" or "all." Generally, we find some limit on the type of problem or kind of object being acted on. It is unlikely that a student in a leadership school can solve all management problems. It is just as unlikely that a technical student could make all repairs on any object. The scope of the performanc e may be limited in the performance statement or in the condition statement. It does not matter which item does the limiting as long as the limits are communicated.
5.12. Summary. The performance section of the criterion ob jective must contain an action verb (do what?) and an object (to what?) . Along with the condition statement, the performance description describes everything of significance that will occur during the test or measurement situation. All that remains is for us to tell our students what criteria or standards to use to evaluate their performance.

Section 5C—Standards

5.13. Overview. The standards for criterion objectives are often difficult to write. Many difficulties show up quickly as we try to express standards to students so they can be understood. Some of our problems can be traced to the limits of written la nguage, others to a lack of consci ous awareness of our own standards, and still others to philosophical problems many of us have with revealing our standards to students.66 AFMAN36-2236 12 NOVEMBER 2003

5.13.1. Many standards are difficult to express in wo rds. Standards should describe how well students must do a task to be acceptable, but often langua ge lets us down. How smooth should a "smooth" finish be? When is a description of an object or ev ent "complete?" How clear is a "clear" explanation? This problem with language cann ot be completely overcome. A ca reful, honest attempt to describe standards, along with trial and error, is the best we can hope to do.
5.13.2. The creative use of models and other media to supplement our word pictures may be helpful. We must realize, however, that the perfect (satisfyi ng to all readers) set of standards for a complex criterion objective is probably yet to be written, particularly for qualitative standards. This problem may be best summarized by the instructor who says, "I knowa passing effort when I see one, but I just can't express it in the right words." 5.13.3. Some standards, particularly quantitative, s hould be easy to express because they lend themselves to precise terms. However, even thes e standards are often poor ly written and incomplete because of an instructor's inability to verbalize all of the standards that will be used to evaluate student performance. 5.13.4. When there are many standards for an objective, there is a temptation to list the most obvious and easiest to express and measure. The student is put at a great disadvantage, however, when additional unstated standards are used to determine satisfactory performance. Instructors often apply unstated time standards and assumed quality standards to student work. No standard should be used to evaluate student performance that is not written into the criterion objective or reasonably assumed. Ifa student can meet all of the criteria in the criterion objective, but the performance is still unsatisfactory, we should reevaluate the objective for additional standards. 5.13.5. A third and more fundamental reason for poor statements of standards is negative instructor attitude about revealing standards. In spite of the Air Force commitment to ISD and instructional objectives, many instructors feel there is something wrong with telling students exactly what is expected of them. Most of us are products of trad itional schools that were not "systems" oriented. Few of us have had more than a course or two, in or out of the Air Force, where objectives were carefully spelled out. As successful student s in those traditional schools, some instructors question ISD in general and criterion objectives in particular.
5.13.6. If instructors cannot see the benefits of this approach in terms of more relevant course content, as well as more effective, efficient instructional techniques, they may have trouble expressing their own objectives. Some would say, "I never had anyone spell the course out for me like that. Why should I spoon-feed my students?" This weak argum ent against writing criterion objectives is best summarized by an instructor who was overheard saying, "I'm not going to write criterion objectives. Why, if I tell my students exactly what they have to do and how well they have to do it, they will only study that and nothing else!"
5.14. Types of Standards. Many different types of standards may be used to evaluate student performance. Some of these types include:
5.14.1. Referring To an External Authority. Making reference to an external authority in criterion objectives may save time and space. Authors of source material often state qualitative and quantitative standards very well. Plus, rephrasing may be a wa ste of time with no improvement. Reference to specific instructions, manuals, authors, checklists, or rating scales is appropria te and should be encouraged. Figure 5.3. through Figure 5.8. illustrate ways in which extern al authority may be referenced.AFMAN36-2236 12 NOVEMBER 2003 67

5.14.2. Expressing or Implying a Percentage or Ratio. We frequently see a percentage or ratio in criterion objectives, as when a student must solve 80 percent of a set of probl ems or locate a malfunction in 2 of 3 cases. Where this type of standard is appropriate, it is easy to state and easy to measure. Figure 5.5. through Figure 5.8. illustrate percentage or ratio standards.
5.14.2.1. We often find it natural to write a stan dard that says "with 100 percent accuracy" or "without error." If there is any doubt the reader w ill not assume this standard we should include it, although we may find that the phrases become veryrepetitious. Often, we can assume the standard of "no error" in our objectives without spelli ng it out as long as our students understand what we

mean. 5.14.2.2. Figure 5.3. and Figure 5.6. spell out the standard of "no error." Other objectives, such as in Figure 5.7. and Figure 5.11. assume the "no error" standard, and a percentage or ratio are not stated.
5.14.3. Specifying Repetition or Redundancy of Testing. Standards that express a percentage or "no error" usually indicate a one-time test; that is , if the student does something without error once or locates two or three malfunctions one time, the objecti ve has been met. If this is what we intended, we have said all we need to say, but we may want the student to show mastery of the objective more than once to prove that learning occurred. In that case, to a standard such as 75 percent as shown in Figure 5.5. , we would add another statement to the standard, such as "in two consecutive tests" or "in two of three examinations separated by at least 2 weeks." We may even wish to extend our test or retest period beyond graduation and use the retest as part of the graduate follow-up program.
5.14.4. Specifying Physical Measurements and Tolerances. Many types of physical measurements will appear in criterion objectives, such as tole rances, weight, number of words, pounds of pressure, distance, degrees, and rate. We should use these standards, which are generally easy to express, when appropriate. Figure 5.7. provides an example of this type of physical measurement in the rate of flow of the intravenous solution.
5.14.5. Specifying the Time Requirements. We should express time as a standard when it may affect student performance. If time is not included as a standard, we assume that a "reasonable" period of time will be allowed for performance testing. A reasonable period allows any learner who has mastered the skill to complete the task without running outof time. If we plan to time the test or measurement in any way, we normally include time as a standard in the criterion objective. In paper-and-pencil te sting, we may or may not include time for each objective being tested. In any case, the total time for the test should be the sum of times estimated for each of the objectives being tested. Figure 5.5. and Figure 5.10. illustrate the use of testing time as a specific standard. "Reasonable time" is an assumed standard in the other examples.
5.14.6. Specifying the Rate of Production. Many performance skills are m easured in rate of production. This measure should be included in appropr iate objectives. Skill development situations that might use this standard include keyboarding, typing, shorthand, and production work. The typing problem described in Figure 5.6. could be modified to include pr oduction rate as a standard by adding "at a rate of 12 forms per hour."
5.14.7. Specifying Qualitative Requirements. Many of the most significant standards applied to student performance are qualitative and may require judgments involving sight, sound, taste, and other inexact measures. To the greatest extent possi ble, we should legitimatelyquantify standards. If there is a color or smoothness scale, we should use it . If there is a format for a written course require-68 AFMAN36-2236 12 NOVEMBER 2003

ment, we should refer to it. If a "properly supported" argument means, among other things, that it includes at least three acceptable statistics or examples, we should state that in the objective. But often no such external standard exists, and qualitative st andards are often genuinely difficult to express. If there is no reasonable way to define the qualita tive standard objectively (e.g., run smoothly; fold neatly; develop fully), we use the qualitative term. We must keep in mind, however, that the qualitative term may not be interpreted the way we mean it. All of the criterion objectives in this chapter contain one form or another of a qual itative standard, but particularly Figure 5.3. (standard 2) and Figure 5.7. (standards 2 and 5).
5.14.8. Specifying Degrees of Supervision or Assistance. The degree of supervision or assistance provided during a test may affect the quality of th e results. Students require a much higher degree of skill and need a greater depth of understanding to perform a task without supervision or assistance. If we do not specify a level of supervision or assist ance, it is assumed that the student must perform without supervision and without assistance. Figure 5.7. illustrates a specific assistance standard. The other examples may be interpreted to read "without supervision and without assistance."

5.15. Common Errors. A wide range of difficulties may be en countered in writing standards for criterion objectives. There is no way, short of extensive experience, to identify all the possible problems we may encounter. The information that follows deals with many of the more common errors and should help in writing better objectives.
5.15.1. Overestimating the Value of an External Authority (Regulation, Pamphlet, Rating Scale). Information actually contained in the cited authority may not be precise enough or complete enough to use for student evaluation. The authority cited might include unacceptable or inappropriate information for the learning situation. In many cases, since these documents were not written to be used as part of a criterion objective, they may contain generalities that are not easily understood or measured.
5.15.2. Inappropriate or Inexact Percentage or Ratio. Few standards are as easy to express and measure as percentages. Unfortunately, we can be tempted to use exact percentages when other more difficult criteria are appropriate. Instructors often mistakenly use percentages because of past practice rather than a careful analysis of real criteria. Of ten we express or imply a standard of 100 percent accuracy when it is unwarranted and unrealistic. We can easily fail to take into account such factors as fatigue or simple errors in math in cases where pe rformance at less than 100 percent is acceptable for mastery.
5.15.3. Assuming Retention Based Upon a Single Evaluation. If retention is important, we need to program reasonable retesting into our learning situa tions. Students should be told that they are going to be tested on the same objective more than once (for example, query, block exam, or phase exam).
5.15.4. Understating or Oversimplifying Measurements. Some instructors write standards for criterion objectives to the lowest acceptable level of pe rformance. Such a standard may be invalid if performance on the task is generally expected to be higher than the bare minimum. We may have many reasons for setting classroom standards higher than th e lowest acceptable level of performance on the job. Additionally, we may make the mistake of using convenient, easily measured standards when other more difficult to express standards are more appropriate.AFMAN36-2236 12 NOVEMBER 2003 69

5.15.5. Overlooking Time As a Factor in Testing. Many criterion objectives are written without reference to time; however, time constraints are imposed upon students during testing. When time standards are set, they are often guesses and not verified to determine their reasonableness.
5.15.6. Incomplete or Inaccurate Production Rates. The per-minute or per-hour production rate is often an important standard. Additionally, it is ofte n important to specify the period over which the per-minute or per-hour standard is to be maintaine d. For example, there is a big difference in typing 60 words per minute for 1 minute rather than for 10 minutes. We may also make an error in the actual unit of production being measured. It ma y be more appropriate to measur e the number of business letters typed rather than the number of words type d or the number of words typed per minute.
5.15.7. Poorly Stated Quality Standards. Quality standards are generally difficult to write as compared to measures of quantity. However, we ofte n excuse poorly written standards simply because they contain quality standards. We should avoid such expressions as "to the satisfaction of the instructor" or "until the work is acceptable." With these statements, the instructor has succeeded in keeping the real standards from the students. Students often have no idea what these types of expressions may

mean. 5.15.8. Failing to Account for Supervision or Assistance. We may assume that students will perform a task without supervision or assistance and omit that information from the criterion objective. However, when students don't realize they must perform without supervision, we may be forced to provide additional instruction during the exam, answer questions, or inadvertently correct obvious errors as they are being made.
Section 5D—Planning Tools
5.16. Task Steps As Planning Tools: 5.16.1. Criterion objectives give a great deal of info rmation about what a student should be able to perform after instruction. But criterion objectives do not tell us how we are going to get our students from where they are to where we want them to be . An effective technique for planning our instructional strategy is to analyze the mission, job, duties and tasks a person may be expected to perform. While it is beyond the scope of this publication to go in depth with an explanation of this analysis, the reader must have an overall knowledge of the terms and how they are used to develop CROs. 5.16.2. The curriculum designer must be aware of th e mission goals and the job a person will perform to accomplish that mission. The following information is paraphrased from AFH 36-2235, Volume 10: 5.16.2.1. A job is made up of the duties, tasks, and task elements that a person performs. It is the basic unit used in carrying out the personnel actio ns of selection, traini ng, classification, and assignment. 5.16.2.2. We can then examine the job to determine all the duties and tasks performed by a person at a given skill level. Duties are large segments of work performed by an individual. 5.16.2.3. A given duty is made up of tasks or units of work activity. Tasks usually have a clear beginning and end that is observable and measurab le and frequently results in a product that can be evaluated. A task is not dependent upon other tasks, although it may fall in a sequence with other tasks in a duty or job array.70 AFMAN36-2236 12 NOVEMBER 2003

5.16.2.4. Task steps are the specific actions a person performs to accomplish a task. These are often identified as checklists, technical order procedures, operating instructions, etc. Once you know the level of the objective, (for example, mission level, job level, duty leve l, or task level) the curriculum designer can write the appropriate CRO. In this case we are interested in the task steps required to perform a given task so th e designer would write a task level CRO. 5.16.3. For planning purposes, task steps are expre ssed behaviorally, but ar e usually written without conditions or standards, as with criterion objectiv es. If we are individualizi ng or self-pacing instruction, we may wish to write task steps as criterion objectives, but this is generally not required. If we can enable our students to perform all the task steps correctly, then they will be able to accomplish the CRO for that task. 5.16.4. Analysis of a criterion objective into a relati vely simple sequence of task steps might look like Figure 5.13. In this illustration, a student who starts withno skill must learn to perform four task steps or gain four items of supporting know ledge in order to show mastery of the objective. An analysis of this type is useful to make sure that all signi ficant task steps and the necessary background are accounted for in planning the course or lesson. Listi ng these steps in order will provide a checklist if we wish to evaluate our student s as they progress independently.
Figure 5.13. Task Analysis of a Criterion Objective.

5.16.5. Although the task steps must be performed ina 1-2-3-4 sequence, our students may not needto learn them in that order. For instance, we may not have a workstation for each student, or some students may already possess one or more of the sub-skills. Figure 5.14. illustrates two of the possible instructional designs that might result from such an analysis of a criterion objective.AFMAN36-2236 12 NOVEMBER 2003 71

Figure 5.14. Sample Instruction Designs For a Task Analysis of a Criterion Objective.

5.16.6. This criterion objective represents the same learning outcome illustrated in Figure 5.13. In that figure, however, we were illustrating a sequence inwhich all of the task steps had to be performed in sequence to demonstrate mastery in the skill described in the criterion objective. Figure 5.14. shows two of several possible sequences in which the students might learn the skill. The skill and the component steps are imaginary, but the diagrams can be used to make the desired point. In Figure 5.14. , example 1, the analysis of the task has revealed th at task step 1 must be learned before task step 2 can be learned and that task step 3 must be learned before task st ep 4. But, we have also discovered that task steps 3 and 4 are not dependent upon learni ng to perform task steps 1 and 2. Such a situation might exist when task step 1 is "measuring," task step 2 is "computing," and task steps 3 and 4 are "sawing" and "fastening." Example 2 represents another analysis in which the performance of task steps 3 and 4 are dependent upon learning the first two task steps, but the order in which students learn the first two task steps is not significant. 5.16.7. The value of this type of analysis is twofol d: (1) We can account for all component tasks and spell them out in detail, and (2) As we discover which skills are dependent upon other skills in the learning process, we become aware of appropriate strategies for instruction sequencing. These illustrations are quite simple, but the analysis of more complex skills may look like the electrical wiring diagram of a color television set. Refer to other in structional design texts for more comprehensive and detailed explanations of this procedure.72 AFMAN36-2236 12 NOVEMBER 2003

5.17. Summary. The criterion objective is an effective way to communicate precise outcomes of learning. Each criterion objective describe s the proof an instructor will accept that a learning outcome has been mastered. 5.17.1. While many formats for writing criterion objectiv es may be used, they all require three basic items of information: performance, c onditions, and standards. The checklist in Figure 5.15. will help us write criterion objectives with all the required el ements. Criterion objectives provide the basis for tests that measure student performance fo r comparison with an objective standard.
Figure 5.15. Criterion Objective Writer's Checklist.

5.17.2. In spite of a tendency for be ginning writers to make fairly pr edictable errors and to oversimplify the learning process, criterion objectives have changed the face of instructional programs. Criterion objectives are invaluable to the technical instru ctor for specifying exact opt imal levels of training.They are also valuable to the academic instructor in establishing the base line of instruction or minimum objectives upon which more ope n-ended courses, such as prof essional military education, are developed. Criterion objectives also he lp us to consider the environment for testing as well as the standards for evaluation so we can deal fairly with our students. In addition, criterion objectives help us to Conditions : 1. Are there any physical factors, such as, materials, equipment, supplies, or references in the performance situation that should be noted as being present or specifically excluded? No ( ) Yes ( ) If yes, specify: 2. Is there any problem or situation that will confront students to which they must respond? No ( ) Yes ( ) If yes, specify: Performance : 1. What are students required to do? Specify with an action verb: 2. To what? Specify: Standards : Which of the following standards will be used to evaluate student performance? ( ) 1. External authority. Specify: ( ) 2. Percentage or ratio. ( ) Without error/100 percent correct. ( ) Other. Specify: ( ) 3. Requirement to be tested more than once. Specify: ( ) 4. Physical measurements and tolerances. Specify: ( ) 5. Time limitation. Specify: ( ) 6. Production rate. Specify: ( ) 7. Qualitative standards. Specify: ( ) 8. Degree of supervision. Specify:AFMAN36-2236 12 NOVEMBER 2003 73

better visualize our measurement in struments so we can make valid reliable assessments of student achievement.74 AFMAN36-2236 12 NOVEMBER 2003

Chapter 6

DEVELOPING THE LESSON PLAN
6.1. The Importance of a Lesson Plan: 6.1.1. A lesson plan is a plan for learning. As is true in most activities, the quality of planning affects the quality of results. Successful executives and professional people know the price of excellence is careful preparation. A lawyer spends hours planning a case before appearing in court. A minister does not ad-lib a sermon, but plans days or weeks in a dvance. In anticipation of the big game, the coach spends hours planning the plays and watching the te am execute them. Should we attempt such a complicated process as learning with less attention than is given to other important activities? The answer is obvious: of course not. The effective instructor devotes much time and energy in carefully planning and preparing each lesson, whether the lesson enco mpasses one or several periods of instruction. 6.1.2. To ensure the greatest probability of learni ng, we must carefully select and arrange activities that will produce the desired learning outcomes in our students. Only through careful planning can we be certain we include all necessary informati on and have our lesson plan properly organized to achieve the lesson objective. 6.1.3. The complete cycle of lesson plan ning includes the following steps (paragraph 6.2. through 6.10. ): determining the objective, researching the to pic as defined by the objective, selecting the appropriate instructiona l method, identifying a usable lesson planning format, deciding how to organize the lesson, establishing a stra tegy statement, choosing appropriate support material, preparing the introduction and conclusion of the le sson, and preparing a final outline.
6.2. Determining the Objective. We often begin our lesson planning with an objective or objectives clearly in mind. At other times, the objective may be shaped by the research and additional planning we do. In other words, although the first step of the less on planning process is to determine the objective, our objective may not fully evolve until after we have completed other steps of the process. 6.2.1. Objectives need to be student-centered. We s hould not state them in terms of what we want to teach, but rather in terms of what we want our stud ents to learn. For instance, the objective of a lessonon developing a lesson plan might be for each student to know the eight steps of effective lesson planning as listed in this chapter. 6.2.2. Of course we might teach the lesson at a hi gher level than the knowledge level. We might want each student to comprehend the eight steps appropriate to effective lesson planning or even to be able to apply the eight steps of lesson planning. But wh atever the level, the student-centered objective should guide our subsequent planning. Without a clear objective, we won't know if we, or more importantly, our students ever get there.
6.3. Researching the Topic. After writing or determining an instructional objective, we are ready to decide on the main points of the lesson and gather materials about the lesson topic. Normally we do not collect a mass of research materials and then devel op an objective to match the findings. Not only is this approach inefficient, but it is also likely to be ineffective. It may well ignore the specific needs of the students and the Air Force. The objective should determine the research needed. On the other hand, research may justify a decision to modify an objective or rearrange main points for greater accuracy or clarity.AFMAN36-2236 12 NOVEMBER 2003 75

6.3.1. Appropriateness and usef ulness are two important criteria for selecting relevant material. To be appropriate, information should relate to the lesson objective and have a high possibility for student retention. To be useful, it should aid both the instru ctor and the students in the teaching-learning process. If the instructor selects material solely on the basis of its interest, a lesson may be filled with interesting information of little learning value. On the other hand, dry, uninteresting facts—even though very important—may also defeat the instruct or's purpose. Students are more likely to grasp and retain facts and concepts enriched with interesting support material and arranged in a way to enhance learning. 6.3.2. With the objective clearly in mind, we are now ready to gather actual material or do research on the subject. The sources for this material are our ow n experiences, the experience of others we gain through conversation and interviews, and written or observed material. Instructors concerned with teaching a good lesson will often draw from all of these sources.
6.3.2.1. Self. The first step in researching a lesson topic is to see what we ourselves know about the subject. Our personal knowledge may suggest a tentative organization, but more importantly, it will point to gaps in our know ledge needing further research.
6.3.2.2. Others. The second step in the research process is to draw on the experience of others. The most fruitful source is the expert who may he lp us clarify our thinking, provide facts and testimony, and suggest sources for further research.
NOTE: While personal experience, conversation, and inte rviews provide valuable content for lessons, we must usually do further research. If we have properly narrowed our subject and kept the purpose in mind, our research will be easier.
6.3.2.3. Library and Internet. Libraries and the internet provide an abundance of sources such as books, newspapers, magazine articles, scholarl y journals, abstracts, subject files, and microfilms. Quantity is no problem; quality is more diff icult. We must always concern ourselves with the accuracy and relevance of the material we select. For example, using an article from 1950 to discuss atomic physics today might well lead to inaccurate, irrelevant conclusions. 6.3.3. The next step in the research process is to evaluate the material gath ered. We will probably find we have enough material for several lessons. We must now combine some ideas, eliminate others, and perhaps expand on what we found in the research materi als. We will also want to give special attention to the types of support material we have selected (definitions, examples, comparisons, statistics, and testimony). Later in this chapter we will di scuss types of support material in detail. 6.3.4. Sometimes we have an organi zational pattern in mind before we start. If not, as we gather our material, we will probably see the ideas begin to forminto some type of pattern. Later in this chapter, we will discuss ways of organizing the lesson.6.3.5. During the research phase, the instructor is likely to find material students should read to prepare for a given class session. If we keep this possi bility in mind when we begin our research, we can prepare a suggested student reading list and save time in selecting student references. When deciding on supplementary reading for the students, choose inte resting and informative materials that reinforce or support the lesson objectives.
6.4. Selecting Instructional Methods. After deciding exactly what to teach, the instructor determines how best to teach it and what instructional method to use. When related to instruction, "method" refers to a combination of techniques or skill s used by the instructor to engage students in meaningful learning76 AFMAN36-2236 12 NOVEMBER 2003

experiences. A method is a broad approach to instru ction—for example, the lecture method or the guided discussion method. A technique, on the other hand, is a sp ecific concrete skill or procedure used in implementing a method—for example, the technique of usi ng the chalkboard or of using an analogy as support material. Chapter 12 provides an overview of several instru ctional methods available to Air Force instructors. Chapter 13 through Chapter 18 present a number of these methods in detail.
6.4.1. Philosophy Underlying Selection. We should choose a teaching me thod suited to the student's needs as a learner. In making the selection, we consider the ways people learn—by doing, discussing, listening, observing, and participating. We should select the instructional method most effective for guiding students toward desired learning outcomes. Ou r role is to select the method and the techniques that will result in a meaningful learning experience.
6.4.2. The Selection Process. No one method is suitable for all teaching situations, because no single method is sufficiently flexible to meet the needs of students in every learning situation. Chapter 19 presents an expanded discussion of the factors to consider in the selection of teaching methods. In general, the nature of a learning outcome suggests the type of activity most helpful to students in achieving that outcome. 6.4.2.1. If, for example, we want students to gain skill in performing a certain task, one of the activities should be practice in performing the task. If the desired outcome is knowledge, students should observe, listen, and/or read so they can re late what they are learning to their own experience. If students must learn to apply a principle, the instructor should ask them to solve problems or perform tasks requiring an a pplication of that principle. 6.4.2.2. The instructional approach we choose for one learning outcome may be different from the approaches we select for other outcomes in the same lesson. Our primary concern is to plan and select the most appropriate approach for students to achieve each outcome.
6.5. Lesson Planning Format. Good lesson planning is essential for any systematic approach to instruction. Although many instructors become discouraged with the time required for good lesson planning, a well written and properly used lesson plan can be a very worthwhile teaching aid. 6.5.1. Experienced instructors use written lesson plan s for a variety of purposes. They can be checkpoints to ensure well-planned learning experiences . They can serve as teaching guides during lessons and as references for other instructors who may teac h for us in emergencies. They can also serve as convenient records of an instructor's planning techniques and methods of teaching. One of the most practical functions of lesson plans is they serve as step-by-step guides for instructors in developing teaching and learning activities.
6.5.2. Authorities differ about the content and form of lesson plans, and many commands and schools have developed their own formats to satisfy particular needs. On the whole, however, most authorities generally agree on the essential ch aracteristics of a good lesson plan. Figure 6.1. lists these characteristics as well as those items of information routinely included.AFMAN36-2236 12 NOVEMBER 2003 77

Figure 6.1. Recommended Content an d Components of a Lesson Plan.

6.6. Organizing the Lesson. After we have researched the topic, selected the appropriate instructional method, and identified the lesson planning format to use, we must decide how to organize the lesson. Every lesson needs an introduction, body, and conclusion. In most instances, the body of the lesson should be prepared before the introduction or conclusion. Af ter we prepare the body or main part of the lesson, we will be in a better position to be gin or conclude the lesson. The first consideration in planning the body is how to organize the main points. Organization of subpoints is also important. Arranging the main points and subpoints of a lesson will help both the instructorand the students—the instructor in teaching, the students in learning. Most lessons, regardless of their le ngth, divide nicely into two to five main points. Major Components Information/Materials to be Included

Part I: Cover Sheet Lesson identification Instructor's name Method used Objective (with behavioral indicators of achievement) Main teaching points or task steps References consulted Instructional aids used Handouts needed

Part II: Lesson Development Content outline Notes on delivery techniques Cues for use of visual aids Notetaking space for instructor inputs Comments on effectiveness of plan made after lesson is presented

Part III: Evaluation Test items Record of student performance on test items Statistical analysis Record of test item revisions

Part IV: Related MaterialsHandouts Homework assignments Reading assignments Supporting documents Actual instructional aids Class text78 AFMAN36-2236 12 NOVEMBER 2003

6.6.1. Typical ways of organizing main or subpoints of a lesson are by chronol ogical, sequential, spatial, cause-effect, problem-solution, pro-con, or topical patterns (paragraph 6.6.1.1. through 6.6.1.7.2. ). Furthermore, certain strategies can be used with each pattern from known to unknown, for instance, or from simple to complex. How does an instructor decide which patterns and strategies to use? The lesson material will often organize itself easier with one pattern and strategy than with another. Let us consider how various patterns and strategies can be us ed to organize the main points of a lesson.
6.6.1.1. Chronological. Our vocabularies are filled with words that refer to time: now, tomorrow, yesterday, today, sooner, later, earlier, last week, a month from now, 4 years ago, next time. We work, play, sleep, and eat at certain times. Major events in our lives are organized by time: births, engagements, marriages, deaths. The chronological ortime pattern of lesson organization is a natural way of arranging events in the order in whic h they happen. Similar to this is an arrangement in the way certain things are carried out, their sequence.
6.6.1.2. Sequential. Certain processes, procedures, or hi storical movements and developments can often be explained best with a time sequence pattern. 6.6.1.2.1. The medical technician presenting a le sson on mouth-to-mouth resuscitation would probably use a sequential ordering of the ma in points: (1) preliminary steps—proper body position, mouth open, tongue and jaw forward, (2) the mouth-to-mouth process, and (3) caring for the patient once breathing resumes. The sequent ial order is also a logical approach to lessons dealing with such subjects as how to pack a parachute, development of the F-15 fighter, or how to prepare a speech. 6.6.1.2.2. Furthermore, any lesson on a subject with several phases lends itself well to the sequential pattern. For example, given an objec tive for students to know the three planned phases of the Common Market (where phase one precedes phase two, and phase two precedes phase three), a lesson might have these main points: (1) Phase one—a customs union where nations agreed to reduce duties, (2) Phase two—an economic union allowing laborers and goods to move freely across national borders, and (3) Phase three—a political union with national representatives as members of a co mmon parliament and using a common currency. 6.6.1.2.3. Of course, rather than looking forward in time from a given moment, the strategy might be to look backward from a point in time. In other words, the strategy might be to move from recent to earlier time rather than from early to late. Regardless of which strategy is used, the flow of the lesson and the transitions shoul d make the chronological relationships between main points clear to the students.
6.6.1.3. Spatial. A spatial or geographical pattern is effective in describing relationships. When using this pattern, the lesson mate rial is developed according to some directional strategy such as east to west or north to south. For instance, if an instructor was describing the domino theory of guerrilla infiltration, a good organization for the le sson would make the main points of the lesson correspond to the geographical locations of various nations. 6.6.1.3.1. With lessons about certain objects, the strategy might be to arrange the main pointsfrom top to bottom or bottom to top. A fire extinguisher might be described from top to bottom, an organizational chart from the highest ranks to the lowest in the organization, a library according to the services found on the first floor, then the second, and finally those on the third.AFMAN36-2236 12 NOVEMBER 2003 79

6.6.1.3.2. Sometimes the strategy is to organize the lesson from the center to the outside. For example, the control panel in an airplane mightbe discussed by describing first those instruments in the center most often used, then by moving out toward the surrounding instruments that are used least often. 6.6.1.3.3. In all lessons arranged spatially, we need to introduce each aspect or main point according to some strategy. Just as with a lesson organized by time, the subject matter and the transitions should include elaboration and clarif ication of how the main points relate to one another. A simple listing of the various objects or places without elaboration as to how they are related may confuse the students and ma ke the points harder to remember.
6.6.1.4. Cause-Effect. We use a cause-effect pattern of organization in a lesson when one set of conditions is given as a cause for another set. In such lessons we may use one of two basic arrangements for our main points. We begin witha given set of conditions and show how these produce or have already produced certain results or effects. With an effect-cause strategy, we take a certain set of conditions as the effects and allege they resulted from certain causes.6.6.1.4.1. The cause-effect strategy might be used in a lesson concerning the increasing number of women in the Air Force. The lesson might first discuss the fact that women are now assuming more responsible leadership roles in the Air Force. One effect of women assuming such roles might be that women are joining the Air Force with increasing frequency. 6.6.1.4.2. The effect-cause strategy might be used in a lesson on child abuse. The first point might explain the effects of child abuse upon the children themselves, the parents, and even on society. The second point mightsuggest that the causes are that parents themselves were abused as children or they lack proper education on parenting. 6.6.1.4.3. Whichever strategy is used, you must bewa re of false causes and of single causes. For false causes: just because one event or ci rcumstance precedes another does not mean the former causes the latter. Many persons assume that "First A happened, and then B took place, so A must have caused B." For single causes: fe w things result from a single cause. There may be several causes and they may not act independe ntly. Their effect may be greater or less than the sum of their parts. For example, lack of safety features on automobiles does not by itself cause most highway accidents, but this cause plus careless driving and unsafe highways may, in combination, account for many highway accidents.
6.6.1.5. Problem-Solution. This pattern, sometimes called the disease-remedy pattern or the need-satisfaction pattern, presents students witha problem and then proposes a way to solve it. With this pattern, we must show that a problem exists and then offer a corrective action that is practical, desirable, capable of being put into action, and able to relieve the problem. It must also be one that does not introduce new and worse evils of its own. For example, the issue of controlling nuclear weapons has long been debated. Those against control argue that erosion of national sovereignty from arms control is more dangerous than no control. 6.6.1.5.1. There are different strategies we might employ when using the problem-solution method. If the students are aware of the problem and the possible solutions, we might discuss the problem briefly, mention the possible solutions, and then spend more time in showing why one solution is better than others. For instance, our objective is for students to comprehend that solar energy is the best solution to the energy crisis. Our main points might be that the world is80 AFMAN36-2236 12 NOVEMBER 2003

caught in the grip of an energy crisis, several solutions are possible, and solar energy is the best long-term solution.
6.6.1.5.2. If the students are not aware of the problem or need, we may describe the exact nature of the problem in detail. Sometimes wh en students become aware of the problem, the solution becomes evident, and little time is neededto develop the solution in the lesson. At other times, we need to spend time de veloping both the problem and the solution. 6.6.1.5.3. Still another strategy is to alternate or stagger portions of the problem with portions of the solution. For example, the cost of a pr oject may be seen as one problem, workability another, time to do the project as a third. Taking each in turn and providing solutions to cost, work ability, and time as we present these aspe cts of the problem may be more satisfying to students than if we had discussed all of the problem and then its total solution. 6.6.1.5.4. Whatever strategy is used, with the problem-solution pattern students must become aware that a problem exists before a solution will be agreed upon.
6.6.1.6. Pro-Con. The pro-con pattern, sometimes called the for-against pattern or advantages-disadvantages pattern, is similar to a problem-solution pattern. A major difference, however, is that fairly even attention is usually directed toward both sides of an issue with a pro-con pattern.6.6.1.6.1. There are various strategies to consider when using the pro-con pattern. One consideration is whether to present pro or con first. Another is whether to present both sides and let students draw their kown conclusions or to presen t the material in such a way students are led to accept the "school solution." For instance, with a lesson on the effects of jogging, we have to decide whether to present the advantages or disadvantages first. Then we must decide whether to let students decide for themselves whether the advantages outweigh the disadvantages. Pro-con plus one is the label given to the organization used when we draw a final conclusion based on the two sides. 6.6.1.6.2. When deciding the specific strategy to us e with the pro-con pattern and determining how much time to spend on each, the following guidelines may be helpful: (1) giving both sides fairly even emphasis is most effective when the weight of evidence is clearly on the favored side; (2) presenting both sides is mo re effective when students may be initiallyopposed to the school solution; (3) presenting only the favored side is most effective when students already favor the school solution or conclu sion; and (4) presenting the favored side last is generally more favorable to its acceptance, es pecially if the side not favored is not shown in too strong a light.
6.6.1.7. Topical. A topical division of the main points of a lesson involves de termining categories of the subject or lesson objective. This type of categorizing or classifying often springs directly from the subject itself. For instance, a lesson a bout a typical college population might be divided into topical divisions of freshmen, sophomores, juni ors, and seniors, with each class division serving as a main point. Housing might be discusse d in terms of on-base and off-base housing. A lesson on the Minuteman intercontinental ballistic missile might be arranged with the main points of warhead, guidance, and propulsion systems. 6.6.1.7.1. At times, the material itself suggests ce rtain strategies for ordering the main points. For instance, a lesson on levels-of-learning lesson planning would most likely begin with knowledge-level planning as the first main poi nt because knowledge-level lessons are generally simpler to understand. Then the lesson w ould move on through the hierarchy to compre-AFMAN36-2236 12 NOVEMBER 2003 81

hension, application, analysis, synthesis, and fi nally, evaluation levels. In other words, our lesson would follow a simple-to-complex strategy inorganizing the "topics" or levels of learning. 6.6.1.7.2. Other topically organized lessons might follow strategies of known to unknown, general to specific, or specific to general. The number of strategies for arranging topical main points is practically infinite. The important cons ideration, as with any pattern, is that we give thought to the strategy of arrangement in or der to improve student understanding and learning.
6.6.1.8. Combining Patterns. Our lessons will make more sense if we use a single pattern to organize the main points. We will be able to remember more readily what the main points are when we teach the lesson. Even more important, st udents will be able to follow the lesson more easily and retain the material if we us e a single, logical pattern of organization. 6.6.1.8.1. While we may choose a certain organizati onal pattern for the main points, we may decide to use different patterns for subpoints. Figure 6.2. provides a tentative outline of a lesson on the importance of nonverbal factors of communication.
Figure 6.2. Outline of a Lesson on Nonverbal Communication.

6.6.1.8.2. Notice the main points (performance and nonperformance factors) are arranged topically. The subpoints for main point I (upper, middle, and lower body) are organized spatially. A pro-con pattern is used to discuss positive and negative effects from each body performance I. Performance factors A. Upper body (head and face) 1. Positive effects 2. Negative effects B. Middle body (arms, hands, torso) 1. Positive effects 2. Negative effects C. Lower body (hips, legs, feet) 1. Positive effects 2. Negative effects

II. Nonperformance factors A. Objects B. Space 1. Personal or body 2. Furniture arrangement C. Time82 AFMAN36-2236 12 NOVEMBER 2003

factor. The subpoints of main point II (objects, space, and time) are organized topically, similar to the two subpoints under space. The important thing to remember is that each set of main points or subpoints should follow a given pattern of organization as shown in Figure 6.2. 6.6.1.8.3. Of course, it may be that none of the formal patterns of organization discussed in this chapter adequately fit our content. If this is the case, we must simply strive to organize our lesson in the way that will help present the information to our students in the most meaningful fashion. As we construct our tentative outline, we must do so with our students' needs in mind. But whatever pattern or strategy we choose, it should be a conscious and rational choice and we should be able to defend or explain it. Quite often, the experienced teacher revises the outline three or four times before satisfactorily putting it into final form on the lesson plan.
6.7. The Strategy Statement:
6.7.1. Character and Purpose. The strategy statement is simply a detailed plan that explains your overall lesson objective and the steps you intend to take in achieving that objective most efficiently and effectively. A well-written strategy statement benefits the writer of the lesson plan by helping to determine the best options to adopt when deciding on methodology, teaching techniques, interim objectives, and type and amount of proof and clarif ication support. It also helps anyone else who is tasked to teach or modify the lesson later by spel ling out the detailed rationale for choosing these options. 6.7.1.1. When the teachers understand why the differ ent elements of a plan are included and when these reasons are sound, the teachers can more easily adopt the process as their own or adapt the plan more coherently—both internally and as it rela tes to other lessons in the curriculum. And, just as importantly, the strategy can also benefit the students immensely because it provides a well-formulated overview for the lesson introduction itself by telling the students exactly what will be covered in the lesson without exposing the lesson itself. 6.7.1.2. The strategy statement should be designed in such a way as to walk the instructor through the entire lesson, focusing on every element of the lesson. A comprehensive strategy statement helps the writer of the plan by forcing the write r to consider these questions often taken for granted: Is the overall lesson outline and order of main points and subpoints the most logical and intuitively acceptable? Are the teaching technique s decided upon the most appropriate for the lesson? How much leeway can you take in the pres entation before you change the actual objective? Moreover, it provides a quick mental outline of the entire lesson helping to prevent the instructor from having to script, or slavishly rely on, the lesson plan (thereby destroying the spontaneity of the presentation).
6.7.2. Method of Presentation. The strategy statement should move the reader through the lesson simultaneously from the general to the specific elements as well as sequentially, from attention step through the closure. No part of the lesson plan a nd no decision about teaching techniques (use of demonstrations, use of questions, where to insert mini-lectures) should be overlooked or assumed in the statement. Spelling out each element about what we sometimes make subconscious decisions on lesson formulation often reveals conventions we sometim es misuse or overuse in the practice of education—again, a benefit to the writer an d any other teacher of the lesson. 6.7.2.1. We suggest you start by looking at your lesson objective and organizational pattern to decide on an overall strategic ordering of the ma in points. State this ordering up front; for exam-AFMAN36-2236 12 NOVEMBER 2003 83

ple, general to specific, specif ic to general, most to least important, known to unknown, simple to complex. This decision will give you a general focus to lead you toward the objective efficiently. 6.7.2.2. Follow this with a sequential statement of each main point and its associated subpoints to include method of presentation and rationales fo r the method and order of presentation. Each of these steps is important to help the lesson planner fit the elements together and consider all the factors necessary for justifying each decision. In effect, this method can be simplified by meticulously asking and answering the three questions indispensable for comprehensive lesson development—what, how, and why. A practical approach to writing a strategy statement and answering those questions is to use a matrix out lining your main points. The process then becomes a matter of filling in the information ( Figure 6.3. through Figure 6.5. ).84 AFMAN36-2236 12 NOVEMBER 2003

Figure 6.3. Strategy Statement—Example 1.

WHAT? HOW? WHY?

LESSON TYPE: Informal Lecture.

PART IB
ORGANIZATIONAL PATTERN: Topical.
STRATEGY: This lesson is the second lesson in a series of three that cover the problem of positional warfare in World War I. It logically builds upon the theo ries of Fuller and Hart which were studied in the first lesson and introduces the concep t of Blitzkrieg, a combined arms a pproach to warfare, which will act as a bridge to the third lesson of the series on applied airpower theory from World War I.
This lesson discusses the Blitzkrieg theory of co mbined arms warfare by Germany during the interwar years and examines the contextual and operational art el ements that affected its development. It is best presented using the informal lecture method and a topical format centered on the contextual and operational art elements introduced in the Nature of War course. Begin the lesson with a definition of the key term "Blitzkrieg" and clarify the distinction betwee n Mechanized and Motorized forces. This will help provide the student with a clear understanding of these key terms and concepts, which is necessary to fully understand the lesson. Review with the students the problem of trench warfare in WWII, the difficulty of assaulting prepared positions, and the tactics empl oyed by Germany and other countries during WWI to try and deal with this problem. This will set the stage for future lessons that will contrast how development of combined arms and armored warfare evolved in Germany with how mechanized warfare theory evolved in other European countries.
A majority of the classroom time should be spent disc ussing the contextual and operational elements that existed in Germany during the interwar years and e xplaining how those factors influenced development of the Blitzkrieg theory in Germany. The information should be presented in a t opical format using each applicable contextual and operational art element as a topic. This approach is favorable since it helps to reinforce the importance of contextual and operationa l art elements in any wartime situation and also because the framework is already familiar to the students.
After presenting the influences that affected the devel opment of the Blitzkrieg theory, the instructor should present Guderian's theory or vision for employing m echanized arms. Guderian is credited as being the "Father of the Blitzkrieg." With the rise of Hitler and nazi Germany, Guderian was given command of a Panzer force and became in strumental in molding how the theory of mechanized arms was actually put into practice. The body of the lesson should conclude by using three historical examples from the early battles of WWII that illustrate the resounding su ccess that this new method of warfare produced.
To conclude, summarize the main points of the lecture by highlighting the significant subpoints and by emphasizing the significance that technology and contex tual elements have on the development of new warfare theory. This conclusion is important because inlater lessons a similar look at the development of Air Power, which also had its roots in WWI and the interwar years, will be undertaken.AFMAN36-2236 12 NOVEMBER 2003 85

Figure 6.4. Strategy Statement—Example 2.

WHAT? HOW? WHY?

LESSON TYPE: Demonstr ation-Performance.

PART IB

ORGANIZATIONAL PATTERN: Sequential.

STRATEGY: The lesson on determining cash value will proceed in sequential order to show how each step of the calculation builds on the next. The lesson will start with a brief explanation of present value and cash flows. This explanation will help stude nts understand the benefit of using this method in determining the time value of their money. Next, I will give a demonstration and explanation of each task step of the process of calculating th e present value of an unequal stream of payments to show each student how to extract necessary data from a scenario and obtain the present value of each amount. This demonstration and explanation step will also teach stude nts how to use the present value table. I will then have the students perform the calculations themselves while I read the task steps to give the students practice under controlled conditions. Once the students have completed the task steps, they will practice once more on their own with the last practice occurring under conditions duplicating the final evaluation so the instructor can be sure each student is ready. The demonstration and explanation of problem, the controlled practice problem and the independe ntly done practice problem should provide enough repetition of the required sequence of steps while providing adequate time for questions to ensure students are learning the procedure. I will then have the stude nts perform the computations without assistance, and then grade the results to ensure the student s have performed to the required standards.86 AFMAN36-2236 12 NOVEMBER 2003

Figure 6.5. Strategy Statement—Example 3.

6.7.3. Interrelating the Questions. Whenever there is a significant lesson element, it constitutes an answer to the "what" question that must be answered in the strategy. A significant lesson element is every main point or subpoint the instructor includes in the lesson. Each point must be denoted in the strategy statement. Often, segments of the lesson, such as the introduction, attention step, interim summary, or conclusion, represent significant elements that should also be mentioned in the strategy. Significant lesson elements tell the teacher (and sometimes the student) what will be covered or where there is an important or required tactical decision. But this is just the beginning of the process because if we were to stop with these determinations of what we intend to do and where we intend to do it, we would be left with simply a laundry list of elemen ts to be touched on during the lesson. Of more significance to the lesson planner are the an swers to the "how" and "why" questions. 6.7.3.1. Asking and answering "how" to each of thes e "whats" forces us to determine and delineate the method, manner, or sequence of presenti ng the significant elements included. It is important to make these determinations because we ha ve a palette full of teac hing techniques, patterns, and methodologies from which to choose. And it is often helpful to specify these decisions to WHAT? HOW? WHY?

LESSON TYPE: Teaching Interview.

PART IB

ORGANIZATIONAL PATTERN: Topical.

STRATEGY: The interview will start with lead-off and follow-up questions about the general role and purpose of the Security Assistance Training Program (S ATP) to show the main point, "The SATP promotes national security." Then a series of lead-off and fo llow-up questions addressing the particular role the International Officers School (IOS) plays in the traini ng process will demonstrate the other main point, that "IOS promotes national security." Students will learn what the SATP is from the point of view of its initial purpose as well as how it operates today. This will be done by use of a practicing expert within the SATP. The interviewing of the expert is chosen for two reasons : 1) to effectively present the concept of the SATP in an understandable and interesting format with the us e of personal experience and real-life examples, and 2) to foster an affective response on the part of th e students by testimony of one who believes in the goals of the SATP. I will use a topical pattern to explain the goals and importance of the SATP as well as the goals and importance of IOS as it contributes to reaching that goal through the use of Air Force training resources. The interview will proceed from a genera l explanation and understanding of the role of IOS. This strategy should be more effective in fosteri ng a deeper comprehension of the importance of the program as well as pointing out the personal relevance of the program to the ordinary Air Force civilian or military member, all of whom are likely to encounter aninternational military student during their careers. After the formal interview of the guest, there will be a question-and-answer period to further illuminate the topic.AFMAN36-2236 12 NOVEMBER 2003 87

encourage precision in our lesson formulation. For example, very often we state that we will discuss a particular topic when we have absolutely no intention of discussing it at all (as evidenced by our failure to plan discussion questions in that part of the plan). Answering the "how" question helps us to focus our intentions and prevent us from making the mistake of saying we will discuss something without providing discussion questions wi thin the lesson plan. Alternatively, we might want to demonstrate, or even have a student de monstrate, something within the lesson or we might want to show a movie or slides to illustrate a lesson element most effectively. But if we state in the strategy that we will explain a given concept in this part of the lesson, this disconnect with what we actually plan in the lesson will be more evident. 6.7.3.2. One cannot determine the final answer to "how" without giving full consideration to why the decision is made. Therefore, the answer to "w hy" must be fully integrated into the strategy statement. This answer provides the intellectual gl ue that binds the parts of the strategy statement into a cohesive whole. It justifies each decisionthe planner makes and f its the elements of this puzzle together. It helps to prevent our adding th e dazzle and flash that add no educational value to the lesson. Everything must have a sound reason for its inclusion or it should be left out of the lesson. The "why" should ask why the main points and subpoints are in the order we choose as well as force us to provide a reason for our choice of techniques, patterns, methods, and the inclusion of every other "what" in the lesson. 6.7.3.3. This explanation of the role of "what, how, and why" in writing a comprehensive strategy should stimulate at least a preliminary understand ing of how the strategy statement can help us plan much better lessons. A full appreciation can only be aroused, however, by our use of this technique and by comparing lesson plans based on th is process to those written without comprehensive strategies. But beyond the immediate benefi ts to lesson planning, there are other benefits just as valid and sometimes more helpful. For example, lesson plans are continuously modified or updated—often by other than the origin al lesson developer. What a bene fit it is to be able to make minor adjustments with a clear understanding of th e reasoning dictating the original form of the lesson!
6.7.4. Significance of the Questions. Why is it so important to be this specific in writing the strategy? All techniques and methods (o r combinations of these) are potentially appropriate in a lesson until the lesson planner determines the most effective. If the writer does not make this decision or doesn't realize what is best to do in a given scenari o, the lesson will never be as effective as it could be. In almost any human endeavorthe effectiveness of interpersonal communication is inversely proportional to how much one allows to be left unstated or "understood." 6.7.4.1. In order to emphasize the importance of including the answers to each of these questions in the strategy statement, an appropriate compar ison can be made between the strategy statement and a cooking recipe. In each case, the intended resu lt is a palatable concocti on of ingredients. But none of us can combine a list of ingredients (wha t) of a dish (what) to produce the desired result unless we know the correct measurements of each (how) as well as the method (how) and time (how) of cooking. Even the order of combining the ingredients (how) at each stage of the cooking process is often important to ge t the desired result. For instance, adding salt too early in a meat recipe could reduce the succulence of the dis h. But without a working understanding of what each of the ingredients does in a recipe (why), we would be unable to easily and successfully modify the recipe for the microwave in lieu of the oven or to substitute elements such as mayonnaise for oil and eggs in a cake recipe.88 AFMAN36-2236 12 NOVEMBER 2003

6.7.4.2. Therefore, it is imperative to ask and answer these three questions in a well-developed strategy statement. It is equally important to interweave the answers throughout the strategy. This careful choice and justification of all lesson plan elements will result in better preparation.
6.7.4.3. Figure 6.3. , Figure 6.4. , and Figure 6.5. are examples of strategies that demonstrate the principles we have just discusse d. Notice first that the "what, how and why" statements are interwoven throughout the statements. This helps to ensure everything in the plan has a justifiable purpose. Secondly, notice that some of these statements serve double duty, for example, some "what" statements are also "how" or "why" statements. This is perfectly legitimate—any type statement can serve more than one purpose. This may even help to strengthen the cohesion of the overall strategy.
6.8. Choosing Support Material:
6.8.1. Developing the Lesson Outline. A major factor in developing the lesson outline is determining the kind of support material to use. While organi zing the ideas forms the basic structure of any lesson, almost all ideas need some form of clarif ication or proof if th e student is to learn.
6.8.1.1. Most students find it difficult to understand unsupported ideas or assertions. Suppose, for instance, we tell our students it is important to organize a speech or a lesson according to one of several patterns of presentation. We then tell them the most common patterns are chronological, sequential, spatial, cause-effect, problem-solutio n, pro-con, and topical. Most likely, we will not have provided enough information so our students ca n actually use these patterns of organization. We must go on to explain each of these patterns, as has been done in paragraphs 6.6.1. through 6.6.1.7.2. This explanation would include supporting ideas or assertions. 6.8.1.2. The subject, the method, the ability of students, the size of the class, and similar factors will help determine the amount and kinds of suppor t material we need. We may also want to use visual support. Chapter 20 discusses types of visual support materials and some rules governing their use. 6.8.1.3. Verbal support is needed either to clarify or explain our points or to prove our assertions. Definitions, examples, and comparisons are used primarily for clarification support. Their use as proof is limited. Statistics and testimony of experts can be used either for clarification or proof. With lectures and other presentational methods, the instructor actu ally furnishes the support. With methods involving student interaction, such as the guided discussion, instructors use questions to encourage support materials from the students. Chapter 11 discusses how to use questions effectively.
6.8.2. Definitions. Definitions are often needed to clarify or explain the meaning of a term, concept, or principle. But like so many words, the term defi nition can mean different things and function in different ways.6.8.2.1. In some lessons we need to use technical or complex words. With the increasing specialization of Air Force schools in both theoretical and applied subjects, the output of words races ahead of dictionaries.
6.8.2.2. At other times we need to define words we employ frequently and loosely. Some words simply have different meanings for different pe ople. Words such as "democracy," "equal rights," "security needs," and "loyalty" can often be defined easily. For instance, "disseminate" can be defined very simply as "spread widely." At other times we might seek novel and memorable waysAFMAN36-2236 12 NOVEMBER 2003 89

to define our terms. "Pragmatism" might be defined as "a fancy word to mean that the proof of the pudding is in the eating." 6.8.2.3. Sometimes it takes a little longer to full y define what we mean by a certain term. A former prisoner of war (POW) might define the sacrifice of one prisoner for another as, "When you see an American prisoner giving up his meager ra tion of fish just so another American who is sick can have a little more to eat, that is sa crifice. Because when you don't have anything, and you give it up, or you have very little and you give it up, then you're hurting yourself, and that is true sacrifice. That's what I saw in the prison camp." 6.8.2.4. Use definitions to explain the meaning of acronyms—words formed from initials. For instance, when discussing PME at AU, we might have to explain that PME at AU means p rofessional m ilitary e ducation that is taught at A ir U niversity. We might go on to mention that PME includes AWC, ACSC, SOS, ABC, and SNCOA— that is, Air War College, Air Command and Staff College, Squadron Officer School, Aerosp ace Basic Course, and Senior Noncommissioned Officer Academy. 6.8.2.5. Finally, instructors may need an entire le sson to define or otherwise introduce students to a new term, concept, or principle. For example, when talking about the meaning of communication as transaction, it would probably be insufficient simply to say the transactional approach means to consider the total communication process and the interaction of the various parts of the process on each other. Other forms of support ma terial such as examples and comparisons might be needed to fully define what we mean.
6.8.3. Examples: 6.8.3.1. Any time students ask us to give a "for instance," they are asking for an example to clarify the point we are trying to make. Sometimes the ex amples we use may be reasonably long. At other times a short example is sufficient. In some cases, short examples are similar to definitions. The earlier definition of "sacrifice" given by the form er POW might also be considered a short example. The fact that some support materials might be classed either as definitions or examples should not overly concern us. As classroom inst ructors, we are more interested in using effective support material than in classifying it. Consider the questions in Figure 6.6. when deciding whether to use a particular example.90 AFMAN36-2236 12 NOVEMBER 2003

Figure 6.6. Choosing Appropriate Examples.

6.8.3.2. Often short examples can be clustered togeth er in order to help students gain a more complete understanding of the point. In a lesson on a barrier to effective communication, we might cluster examples of spoonerisms: "Is the bean dizzy?" (Is the dean busy?); "I'll have a coff of cuppee" (I'll have a cup of coffee); "A half-warmed fish within us" (A half-formed wish within us).
6.8.4. Comparisons. Description often becomes more graphic when we place an unknown or little understood item beside a similar but better known item. We might want to compare things that are unlike or things that are very much alike to help our students visualize what we are talking about. 6.8.4.1. Metaphors such as Winston Churchill's "iron curtain" or similes (using the words "like" or "as," such as Robert Burns' "My love is like a red, red rose," or the saying "strong as an ox") are comparisons of things that are unlike in most ways. We often use comparisons of unlike things in lessons. For instance, we might say, "The flow of knowledge is like the relentless and uncompromising flow of a river after the spring thaw as it imposes on us the requirement that we not only adjust but anticipate the future." Or we might sh ow that being a member of a branch in an Air Force organization is like living in a family where we have intimate contact with each other. Wemight carry the analogy or comparison further by pointing out that in a unit, as in a family, members can protect, help, or irritate one another. 6.8.4.2. Although analogies that compare things that are unlike serve as an excellent method of clarification, they have limited utility as proof. To prove an assertion, we must compare "like" things. Comparing the airpower of an adversary with US airpower or a mayor and city council with a base commander and his or her staff are "like" comparisons. Arguing for a longer orientation session for students in one NCO academy beca use it has improved academic performance at another would be comparing "like" phenom ena—in this case, two NCO academies. Questions to Ask About Using Examples in Lessons: Do they accurately represent the point? Will students clearly understand their meaning? Do they fit the content? (Avoid those that may confuse.) Do humorous ones add rather than distract from the lesson? ( Chapter 13 presents some guidelines for using humor.) Do they come from personal experience or can other ex amples be personalized in such a way as to seem real? Can anything be gained from clustering more than 3 or 4 examples? (Usually not.) Do long ones take too much time? (At times affectiv e or attention-getting value of long examples may justify their use.) Are they interesting? NOTE: The appropriate answers to these questions should be obvious.AFMAN36-2236 12 NOVEMBER 2003 91

6.8.4.3. Contrast is a special form of comparis on. For instance, showing how Air Force training differs from civilian training or how today's standa rd of living differs from that of a generation ago clarifies and explains a point by showing contrast or differences. 6.8.4.4. Obviously, any kind of comparison may be verybrief like those given here or they may be quite long. We need to decide what will work best in a given situation. But whether long or short, comparisons are a valuable and ge nerally underused method of verbal support.
6.8.5. Testimony. Words and thoughts of others become particularly useful when we wish to add strong proof support for assertions or points we make. None of us are expected to be experts on all subjects; we must often rely on what others have sa id. At times, we use the testimony of others simply to clarify or explain an idea, but of ten testimony provides proof for a claim. 6.8.5.1. A lesson dealing with managerial effectiv eness in an organization may have the importance of effective downward communication as one of its main points. In other words, we want to stress how important it is for supervisors to keep their subordinates informed. We might quote from a recent Air Force policy letter that states, "Commanders and supervis ors have an increased responsibility to keep Air Force military and civilian members informed." We might also report the findings from a recent study by the Internati onal Association of Business Communicators that show "face-to-face communication, including group meetings and one-on-one dialogue, proved the most effective means of communicating with employees." 6.8.5.2. Sometimes we will use direct quotations as we have done here. Other times we will paraphrase what another has said. What ever the case, there are two questions we will want to ask about the sources of testimony we plan to use: (1) Are the sources competent—do they know what they are talking about? and (2) Can they be trusted—are they free from bias? We might also consider whether the testimony is relevant, clear, and interesting and whether quotations are longer than necessary. 6.8.6. Statistics. Statistics are probably the most misused and misunderstood type of verbal support. When properly collected and wisely used, statistics can help instructors clarify their ideas. Statistics are also the most powerful proof support we can use. However, not all figures are statistics; some are simply numbers. Statistics show relationships (largene ss or smallness, increases or decreases) or summarize large collections of facts or data. When we choose statistics to use in our lessons, there are some questions we should ask. 6.8.6.1. Are the statistics recent? Figures concerni ng the cost of living in 1960 would have limited usefulness for planning a budget for today's family. Wh en selecting statistics to use in our lessons, we should be on guard for outdated sta tistics or those not dated at all. 6.8.6.2. Do the statistics indicate what they purport to? A single test score may not be a true measure of a student's ability. For instance, comparing the simple number of planes may not indicate the comparative strength of two countries' air forces. 6.8.6.3. Do the statistics cover a long enough time to be reliable? The results of how one class responded to a new curriculum change would be le ss meaningful than how three or four classes responded to the change. 6.8.6.4. If the statistics are drawn from a sample, does the sample accurately represent the group to which we are generalizing? Public opinion surveys and experimental researchers are generally92 AFMAN36-2236 12 NOVEMBER 2003

sensitive to the importance of obtaining a representative sample. Instructors also need to be sensitive to this need. 6.8.6.5. When statistics report differences, are the differences significant? Minor variations can often be attributed to chance. In other words, if we were to collect our statistics again, the results might differ. 6.8.6.6. When comparing things, are the same units of measure used to make the comparison? Failure in one course might have a different mean ing from failure in another. If more students fail one course than another, we cannot necessarily c onclude the content of one course is more difficult. Perhaps the grading scale rather than the content was more difficult. 6.8.6.7. Do the statistics come from a reliable source? And is the source clearly indicated? It is more effective to state the source of the information than to say, "recent surveys show." 6.8.6.8. Are the statistics presented to their best advantage to aid student understanding? Could we use visual aids to present the statistics in graphic or tabular form for easier understanding? Have figures been rounded off where possible? St udents are more likely to remember nearly $45,000 than $44,871.24. Is the number of statistics limited so students ar e not overwhelmed by them? Could the significance of statistics be cl earer with meaningful comparisons? To say that World War II cost the United States $200 billion woul d be clearer if we converted the figures into today's dollars or compared the figures to the cost of other wars using a standard measure.
6.9. Beginning and Ending the Lesson. So far we have selected the verb al and visual material that best supports our lesson and made necessary changes in the original tent ative outline. We are now ready to cast our lesson into a final content outline. However, we us ually want to consider how to begin and end the lesson before we outline. If the lesson is not the first in a block of instruction, we ma y have little to do in the way of beginning or introducing the lesson. If other less ons in the same block of instruction are to follow this lesson, we may not need an extensive conclusion. But, especially if the lesson is to stand alone, we need to give some attention toward preparing an introduction and conclusion.
6.9.1. Introduction. The introduction to a lesson should serv e several purposes—to establish a common ground between the instructor and students, to capture and hold attention, to outline the lesson and relate it to the overall course, to point out benefits to the student, and to lead the student into the lesson content. While humor may be appropriate, the introduction should be free of irrelevant stories, jokes, or incidents distracting from the lesson objective. It should not contain long or apologetic remarks likely to dampen student interest in the lesson. Educators often speak of three necessary elements in the introduction of a lesson: gain attent ion, motivate, and provide an overview of lesson material.
6.9.1.1. Attention. To gain attention, the instructor may re late some incident focusing students on the subject and providing a background for the less on. Another approach may be to make an unexpected or surprising statement or ask a question that relates the lesson to group needs. A rhetorical question (Have you ever...? or Can you imagine...?) ma y be effective. In all instances, the primary concern is to focus student attention on the subject.
6.9.1.2. Motivation. The instructor should use the introduc tion to discuss specific reasons why the students need to learn the lesson. In the mo tivation, the instructor should make a personal appeal to students and reinforce their desire to learn. The appeal may relate the learning to career advancement or to some other n eed. But in every instan ce, the instructor should cite a specificAFMAN36-2236 12 NOVEMBER 2003 93

application for student learning experiences. In many cases, the need for this lesson as a foundation for future lessons is strong motivation. Th is motivational appeal should continue throughout the lesson. If a brief mention of needs is made only in the introduction, the instructor is filling squares—not motivating.
6.9.1.3. Overview. For most instructional methods, the introduction should provide an overview of what is to be covered during the class period. An overview should have a clear, concise presentation of the objective and main points as a road map for learni ng. Effective visual aids can be helpful at this point. A clear overview can contribute greatly to a lesson by removing doubts in the minds of the students about where the lesson is going and how they are going to get there. Students can be told what will be covered or left out and why. Inform them about how the ideas have been organized. Research shows students understand better and retain more when they know what to expect. The purpose of the overview is to prepar e students to listen to the body of the lesson.
6.9.2. Conclusion. The conclusion of a lesson may stick with the students longer than anything else said. For this reason, we should give much care to its preparation. The conclusion of most lessons should accomplish three things: summarize, remotivate, and provide closure.
6.9.2.1. Final Summary. Short or interim knowledge-level summaries may be appropriate at various places in a knowledge-level lesson; for example, after each main point has been made. But final knowledge-level summaries come after all main points of the lesson have been made. An effective knowledge-level summary retraces the important elements of the lesson. As the term suggests, a final knowledge-level summary revi ews the main points in a concise manner. By reviewing the main points, it can aid students' rete ntion of information and give them a chance to fill in missing information in their notes. 6.9.2.1.1. In lessons designed to reach a conclusi on (principle), a comprehension-level summary is desired as the final summary. Short or interim comprehensi on-level summaries may come at the conclusion of main points. But the final comprehension-level summary comes after covering all main points of the lesson and se rves as the first part of the lesson conclusion. 6.9.2.1.2. The comprehension-level final summarymay require several minutes. While containing a brief restatement of significant info rmation, it requires an expansion of key items to establish relationships that lead to a generaliz ation. The generalization is the lesson objective. New support material can be introduced when needed to establish the generalization.
6.9.2.2. Remotivation. The purpose of the remotivation is to instill in students a desire to retain and use what they have learned. Effective instructors provide motivation throughout the lesson. But the remotivation step is the instructor's last chance to let students know why the information presented in the lesson is so important to the students as individuals. The remotivation may also provide the groundwork for future lessons or r easons the information will help the students do their jobs more effectively. But whatever the r easons given, they should be ones appealing directly to the students and show the importance of the subject matter learned.
6.9.2.3. Closure. For many instructors, the closure presents the most difficult challenge in planning a lesson. Students need to be released from ac tive participation. In lectures, they need to be released from listening. In interactive methods, th ey need to know that it is time for their verbal participation to end. Sometimes instructors w ho don't know how to close will say, "Well that's about all I have to say," or "I guess I don't have anything else." Th is type of closure is not very satisfying. There are many more effective ways of closing. Sometimes vocal inflection can signal94 AFMAN36-2236 12 NOVEMBER 2003

that the lesson is ending. Quotations, stories, or humorous incidents can also provide effective closure. Sometimes when the lesson is to be followed by others in the same block of instruction, we might say something such as, "Next time, then, we will continue with our discussion of... . Between now and then if you have any questions, co me to my office and I'll see if I can answer them for you."
6.10. Preparing The Final Outline: 6.10.1. After researching the topic, selecting an instructional method, identifying the lesson planning format to use, organizing the lesson, choosing the support materials, and deciding how to begin and end the lesson, you are ready to prepare the final c ontent outline. In fact, you may want to prepare two versions of the outline. One that is very complete—almost in manuscript form—so you can return to it several weeks or months later when you have to teach the lesson again or when someone else must teach the lesson. Another version will be much briefer—perhaps only one page long, or written on cards so you can carry it to the classroom and teach from it. 6.10.2. This brief outline, or keyword outline, may have key words and phrases of main points, subpoints, support material you plan to use, questions to ask, and the things you want to mention in the introduction and conclusion of the le sson. This keyword outline is a basic minimum for most of us to take into the classroom. The following discussion focuses on its construction: (NOTE: The longer version of the outline will follow the same princi ples, but will include much more information.)
6.10.2.1. Division. Divide the outline into three main parts: introduction, body, and conclusion. As discussed previously, the introduction will gene rally have three subparts: attention, motivation, and overview. The body will have the main points as major subdivisions. The conclusion will have three subdivisions: final summary, remotivation, and closure.
6.10.2.2. Symbol System. To show the relative importance of lesson materials in the body of the lesson, use a number or letter symbol before each entry. A roman numeral may designate main points, capital letters for subpoints, Arabic numer als for sub-subpoints, and so forth. When outlining, remember that only one symbol should be used per point or idea, subordinate points should be indented, and the principle of subpoints or subor dination means that a point follows logically or supports the point above it.
6.10.2.3. Sample Keyword Outline. Figure 6.7. provides a sample keyw ord outline. Earlier we considered how to make a tentative outline for a lesson on nonverbal communication. Now consider how to revise that outline to teach from. Th is outline has been prepared to use with the lecture method. The same outline could be revised slightly to use with a guided discussion orteaching interview method by simply replacing references to support material the instructor supplies with questions that would prompt students or an expert to supply appropriate support material.AFMAN36-2236 12 NOVEMBER 2003 95

Figure 6.7. Sample Keyword Outline – Nonverbal Communication.

INTRODUCTION Attention: "Actions speak louder than wo rds." "Dinner jacket" example Motivation: Dr. Ray Birdwhistle—65 percent of message communicate nonverbally Importance—jobs, family, church, clubs Overview: Chart listing main points and first level subpoints Define "performance factors" and "nonperformance factors" BODY Know performance factors of nonverbal communication Upper Body— importance capitalized on by F.D.R. Head Theory of origin of head gesture Cultural differences Eyes—very important Show interest in others—example of blind student Nonverbal feedback—cultural differences Increase credibility—describe U of Mo. studies Facial Expression Affect displays—read Charles Darwin quote on expression Affect recognition—use everyday examples Middle Body Arms—demonstrate how we use them Hands—primary means of gesturing Compare meanings from different cultures—OK and Victory signs Demonstrate use of hands Torso—demonstrate shoulder, chest, stomach—belly dancer example Lower Body Hips—Elvis example Legs—compare with foundation of building Feet—show different angles Nonperformance Factors Objects Present—clothes, home, office Past—things we have constructe d—example of my former home Space Personal Stress cultural differences—give example of visit to Greece Space bubble—example of waiting for bus or in line Acceptable distance—cite statistics by Hall Constructed—office arrangement, fences, etc. Time—humorous definition from Esquire, Wetumpka example CONCLUSION Summary: Reteach main points Remotivation: Stress importance of nonverbal to each student Closure: Tell humorous story of how deaf man solved problem; challenge students to do likewise96 AFMAN36-2236 12 NOVEMBER 2003

6.11. Summary. The complete cycle of lesson planning may include eight steps: (1) Determine the objective and state in terms of what the students are expected to learn, (2) Research the topic as defined by the objective. We can draw appropriate subject matter material from our own personal knowledge of the subject, the expertise of others and finally make effective use of a library, (3) Select the appropriate instructional method with regard to designated stud ent outcomes and acceptable student activities, (4) Identify a usable lesson planning format. We need to select a format that se rves as a ch eckpoint for well-planned learning experiences and provides a worthwhile teaching aid, (5) Decide how to organize the lesson. Organization of main points and subpoints is important because it helps instructors and students remember the material. Commonl y used patterns of organization are time, space, cause-effect, problem-solution, pro-con, and topical. Strategies for how we organize material with these patterns are also important. Although we may choose one pattern for organizing our main points, we may choose a different pattern for organizing subpoints, (6) Choosing s upport material. Students understand supported ideas or assertions better than if suppor t is not given. Definitions, examples, and comparisons are used mainly as clarification support. Statistics and testimony from experts can be used for either clarif ication or proof, (7) Beginning and ending the lesson. The beginning or introduction generally has three necessary elements: attention, motivation, and overview of what is to follow. Th e conclusion has three parts: final summary (or summation in lessons teaching principles), remotivation, and closure, and (8) Preparing the final outline. We may prepare two versions of the outline. On e version will be very complete so the information remains intact if we want to return to it at a late r time. A second version will be much briefer so we can teach from it in the classroom.AFMAN36-2236 12 NOVEMBER 2003 97

Chapter 7

DEVELOPING KNOWLEDGE-LEVEL LESSONS

7.1. Introduction. Most of us could not do our jobs without knowing a great many facts. As described in previous chapters, the ability to recall or recognize fa cts is classified as learning at the knowledge level. According to most systems for classifying learning obj ectives, the task of memorizing factual information is a low-level mental process (see Chapter 3 ). 7.1.1. According to Bloom's taxonomy, the basis of this chapter, knowledge-level learning is the most elementary and least complex level of learning in the cognitive domain. Although it is the lowest level of learning, it is absolutely necessary as the foundation for all higher levels of learning. 7.1.2. Learning facts is not necessarily easy. Some information is fairly simple to remember; for example, the US Air Force rank structure, the ma jor commands, or the range and payloads of various aircraft. Other information, such as complex operati ng procedures and the wide range of complex regulations is extremely difficult to learn and remember.
7.2. Appropriateness of Classroom In struction at the Knowledge Level. As difficult as knowledge-level lessons can be for the student, they are ge nerally the easiest for instructors to plan and teach. Because of this, instructors may be tempted to teach at the knowledge level to avoid the more difficult task of developing understanding. To teach facts, we should normally use reading assignments, video or computer-based training, and other noninstructor controlled methods whenever possible to save available classroom time for other uses. There are many classr oom situations, however, when we need to teach knowledge-level information. Use class time for knowle dge-level lessons if any of the following conditions exist:
7.2.1. Rapidly Changing Subject Matter. Many areas of instruction change too rapidly to keep printed materials and other media up to date. To keep instruction current, we may have to teach knowledge-level lessons in class, even though other methods of presentation could better do the job.
7.2.2. Cost-Effectiveness. In many cases, it may simply cost less to teach knowledge-level information in a classroom setting. We may discover that the cost of preparing another medium is more than the cost of the classroom presentation. Considerations that might affect this decision include the costs for media production, course administration, study carrels, and costs of changes or updating.
7.2.3. Need for Interpretation of Facts. No matter how clear and straightforward, many facts are difficult to learn without instructor assistance. We rarely get a class of adult students with backgrounds and reading levels that matc h perfectly with what they need to learn. An instructor may be of assistance in interpreting technical terms, important data, and charts or graphs. It is difficult for students to memorize knowledge-level information if they do not understand the basic terminology used.
7.2.4. Increasing the Significance of Subject Matter. We can make knowledge-level material more significant to students by bringing it out of the text and into the classroom—this gives it special attention. Of the many facts that could be learned from any course, we may signal to our students that some material is more important than others by devoting so me class time to it. A review of a reading assignment, going over a handout in class, or bringing in an outside speaker to impart information are all cues that the subject matter has special significance.98 AFMAN36-2236 12 NOVEMBER 2003

7.2.5. Need for Human Interaction. Because much of the knowledge-level information that must be learned is dry and di fficult, students become discouraged or disinterested. Even highly motivated students quickly tire with this type learning. It may be better to have some instruction in a group to raise interest or sustain motiva tion. Individualized study assignments, especially over a long period of time, often fail to satisfy basic needs for interaction with others . An awareness of group dynamics is important even at this level of instruction.
7.3. Planning Knowledge-Level Lessons. Teaching information at the knowledge level is relatively simple compared to the higher levels of learning. Good planning, however, is as vital at this level as it is at any other. Planning for students to learn and remember important factual information can be a challenge. Much of that challenge comes from the need to keep instruction on target and free from unnecessary complications. 7.3.1. Factual information should be well organi zed for learning. Although the reasons for learning many facts are obvious, the pattern and strategy of the plan can affect learning. Although the material on organizational patterns and strategies is introduced in Chapter 6 , a brief discussion of how they apply to teaching at the know ledge level is appropriate. 7.3.2. The pattern of lesson development can help a student organize and retain information. In knowledge-level lessons, the organizational patter n should be obvious and logical. Information about the structure of a unit may be learned more easily a nd retained if it is presented spatially (top to bottom as in an organization chart, or by geographic regions going from one direction to another in a logical fashion). Safety information, first aid, and many other similar topics can be organized into a cause-and-effect pattern. Sequencing or ordering is a very successful way to organize for teachingprocedures. Topical planning is a very useful pattern for knowledge-level instruction. Topical planning is especially effective since much of the mate rial we teach naturally groups itself this way when the topic outline is made clear to the students. 7.3.3. Strategies for dealing with factual information may be looked at as applied common sense. Most of us prefer to learn new information by relating it to what we already know. There are many strategies for presenting facts to our students that capitalize on this need to relate the new to the old. The obvious strategy of known to unknown works well at the knowledge level. 7.3.4. Building from simple information to more co mplex information, still atthe knowledge level, is generally another good strategy. Un fortunately, many experts fail to plan for linking new information to old because the relationship seems so obvious to them—but it's not so evident to the student. The plan for teaching and learning should be focused on the needs of students and must take their beginning level of knowledge into account.
7.4. Developing the Teaching Outline. By the time we sit down to develop an actual outline for teaching and learning, several important steps in the pl anning process are already done. Lesson objectives have been written by or for us. We have researched the subject, and we begin to see a logical organization of the material and a basic strategy for teaching. We may evenbe given a skeleton lesson outline, which we are to expand on and "personalize." In most cases, we have already selected an appropriate te aching method. 7.4.1. The teaching outline of our lesson plan will be the product of our decisions in all of these matters. Whether we are writing our own teaching outline from scratch or personalizing one given to us, the teaching outline must make sense both to us and to our students. The lesson organization and plan-AFMAN36-2236 12 NOVEMBER 2003 99

ning strategy should be obvious. In addition to keeping us on track with our lesson, we should also share our outline with the students in an overvie w so they will know how the lesson will proceed. 7.4.2. The teaching outline for this level of learning may range from a few note cards to several pages, but the outline should always be simple and strai ghtforward. The following examples illustrate three simple knowledge-level teaching outlines. Each ex ample contains objectives written at the knowledge level and main points in outline form. Considerably more detail would be required to satisfy the need for a teachable plan, but Figure 7.1. , Figure 7.2. , and Figure 7.3. illustrate the relatively simple teaching outline of most knowledge-level lessons.
7.4.2.1. Figure 7.1. illustrates a plan organized topically; the teaching strategy is probably known to unknown. The instructor has calculated that these three objectives represent a logical way toorganize this material for both teaching and learning. In addition, the instructor has started the lesson plan with information on the mission because it is assumed that students have some previous knowledge in this area. After receiving the info rmation on mission and history, students are now prepared to tackle the most complex and lengthy area—organization.
Figure 7.1. Teaching Outline—Example 1.

Objectives and Main Points

1. Know the mission of the ____________________.

a. .....

b. .....

2. Know the history of the _____________________.

a. .....

b. .....

3. Know the organization of the _________________.

a. .....

b. .....100 AFMAN36-2236 12 NOVEMBER 2003

7.4.2.2. Figure 7.2. shows the nature of the ISD model and the three items concerning each and provides us with at least two logical patterns fo r instructional system lesson development. The first outline follows a topical development for the objectives with a sequential development for the main points. An equally good alternative is illustrated in the second outline. In this case, the objectives in the lesson outline are the sequential proce ss steps in the ISD process, and the three areas of information for each are set up as topical mainpoints. The strategy is also driven by the ISD model. The philosophy behind this approach to teaching and learning is already built into the model and forms the strategy of the lesson.AFMAN36-2236 12 NOVEMBER 2003 101

Figure 7.2. Teaching Outline—Example 2.

7.4.2.3. Figure 7.3. shows the fairly common cause-effect pattern of organization. Depending on the actual topic being developed, the main points might have been arranged in a simple-to-complex or known-to-unknown strategy. Objectives and Main Points

1. Know the phases of the USAF Instructional System Development (ISD) Model. a. Analyze b. Design c. Development d. Implementation

2. Know techniques required to use the ISD Model. a. Analyze b. Design c. Development d. Implementation

3. Know major problems expected in implementing the ISD Model. a. Analyze b. Design c. Development d. Implementation

OR

Objectives and Main Points

1. Know the analysis phase of the USAF Instructional System Development (ISD) Model. Description Techniques Problem areas

2. Know the design phase of the USAF Instructional System Development (ISD) Model. Description Techniques Problem areas102 AFMAN36-2236 12 NOVEMBER 2003

Figure 7.3. Teaching Outline—Example 3.

7.4.3. Each of these examples illustrates simple but logical development for a knowledge-level lesson. Basic outlining skills and a little planning will add greatly to effective teaching at the knowledge level.
7.5. Support Material: 7.5.1. All lessons, no matter what level, require ade quate and appropriate s upport material. But unlike support material for lessons at higher levels of learning, support for knowledge-level lessons is provided for its own sake—to contribute directly to the student's ability to remember what is being taught. Examples are not given to enable the student to generate new examples. Steps in a procedure are taught in a way that will make the student remember, not so the student can interpret the reason for the procedure. Statistics and testimony are taught to be remembered themselves—not to raise the students' understanding to a hi gher level of abstraction. 7.5.2. Testimony, supplemental data, and statistics— important for the higher levels—should be kept to a minimum if they interfere with the learning of important information. For this reason, very little information, which is not important enough to be remembered, should be contained in a knowledge-level plan. Instructors who use support material to develop understanding rather than to promote retention should reexamine the lesson objective. Continued need for new examples, need for discusObjectives and Main Points

1. Know the primary causes of child abuse.

a. Cause 1

b. Cause 2

c. Cause 3

2. Know the primary effects of child abuse.

a. Effect 1

b. Effect 2

c. Effect 3AFMAN36-2236 12 NOVEMBER 2003 103

sion to clarify points, or a dissatisfaction with the lo w level of test items may be a clue that we should be teaching to a higher level of learning.
7.6. Teaching the Lesson. Instructors teaching at the knowledge level must make a real contribution to learning. If not, they come close to being little more th an audiovisual aids. Instructors can best add those benefits that come from interaction among students. Keeping in mind that the flow of knowledge-level information lessens the importance of a live instructor, we should at least make certain we are no less effective than a well-prepared visual aid.
7.6.1. Clarity of Presentation. Our lesson plan should be devel oped well enough that we have little need to restate ourselves for the sake of clarity. The presentation in a knowledge-level lesson should add to the student's ability to learn the factual in formation. The more complex the facts or the processes being taught, the greater the need for a clear presentation.
7.6.2. Redundancy of Information. Some redundancy is probably desirable in a knowledge-level lesson. Don't be afraid to say something more thanonce or through a different medium if it can help students to remember. We can build desirable redundancy into knowledge-level lessons in several ways: 7.6.2.1. Summarize When Needed. Interim and final summaries ensure that all information has been noted.
7.6.2.2. Supplement Presentations. Readings and handouts are invaluable study aids for knowledge-level lessons, particul arly if we show students how they support the lesson.
7.6.2.3. Use More than One Teaching Medium During the Lesson. Seek opportunities to support verbal information with visuals. Overhead transparencies, slides, and other projected media allow us to go after learning by sight as well as sound. Perhaps our best teaching aid for the average size class or seminar is still the dry erase board. We owe our students the chance to have what they hear reinforced by what they see.
7.6.2.4. Help Students Remember. Knowledge-level information can be presented in group situations, but students normally need to study on their own to memorize. If we have to learn a long string of facts, we would probably not try to do it while someone is lecturing. We should organize and present knowledge-level lessons so that we have done all we can to help students remember what has been taught. Take advantage of what we know about memory to help students get the most from the lesson. For instance: 7.6.2.4.1. Avoid needless distracters during the presentation. Humor, for instance, can add greatly to a lesson, but it may also interfere with the orderly transfer of information. See Chapter 13 for a further discussion of humor while lecturing. 7.6.2.4.2. Stick to a well-thought-out plan. When we teach in a system using objectives, we must resist the temptation to take off in an unplanned direction because it seems more attractive at the moment. 7.6.2.4.3. Build in memory aids where possible. We should suggest acronyms, word pictures, and other memory aids when possible. These devices can help the memory retain facts. 7.6.2.4.4. Encourage and plan for study to aid retention. Students should develop effective study systems for learning facts. Suggest flash cards, drill and repetition, studying with another, or other methods that seem appropriate.104 AFMAN36-2236 12 NOVEMBER 2003

7.6.2.4.5. Quiz and review frequently. Students s hould have an opportunity to test their memories before formal evaluation. By the time they are tested for real, students should already know whether they have been able to store and retrieve the information given them. Periodic checkups on the subject matter are also good ways to lessen the rate of forgetting. Material taught once and tested once may be forgotten at once.
7.7. Summary. Teaching at the knowledge level requires logical planning and effective presentation. Classroom lessons at the knowledge level should be carefully looked at to see if they can be replaced by readings or other media. Knowledge-l evel material presented in class should be presented in the clearest way possible so that students truly benefit from the presence of an instructor. 7.7.1. The pattern of organization for this level of instruction should be clear and help the student learn and retain the material. The strategy for teac hing should take into account the need for students to relate new material to what is already known. 7.7.2. Lessons should be planned and taught with a focus on memory. Aids to memorization such as redundant teaching and use of multimedia are importa nt points at this level of learning. While knowledge-level lessons are at the lower end of the cognitive taxonomy, learning and remembering facts are the foundation stones upon which the understanding of concepts and principles is built.AFMAN36-2236 12 NOVEMBER 2003 105

Chapter 8

DEVELOPING COMPREHENSION-LEVEL CONCEPT AND PRINCIPLE LESSONS

8.1. Introduction. Most instruction in American schools and colleges involves comprehension and not just simple recall. Similarly, we conduct much Air Fo rce academic instruction at the comprehension level of learning or higher. Unless students progress through the comprehension level to determine how and why something is done, they will experience difficulty applying information or skills in new situations on the job. 8.1.1. Comprehension is defined as the ability to gene ralize or to grasp the meaning of material. Students are expected to understand what is being communi cated and to be able to make some use of the material or ideas contained in it. A student can demonstrate this ability by translating material from one form to another, interpreting material, or estimating future trends. Comprehension is one step beyond simply remembering the material and repr esents the lowest of the understanding levels. 8.1.2. When conducting lessons at the comprehension level, instructors often teach either concepts or principles. Using this classification system, concept teaching needs to be addressed first, since principles are statements of relationshi p between two or more concepts.
8.2. The Teaching of Concepts. A concept is a class of people, objects, events, ideas, or actions grouped together on the basis of shared attributes or characteristics, and are called by the same name. When we teach a concept, we are referring to one of these classes or categories. The members in the class are put into the same category on the basis of shared attributes. These shared attributes are critical to identifying the concept. All members of the class sharing the critical attributes are called by the same name. 8.2.1. The term "instructor" is a concept. It refers to a class of people distinguished from others by distinctive duties and responsibilities. Only those performing these duties can be called instructors. The concept "bomber" refers to a class of objects with critical attributes that distinguish it from fighter, transport, and reconnaissance aircraft. Terms such as doctrine, strategy, and tactics are also concepts. Most of the terms we use in the classroom are concepts. 8.2.2. A specific individual, object, event, idea, or action is not a concept. No class is represented. Sergeant Ed Jacobs is an individual and not a category of people. While "war" is a concept, World War II is a specific event and not a class of wars. "Book" is a concept, but Tolstoy's War and Peace is not, since no class or category is presented. 8.2.3. For planning purposes, instructors need to understand the reasoning process students go through in comprehending concepts, such as generalizing and discriminating, so they can organize their concept teaching plan.
8.2.3.1. Generalizing. Humans have the mental ability to make the same response (give the same name) to a new example of something that differ s in some way from previously met examples. For example, when children reach the comprehension level on the concept "dog," they can point to a schnauzer, poodle, or collie and say "dog." Although each new example of a dog differs from theirown beagle or German shepherd, they can correctly call each new example a dog. Similarly, a student who comprehends the concept "strategy" can correctly label a new grouping of actions by a military commander as strategy. In each instance, the person is generalizing.106 AFMAN36-2236 12 NOVEMBER 2003

8.2.3.2. Discriminating. Humans also have the mental ability to make a different response for a nonexample of something that shar es some properties or characteri stics with previous examples. This process is called discrimination. For inst ance, children who comprehend the concept "dog" can correctly differentiate a dog from a cat or fox even though all share certain attributes such asfour legs and sharp teeth. Similarly, a student who comprehends the concept "strategy" can differentiate the term from "tactics" even though the actions share certain attributes or characteristics. 8.2.4. When we evaluate student performance at the comprehension level, we often ask students to give or recognize a new example of something we have taught. Students who respond correctly are able to generalize. When we ask students to differentiate between examples and nonexamples ofsomething we teach and they respond correctly, they are able to discriminate. Students who comprehend a lesson on the concept "prejudice" should be able to identify which written summaries of human interactions are examples illustrating prejudice and which are not. 8.2.5. Concept teaching is fundamental to instructi on at the comprehension level. As instructors, we want our students to be able to solve problems on the job (application). Problem solving requires that certain rules or principles have been learned. Thes e principles cannot be learned until certain concepts have been learned. In short, some educati onal objectives are prerequisites to others. 8.2.6. Being able to spout a memorized definition of an important concept is of little or no value on the job if a student cannot recognize new examples and nonexamples of that concept and use the concept in the work setting. Reciting the definition and ch aracteristics of leadership (a concept) is of little value on the job unless the person can apply leadersh ip in real-life situa tions. Thus, knowledge-level instruction for teaching concepts is insufficient . Comprehension-level instruction provides the missing link between recalling information about a con cept and the direct appli cation of the concept in real-life settings.
8.3. Organizing a Concept Teaching Plan. Since concept teaching is fundamental to instruction at the comprehension level, focus on organizing the lesson plan. Whether 5 minutes or several hours are required to teach a concept, the following format can be used: define the concept, teach its critical attributes, give examples and nonexamples.
8.3.1. The Definition. Defining the concept is the first step in concept teaching. Concept definitions can be found in dictionaries, glossaries, and in th e content of textbooks. Subject experts can form their own definitions. Concept definitions are statements that identify critical attributes and indicate how these attributes are combined. Often, we can start with the critical attributes and use them to write a definition.
8.3.2. The Attributes. A difficult step in teaching a concept is to differentiate among its attributes. An attribute is a special name used to refer to characteristics that determine whether a person, object, event, idea, or action is a member of a particular class (a concept). Critical attributes must be present in all members of the class. Variable attributes are characteristics shared by some but not all members of the class, but are not necessaryfor determining class membership. 8.3.2.1. If we wanted to teach the simple concept "s tool" to students, we might start with a dictionary definition. A stool is a single seat supported on legs or a pedestal and without arms or a back. In this definition, the critical attributes are (1) a single seat with (2) the seat resting on legs or a pedestal, and it cannot have (3) arms or (4) a bac k. Variable attributes would include characteristics such as a round seat, a rotating seat, four le gs, and a covering of leather material. The variableAFMAN36-2236 12 NOVEMBER 2003 107

attributes are shared by some members of the cl ass, but do not determine class membership. A stool may have a square seat, a fixed seat, a vary ing number of legs, or have a cloth covering or no covering at all. 8.3.2.2. If we do not have the expertise to label characteristics pr operly as critical or variable attributes, then we should turn to experts for assistance.
8.3.3. Examples. To make conceptual learning meaningful, we need good sets of examples and nonexamples. We build an understanding of concepts on examples. We vary the examples in such a way that students are able to make the discrimination s and generalizations on which real understanding depends. In selecting teaching examples, use thos e that are as different as possible in variable attributes. Otherwise, students may believe a particular variable attr ibute is critical to comprehending the concept. If each example of a stool shows a r ound seat, the student may be lieve that all stools have round seats. However, all examples must illustrate the critical attributes possessed by members of the class.
8.3.4. Nonexamples. Select nonexamples that vary in the cr itical attribute omitted. For the stool example, a good nonexample would be a seat for two people, such as a love seat. Since a stool must have legs or a pedestal, a single seat suspende d from the ceiling would serve as another nonexample. Nonexamples containing arms and a back should also be included.8.3.4.1. Technically, everything that is not an example is a nonexample. However, most nonexamples are not useful for concept instruction. Onlynonexamples that resemble examples, thus representing a source of confusion for students, are useful. Generally, when we use the word nonexample we are referring to those instances a student might incorrectly call an example. For each example we provide we should try to give a matched nonexample. 8.3.4.2. When a comprehension-level lesson calls for a definition, a concept lesson or concept teaching point is probably needed. The recommended organization in this chapter can be used for simple concepts like stool or for complicated concepts like ISD.
8.4. Testing Concept Learning. Students who learn new concepts shou ld be able to generalize their learning to the point that they can recognize new examples of the concept when they see them. In addition, they must be able to discriminate between an example and a nonexample of a given concept. 8.4.1. Test items that require generalizing and disc riminating are essential for testing concept learning. Test items that test concept learning may be either selection or supply items (see Chapter 22 ). These items generally require the use of a scenario or other description of an alleged example of a concept. 8.4.2. In supply questions, you may ask students to identify a concept from a scenario or require themto describe a scenario that meets all the critical attributes of a given concept. Selection items, particularly multiple choice, often contain a description or scenario in the stem. Then ask students which concept the scenario illustrates or whether there is enough information given to qualify as an example or nonexample of the concept.
8.5. Sample Concept Teaching Plans (Part I): 8.5.1. The Part I lesson plan illustrated in Figure 8.1. can be used for teaching a single concept. In this model, the term "concept" is included in the less on objective. The objective might also be written simply as "comprehend prejudice." Samples of behavior are based on the lesson objective. Notice that the108 AFMAN36-2236 12 NOVEMBER 2003

first main point is taught at the knowledge level, butthe first sample of behavior requires an explanation in the student's own words. In teaching the concept, at the end of the first main point the student should simply be able to recall the definition the instructor gave. However, by the time the lesson is complete and the concept is developed, the instructor would want students to define the concept intheir own words to show comprehension. The main points outlined in this model plan illustrate a recommended approach for teaching concepts at the comprehension level.
Figure 8.1. Sample Part I Lesson Plan.

8.5.2. The lesson plan cover sheet illustrated in Figure 8.2. can be used for teaching multiple concepts in a single period or several periods of in struction. Here, three concepts are being taught. Although the term "concepts" is omitted from the objective, each term represents a class or category of leadership. Samples of behavior are taken from th e concepts. The instructor should ensure an adequate sampling of each objective. Each concept is developed using definitions, attributes, examples, and nonexamples. Lesson Objective: Comprehend the concept of prejudice.

Samples of 1. Define prejudice in student's own words. Behavior: 2. Distinguish critical attributes of prejudice from variable attributes and nonattributes.

3. Give new examples and nonexamples of prejudice.

4. Given scenarios containing transactions between persons, differentiate between prejudicial and nonprejudicial interactions.

5. Explain the concept of prejudice.

Main Points: 1.*(K) The definition of prejudice.

2.**(C) The critical attributes of prejudice.

3.**(C) Selected examples and nonexamples of prejudice.

* Knowledge Level - Bloom's Cognitive Taxonomy ** Comprehension Level - Bloom's Cognitive TaxonomyAFMAN36-2236 12 NOVEMBER 2003 109

Figure 8.2. Sample Lesson Plan Cover Sheet.

Lesson Objective: Comprehend (1) authoritar ian, (2) laissez-faire, and (3) democratic leadership.

Samples of 1. Define authoritarian, laissez-faire, and democratic leadership in student's Behavior: own words.

2. Distinguish critical attributes of aut horitarian, laissez-faire, and democratic leadership from variable or nonattributes.

3. Distinguish examples from nonexamples of authoritarian, laissez-faire, and democratic leadership.

4. Explain the concepts of authoritar ian, laissez-faire, and democratic leadership.

Lesson Objective: 1. (C) Authoritarian leadership.

Main Points: a. (K) The definition of authoritarian leadership.

b. (C) The critical attributes of authoritarian leadership.

c. (C) Selected examples and nonexamples of authoritarian leadership.

Lesson Objective: 2. (C) Laissez-faire leadership.

Main Points: a. (K) The definition of laissez-faire leadership.

b. (C) The critical attributes of laissez-faire leadership.

c. (C) Selected examples and nonexamples of laissez-faire leadership.

Lesson Objective: 3. (C) Democratic leadership.

Main Points: a. (K) The definition of democratic leadership.

b. (C) The critical attributes of democratic leadership.

c. (C) Selected examples and nonexamples of democr atic leadership.110 AFMAN36-2236 12 NOVEMBER 2003

8.5.3. As we have seen thus far, the terms we use for communicating in the classroom usually represent concepts. When understanding key concepts is essential to comprehending a lesson objective, we need to take the time to ensure students reach th e comprehension level on each concept in the objective. Once students understand the concepts we are teaching, we can build on these concepts to more complex learning.
8.6. The Teaching of Principles. A principle is a statement of the relationship between two or more concepts. The principle is stated in terms of a conclusion (generalization) about a class of people, objects, events, ideas, or actions.8.6.1. The statement, "Barriers to creativity inhibit innovation in classroom instruction," is a principle. A relationship is established between concepts . To comprehend this relationship in a lesson, students need to comprehend the following concepts: barriers, creativity, inhibition, innovation, classroom, and instruction. In addition, students ne ed to comprehend the new concept formed by combinations of these concepts. When barriers and cr eativity are combined, we have a new concept—barriers to creativity. Similarly, when the concepts in the statement's predicate are combined, a new concept is formed representing actions (a class) th at "inhibit innovation in classroom instruction." The relationship between the combined concept in the s ubject and the combined concept in the predicate becomes very important when we plan lessons using deductive reasoning. 8.6.2. Other examples of principle st atements are: visual aids enhance an instructor's presentations; values influence a person's motivation; and the tactics employed by urban guerrillas pose a direct threat to the international order. Each statement ha s a subject and a predicate with a relationship established between the concepts. Each statement could serve as a lesson objective or as a main supporting point for another principle or concept being developed as an objective. 8.6.3. The statement, "Sergeant Jones teaches military science," is not a principle. The subject, Sergeant Jones, is not a concept. The predicate is a concept representing a class of people who teach military science. Since the subject of the statement is not a concept, we do not have the statement of a relationship between concepts. Neith er is "military strategy in uncon ventional warfare" a principle; while the phrase involves more than one concept, a critical attribute of a principle is missing—the statement of a relationship between concepts. Clos er examination of the phrase "military strategy in unconventional warfare" will reveal that it is a conc ept, which could be effectively taught using definitions, attributes, examples, and nonexamples. 8.6.4. The teaching of principles is very important in Air Force instruction. If students are going to comprehend the principles taught, they must be able to reason logically. Logically organized lesson plans will assist students in reasoning through to the principles we are teaching.
8.7. Logical Lesson Planning. As we organize our lessons to teachprinciples, some understanding of approaches that may be used will assist in planning topically organized lessons. The following provides information on these approaches:
8.7.1. General-to-Specific: 8.7.1.1. The general-to-specific pres entational approach (often referred to as "deductive reasoning") is a mental process in which we start with a principle to be taught and then apply this principle in specific instances to form supporting main points. The reasoning is from general to specific.AFMAN36-2236 12 NOVEMBER 2003 111

8.7.1.2. For example, suppose our lesson objective read: "The objective of this lesson is for each student to comprehend that barriers to creativity inhibit innovation in classroom instruction ." The italicized portion of the objective is a principle representing the content to be taught. If a topical pattern is chosen for lesson development, the focu s should be on the subject of the objective statement—barriers to creativity. By moving from the general to the specific, we know from our review of lesson content that fourcommon barriers to creativity are fear, prejudice, habit, and inertia. 8.7.1.3. If the lesson content provides logical support for the principle in the objective, then we can reason that fear, prejudice, habit, and inertia (the specific instances) each inhibit innovation in classroom instruction. The reasoning might be depi cted as follows: Barriers to creativity inhibit innovation in classroom instruction. Fear, prejud ice, habit, and inertia are barriers to creativity. Therefore, fear inhibits innovation in classroom instruction.
NOTE: By starting with the principle, we topically divide the subject (first concept of the principle) into more specific instances and then draw a ne w conclusion about these specific instances. 8.7.1.4. The next step is to write main points as principles, which support the lesson objective. The simplest way is to use the principles deve loped through the general-to-specific approach. The first main point might read, "Habit inhibits in novation in classroom instruction." Using a second approach, we might write the principle, "Habit causes instructors to resist change." The second principle is more specific than the first in showing how habit inhibits innovation in classroom instruction. Notice that the predicate in the second principle supports the predicate of the principle stated in the lesson objective. When the main points are made more specific, this requirement must be met. We could continue the process and write principles for fear, prejudice, and inertia using either approach. NOTE: If the more specific statement is usedin the first main point, then the same approach should be used in writing the other main points.
8.7.1.5. Since instructors seldom have time to deve lop all aspects of a topic, sampling is usually required. In this lesson, the instructor must provide sufficient evidence to support the lesson objective. If two barriers to creativity are discussed indepth, this support should provide sufficient evidence for students to accept the conclusion stated as the principle in the objective. In this illustration, sampling of two of the four barriers should provide suff icient evidence to get students to accept the broader generalization.
8.7.1.6. On another occasion we could be faced withfourteen defense mechanisms for use in supporting the principle "defense mechanisms allow the individual to cope with stress." Three might be chosen as main points to show how each allows the individual to cope with stress. Then wewould ask the students to accept a conclusion about all defense mechanisms based upon a sampling of three. Samples should be representati ve of all members of the class. Enough samples should be chosen to provide sufficient ev idence for drawing the correct conclusion. 8.7.1.7. When a topical organization is chosen as the best means for developing the lesson, the process just described can be used effectively to develop main points from an objective stated as a principle. Similarly, a main point stated as a principle can be divided topically into subpoints by using the same method. Other common organizational patterns for use on comprehension-level plans will be developed in the sample plans, which follow in this chapter.
8.7.2. Specific-to-General. While the general-to-specific approach assists us in breaking down a principle into its component elements, the specific-t o-general approach (often referred to as "induc-112 AFMAN36-2236 12 NOVEMBER 2003

tive reasoning") can assist us in developing principl es from specific examples. In teaching principles, we ask students to take pieces of evidence we pres ent to form a related pattern and then draw a conclusion from the evidence. The pieces of evidence are definitions, examples, statistics, comparisons, and quotations or testimony used as support material . When skillfully put together by the instructor,these pieces of evidence and their relationship form a pattern, which leads to a generalization. Movement in the lesson is from specific instances (support) to a general conclusion (the principle). 8.7.2.1. For instance, a security forces instructor presents the following evidence: A security forces vehicle overturned on perimeter road last week during an emergency run in the rain. On Monday, a dependent's car slid on a wet road into a base exchange oil truck. And, an airman's motorcycle went out of control on a base street following a rain shower. Based on this evidence, we would conclude that base roads are slippery when wet. The conclusion would be even stronger if a statistic was then added: "Last year, accord ing to security forces records, 45 accidents occurred on base roads following rainstorms." The statistic serves as a summary of specific instances and as proof support for the conclusion drawn. 8.7.2.2. In teaching principles at the comprehensi on level, instructors should keep in mind both the inductive and the deductive nature of lesson de velopment. In lecturing, we should include only those items of support material needed to clarify and prove the principle being taught. In discussions, we should keep students on track so material that is irrelevant to support of the principle does not develop. 8.7.2.3. No matter what method is used, we should en sure that we develop sufficient support for comprehending of the principle. Often, strong support is lacking and we can only say that, based upon available evidence, the conclusion stated as a principle is probably correct. In all cases, we must ensure that logical organization is used.
8.8. Testing Principle Learning: 8.8.1. Students who learn new principles should be able to understand the relationship between two or more concepts. They ought to be able to translate these relationships into their own words, interpret instances where the principle is applied or ought to be applied, and make si mple predictions based on their understanding of the principle involved. It is essential that test items that measure principle learning determine the extent to which students unde rstand the relationship expressed in the principle rather than just an understanding of the indi vidual concepts contained in the principle. 8.8.2. Test items that test concept learning ma y be either selection or supply items (see Chapter 22 ). Like concept test items, items that measure prin ciple learning often require scenarios to set up the proper conditions for testing. In supply questions, stude nts may be asked to identify a principle that is working or ought to be working in a given scenari o. Selection items, particularly multiple choice, often contain a description or scenario in the stem. Students are then asked to identify the principle at work, explain the principle involved, or make a lim ited prediction about the situation based on their understanding of the principle.
8.9. Sample Plans for Teaching Pr inciples (Part I—Cover Sheet). Various organizational patterns can be used successfully in teaching principles at th e comprehension level. Some of the most common patterns are illustrated in Figure 8.3. and Figure 8.4.
8.9.1. Topical Pattern:AFMAN36-2236 12 NOVEMBER 2003 113

8.9.1.1. The lesson plan cover sheet (Part I) provided at Figure 8.3. illustrates the use of topical organization to teach a principle. Main points one and two are supporting principles, which serve as specific instances, developed from the less on objective. Because only two of four common barriers are covered, the main points are samples that serve as evidence in support of the objective. Ifthe students comprehend the two conclusions stated as main points, then the instructor has sufficient evidence to support the lesson objective.
Figure 8.3. Topical Pattern Sample.

8.9.1.2. On the sample plan, notice that the lesson obj ective is repeated as the third main point. This approach is recommended to ensure that the instructor gives attention to development of thelesson objective. Additional barriers to creativity can be introduced in the third main point showing how they inhibit innovation in classroom instruction. The instructor has the final responsibility for tying the lesson together to show how all the lesson parts lead to the conclusion stated as the principle in the lesson objective.
8.9.1.3. Figure 8.3. illustrates a single objective plan. The key principle in the objective was supported by the principles stated as main points. However, it would be possible for each main point to stand by itself as an objective for a lesson.
8.9.1.4. Figure 8.4. illustrates a multi-objective plan. Tw o objectives are being taught in the same period of instruction. Each objective is then s upported by main teaching points. The main points under each objective are developed topically. Each is a specific instance (principle) that supports the broader principle stated in the objective. Notice that the predicates of the objectives are repeated as predicates for the main points. While repeating the predicates in the main points appears redundant, such action ensures that principl es are taught instead of concepts. For instance, teaching "fixed attitudes" as a concept requires different organization and support than teaching that "fixed attitudes cause instructors to resist creative change." While the experienced instructor may not repeat the predicate each time on the lesson plan, the mental connection is still made between the concepts to form the principle (betwe en fixed attitudes and things that cause instructors to resist creative change). Lesson Objective: Comprehend barriers to creativity inhibit innovation in classroom instruction.

Main Points: 1. (C) Habit causes instructors to resist change.

2. (C) Prejudice restricts an instructor's thinking.

3. (C) Barriers to creativity inhibit innovation in classroom instruction.114 AFMAN36-2236 12 NOVEMBER 2003

Figure 8.4. Sample Multi-Objective Plan.

8.9.2. Problem-Solution. Another common pattern is problem-solution. Figure 8.5. illustrates a plan organized with a problem-sol ution organizational pattern. Lesson Objective One: Comprehend that habit caus es instructors to resist creative change.

Main Points: 1. Reliance on old solutions for solving new problems causes instructors to resist creative change.

2. Fixed attitudes cause instructors to resist creative change.

3. Fixed behavioral patterns cause instructors to resist creative change.

4. Habit causes instructors to resist creative change.

Lesson Objective Two: Comprehend that prejudice restricts an instructor's creative thinking.

Main Points: 1. Emotional bias restricts an instructor's creative thinking.

2. Sticking to favorite teaching methods restricts an instructor's creative thinking.

3. Any irrational attitude of hostility dire cted against an individual student or group of students restricts an in structor's creative thinking.

4. Prejudice restricts an instructor's creative thinking.AFMAN36-2236 12 NOVEMBER 2003 115

Figure 8.5. Sample Problem-Solution Organizational Pattern.

8.9.2.1. On this Part I, the lesson objective is a statement of a relationship between concepts, so a principle is involved. Main point one states th e problem and main point two, the solution. Main point two is a repeat of the objective indicati ng the instructor is putting emphasis on the solution for evaluation purposes. 8.9.2.2. If the instructor wanted to put equal em phasis on both the problem and solution in this lesson, it would be a multiple-objective plan w hose objectives might read: "Comprehend that (1) child abuse damages the health of children and (2) widespread public concern is the most effective weapon available for combating child abuse." 8.9.2.3. With a problem-solution pattern, the problem is normally covered before the solution although the reverse strategy might be used. Another common strategy is to develop the problem as a main point, examine several possible solutions as a main point, and then pick the best solution for development as the final main point. 8.9.2.4. Although the main points are organized us ing a problem-solution pattern, the subpoints could be developed topically. Using the principl e in the first main point, child abuse could be divided into physical and mental abuse. Supporti ng subpoints might read: (1) Physical abuse damages the child both physically and psychologically and (2) mental abuse damages the child psychologically. Or, more simply, (1) Physical abus e damages the health of children and (2) mental abuse damages the health of children. Similarly, the subpoints for the solution could be developed topically. 8.9.2.5. Thus far we have examined topical and problem-solution patterns commonly used in developing comprehension-level lessons for teaching principles. A cause-effect pattern can also be used effectively.
8.9.3. Cause-Effect. Figure 8.6. illustrates a cause-effect organizational pattern. Lesson Objective: Comprehend that widespread public concern is the most effective weapon available for combating child abuse.

Main Points: 1. (C) Child abuse damages the health of children.

2. (C) Widespread public concern is the most effective weapon available for combating child abuse.116 AFMAN36-2236 12 NOVEMBER 2003

Figure 8.6. Sample Cause-Effect Organizational Pattern.

8.9.3.1. This particular lesson objective is a stat ement of the relationship among three concepts. Usually only two of the concepts are stated in th e objective, but as the lesson will point out, a third concept (the distant problem or condition) will be used to cancel or explain the concept stated as the proximate problem or condition. The statement will consist of two majo r concepts representing sexism (the proximate problem or condition) and actions inhibiting effectiveness of skilled female employees (the result). 8.9.3.2. The last main point is a statement of the lesson objective because instructor emphasis in a cause-effect pattern of organization is usually on the effect (linking of the proximate problem and the result). However, if the principles stated as cause and effect are of equal importance, then the plan would reflect a compound objective. The objec tive might read, "Comprehend that (1) a negative self-concept fosters sexism in the workplac e and (2) sexism inhibits the effectiveness of skilled female employees."
8.9.4. Pro-Con Plus One. A fourth pattern for teaching principles might be labeled pro-con plus one. Students are often asked to examine both sides of anissue. If a discussion is being conducted, the lesson might end with students comprehending the pro a nd con positions but not drawing a final conclusion. This development is acceptable for certain subjects, especially those involving a great deal of controversy. At other times, the in structor can lead students to a final conclusion if the lesson is planned properly. Figure 8.7. illustrates this type pattern. 8.9.4.1. While each main point is an important principle, the key principle in the objective goes beyond the main points in the reasoning process. Therefore, Figure 8.7. illustrates the Part I of a single objective plan. The objective is to draw a final conclusion. Main point three, the lessonobjective, requires development that will encompass comparing and contrasting the pro-con arguments. Lesson Objective: Comprehend that sexism inhibits the effectiveness of skilled female employees.

Main Points: 1. (C) A negative self-concept fosters sexism in the workplace.

2. (C) Sexism inhibits the effectiveness of skilled female employees.AFMAN36-2236 12 NOVEMBER 2003 117

Figure 8.7. Sample Pro-Con Plus One Organizational Pattern.

8.9.4.2. For strategy, the instructor wants to deve lop a logical arrangement to support the lesson objective. By starting with the con argument, students comprehend how planned obsolescence exploits our natural resources, then attention turn s to the value of such a program. Instead of leaving students wondering what conclusion should be dr awn, the instructor leads them to a final conclusion (the lesson objective).
8.10. Sampling Achievement of the Objective. Paragraphs 8.9. through 8.9.4.2. addressed placing emphasis on the organization of main points in suppo rt of a lesson objective. We will now focus on samples of behavior written on the same objectives.
8.10.1. Sample One: Topical Pattern. Figure 8.8. provides a sample of a lesson organized topically. Two principles were used as main poi nts in support of the lesson objective. Lesson Objective: Comprehend that a program of planned obsolescence must represent a balance between the exploitation of our natural resources and the generation of a higher standard of living.

Main Points: (C) Planned obsolescence exploits our natural resources.

(C) Planned obsolescence generates a higher standard of living.

(C) A program of planned obsolescence must represent a balance between the exploitation of our natural resources and the generation of a higher standard of living.118 AFMAN36-2236 12 NOVEMBER 2003

Figure 8.8. Sample 1—Topical Pattern.

8.10.1.1. Notice that the samples of behavior are written on the lesson objective—not on main points. The principle in the objective is a broader generalization than the principles stated as main points. Although samples of behavior written on th e main points might provide some evidence of achievement of the objective, significant evidence must develop from samp les of behavior written directly on the objective. 8.10.1.2. If samples are written on the principles statedas main points, they should be used for test questions written for purposes of formative evaluation (sometimes referred to as diagnostic testing where results are used to assist students in overcoming deficiencies before a final evaluation or posttest is given). Samples written on the objective would be used for summative evaluation (the final evaluation or posttest).
8.10.2. Sample Two: Parallel Principles. Figure 8.9. shows two parallel principles stated as individual objectives. The objectives are of equal impor tance, so a similar number of samples of behavior are written on each. Lesson Objective: Comprehend that barriers to creativity inhibit innovation in classroom instruction.

Samples of Behavior: 1. Describe the relationship between ba rriers to creativity and classroom instruction.

2. Estimate the consequences on classroom instruction when barriers to creativity are in operation in a classroom.

3. Explain why barriers to creativity inhibit innovation in classroom instruction.

4. Interpret the relationship of the barriers to creativity to each other as they interact to inhibit innovation in classroom instruction.AFMAN36-2236 12 NOVEMBER 2003 119

Figure 8.9. Sample 2—Parallel Principles.

8.10.3. Sample Three: Problem-Solution Pattern. Figure 8.10. shows Part I organized using a problem-solution pattern. The solution represented the last main point and the lesson objective. In this lesson, the instructor is putting emphasis on the so lution for evaluation purposes. Therefore, the samples of behavior are only written on the solution. Inmost problem-solution lessons, it is only necessary to test the solution because an understanding of the solution can only be based upon an understanding of the problem. However, if the problem and solution are of equal importance, then a multi-objective format can be used and samples of behavior would be written for the problem and the solution. Lesson Objective: Comprehend that (1) habit causes instructors to resist creative change and (2) prejudice restricts an instructor's creative thinking.

Samples of Behavior: 1a. Predict the effect on a group of instructors when a new supervisor takes over and makes an effort to innovate.

1b. Explain the relationship between an instructor's habits and the desire to be creative in the classroom.

1c. Give original examples of habits that cause instructors to resist creative change.

2a. Interpret the relationship between prejudice and an instructor's reactive thinking.

2b. Explain how prejudice acts as a barrier to an instructor's creativity in the classroom.

2c. Give original examples of prejudice operating to restrict an instructor's original thinking.120 AFMAN36-2236 12 NOVEMBER 2003

Figure 8.10. Sample 3—Problem-Solution Pattern.

8.10.4. Sample Four: Cause-Effect Pattern. Figure 8.11. illustrates Part I using a cause-effect pattern. The objective and main point two stated the ef fect. In this lesson, emphasis is placed on the effect and not the cause. Therefore, samples of behavior ar e needed for the effect alone. In most cause-effect lessons, testing of the effect is all that is need ed because an understanding of the effect is based upon an understanding of the cause. However, if the cause and effect are of equal importance, the plan would be multi-objective with samples of behavior written on both the cause and the effect.
Figure 8.11. Sample 4—Cause-Effect Pattern.

Lesson Objective: Comprehend that widespread public concern is the most effective weapon available for combating child abuse.

Samples of Behavior: 1. Estimate the consequences of an aroused public on the problem of child abuse.

2. Interpret the relationship between publ ic arousal over child abuse and the number of cases of child abuse.

3. Explain how a concerned public can reduce the number of child abuse cases.

4. Generalize as to actions a concerned public can take that will reduce the number of child abuse cases.

Lesson Objective: Comprehend that sexism inhibi ts effective use of skilled female employees.

Samples of Behavior: 1. Give original examples that illustrate ways sexism inhibits effective use of female employees.

2. Generalize as to the relationship between sexism and the effective use of female employees.

3. Estimate the consequences when sexism prevents effective use of skilled female employees.

4. Explain how sexism inhibits effective use of skilled female employees.AFMAN36-2236 12 NOVEMBER 2003 121

8.10.5. Sample Five: Pro-Con Plus One Pattern. Figure 8.12. illustrates a pro-con plus one pattern. Because there is a final conclusion in this le sson (the objective and main point three), samples of behavior should be based upon th is conclusion. Samples based upon main points one and two would not provide the best source of evidence for evaluatin g achievement of the objective. If samples are written for main points one and two, they should be used as a basis for interim evaluation of students and not a final evaluation.
Figure 8.12. Sample 5—Pro-Con Plus One Pattern.

8.11. The Comprehension-Level Summary. In developing a comprehensi on-level lesson plan, we mustplan summaries carefully. This rule holds true even in guided discussions where most of the support material comes from students. 8.11.1. A comprehension-level summary (or summation) reviews and expands on key material and develops relationships that lead to a generalizati on that is (or supports) the instructional objective. Only the most significant information is restatedand expanded. The focus is on how the items are related. This relationship allows a general conclusion to be drawn. The generalization will either be a concept or a principle. The concept or principle de veloped is the stated lesson objective or is used to support the objective. 8.11.2. This type of summary permits the introduction of new support material (from instructor or students) needed to establish the generalization. Th e summary may require anywhere from a few minutes to a much longer period of time depending on the amount and complexity of the material. 8.11.3. Use a comprehension-level summary to summarize support material needed to achieve a comprehension-level main point. A summary of this type is not used for a knowledge-level objective or a knowledge-level main point. Lesson Objective: Comprehend that a program of planned obsolescence must represent a balance between the exploitation of our natural resources an d the generation of a higher standard of living.

Samples of Behavior: 1. Explain the economic cycle initiated by the practice of planned obsolescence.

2. Differentiate between positive and negative effects on our national economy of a program of planned obsolescence by the US auto industry.

3. Estimate the effect on our economy of a program of planned obsolescence.

4. Predict ways in which industry and the government can work together to achieve a balance in conserving resource s and a higher standard of living through planned obsolescence.122 AFMAN36-2236 12 NOVEMBER 2003

8.11.4. Figure 8.13. is an example of a comprehension-leve l summary for a lesson illustrated (the Part I) earlier in this chapter. While comprehens ion-level summaries would normally be given after each of these main points, only the summary in the lesson conclusion is illustrated.
Figure 8.13. Comprehension-Level Summary.

8.12. Summary. Comprehension is a step beyond the simple re membering of material and represents the lowest level of understanding. Stude nts who comprehend are able to grasp the meaning of material. Objective: Barriers to creativity inhibit innovation in classroom instruction.

Main Points: 1. Habit causes instructors to resist change.

2. Prejudice restricts an instructor's thinking.

3. Barriers to creativity inhibit innovation in classroom instruction.

Summary: In this lesson we have examin ed two barriers to creativity that inhibit innovation in classroom instruction. As a barrier to creativity, habit causes in structors to resist change. We find instructors using old solutions in trying to solve new problems because su ch an approach is comfortable. Instructors also develop fixed attitudes, which prevent them from acce pting something that is new or different. Many instructors develop fixed behavioral patterns that establish a very comf ortable instructional routine. Old solutions, fixed attitudes, and fixed be havioral patterns are habits, which ca use instructors to resist change. Such habits become a barrier to the instructor's creativity. Prejudice, a second barrier to creativity, restrict s an instructor's creative thinking. When instructors are prejudiced, emotional bias can cause them to reject new ideas. Prejudice can also cause instructors to stick to favorite teaching methods instead of trying innovative instructional methods. Or instructors can reserve the assignment of key curriculum hours for personal frie nds, often preventing new and talented instructors from improving a course. Emotional bias, sticki ng to favorite teaching methods, and assigning key curriculum hours to friends are forms of prejudice that restrict an instructor's creative thinking. Such forms of prejudice become barriers to the instructor's creativity. In viewing barriers to creativity, four are often mentioned. In addition to habit and prejudice, fear and inertia can serve as barriers to an instructor's creativ e efforts. The fear of ridicule, failure, being laughed at, or being branded as an oddball limits the freque ncy of new, original, or unconventional ideas among instructors. Inertia causes groups to accept things as they are and discourages members from looking for new and better ways of doing things. The Air Force has a continuing need for instructors with the imagination to develop new and useful educational tools. However, corrective action will not occur until instructors understand that barriers to creativity inhibit innovation in classroom instruction.AFMAN36-2236 12 NOVEMBER 2003 123

8.12.1. Concept teaching is fundam ental to instruction at the comp rehension level. A concept is a class of people, objects, events, ideas, or actions grouped together on the basis of shared attributes (characteristics) and are called the same name. 8.12.2. A student who comprehends a concept is able to generalize and recognize new examples of the concept that differ in some way from previouslymet examples. The student is also able to discriminate by identifying nonexamples that share propertie s or characteristics with previous examples. 8.12.3. In teaching a concept, instructors define the c oncept, teach its critical attributes, give examples, and then nonexamples. A very difficult step in planning for concept teaching is to identify the critical attributes. 8.12.4. A principle is a statement of the relationship between two or more concepts. Principles represent conclusions drawn in comprehension-level lessons and are based upon support material used in the lesson. Deductive reasoning is often used to de velop main points that provide logical support to the lesson objective. Deductive or inductive reason ing is used in the classroom when an instructor puts supporting points together to draw conclusions. 8.12.5. Common organizational patterns used in comp rehension-level lessons for teaching principles include topical, problem-s olution, cause-effect, and pro-con plus one.124 AFMAN36-2236 12 NOVEMBER 2003

Chapter 9

DEVELOPING HIGHER-LEVEL COGNITIVE LESSONS

9.1. Introduction. Teaching and learning at the higher levels of understanding are difficult and challenging. These higher levels of understanding include the ability to apply, analyze, synthesize, and evaluate. This is not to say that the two lower levels of the cognitive taxonomy—know ledge and comprehension— necessarily come easily to students. They may not. But when students learn at the higher levels, they are called upon to put theory into prac tice. Both the instructor and the st udent are challenged to determine whether they can put into use what is being learned.
9.1.1. Little of what we learn in the classroom has real value in and of itself. Few concepts or principles have value unless students can apply them in real work situations. We should seriously question the learning of concepts and principles for which there is no apparent need for application. 9.1.2. Not all lessons should be taught and tested at the higher levels of learning. Efficient, effective teaching should be directed at le vels that are neither higher nor lo wer than those appropriate for the desired learning outcomes.
9.2. Higher Levels of Cognitive Learning:
9.2.1. Application. Students who can apply concepts and principles in new and different situations are operating at the application le vel of understanding. When student s apply or recognize the application of principles and other generalizations, th ey have clearly gone beyond simple understanding. Students who can answer theory questions about prin ciples of management, hum an relations, and other significant topics but are not able to put their unde rstanding into practice are not yet at the application level of learning. 9.2.1.1. Students who solve problems by applying principles and other generalizations and are able to explain why are working at least at the a pplication level. It is lik ely that any educational experience beyond an introductory course or basi c principles course will and should involve the application level of learning. Instructors and course developers should plan lessons to this level with the realization that they require more advanced planning and more effective classroom skills than the lower levels of learning.
9.2.1.2. Figure 9.1. contains typical application-level st udent-centered instructional objectives.AFMAN36-2236 12 NOVEMBER 2003 125

Figure 9.1. Instructional Objectives at the Application Level.

9.2.2. Analysis. The ability to analyze a problem or situa tion into its component parts and to determine relationships that exist between them character izes the student at the analysis level. Students retain and use analytic skills long after they have forgotten the specific subject matter used as a vehicle to develop the skills. "Scientific problem solv ing" and other examples of management tools for decisionmaking characterize this level of learning. 9.2.2.1. Learning and testing at the analysis level and higher can be very complicated. There are few opportunities for "school solutions," and solutions to problems may not be clearly right or wrong. One student's analysis may be better than another, but both may be correct. Often students will discover relationships between elements of a situation not anticipated by the instructor. Being unable to anticipate and plan for all student interp retations can make teaching at the analysis level very challenging for even the most experienced instructor.
9.2.2.2. Figure 9.2. contains some typical student-centered instructional objectives at the analysis level. - Apply the concepts of ISD to actual curriculum development situations.

- Apply basic human relations skills to situations involving superior-subordinate interpersonal relations.

- Apply their understanding of the Russian de finition of democracy to the interpretation of real or hypothetical situations.

- Apply the principles of war to scenario s in simulated, fictitious theaters of operation.126 AFMAN36-2236 12 NOVEMBER 2003

Figure 9.2. Instructional Objectives at the Analysis Level.

9.2.3. Synthesis. Students who can rearrange elements of a problem or situation in new and creative ways to form relationships not previously apparent are functioning at the very high level of cognition known as synthesis. Since creativity is so importan t to this level of understanding, instructors may have to evaluate learning at this level without a model to compare to the student's answer or solution.Previously learned skills of deductive and inductive reasoning play a critical role in the student's ability to synthesize. 9.2.3.1. Learning and planning for learning at the synthesis level are certain to be complex and time consuming. Only experienced instructors shou ld attempt to bring students to such a high level of understanding. Fortunately, or unfortunately depending on your point of view, few right or wrong answers to problems exist at this level. Rather than judge student work against a set of absolute standards, instructors must engage in a shared experience to explore new alternatives with students while providing scholarlyguidance and seeking expert opinion.
9.2.3.2. Figure 9.3. contains some typical student-centered instructional objectives at the synthesis level.
Figure 9.3. Instructional Objectives at the Synthesis Level.

- Analyze the communication network of an organization to determine discrepancies between the formal and informal organization chart.

- Conduct an analysis of work flow within an organization to determine potential areas of improved productivity.

- Analyze several sources of input into an accident investigation to determine fact and uncorroborated assumptions or allegations.

- Analyze apparently unrelated government doc uments to identify principles of strategy or tactics.

- Synthesize data from recent armed conflicts to propose new principles of warfare that may affect future conflicts. - Synthesize a variety of inputs from manage ment decision-making techniques to determine a course of action that is highly supportive of the mission. - Prepare an accident prevention plan for a new facility that synthesize s appropriate regulations as well as the most applicable principles of safety engineering. - Synthesize relevant research and expert opini on into the revision of existing USAF instructor manuals and other related literature.AFMAN36-2236 12 NOVEMBER 2003 127

9.2.4. Evaluation. Carefully considered judgments made by students based upon objective standards are characteristic of students at the highest level of learning, evaluation. Informed judgment on the part of students requires them to separate personal feelings from obj ective analysis and apply a set of quantitative and qualitative criteria to given situat ions. This level of learning requires understanding so complex that few Air Force schools will include it in their curriculums. Where it is applicable, however, it represents the highest level of cognitive behavior and creative problem solving.9.2.4.1. Teaching and testing at this level is the most difficult task facing an experienced instructor. ( NOTE: Those instructors not at this level themselves should not attempt to teach these lessons.) The constant search for new and different problems to be evaluated for creative solutions is time consuming and difficult. Lack of time prev ents many instructors from attempting higher learning outcomes. Teaching at this level represents the best of what we mean when we refer to the instructor as a real expert or a genuine scholar.
9.2.4.2. Figure 9.4. contains some typical student-centered instructional objectives at the evaluation level.
Figure 9.4. Instructional Objectives at the Evaluation Level.

9.3. Principles of Teaching and Learning at the Higher Levels. There are several important principles to keep in mind as we plan and teach at all the higher levels. The significance of one principle over another varies from situation to situation. However, they all apply generally to instruction at the higher levels. It is unlikely that instruction can be successful at these levels if any of these principles have been violated. Several of these principles are as follows: 9.3.1. Carefully planned learning experiences at the de sired level of learning facilitate the transfer of learning. 9.3.1.1. Students who are attempting to reach the hi gher level of learning need sufficient opportunity to practice their newly acquired skills. If we take our students through the understanding of theory and then ask them to apply the theory w ithout appropriate guidance, we are violating this critical principle. 9.3.1.2. Students need to be guided through experiences at each level of the taxonomy so they can benefit from the constructive feedback of their instructor and other experts. Giving good feedback in carefully structured learning situations isn't spoon feeding; it's good teaching and critical for learning. 9.3.2. Experience with new, unfamiliar, or substantia lly different learning and testing situations promotes insight at the higher levels of learning. One of the most demanding elements of learning at the higher levels is the need to come up with many new and different situations for applying, analyzing, synthesizing, or evaluating principles and other gene ralizations. Essential principles may be the same - Evaluate alternative operational plans to determine a justifia ble course of action. - Evaluate an original research project including data-gathering techniques, research methodology, validity of findings, and logic of recommendations. - Evaluate a system for the comparative asse ssment of the performance of subordinates to determine its applicability to a given situation.128 AFMAN36-2236 12 NOVEMBER 2003

in apparently diverse situations, but there must be enough newness about the problems that students cannot solve them from rote, memory, or recollection. This critical need for instructors to develop apparently different teaching and learning situations with common principles places great demands on their subject matter expertise.9.3.3. Effective feedback for learning demands flexibility in evaluating the products of student outputs at the higher levels. 9.3.3.1. Teaching, learning, and testing at the higher taxonomy levels places considerable extra demands on the flexibility of both the instructor and the student. Creativity is characteristic of all the higher levels and should be encouraged! Be cause instructors and students bring their unique backgrounds with them into the classroom, it is often impossible to anticipate the exact solution any student may offer to a given problem. 9.3.3.2. While it is possible to retain the notion of a "school solution" for some application level learning, most learning at the application leve l and beyond probably does not lend itself to predetermined model answers. Feedback to the student s becomes much more difficult and subjective. Evaluation of the student becomes more complex because a solution or strategy proposed by a student may be 180 degrees in the opposite direction of one proposed by the instructor, and yet may be of equal or greater value. 9.3.3.3. As complex as this situation may be, the instructor should remember that feedback and evaluation must remain objective to be worthwhile. Students are to solve problems and propose alternatives based on solid fact s and a thorough grasp of higher principles. Learning at the higher levels is disciplined and scholar ly and must not be reduced to expressions of personal preference and unsubstantiated opinion, as is often the case. 9.3.4. The need for supplemental expert evaluation ofstudent outputs increases as the level of learning increases. 9.3.4.1. The increased use of expert judgment in eval uation is closely related to the need for flexibility in evaluating the products of student learni ng we have just described. By definition, students must produce creative insights or solutions when working at the higher taxonomy levels. Often, this creativity is best judged by an obj ective evaluation of proposed student evidence of learning by one or more experts in a given subjec t matter area. The instructor may or may not be one of the expert judges. 9.3.4.2. Since learning activities at the higher leve ls are often highly individualized, students may be doing much, if not most, of their work outside of class. Often, instructors and students engage in mutual goal setting through negotiation. Resear ch proposals and individual learning contracts are often characteristic of learni ng at these levels. In these cases, a mutually agreed upon plan is developed so that both the instructor and the student agree to the scope and procedures of the learning activity as well as the goals. Often, such research and individual contracts contain a provision to have them evaluated by third parties. Th e instructor may also evaluate the student product, but a more qualified expert may be consulted if the project is highly specialized or technical. 9.3.5. The focus of evaluation at the higher levels gradually shifts from a primary concern about the process used to arrive at a decision on the quality of the decision itself. 9.3.5.1. As students function at the higher levels of taxonomy, it becomes le ss important to predict their exact behavior in the process of insight or problem solving. The product of their efforts even-AFMAN36-2236 12 NOVEMBER 2003 129

tually becomes more important than the proc ess they go through. Sooner or later the student's work needs to be evaluated on whether or not a proposed solution will work. 9.3.5.2. Eventually the books must balance and the solution must have the potential to relieve a problem. A solution that comes about as a result of applying a scientific problem-solving method but doesn't work is unacceptable at the highest leve ls of learning. Checklists or models often helpful at the application level may be of little or no value at the ev aluation level. In fact, solutions generated by students functioning at this level of learning are characteristically different from one student to another in their approach and process. 9.3.5.3. When giving students feedback on their produc ts, it may be necessaryto help them with difficulties they had in their process of problem so lving. This feedback should aid them in their next attempt to produce a product. 9.3.5.4. The concept of sampling described in Chapter 4 still has validity but must be approachedwith more caution at the higher levels. Predictable samples of behavior characterize the knowledge and comprehension levels but become increasingly difficult from that point upward in the taxonomy. Many samples of behavior are broad in dicators of student activity which serve as a baseline for evaluation. The samples can still be of va lue if they sufficientlydescribe the essential elements of the product at these levels. However, efforts to write criterion objectives at these levels may produce such general objectives that they are meaningless.
9.4. Lesson Planning. The basic skills for planning lessons at th e higher cognitive levels are generally as described elsewhere throughout this manual. The questioning skills discussed in Chapter 11 are critical to promoting learning at these levels. Chapter 27 describes feedback considerations that become more important as we go up the ladder of the taxonomy. Discussion skills ( Chapter 14 ), student differences ( Chapter 28 ), and counseling skills ( Chapter 30 ) take on new and more important dimensions as we attempt to cause learning at the highest levels. Di fferences in planning techniques between lower and higher levels are most often in terms of degree rather than totally different skills.
9.4.1. Style and Format. Actual lesson plans are often quite simple for conducting class sessions at these levels. The detailed step-by-step outline thatcharacterizes lower-level lesson plans may not be appropriate here. The sample le sson plans in the attachments ( Attachment 2 through Attachment 10 ) illustrate the amount of detail recommended for teaching at the knowledge and comprehension levels. 9.4.1.1. Typical application-level lesson plans, as we ll as those for higher levels, might be brief outlines of activity or as complex as those shown in Attachment 2 through Attachment 10 . These lesson plans go into great detail to include all of the information needed to teach the lessons. Often, an outline is developed around a set of lo cally produced or commercially prepared materials. 9.4.1.2. The exercises involved might include case st udies, gaming exercises, in-basket exercises, role-playing scenarios, or other elaborate simulations. See Chapter 12 for a description of these and other teaching methods. (These complex exer cises will not be attempted in this manual beyond the case study in the attachments found at th e back of the manual. Good simulation exercises are often many pages long and go beyond the scope of this chapter.)
9.4.2. Classroom Instruction as a Part of a Total Learning Package. The higher the level of learning, the less relative significance traditional cla ssroom instruction and time in class are likely to play. Much of what is to be learned will be learned outside of class. Readings, research, personal inter-130 AFMAN36-2236 12 NOVEMBER 2003

views, and other individualized learning experiences pl ay more of a role than at the lower levels. The classroom experiences may become the group anc hor for individualized experiences. At higher level-of-learning sessions, students may share what they have gleaned from their personal research and share their creative ideas for peer and instructor reaction. Depending on the overall methodology chosen, actual classroom time may account for just a fraction of the time spent in the total learning experience.
9.4.3. Quality of the Simulation in the Learning Experience. Perhaps the single biggest factor that discourages teaching at the higher cognitive levels is the need for quality simulation exercises. In order to develop a deep understanding in a subject ar ea, it may be necessary for students to have several different opportunities to practice their new insi ghts and skills in problem solving. Each practice exercise must include the elements critical to developing the higher levels of understanding and must be sufficiently different so that it cannot be reso lved through simple recall. Case studies, in-basket exercises, games, and role-playing scenarios are a few of the complicate d, time-consuming, and often expensive approaches used to bring these simulations to the students.
9.4.4. Time Devoted to Instruction. Although classroom activity may constitute a smaller percentage of the time spent on learning at the higher levels , more total classroom time may be required than at lower levels. That is, actual classroom time may be just 3 or 4 hours of a learning experience (reading, research, writing), but this time may be double thattypically devoted to any single topic in the curriculum. 9.4.4.1. Instructors often raise the level of learning and increase the amount and complexity of outside work only to find they are unsuccessf ul because no additional classroom time was built into the schedule. Classroom time required for a higher level lesson may be two, three, or more times that of a lower-level lesson. 9.4.4.2. A common mistake is to take a sequence of instructional periods and allocate them evenly without regard to the level of learning. One-th ird of the periods may be allocated to knowledge-level lesson, one-third to comprehension, and the remaining third to application level and beyond, or an even distributi on of time may be made among departments competing for scarce time. A good course analysis might reveal that th e time allocation should be more like 10 percent to knowledge lessons, 30 percent to comprehension, and a full 60 percent or more of the available time allotted to the higher levels.
9.4.5. Maturity and Sophistication of Students. Students must take a more active role in their own learning process at the higher levels. Those used to being passive learners may suffer some initial trouble coping with learning at these higher levels . They may mistake the lack of lectures by the instructor as weakness or disinterest. They may need close attention and assistance during this period. In the opposite vein, relatively sophisticated students may resent interference from the instructor as they pursue objectives with a higher degree of motivation. The instructor may become a barrier to learning in their eyes. As instructors, we will have to use our teaching and counseling skills to the fullest to try to map our individual strategy with each student. See Chapter 27 and Chapter 28 for further discussion about dealing with the individual student.
9.4.6. Subject Matter Expertise of Instructors. The level of learning to which we teach is limited by the level of our own understanding. To be effective at the higher levels, we must be genuine experts in our subjects. We can stay with a canned lesson pl an at the lower levels and be quite successful. WeAFMAN36-2236 12 NOVEMBER 2003 131

are doomed to failure if we try that at the higher levels. We cannot apply, analyze, synthesize, and evaluate student learning outcomes unless our own level of understanding permits us to do so.
9.4.7. Special Emphasis Upon Feedback. The highly individualistic nature of higher level learning makes quality feedback to students especially critical. Feedback he re has the same meaning as that described in Chapter 27 of this manual. Evaluation, or the pa ssing of judgment on a student effort, is far less important than constructive feedback at these levels. At some point, however, the instructor has to decide whether learning has occurred. But the primary emphasis should be placed on feedback to foster learning rather than labeling or grading. Good lecturers and even good discussion leaders may be uncomfortable with this feedback role. Bu t the increased use and importance of feedback must be anticipated and planned into the curriculum. It is obvious that the quality of this feedback is related directly to both the subject matter expertise and the teaching and counseling skills of the instructor. The degree to which these skills are available mu st have a significant impact on the planning of individual lessons and the entire course.
9.5. Summary. The five important principles described earlier in this chap ter for teaching and learning at the higher taxonomy levels place heavy demands on bo th instructors and stude nts. Instructors become more like helpers or counselors and become partners in the learning proc ess to a much greater extent than most of us are used to experiencing. While this change in roles may be very uncomfortable to some, most expert instructors find teaching at this level to be mo st rewarding. At these levels of learning, it is not unusual to hear instructors say, "I've learned as much as my students!" 9.5.1. As could be expected, there is a great deal of extra energy and effort expended by both instructors and students in working at the higher levels. Expenditures of energy and effort are very difficult to measure; however, there is no doubt that ther e are increased demands upon both students and instructors. But this increased energy and effort does affect a very tangible asset—time. The time necessary to plan for and conduct higher level lear ning experiences is substantially greater for bothinstructors and students. In addition to the time spent in class by both, there is the need to do considerably more planning and preparation. One-on-o ne conferences, individualized evaluations, and increased emphasis on feedback all place additional demands on time. Time, as a critical factor, must be accounted for by the instructor and the curriculum planner as instruction is developed at the higher levels of the cognitive taxonomy.9.5.2. Instruction at these higher levels becomes increasingly student-centered. Being student-centered shifts much more of the burden for learning from instructors to students. Often, more learning occurs outside of class periods thanwithin. Trial and error, discovery , and creative insight characterize student activity. Students may incorrectly sense a lack of direction or shortcomings in instructor expertise as they discover fewer and fewer answers that are clearly right or wrong. With the patient and skillful help of instructors with great subject matter expertise, these students can be guided to the highest and most satisfy ing levels of learning.132 AFMAN36-2236 12 NOVEMBER 2003

Chapter 10

WRITING AND MEASURING AFFECTIVE OBJECTIVES

10.1. Introduction. All instructors should care about the attitude s of their students. We want our students to have positive feelings about our courses, our schools, and us as instructors. Some courses are designed specifically to affect attitudes but do so only informally , or with no real plan. Whether we plan to or not, much of what we do as instructors affects the attitudes of our students. But, the potential to affect student attitudes about instructors and the way they feel about what is taught is too important to leave to chance. 10.1.1. As we have seen in previous chapters, we can and should plan for measurable learning outcomes. Although we have focused on cognitive or subject matter outcomes to this point, we should not ignore the need to plan for attitude developm ent. Attitude development is complicated and must be approached with caution, but the affective domain s hould still be of great interest to us as instructors. We should realize our potential effect upon st udent attitudes. Further, we should be aware that many of the techniques for writing and measuring c ognitive objectives apply to the affective domain. We may find affective objectives are more difficult to write and the measuring tools are less precise, but paying more systematic attention to th e affective domain will improve our course. 10.1.2. There are at least four elements of writing and measuring objectives for student attitude development that should be of concern to the instructor : clarifying what is meant by the term attitude (or affect), writing attitude development objectives, measuring student attitudes, and designing a curriculum to develop attitudes in a desired direction. This chapter will discuss the first three of these topics (which are of practical concern to the classroom instructor). 10.1.3. The fourth topic, designing a curriculum to develop attitudes in a desired direction, is beyond the scope of this manual. Classr oom instructors are not usually involved with curriculum development at this level. Among the many schools and courses th at successfully develop attitudes are Basic Military Training, Officer Training School, pilot training, survival training, and virtually all safety courses. A person who wants to develop an effect ive curriculum plan for developing student attitudes on a systematic basis would be wise to study the programs of successful schools. 10.1.4. Although most of our objectives are cognitive, often we want to know how our teaching affects student attitudes. In those cases, it is not ne cessary to plan for affective outcomes separate from cognitive outcomes. Much of the affective impact of cognitive lessons is in the way teachers deal with the subject matter. If we teach effectively, student attitudes will probably be positive. Conversely, if we do a poor job, student at titudes might be negative.
10.1.5. There are still the other areas of interest to th e instructor in developing student attitudes without getting into the broad implications of curriculum development. The definition and structure of student attitude development are important to us. Th e way we write and measure attitude objectives for our own classes is something we should understand. We should also care about the ways our daily classroom behavior affects student attitudes.

10.2. A Structure For Attitude Deve lopment—The Affective Taxonomy. A basic structure for attitude development was first presented by Krathwohl and associates in 1956. Their work provides a taxonomy we can use to plan and classify affec tive instructional objectives. As introduced in Chapter 2 , the affective taxonomy is developmental. Attitudes seem to develop through se veral identifiable stages; a per-AFMAN36-2236 12 NOVEMBER 2003 133

son cannot reach a level of attitude development with out going through the level below. The affective taxonomy levels are:
10.2.1. Receiving. The first step in teaching is to get the students to receive—to pay attention or listen to the presentation. Awareness of what is being communicated is the issue at this level. Willing attention is a logical follow-on to awareness, foll owed by selective attention. Selective attention is important because students choose the stimulus they will attend to.
10.2.2. Responding. For learning to occur, students must do more than simply listen to a message. After receiving information, we want them to do something with it. Responding involves some sort of action or response on the part of the student. At first, the student may respond by merely complying. Later, the student may willingly perform an acti on and obtain satisfaction from the act. Responding behavior at a high level is reflected in what we co mmonly refer to as "interests," that is, those activities bringing personal satisfaction.
10.2.3. Valuing. As a person responds, some worth or value may be seen in what is done. Students may accept, prefer, or commit to an object or activ ity because of its perceived worth or value. When students value something, we can also say they "appreciate it."
10.2.4. Organization. At this level, students compare, relate, and synthesize values into their own value system. Conflicts between the new material and the existing value system must be resolved because we cannot hold two strongly opposing values orattitudes at the same time. Hence, this synthesizing step must occur as part of the developm ent of a philosophy of life which takes place in the next level.
10.2.5. Characterization. At the highest affective level, the student incorporates values into a system and that system now becomes characteristic of the student. These values now are the student; they represent a student's way of life or lifestyle. Behaviors resulting from such values are considered typical for a given individual.
NOTE: Figure 10.1. summarizes the affective taxonomy, its levels of learning, and those activities characteristic of each level.134 AFMAN36-2236 12 NOVEMBER 2003

Figure 10.1. Krathwohl's Affective Taxonomy.

10.3. Lesson Planning for Affective Development. As an example of attitude development, let us take the topic of "lesson planning" and trace the building of a value through the various levels of development within the affective taxonomy. When potential instructor s take an instructor training course, a goal is to teach them how to plan a lesson. Lesson planning may be a pretty dry subject to some, so we have to capture our students' attention and fo cus it on lesson planning as best we can. Upon attaining that focus, we have made students aware at the receiv ing level that lesson planning exists. 10.3.1. To further illustrate the point, considerthe basic design for lesson planning introduced in Chapter 6 . For cognitive reasons we can see that a good attention step can help students focus on the subject and get them ready to learn. Affectively, we stimulated the students to tune in and begin receiving. Recall that "receiving" is the lowest level in the affective taxonomy. Remember, the student must begin receiving before the worth of the topic can even be considered. The other lesson parts have important cognitive and affective consequences as well. These consequences are summarized in Table 10.1.AFMAN36-2236 12 NOVEMBER 2003 135

Table 10.1. Cognitive and Affective Consequences.

10.3.2. Awareness is not enough. Our next step is to get the students to act on the information. At first,students plan lessons because they are told to do so—simple compliance at the responding level. Willingness to respond emerges and la ter develops into satisfaction with responding. As students go through lesson planning procedures, some of them will begin to see some worth or value in planning a lesson. A lesson plan helps them clearly state whatis to be accomplished. It is a handy outline they can use in giving a presentation. It helps them rememb er visual aids, stories, and examples. Most students will begin to feel more conf ident with a lesson plan available. Hence, we have obviously arrived at the valuing level of the affective taxonomy. 10.3.3. As experience increases, the value of lesson planning may be assimilated into the students' value system through the process of organization. Th is phase helps the student fit the new value into his or her existing structure of values. 10.3.4. At some point in the future someone may say, "Let's do a lesson on a new topic," and the immediate reply will be, "First, we'll have to devel op a lesson plan." This positive attitude toward lesson planning now seems to be typical or characteristic of the student. Student behavior can then be generally predicted in terms of the value of lesson planning. The characterization level has been achieved. I T E M A B C

Lesson Part Cognitive Consequence Affective Consequence
1 Attention Step Students focus on subject matter and get ready to learn. Students are stimulated to be attentive, to begin receiving, and to consider the worth of the topic.
2 Motivation Students know why they should listen. Students are urged to see the worth of the subject and to respond. They are encouraged to develop an interest in the subject.
3 Overview Students follow more effectively when they see the logical sequence of ideas and methods the lesson will include. Students are oriented when they have the security of knowing where they are going and how they are to get there.

4 Body (Lesson Development) Students acquire knowledge or gain understanding of the subject. When support is credible, students see and develop positive attitudes and appreciation.

5 Summaries Students see relationships between main ideas. The essential ideas are summarized. Students feel confident about lesson content and have a feeling of security and completeness.
6 Remotivation Students realize how the lesson can be put to use. Students see worth in the lesson and leave with a positive feeling.
7 Closure Students recognize the lesson is

over. Students sense a psychological completeness.136 AFMAN36-2236 12 NOVEMBER 2003

10.4. Relationship of Affective Domain to Other Domains. Although it is very convenient to think of the cognitive, psychomotor, and affective domains of learning as being separate, they are not. Educators have treated the domains of learning as separate entities to make it easier to conceptualize them. In reality, learning takes place and behavior is affected in more than one domain at the same time. 10.4.1. Consider, for instance, the Air Force specialt y of pararescue. This job involves highly dangerous situations where a person may parachute or be lowered by cable from a moving helicopter into a combat situation or to the scene of a natural di saster for the purpose of aiding the stranded or wounded. In what domains of learning do the pararescue specialists operate? 10.4.2. Certainly the psychomotor domain is involved: lines must be operated, knots tied, and first aid rendered. Obviously cognitive learning also comes into operation: procedures must be remembered, situations evaluated, and the best courses of action taken. It is not difficult to see that the affective domain is also involved. The mere fact that people would volunteer for such a job tells us something about their feelings about the job and other human beings. They probably operate at the valuing level or even higher. 10.4.3. It is clear, then, that learning often takes pl ace in all three domains at the same time. While we separate the domains to plan better for learning and evaluation, it is often impossible to distinguish between them in real life situations.
10.5. Affective Objectives Can be Specified and Measured. Few military or civilian schools have undertaken the task of specifying affective outcome s, planning and developing lessons to account for affective outcomes, or measuring such outcomes. A lthough little has been done to account formally forchanges in student feelings and attitudes, in many cases there is at least an implied concern for these affective changes. Many schools have demonstrated thei r concern by putting affective goal statements into their curriculum catalogs. These statements, which quite often sound and look good, might be useful as mission statements, but they represent a level of abstraction too high for measurable objective statements. 10.5.1. Consider the following two unmeasurable objectives: Students should be stimulated toward reaching their maximum potential as teachers and to strengthen those professional values necessary for a full career of dedication and service to their country. 10.5.2. These statements are worthwhile goals, but th ey are of little value as objectives because they are not measurable. How do we know when studen ts have been stimulated toward their maximum potential as a teacher? How do we know when airmen have strengthened their professional values? The answer is that we have no effective way of me asuring such broadly stated objectives. That does not mean we should not be concerned about affective outcomes. The problem is that the objectives were not stated so they could be measured. Th ese value objectives do indicate a concern and an acknowledgment of the importance of affect in edu cation, but the writers of these statements simply do not go far enough to specify a measurable affect ive objective. Let's begin looking at how to write or specify measurable affective objectives by looking at their similarity to cognitive objectives.
10.6. Techniques for Specifying and Measuring Affective Objectives. A wide range of techniques for specifying and measuring affective obj ectives is available. What follows will introduce a few of these techniques:
10.6.1. Stating Affective Objectives. Stating affective learning outcomes is not especially difficult given the complexity of the area with which we are dealing. Many of the objectives we have seen orAFMAN36-2236 12 NOVEMBER 2003 137

perhaps have already written reflect conc ern for affect. Consider the example at Figure 10.2. of a lesson objective and purpose statement.
Figure 10.2. Lesson Objectiv e and Purpose Statement.

10.6.1.1. In Figure 10.2. , concern for affect is seen in both the objective and the italicized purpose statement. Not every school requires that purpose st atements be written. It is almost certain, however, that every instructor has some purpose in mind when planning a lesson. Purpose statements answer the question, "Why does the student need to learn this material?" The lesson objective itself deals with what the lesson is about. Purpos e statements, whether written or not, very often reflect the instructor's concern for student feeli ngs and attitudes that should result from the lesson. This instructor wants the students to comprehend and feel that overpopulation is detrimental. Theuse of the word detrimental implies a value judgment about overpopulation. In this case, the purpose statement also reflects a concern for affect.10.6.1.2. A look back at the taxonomy will reveal that "appreciation" is synonymous with "valuing." We have an objective and a purpose statement th at reflect affective concern at a specific level within the affective taxonomy. The writer of this obj ective may not have realized attitudes as well as understanding were under development.
10.6.2. Sampling Changes in Student Feelings. Using another example closer to the classroom, consider the lesson objective and outline of main teaching points at Figure 10.3. Notice that the third point is clearly affective.
Figure 10.3. Lesson Objective and Ou tline of Main Teaching Points.

10.6.2.1. We must now attempt to determine whethe r students do in fact "value classroom examinations as learning experiences" as stated in this objective. Using the same concepts as were used to measure cognitive learning, we proceed to sample student behavior. Using behavioral terms, we construct samples of behavior that will prove the student s do value classroom examinations. Thesamples might be listed as follows: (1) Propose using examination feedback to improve performance. (2) Select use of examinations to improve performance when given the opportunity. (3) Justify the continued use of classroom examinations to improve performance.
10.6.2.2. The idea is that if our students do one or mo re of these three tasks (out of many we could have listed), we will have some evidence students achieved this attitude objective. We can see that the same sampling procedures apply in affective lessons as in cognitive lessons. Of course, the samples we would require as evidence of lear ning might not be the ones listed paragraph 10.6.2.1. We might select others. We might also decide th at three is not enough, or that three is too many. The objective of this lesson is for each student to comprehend that overpopulation is detrimental to world development so that the student may better appreciate the need for population control

.

The objective of this lesson is for each studentto comprehend that classroom examinations play a valuable role in the learning process. Comprehend that examinations aid teachers by.... Comprehend that examinations aid students by.... Value classroom examinations as learning experiences.138 AFMAN36-2236 12 NOVEMBER 2003

10.7. Similarity of Cognitive and Affective Learning Outcomes. If we think for a moment about the schools where we have taught or attended as students, we realize that most schools have specifically stated cognitive objectives. Perhaps some have specified psychomotor objectives as well. Few, however, specify affective objectives. But if we closely exam ine the purpose for most schools, we will discover they are also very interested in whether or not their students are conscientious, empathetic, receptive, interested, willing, participating, dependabl e, and ethical.
10.7.1. What are some of the lessons learned fromworking with the cognitive domain that will help us get started on writing and measuring affective objectives? We will use and build upon the lesson planning concepts illustrated earlier in this manual. We will not have to learn new approaches to lesson planning. Devices used to measure changes in learning are imperfect. Anyone who has attempted to test understanding or rate skilled performance knows the difficulties involved.
10.7.2. The same is true for the affective domai n. In the affective domainwe will often observe behavior and make judgments or inferences about what the behavior means. Many of the assumptions made when working in the cognitive domain will al so apply in the affective domain. For instance, we will sample behavior and assume it is typical of the way students behave. We cannot possibly measure all behaviors, so we will use the concept of sampli ng. Behavior is the primary indicator of what people feel, just as it is the primary indicator of what people know. We cannot look inside the brain to determine how a student feels about something any mo re than we can determine what is understood. 10.7.3. Consider a brief example on using behavior inthe affective domain. Joe is a student in one of our classes. While talking to a nother instructor we might say, "Joe has a bad attitude about this school." Our colleague may respond by asking, "What makes you think so?" The evidence we give for Joe's having a bad attitude will probably sound like th is: "Why do I believe that Joe has a bad attitude? Well, it's pretty obvious because he comes to clas s late (or not at all), doesn't bring his textbook, doesn't take notes, sleeps in class, stares out of th e window, cleans his fingernails in class, talks during lectures, doesn't participate in class disc ussions, or reads magazines during class." 10.7.4. What has happened is we have sampled Joe's be havior and inferred from it that he has a bad attitude about our school. We frequently reach conclu sions about people's attitude s and feelings in this way. How can we possibly know how people feel exce pt through their actions or behavior? We infer how people feel from what they do. We need behavioral indicators of attitude just as we need them in the other domains.
10.8. Interpreting and Modifying Criterion Objectives. The previous examples addressed a level-of-learning objective. The same concern for affectmay be interpreted in even the simplest criterion objective. Consider the following criterion objective: "Without the use of references, correctly identify 200 of the 206 bones of a human skeletal model." 10.8.1. At first we may question whether there is anything at all affective about this objective. Let us examine it more closely, however. It takes a great deal of time and effort to memorize 206 bones, their Latin names, and to be able to point to each on a human skeleton. The mere fact that people would take on this task indicates they are probably at the responding level of the taxonomy. Learning the names of these bones reflects at least compliance a nd possibly an even higher level. The point is that even criterion objectives may reflect implied fee lings of a student toward the learning task. 10.8.2. Minor changes can be made to criterion objec tives to help us better express our affective intent. Consider the following criterion objective thatmight be part of an Air Force school curriculum:AFMAN36-2236 12 NOVEMBER 2003 139

"Upon completion of tasks in the school laboratory, replenish supplies to maintain the stock levels specified in the laboratory handbook." 10.8.3. With minor changes we can attempt to account fo r changes in student attitudes in this straightforward objective. To add this element to the objective, we can add a phrase such as "without supervision." The criterion objective would then read "Upon completion of tasks in the school laboratory, replenish supplies to maintain the stock levels specified in the laboratory handbook without supervision ." 10.8.4. If students meet this objective without supe rvision, no one will watch them, pressure them, or remind them to replenish supplies—they must now do it on their own. If students meet such an objective, they are functioning at the responding le vel of attitude, or perhaps even higher. 10.8.5. Consider another criterion objective: "Upon completion of assigned laboratory tasks, return all tools and equipment to designated storage places." 10.8.6. We might add a different phrase to this ob jective to account for attitude and end up with:

" Without being told to do so and upon completion of assigned labo ratory tasks, return all tools and equipment to designated storage places."
10.8.7. Now the student must do the task without be ing told. Here again, we are at least at the responding level of att itude development. 10.8.8. Any number of changes to criterion objectives can help better express affective objectives. Consider the list of suggested changes in Figure 10.4.
Figure 10.4. Suggested Changes to Affective Objectives.

10.8.9. Notice the last three of these phrases in pa rticular. If students do something on their own time, initiate something on their own, or do something in sp ite of social pressure to do otherwise, they are displaying behavior at the valuing level of the taxonomy or perhaps higher.
10.9. Stating Purely Affective Objectives. The changes described so far are used when we want to "add on" some affective outcomes. But what if the primary purpose of a lesson is affective? If the main purpose of a lesson is affective, we should plan and conduct the lesson accordingly. Sometimes, it might be best to have both a cognitive and an affective objective for the same lesson. In either case, we need to know how to write an objective that describes affective outcomes. Voluntarily takes action to.... Without being told to do.... Without promise of reward.... Without threat of punishment.... Without being reminded.... On student's own time.... In spite of social pressure to do otherwise.... Initiates on his or her own....140 AFMAN36-2236 12 NOVEMBER 2003

10.9.1. If we teach management, we have probably sp ent time ensuring our students know it is important for good interpersonal relations to exist betw een supervisors and subordinates. However, knowing what to do is one thing and being willing to do it is something else. We want to determine whether our students will, in fact, establish good relations between supervisors and subordinates when given the chance. What sort of affective lesson objective could we plan to help express our desired goal? 10.9.2. First, we select the desire d level of learning from the affective taxonomy and state the lesson objective using that level. We might choose the re sponding level and begin with the following lesson objective: "The objective of this lesson is for each student to respond to the desirability of having good interpersonal relations between s upervisors and their subordinates." 10.9.3. As with a cognitive lesson plan, we must outline the main points we want to teach in support of our lesson objective. In this instance, we will se lect a main point at the responding level of learning as indicated by the lesson objective. The first ma in point (MP) might look like the sample in Figure 10.5.
Figure 10.5. Main Point Sample.

10.9.4. This main point also points toward a te aching method—guided discussion. Here we will use the discussion for something other than a cognitive purpose. 10.9.5. As in previous examples, the next step is to write several samples of behavior using behavioral terms. The samples in Figure 10.6. might be appropriate in this situation.
Figure 10.6. Samples of Behavior.

10.9.6. Putting the objective, the first main point, and the samples of behavior together, we end up with the plan in Figure 10.7. The objective of this lesson is for each student to respond to the desirability of having good interpersonal relations between supervisors and their subordinates. MP 1 Respond, during guided discussion, to the importance of effective supervisor-subordinate communications.

1. Present arguments to support the desirability of open communications between supervisors and subordinates.

2. Voluntarily tell personal experiences that illustrate problems that arise from a lack of good supervisor-subordinate communication.

3. Select organizational designs that emphasize interaction between supervisors and subordinates over those that do not.AFMAN36-2236 12 NOVEMBER 2003 141

Figure 10.7. Sample Lesson Plan.

10.9.7. We would of course continue this process fo r subsequent main points and samples. Notice that in this lesson plan, we deal primarily with the affective domain and the major purpose of the lesson is to get students to respond positively to our topic. At the same time, we provide built-in evaluation criteria that tells whether or notwe have met the objective.
10.10. Measuring Changes In Student Attitudes: 10.10.1. Let us look again at the samples of behavior in Figure 10.6. These samples actually form a small attitude assessment instrument—an attitude scal e. The behaviors students exhibit in the samples will tell us something very specific about their at titude toward the task. Such evidence gives us some indication the affective objective was met. In measuring attitudes, we cannot be concerned with hair-splitting measurements. We are primarily conc erned with general tendencies and whether these tendencies are positive or negative. 10.10.2. Because of the broad range of possible behavi ors, we should not be satisfied with a measure of just one sample of behavior by itself. Attitudes are so very complex—we run a big risk of misinterpreting a student's overall attitude if we use only one or two samples of beha vior. Always use several samples in an important attitude measurement to have confidence in the resulting decision.
10.11. Using Published Attitude Scales. When we first discussed measur ing attitudes, we were probably reminded of some sort of attitude scale or test written by a psychologist. Most of us have taken such tests or completed such scales at one time or another. Most libraries have copies of Buro's publication, Mental Measurements Yearbook , which reviews a wide variety of commercially available scales and inventories. Scales can measure attitudes toward suchitems as teachers, education, politics, student ratings, self, study habits, and many more. If we find a scal e in Buro that measures what we are interested in, it may save time in de veloping one of our own.10.11.1. If we are faced with measuring the attitudes of a large group of people, a commercially published attitude scale may be just what we need. Ifwe go back to the example level-of-learning objective just used and its accompanying samples of behavior, we can see that to determine whether or not students can perform each of those samples withouta written test would require close observation during the learning process. If we had a large group of students, we might find it difficult to observe and record the necessary responses accurately and object ively. When we are tasked to measure the attitudes of a large group of people, some sort of written instrument will probably be most accurate, efficient, and reasonable to use. MP 1 Respond, during guided discussion, to the importance of effective supervisor-subordinate communications. a. Present arguments to support the desirability of open communications between supervisors and subordinates. b. Voluntarily tell personal experiences that illustrate problems that arise from a lack of good supervisor-subordinate communication. c. Select organizational desi gns that emphasize interaction between supervisors and subordinates over those that do not.142 AFMAN36-2236 12 NOVEMBER 2003

10.11.2. We might find the following affective criterion objective for these commercially available scales to be of value: "Given the (name of the scale) attitude scale, show a statistically significant gain in (whatever the scale measures) on a posttest as compared to a pretest."
10.12. Making Our Own Attitude Assessment Scale. As useful as the published attitude assessment instruments may be, the one we need may not be availabl e in print. In such cases, we might write our own. Constructing a scale to measure attitudes and feelin gs is not an easy task, however. The complexity of human behavior makes us appreciate the difficulty of accurately measuring changes in attitudes. Whether instructor-made evaluation instruments can accurately and consistently record changes in student attitudes is questionable. But these instruments can be of value. To the instructor using self-made attitude assessment devices, some words of caution: use the results carefully and do not generalize results to a large population. Use the data primarily for your own information. With those reservations in mind, we can approach the process of developing attitude assessment scales.
10.13. Pretest-Posttest Design. Probably the most widely used strategy in affective measurement is the pretest-posttest design. In this design, students' attitude s are measured at one point in time to determine their attitudes before instruction. Students are then e xposed to some sort of intervening learning experience. After the intervening activity, attitudes are again measured and the results compared to those obtained in the pretest. What we want to measure is a change of attitude and its direction (positive or negative). Figure 10.8. illustrates the pretest-posttest design. Many formats are available to use in the pretest-posttest design, such as:
Figure 10.8. Attitude Test—Pretest-Posttest Design.

10.13.1. Word List Format. Suppose we were interested in the attitudes of Air Force middle management personnel toward a new management technique . One way to begin would be to survey these 1. Administer a pretest word list to the managers who had not used the technique before coming to our school. The word list might look like this. Circle 10 of the words listed below that best describe your reaction to management technique "A."

Rational Efficient Helpful Bothersome Inefficient Irrational Stable Effective Traditional Volatile Ineffective Modern Rewarding Hindrance Workable Unworkable

2. Teach the managers the concept and principles behind the new technique and give them the opportunity to apply it successf ully to relevant situations. 3. Re-administer the word list following the course.AFMAN36-2236 12 NOVEMBER 2003 143

managers and ask them, at random, to list words that describe other ma nagement techniques they know. These words would come from techniques with which they have been both successful and unsuccessful. With the words obtained, we could de velop a master list of both positive and negative adjectives such as effective, efficient, and logical. 10.13.1.1. Assume that we then have a group of mana gers in our course who have never used a particular management te chnique. Further assume that we are interested in finding out what happens to the attitudes of these managers as a re sult of attending our school and learning this new technique. Our attitude test could be conducted as in Figure 10.8. 10.13.1.2. The idea is that once the managers ac tually learn and use the new technique in the school setting, their attitudes should change toward the positive. If we are successful, they will likely conclude that the new technique is of real value to them as managers. Therefore, when theword list is re-administered the number of positive words selected should increase. We can construct a criterion objective for this pretest-posttest situation that would look like this: "Given a list of mixed (positive and negative) adjectives used to describe a technique of management, select 10 that include more positive connotations on a posttest as compared to a pretest."
10.13.2. Multiple-Choice Format. A multiple-choice question for measuring the affective domain will have no right or wrong answer. Rather, it will permit students to express their feelings about a topic in a structured way ( Figure 10.9. ).
Figure 10.9. Multiple-Choice Format.

10.13.3. Bipolar Scale Format: 10.13.3.1. A variation of the word list technique, the bipolar scale format does not require the person taking the test to select single words, but instead asks them to rate something along a line between two extremes. This format could also be used to measure attitudes about the new management technique example in paragraph 10.13.1. The major difference from the word list format is that the managers would now be given the opportunity to indicate their feelings at some point along a line between two extremes. For example, in stead of making the adjective "irrational" an option, we might put the word "irrational" at one e nd of a continuum and the word "rational" at the other end. 10.13.3.2. In this example, the respondents would pl ace an "X" or check mark at some point on each line. The various points could be given values that would enable us to calculate total or averI find the subject of (fill in)

a. very interesting

b. somewhat interesting

c. somewhat uninteresting

d. very uninteresting144 AFMAN36-2236 12 NOVEMBER 2003

age scale scores permitting pretest-posttest comparisons. Note that the values of the qualities are purposely mixed with positive and negative terms on both sides, forcing the student to think about the responses.
NOTE: The bipolar scale is an excellent device to us e when your lesson involves a concept or concepts as described in Chapter 8

. 10.14. Measuring Attitude Changes Following the Experience. Sometimes it is impossible to give anattitude test during both a pre- and posttest. In certain situations there is only one chance to measure student attitudes. The bipolar scale just discussed ca n be used as a one-time measure. Another example would be an end-of-course critique. On such critique s, students indicate how they feel about various aspects of the experience. For example, we might give the students a set of instructions and a scale and ask them to rate a series of curriculum topics su ch as methodology, communication skills, or evaluation ( Figure 10.10. ).
Figure 10.10. End-of-Course Critique—General.

10.14.1. Rating Statements Format. We could further break down the major areas into topics to get more specific feedback. For example, under the major curriculum area of communication skills we might ask students to rate statements in the follow ing two examples using the same five-point scale (A, B, C, D, E) ( Figure 10.11. ). I believe that the performance of this school in the tasks described below has been:

A. Outstanding B. Excellent C. Good D. Fair E. UnsatisfactoryAFMAN36-2236 12 NOVEMBER 2003 145

Figure 10.11. End-of-Course Critique—Specific.

10.14.1.1. Most end-of-course critiques obtain responses to questions like those listed in Figure 10.10. and Figure 10.11. , but do not use results to determine whether an attitude objective has been reached. We could also write a school crite rion objective that would set a goal for how well we are meeting student needs ( Figure 10.12. ).
Figure 10.12. Sample School Criterion Objective.

10.14.1.2. In this instance, we have decided that if the students at our school rate 80 percent of the areas we ask about as average or higher, we are satisfied our students have positive feelings about our school. The degree to which the curriculum:

Prepared me to organize and prepare lessons that will ensure clarity and effectiveness in communication.

A. Outstanding B. Excellent C. Good D. Fair E. Unsatisfactory

Prepared me to listen more effectively to others.

A. Outstanding B. Excellent C. Good D. Fair E. Unsatisfactory

Given a series of curriculum topics and a related rating scale, students will rate at least 80 percent of the topic areas as average or higher.146 AFMAN36-2236 12 NOVEMBER 2003

10.14.2. Likert-Type Scale. Another approach to posttest only or pre-post evaluation is the Likert-type scale. Students respond to clearly favorable or unfavorable statements by selecting one of the following responses: Strongly agree, agree, no opi nion, disagree, or strongly disagree. This approach is useful for measuring attitudes toward concepts or principles as described in Chapter 8 . The Likert-type scale is relatively easy to quantify by assi gning numeric values to each point on the scale. For example, we might assign a 5, 4, 3, 2, or 1 respectiv ely from the strongly agree to strongly disagree end of the scale.
10.14.3. Either-Or Format. This simple format gives the student a choice of expressing an attitude by selecting one alternative over another ( Figure 10.13. ).
Figure 10.13. Either-Or Format.

10.14.4. Open-Ended Question Format. Sometimes we have to permit our respondents to express themselves in a somewhat less structured way th an those already mentioned. The open-ended question can help us in this regard. This format structur es what is asked, but allows freedom of response ( Figure 10.14. ). The major drawback to this format is that the responses must be read and interpreted, making it very difficult to code and quantify data.
Figure 10.14. Open-Ended Question Format.

10.15. Dealing With the Higher Affective Levels. Written scales and tests like these are useful when measuring the lower affective levels, especially in changes in attitudes over short periods of time. Short courses and brief exposure to material can be expected to change attitudes very little and only at the lower affective levels (receiving, responding, and [possibly] valuing). 10.15.1. Changes at the higher affective levels (val uing, organization, and characterization) require much longer exposures, a block of instruction or an entire course rather than a lesson, and are more difficult to measure. We cannot be sure a particul ar attitude characterizes a person when our only measure or index is a paper-and-pencil test or scale. It is very difficult to measure the higher affective levels at all, much less withpaper-and-pencil tests.10.15.2. The methods for measuring higher affective levels are not objective or precise, but it is possible to measure the higher levels. The best way to get evidence of this level of attitude is to place students in specific situations for a long period of time and then observe their behavior for indicators of If I had it to do all over again, I (would) (would not) have taken this course.

If I were teaching this class, I would....

If I caught a student cheating, I would....AFMAN36-2236 12 NOVEMBER 2003 147

attitude. For example, if we are interested in dete rmining if some value or attitude "characterizes" our students, we might have them role-play in a realis tic situation. Observing their role-playing can tell something about student attitudes. 10.15.3. Similar situations might include case studies, games, simulations, and class projects. Each of these situations provides an opportunity to observ e a student under realistic conditions. Short experiences that do not get students to act out or somehow show their real attitude are of no particular value at the higher attitude levels.

10.16. Which Affective Measure Should Be Used? We may wonder which techniques for measuring attitude change should be used in our own teaching. Unfortunately, there is no easy answer to the question, just as there is no easy answer to which m easurement techniques we should use to measure knowledge or skills. Many methods and te chniques adequately measure attitude, just as many methods and techniques measure other learning. Who decides whet her to use true-false, matching, multiple-choice, or essay items at a particul ar school? The faculty members, curriculum advisor, or commandant? The same person might also make similar decisions with regard to attitude measures. Someone must decide which techniques would be most a ppropriate and practical for the particular situation.
10.17. Summary. Writing and measuring affective objectives is an important but difficult task. In many ways, we have more difficulty specifying and measuri ng attitudes than we do for any other type of learning. We are forced to accept indicators of behavior, which is second-hand evid ence of student attitudes. Our measurement devices are often inconsistent and occasionally marred by error. But we do have attitudinal objectives, and our difficulties in writing a nd measuring them should not keep us from making attitude development a critical part of our courses.148 AFMAN36-2236 12 NOVEMBER 2003

Chapter 11

USING QUESTIONS FOR LEARNING

11.1. Introduction. Questions are so much a part of teaching that they are often taken for granted. Many professionals, such as lawyers and doctors, are taught how to ask questions, but few instructors receive formal training in questioning skills. This lack of training, coupled with the average instructor's lack of planning for questions, often results in ineffective or inefficient use of questions. But it doesn't have to be that way. 11.1.1. Why should we care about questions? Because the effective use of questions may result in more student learning than any other single techni que used by instructors. A considerable amount of classroom communication is in the form of questions ; they play a key role in most teaching methods. Thus, it is important for us to learn to use questions correctly. 11.1.2. In this chapter we will begin by comparing questions at the knowledge and comprehension levels to show that the level of learning is alwa ys the starting point in formulating questions. Having determined the level of learning and organizational pattern, we will see that questions are categorized by purpose depending on our objective. We also see that questions are categorized by direction to ensure all students are involved in the process and reach the appropriate leve l of learning. Finally, we will discuss some common problems associated with the use of questions and some techniques that can be effectively used to overcome these problems.
11.2. The Starting Point. In order to understand that the level oflearning expressed in the lesson objective is the starting point in designing questions, let us compare the first two levels of learning in Bloom's taxonomy. At the knowledge level, the goal for our stude nts is to recall specific information. The process of critical thinking begins with data or facts; ther efore, recall questions are a necessary beginning. Recall questions help guide the students to the higher intellectual pr ocesses of thinking. 11.2.1. During the lesson, we want our students to re member factual information and to repeat what they have learned. What kinds of questions should we ask during the class session to produce this behavior? Knowledge-level questions are usually characterized by such words as who, what, when, where, and which. For example, "Which presidents of the US had military service before their election?" Students either know the answer to this question or they don't. 11.2.2. There are many legitimate reasons for asking knowledge-level questions. In a demonstration-performance lesson you may want to determine if the students have been able to keep up with your explanation. In an informal lecture you may want to determine if the students have the background information necessary for the class to proceed. Perhaps you want to determine if the students can recall basic principles and generalizati ons before you begin more indepth instruction.11.2.3. There is a danger in asking only these kinds of low-level questions. Simply because your students can parrot back what you said in class or wh at they have read does not necessarily mean they understand the material. If we ask solely knowledge-level questions, we run the risk of slighting other intellectual processes of our students. If we want our students to engage in more than just memory work, then more work in formulating questions is required. 11.2.4. At the comprehension level of learning, the emphasis is on understanding rather than mere recall. We want our students to grasp concepts, to explain similarities and differences, and to inferAFMAN36-2236 12 NOVEMBER 2003 149

cause-and-effect relationships. To achieve these re sults we should ask open-ended questions that are thought provoking and require more mental activity than simply remembering facts. For instance, questions containing the word "how" allow the students to compare and contrast; "why" questions encourage the students to question the causes of even ts or actions; "what if..." questions prompt the students to predict. For example, "With all the empha sis put on safety in the Air Force, why do accidents continue to be such a problem?" Notice thatstudents must expend greater effort to answer this question than they did to answer the simple reca ll question about which pr esidents had military service. 11.2.5. In many Air Force classes, instructors must bring their students beyond the comprehension level to a position where they can use what they have learned to solve Air Force problems. Most of what we say about questioning at the comprehensi on level also holds true for the higher levels of learning. Even though the remainder of this chapter addresses the comprehension level, it also applies to the higher levels.
11.3. Determine Purpose. Questions serve a variety of purposes. A good question posed by the instructor can command the students' immediate attention. Properly planned and aske d questions will stimulate student thinking. Questions help instructors see how effective their teaching has been. 11.3.1. One of the most important purposes of questioning is to develop the subject of the lesson. As an example, in a guided discussion or case study, the subject would be developed through questions that foster student participation. In a teaching interview, development would occur through questions directed toward the expert guest. 11.3.2. Regardless of the method, if your purpose is to start participants communicating, plan a leadoff question that will generate discussion. Your leadoff question should stimulate thinking and be phrased so that participants are aware of the main point. For instance, a guided discussion on the topic of time management might have as a main point, "Effective management begins with proper time management." The discussion might be initiated w ith a leadoff question such as, "Why is there a need for all of us to manage our time properly?" 11.3.3. After you have initiated the lesson with a le adoff question, your purpose is to guide the direction of the discussion. Plan followup questions that are designed to guide the lesson progression by supplying subideas. Your followup questions should promote reasoning. In the same discussion on time management, an appropriate followup questio n might be, "How do we mismanage our time?" Notice how the instructor's question has provide d a subidea—mismanagement of time. The students will now consider this aspect of the issue and relate it to the main point. 11.3.4. Once your lesson is in progress, you may notice that one of the students is starting to get sidetracked. Ask a spontaneous question to lead the straying student back to the main point. A spontaneous question can control the content of the less on by ensuring that responses are pertinent. A spontaneous question can be used to seek clarificat ion. For instance, in a t eaching interview, if the guest gives an unclear response, prompt the guestby asking for an explanation or an expansion through such questions as, "Would you restate your answer in another way?" or "What else can you add?" or "Are there other reasons?" 11.3.5. Spontaneous questions are also used to seek support when a statement is made that appears to be shallow, lacks focus, or merely parrots informat ion from a text or previous lecture. You need examples or statistics as support for the assertions ju st made in order to promote understanding. Hinting or150 AFMAN36-2236 12 NOVEMBER 2003

further prompting, from the instructor, may be nece ssary before the lesson can continue. For example, in an informal lecture on the strategy of deterre nce, you might ask, "How would a countervalue strategy affect the free world order of battle?" If the students cannot reply, ask for a definition of countervalue at the knowledge level. With that answer in mind, you could then provide a hint such as, "Opposed to a countervalue strategy is a counterfo rce strategy. How would the targets differ?"
11.4. Questions Categorized By Direction. After you have determined th e purpose of a particular question, you are ready to ask the question. Your first decision should be your target—who do you want to respond?
11.4.1. Directed to Entire Group. A question projected overhead is directed to the entire group rather than to a specific individual and is one to which you expect an answer. Overhead questions work well in stimulating thinking and getting participation started. Leadoff questions are often projected overhead since they are usually addressed to the class rather than one person. Questions of this kind alert all members of the class and keep them following the discussion.11.4.1.1. Instructors frequently ask questions for which no answer is expected. These are rhetorical questions because they are either answered by the instructor or are left unanswered. Why would an instructor ask questions for which no answer is expected? Such questions can be found throughout a lesson, from the attention step to the conclusion. "Have you ever wondered why...?" can be an arresting opening. "What could Captain Jones have done to prevent...?" could motivatestudents by putting them mentally into a specific situation. "What do you think the Inspector General's recommendation might have been?" could lead into an overview of the lesson.
11.4.1.2. As the instructor nears the end of a less on, there are also many ways rhetorical questions can be useful. The summary often begins with so mething like "Well, what have we learned about NCO responsibilities today?" In an interim summ ary it is very natural to say "What have we learned so far?" As we remotivate students we mi ght say, "If you faced the same situation as Captain Jones, do you see where this approach would have worked?" Even in the concluding statement we might find such questions as, "What will you do the next time a traffic fatality is reported in your squadron?" or "Based on what we have observed today, what would you recommend to the new wing commander?"
11.4.2. Directed to Individuals. Many times during a class session, the instructor will find it necessary to direct a question to a speci fic individual. A direct question is addressed to a particular person to elicit involvement, seek an opinion, or draw out support. For example, "Sergeant Lopez, you've taught for a while. How have you handled individual differences in your classroom?" 11.4.2.1. When a student asks you a question, you may prefer not to answer it because you want the other students to continue to think and disc uss. Or you might ask a reverse question, one that directs the question back to the student who asked it. Lieutenant Michaels might say to you, "I really can't see your point. Why is it necessary to have authoritarian leaders?" You could reverse the question to him or her by saying, "Think about it for a minute, Lieutenant Michaels. Why do you think it would be necessary to have an author itarian leader, say, in a combat situation?" 11.4.2.2. Similarly, if a student asks you a question and if, again, you prefer not to answer it, you may choose to redirect the questi on to another student. This relay question will help keep the discussion among the students. Ms Cannon might ask, "But how does an authoritarian leader fit into the modern Air Force?" Your reply could be, "Your question is one we ought to consider. AirmanAFMAN36-2236 12 NOVEMBER 2003 151

Cook, how would an authoritarian leader fit into today's Air Force?" A relay question, then, is one in which student A asks a question of the instruct or and the instructor passes it on to student B.
11.5. Effective Questioning Techniques. Now that we have discussed the factors involved in planning and selecting comprehension-level questions, the real challenge is in asking the questions. Even the right type of question can be carelessly asked. Let's assu me you have one overall problem at this time in your class—no one is saying anything, or they are saying th e wrong thing! What are some of the things you can do to avoid this situation?
11.5.1. Avoid Do-Nothing Questions. Your first course of action can be to avoid questions that do nothing to promote thought and discussion. If not caref ully planned, questions can be too simple or too complex. We should avoid the kinds of questions that actually limit the discussion. Types of do-nothing questions are:
11.5.1.1. Dead-End Questions. A question that requires no more than a yes-or-no response is really a dead-end question. If you asked your student s, "Is Colonel Doaks an example of a democratic leader?" as a leadoff question, what would they discuss? This kind of question does not promote discussion; it encourages guessing and can be a waste of time. If you find that you must ask a yes-or-no question, be sure to follow it up by asking howor why to get the students to explain their answers and have something to discuss.
11.5.1.2. Foggy Questions. As the name implies, foggy questions are unclear or nebulous, usually because the instructor has not thought about the desired answer. Foggy questions are often ambiguous, such as "What happened in Clevela nd in 1873?" Students will not understand what is wanted or what the teacher is suggesting. Indefin ite questions are also unclear because they are not limited. "How does a manager and a staff member differ?" is a foggy question because the students have not been provided any focus.
11.5.1.3. Multiple Questions. Students become confused when you ask double- and triple-barreled questions, such as "What are some of the Federal laws passed by C ongress in support of education, and what have they accomplished, if a nything?" This double-barreled question is confusing because the students will not know which part the teacher wants answered first. As a result, they just sit there with puzzled looks on their faces.
11.5.1.4. Catch Questions. Catch questions are where you already know the answer and imply it in the question. Some kinds of leading questions fall in this category, such as "Now, this is the first step, isn't it?" Because the question implies the expected answer, it prevents the students from reaching their own conclusions. Ofte n, they elicit no more than a yes-or-no answer or a nod of the head, and therefore genera te very little discussion.
11.5.1.5. Loaded Questions. "Have you quit pressuring your subordi nates?" is a loaded question because no matter what the student says, it coul d be wrong. Loaded questions should be avoided because they reflect the bias of the teacher.
11.5.2. Avoid Stifling the Discussion. If you find that despite the nature of your questions the students still are not talking, perhaps you are somehow hampering the flow of discussion. You can avoid stifling the discussion by allowing enough time for students to think and respond. Waiting for an answer is perhaps the most difficult task for an in experienced instructor who is intimidated or threatened by moments of silence. Too many rapid-fire questions will not allow for thought. You may be encouraging short responses and, therefore, little discussion. Patience is necessary.152 AFMAN36-2236 12 NOVEMBER 2003

11.5.2.1. Allowing enough time is especially critical after the leadoff question. You might indicate to your students that you do not want an immedia te answer by saying something like "I want you to think about this for a minute before answering," and then ask the question. This tactic tends to discourage aggressive, hand-waving students and gives the quieter ones a chance to think and then respond.11.5.2.2. To compensate for students who may not be responding, impatient instructors sometimes feel tempted to provide the information themselves. Avoid this habit—don't answer your own questions. This habit can be very annoying because it increases the amount of teacher talk and minimizes student participatio n time. Students will learn quickly that it is not necessary for them to prepare for class, because "the teacheranswers all the questions anyway." Also, try to allow more than one student response per question. Don't rush the responses; give the students a chance to explain. Rushing responses inhibits students and tends to stifle the discussion.
11.5.3. Be Clear and Concise. Energy can be wasted when we persist in trying to answer vague and meaningless questions. How often have you sat in cla ss and listened to the instructor ask a question for which your response was "Huh?" The wording of your questions is very important, especially to avoid smart-aleck remarks. As an example, to the question, "When do you use Boyle's law?" the answer might be, "Never. I don't even know the man." Also, if you rephrase or ask questions several times, the students may become confused and unable to discuss your material.
11.5.4. Show Concern for Individual Differences. The last technique for asking effective questions pertains more to your presence in the classroom and your acceptance of the students. If there is little discussion, perhaps it is because you are not showi ng concern for your students. Rather than allowing one student to dominate the conversation, encourage a ll students to participate. If you make it a practice to call on nonvolunteers, everyone soon gets the id ea they are responsible for participating. If you encounter students who are quiet and do not say muc h, be tactful in drawing them out. Probably the least effective approach is surprising the student with, "What do you think about this point, John?" 11.5.4.1. Often, simple eye contact will encourage a response, rather than using direct by-name confrontation. The reasons student s are quiet and do not say much vary; the student may not be prepared, may feel ill at ease in the group, or ma y simply be thoughtful or thoughtless. If a topic is new, challenging, or complex, some students may ne ed more time than others to get into it. 11.5.4.2. When considering individual differences, be sure you don't embarrass your students. If someone has been inattentive, alert that pers on by calling his or her name and choose a question you think that person can answer, or start with a recall question. Do not embarrass an inattentive student by asking a difficult question. Make all the students feel they have a share in the class. 11.5.4.3. Be accepting of your students and their responses. Student answers should be genuinely sought. Your body language, facial expressions, and tone of voice all play a part in encouraging ordiscouraging students to participate. Be aware that mannerisms as well as words can deter participation. Also, be careful not to cut someone off too quickly. Even though an answer may sound inappropriate, if the student is given a chance to explain, part of the answer may be correct. Even if the answers are not correct, they may provide va luable clues to student difficulties and provide a basis for reteaching. Criticism from you at this poi nt might cause withdrawal from participation.When students ask questions that are not pertinent, tactfully defer the answer until later in the lesson or arrange to discuss the issue later.AFMAN36-2236 12 NOVEMBER 2003 153

11.5.4.4. If you adopt these suggestions, your st udents are likely to realize you are concerned about them and will be more willing and able to discuss the material.
11.6. Summary. Questioning is an important means of producing mental activity in students. It is possible to reach a specific level of learning by designing questions to elicit the responses necessary for support. Learning at the comprehension level and beyond will require more than responses to who, what, where, which, and when questions. Determining the purpos e of a question will also help to reach a given level of learning. 11.6.1. The leadoff question initiates the discussi on, the followup question guides discussion and the spontaneous question controls and supports the disc ussion. Questions can be directed to the entire group or to specific individuals, depending on whether you are trying to motivate the students, get them involved, or keep the discussion going. Planning questions to include everyone will help ensure that all students reach the object ive. Promote a lively discussion by avoiding questions that do not stimulate thought. Word the questions so students unde rstand what is being asked, and be considerate of the individuals in the group. 11.6.2. Questions are used for many purposes, but they should always develop the lesson's central theme or thought. If the objectives of the lesson ar e clearly determined in advance, instructors will find it much easier to handle questions and keep the discussion moving in the planned direction.154 AFMAN36-2236 12 NOVEMBER 2003

Chapter 12

SURVEY OF TEACHING METHODS

12.1. Introduction. Which is more important—what you teach or the way you teach? This is like asking which is more important--the payloa d or the delivery vehicle? Obviously, both are critical. Good objectives are wasted on poor teaching methods. Good teac hing methods aid learning, but have little value if course objectives are poor. Most of the previous chap ters have been concerned with design. This chapter, and several that follow, discusses the many ways to deliver instruction—the teaching methods. 12.1.1. After determining the lesson objective, the in structor should choose a method of instruction, not in terms of instructor activities, but in terms of the students' activities as a learner. In making this decision, the instructor considers the ways people learn—by doing, discussing, listening, observing, and participating. The instructor's role is to sele ct an organized set of activities that will result in meaningful, learning experiences for the students. 12.1.2. This chapter is a survey of many popular teaching methods. Although the descriptions are short, there should be enough detail to identify t hose methods that deserve further study. Six methods of teaching are described in considerable detail in subsequent chapters. Thes e are lecture, guided discussion, teaching interview, case study, demonstrat ion-performance, and experiential methods. They have been selected for expanded treatment because of their applicability to Air Force classroom instruction. 12.1.3. For the sake of clarity, the individual methods described in this chapter have been grouped into broad major categories—pres entational methods, student verbalinteraction methods, and application methods. Because no one particular method is su itable for all teaching situations, examples of each category are included. If, for example, students are to gain skill in performing a certain task, one of their activities should be to practice performi ng the task. If the desired outcome is knowledge, students should probably observe and listen so they can relate what is seen and heard to their own experiences. If students must learn to apply a principle, the instructor might ask them to solve a problem or perform some task requiring an application of that principle.

12.2. Presentational Methods. Presentational methods provide situations in which the skill or material to be learned is in some way presented to or demons trated for the learner. Insome presentational methods there is little, if any, activity or interaction require d of students other than their attention and desire to learn. In other instances, there is considerable stude nt activity involved. What distinguishes these methods from the other categories is that students begin the le arning experience with little or no previous exposure to the material or skills to be learned.
12.2.1. Lecture Method. The teaching lecture is a formal or informal presentation of information, concepts, or principles by a single individual. 12.2.1.1. The formal lecture is usually given to large groups of people (more than 100) with no active participation by the students. The l earning experience is essentially passive. 12.2.1.2. In the informal lecture, the size of the group is usually smaller than the formal lecture and student participation develops when the instru ctor questions the students or they question the instructor on points presented.AFMAN36-2236 12 NOVEMBER 2003 155

12.2.1.3. A briefing is a formal or informal presentation in which a variety of significant facts are presented as concisely as possible. The briefing is rarely concerned with material beyond the knowledge level and is almost always accompanied by visual representation of the material in the form of charts, graphs, slides, and other aids. Strictly speaking, the briefing is not a teaching method, but is sometimes used in school situations. 12.2.1.4. A guest lecture is a presentation by a person other than the instructor who is usually an expert. It is used to give variety to the class pe riod or to supply information in an area where the instructor is not an expert.
12.2.2. Indirect Methods of Discourse. Indirect discourse involves ve rbal interaction among two or more persons, which is seen and heard by students. Some examples include:
12.2.2.1. Dialogue. A dialogue is the interaction between tw o or more persons, one of whom may be the instructor, generally to present sharpl y opposing points of view for students. The dialogue is often highly structured toward preplanned goals and may take the form of questions and answers between the participants.
12.2.2.2. Teaching Interview. In a teaching interview, the instructor questions a visiting expert and follows a highly structured plan that leads to educational objectives. The advantage of the teaching interview over the guest lecture is that th e instructor controls the expert's presentation. The expert normally requires little or no adva nce preparation, but responds extemporaneously from general experience. When a question-and-answ er period follows the interview, students can interact with the expert.
12.2.2.3. Panel. A panel is a structured or unstructured discussion between two or more experts (generally excluding the regular instructor) presented in a variety of ways, such as constructivearguments followed by debate, response to questions from the instructor or the students, a preplanned agenda, a fixed or a random order of speakers, or free discussion.
12.2.2.4. Skits, Mini-Plays, and Other Dramatizations. These are often effective means of introducing variety into instruction and/or reachi ng objectives directly or indirectly. A subdivision of dramatization is role-playing by an instructor to point out good or bad examples. Role-playing by the instructor differs from role-playing by students—a simulation method.
12.2.3. Demonstration-Performance Method. The demonstration-performance is the presentation or portrayal of a sequence of events to show a pr ocedure, technique, or ope ration, frequently combining oral explanation with the operation or handling of systems, equipment, or material. This method is the most commonly used small group learning experience in a classroom or laboratory (which requires significant instructor intervention) to develop learner skills in the operation of equipment or the acquisition of mental skills. 12.2.3.1. Coaching is an intensive learning experience for an individual or for small groups, characterized by significant student involvement an d immediate instructor feedback. A videotape of student performance is an excellent teaching aid when supplemented by an instructor's analysis and critique. This technique is particul arly effective in instructor training. 12.2.3.2. Tutoring is an informal, student-centered activity generally involving instructor and learner in a one-to-one relationship. It is often us ed for remedial reasons, for test taking, for students to get ahead, or for learners with special needs.156 AFMAN36-2236 12 NOVEMBER 2003

12.2.4. Reading. Reading is the assignment to a student of printed text, which includes books, periodicals, microforms, manuals, instructi ons, and handouts (instructor-produced).
12.2.5. Self-Paced Methods. Self-paced instruction is a learning program organized so students are allowed to move through it at their own pace under the guidance of an instructor. Some typical applications are:
12.2.5.1. Programmed Instruction. This method of instruction usually includes a carefully planned sequence of small units of instruction that require the learner to respond to cues and receive immediate feedback. Various media (books , teaching machines, and computers) are used to deliver the programmed in struction to the learner.
12.2.5.2. Modular Instruction. Modular instruction are prepackag ed units of instruction that typically contain a clear statement of objectives and all necessary learning resources to permit the learner to achieve these objectives. A module can be a complete unit or part of a course.
12.2.5.3. Computer-Assisted Instruction. This learning experience uses a computer as the vehicle for interaction between the learner and the planned course of instruction.
12.2.5.4. Mediated Instruction. This type of instruction uses such devices as slides, films, tapes, and cassettes to present the planned cour se of instruction to the learner.

12.3. Student Verbal Interaction Methods. As the designation implies, verbal interaction methods present situations in which students interact verbally with an instructor, group leader, or with each other. Learning is enhanced as students deal with the ma terial as a group. These me thods presuppose a certain amount of previous preparation by the students.
12.3.1. Questioning Method. Questioning as a method is used to emphasize a point, stimulate thinking, keep students alert, check understanding, review material, and seek clarification (see Chapter 11 ). Examples of this method are:
12.3.1.1. Socratic Method. While rarely seen in its pure form, instruction by asking students questions is a method as old as ancient Greece a nd as modern as a great book's course. The method may resemble a guided discussion, but the goal is of ten to obtain specific an swers to specific questions (reiteration) and not to stimulate discussi on. An instructor may use the method for "trapping" students into inconsistencies in logic which sharpe n their thinking skills. Law professors often use the method for "interrogating" specific students usi ng a series of questions as they might be used in a court of law.
12.3.1.2. Student Query. "Students asking questions" is often used in combination with other methods such as the lecture, panel discussion, or te aching interview. It could also be used by itself, either on a one-to-one basis in tutoring or coaching or as part of small or large groups. The method is student controlled, although the responder can also control the session to a certain extent if he or she is skillful enough. Students' questions may of ten be a measure of how well they understand a particular matter; that is, they know enough to ask the right questions.
12.3.2. Nondirected Discussion Method. Nondirected discussion is a group interactive process in which task or objective-related information and e xperiences are evoked from the student. The instructor normally plays a very limited or passive role. Some examples are: 12.3.2.1. In its original form, the peer-controlle d seminar is a group of highly qualified peers (such as a doctoral-level faculty) who meet periodica lly for the exchange of ideas, usually in theAFMAN36-2236 12 NOVEMBER 2003 157

form of prepared papers with discussion or que stions following. The research seminar resembles a peer-controlled seminar when the instructor allo ws qualified students to lead the discussion with the instructor providing proper supervision. In pr ofessional military education, a peer often acts as a facilitator to lead discussions or conduct work shops. When this process is used, the instructor should provide a statement of the educational objectives and a suggested discussion guide. It should also require some tangible evidence of the discussion results. 12.3.2.2. Akin to the "bull session" or "war story" hour, free discussion can be a valuable adjunct to participatory manageme nt or brainstorming. But, by its verynature, it seldom supports measurable objectives.
12.3.3. Guided Discussion Method. The guided discussion is an instructor-controlled, interactive process of sharing information and experiences related to achieving an educational objective. The difference between nondirected and guided discussion is the instructor's active involvement in asking questions and summarizing the concepts and principles learned. The instructor interacts with the group as a whole through questions, but tries not to dominate the discussion. Students are encouraged to learn about a subject by actively sharing ideas , knowledge, and opinions. The flow of communication is a transaction among all the students rather than question and response between individual students and the instructor. The method employs the ge neral-to-specific presentation to help students form generalizations.
12.4. Application Methods. Application methods provide learners with opportunities to apply previously learned material in situations calling for the practical use of the materi al. Some application methods require students to relate material already learne d to new experiences and mentally recognize how the material applies; that is, to transfer concepts to new situations. Other applicat ion methods require students to apply previously learned materials to new situations for the purpose of making decisions or solving problems.
12.4.1. Individual Projects. In all practical exercises, students interact with things, data, or persons, as necessary, to attain training objectives. Student projects usually take place outside the classroom setting, but not necessarily. An important aspect is instructor feedback, which is given periodically as needed. Some examples of the uses of this method are research papers, staff instruments, theses, dissertations, construction or assembly of equipment or models, creative writing, and making of graphics, pictures, or displays.
12.4.2. Field Trips. A field trip is an out-of-classroom experi ence where students interact with persons, locations, and materials or equipment to attain instructional objectives. An important aspect of the field trip is the student's encounter with real settings.
12.4.3. Case Studies. The case study is a learning experience in which students encounter a real-life situation in order to achieve some educational obj ective. By studying realistic cases in the classroom, students develop new insights into the solution of specific on-the-job problems and also acquire knowledge of the latest concepts and principles used in problem solving. Case studies designed to reach the application, analysis, synthesis, and/or evaluation levels are within the scope of the term Application Method. However, case studies designed to reach only the comprehension level may be defined better as a Student Verbal Interactive Method. The complexity of the case, the level of the objective, and how the case is conducted will have a ma jor impact on whether it is one or the other.158 AFMAN36-2236 12 NOVEMBER 2003

12.4.4. Experiential Learning. The experiential method of learni ng focuses on the students participating in structured learning activities that focus on a specific learning objective. Ideally, the activity has a direct real-world relevancy. The following are some of the types of activities considered experiential learning:
12.4.4.1. Simulations. Simulations are low-risk, educational experiences that substitute for a real-life situation. Simulations may involve individuals, groups, or whole units. They may preempt normal classroom time and they are especi ally effective as capstone methods following a block or several blocks of instruction. Elaborate versions may require special equipment, simulation areas of various sizes and configur ations, and specially trained staff.
12.4.4.2. Role Playing. In this activity, students project themselves into simulated interpersonal situations and act out the parts of the persons and situations assigned by the instructor. Role playing is generally limited to practicing the skills i nvolved in interpersonal relations, such as counseling, interviewing, and conference leadership.
12.4.4.3. In-Basket Exercises. These exercises are used in random order to simulate a series of matters or decisions a leader might actually enc ounter. Students are confronted with a rush situation, limited information, and a list of action-demanding items that actually might be found in an in-basket. After sorting out priorities, student s dispose of matters by delegating, replying by mail, setting up meetings, delaying action, and deciding who should do some of the urgent actions.
12.4.4.4. Organizational or Management Games. Students manipulate an organization or some component part to produce certa in outcomes in these games. Various degrees of competition between teams of students may be built into these exercises. If they entail any degree of complexity, organizational games almost always employ a computer to carry out the simulation.
12.4.4.5. Hardware Simulation. For hardware simulation, students use trainers that resemble, to some degree, the equipment to be used on the job. (Aflight simulator, for instance, has some characteristics of an airplane.) Such devices are su bstituted when using the actual equipment is too costly or otherwise impractical.
12.4.4.6. Frame Games. An educational game designed to inco rporate elements of conflict along with rules for the control of moves and game ter mination that uses changeable content to reach a specific lesson objective is a frame game. These games allow the instructor to reach cognitive, affective, and (at times) psychomotor objectives. Frame games provide the instructor an excellent way to motivate students, get them to interact with each other, and to teach them at the same time. They also provide a way for the instructor to test students in a nonthreatening manner. Frame games can include established games such as monopoly, tic-tac-toe, Jeopardy, crossword puzzles, word hides, and bingo, or they can use many new games available be yond the classic games.
12.5. Summary. This chapter has provided a brief overvie w of several teaching methods grouped under the broad categories of presentational, student verbal interaction, and application. Most familiar methods have been identified with enough deta il to determine whether further study might be worthwhile. Six individual methods will be dealt with in detail in the chapters that follow. A pr ocess for selecting teaching methods appropriate to gi ven situations from those described in this chapter is the subject of Chapter 19

.AFMAN36-2236 12 NOVEMBER 2003 159

Chapter 13

THE TEACHING LECTURE METHOD

13.1. Introduction. Traditionally, lecturing has been the most popular teaching method in the military. We sometimes lecture to large groups of students in auditoriums for extended periods. Other times we explain a subject or a process briefly and informally in small groups before pr oceeding with some other activity or method—a kind of mini-lecture.
13.2. Types of Lectures. A teaching lecture is a formal or info rmal presentation of information, concepts, or principles by a single individual.
13.2.1. Formal Lecture. Normally, communication in a formal lecture is one-sided—from the instructor with no verbal participation by students. Because it is impersonal, a formal lecture is usually given to a large audience. Even though structured and often rehearsed, the formal lecture should have a natural and conversational deliver y style without overuse of notes.
13.2.2. Informal Lecture. The audience for an informal lecture is usually smaller than for a formal one. Considerable verbal interaction between instructor and student is often possible in the form of both questions and discussion. The delivery style is even more conversational, with students often addressed directly by name.
13.3. Other Types of Oral Presentations. Two other speaking situations—briefing and speech—are common in the Air Force yet differ from the teaching lecture.
13.3.1. Briefing. The main purpose of the military briefing is to inform rather than to teach. The briefing format or organization is often fixed, with different subjects arranged topically that may be in the same order day after day. The briefing often forms the basis for decision-making or operations. The formal briefer minimizes self, while effective l ecturers often project more personality or humor. The briefer is usually not concerned with such le arning techniques as interim summaries, repetition, and extended conclusions. If there is interchange be tween the briefer and the audience, questions are usually restricted to the material being presented. As Air Force teachers, we may occasionally present briefings. We should avoid the strict briefing style in the classroom.
13.3.2. Speech. A speech generally has one of three basic purposes—to inform, persuade, or entertain. The informative speech is a narration concerni ng a specific topi c, but does not involve a sustained effort to teach. Orientation talks or presentations at commander's call are examples of speeches to inform. The persuasive speech is designed to m ove an audience to belief or action on some topic, product, or other matter. Recruiting speeches to a high school graduating class, budget justifications, and courts-martial summations are all primarily persuasive speeches. The entertaining speech gives enjoyment to the audience. The speaker often relie s on humor and vivid language as a primary means of entertaining the listeners. A speech at a di ning-in is frequently a speech to entertain. 13.3.2.1. Instructors may, at times, want to accomplish all three basic purposes of speaking. We often attempt to inform students on a given subjec t. When we try to influence students to modify their behavior we become persuade rs. Finally, it may be advantageous at times to gain attention by entertaining. Still, our primary purpos e is to facilitate student learning.160 AFMAN36-2236 12 NOVEMBER 2003

13.4. Advantages and Disadvantages of the Lecture Method. In many Air Force teaching situations the method of instruction to be used is predetermined. As instructors, we may have a limited role in deciding how the material will be presented. In some case s, however, it may be our job to choose the method to use for a lesson or a series of lessons. We should sele ct the appropriate method on ly after we write the lesson objective and complete our initial research on the subject. The selection process will be more effective if we are aware of the advantages and disadvantages of the lecture method. While these advantages and disadvantages pertain especially to the formal teachi ng lecture, most also apply to the informal teaching lecture.
13.4.1. Advantages. The lecture is one of the most efficient teaching methods for presenting many facts or ideas in a relatively short time. Material th at has been logically or ganized can be presented concisely in rapid sequence. Becaus e of its advantages, a majority of Air Force instructors use the lecture method at least part of the time. 13.4.1.1. The lecture is particularly suitable for introducing a subject. To ensure all students have the necessary background to learn a subject, we can present basic information in a lecture. By using the lecture in this manner, we can offer students with varied backgrounds a common understanding. A brief introductory lecture can give di rection and purpose to a demonstration or prepare students for a discussion.
13.4.1.2. The lecture is a convenient method for inst ructing large groups. If necessary, we can use a public address system to ensure all students can hear us. The lecture is sometimes the only efficient method to use if student-to-faculty ratio is high. 13.4.1.3. The lecture is often used to supplement material from other sources or for information difficult to obtain in other ways. If students do not have time for research or they do not have access to reference material, the lecture can fill the bill. In subject areas where information is available in widely scattered places (textbooks, journals, tapes), the lecture allows the instructor to summarize and emphasize pertinent material. Reports, current research, and information that changes frequently may not be easily available inwritten form. The lecture can give students the most up-to-date information. 13.4.1.4. The lecture allows a large number of student s to receive information from real experts in a particular subject. In general, a person who can speak from actual experience or a scholar who has carefully analyzed the results of research will have great credibility with students. The lecture is often the most effective way of communicati ng the energy and enthusiasm of a person who has actual experience in a field, thus motivating students.
13.4.2. Disadvantages. Although the lecture method can be an effective and efficient teaching method, it has a number of disadvantages as follows:
13.4.2.1. The lecture does not lead to maximum achi evement in certain types of learning. Speech skills, cooperative group thinking, and motor skills, fo r example, are difficult to teach with the lecture method. Students can develop such skills well only through practice. Moreover, the formal lecture alone is generally not appropriate for pr esenting material above the comprehension level of the cognitive domain. Because it allows for little orno student verbal participation, the formal lecture may also be inefficient for comprehension-le vel lessons in which concepts and principles are developed. 13.4.2.2. The lecture does not provide teachers with an opportunity to estimate student progress before an examination. Within a single lecture period, we may unwittingly present more informa-AFMAN36-2236 12 NOVEMBER 2003 161

tion than our students can absorb , and we have little accurate means during the lecture to determine what they have learned. 13.4.2.3. Too often, the lecture makes no provision for student participation. As a result, many students willingly allow the instructor to do all the work. Learning is an active process—the lecture method, however, tends to foster pa ssiveness and dependence on the instructor.13.4.2.4. Instructors may have to spend much time preparing for the lectures. With the demonstration-performance method of inst ruction, students participate actively. With the case study and guided discussion methods, students participate verb ally. The teaching interview relies heavily on the knowledge of an expert and provides for st udent involvement through a question-and-answer period. But with lecture, a greater burden fo r the total lesson rests on the instructor. 13.4.2.5. Finally, many instructors find it difficult to hold the attention of their students when they lecture for an entire class period. To use the lectur e method effectively, we obviously need considerable skill in speaking. 13.4.2.6. The first two disadvantages of the lecture—di fficulty in achieving certain types of learning and ineffectiveness of assessing student learning—are inherent to the method. The third—student passivity—while characteristic of the method, can be alleviated with an effective informal lecture. The final two disadvantages—preparation time and speech skills necessary for an effective lecture—provide the focus for the remainder of this chapter.
13.5. Preparing the Lecture. When faced with the responsibility for planning and preparing a lecture, a primary requirement is to analyze the students. This requirement is especially critical because students tend to be passive when hearing a lecture. Your analysis of the students will greatly influence how you plan your instruction.
13.5.1. Student Analysis. There are two reliable methods for gaining student information. Used together they can be extremely useful. The first is to organize information you already have about the students. Knowing such variables as age, sex, grade, and experience can help you relate to your students. If one or more of these variables separate s you from your students, you may want to give special attention to ways of emphasizing your si milarities and reducing diff erences. The second method, when you have not talked to the group before, is to check with someone who has. Perhaps a friend or colleague has already lectured to the same group and can tell you what to expect. 13.5.1.1. Especially important are the questions, "What do students already know about my topic?" and "What do they need to know?" If some students are already experts, you will want to reveal your credentials in a pleasing but info rmative manner. If they know much less about the topic than you do, there may still be a problem. One of the greatest difficulties a lecturer faces is not to talk up or down to students. Only by carefu lly analyzing the needs and characteristics of the students can you be certain your comments are at the appropriate level.13.5.1.2. Assuming you have carefully assessed the students and written an appropriate student-centered objective, as discussed in Chapter 4 , you are now ready to collect and arrange your information. What you already know and what you learn from others or from books will probably yield more information than you have time to pr esent. Possession of extra and extended information is especially important in the informal lect ure where students may ask questions, but it is also important in the formal lecture. Students can rightfully expect the lecturer to know more about the topic than is presented in the lecture.162 AFMAN36-2236 12 NOVEMBER 2003

13.5.1.3. In preparing the formal teaching lecture, you should follow a lesson planning process like the one discussed in Chapter 6 and illustrated in the sample lesson plan at Attachment 2

. Choosing an appropriate organizational pattern, effectively using support material, and having a clear introduction and a complete conclusion (all discussed in Chapter 6 ) will facilitate student understanding and retention. Although the major concerns of lesso n planning are more or less common with all methods, some are of special importance to the lecture.
13.5.2. Lecture Development: 13.5.2.1. Research indicates that clear attention, motivation, and overview steps in the introduction of the lesson significantly improve learning by appealing to students, giving them a reason to listen, and providing a forecast of what will be taught. (See the introduction of the sample plan at Attachment 2 .) Strong organization and clear verbal and visual support in the body of the lesson help students understand and retain the material. In the sample plan for the formal lecture, see how organization is planned and verbal support (defin ition, example, comparison, statistics, and testimony) are all used. Also, visual support is planned. A complete conclusion to the lesson—final summary, remotivation, and closure—reteaches, chal lenges, and leaves the students with a satisfying sense of completeness. 13.5.2.2. With the teaching lecture, each of these concerns becomes especially critical since the burden for presenting the lecture falls directly upon th e instructor. In fact, every part of the lecture must be designed to elicit attention and challe nge students to listen. If you do any less, you will not be totally effective. All verbal and visual support should be selected and used carefully.
13.5.3. Audiovisual Support. Chapter 20 discusses visual aids available to the instructor. The sample lesson plans in the attachments may indicate th at the instructor plans to use a video cassette recorder, an overview on chart paper, a computer-generated presentation, and a student note-taking handout. Careful planning can help you choose appropriate and timely aids. Although audiovisual aids are both appropriate and useful with all teaching methods, they are especially important with the lecture. The following guidelines should help you in planning the use of visual support, not only for the lecture but for all methods: 13.5.3.1. Use only relevant materials. Avoid using mate rials solely for aesthetic or interest value. Visual materials should certainly be interesting, but the primary purpose of any visual aid is to portray or support an idea graphically. Irrelevantmaterials distract from the idea you are presenting. 13.5.3.2. Use visual materials large enough to be seenby all the students. Nothing is so disturbing as to be seated in the back of the room unable to see the visuals. In preparing for your presentation, display the visuals, then move yourself to the location of your most distant listener. If you can't readily see the material, consider repl acing it with somethi ng more appropriate. 13.5.3.3. Use visual materials only at the proper time. Do not expose the visual material until the proper point in the lecture. As illustrated in Part II of the sample plan, cues for audiovisual aids are clearly marked. Materials that are visible too soon, or those that remain in view after the point has been made, distract from and interrupt the continuity of the lecture. You may want to use the "striptease" or buildup method for revealing a series of points. Don't list 10 main points for the students and then discuss each one. Instead, uncover th e points one at a time to keep student attention focused.AFMAN36-2236 12 NOVEMBER 2003 163

13.5.3.4. Keep visual materials as simple and clear as possible. Emphasize only the most important information. Omit unnecessary details. A series of simple charts is preferable to a single, complicated one. 13.5.3.5. Talk to the student, not to the visual aid. If you are explaining a chart, look at your students as much as possible. By the time you ma ke your presentation, you should be so familiar with your visual aids that it will not be necessary for you to look at them closely. When possible, paraphrase the visual instead of simply reading it. 13.5.3.6. Place visuals away from obstructions. Don't allow other objects or persons—including yourself—to obstruct the view of your students. Y ou decided to use visual materials to support and clarify your lecture; don't hinder their effectiveness by obstructing the student's view.
13.5.4. Humor. Appropriate humor (that which supports the subject and/or lesson) is useful in a lecture. Humor relaxes both teachers and students. Humor serves this role especially well at the beginning of a lecture because it places the teacher directly in communication with the students. There are two reasons to use humor in the body of the lecture. One is to bring back the attention of the students; humor regains attention. The second reason is to emphasize an important point. Although a humorous story or anecdote is seldom real proof, it may be the most powerful memory or clarification device a speaker can use. Notice how the sample lesson plan ( Attachment 2 ) for the lecture incorporates humor into the planning. 13.5.4.1. Humor must be used properly to be effectiv e. There are five essentials to using humor.
13.5.4.1.1. Know the Item Thoroughly. If you know the story and have told it before, you will be able to tell it again and know the kind of response to expect. Generally a good rule to follow is not to use a story or humorous item unl ess you have told it several times in informal situations so you can both practice and gauge the reactions of others.
13.5.4.1.2. Don't Use Off-Color Stories to Get a Cheap Laugh. Even people who laugh at such stories in private often lose respect for the teacher who uses them in public.
13.5.4.1.3. Vitalize Humor. Stories should be personalized so they are believable and sound as if they really happened. Rather than talkabout "this guy I heard about" or "this truck driver," give names to the characters in the stories. Successful comedians and speakers nearly always vitalize their humor.
13.5.4.1.4. Don't Laugh Before the Audience Laughs. If you fail to get the story across, laughing alone on a platform is disaster. If the joke fails, leave it and go on.
13.5.4.1.5. Capitalize on the Unexpected. One of the primary elements of humor is that people laugh when they are surprised. A few years ago, streaking was a fad on college campuses. Most firsthand observers laughed when confronted by a streaker. Some of the laughter was no doubt due to embarrassment, but much of it was due to the element of surprise. Types of humor that depend on the unexpected are: quips (o f course men aren't what they used to be— they used to be boys); puns (try our bread, we knead the dough); exaggeration (the heat was so terrific last week that I saw a hound dog chasi ng a rabbit; they were both walking); and understatement (if at first you don't succeed...well, so much for skydiving).
13.5.5. Transitions. Transitions are statements used by the instructor to move from the introduction to the body of the lecture, between main points, between subpoints within each main point, and from the body to the conclusion of the lecture. Transitions signal to the students that we are progressing to164 AFMAN36-2236 12 NOVEMBER 2003

a new point, but they are also important in mainta ining the continuity of the information given. Consider this sample transition: "We have discussed the precedents for a mandatory physical fitness program in the military. Next we will consid er the benefits of such a program." NOTE: This transition indicates a change in direction, but does not indi cate the reason for or importance of the change. 13.5.5.1. For transitions to be effective, they shoul d (1) mention the point just discussed, (2) relate that point to the objective, and (3) introduce the next point. Suppose the objective is for students toknow the need for a mandatory physical fitness program. Notice how all three steps are incorporated in the following transition: "(1) We have discussed the precedents for a mandatory physical fitness program in the military, (2) but these preced ents alone will not prove a need for such a program. To understand that need more fully, (3) we must next examine, in several practical situations, the benefits of mandatory physical fitness." 13.5.5.2. When planned and used correctly, transitions contribute substantially to the continuity of the total lecture. The lesson plan shows how the lecturer plans to use transitions.
13.5.6. Summaries. Summaries are useful tools for maintainin g continuity within a lecture and for highlighting areas of particular importance. Summaries prepared for use between main points are not always necessary in the lecture. In fact, if the point is very clear, a summary may be redundant and boring. You should use them, however, when main points are unusually long or contain complex or unfamiliar information. 13.5.6.1. With summaries, we repeat information concisely to reinforce student understanding before new information is presented. Summaries should not take the place of transitions. They should provide a means for us to progress logically from one main point through the transition and into the next point. 13.5.6.2. The summary given in the conclusion of the lecture should be designed so that it reviews for the students those facts or aspects of a concep t or principle you consider particularly important. It may be helpful to think of this summary as a "c apsulated" version of the lecture itself, in which key ideas are related both to each other and to th e lesson objective. It is your final opportunity to reinforce critical aspects of the lesson.
13.5.7. Questions. In both the formal and informal lecture, questions may be used to add continuity to the lesson. The instructor may plan rhetorical quest ions to use within a main point or in the conclusion of the formal lecture. Questions encourage students to review in their own minds the information presented. They also indicate areas of special im portance and should be phrased to allow students to see the relationship of these areas to the lesson objective. Chapter 11 discusses the use of questions in more detail.
13.6. Presenting the Lecture. Although preparing a lecture can be laborious, for many instructors the hardest part is actually presenting the lecture. Ques tions instructors most often ask are: How many notes should I use? How can I overcome nervousness? What ki nd of physical behavior is appropriate for me to use when I speak? What if my voice isn't suited to speaking before a group? How can I project sincerity and enthusiasm?
13.6.1. Methods of Presentation. Instructors, like all other speakers, use one of four common methods of presentation: (1) speaking from memory, (2 ) reading from manuscript, (3) speaking impromptu with no specific preparation, and (4) speaking extemporaneously with, ideally, a great deal of prepara-AFMAN36-2236 12 NOVEMBER 2003 165

tion and a limited number of notes. The fourth method us ually allows us the most freedom in adjusting to an audience as we speak and is best suited for almost all instructing in the Air Force.
13.6.1.1. Memorizing. Speaking, based on memorization, is the poorest method of delivering lectures and should be used very sparingly or not at all. While this method may seem to be helpful for persons who cannot think on their feet, the memo rized lecture is a straitjacket. Such a lecture cannot be adapted to the immediate situation or to student reactions. It does not allow the teacher to adjust to the particular situation. Moreover, the method is almost sure to destroy spontaneity and a sense of communication. The method also requi res an inordinate amount of preparation, and the danger of forgetting is ever present.
13.6.1.2. Manuscript Reading. Reading a lecture from a manuscript allows for planning the exact words and phrases to use. The disadvantages of this presentation method far outweigh the advantages. Many instructors use the manuscript as a crutch instead of fully thinking through the ideas in the lecture. All too often the written lecture is regarded simply as an essay to be read aloud. Therefore, the lecture is too broad and has language too abstract to be understood when presented orally. Also, very few people can read fr om a manuscript with the same emphasis and spontaneity used with extemporaneous delivery. If you must adhere closely to a manuscript, consider the guidelines in Figure 13.1.
Figure 13.1. Guidelines for Manuscript Lessons.

13.6.1.3. Impromptu. Impromptu speaking requires a tremendous amount of skill and knowledge. It may be necessary at times to lecture on the spur of the moment without any preparation. This method should only be used by experienced instructors who are very knowledgeable of their - Keep spoken words simpler, clearer, and more vivid than in writing. - Make sentences shorter and ideas less complex than in writing. - Clarify the transitions between thoughts and id eas. Provide punctuation with vocal inflection, variety, and pauses. - Use repetition to emphasize main ideas and key points. - Use direct address when speaking about people. Pe rsonal pronouns such as I, we, our, us, and you are better than they, people, a person, the reader, the listener. - Use concrete language where possible. Follow abstract or complicated reasoning with specific examples, comparisons, and definitions. - Prepare your manuscript to facilitate reading. D ouble or triple spacing, marking the manuscript, using only one side of the paper, and us ing shorter paragraphs may be helpful. - Practice the lecture aloud severa l times to see how it sounds. Reco rding yourself on a video cassette recorder and watching it will help you to discover places where you may not be communicating effectively. - Try to make your lecture sound like conversation, as if you were thinking the words for the first time as you read them. - Practice looking at your audience most of the time as the manuscript becomes more familiar to you.166 AFMAN36-2236 12 NOVEMBER 2003

subjects and have the ability to organize their thoughts for learning as they speak. Even these experienced instructors fall back upon thoughts a nd phrases they have used before. They have spent years, so to speak, in preparing to give an unprepared lesson.
13.6.1.4. Extemporaneous: 13.6.1.4.1. This technique is used most wide ly by effective speakers. It produces the most fruitful results when based upon full preparation and adequate practice. The lesson is carefully planned and outlined in detail. The instructor's only guide is a well-constructed outline like that given in Part II of the model ( Attachment 2 ). It is a lesson planned idea by idea, rather than word by word. 13.6.1.4.2. The advantages of teaching from a well-planned outline are many. The method compels instructors to organize ideas and puts pr essure on them to weigh materials in advance. It gives freedom to adapt a lesson to the occasion and to adjust to student reactions. It enables instructors to change what they plan to say right up to the moment of utterance. In short, the extemporaneous method permits the instructor to adhere to the two vital needs of effective teaching—adequate preparation and a lively sense of communication.
13.6.2. Nervousness. If a teacher suffers from stage fright, nervousness, or fear of speaking, students may also become uneasy or anxious. Yet some ner vousness is both natural and desirable. Even skilled instructors often experience the queasy feeling of "but terflies in the stomach" as they prepare to speak. The secret is to get the butterflies "flying in form ation" through practice. A vi siting athletic team willpractice on the home field before game time to accustom themselves to differences in terrain and environment. Similarly, the speaker may need to prac tice a new lecture several times, preferably in the room where the lecture will be given, before actually presenting it. Practice reminds us to concentrate on the pronunciation of a new word or to check an additional piece of information on an important point. Consider the following suggestions for overcoming nervousness:
13.6.2.1. Be enthusiastic. At times we may lecture on subjects we find dull, but as we get more involved, the subject becomes more interesting. There is no such thing as a dull subject—only dull teachers. Be enthusiastic about a subject; enthus iasm can replace fear. The more enthusiastic youare about the subject, the more the students will be involved with both you and what you are saying. 13.6.2.2. Hold good thoughts toward your students. The students in the audience are the same ones you enjoy speaking with in a less structured environment. Most classes are made up of warm human beings with an interest in what you have to say. Students rarely "boo" the teacher or throw vegetables. Most students have great empathy for teachers and want them to do a good job. 13.6.2.3. Do not rush as you begin to speak. Many teachers are so anxious to get started they begin before they are really ready. The little ex tra time taken to arrange your notes will generally pay big dividends. When you are ready to begin, look at the various parts of the class, take a deep breath, and begin to speak.
13.6.3. Physical Behavior. Communication experts tell us that over half of our meanings may be communicated nonverbally. Although some nonverbal meaning is communicated through vocal cues, eye contact, body movement, and gestures carry mu ch meaning. As teachers, we need to know how physical behavior can improve our lect uring skill and thus enhance learning.AFMAN36-2236 12 NOVEMBER 2003 167

13.6.3.1. Eye Contact. Eye contact is one of the most important factors of nonverbal communication. Nothing will enhance your delivery more than effective eye contact with your students. Eye contact is important for three reasons. First, it lets the students know you are interested in them. Most people like others to look at them when talking. Second, effective eye contact allows you to receive nonverbal feedback from your students. With good eye contact, you can gauge the effect of your remarks. You can determine ifyou are being understood and which points are making an impact and which are not. You will be able to detect signs of poor understanding as well as signs that the students are learning. You can then ad just your rate of delivery or emphasis, rephrase or summarize certain points, or add more supporting data. Third, effective eye contact enhances your credibility. Students judge teachers with gr eater eye contact as being more competent.13.6.3.1.1. To achieve genuine eye contact, you must do more than merely look in the direction of your listeners. You must have an earnest desire to communicate with them. The old advice of looking over the tops of your listeners' heads or attempting to look at all parts of the class systematically simply does not describe effective eye contact. Furthermore, looking at only one part of the audience or directing atte ntion only to those students who seem to give you positive feedback may cause you to ignore large parts of the audience. Make it evident to each person in a small class, and each part of the audience in larger auditoriums, that you are interested in them as individuals and eager to have them understand the ideas you are presenting. In this way you will establish mental as well as sensory contact with your listeners. 13.6.3.1.2. Effective eye contact can be described as direct and impartial. Look directly into the eyes of your listeners and look impartially at all parts of the audience, not just at a chosen few.
13.6.3.2. Body Movement. Body movement is one of the important factors of dynamic and meaningful physical behavior. Good body movement is important because it catches the eye of the listener. It helps hold the attention needed for learning to occur. Movement also represents a marked departure or change in your delivery pattern; it is a convenient way of punctuating and paragraphing your message. Listeners will know you are done with one idea or line of thought andready to transition to the next. Finally, aside from its effects on the listeners, it helps you as a lecturer. Movement helps you work off excess ener gy that can promote nervousness. Movement puts you at ease. 13.6.3.2.1. How much movement is desirable? Un less the formality of the situation or the need to use a fixed microphone keeps you inone position, you should probably move frequently when presenting a teaching lecture. M ovement from behind the lectern can reduce the psychological distance between you and the students and place them more at ease. Some instructors feel they need the lectern to hold th eir notes. However, instruction is actually more effective if you carry your notes with you instead of looking down at the lectern to see them. When you look at your notes, remember to direct your eyes—not your head—to the paper

. 13.6.3.2.2. Some instructors, however, move too much. Perhaps out of nervousness they pace back and forth in front of the class. Still others have awkward movements that do not aid communication. Some leave their notes on the lectern, then move in and out from behind it like a hula dancer. Others plant their feet firmly in one place, then rock from one side to the other in regular cadence.168 AFMAN36-2236 12 NOVEMBER 2003

13.6.3.2.3. Effective body movement can be descri bed as free and purposeful. You should be free to move around the class. You should not feel restrained to stay behind the lectern, but should move with reason and purpose. Use your m ovement to punctuate, direct attention, and otherwise aid learning.
13.6.3.3. Gestures. Gestures may be used to clarify or empha size ideas in the lecture. By gestures we mean the purposeful use of the hands, arms, sh oulders, and head to reinforce what is being said. Fidgeting with a paper clip, rearranging and shuffling papers, and scratching your ear are not gestures. They are not purposeful and they distract from the verbal message. Placing both hands in your pockets, behind your back, or in front of you in the fig-leaf position severely limits their use for gesturing. Holding your shoulders and head inone position during the lecture will also rob you of an effective means of strengthening your communication. 13.6.3.3.1. Gestures will be most effective if you make a conscious effort to relax your muscles before you speak, perhaps by taking a few short steps or unobtrusively arranging your notes. Effective gestures are complete and vi gorous. Many speakers begin to gesture, but perhaps out of fear, they do not carry through and their gestures abort. 13.6.3.3.2. Comedians get laughs from the audience by timing gestures improperly. A gesture that comes after the word or phrase is spoken appears ludicrous. Good gestures should come atexactly the time or slightly before the point is made verbally. Poor timing results from attempting to "can" or preplan gestures. 13.6.3.3.3. Finally, good gestures are versatile. A stereotyped gesture w ill not fit all subjects and situations. Furthermore, the larger the au dience, the more pronounced the gestures will need to be. As with all aspects of comm unication, gestures must fit the transaction. 13.6.3.3.4. We are not advising that you adopt a dynamic forceful mode of delivery if by nature you are quiet and reserved. As with movement, gestures should spring from within. Effective gestures are both natural and spontane ous. Observe persons talking with each other in a small group; we should try to approximate the same natura lness and spontaneity of gestures when we are lecturing.
13.6.4. Use of the Voice. A good lecturing voice has three importantcharacteristics: it is reasonably pleasant, it is easily understood, and it expresses diff erences in meaning. Technically we might label these three properties as quality, intelligibility, and variety. 13.6.4.1. Quality refers to the overall impression a voice makes on others. Most people have reasonably pleasant voices suitable fo r lecturing. Certainly a pleasing quality or tone is a basic component of a good speaking voice. Some persons have a full rich quality, others are shrilly and nasally, still others may have a breathy and muffled tone or quality. While basic aspects of quality may be difficult to change, your voice may beco me more breathy when you are excited, tense when suspense is involved, or resonant when reading solemn language. Students can often tell from the voice if the teacher is happy, angry, sad, fearful, or confident. Similarly, vocal quality can convey sincerity and enthusiasm. Some teachers are overconcerned about the basic quality of their voices; at the same time, they pay too little attention to the effect of attitude and emotion on the voice. 13.6.4.2. Intelligibility of your speech depends on se veral factors. Attention to articulation, pronunciation, and volume, as well as avoiding voca lized pauses, stock expressions, and substandard grammar can make your voice more intelligible.AFMAN36-2236 12 NOVEMBER 2003 169

13.6.4.2.1. Articulation, or enunciation, refers to the precision and clarity of speech. Good articulation is chiefly the job of the jaw, ton gue, and lips. Most artic ulation problems result from laziness of the tongue and lips or failing to open the mouth wide enough. You should overarticulate rather than underarticulate spee ch sounds. What sounds like overarticulation to you will come out as crisp, understandable words and phrases to your students.
13.6.4.2.2. Pronunciation refers to the traditional or customary way words sound. Standards of pronunciation differ, making it diff icult at times to know what is acceptable. Dictionaries are useful, but as they become outdated, they should not be adhered to excessively. Generally, educated people in your community, as well as national radio and te levision announcers, provide a good standard for pronunciation. Common faults of pronunciation are to misplace the accent saying "device" instead of de-vice, to omit sounds by saying "guh'mnt" for government, to add sounds by saying "ath-a-lete" for ath-lete, to sound out silent letters by saying "mort-gage" instead of mor-gage or "of-ten" instead of of-en. Do not overcompensate to the point that you call attention to your speec h. Remember, though, that pronunciation acceptable in informal conversation may be substanda rd when presenting a formal lecture. 13.6.4.2.3. Vocalized pause is the name we give to syllables "a," "uh," "um," and "ah," often used at the beginning of a sentence. While a fe w vocalized pauses are natural and do not distract, too many impede the communication and learning processes. 13.6.4.2.4. Avoid overusing stock expressions, such as "OK," "like," and "you know." These expressions serve no positive role in communication and only convey a lack of originality by the speaker. 13.6.4.2.5. Substandard grammar has no place in the teaching lecture. It only serves to reduce teacher credibility with students. Research show s that even persons who have been using substandard grammar all of their lives can, with d iligent practice, make significant gains in this area in a relatively short time. 13.6.4.3. Variety is the spice of speaking. Students rapidly tire from listening to a teacher who doesn't vary delivery style or has a monotonous voice . A teacher's voice that is intelligible and of good quality still may not appeal to students. Y ou may vary your voice and, at the same time, improve your communication by considering the vocal fundamentals of rate, volume, force, pitch, and emphasis. 13.6.4.3.1. Most teachers speak at a rate of from 120 to 180 words a minute when presenting a lecture. In normal speech, however, we vary the rate often so that even within the 120 to 180 word-per-minute constraints there is much change. The temperamentally excitable person may speak at a rapid rate all the time, and the stolid person generally talks in a slow drawl. The enthusiastic but confident individual, though, will vary the rate of delivery to emphasize ideas and feelings. A slower rate may be appropriate for presenting main points, while a more rapid rate may lend itself to support material. The e xperienced lecturer also knows that an occasional pause punctuates thought and emphasizes ideas. A dramatic pause at the proper time may express feelings and ideas even more effectively than words. 13.6.4.3.2. Volume is important to the lecturer. Always be certain all students can hear you. Nothing hinders the effect of a lecture more thanto have some students unable to hear. On the other hand, the lecture should not be too loud fo r a small room. A bombastic or overly loud speaker tires out listeners very quickly.170 AFMAN36-2236 12 NOVEMBER 2003

13.6.4.3.3. Force or vocal energy is needed at times to emphasize and dramatize ideas. A drowsy audience will quickly come to attention if the teacher uses force effectively. A sudden reduction in force at times may be as effective as a rapid increase. By learning to control the force of your voice, you can help to add emphasis and improve communication. 13.6.4.3.4. Pitch is the highness or lowness of the voice. All things being equal, a higher-pitched voice carries better than a low-pitched one. On the other hand, students will tend to tire faster when listening to the higher-pitched voice. If your voice is within normal limits—neither too high or too low—work for variety as you speak. Try not to become firmly entrenched in your habitual pitch level. 13.6.4.3.5. Emphasis obviously stems from all forms of vocal variety, and any change in rate, volume, force, or pitch will influence the emphasis. The greater or more sudden the change, the greater the emphasis will be. As a lectureryou will want to use emphasis wisely. However, avoid overemphasis and continua l emphasis. Be judicious. Emphasizing a point beyond its real value may cause you to lose credibility with your students.
13.6.5. Sincerity: 13.6.5.1. A lecturer certainly needs to prepare well and possess strong delivery skills to do an effective job in the classroom, but something more is needed to be really effective—a teacher must be sincere. Students will be amazingly tolerant of weakness in both prep aration and delivery, but give them a chance to suspect your sincerity and you lose effectiveness. And, once lost, effectiveness is nearly impossible to regain. What is sincer ity? It may be defined as the state of appearing to be without deceit, pretense, or hypocrisy—a state of honesty, truthfulness, and faithfulness. 13.6.5.2. Sincerity toward students is reflected in your eye contact, enthusiasm, and concern about students, both as individuals and as learners. Sincerity toward the subject is judged by whether or not you seem involved and interested in the subject or topic of the lecture. Sincerity toward self is displayed in the confidence and concern you have that you are doing the best job possible in the classroom. Lack of sincerity in any of these areas will almost certainly hinder learning.
13.6.6. Verbal Interaction. Verbal interaction in the informal lecture takes place in two ways. First, students should ask questions to clarify confusing poi nts or to ensure their understanding of the information. Second, the instructor should also question th e students during an informal lecture. By asking both planned and spontaneous questions, the instru ctor can stimulate participation, emphasize important points and, most importantly, judge whether or not students understand the material. (See the informal lecture sample lesson plan at Attachment 3 .) 13.6.6.1. To be most effective, verbal interactio n should occur consistently throughout an informal lecture. The instructor must allow ample t ime for questions and answers when planning andpracticing the lesson. During the introduction, he or she should encourage questions. By comparing the sample lesson plans at the attachments, we can see the substantial differences between planning for the informal and formal lecture. Students will be more likely to participate if the instructor indicates through direct eye contact, fr equent pauses, and a relaxed delivery that he or she is sincerely interested in student participat ion. Instructor questions are especially effective when they require students to summarize important information or to provide additional support in the form of personal examples.AFMAN36-2236 12 NOVEMBER 2003 171

13.6.6.2. Although frequent verbal interaction is a goal of the informal lecture, it should not take priority over achieving the lesson objectives. If a portion of the material is complex, unfamiliar to the students, or follows a necessary sequence, the questions may be distracting or cause confusion. In this case the instructor should ask students to hold their comments until after that material has been presented. This additional structure may also be necessa ry when time constraints do not allow student participation toward the end of a l ecture. Careful planning is needed to ensure a comfortable balance between the material to be presented and the questions to be shared.
13.7. Summary: 13.7.1. Formal and informal lectures are popular teac hing methods in the military. Advantages of thelecture method are (1) presentation of many ideas in a short time, (2) suitability for introducing a subject, (3) convenience for instructing large groups , (4) value for supplementing material from other sources, and (5) allowance for a large number of stude nts to hear an expert. Disadvantages of the formal and informal lecture methods are that they are (1) not appropriate for certain types of learning, (2) require too much preparation, and (3) depend on speaki ng skill. In addition, the formal lecture also has no provision for the teacher to estimate student progress and no active participation by students. 13.7.2. To prepare for a lecture we must analyze our student audience, carefully plan the beginning and ending of the lecture, and or ganize and choose verbal supporting materials to help students listen and understand. In addition, we should consider using humor when appropriate, choose visual aids wisely, and plan transitions and interim summarie s when needed. When presenting the lecture, we need to select the method for pres entation; overcome nervousness; atte nd to the physical factors of eye contact, movement, and gestures; and strive for a pleasant, communicative, and expressive voice. Above all, we need to show sincerity towa rd our students, our subject, and ourselves.172 AFMAN36-2236 12 NOVEMBER 2003

Chapter 14

THE GUIDED DISCUSSION METHOD

14.1. Introduction. Discussion is one of the most used teachi ng methods in civilian and military educational institutions. One type, the guided discussion, is an instructor-controlled group process in which students share information and experiences to achieve a learning objective. A guided discussion is different from free discussion and a peer-controlled seminar. Like the "bull session," free discussion can be valuable for brainstorming or as a management aid, but it seldom supports measurable objectives. The peer-controlled seminar relies on qualified students to lead discussion among peers, which may significantly reduce the likelihood of reaching the learning outcomes. Attachment 4 provides an example lesson plan for the guided discussion method. 14.1.1. In a guided discussion, the instructor carefullyplans the lesson to reach desired learning outcomes. The group interacts in response to questions; the instructor refrains from entering the discussion as an active participant. Students are encouraged to learn about the subject by actively sharing information, experiences, and opinions. The flow of communication is a transaction among all the students rather than between individual students a nd the instructor. Questioning and summarizing skills become critical as the instructor guides the students to the lesson objective. 14.1.2. In this chapter we will treat the guided disc ussion in terms of a teaching method with a set of teaching skills that differentiate it from other teaching methods.
14.2. Selection and Planning Factors. Proper consideration of selection and planning factors contributes to the success of guided discussions. Some of the most important factors follow:
14.2.1. Lesson Objectives. The discussion method is a superiormethod for teaching more complex cognitive and affective objectives. Thus, the met hod is appropriate for promoting the understanding of concepts and principles and for the development of problem-solving skills. It is inappropriate for transmitting knowledge-level mate rial where lectures or read ings are more efficient. 14.2.1.1. The degree of consensus in a subject area is also an important consideration. In some subjects there is a high degree of consensus among authorities in the field because of established concepts, principles, laws, and fa ctual information. In such subjects the objective is to communicate these facts, concepts, and principles to stude nts. Thus, Air Force instructors in civil engineering, nuclear physics, mathematics, and highly te chnical subjects might find forms of one-way verbal communication, such as the lecture method, more appropriate. They might also use individualized instruction or the demonstration-performance method in achieving lesson objectives. 14.2.1.2. If authorities in a field have a low degree of consensus, or if there are two or more widely held schools of thought, the guided discussi on method is a very appropriate way to achieve lesson objectives. In such subjects the experts do not always agree. Concepts and principles differ considerably from one school of thought to another and controversy often exists. Air Force instructors who give instruction in history, ma nagement, leadership, fina ncial management, logistics, and Air Force organization, for instance, will find many objectives that lend themselves to guided discussion. Through discussion students begi n to understand the controversy and differing points of view. They have the opportunity to de velop and defend a logical position of their own.AFMAN36-2236 12 NOVEMBER 2003 173

14.2.1.3. Another consideration is the needs of particular students. If your students need to develop the ability to think critically in apprai sing the ideas of others, the guided discussion method works well. Students also get the opportu nity to reason together and to develop group decision-making skills. In the discussion, they learn to evaluate ideas while listening to others, to formulate their own ideas, and to present and defend these ideas in the process of achieving an instructor's planned learning outcomes. 14.2.1.4. The guided discussion may also be effective in changing student attitudes and their behavior. Research in social psychology supports the position that discussion is more effectivethan the lecture for achieving such objectives. When students make a public commitment in a discussion, they are more apt to follow through with a change in behavior. Instructors in fields such as religious education, social actions, and safe ty will find the guided discussion method serves as an effective tool for modifying attitudes and behavior of Air Force personnel.
14.2.2. Group Size. If the instructional objective lends itsel f to the guided discussion, the next consideration is group size. With one student, individualiz ed instruction is appropriate. When the students number 2 to 20, the guided discussion method is frequently used. In smaller groups, there is more opportunity for each student to participate. Reticent students may not participate at all when the group becomes too large. Five to seven students would se em to be the ideal size, but few instructors will have such a low instructor-to-student ratio in thei r classes. When the number is 20 to 40, instructors may find that a combination of methods works best. For instance, a combination lecture-discussion is often used. With over 40, the lecture method is typically used.
14.2.3. Time Available. Instructors should keep in mind the relationship between group size and time. As group size increases, so should the av ailable discussion time. Ideally, enough time should be scheduled for the contributions of all members to be heard and discussed, but this goal may not be achieved even with groups of five to seven students. When costs are a significant factor and a given amount of material must be cove red in a short time, the teaching lecture may be more efficient than the discussion.
14.2.4. Instructor Personality: 14.2.4.1. Some instructors have personalities better suited to the guided discussion than others. When this method is used, instructors must be willing to relinquish some of their authority over the lesson. A key to success for this method (like a ny other method) is that the instructor needs to be flexible enough to adjust his or her teaching style to achieve the lesson objective. The guided discussion method requires a little more give-and-take between the instructor and students. Instead of presenting material directly to students, they must be confident in their ability to use questions and other control devices skillfully and to deve lop the concepts and principles based upon student input. 14.2.4.2. If your lessons must be highly organize d, tightly controlled, and instructor centered, then you will have difficulty using the guided di scussion. Similarly, if you have difficulty following the twists and turns of a discussion without losing track of the argument, losing patience with its complexity, or in pulling scattered points toge ther in summaries to reach conclusions, the discussion method may be difficult for you to use successfully.
14.2.5. Establishing a Common Base for the Discussion-Prerequisite Knowledge. If you decide to use a guided discussion after considering these fact ors, your next planning consideration is to establish a common ground for the discussion. Students shoul d be at least at the knowledge level on a dis-174 AFMAN36-2236 12 NOVEMBER 2003

cussion topic when asked to engage in a guided discussion. Lectures, programmed texts, reading assignments, films, and field trip s can provide basic knowledge. Gu ided discussions conducted near the end of a block of instruction should be base d on a consideration of the common ground established prior to the discussion.
14.2.6. Use of a Problem. One way to make sure students in a discussion group are able to sustain a worthwhile discussion is to give them a problem or short case to read or view in advance. This problem, along with the knowledge stude nts already have, should keep the discussion on track and productive. The problem is usually a printed paragraph or two that illustrates the concept or principle under discussion. Unlike a higher-level case study, we are not yet attempting to apply specific principles to solve complex problems. Guided discussions centered around a problem are designed to develop a comprehension of concepts and principles, with the problem providing the discussion vehicle. The techniques for leading this type of discussion ar e essentially the same as those described throughout this chapter.
14.2.7. Planning the Room Arrangement: 14.2.7.1. Because of its effect on group dynamics, roomarrangement is an important factor in a successful guided discussion. (An understanding of group dynamics is also a prerequisite for maximizing learning in a guided discussion; see Chapter 29 .) Research shows that eye contact is one of the most important variables in conducting a su ccessful guided discussion because direct visual contact between members of a group increases communication. 14.2.7.2. Therefore, a circular arrangement is re commended in which no one is placed in a position of dominance, and each member can establish direct eye contact with each of the other members. With tables, an arrangement involving a pentagon, hexagon, or octagon also works well.Avoid rectangles that tend to put those at each end in a position of authority. To foster group discussion, instructors should avoid placing themselves in a position of authority where student eye contact (and most student comment) is directed toward the instructor. Instead, the instructor should become a group member without relinquishing control.
14.3. Organizing the Guided Discussion. Once you have considered the planning factors and have decided to use a guided discussion, you are ready to de velop the lesson plan. Note the sample lesson plan in Attachment 4 . Part I of the sample lists such things as lesson title, references, statement of the lesson objective, samples of behavior, and the lesson outline. Assistance in developing this part of the lesson plan is provided in the lesson planning chapters ( Chapter 3 , Chapter 4 , Chapter 6 , Chapter 8 , Chapter 9 , Chapter 10 , and Chapter 11 ). This chapter is directed toward preparing Part II—the actual teaching plan.
14.3.1. Introduction. A guided discussion is introduced in a way similar to other teaching methods with attention, motivation, and overview steps; see Chapter 6 . However, the overview is especially important to the success of a guided discussion and deserves further comment. 14.3.1.1. A good overview pinpoints the topic for discussion and the key areas to be developed. If the topic is motivation, the two key areas for disc ussion might be extrinsic and intrinsic motivation. You should tell the students what will be discussed; visually show the overview on a chart, chalkboard, or handout. The overview visual should be available throughout the lesson to aid students in following the lesson organization and to assist the instructor in summarizing and transitioning between main ideas. A lesson overview also provides the instructor with an opportunity toAFMAN36-2236 12 NOVEMBER 2003 175

review with students pertinent information al ready known or understood about a subject. Define or clarify terms or concepts to prevent confus ion. Also, you should make students feel that their ideas and active participation are wanted and needed.
14.3.2. Development. When developing concepts, principles, generalizations, or other ideas, it is important to anticipate any potential questions your students might ask.
14.3.2.1. Leadoff Questions (LOQ). The first item listed on the plan under development is the concept or principle you are teaching. How can you best lead your students to an understanding of this concept or principle? The logical reasoning process starts with the LOQ. 14.3.2.1.1. To develop a concept, you might start the discussion with an LOQ that calls for a definition of the concept, such as "How woul d you define prejudice?" If you have already provided the students with a definition in your introduction, your LOQ might focus on the attributes (characteristics) of the concept, su ch as "What behaviors might we expect from a person who is prejudiced?" Followup and spontaneous questions will be used to further develop the concept. 14.3.2.1.2. To develop a principle, you might rephrase the principle as a how or why LOQ. For instance, if your main point was "An authorit arian leadership style is appropriate in crisis situations," your LOQ might be "Why is the author itarian leadership style appropriate in crisissituations?" This question will identify supporting points for development through spontaneous and followup questions. 14.3.2.1.3. Another acceptable approach for developing a principle is to ask an LOQ based upon a key supporting point. If this approach were used with the principle just cited, you might ask "How would you describe the authoritari an leadership style?" Additional supporting points would be developed which contribut e to development of the principle.
14.3.2.2. Followup Questions (FUQ). Once your LOQ is identified, you must develop appropriate FUQs. If a concept is being developed, FUQs w ill ensure that characteristics, examples, and nonexamples of the concept are de veloped. When principles are taught, FUQs will be planned for key supporting points important in the logical deve lopment of the principle. Since guided discussions are normally conducted at the comprehension le vel, devote little time to FUQs written at the knowledge level unless they are necessary for esta blishing a base for the discussion. Instead, use questions dealing with "how," "why," and "what ef fect." Knowledge-level facts can often be given in the introduction or in a handout.
14.3.2.3. Anticipated Responses (AR). Notice the right column on the sample plan ( Attachment 4 ). By listing anticipated responses, the instructor is forced to think about possible answers students might give. Leave space in the right co lumn after the ARs of each subpoint so you can write in additional responses of students for us e in your summaries. As an alternate recording method, you may wish to list the student responses on an overhead transparency or a sheet of paper for use in summarizing. If you intend to us e the lesson plan again, you can add the best student responses later under ARs.
14.3.2.4. Transitions. Since the instructor is working toward desired learning outcomes in the discussion, major transitions can be planned in advance of the discussion. A good transition reviews the key point just made, reminds the students of the central idea (lesson objective), and introduces the next point. (Note the transitions on the sample lesson plan at Attachment 4 .) Transitions written on the lesson plan serve as a good memory jogger each time the lesson is taught.176 AFMAN36-2236 12 NOVEMBER 2003

14.3.2.5. Planned Summaries. If the guided discussion has a single conclusion you are working toward (single objective), then comprehension-level summaries will occur after main points in the discussion. If the lesson has more than one objec tive (multiobjective), then comprehension-level summaries will occur after each main point and objective. These comprehension-level summaries are not tightly planned but develop primarily fr om student contributions. However, you can mention planned ARs that were not discussed. Als o, you may add additional support material in the summaries in the form of examples, statistics, and expert testimony to aid students in achieving lesson objectives.
14.3.2.6. Conclusion. The conclusion to a guided discussion has three parts as discussed in Chapter 6 . Remotivation and closure are planned, as are certain aspects of the summary. When asingle objective is taught, a comprehension-level summary is required in the conclusion. Additional support material can be introduced from the lesson plan to ensure achievement of the lesson objective. Once the lesson is planned and organize d, the instructor is ready to conduct the lesson.
14.4. Conducting a Guided Discussion. A number of skills are involved in conducting a guided discussion. The most important ones follow:
14.4.1. Controlling. How active should your role be in the discussion? You can do most of the talking (which results in overcontrol), you can remain almost silent (which usually results in undercontrol), or you can find an active intermediate role. As the instructor, you define your role. However, the more you talk the less students can talk, which lessens the benefit of using the guided discussion method. Thus, it is wise to limit your role to th at of questioning, clarifying, probing, and summarizing. 14.4.1.1. As discussions with new groups begin, st udents will look to you for a definition of your role. While you may tell them what your role willbe, your actions will be closely observed. Eye contact of group members will give you good feedback on your control. If all eyes turn to you (as the instructor) when silence occurs, perhaps you ar e exercising too much authority in the discussion. Analyzing the lines of communication (soc iogram) will also provide valuable feedback. Have a student plot the lines of communication on a piece of paper. If the lines of communication usually flow back and forth between you and the students rather than among the students then you are probably overcontrolling the discussion. 14.4.1.2. If you undercontrol the guided discussion, you may never reach your desired learning outcomes. Minimal instructor part icipation may result in excessive disorganization or aimlessness. Aimless discussion can quickly lose the respect of students and have a negative effect on their motivation to learn. To prevent this deterior ation in the discussion, the instructor must judge quickly and accurately when intervention is necessary. 14.4.1.3. As a guideline, remember the desired lear ning outcomes to be achieved in a designated period of time. Proper control w ill ensure maximum student participation in achieving each of these desired learning outcomes in the time allotted.
14.4.2. Questioning. To be successful in conducting a guided discussion in which desired learning outcomes are achieved, an instructor must have anunderstanding of questions and the role they play in this method. (See Chapter 11 for detailed coverage of questions and their uses.)
14.4.2.1. Leadoff Questions. The instructor opens the discussion by asking a prepared LOQ and then waits for responses. The students must be given a chance to react. The instructor has the answers in mind before asking the question, but a student must think about the question beforeAFMAN36-2236 12 NOVEMBER 2003 177

answering. Avoid asking two questions at once, questions requiring a yes or no response, and complex questions that leave students confused. 14.4.2.1.1. The discussion begins with the first student response to the LOQ. The instructor should listen attentively to the ideas, experi ences, and examples contributed by the students. As the discussion proceeds, the instructor guides the direction of the discussion, stimulates the students to explore the subject in depth, and encourages them to discuss the topic in detail by using both planned and unplanned (spontaneous) FUQs.
14.4.2.2. Spontaneous Questions. As the name implies, these questions cannot be planned, but they serve a very useful purpose. When an LOQ or FUQ is asked, the instructor can never be sure of the response students will give. The discussion ma y start to wander in ways the instructor had not anticipated and the spontaneous question canget the discussion back on track. While the instructor hopes that indepth comprehension-le vel support material will develop, the opposite may be true. Consequently, to develop the topic properly, the instructor uses spontaneous questions. Direct questions, which bring reticent students into the discussion, may be spontaneous and are based upon an observed need. Spontaneous questions al so work well when the instructor wishes to probe for more information or to seek clarification on a point.
14.4.3. Summarizing. One of the key skills involved in conducting a discussion is summarizing. You should plan for summaries to occur after discu ssing main points and objectives. Unplanned summaries occur because of need and may be used period ically to designate rela tionships, get students back on track when the discussion wanders, or to transi tion between subpoints. No matter when you summarize, use ideas developed by the group that support achievement of your objectives. Try to use the students' words and show how their id eas developed the learning outcomes. 14.4.3.1. Not all student contributions will support a lesson objective, nor will you have time to cover each student response. Therefore, avoid cr editing students by name for their contributions ormaking quality judgments about the various responses. Such comments may cause certain students to wonder why their contributions were ignored or treated differently. Consequently, they may participate less and ope n discussion is stifled. 14.4.3.2. After summarizing student contributions, you may need additional support for a main point or objective. At this point you should introduc e new support material to aid in achieving the learning outcome intended.
14.4.4. Knowing When to Intervene. As you develop your skill as a guided discussion leader, you will become more skillful in knowing when to interv ene in your role as ques tioner, clarifier, prober, and summarizer. These following guidelines might prove helpful:
14.4.4.1. If several contributions in a row indicate a drift away from the main point or subpoints, then a brief interim summary followed by a spontaneous or followup question may be appropriate for getting back on target. 14.4.4.2. If pauses between contributions become too long, there are three possible causes: the participants may be conf used, your question was not clear in st arting the discussion, or perhaps the students have no additional contributions in the ar ea. At this point you may want to find out the reason for the pauses by asking a spontaneous question.178 AFMAN36-2236 12 NOVEMBER 2003

14.4.4.3. A student who mistakenly states something as a fact may have a negative effect on the discussion. If other students do not correct this error, you should intervene. A probing question or asking a student to clarify what was said may suffic e, but the error should not be allowed to stand.
14.4.4.4. Watch to see if serious fallacies in r easoning are going undetected. If so, the fallacies should be pointed out before they have a chance to damage the validity of the discussion. For instance, the students may be divided along two extr emes on an issue and be unable to see that a reasonable course of action exists between the extremes. Or students may omit important facts ordistort and arrange evidence in a special way to make a point (stacking the evidence). While analogies might be helpful in suggesting new ideas orin illustrating a point, students should not usethem to prove a point (misuse of analogy). Generalizations may be reached based upon insufficient evidence or instances supporting an opposing point of view may be ignored (hasty generalization).
14.4.5. Dealing with Nonparticipation. What happens if you ask your LOQ and nothing happens? Several possibilities exist. Perhaps you have not cl early established a context for the question in your introduction. The question may be double-barreled, too long, or confusing. Students may not understand their role in the discussion or they may be a fraid to say anything because of an authoritarian role you played in a previous discussion. The question may simply be one that requires some thought before giving a reply. Try waiting out the silence for several seconds. At that point, you may need to ask the group what the silence means, rephrase th e question, or make a few content comments about the subject and follow up with a new question. If th e question does not work successfully, be sure to change it on the plan before conducting th e discussion again with another class. 14.4.5.1. Another problem is uneven distribution of responses. Certain students may refrain from discussing a particular topic or any topic pres ented for discussion. Other students may tend to monopolize the discussions. Through awareness, th e instructor can take certain actions to gain more participation of nonparticipants. The directquestion and relay ques tions to nonparticipants can be used to draw them into the discussion. Byavoiding eye contact with the active participants when a question is asked and focusing on others, th e quieter members can be encouraged to reply. 14.4.5.2. Nonparticipants can often be encouraged to participate when you give them positive reinforcement for contributions (when such rec ognition is appropriate). Tying the quiet student's ideas to other ideas and referring to these ideas often is a kind of positive reinforcement repeatedly found to promote achievement and favorable attitudes. Head nods and positive remarks such as "Good point" and "That's interesting!" reinforce a person's contributions, while frowns and withholding reinforcement suppress the responses of group members.
14.4.6. Dealing with Hurt Feelings. As you gain more experience with the guided discussion, you will become increasingly aware of student feelings in a discussion. Instructors should ensure that student feelings are not hurt unnecessarily. Sarcasm and ridicule have no place in a guided discussion, whether from the instructor or other students. By example and guidelines (if necessary), the instructor should rule personal attacks out of order. On the other hand, group members should be encouraged to accept criticism or correction without taking it persona lly. Mature students should be able to learn to accept valid criticisms of their ideas without bei ng irrationally defensive about the ideas or themselves. Instructors can help by acting in a rationa l and nonemotional way to criticisms of their own ideas.AFMAN36-2236 12 NOVEMBER 2003 179

14.4.7. Avoid Biasing the Discussion. Instructors must be constantly reminded of the power they have in the classroom. Casual mention of their own opinions, attitudes, and beliefs will influence some class members. Over a period of time, students ha ve honed their senses to be able to detect what the instructor wants. The instructor must be care ful that opinions expressed do not close discussion or force student conformity.
14.4.8. Avoid Encouragement of Yielding. Groups exert enormous pressure on members to conform. The instructor and one group of students may agree on something and, through group pressures, try to get the other students to conform. Guided discussion leaders should encourage the expression of minority points of view. If the instructor insists on unanimity of opinion, then minority views will be stifled and the discussion will suffer.
14.4.9. Avoid Withholding Crucial Information. The discussion may suffer at times from the lack of a fact, definition, concept, or principle. It is not wrong to share your expertise with the group when the discussion would profit by it. However, you should not use the expertise if your only goal is to enhance your own image of competence in the eyes of students.
14.4.10. Avoid Sticking to a Dead Topic. When students have exhausted a topic, the instructor should be prepared to move on with a short inte rim summary or a new question. A dead topic is one that no longer helps achieve your objective. Signs of a dead topic include periods of silence, repetition of points already made, boredom, or inattention. Normally, you should not let a discussion get to this point before you intervene.

14.5. Post-Discussion Actions. After the lesson is complete, you should go back to your lesson plan to make notes and then evaluate the discussion.
14.5.1. Annotating the Lesson Plan. Annotate your lesson plan while the material is still fresh in your mind. If unanticipated materi al entered the discussion, make note of it on the plan. If certain questions did not work well, rewrite or eliminate them. Perhaps you became aware of points or definitions that would have aided the discussion if they had been covered or clarified in the introduction. Now is the time to make such annotations or corre ctions on your lesson plan. It is important to establish records on the plan that allows you to improve your instruction.
14.5.2. Evaluating the Discussion: 14.5.2.1. After the discussion has been completed, you should take time to evaluate its effectiveness in achieving your lesson objective. Did stude nts have the required knowledge to discuss the topic adequately? Should the readi ng assignment be changed, or is there some better way to get students to the knowledge level before the discussi on? If you or another student maintained a diagram of the discussion, what do the lines of co mmunication say? Such evaluation may be subjective but still valuable in improving instruction. 14.5.2.2. More formal evaluation comes when tests are administered. When you develop test questions, you will have direct evidence of how well the guided discussion achieved your lesson objective. Such evidence will also allow you to make changes to the lesson plan to help you reach your objective.
14.6. Summary. The guided discussion is a teaching method inwhich students learn by sharing experiences and opinions with each other. The instructor carefully plans the lesson to reach desired learning outcomes and then guides the discussion through questioning and summarizing.180 AFMAN36-2236 12 NOVEMBER 2003

14.6.1. Consider student needs when planning the gui ded discussion. The discussion method is effective in teaching cognitive and affective objectives. It helps students develop skill in group reasoning and problem-solving and in facilitating changes in attitudes and behavior. Instructors should use the guided discussion when group size is from 2 to 20 st udents and when there is sufficient time for all students to participate. Arrange the classroom to allow maximum eye contact among students. Each student should have a basic knowledge of th e subject before the discussion begins. 14.6.2. The guided discussion is organized to encour age maximum student participation. The instructor tells students what will be discussed in the overv iew, gives any necessary definitions or concepts, and encourages participation. Carefully planned LOQs and FUQs then elicit this participation. The instructor should also anticipate possible studen t responses and include these in the lesson plan. Although remotivation and closure are planned, the fi nal summary is developed using inputs from the students. 14.6.3. Certain specific skills ar e needed to conduct a successful guided discussion. The instructor should control the group so that discussion flows among the students. LOQs and FUQs are asked as planned, but spontaneous questions should also be used frequently to clarify student responses and eliminate errors in fact and reasoning. The instruct or should avoid biasing the discussion or withholding necessary information from the students. Encourage all students to participate and give positive reinforcement to student inputs.AFMAN36-2236 12 NOVEMBER 2003 181

Chapter 15

THE CASE STUDY METHOD

15.1. Introduction. The case study method presents students with real-life challenges. It helps bridge thegap in the classroom between theory and practice by using a real world situation that focuses on previously learned concepts and principles in realistic situations.
15.1.1. Some Air Force schools teach manual tasks—the practice of a job—such as missile operations or vehicle maintenance. Usually, the more specific the task to be learned, the more closely the school tends to fit job requirements. Theory and practice come together. Engine mechanics, for instance, are trained on actual engines or mock-ups in shops similar to those on the job.
15.1.2. Other schools teach abstract or mental skills—the theory of a job—like strategies for limited warfare or an aspect of professional military education. In schools where the subjects are abstract, however, students tend to be less involved with the sp ecific tasks they will encounter later on the job. Theory and practice may be far apart. The curricu lum of an NCO leadership school, for instance, does not relate directly to NCO specialty codes. Thus , bridging the gap between school concepts and reality is a challenge to many Air Force schools or curriculums.
15.2. Using Case Studies. Using case studies is one approach in teaching to meet this challenge. In the case study method (hereafter referred to as the case method), students meet a simulated, real-life situation in the classroom in order to achieve an educati onal objective. The case study, typically written, is an account provided to students, usually as a homework a ssignment, that serves as a basis for class discussion. Usually, a case will describe a problem already fa ced by others in a given fi eld. The greatest value of the case study is that it challenges students to apply what they know and comprehend to a realistic situation. The case method takes students out of the role of passive listeners and makes them partners with the instructor in applying the concepts and principles under study. The method enables students to think, reason, and employ data in a logical fashi on, just as they will in a real job. 15.2.1. The case method is a flexible teaching approach. It can be the basis of an entire curriculum, a course, or simply an individual lesson. While the ca se method is versatile, there are times when other teaching methods are suggested. The case method may not work with immature students or with large classes. It is not applicable where students are l earning a procedure or a single solution that has been accepted as "correct." Case studies do not lend themselves to developing objectives at the knowledge level, where the lecture and readi ng are usually more efficient. Once the knowledge level of learning has been achieved, the case study is an excellent way for students to progress to the higher levels.
15.2.2. Variations of the case method were first used in teaching medicine and law, but over the years, the case method has been widely used in the study of business, management, and education. It can be used in almost any learning area that can draw cases from real situations for study and discussion.
15.3. Teaching Value and Limitations: 15.3.1. A principle advantage of the case method is to provide students experience in solving problems. Many students can repeat isolated facts or princi ples, but they have difficulty relating these facts and principles to other situations. The case method provides an opportunity for them to gain experience in making decisions and in working with othe r people. It exposes them to a number of ways to182 AFMAN36-2236 12 NOVEMBER 2003

solve problems. As they learn various problem-solv ing techniques, they make their mistakes in the classroom and not on the job where an error is much more costly. 15.3.2. Students learn by encountering situations from reallife instead of listening to lectures or reading theoretical descriptions of the solution to problems. By interacting with others, they learn to respect the opinions of others. Because the case may notlist all pertinent facts, the students also learn how to cope with ambiguity and uncertainty. As w ith real life, we sometimes ask questions and get answers; other times, we can only ask questions.
15.4. Typical Case Applications. Because of its versatility, the case lends itself to a wide variety of teaching conditions.
15.4.1. Case Courses. Some schools use the case method exclusively, especially graduate schools of business. The assumption is that the best way to pr epare for a business career is to have experience in analyzing data and making decisions. Students in these courses are given complex cases to analyze and solve. The quality of the analysis and the reasoning behind the suggested decisions are often moreimportant than arriving at a single solution. In the classroom, students participate in collective analysis and decision-making. As more and more cases ar e analyzed, students begin to form generalizations they can apply to new situations. Thus, case studies substitute for a period of on-the-job training. 15.4.1.1. Normally in the case study course, concepts and principles are not taught directly. Instead, they emerge gradually as students are forced to formulate theories to support their case decisions. Because these theories arise from practical work with problems, students remember them better and are able to recall them for similar solutions. 15.4.1.2. The case study course is usually c onducted using group problem -solving techniques in open-class sessions. But the case may serve as anout-of-class written assignment, with students asked to bring solutions to class along with a ra tionale for their decisions. These solutions may form the basis for class discussion, and may also be turned in for the instructor to grade. A variation like this may overcome the need for rela tively small classes in the pure case method.
15.4.2. Capstone Method. A case or a series of cases is often us ed at the end of a body of instruction to help show the application of the course content. Often a course will proceed by lectures, discussions, and other more-used methods. Toward the end of the instruction, students will apply what they have learned to a series of cases specifically designed to support th e course objectives. This capstone method is particularly appropriate when students start at lower levels of learning (such as the knowledge level) and are brought gradually to the higher levels (such as the application level). The capstone method is particularly appropriate in Air Force l eadership schools where readings and lectures can supply the basic material and case studies can allow an opportunity to practice the theory as a capstone experience.
15.4.3. The Short Case or "Problem." The case also has applications at lower levels of learning. This technique is called "use of the problem in te aching." A realistic problem situation, often a page or less in length, is used as a discussion vehicle. The instructor plans the lesson much like a guided discussion. For example, the objective of the lesson ma y be to have students comprehend a principle of management called span of control. The instructor might introduce a situation in which a supervisor failed to take this principle into account. A serious accident or a dramatic event might then result.15.4.3.1. The primary object is not to find a "correct" solution to the problem posed, but to understand the principles involved. The problem posed should be sufficiently interesting and difficult toAFMAN36-2236 12 NOVEMBER 2003 183

involve all the class members for the time allotted. Because the written problem provides the discussion vehicle, the class can also be broken up into smaller discussion groups. 15.4.3.2. A variant of the short case can also be used in group dynamics exercises where the emphasis is not on solving the problem, but on th e interaction of group process. Similarly, the short case can be used to demonstrate a decisionm aking process where the process is more important than the solution arrived at. A series of s hort cases or scenarios often works for reinforcing affective objectives; for instance, in human relations or equal opportunity exercises.
15.5. Types of Cases. Teaching cases are situations from real lif e. There is no single format for cases— in fact, there can be great variation in the types of cases. They may vary in length from a paragraph or two to many pages. For convenience sake , types of cases will be discusse d in two categories—mode of presentation and case format.
15.5.1. Full Text. A full text case contains all the information the student will need to deal with the situation and requires access to no other source. Busi ness school cases, for instance, often illustrate a real situation faced by managers in the past. Theycontain the problem to be addressed, how the problem arose, the organization's structure, and the employees involved in the situation, as well as their perceptions at the time of the original incident. Ca ses of this type are usua lly of 8 to 15 pages in length, take considerable time to analyze, and are usually the basis of a case course. Students are required to read the case, perform an analysis, make a decision, and support that decision before their peers. The instructor is particularly interested in the student's ability to perform an analysis and make a decision.15.5.1.1. Another full text case is the abbreviated case, which is much shorter—from one paragraph to several pages in lengt h. An obvious consequence of the length is the diminished content. Since it is shorter, the focus is usually built in and the solution or solutions may be more limited. The abbreviated case may deal with one "problem," or it may define a scenario with a series of limited choices. The abbreviated case may lend itself to lower levels of learning.
15.5.1.2. A third variation of the full text case is the unsifted case. Although all the data required by students is presented at one time, it is given in an unorganized form (simulating an in-basket exercise). Extraneous information may be included. The student's task is to select and arrange the information into some meaningful pattern for action.
15.5.2. Partial Text. With partial text cases, students are gi ven only limited information about the situation under study, and part of their job is to find or ask for the additional information they need for solution.
15.5.2.1. In the incident-process case, the instructor presents a brief incident and the students are required to take some action, make a decision, or suggest recommendations. The instructor usually provides additional information onlywhen the students request it. Partial text cases are designed to teach students analysis and problem solving and al so how to ask the right kinds of questions. The situation parallels real events because we ofte n make decisions based on partial information. The incident-process case prepares students for this contingency.
15.5.2.2. In an interactive case , students also receive limited information. Then they interact with other sources for additional data, return to clas s and ask for additional information, or receive another segment of the case from the instructor, re turn to interaction, and so on. Other sources include additional readings, interviews, and library and/or internet research. This process is simi-184 AFMAN36-2236 12 NOVEMBER 2003

lar to doing a research project. It is an artful blend of dealing with reality while acquiring additional knowledge and skill at the same time. 15.5.2.3. Like other partial text cases, the sequential case begins with limited information. The situation unfolds in succeeding installments of information to the students, much like a command post exercise. At the end of each installment, student s decide if intervention in the situation at that point is called for or not. Some incidents resolve themselves and intervention will only aggravate the situation. Other incidents continue to deterior ate and might even become unresolvable if intervention comes at an inappropriate time, or even too late. The sequential case is particularly suited for personnel management training.

15.6. Mode of Presentation: 15.6.1. Thus far, we have assumed that all cases are written. While the major ity of cases do come in this format, other modes of presentation add variety, drama, and realism to the case class. Movies and visual aids can offer the participants the incident and the emotions of the case in a dramatic and life-like way. Typically the film will unfold the situa tion to a decision point, then the students take over. After some individual or corporate decision ha s been made, the presentation can be continued by presenting more information by using the outcome as an illustration and as a basis for further discussion. Or, the case can be closed, whichever suits the student and instructor needs. 15.6.2. Videotape cases have the potential of increa sing realism by having the original case participants give their perceptions in their own words. Ho w the original participants saw the case is often more critical and important to the student than the interpretations of these perceptions by a case writer. Another step toward case realism is to have a live presentation where the participants come in person before the class to recreate the case experiences and respond to questions. A variation is to have students meet with the case participant, perhaps in the actual case environment.
15.7. Case Methodology. More is required in teaching with the case method than simply reading the case and asking a few questions in class. Proper us e of the case method requires conscientious preparation, presentation, and followup.
15.7.1. Preparation. While there is no "best" way to approa ch case preparatio n, some generalizations can be made. The instructor should identify th e specific learning objective for the class. Here also, some appraisal of the relationship between the lesson and the rest of the curriculum is appropriate. Consider the instructor who has taught the pr inciples of management to students during a 2-week period. The objective now is to see if the students can apply these principles to a given situation. Using the case in this way makes it a "capstone me thod" because the case reinforces and extends the teaching that has gone before. Most Air Force case teaching uses this approach.
15.7.1.1. Sources of Cases. Once instructors decide to use the case method, where can they find cases? They can either use prepared cases or write original ones.
15.7.1.1.1. Prepared Cases. Because of its long involvement with the case method, the Harvard Business School, through the Intercollegiate Case Clearing House, offers the most centralized source of prepared cases. Cases can be ordered from the published catalogue for a nominal fee. The Air University Library also ha s an extensive collection of cases that have been used in AFIT and in professional militaryschools. However, some of these cases cannotAFMAN36-2236 12 NOVEMBER 2003 185

be released for general use. The base or community libraries as well as the internet are also sources of cases.
15.7.1.1.2. Case Writing. Quite often the instructor is unable to find the exact kind of case to achieve a specific objective. The solution may be to write a case to fit the specific school need. The idea for a case may come in some current event, an article, periodical, or personal experience. 15.7.1.2. Case Selection. In evaluating a case for use, regardless of the source, we should ask ourselves five questions:
15.7.1.2.1. Is it realistic? A case is realistic if it describes an actual problem or situation, even if the organization or participants are disguised. Obviously fabricated or fantasy cases are unlikely to have credibility. If the case is not rea listic, it is hard to make the students' solution, decision, or action seem real. While it is theore tically possible to "invent" a case or use a composite of several situations, in practice such an approach is rarely as successful as dealing with an actual real-life case. 15.7.1.2.2. Is it meaningful? The case should be meaningful to students so they can identify with it in some way. They must be generally familiar with the problem either through their past experience or in the experiences they expect to face in the future. A case on a moonwalk, for instance, may not work for students who are not involved with space travel or who do not have the necessary technical background. 15.7.1.2.3. Is it challenging? The case should contain actual or potential controversy. If the case is one in which the solution is obvious, it should be rejected.15.7.1.2.4. Is it complete? Usually a case is complete within itself. It presents enough information so students can deal with the problems without reference to outside sources or the instructor. 15.7.1.2.5. Does it provide for a logical difference of opinion? A case study that prompts a single line of analysis may not result in produc tive discussion. An acceptable case should provide an opportunity for reasonable people to differ rationally.
15.7.1.3. Writing the Teaching Note. A good case allows both the instructor and student to achieve educational objectives—the instructor to re inforce general principles with specific examples and the student to gain experience from the pa st in a close-to-real-life role. The instructor who analyzed the case meets these objectives beforehand in what is called a teaching note. This may include essential details in the case, major issues , analysis of these issues, evaluation of the case characters (their relationships, goals, values), pr ior knowledge students need to work the case, and questions to be included in the lesson plan to guide the discussion. (See the teaching note in the sample lesson plan at Attachment 7 .) 15.7.1.3.1. The teaching note is not the solution to the case, but it should detail typical student activities the case will stimulate and define the specific student outcomes the instructor expects. It should also define the student level for which the case is written and relate the case to readings and preceding or subsequent classe s. The note should include at least an outline of the instructor's analysis of the case to show it is internally consistent and able to hold up under analysis. After writing the teaching note, inst ructors will often find that the case contains insufficient or misleading information; therefore, they may want to revise or amend it before186 AFMAN36-2236 12 NOVEMBER 2003

giving the case to students. For certain groups, the case may assume knowledge of a process or background material that may need to be summarized in an attachment. A case on discrimination in promotion, for instance, may depend on a detailed knowledge of the promotion system and its appeals, which the average social actions specialist may not have. 15.7.1.3.2. The instructor's preparation should then extend beyond the limits of the case to include political, economic, or social factors; po licies and procedures in effect at the time of the case; and concepts and principles alluded to but not explained in the case. The more expert the instructors are about a case, the greater the chance of teaching it successfully, especially if the subject area is unfamiliar or the details skimpy.
15.7.1.4. Lesson Format. Before the actual lesson planning begins, the instructor must select some logical sequence for analyzing the case. An an alytic format will often be inherent in the concept, principle, or objective the case is to reach. If so, the concept, principle, or approach may serve as the lesson objective and its format as ma in points in the lesson plan. These main points also become the procedures the students will use to develop the lesson during the class period. They also might be used as broad categories to be outlined on the chalkboard during class to help with lesson development. Figure 15.1. is an example of this process.AFMAN36-2236 12 NOVEMBER 2003 187

Figure 15.1. Lesson Plan Process.

15.7.1.4.1. Other examples of case lesson objec tives, main points, and associated board arrangements are in Attachment 5 . These examples are merely illustrations of possible approaches. Two different instructors teaching identical objectives an d using the same case might use completely different main points and an alysis procedures with equal efficiency and effectiveness. 15.7.1.4.2. The instructor is now prepared to develop the lesson objective and the lesson's main points. The objective is often stated atthe application level, although many cases lendthemselves better to the comprehension level. Examples of acceptable objectives are illustrated on Parts I of the lesson plans in Attachment 6 and Attachment 7

. 15.7.1.4.3. With a specific well-defined object ive and logical main points, we are ready to develop the lesson plan. Lesson Objective: "...for the students to highlight Gelatt's decision-making model in given situations." Main Points: 1. Analysis procedure to identify the problem. 2. Research resources to collect data. 3. Creativity in determining alternatives. 4. Assessment of possible outcomes of each alternative. 5. Assessment of probable outcomes of each alternative. 6. Individual choice of acceptable solutions.

BOARD ARRANGEMENT Identify Possible Probable ProblemDataAlternatives Outcomes Outcomes Choice At times, the case under consideration has no obvious fo rmat for main points or an alysis, but an approach will emerge from the instructor's own study of the case. For example: Lesson Objective : "...for the students to highlight the principl e of delegation of aut hority in management situations."
Main Points : 1. Past management training and experience to separate the facts and the assumptions. 2. Group problem-solving techniques to clearly define the problems. 3. Individual creativity to reach the best possible group solution to the problems.

BOARD ARRANGEMENT Facts Assumptions Problems Solutions188 AFMAN36-2236 12 NOVEMBER 2003

15.7.1.4.4. The introduction of the case lesson plan development differs only slightly from the other teaching methodologies. The attention and mo tivation steps are the same, with the major difference occurring in the overview. Here the students are given an opportunity to read the case if it is short or if they have not read it before coming to class. Students are told how the lesson will proceed and the point of view they should adopt. They may or may not role play. 15.7.1.4.5. The development phase elaborates on ea ch main point in the lesson plan through the use of well-formed preplanned questions. "Wha t are the facts in this case?" "Who is causing the biggest problem?" "Which of these vi olations has the most serious consequences?" Instructors who have prepared thoroughly will be able to transition from one main point toanother and from issue to issue within main points, to guide the class into critical areas for discussion, and ultimately to lead them to the overall lesson objective. 15.7.1.4.6. The conclusion summarizes the case, adds additional information, and relates it to the concept or principle to be illustrated. It th en proceeds to show how this same concept or principle can be used to deal with other sim ilar situations. The remotivation and closure steps are similar to those in other methodologies.
15.7.2. Presentation:
15.7.2.1. Guidelines. As with other aspects of the case method, there is no set procedure for conducting the class, but a number of general guidelines have worked well in past case sessions.
15.7.2.1.1. Refraining From Lecturing. The case method is inherently a student-centered approach. Keep instructor comments to a minimum and let the students do the talking.
15.7.2.1.2. Starting the Discussion. Some case leaders begin with the question, "What is the issue here?" then go on to, "What are the pertinen t facts?" Others begin with the more general question, "What action should be taken?" The appr oach depends on the inte llectual maturity of the students and with the subject matter.
15.7.2.1.3. Controlling Participation. The case discussion is controlled much like the guided discussion except that in the case, the instructor may feel freer to enter the discussion. The case instructor often keeps track of the discussion at the board so the entire class has avisual record of where the discussion has been and where it is going. The questioning techniques used in the case method are the same as for the guided discussion.
15.7.2.1.4. Being Nondirective. No single right answer often exis ts in case studies. It is more important to lead students toward sound applicati on of principles than to persist in an endless search for one right answer. In the body of the lesson, the instructor should, of course, "guide"the discussion. But imposing the instructor's views on the students, passing judgments on contributions, and arguing do little toward encourag ing independent thinking or achieving the lesson objective. Attempts to force a conclusion on the group often fail. Save instructor comments for the conclusion.
15.7.2.2. Case Roles. In the process of presenting a case, th e instructor and students have roles they are responsible for. Fulfilling these roles increases the probability of a successful lesson.
15.7.2.2.1. The Instructor's Role. As case leaders, how should we behave? We can dominate, control, and structure the discussion of a case and emerge from the discussion with our own solution. On the other hand, we can demonstrate a high degree of leadership skill in guid-AFMAN36-2236 12 NOVEMBER 2003 189

ing and involving students in the discussion and solution of the case. In either instance, we serve in the role of recorder, questioner, and occasionally as clarifier or expert.15.7.2.2.1.1. In our role of recorder, we provide direction and organization to the discussion of a case by writing ideas on a board as the students submit them. We also record problem areas and items for further analysis and solutions by the discussion group. To avoid confusion or distraction, we might list th e major elements of the selected decision or analysis process on the board. Under each cate gory, we might list specific student contributions. This task requires disciplined listening and undivided attention to every statement made during the discussion period. 15.7.2.2.1.2. One of the most important requirements of the case study method is the instructor's ability to ask good questions. Ques tions are the principal device for clarifying contributions to the discussion and for relating ideas to the problem under discussion. Another questioning technique in the transfer of ideas is to review past experiences as they relate to a new problem. Frequently, to avoi d student superficiality, a simple "why" question is used to confront a student's assertion and stimulate an orderly problem analysis. The instructor also has a responsibility to assure that the discussion is meaningful. We are the experts in teaching the subject and also in helping students express themselves. We should see that every student has an opportunity to participate. It is also our responsibility to establish rapport and maintain student interest in the case under discussion.
15.7.2.2.2. The Student's Role: 15.7.2.2.2.1. The case method of instruction may require more student preparation thanany other teaching method. If students do not prepare for class participation, they do themselves an injustice and deprive other stude nts of possible discussion. As minimum preparation, they must read and study the case thoroughly. If there is time, they might also refer to as many related references as possible. 15.7.2.2.2.2. During class discussion of the case, students should think reflectively and strive for cooperation rather than competition. They should share responsibility to contribute briefly and directly to the discussion and to assist in developing group concepts based on specific items of information. Thus, self-mot ivation is a significant element of the case study method. This motivation obviously enhances the learning process.
15.7.3. Followup. What we do after the class session is over is often as important as what preceded the period. If student participation is part of the course grade, we should establish the criteria for grading beforehand, make it known to the students, and ta ke notes while the impressions are still fresh in our minds. Even more important for future use of th e case, we should review the lesson plan and note corrections needed in the case, possible changes in questions, and new lines of thought or different methods of analysis brought out by the students. This keeps the case fresh and makes it a more refined tool for use with subsequent classes.
15.8. Summary. The case, properly used, initiates students in to the ways of independent thought and responsible judgment. It faces them with real situations (not hypothetical); it places them in the active role, open to criticism from all sides; it puts th e burden of understanding and judgment upon them; and it gives them the stimulating opportunity to make contributions to learning.190 AFMAN36-2236 12 NOVEMBER 2003

15.8.1. In the problem-solving environment of the cl assroom, the students develop skills in communicating their ideas to others. At times, they may a dd to the ideas contributed by other members of the group, and, at other times, they may take exception to their peers' ideas. Both actions require effective communication techniques, and both involve a type ofinteraction that leads to consensus and understanding. 15.8.2. The case study method increases the student's ability to appreciate other points of view, to explore and discuss differences of opinion, and, eventually, to reach an agreement. It forces students to think analytically, constructively, and creatively , and gives them the satisf action of participating in an orderly social relationship with others.AFMAN36-2236 12 NOVEMBER 2003 191

Chapter 16

THE TEACHING INTERVIEW METHOD

16.1. Introduction. As part of our daily lives, conversation occurs when two or more people exchange information or viewpoints. An inte rview is a special type of contro lled conversation aimed at specific objectives. Interviews vary broadly in content accordi ng to their purpose or the situation in which they are conducted. For example, we use personal interviews wh en we evaluate performance and when we reassign, retain, or separate. Managers use interviews for giving orders and handling complaints or reprimands. Interviews are also used in sales, counseling, and information gathering, including polls and surveys, and for such specialized purposes as medica l diagnosis, criminal investigation, and legal inquiry. The teaching interview is another type of inte rview: its main use is to promote learning.
16.2. Background: 16.2.1. Much like the guided discussion, the interview as an educational technique started when thefirst student asked the first question and the first teacher replied. Tradition credits the Greek philosopher Socrates with a pupil-teacher dialogue that replaced the one-sided lecturing of the Sophists, a class of professional teachers in ancient Greece. Still in use today, Socratic dialogue relies on the assumption that each person has a storehouse of knowledge and understanding that can be discovered through questioning techniques. Student s who participate in this exchange of ideas and opinions learn by relating what they know to what they don't know. The dialogue is the basis of most discussion methods of learning. 16.2.2. The teaching interview is a dialogue in a classroom situation that pairs a skilled instructor with a recognized expert. The expert has speciali zed knowledge or a broad understanding of a subject; the instructor draws out that knowledge and unders tanding by means of questions. The students learn by observing the interview and asking questions.
16.3. Uses of the Teaching Interview. The teaching interview can be used as a primary method for achieving planned learning objectives . The instructor's job is to elicit espouses from the guest expert that allows the students to reach these desired learning outcomes. Many schools alrea dy have a guest speaker program. Because guests may depart from the planned topic or lesson outline, they often do not help to achieve planned lesson objectives. Reasonable control over the actions of a guest speaker is the primary advantage of the teaching interview. 16.3.1. The teaching interview may also be used to supplement the instructor's knowledge of a specific subject area. In the Department of Defense, we often are asked to teach subjects in which we are not completely expert. When the subject matter is very broad, an outsider may help cover our knowledge gap. For example, management courses could easily profit from the interview of an expert who has had experience in conference leadership. In e ducational fields, experts who have worked with motivating adults could contribute significantly to an understanding of the learning process. Professional Military Education schools frequently benefit by bringing management experts to the classroom and interviewing them for the students. 16.3.2. Another use of the teaching interview is curriculum enrichment . The appearance of an expert or authority enhances the credibility of a classroom difficult to duplicate by the average teacher. The teaching interview lesson may also be used to ad d variety to the curriculu m. For example, if the192 AFMAN36-2236 12 NOVEMBER 2003

instructor usually lectures, a teaching interview can be an exciting change of pace to the regular routine. 16.3.3. The teaching interview can also be a motivational device . It can be used to introduce a block of instruction and gain the students' interest. For example, a physician might introduce a series of lessons on first aid, or a former prisoner of war c ould stimulate interest in the Code of Conduct. 16.3.4. The teaching interview can also be used as a problem-solving activity . The instructor can interview an expert in a particular area, presentin g a problem to the expert for solution. During the course of the interview, the expert can present the solution to the problem.
16.4. Initial Planning Factors. The instructor must examine the cu rriculum closely to determine the need for interview lessons. The inte rview lesson is best suited for exploring attitudes, experiences, and opinions. Generally speaking, teaching interview lesso ns should be planned at the comprehension level within the cognitive domain and at the lower levels of the affective domain. The instructor should also make a value judgment about the expert's ability to contribute to the overall learning experience. 16.4.1. An interview can examine an expert's reasoni ng process and the rationale used for arriving at conclusions. The interview probes the "whys" and "hows" of a thinking mind. The interview lesson discusses "the importance of...," "the reasons for...," and "the value of..." particular events or actions rather than the events or actions themselves. These valuable additions to the learning process influence the focus of the lesson objective. The teachi ng interview lends itself to teaching both concepts and principles. 16.4.2. After stating the lesson objective and selecting the teaching interview as the method, we must select the expert to be interviewed. The first step in any sort of research is to explore our own resources. So, the first step in selecting an expert is to examine our own faculty or colleagues. The best available resource may be in the next office. Military bases, local communities, and nearby colleges often provide experts for interview lessons. 16.4.3. Select a recognized expert if one is available. Students respond to people they acknowledge as experts and generally accept their remarks readily. Since students frequently accept the expert's testimony at face value, you can save time by not having to develop support materialto substantiate his or her ideas. 16.4.4. When selecting a prospective expert, find out all relevant information about the person. Make sure the expert can set forth views with candor and frankness and clearly explain the basis for opinions. If you discover the person generally thought to be anexpert really is not or that his or her personality or style of presentation might be a se rious barrier to learning, find someone else. 16.4.5. Spend time researching the subject that the interview lesson will cove r. If the expert has published works or made public statements, examine th em to gain insight into the views expressed. If you have a thorough grasp of the expert's opinions and viewpoints, you can choose the ideas with the greatest value to the students. 16.4.6. You are now ready to narrow your research to the subject of the interview itself. What specific topics will be covered? What sorts of questions will prompt the best answers? How should you react to the responses of the expert? Obviously, you do not need to know the subject matter of the interview as well as the expert. Perhaps a method other than the teaching interview should be used if you have as much expertise as your guest expert.AFMAN36-2236 12 NOVEMBER 2003 193

16.4.7. The last step in the preliminary planning of the teaching interview is to prepare a logically organized outline of the lesson. (See Part I in Attachment 8 .) Consider how the expert's knowledge can best be used to satisfy the overall lesson objective. As you prepare this outline, you will probably discover that some limitation of the subject, as well as the lesson objective, is necessary. When you consider the time allotted for the lesson, the depth of detail, and the degree of understanding desired, you will probably find that you should limit the scope of the interview. 16.4.8. The outline will also suggest question areas. Place yourself in the position of the students and decide what they need to know in order to increase their understanding of the expert's attitude or experience. You need not write out specific questions fo r the interview at this point; instead, establish in your own mind the general areas of questions to be covered.

16.5. Coordination. Three major coordinating steps precede th e actual teaching interview lesson. Two of these steps involve meetings between instructor and expert, while the other deals with the students themselves.
16.5.1. Preview the Subject With the Expert. The first coordinating step involves interaction between the instructor and the expert. The aim of the initial contact should be the previewing of the subject of the interview, where the essential facts and specific topics or subject areas to be covered are discussed. However, you may not want to give the expert the actual questions you will ask since you want the interview to be spontaneous. A common outline of the lesson may be prepared with the expert. This can serve as a tentative guide for the interview itself. Decide upon any visual aids you will prepare or provide and whether they are for your own requirements or for the guest to use. 16.5.1.1. One of the most important considerations in the preview is to help the expert gain an appreciation for which areas of subject content will be covered. You should add or delete items that either the expert does not feel qualified to discuss or may wish to avoid in public discussion. This will help to eliminate any potentially embarrassing situations that might inhibit a feeling ofmutual trust. Remember that an interview lesson is not an interrogation in which the expert is badgered into an admission of guilt or error, nor is the interview lesson an opportunity to make the expert appear foolish, ignorant, or upset. 16.5.1.2. Mutual considerations during coordination will lead to good rapport, which is essential to the successful outcome of the interview less on. If you cannot arrange a face-to-face meeting, arrange the topics for the interview by correspondence or by telephone. Building rapport and establishing a good working relationship will have to wait until shortly before the lesson begins.
16.5.2. Announce the Visitor's A ppearance to the Class. The second coordinating step involves the students themselves. The interview should not be a surprise. Motivate your students by announcing that a guest will be visiting the class. You may provide the students a short biographical sketch of the expert or a brief outline of the lesson to be pr esented. This material will help arouse student interest and curiosity and may be highly motivational as well. Students may become more involved if you ask them for questions to be used during the interview. When the students know who is coming, what the expert's qualifications are, and what the subjec t is, the interview has a better chance for success.
16.5.3. Meet With the Expert. The final coordinating step is a br ief meeting between the expert and instructor shortly before the actual interview pr esentation. This final meeting gives the expert a chance to become physically adjusted to the interv iew situation and to see the classroom and its equipment. It also helps develop informality and cordiality that are particularly important, especially if the194 AFMAN36-2236 12 NOVEMBER 2003

expert is meeting the instructor in person for the fi rst time. The instructor should put the expert at ease and establish a relaxed and permis sive atmosphere for the interview lesson. In addition, it allows for any last-minute changes in the interview outline.

16.6. The Interview Lesson:
16.6.1. The interview lesson is not a series of short speeches, nor is it solely a question-and-answer period. (See the sample interview lesson plan at Attachment 8 .) We only need to recall the informality and spontaneity of the popular TV talk shows to understand the atmosphere sought in a teaching interview. 16.6.2. The classroom arrangement is as important as in other methods. You and the expert should have enough desk or table space. Students should be able to see and hear both of you without effort. You may choose to be seated to start the less on or you may remain standing depending on which is more comfortable and appropriate . Normally, you should sit after the questioning begins. If possible, arrange a "warm-up" session in private with the expe rt just before the interview lesson. If this is not possible, begin the interview with easily answeredquestions geared toward opening up the subject and the person, such as "When did you first become interested in ecology?" or "What led to your interest in military history?" A preliminary conversation is a smooth way to ease the expert into the main thrust of the lesson, while avoiding mere social pleasantries.
16.7. Introducing the Interview Lesson. Like any other well-organized lesson, the introduction of an interview lesson focuses attention on the subject, prepar es students to recognize the value of listening, and provides them with a good idea of how the interview will proceed. Introduce the guest yourself so you can stress the pertinent qualifications, but keep the intr oduction simple and take care not to embarrass the guest by overstatement. Make sure you have accurate details. 16.7.1. In the introduction, announce that you will have a question-and-answer period at some point during or following the lesson. It is also critical that you present a thorough overview. Do not forget that this overview is for the students and the expert. The overview gives the students a "road map" for the lesson and serves to remind your expert of exactly what is to be addressed. 16.7.2. Students normally know you have invited a guest to class, and they will be anxious to meet and hear from the expert. Therefore, you should try to involve the expert in the lesson as early as possible. Be brief in your attention and motivation step s, while not slighting their importance and effect. Introduce the class to the expert if you feel it will enhance the setting. This is an optional procedure, but it often helps in channeling th e expert's comments toward studentneeds. The expert's involvement in the lesson should start as an integral part of the introduction.
16.8. Conducting the Interview. The development section of the interview lesson centers on the conversation between the instructor and the expert. The instruct or's task is to serve as a stimulus to conversation by asking questions that bring out ideas from the expert in support of the lesson objective. As in theguided discussion, you should plan the lesson so that it will develop along certain lines to satisfy the specific learning objective. For this reason, carefully plan the leadoff question for each main point. When you ask the leadoff questions, ask them as planned. This will ensure you are headed in the right direction. 16.8.1. Even though your lesson is somewhat structured, be careful not to become so directive that conversation is stifled. However, the interview is controlled at all times, even to the extent of tactfully interrupting the expert if necessary. By remaining natural, friendly, permissive, and conversational,AFMAN36-2236 12 NOVEMBER 2003 195

you can show genuine interest in both the guest and the subject. Listen to what is being said and be prepared to ask for further clarif ication, examples, details, and other support material if required, or to pursue new lines of discussion when necessary. Notes taken during the lesson will be helpful when the time comes to summarize. 16.8.2. Your job as instructor, then, is to bridge the gap between the expert and the students. You interpret what the expert says and provide clarific ation when needed for the students. However, avoid using such phrases as "What you're saying is..." or "Let me clear up what you've just said... ." You can ask the expert to clarify points without imply ing poor communication. You are the best judge of students' needs and also of how well the informati on given in the interview is satisfying those needs. Divide your attention, between looking at the expert and monitoring the students for any cues that would prompt you to seek further clarification fro m the expert. Carefully positioning the chairs will eliminate any awkwardness in directing eye contact to both the expert and the students. 16.8.3. Your questions are the vehicles for achievi ng the learning objective of the lesson and satisfying students' needs. The successful instructor organi zes questions in a definite sequence relating to the subject's central theme and the lesson objective. Questioning sequences may follow any acceptable organizational pattern (chronological, spatial, topical, and so forth) provided th at pattern is appropriate to the lesson. 16.8.4. Each question should be clear and definite for both the expert and the students. The expert should not have to guess what the question means, wh at it implies, or what kind of answer to give. If the expert seems to be having difficulty with a question, you can restate or rephrase it. Lengthy, involved, and ambiguous questions hardly ever pr oduce the best possible response. Simple, precise, and direct questions are better for communicating wi th the expert. Avoid questions that contain unfamiliar or technical vocabulary beca use they may be misunderstood by the students or, for that matter, by the expert. In addition to your planned questions, do n't hesitate to use spont aneous questions to get the lesson back on track and to explore new ideas and points. You should have an interim summary before going on to another main point, but you may also wish to use interim summaries to curtail and redirect discussion if the expert gets off track. 16.8.5. Remember to ask questions that direct attent ion to ideas, elicit comments, and seek explanations to clarify ideas in depth. Some questions perform these tasks better than others. Thought-provoking questions require the expert to explain the responses and are better for promoting discussion. With an outgoing, talkative expert, you may not have a problem keeping the conversation moving. However, with a quiet, reserved expert, you must ensure the questions prompt more elaborate responses than simple agreement or disagreement. Questions phrased using "how" or "why" tend to promote more thought and discussion. They will more readily seek out the expert's special knowledge or opinions on the subject. Additional examples of questions to open up conversation include: What are your ideas concerning...? What is your opinion of...? Why do you feel...? How do you view...? 16.8.6. The instructor must also continually remember that the purpose of the interview is to aid student learning. Therefore, assess the expert's res ponses on the basis of student understanding and follow up as necessary when the replies are vague, confusing, or incomplete. Carefully plan your followup questions, but be prepared to bypass some of them if the guest has already covered the point involved. If appropriate, ask the que stions verbatim from the lesson plan. However, be ready to modify a followup question based on the expert's comments that preceded it. Be sure to plan a sufficient number of questions to keep the interview lively. Followup questions useful for clarification include:196 AFMAN36-2236 12 NOVEMBER 2003

What are some other examples? Under what circumstances would that apply? What is the basis for your opinion? 16.8.7. The instructor may also guide the conversation by repeating phrases the expert has already used. This technique invariably causes the expert to expand ideas further. Nodding to answers is reassuring and encourages the expert to continue. Certai n neutral questions may also be used to obtain a more complete, clearer response. Examples of ne utral questions include: What do you have in mind? Why do you feel that way? Why do you think so? 16.8.8. Allow the expert adequate time to refl ect on questions, decide on answers, and to word replies. The short gaps in the conversation these paus es will create are not harmful to the expert or the students. Actually, they will give the expert a ch ance to gather thoughts and give a more complete response than if a followup question is asked too qui ckly. In addition, the students will have a chance to finish notes and do reflective thinking.
16.9. Ending the Interview. The instructor's role in concluding the teaching interview is important. Schedule a question-and-answer session before the final summary because you will want that summary to include any pertinent comments made in response to the students' questions. A brief interim summary may follow the last main point, then the question-a nd-answer period may begin. You may choose to stand up or remain seated for the conclusion, depending on your personal teaching technique. Summarize the ideas presented by the expert and show how they support your conclusion and the lesson objectives. Remotivate the students to retain and build upon the insights gained during the lesson. This remotivation can occur as a separate step or as part of the summary. Finally, close the lesson by thanking the expert. Do not thank your guest until you are ready to dismiss th e class; once you do, the lesson is over as far as the students and the expert are concerned.
16.10. Sample Lesson Plan for a Teaching Interview. The sample teaching interview lesson plan is at Attachment 8 . Pay particular attention to how the questi ons are phrased and how the summary is developed. Note the wording of the objective, the fact thatthe samples of behavior provide evidence of comprehending the overall objective, and that the main points add up to the conclusion stated in the objective. Also, notice the similarities between this pl an and that for a guided discussion found at Attachment 4

. 16.11. Summary. The teaching interview is essentially a method of instruction through which expert resource persons can be used efficiently in the classroom. This method gives the instructor an opportunity to be more flexible and vary presentation modes of teaching, while at the same time satisfying planned learning objectives. The degree of success in the use of the teaching interview depends on a good lesson plan and questions that will guide the expert's re sponses. If you understand the strengths of this method and when to use it, you will improve your use of guest experts. The teach ing interview requires little preparation on the part of the expert compared to prepar ing for a lecture. As such, you will most likely find many individuals enthusiastic and willing to he lp you present a dynamic teaching interview lesson.AFMAN36-2236 12 NOVEMBER 2003 197

Chapter 17

THE DEMONSTRATION-PERFORMANCE METHOD

17.1. Introduction: 17.1.1. The demonstration-performance (D-P) method is a proven method for teaching mental or physical skills that require student practice for skill mastery. This method is based on the principle that students learn best by doing. A person learns to swim by swimming, drive by driving, and teach by teaching. Skills requiring the use of tools, machin es, and equipment are suited to this instructional method. Equally well suited are skills that requir e mathematical computati ons and those that are a combination of mental know-how and physical dext erity, such as taking fingerprints and fitting earplugs.
17.1.2. The science teacher uses the D-P method to teach laboratory procedures. The shop foreman teaches the apprentice almost entir ely by some version of this method. The instructor pilot uses it to teach flying skills. The professor of medicine uses itto teach surgical skills. In each case, the instructor demonstrates the particular procedure to the stude nts and then acts as a supervisor while they practice the skill. During the practice period, the instru ctor points out errors an d helps students eliminate rough spots and/or errors in their performances. Inthe D-P method, the instructor tells and shows students how to do the skill, they practice under the instructor's supervision, and they are evaluated on their performances against predetermined criteria.
17.2. Planning and Developing a D-P Lesson. The D-P lesson or block of lessons consists of explanation, demonstration, performance-supervision, and evaluation phases. If all these phases are included in one continuous block or period of instruction, they are preceded by an introduction and followed by a conclusion.
17.2.1. The Introduction. The introduction to a D-P lesson generally consists of the three elements recommended for all lessons—attention step, motivation step, and overview. The attention and motivation steps are the same as those discussed in Chapter 6 ; however, the overview is somewhat different. In a D-P lesson, students need to know not only what they will learn but how they will learn it; that is, how the lesson will proceed and how they w ill be evaluated. Where there is potential for physical harm to students, the overvi ew must include safety procedures. Since beginning students often run the risk of injury while working with unfami liar tools and equipment, instructors should continually stress safety throughout a lesson, even if only slight hazards are involved.
17.2.2. The Body: Explanation Phase. The instructor tells the students how to perform the skill in this phase. This explanation should be planned as a s hort lecture with the nature of the task determining the appropriate organizational pattern. Most skills lend themselves to a sequential pattern where the skill is explained in the same step-by-step order normally used to perform it. When the skill being taught is related to previously taught or alrea dy known material, the instructor may use the known tounknown strategy. When teaching more than one skill in the same lesson, the simple to complex strategy works well. By starting with the simplest skill, students build confidence and are less likely to become frustrated when building to more complex skills. 17.2.2.1. Another important consideration in the e xplanation phase is the language used. Instructors should attempt to speak at the learner leve l and avoid unnecessary ja rgon and technical terms the students may not understand. Clear communication is the key. Take care to clearly describe the198 AFMAN36-2236 12 NOVEMBER 2003

actions students are expected to perform. It is neither appropriate nor effective for instructors to try to impress the students with their expertise by using unnecessarily complicated language. 17.2.2.2. Instructional aids are particularly important in a D-P lesson. The best instructional aid is the actual equipment to be used. Other useful ai ds are charts, mock-ups, and models. When using aids, be sure they are readable, accurate, and that all students can see them.
17.2.3. The Body: Demonstration Phase. In the demonstration phase (which may be combined with the explanation phase) the instructor shows the students how to do the skill. At times, an explanation alone is too abstract and may need a simu ltaneous demonstration to aid in understanding. However, with a complicated or dangerous skill, the two phases are often better separated. For example, students will probably understand a combined explanat ion or demonstration of flag folding, but they would probably need separate st eps to construct visual aids.
17.2.3.1. The instructor must demonstrate the skill co rrectly and safely the first time it is demonstrated. When the skill is demonstrated incorrectly, the instructor may lose credibility, and students will have to unlearn the incorrectly presented material before they can learn it correctly.
17.2.3.2. The skill should be demonstrated in the same sequence in which it was explained, thus avoiding confusion among the students and reinforc ing the steps. Since the students generally imitate the instructor's performance, the instructor must demonstrate the skill exactly the way the students are expected to practice it, including al l safety procedures the students must follow.
17.2.4. The Body: Performance-Supervision Phase. Before the students actually begin to practice the skill, the instructor must decide how much control to use. In the inde pendent approa ch, the students practice the entire skill after seeing the demons tration, going from step to step at their own individual rates. In the controlled approach ("by the numbers"), students practice each task step (or small group of task steps) after seeing them demonstrate d. With dangerous or difficult skills, the controlled approach is recommended for the first practice as a minimum. In each case, the students should practice the entire skill independently as many times as practical to achieve mastery before they are evaluated. In many cases, over-learning to ensure profic iency may be desirable. Allow students to practice at least once under evaluation conditions. 17.2.4.1. Each student's work must be supervised to ensure safe, correct performance. If any common error or safety hazards develop, the instru ctor should stop the group and reteach the area ofdifficulty. Students should be permitted to work on their own as much as possible without unnecessary interruption or too much assistance. Th e instructor should avoid distracting or nonpurposeful talking or wandering. However, the instructor s hould not hesitate to interrupt if a student has made a mistake or is about to make a mistake. 17.2.4.2. The time to identify errors is during the learning activity rather than the evaluation phase. The stronger, more proficie nt students may assist the less pr oficient ones. The stronger students may also be able to make some comments or suggest techniques all the students can use. Weaker students often make comments concerning their areas of difficulty. These comments provide excellent sources of information for improvi ng subsequent instruction. In a self-paced environment, proficient students s hould not be held back when they are able to perform better and more quickly than their peers.
17.2.5. The Body: Evaluation Phase. The most important consideration in the evaluation phase is to develop an appropriate rating in strument or written test (see Chapter 22 and Chapter 24 ). To be valid, the rating device must accurately measure achievement of the criterion objective. For example,AFMAN36-2236 12 NOVEMBER 2003 199

if the conditions of the objective state that the skill will be accomplished without assistance, then the instructor must cover or remove all instructional aids, erase the board, put away work sheets, and ensure that the students actually do the skill without references.17.2.5.1. When beginning the evaluation phase, instructors should give clear, complete instructions to the students. They should review the task steps, if necessary, and emphasize acceptable standards (time, accuracy, quality, etc.). They must allow for enough time, equipment, and supplies to evaluate all students on all standards. 17.2.5.2. At the conclusion of the evaluation phase, instructors should record student performance on a suitable form and reveal the results privately to each student if possible. Students will naturally be interested in their individual performances; however, the instructor must withhold comments that may influence other students until all have completed the evaluation. 17.2.5.3. Regardless of how well a skill is taught, there may still be failures. The rule is that "success motivates," so instructors should always be positive in revealing results. When pointing out areas that need improvement, instructors should offe r concrete suggestions that will help. If possible, they should avoid ending the evaluation on a ne gative note or criticizing a student in front of other students. Afterwards they should evaluate overall student achievement of the lesson objective and revise the instruction as necessary for future classes.
17.2.6. The Conclusion. The conclusion to a D-P lesson consists of the three elements recommended for all lessons—summary, remotiva tion, and closure. The remotivation and closure are generally the same as discussed in Chapter 6 . In the summary, instructors should review the task steps with emphasis on any areas of difficulty experienced by a majority of the students. Areas of difficulty experienced by only one or two students may be be tter addressed on an individual basis.
17.3. Factors to Consider When Using the D-P Method:
17.3.1. Lesson Content. A D-P method is best used for teaching mental or physical skills that require student practice for mastery. It is not recommended for cognitive or affective lessons in which student practice is not required.
17.3.2. Evaluation. One advantage of the D-P method is that evaluation is a part of the lesson format. The criterion objective dictates the evaluation proce dures. If the criterion objective is complete, clear, and concise, there should be no surprise s for the student during the evaluation.
17.3.3. Student-Instructor Relationship. When using the D-P method, the instructor must be aware of individual differences among students and be willing and able to give as much or as little assistance as required by each student. Students may work inde pendently or the instructor can provide close, individual attention.
17.3.4. Time. The majority of time in a D-P lesson should be devoted to supervised practice. There is no alternative to providing sufficient time for student practice. If the students do not require practice to learn the skill, then another me thod is probably more appropriate.
17.3.5. Equipment and Materials. The instructor should have all handouts, equipment, and materials ready before the lesson begins. Arrange the room or work area to facilitate safety, close supervision, and maximum use of space.
17.3.6. Student Role. Many skills cannot be performed on an individual basis; therefore, teams or partners must be assigned. Only through intensive practice can groups know exactly what to do and200 AFMAN36-2236 12 NOVEMBER 2003

how to do it. If the activity is normally accomplished as a team effort, it should be practiced and evaluated as a team effort.
17.3.7. Group Size. In many skills, the size of the class or th e number of workstations is critical. All students must be able to see the demonstration, and the group must be small enough so the instructor can effectively supervise all students during the practice session. All students must have sufficient access to needed equipment, supplies, and materials.
17.4. The Sample D-P Lesson Plan: 17.4.1. The lesson plan at Attachment 9 illustrates a format for D-P lessons from among many acceptable ones. The objective, task steps, and actual teaching plan arrangement may be varied to meet the administrative requirements of different schools or situations. At Attachment 9 , the criterion objective is broken down into three elements—performance, conditions, and standards—rather than written in narrative form. This separation of the elements clearly specifies the instructional intent and may be easier to write than a narrative format. 17.4.2. The process/product evaluation sheet shown as Part III of the sample lesson plan in Attachment 9 is one of many possible formats for recording student evaluation results. The important thing to remember when designing formats for lesson plans and evaluation sheets is to use those that will work for the particular situation within the administrative constraints of the individual school.
17.5. Summary. The D-P method provides students with the oppor tunity to perform skills or processes that have been explained and demonstrated under controlled conditions and close supervision. Through this supervised performance, the student gains necessary skills to apply on the job or in subsequent lessons. 17.5.1. When using the D-P lesson, the instructor mustcarefully consider the following factors: lesson content, evaluation, student-instructor relations hips, time, equipment and materials, instructor preparation, student roles, and class size. 17.5.2. A successful D-P lesson requires a thorough explanation, a proper demonstration, studentpractice with close instructor supervision to eliminate student mistakes, and a valid evaluation of student performance. If these four primary phases ar e effectively planned for and carefully implemented, students will have the maximum opportunity to master the skill taught.AFMAN36-2236 12 NOVEMBER 2003 201

Chapter 18

USING GAMES FOR LEARNING

18.1. Introduction. Games in the classroom? You must be kidding! This is the reaction you may get from educators who have never experienced the benefits of using games in the classroom to facilitate the learning process. However, adults, like children, enjoy the interaction and challenge games provide. Games have been used for years by parents and school teachers to provi de children an enjoyable way to learn. Adults can benefit from this method as well, and you'll find (as an instructor) that games help provide variety to your standard less ons. Your students can benefit from ga mes if the games are used to help reach your lesson objectives.
18.2. Benefits of Using Games. Using games in the classroom can be beneficial to your students in a number of ways. One thing games will help you do is to gain your students' attention and increase their motivation by providing a diversion from the normal lecture methods. Games al so help students retain information more readily and help to create a nonthreatening learning environment. Students are often reluctant to take risks in the normal classroom setting. When involved in an experiential method ofinstruction, students can make mistakes without feeling their reputation will be threatened. Games provide individuals with opportunities to participate in si tuations in a safe environment before facing the situation in real life. It's always better to try out ne w ideas or methods in a classroom environment before using them in real-life situations. Another big plus is that an experiential method gets the students involved in reaching the lesson objective, which d ecreases passive learning. Finally, the majority of games help add that needed level of competition to the classroom by pitting teams against each other in a friendly way.
18.3. Classifying Games. Games are defined as "student interac tive educational methods where students participate in structured activities that focus on specified learning objectives." The key ingredient to games is the aspect of student in teractivity. Students must get actively involved in the learning process and not be passive listeners. This activity engages students in the c ognitive, affective, and psychomotor domains. While the number of games used for learning is almost limitless, games can be divided into two major areas—climate setting and learner development games. Climate-setting games include openers or icebreakers and pacers or energizers, while learne r development games can be divided into cognitive development games and skill practice games.
18.3.1. Climate-Setting Games:
18.3.1.1. Openers and Icebreakers. These types of games are designed to help individuals in an unfamiliar setting get comfortable with each other, pl us get comfortable with the instructor and the learning environment. In other words, they help break the ice.
18.3.1.2. Pacers and Energizers. At times, a class may need a boost in the middle to get the students' interest back. This is especially true afte r lunch or after several hour s of sitting in class. Pacers and energizers help by getting the students actively involved in a learning activity so they are energized enough to continue with the class. Climate-setting games for the most part are relatively brief in duration.
18.3.2. Learner-Development Games:202 AFMAN36-2236 12 NOVEMBER 2003

18.3.2.1. Cognitive Development Games. These games involve students using previously learned material to accomplish the game and reach the lesson objective, or they may be used to introduce new concepts or principles to students. These types of games are very effective in reinforcing student learning by providing capstones to the lessons.
18.3.2.2. Skill Practice Games. These types of games allow students to try out or improve their abilities on a task or skill. Students can practice real-life situations in a safe environment. Such games include role plays, simulation games, orga nizational and management games, and in-basket exercises. These games are relatively longer in duration than cognitive development games. Some skill practice games may last throughout the class duration.

18.4. Various Games for Classroom Use. Some of the games you may want to use in your classroom include:
18.4.1. Frame Games. Frame games allow you to plug your content into an existing game to fit individual needs and objectives. For example, you may use the game of monopoly, tic-tac-toe, bingo, jeopardy, or some other board game and develop questions, rules, etc., to fit your need. In other words, use the frame (format) of an established game in stead of reinventing the wheel. Frame games are effective at the knowledge level of learning and ma y be taken to the comprehension level by having students translate, interpret, or extrapolate.
18.4.2. Matching Games. Matching games are great for helping students remember definitions, sequences, steps, phases, etc. You provide the te rms and definitions in random order and the student must match the correct term with its correspondi ng definition. Matching games can even be placed on room walls for students to solve at their leisure. Matching games are effective at the knowledge and comprehension levels of learning.
18.4.3. Word Hide. Also known as word search games, stude nts enjoy the challenge of locating the hidden word. This game can be used as a pacer or energizer during class.
18.4.4. Crossword Puzzles. In crossword puzzles, students read a sentence or phrase, then mustdetermine what word corresponds to the sentence or phrase. Crossword puzzles can be used effectively at the knowledge or comprehension level of learning. To help ensure students become familiar with the reading material, you can develop a crossw ord puzzle that can only be solved by reading the chapters in their books.
18.4.5. Role Play. Role plays require students to place themselves into a simulated interpersonal situation and act out various roles assigned them. The va rious role plays include single role plays, multiple role plays, and spontaneous role plays. Role plays are very effective in human relations and communication scenarios. Role plays can work effectively at the higher levels of learning.
18.4.6. In-Basket Exercises. These exercises simulate a series of situations or decisions a leader or manager may encounter within his or her profession. They are effective in allowing students to practice skills such as decisionmaking, delegating, pl anning, and scheduling. So, in-basket exercises are effective at the higher levels of learning.
18.4.7. Simulation Games. Simulations are low-risk, educational experiences that substitute for real-life situations. Simulations involve rules, competition, cooperation, and payoffs. They require students to make various decisions and plan strategi es in order to reach a goal or outcome. There are numerous computer simulation games on the market today that can be adapted for use in the classroom. Simulation games are effective in helpi ng students reach the higher levels of learning.AFMAN36-2236 12 NOVEMBER 2003 203

18.4.8. Organizational or Management Games. Students are allowed to manipulate an organization or some component part to produce certain desired outcomes. Students are able to practice the various roles of manager, employee, and customer to get a feel of what it is like to function in these roles. Organizational or manageme nt games are effective at the higher cognitive levels of learning.
18.5. Designing a Game. If you can't find a ready-made game to fit your need, design your own. An example game lesson plan is at Attachment 10 . As with other methods of instruction, the first step is to identify the lesson objective. The objective helps you ensure the game is relevant to student learning— otherwise, don't attempt it. 18.5.1. Second, select or adapt a game to fit your needs. You may be able to find some preexisting game that will fit your need; however, you may need to modify it so that the game rules fit your need. If you are unable to modify a game, you can always develop your own. The choice is really up to you. 18.5.2. Third, determine the process orsequence of events for the ga me. The process may be a given with some preexisting games, but you can always m odify the sequence of events. The key is to have the game run smoothly. 18.5.3. Fourth, determine the end of the game. You n eed to determine how you will identify the winner of the game—provided the game has a winner. 18.5.4. Fifth, if you are developing a cognitive devel opment game that lends itself to asking students questions, collect the curriculum you will be teach ing and write out your que stions. Reading assignments, handouts, class discussions, etc., wo rk well in helping you develop questions. 18.5.5. Sixth, construct your game or find a preexisti ng board or computer game to use. If constructing your own game board, game pieces, game cards etc., ensure they are of good enough quality to last and are easily duplicated. 18.5.6. Finally, try out your game with your coworker s to see if it flows well. This will help iron out any rough edges before introduci ng the game to your students.
18.6. Writing Game Questions. Questions are important in a large portion of games. Steve Sugar, of The Game Group, suggests the following tips on writing game questions: 18.6.1. Write your questions in a conversational format. 18.6.2. Questions and answers should be stated simply. 18.6.3. Make your questions short in length. Since you usually read them aloud, a lengthy question will slow the game down. 18.6.4. Ensure your question focuses on one fact . Therefore, avoid multiple questions. 18.6.5. Include the rationale or elaboration along with the answers. This will aid additional learning. 18.6.6. Develop a difficulty mix among your questions. A recommended mix might be 20 percent challenging, 60 percent moderate, and 20 percent easy. 18.6.7. If using note cards for your questions, number your cards just in case you drop them.
18.7. Writing a Game Lesson Plan. Where in your lesson or course do you want to place your game method? You may decide to use the game for just a portion of your lesson (p artial period game) or you204 AFMAN36-2236 12 NOVEMBER 2003

may want to design your entire class around a game (whole period game). If writing your own game lesson, you will need to: 18.7.1. Identify your lesson objective, the timeframe yourgame will take up, and the number of players required to play the game (minimum and maximum). 18.7.2. Identify all the supplies and materials requir ed to play the game to include boards, game pieces, pens, paper, chips, etc. 18.7.3. Lay out the process of your game in a step-by- step sequence. This will help the game to run smoothly and as planned. 18.7.4. Include a debriefing section, if needed. Don' t confuse debriefing with a briefing. They're not the same. In a game debriefing you help the students reflect on the game experience. Providing a debriefing will help increase student learning insight s. You can also clear up any areas of student confusion in the debriefing. According to Dr. Sivasailam Thiagarajan, of Workshops by Thiagi, you should prepare special questions for the debriefi ng, such as: How do you feel about what took place? What happened during the exercise? What did you lear n from this experience? How does this relate to the real world? What if ... ? What is next?
18.8. Administering the Game. The following tips should help your game run smoothly: 18.8.1. Provide an introduction to your game as a lead -in. Be enthusiastic about the game, for enthusiasm is contagious. If you present a negative att itude about the game, your students will mirror that negativism. 18.8.2. Provide instructions to the game in your se tup and distribute the game materials as required. In order to get the game going, only explain the important rules and instructions in outline form to keep it quick and simple; however, ensure all rules are covered. Provide clarification if the need arises during the game. 18.8.3. To make sure you reach your objective, ensure students remain on track as the game progresses. Games are a lot of fun and can sometimes get out of hand if you're not careful. Some individuals may get very competitive dur ing the game while others may not participate. As with any instructional method, use your teacher skills to ensure everyone is given the opportunity to participate. You will want to maintain an appropriate pace a nd level of competition during the game so you can move smoothly from one stage of the game to the next. 18.8.4. One important key to remember is to alwa ys be flexible. When conducting a game you must always be ready for a new twist; being flexible wi ll keep you from getting upset when things don't go exactly as planned. 18.8.5. Finally, debriefing students as necessary will help them reflec t on the game, clear up any areas of confusion, and close your game on a positive note.
18.9. Cautions on Using Games. While games are a lot of fun to use in the classroom and relatively easy to administer, there are a few precautions to keep in mind. 18.9.1. Your attitude is paramount when conducting a game in the classroom. Ensure you have a proper attitude before proceeding. Ensure you are us ing the game to help your students learn and not just to have fun or kill time. While games are de signed to be fun, you have a bigger picture to keep in mind—student learning. Maintain a bala nce between having fun and seriousness.AFMAN36-2236 12 NOVEMBER 2003 205

18.9.2. Make sure the game fits into the class context. If the game isn't relevant to the class, leave it out. Maintain overall control of the game to ensure your objective is met. Watch out for those domineering students or those who don't want to partic ipate. If you have a domineering student, direct participation to others in the class. If a student doesn't like talking, direct questions to him or her that can be easily answered. Peer pressure will often take care of these situations; however, intervention may be necessary from time to time. Re member that everyone should gain from the game experience.
18.10. General Considerations When Using Games. There are several things you will need to consider before using a game.
18.10.1. Determine Your Lesson Objective. You need to determine if using a game will help your students learn and help reach your lesson objective. Do you want your students to simply get to know each other or is your objective to develop thei r cognitive, affective and/or psychomotor skills?
18.10.2. Group Size (Number of Players). What is the minimum and maximum number of students who can play your game? How much time do you have during your class or course for an experiential method? Do you want to use the game during the introduction, body, conclusion, or as a complete period of instruction?
18.10.3. Supplies or Materials. What materials will your game require? Can you make your own game pieces or will you need to purchase them?
18.10.4. Physical Setting. Finally, do you have enough classroomspace to conduct your game? Most games will run very smoothly in your classroom; howeve r, some games may require special equipment or locations.
18.11. Dr. Sivasailam Thiagarajan's Considerations. To ensure your game will be effective, consider Dr. Thiagarajan's (more simply known as Thiagi) checklist at Figure 18.1. for using experiential activities. He developed this checklist to give trainers and educators who use games a tool to ensure the validity of their efforts. It is available on the worldwide web at http://www.thiagi.com

.206 AFMAN36-2236 12 NOVEMBER 2003

Figure 18.1. Dr. Thiagarajan's Experiential Checklist.

18.12. Summary. Games will provide that extra something for you in the classroom if you plan and prepare well. As the instructor, you will enjoy watchi ng your students learn while having fun. So experiment, be creative, and enjoy the game. For further information on using games in the classroom or finding a ready-made game for use, consult your local library and look under games, teambuilding, or classroom activities. The internet also contains a wealth of resources in this area. 1. Real-World Relevance. Does the training game help participan ts learn skills and concepts that are applicable to the workplace? Do the roles in the tr aining game relate to easily recognizable real-world counterparts? 2. Appropriate Frame. Is the basic structure of the training game appropriate for the instructional objectives, trainee characteristics, type of learning, and intended use? 3. Flexible Format. Does the training game permit easy modifications to suit local resources and constraints in terms of schedule, number and type of participants, and physical facilities? 4. Participant Involvement. Are all participants involved in the training game at all times? 5. Effective Packaging. If the game uses different components (such as game boards, cards, and dice), are they conveniently packaged in a box? Are the ma terials produced in an attractive and durable form? 6. Effective Instructions. Does the training game include clear and concise instructions? Do the rules avoid unnecessary and trivial items? 7. Intellectual Stimulation. Are the participants engaged in challenging tasks instead of trivial rote memory activities? 8. Criterion Reference. Does the scoring system reward achievement of the performance objectives rather than chance occurrences? Is the mastery of useful skills and knowledge obvi ous to the participants? 9. User Friendliness. Can a typical trainer use the game without having to spend too much time preparing the materials or learning the rules? 10. Cost-Effectiveness. Is the training game inexpensive? Can a cheaper alternative produce the same training outcomes?

Copyright © 2000. Workshops by Thiagi, Inc.
URL: http://www.thiagi.com/fac-028.htmlAFMAN36-2236 12 NOVEMBER 2003 207

Chapter 19

SELECTING TEACHING METHODS

19.1. Introduction. After deciding exactly what to teach in a lesson, the instructor must determine how to teach it and what instructional method to use. Selec ting an appropriate method is an integral part of lesson planning. Certain limiting factors may prevent an inst ructor from using certai n instructional activities. In addition to considering the lesson objective, the instructor must consider the background and abilities of the adult learners, their motivation and learning styles, the number of learners, the time allotted to the lesson, and available facilities.
19.1.1. The final selection of an instructional method may be a compromise between the method most suitable to the outcome (an ideal approach) and the method possible under the circumstances (a realistic approach). Many vari ations, combinations, and adaptations can be made of any method or methods of teaching; in fact, the number of possibilitie s is limited only by the instructor's imagination. For this reason, the instructor should recognize the many available opportunities to help students accomplish desired outcomes—whether process or product. 19.1.2. In previous chapters, we surveyed many methods in general and examined five methods in detail. This chapter may help us determine whic h teaching method could and should be used under a given set of circumstances. 19.1.3. The decision about which teaching method to use comes in the design phase of the instructional systems development (ISD) process (see AFMAN 36-2234). In determining the means for teaching the block or period of instruction, course pl anners need some systematic way to deal with the available options. Table 19.1. may help you decide. The remainder of this chapter explains this table and describes how to use it. 19.1.4. The intersection of the factors listed in column A and the methods listed in columns B through M provide the key. The explanation of the key is located below the table. Broad, general descriptions are used in the key because a comment too specific could be misleading. 19.1.5. We can use the grid both for evaluating current courses and planning new ones. The grid is a synthesis of options proposed by several qualifie d educators and their estima tes based on research, logic, and practical experience. The recommendations are only guidelines. So, use the grid with caution because these generalizations may not be appropr iate for a specific situation or for combinations of methods. 19.1.6. First, the instructor should consider the primary domain of learning involved—the cognitive, psychomotor, or the affective. In the cognitive domain, the grid considers knowledge and comprehension levels separately and groups the remainder of the taxonomy levels together as higher levels. Generally, the same teaching methods work for application, analysis, synthesis, and evaluation. Similarly, the affective domain is divided roughly into lower levels (receiving and re sponding) and higher levels (valuing, organization, and character ization). The psychomotor domai n also has a taxonomy, but for purposes of this grid, it is divided simply into motor and mental skills. 19.1.7. All who plan instruction must deal with the realities of their situation—resources, space, class size, etc. These items are grouped together as factors and constraints. Instructor expertise is always a consideration, particularly when the instruction cont ains complex procedures or involves high levels of cognitive and/or affective learning. Class size is another important factor; a very large class may208 AFMAN36-2236 12 NOVEMBER 2003

cause planners to select a less desirable method th an they prefer. Evaluation becomes more difficult at the higher levels of learning. But some methods provide ways for instructors to evaluate during the learning process. Case studies and other simulations are good examples. Similarly, some instructional methods permit instructors to respond to individual student needs better than others. The lecture, for example, rates low in providing opportunities for in structors to modify their instruction in order to help the fast or slow learner. On the other hand, small group methods lend themselves to more individualized student treatment. 19.1.8. Many other critical factors and constraints affect the selection of teaching methods. These additional considerations may be as significant as those just discussed, and often combine to create even more difficult decisions on selecting appr opriate teaching methods. Among the other significant factors that affect the selection of teaching methods are: the need for a specially equipped facility, amount of time needed to develop instruction, the co st to develop and mainta in both the hardware and software to support the method, and availability ofsupport people to develop and maintain the hardware and software necessary to use the method effectively.
19.2. Assumptions. The grid ( Table 19.1. ) can be of considerable value as a basis for evaluating instruction methods. To realize the most practical value from this grid, we should be aware of and understand five assumptions (paragraphs 19.2.1. through 19.2.5. ). While these assumptions may affect the interpretation of the grid, the grid's recommendations should be useful for all schools and courses under most normal situations. At the least, these recommendations will help identify periods or blocks of instruction that may require further analysis. The assumptions are as follows:AFMAN36-2236 12 NOVEMBER 2003 209

Table 19.1. Teaching Methods Grid or Decision Table.210 AFMAN36-2236 12 NOVEMBER 2003

NOTE: A school may find itself with an effective instru ction system where the methods used are as good as or better than those recommende d. Carefully examine situations markedly different from these recommendations to ensure the methods c hosen are not simply a result of an error in judgment or blind luck. 19.2.1. The terms and categories follow the definitions and conditions found in the accompanying chapter narrative. Terms used in a specific course or school may be define d differently. Most terms, however, are consistent with AFMAN 36-2234 and the taxonomies of Bloom and Krathwohl. 19.2.2. The periods or units of instruction being an alyzed are assumed to be relatively short—from one period to a short block of instruction. Extended periods of instruction present many considerations beyond the scope of this grid. 19.2.3. The methods are analyzed in their "pure" form; that is, not combined with other methods. The greater the combination of methods, the more we must exercise caution in using this grid. For instance, a method not recommended as an exclusiv e approach to instruction may be highly recommended in combination with another method. 19.2.4. A reasonable quality of material s and adequate teaching skills mu st be available. Poorly prepared materials and weak instructor skills have a serious, negative effect on all recommendations. Conversely, the "great teacher," the "master" of a particular method, or extremely well-prepared instructional materials may create an impact well beyond those normally expected. 19.2.5. The learners dealt with in this chapter are "a dult" learners. Adult learners tend to be task oriented, highly motivated, possess necessa ry prerequisite skills for a given learning situation, and often prefer interactive methodologies.

19.3. Definition of Terms. Keep in mind the following term definitions as we discuss the methods and their ratings:
19.3.1. Cognitive Domain—Lower Levels:
19.3.1.1. Knowledge. The recall of previously learned materi al (facts or theories) in essentially the same form taught.
19.3.1.2. Comprehension. Seeing relationships, concepts, principles, and abstractions beyond simply remembering material. Typically involves translating, interpreting, and estimating future trends.
19.3.2. Cognitive Domain—Higher Levels:
19.3.2.1. Application. The ability to use learned material in new and concrete situations, including the application of rules, methods, c oncepts, principles, laws, and theories.
19.3.2.2. Analysis. The ability to break down material into its component parts so that the organizational structure is understood, including iden tifying the parts, analyzing the relationships between parts, and recognizing the organizational principles involved.
19.3.2.3. Synthesis. The ability to put parts together to form new patterns or structures, such as a unique communication (a theme or sp eech), a plan of operations (a research proposal), or a set of abstract relations (schemes for classifying information).AFMAN36-2236 12 NOVEMBER 2003 211

19.3.2.4. Evaluation. The ability to judge the value of material for a given purpose. Learning in this area is the highest in the cognitive hierarc hy because it involves elements of all the other categories, plus conscious value judgments based on clearly defined criteria.
19.3.3. Skill Development. Although skill development is not normally considered a domain of learning, it is useful here to differentiate between those skills primarily motor in nature and those primarily mental.
19.3.3.1. Motor Skills. Those skills associated with neuromuscular responses learned primarily through physical repetition (such as typing, most athletic skills, shop work, etc.).
19.3.3.2. Mental Skills. Those skills in which cognition is of primary emphasis (such as math skills, computer operation, speaking, writing, etc.).
19.3.4. Affective Domain—Lower Levels:
19.3.4.1. Receiving. The getting, holding, and directing of the student's attention, from the simple awareness that a thing exists to selective attention on the part of the learner.
19.3.4.2. Responding. The student not only attends to a particular phenomenon but also reacts to it in some way, such as acquiescence (reads assigned material), willi ngness to respond (voluntarily reads beyond assignment), or satisfaction in responding (reads for pleasure or enjoyment). Includes instructional objectives relate d to "interests" or "disinterest."
19.3.5. Affective Domain—Higher Levels:
19.3.5.1. Valuing. The worth or value a student attaches to a particular object, phenomenon, or behavior, ranging from acceptance of a value to commitment. Includes instructional objectives related to "attitudes" and "appreciation."
19.3.5.2. Organization. The bringing together of different values, resolving conflicts among them, and building an internally consistent valu e system. Includes instructional objectives related to an integrated, coherent "philosophy of life."
19.3.5.3. Characterization By a Value Complex. Pervasive, consistent, and predictable behavior (life style) developing from a value system which controls behavior for a significant period of time. Instructional objectives focusing on personal, social, and emotional adjustment are in this category.
19.3.6. Factors and Constraints:
19.3.6.1. Minimum Level of Instructor Expertise. The minimum level of proficiency in the cognitive domain required to teach a particular method of instruction effectively. Unless an instructor is just delivering a lesson planned by an expert designer, most methods require the instructor to be at the applicat ion level or higher. In several instances throughout this chapter, the "one level higher" rule is reco mmended. Roughly translated, this ru le cautions that instructors should be at least one cognitive level higher than the level of le arning for the lesson being taught. If knowledge-level material is be ing taught, instructors should be at least at the comprehension level (or higher). If application-level objectives are being used for a lesson, instructors should be at least at the analysis level of learning (or higher).
19.3.6.2. Class Size. The optimum number of students for teaching a particular method of instruction. Recommended numbers (where applicable ) are given in the narrative for each method.212 AFMAN36-2236 12 NOVEMBER 2003

With advanced electronic media often used in today's classroom, class size is extremely difficult to estimate.
19.3.6.3. Evaluation Inherent in Method. An evaluation to determine if the method itself provides the basis for evaluating attainment of ob jectives, or if a followup evaluation device is required.
19.3.6.4. Responsiveness to Individual Needs. Determining if the method allows for active student participation and produces opportunities to express viewpoints. Also, determining if the method allows for different levels of student achie vement or allows students to work at their own pace.
19.4. Recommendations for Teaching Methods. The remainder of this chapter is a detailed narrative describing the rationale for the ratings given in Table 19.1. for the various teaching methods based on the definitions in paragraphs 19.3.1. through 19.3.6.4.

19.5. Lecture:
19.5.1. Domains and Levels—Cognitive Domain:
19.5.1.1. Knowledge (HR). Highly recommended when the basic purpose is to disseminate information and the material is not available else where. It works well for arousing interest in a subject and in organizing content for presentation in a particular way for a specific group. It is especially suitable for content that must be c ontinually updated or revised by the expert. However, the instructor should realize that material pr esented will be remembered for only a short time unless it is reinforced through use of another teaching method or real-life experience.
19.5.1.2. Comprehension (r). Probably the most efficient at th e knowledge level, but it may also be used at the comprehension level, particularly when the lecture is informal. Recommended if thestudent-faculty ratio is too high for small group activities. A more formal approach is recommended for a very specialized subject area if facu lty expertise is at too low a level to use small group activities effectively.
19.5.1.3. Higher Levels. Many adult learners can increase their level of cognitive learning even at the higher levels. Learner motiva tion, need, and prerequisite skills are factors that must be considered. For all practical purposes, the expertise of the instructor (or at least of the resource person who designed the lesson) must be at least one level higher than the educational outcome of the period.
19.5.2. Domains and Levels—Skill Development:
19.5.2.1. Motor Skills. Limited use because the lecture is primarily a "saying" method which makes it less appropriate for instruction in "doing." The explanation phase of a D-P lesson used in teaching psychomotor skills might be presented as a formal or informal mini-lecture; the one level higher rule applies.
19.5.2.2. Mental Skills. Useful in the introductory phases of mental skills for such topics as rule formation, formulas, and factual material associ ated with equipment use. Although such material could also be taught through readin g or programmed texts, the informal lecture with its inherentuse of questions might be more effective in dealing with differing student backgrounds and complex factual material.AFMAN36-2236 12 NOVEMBER 2003 213

19.5.3. Domains and Levels—Affective Domain:
19.5.3.1. Lower Levels (r). A lecturer may develop a rational argument aimed at producing actual changes in overt student behavior. Howeve r, passive participation by the students is likely to lead to little more than an awareness of the new material and perhaps some intellectual conviction about appropriateness of the new behavior. It may be used quite successfully for reaching lower affective levels, such as an informal le cture near the end of a block of instruction.
19.5.3.2. Higher Levels (nr). Adult learners may respond to highe r level affective lessons. Their affective orientation may be such that they are "l ooking for" more stimulation to increase their already strong affective commitment. For example, a very spirited speaker could pull his students to higher affective levels. Going from receiving to characterizing during a lecture or series of lectures is unlikely. However, over a period of time, it would be quite possible for an adult learner to increase his or her affective commitment to a cause or a principle by using just the lecture method as a tool. The instructor would probablyhave more success using other methods.
19.5.4. Factors and Constraints:
19.5.4.1. Minimum Level of Instructor Expertise (h). The minimum level of instructor expertise is at least one level higher than the anticipa ted educational outcome. Fo r example, if a lecture is to the knowledge level, instru ctor expertise at the comprehens ion level is the minimum requirement.
19.5.4.2. Class Size (lg). As long as the instructor can be seen and heard, the size of the class is not important. Under certain circumstances, an informal lecture to a small group may be just as defensible as a formal lecture to a larger one. In a very large hall, if students cannot be close enough to make eye contact with the lecturer, communication may suffer.
19.5.4.3. Evaluation Inherent in Method (no). Very little opportunity to estimate how well the students are learning the material exists, except in an informal lecture to a group of reasonable size.
19.5.4.4. Responsive to Individual Needs (no). Except in an informal lecture to a small group, lecturing usually does not allow students to form ulate questions and have the questions answered before proceeding to the next area of instruction.

19.6. Indirect Discourse:
19.6.1. Domains and Levels—Cognitive Domain:
19.6.1.1. Knowledge (r). While these methods are cautiously recommended to teach factual information, many other methods are probably be ttered suited at this cognitive level.
19.6.1.2. Comprehension (HR). Especially suited for providing th e bases for abstractions, airing multiple points of view, and drawing together data to form generalizations for the student to arrive at conclusions. These methods are suite d to developing student comprehension.
19.6.1.3. Higher Levels (nr). Since there is little requirement for the students to put any abstractions to use in new situations, these methods are not generally recommended for the higher cognitive levels. For adult learners, however, these methods may provide an excellent basis for use at the higher levels, particularly incombination with other methods.
19.6.2. Skill Development (nr). Generally not suitable for developing motor or mental skills.214 AFMAN36-2236 12 NOVEMBER 2003

19.6.3. Domains and Levels—Affective Domain:
19.6.3.1. Lower Levels (HR). A great strength of indirect discourse is the development of lower-level affective outcomes. Activities such as panel discussions, teaching interviews, and dramatizations go a long way toward encouraging student reception and response.
19.6.3.2. Higher Levels (nr). While it is possible to obtain valuing under certain circumstances, the higher affective levels are often beyond the scope of methods of controlled conversation because, with the exception of question-and-an swer periods, active student involvement and application are usually not possible. Such methods are generally not the first choice but may prove to be effective at the higher affective levels with a group of adult learners in just the right situation.
19.6.4. Factors and Constraints:
19.6.4.1. Minimum Level of Instructor Expertise (h). The methods of indirect discourse can best be used to develop knowledge and compre hension-level understanding. Often, however, they are suited to the higher levels of learning as long as the minimum level of instructor expertise follows the one-level-higher rule.
19.6.4.2. Class Size (lg). While there is no optimum class size for these methods, an advantage of indirect discourse is that it can be used for extr emely large groups of students, as long as they can see and hear. These methods have also been very successful with small gr oups of 8 to 12 students.
19.6.4.3. Evaluation Inherent in Method (no). Measuring or evaluating student learning is not an inherent part of indirect discourse. The teac her has no way of knowing if, at the end of the lesson, the students learned. Some additional form of evaluation is necessary to determine if students met the planned objectives. The only exception w ould be a question-and-answer period where the nature and quality of student questions might be an indirect way of evaluating students. However, not all students may get to ask questions, others may choose not to ask.
19.6.4.4. Responsive to Individual Needs (no). If these methods do not include a question-and-answer period as is often the case, they are not very responsive to individual needs. Issues or questions of interest will never surf ace unless students are allowed to pursue them in some way. By themselves, the methods of indire ct discourse do not permit students' needs to be satisfied.

19.7. Demonstration-Performance:
19.7.1. Domains and Levels—Cognitive Domain (nr). Methods of teaching which rely heavily on demonstration are often not efficient and effect ive ways of transmitting cognitive subject matter. While factual information is ofte n transmitted during the course of a lesson involving demonstration, such methods are generally not recommended for ach ieving cognitive objectives other than mental skills. For mental skills, the D-P method is often us ed at both the lower as well as the higher levels.
19.7.2. Domains and Levels—Skill Development (HR). Particularly well-suited to the development of both physical and mental skills. Lessons that rely on teacher-demonstrated skills generally lend themselves to precisely stated, easy-to-meas ure objectives. These lesson objectives require practice to acquire. Although the method is usually associat ed with psychomotor skills, it is also effective with mental skills, such as computation, wr iting, and construc tion of graphs. Because students actively participate in learning physical or mental skills, teachers have ac cess to trial behavior toAFMAN36-2236 12 NOVEMBER 2003 215

which they can respond and provide precise feedbac k. Such feedback is essential in shaping student behavior to achieve the skill level desired.
19.7.3. Domains and Levels—Affective Domain (nr). Use caution with less ons designed primarily to affect attitudes. Although a change in attitude ma y come about from any learning experience, other methods may lead to attitude change more directly . One notable exception is technical training, which often seeks to develop positive attitudes for factor s like safety and accuracy during demonstration lessons. Participation in lessons such as these w ould make the use of the D-P method highly recommended.
19.7.4. Factors and Constraints:
19.7.4.1. Minimum Level of Instructor Expertise (h). Considerable proficiency in the skill being demonstrated is critical to the success of this method. As observation is usually an important element in D-P, instructor error in the physical or mental skill can be particularly confusing to the learner. In addition, the instructor must be able to anticipate learner error, analyze the causes, and provide exact feedback that will allow the learner to perform satisfactorily.
19.7.4.2. Class Size (sm). The size of the learner group is a pa rticularly sensitive variable in a D-P lesson with class size varying from indivi dual instruction to small groups. Generally speaking, educational technology has had little effect on increasing group size in skill development lessons. To the contrary, most sophisticated appli cations of educational technology tend to make demonstration-learning experiences more and more individualized while increasing the size of the potential audience being served.
19.7.4.3. Evaluation Inherent in Method (yes). Easily accommodates evaluation. The instructor generally should not accept the practice of the skill as proof of skill attainment, and a separate evaluation step should be incorporated into the lesson.
19.7.4.4. Responsive to Individual Needs (yes). Because the D-P method relies so heavily on student-teacher interaction, it readily accommodates individual needs. Instructors can readily tell what students can and cannot do and prescr ibe appropriate learning experiences.
19.8. Reading:
19.8.1. Domains and Levels—Cognitive Domain:
19.8.1.1. Knowledge (HR). For the majority of students, the most effective and time-efficient means of presenting knowledge-level material. Students pace themselves and are free to adjust at will to the learning experience. Their only limitations are reading speed and comprehension, both of which may be improved through special courses.
19.8.1.2. Comprehension (r). Although a student can reach the comprehension level through carefully selected readings, other methods that ensure interaction with the instructor or other students may be preferable. When such interaction is not possible, reading assignments are acceptable alternatives, especially when combined with study or review questions or other activities that require the student to manipulate the material.
19.8.1.3. Higher Levels (nr). Use caution at the higher cognitive levels of developmental learning. Note, though, that it is quite possible for an a dult learner to be take n through one or more of the higher levels with an indepth, lengthy, carefully planned reading program.216 AFMAN36-2236 12 NOVEMBER 2003

19.8.2. Skill Development (nr). Although students may read how to do a motor or mental skill, the method by itself does not ensure they can actually perform the task. Formulas, procedures, and task steps can be read, but to expedite skill developm ent, a D-P is recommended if at all possible. Some relatively simple motor or mental skills might be learned from reading if it is the only method available; however, some sort of programmed learning is probably preferable to straight reading. Given certain prerequisite skills, reading is often quit e effective for developing certain motor and mental skills for adult learners.
19.8.3. Domains and Levels—Affective Domain:
19.8.3.1. Lower Levels (r). Whether attitudes can be influe nced through reading depends on such factors as the writer's skills and the learne r's predisposition. Therefore, reading is recommended for teaching the lower level of the affective domain (receiving and responding) even though there is no assurance any of the higher levels will be reached.
19.8.3.2. Higher Levels (nr). Because of individual differences, it is difficult to be sure that reading materials will actually lead to the desired affe ctive objective. But, we all can identify with the adult learner who is deeply moved or profoundly affected by a written work. Reading, by itself, will not provide evidence of change at the higher affective levels. It may, however, be the vehicle for significant progression up the affective developm ental ladder. Adult learners at the very highest levels can plan their own reading programs for learning.
19.8.4. Factors and Constraints:
19.8.4.1. Minimum Level of Instructor Expertise (h). Because the instructor's only control in this method is the selection of what students read or the direction of their research, the instructor's expertise should be at the higher levels of the cognitive taxonomy.
19.8.4.2. Class Size (Indiv). Class size is a factor only when all members of a class must have access to a limited number of copies of a partic ular reading. With preplanning and modern methods of duplication, such problems are easy to avoid. Some techniques for making reading materials more available to students are textbook issu e (especially a "walking library"), reserve book rooms or shelves, classroom bookshelves, the repr oduction of microforms into readable copy, and multiple library copies. For an effective research program, specialized facilities such as libraries, archives, and reference materials are essential.
19.8.4.3. Evaluation Inherent in Method (no). Evaluation is not built into the method so the instructor must provide evaluation separately.
19.8.4.4. Responsive to Individual Needs (no). — Although reading is highly responsive to the needs and differences of individual students, the in structor must not neglect the factor of student motivation. Many students do not like to read or have not developed disciplined study habits. These factors may be more significant than th e relatively few students who have problems with comprehension or reading speed. Because the instructor is normally not present when the reading takes place, controls and well-defined goals are even more important.

19.9. Self-Paced Methods:
19.9.1. Domains and Levels—Cognitive Domain (HR or r). The wide-ranging applications of the various self-paced methodologies i ndicate potential for all of the cognitive levels. Great numbers of learners have mastered basic factual information as well as advanced concepts and principles throughAFMAN36-2236 12 NOVEMBER 2003 217

the use of programmed instruction and mediated obj ectives from simple knowledge to complex tasks requiring application level and beyond. Materials and methods such as th ese are very tolerant of learners, permitting them to select their own pace of l earning, review the material as they see fit, and redo the lesson until the material has been mastered. Th ese characteristics allow self-paced methods to accommodate individual learning rates, styl es, and methods for cognitive objectives.
19.9.2. Domains and Levels—Skill Development:
19.9.2.1. Motor Skills (r). Self-paced methods of instructi on can be successfully employed in motor skill development, but are particularly well suited for the development of mental skills.
19.9.2.2. Mental Skills (HR). A combination of self-paced materials and instructor intervention is strongly recommended as a highly efficient and effective approach to mental skill development. The same characteristics of these methods that make them well suited for general cognitive development also make them appropriate for mental skill development.
19.9.3. Domains and Levels—Affective Domain:
19.9.3.1. Lower Levels (r). Most of the methods that have ma n-material or man-machine interaction rather than human interaction may be ca utiously recommended for affective development. Self-paced methods may enhance a student's attitu de toward the subject matter, but they are unlikely to affect change in value systems or to help reach higher affective levels.
19.9.3.2. Higher Levels (nr). It is possible, though, to construct a self-paced methodology that will enhance a learner's abilitie s to progress up the lower levels of the affective taxonomy. As more and more progress is made in the use of computer learning applications, it is likely that the higher levels of the affective taxonomy will be more easily reached through such applications.
19.9.4. Factors and Constraints:
19.9.4.1. Minimum Level of Instructor Expertise. Preparing materials for use in the self-paced methods requires a very high level of instructor expertise in the subject matter, perhaps rivaled only by simulation and case study methodology. Self-paced methods also require considerable technical skill in their design. Programmed instru ction and modules demand careful attention to a fairly rigid set of rules as well as considerable development time and field-testing. This requirement for both subject matter and technical expertise puts the various self-paced methods among the most difficult, time consuming, and costly to produce. The educational benefits realized from the methods must be carefully weighed against these factors.
19.9.4.2. Class Size. By the nature of their construction, se lf-paced exercises are directed toward individualized learning. Because most of these exercises are capable of being mass-produced, large audiences can be reached by these methods, although the mode of delivery is individualized.
19.9.4.3. Evaluation Inherent in Method. A basic ingredient in all self-paced instruction is ongoing evaluation of learner progress toward established learning objectives. Many forms of self-paced instruction have carefully prepared systems for pre-, post-, and formative testing. Theevaluation step is often so intermingled in the instructional materials that the learner may not consider it to be a "test" in the usual sense.
19.9.4.4. Responsive to Individual Needs. This method is particularly responsive to student needs and differences. Properly prepared, the me thod can incorporate such sound learning principles as positive reinforcement and immediate fee dback. These principles allow students with var-218 AFMAN36-2236 12 NOVEMBER 2003

ious needs and motivation levels to be successful . In addition, diverse technical treatment such as branching programs (special tracks) provides additi onal capacity to deal with individual needs and differences.
19.10. Questioning:
19.10.1. Domains and Levels—Cognitive:
19.10.1.1. Knowledge (nr). Although possible at the knowledge le vel, questioning is not generally recommended because most questioning results in too much recitation and response in an elementary, rote-learning classroom manner. Other methods may be more productive to present and reinforce knowledge-level material.
19.10.1.2. Comprehension (HR). This method lends itself best to material at this level becausethe instructor can lead the student or class to form concepts, test them, and see their interrelationships through a series of skillfully chosen questions.
19.10.1.3. Higher Levels (r). Although it is possible to question at the higher cognitive levels (application, analysis, synthesis, and evaluation) , these levels lend themselves to more student interaction than is common with one-on-one ques tioning. Questioning can be used to stimulate thinking at the higher levels as a preliminary step for the student.
19.10.2. Domains and Levels—Skill Development:
19.10.2.1. Motor Skills (nr). Except at very low levels of p hysical skill development where an instructor wants to check on simple knowledge of facts or procedures, questioning by itself does not develop motor activity.
19.10.2.2. Mental Skills (r). An effective technique for developing mental skills. Inductive questioning, deductive questioning, and the Socratic method are often chosen as the best ways to foster mental skills development.
19.10.3. Domains and Levels—Affective Domain:
19.10.3.1. Lower Levels (r). Questioning often draws the learner's attention to a new topic or area of study in a very effective fashion. The in teraction of the question- and-answer routine tends to focus the affective concern of the learner on th e chosen topic. There ar e no guarantees, but theinteraction often causes (if that's possible) an adult learner to receive and respond to a new or different topic for discussion.
19.10.3.2. Higher Levels (nr). Questioning is generally not recommended for the higher levels (valuing, organization, and characterization) of the affective domain because it provides for too narrow a range of student response and initiative. However, history tells us that questions in thehands of a master teacher can accomplish almost any affective outcome—including the higher levels.
19.10.4. Factors and Constraints:
19.10.4.1. Minimum Level of Instructor Expertise (h). To fully exploit the potential of the questioning method, instructors should be at the evaluation level. They should be highly skilled because the method requires immediate evaluation of student responses. It also requires expert competence in the subject to see the logical consequences of a line of reasoning or to form new problem-solving approaches. Instructors with especially analytical minds who enjoy theAFMAN36-2236 12 NOVEMBER 2003 219

give-and-take of lively intercha nge will find this method effective in achieving instructional objectives.
19.10.4.2. Class Size (sm). Although some law schools reportedl y use the questioning method to good purpose in very large lecture halls (100 and more students), the method seems to lend itself best to one-on-one or small group (8 to 12) instruction.
19.10.4.3. Evaluation Inherent in Method (yes). The fact that instructors receive immediate response to their questions and are in a position to evaluate these responses before proceeding to the next question rates this aspect very high.
19.10.4.4. Responsive to Individual Needs (yes). If the instructor is able to use a number of spontaneous questions instead of relying on planned questions, the method can be very responsive to student needs and differences.

19.11. Nondirected Discussion:
19.11.1. Domains and Levels—Cognitive Domain (nr). Although the peer-controlled seminar can successfully discuss lower-level or even higher-leve l cognitive materials, there is also the danger thatthe seminar will pool ignorance. Clearly defined objectives as required by ISD and a means of measuring their achievement can, however, provide th e focus for learning at any cognitive level. Such learning can be substantial fo r a group of adult learners.
19.11.2. Domains and Levels—Skill Development (nr). This kind of discussion does not lend itself to developing either motor or mental skills.
19.11.3. Domains and Levels—Affective Domain:
19.11.3.1. Lower Levels (r). A basic use for the peer-controlled seminar is receiving and responding to affective material. It can be very difficult to write affective classroom objectives; therefore, use caution. A possible use of prof essional military education might be for a seminardiscussion following a guest speaker (such as a war hero or a popular leader) whose primary reason for addressing the group was motivational. If properly motivated, the group members might share and reinforce the affective experience am ong themselves. But without strong motivation and interest, this limited objective might not be met a nd the session might just as easily deteriorate into free discussion.
19.11.3.2. Higher Levels (nr). Under the proper conditions and with a group of adult learners, nondirected discussion can be used to further affective learning at the higher levels. Motivated adults pursuing a common goal can share a po werful learning experience with or without an instructor being present to direct the flow of an unstructured discussion. The obvious caution still exists as with the lower affective level, but student-controlled discussion can be and is a powerful tool for affective development.
19.11.4. Factors and Constraints:
19.11.4.1. Minimum Level of Instructor Expertise (na). Not applicable because of the instructor's limited or passive role in a peer-controlled seminar. This is, of cour se, one of its weaknesses.If qualified students are available and properly supervised, such seminars can still be highly successful.220 AFMAN36-2236 12 NOVEMBER 2003

19.11.4.2. Class Size (sm). The small group (8 to 12 students) is probably the most common and workable size for the peer-controlled seminar. Any larger class would probably become too unwieldy.
19.11.4.3. Evaluation Inherent in Method (no). With the instructor playing a passive role, there is little evaluation in the usual sense. However, it is possible to obtain evaluation information if an individual or group product is required.
19.11.4.4. Responsive to Individual Needs (yes). As the seminar is entirely run by the students, it is obviously responsive to their individual intere sts, but not necessarily their educational needs. Be aware that the natural leaders in the group may dominate the class and exclude the weaker students, which will be to their detriment.

19.12. Guided Discussion:
19.12.1. Domains and Levels—Cognitive Domain:
19.12.1.1. Knowledge (nr). The guided discussion is not particularly recommended for simple recall of factual information. Other methods, such as the lecture, reading, or self-directed instruction are more efficient for reaching this level. On the other hand, many learners like to participate in guided discussions because they can interact wi th the content rather than passively taking lecture notes.
19.12.1.2. Comprehension (HR). This method is designed primarily for the comprehension level and is one of the most efficient ways of reaching it. It develops concepts and principles through group process and the unobtrusive guidanc e of the instructor. Properly conducted, the guided discussion also ensures each student learns atthis level, as the instructor can draw out an individual who may not be participating voluntar ily. Unlike free discussion, which probably has no objective and develops solely by the natural di rection the group happens to take, the guided discussion is highly structured, with planned questi ons that lead the group to a specific, predetermined instructional objective.
19.12.1.3. Higher Levels (nr). This method rarely reaches the hi gher levels of learning because students are not required to put the concepts they have learned to use in new situations as in some of the application methods. But, once again, cauti on must be exercised with a group of adult learners. It is not unusual for such a group to pursue pr oblem solving (and its component skills of analysis, synthesis, and evaluation) during a guided discussion. The goals of the group, their motivation, prerequisite skills, etc., must be take n into account when looking to the guided discussion as a tool for developing the higher cognitive levels.
19.12.2. Domains and Levels—Skill Development:
19.12.2.1. Motor Skills (nr). As there is little student "doing" in the guided discussion—only verbal interaction—this method is not gene rally recommended for teaching motor skills.
19.12.2.2. Mental Skills (nr). Once the need to display physical (motor) skills is taken away, the situation for using the guided discussions may change dramatically. As with any method of instruction involving a large amount of verbal inte raction, mental skills may be greatly affected. The direction of the learned change may be small or quite significant, the speed of the change is variable, and the function of prerequisite skills becomes unusually significant. While cautionAFMAN36-2236 12 NOVEMBER 2003 221

should be exercised when using guided discussion with mental skills, to overlook it out of hand would be a mistake.
19.12.3. Domains and Levels—Affective Domain (HR or nr). By participating in a guided discussion, students are exposed to the opinions of others and are forced to defend their personal positions. For this reason, the method is more generally suited to receiving, responding, and valuing than to organization and characterization. The method can and often does work at the higher levels as well, however, and should be considered when le sson planning for higher affective outcomes.
19.12.4. Factors and Constraints:
19.12.4.1. Minimum Level of Instructor Expertise (h). Although it is possible for instructors in guided discussions to be at the comprehension level, ideally they should be at the higher cognitive levels. One primary responsibility of instruct ors is to judge the worth of student responses, since achieving the planned objective is totally de pendent on the use of student responses to form the generalization. Since instructors cannot possibly use all student responses, they must select those most relevant to the concept or principle under discussion. This method is a new one for most instructors, so a certain amount of training is necessary. Once an instructor has the basic idea, there is no difficulty in applying the technique to different situations.
19.12.4.2. Class Size (sm). Although 8 to 12 students might be considered an optimum size, the method can be used very satisfactorily with a slightly larger or smaller group. If the size of the group gets too large, it becomes somewhat harder to control the discussion, thus calling for more instructor expertise to conduct discussions.
19.12.4.3. Evaluation Inherent in Method (no). Students are not normally asked to actually formulate and verbally express their own generalizations; therefore, some other type of follow-on evaluation device is necessary. However, if time is available and students are asked to express their own generalizations, then instructors might be able to evaluate achievement of the objective.
19.12.4.4. Responsive to Individual Needs (yes). This method does an excellent job of meeting individual needs. Each student is encouraged to express opinions on issues and to ask questions about issues raised by others. The instructor is responsible for ensuring all students have the opportunity to participate.
19.13. Practical Exercises—Individual Projects:
19.13.1. Domains and Levels—Cognitive Domain:
19.13.1.1. Knowledge and Comprehension (r). Projects of this type can be used to teach at these levels, but they do not encompass the method's full potential. Students usually need to acquire mastery at these levels before the proj ect starts through other means, such as reading.
19.13.1.2. Higher Levels (HR). This method is one of the best for ensuring learning at the higher levels of application, analysis, synthesis, and evaluation—all of which are difficult to reach by most other means. Properly directed, individual research gives the students maximum flexibility to pursue interests at their own speeds, while at the same time allowing them to reach their capabilities and maximum insights. Indi vidual projects combine well with other methods such as the research seminar that allows students to work i ndependently and also interact with peers and the instructor.
19.13.2. Domains and Levels—Skill Development:222 AFMAN36-2236 12 NOVEMBER 2003

19.13.2.1. Motor Skills (nr). Generally, individual projects are not recommended to develop motor skills as a standalone methodology. While students may sharpen such skills working on a project, other instructional methods such as D-P may be more efficient and effective.
19.13.2.2. Mental Skills (r). These projects may be useful in developing mental skills; the skills could be integral parts of the desired objective . Individual projects are not as highly recommended for mental skills as is the D-P method. The adult learner's level of exper tise may be the deciding factor. It may boil down to an issue of individual expertise—the more the expertise, the more appropriate the individual project as a learning methodology.
19.13.3. Domains and Levels—Affective Domain:
19.13.3.1. Lower Levels (r). There could be an affective influe nce on the student during an individual project. This could result in paying attention (receive) and actually taking some action (respond) toward the material. However, for the most part, receiving and responding skills are assumed to be present before a student undertakes a project.
19.13.3.2. Higher Levels (HR). This method is an excellent mean s to judge attainment of the higher affective levels. Whether a student values certain materials, techniques, or ideas is often easy to determine during a project. The same can be said of organization and characterization. Frequent contact between student and instructor ar e an important part of the method, and there is opportunity for regular feedback and guidance. An aware instructor will find indicators of the higher affective levels in this method.
19.13.4. Factors and Constraints:
19.13.4.1. Minimum Level of Instructor Expertise (h). The higher cognitive levels are desired in this method and students are required to sift through and evaluate data, equipment, or persons. Therefore, the instructor must be operating at the higher levels of the cognitive taxonomy.
19.13.4.2. Class Size (Indiv). The actual instruction and interaction between student and teacher is usually on a one-to-one basis. There is a practi cal limit to the number of individual projects that can be supervised at one time. This me thod is outstanding, but very time consuming.
19.13.4.3. Evaluation Inherent in Method (yes). Individual projects are superb because student and teacher interact regularly. Finished portions of the project are evaluated and feedback given as the endeavor proceeds, so it is clear that stud ent progress is monitored constantly. Obviously, knowledge of what students must be able to do and how they are doing it is readily available.
19.13.4.4. Responsive to Individual Needs (yes). With the continual interchange betweeninstructor and student, instructors will find it easy to identify and deal with students' special weaknesses, interests, and needs.
19.14. Practical Exercises—Field Trips:
19.14.1. Domains and Levels—Cognitive Domain. With some exceptions, field trips are typically used for affective purposes rather than to meas ure cognitive development. For this reason, it may be difficult to visualize how comprehension, applicati on, and the higher levels can be developed in the student as the result of a field trip. Certainly so me cognition may develop—in some cases a great deal of cognition—but this method is not highly reco mmended for any higher than the knowledge level because there may be other more appropriate methods (such as lect ure, reading, etc.) available.AFMAN36-2236 12 NOVEMBER 2003 223

19.14.2. Skill Development (nr). Typically, the field trip provides no opportunity for the development of either motor or mental skills.
19.14.3. Domains and Levels—Affective Domain (HR). Highly recommended as a method to influence all levels of student affe ct. The field trip experience may ga in the attention of some students (receiving) and evoke responses in others (responding). Value or worth may be seen in an object or idea as the result of a field trip and, while it is diff icult to attain, this method can be a strong factor in achieving the highest affective levels.
19.14.4. Factors and Constraints:
19.14.4.1. Minimum Level of Instructor Expertise (c). An instructor does not need to be at the highest cognitive levels to partic ipate in a useful field trip for students. However, the instructor must be at least at the comprehension level to make proper use of the method. Selecting the field trip location and activities, asking meaningful questions during the trip, and possibly discussing the trip afterward all require comprehension on th e part of the instructor. Because the field trip may be used primarily for affective purposes, the in structor should be at the higher levels of the affective taxonomy as well.
19.14.4.2. Class Size (sm). Many variables determine optimum class size for a field trip. All students need to observe and experience the important elements of the field trip and be able to interact with either the instructor or an expert at th e field trip site. Therefore, the number of students per instructor should be small.
19.14.4.3. Evaluation Inherent in Method (no). As an experiential teaching method, formal evaluation is generally not possible. Some additional form of evaluation would be necessary to measure achievement of the instructional objective.
19.14.4.4. Responsive to Individual Needs (no). Generally speaking, this method does not meet an individual student's needs. The typical field trip is a highly stru ctured and scheduled affair that leaves little time for students to satisfy their needs with regard to the topic. Adding a question-and-answer period or having a post-field trip discussion will help the students.
19.15. Practical Exercises—Experiential. (Experiential exercises are learner development games, such as role plays, simulation games, organizational and management games, a nd in-basket exercises.)
19.15.1. Domains and Levels—Cognitive Domain:
19.15.1.1. Knowledge (nr). Learner development games are not usually recommended for imparting knowledge to students. As a matter of fact, knowledge is presumed to be a prerequisite for this method in most cases.
19.15.1.2. Comprehension (r). While learner development games can be the vehicles through which comprehension occurs, most of the games assume a prerequisite of comprehension-level functioning.
19.15.1.3. Higher Levels (HR). A major strength of learner development games is that an opportunity exists for students to operate at the highest cognitive levels within a low-risk environment, as in war games, role playing, and computer simulations. Most of these games require some sort of analysis, synthesis, and evaluation, while practi cally all of them require the application of concepts and principles in new situations.224 AFMAN36-2236 12 NOVEMBER 2003

19.15.2. Domains and Levels—Skill Development (r). Hardware simulators (such as a cockpit or missile trainer) are excellent adj uncts to both physical and mental skill developments (coordination in flight, locations of instruments and c ontrols, emergency procedures, etc.).
19.15.3. Domains and Levels—Affective Domain (HR). Another strength is the direct relevance to affective development. Learner deve lopment games can be instrumental in achieving all affective levels. For example, the purpose of observing or partic ipating in one of these games may be to get students' attention and action (recei ving and responding). Since these games deal with putting learned material to use, the higher affective levels are al so addressed. Students begin to see worth or value in certain principles, ideas, equipment, etc.. This insi ght is the stepping stone to the higher affective levels.
19.15.4. Factors and Constraints:
19.15.4.1. Minimum Level of Instructor Expertise (h). The instructor and/or instructional designer in a learner development game must be at the highest cognitive level possible to use these methods properly. Since students ar e often operating at the highest le vels, the instructor must also be with them at these levels. Though more passive in this method than in some others, the instructor must be ready to act and answer upon request or whenever the situation dictates.
19.15.4.2. Class Size (sm). While there may be large numbers of students involved in any given learner development game, there must be few stude nts per instructor during the game itself. Otherwise, students will flounder and not get all they might or should from the game.
19.15.4.3. Evaluation Inherent in Method (yes). Most learner development games provide immediate feedback to instructor and student alike. Unless one is interested in measuring knowledge or comprehension prerequisi tes or game outcomes, the proof of learning in a learner development game is the adequate handling of whatever task is involved. No other evaluative device is usually required.
19.15.4.4. Responsive to Individual Needs (yes). Because of the game/student and instructor/ student interaction and immediate feedback, th ese games provide for individual student needs. Various learner development games (or portions of them) can be repeated until adequately dealt with by the student (achievement of instructio nal objectives). Weaknesses and strengths can be quickly identified and worked with as appropriate.

19.16. Case Studies:
19.16.1. Domains and Levels—Cognitive:
19.16.1.1. Knowledge (nr). Case studies are not recommended for teaching at this level. Their efficiency compared to other methods is poor.
19.16.1.2. Comprehension (r). Although the case study can be used to teach to this level, other methods such as the guided discussion or the controlled conversation me thods are normally more efficient.
19.16.1.3. Higher Levels (HR). This method is one of the best for reaching the higher levels ofthe cognitive domain. It is very effective for applying a single principle or concept and for evaluating a situation involving total analysis and synthe sis. Its primary limitation is the absolute necessity for students to have a thorough comprehension of the concepts and principles involved beforeAFMAN36-2236 12 NOVEMBER 2003 225

the case study begins. For this reason, the method is often used as a "capstone" to other methods at the end of a block of instruction.
19.16.2. Domains and Levels—Skill Development:
19.16.2.1. Motor Skills (nr). Not recommended in this area, as the D-P is far more efficient in developing motor skills. Also, there is very little (if any) motor skill activity occurring in a case study.
19.16.2.2. Mental Skills (nr). Cases can be adapted to include skills that may have their roots in other methods—guided discussion, simulation, D-P, etc. Such simple, logical adaptations, designed to suit an audience of motivated adult learners, gives the creative teacher the opportunity to gain the maximum effect for learning within the group. Statistical applications for a case study on total quality management (TQM), application of ISD principles in an Air Force curriculum development case, etc., are just a few examples of how mental skills development can be worked into a case study situation.
19.16.3. Domains and Levels—Affective:
19.16.3.1. Lower Levels (r). Using the case study in this domain very closely parallels its use in the cognitive domain. Although a good method for re aching lower-level objectives, other methods such as the guided discussion and field trips may be more efficient if the affective objectives are solely at the lower levels.
19.16.3.2. Higher Levels (HR). Highly recommended for the higher levels as students are forced to recognize the role of systematic planning and objective approaches to problem solving. They are also forced to defend positions that in many cases are a synthesis of their own value systems.
19.16.4. Factors and Constraints:
19.16.4.1. Minimum Level of Instructor Expertise (h). Because of the high levels of student participation associated with this method, the instructor should be at the higher levels to evaluate student responses. The factor of instructor expert ise is one of the primary limitations of the case study method in schools with large faculties where all students are receivi ng the same high level of instruction in seminars demanding more fr om all instructors than may be possible.
19.16.4.2. Class Size (sm). Class size in the case study method is perhaps more a function of time than any other one factor. All students must be prepared to present their thoughts on the subject under consideration. The other students or the inst ructor must have sufficient time to critically analyze the proposals presented by each student. Although some authors writing on the case study recommended 8 to 12 students as an optimum numb er, a variation is used quite satisfactorily in large classes where adequate time and facilities are available. The average seminar size in Air Force professional military edu cation schools is generally very well suited to the use of this method.
19.16.4.3. Evaluation Inherent in Method (yes). Students must give their own proposals or at least critically analyze the proposals of others; th erefore, the instructor has an excellent opportunity to evaluate student achievement of objectives on the spot. Under optimum conditions, no followup evaluation may be necessary. In a great number of instances, written or selection tests are successfully used to measure achievement of learning outcomes.226 AFMAN36-2236 12 NOVEMBER 2003

19.16.4.4. Responsive to Individual Needs (yes). Because students are free to develop their ownapproaches to the situation, the method is easily accommodated to individual student needs, differences, and creativity.
19.17. Summary. Using the teaching methods grid ( Table 19.1. ) can be of real value in determining the appropriateness of proposed or exis ting methods of teaching. Used with some caution, the recommendations within the grid will be useful for most school s and courses. Such recommendations might form the basis to identify periods or blocks of in struction that may require further analysis.
19.17.1. A situation inconsistent with the recommendati on of this grid is not necessarily a poor one. The methods already being used may be as good as or better than those re commended. However, situations markedly different from these recommendations should be examined to ensure the methods chosen are not a result of error or inertia but are the result of creative teaching and the presence of adult learners. 19.17.2. This grid is recommended for use in an y educational setting. The impact of methodologyupon learning can be significant. Appropriate methodology should enhance the learning of knowledge, skills, and attitudes. Using th e teaching methods grid can be ofvalue to maximize the achievement of objectives in the educational setting.AFMAN36-2236 12 NOVEMBER 2003 227

Chapter 20

INSTRUCTIONAL MEDIA

20.1. Introduction. In the preceding chapters, we learned that selecting a method for a particular teaching situation is far from an accidental process. In orde r to reach our teaching objective most efficiently, we must carefully consider the advantages and limitations of each method. The same care is needed to select instructional aids that will support the lesson and make your teaching task easier.

20.2. Instructional Aids: 20.2.1. Besides giving support for our teaching, instructional aids have other roles. They can be used to "teach" an entire lesson. This most often occurs with television or computer programs. Designers of courses that rely heavily or primarily on audio and vi sual materials to deliver instruction must be particularly concerned about the validity of their courseware. But, in most Air Force instruction, instructional aids are used primarily to support classroom lessons. The material in this chapter should help you select and use the various instructional aids available for today's classroom instructor. 20.2.2. In a broad interpretation, instru ctional aids include anything thatassists students in their learning, even instructors and textbooks. In this chapter, however, we will look at the more basic forms of instructional aids (instructional media) that give audio or visual support in a classroom setting. While detailed coverage of all aids is beyond the scope of this manua l, we will describe the primary advantages and disadvantages of each.
20.3. Selecting Instructional Aids. What are the factors an Air Force instructor must consider in selecting instructional aids? Ideally, the type of aid s hould be determined by the lesson objective. For example, if a learning outcome requires students to identify or recognize an item or process, the instructor wouldlikely select a medium that visually displays the item or process. If the objective is for the student to recognize an alarm bell or a particular set of sounds, th en, or course, an audio tape or some other audio device is needed. 20.3.1. Having made this decision, the next factor to determine is which form of visual aid is most appropriate, by asking the following questions: Must th e actual item or process be seen, or will a replica or simulation do as well? Is motion required? Is color required? Will a large number of students view the aid at the same time and place, or will the number and the location vary? Have existing aids been surveyed to determine whether the materials required are already available? Is the equipment or other capability available to display or present th e instructional aid? The answers to these questions will help you select your aids. 20.3.2. Our instructional aid options are as broad as th e ocean is deep. Instructional aids are as simple as writing on a chalkboard or as complicated as usi ng a computer to create an electronic presentation. However, with any form of instructional aid, there are rules of thumb that must be followed. These rules—and common sense—will assist you in creating an effective instructional aid. 20.3.3. An important point to remember when creating instructional aids is to constantly put yourself in your students' positions. Walk through the classr oom to view or listen to your aids from your students' perspective. Ask yourself the following quest ions: Is this instructional aid doing the job I want it to do? Are the students able to read the inform ation I'm trying to present? Can the students read my228 AFMAN36-2236 12 NOVEMBER 2003

writing? Give each of your instructional aids an honest evaluation before you present it. The answers to these questions will help you select your aids.
20.4. Types of Instructional Aids. The list of instructional aids you can use might be endless; however, we will look at the ones most commonly used. One such aid in the classroom is the writing board. These are chalkboards, dry erase boards, turn charts, a nd other similar items. Other aids you may use in the classroom are overhead transparencies, posters, magneti c cards, video clips, and electronic presentations. The rule of thumb for these different types of aids can overlap. One instructional aid to keep in mind is simply writing.
20.4.1. Handwritten Aids. Not everyone possesses the perfect penmanship we all want to display. Great penmanship or freehand is a skill that comes from many hours of practice. It is important toknow your writing limitations. For the most part, slowing down and taking your time is the key ingredient required to having better penmanship. 20.4.1.1. The rule of thumb for having a legible instru ctional aid is to always print versus using the cursive style. There are devices you can use to assist you in achieving readable print. Stencils or a system called the Leroy writing set can be f ound in your office supply store. Both systems are time consuming, require patience, and can be restri ctive as to size. Freehand is by far the easiest, fastest, and the least restrictive. Remember, your penmanship skill will develop as you practice, practice, practice. 20.4.1.2. For spontaneous writing, as you might need in a case study, remember to slow down and accurately develop each letter of the alphabet. Y ou might want to read out loud what you have written. This will ensure your students understand wh at you are writing for them to read. The size of your printed information is veryimportant. The rule of thumb is 1 inch high for every 10 feet of classroom, up to 30 feet. If you are going to teach st udents who are sitting more than 30 feet away, consider using a different type of instructional aid. An easy test is for you or a coworker to sit and evaluate your visual aid from the distance of the farthest student. This will allow you to adjust your instructional aid as required.
20.4.2. Electronic Presentations. Electronic presentation is an option in wide use. Electronic presentations are put together with the use of a com puter system. The computer gives us many options in letter style (font), size, and colors. However, not all fonts, sizes, and colors are appropriate for an effective visual presentation.
20.4.2.1. Font Selection. There are a multitude of fonts (letter styles) to select from; however, the ones that seem to work best are the block type f onts, such as Arial, Avant-Garde, and Times New Roman. If a fancier font is desired, boldface or italicize fonts. Remember, simple fonts allow your students to read the information comfortably.
20.4.2.2. Font Size. When choosing a font size, use 32- to 40-point size for the titles and between 20 to 24 points for your main and subpoints. Any sm aller, and your information will become difficult for students to read.
20.4.2.3. Color Selection. Color has a significant role in cr eating effective instructional aids. Always use contrasting colors. If you choose a da rk font, use a light background, and if you select a light font then select a dark background. Neve r choose a font color similar to your background color. This makes reading the information difficult because your words blend into the background.AFMAN36-2236 12 NOVEMBER 2003 229

20.4.2.4. Presentation Design. When creating your presentation, be consistent in design. If you decide to use yellow print on a blue background with your unit logo on the lower-right side, continue to use this same design throughout the entir e presentation. Changing the design of each slide can become a distraction for your students. Also, using a different background color on each slide may cause your students to wonder why you are pres enting information this way. They will question the meaning of the slides, such as, "Is the red slide more important than the blue slide or is the yellow slide the one I need to remember?" Staying consistent helps eliminate confusion—it's important in all aspects of your aid.
20.4.2.5. Emphasizing Key Points. The way you emphasize key words or ideas can also become a distraction for some students. The computer system has given us many options in this area as well. You might emphasize key words or ideas by boldfacing, underlining, italicizing, and increasing the font size. Again the key to success is c onsistency. Choose one of the options and stick with it throughout your presentation. This will do away with any and all distracti ons related to what is important.
20.4.2.6. Using Clipart. The computer has made everyone an artist. You can now easily dress up your slide with the click of a button. Only use ima ges that relate to the information presented on the instructional aid. Don't just use a piece of c lipart for the sake of having something on the aid. If the clipart doesn't enhance your instruc tional aid information, then don't use it.
20.4.3. Transparencies. Overhead transparencies are an old technology surviving the test of time.Transparencies can be created in different ways. You can create them with the assistance of the computer or they can be handwritten. Either way provides an easy, effective instructional aid. Computer-generated transparencies can be developed by using electronic presentation software and printed out on transparency, or they can be printed on pape r and copied onto the transparency. The same rules of thumb for electronic presentations also apply to computer-developed transparencies as well. For handwritten transparencies, use pens designed for wr iting on transparencies. They're available at any office supply store. Always evaluate your transpar encies by putting yourself in the students' positions.
20.4.4. Video Footage. Video clips are being used more and more within the classroom. They are the perfect aid to give your students a moving example of a subject. The video clip is a tool to support you in helping your students gain a better understanding of the material. A video clip should only be a few minutes in length, and should not extend through your entire presentation time. To be effective, the selected clip should be in good taste and recorded in good quality. Follow the copyright guidelines provided in Title 17, United States Code, Copyrights

. 20.4.5. 35 mm Slide Projector. 35 mm slides are a good instructional aid racing toward extinction. Many auditoriums are converting to electronic presentation systems and doing away with the 35 mm slide projector. However, there is good news for those who still have their favorite presentations in the 35 mm slide format. Slides can be scanned into the computer and transformed into an electronic presentation.
20.4.6. Handouts. Handouts are effective and very reliable. A handout can be created by printing out an electronic presentation on paper or just handwri ting your main points on a sheet of paper. A handout should give the student something to refer to at a later time. It can be the key to a successful presentation if the electronic options are not possible.230 AFMAN36-2236 12 NOVEMBER 2003

20.5. Proofreading. Be on a constant lookout for misspelled wo rds or grammatical errors in all visual aids. Computers have software installed to check for these things, but the human eye is the best line of defense when it comes to checking your instructiona l aids. Have a coworker look over your work with a careful eye. By doing this you are almost certain to have an error-free instructional aid ready to support your lesson objective.
20.6. Distance Learning. Everything discussed in this chapter is steered toward the traditional classroom setting where the instructor and the students are physically in the classroom. However, distance learning is another classroom presentation medium in creasing its popularity in the Air Force. Distance learning is defined as a structured learning that takes place without the physical presence of an instructor. This definition covers various formats. 20.6.1. The live broadcast, video production, or other similar formats for instructing creates different design rules for constructing instructional aid support. The instructional aids used in a traditional classroom setting may not be effective in a distance learning setting. The use of color, size, and creativity are definite areas of interest. The rules fo r building instructional aids for distance learning are just as broad as those for traditional classroom instructional aids. A couple of rules to keep in mind are: (1) KISS--Keep It Short and Simple; and (2) KILL--Keep It Large and Legible. Remember, the instructional aid must be seen on a television or a projection screen. 20.6.2. If you are faced with the possibility of teachi ng in this environment, realize you are going to encounter different rules of engagement. Prepar ation, coordination, and be ing open-minded are key toa successful presentation. Meet with the experts in this unique area; they will help you present yourself and your lesson in a professional manner. 20.6.3. Frequently, a trade-off will be necessary. Fo r example, the cost of producing a color film may be prohibitive or a closed circuit television system may not be available. Because the availability of equipment and material may impact the teaching method we select, it is important to combine the method and instructional aid selection process. When a long lead-time or budget limitation is involved, the aid selection process may have to occur early in the instructional system development process.
20.7. Summary. It doesn't take an enormous amount of work to create an effective instructional aid. Simple rules of thumb and common sense are good tools to have in your academic toolbox to create the aid needed to support your valuable information. Constantly put yourself in the students' positions and evaluate your instructional aids. Always proof, reproof, and final proof your instructional aids for effectiveness and accuracy. The result will be a powerful tool to bring your students to a higher level of learning.AFMAN36-2236 12 NOVEMBER 2003 231

Chapter 21

INTRODUCTION TO EVALUATION

21.1. Introduction. Instructors continually evaluate their students in one way or another. This practice provides the instructor with the necessary feedback to judge whether the teaching is effective or not. The evaluation may be part of a formal testing program or it may be quite informal. It may be as elaborate as a simulation of a real-life problem or situation or as simple as questioning a single student. Teachers may evaluate just to see how students are progressing or th ey may need to give grades and make pass or fail decisions. So, in this sense, every instructor must be an evaluator. 21.1.1. Not all instructors, however, are involved in actually writing and preparing tests. However, they usually give tests or at least make informal ev aluations of students in and outside of class. Therefore, all instructors should be thoroughly familiar w ith evaluation procedures and principles that govern the creation of test items to determine whether students are successfully learning the material presented. 21.1.2. We will now propose a process for writing test items and testing instruments that is informative and relevant—criteria for telling us what the student has learned. Chapter 3 provided a process for writing student-centered objectives and tests. This chapter on evaluation and Chapter 22 , Chapter 23 , Chapter 24 , and Chapter 25 require our understanding of material presented in earlier chapters, specifically Chapter 3 , Chapter 4 , Chapter 5 , Chapter 7 , Chapter 8 , Chapter 9 , and Chapter 10

. 21.1.3. The process includes all the steps necessary to plan and prepare student-centered objectives and tests; each step affects the others. Level-of-lea rning objectives (Step 1) are of little value in planning if we can't write them as behavioral outcome s (Step 2) because the behavioral outcomes are the visible representations of the objectives. Test item s (Step 3) flow naturally from the samples of behavior. A change in any of these steps has a ripple effecton the other steps. All of the steps in this process are interdependent; therefore, a general or specific objective is not complete until we write test items to measure it.
21.1.4. We need to master skills of constructi ng and analyzing tests an d other measurement tools. While all instructors may not require the same depth of understanding as the evaluation specialists, they all need basic evaluation skills to help them do their best in the classroom. We teach more efficiently when we understand the way our students will be evaluated. We tend to teach the learning outcomes more directly when we understand the way the outcomes will be measured. 21.1.5. Whether or not we have the opportunity to write test items, we must understand evaluation terms, principles, and practices. Our students expect us to prepare them for evaluation. Therefore, it is our obligation to know as much about their evaluati on system as possible so we can teach better, counsel more effectively, and be more responsive as Ai r Force instructors. Even if we do not construct our own tests, with a knowledge of these principles we are better able to provide feedback to evaluation specialists on behalf of our students.

21.2. Nature of Evaluation. Evaluation is an important part of the learning process. In educational evaluation we define, observe, measure, or judge learned be havior. Once instruction begins, we need to evaluate constantly to dete rmine what and how well students learn. Only with this information can we intelligently plan or modify our instruction.232 AFMAN36-2236 12 NOVEMBER 2003

21.2.1. Educational evaluation is a systematic pro cess of judging how well i ndividuals, procedures, or programs have met educati onal objectives. However, we can evaluate only learned behavior we cansee, hear, or in any other way sense. Moreover, through such observations, we can include only samples of student behavior as a basis for judgment. To be reliable and effective, then, the samples we evaluate must be representative of the student's overall ability in the measured area. 21.2.2. Furthermore, that evaluation must be a fair one. Instructors see students differently because of differences in expectations, intelligence, learning ability, aptitudes, attitudes, job knowledge, and ability to verbalize. We must try to control the effect s these differences might have when we establish and operate an evaluation program. Not recognizing our differences and their effects on our judgment can result in unfair and inaccurate requireme nts for measuring student achievement. 21.2.3. Besides the issue of fairness, we need to en sure our observations are objective. We can often observe the students' progress toward course objectives in class disc ussions, interviews, term papers, laboratory exercises, special projects, and tests. But this is where care is needed. We must weigh all such observations dispassionately in our overall evaluation program. Only by taking this precaution can we minimize the effects of our subjective judgm ents that often influence informal day-to-day evaluations and make them invalid. Otherwise, we may tend to remember only favorable student responses that indicate progress toward learning goals and overlook unfavorable responses. Or we may remember an unpleasant situation with another student that overrides our judgment of a current brilliant essay response. 21.2.4. To avoid the pitfall of subjectivity in the cl assroom, we need to follo w a systematic approach in evaluating student achievement. An evaluation progr am planned in advance is less subjective to the personal bias of individual instructors. A planned system of evaluation will measure what was taught rather than what might or might not have been taught at the whim of the instructor.

21.3. How We Evaluate. We use data from evaluation programs in two ways: to compare students to stated objectives and to compare students to each othe r. If our goal is to compare the achievement of our students to specific objectives, we should use crite rion-referenced or objective-referenced testing and analysis programs. ISD encourages criterion-referenced testing and analysis. 21.3.1. In contrast to this type of evaluation, there are norm-referenced programs for comparing the performance of individual students to the group. Th ese programs usually result in rank ordering students, typically represented by le tter grading. We must understand the characteristics of each method to determine which to adopt.
21.3.2. Chapter 22 and Chapter 24 describe proper techniques for constructing paper-and-pencil tests and performance rating instruments. Alth ough we use similar construction principles for norm-referenced and criterion-referenced evaluati on programs, the analysis techniques are quite different. Chapter 25 addresses criterion-referenced test analysis procedures and Chapter 26 covers norm-referenced procedures. Measurement instruments constructed for use in one system may be used in the other, but normally we do not use a single set of instruments for both criterion- and norm-referenced analyses.

21.4. Characteristics of Evaluation. The instruments we use to measure achievement must be reliable, valid, objective, comprehensive, and capable of diff erentiating. Any given test or rating scale may possess each of the characteristics in varying degrees. However, few Air Force tests are completely valid or invalid or completely reliable or unreliable. Our role as evaluators is to select those instruments with theAFMAN36-2236 12 NOVEMBER 2003 233

highest levels of the characteristics that give us the most useful evaluative information in a given setting. What follows is a detailed expl anation of each characteristic:
21.4.1. Reliability. A reliable measuring instrument is one that yields consistent results. If a tape measure provides identical measurements each time we use it to measure a certain length, we consider it a reliable measuring instrument. An unreliable instrument will not yield consistent results. For example, an altimeter that isn't set to the local barometric pressure, a steel tape that expands and contracts with temperature changes, or cloth tapes affected by humidity will not yield reliable measurements. 21.4.1.1. No instrument is completely reliable, but some instruments are more reliable than others. We can sometimes determine the relative reliab ility of certain instruments, however, to aid ourselection of testing media. For example, we can determine, to some extent, the reliability of a thermometer by using it to take several readings of th e temperature of a fluid maintained at a constant temperature. Discounting our own reading errors, we should question the thermometer's reliability if we don't get consistent readings. 21.4.1.2. Reliability means the same whether we apply the term to balances, thermometers and altimeters, or educational tests and ratings. It re fers only to the consistency of results obtained from a given instrument. However, measuring the reliability of a rating scale or examination is more complex than measuring the reliability of a mechanical device. If we use the same calipers at different times to measure the bore of the same cylinder, our results should be almost identical each time because the size of the cylinder does not change appreciably between measurements. However, change occurs often during educat ional evaluation wherein knowledge, understanding, and skills do not remain constant. For example, in educational settings, students change between two attempts on the same test because they ga in new knowledge and understanding. Therefore, applying the reliability test too strictly might le ad us to the wrong conclusions. Where repeated testing might reflect growth in knowledge, we might incorrectly judge that the differences point to the unreliability of the test. 21.4.1.3. The reliability of a measuring instrument is equally important in criterion-referenced and norm-referenced testing. The results must be consistent whether students are judged against an absolute standard or compared with one another.
21.4.2. Validity. A measuring instrument is valid when it measures exactly what it was designed to measure. Mechanics who use a micrometer to measure th e diameter of a bearing must be sure that the contacting surfaces of the bearing and the micromete r are free from grease and dirt; otherwise, they will measure the coating in addition to the diam eter of the bearing, and the measurements will be invalid. 21.4.2.1. Although a measuring instrument may be highly reliable, it does not necessarily follow that it is valid. In other words, an instrument may simultaneously have a high reliability and a low validity. The instrument with maximum consistency (high reliability) may not measure what we intend it to measure. For example, merchants w ho use false scales may obtain reliable measurements but invalid results, at least from the customer's point of view.
21.4.2.2. An educational test must e xhibit the same characteristic of validity we expect of other measuring instruments. If we want an examinatio n to measure the ability of students to apply a skill, it must measure the ability to manipulate appropriately, and not simply the ability to recall facts about the skill or merely explain the signifi cance of the activity. If we design an examination234 AFMAN36-2236 12 NOVEMBER 2003

to measure how well students understand the operati on of a four-cycle internal combustion engine, we should make sure we're not just measuring th eir ability to read the specifications sheet. 21.4.2.3. We can determine the validity of a measur ing instrument by comparing it with a criterion or standard relevant to the test or test ite m. For example, when the purpose of the test is tomeasure a special aptitude (ability or talent), the test is valid if it requires the student to demonstrate that aptitude under realistic conditions. No written test can validly measure aptitude for piloting an airplane. However, written tests can more validly measure one's ability to navigate since that skill is primarily a pen-and-paper pla nning process. The only valid test for demonstrating aptitude to pilot a plane is a physical test pe rformed in the plane itself or in a simulator (the former yielding more reliable judgments than the latter because of slightly different handling characteristics). 21.4.2.4. Sometimes we verify the validity of a me asuring instrument by comparing results with later on-the-job performance. The real objectives of Air Force instruction, after all, are usually aimed at improving job performance. If the course objectives relate to job performance, then subsequent job performance may be the criterion for dete rmining the validity of te sts used in a course. We can verify our tests by testing our students ag ain, after they graduate, or by giving the tests to experts in the same field. 21.4.2.5. At times, our goal in Air Force schools is not to improve performance at a specific job, but rather to raise the overall intellectual ability of the students or to improve their judgment. Most advanced courses in military and civilian nontechni cal schools seek this type of educational outcome; many measuring instruments used in these schools measure achievement that cannot be related to specific job performance. Consequently, it is hard to determine validity. In theseinstances, the best we can do is to identify analytical steps, processes, and actions that, if accomplished correctly, will lead to the type s and quality of judgment we expect. 21.4.2.6. It should now be clear that when we state obj ectives in terms of specific abilities, the test should require students to demonstrate these abilitie s. Likewise, when we state objectives in terms of factual knowledge, understanding, and application, the test should have students demonstrate behavior we are willing to accept as evidence of their achievement.
21.4.3. Objectivity:
21.4.3.1. This third characteristic of evaluationrelates to the ability to grade with a relative absence of personal judgment or subjectivity. The bi as of the rater or test scorer should not affect the score. For example, a supervisor in a civil engineering squadron must select three workers for a team to complete an emergency construction jo b. Six available people are called into the office for interviews and evaluations. The supervisor can compare their weights, heights, ages, and other physical characteristics that can be rated objecti vely. But these measurements do not tell the supervisor enough because he or she also needs to know something about the workers' experience levels and then judge just how much experience will qua lify them to do the job right the first time. The supervisor needs to know how well they work unde r stress, their emotional stability, their attitudes toward work, etc. In making these evaluations, the supervisor must try to be objective. However, personal considerations, attitudes, and judgments will undoubtedly influence the evaluation of all considerations not directly measurable. Thus, part of the evaluation lacks objectivity because the judgments about personal qualities, attitudes, and the effect of experience on ability are subjective. These subjective judgments need to be disc ounted in the decision as much as possible.AFMAN36-2236 12 NOVEMBER 2003 235

21.4.3.2. In classroom measurement, essay tests te nd to be subjective because our opinions of a student's organization, handwriting, seriousness, etc., may influence the grade we give the test. The grades we give on essay tests may vary from da y to day, student to student, or from one scorer to another. The more objective we can make the cr iteria for judging a test's score, the better graders we will be because students of comparable ab ility will have a better chance of receiving comparable or equal grades. In the case of essay questions, this requirement may preclude judgments of writing style or word choice and force us to look at measurable items, such as inclusion of required facts and definitions, order of presen ting material, etc., to ensure objectivity.
21.4.4. Comprehensiveness: 21.4.4.1. A measuring instrument must liberally sample the skills or knowledge being measured. For example, suppose the owners of grain elevat ors want to classify the grade of carloads of wheat. If they analyze a bushel of wheat from the top of each carload, they will have a poor test because it is not sufficiently comprehensive. Th e wheat at the bottom of the car may be infested with insects. But, if the owners take 50 test t ubes of wheat from different positions in the carload, even though they may analyze less than a bushel of wheat, they will have a more comprehensive evaluation of the contents of the car. By the same token, a contender who only fights rookies or spars with one partner can never really arrive at the conclusion that he's ready for the title fight because he doesn't have enough datato make that judgment. He ha sn't been matched against different skill levels or styles of boxing to comprehensively measure his ability. 21.4.4.2. Similarly, in classroom measurement, a test or rating must liberally sample the skills representative of the stated instructional objectives . An exam should not test just one area of the curriculum in depth at the expense of other areas th at also need evaluation. Just as the owner of the grain elevator must select samples of wheat from different positions in the car, we must be sure we include a representative and comprehensive samp ling of course objectives. In the ideal case, we would measure all course objectives. However, we usually have limited testing time available. Sometimes we write too ma ny noncritical objectives rather than focusing on the critical ones— this would have the same effect of not sampling comprehensively. If we try to limit our objectives to those important enough to measure, we can c onstruct an evaluation program that tests a much larger sample of our more important objectives. In any case, we must deliberately take comprehensive samples of as many objectives as possible.
21.4.5. Differentiation. We must construct measuring instrument s that detect significant differences in ability. For example, suppose a machinist wishes to measure bearings that are slightly graduated in size. If we use a ruler to measure the diameters of the bearings, the differences among them may not be obvious. But if we take measurements with a mic rometer, we can easily differentiate between the diameters of the first and second bearing, the sec ond and third bearing, and so on up the scale. Similarly, we can design classroom tests to differenti ate among our students. There are two methods of differentiation we need to recognize.
21.4.5.1. Norm-Referenced Differentiation. This is the comparison of items to each other. For example, one goal the machinist might have in taking the measurements is to arrange the bearings from smallest to largest. When we want to show th e relative difference in sizes of the bearings, our measurement is norm-referenced. In a norm-referenced evaluation program, a test or rating must measure small and sometimes relatively insignifi cant differences among the elements we're measuring.236 AFMAN36-2236 12 NOVEMBER 2003

21.4.5.1.1. When we need to use the evaluation data to rank order or give a letter grade to students to distinguish the higher achievers from the lower, our goal is to compare individual performances against each other to show rela tive position within the group (comparing to a norm). Often we make this comparison without caring whether or not the individual or the group has actually met the stated objectives, or so metimes there is no stated learning objective. 21.4.5.1.2. When we construct a test or rating scal e to differentiate achievement for norm-referenced analysis, we look for three features: a range of scores, different levels of difficulty, and the distinction between students who are generally low and those who are generally high in achievement. Such a basis for judgment has l ittle significance beyond helping us give letter grades or achievement rewards. It certainly does not tell us what the student learned or whether the student learned what we had hoped.
21.4.5.2. Criterion-Referenced Differentiation. This is the comparison of elements or items to a standard. A more significant way our machinist might have used the measurement of the bearings is to compare each bearing to a standard to find out whether it meets established tolerances. The question then, is "Does each of the items bei ng measured meet the tolerances?" In the classroom we should ask whether each student met the stated objective instead of how the student compared to the average. The former approach to te sting distinguishes criterion-referenced testing from most traditional (norm-referenced) testing pr ograms. The primary function of differentiation in criterion-referenced testing is to make a cl ear distinction between those students who have mastered an objective and those who have not. Rank or dering or letter grading is of secondary importance.
21.5. Determining Means of Measurement. An effective measuring instrument must be based on allfive of the characteristics we've covered—reliability, validity, objectivity, comprehensiveness, and differentiation. In determining appropriate methods of measur ement, we must select instruments that includethe largest representation of these essential characteristics. Now let's turn to another important consideration in evaluation—the purpose of the evaluation. Th e purpose we have for using an evaluation instrument depends on the unique circumstances of the school, the instructor, and the students.

21.6. Reasons for Evaluation. Why are we testing? Do we test to improve instruction, pinpoint student deficiencies, identify specific degrees of skill achie vement, or to predict success on a future job? Our answers to these questions will determine the procedur es needed to reach these goals. If we are evaluating solely to assign grades, measuring instruments will be much simpler. In Air Force schools, however, our primary reason for evaluating is usually to determ ine whether students can perform a job (represented by our stated objectives). Our task, then, is to ensure there is a strong correlation between the objective and the tasks required by the job.
21.7. Nature of Required Behavior: 21.7.1. To determine whether an objective realisticallyrepresents the job our graduates will be taskedto perform, we must ask the following questions: Can the student's behavior be realistically represented by word or pictures on a pencil-and-paper test ? Must students perform in real situations for the evaluation to be valid? Do students need an active or passive knowledge of the material in our course? Will they respond in much the same form as we taught them or do we want them to translate or synthesize the material in some way? Do we need to modify our curriculum and course materials to getAFMAN36-2236 12 NOVEMBER 2003 237

the kind of evaluation we really need? NOTE: Physical performance and essay testing require more time to accomplish and evaluate, so we must plan for them if they are necessary

. 21.7.2. When the knowledge, activity, or skill is important for safety or to protect life, or if it threatens the destruction of an expensive piece of equipment, then we should expect the student to perform correctly the first time, every time on the job. In some fields, such as medical or maintenance, students must perform at least the critical elements of th e required skill under realistic conditions for evaluation. Less important or less critical skills may be evaluated by administeri ng a pencil-and-paper test.
21.8. Availability of Time and Facilities. Are students required to perform the skill during the learning process to certify learning, such as in flight training? If so, we must evaluate as an extension of the learning activity. If students are not required to demonstrat e the exact skill during the learning process, we can save time by not having to design a formal evaluation using on-the-job conditions. This flexibility can save us time, space, and money through our ability to use simulation. In either case, the facilities and equipment available for evaluation ar e essential factors to consider. Time may also influence the decision. For example, how much time must we set aside to evaluate? Do we have the time available to evaluate the objective fully? Sometimes our answers will influenc e our needs and our cons traints will determine our capabilities.
21.9. Availability of Qualified Faculty:21.9.1. When choosing the best means of evaluating what we have taught, we must consider the qualifications of the faculty. Can the teachers judge student performance using a rating scale? Can they judge student responses effectively using less objecti ve instruments such as essay tests and other supply-type tests? 21.9.2. Only when these factors have been considered and weighed can the instructor or evaluator choose the best means of testing students. Quite likely, no single means will accomplish all purposes. Under such circumstances, the instructor must use a variety of measuring instruments or compromise by accepting less than the ideal from the evaluation program.
21.10. Summary. Educational evaluation is the systematic process of judging how well individuals, procedures, or programs have met educational objectiv es. Criterion-referenced testing is concerned with determining educational ef fectiveness and is, therefore, insepara ble from the learning process itself. Every effective instructor must be an evaluator b ecause it is through evaluation—internal or external, formal or informal—that the instructor will get the required feedback that tells whether the instruction is successful. The process begins with a clear definition of objectives, regardless of the means of evaluation. In practice, testing and performance ratings are almostthe exclusive means used to estimate how well students achieve objectives. A secondary purpose of evaluation may be to rank or to give letter grades to students as some measure of general achievement through norm-referenced testing and analysis. 21.10.1. The evaluation process must be systematic and continual. Unplanned or disorganized observations usually lack the desirable characteristics of comprehensiveness, validity, and reliability. To be usable, evaluative judgments must be presented in a form that can be properly interpreted by the school, the instructor, and the student. No matter what means we use, the test should possess, to the maximum degree possible, the five characterist ics of evaluation. They include: validity—measuring what it is supposed to measure as defined by th e objective; reliability—yielding consistent results; objectivity—eliminating personal biases of the scorer; comprehensiveness—measuring a representa-238 AFMAN36-2236 12 NOVEMBER 2003

tive sample of instructional objectives or behavi ors associated with one objective; and differentiation—detecting significant differences in student ac hievement, particularly between those who have and have not mastered instructional objectives. 21.10.2. The basis for choosing one measuring instrume nt over another is the degree to which they exhibit the five characteristics of effective evaluation. A variety of methods should be used to evaluate students correctly. In choosing between any two equally valid means of evaluation, the instructor should decide on the basis of simplicity.AFMAN36-2236 12 NOVEMBER 2003 239

Chapter 22

CONSTRUCTING AND ADMINISTERING CLASSROOM TESTS

22.1. Introduction. Undoubtedly, written tests are the most frequently used means of measuring how well students achieve learning objectives. Ideally, the test items we write will be so mechanically sound that only students who have masteredour course content will be able to answer them correctly. Others, including "test-wise" students, will be able to do no more than guess. Writing such items is not easy, but the results of writing sound test items are highly rewarding. In Chapter 23 we will learn more about how to write test questions that meas ure our instructional objectives. 22.1.1. This chapter contains general suggestions for preparing test items and specific suggestions for writing selection items (multiple-choice, matching, and true-false) and supply items (essay, short answer, and completion). The relative advantages and disadvantages of selec tion and supply items are also discussed in detail. To further assist your understanding of test-construction principles, a poorly written test item and a corrected example follow the discussion of each type of test item. 22.1.2. Finally, this chapter also addresses impo rtant considerations involved in structuring and administering written tests and the scoring of suppl y items. Approaches to the scoring and grading of selection items are discussed in Chapter 26

. 22.2. Suggestions for Preparing Any Test Item. Consider the following general suggestions when writing test items of any type: 22.2.1. Keep the wording simple and direct. Test items are not the place to show off one's vocabulary. State items in the working language of the student and the job. 22.2.2. Avoid tricky or leading questions. Many people attempt to write tr icky items in hopes that only the better students will pick up the trick. Unfortunately, they may not and, therefore, do not receive credit for items they are actually able to answer. 22.2.3. Keep all items independent of other items on the test. No item should give away the answer to another question. Also, the answer to one item should not depend on having to know the answer to another item. 22.2.4. Crucial words or phrases in the stem should be underlined, capitalized, it alicized, or otherwise highlighted. If possible, avoid negatives altogether; it is amazing how many people miss negatives even when they have been capitalized and underlined. 22.2.5. Include sketches, diagrams, or pictures wh en these will present information to the student more clearly than words. 22.2.6. Be fair, clear, and straightforward. If we write and use enough good test items, we will identify the students who have mastered our curriculum and those who have not.

22.3. Selection Test Items. Selection test items require students to select the correct response from a list of responses. Multiple-choice, true-false, and matching are examples of selection items. 22.3.1. Selection test items have three major advantages over supply test items (paragraphs 22.7. through 22.11. ) that require students to furnish their own answers (see Table 22.1. ). First, since students only have to identify the correct answer, tw o or more people can score selection items without240 AFMAN36-2236 12 NOVEMBER 2003

letting personal bias or opinions affect the result. Th ese items lend themselves well to scoring by clerical assistants or a computer.
Table 22.1. Comparison Between the Two Major Types of Test Questions.

22.3.2. Second, selection items take comparatively little time to answer. Students only have to read the item and choose between the responses provided rather than write out their answers. Thus, in a given period of time we can give students more selection items to answer than we could give themsupply-type items. If testing time is limited, this enables instructors to test course material more comprehensively. 22.3.3. Finally, selection items can be readily analyz ed statistically. Since an swers to selection items are either right or wrong, statistical analysis is relatively easy. The quantitative measures referred to in Chapter 25 and Chapter 26 , such as the ease index and the differentiation index, can be computed by machine quite easily. I A B C T Selection (i.e., Multiple E Supply (i.e., short answer Choice, True/False, M Factor completion) Matching) 1 Chance for guessing Student has to furnish information with no help provided. Student may be able to guess the answer or eliminate incorrect

ones. 2 Cover a lot of material in a short period of time Varies with the different supply-type tests. The completion test can cover more of a given topic than the essay test, which provides only a limited sample. Generally true. A large sampling can be obtained within a regular class period. However, the multiple choice may not give the adequacy of the sample of the true/false or matching tests. 3 Easy to score Needs evaluator to make judgment, even with short answer questions. A definite advantage is the machine-scored answer sheet.

4 Easy to construct Except for the essay, which requires an extremely well-worded question to test the required level of learning, the supply-type test item is no more difficult than the selection-type test item to construct. The multiple-choice test is more difficult to construct than all other types of test items. Extreme care must be taken in writing stems and distracters. True/false and matching are relatively easy to construct.
5 Measures all levels of learning Th e essay is the best type of test item for this factor. Completion type items are useful for lower levels only. Well-constructed multiple choice items can measure all levels, while true/false is primarily for lower levels only. 6 Statistically analyzed Cannot be statistically analyzed because variety is too great. All easily analyzed.AFMAN36-2236 12 NOVEMBER 2003 241

22.3.4. A distinct disadvantage of selection test questions is the possibility of successful guessing. Students have a 50 percent chance of correctly answ ering true-false items and about a 25 or 33 percent chance (depending on the number of choices) of an swering multiple-choice items correctly. The percentage for matching items depends upon the format. To offset this weakness, test writers should adhere to the mechanics for writing good test items that help minimize successful guessing.
22.4. Multiple-Choice Items. The multiple-choice item can be used to measure student achievement and works equally well when a test problem has one correct answer or one best answer from an assortment of plausible answers. When possible, use completion a nd essay items to measure such student behaviors as explain, define, and describe. If constraints on the te sting program do not permit use of supply-type questions, multiple-choice items may prove to be a suitable alternative.
22.4.1. Constructing Multiple-Choice Items. Certain standard terms are used in the construction of multiple-choice items. The preliminary sentence that poses the question or states the situation is known as the "stem" (see Figure 22.1. ). Possible answers that can be selected by the students are known as "alternatives" (see Figure 22.2. ). The correct answer is the "keyed response," and incorrect answers are called "distracters." Distracters are de signed to attract less-informed students away from the correct answer. The example in Figure 22.3. illustrates this terminology.
Figure 22.1. Stems.

In preparing or revising the stem of a multiple -choice item, the instructor should observe the following general principles: - Write the stem so it clearly represents the central problem or idea. The function of the stem is to set the stage for alternatives. - Only place the material in the stem relevant to the idea or to the solution of the problem unless selection of relevant material is part of the problem. - Make sure the stem does not reveal the correct response. Avoid clue words or phrases. - Include language in the stem common to all alte rnatives to avoid repetitious wording and to save reading time for the student. - Avoid any wording unnecessary to answer the question. - Avoid negative statements whenever possible because they often confuse the student. - Exercise caution when using the articles "a" or "a n" at the end of the stem. These articles may reveal the correct response if all alternativ es do not conform grammatically to the stem.242 AFMAN36-2236 12 NOVEMBER 2003

Figure 22.2. Alternatives.

Figure 22.3. Multiple-Choice Alternatives.

22.4.1.1. Multiple-choice items usually provide three, four, or five alternatives ( Figure 22.3. ). In most instances, it is difficult to design more th an five plausible alternatives (alternatives that appear correct to a student who has not mastered the subject matter). The alternatives are as important as the stem. To prepare or revise the alternatives to a multiple-choice item, the instructor should observe the following principles: - Make sure that incorrectness is not the only crit erion when developing distracters. Distracters may include any of the following: (1) a response that relates to the situation and sounds plausible but is incorrect, (2) a common misconception, (3) a statement that is true in itself but does not satisfy the requirements of the stem, or (4) a statement that is either too broad or narrow for the requirements of the stem. - Avoid clue words. Students with experience in ta king tests usually recognize that statements containing such words as "all," "always," and "never" are likely to be false. Similarly, statements containing such words as "usually" or "sometimes" are likely to be true. - Make sure all alternatives are approximately the same length. We often use longer statements in the correct alternatives than in the incorrect ones, and this may be a clue to the correct answer. - When alternatives are numbers, list them in ascending or descendi ng order of magnitude. - Make all alternatives plausible to the student w ho has not achieved the required level of learning in the subject matter being tested. - Place correct alternatives in random positions th roughout the total test. The instructor should avoid a consistent pattern of correct answers within a test or in a series of tests. - Avoid using the alternative "all of the above" and/or "none of the above." If you must use them, be extremely cautious. The "all of the above" opti on makes it possible to answer or guess the answer to the item on the basis of partia l information by noting other correct or incorrect alternatives. The "none of the above" option may be measuring nothing more than the ability to detect incorrect answers, which is no guarantee the student knows what is correct.AFMAN36-2236 12 NOVEMBER 2003 243

22.4.1.2. Properly designed multiple-c hoice items do not permit students to easily guess the correct answer. An effective and valid means of di verting the less-informed student from the correct response is to use incorrect answers co mmonly made by students as distracters. 22.4.1.3. Questions designed to measure the knowle dge level of learning should provide only one correct alternative; all other alternatives should be factually incorrect. In questions designed to measure achievement at higher levels of learning, so me or all alternatives should appear to be reasonably acceptable responses, but one alternative is a better response than all the others. In either case, instructions to students should direct them to select the correct or best alternative.
22.4.2. Examples of Multiple-Choice Items. Multiple-choice items may be written in a variety of forms, as shown in Figure 22.4. , Figure 22.5. , Figure 22.6. , Figure 22.7. , Figure 22.8. , Figure 22.9. , Figure 22.10. , and Figure 22.12.

. 22.4.2.1. Stem Written As a Question. (See Figure 22.4. ) Writing the stem as a question is usually a better form than the incomplete stem because it is simpler and more natural.
Figure 22.4. Stem Written As a Question.

22.4.2.2. Stem Written As an Incomplete Statement. (See Figure 22.5. ) Be careful in using this form to avoid ambiguity, clues, and unne cessarily complex or unrelated alternatives.
Figure 22.5. Stem Written As an Incomplete Statement.

22.4.2.3. Multiple Response. (See Figure 22.6. ) In this form, more than one of the alternatives is correct; students are instructed to mark all correct answers.
Figure 22.6. Multiple Response. What is the purpose of the Oak Leaf Cluster? To a. be awarded in lieu of a repeat decoration. b. denote the award of a decoration in lieu of a ceremony. c. denote presentation of a decoration at an official ceremony. d. use with miniature ribbons on the mess dress uniform.

The order in which military persons enter staff cars is a. according to the commander's desire. b. ascending according to rank. c. descending according to rank.

Why are summer days hotter than winter days? In the summer, the244 AFMAN36-2236 12 NOVEMBER 2003

22.4.2.4. Definition. (See Figure 22.7. ) Two varieties of this form are shown. The form illustrated by the first item is normally more difficult for the instructor to prepare and for the student to choose the correct response. The criterion objective used to describe the first item would be quite different than that for the second item.
Figure 22.7. Definition.

22.4.2.5. Stem Supplemented By an Illustration. (See Figure 22.8. ) This form is useful for measuring the ability to read inst ruments, identify objects, use principles in hypothetical situations, etc.
Figure 22.8. Stem Supplement ed by an Illustration. a. Earth is nearer the sun. b. Sun shines almost straight down on the Earth. c. Earth's axis is at a greater angle to the sun.

Validity is defined as a. the degree to which a test item measures the objective from which it was written. b. a consistent percentage of right answ ers for a test item over several administrations. c. the process of developing a test item whic h does not allow for personal bias to be present. OR The degree to which a test item measures the object ive from which it was written is a definition of a. validity. b. reliability. c. objectivity.AFMAN36-2236 12 NOVEMBER 2003 245

22.4.2.6. The Negative Variety. (See Figure 22.9. ) This form always emphasizes the negative word. Avoid using the negative variety because test s should be used to measure what something is instead of what it isn't.
Figure 22.9. The Negative Variety.

22.4.2.7. Association. (See Figure 22.10. ) This form is useful if a limited number of associations must be made. It asks the student to tie one thing to another. (Matching items serve better if a large number of related associations are required.)
Figure 22.10. Association.

22.4.2.8. Problem-Situation:22.4.2.8.1. A problem-situation format poses a realistic set of circumstances leading to a problem for students and asks them to find the correc t or best solution from among the alternatives. The problem-situation may require students to recall and select facts or it may require more analytical and reflective thinking. 22.4.2.8.2. The situation should, however, require the use of applicable course content. As the problem-situation format is discussed under other types of items in this chapter, refer to the principles described in Figure 22.11. to assist you in developi ng this useful format. See Figure 22.12. for an example.
Figure 22.11. Problem-Situation. Which is NOT a surface-to-surface missile? a. Minuteman. b. Polaris. c. Pershing. d. Bomarc.

Which of the following is a rotary wing aircraft? a. OV-10 Bronco. b. UH-1 Huey. c. T-38 Talon. d. C-130 Hercules.

Certain principles should be followed in developing the problem-situation format: - Make both the situation and the problem realistic. Whenever possible, an actual situation should be presented.246 AFMAN36-2236 12 NOVEMBER 2003

Figure 22.12. Example of a Problem-Situation Format.

22.4.3. Suggestions for Writing Multiple-Choice Questions. The checklist at Figure 22.13. contains suggestions for constructing sound multiple-choice questions. Use this checklist as a guide for writing multiple-choice questions, as well as for an alyzing one that has already been written. Use this checklist to identify errors, rewrite items, and compare the rewrite to the corrected version.
Figure 22.13. Checklist for Writing Multiple-Choice Items.

22.4.3.1. Checklist Analysis. The multiple-choice question found in Figure 22.14. violates one or more of the suggestions included in the checklist ( Figure 22.13. ). Using the checklist at Figure - Make both the situation and the problem specific. Allow no confusion regarding details or requirements. - Limit problems so students may respond to them in the allotted time. - In the interest of economy and efficiency in te sting, construct several test items to exploit as many aspects of a single situation as possible. - Add unimportant details at times to determine th e student's ability to select the important facts. - Do not present a situation previously presented to the student.

The following is an example of the problem-situat ion format used with the multiple-choice item type: Sgt Smith and Sgt Jones are in the same academic sec tion. Every time the instructor asks a question, these two individuals banter and bicker among themselves tr ying to prove the other incorrect. This disrupts the entire class, and no one concentrates on the subject athand. According to the text , which type of military discipline should the instructor use? a. Preventive—the instructor should educate th e students on behavior appropriate for the classroom. b. Corrective—the instructor should separate th e disruptive individuals before the situation becomes uncontrollable. c. Punitive—the instructor should warn the i ndividuals that if disruptive behavior continues, punishment will follow.

No. Stem: Yes No

1 Key words and negatives emphasized? 2 Simply and clearly worded? 3 Free of clue words and giveaways? Alternatives: 4 Brief; all common wording in stem?
5 Only one clearly correct or best answer? 6 All distracters plausible?
7 All about same length or arranged by length?AFMAN36-2236 12 NOVEMBER 2003 247

22.13. , let's analyze the mechanical soundness of the test item in Figure 22.14. Read the first question on the checklist and see if the item is satisfactory. Then go to the second question and so forth until you've read all seven questions.
Figure 22.14. Multiple-Choice Test Item.

22.4.3.1.1. (Question 1) Are key words and negatives in the stem emphasized? NO. Normally such words as "probably," "most likely," "best," and "no" in the stem are capitalized, underlined, or otherwise brought to the student' s attention as having particular importance in the context of the test item. Depending on the writer's intent, almost any word in the stem may be a key word.
22.4.3.1.2. (Question 2) Is the stem si mply and clearly worded? NO. This issue is frequently open to subjective judgment; however, in this case it is obvious we should check. A good way to keep the stem simple is to ask other instructors if they understand it.
22.4.3.1.3. (Question 3) Is the stem free of clue words and giveaways? NO. "Essay test" is the only answer that agrees grammatically with the article "an" in the stem.
22.4.3.1.4. (Question 4) Are alternatives brief and is all common wording in the stem? NO. "Test" is unnecessarily included in all alte rnatives. Eliminating repetition in the alternatives often gives the student a more clear and concise answer from which to choose.
22.4.3.1.5. (Question 5) Is there only one clearly correct or best answer? In this item, YES, there is one best answer. We must unders tand the material ourselves to make this judgment. The best way to assure there is only one co rrect or best answer is to write out why it is correct and why the distracters are incorrect. This explanation is called the "rationale." Writing out a rationale for each item not only helps assure one best answer, but also aids us or other instructors administering the test to explain the answers.
22.4.3.1.6. (Question 6) Are all distracters plausible? NO. A performance-rating test is not a written test. Except in obvious cases, this can be a real judgment call. We frequently learn only after the test is given whether distracters are really plausible or not. Then we can see how many students selected each alternative. In this particular item, the last distracter (d. performance rating test) is not plausible.
22.4.3.1.7. (Question 7) Are alternatives about the same length or arranged by length or other logical order? YES. However, inexperienced test item writers usually make the correct answer the longest answer. This is often because they wish to be fair by spelling out the correct answer as accurately as possible. No such effort is required to make an incorrect answer more An instructor wants to use a written test to ascertain a student's organizational ability. Compared with other types of tests, of the follow ing the instructor would probably use an a. multiple-choice test. b. true-false test.
X c. essay test. d. performance rating test.248 AFMAN36-2236 12 NOVEMBER 2003

incorrect. It is also frequent practice to "hide" a correct answer in the middle of the alternatives. The tendency toward writing longer, correct answers can be reduced with practice. Arrange alternatives in logical order to eliminate the tendency to hide answers. When no other logical order is apparent, arrangement in alphab etical order or by length will suffice. In this item, the alternatives could have at least been arranged alphabetically.
22.4.3.2. Checklist Summary. We've answered NO to Items 1, 2, 3, 4, and 6 on our checklist. See Figure 22.15. for appropriate changes to our corrected item.
Figure 22.15. Corrected Item.

22.5. Matching Test Items. The matching item, with several variati ons, presents many advantages of the multiple-choice item. It is particularly useful in measuring understanding of closely related concepts orfacts. The matching item is actually a collection of related multiple-choice items. Thus, the matching format provides a more compact form for measuring the same learning and can allow more efficient use of testing time.
22.5.1. Constructing Matching Test Items. The instructor should observe the following principles in constructing matching test items: 22.5.1.1. Give specific and complete instructions. Do not make students guess what is expected of them. 22.5.1.2. Test only essential information; never test for unimportant details. 22.5.1.3. Use closely related materials throughout an item. When students can divide the set of alternatives into distinct groups, the item is redu ced to several multiple-choice test items with just a few alternatives. This increases the possibility of guessing the correct answer. 22.5.1.4. To minimize guessing by elimination, make all alternat ives plausible. 22.5.1.5. Arrange the alternatives in some logical order. An alphabetical arrangement is common.
22.5.1.6. If alternatives are not to be used more th an once, provide three or four extra to reduce guessing.
22.5.2. Examples of Matching Items. Like multiple-choice items, matching items can be constructed in a variety of forms—equal colu mns, unequal columns, inference matching, cause-and-effect matching, and coded alternatives for scientific reasoning. See below explanations and examples of each in Figure 22.16. through Figure 22.20. Figure 22.21. provides a checklist for writing matching test items. A student's ability to organize material can BEST be measured by using which type of written test? a. completion. b. essay. c. multiple-choice. d. true-false.AFMAN36-2236 12 NOVEMBER 2003 249

22.5.2.1. Equal Columns. When using this form ( Figure 22.16. ), the instructor should either plan to use items in the response column more th an once or not at all to preclude guessing by elimination.
Figure 22.16. Example of Equal Columns.

22.5.2.2. Unequal Columns. This form ( Figure 22.17. ) is preferable when alternatives may not be used more than once. Directions: In the blank before each base in Column A, write the letter from Column B corresponding to the name of the state where it is located. Each state may be used more than once; some not at all. Column A Column B Air Force Base State 1. _____ Eglin a. Alabama 2. _____ England b. Florida 3. _____ Plattsburg c. Indiana 4. _____ Maxwell d. Louisiana 5. _____ Grissom e. New York 6. _____ Wright-Patterson f. Ohio 7. _____ Griffis g. Tennessee250 AFMAN36-2236 12 NOVEMBER 2003

Figure 22.17. Example of Unequal Columns.

22.5.2.3. Inference Matching. The item in Figure 22.18. would probably measure learning at the comprehension level.
Figure 22.18. Example of Inference Matching.

22.5.2.4. Cause-and-Effect Matching. This is a useful form ( Figure 22.19. ) to measure learning at the comprehension level. Directions: In the parenthesis before each statement, write the number of the location in Column B to which the statement in Column A applies. Each location may be used on ce, more than once, or not at all. Column A Column B ( ) 1. The eastern end of this island is located directly south from the 1. Bermuda easternmost part of the United States. 2. Cuba ( ) 2. Would be crossed in a direct flight from Key West to the Panama Canal. 3. Haiti ( ) 3. Etc., to a maximum of nine statements. 4. Jamaica 5. Nicaragua

Directions: Which of the following characteristics of a good test are implied or referred to in the statements below? Fill in the space on your answer sheet corresponding to the letter of the term you believe is implied or referred to by the speaker. Each of the characteristics may be used once, more than once, or not at all. a. Differentiation b. Validity c. Objectivity d. Comprehensiveness _____ 1. Well, I've finally finished constructing this test and it contains items at all levels of difficulty. _____ 2. After grading this test, I have the feeling that even the students who did well don't understand the subject material. _____ 3. Thirty test items cover this entire unit? _____ 4. I wonder if this test will really tell me what the students know about the subject matter? _____ 5. Be sure you follow the key because it is the only fair way to judge students. _____ 6. Etc. (more statements can be added)AFMAN36-2236 12 NOVEMBER 2003 251

Figure 22.19. Example of Cause-and-Effect Matching.

22.5.2.5. Coded Alternatives for Scientific Reasoning. This type of matching item ( Figure 22.20. ) can be used to differentiate the achievement levels of better students. Items of this type normally measure achievement to at least the comprehension level.
Figure 22.20. Example of Coded Altern atives for Scientific Reasoning. Directions: Listed below are certain facts followed by several statements. Fill in the space on your answer sheet corresponding to the letter (a, b, or c) that represents the best statement of the relationship between each statement and the fact that precedes it. a. The statement is the cause of the fact. b. The statement indicates a result of the fact. c. The statement is not related to the fact. Fact: A flash of light occurs. Statement: 1. A roar of thunder can be heard. 2. Electricity passed between clouds and the earth. Fact: Metals, generally speaking, expand when heated. Statements: 3. Like poles of a magnet repel each other. 4. After driving the car for several miles, the indi vidual found it difficult to remove the spark plugs.

Directions: Below are some hypotheses and methods for testing them. On your answer sheet, fill in the space corresponding to the following conditions: a. The item directly helps to prove the hypothesis true. b. The item indirectly helps to prove the hypothesis true. c. The item directly helps to prove the hypothesis false. d. The item indirectly helps to prove the hypothesis false. e. The item is unrelated to pr oof or disproof of the hypothesis. Hypothesis: Thunder causes rain. 1. Observations show that, during a rain storm, it always begins to rain immediately after a clap of thunder. 2. The meeting of cool and warm air masses simultaneously causes thunder and precipitation. Hypothesis: High temperatures for a few days cause rain. 3. Heated water in a pan evaporates. 4. Many desert areas exist with virtually no vegetation.252 AFMAN36-2236 12 NOVEMBER 2003

22.5.3. Suggestions for Writing Matching Items: 22.5.3.1. The sample test at Figure 22.21. violates one or more of the checklist questions in Figure 22.22. Again, please use the checklist to analyze this sample test.
Figure 22.21. Sample Matching Test Item.

Figure 22.22. Checklist for Writing Matching Test Items.

22.5.3.2. As we can see by entering NO in Questions 1 through 8 in Figure 22.23. , what this item primarily needs is a set of inst ructions plus a little cleaning up. Check the corrected version in Figure 22.24. 1. Uniform Code of Military a. Establishe s rights, privileges, rules, and punishment for military Justice personnel. 2. Status of Forces Agreement b. Provides for controls over duties, customs, and privileges of military personnel in a foreign country. 3. Command c. Implements controls over duties, customs, and privileges of military personnel in a foreign country.

No. Instructions Yes No

1 Clear what is to be matched? 2 Where and how to match? 3 Indication of number of times each alternative can be used? 4 One column contains more entries than the other?
5 Spaces provided for responses? 6 Column requiring more reading on the left?
7 Each column composed of similar materials and appropriately titled? 8 Alternatives arranged in some logi cal order (alphabetical, numerical, or chronological)?AFMAN36-2236 12 NOVEMBER 2003 253

Figure 22.23. Questions to Ask.

Figure 22.24. Corrected Sample Matching Test. Question 1. Makes clear what is to be matched? NO Question 2 . Indicates where and how to match? NO Question 3 . Shows number of times alternatives may be used? NO Question 4 . One column is longer than the other? NO (When one or both of these last two suggestions is ignored, it usually means there is one and only one correct response in the second column to each problem posed in the first column. Therefore, students who can match any alternative find it easier to match remaining alternatives.) Question 5 . Provides space for responses? NO Question 6 . Column requiring more reading than the other is on the left? NO (In the sample question, students may have to read through the long definitions in the second column several times to complete the question.) Question 7 . Each column composed of similar materials and appropriately titled? YES Similar materials? YES Titled? NO Question 8 . Alternatives arranged in logical order? NO (As in multiple-choice items, alphabetical order or length is useful when no more logical order is obvious.)

Instructions: Match the legal instruments in Column B to their definitions in Column A in the space provided. Each instrument may be used once, more than once, or not at all. Column A—Definitions Column B—Legal Instruments _____ 1. Establishes rights, privileges, rules, and a. Code of Conduct punishment for military personnel. b. Command Regulations _____ 2. Implements controls over duties, customs, and privileges of military personnel ina foreign country. c. Status of Forces Agreement _____ 3. Provides for controls over duties, customs, d. Uniform Code of Military Justice and privileges of military personnel in a foreign country.254 AFMAN36-2236 12 NOVEMBER 2003

22.6. True-False Items. True-false items are useful in testing know ledge of facts, especially when there is little question whether a statement about a fact is either right or wrong. True-false items may also be used to determine the persistence of popular misconc eptions when the suggestion of a correct response in a multiple-choice item would be too obvious. The chief disa dvantage of the true-false item is that the possibility of correct guessing, particularly in simple items, is greater than in any other type of selection test item.
22.6.1. Constructing True-False Items. True-false items have probably been used and misused to a greater extent than any other select ion test question type. Too often, in structors merely scan text material, select statements more or less at random, and then insert negatives in approximately one-half of the statements to make them false. They mistakenl y believe they can justify answers to these items by referring students to pages in the text. When tests ar e constructed in this manner, the principal attribute measured is photographic memory. Tests of this sort have brought objections to selection tests in general and true-false items in particular. The instructor should observe the following principles in constructing true-false test items: ( NOTE: Include only one idea in each true-false item.) 22.6.1.1. Do not make part of a statement true and another part false. 22.6.1.2. Avoid the use of negatives. They confuse the reader. 22.6.1.3. Avoid involved statements. Keep wording and sentence structure as simple as possible. Make statements clear and definite. 22.6.1.4. Whenever possible, use terms that meanthe same thing to all students. Short, simple statements will increase the likelihood that the point of the item is clear and that passing or failing it will be determined by the students' learning—not their reading ability. 22.6.1.5. As a rule, avoid absolutes, such as all, every, only, no, and never. Such unequivocally true or false statements are rare; statements containing absolutes are usually false. Similarly, avoid statements containing some, any, and generally, which are most frequently true. These terms are often called hedgers. 22.6.1.6. Avoid patterns in the sequence of re sponses because students can often identify sequence patterns. Instructors sometimes deliberately use patterns to make hand scoring easier. This is poor practice.22.6.1.7. Make statements brief and uniform. Some instructors unconsciously make true statements longer than false statements. Students are quick to take advantage of this tendency.
22.6.2. Examples of True-False Items. True-false items may be written in the forms shown in the examples in Figure 22.25. through Figure 22.28. . Other items of this general type have also been devised to reduce guessing to a minimum. These incl ude adding to the basic true-false choice the choices of "sometimes true—sometimes false," and s imilar variations. In additi on, this type of item can be made more difficult and di scriminating by requiring the student to suggest what will make false statements true.
Figure 22.25. Declarative Statement.

T F Winds circulate clockwise around a high pressure area in the Northern Hemisphere.AFMAN36-2236 12 NOVEMBER 2003 255

Figure 22.26. Cluster.

Figure 22.27. Statement with Illustration.

Figure 22.28. Double True-False.

22.6.3. Suggestions for Writing True-False Items. The checklist at Figure 22.29. provides another checklist that summarizes the principles for constructing mechanically sound true-false items. The M-16 Rifle T F stock is constructed of high grade walnut. T F is chambered for a .223 caliber cartridge. T F may be set to fire either automatic or semi-automatic. T F disassembly kit is stowed under the butt plate.

Directions: Indicate your answer in the space provi ded to the left of each statement, as if the statement is a. true and the reason is correct for the statement. b. false and the reason is incorrect. c. true and the reason is incorrect. d. false and the reason is correct. _____ Statement: Aluminum is used in transportation. Reason: Aluminum is light and strong. _____ Statement: Most items made of metal corrode. Reason: Many metals corrode.256 AFMAN36-2236 12 NOVEMBER 2003

Figure 22.29. Checklist for Writing True-False Items.

22.6.3.1. Sample True-False Item. The true-false item in Figure 22.30. violates one or more of the principles of sound construction listed in the Figure 22.29.
Figure 22.30. Sample True-False Item.

22.6.3.2. Checklist Analysis. Using the checklist provided at Figure 22.29. , let's examine the true-false item at Figure 22.30. for quality of construction.
22.6.3.2.1. (Question 1) Is the item short and simply stated? NO. Although the informed student may well understand the words used in the item, the sentence structure is awkward and fragmented, making it difficult for the student to grasp its meaning.
22.6.3.2.2. (Question 2) Does the item test only one idea? NO. The item addresses two distinct points, one regarding the violation of a certain characteristic of evaluation and the other regarding a possible explanation fo r such a violation. Not only is this confusing to students, it creates problems in interpreting what student s understand or do not understand about the question's content.
22.6.3.2.3. (Question 3) Is the item free of absolutes and hedgers? NO. In this question, the hedger "In general" and the absolute "never " are not only misleading and imprecise, but are also contradictory. Avoid these types of terms whenever possible.
22.6.3.2.4. (Question 4) Is the item stated positively? YES.
22.6.3.2.5. (Question 5) Is the item totally true or false? NO. This is a frequent occurrence when attempting to test more than one idea with one item, as discussed in paragraph 22.6.3.2.2.
22.6.4. Checklist Summary. We've answered NO to items 1, 2, 3, and 5 on our checklist. See Figure 22.31. for appropriate changes to our corrected item. No. Item Format Yes No

1 Short and simply stated? 2 Tests only one idea? 3 Free of absolutes and/or hedgers? 4 Stated positively?
5 Totally true OR false?

T F In general, the characteristic of evaluation that is never viol ated by experienced writers of selection test items is validity, and this is a function of training and experience.AFMAN36-2236 12 NOVEMBER 2003 257

Figure 22.31. Corrected Item.

22.7. Supply Test Items. A supply test item requires students to furnish their own answers. They are not given alternative responses from which to choose. The basic forms of supply questions are completion, short answer, and essay. Supply test items are mechanical ly easier to construct than selection items but far more difficult to evaluate. 22.7.1. When the ability to express ideas or original thinking are to be measured, supply items have adistinct advantage over selection items. The ability to solve problems and to think creatively are seldom worthwhile end products in themselves; it is usually necessary to communicate the solutions or ideas to others, frequently in writing. When deve loping the ability to write clearly is a legitimate course objective, supply items may be used effectively. Test items that call for a written discussion, such as the essay form, also give students an opportunity to express themselves—something students often like to do. 22.7.2. Constructing a supply item for which several equally competent instructors can give comparable scores is difficult. This difficulty in attain ing objectivity often leads to reduced test reliability. Clerical assistants usually can not score supply test items because they must understand the subject matter area covered by the test. As a rule, this mean s that the instructor must score supply items. The supply item also requires considerably more time to score than a selection item. 22.7.3. Another disadvantage of the supply item stems from the ability of students to think and read more rapidly than they write. Consequently, in a gi ven time block, this type of test may be less comprehensive than a selection item test.
22.8. Completion Items. Completion items require the student to provide one or more words omitted from a statement. The student must supply at least pa rt of the idea expressed in the statement. When the correct word or words are supplied in the proper blank, the statement is complete and true. Completion item testing virtually eliminates the possibility of guessing and is a time-saving device in comparison with the essay test.
22.8.1. Scoring Key. The completion item can be used in testing student ability to make verbal associations of the who, what, when, where, and why types. However, it is difficult to construct items that require but one correct response; therefore, the sc oring of completion items is normally less objective than scoring of selection items. Even after determin ing that a given response is correct, instructors frequently must decide to give part or full credit for responses they did not anticipate. For this reason, they should make a complete scoring key (for any supply item) when they prepare the item.
22.8.2. Constructing Completion Items. Instructors should observe the following principles to construct completion questions: 22.8.2.1. Construct a completion item so it cont ains only one blank in each sentence. Multiple blanks in a sentence can quickly lead to ambiguityand, therefore, to reduc ed validity and reliability. T F Improved validity of test items is the most frequent re sult of training the writers of test questions.258 AFMAN36-2236 12 NOVEMBER 2003

22.8.2.2. Write the statement so that the blank appears at or near the end of the item. This arrangement provides the student with necessary informa tion to fill in the blank without having to read through the item twice. 22.8.2.3. Be sure there is only one correct or best re sponse for each blank. In the interest of scoring objectivity and overall test reliability, both th e students and the instructor should have a clear understanding of the criteria for acceptable responses. 22.8.2.4. When testing comprehension and higher le vels of learning, word completion statements differ from the way they were worded in texts or lectures. If not expressed differently, the test will measure a level of learning no higher than the know ledge level regardless of the form of the question or the subject matter. 22.8.2.5. Make all blanks uniform in length. Blanks adjusted to the length of the anticipated response may provide a clue to the correct answer. Directions for completion items should state that the length of the blanks does not indicate the length of the nature of the response. They should also indicate whether a single bl ank always requires one word or whether short phrases may sometimes be supplied. 22.8.2.6. For convenience and accuracy in scoring, incl ude a separate series of blanks arranged in a vertical column on the test page. Instruct st udents to provide their responses in these blanks rather than in the blanks within the items.
22.8.3. Examples of Completion Items. Completion test items may be constructed either as incomplete statements or as sentences or paragraphs with key words omitted. Some acceptable examples of completion test items are incomplete statem ents and sentence or paragraph completion.
22.8.3.1. Incomplete Statement. This type ( Figure 22.32. ) is more difficult for the student than the direct question form of short answer item. The instructor must constantly be on guard to avoid ambiguous, incomplete statements such as "Col umbus was born in...," which could be answered with a city, country, or year.
Figure 22.32. Example 1 – Incomplete Statement.

22.8.3.2. Sentence or Paragraph Completion. Omit only key words or phrases. Avoid copying from textbooks ( Figure 22.33. ). The name of the man who first successfully flew the airplane is __________. The Air Force became a separate branch of the military in 19___.AFMAN36-2236 12 NOVEMBER 2003 259

Figure 22.33. Example 2 – Senten ce or Paragraph Completion.

22.9. Checklist for Writing Completion Items. The brief checklist at Figure 22.34. can be of assistance in implementing the principles for writing sound completion items. The question at Figure 22.35. violates one or more of these checklist items.
Figure 22.34. Checklist for Writing Completion Items.

Figure 22.35. Sample Item.

22.9.1. Checklist Analysis:
22.9.1.1. (Question 1) Is there a total of one or two bl anks at or near the end of the sentence? NO. As blanks increase in a completion item, so does the ambiguity of that item. Also, blanks at the beginning of the sentence cause students to re read frequently in attempting to grasp the sense of the statement. 1. _____ The (1) __________ of a sound varies (2) __________ as the square of the distance 2. _____ from the source.

OR 1. _____ In 1795, Napoleon was given command of a small army in (1) _____. The wonderful 2. _____ genius of the young leader made the (2) _____ campaign the decisive factor. By rapid 3. _____ movements he separated (3) _____ and (4) _____ forces. In eleven days he forced 4. _____ (5) _____ to conclude peace. 5. _____

No. Item Format Yes No

1 Blank(s) at or near end of sentence? 2 Blanks approximately same size? 3 Blank(s) related to key terms? 4 Worded so that response possibilities are limited?

_______, ________, ________, ________, and are usually presumed to be the most common functions of management taught by educators in __________.260 AFMAN36-2236 12 NOVEMBER 2003

22.9.1.2. (Question 2) Are blanks approximately the same size? NO. The variable lengths of blanks may provide clues to the keyed respons es and may confuse students unnecessarily. Keep blanks the same length and provide clea r instructions regarding their use.
22.9.1.3. (Question 3) Are blanks related to key terms? Probably NO, although it would depend upon the actual keyed responses. It appears thatthe blank at the end of this item is requesting something less than key information.
22.9.1.4. (Question 4) Is the question worded so that response possibilities are limited? NO. Besides the fact that the information requested in the final blank may be interpreted by students in a number of ways, the lack of a source reference in the sentence would seem to permit student opinions as acceptable responses.
22.9.2. Corrected Completion Test Item. Figure 22.36. shows a corrected completion test item based on the checklist.
Figure 22.36. Corrected Item.

22.10. Short-Answer Items. In general, the short-answer item as a type of supply test item includes features of both the completion and the e ssay item. It may be used in measuring ability to recall facts, basic concepts, and principles. The advantages and disadvant ages of the short-answer item are essentially the same as those of completion questions. However, the short-answer item often tests ideas more fully because the recalled idea must be stated in a more complete form.
22.10.1. Constructing Short-Answer Items. Instructors should observe the following principles in constructing short-an swer test items: 22.10.1.1. Be specific. The student should know exactly what is expected. If the instructions are not specific, test results are lik ely to be unrelia ble and invalid. 22.10.1.2. Be sure each required answer involves a simple idea, concept, or fact. Otherwise, a meaningful evaluation of student responses becomes quite difficult. 22.10.1.3. Be sure students know how complete to make their responses. Usually, the wording of the item can provide this inform ation. Sometimes, the amount of space will indicate the length of the response. However, the amount of space provided for a response should not be a clue to the correct answer; it should serve only as a guide to the desired length of the response.
22.10.2. Examples of Short-Answer Items. Short-answer test items may be constructed in the following forms: direct question, association, short explanation, list, and pictures or diagrams.
22.10.2.1. Direct Question. This form is usually easier for the average student than a completion item because it helps eliminate ambiguous elements ( Figure 22.37. ). According to the article written by George S. Odiorne, the two most common functions of management are _____ and _____.AFMAN36-2236 12 NOVEMBER 2003 261

Figure 22.37. Direct Question Example.

22.10.2.2. Association. This form is acceptable when only a few correlations are being made ( Figure 22.38. ). Matching items are more functional if se veral different associations are required.
Figure 22.38. Association.

22.10.2.3. Short Explanation. This form can be used to obtain maximum comprehensiveness with a minimum of guessing ( Figure 22.39. ).
Figure 22.39. Short Explanation Example.

22.10.2.4. List. This form is especially useful for informal classroom teaching ( Figure 22.40. ). What are the two main classifications of rocket fuel?__________ and __________. OR Who was the first Secretary of Defense? __________

After each level of learning, write the name of th e author of the taxonomy from which that level was drawn. (Bloom, Krathwohl, or Simpson) Responding _______________ Guided response _______________ Synthesis _______________ Comprehension _______________ Adaptation _______________ Characterization _______________

In one sentence, explain a significant contribu tion each of the following men made to the development of military aviation. General Benjamin Foulois _______________ General Billy Mitchell _______________ Captain Eddie Rickenbacker _______________ General James Doolittle _______________262 AFMAN36-2236 12 NOVEMBER 2003

Figure 22.40. List Example.

22.10.2.5. Picture or Diagram. Pictures or diagrams often add clarity, speed the testing process, and make testing more interesting and valid ( Figure 22.41. ).
Figure 22.41. Picture or Diagram Example.

NOTE: Since short answer items are so similar in cons truction to essay items, suggestions for writing short answer items are included in the next section.
22.11. Essay Items. Any sharp differentiation between short answ er and essay items would be a compromise at best. Fortunately, such a distinction is not necessary. The difference between the two forms is perhaps more noticeable in the length of responses than in the wording of the items themselves. The essay is probably the least efficient form of item for measur ing simple recall (knowle dge level of learning). It should be used only when students are required to think reflectively or creatively, to organize knowledge in the solution of a problem, and to express their solution in writing. List at least four significant reasons wh y flying safety practices are important.
a. ________________________________________ b. ________________________________________
c. ________________________________________ d. ________________________________________
e. ________________________________________AFMAN36-2236 12 NOVEMBER 2003 263

22.11.1. Major Disadvantage of Essay Items. The major disadvantage of e ssay items is the scoring. For this reason, instructors should make a complete scoring key for each essay item when they first prepare it. The construction and use of the scoring key will be explained later in this chapter.
22.11.2. Constructing Essay Items. One of the difficulties in constructing essay items is that the process appears too easy. Like se lection items, however, essay ite ms with good scoring keys are quite difficult to construct. Before preparing an e ssay question, instructors must understand the objectives to be measured. They must also give clear instructions concerning the kind and length of required responses. Items of equal value on a test should be balanced in difficulty. Instructors cannot evaluate the students' achievement of the same objective unl ess all students provide answers to the same test items. Therefore, instructors should not offer students a choice of questions. 22.11.2.1. Generally, use essay items for purposes for which they are best suited; that is, to measure achievement at the comprehension or higher level of learning. 22.11.2.2. State the items clearly so students will know exactly what type of discussion is required. Do not use vague essay questions in an ef fort to test ingenuity. If the items are skillfully designed, students will have ample opportunities to apply their ingenuity and creativity. Essay test items should mean essentially the same thing to all students who have achieved the desired level of learning. Revise or eliminate all items with a double meaning. 22.11.2.3. Whenever possible, increase the number oftest items and reduce the amount of discussion required for each. This will not only make scoring easier, but will also probably increase the comprehensiveness of the test. 22.11.2.4. Suggest a time limit for each item. Indicate the desired length of response, or otherwise limit responses. 22.11.2.5. As part of the instructions to students, explain how each item affects the final score ofthe overall test and the possibilities of partial credit for each item. Every student should understand the criteria used in the scoring process. 22.11.2.6. Avoid making the answer to a first item the basis for a second item. Items linked in this manner may cause students to miss an entire seri es of test items although they may be wrong on just the first. It is nearly impossible to interpret results in such situations.
22.11.3. Examples of Essay Items. Essay items may be designed and classified in various ways. One classification can be made according to the type of response required of the student. Seven major types of essay items based on this classifi cation are shown in the following examples:
22.11.3.1. Comparison. In 100 words or less, compare norm- and criterion-referenced measurement. This question is worth 20 points and no partial credit will be awarded. Up to five points may be deducted for grammar and organization.
22.11.3.2. Decision. Defend the assertion that the death penalty should be abolished. You will have 20 minutes to answer this question. The ques tion is worth 25 points and partial credit may be awarded. No points will be deducted for spelling and grammar.
22.11.3.3. Causes or Effects. In the space provided below, discuss how institutional racism inhibits social progress in the United States. Support you answer with specific examples. The value of this question is 15 points with partial cr edit available. Spelling, grammar, and organization will not be graded.264 AFMAN36-2236 12 NOVEMBER 2003

22.11.3.4. Explanation. In 250 words or less, explain the principles involved in airfoil lift. This question is worth 30 points; partial credit is ava ilable. Up to three points will be deducted for spelling, grammar, and organization.
22.11.3.5. Summary. Summarize in 300 words or less the f unctions of management. Your answer should be supported with specific, personal exampl es. This question is worth 50 points with partial credit available. Up to 10 points may be deducted for organization, clarity, and grammar.
22.11.3.6. Illustration. In the space provided below, illustrate the peculiarities of high-speed aerodynamics. Limit your answer to verbal descriptions and use examples when appropriate. This question is worth 25 points with partial credit available. No points will be deducted for spelling, grammar, or organization.
22.11.3.7. Guided Essay. Another classification of essay items can be made according to format. The essay item at Figure 22.42. represents an attempt to present a realistic situation and to guide the student's response.AFMAN36-2236 12 NOVEMBER 2003 265

Figure 22.42. Guided Essay Item.

22.12. Checklist for Writing Essay or Short Answer Questions. The checklist at Figure 22.43. summarizes several of the principles for constructing s ound essay and short-answer items. Guidelines for the scoring of these types of supply items are provided la ter in the chapter (paragraph ___). The principles for the construction of each are virtually the same. Use th e checklist to analyze the mechanical quality of the item in Figure 22.44. Read the following situation, then answer the two questions provided. Question 1 is worth 10 points; question 2 is worth 15 points. Partial credit is ava ilable. For full credit, your answer should contain specific, relevant support for all conclusions made. No points will be deducted for spelling, grammar, or organization. SITUATION: Captain Henry Jones is a management analyst at Randolph Air Force Base. You are the comptroller. Five months ago the maintenance squa dron commander requested a study of his maintenance control system. You gave Captain Jones the assignm ent. He is young and aggressive, and although he had no maintenance experience, you felt he could handle the job. A recent check on his progress showed the following developments: 1. He had not completed any studies but showed vary ing degrees of progress on seven different studies, with "three more in the planning stages." Of the se ven, two were held up because of a deadlock between Captain Jones and the officers in charge of the areas in which he was making the studies. The two officers in charge stated that he did not understand main tenance problems and that his recommendations were faulty. The other five studies were apparently stagnant. 2. Captain Jones had established requirements for nine temporary reports fro m maintenance squadron activities to provide him with data in various mainte nance areas. The requests for these reports had been initiated at different times.

_____________________________________ Directions: Respond to each of the following questions in no more than two handwritten pages. You will have approximately 45 minutes to complete the exam. Your answers will be evaluated on their content alone, and from 0 to 20 points may be awarded for each answer. QUESTIONS: 1. In what respect did Captain Jones err most seriously? 2. In the present situation, how can the two studies, wh ich have been held up because of non-concurrence, be best completed?266 AFMAN36-2236 12 NOVEMBER 2003

Figure 22.43. Checklist for Writing Short-Answer and Essay Items.

Figure 22.44. Short Answer or Essay Item.

22.12.1. Checklist Analysis:
22.12.1.1. (Question 1) Is the item stated clearly? NO, not really. "Discuss" by itself does not adequately describe the nature of the desired response. The words "discipline" and "environment" are particularly ambiguous and require more specific direction for the student.
22.12.1.2. (Question 2) Is the point value of the item clearly indicated? NO. Students need to be appraised of the relative weight of the item in the overall test, just as the scoring procedures of other item types are revealed.
22.12.1.3. (Question 3) Is the length of th e expected response indicated? NO. Instructions to students should include a realistic guide in terms oftime, words, or space for adequately answering the item.
22.12.1.4. (Question 4) Does the item address partial credit? NO. Particularly in an essay item, partial credit may be available. Specifically on what basis that credit will be awarded is of interest to the student.
22.12.1.5. (Question 5) Are factors other than content indicated that will affect the score? NO. Sometimes the total score for the item is notbased on the content of the response alone; grammar, spelling, neatness, and organization may be factors consider ed important enough to score in an essay item. If such factors will be j udged, let the student know what and how much.
22.12.2. Checklist Summary. We have answered NO to all five questions on our checklist. This question can be fixed as shown in Figure 22.45. No. Item Format Yes No

1 Item stated clearly? 2 Point value of the item clearly indicated? 3 How much response is expected (in time, words, or space)? 4 Whether or not partial credit is available?
5 Factors other than content affect the score (e.g., grammar, organization, neatness, spelling)?

ITEM: Discuss the concept of discipline as it relates to the Air Force environment.AFMAN36-2236 12 NOVEMBER 2003 267

Figure 22.45. Corrected Essay Test Item.

22.13. Assembling the Test. Once the desired test items have been prepared, pay careful attention to putting the overall test together. All information required to identify the test should be printed on the first page or the cover page or the st udents may fill in this information. The name and location of the school, name of the course, section number, date, and student c ode number are often included. 22.13.1. A complete set of instructions, which may be provided in written form, orally, or by visual aid, should specify what text and other references or tools may be used during the test and the time allowed for parts or for the total test. The instruc tions should include a statement of the number of items in the test and indicate clearly how each st udent should respond. A good plan is to give a sample item with the correct response. The instructions should also tell whether to proceed individually from part to part or page to page or whether to wait for a signal or further instructions. Students should alsobe told the procedure to follow when they have completed the test. If reliable testing data is to be compiled for administrative purposes, testing instructions should be kept uniform from class to class. 22.13.2. Test items should be placed on the page so that each item stands out clearly from the others. For example, true or false items two lines long sh ould be single-spaced, but there should be two spaces between items. A space should separate the stem of multiple-choice items from the list of alternatives. The alternatives should appear in a singl e column beneath the stem and indented beyond the paragraph margin. 22.13.3. For machine-scored tests, take precautions to ensure the items can be used with machine-scored answer sheets. Special answer shee ts can be designed for specific test objectives, while stock answer sheets generally can be used fo r true-false items, multiple-choice items with 5 or less alternatives, and for matching items with 15 or fewer alternatives. 22.13.4. As nearly as can be anticipated, arrange items in ascending order of di fficulty within groups. Items of known difficulty used before are readily arranged in order of difficulty. Students are put at ease when they find relatively easy items in the early portion of the test. 22.13.5. A test should be designed to ensure that most students can complete it. When many students cannot complete a test, much of its efficiency is sacrificed and student morale is likely to suffer. On the other hand, it may not be desirable for all students to complete the test before the end of the allotted time. In that event, a large portion of the gro up may have to wait for the remainder to finish. Allowing students to leave as they complete a test (as in some room arrangements) may distract and confuse those who are still working.
22.14. Administering the Test. Although the decision to administer a test does not always rest with instructors, they are often allowed to decide when to test. Instructors should arrange for tests to be administered in the morning when students are fresh and alert, rather than in the afternoon when they are tired and less likely to be at their best. In no more than 500 words, explain the relationshi p between the concept of self-discipline and the accomplishment of the Air Force mission. Provide support for your statements in the form of specific examples—both hypothetical and real. This question is wo rth a total of 25 points, 5 points of which will be for organization and grammar. Partial credit may be awarded.268 AFMAN36-2236 12 NOVEMBER 2003

22.14.1. Selecting a place to administer the test is equally important for reliable testing results. Students are more likely to perform at their best in a quiet, well-lighted, well-ventilated room with a comfortable temperature. Ample working space is particularly important when separate answer sheets are used and when the students may have to make computations on scratch paper. If students are uncomfortable or distracted by physical surroundings, expect their scores to suffer. Also, since some students are affected more than others, scores will be less reliable.
22.14.2. Arrive at the examination room well in advanc e of the class to assemb le test materials. Some instructors prefer to have test papers and other materials in plac e before the students arrive. Extra tests, answer sheets, and pencils should be available. In administering the test, provide clear and concise instructions to avoid confusion. Students are less likely to become nervous or tense, when they understand exactly what they are supposed to do. Therefore, their test scores will represent a more accurate picture of their achievement. 22.14.3. Whether you win the confidence of your students in the testing situation depends largely upon your personality and the effectiveness of your te aching. Make every effort to encourage the students and put them at ease. 22.14.4. Although carefully written directions for taking the test should be a part of the test, you may wish to give oral directions as well. Before begi nning the test, invite the students to ask questions concerning procedures and make it clear whether questi ons may be asked of you after the test begins. Any errors, misprints, omissions, or other defects inthe test should be found and explained beforehand. 22.14.5. If the test has time limits, these should be announced and observed. Tell the students whether they can turn in their papers and leave the room during the period or whether they must remain seated until all materials are collected.
22.15. Scoring Supply Items. You should explain the procedures for scoring supply items at the time students take the test. The explanation should include a discussion of the relative weight of each item or problem and its separate parts. With this information, the students can determine how to allocate their time and effort. 22.15.1. Scoring supply items often presents serious difficulties. For reliable test results, the rating assigned to a given paper must be comparable whenit is scored by several competent instructors. If one instructor assigns a "Poor" scor e to a test paper and another in structor assigns a "Good" score to the same paper, the grades may express inst ructors' bias and not student proficiency. 22.15.2. To achieve scores that are as objective and reliable as possible, you should arrange for more than one person to score each paper. Supply items are highly subjective and difficult to evaluate; therefore, they require a great deal of time to gr ade. You may be tempted to assign the best grades to students who use their phraseology or who have otherwise done well in the course.
22.16. Scoring Essay Test Items. The principles for achieving objectivity apply to completion, short-answer, and essay test items. Si nce the essay test is the most difficult test to score objectively, this paragraph includes a detailed disc ussion of the principles involved in scoring essay tests. 22.16.1. After you are satisfied the essay item is carefu lly and accurately prepared, the next task is to make certain as much objectivity as possible is used in grading the responses. The effectiveness of an essay examination depends to a large degree on how well it is graded.AFMAN36-2236 12 NOVEMBER 2003 269

22.16.2. In grading essay responses, you must (1) use appropriate methods to minimize biases, (2) pay attention only to the significantand relevant aspects of the answ er, (3) be careful not to let personal idiosyncrasies affect grading, and (4) apply uniform standards to all the papers. Undoubtedly, the uniformity of grading standards (hence the reliability of the scores) is probably the most crucial aspect of essay grading for without uniformity, ther e is no valid way to measure achievement of objectives. 22.16.3. Decide in advance what qualitie s to consider in judging the adequacy of the answer. If more than one distinct quality is to be appraised, make separate evaluations of each. In an essay test, instructors are generally trying to appraise the achievement of objectives directly and explicitly related to the content areas. These may include areas such as knowle dge of facts, principles, theories of the content area, and the application and development of thes e to novel situations and problems. They may also wish to appraise the development of generalized skills of writing, such as logical organization, English usage, and spelling. These two dimensions are quite different and should be rated and reported to the student separately. This separation is necessary to enable the instructor to make a judgment of the extent to which the student is achieving the distinctive objectives of the subject matter and to reveal to the student whether any deficiency in answering the question is due to weakness in subject matter or weakness in written expression or both. 22.16.4. Prepare an answer guide or model answer in advance showing what points should be covered. This measure helps provide a common frame of reference for evaluating each paper and is especially important if the scoring of the papers is spread across a number of days. After the preliminary guide has been prepared, check it against a sample of responses. This step helps determine the adequacy of the scoring guide; therefore, do not rate this sample of papers. If the check reveals that students have consistently interpreted the question at a consistently lower level than the scoring guide, then revise the specifications of what represents an acceptable answer to correspond more closely to the performance. Instructors frequently object to this procedure as representing a lowering of educational standards, but it is not. Usually, difference s between the original scoring guide and student answers are due to lack of clarity in the original que stion or the instructor's unrealistic expectations of what students can do within the limits of the testing time.
22.16.5. Read all answers to one item for all student s before going on to the next. This allows you to maintain a more uniform set of standards of grading across papers and you will be less likely to be influenced in your judgment of the quality of the answer to another item by how the student answered the previous one. 22.16.6. Grade the papers as anonymously as possible. The less you know about who wrote an answer, the more objectively you can judge what was written. 22.16.7. Write comments and correct errors on answers to essay items. A test is most effective for motivation and learning when students get prompt, sp ecific information on their strengths and weaknesses. Also, if you make a tally ofthe types of comments and errors, you will gain valuable information to judge the effectiveness of your teaching. 22.16.8. Try to score all responses to a particular item without interruption. Scores may vary markedly from one day to the next and even from morn ing to afternoon of the same day. We would be very unfair to our students if we allowed a personal argument with a spouse, a migraine headache, or an upset stomach to influence the grades we give students for their answers. Our scoring is valid only to270 AFMAN36-2236 12 NOVEMBER 2003

the degree that it is based on the quality of the s ubject's answer and not on the reader's disposition at a particular moment.
NOTE: Although we recommend all responses to a particular item be read at the same time, we also suggest there be short breaks during the reading peri od. Otherwise, you can become fatigued very easily. 22.16.9. If possible, have two independent readings of the test. A double reading by two independent readers will make the scores more reliable. If i ndependent readings are done, the scores should not be recorded on the test booklet—instead, they shoul d be written on a separate sheet. Seeing the scores from the first reading could markedly influence the scores on the second reading, and thus defeat the aim on maintaining independent scoring. 22.16.10. The sample essay test question at Figure 22.46. provides us with a sample essay item and instructions. Instructions for answering an essa y item should include the length of the essay, time allowed, worth of the item, if partial credit will be given, and if spelling, organization, and grammar will be graded.
Figure 22.46. Sample Essay Test Question.

22.16.11. The sample scoring key for essay test items ( Figure 22.47. ) lists the most important elements of an answer. The key is helpful to remind you of the ideas to look for in an answer. However, your knowledge is still needed to judge the quality of the answers.
Figure 22.47. Sample Scoring Key for Essay Test Items. ITEM: Explain how implementing a Management by Ob jectives (MBO) system might improve worker performance and organizat ional effectiveness. INSTRUCTIONS: Answer this question in 150-200 words using specific hypothetical or actual examples where appropriate. This question is worth 20 points; up to 5 points ma y be deducted for organization, grammar, and spelling. Partial credit may be awarded for th e content section of the question.

For full credit the student should address, with sp ecific examples, the following significant teaching points: a. MBO is a good motivational tool for the supervisor, although it does not guarantee success. (5 points) b. MBO can have a positive impact upon the meeti ng of individual and organization needs. (5 points) c. Increased involvement and participation of workers with the job goals and objectives lead to an increase in satisfaction, creativity, and productivity. (5 points) d. Grammar, spelling, punctuation, etc. (5 points)AFMAN36-2236 12 NOVEMBER 2003 271

22.17. Summary. A primary purpose for testing is to improve instruction and, thereby, to increase learning. The grades derived from testing are, at best, a secondary testing objective. 22.17.1. Two broad categories of test items are selection and supply. The selection category includes true-false, multiple-choice, and matching items, all of which provide alternativ es from which the student must choose. The supply category includes completion, short-answer, and essay items, all of which require the student to supply whatever information is requested. Advantages of selection test items are a high degree of objectivity in scoring, the capability to measure the acquisition of large numbers of facts, ideas, or principles in a relati vely short time, and convenience in the development of statistical analyses. 22.17.2. Supply items have a distinct advantage over selection items when you are measuring the ability to express ideas or original thinking. When properly constructed, ei ther selection or supply items can measure learning at any level. The probl em-situation variety of either supply or selection test items can measure learning at the higher levels. It can aid in differentiating among students of high achievement and in estimating the degree of learning achieved by students. 22.17.3. The actual writing of test items is one of our most difficult tasks. We must understand the basic concepts involved. When assembling items for a test, the instructor should include a complete set of instructions. Students should know exactly what is expected of them, the relative value of test items, and time limits. Normally, test items should be arranged in order of difficulty, starting with the easiest, and grouped according to subject matter and item type. 22.17.4. Instructors should understand the effects of phys ical facilities, temperature, humidity, lighting, time allowed, and time of day on the reliability of test results. Similarly, they should understand the function of oral instructions preceding a test . For reliable test results, the circumstances surrounding each testing session must be consistent. 22.17.5. A test may be valid, reliable, and comprehensiv e, but if it is not scored properly, individual scores are useless. To increase objectivity, model answers and a scoring key should be used to score a test with supply-type items. One of three basic procedures can be used to score a test with selection items. Whatever the method of scoring, students should be told specifically about the method when the test is administered. Obviously, if a good test is to be used effectively, scoring is a vitally important procedure.272 AFMAN36-2236 12 NOVEMBER 2003

Chapter 23

MEASURING LEARNING OUTCOMES

23.1. Introduction. An orderly process for writing sound objectives and worthwhile test items was introduced in Chapter 3 . This approach to planning for learning outcomes produces behavioral objectives and criterion-referenced tests as required by AFMAN 36-2234. The process for writing student-centered objectives and tests works well for most Air Force in struction with the possible exception of wholly physical skills. This process will produce sound objectives fo r cognitive and affective learning as well as job tasks that include, but are not limited to, physical skills. Using criterion objectives (as explained in Chapter 5 ) might best develop wholly physical skill objectives , aimed at developing de xterity or manipulative tasks (touch typing, shooting a firearm, or athletic skills).

23.2. Overview and Review. Although the process for writing sound instructional objectives and tests has been explained throughout this manual, this chapte r will show additional, more complete applications using examples from some of the Professional Military Education institutions in our Air Force. We will illustrate the process applied to a knowledge-level lesson objective ( Figure 23.1. ) and two comprehension-level objectives ( Figure 23.2. and Figure 23.3. ) in Bloom's Cognitive Taxonomy.AFMAN36-2236 12 NOVEMBER 2003 273

Figure 23.1. Sample Knowledge-Level Test Question.

STEP 1 (note 1) LEVEL-OF-LEARNING LESSON OBJECTIVE. Know the fundamentals of the US Air Force Instructional Systems Development (ISD) model.

STEP 2 SAMPLES OF BEHAVIOR (note 2) 1. Define ISD. 2. List the phases of the ISD process. 3. State the purpose of ISD. 4. Label each phase of the ISD m odel when given a blank graphic.

STEP 3 ISD is the Air Force procedure for replaci ng live classroom instructors with computer- (note 3) based instruction.

T ( ) F( )
(Key: False)

NOTES: 1. This knowledge-level lesson is typical of many factual lessons taught for recall or recognition. The lesson might be an introduction to a larger block of instruction that would look at ISD in much more depth, or it could be a one-time lecture to make students aware of the model. In this objective, "fundamentals" is a "catch all" term used to save space and keep the objective relatively short. The writ er of this objective could have spelled out which fundamentals (process for assessing needs, format for writing objectives, etc.), but the objective could have become so long as to be unreadable. An outline of the lesson plan would show which fundamentals were to be included in the lesson.

2. The samples of behavior list ed here describe the behavioral evidence of learning. These behaviors, and others equiva lent to them, can be used to show mastery of the level-of-learning lesson objective. Students may be asked to recall or recognize any of this information essentially as it was taught to them. For the purpose of illustration, a test question is written from Sample of Behavior #1. Each of the other samples will work equally well as long as the anticipated learning outcomes are limited to recall or recognition activities.

3. Although a true-false test item was written from this sample of behavi or, several other test item types could be used to measure this same sample. Figure 23.3. illustrates how a sample of behavior may be used to generate several test item types. By answering this item correctly, students will provide some evidence of reaching the lesson objective. If more evidence of mastery is needed, students may be tested on additional items that measure other behavioral samples of learni ng at the same level of learning.274 AFMAN36-2236 12 NOVEMBER 2003

Figure 23.2. Sample Comprehension-Level Test Question.

STEP 1 LEVEL-OF-LEARNING LESSON OBJECTIVE. Comprehend the behaviorist (note 1) learning theory concepts of chaining, shaping, and reinforcement.

STEP 2 SAMPLES OF BEHAVIOR (note 2) 1. Define each concept in terms other than those used in class. 2. Identify new examples of each concept. 3. Compare the critical attributes of each concept to those of the other concepts. 4. Explain why an apparent example of any of these concepts is or is not a correct example.

STEP 3 An example of reinforcement is (note 3) ( ) a. Offering a child 50 cents to clean a room. ( ) b. Patting a dog on the head after it performs well. ( ) c. Placing a child in timeout after a temper tantrum.
(Key: b)

NOTES: 1. This objective identifies several concepts that are to be taught for understanding. The comprehension level of learning, called for in this objective requires more than simple recall or recognition. This objective is more difficult to achieve than knowledge-level objectives because students who comprehend concepts must be able to exhibit more demanding evidence of learning. The lesson objective names three separate concepts rather than just one or two. This is neither good nor bad. It indicates, however, that the instructor feels sufficient time is devoted to the lesson to develop comprehension of all three concepts.

2. Students may be asked to show their understanding of these concepts on a written test which cannot be answered by recall or re cognition. Students must go beyond the facts taught to them and generalize to new a nd unfamiliar material. For the purpose of illustration, a test question is written from sample of behavior #2. Each of the other samples will work equally well as long as the anticipated learning outcomes require students to be able to generalize on the three concepts taught.

3. Several other test item types could be used to me asure this same sample. Figure 23.3. illustrates how the same sample of behavior may be used to generate several test item types. By answering this item correctly, students will provide some evidence of reaching the lesson objective. If more evidence of ma stery is needed, students may be tested on additional items that measure other behavioral samples of learning at the same level of learning.AFMAN36-2236 12 NOVEMBER 2003 275

Figure 23.3. Sample Test Questions at the Comprehension Level.

SCENARIO: All questions below are base d on the following scenario:
Audiovisual specialists do not need to know ge ometry or trigonometry to do their jobs. The audiovisual specialist course director, Captain Ri chard Kline, believes thatall personnel should take geometry and trigonometry to increase thei r mental faculties. He has prepared a course proposal to submit to the school's curriculum committee.

STEP 1 LEVEL-OF-LEARNING LESSON OBJECTIVE. Comprehend the concepts of Perennialism, Essentialism, Progressivism, and Reconstructionism as they relate to education.

STEP 2 SAMPLES OF BEHAVIOR 1. Define each concept. 2. Identify examples of each concept. 3. Compare the critical attributes of each concept. 4. Predict the reaction of a person identified with each of the concepts to a specific educational issue or innovation. 5. List the critical attributes missing from a scen ario that would be necessary to illustrate a given concept.

STEP 3 Test Item (True/False) The curriculum committee, with a past voting record which is clearly Perennialist in nature, will probably support the proposal to re quire all audiovisual specialists to take geometry and trigonometry. T ( ) F ( )

Test Item (Matching) Match the predicted responses to the situation described in the scenario in column B with the major schools of educational philosophy liste d in column A by placing the appropriate letter in the space provided. Each of the responses may be used once, more than once, or not at all. Column A Column B Philosophy Responses ( ) 1. Essentialism a. "Yes , require all students to take these subjects." ( ) 2. Perennialism b. "No, students should not be required to take these subjects." ( ) 3. Progressivism c. "Yes, students should have a choice on which subjects to take." ( ) 4. Reconstructionism

Test Item (Multiple Choice) From the situation described above, the Perennialist committee members will: ( ) a. support the proposal. ( ) b. not support the proposal.276 AFMAN36-2236 12 NOVEMBER 2003

23.3. Test-Writing Process. The figures in this chapter are organized to illustrate the writing of test items. Although the subject matter varies, each figur e contains an appropriate example of the kind of information generated by each step in the process. The following general comments may be helpful in interpreting these figures:
23.3.1. Step 1. Each figure contains an acceptab le level-of-learning objective.
23.3.2. Step 2. Several samples of behavior are included fo r each level-of-learning objective. There is no significance to the number or order of these sa mples. Nor is there any significance to the specific samples of behavior used as illustrations. ( Figure 23.3. further illustrates how a single sample can generate any of severa l test item types.)
23.3.3. Step 3. These criterion-referenced test items have high content validity. Each figure shows a clear trail from the level-of-learning objective to th e test item. As products of the first two steps, use ( ) c. support teaching trigonometry only.

Test Item (Completion/Fill-in) In the situation de scribed above, the vote of the curriculum committee with a Perennialist philosophy will be __________________.

Test Item (Short Answer/Essay) What will be the vote of a curriculum committee in the situation described above if the committee is clearly:
(1) Essentialist (2) Perennialist (3) Progressivist (4) Reconstructionist

In general, the analysis of Figure 23.2. applies to this figure as well. This is another example of a lesson in which several concepts are to be taught, then test ed. These concepts happen to be philosophical theories as they relate to education. This general objective format applies e qually well to several alternative theories of management, different doc trines of strategy and tactics, or alternative concepts of leadership.
This figure is unique from the previous examples in th at it illustrates how the same sample of behavior can generate several alternative test items. The behavi or in this sample is to "predict." Whether the students let us know of their predictions by circling, ch ecking, matching, filling in, or writing them out is not important. The sample of behavior is particularly valuable to the student as a study guide if we change test item formats from time to time, or if we intend to test the same behavior with more than one item type.
Each of the samples illustrated in this chapter could be developed into the whole range of test items in the same way as shown for this figure. The significant student behaviors for this objective include define, identify, compare, predict, and list. The actual student behaviors required to complete the answer to the test item--select, match, and write—are of no particul ar consequence. This illustration further emphasizes the importance of well-stated and a ppropriate samples of behavior. Agai n, regardless of the test item type, the ability to predict is what is important about the several test item examples in this table.AFMAN36-2236 12 NOVEMBER 2003 277

these test items to provide evidence of achieving the level-of-learning objective. The test items measure what they are intended to measure.
23.4. Summary. The process for writing student-centered objectiv es and tests is a valuable planning tool. Almost all cognitive and affective objectives easily lend themselves to this treatment, as do many complex tasks involving psychomotor skills. General objectives, which specify a level of learning in some acceptable taxonomy, give us a starting point (Step 1). We then name the samples of behavior we will accept as evidence of learning (Step 2). Test items are then written to provide us with this evidence (Step 3).278 AFMAN36-2236 12 NOVEMBER 2003

Chapter 24

EVALUATION BY RATING

24.1. Introduction. When we evaluate student performance and traits, instructor observation and judgment are often the only satisfactory bases for evaluation. A paper-and- pencil test can measure understanding of administrative procedure, but we must exerci se judgment to determine how well procedures are administered. It is possible to devise examination it ems to determine a supervisor's understanding of management principles, but we need judgment to determine how well the person actually applies those principles on the job. 24.1.1. Rating instruments should be valid and reliabl e. Rating instruments are valid when they measure the trait or ability they are supposed to. Instruct or ratings are reliable when they measure traits or abilities consistently. We determine the validity of a rating and supporting rating instruments in much the same way as we determine the validity for any measurement of learning. We may need to establish content, concurrent, or predictive validity for a rati ng instrument. Validity is the most important characteristic of rating instruments as well as other measures of learning. ( Chapter 25 provides further discussion on ways to establish validity.) 24.1.2. The reliability of ratings is not difficult to determine. If we are rating supervisors on their ability to manage, we ask a number of qualified people to observe the performance and rate the supervisor. The extent to which the observers agree on the ability being rated is a measure of reliability. We can consider the rating highly reliable if the observers are in complete agreement. 24.1.3. Several people will rarely agree on the adje ctive describing the exact quality of a performance. Two observers who agree on the high quality of a performance may use different words to describe it in the rating. For example, in using the rating scale superior, excellent, good, fair, and unsatisfactory, one observer might describe a performance as "excellent" and the other describe the same performance as "superior." Ea ch may be trying to say the same thing, but in different words.
Figure 24.1. Rating Scale.

24.1.4. To avoid the apparent unreliability that aris es from semantic difficulties, the instructor may use a scale that has numbers instead of words, such as in Figure 24.1. Observers using this scale to rate a performer on ability to drive a car might circ le 9 if they thought the car was driven in the best possible manner; 1, if in the worst possible manner; 5, if average; and so on. However, simple numerical scales of this sort may provide observers wi th too meager a frame of reference. Numbers may mean the same things to an observer as words do. Numbers have the advantage of appearing to be equally spaced, easier to rank, and easier to tabulate.AFMAN36-2236 12 NOVEMBER 2003 279

24.2. Rating Methods. For the purpose of rating student perfo rmance, the instructor must assign an absolute value to the trait, performance, or ability of a student without reference to the performance or ability of any other student. In effect, the instructor rates the person, trait, or performance according to a fixed scale. Our instructional objectives specify the scale to be used in the rating and the fixed point at which the rated performance is accep table. Regardless of the number of points on a rating scale, we should divide the scale at some point into acceptable and unacceptable zones. Gradations within those zones are indications of how accep table or unacceptable the instructor rated the performance when compared to the standard. The acceptable and unacceptable data from these ratings can be analyzed in the same criterion-referenced way as other test data. (See Chapter 25 .)
24.2.1. Numerical Scale: 24.2.1.1. The simplest form of rating scale is th e numerical scale. Any number of points on the scale are feasible, but an odd number is commonly us ed so the center figure can stand for average or passing. The number of points on the scale should depend upon the number of differentiations required and upon the ability of instructors to differentiate.
24.2.1.2. Most instructors can make at least five differentiations, which break a group into those who far exceed standards, slightly exceed standard s, meet standards, fall slightly below standards, and fall far below standards. Few trained instructor s can reliably make more than nine differentiations. As a result, most rating scales contain five to nine points.
24.2.2. Descriptive Scale. The descriptive scale uses phrases to indicate levels of ability. Figure 24.2. shows a descriptive scale for rating pilot ability. Fi ve levels of ability are described. Such a scale is more versatile than the numerical scale because the degrees of excellence can be varied to suit the occasion. For example, suppose an instructor wants to evaluate a particular student trait. He or she feels all the students are notably satisfactory, but th e course instructor wants to know to what degree each is better than satisfactory. A numerical sc ale might be useful, except for the common impression that the lowest numbers on the scale i ndicate very inferior performance. 24.2.2.1. A good choice of descriptive phrases helps to reduce rating errors. If instructors do not use extreme descriptions, the categories at the e nd of the scale are less objectionable. By using a descriptive scale, like the one shown in Figure 24.2. , the instructor has a better frame of reference. Here, only the lowest rating possible is labeled una cceptable. All other ratings indicate degrees of acceptability. If we had used a numerical scale in this instance, instructors might have tended to shift ratings to the high end of the scale.
Figure 24.2. Descriptive Scale for Rating Student Performance.280 AFMAN36-2236 12 NOVEMBER 2003

24.2.2.2. The major disadvantage of using descriptive scales is a semantic one. An "excellent" problem solver does not mean the same thing to a ll instructors. Another disadvantage is the difficulty in selecting phrases that describe degrees of performance that are equally spaced. When the scale shown in Figure 24.2. is used, most people feel there is less distance between "excellent" and "superior" than between "fair" and "good." We ll-written instructional objectives, with clearly defined standards for each point on the scale, can help to lessen this problem.
24.2.3. Graphic Scale. The graphic scale is a combination of the numerical and descriptive scales. Besides a numerical scale, various adjectives or phrases are set below a continuous horizontal line, which represents the range of the ability or tra it being measured. In using the graphic scale, the instructor must consider the numerical range of th e scale, the phrases that describe the various positions on the scale, and the point on the scale used to separate acceptable from unacceptable performance. 24.2.3.1. Three typical forms (examples A, B, and C) of the graphic scale are shown in Figure 24.3. In example A, the instructor is given instructions for judging the energy level of students. The instructor is instructed to mark the scale after considering energy and application to duties on a daily basis. These instructions help reduce rating errors and encourage instructors to consider the same things about each student.AFMAN36-2236 12 NOVEMBER 2003 281

Figure 24.3. Examples of Typical Forms of the Graphic Scale.

24.2.3.2. Example B in Figure 24.3. shows a graphic scale in which certain types of behavior are described for each point on the scale. With most scales, the instructor must not only observe but also evaluate to form a rating. People can observe more accurately than they can evaluate what they have observed. The difficulty of evaluation increases errors of rating. When ratings can be based on observations alone, reliability is greatly improved. With the scale in example B, the instructor is only required to record—not evaluate—the actions of the person being rated. Hence, this type of graphic scale incor porates much objectivity. If a trained instructor observes accurately and records honestly in using this scale, most rating errors should be eliminated. In preparing this type of scale, the instructor must make sure the behavior described for each point is actually an improvement over that for the point just below it. In each cas e, distances between the points should appear to be about equal. 24.2.3.3. The scale in example C is similar to that inexample B except that descriptive phrases are not provided for all points on the scale. In some respects this is an improvement. Many times instructors feel the rating should fall somewher e between two points. Such a rating is possible with this form of the graphic scale. The fuller descriptions in example C increase the likelihood that observed behavior could be pinpointed on the scale. Generally speaking, the more detailed the descriptions, the better the rating results. At so me point, however, the descriptions may be too UNSATISFACTORY SATISFACTORY
Example A 1 2 3 4 5 Industry: Consider energy and application to duties day in and day out. Lazy Indifferent Diligent Energetic Untiring

Example B 1 2 3 4 5 Cooperation: Creates Indiffer ent Gets along An Actively Demonstration of friction. to others. with most harmonious promotes willingness to work with people. team harmony in others. worker. working with others.
Example C 1 2 3 4 5 Initiative: Action taken on Slow to act, Takes Anticipates own responsibility. even when a decision is much needed. Waits for others. Lets opportunities pass. Does not volunteer. Reticent. needed action without delay. Volunteers for some tasks. Undertakes all routine jobs without supervision. Dependable. needs. Works ahead and prepares for possibilities. Actively seeks opportunities. Eager.282 AFMAN36-2236 12 NOVEMBER 2003

long to print directly on the rating form. At that point, brief descriptors may be used as reminders of the more detailed descriptions published elsewhere.
24.2.4. Checklist. The checklist is sometimes considered a separate rating method, but is actually a two-point rating scale. A checklist is useful for rati ng the ability to perform a set procedure. It is also a simple method of rating skills when the purpose is to determine whether students have reached a certain minimum level, without regard to degr ees of acceptable or unacceptable performance. Figure 24.4. shows a portion of a checklist that might be used to rate instrument flying proficiency. In usingthe scale, the instructor indicates whether the completion of each step was satisfactory or unsatisfactory. Breaking a performance into many observable el ements eliminates or reduces many rating errors.
Figure 24.4. Checklist for Rating Pr oficiency in Instrument Flying.

24.2.4.1. Reliability is usually high in checklist rating because of the "go" or "no go" nature of thedecisions required. For example, if apples are to be sorted into a hundred different categories ranging from rotten to top quality, the possibility of error would be enormous. If the categories are reduced to rotten, partially rotten, fair, good, and excellent, the number of possible errors is greatly reduced. Fewer choices require fewer and less-re fined judgments. When the number of choices is reduced to two, rotten or sound, the chance for eithererror or bias is still further reduced, and reliability and objectivity improve correspondingly. Because of its many broad differentiations, the checklist is comparatively reliable. 24.2.4.2. On the down side, however, only a limited amount of information can be obtained from a checklist. For example, the checklist in Figure 24.4. does not indicate how far above or below standard the satisfactory pilot pe rformed each maneuver. For purposes of an instrument check, the information obtained would be sufficient to dete rmine whether each pilot reached a certain minimum level of proficiency, but little else is determined.
24.2.5. Product Scale: 24.2.5.1. At times, we rate the tangible product of a performance rather than the performance itself. Figure 24.5. shows a product scale for rating the ability to fly an instrument trainer. We require the pilots being tested to fly a pattern that corresponds to the pattern shown at E. After completing the exercise, we compare the patte rn produced to the patterns on the scale. INSTRUCTIONS: If the performance is satisfacto ry, place a "+" in the space provided. For unsatisfactory performance, place a "-" in the space provided. 1. Maintains constant heading (within 5 o of course). [ ] 2. Maintains constant altitude (within 50 feet). [ ] 3. Can make a timed turn (gyros caged) (within 10 o of new heading). [ ] 4. Can make a steep turn (within 50 feet of altitude). [ ]AFMAN36-2236 12 NOVEMBER 2003 283

Figure 24.5. Product Scale.

24.2.5.2. If followed carefully, this procedure can e liminate nearly all rating errors. The scale provides a tangible standard a rater can use to me asure the product. Product scales tend to reduce or eliminate most rater errors.
24.2.6. Forced Choice. Another approach to performance rating is the forced-choice technique, which uses two to five descriptions of behavior in each block. Both favorable and unfavorable statements may be included in a block. Most rating instruments contain 25 to 30 blocks. Figure 24.6. shows a single block containing four descriptions —two favorable and two unfavorable. The rater is asked to select the most characteristic and the least characteristic statements. Although the favorable and unfavorable statements are easily distinguished, the rater does not know whether A or B receives more credit.284 AFMAN36-2236 12 NOVEMBER 2003

Figure 24.6. Single Block of Forced-Choice Instrument.

24.2.6.1. The decision is so precise that the observers are usually forced to ignore their general impression and think back to specific instances of behavior. Because the rater reports behavior without being permitted to judge the quality of the behavior, reliability is greatly improved. 24.2.6.2. An objection to the forced-choice technique is that considerable research is necessary to produce a scale for any given purpose and to devise a fair method of keying the responses. For example, before the scale is constructed, item s known to differentiate between good and poor performances are paired with items that do not differentiate. Both items in the pair must be shown to have approximately equal appeal to the rater. Because of such factors, instructors seldom try to construct forced-choice rating scales.
NOTE: The scales described so far are probably the most common. Except for the forced choice, most of the scales considered (or variations of them) can be designed by classroom instructors. Naturally, they should be constructed carefully to serve specific pur poses and keyed to carefully written instructional objectives.
24.2.7. Other Rating Scales. It is possible but quite difficult to use more sophisticated rating scales in a criterion-referenced situation. These types ofscales are normally developed only by experts. Generally speaking, better data for criterion-referen ced testing is obtained through numerical scales, descriptive scales, graphic scales, checklists, and product scales. Some of these types of scales include:
24.2.7.1. Sociometric Techniques. This allows individuals to rate themselves or others according to their performance in a group. Some sociom etric techniques are usef ul to evaluate social traits and attitudes. Others can be applied in the classroom to j udge the amount or quality of participation by students in projects and discussions . Some techniques employ diagrams, often called sociograms, which are easy for an instructor to use.
24.2.7.2. Q-Techniques. This is a type of rank-order scaling. It rates traits as well as performances of students and teachers. Of ten the rater sorts and scores a predetermined set of descriptive terms or makes a selection of terms after a specific period of observation. Most characteristic Least characteristic

A. Can take over in an emergency. [x] [ ]

B. Fair and just in dealing with others. [ ] [ ]

C. Lacks interest in job. [ ] [ ]

D. Questions orders from supervisors. [ ] [x]AFMAN36-2236 12 NOVEMBER 2003 285

24.2.7.3. Semantic Differential Scale. These depend upon key words with opposite meanings. The words are placed at opposite ends of a line w ith numbers representing a spectrum of values, in this manner:

NOTE: Surprising success has been reported by re searchers working with such scales.
24.2.7.4. Self-Anchoring Scales. These use the rater's opinion as a point of departure and are frequently written for self-evaluation. One might begin: Here are the numbers 1 through 10. Let 1 represent the untrained beginning instructor, and le t 10 represent the ideal master instructor. Now circle the number you feel best represents your abilit y level as an instructor today. From this point, instructors can estimate their ability in the past and future or even estimate the abilities of others.
24.3. Constructing the Rating Device. The process for constructing a rating instrument involves certain general steps that will naturally vary according to the type of scale being made. In all cases, however, the type of rating scale used is dictated by the performance or trait to be measured as described in the instructional objective.
24.3.1. Determine Objective and Purpose. The initial step is the same for constructing any other measuring instrument—determine the objective and carefully define it in terms of observable behavior (see Chapter 3 ). An objective may be to determine a student's effectiveness in a particular skill, to evaluate the level of achievement in performing a co mplex task, or to predict the proficiency in applying what has been learned. After the objective has been clearly defined, the task is to determine not only what to evaluate (the objective), but also why the evaluation is to be made (the purpose).
24.3.2. Select Traits or Factors. This step requires careful consideration of the job that the subject is now or will be doing. In light of the objective and purpose of the rating, what traits or factors determine the success or failure of the person being rated? Selected traits or factors must be critical, observable, distinguishable, specific, differentiating, and limited in number. 24.3.2.1. Only select traits or factors critical or essential to the person's success. If promptness is not a trait that must be considered to determine a degree of proficiency, do not include it. On the other hand, if it is essen tial and an objective of your course, it must be included to make the rating valid. The fact that a trait is highly important or de sirable in a rating is no guarantee it is essential. 24.3.2.2. Traits or factors must be observable. Base ratings on what you can see instead of yourinterpretation of the situation. Base the rating upon an eyewitness report—circumstantial evidence. 24.3.2.3. Select distinguishable traits or factors. If the traits are so closely related that a rating on one ensures a similar rating on the other, logical error and loss of comprehensiveness will lower the validity of the rating.
24.3.2.4. Select traits or factors that can be define d in terms that have the same meaning for all instructors. They should be specific and defini te, rather than vague or general in nature. 24.3.2.5. After studying the traits that seem to meet the criteria, eliminate those that do not differentiate. If all people possess the trait to the same degree or level, it serves no useful purpose in the rating scale. autocratic 1 2 3 4 5 6 7 8 democratic286 AFMAN36-2236 12 NOVEMBER 2003

24.3.2.6. The number of traits or factors may vary from 4 to 10. Having too many factors or traits may overburden instructors. Accurate observations of a critical sampling are superior to sporadic observations of a bulky list of traits.
24.3.3. Define Traits. The traits or factors selected must be clearly and accurately defined in terms of observable behavior pertinent to the situation. De finitions should show what students do rather than how well they do it. For example, voice (as used by a drill instructor) might be appropriately defined as "volume and clarity in giving commands." Such comments as "uses voice well" or "an able and effective communicator" are inadequate. The first gives no guidance as to what is to be considered under voice and the second is, in itself, an evaluation.
24.3.4. Determine Degrees of Success: 24.3.4.1. The levels of performance should relate to the purpose of the evaluation and the instructor's ability to differentiate. If the pulse of the rating is to identify only three groups—poor, average, and good—there is no justification for using a fi ve-point scale. If the nature of the evaluation is such that the rater can identify or distinguishbetween only two levels of performance, a scale of seven or more levels of atta inment would be inappropriate. 24.3.4.2. Rating scales used for criterion-referenced measurement should have an understood or labeled point to distinguish acceptable from unacceptable ratings. This distinction is necessary regardless of the number of points on the rating scale used to grade levels of acceptability and unacceptability.
24.3.5. Describe Degrees of Success. Each degree of success or level of performance in each trait or factor should be described in terms of observable be havior. These word pictures indicate how well the person performs. Quality of performance ra ther than frequency should be stressed.
24.3.6. Assign Weights. Determine how important each factor is a nd weight it accordingly. It is possible that all factors are not of equal importance. Figure 24.7. provides an example of a rating system for two duty assignments—one administrative, the othe r technical. The weight given each factor can be determined by pooling the opinions of several persons thoroughly familiar with the performance requirements specified in the in structional objective or supporting documents. Examine these weights after the scale has been used to determine if the ratings actually differentiate between those who are effective and those who are ineffective on the job.AFMAN36-2236 12 NOVEMBER 2003 287

Figure 24.7. Example of a Rating System for Two Duty Assignments.

24.3.7. Validate. A validation study is a method or procedure used to examine, review, and improve the rating instrument itself. In the validation of a rating scale, the following considerations are important: 24.3.7.1. Base the study on a significant number of cases, normally 50 to 100. With small groups, it may be necessary to use the scale seve ral times before attempting to validate it.
24.3.7.2. In order to determine the reactions of ra ters, compare the ratings of one individual made by several raters, as well as the ratings of se veral individuals made by a number of raters.24.3.7.3. Note variations in ratings and determine the cause. The most common causes of variations are poorly defined traits, in adequate description of performan ce levels, and improper weighting (see Chapter 21 ). 24.3.7.4. Validation of the rating scale is a continuous process. If the instructor does not analyze, critique, and improve the rating instrument , it will not serve its intended purpose.
24.4. Common Rating Errors. Instructors may differ in their ratings when scales are used to record observations and judgments of quality. These differe nces are often called rating errors. Some errors appear to be caused by the design of the scale, some occur only with certain groups of observers, and some happen with individual observers. Some observe rs are subject to errors when rating all persons, some when rating certain groups, and some when rati ng only certain individuals. Other errors occur only when certain traits or attributes of students are rated.
24.5. Classifying Rating Errors. Errors can be classified into fourbroad groups: error of central tendency, error of standards, error of halo, and logical error. The first two kinds of errors, central tendency and standards, affect all students rate d by an instructor who is subject to either of the errors. The third type of error, halo, affects only certain students within a group. The fourth, logical error, appears only when two or more traits of students are being rated. These common errors may all be reduced; some may be eliminated by training the instructors and carefully designing a rating scale. POINT VALUE

Administrative Technical I. Job knowledge 15 10 II. Cooperation 20 10 III. Judgment 20 20 IV. Personal responsibility 20 10 V. Leadership 15 10 VI. Growth potential 5 35288 AFMAN36-2236 12 NOVEMBER 2003

24.5.1. Error of Central Tendency. Many instructors hesitate to give either extremely good or extremely bad ratings and tend to group their ratings close to the center of the scale. This tendency generally occurs with inexperienced instructors. Ev en with experienced instructors, this error may appear when rating personal qualities or abilities that are difficult to identify, such as teaching ability or personality. Analyzing a number of ratings made by a single instructor may reveal the error of central tendency. For example, on a 9-point scale, an observer made 100 ratings of administrative ability, all of which were either 4, 5, or 6 ( Figure 24.8. ). You would assume that administrative ability would vary more in a sample of this size and that the observer made an error of central tendency.
NOTE: Some observers place too many ratings at the extr emes and too few near the center of the scale, but this tendency is rare. Although the error is the opposite of central tendency, it may be considered in that category.
Figure 24.8. Error of Central Tendency.

24.5.2. Error of Standards. Some instructors tend to overrate or underrate because of the differences in their standards. Standards of physical measurement are fixed units—inches, centimeters, ounces, and grams. But in rating with only mental standards for comparison, there may be as many different standards as individual instructors. 24.5.2.1. Well-trained and experienced instructors us ually distribute their ratings similarly, which usually indicates that similar standards are use d. Inexperienced and untrained instructors are less likely to have similar distributions. 24.5.2.2. Thirty persons, representing an average group, were simultaneously rated on a single trait by instructors A and B who had different interpretations of the standards of acceptability for that trait. The differences in standard are reflected by the ratings in Figure 24.9. If instructors A and B were to rate an additional group of 30 persons, their standards should probably be more nearly alike. As instructors A and B receive more training and gain experience in rating this trait, their distributions should tend to agree more closely, particularly as they relate to the point of acceptable or unacceptable behavior.AFMAN36-2236 12 NOVEMBER 2003 289

Figure 24.9. Error of Standards.

24.5.2.3. If instructors A and B were already experien ced raters, it would appear that A has a strict interpretation of the standards and B has a lenient interpretation. If experienced instructors have consistent yet quite different interpretations of standards, training should reduce the differences. One partial administrative correcti on procedure would either be to add a certain number to all of A's ratings or to subtract that number from all of B's; another would be to add to A's ratings and subtract from B's. An effective correction procedure can be determined after comparing the ratings given by A and B with the ratings of several othe r competent instructors. Better still, the course director can inform instructors A and B of their er ror in standards, give them further training, and encourage them to make more consistent yet correct ratings.
24.5.3. Error of Halo or Horns. Instructors sometimes allow their ratings of performance to be influenced by their general impression of the student. Such an impression is usually formed on the basis of observations or knowledge that have little or nothing to do with the rating. If allowed to influence judgment, this outside influence would result in an upward or downward shift of all ratings. If instructors are favorably impressed, they may rate students too high (error of halo). If they are unfavorably impressed, the shift is toward the low end of the scale (error of horns). The halo and horns errors may affect an entire group or only a few individuals within a group. 24.5.3.1. Halo and horns errors can be traced to the likes, dislikes, opinions, prejudices, and moods of raters. When considering friends or close acquaintances, instructors may tend to give undeserved high ratings on all favorab le traits. Such a halo error is often called error of leniency. 24.5.3.2. Some people believe that close-set eyes s how dishonesty. An instructor who holds this belief is likely to be affected by it when rating a student with close-set eyes. Some people havepreconceived concepts about certain racial groups. They may believe that one group is industrious, another thrifty, yet another excitable. Others have an aversion to or a special liking for religious groups. All these preconceptions may influence instructors. When the halo or horns error is traced to such sources, it is sometimes called error of stereotype. 24.5.3.3. For example, Jones and nine other instru ctors simultaneously rated six persons on teaching ability. In Figure 24.10. , all 10 instructors agreed reasonabl y well on the teaching ability of 5 of the 6 persons rated. All but Jones agreed on the teaching ability of the sixth (F). Apparently Jones allowed some general impression regarding F to influence the rating unduly.290 AFMAN36-2236 12 NOVEMBER 2003

Figure 24.10. Halo or Horns Error.

24.5.3.4. The error of halo or horns may be susp ected in some situations, but frequently goes undetected. These errors can be positively identif ied only when many competent and experienced instructors rate a number of persons under identi cal conditions. However, the simultaneous use of a large number of instructors is often unecono mical and administratively inadvisable. Usually only extreme cases are detected, even under c ontrolled conditions. Even when halo and horns errors have been identified, their reappearance ca nnot usually be predicted. This is the most difficult error to overcome.
24.5.4. Logical Error. A logical error may occur when two or mo re traits are being rated. An instructor who tends to give similar ratings to traits that do not necessarily go together is making a logical error. For example, some instructors may think an i ndustrious person is also efficient, or that a prompt person is also industrious. This is not necessarily so.AFMAN36-2236 12 NOVEMBER 2003 291

24.5.4.1. The term logical error is used to mean that the traits are related in the mind of the person making the error. The relationship may not appear to be logical to someone else. As a matter of fact, the instructor who exhibits an error of this sort is probably not really aware of it.
24.5.4.2. Figure 24.11. shows how six observers (A, B, C, D, E, and F) rated a certain person on four traits—industriousness, promptness, efficiency, and courtesy. On three of the traits—industriousness, promptness, and courtesy—the six observe rs agree reasonably well. However, E gave a much higher rating on efficiency than did the othe r observers. Since E assigned the same rating to both efficiency and industriousness, it appears the instructor thinks industriousness and efficiency are much the same, a logical error. There may be many reasons for this error: E may not have had adequate opportunity to observe the performer's efficiency, may have an erroneous conception of efficiency, or may not be able to distingui sh between efficiency and industriousness.
Figure 24.11. Logical Error Ratings.

24.6. Improving Ratings. Ratings can be improved and errors re duced or eliminated by using well-constructed scales, but greater improvement is possible through the training of raters. Satisfactory reliability can be achieved only when an instructor has gained experience in rating and is familiar with all causes of error. Instructors should thoroughly understand the common rating errors, correct rating procedures, and the various rating techniques. The more an instructor knows about causes of unreliability, the easier it is for that instructor to avoid errors.
24.7. Summary. By definition, rating contains a subjective element. Therefore, it is subject to the following types of errors: logical, central tendency, standard s, and halo (or horns). Despite these possible errors, the rating process may be the most effective method available for evaluating a person's overall ability orthe quality of performance. Because of common rating errors, differences in ratings may indicate the ability of the instructor to rate rather than the actual di fferences between the people rated. In view of that possibility, it is necessary to frequently analyze rating re sults. Instructors can further improve ratings through careful consideration of the various steps in the construction of the rating scale, and most important of all, by thorough and continuous training.
NOTE: For additional examples of rating instruments, see Chapter 10 and Part III to Attachment 9

.292 AFMAN36-2236 12 NOVEMBER 2003

Chapter 25

CRITERION-REFERENCED EVALUATION

25.1. Introduction. Tests should measure how well students meet our instructional objectives. From true-or-false tests to simulated war games, we gath er measurement data to compare student performance to what we intended to teach. Before we start to teach, tests tell us which students need some or all of what we have to offer. During the course of instruction, tests tell us and our student s how they are progressing. At the end of instruction, tests tell us whether indi viduals have achieved each instructional objective. 25.1.1. How we report and analyze criterion-referenced test (CRT) items is the focus of this chapter. Generally speaking, CRTs are different from the norm-referenced tests we have used for many years.Some consider CRT analysis to be a new point of view brought on by our increasing use of student-centered objectives. When we are no longer as c oncerned about using test data to rank order students or to give comparative grades (such as A, B, C), then norm-referenced testing (see Chapter 26 ) is of secondary importance. The pr imary concern of a CRT is to measure, report, and analyze student performance as it relates to specific, significant instructional objectives. 25.1.2. The placing of norm-referenced testing and anal ysis in a secondary role may be the most difficult concept to accept in this manual. See Chapter 26 for a detailed discussion on the use of test data in norm-referenced applications. Mo st of us have experience with normative testing systems. We have consciously or unconsciously grown to expect tests and test data to be used to rank order or compare students to each other. 25.1.3. Sometimes, we are concerne d about how many students passed with a score of 70 percent or what percentage of test takers are distinguished graduates. But we often forget the primary purpose of evaluation programs in Air Force schools is to me asure student achievement of instructional objectives. No other use of test data should interfere with our need to compare what students can do to what they should be able to do, as descri bed by our instructional objectives. 25.1.4. It is important, then, to become familiar withthe ways we can obtain, report, and analyze CRT data. Chapter 3 through Chapter 5 and Chapter 21 through Chapter 24 describe in detail the techniques needed to obtain CRT data from well-written objectives and tests. In the remainder of this chapter, we will discuss useful ways in wh ich CRT data may be reported and analyzed.
25.2. A CRT Reporting Format. We have better ways to report CRT da ta than percentage test scores or traditional letter grades. Neither of these old standbys te lls us what we really need to know. Yes, they do tell us how our students performed relative to others in the class. But the more important question is: Did any or all of the group members reach the course obj ectives? Relative grading does not report that critical piece of information. 25.2.1. A better way to present CRT data is a simp le reporting grid. Already in use in many schools, such a grid lists the objectives to be mastered along one axis and the names of individual students along the other. As each student passes a measure of each objective, an appropriate notation is recorded in the correct square on the grid. If ov erall class performance on specific objectives is of interest, we can tally individual scores and report the number or percentage of the group who have achieved the objectives.AFMAN36-2236 12 NOVEMBER 2003 293

25.2.2. Figure 25.1. illustrates a quite simple reporting format for CRT data. In this example, CRT test results show how each student is progressing inthe course based on CRT performance. Each CRT gives us pass or fail or "go" or "no go" data of eac h student for each objective. Student A achieved two of the objectives but failed an attempt at passing a CRT on the third objective. Student D attempted and passed a CRT on each of the four objectives. Student E failed the CRT for objective 1 and has been unable to go on to the other objectives. We can make many sophisticated variations to this simple grid, but the basic idea remains the same. In order to analyze the effectiveness of our instructional program, we need to test and report CRT data for each student on each objective.
Figure 25.1. Simple Reporting Format for CRT Test Data.

25.2.3. Figure 25.2. is a more sophisticated reporting formatthat achieves the same basic purposes as Figure 25.1. Actual test scores are reported in Figure 25.2. , and the standard for passing is shown. For example, student D scored only 43 on objective 1 and failed because a minimum passing score of 85 is required by the instructional objective. This student passed objective 2 with a score of 13 (9 is listed as the minimum passing score). Student D also scored what appears to be a very fine 98 on objective 3 but failed that objective because the standard is 100. On the fourth objective, student D received a passing rating for a task which has no numerical value and is simply rated "pass" or "fail." Overall, student D failed, or at least has not yet satisfactor ily completed, the course of study. The data on the grid shows the minimum course standard is passing at least three of the four stated objectives. StudentD achieved only two objectives—one short of the required minimum. Even though student D performed well above standard and well above all other students in the group on objective 2, the test results on individual objectives are not average d, so outstanding performance on one objective cannot make up for poor performance on others. No averag ing, grading on a curve, or using other leveling devices is another significant di fference between CRT analysis and more traditional normative techniques.

Student Objective 1 Objective 2 Objective 3 Objective 4 A Yes Yes No B Yes Yes Yes No C Yes No D Yes Yes Yes Yes E No294 AFMAN36-2236 12 NOVEMBER 2003

Figure 25.2. Detailed Reporting Format for CRT Data.

25.2.4. An even more detailed display of CRT data is illustrated in Figure 25.3. This figure displays two important additional items of data not found in Figure 25.1. and Figure 25.2. —the number of behavioral samples (keyed to the actual number of the test item) used to measure achievement of the objective and the level of performanc e on the test items (mastery leve l) required to show satisfactory achievement of the objective.
Figure 25.3. Summary Table of Mastery or Nonmastery by Curriculum Objectives. OBJ OBJ OBJ OBJ Total Minimum
Raw 1 Raw 2 Raw 3 4 OBJS Course Course
Score STD P/F Score STD P/F Score STD P/F P/F Passed Standard Grade Student A 80 85 F 5 9 F 97 100 F P 1 3 of 4 Fail
Student B 85 85 P 11 9 P 100 100 P F 3 3 of 4 Pass
Student C 97 85 P 9 9 P 100 100 P P 4 3 of 4 Pass
Student D 43 85 F 13 9 P 98 100 F P 2 3 of 4 Fail

__ Totals 4 2 Pass 3 Pass 2 Pass 3 Pass 10 Pass 2 Pass
2 Fail 1 Fail 2 Fail 1 Fail 6 Fail 2 Fail

COMPREHENSIVE SUMMARY OF INDIVIDUAL STUDENT PERFORMANCE

STUDENT RECORD #226638

OBJECTIVE 1 OBJECTIVE 2 OBJECTIVE 3 Behavioral

Sample Test Item # Pass/ Fail Behavioral Sample Test Item # Pass/ Fail Behavioral Sample Test Item # Pass/ Fail 1 8 F 2 16 P 1 1 P 2 9 P 3 22 F 1 2 P 3 10 F 6 23 P 2 7 F 4 11 P 7 24 P 2 17 P 5 12 P 8 30 P Summary 5 3 5 4 4 3 Mastery Level4 of 5 4 of 5 3 of 4 (Passing) Mastery Decision No Yes Yes (Pass?)AFMAN36-2236 12 NOVEMBER 2003 295

25.2.4.1. In this figure, student #226638 attempted to pass three objectives. The data from this grid reveals that objective 1 was not passed but obj ectives 2 and 3 were (see bottom line of grid, mastery decision). The data reported for each objec tive supports the pass or fail decision. Five different samples of behavior were tested for objective 1 (samples 1 through 5). Each sample was tested by a single test item (items 8 through 12). Th e pass or fail data for objective 1 shows the student failed items 8 and 10 but passed items 9, 11, and 12. The mastery level for this objective is set at 4 of 5 items passed. Therefore, the student failed to achieve objective 1, indicated by the word "No" on the bottom indicating the mastery decision. 25.2.4.2. Data for objective 2 reveals the student passed the objective by answering 4 of 5 testitems correctly. The minimum standard for this objective is 4 of 5, so a "Yes" is entered on the bottom indicating the mastery decision. 25.2.4.3. The student also passed objective 3. Notice that the test on this objective was put together a little differently and the mastery leve l has changed. Although there are four test items, only two samples are being measured for the objective. (This may be a math objective where more than one problem is typically tested on the same math function.) Also note that there are only four test items for this objective instead of five as for objectives 1 and 2. We can test any number of behavioral samples with any numbe r of test items. In objectives 1 and 2, we tested five samples each with one test item per sample. For objective 3, we only tested two samples but used two test items for each sample. The number of samples test ed and the number of test items is a value judgment on our part, in the sense that there may be no precise statistical data or job data to help us make a decision on the number of samples to test. We may have to rely heavily on expert opinion to decide what level of performance is acceptable. 25.2.5. Close in importance to the data on indi vidual students is dataon group performance. Figure 25.2. gives us totals about the class as a whole so we can evaluate course effectiveness. CRT analysis shows that no objective was mastered by all of the students—an important fact for school managers. Further, students achieved only 10 of 16 objectives (4 students X 4 objectives). This data may raise further questions about the effectiveness of the curric ulum or at least about the efficient and effective use of Air Force resources.
25.2.5.1. Figure 25.1. , Figure 25.2. , and Figure 25.3. report individual student achievement of course objectives. We determine how well students have mastered these objectives by tests that measure carefully stated objectives. This display of CRT data for analysis reports the outputs of these courses. We can make the grids more sophis ticated, if required, by adding items such as the number of attempts to pass each CRT, the dates of the CRT, and the length of instructional time between CRTs. 25.2.5.2. Some of the data from these grids could be used to rank order student performance so as to arrive at comparative grades or distinguished graduates. However, these uses of the data should not interfere with the primary purpose of CRT analysis: to account for the performance of each student on each objective. One unique way, then, to use the data from these figures for comparing students is to rank them by the number of objectives achieved rather than the average of their test scores or letter grades. Based on CRT data, the ranking for students (from highest to lowest) in Figure 25.2. is student C, student B, student D, and student A. 25.2.6. Ideally, none of the students who come to our schools for group instruction should already be masters of our course objectives. If they are able to perform at the level described in our instructional296 AFMAN36-2236 12 NOVEMBER 2003

objectives, they don't need the school. By the sa me token, all students who complete our course should ideally have mastered all of the instructional objectives. If the objectives really satisfy Air Force needs and if we really teach and test with a focus on the objectives, we should have all students meet all objectives. This is an ideal we may never ac hieve, but it is an ideal toward which we work. 25.2.6.1. If too many students show up in our classrooms already knowing what we intend to teach, we are wasting time and resources by maki ng them take the course. To reduce the number of students who show in a pretest that they do not need some or all of our school, we can take some of the following actions: 25.2.6.1.1. More carefully describe prerequisites and course objectives to eliminate overlap between courses. 25.2.6.1.2. Design special tracks for individuals through self-paced, individualized instruction rather than lock-step group instruction. 25.2.6.1.3. Provide exemption procedures that allow students to skip some or all of a course while still receiving credit based on a pretest. 25.2.6.2. If we must use large group instruction with instruction time as a constant, we may have to settle for less than 100 percent of the stude nt body achieving all course objectives. However, we should not take lightly the figure we do settle for. It may be more reasonable to accept a lesser goal because of scarce resources, limited time, or lack of specific student motivation or aptitude. The effectiveness of group instruction is often judged by arbitrary standards such as 90 percent of the students will achieve 90 percent of the objectives (or 80 percent of the students will achieve 80 percent of the objectives). These figures may seem far more reasonable than the 100 percent of thestudents achieving 100 percent of the objectives, but we must realize that the 90 percent/90 percent standard means we are satisfied with the course if 81 percent of possible objectives are achieved (90 percent of possible objectives X 90 percent of the student body to achieve them). In applying the 90 percent/90 percent criterion to a c ourse with 20 objectives and 50 students (a possible total of 1,000 objectives to be achieved), we would only need to accomplish 810 objectives overall (18 objectives X 45 students) to consider group instruction successful. 25.2.6.3. There are no statistical techniques to tell us what pretest and posttest percentages are acceptable. Statistics do not make decisions for us ; they simply give us data for decisionmaking. Value judgments are ours to make, and we need to make a value judgment on acceptable pretest and posttest percentages. We can arrive at these value judgments in many ways, some of which have little or no educati onal basis. Personnel problems, economic factors, or other considerations may influence the decisions.
25.3. CRT Item Analysis. Since the term "criterion-referenced" was first introduced in 1962, there has been a great deal of research to improve methods of evaluating student performance against established standards as opposed to the traditional norm-referenced methods of evaluating students. As a result of that research, there have been a number of different statisti cal formulas developed to use in CRT item analysis. We will look at two very effective formulas that give us useful data to determine the effectiveness of criterion-referenced evaluation programs.
25.3.1. Ease Index (EI). As indicated earlier, the ideal CRT situation includes a pretest and a posttest. These tests give us information about what the students know about our objectives before taking our course (nonmasters) and then measure how much was learned as a result of our course (masters).AFMAN36-2236 12 NOVEMBER 2003 297

To make this comparison, we must look at each indi vidual test item to determine how easy or difficult the item is. If a majority of our students are answering a particular item correctly, this indicates the students know the objective tested (provided it is a well-constructed an d valid test item [see Chapter 22 ]). 25.3.1.1. The EI is found by identifying the number of students who correctly answered the test item (R) divided by the number of students who took the test (N). This total is then multiplied by 100 to give a percentage ( Figure 25.4. ).
Figure 25.4. Ease Index Formula.

25.3.1.2. This formula is frequently used for both criterion- and norm-referenced evaluation. Although the statistical results will be the same, the interpretation of the results will differ (see Chapter 26 for norm-referenced interpretation). In general, a test item with an EI of 0 to 40 percent is a difficult item, an EI of 41 to 75 percent is considered to have a medium difficulty, and a 76 to 100 percent EI is considered an easy item. 25.3.1.3. For example, if 20 students out of 80 answ ered an item correctly, the formula for computing the EI would be 20/80 (100) = .25. For a pretes t, this may well be an indicator the students need to be taught the material. If this were the EI for a test item after the material has been taught, it might indicate there was limited learning of this particular item. However, remember that the numbers are only indicators. They do not identify or solve problems. They should get our attention to give us an idea of how our educational program is working. 25.3.1.4. When using the EI with a pretest in CRT, an EI of 50 percent or above on a given item is considered unacceptable. The 50-percent point is an arbitrary number and the actual cutoff point or standard for a pretest item is a decision th at depends on the school objectives, curriculum, and a wide variety of data.25.3.1.5. When using the EI with a posttest in CRT, an EI of less than 80 percent may be considered unacceptable. After having taught the material, ifwe have constructed test items well, the students should perform much better on the test. Mu ch like a 50 percent EI on a pretest (nonmasters), the 80 percent EI for a given item on a posttest (masters) is an arbitrary figure. This figure may vary depending on a number of educat ional factors and value judgments. 25.3.1.6. By looking at the EI of individual test items on pretests and posttests, we can gain a great deal of valuable generalized information in mana ging an educational program. As a result of using the EI formula, we can gain insight into howmuch educational growth has taken place.
25.3.2. Differentiation Index (DI). The Cox & Vargas DI simply measures the difference between the EI of a test item on the pretest and that same item on a posttest. More often than not, when using criterion-referenced analysis, the same test items are used for the pretest and posttest. Although this is R/N (100) = % Ease Index R = Number who answered item correctly divided by N = Number who took the test. Multiply the result by 100.298 AFMAN36-2236 12 NOVEMBER 2003

not the only method used in CRT item analysis, it is a very effective method and provides very usable data. 25.3.2.1. The Cox & Vargas DI is found by subtracting the EI of a given test item on the pretest from the EI of the same item on the posttest. The di fference is an indicator of the growth for that item from the pretest to the posttest. Once again, it is critical to keep in mind that the statistics from any test item analysis are only indicators that problems may exist. They do not identify the nature of these possible problems. 25.3.2.2. When using written evaluation instruments, areas of concern include the construction of the item, its validity, how the material is taught, and other factors that influence test taking. However, statistical analysis gives us some strong i ndicators and waves a red flag to get our attention when potential problems are indicat ed in individual test items.
25.3.3. The Ideal CRT Item Situation. The ideal situation for any individual CRT item is much like that for the overall test. When any group of student s arrives at our school, we should test them to determine their need for our instruction. We may fi nd they know nothing about one, several, or all of our objectives. The less they know, as shown by the pretest, the more ideal the situation: the more they need our course. Our selection and screening procedures have worked.25.3.3.1. We discussed objectives as a group and the value judgments involved in setting an overall pretest or posttest st andard. As we look at individual CRT items that measure objectives, however, we need to examine the items one by one. An analysis of group performance on each item in a CRT can be a powerful tool for managing the curriculum. Figure 25.5. provides data on threetest items from a CRT pretest and posttest. In this example, each test item happens to sample different objectives, but the same discussion would apply if all test items measured the same objective.
Figure 25.5. Percent of Correct Responses to Test Items.

25.3.3.2. The data for test item #1 shows that only 20 percent of the students coming into the course knew the material well enough to correctly an swer the item. Whether this percentage is too high or too low depends on the value judgments we make. But it is a reference point to which we can make comparisons. An obvious comparison comes about when we see that only 56 percent of the students correctly answered that same test it em on the posttest. Only 36 percent more of the students were able to pass this CRT item on the posttest than on the pretest (56 percent [post] - 20 percent [pre] = 36 percent [growth]). This growth does not seem high enough when we compare the 56 percent posttest percentage with the ideal 100 percent posttest goal. On the other hand, 56 percent group performance on the posttest may be acce ptable in a short cour se on a less-than-critObjective 1 Objective 2 Objective 3 Test Item #1 Test Item #2 Test Item #3

Pre Post Pre Post Pre Post 20% 56% 67% 71% 12% 89%AFMAN36-2236 12 NOVEMBER 2003 299

ical objective. However, 36 percent growth is ne ither bad nor good by itself. Questions need to be asked about the curriculum, the selection procedur es, or the evaluation program if only 56 percent of our students can respond correctly to a posttest ite m that measures an objective. Is the test item bad? Is the objective poorly taught? Is 56 percen t an acceptable posttest achievement percentageconsidering the possible high level of learning involved? Obviously, the answers to these questions might not be easy.
25.3.3.3. Data for test item #2 of Figure 25.5. shows a high percentage of the group is already able to pass this test item when they reach our course. The 67 percent entry figure detracts from the luster of the 71 percent posttest statistic. Thes e figures may reveal at least two problems. First, a very high entry percentage may raise the question of appropriate content. By itself, the 67 percent is neither good nor bad, but such a high per centage of correct answers in an incoming group might have curriculum implications concerning the need for the course, the level of the curriculum, or the screening of potential students. The sm all 4 percent growth figure from pretest to posttest may tell us more. It is hard to imagine a case involving an Air Force school where such a growth figure would be acceptable. Whether in re lative terms (4 percent growth on this individual test item) or absolute terms (71 percent of the ideal 100 percent), this pos ttest statistic tells us something about our curriculum or the delivery of instruction—and it probably isn't something good.
25.3.3.4. The third test item from Figure 25.5. comes much closer to the pretest and posttest ideal. It appears that the vast majority of our incomi ng class needs this instruction (100 percent - 12 percent correct on pretest = 88 percent potential growth for class). Of that potential 88 percent, we were able to accomplish quite a bit—a 77 percen t class growth. This is probably good growth and is certainly superior to either of the other sets of test item data.
25.4. Group CRT Data as a Management Tool. To the extent that what we want in the way of group performance differs from what we actually get, we have a curriculum problem of some type. We may be able to do a better job managing our curriculum by paying special attention to group or aggregate CRT item data. Take, for example, an attempt to impr ove a curriculum through test item management. The variables that may be different in each of our own situations are the acceptable group percentages for pretest and posttest items, the time to respond to curricul um problems revealed by test item analysis, and the individual curriculum responsibilities in cases of unsat isfactory test item data. This example is typical of what can be done to the management of curricu lums by improving the use of group CRT item data.

25.5. Characteristics of CRT Evaluation. As we report and analyze CRT data, we assume we are using good tests and test items to get data. Good CRTs and testitems must have each of the critical characteristics demanded of all effective evaluation devices (see Chapter 21 ). CRT items must be reliable, valid, objective, comprehensive, and capable of differentiat ing. These characteristics are the same as those we have always demanded from Air Force measurement devices, but the characteristics take on a new and unfamiliar look to instructors who have only worked with norm-referenced testing (see Chapter 26 ).
25.5.1. Reliability. A reliable CRT item or other measure is one that yields consistent results. In other words, will the same person taking the test twice an swer the test items much the same way. Reliability is not generally a severe problem with performan ce rating provided conditions in the evaluation situation remain constant. Reliability can be a problem in paper-and-pencil testing, however, because constructing a good test item is difficult.300 AFMAN36-2236 12 NOVEMBER 2003

25.5.1.1. CRT items with poor reliability are easy to spot if we can recognize the symptoms. If students who are about equal in knowledge or ability have scores that vary widely on a CRT or test item, we may have an unreliable exam. Likewise, if we test the same student twice on the same CRT or test item (within a short period of time), and he or she passes onc e and fails the next time, we may have an unreliable exam. 25.5.1.2. In neither of these cases could we say with certainty that the exam is not reliable, but we should be suspicious. We can use statistics to help us write reliable CRTs.
25.5.2. Validity. For a test to be valid, it must measure exactly what we intended it to measure as defined in our objective. We should ask ourselves the questions at Figure 25.6. to determine the kind of validity we're checking.
Figure 25.6. Three Usable Appr oaches to Determine Validity.

25.5.2.1. Content Validity: 25.5.2.1.1. Content validity is probably the easiest and best way to assess whether or not the CRT measures the objective. We can establish content validity by examining the test and comparing the items to carefully written objectives. No statistical test is used to establish content validity. If subject matter experts feel it measures what the objectives call for, it is valid. If they do not, it is not valid. 25.5.2.1.2. This technique for establishing validit y is practical and useful for Air Force instruction. A test of content validity is a reputable technique and should be rigorously applied. The usefulness of content validity as a tool in a school will vary with the quality of the objectives. The test may validly measure the obj ectives, but valid measurement of worthless objectives yields valid but worthless data for the school.
25.5.2.2. Concurrent Validity. If we already have another good CRT measure of what we want to test, we can determine the degree of associati on between the results of the first CRT and the new one we want to use. To the extent that they are related, we have establishe d their level of concurrent validity. We may wish to determine a level of concurrent validity between our own exam on management and a nationally recognized exam that has been accepted as valid. We may wish to check the validity of a newly developed mu ltiple-choice management exam against a very time-consuming, but valid essay exam now in use. A statistical test is required to establish a level of concurrent validity.
25.5.2.3. Predictive Validity. We can establish predictive validity for our CRT in much the same fashion as we can determine conc urrent validity. When we have two CRT measurements of what we believe to be the same skill or knowledge taken at a considerable length of time from each other, we may wish to determine how well the first CRT predicted success on the second CRT. We may wish to see how our school posttest predicts success on the job as measured by supervisor ratings. Or we may wish to determ ine how well a pencil-and- paper test can be used to predict future Content Validity — Does the CRT measure what the instructional objective specifies? Concurrent Validity— Does one CRT compare favorably with another, already validated CRT? Predictive Validity— Can student scores on one CRT be used to predict success on a second CRT?AFMAN36-2236 12 NOVEMBER 2003 301

success on a performance exam. In these and similar situations, we can use various statistics to establish predictive validity between two CRTs provided they are both scored on a pass-or-fail basis and the tests are separated by a substan tial period of time.
25.5.3. Objectivity. One of the most obvious benefits of CRT is the positive effect upon objectivity in testing. While it is almost impossible to remove all personal bias from measurement, CRT allows us to come closer to that ideal. We can improve the objectivity of a CRT by ensuring that objectives are appropriate and well written. Further, careful atten tion to content validity greatly improves objectivity. To the extent that any element of the objective is ambiguous, the CRT has the potential to lose its objectivity. Instructor bias may creep into any measurement situation. For help with this problem, refer to Chapter 22 , Chapter 23 , and Chapter 24

. 25.5.4. Comprehensiveness. CRT should be comprehensive by its very definition. CRTs measure instructional objectives; therefore, our tests should be comprehensive by the very act of finding out whether we accomplished what we set out to accomplish. Ideally, we should be 100 percent comprehensive in our testing programs. Any objective wo rth stating should be important enough to teach and to measure. There may be some reason to have so me objectives that are not measured, but we should be able to justify each case. If our objectives re present our needed and desired learning outcomes and we test them, we have a built-in mechanism to ensure comprehensiveness.
25.5.5. Differentiation. We can use statistical tests to help us tell whether our test items can tell the difference (differentiate) between persons who ha ve mastered our course and those who have not. Masters of our course include our graduates, those persons exempted from our course because of equivalent training or experience, and those acknowledged experts already working in the field for which we are preparing our students. Nonmasters are essentially any group of students or potential students who do not possess or are not supposed to possess the knowledge and skills we teach. If our test items cannot differentiate those who are graduates of our course from those who are getting ready to take it, we have some sort of problem ( Figure 25.7. ).
Figure 25.7. Possible Problems Causing Lack of Differentiation.

25.6. Summary. To report and analyze CRT data correctly, we need to use techniques different from those we traditionally use. The need to change techniques comes about because CRT is concerned with comparing a student's performance with instructional obj ectives rather than with the level of performance of other students. 25.6.1. An appropriate display of CRT data will assist us as we analyze the results of our instruction. A reporting format that accounts for individual stude nt achievement for each si gnificant instructional objective is a valuable tool. This reporting format may be as simple as a grid listing each student on Poor Selection of Students —Too many students coming to our cour se already possess the skills we intend to teach. Poor Curriculum and Instruction— There is little growth on the part of the students during our course.
Poor Test Items— Something in our test items may be causi ng qualified students to perform poorly while not affecting unqualified persons.302 AFMAN36-2236 12 NOVEMBER 2003

one axis and each objective on the other. Or, it may be so complex as to need computer support to keep track of and analyze instructional variables suchas learning time and attempts at mastery. 25.6.2. Except for extreme examples, the passing or fai ling point in a CRT situation is quite subjective. We must often settle for less than perfect performance on our examinations. Time, money, student characteristics, and other variables often af fect our standards negatively. Although pretest scores of 0 and posttest scores of 100 percent are ideal, we must often accept less than the ideal. The extent to which we exceed the ideal score at entry or fall short of the ideal at exit is a subjective decision. Many complex variables affect that decision.AFMAN36-2236 12 NOVEMBER 2003 303

Chapter 26

NORM-REFERENCED ANALYSIS

Section 26A—Overview

26.1. Introduction. Schools that use evaluation data to compare the performance of one student to the group use some form of norm-referenced analysis. Awards to the commandant's trophy winner, the best writer, distinguished graduates, and similar rankings of outstanding groups or individuals are usually the products of norm-referenced analysis. 26.1.1. Norm-referenced analysis is a classification or label applied to the use of evaluation data for comparing students to each other rather than to a specific instructional objective. Many different statistical techniques can be used to evaluate students' scores on written tests and performance rating instruments. In some schools, especially in profe ssional military education programs, there has been a tradition of identifying and rewarding the best pe rformers in each class by comparing students on many different tasks. Norm-referenced analysis is characterized by using data to compare students to the "norm" or average. Any or all of the following items may be used for the purpose of comparing students: written test scores, grades on writing and speaking assignments, grades in drill and ceremonies, grades in physical training, and numericalestimates of various leadership traits.26.1.2. Consider the following hypothetical case to help distinguish between criterion- or objective-referenced analysis (see Chapter 25 ) and norm-referenced analysis. Suppose we have 50 students in a 1-week course and we use only one evaluation instrument—a final written test of 100 questions. If we set minimum passing at 70 percent of the objectives being tested and compare student scores to the 70 percent standard, we use a form of criterion-referenced measurement. If we are more interested in rank ordering our students than holding them to a specific cutoff score for passing, the process would be called norm-refere nced measurement. If we set a minimum standard for passing and we also desire to rank order students for the pur pose of recognizing the students who performed better than others, then we are using both criterion-refe renced measurement and norm-referenced analysis. 26.1.3. This chapter presents several statistical measurements and procedures commonly used to compare students with each other. The advent of inexpensive calculators with many functions has decreased the need to be thoroughly familiar with th e computations for most statistical measures.However, understanding how to compute various functions helps to reinforce the meaning of statistical concepts.
Section 26B—Grading Methods

26.2. Overview. This section is concerned with several gr ading methods. The basic data used throughoutthe examples are the final school grades made by 150 students in a hypothetical Air Force school. Whatever grading method is used, a frequency distribution of scores should be established first. The scores may be the result of a single administr ation of the test or they may have been accumulated over a period of time. 26.2.1. Construct the frequency distribution of the scores made by the 150 students ( Figure 26.1. ) as follows:304 AFMAN36-2236 12 NOVEMBER 2003

Figure 26.1. Tabulation of a Frequency Distribution.

26.2.1.1. Step 1. In column 1, list the scores obtained by the group from the highest to lowest.
26.2.1.2. Step 2. In column 2, enter tally marks for each score.
26.2.1.3. Step 3. In column 3, record the number of tallymarks for each of the scores. Because this column tells how frequently each score occurred, it is known as the "frequency (f) column." The total of this column should be the total numbe r of persons in the group. This frequency distribution (columns 1 and 3) is used to compute grades.

26.3. Rank-Order Grades. Rank-order grades are best computed from the basic frequency distribution shown in Figure 26.1. Figure 26.2. shows the rank order that would be assigned to all 150 students. Two students tied for the highest rank in the class. If evaluation instruments had the capability of making finer differentiations between the two students, one would have ranked first and the other second. Because it is not possible to differentiate between them on the basi s of available information, they should both receive the same rank-order grade.AFMAN36-2236 12 NOVEMBER 2003 305

Figure 26.2. Computation of Rank-Order Grades.

26.3.1. In cases of a tie, it is customary to assi gn all members the median rank of the tying group. Therefore, in Figure 26.1. the highest two would be assigned the rank-order grade of 1.5/150. Seven students tied for the ranks three through nine; theref ore, each would receive the rank-order grade of 6/ 150. 26.3.2. To find a rank-order grade of tying students, the instructor should note the lowest and highest ranks of those in the tying group and average these two ranks. You may use the following formula to compute rank-order grades: A B C

School Score Number of Students (f) Rank-Order Grade

47 2 1.5/150 46 7 6/150 45 8 13.5/150 44 8 21.5/150 43 12 31.5/150 42 11 43/150 41 15 56/150 40 12 69.5/150 39 8 79.5/150 38 12 89.5/150 37 12 101.5/150 36 10 112.5/150 35 7 121/150 34 1 125/150 33 7 129/150 32 6 135.5/150 31 4 140.5/150 30 2 143.5/150 29 4 146.5/150 28 1 149/150 27 1 150/150306 AFMAN36-2236 12 NOVEMBER 2003

26.3.3. For example, 25 students received scores higher than 43 and 12 students received 43. The highest person in the group receiving 43 would ther efore have a rank of 26; the lowest, 37. The rank-order grade for each of these 12 students is computed as follows:

NOTE: Here, T is the rank of the student at the top of the tying group; B, the rank of the student at the bottom of the tying group; and N, the total number in the class. 26.3.4. The formula may be used to obtain the rank-order grade for the 8 students with the score of 44:

NOTE: We would say that the rank order of 8 students with a score of 44 is 21.5 out of 150 students.

26.4. Percentile-Rank Grades. The percentile rank for a specific score is defined as the percentage of persons in the reference group who obtained scores equa l to or less than the given score. Thus, if a raw score of 50 equals a percentile ra nk of 87, called the 87th percentile, then 87 percent of the people in the reference group obtained scor es equal to or less than 50. A percentile rank of five indicates that only 5 percent of the group obtained equal or lower scores. A percentile rank involves percentages of people and indicates a student's relative standing in percentage terms.
26.4.1. Scores on the Airman Classification Test and the Air Force Officer Qualifying Test are reported in percentiles. By contrast , percentage scores express performance in te rms of percent of content mastered. A percentage score of 85 means that a student answered 85 percent of a test correctly. 26.4.2. Percentile rank grades are computed from a frequency distribution in much the same way as are rank-order grades. The main difference is that percentile-rank grades are computed from the lower end of the scale. 26.4.3. Probably the easiest method of converting a fr equency distribution into percentile-rank grades is to use the basic distribution of student scores as follows:
26.4.3.1. Step 1. Construct a cumulative frequency column ( Table 26.1. , column C) which shows the number of students below the score given. For example, no one scored below 27. Therefore, the entry in column C for 27 is 0. One student had a score lower than 28; the entry in column C forAFMAN36-2236 12 NOVEMBER 2003 307

28 is 1. Two students scored below 29; the entry in column C for 29 is 2. This process is continued to the top score. Obviously, the top entry in column C is 148. The cumulative frequency for the top percentile rank can be checked against the total number in the distribution.
26.4.3.2. Step 2. Construct a column ( Table 26.1. , column D) that shows the number of students below a particular score, plus one-half the number of students obtaining that score. Entries for column D may be obtained by adding 1/2 the number in column B to the entry in column C. For example, in Table 26.1. the entry in line 12, column D, for the school score of 36 is 38 because 10 (1/2) + 33 = 38. For the score of 41, the entry at line 7, column D, is 94.5 because 15 (1/2) + 87 = 94.5.
Table 26.1. Computation of Percentile-Rank Grade.

26.4.3.3. Step 3. It is now a simple matter to compute the percent of the total group that lies below the middle of each score. To do this, divide each entry in column D by the total number of L I N E A B C D E

School Score Number of Students (f) Number of Students Below Each Score (Column 3 Plus Half the Number in

Column B) Percentile-Rank Grade 1 47 2 148 149.0 99 2 46 7 141 144.5 96 3 45 8 133 137.0 91 4 44 8 125 129.0 86 5 43 12 113 119.0 79 6 42 11 102 107.5 72 7 41 15 87 94.5 63 8 40 12 75 81.0 54 9 39 8 67 71.0 47 10 38 12 55 61.0 41 11 37 12 43 49.0 33 12 36 10 33 38.0 25 13 35 7 26 29.5 20 14 34 1 25 25.5 17 15 33 7 18 21.5 14 16 32 6 12 15.0 10 17 31 4 8 10.0 7 18 30 2 6 7.0 5 19 29 4 2 4.0 3 20 28 1 1 1.5 1 21 27 1 0 0.5 0308 AFMAN36-2236 12 NOVEMBER 2003

students, and then multiply by 100. For example, the l0 students with a score of 36 receive a percentile-rank grade of 25; they are at the 25th percentile, obtained as follows:

26.4.3.4. Step 4. These arithmetical procedures can be e xpressed by the formula presented below. In this formula, Y is the number of students be low a particular score; f, the number of students receiving that same particular sc ore; and N, the total number of st udents in the class. Usually the decimal portions of percentile-rank grades are not considered important; consequently, percentile rank is usually reported to the nearest whole percent.

26.5. Percentile-Group Grades. Many methods of percentile-group grading are used. Instructors may invent one to suit their needs. However, all students in a group with identical scores should receive the same grade. Whatever groups are used, some of the di viding lines are almost sure to fall where they will separate the members of a group who have the same score. If this occurs, you must decide on which side of the group to place the line. A basic distribution of scores may be grouped in several ways ( Table 26.2. ).AFMAN36-2236 12 NOVEMBER 2003 309

Table 26.2. Computation of Several Percentile -Group Grades with Different Percentage Groupings.

26.5.1. For the method in column C, the class is divided into four groups: the top 20 percent of the class (superior); the next 30 percent (above average) ; the next 40 percent (satisfactory); and the bottom 10 percent (unsatisfactory). Twenty percent of 150 (the number of students who took the test) is 30; therefore, a count from the top of the distri bution should include 30 students. However, the 30th student from the top is among the 12 students who received a score of 43. If we place the dividing line just above this group, only 25 students are within the top 20 percent. If we place the dividing line so as to include the whole group with a score of 43, then 37 students would be within the top 20 percent. Since 25 is closer to 30 than 37 is, we placed the dividing line between the scores 43 and 44. The position of this dividing line will vary with instructors; for the individual instructor, it is likely to vary according to mood and general impressions. Subjectiv ity is involved even more in this method inidentifying the bottom l0 percent. Ten percent of 150 is 15, so a count from the bottom of the distribution should include 15 persons. This dividing line falls at the middle of the group that received a score of 32, making an arbitrary decision necessary. A B C D E F L Percentage Groupings I 10%, 20%, N School Score Number of 20%, 30%, 40%, 20%, E (Raw Score) Students (f) 40%, and 10% and 10% Tenths Quarters 1 47 2 Top 20% Top 10% 1 1 2 46 7 (Superior) (A) 3 45 8 4 44 8 Next 20% 2 5 43 12 Next 30% (B) 3 6 42 11 (Above 2 7 41 15 Average) Next 40% 4 8 40 12 (C) 5 9 39 8 Next 40% 6 3 10 38 12 (Satisfactory) 11 37 12 7 12 36 10 Next 20% 8 13 35 7 (D) 9 4 14 34 1 15 33 7 16 32 6 10 17 31 4 Bottom 10% Bottom 10% 18 30 2 (Unsatisfactory) (F) 19 29 4 20 28 1 21 27 1310 AFMAN36-2236 12 NOVEMBER 2003

26.5.2. The method in column D is similar to th e method in column C with these exceptions: five groups are used, the percentages are intended to be symmetrical, and letters instead of adjectives are used to identify the groups. In the method at column E, the class is divided in to l0 approximately equal groups. Column F divides the class into quarters.

26.6. Arithmetic Mean. The arithmetic mean of a set of scores can be found by totaling them and dividing that sum by the number of scores. For example, the mean of 103, 101, 98, and 98 is 100, computed as follows:

NOTE: We can further illustrate the method of computi ng the mean by considering grades obtained from the class of 150 students ( Table 26.3. ). The numbers in column B tell how many students obtained each raw score. The mean is computed using the following steps:
26.6.1. Step 1. Arrange the raw scores of the test (referred to as X) in numerical order from highest to lowest without repeating identical scores.
26.6.2. Step 2. In column B, list the number of students who made that score (referred to as frequency or f). Total this column to determine the number of students in the class, referred to as N.
26.6.3. Step 3. Multiply each score by the number of students who made that score and place that number in column C, fX. The table shows that two pe rsons obtained a score of 47; therefore, this score should appear twice in the list. To save space, how ever, we can enter 94 (47 multiplied by 2) in line 1, column C. The next entry is 46 times 7, or 322. The column is completed in this manner.
26.6.4. Step 4. Find the sum of column C. This is the sum of all 150 scores obtained on the test. In the example, the sum is 5,830.
26.6.5. Step 5. Divide the sum of scores (5,830) by the number of scores (150) to obtain the mean (see the following formula). The result is usually reported to the nearest tenth.

Here, M is the mean; the Greek letter sigma, the "summation of" the frequency (f); X, a test score; and N, the number of scores.

26.7. Standard Deviation. The same frequency distribution of grades ( Table 26.3. ) is used here to show how to find the standard deviation of any distribution. The following method is used:
26.7.1. Step 1. Determine how much each test score deviat es from the mean (column D) by subtracting the mean from the test score. For example, 47 - 38.9 = 8.1, the deviation of the highest score; andAFMAN36-2236 12 NOVEMBER 2003 311

27 - 38.9 = -11.9, the deviation of the lowest score. Note that the deviation of all scores below the mean are negative.
26.7.2. Step 2. Square the deviation, referred to as d 2 (see Table 26.3. , column E). If 8.1 (the deviation of the test score 47) is multiplied by itself (that is, squared), the product is 65.6. The column is completed in this manner.
26.7.3. Step 3. For a particular score, multiply the freque ncy of the score by the deviation squared, referred to as fd 2 (see Table 26.3. , column F). Just as the test score is multiplied by the frequency to determine the mean, the square of the deviation is multiplied by the frequency to determine the standard deviation.
Table 26.3. Computation of Mean, Standard Deviation, Standard Scores, and T-Scores.

N

OTES: 1. Sum of this column is 150. A B C D E F G H L I Test Frequency Standard N Score (f) fX Deviation fd 2 Score
E (X) (note 1) (note 2) (d) d 2 (note 3) (d/s) T-Score 1 47 2 94 8.1 65.6 131.2 1.76 68 2 46 7 322 7.1 50.4 352.8 1.54 65 3 45 8 360 6.1 37.2 297.6 1.32 63 4 44 8 352 5.1 26.0 208.0 1.10 61 5 43 12 516 4.1 16.8 201.6 .89 59 6 42 11 462 3.1 9.6 105.6 .67 57 7 41 15 615 2.1 4.4 66.0 .45 55 8 40 12 480 1.1 1.2 14.4 .23 52 9 39 8 312 .1 .0 .0 .02 50 10 38 12 456 -.9 .8 9.6 -.19 48 11 37 12 444 -1.9 3.6 43.2 -.41 46 12 36 10 360 -2.9 8.4 84.0 -.63 44 13 35 7 245 -3.9 15.2 106.4 -.85 42 14 34 1 34 -4.9 24.0 24.0 -1.07 39 15 33 7 231 -5.9 24.8 243.6 -1.28 37 16 32 6 192 -6.9 47.6 285.6 -1.50 35 17 31 4 124 -7.9 62.4 249.6 -1.72 33 18 30 2 60 -8.9 79.2 158.4 -1.93 31 19 29 4 116 -9.9 98.0 392.0 -2.15 29 20 28 1 28 -10.9 118.8 118.8 -2.37 26 21 27 1 27 -11.9 141.6 141.6 -2.59 24312 AFMAN36-2236 12 NOVEMBER 2003

2. Sum of this column is 5,830. 3. Sum of this column is 3,234.
26.7.4. Step 4. "Standard deviation" may be defined as the square root of the average of the squares of the deviations from the mean of the distribution. Compute the standard deviation of this distribution by using the following formula where s is the standa rd deviation; f, the frequency; d, the deviation of a score from the mean; and N, the number of scores.

26.8. Standard Score. Now it is a simple matter to find the standard score corresponding to any given raw test score. Standard sc ores (SS) are defined as scores expresse d in terms of their standard deviation distances from the mean. This definition may be expressed in terms of the following formula where X is the test score, M is the mean, and s is the standard deviation.

26.8.1. The basic formula can be reduced to d/s because d is the amount each raw score deviates from the mean and is the same as X - M. 26.8.2. To calculate the standard score ( Table 26.3. , column G), divide the amount each score deviates from the mean (column D) by the standard devi ation. For example, the test score 47 expressed as a standard score may be found as follows:

26.9. T-Scores. Standard scores can be converted to T-scor es, which have a mean of 50 and a standard deviation of 10. ( NOTE: T-scores are usually reported to the nearest whole number.) This conversion is expressed in the following formula:AFMAN36-2236 12 NOVEMBER 2003 313

26.9.1. For example, the test score 47 expressed as a T-score would be found in the following manner—

26.9.2. In actual practice, the labor of computing standard deviation, standard, and T-scores is reduced considerably by using a calculator or com puter software. Instructors will find many shortcuts as they become more familiar with the process.
26.10. Grading Methods Compared. To compare grading methods, suppose two students in a class of 90 received the scores shown in Figure 26.3. for five subjects. The raw scores tell nothing about the real proficiency of either student in the five subjects. Itis only possible to draw conclusions about their comparative standings. Inspection reveals that Frank Brown excels John Doe in three of the five subjects, but there is no way of knowing the quality of the differen ce. If the scores of the two students are averaged, John Doe has the higher average. It might, therefore, be inferred that John Doe is the better student, assuming the five subjects are equally important.
Figure 26.3. Raw Scores Made by Two Students.

26.10.1. In Figure 26.4. , Brown's and Doe's rank-order grades are shown. These rank-order grades give more information than the raw scores alone. Differences in rank order are not proportional to differences in ability; however, there is a tendency to infer such a relationship. Furthermore, although Frank Brown ranked above John Doe in writing, such fine discriminations are not warranted near the center of so large a group. Students Subject Frank Brown John Doe

Practice teaching 156 186 Curriculum construction 16 6 Special laboratories 30 25 Evaluation 11 2 Writing 129 130

Average Score68 70314 AFMAN36-2236 12 NOVEMBER 2003

Figure 26.4. Rank-Order Scores Made by Same Two Students.

26.10.2. The scores of Brown and Doe were convert ed to the percentile-rank grades shown in Figure 26.5. In general, the situation is the same as with the rank-order grades. Again, the apparent difference in writing abilities of two students is not genuine.
Figure 26.5. Percentile-Rank Scores Made by Same Two Students.

26.10.3. The mean and standard deviation of the distri bution of the 90 scores in each of the 5 subjects were computed and are recorded in Figure 26.6. When these values were compared with the scores of Brown and Doe, their T-score grades for each of the five subjects were obtained. If you keep in mind the normal probability curve and the scales shown in Figure 26.7. , you can obtain a clear picture of the proficiency of each student re lative to that of other students: Students Subject Frank Brown John Doe

Practice teaching 45/90 9/90 Curriculum construction 11/90 89/90 Special laboratories 47/90 79/90 Evaluation 6/90 89/90 Writing 43/90 42/90

Average Score32/90 61/90

Subject Frank Brown John Doe

Practice teaching 50 90 Curriculum construction 88 4 Special laboratories 48 12 Evaluation 82 1 Writing 52 53

Average Score 64 32AFMAN36-2236 12 NOVEMBER 2003 315

Figure 26.6. T-Score Grades Made by Same Two Students.

T-Score Subject Mean Score Standard Deviation Frank Brown John Doe

Practice Teaching 56 20 50 65 Curriculum construction 2 4 60 35 Special laboratories 30 5 50 40 Evaluation 8 3 60 30 Writing 130 36 50 50

Average Score— — 54 44316 AFMAN36-2236 12 NOVEMBER 2003

Figure 26.7. Comparison of Grading Methods.

26.10.4. Frank Brown is average, or well above average, in all five subjects. John Doe, on the other hand, is above average in only one subject, and clearly below average in three subjects. Furthermore, Brown's mean T-score is well above the average of the total group, whereas Doe's mean T-score is clearly below average. One of the advantages of gr ades based on standard scores is that they may be averaged fairly. Averages based on other grading methods are open to question. For example, Doe had a higher course average based on original raw scores, yet he is clearly a poorer student. 26.10.5. When grades in several subjects are to be weighted and then expressed as a single composite grade, the grades should be based on standard scor es. By doing this, the instructor can be sure each subject has been truly weighted according to the wei ghting factor. For example, if scores in each subject are first converted to T-scores and each set of T-score grades is multiplied by the weighting factor, then each student's sum may be used as a measure of his or her relative position in the class. This approach to weighting subjects is shown in Figure 26.8. for Brown and Doe.AFMAN36-2236 12 NOVEMBER 2003 317

Figure 26.8. Weighted Scores Made by Same Two Students.

NOTE: T-scores may be weighted, but the original raw scores and accumulated points should not be weighted; weighting original raw scor es may only make weightings less re liable, as in the case of Brown and Doe.26.10.6. Most of the grading methods used to compare students with each other are based on the characteristics of the bell-shaped normal curve. The pr operties of the normal curve never vary; therefore, it is used widely in normative analysis of testing data. Figure 26.7. shows the normal curve and a comparison of the measurement units used with va rious normative grading methods. You can find an indepth discussion of the properties and use of th e normal curve in any textbook on tests and measurement.
Section 26C—Effectiveness of Individual Test Items

26.11. Ease Index of Test Items. The ease index (EI) of a test or a particular test item is a measure of how easy that test or item was for the group taking th e test. The ease of an item is also a measure of its difficulty, so it is sometimes called the difficulty in dex. The distribution of norm-referenced test scores can be adjusted by changing the ease of items or by eliminating some items and selecting others on the basis of their difficulty. If the difficulty level of item s has been adjusted properly, the average score of the group should fall between 50 and 60 percent of th e maximum possible score and scores should cover nearly the whole possible range—from the guessing level to almost perfect. This situation is ideal in norm referencing.
26.12. Simple Formula for Ease Index. The simplest way to determine the EI of an item is to divide the number of persons who answered the item correctly bythe total number of persons who took the test. Multiply this quotient by 100 to convert the result to a percentage figure. The following formula is used to compute the simple EI of an item, where R is the number responding correctly and N is the number taking the test: Frank Brown John Doe Weight Factor Weighted Weighted (percent) T-Score Score T-Score Score

Practice teaching 35 50 18.50 65 22.75 Curriculum construction 10 60 6.00 35 3.50 Special laboratories 10 50 5.00 40 4.00 Evaluation 25 60 15.00 30 7.50 Writing 20 50 10.00 50 10.00

Total 54.50 47.75318 AFMAN36-2236 12 NOVEMBER 2003

26.12.1. If everyone answered the item correctly, the numerator and denominator of the fraction would be the same, and the EI would be 100 percen t. If no one in the group answered the item correctly, the numerator would be 0, and the EI would also be 0. Consequently, the range of the EI is from 1 to 100 percent. The higher the percentage, the easier the item; that is, an item with an EI of 80 percent is easier than an item with an EI of 35 percent. 26.12.2. The responses of 150 airmen to a question on a management test are shown in Figure 26.9. On the basis of 47 correct answers, the EI for this item would be computed as follows:

Figure 26.9. Distribution of Responses to Test Item.

26.13. Interpreting the Ease Index: 26.13.1. The EI is only a statistical indicator. It should not be used by itself to judge the worth of an item; instead, it should be used with other data. All statistical data (including the EI) used in comparisons must be based on equivalent groups. For example, an item having an EI of 85 percent with an advanced group of students might well have an EI of 45 to 60 percent with a group of basic students. Response (For One Item)

Groups According Incorrect to Total Test Score or Omitted Correct Number of Students

High third (above 65) 22 28 50 Mid third (61-65) 38 12 50 Low third (35-60) 43 7 50

Total103 47 150AFMAN36-2236 12 NOVEMBER 2003 319

26.13.2. An effective norm-referenced test should in clude items representing a considerable range of difficulty, but most of the items should be of about average difficulty (EI of 50 to 60 percent). If instructors have a pool of test items, they can select items to cover the range of abilities in the class in such a way that the distribution of test scores fills the scale and approximates a normal curve.

26.14. Differentiation Index (DI) of Test Items. When an item is answered correctly by some students and not others, it divides the class into two groups. This question arises: In what other way are the two groups different? A really effective instructor want s to know not only those students who answer an item correctly, but also the kind of student who has the ability to answer it correctly. 26.14.1. One means of analyzing an item is to determine the total test standing of the students who answer the item correctly. If the students who answer the item correctly are predominantly those with high total test scores, the item is adding to the effe ctiveness of the test as a whole. It is contributing positively toward the differentiating ability of the test. 26.14.2. If students who answer the item correctly ar e about equally divided between those with high total test scores and those with low scores, the item contributes little or nothing to the differentiation of the test as a whole. In fact, it may detract from the efficiency of the test by unfairly adding to the scores of some students and subtra cting from others. Do not overlook the possibility that the quality of instruction—rather than a poor test item—c ould be the reason for little differentiation. 26.14.3. In general, the group that responds correctly to any item should be composed principally of students who have high total test scores. This score grouping is particularly necessary if the test is designed to cover closely related material. In this situation, the test should have high consistency within itself. An item that can be used to separate students who achieve high test scores from those who do not is said to have differentiation ability. For purposes of analysis and comparison, this ability can be represented by an index number.
26.15. Computing Differentiation Index. The DI measures the ability of an item to differentiate between students with high scores on the total test and those with low scores. There are a number of formulas for computing this index. The formula selected mustbe the one that best fits the needs of the user. (Some testing authorities use the term discrimination index or validity index when discussing this type of test analysis.) 26.15.1. One method of computing the DI is to divide the test papers into groups based upon the overall test scores. For example, if three groups are used to compute the DI of 150 students, the best 50 test papers are placed in the high group, the next 50 pa pers in the middle group, and the last 50 papers in the low group. The number of students in each group who responded correctly to the item are then counted. For the management test ( Figure 26.9. ), 50 airmen were in each third of the class. In the high third, 28 airmen answered the item correctly; in the middle third, 12; in the bottom third, only 7. 26.15.2. Subtract the number of persons in the low third who answered the item correctly from the number of persons in the high third who also answered it correctly. This difference is then divided by one-third the total number of students in the gr oup, which represents the total number of students divided by the number of groups used. In this partic ular case, three groups were used; therefore, N is divided by three. The formula for computing the DI when three groups are used is as follows, where H is the number of students in the high third who an swered the item correctly, L is the number of students in the low third who also answered the item correctly, N is the number of students who took the test, and 3 is the number of groups:320 AFMAN36-2236 12 NOVEMBER 2003

NOTE: The index of 0.42 indicates the degree to which the item differentiates between those students who obtained high total test scores and t hose who obtained low total test scores. 26.15.3. Note that the symbols in the formula for the DI duplicate those of the formula for the validity index discussed earlier. However, the indexes represent different things; the difference lies in the criteria used to obtain either index.
26.16. Interpreting the Differentiation Index. The theoretical range of the DI is from -1 to +1. If all students in the high group answered the item correctly and no one in the low group got the item correct, a +1 DI would result. If all students in the low group got the item correct, when none in the high group did, a -1 would result. If the same number of students inboth the high and low groups responded correctly, the DI would be 0. 26.16.1. As can be seen, when the test scores indicate that more students in the upper third of a class answer an item correctly as opposed to those in the lower third, the item is said to have a positive (or plus) differentiation. On the other hand, if more students in the lower third get the item correct, it is said to have negative differentiation. 26.16.2. Any item may be neutral or have zero diff erentiation because of an equal distribution of scores. If this occurs with very easy or very difficult items, it may be acceptable. However, items with very low positive differentiation, zero differentiatio n, or negative differentiation should be qualitatively examined for validity. If neutral or negative differentiation occurs with items of medium difficulty, you need to determine the reason and correct it before the item is reused.
26.17. Analyzing All Responses To a Test Item. In addition to the inform ation available from the EI and DI calculations, another useful source for test item analysis is to look at all the responses to a test item. This can be accomplished by portraying response information in the same general format as is done for the DI, except attention is directed to all res ponses—not just those made to the correct answer. Quite often the way students respond incorrectly can provide valuable information on the effectiveness of a test item.
26.17.1. Figure 26.10. lists the responses to a four alternat ive multiple-choice item and illustrates this type of analysis. This method can be adapted to othe r types of test items. When the distribution of the 150 responses to this question are analyzed, the following aspects are worth noting:AFMAN36-2236 12 NOVEMBER 2003 321

Figure 26.10. Analysis of All Responses To a Test Item.

26.17.1.1. The item has a substantial DI (0.42), as has been shown. The distribution shows a definite relationship between the total test scor e and correct responses (alternative C). Nearly two-thirds of the airmen who selected the correc t response were from the high third of the class in terms of total score. More than half of the airm en in the high third of the group selected the correct response. 26.17.1.2. None of the students chose alternative A. Since the item has esse ntially three alternatives, alternative A should be reworded, removed, or replaced. This alternative should be constructed to appeal to some of the 90 airmen who either answered the item incorrectly or deliberately omitted it.
26.17.1.3. Half of the airmen who selected altern ative B came from the high third of the group. This indicates that these airmen may have been given some misinformation or they have been misled by the wording of the alternative. Only the correct response should be positively related to the total test score. This alternative should be rewritten.
26.17.1.4. Alternative D has been selected most ofte n by airmen in the low third. It differentiates in terms of the total test and is, therefore, a good distracter. 26.17.2. Data furnished by tabulation, such as in Figure 26.10. , is useful to the test constructor and interpreter. If the item is valid, the data may be interpreted as follows: 26.17.2.1. Students who selected the correct response to the item can be assumed to have achieved that objective. Groups According to Items Number Total Test Score Alternatives Items Not of A B C* D Omitted Reached Students

High third 95 65 0 16 28 6 0 0 50 Mid third 64 61 0 10 12 10 10 8 50 Low third 60 36 0 9 7 12 17 5 50

TOTAL 0 35 47 28 27 15 150

* Correct response322 AFMAN36-2236 12 NOVEMBER 2003

26.17.2.2. Students who selected incorrect alternativ es or who omitted the item can be assumed to not have achieved that objective.
26.17.2.3. Alternative A is ineffective and must be replaced, further investigation is required regarding alternative B, and alternat ive C is apparently good as it is. 26.17.2.4. Students who reached an item, but omitted it, did so by choice. 26.17.2.5. Subsequent items were omitted because the students failed to reach them.
26.18. Summary. Instructors should use norm-referenced analys is when there is a need to compare students with each other. We may want to provide recognition and awards to students who perform very well. Also, by using norm-referenced analysis of evaluation data, we can identify those students who are not doing well in our classes when their performance is compared with the performance of other students.
26.18.1. Statistical measures used in norm-referencedanalysis include rank ordering, percentile ranking, percentile grouping, standard scores, and T-sc ores. Also, the mean and standard deviation are used to calculate many statistical concepts. 26.18.2. Several important phases are considered in evaluating the worth of a norm-referenced test. When a test is working effectively the scores re semble a normal curve of distribution, having a wide range with the majority of the scores grouped near the center of the distribution. It is useful to analyze individual test items in terms of difficulty level and ability to differentiate. Through proper interpretation and use of this information from each item, the instructor can analyze each test to see if each test item is achieving its purpose as planned.AFMAN36-2236 12 NOVEMBER 2003 323

Chapter 27

USING FEEDBACK IN THE CLASSROOM

27.1. Introduction:
27.1.1. We are familiar with courses that consist of readings, lectures, an assigned paper or project, and tests. In traditional courses like these, indi vidualized comments from instructors to students are limited to grades on papers, quizzes, exams, and the final grade. But comments of this sort come well after students have been evaluated on their course work. If there is any impact on learning, it will come during the next phase or in another course or in some follow-on activity on the job. 27.1.2. Such after-the-fact comments, called summat ive evaluation, often contribute little to student learning because they come too late for the student to take corrective action. On the other hand, the most important job of an instructor may be to pr ovide information which students can use to improve themselves throughout the course. Such information, called formative evaluation, guides students while they can still take corrective action and indicates to instructors if a change is needed in the teaching approach.
27.2. Definition and Theory. In order to differentiate between end-of-course grading and information an instructor may provide during a course, many educators use the word feedback. In general, feedback is any information about the results of a process. When we use a computer, for instance, we feedin information and get back responses. In the social sciences, feedback is information that returns to the source of a process (or to a preceding stage) so as to reinforce or modify it. For instance, if the coach finds that the football team is weak in defensive tactics, players are scheduled for more blocking practice. In psychological jargon, feedback is called knowledge of results. 27.2.1. In the classroom, then, feedback can be defined as information students receive from their instructor about their performance that will cause them to take corrective action and guide them in attaining the goals of the course more effectively. 27.2.2. This notion of feedback or knowledge of resu lts is actually quite complex. Textbooks use such terms as primary versus secondary feedback, extrin sic versus intrinsic, augmented versus summary, and specific versus general. Students receive feedback from at least five sources—self, the learning task, fellow students, the instructor, and the school . We do not understand completely how the process works in each instance, but the broad outline is clear. 27.2.3. Feedback generally serves one or two purpos es—informational and motivational. The first is generally responsible for correcting student errors. The second encourages the student to try harder. Informational feedback should be motivating, but motivational feedback doe s not necessarily provide information. A pat on the back or a word of enc ouragement may motivate a student, but will not necessarily point out errors in that student's performance. 27.2.4. Feedback seems most effective when narrative comments are accompanied by numerical scores of ratings, especially when students clearlyunderstand the scales or numbers used. Narrative summaries have a higher chance of being biased than scores or ratings, and if the narrative is too specific or goes too far the feedback may actually be counterproductive. 27.2.5. Feedback need not always be negative or de structive. In fact, positive feedback is normally seen as warmer and more sincere than negative fee dback, even when given in identical ways. Be care-324 AFMAN36-2236 12 NOVEMBER 2003

ful, though, when giving feedback to students becaus e they may appear to tolerate criticism, but perceive instructors' attempts to point out weak areas as threatening. This perceived threat often causes defensive behavior on the part of the student, which may result in a lack of improvement. 27.2.6. It might appear that the best advice is "to accentuate the positive," or to discuss positive feedback and ignore the negative. This technique, however, is not the answer. Students reinforced only for success tend to lower their goals in order to ensure success. Criticism may hurt, but praise may not always help. 27.2.7. We cannot reconcile these conflicting statements about feedback more than to say that positive information is more readily received than negative. Negative information generally tends to arouse defensive behavior on the part of the student. Yet, teachers must stress positive aspects of student performance and, at the same time, adequately communicate weaknesses and needed improvement. Emphasizing the positive should enhance student reception or acceptance and hopefully foster some attempts to apply the feedback in the future.
27.3. Giving Feedback to Students. The purpose of feedback is to improve student performance. In its most effective form, it provides constructive advice, di rection, and guidance to students in their efforts to raise their performance levels. (See Attachment 11 and Attachment 12 for examples.)
27.3.1. Feedback is a communication medium in the sens e that the instructor can review course standards with the students and provide feedback on their performance in relation to these standards. Students must understand the purpose and role of feedback in the learning process. Otherwise, they may reject the feedback and make little or no effort to improve.
27.3.2. Feedback can also be used as a device to reinforce learning. Although not all feedback can be used this way, the instructor should take every oppor tunity to use feedback as a means of clarifying, emphasizing, or reinforcing instruction. For example, if several students falter when they reach the same step in lesson planning, student feedback may indi cate a need to repeat an explanation or to give special emphasis to the step in subsequent critiques.

27.4. Characteristics of Effective Feedback:
27.4.1. Objectivity:
27.4.1.1. Effective feedback focuses on the student and student performance; it should not reflect the instructor's personal opinions, likes, dislikes, or biases. For example, if a student demonstrates a complicated test construction se quence, it would be unfair to give feedback on personality traitsunless they interfere with performance. If a student makes a speech and expresses views that conflict with the instructor's beliefs, the instructor should give feedback on the merits of the speech, not on the basis of agreement or disa greement with the student's views.27.4.1.2. The "error of halo" is a pitfall in giving feedback, much the same as it is a pitfall in summative evaluation. Instructors sometimes permit fa vorable or unfavorable impressions of students to influence their judgment. Sympathy or over-ide ntification with a student can be a barrier to objectivity. A conflict of persona lities can unwittingly color a judgment. Feedback must be honest to be objective; it must be based on actual perfo rmance—not performance as it could have been or as the instructor and the student wish it had been.
27.4.2. Acceptability. Students must first accept the instructor before they can willingly accept feedback. They must develop confidence in the instru ctor's qualifications, teaching ability, sincerity, com-AFMAN36-2236 12 NOVEMBER 2003 325

petence, and authority. The instructor usually has the opportunity to establish rapport and mutualrespect with students before feedback is needed. If there is no such opportunity, however, the instructor's manner, attitude, and knowledge of the subject must serve instead. Students usually accept feedback when instructors present it with conviction and sincerity. The competent instructor does not rely on rank or position on the faculty as a basis for feedback. Although these are important elements of an instructor's authority, student acceptance depends on more active and demonstratable qualities.
27.4.3. Constructiveness: 27.4.3.1. Feedback is pointless unless a student profits from it. Praise just for the sake of praise has no value unless the only goal is to motivate or improve self-concept. The instructor should identify a fault or a weakness and also provide positive guidance for the student to improve. Negative criticism that does not point toward improve ment or a higher level of performance should be omitted from feedback.
27.4.3.2. Effective feedback reflects the instructor's consideration of the student's need for self-esteem, recognition, confiden ce, and the approval of others. Ri dicule, anger, or fun at the expense of the student have no pl ace in constructive feedback. The instructor should occasionally offer comments to the student in private. In so me cases, discretion may rule out any feedback at all. For example, feedback may not help a studentimpaired by a physical defect that the student cannot change. Although instructors need to be straightforward and honest, they must also respect the student's personal feelings.
27.4.4. Flexibility. As effective instructors, we should remainflexible in giving feedback. We should avoid mechanical, predetermined techniques and pr econceived opinions regarding content, subject matter, and student capability. We should consider the actual content of a student effort or what actually happens during a student activity before givi ng feedback on those observed factors that affected performance. Sometimes a good st udent will do poorly and a poor stud ent will do well. A friendly student may suddenly become hostile or a hostile student may suddenly become friendly and cooperative. We should be honest enough to evaluate each effort on its own merits. 27.4.4.1. We should also be sensitive to student reaction to feedback. If a student does not respond to a given comment, we should not dwell at lengt h on that area because our criticism will likely fall on deaf ears. On the other hand, if the student shows a keen interest or appears motivated in a given area, we should offer a more detailed analysis of the area and other suggestions for improvement. 27.4.4.2. We should vary our organization and methods of giving feedback. We may choose to begin with our own comments, continue with a group critique, if appropriate, and finally requestindividuals to give feedback on their own performances. Additional ideas on organizing and conducting a feedback session are included later in this chapter. 27.4.4.3. Use different approaches to cope with diff erent situations and adapt the tone, technique, method, organization, and content of the feedback to the occasion and the student. We frequently confront the problem of selectivity—what to say, what to omit, what to stress, and what to minimize. Consider the class situation, student ability, subject matter, the time allotted for the feedback, and other factors when determining your ap proach. Feedback should never be so rigidly designed or executed that you cannot allow for variables.
27.4.5. Organization:326 AFMAN36-2236 12 NOVEMBER 2003

27.4.5.1. Unless feedback follows some pattern of organization, a series of otherwise valid comments may lose their impact. Almost any pattern is acceptable if it is logical and understandable to the student and instructor. Therefore, the instructor should tell the student what the pattern will be in order to improve the student's comprehension ofthe feedback. For example, an effective organizational pattern might be the sequence of the stude nt activity itself. In certain instances, feedback can begin at the point of failure and work backward through the steps that led to the failure. A successful performance can be analyzed in a similar fashion. 27.4.5.2. Sometimes a defect is so glaring or a strength so obvious it overshadows the remainder of the student activity and can serve as the focal point of feedback. Breaking the whole into partsor building the parts into a whole has strong possibilities. Whatever the organization of the feedback, the instructor should be willing to change if the student cannot understand it.
27.4.6. Comprehensiveness. Comprehensive feedback does not need to be extremely long or treat every detail of a student's performance. As instructors, we must decide whether we can achieve the best results by discussing a few major points or a num ber of minor points. Base the feedback on areas that need improvement or on areas we reasonably expect a student to improve in. 27.4.6.1. Effective feedback includes both strengths and weaknesses. Only the instructor can determine a proper balance between the two. It is a disservice to students to dwell on the excellence of their performances and neglect areas that need improving. 27.4.6.2. As instructors, we should be specific wi th our comments and recommendations. Such a statement as "Your second phase wasn't as good as the third phase," has little value to the student until the student learns exactly why the second pha se was not as good as the third. If we have clear, supportable ideas in mind, we should express them firmly and authoritatively in easily understood terms. Specific examples followed by practical s uggestions are more meaningful than generalities. Students cannot act upon recommendations unl ess they understand what the recommendations are. At the conclusion of a feedback session, students should have no doubts concerning what they did well, what they did poorly, and, most importantly, how they can improve.
27.5. Methods of Giving Feedback. Critiquing a student's performance is the instructor's responsibility—never delegate it in its entirety. However, we can add interest and variety to our feedback if we draw on the talents, ideas, and opinions of others. The fo llowing methods for giving feedback may be used individually or in combination:
27.5.1. Instructor Feedback. Most critiquing results from the direct leadership and guidance of the instructor. For example, we conduct workshop sessions to refine skills, hold individual conferences or critique in front of a group, examine outlines and drafts, and coach and conduct practice sessions.
27.5.1.1. Group Critiques. Sometimes we give feedback in a one-to-one, student-to-teacher setting. A more efficient way is often within a gene ral class session because students may make identical mistakes as they perform an exercise or explore new experiences. Giving feedback to a group of students means that more aspects of the subject can be covered.
27.5.1.2. Written Feedback. Writing down feedback allows us to devote more time and thought to preparing it than when we give immediate oral feedback in the classroom. What we lose in spontaneity we often make up for in more comple te, carefully considered comments. Written feedback also gives students a permanent record they can refer to as necessary. Rating scales (a formAFMAN36-2236 12 NOVEMBER 2003 327

of written feedback) often provide for standardization among instru ctors, which students appreciate.
NOTE: We provide graphic rating scales and brief, but specific guidance for giving feedback in these two areas because so many academic instructors evaluate writing and speaking.
27.5.2. Student Feedback. Other methods of feedback focus on student leadership under the instructor's supervision. We often involve students in the feedback session because they learn as they help teach others. Many civilian schools use a system of t eachers' aides or graduate assistants to help with the feedback because direct involvement with stude nts can be extremely valuable; more experienced students can give more effective student-centered feedback.
27.5.2.1. Self-Evaluation. One goal of education is to give students enough confidence to be self-critical; allowing them to evaluate their own performances can help wi th this process. Beginner writing students, for instance, can often spot el ementary errors in their own work if they participate in a supervised proofreading exercise. When television facilities are available, an effective way to critique public speaking is for students to see themselves on TV. Giving students a checklist or a scale of some sort often aids them in self-evaluation. When students evaluate their own work, the instructor should follow up to make su re the feedback is complete and accurate.
27.5.2.2. Student-Led Evaluation. Students can give feedback ina variety of ways, but their feedback should not supersede that of the instru ctor. Sometimes a student may lead the discussion in a group feedback session, but the instructor should set the ground rules. The efficiency of this type of feedback may be limited by the inexperience of participants, but it may also generate a high level of student interest, participation, and learning. In student-led evaluation, the instructor invites members of the class to comment on student activity or singles out one student to present all the feedback. Another technique is to divide the class into small groups and assign each group a specific area on which to comment. Using basic criteria and guidelines issued by the instructor, these groups then present their fi ndings to the class. Their combined reports provide comprehensive feedback of individua l and class performance.
NOTE: Whatever the method, the instructor remains re sponsible for the overall quality and completeness of the feedback ( Figure 27.1. ). If students participate in the feedback, the instructor must make allowances for their relative experience. Normally we shoul d reserve time at the end of student-feedback sessions to cover those areas that may have been omitted or not given sufficient emphasis.328 AFMAN36-2236 12 NOVEMBER 2003

Figure 27.1. Ground Rules To Aid th e Instructor in Giving Feedback.

27.6. Summary. By the nature of the job, instructors are evaluators; therefore, they should be able to give feedback in the classroom. Feedback is not part of th e grading process, but a step in the learning process. Feedback is intended to improve future performances of the students and, when possible, to reinforce learning. Effective feedback stresses student strengths as well as suggestions for improvement. Feedback should be acceptable, constructive, flexible, organized, and comprehensive. Classroom feedback may be varied for interest, and it may be both written and or al. Student participation in feedback should be carefully supervised; instructors should reserve time for themselves to ensure adequate coverage. 1. Establish and maintain rapport with the students. 2. Tell the students the organizational pattern to be used in the feedback. 3. Cover the major strengths and we aknesses. Try to be specific and give examples if possible. 4. Avoid trying to cover everything. A few well-made points may be more beneficial than numerous but inadequately developed points. 5. Do not extend feedback beyond its scheduled time . A point of diminishing returns can be reached quickly. 6. Allow time for a summary of the feedback to re emphasize the most important things a student should remember. 7. Try to avoid comments with "never" or "always" —most rules have exceptions. Your feedback may be wrong. 8. Do not criticize something that cannot be corrected. 9. Do not criticize when you cannot also suggest an improvement. 10. Avoid controversies with the class, and try not to take sides with group factions. Stop the discussion before arguments get started. 11. Avoid being maneuvered into the unpleasant positi on of defending feedback. If feedback is honest, objective, constructive, and supported, no defense should be necessary. 12. If part of the feedback is written, it shoul d be consistent with the oral feedback.AFMAN36-2236 12 NOVEMBER 2003 329

Chapter 28

STUDENT DIFFERENCES

28.1. Introduction. Since we were old enough to notice the world around us, we have seen that peopleare different. The physical differences are most obvious; psychological differences (personality, motivation, and intelligence) are less obvious. Physical an d psychological differences among individuals are important concerns in such areas as business, politic s, and recreation, but they are especially important in education. Along with objectives, teaching methods , and evaluation, an understanding of our students is one of the most important factors in education.
28.1.1. While there is much research on student differ ences, firm conclusions are often hard to draw because of conflicting results. Conclusions about student differences often prove to be incorrect or only partially true as additional or more exacti ng research is conducted. Many educational decisions have been made based on faulty ge neralizations. Therefore, caution must be our guide as we examine this important area. 28.1.2. When studying human differences, we use m easurement techniques to obtain a range and distribution of scores and to describe how individuals differ. Most human differences cluster around a value known as the average. If we look at many individuals, we find some above and some below the average, resulting in a distribution known as the normal curve (see Chapter 26 ). Thus, in measurements of differences in intelligence, the major clus tering of cases occurs at a score of 100 (the mean). Intelligence quotients below 70 and above 130 are few innumber compared to the cases at or near the mean of 100. Similarly, the clustering of cases on the individual difference of height recently reflected an average of 5 feet 8 inches for adult males in the United States. 28.1.3. Because students vary so widely on many impor tant dimensions, instructors must be aware of these variations within a single classroom. Educators have normallytaken one of two approaches in planning curriculums for students with a wide range of individual differences.

28.2. Match the Method to the Student. This approach recognizes that students differ in abilities and other characteristics that affect their learning. Instructors should not accept all teaching methods and curriculums when they don't produce the desired learning outcomes. 28.2.1. For example, if a student or group of stud ents have trouble succeeding under a given teaching method, the instructor should change or improve the method so the students will succeed. Special efforts are directed toward helping students performi ng below the average. The instructor also tries to allow above average students to move ahead at gr eater speed to capitalize on their special talents. 28.2.2. However, better teaching me thods and means of reinforcing student behavior are not always practical. Drawbacks to this approach include the costs involved and the difficulty of designing and implementing effective programs.
28.3. Match Students to the Planned Learning Experience. This approach accepts the instructional system as it exists. Using measures of student differences to predict success minimizes failure. Using tests and other evaluative data, forecasts are made in se lecting, guiding, and admitting students to different curriculums, units, or courses of study. The school does not adapt its curriculum to the students.330 AFMAN36-2236 12 NOVEMBER 2003

28.3.1. This approach protects students by keeping them away from situations in which they are not likely to succeed. If students have too little ability to succeed in a school, course, or given curriculum, they are not admitted or are placed in an environment in which they can succeed. Information about student differences is used to predict and select . Only students with a high enough probability of success are admitted into the specified level or curriculum. 28.3.2. However, testing and evaluation instruments ar e often inaccurate or unreliable for predicting success. Students are forced to adapt to the teachi ng methods rather than having the methods adapted to their needs. Students who fail may not be as much at fault as the methods of instruction used on them.
28.4. Combining the Two Approaches. Some combination of the two approaches described in paragraphs 28.2. and 28.3. is probably superior to either one alone. If a teaching method is not succeeding with a student, then we should use another method. We should not consider teaching methods as fixed, nor should we disregard the students' abilities. We shoul d look for some optimal match of students with methods. 28.4.1. Students at one level of a given aptitude may yield greater achievement on one teaching method while students at another le vel of that aptitude may yield greater achievement using a second method. For instance, a student with fairly low mechanical aptitude might profit most from one-on-one instruction, while a student with higher aptitude may learn as efficiently in a group using the demonstration-performance method. No longer would a search be made for the one best way to teach. Many ways to teach would be investigated, each of which may be "best" for particular types of students. 28.4.2. Too little is known as yet to make possibl e the widespread design of educational programs based upon a combination of these two approaches. Many educators, however, are currently attempting to meet this need. Instead of arguing about wh ich of two teaching methods is best for everyone, chances are that both claims have some validity for some students. 28.4.3. Because of student differences, no one met hod, test, curriculum, or version of any teaching-learning activity is likely to be equally success ful with all students. Student differences and teaching methods interact to help or hinder learning. Id eally, we should never lose sight of the need to monitor and then match students to methods in so me fashion that best fit the individual student.
28.5. Group Differences. People are often grouped and studied on the basis of their age, sex, race, or background. When differences are found, they are referr ed to as group differences. For instance, school officials might gather data to compare achievement of male and female members at a military academy or the success rate of black and white undergraduate pi lot trainees. Managers mi ght draw conclusions from the accident rate of Air Force pilots under 30 ye ars of age compared with that of pilots over 30. 28.5.1. The results of research on group differences are often of limited use to educators because of the controversy they generate or because one study of ten conflicts with another. However, useful and tentative conclusions are being drawn in some areas of study, with implications for classroom instructors. For instance, much research has investigatedsex differences in the classroom. Although differences in verbal, visual-spatial, and mathematical ability have been found between the sexes, the cumulative research on ability and actual school ac hievement does not offer sufficient evidence for treating adults differently based on their sex. More importantly, current findings suggest that instructors should not hold different expectations of cognitive performance for the sexes.AFMAN36-2236 12 NOVEMBER 2003 331

28.5.2. While the extensive amount of research on group differences cannot be summarized in this chapter, here are some guidelines for dealing with these differences in the classroom:
28.5.2.1. Work to Eliminate Stereotypes. Stereotypes are oversimplified, rigid beliefs about categories of people. These beliefs tend to be i nvalid because they are overgeneralizations. If we believe that all noncommissioned officers (NCO) conform to the image of the drill sergeant, all second lieutenants are naive, all civil servants are not dedicated, or all fight er pilots are adventuresome, then we are guilty of forming stereotypes. 28.5.2.1.1. Today, many work positions are filled (and wages set) based upon stereotypes fostered by such innocent means as textbooks and TV shows. These communication mediums often show groups of people in stereotypical role s. The portrayals and values they project may actually interfere with or prevent pers onal growth for members of these groups. 28.5.2.1.2. While there is no easy way to eliminate stereotyping from our behavior, we need to make a concerted effort to eliminate the hundred s of subtle ways in wh ich people are treated based upon their sex, race, age, or background. These unsupported beliefs about people can do great harm. An awareness of our own use of stereotypes is a good place to start; we should examine our use of stereotypes and decide how th ey might be influencing our behavior in the classroom. We should then work to eliminate these unsupported beliefs about people and the influence they might have on our behavior.
28.5.2.2. Use Caution in Using Group Averages. Suppose we read a research article that indicates adult males are superior to adult females on visual-spatial tasks, or adult females are superior in tasks that involve understanding and producing language. How might this information have a negative effect on our relationships with students? 28.5.2.2.1. When we speak of group averages or group characteristics, we should be extremely careful in applying these figures or de scriptions to an individual student. While the average height of adult males may be 5 feet 8 inches, this average certainly means little when dealing with a student who is 6 feet 4 inches tall. Similarly, take greatcare when using knowledge that males are superior to females on visual-spatial tasks. I ndividual students in your class may deviate from the average of their group and the women may surpass the men in visual-spatial tasks in that particular class. A dditionally, expectations come into play. Students often perform to the level of their instructor's expectations. If we expect female students to outperform males in a language class, our e xpectations may influence actual performance. 28.5.2.2.2. If we are tempted to apply group averages to individuals, we might consider the comment of a famous British writer who was once asked, "Who has the most brains, man or woman?" His reply was, "Which man? Which woman?" Instead of treating students as members of a group, we should treat them as individuals.
28.5.2.3. Recognize an Instruct or's Limitations. When we focus on group differences in our classrooms, we soon realize they are a fact of life we cannot change. We cannot alter a person's age, sex, race, or background. Most students do not want to be treated differently based upon their group membership. Therefore, we can make certai n we do not say or do something to offend them or cause them to lose interest. At the same time , they deserve to be respected for who and what they are. 28.5.3. In summary, we should be very careful whenbasing educational polic ies and classroom practices on group differences.332 AFMAN36-2236 12 NOVEMBER 2003

28.6. Individual Differences. Our students will arrive with different personalities, varying motivation, a wide range of past achievements, and differing abilitie s. There is little we can do about their past achievements or personalities. We can, however, provide an environment in which they will be motivated and opportunities to help them make th e best use of their abilities.
28.6.1. Differences in Motivation. Probably the individual difference of most concern to Air Force instructors is motivation. Motivated students learn more effectively. Unmotivated students are likely to be a disturbing influence on others. Some st udents will possess an inner motivation, while others will be motivated to learn only when external rewards are offered, such as grades or the opportunity to be a distinguished graduate. Certain students will memorize the textbook if you ask them to since they want to please you. Some students will strive for excellence in all aspects of a course, while others will exert only enough effort to meet the minimum criteria for passing.
28.6.2. The Concept of Motivation. Psychologists use the concept of motivation to account for changes in and the frequency and vigor of a pers on's activities. We are always doing something, but some of our activities occur more often than others and continue longer. A dditionally, some activities are pursued more vigorously than others. Outcomes that usually give rise to pleasant feelings will be pursued; those that usually give rise to unpleasant feelings will be avoided. While a motive cannot be observed directly, when activated by a situation we can see its effect on behavior. We say persons are motivated or we speak of their motivation.
28.6.2.1. Understanding Motivation. The concept "motivation" is complex and can be viewed as encompassing other closely related concepts.
28.6.2.1.1. Needs. When students have a need, they lack something a given activity or outcome can provide. The need to belong, for instance, can motivate a student to seek group acceptance.
28.6.2.1.2. Interests. If students have an interest in a subj ect, they tend to pay attention to it. For instance, the student who is interested in computers is more likely to pay attention to instruction in that subject.
28.6.2.1.3. Values. Students with a particular value have an orientation toward a class of goals considered important in their lives. A student who values patriotism will most likely be motivated in a lesson on the flag or the Code of Conduct.
28.6.2.1.4. Attitudes. Attitudes consist of feelings for or against things (people, objects, ideas). If students like mathematics, they get pl easure from activities that involve mathematics. Students seek activities toward which po sitive attitudes are held. Attitudes direct and arouse purposeful activity.
28.6.2.1.5. Incentives. Incentives can satisfy an aroused motive. Incentives such as good grades, awards, and selection as a distinguish ed graduate motivate students who want to achieve.
28.6.2.2. Aspiration. Aspiration is the hope or longing for ach ievement. A certain level is needed if the student is to make an effort. For instance , the NCO may aspire to be a college graduate and be willing to devote evening hours to complete degree requirements.
28.6.2.3. Achievement Motivation. The concept of motivation helps to explain why students with the same scholastic aptitude or intelligence perform differently in our classrooms. When stu-AFMAN36-2236 12 NOVEMBER 2003 333

dents achieve beyond their predicted capability, we speak in terms of overachievement. When they achieve below predicted capa bility, we say they underachieve. 28.6.2.3.1. It will not take us long to note that our students differ in achievement motivation. Students with high and low achievement motivation will perform differently on tasks when working independently. Students low in achieveme nt motivation will need more of our assistance in the form of supervision, direction, and structure. Highly motivated students will require little encouragement or direction when working in areas they value. 28.6.2.3.2. Students high in achievement motivation will persist in task completion and have the desire to complete interrupted tasks. They also have the desire to keep up orderly progress toward distant goals. During group activities, instructors will find these students want to work with partners who can get the job done.
28.6.2.4. Fear of Failure:
28.6.2.4.1. Not all student motivation comes from the desire to succeed. Some students will be motivated by the fear of failure. In the past, st udents may have experienced shame, ridicule by peers, or punishment for failure. The resulting fear of failure may be more motivating than a desire to do well. The fear may even keep students from attempting certain activities, or they may be motivated to attempt unrealistic tasks. 28.6.2.4.2. Suppose your course uses programmed in struction with its emphasis on errorless performance. You may find a decrease in mo tivation among students whose motivation to succeed is higher than their fear of failure. However, you may see an increase in motivation among students whose fear of failure is higher than their motivation to succeed. If you use strict grading with your students, those whose motivation to succeed is high should be more motivated, students whose fear of failure is grea ter than their desire to succeed will probably perform worse.
28.6.2.5. Affiliation, Power, and Approval. Some students will be motivated by the want and need of friendly relationships with other persons (affiliation). By mixing teaching methods, wecan provide opportunities for these students to participate in small groups. Others will be motivated by the desire to influence others (power). We can provide leadership opportunities in group activities or we may need to counsel these st udents on their tendencies to dominate or manipulate others as a cause for their being disliked. Other students will have strong needs for recognition or approval. We will have to give special attention to this group in our counseling program. We can provide opportunities for these stude nts to participate in classroom activities where their particular talents can be recognized.
28.6.2.6. Motivation and Reinforcement. From the viewpoint of psychologists who emphasize the role of reinforcement in learning, all of us are motivated by the reinforcing consequences of our past behavior. Our motivated students may not have strong motives to learn or be driven by powerful drives and needs; rather, they may be students who have been properly reinforced for certain behaviors in their past. The students may have been rewarded in the proper amount and at the proper time when a certain behavior was exhi bited. The truly motivated student may be theperson who has learned to get along for long periods of time without much apparent reinforcement. From the behavioral perspective, we can motivate students by a great deal of reinforcementin the early stages of learning (for instance, praise for each correct answer) followed with a gradual thinning out or lowering of reinforcement as the student matures.334 AFMAN36-2236 12 NOVEMBER 2003

28.6.2.7. Frustration and Motivation. Incentives such as grades, awards, and distinguished graduate programs provide a degree of control over most students be cause the expectation of reinforcement is usually met with actual reinforcement. But what happens when such expectations are not reinforced? Suppose we have students working for good grades, wins in some form of athletic competition, or verbal praise. In the past we have reinforced this behavior with these incentives. 28.6.2.7.1. But suppose we stop these incentives. The students' behavior may show changes in energy and direction leading to frustrated behavi or—expected grades are not received, the students do not or cannot score points for their team, or students may be moved from a group in which they are doing well to one in which they do not receive their expected reinforcement— frustration results. Similarly, keeping students from finishing a task they are engrossed in because time ran out leads to frustration. Eli minating students from athletic competition with their peer group because their peers are too skilled blocks their chances for reinforcement. 28.6.2.7.2. Frustrating situations like these mo tivate students because they change student energy level and direction. However, the motivatedbehavior is usually in a socially undesirable direction resulting in emotionality, withdrawal, regression, over-activity, and aggression. 28.6.2.7.3. As instructors, we should guard agains t frustrating situations. When we see signs of aggression, withdrawal, or in appropriate displays of emoti on exhibited by our students, we should regard such behavior as a cue to the occurrence of frustration. We should make an effort to determine the source of the frustration.
28.6.3. Techniques for Motivating Students. Figure 28.1. lists techniques to use to motivate students. In addition, educational research suggests the following techniques to motivate students:
Figure 28.1. Techniques for Motivating Students.

Verbal Praise Words such as "good," "great," and "fine" work as powerful motivating devices when given after a proper response or pe rformance or after a close approximation of a proper response or performance. Verbal praise is easy to use and serves as the most natural of the motivational devices available to an instructor. Praise given after a certain behavior increases the frequency of that behavior.

Written Comments Studies indicate that a short enc ouraging note based on a stude nt's test performance has a more positive effect on future testperformance. Additi onally, brief comments are more effective than grades alone in motivating students to do better on future tests.

Grades Students have learned that benefi ts are associated with getting good grades. Consequently giving tests and assigning grad es have come to have the effect of motivating students to learn. Not only do grades affect what is learned, they affect how much is learned. They become incentives and reinforcers.AFMAN36-2236 12 NOVEMBER 2003 335

28.6.3.1. One of the strongest arguments against using tests and grades as motivators is their link with anxiety. When a student's anxiety is very low or very high, test performance is lowered. Almost always, very high anxiety will hurt perfo rmance. However, moderate anxiety motivates in a beneficial way by assisting performance. For stude nts with high anxiety, we should seek ways tolower it in the classroom; for those with low anxiety, we might find ways to raise it to improve performance. 28.6.3.2. When students receive either a pass or no credit for a course instead of conventional grades (such as A through F), the result may be decreased effort. Some students may do less work. However, many students and faculty members fr equently mention the positive consequences of this type of grading system. There is less pressure on students, they can experiment in new areas, and a healthier attitude develops toward the course. These benefits may outweigh any reduced quality or amount of student achievement occurri ng under a pass or no credit type of grading as shown in Figure 28.2.336 AFMAN36-2236 12 NOVEMBER 2003

Figure 28.2. Benefits Associated With Grading System.

Creative Behavior Doing the unexpected or someth ing novel has the effect of motivating by arousing the student's curiosity. One Air Force in structor established a reputation based upon this technique. During a class period this instructor might appear as an old west gunfighter to emphasize points. The ne xt day the instructor might be dressed as an astronaut. Students looked forward to classes because of the unexpected. Instructors might turn the tables and ask students to prepare a test for them to take. Or students can be asked to prepare test questions for use on the next exam. In a case study, the instructor might vary the approach by giving students several solutions to the problem and asking them to determine which is best.

Periodic Reinforcement Providing reinforcement to students as a course progresses can keep them motivated to the end. If we make early stag es of a learning task easy students will have some initial success in task acco mplishment. Introduce the difficulties and complexities of a task gradually and ensure that all students have a chance for reinforcement so their appetite will be wh etted. This action will require us to know the variety of talents and skills possessed by our students.
Familiar Examples The value of examples as support material was discussed in Chapter 6 . Examples when based on the experiences of your students serve to motivate.
Building Block Concept Building upon what has previously been taught is motivational. Using the cognitive taxonomy of learning we build upon prior instruction. When students see how we use prior instruction to build the current lesson the effect is motivational. We should emphasize that the material we are now learning does not stand in isolation. We will be using it again later in the course or unit of instruction. When possible, we should call upon previously ac quired facts, concepts, and principles to develop a current lesson.
Computers, Simulations, and Games In addition to dealing with higher levels of learning in the cognitive domain, the use of simulations, games, and computer s can be fun and may provide important learning experiences and usually keeps students highly involved and motivated. Using and developing such devices usually increases classroom motivation.

Classroom Leader Support We should try to get the leaders among the students to adopt the goals and objectives of the course. If the formal a nd informal leaders—not always the same person—in a class support us and our efforts, other students will be motivated to do likewise.
Unpleasant Stimuli The list of unpleasant stimuli to avoid is long and includes items such as having students sit too long in an auditorium, having them try to listen in a room with poor acoustics, putting them too far away from a board or screen, not telling them how they are doing in the course until it is t oo late to do something about it, or not meeting their requests for help. These st imuli have a negative effect on motivation and should be eliminated.AFMAN36-2236 12 NOVEMBER 2003 337

28.6.3.3. As we work to motivate students and fi nd techniques that work we should take the time to annotate our lesson plan so the techniques can be used the next time the lesson is taught. If a motivational technique does not work, drop it from th e lesson plan. In time we will have a lesson that motivates as well as teaches.
28.6.4. Differences in Ability. The testing movement during the 1920's revealed a wide range of student intelligence quotients in typical public school cl assrooms. Studies indicated that the brightest students could move at a more rapi d pace and learn much more than less able students. The findings raised questions about accepted teaching methods and indicated that many materials, texts, and topics were inappropriate for some students. Educators c oncluded that any attempt to teach students in a group would likely miss the mark for some students. The findings greatly complicated the task of teachers. 28.6.4.1. As a solution, educators turned their attention to individualized instruction. Students should work on tasks appropriate to their particul ar abilities. To the extent possible, they should move ahead at their own pace. Accounting for i ndividual differences in student ability became one of the primary purposes of individualized instruction.
28.6.4.2. Individualized instruction occurs when the goals of instruction, learning materials, subject matter, or methods of instru ction are specially chosen for a pa rticular student or a small group of students with common characteristics. The memb ers may be placed in a special seminar and the academic program tailored to their special needs.
28.6.5. Independent and Self-Directed Study. Air Force instructors may have an opportunity to allow students to work on their own as a supplement to, or replacement for, segments of a course. When this opportunity exists, instructors can provide a student or small group of students with an assignment that lasts for a few hour s, several days, or weeks. Stude nts may not be required to attend formal classes during this period.28.6.5.1. If we use this approach, we should ensure that students clearly understand the objectives, resources available, task steps involve d, time allotment, and eval uation techniques prior to the start of the study. The degree of supervisi on given depends on the maturity and objectives of the students. We may need to supervise and guid e the students fairly closely or they may work almost entirely on their own and be responsible for a finished product which we will evaluate. 28.6.5.2. Research on self-directed study suggests that most students will need some guidance,even if they are going to work independently. Many students are not ready for intellectual independence and will complain if thrown unprepared in to an extreme form of self-directed study. The method seems to work best in more advanced c ourses where students have previously dealt with the same material at a basic level.
28.6.5.3. Be aware that initial student reaction to self-directed study course s is often unfavorable. However, students do develop more favorable attitudes toward future opportunities for intellectual independence. The unpleasantness from initial exposur e to this approach probably comes from the big change in teaching methodology. But the broadening experience eventually makes studentsaccept more independent work. When using this approach, we should keep in mind that the unfavorable initial reaction may last and could affect student response to any additional self-directed studies.
28.6.6. Contracting. One variation of the use of independentand self-directed study involves using learning contracts between individual students and the instructor ( Figure 28.3. ). The approach can be338 AFMAN36-2236 12 NOVEMBER 2003

used for a total course, as a means of supplementing a basic course for all students, or for use with selected students. A contract involves an agreemen t, usually written, between the instructor and student.
Figure 28.3. Elements Found in Learning Contracts.

28.6.7. Mastery Learning. This approach to individualizing in struction operates on the premise that time is a crucial variable in the learning process. In this approach, the instructor fixes the degree of learning expected of students at some mastery level and allows time for learning to vary so that all or almost all students achieve the desired level of mastery. 28.6.7.1. Mastery is defined in terms of a specific set of major objectives students are expected to exhibit by subject completion. The subject is thenbroken into a number of smaller learning units. Unit objectives, whose mastery is essential for mastery of major objectives, are defined. Students are allowed to demonstrate mastery of a unit at any time they choose. Br ief diagnostic (formative) tests provide feedback to the students and instructor. 28.6.7.2. When unit learning problems are identified, supplementary instructional correctives are applied to help students. These correctives invol ve use of small group problem sessions, individual tutoring, alternative textbooks, workbooks and programmed instruction, audiovisual methods, Objective Specify in the contract the particular objectives to be achieved.

Evaluation Techniques Agree upon the formal means of evaluation whether it be a test, oral or written report, or a project.

Available Resources Make the student aware of available res ources for starting the learning experience.

Task Steps Establish a series of task steps leading to achievement of the objectives.

Checkpoints Meet periodically with the student to discuss problems and progress. These meetings serve to motivate the student.

Deadlines Establish and require the student to meet realistic deadlines. If the student fails to meet a deadline, counseling is usually required. Renegotiate the contract, if necessary.

Reinforcement When possible, reward the student for contract completion. An opportunity to brief others, participation in a conference, or an opportunity for extended study in an area of interest might serve as the reward.AFMAN36-2236 12 NOVEMBER 2003 339

academic games and puzzles, and reteaching. Th e students demonstrate achievement of course objectives through end-of-instruction (summative) evaluation. The instrument used defines mastery in terms of a specific set of skills each st udent is expected to have learned by completion of the subject. 28.6.7.3. The mastery approach is very student orient ed. Students are graded solely on the basis of the final (summative) examination performance. Grades are based on performance against a predetermined standard and not relative to peer performance. All students who attain the standard receive appropriate grade rewards. There is no fixed number of rewards. Students are given constant feedback through a series of ungraded, diagnosti c progress tests, and each student is given all the help needed to learn. The mastery learning approach is particularly effective when students have a wide, but manageable , range of entry skills. 28.6.7.4. While the mastery learning approach has high potential, there are limitations. First, the approach works best when the course requires minimal prior learning or previous learning that most of the students already possess. We would not want to institute the approach in an advanced course if most of the students did not possess certai n basic skills or if time or money did not permit us to modify the instructional design to take this into account. The approach may not be able to offset the negative effects of prior learning defi ciencies. Second, the approach has been most successful for subjects that are sequentially learned. The learning of a unit builds on the learning of all prior units. Finally, the subjects in which mastery learning strategies are most effective tend to be those for which there is a single definite right answer to each question.
28.6.8. Personalized Instruction. One approach that appears to work successfully with adults involves personalized instruction ( Figure 28.4. ). Using the concepts of i ndependent and self-directed study, personalized instruction is supplemented with traditional inst ructional techniques.340 AFMAN36-2236 12 NOVEMBER 2003

Figure 28.4. Personalized Instruction.

28.6.8.1. Courses taught using the guidelines ( Figure 28.4. ) for personalized instruction show promising results. Student reaction tends to be pos itive and examination scores are higher than in courses conducted in a traditional manner. 28.6.8.2. We should attempt conducting such a course only when we have carefully decided what we expect students to learn. The objectives s hould clearly state responsibilities. This knowledge can help overcome negative feelings about the lack of structure that often accompany participation in independent study programs. Many students like self-paced instruction because it allows them to study where and when they please. They also like the personal interaction with tutors.
28.6.9. Cognitive Learning Style. Students learn in many ways. They perceive, think, and solve problems differently. The general patterns students use as they perceive, think, and solve problems can be called their cognitive style or cognitive learning style. 28.6.9.1. Learning research is beginning to identifyseveral distinct styles of learning that can affect how much and how well students learn. There are several models for analyzing cognitive learning style in current educational literature. So me deal with preferences students have for learning through passive listening rather than active participation. Other models attempt to link Individual Pacing Students can rapidly complete a c ourse, often using a fraction of the time required for group instruction; however, they ma y use more time than is allotted for traditional instruction within constraints imposed by the administration. Students work to meet established criteria and do not compete against each other.

Mastery Oriented Before students can progress to ne w material, they must demonstrate mastery of a particular area of instruction. Students are not penalized for failure. If failure occurs, students study the material again be fore retesting on a parallel form of the test. They continue the cycle until master y is achieved, which usually means 80 or 90 percent correct responses. When students finish the unit of work, a new study guide is provided for the next unit.

Tutoring More advanced students can often serv e as tutors for those needing assistance. Encouragement and support are provided.

Study Guides Students are provided with study guides for each unit. The guides contain unit objectives, study references, suggestions for study, experiments and projects that can be undertaken, and sample test items.

Supplemental Materials Lectures, demonstrations, discussions, teaching interviews, case studies, and movies can be provided for motivation or clarification. Students normally attend on a voluntary basis.AFMAN36-2236 12 NOVEMBER 2003 341

self-concept with willingness to accept group norms or to take risks. Many other personality characteristics are included in these models. Consider able research suggests that various aspects of cognitive learning style should be explored further. 28.6.9.2. Many schools, particularly at the communitycollege level, have employed an analysis of cognitive learning styles for placing students into courses taught differently. Grades seem to improve and anxiety is reduced when students and their learning experiences are matched on the basis of cognitive learning style. Individual instru ctors can do very little, however, to take advantage of the various learning styles in typical classrooms. Some models for determining learning style are very theoretical; many of the techniqu es are very sophisticated, time consuming, and expensive. Further, if a cognitive learning style can be determined, we ha ve the additional problem of matching several altern ative ways of teaching to the many and varied student cognitive learning styles. 28.6.9.3. The growing interest in cognitive learning style may yet be one of the most fruitful approaches to dealing with individual differences in the classroom. Like so many other attempts to deal with students as individuals, however, the fate of cognitive learning style analysis will depend on results as compared to costs. From what we have learned so far, the approach appears to deserve further serious study.
28.7. Summary. One of the basic themes of psychology is that of human differences—no one is exactly like anyone else. These differences co mplicate the task of classroom instru ctors, but they also provide the challenge. 28.7.1. Many educators have chosen to adapt the system to account for differences in abilities and other characteristics that affect student learni ng. Teaching methods are changed or improved so students can succeed. Other educators have chosen to accept educational systems as they exist and they try to minimize failure by admitting students w ho will most probably succeed. Each approach has shortcomings so attempts are be ing made by educators to synthesi ze the two approaches to overcome the effects of student differences. 28.7.2. As classroom instructors, there is not muchwe can do to overcome group differences based on sex, age, race, or background. However, we can workto eliminate stereotypes based on these differences, use caution in interpreting group averages, and recognize our own limitations in coping with these differences. 28.7.3. Students vary in their motivation. Many students are motivated by the need to achieve, while others may be motivated more by a fear of failure . The want and need for affiliation, power, and recognition also serves to motivate students. Frustration can motivate students, but the motivated behavior is usually in a socially undesirable direction. 28.7.4. Techniques for motivating students include ve rbal praise, personal notes on their test performance, grades or similar rewards, creative behavior on the part of the instructor, reinforcement at the appropriate time, use of familiar examples, building upon previous instruction, obtaining classroom leader support, avoiding unpleasant stimuli, and using computers, simulations, and games. 28.7.5. Differences in ability in a classroom ma y be met through such means as independent and self-directed study, contracting, mastery learning, and by personalizing instruction.342 AFMAN36-2236 12 NOVEMBER 2003

Chapter 29

THE DYNAMICS OF SMALL LEARNING GROUPS

Section 29A—Overview

29.1. Introduction. Appropriate teaching methods are a continui ng concern of Air Force instructors. The formal teaching lecture, for exampl e, may place students in a passive role and fail to provide enough incentive or motivation for students to learn. On the other hand, self-paced or individualized instruction, which often provides more incentive or motivation, may not be feasible.
29.2. Small Learning Group Defined. Small group learning is a way to overcome limitations of some methods while promoting verbal and nonverbal intera ction among instructors and students. A small learning group is a collection of students who share a common learning objective and occupy a space so limited there can be direct verbal and nonverbal communication among them. A small learning group might consist of as few as 3 or as many as 20 students. Th e upper boundary is arbitrary, but the benefits of small group learning are reduced if there are many more than 20 persons. 29.2.1. The demonstration performance, guided disc ussion, and case method discussed in this guide are instructional methods generally designed for sm all learning groups. However, these aren't the only methods designed for small group learning (see Chapter 19 ). Each method helps students become socialized and adjust to the learning environm ent by promoting interaction in the classroom. 29.2.2. Simply forming small groups of students, however, does not guarantee effective group learning. Human communication is comprised of two ma jor ingredients—content and process. The first deals with the subject matter or the task the group is working on. In most learning settings, people focus on the content. The second ingredient, proce ss, is concerned with what is happening among members while the group is working. Group process or dynamics deals with such items as norms, cohesiveness, consensus, individual roles, group feeling, needs, and leadership. In many learning groups, too little attention is paid to process, ev en though it is a major cause of ineffective learning. 29.2.3. Analysis of verbal behavior in learning gr oups indicates that about half of the discussion focuses on process and half on content. Certainly e ducational content is of prime importance, but process statements that encourage a group's progress are also important. Attention to process serves to keep students on track by orienting the discussion and satisfying their personal and social needs. Sensitivity to group process will help an instructor diagnose students' problems early and deal with them more effectively.
Section 29B—Small-Group Learning Influences
29.3. Benefits of Small-Group Learning. Although small-group learning has some limitations, such as the higher cost of a smaller student-to-instructor ra tio, the benefits generally outweigh the limitations. Some benefits of sma ll-group learning include: 29.3.1. Most students tend to perform better on many c ognitive and skill-oriented tasks when working with others in small groups. The presence of othe rs working on the same problem increases motivation to perform well. This motivation may be lacki ng with either large groups or individualized learning.AFMAN36-2236 12 NOVEMBER 2003 343

29.3.2. Students can gain a clearer understanding of the material when they interact with the instructor and other students. Much informal learning occu rs in families and other groups; therefore, students are accustomed to learning in groups. 29.3.3. Most students enjoy interacting with othe rs in social, recreational, and educational groups. Satisfaction and enjoyment of learning is in creased when participating with others.29.3.4. Students often learn faster in groups than individually. Obviously, such factors as content matter and individual motivation influence whether individuals learn faster in groups.
NOTE: In spite of the positive influences a group can have on students, there are other influences that may or may not be beneficial. For example, member ship in a group places much pressure on individual students to conform to group norms.

29.4. Norms. Every group develops norms; that is, shared e xpectations of conduct and participation that guide and regulate group behavior. Norms reflect how st udents speak, to whom they speak, where they sit, what the classroom activities are, and what the limits are for expressing their own attitudes and values. Usually such norms are not even stated, but are still readily apparent to students who have been in the class even for a short time. In a large lecture class, for instance, instructors have greater responsibility for determining norms than in smaller groups where students and instructors can interact more easily. On the other hand, in small groups instructors must be pa rticularly careful not to impose norms on the group unnecessarily. Group atmosphere or feeling will be improved if students are allowed to help develop

norms. 29.4.1. Classroom norms become apparent in two ways: (1) certain behaviors either recur often or are avoided, and (2) punishments or sanctions are take n against persons who deviate from the norm. Others may frown, ignore, or even comment negative ly about the behavior. Some norms are productive: students should relate their comments to what has be en said previously; student s should arrive at class on time; the instructor should include all students in the discussion, etc. Some norms, however, are largely counterproductive: students s hould talk about whatever they want; the instructor should "put down" students who express different points of view ; the instructor and students do not need to prepare for class, etc. 29.4.2. Students must conform to productive norms if they are going to learn efficiently in groups. Ifproductive norms are clearly understood but violated by one or more students, then other class members will generally act to bring violators in line. They may exert pressure by talking to them directly. Usually the first comments are light and humorous, but if gentle teasing and light comments do not change the behavior, the language may become more biting and satirical. Inst ead of laughing with the nonconformists, others begin to laugh at them. If the member still does not conform, direct ridicule and finally rejection of the nonconformist may result. Obviously, the instructor should take action to prevent behavior that may threaten the cohesiveness of the learning group.

29.5. Cohesiveness. Cohesiveness, or the ability of a group to stick together, is essential in small group instruction. Cohesive groups—groups where students have high group loyalty—exhibit higher morale, greater productivity, and more communication.
29.5.1. Morale. The morale of members is related closely to cohesiveness. If the group is important to them, students pay attention to its problems and ge t more involved in the subject matter. They spend more time on behalf of the group and wa nt others in the group to do well.344 AFMAN36-2236 12 NOVEMBER 2003

29.5.2. Productivity. Cohesive learning groups ar e more productive because class members help one another. If one student is having trouble, others often offer to assist. Students in groups with little cohesiveness often lack initiative and have to wait for instructions from the instructor. They often do only what they are told and no more.
29.5.3. Communication. Cohesiveness encourages communication among students and between the students and the instructor. Students feel secure as a part of a group and are not afraid to speak up. The instructor and class leader do not feel threatened when questions are asked. Everyone is interested inthe welfare of the group. Groups lacking cohesiveness may be quiet, polite, boring, and even apathetic. Cohesive groups tend to be noisier, tend to disagree more, and of ten run overtime in class because there is generally a good deal more discussion on each point raised.
29.6. Increasing Cohesiveness. With small group interaction, we want to increase cohesiveness to promote learning. Some steps for improving cohesivene ss in the learning group consist of the following: 29.6.1. Set obtainable objectives. Clear obtainable obj ectives are important to efficient planning and teaching. Course objectives, objec tives for large blocks of instruction, and individual lesson objectives not only enhance learning but also increase morale. Achieving an objective rewards a group. Making daily and weekly progress rewards the group regularly. Of course, for objectives to build morale, they must be obtainable and clearly specified and understood by all. 29.6.2. Reward the group. Reaching an objective is in itself a reward for the group, but the instructor can provide incentive and promote teamwork by rewarding the entire group for achieving a goal. Praising the class for good work and planning social affairs recognizing a job well done are ways of rewarding the group. Many classrooms are geared entirely toward rewarding only individuals. Yet as any athletic coach and most e xperienced teachers know, encouraging persons to look out only for themselves to the exclusion of others leads to tr ouble. The wise instructor uses both individual incentives and group rewards to enhance small group learning. 29.6.3. Build a group tradition. As discussed in paragraph 29.4. , once a group is established, norms develop. Norms are a part of group tradition. Other things also happen. Unusual, humorous, or otherwise memorable incidents become a part of the gr oup's history and tradition. An instructor can emphasize that the group exists over time by recalling such incidents at class meetings. Highly cohesive, social organizations such as churches and service clubs enact ceremonies and traditions over and over. One instructor built a group tradition and aided cohe siveness by slipping into the classroom early eachmorning to write a little saying on the dry erase board. When the instructor failed to do so one morning, the seminar group was genuinely disappointed.

29.7. Consensus. Certain cohesive groups will express disa greements more openly than less cohesive ones. Such expression is both desirable and expecte d. But there comes a time when consensus or agreement on issues must come if closure is to result. Inlearning groups as with other types of groups, success can generally be measured by the group's ability to reach its objectives. Although consensus in the sense of unanimity may be nearly impossi ble to achieve, there are some guidelines the instructor can follow to help the group reach general agreement. 29.7.1. Clarify the discussion. Make sure the group's activity is understandable, orderly, and focused on one issue at a time. Consensus comes more easily if each of the factors is weighed individually and systematically. Encourage each student to stick to the subject, avoid side discussions, and clarify theAFMAN36-2236 12 NOVEMBER 2003 345

issues with questions so everyone can have equa l understanding. Instructors can keep the discussion focused and prevent the introduction of extraneous or unrelated matters. 29.7.2. Use process statements. As suggested earlier, process statements deal with what is happening in the group. While process statements may relate to the content, theyserve primarily to stimulate and facilitate discussion. Use statements such as, "Wha t you've said seems to make sense. How do the rest of you feel?" "So far we seem to agree on the firs t two points; let's move on to the third." "Have we heard from Joe yet?" "This is really a productive di scussion." Questions that ask for clarification and statements that get the class discussion back on track are also types of process statements. When both instructor and students use process statements effectively, agreement will come more readily and learning will increase. 29.7.3. Seek different views. Encourage all students to present their views and provide information and evidence to support their views. Expressing a wide range of opinions and views allows a great opportunity for learning to take place. At the same time, participation by a ll students will allow them to have their voices heard and will increase thei r satisfaction with the discussion and conclusions reached. 29.7.4. Remain open to different views. This discussion is clearly the corollary to the preceding guideline. We have all known people who seek the view s of others with no intent to be influenced by them (don't confuse me with the facts; my mind is made up). When others present irrefutable facts and figures or even a good idea you may not have thought of before, don't be afraid to alter your position or admit you may have been wrong. Good instructors often learn from their students. Also, instructors can serve as models for the behavior of students in the matter of not being overly opinionated. Studies show that low or moderately opinionated instructors and students are held in higher esteem by others than highly opinionated ones. 29.7.5. Use group pronouns. Studies show that less cohe sive groups, that is, groups less successful in reaching consensus, tend to use more self-referent words such as I, me, my, and mine. On the other hand, groups that reach consensus and are more cohesive are more apt to use group-referent words such as we, our, and us. As an instructor, talk about the seminar or class as a group. Talk about what "we" hope to accomplish and how "we" can work toge ther to achieve "our" objectives. Do not emphasize what "I" want done or what is best for "my" interests. Stress that while all students should be concerned with their own learning, they should also be interested in the learning of others in the group. The roles each student assumes can help with ef fective functioning of the group and with creating an atmosphere where learning can take place.
Section 29C—Roles of Class Members and Instructors

29.8. Class Member Roles. In all small learning groups, students begin to play different parts or roles after they have been together for a while. When a st udent becomes comfortable with a role and the rest of the group offers rewards for assuming the role, that student has found a place in the group. The student expects to play the role and others expect it as well. This situation is comparable to roles in plays and, interestingly, just as people play different roles in diff erent plays, they also play different roles in different groups. The gregarious life of the class may be quiet , devout, and reserved in church. The person who is ill tempered and bossy at home may be a likeable, even-tempered person in the classroom. A student may be outspoken in one class and a good de al more reserved in another wher e the content matter is less famil-346 AFMAN36-2236 12 NOVEMBER 2003

iar. The basic principle here is that a student's role is worked out jointly by the student and the rest of the group. 29.8.1. Student behavior in learning groups is similar to role behavior in other small group situations. Obviously there are many different roles and one person might fill more than one role in a group. Some task roles found often in learning groups are: idea-initiator—proposes new ideas, new goals, possible solutions; information giver—offers facts and information, personal experiences, and evidence useful for the group; and coordinator—develops, describes, and shows relationship among facts, ideas, and suggestions. 29.8.2. Common roles for group maintenance are: supporter—agrees, praises, shows warmth and solidarity to others; harmonizer—mediates differen ces between others, finds common ground, reconciles disagreements, conciliate s, and suggests compromises; and tens ion reliever—jokes, points out humor in the situation, and puts others at ease by reducing formality and status differences.
29.9. Dealing with Nonproductive Roles. Instructors are pleased to have students assume productive roles like those described in paragraph 29.8. With students like these, the learning group becomes more productive for all its members. However, instructor s are often faced with the problem of students whoplay nonproductive roles. Here are some common problems and suggestions for dealing with the problems: 29.9.1. Monopolizers talk too much and hinder partic ipation by others. If they know a lot about the subject and are eager to be helpful, then don't embarrass them for you may need them in this role later on. Do not, however, let them monopolize or give long speeches. Politely turn the conversation to another student by asking a direct question. Do not use an overhead question (a question asked of the group as a whole) because a monopolizer will mostlikely answer. If monopolizers are simply trying to impress others with their knowledge or status , encourage them only when their comments are productive; otherwise, politely interrupt them and move to another student or a different topic. Other members of the group will generally handle the problem for you. Give them positive reinforcement when they do. 29.9.2. Arguers take issue with what the instructor and other students say. They may simply have strong personal convictions about the topic. If this is the case, don't close your mind to some possibly good ideas just because of the opinionated manner in which they are presented. Talk to arguers outside of class and explain that while you consider their views important for the group to consider, they must not hinder overall group effectiveness. It is possible that arguers are attempting to gain recognition or perhaps even attempting to disrupt the group. Try not to get too excited—above all, don't let the group become antagonistic. Remember that the group is at least partly responsible for a person playing this role. You may want to talk to other students outside of class and ask them what they think can be done to understand the person and help change the behavior.29.9.3. Nonparticipants show reluctance to participate verbally in class discussions. Although nonparticipation may result from not being prepared, students often don't participate because they feel insecure as individuals or insecure with the particular group, perhaps due to perceived status or rank differences. Spend extra time with these people outside of class. Discover their interests. In class, ask such students direct open-ended questions. Make ce rtain the question can be answered easily, but not with either a "yes" or "no" response. When these students take part in the class discussion, give positive reinforcement. Say something like, "That's interesting, Mary. We need to hear from you more often." Or, "I hadn't thought of it that way before; good answer."AFMAN36-2236 12 NOVEMBER 2003 347

29.9.4. Clowns don't take things seriously and as a resu lt distract others and hinder learning. These students generally enjoy attention. Encourage th em when tensions mount. Laugh and let them know you admire their humor. Ignore them when "clowning around" is inappropriate. They will not like to be ignored and will soon learn when and when not to supply humor.
29.10. Dealing with Nonproductive Groups. Instructors occasionally find that rather than having problems with individual students, th ey have problems with nonproductive groups. Some common causes are: 29.10.1. Apathy—lack of enthusiasm, low response rate, yawning, quietness. Apathy may result from tension often felt in first meetings of the group, lack of interest in the topic, or a boring instructor. Whatever the cause, don't give up too soon. Even experienced instructors find that their learning groups sometimes become apathetic. Make time for small talk, kidding, and humor. Show as much enthusiasm and energy as possible. Do not give up if first attempts at motivating the group are unsuccessful. Use explicit and interesting examples to stimulate interest.
29.10.2. Hostilit y— arguing, sarcasm, personal antagonism toward other students and the instructor. Hostility may result from tensions among students, frustration or dissatisfaction with the school, or perhaps from some external or seemingly unrelatedsource. Dissatisfaction with housing or transportation problems can transfer dir ectly to antagonism for the school and even for the learning group. Attempt to alleviate external and internal proble ms. Support students who assume suitable roles. Relieve tension with small talk and humor. If nece ssary, bring the situation into the open. Remind the group of its objectives. 29.10.3. Confusion—students ask directions excessively and complain their time is being wasted. Listen to what students have to say and take the time to be especially understanding and provide needed direction. Structure will most likely be welcom ed as students want to get to work. Do not overemphasize the social aspect of the group. Th is will only postpone resolving the problem.

29.11. Student Needs. Individual roles and group behavior come as a direct response to needs of students in the group. Although psychologists may disagree on the number and nature of human needs, two facts important to the learning group are commonly agreed upon: (1) some behavior is goal directed and results from a student being internally motivated (for example, a student will study to get a good grade in a course), and (2) other behavior can be brought ab out by teachers who are aware of student needs (for example, a teacher can show how the subject matter is important to the student). Whatever the case, student needs trigger behavior, and the instructors can influence student behavior by knowing student needs. 29.11.1. Probably the best known classification of needs is one developed by Maslow. Body—basic physiological needs for food, water, sleep, and air. Safety (security)—protection from harm or injury. Belonging—social need for warmth, acceptance, approval. Self-esteem—for self-enhancement, competence, importance. Self-actua lization—for self-fulfillment, accomplishment, having arrived. 29.11.2. Maslow suggests that these needs must be satisfied sequentially. That is, once body needs are satisfied a person is concerned withsatisfying safety needs. After safety needs are satisfied, then concern is for belonging and then self-esteem needs. On ce the first four kinds of needs are satisfied, the person is concerned with becoming self-actualized.
29.11.3. Instructors who understand the direct relati onship between student needs and the roles students play in the group will be be tter equipped to adjust their own teaching behavior within the group.348 AFMAN36-2236 12 NOVEMBER 2003

More specifically, instructors will be able to find the most appropriate role or behavior for themselves in the learning group.

29.12. The Instructor As Leader. Instructors often need to assume task and maintenance roles like class member roles to help the group functi on effectively. Rather than simply play roles, instructors must vary their entire task and maintenance lead ership behavior to fit the situation. Task behavior in the learninggroup relates to how the instructor organizes and defines student roles and structures learning. Maintenance behavior is concerned with how the instructor maintains good human relations with students by using open, supportive, helping actions.
29.12.1. The amount and kind of instructor involvement relates directly to th e maturity of the group. In other words, although task and maintenance behavior by the instructor should vary with the situation, there are some predictable ways on how they should vary ( Figure 29.1. ).
Figure 29.1. Instructor Task and Maintenance Roles.

29.12.2. The reasons for adjusting leadership behavior to the situation are many, but one of the prime reasons is that students' needs change. Remember Ma slow's classification of needs described in paragraph 29.11.1. Consider the example of student seminar gr oups at a typical United States Air Force professional military education course ; it seems safe to assume that students arrive with basic needs fairly well satisfied. 29.12.2.1. High task and low maintenance. Students gather in the auditorium to receive briefings and information important to them as new students. They are told where to go, what to do, what is expected of them, and how the school is run. Emphasis is on the task and structure of the school with very little consideration shown to them as individuals. School officials engage in very little maintenance behavior. 29.12.2.2. High task and high maintenance. Students are soon assigned to se minars or small learning groups with a seminar advisor or instructor. Still there is strong emphasis on the task. Students are told about general course requirements, co mmunication skills requirements, and evaluation procedures, but the instructor also exhibits strong maintenance behavior. Students get to know each other and the instructor and begin to feel more at ease. The instructor, knowing their body and security needs are satisfied, is appealing to their need to belong. Certainly the need for task orientation and structure cannot be ignored earl y in the course when students need direction, but maintenance behavior is needed to give students a sense of belonging. 29.12.2.3. Low task and high maintenance. Students gain confidence as the course progresses. They understand what is required of them and how to meet the requirements. They also feel as if they actually belong to their home seminar learning group. The work may still be demanding, but the atmosphere is more relaxed and the group is cohesive. Students are now striving for a sense of 1. High task and low maintenance with a new group. 2. High task and high maintenance as group begins to mature. 3. Low task and high maintenance as group reaches greater maturity. 4. Low task and low maintenance for fully mature groups.AFMAN36-2236 12 NOVEMBER 2003 349

achievement—they want to do more than just be long to the group; they want to be thought highly of by the group. Recognizing the student's need fo r self-esteem and accomplishment of tasks on their own, the instructor should maintain high maintenance behavior, but reduce task behavior and say in effect, "I have confidence in you to make decisions and do the work without my looking over your shoulder."
29.12.2.4. Low task and low maintenance. When the course is nearly over and students have met the objectives, they are ready to graduate. The group will soon cease to meet, and the instructor will no longer lead. At this time, very low task and low maintenanc e behavior by the instructor is appropriate. The group has fully matured and its members will go to their assignments. 29.12.3. The really effective instructor, then, is one w ho is able to use the correct balance of task and maintenance behavior to ensure that student needs are met and maximum learning takes place.
29.13. Summary. The small learning group is an important pa rt of education. In learning groups, the instructor must consider both the content or subject matter of the group and the process or interaction among group members in order to increase the benefits and decrease the limitati ons of small-group learning.29.13.1. Every group develops norms that regulate beha vior and influence cohesiveness. In cohesive groups, students have high group loyalty and exhibit higher morale, greater productivity, and more communication among members. Group cohesivene ss can be increased by setting obtainable objectives, rewarding the group, and building group tradi tion. Cohesiveness promotes needed consensus or agreement by the group on major points. Consensus is encouraged when the instructor and students clarify the discussion, use process statements, seek different views, remain open to different views, and use group pronouns. 29.13.2. Students play roles in the learning group. Nonproductive student roles such as monopolizer, arguer, nonparticipant, and clown, and group problem s of apathy, hostility, and confusion need to be dealt with by the instructor. Attention to student ne eds will help the instructor understand student roles and be able to assume the proper balance of task and maintenance behavior needed to promote a productive learning group.350 AFMAN36-2236 12 NOVEMBER 2003

Chapter 30

THE INSTRUCTOR AS A HELPER

Section 30A—Overview

30.1. Introduction. Effective counseling can improve student performance. Since few Air Force schools have full-time counselors, Air Force instructors should be prepared to help students in this regard when the need arises. This chapter will assist you in meeting this responsibility. 30.1.1. One purpose is to help people make wise choi ces and decisions; the second is to help people be better adjusted or to promote their mental he alth. When discussing Air Force academic counseling, our emphasis is on helping students improve or change their behavior by making wise choices that will help their academic progress. The second purp ose, to promote adjustme nt or mental health, should be left to professionals in the field of mental health. 30.1.2. In discussing counseling, we will first define the term and classify student requests for assistance. Next, we will examine the helping relations hip so important in effective counseling. Third, we will discuss techniques and practices taken from three different counseling approaches. Paragraphs 30.16. and 30.17. provide direction for referring students wh en the need arises and legal guidance concerning confidentiality of information in a military counseling setting.
30.2. Counseling Defined. There are many definitions of counseling, but most of them do not seem appropriate to counseling students in military school s. By extracting three common characteristics from the various definitions, however, we can define couns eling as (1) a process (2) involving a helping relationship (3) directed toward improve ment or change in student behavior. Let us examine these characteristics in more detail.
30.3. Counseling as a Process. Counseling may be viewed as a three- phase process or cycle of helping.
30.3.1. Self-Exploration. In the first phase, the instructor directs efforts toward establishing a base or building a good relationship with the student. The stude nt is allowed to explore the problem in depth. The instructor uses listening skills, proper res ponses, and effective questioning techniques, along with empathy, respect, warmth, and genui neness in building the relationship.
30.3.2. Self-Understanding. Once the foundation for the relationship is established, the instructor helps the student gain self-understanding. The instru ctor assists the student in making some sense out of the many pieces of the puzzle. Although students think about their problems a great deal, they frequently are unable to change their behavior. A student may recognize that nightly visits to the club interfere with academic preparation and yet persist in this behavior. What is missing is commitment to change. Therefore, in this phase the student must not only understand the problem in depth, but make a commitment to follow through with a plan or progr am designed to correct the inappropriate behavior.
30.3.3. Student Action. The final phase may be the most difficult. The instructor and student must devise a plan the student can follow to resolve his or her problems. The instructor and student consider alternate plans and the possible consequences of th ese plans before selecting one. The emphasis is on outcomes and achievement of attainable goals.AFMAN36-2236 12 NOVEMBER 2003 351

NOTE: These three phases illustrate counseling as a process. At the core of this process is the helping relationship.
30.4. The Helping Relationship. The second characteristic of c ounseling—the helping relationship— refers to interactions in which the instructor makes a determined effort to contribute in a positive way to the student's improvement. While shoe clerks, policem en, bankers, and politicians all provide help, the term "helping relationship" is not used to describe their services because their primary purpose is not to assist in the personal development of the individual. In counseling, we establish a helping relationship by drawing upon ourselves in ways that help students live more in harmony with themselves and others, and with a greater understanding of themselves. The relati onship develops because students need assistance, instruction, or understanding. The helping relationship is so very important.
30.5. Improvement or Change in Student Behavior. The third characteristic of counseling, improvement or change in student behavi or, describes the goal of counseling. Counseling can be considered successful when some kind of change or improveme nt occurs in a student's observable behavior. 30.5.1. After counseling, it is not enough for a student merely to speak of renewed confidence in, for example, lecturing before a group. Such a comment does not allow us to be sure our counseling was successful. However, if the student gets up in frontof the class and presents the lecture with a new degree of confidence, then we have observable evidence the behavior has changed or improved. 30.5.2. Or consider a student who could not perform satisfactorily on tests because of a high anxiety level. Observable evidence afte r counseling intervention would be a report that the student took several tests without experiencing anxiety that interfered with test performance. While we cannot say with certainty that the change was the result of counseling, it is quite probable that counseling intervention contributed to the improved performance.
30.6. Student Requests for Assistance. Students will come to us asking for information and action or understanding and involvement. If a student asks for information, we provide the appropriate verbal response. When the student asks for a handout from our lecture, we take the appropriate action to get it. Neither of these interactions would be classified as counseling. But when students verbalize a request for understanding or involvement, they are generall y seeking a helping relationship and counseling is involved. Most students have occasions when they wi sh to talk with others about matters important to them. As instructors, we must be alert for these needs even when they are not verbalized. With this type of request, we must be ready to enter into a helping relationship.
Section 30B—The Helping Relationship
30.7. Importance of the Helping Relationship. All theories and approaches to counseling stress the relationship between participants as the common ground for the helping process. Such a helping relationship allows the instructor to contribute in a positive way to the improvement of student performance. We often speak of the counselor's role as facilitating the change. Because th e helping relationship is so significant in effective counseling, you need to clearly understand its key components.
30.8. The Counseling Environment. The setting in which counseling is conducted has some bearing on whether the relationship will be facilitated or thwarte d. Our goal is to provide the atmosphere that will contribute most to communication.352 AFMAN36-2236 12 NOVEMBER 2003

30.8.1. Privacy. One of the most important physical consider ations is privacy. Students desire and have a right not to be seen or heard by other students, instructors, or others when they enter into a counseling relationship. Nothing will limit the relati onship more quickly than knowing others are able to hear or see what is taking place.
30.8.2. Seating Arrangement. Little research has gone into the ef fect of seating arrangement on the helping relationship. Available research suggests that a seating position across the corner of a desk works well. Another arrangement is comfortable chairs placed close to each other at a 90-degree angle with a small table nearby. You might find that a chair-facing-chair arrangement works better for you. Most students should be comfortable with distances of approximately 30 to 40 inches between the counselors and themselves.
30.8.3. Limited Interruptions. The instructor must establish rapport and build trust. Outside interruptions can only hinder. Phone calls, knocks on the door, people who want "just a word" with you, and secretaries who must have you sign this document "at once" may destroy in seconds what you and the student have tried hard to build over a period of time.
30.8.4. Student Folders. Information in a student's folder can be useful in preparing for a counseling session. However, place the folder in a drawer or removed it from view before the student arrives. Otherwise, the student may become more concerned w ith what is in the fold er than with discussing the problem at hand. Be careful not to prejudge students based on information in the folder or to expect certain attitudes, behavior, or achievement from students based on written comments of other instructors. To properly help students, rema in objective in your relationship with them.
30.8.5. Confidentiality. Students seeking help from instructors may reveal intimate, personal, and sometimes painful details and experiences. Becaus e such highly personal and private revelations may bring embarrassment or ridicule, students do not want them disclosed. They usually assume others will not have access to their disc losures without their express consent. When a student enters counseling under this assumption, a confidential relationship exists and the instructor is normally obligated tomaintain this confidentiality. However, in addition to the ethical considerations, there are legal implications involved. Paragraph 30.17. outlines your legal rights and responsibilities concerning confidentiality when counseling in a military setting.
30.9. Stages of a Helping Interview. We might think of the actual counseling interview in terms of stages labeled initiation, development, and closing. Earlier we discussed the de velopment stage in terms of self-exploration, self-understa nding and commitment, and action. Let us examine initiation and closing in more detail.
30.9.1. Initiation. During this stage you should work to establish rapport. You might focus upon some neutral topic or event known to you and the st udent. However, avoid trite conversation that may result in an increase in strained feelings, such as "It' s a nice day, isn't it?" Being friendly, attentive, and demonstrating interest to reduce student resistan ce are important, but don't make the mistake of consuming valuable time with extended idle conversation implying that th e weather, sports, or activities are what the student wants to discuss. 30.9.1.1. Keep any opening conversation brief. Fo cus on the student's reason for being there. Statements such as "Please tell me what you wished to see me about," "I understand you wanted to see me," "I've asked you to come in because...," or "Please feel free to tell me what's on your mind" should get you off to a good start. Try avoi ding the use of "May I help you?" and "What isAFMAN36-2236 12 NOVEMBER 2003 353

the problem you would like us to discuss?" The words "help" and "problem" may hinder development of the relationship. 30.9.1.2. Initiate counseling appointments on time or provide a good and sufficient reason why the appointment was not met. Tell students how mu ch time you have available for the counselingsession. This procedure provides an important framework for the interview and allows the students to orient themselves. If available, 30 to 45 minutes is generally sufficient for counseling sessions. However, few instructors will have the luxury of this much time. Structure the interview according to the time available.
30.9.1.3. Initiation ends when the instructor and student understand what is to be discussed and agree that it should be. Although the focus may shift to a more primary concern as the interview progresses, you have agreed to discuss a point and are ready to proceed to the development stage.
30.9.2. Closing. Both partners in the interview should be aw are that closing is taking place. No new material should be introduced or discussed during this stage. (If such information does come out, schedule another counseling session.) A concluding statement by the instructor may suffice. Or a short summation of the session with the agreed-upon course of action by either the instructor or the student can effectively close out the session. Closing is esp ecially important because what occurs during this last stage is likely to determine the student's impression of the interview as a whole.

30.10. Instructor Attitudes. The essential components of the help ing relationship are attitudes rather than skills. For this reason, it has been difficult to analyze what good counselors do to establish a helping relationship so others can be trained to do likewise. During a counseling session, instructors must communicate the attitudes of acceptance, understanding, and sincerity by their actions, words, gestures, and facial expression. The main skill instructors must de velop is that of communicating this understanding of what the student is trying to express, and doing it in a warm, sincere way.
30.10.1. Acceptance (Warmth). Different authorities use different terms to describe this attitude or core condition of counseling. Some call it "uncond itional positive regard" while others use the term "nonpossessive warmth." Primarily nonverbal, this in structor attitude is a direct outgrowth of the instructor's ability to appear nonjudgmental. It invol ves a willingness to allow students to differ from one another and the realization that the experiences of each student are a complex pattern of striving, thinking, and feeling. The student experiences accep tance as a feeling of being unconditionally understood, liked, and respected. Acceptance —a positive, tolerant attitude on the part of the instructor—is one of the conditions that enables the student to change behavior.
30.10.2. Understanding (Empathy). Counseling is basically a perceptual task. Learning to say the right thing at the right time without learning to lis ten, watch, and understand is not possible. While listening, instructors must put themselves in the student 's place and try to see the circumstances as the student sees them—not as they look to an outsider. While simple to say, this attitude is hard to learn. The first step in communicating this attitude is to listen carefully to what students say about how they feel because of what is happening to them. The next step is to think of words that represent the students' feelings and their situations. Finally, use wo rds to tell students you understand their feelings and their situations. For instance, if a student expresse s concern about a relationship to an instructor, you might respond with empathy by saying, "Underneath your frustration, I can sense a great deal of anger. You're giving 100 percent and instead of a ppreciating your effort, your instructor expects even more. That's pretty hard to take."354 AFMAN36-2236 12 NOVEMBER 2003

30.10.3. Sincerity (Genuineness). With this attitude, essential harmony exists between what the instructors say and do and what they really are. Sincerity is the opposite of phoniness. The instructor cannot role-play being a counselor. Each of us can sense from the tone of voice, gestures, or mannerisms of some of the actors used in TV commercial s that they are playing a role. They are putting on a facade in saying things they don't feel. Thus, with sincerity you mean what you say and say what you feel, keeping in mind that you are trying to be helpful to the student.
30.11. Interviewing Skills. The following interviewing or communication skills can help the instructor develop a helping relationship with students:
30.11.1. Attending Behavior. These skills pertain to the physical behavior you exhibit while listening to another person. Your posture, eye contact, and facial expression carry messages to the student while you communicate. Effective attending skills commu nicate you are interested. If the skills are ineffective, it is doubtful a helping relationship will develop. 30.11.1.1. Effective attending behaviors show acceptance and respect for students. Eye contact is regular; movement is toward the students and not away. Your posture should be relaxed, leaning slightly toward the students. Match your facial e xpression with your feelings or the students' feelings. Keep the voice clearly audible—neither t oo loud nor too soft. Maintain a high energy level so you can stay alert throughout a long conversation. 30.11.1.2. Effective attending skills can be devel oped with practice alt hough your initial efforts may seem mechanical. If you work hard at projecting acceptance, understanding, and sincerity, you will find good attending skills developing quite naturally.
30.11.2. Noting Nonverbal Behavior. Student nonverbal responses are more likely to transmit the real message than the spoken one. Facial expression alone can trans mit more than half the meaning of a message. Nonverbal responses are automatic; student s are not generally aware of them. However, do not fall into the trap of believing a certain nonve rbal response can mean only one thing. Arms folded across the chest may mean defensiveness, but they may also mean the person is comfortable in that position or is cold. Nonverbal behavior also varies from culture to culture. Therefore, nonverbal behaviors must always be judged within the context of what is happening in the counseling session and their meaning considered tentative. In couns eling sessions, the students' nonverbal behaviors provide clues to possible underlying feelings or mo tives but do not serve as proof they exist.
30.11.3. Questioning Techniques: 30.11.3.1. Questioning is a common and very overused technique. Used improperly, it tends to make students see the counseling session as an in quisition in which they can sit back and think along the lines indicated by the questions. One quest ion usually leads to another and before long the instructor is searching for additional questi ons to ask. Or the instructor may ask questions accusingly, arousing fear and suspicion on the part of the student instead of cooperation.30.11.3.2. Used properly, questions can solicit needed information and direct student conversations to helpful channels. When used, the questi ons should be open-ended and require more than a "yes" or "no" response. For instance, ask "How did you feel after the test?" instead of "You felt great after the test, didn't you?" Avoid double-barreledquestions such as "Are my questions helpful, and are you learning more about yourself?" Us e indirect questions wh ere possible since they inquire without seeming to do so, such as "You've been here at the school a week now. There mustAFMAN36-2236 12 NOVEMBER 2003 355

be a lot you want to talk about." Another exam ple would be, "You must have many thoughts about our new grading system."
30.11.4. Responding. Responding skills allow you to continue communicating with the studentswithout relying on questions. Through practice, although they may seem mechanical at first, instructors can learn to use the following responding skills:
30.11.4.1. Restatement of Content. Restating the content is an attempt to convey understanding either by simply repeating or rephrasing the communication. In restating, you make no attempt to clarify, interpret, or organize what the student has said.
30.11.4.2. Reflection of Feeling. Here, the instructor's intent is to show that student feelings or experiences are correctly understood. The focus is on student attitudes. Reflection techniques bring student feelings and attitudes to the surf ace. These techniques also bring problems into awareness without making a student feel pushed by the instructor. The instructor mirrors studentattitudes so they can be clarified and understood. When properly achieved, reflection helps students feel deeply understood and clarifies student th inking so the situation can be seen more objectively. For instance, a student might say, "It's so hard knowing my wife is in the hospital while I'm in school and there is absolutely nothing I can do." To reflect feeling, you would respond, "You feel anxious and helpless right now."
30.11.4.3. Reinforcing Responses. The "mm-hm" response accompanied by a nod of the head is one of the most common reinforcing responses. T hough not a word, it is a clearly uttered sound. When used, it indicates "Go on. I'm with you. I'm listening and following you."
30.11.4.4. Silence. Silence as a response can be difficult to master. We might consider two times when it is especially appropriate. A pause may come because students are thinking over what they just expressed and interruption would be inadvisa ble. In another instance, silence may fall just after an emotion-laden expression by students; qui et acceptance of this pause is appropriate.
30.11.4.5. Clarification. Clarification is anothe r important response. Perhaps you simplify what students have said to make it clearer or you try to verbalize ideas and feelings students have had difficulty expressing clearly. When you clarify what students have said or tried to say, you are using this skill. At other times, you may need to have students clarify some thing they said to you. In this situation, an appropriate response may be a question such as "I'm sorry. I didn't quite understand your role in the argument. How did it get started?"
Section 30C—Counseling Approaches

30.12. Overview. Instructors who do even a moderate amount of counseling should give serious thought to the approaches they use. Thought put into the sele ction of an approach should culminate in a theoretical base from which to operate. While th e theoretical base need not be a fully developed, sophisticated theory of the counseling process, it should provide a structure or framework from which to operate. Without that framework or structure, instructors may find th emselves floundering during counseling sessions with vague goals and no means of achieving them. 30.12.1. Once developed, a theoretical base will allow you to make comparisons between the body of unique information supplied by the student in couns eling and a larger body of generalizations you have about human behavior. Because you must make predictions alone or with the student about the effects of each course of action available to th e student, theory provides structure for making appro-356 AFMAN36-2236 12 NOVEMBER 2003

priate selections. A theoretical base can also te ll you what facts you lack to achieve understanding ofthe student. Finally, it can help you identify reasons for your success or failure as you counsel students. 30.12.2. Counseling approaches or theories might be categorized in many ways. The behavioral approach might represent one group, cognitive approaches a second, and affective approaches a third. While you may not have the required training or need to fully understand and use these theories, the techniques and practices from them may be useful in counseling students and in developing your own counseling approach. You may find your self drawn to an approach be cause of your own view of people and how they should behave. As you become more knowledgeable, you may find that proper selection of techniques and practices from the differ ent approaches increases the probability of giving appropriate help to your students. With experience, you may develop a theoretical approach to assist you in giving the best possible help in the counseling situation. To simplify our discussion we will select one theory from each approach for development.
30.13. Cognitive Approach. Rational-emotive counselors view peopl e as being capable of both rational and irrational (illogical) thinking. A major element in th eir theory is that our emotions are caused and controlled by our thinking. What we think becomes our emotion. 30.13.1. Particularly important to this viewpoint is the idea that much of our emotional behavior stems from "self-talk" or internalized sentences. What we tell ourselves is or becomes our thoughts and emotions. Thus, our negative emotions such as fear, anxiety, embarrassment, and shame comefrom our thinking process. Simply stated, if we can learn to control our thinking, we can learn to control our emotions. 30.13.2. Let us consider an example. Before students get ready to give speeches, they start talking to themselves. The students picture all the things thatcan go wrong and anxiety starts to build. They continue to picture the worst possible outcomes and the anxiety increases. Consequently, they then have difficulty performing because of their own irrational thinking that triggered the negative emotion of fear or anxiety. 30.13.3. Another common example occurs among student s at teacher training schools. The thought of getting up in front of a group ofstudents and teaching a lesson produces negative thoughts. The students start repeating the things that might go wrong and fear or anxiety starts to build. This negative self-talk continues. If they got up to teach and fa iled, the experience would be humiliating. If they could not teach the lesson, they would probably fail at the school. If they fail at the school, their entire military career will be affected. If their career starts going badly, the wives and children will be affected. This catastrophic thinking continues until th ey have worked themselves up into an emotional state that interferes with their school performance. 30.13.4. Counselors who borrow from rational-emotive theory would explain to the students how such self-talk brings on the very thing they are trying to avoid. They try to get the students to view the situation realistically. What is the probability thatthe worst possible thing will occur? What if they should fail the course? It might be in their best interest to know they should not go into a field for which they are not suited. By focusing on the faulty reasoning, emphasis shifts to eliminating self-talk that stands in the way of student progress and caus es negative emotions they do not have to experience. The brain focuses on one thing at a time. If students change their negative thoughts for positive ones, they can learn to control their emotions.AFMAN36-2236 12 NOVEMBER 2003 357

30.13.5. At other times students can make themselves miserable through their negative self-talk, telling themselves how much they hate being at your school or attending your cour se. By learning to view the situation realistically and cutting out the negativ e self-talk, students can accept the situation and make the most of the experience.
30.14. Behavioral Approach. Behavioral counseling has many applications in the military academic setting. According to behaviorists, most human behavior is learned and is therefore subject to change. If we can change the students' environment, we can assi st in altering their behaviors. Our effectiveness in counseling and the outcome of the counseling can be assessed by changes in specific student behavior outside the counseling setting. 30.14.1. Basic to behavioral counseling is the principle of reinforcement. Rewards such as praise, affection, grades, and awards are given for appropriate behavior in hopes of getting students to repeat this type of behavior. For instance, if a studentis commended for a good class presentation, this reinforcement should motivate the student to prepare we ll for the next lesson in hopes of again receiving recognition (called positive reinforcement). 30.14.2. At other times, a certain student response st ops disapproval, criticism, or nagging from others. Thus, this behavior is reinforced by the rem oval of the disapproval, criticism, or nagging, and the particular response should occur again. For instance, a student does not get al ong well with other class members and is not accepted by them . After counseling, the student re sponds to class members differently and disapproval stops. This new form of behavi or has been reinforced and should continue to be used by the student (called negative reinforcement).
30.14.3. Student behavior that is not reinforced will be eliminated over a period of time. If students are no longer called on when they raise their hands , this form of behaviorwill eventually be eliminated in that classroom. Students who become overl y dependent on an instructor will usually stop this practice if the instructor does not provide reinforcement for this type of behavior. 30.14.4. What we do and say as instructors serves to reinforce appropriate or inappropriate behavior. If we nod our head in counseling and say "mm-hm," we are giving positive reinforcement to what students say or actions they have ta ken. This action encourages the stude nt to continue. But if we do not like what students say or actions they propose, we should avoid reinforcement with head nods and our responses. 30.14.5. When using behavioral techniques in counse ling, we should help students outline a course of action during which we know a good chance exists for them to receive positive reinforcement. Thus, there is a good chance for a positive change in behavior. However, if we outline a course of actionwhere the probability of failure is high, then student behavior may get worse or show no improvement.
30.15. Affective Approach. One affective (or humanistic) approach is client-centered counseling. Thisapproach stresses student ability to determine the issues important to them and to solve their own problems. Client-centered counselors see students as basi cally good and behavior as being purposeful or goal directed. If placed in the proper counseling relati onship and given the opportunity, students will choose courses of action that will lead toward actualization of their potential. The instructor's primary responsibility is to create a helping relationship in which students can gain self-unders tanding and self-acceptance. Enhancement of the self-concept and actualization of potential are at the core of client-centered counseling.358 AFMAN36-2236 12 NOVEMBER 2003

30.15.1. The client-centered approach might be especi ally effective in situations where students are grappling with personal conflict. Consider, for example, situations in which students must decide to stay in school and complete a course or withdraw to return home to solve or deal with a difficult family problem. Either option is feasible from an administrative standpoint. From your point of view, the student should return home, resolve the problem, an d return to complete the course at a later time. However, no one knows the student and the total situ ation better than the student; consequently, only the student can make the most reasonable choice. 30.15.2. In these cases, for example, the self-con cepts of the students may include attitudes about themselves that preclude their being quitters or leaving commitments unfulfilled. What seemed to you the wisest choice would not necessarily be the bestone from the internal vantage point of these students. Your task as a client-centered instructor-couns elor is to establish a helping relationship in which students know they are accepted and free to make choices without fear of losing your regard and understanding. In this helping relationship, students will generally c hoose the course best for them. 30.15.3. Many military instructors will be dealing with immature students who have never taken responsibility for their own behavior or actions. W ith these students, the client-centered approach is often particularly effective. If certain student s consistently ask your opinion on how to handle problem situations, work at establishing a helping relationship with them. Give them the responsibility to resolve the problem situation. If they are successful , also give positive recognition as a form of reinforcement. 30.15.4. Your own view of people may be similar to that of the client-centered counselor. If so, you will probably be comfortable working at establis hing a strong helping relationship with students and allowing them to solve their own problems. When establishing the helping relationship, remember that in the affective approach it is the attitudes of the counselor rather than techniques that facilitate counseling. Thus, with the client-centered approa ch, your focus will be on the students—not on you as the instructor. You should see a decreasing need fo r your assistance as students mature in problem solving.
Section 30D—Referrals and Communications

30.16. Referral. Referral is the process of finding someone else to help the student. As teachers, we must recognize when the needs of a student call for a ssistance beyond the scope of our training. Occasionally we may encounter students w ho have problems that are not educati onal or which call for specialized service. Referral to a professionally qualified person is more appropriate than attempting to help solve the problem. Marriage and family concerns, drug and alc ohol abuse, and emotional disturbances brought on by stress are common problems instructors encounteramong members of their student body where referral is appropriate, or students may need specialized assistance from the legal office, social actions, or another base or community agency. Even though the pr oblem may not be serious, referral action is appropriate in all of these cases because few Air Force instructors have the required education and training to help students with these concerns.
30.16.1. Instructors should be familiar with the referr al possibilities before suggesting this course of action to the student. What kind of special service doe s this person require? Is it available and, if so, where? First assess the resources of your own school . Perhaps you have staff members who have the professional training required to deal with a partic ular concern. Second, determine the extent of services available on your base. A base mental health c linic may be able to handle a variety of problemsAFMAN36-2236 12 NOVEMBER 2003 359

because of the diverse background of the professi onals assigned. The drug and alcohol abuse unit of your social actions office can give specialized assi stance and one or more of the base chaplains may have been trained as counselors. A variety of comm unity services is also av ailable to military personnel and should not be overlooked as referral possibilities. 30.16.2. When referral becomes a consideration, do not hesitate to discuss the situation with your supervisor or someone on your faculty who has professional training. You may need to call base or community agencies for their recommendations.30.16.3. A student may show resistance, doubt, fear, guilt, or defensiveness when you suggest referral. Be prepared to cope with these reactions. When suggesting referral, communicate your acceptance, understanding, and concern just as you did in establishing the helping relationship. Offer to make the contacts and arrangements with the referral resource and assist in any way possible. Arrange for a followup with either the agency or the student to determine if you need to help further. 30.16.4. When suggesting referral, students may indica te they would prefer making their own contact with the referral source. If so, attempt to arrange a followup procedure to determine how you might be of further assistance. The student may also reject the referral attempt. If so, do not interpret this reaction as a failure on your part. The student may now be more openly facing the problem as a result of the counseling intervention or the student may have decided upon another course of action.

30.17. Communications Between Instructors and Students in a Counseling Environment: 30.17.1. The military does not recognize any legally enforceable "privileged communication" between a counselor and counselee (i.e. instructor and student), as is the case in attorney-client, chaplain-penitent, physician-patient, Military Equal Opportunity-client, or spousal privilege. For further information, see The Military Commander and the Law , 4th edition, 1998. A promise of confidentiality by a counselor cannot override the legal requireme nt to report crimes, and a counselor who is told of a criminal act committed or about to be committed must report this information or be personally subject to criminal prosecution. The breach of confidentiality, in and of itself, does not give rise to grounds for legal action in libel or slander. Further, the counselor has an absolute privilege from libel or slander actions when testifying in court or before an administrative body.
30.17.2. Counselors should consult with their loca l documentation management office and judge advocate officials to resolve any uncertainty.
30.18. Summary. Effective counseling can improve student pe rformance. Therefore, Air Force instructors should be prepared to counsel students when the need arises. 30.18.1. Definitions of counseling stress that counse ling is a process involving a helping relationship directed toward improvement or change in studentbehavior. The purpose of counseling in the Air Force academic setting is to assist students in making wise choices and decisions. 30.18.2. All theories and approaches to counseling st ress the relationship between participants as the common ground for the helping process. A helping rela tionship allows the instructor to contribute in a facilitating, positive way to the improvement of st udent performance. Establishing a proper environment for counseling, developing the interview by stag es, projecting proper attitudes in the session, and using appropriate interviewing skills are importantconsiderations in establishing a helping relationship.360 AFMAN36-2236 12 NOVEMBER 2003

30.18.3. Once fundamentals of the helping relationship are mastered, instructors can give serious attention to developing a counseli ng approach. Techniques and practices from different theories can be useful in counseling students and developing a counseling approach. With experience, the instructor may develop a theoretical approachthat will assist in giving the best possible help in the counseling situation. 30.18.4. Instructors must recognize that the needs of students occasionally call for assistance beyond the scope of their training in counseling. Referral is the process of finding someone else to help the student at such times. Instructor s should familiarize themselves wi th sources of assistance both on base and in the civilian community. 30.18.5. Confidentiality of communications between in structors and students in a counseling relationship is essential. However, there are instance s in military settings when confidentiality cannot be maintained. Instructors should contact the neares t base legal office if guidance is needed.AFMAN36-2236 12 NOVEMBER 2003 361

Chapter 31

SELF-CONCEPT

31.1. Introduction. Educators and psychologists are becoming in creasingly aware that the self-concept of students is closely connected to how they learn a nd behave. Research evidence indicates that lack of academic involvement, misdirected motivation, and low performance in academic subjects characteristic of underachievers, failures, dropouts, and the cultura lly disadvantaged may be due in part to negative self-concepts. These students have difficulty in the classroom because they do not believe they can do academic work. When students say, "I'll never pass this test I just know it," they are reflecting how they feel about themselves. With this attitude, chances are increased they will not do well on the test. Thus, performance depends not only on how intelligent they are, but also on how intelligent they think they are. 31.1.1. Current literature and research also suggest a close link between self-concept and mental health. Such evidence leaves little doubt that the mental health of individuals depends deeply on the quality of their feelings about themselves. Just as individuals must maintain healthy views of the world around them, so must they learn to perceive themselves in positive ways. 31.1.2. Recognizing the importance of self-concept, in structors should also realize that the self-concept of students can be changed, for better or wors e, through their experiences in the academic environment. While successful experiences in scho ols are no guarantee of a generally positive self-concept, they do increase the probability thatsuch will be the case. In contrast, unsuccessful experiences in school show that these individuals will develop negative acad emic self-concepts and increase the probability of their de veloping negative overall self-concepts. 31.1.3. In this chapter we will describe important fa cets of the self-concept, explain certain environmental forces that have shaped the academic self-c oncepts of students, summarize the role of defense mechanisms in protecting the self-concept, and identi fy actions an instructor can take to enhance students' self-concepts.
31.2. Description of the Self-Concept. All of us have mental blueprints or pictures of ourselves. The blueprint is composed of a system of interrelated ideas, attitudes, values, and commitments influenced by past experiences, successes, failures, triumphs, humiliations, and the way others reacted to us especially during our formative years. Consequently, we arrived at more or less stable frameworks of beliefs about ourselves and have proceeded to live in consistent manners within those frameworks. This framework is known as the self-concept, although the terms self-p erception, self-image, and se lf-structure are often used to describe the beliefs. Thus, we act like the people we conceive ourselves to be. In fact, it is difficult to act otherwise even with conscious effort and willpower. 31.2.1. Once established, the mental blueprint or pi cture of ourselves provides a screen through which everything else is seen, heard, evaluated, and unde rstood. The likelihood is increased that our behavior will cause others to respond toward us in ways that validate and support our self-concepts. If the self-concept is generally positive, it engenders self-respect and confidence, while a generally negative self-concept leads to feelings of inadequacy and a lack of confidence. 31.2.2. The self-concept may also be thought of as a th eory we have about ourselves. Just as we use data from our experiences to make inferences about the world and th e people with whom we interact, we also make inferences about ourselves by observing our own behavior and the reactions of other362 AFMAN36-2236 12 NOVEMBER 2003

people to us. Our self-theory affects our behavior; our behavior in turn affe cts our self-theory—a theory continually being tested and revised. 31.2.3. We might speak of a student's self-concept in te rms of levels in a hierarchy. At the top is the general self-concept representing a se t of beliefs students hold about th emselves. At the next level are three major areas of self-concept for students: acade mic, social, and physical. Finally, we come to specific areas of self-concept related directly to a ki nd of activity or subject matter area. These specific self-concept areas are probably the easiest to change. Under academic self-concept would be specific areas such as mathematics, English, and history ab ilities. Under social self-concept would be such areas as friendship, dating, and dancing. The physical self-concept would include athletic and appearance self-images. Successes or failures of students directly affect specific self-concepts. The pluses and minuses in specific areas affect the major areas of self-concept, which, in turn, affect the overall self-concept. 31.2.4. With this knowledge of what the self-concept is, let us turn to an explanation of environmental forces that may have shaped the self-concepts of students we will be dealing with in the academic environment.
31.3. Formation of the Academic Self-Concept. While instructors should strive to improve the general self-concepts of students, their best chance for success is in improving the scholastic or academic self-concept. Our appreciation of a student's academic self-concept willbe enhanced if we understand the role parents, teachers, peers, and the schoolhas played in shaping that self-concept. 31.3.1. Parental feedback and evaluation, caring or lack of it, and acceptance or rejection have a striking effect on how students view their general in tellectual abilities and abilities in specific subject areas. Research suggests that parents who comb ine caring, acceptance, and high, but reasonable expectations are likely to rear children who think well of themselves and who strive to do as well as they can in school. Parents who pay no attention at al l to the academic accomplishments or failures of children, or who attend only to the children's failures and rarely to their successes, are contributing to the development of a negative academic self-concep t. Thus, the type and amount of reinforcement coming from parents has a significant effect on children's academic achievements and their academic self-concepts. 31.3.2. Students who complete secondary educati on have devoted nearly 20,000 hours to school and related activities. During this lo ng period of time, which includes some of the most impressionable stages of a person's development, students are being taught the curriculum of the school and another curriculum based on interactions of persons within the school. The latter curriculum teaches students who they are in relation to others and their places in the worlds of people, ideas, and activities. Theymay learn the latter curriculum more slowly, but it won't be forgotten as easily as the school curriculum.31.3.3. Not only does the school provide the stage upon which most of the drama of a person's formative years is played, it houses the most critical audien ce in the world—teachers and peers. It is here, in the face of their most severe critics that students, whether in kindergarten, graduate school, or Air Force schools, are likely to be reminded again and ag ain of either their failings and shortcomings or of their strengths and possibilities. 31.3.4. Teachers play an enormous role in the deve lopment and change of the self-concept. Research shows that students consider teache rs to be "significant others" in their lives. What they say or do isAFMAN36-2236 12 NOVEMBER 2003 363

important to the students. Teachers dispense rewards, punishment, acceptance, and rejection on a colossal scale. 31.3.5. Peers also play an important role. By admiring those who succeed in school or seeking their assistance with problems, peers provide positive reinforcement for the academic self-concept. Similarly, when peers ridicule a classmate or point outdeficiencies, they are contributing to the development of a negative academic self-concept. 31.3.6. While students may enjoy a few hours in which they are not judged (relative to others) byteachers, peers, family, and others, there are few school hours in which they are not judging themselves against the standards set by themselves, the teachers, peers, and their families. At no other time in their career as workers, members of families, ci tizens, or persons engaging in leisure activities will they be judged so frequently by others and possibly by themselves. These relative judgments arise because almost all of a student's school learning occurs as a member of a group. Judgments are made frequently because schools have stressed competition for so long as a primary motivational technique. Relative rather than absolute norms are the basis for most judgments. 31.3.7. Because of the sequential nature of learning ta sks and their consistency from year to year, the student tends to remain in much the same position rela tive to other students. This results in stability of school marks and test perf ormance. In general, the more public and official the judgments (or marks) the greater the effect they are lik ely to have on the student's perception of adequacy in the subject. Sooner or later such students are forced to accept some judgments about their capabilities with groups or learning tasks.31.3.8. The student gradually acquires a consistent performance as tasks accumulate in large numbers. If performance has been adequate, the student approaches the next task with confidence. If performance has been inadequate, the student comes to believe in his or her inadequacy with respect to this type of learning. 31.3.9. The student approaches the next task with marked reluctance. If the experience was painful enough, the task will be avoided or at least approached with little enthusiasm or marked dislike. Thus, the student's personal feelings about a subject or set of learning tasks are highly influenced by their perceptions of their adequacy or inadequacy with such tasks. Generally, adequacy or inadequacy for most students is defined in terms of their standing in the upper or lower portions of the local distribution of marks. 31.3.10. Success or adequacy in an academic subject opens it up for further consideration and use. Failure in an academic subject may effectively clos e this subject for further consideration. As various indices accumulate, students gradually begin to ge neralize about their adequacy or inadequacy with school learning tasks. If they have positive experience s, they are likely to develop a generally positive view about school and school learning. If their experi ences are negative and the students, their teachers, and the parents regard learning as inadequate , then they are likely to develop a negative view about school and school learning. Given a suffic ient number of unsuccessf ul learning experiences, nearly everyone would succumb to an acceptance ofa negative or inadequate self-view about learning. 31.3.11. Thus, we might conclude that the academic self-concepts of students in our classrooms were formed primarily during the critical years in el ementary and junior high schools. The frequency and consistency with which they received positive or negative feedback from parents, teachers, and peers largely shaped the academic self-concept th ey bring into our Air Force classrooms.364 AFMAN36-2236 12 NOVEMBER 2003

31.3.12. Once the self-concept is formed, students will attempt to defend this concept of self throughthe use of defense mechanisms. These mechanisms will influence behavior of students in our classrooms.
31.4. Self-Concept Defense Mechanisms. Whether we are aware of it or not, all of us use certain defense mechanisms to preserve or protect our self-c oncepts. In fact, our succe ss in meeting daily stresses and strains of living can be traced to our effectivene ss in using certain defenses. However, if we use them to avoid assuming responsibility, to avoid risk taking , or to develop excuses for immature or self-defeating behavior, then they can have a negative effect on our behavior. Their use is normal unless they begin to interfere with the maintenance of self-esteem rather than aiding it. Defense mechanisms can best be understood in view of the objective they serve, which is to safeguard the integr ity and worth of the self. Some of the most common in the academic environment follow:
31.4.1. Compensation. Here we attempt to disguise the pres ence of a weak or undesirable quality by emphasizing a more positive one, or reduce tensi on by accepting and developing a less preferred but more attainable objective for a more preferred bu t less attainable objective. Students who regard themselves as unattractive may develop exceptionally winning personalities to compensate. Students may say they would rather spend their evenings working on lesson plans than anything else they can think of when they would rather be with a person of the opposite sex who has resisted their dating requests.
31.4.2. Projection. With this mechanism we relegate the blame for our own shortcomings, mistakes, and transgressions to others or attribute our own mo tives, desires, characteri stics, and impulses to others. The athlete who fails to make the team may feel sure the coach was unfair, or the tennis player who examines the racket after a missed shot is projecting blame. When students say, "Everybody will cheat on an exam if given the chance," they are projecting.
31.4.3. Rationalization. Rationalization aids in softening disappointment when we don't reach our goals and helps us come up with excuses when we do something we believe we shouldn't but want to anyhow. Heavy smokers rationalize when they say, "The relationship between cancer and smoking isn't conclusive, and if we should get it in the future a cure will be discovered by then." Other students rationalize when they say, "We had to cheat because others were and we might not have passed the course otherwise," when in reality they didn't know the material.
31.4.4. Denial of Reality. We can often ignore or refuse to acknowledge disagreeable realities. We turn away from unpleasant sights, refuse to disc uss unpleasant topics, deny criticism, and become so involved with work that we don't have to deal with marital, child-rearing, or other personal problems.
31.4.5. Reaction Formation. Individuals sometimes protect themselves from dangerous desires by not only repressing them but actually developing conscious attitudes and beha vior patterns that are just the opposite. A student may develop a who-cares- how-other-people-feel attitude to cover up feelings of loneliness and a hunger for acceptance.
31.4.5.1. Information on these and other defense mechanisms such as fantasy, repression, displacement, emotional insulation, regression, and introjection can be obtained from a good psychology text. The instructor should recognize that the mechanisms are used to preserve the self and with learned behaviors they function at relatively automatic and habitu al levels and involve some measure of self-deception and reality distor tion. Because they soften failure, alleviate anxi-AFMAN36-2236 12 NOVEMBER 2003 365

ety and hurt, and protect our feelings of adequ acy and worth, we can consider them normal adjustive reactions unless they seriously interfere with the effective resolution of stress situations.
31.4.5.2. While defense mechanisms are learned behavi ors, student feelings about themselves are learned responses. Students need to unlearn bad fe elings and acquire new feelings. While this is not always easy, it is possible. Through purposeful effort, Air Force instructors can enhance the self-concepts of students in their classrooms by helping them acquire new feelings about themselves.
31.5. Enhancing the Self-Concept: 31.5.1. Thus far in our discussion, we have reasoned th at the past experiences of students can have a vast influence on their current behavior. Their behavior is influenced not only by the accumulation of their past and current experiences, but more importantly by the personal meanings they attach to their perceptions of these experiences. Their behavior is more than simply a function of what happens to them from the outside. It is also a function of howthey feel about themselves on the inside. Although they cannot change what happened to them yesterday, they can change how they feel about it today. They cannot change the event, but can modify the pe rceptions they have about the event. In this manner, they can change their view of themselves or their self-concept. 31.5.2. Before examining specific suggestions for imp roving the self-concept of students, we need to deal with the question of which comes first—a cha nge in academic self-concept leading to improved academic performance or improve d academic performance leading to an improved academic self-concept. It does not seem unreasonable to suggest that each is mutually reinforc ing to the other to the extent a positive change in one facilitates a positive change in the other. If students begin Air Force schools with low levels of self-confidence or self-regard and experience sufficient successes, we would reasonably expect their self-c oncepts would be elevated as far as school ability is concerned. However, if students begin Air Force schools with hi gh confidence in their abilities to do the academic work and experience excessive failures, their se lf-concepts may be lowered. Under the latter conditions students must shift focus to other areas, us ually nonacademic, to mainta in self-esteem or continue to lose self-confidence and self-esteem.
31.6. Suggestions for Improving Self-Concepts:
31.6.1. Give Recognition. In transactional analysis, the term " positive strokes" is used. Behaviorists talk in terms of positive reinforcement. Adlerian psychologists use the term "encouragement." The concept is similar in each case. When students do something worthy of recognition, instructors need to give positive feedback to the students. Such re cognition makes the students feel alive, important, and significant. An achievement, innovative or crea tive idea, or a good effort all deserve recognition from the instructor. Such feedback is most important in improving any student's self-concept. Remember that you have the power to help students rec ognize their strengths and possibilities or to remind them again and again of their weaknesses and shortcomings.
31.6.2. Serve as a Good Model. As an instructor you will have a considerable influence on student self-concepts, particularly as these attitudes are re lated to feelings about being able to think, answer questions, and solve problems. You will quickly be established as a significant person in the eyes of most of your students. Perhaps you will be signifi cant because you are the only person in the school environment who makes a partic ular student feel like an individual of worth and value.366 AFMAN36-2236 12 NOVEMBER 2003

31.6.2.1. While all students deserve to have their total development faci litated by a truly competent human being—their instructor—you cannot te ach what you do not know well yourself. As an instructor, you should strive to grow and live fully if you are to help students learn to do likewise. Research indicates that teachers who have low se lf-concepts tend to have students in their classrooms with lower self-concepts, while students in classrooms where teachers have positive self-concepts tend to have students with high self-concepts. 31.6.2.2. Just as leaders must model appropriate behavior for followers, so must instructors be positive models for their students. Thus, effort directed toward improving your own self-concept may lead to an improved self-concept for students in your classrooms.
31.6.3. Stimulate Cooperation—Not Competition—Among Students. Modern society places a lot of emphasis on competition. While competition can lead to improved performance as students strive to do their best, competition against others can result in negative perceptions of the self if the students happen to lose more than they win. With coopera tion, everyone can experience the success of the group and no one is viewed as the winner or loser.
31.6.4. Consider Mastery Learning. Mastery learning was discussed in Chapter 28 as a concept designed to meet individual needs of students. Usi ng this approach, a student 's performance is measured against objectives rather than against the pe rformance of other students. As the instructor, you determine mastery, which often mean s achieving 80 to 85 percent of the course objectives. Time is the most crucial variable. Provide each student with sufficient time to achieve mastery. Theoretically, when you provide the proper instruction along with proper diagnostic techniques and correctives (tutoring, programmed instruction, reteaching), 90 percent of a give n group of students will achieve mastery. 31.6.4.1. When used properly with appropriate subjects, mastery learning appears to be able to eliminate most of the effects of individual diff erences on level of achievement. Each student is provided with an excellent opportunity for succe ss. Most students get an A or B indicating mastery. Mastery learning becomes both a subjective recognition by students of their own competence and public recognition of this competence by the school or society. 31.6.4.2. While approximately 25 percent of student s achieve the desired performance standards (A or B) in conventional group-based instructi onal programs, research indicates that nearly 75 percent of the students are reaching the same pe rformance standards with mastery learning. The impact on the student's self-concept is significant . Mastery learning seems to have the potential to ensure each student a history of successful and rewarding school learning experiences and, in the process, to shape positive affective development.
NOTE: Despite this potential for enhancing a student's self- concept, mastery learning has limitations that may preclude its use in your educational setting (see Chapter 28 ).
31.6.5. Have High But Reasonable Expectations For Students. Considerable research suggests that students perform up to the expectations teachers have for them. When instructors believe students will perform well academically, they normally do; on the other hand, when expectations for student performance are low, students usually perform poorly. Likewise, when students are expected to behave in certain ways, they often do. Instructors unknowingly encourage and reinforce the very behaviors they expect from their students. If inst ructors are prejudiced or ha ve a tendency to stereotype students, they are setting up expectations that can have a negative eff ect on student performance. What research is beginning to tell us is what co mmon sense has always told us. Students grow, flour-AFMAN36-2236 12 NOVEMBER 2003 367

ish, and develop better in a relationship with someone who trusts them and believes in their capabilities.
31.6.6. Recognize Potential in Students. During the past two decades, an increasing number of behavioral scientists have concluded that humans function at 10 pe rcent or less of their potential. For instance, scientists tell us that the human mind can store as many as 600 memories a second for a lifetime of 75 years without the slightest strain. It is estimated that our brain has the capacity to learn as many as 40 languages and would allow us to comple te the required courses of dozens of colleges. If these statements on the capacity of the brain are true, new means and methods must be designed to actualize a person's vast possibilities and latent pow ers. However, most of today's physical and cultural environment works in opposition to the devel opment of this potential. A negative self-concept certainly stands in the way of releasing the potentia l of students. So the next time you have students who tell you they can't, tell them they can, and even when you suspect that maybe they really can't, tell them they can; then who knows, they just might.
31.7. Summary. In this chapter we have seen that the sel f-concept of students is an important determiner of academic achievement. The belie fs students have about themselves have significant effects upon how they learn and behave. 31.7.1. Students have overall self-concepts that might be thought of as composed of physical, social, and scholastic self-concepts. The latter are composed of specific self-concepts that represent the areas where the instructor can have the most impactbecause they are the easiest to change. 31.7.2. In understanding students, we need to realiz e the impact parents, teachers, peers, and the school environment have had in shaping their aca demic self-concepts. While these groups are judging students, students are also judging themselves. Once th e perceptions of self begin to form, there is a good chance they will be reinforced over a period of time, especially if the students continue to interact with the same groups. 31.7.3. Defense mechanisms are used by students to pr eserve or protect their self-concepts. The common mechanisms encountered in the classroom include compensation, projection, rationalization, denial of reality, and reaction formation. 31.7.4. The self-concept can be changed; instructors are in a strategic position for enhancing a student's self-concept. Giving recognition, modeli ng a positive self-concept, stimulating cooperation (and not competition) in the group, using mastery learning, having high but reasonable expectations, and working to release the potential that exists in each student are suggested means of enhancing the self-concept.

ROGER M. BLANCHARD Assistant DC S / Personnel368 AFMAN36-2236 12 NOVEMBER 2003

Attachment 1

GLOSSARY OF REFERENCES AND SUPPORTING INFORMATION
References AFI 36-1201, Discrimination Complaints AFPD 36-22, Military Training AFMAN 36-2234, Instructional Systems Development AFH 36-2235, Volumes 1 through 11, Information for Designers of Instructional Systems AFPD 36-23, Military Education AFI 36-2706, Military Equal Opportunity and Treatment Program Title 17, United States Code, Copyrights Abt, Clark C., Serious Games , New York: Viking Press, 1970. Ackerman, Jerrold and Lipsitz, Lawrence, eds., Instructional Television: Status and Directions , Englewood Cliffs, NJ: Educational Technology Publications, 1977. Adams, Dennis M., Simulation Games: An Approach to Learning , Worthington, OH: Charles A. Jones, 1973. Anastasi, A., Psychological Testing , 4th ed., New York: MacMillan, 1976. Andre, T., Cognitive Classroom Learning: Unders tanding, Thinking and Problem Solving, Orlando: Academic Press, Inc., 1987. Apps, Jerold W., "Foundations for Effective Teaching," in Effective Teaching Styles , Edited by Elizabeth Hays, San Francisco: Jossey-Bass Publishers. Argyris, Chris, "Some Limitations of the Case Method: Experiences in a Management Development Program," Academy of Management Review , 5:291-298, April 1980. AU-1, Communication Techniques: Volume I: Individual and Group Communications, Maxwell AFB, AL: Air University, 1978. Baker, Frank B., Computer Managed Instruction: Theory and Practice , Englewood Cliffs, NJ: Educational Technology Publications, 1978. Banathy, Bela H., Instructional Systems , Belmont, CA: Fearon Publishers, 1968. Banathy, B. H., Systems Design of Education: A Journey to Create the Future , Englewood Cliffs, NJ: Educational Technology Publications, 1991. Banton, Lee, "Broadening the Scope of Classroom Questions," Virginia Journal of Education , 71: October 1977. Baron, D., and Bernard H. W., Evaluation Techniques for Classroom Teachers , New York: McGraw-Hill, 1958. Bechman, M. Dale, "Evaluating the Case Method," The Educational Forum , 36:489-497, May 1972. Benjamin, Alfred, The Helping Interview with Case Illustrations , Boston: Houghton Mifflin, 1974.AFMAN36-2236 12 NOVEMBER 2003 369

Bennett, John B., "Writing a Case and its Teaching Note ," From a note written as a basis for class discussion. Distributed by the Intercollegiate Case Clearing House, Boston, MA. Bennie, Frances, Learning Centers: Development and Operation , Englewood Cliffs, NJ: Educational Technology Publications, 1977. Berliner, D. C., and Cohen, L. S., "Trait -Treatment Interaction and Learning," in Review of Research in Education , Edited by F. N. Kerlinger, Itasca, IL: F.E. Peacock, 1973. Block, James H., Mastery Learning: Theory and Practice , New York: Holt, Rinehart and Winston, 1971. Bloom, Benjamin S., "Affective Outcomes of School Learning," Phi Delta Kappan , 59:193-198, November 1977. Bloom, Benjamin S., ed. Taxonomy of Educational Objectives: Th e Classification of Educational Goals, Handbook I: Cognitive Domain. New York: Longman, 1956. Bloom, Benjamin S., and others, Handbook on Formative and Summative Evaluation of Student Learning, New York: McGraw-Hill, 1971. Bormann, Ernest G., and Bormann, Nancy C., Effective Small Group Communication , 2d ed., Minneapolis: Burgess, 1972. Boston, Robert E., How to Write and Use Performance Objectives to Individualize Instruction , Englewood Cliffs, NJ: Educational Technology Publications, 1972. Bretz, Rudolf, A Taxonomy of Communication Media , Englewood Cliffs, NJ: Educational Technology Publications, 1971. Brilhart, John D., Effective Group Discussion , 3d ed., Dubuque, IA: William C. Brown, 1978. Broadwell, Martin M., The Lecture Method of Instruction , Englewood Cliffs, NJ: Educational Technology Publications, 1978. Brookfield, S.D., Understanding and Facilitating Adult Learning , San Francisco, CA: Jossey-Bass Publishers, 1986. Brown, George, Lecturing and Explaining , London, Methuen, 1978. Bryant, Donald C., and Wallace, Karl R., Fundamentals of Public Speaking , 4th ed., New York: Appleton-Century-Crofts, 1969. Chapter 9 "The Basis of Outlining" pp 131-162. Buro, Mental Measurements Yearbook Butler, F. Coit, Instructional Systems Development fo r Vocational and Technical Training , Englewood Cliffs, NJ: Educational Technology Publications, 1972. Campbell, J. P., and others, Managerial Behavior, Performance, and Effectiveness , New York: McGraw-Hill, 1970. Canfield, Jack, and Wells, Harold C., 100 Ways to Enhance Self-Concept in the Classroom , Englewood Cliffs, NJ: Prentice-Hall, 1976. Caplow, T., "The Dynamics of Informational Interviewing" American Journal of Sociology , 62:165-171, September 1956.370 AFMAN36-2236 12 NOVEMBER 2003

Carey, L. M., and Dick, W., "Summative Evaluation," in Instructional Design: Principles and Applications , 2d ed (pp 227-268). Englewood Cliffs, NJ : Educational Technology Publications. Carkhuff, Robert R., Helping and Human Relations: A Primer for Lay and Professional Helpers, Vol II, New York: Holt, Rinehart, and Winston, 1969. Catania, A. Charles, Learning , Englewood Cliffs, NJ: Prentice-Hall, 1979. Cathcart, Robert S., and Sarnovar, Larry A., Small Group Communications: A Reader , 3d ed., Dubuque, IA: W. C. Brown, 1979. Cavert, C. Edward, An Approach to the Design of Mediated Instruction , Washington: The Association for Educational Communication and Technology, 1974. Charan, Ram, "Classroom Techniques in Teaching by the Case Method," Academy of Management Review , 1:116-123, July 1976. Claxton, Charles S., and Ralston, Yvonne, Learning Styles: Their Impact on Teaching and Administration
, Washington DC: American Association for Higher Education, 1978. Copeland, Melvin T., "The Genesis of the Case Method in Business Instruction," in The Case Method at the Harvard Business School . Edited by Malcolm P. McNair, New York: McGraw-Hill, 1954. Copperud, Carol, The Test Design Handbook , Englewood Cliffs, NJ: Educational Technology Publications, 1979. Corey, Raymond E., "The Use of Ca ses in Management Education," Paper prepared at the Harvard University Graduate School of Business Administration and distributed by the Intercollegiate Case Clearing House, Boston, MA. Cragon, John F., and Wright, David W., Communication in Small Group Discussions: A Case Study Approach , St. Paul: West Publishing Co., 1980. Crawford, R. P., Direct Creativity , Wells, VT: Fraser, 1964. Cronbach, L. J., Essentials of Psychological Testing , New York: Harper & Brothers Publishers, 1949. Cronback, Lee J., Educational Psychological , 3d ed., New York: Harcourt Brace Jovanovich, Inc., 1977. Cronback, Lee J., and Snow, R. E., Aptitudes and Instructional Methods , New York: Irvington, 1977. Cross, P., Adults as Learners , San Francisco, CA: Jossey-Bass Publishers, 1981. Davies, Maj Kirk L. and Norton, Capt Lynn J., The Military Commander and the Law, 4th ed., Air Force Judge Advocate General School, 1998. Davis, G.A., Psychology of Problem Solving: Theory and Practice , New York: Basic Books, 1973, Chapter 9. Davis, G. A., and Houtman, S. E., Thinking Creatively: A Guide to Training Imagination , Madison: University of Wisconsin Research and Deve lopment Center for Cognitive Learning, 1968. Davis, G. A., and Roweton W., "Using Idea Checklis ts with College Students: Overcoming Resistance," Journal of Psychology , 70:221-226, November 1968. Davis, James R., Teaching Strategies for College Classroom , Boulder, CO: Westview Press, 1976

.AFMAN36-2236 12 NOVEMBER 2003 371

DeCecco, John P., The Psychology of Learning and Instruction , Englewook Cliffs, NJ: Prentice-Hall, 1968. Dedman, Donald N., "Criticizing Student Speeches: Philosophy and Principles," Central States Speech Journal , 20:16-18, Winter 1977. Delaney, Daniel J., and Eisenhery, Sheldon, The Counseling Process , Chicago: Rand McNally, 1973. Denton, Jon J., and others, "Matching Events of Instruction to Objectives," Theory into Practice
, 19:10-14, Winter, 1980. Dick, W., and Carey L., "Formative Evaluation," in Instructional Design: Principles and Applications , 2d ed (pp 227-268), Englewood Cliffs, NJ: Educational Technology Publications. Dick, W., and Carey L., The Systematic Design of Instruction , Glenview, IL: Scott, Foresman, 1985. Dooher, M. J., and Vivienne, M., Rating Employees and Supervisory Performance , New York: American Management Association, 1950. Downs, Cal W., and others , Professional Interviewing , New York: Harper & Row, 1980. Drumheller, Sidney J., Handbook of Curriculum Design for Individualized Instruction: A Systems Approach , Englewood Cliffs, NJ: Educational Technology Publications, 1971. Duane, James E., and others, Individualized Instruction-Programs and Materials: Selected Readings and Bibliography , Englewood Cliffs, NJ: Educational Technology Publications, 1973. Durost, Walter N., and Prescott, George A., Essentials of Measurement for Teachers , New York: Harcourt, Brace & World, 1962. Ebel, Robert L., "Behavioral Objectives: A Close Look," Phi Delta Kappan, 52:171-173, November 1970. Ebel, Robert L., Essentials of Educational Measurement , 3d ed., Englewood Cliffs, NJ: Prentice-Hall, 1979. Ebel, Robert L., Measuring Educational Achievement , Englewood Cliffs, NJ: Prentice-Hall, 1965. Eble, Kenneth E., The Craft of Teaching , San Francisco: Jossey-Bass, 1977. Eble, Kenneth E., Professors as Teachers , San Francisco: Jossey-Bass, 1972.
Educational and Industrial Television , Danbury, CT:C.S. Tepfer Publishing Company. Monthly. Egan, Kieran, "How to Ask Questions that Promote High-level Thinking," Peabody Journal of Education
, 52:228-234, April 1975. Eisele, James E., and others, "Comput er Assisted Planning of Curricul um and Instruction: How to Use Computer Base Resource Units to Individualize Instruction," Englewood Cliffs, NJ: Educational Technology Publications, 1971. Ellis, Albert, and Harper, Robert A., A New Guide to Rational Living , Englewood Cliffs, NJ: Prentice-Hall, 1975. Engel, Herbert M., Handbook of Creative Learning Exercises , Houston: Gulf Publishing Company, 1973. ETS Tests and Measurement Kit, Princeton, NJ: Educational Testing Service, 1973. Feder, Bernard, "Case Studies: A Flexible Approach to Knowledge Building," Social Studies , 64:171-178, April 1973.372 AFMAN36-2236 12 NOVEMBER 2003

Fisher, Charles F., "Being There Vicariously by Case Studies," in On College Teaching . Edited by Ohmer Milton and Associates, San Francisco: Jossey-Bass, 1978. Fitts, Paul, "Factors in Complex Skill Training," in Training Research and Education . Edited by R. Glasser, Pittsburgh: University of Pittsburgh Press, 1962. Ford, Leroy, Using the Case Method in Teaching and Training , Nashville, TN: Broadman Press, 1969. Fox, W. M., Effective Group Problem-Solving , San Francisco: Jossey-Bass Publishers, 1987. Furst, Edward L., Constructing Evaluation Instruments , New York: Longuano, Green and Co., 1968. Gage, N. L., and Berliner, David C., Educational Psychology , 2d ed., Chicago: Rand McNally, 1978. Gagne, R. M., Instructional Technology: Foundations , Hillsdale, NJ: Lawrence Erlbaum Associates, 1987. Gagne, Robert M., Brigg, Leslie J., and Wager, Walter W., Principles of Instructional Design , Chicago, IL: Holt, Rinehart and Winston, Inc., 1988. Gagne, Robert M., "Behavioral Objectives? Yes!" Educational Leadership, 29:394-396, February, 1972. Gagne, Robert M., The Conditions of Learning , New York: Holt, Rinehart and Winston, 1965. Gagne, Robert M., The Conditions of Learning , 2d ed., New York: Holt, Rinehart and Winston, 1970. Gagne, Robert M., The Conditions of Learning and Theory of Instruction , 4th ed., New York: Holt, Rinehart and Winston, 1985. Gall, Meredith D., and Gillett, Maxwell, "T he Discussion Method in Classroom Teaching," Theory into Practice , 19:98-103, Spring 1980. Gazda, G., and others, Human Relations Development: A Manual for Educators , Boston: Allyn & Bacon, 1977. Gerlach, Vernon S., and Ely, Donald P., Teaching and Media: A Systematic Approach , Englewood Cliffs, NJ: Prentice-Hall, 1971. Gorden, Raymond L., Interviewing: Strategy, Techniques, and Tactics, Homewood, IL: Dorsey Press, 1980. Gorth, William P., and others, Comprehensive Achievement Monitori ng: A Criterion Referenced Evaluation System , Englewood Cliffs, NJ: Educational Technology Publications, 1975. Gragg, Charles I., "Because Wisdom Can't Be Told," in The Case Method at the Harvard Business School

. Edited by Malcolm P. McNair , New York: McGraw-Hill, 1954. Graham, P. Tony, and Cline, Paul C., "The Case Method: A Basic Teaching Approach," Theory into Practice , 12:112-1165, Spring 1980. Green, John A., Teacher Made Tests , New York: Harper and Row, 1963. Greene, E. B., Measurement of Human Behavior , New York: The Odyssey Press, 1952. Gronlund, Norman E., Measurement and Evaluation in Teaching , New York: MacMillan, 1965. Gronlund, Norman E., and Linn, Robert L., Measurement and Evaluation in Teaching , 6th ed., New York: MacMillan Publishing Company, 1990.AFMAN36-2236 12 NOVEMBER 2003 373

Gronlund, Norman E., Preparing Criterion-Referenced Te sts for Classroom Instruction, New York: MacMillan, 1973. Gronlund, Norman E., Stating Objectives for Classroom Instruction , 2d ed., New York: MacMillan, 1978. Gronlund, Norman E., Constructing Achievement Tests , Englewood Cliffs, NJ: Prentice-Hall, 1968. Gronlund, Norman E., How to Make Achievement Tests and Assessments. Englewood Cliffs, NJ: Prentice-Hall, 1993. Gronlund, Norman E., How to Write and Use Educational Objectives. Englewood Cliffs, NJ: Prentice-Hall, 1995. Gropper, George L., Instructional Strategies , Englewood Cliffs, NJ: Educational Technology Publications, 1974. Gropper, George L., Diagnosis and Revision in the Development of Instructional Materials , Englewood Cliffs, NJ: Educational Technology Publications, 1975. Gropper, George L., and others, Criteria for the Selection and Use of Visuals in Instruction, Englewood Cliffs, NJ: Educational Technology Publications, 1971. (Two-volume handbook and workbook.) Gruber, H. E., and Weitman, Morris, 1962 Self-Directe d Study: Experiments in Higher Education, Report No. 19, Boulder: University of Colora do, Behavioral Research Laboratory. Guilford, J. P., Fundamental Statistics in Psychology and Education , New York: McGraw-Hill, 1965. Guilford, J. P., Psychometric Methods , New York: McGraw-Hill, 1954. Gulley, Halbert E., Discussion, Conference, and Group Process , 2d ed., New York: Holt, Rinehart, and Winston, 1968. Gulley, Halbert E., and Leathers, Dale G., Communication and Group Process , 3d ed., New York: Holt, Rinehart, and Winston, 1977. Hamachek, Don E., Encounters with the Self , 2d ed., New York: Holt, Rinehart, and Winston, 1978. Hambleton, Ronald K., and others, "Criterion-Referenced Testing," Re view of Educational Research, 48:1-47, Winter 1978. Hammond, John S., "Learning by the Case Method," No te based on earlier notes by Professor E. Raymond Corey and Martin Marshall of the Harvard Bu siness School. Distributed by the Intercollegiate Case Clearing House, Boston, MA. Hargrove, Melvin M., "The Case Me thod," Paper prepared at the University of Tulsa and distributed by the Intercollegiate Case Clearing House, Boston, MA. Haase, Richard F., "The Relationship of Sex and Instru ctional Set to the Regulati on of Interpersonal Interaction Distance in a Counseling Analogue," Journal of Counseling Psychology , 17:233-236, May 1970. Heckhausen, Heinz, The Anatomy of Achievement Motivation , New York: Academic Press, 1967. Henson, Kenneth T., "Questioning as a Mode of Instruction," Clearing House , 53:14-16, September 1979. Henson, Kenneth T., "Teaching Methods: History and Status," Theory into Practice , 19:2-5, Winter, 1980. Hergenhahn, B. R., An Introduction to Theories of Learning , Englewood Cliffs, NJ: Prentice-Hall, 1976.374 AFMAN36-2236 12 NOVEMBER 2003

Hill, J. E., and Nunnery, D. N., The Educational Sciences , Bloomfield Hills, Michigan: Oakland Community College Press, 1973. Hinricks, J. R., "Personnel Training," in Handbook of Industrial and Organizational Psychology , Edited by M. D. Dunnette, Chicago: Rand McNally, 1976. Holland, James G., and Skinner, B. F., The Analysis of Behavior , New York: McGraw-Hill, 1961. Hoover, Kenneth H., College Teaching Today: A Handbook for Postsecondary Instruction, Boston: Allyn & Bacon, 1980. Hoover, Kenneth H., The Professional Teacher's Handbook: A Guide for Improving Instruction in Today's Secondary Schools, Boston: Allyn & Bacon, 1973. Horn, Robert E., and Cleaves, Anne, eds., The Guide to Simulations/Games for Education and Training
, 4th ed., Beverly Hills, CA: Sage, 1980. Hunkins, Francis P., Questioning Strategies and Techniques , Boston: Allyn & Bacon, 1972. Hunt, David E., "Learning Styles and Teaching Strate gies," Paper presented at National Council for the Social Studies, Boston, MA, 1972. Jones, Anthony S., and others, Strategies for Teaching , Metuchen, NJ: Scarecrow Press, 1979. Jones, Stanley E., and others, The Dynamics of Discussion Communication in Small Groups , 2d ed., New York: Harper and Row, 1980. Joyce, Bruce R., "Learning How to Learn," Theory into Practice , 19:15-27, Winter 1980. Joyce, Bruce R., Selecting Learning Experiences: Linking Theory and Practice , Washington: Association for Supervision and Curriculum Development, 1978. Kagan, J., "Acquisition and Significance of Sex Typing and Sex Role Identity," in Review of Child Development Research, Vol I, Edited by M.L. Hoffman and L.W. Hoffman, New York: Russell Sage Foundation, 1964. Kahn, Robert L., and Cannell, Charles F., The Dynamics of Interviewing , New York: John Wiley and Sons, 1964. Kapfer, Miriam B., Behavioral Objectives in Curriculum Development: Selected Readings and Bibliography , Englewood Cliffs, NJ: Educational Technology Publications, 1971. Keller, F. S., and Sherman, J. G., The Keller Plan Handbook: Essays on a Personalized System of Instruction , Menlo Park, CA: W. A. Benjamin, 1974. Keoman, Sam, "Teacher Expectations and Student Achievement," Phi Delta Kappan , 60:716-718, June 1979. Kibler, Robert J., and others, Behavioral Objectives and Instruction , Boston: Allyn & Bacon, 1970. King, Carl D., Teaching Social Studies: A Case Approach , New York: MacMillan, 1972. Kline, John A., Speaking Effectively , Maxwell AFB, AL: Air University, December 1989. Kline, John A., and others, "Tips on the Critiquing of Writing and Speaking," AFROTC Education Journal , 20:16-18, Winter 1977. Knowles, Malcolm S., The Modern Practice of Adult Education , New York: Association Press, 1970.AFMAN36-2236 12 NOVEMBER 2003 375

Knowles, Malcolm S., The Modern Practice of Adult Education: Andragogy Versus Pedagogy, New York, NY: Association Press, 1974. Knowles, M., The Adult Learner: A Neglected Species , Houston: Gulf Publishing Company, 1973. Knowles, M., The Modern Practice of Adult Educ ation: From Pedagogy to Andragogy, Chicago: Association/Follett Press, 1980. Knowles, M., Self-Directed Learning: A Guide for Learners and Teachers, Chicago: Association/Follett Press, 1975. Kolb, David, Experimental Learning , Englewood Cliffs, NJ: Prentice-Hall, 1984. Kozma, Robert B., and others, Instructional Techniques in Higher Education , Englewood Cliffs, NJ: Educational Technology Publications, 1978. Krathwohl, David R., and others, Taxonomy of Educational Objectives: The Classification of Educational Goals, Handbook II: Affective Domain, New York: Longman, 1964. Krech, D., and others, Individual and Society , New York: McGraw-Hill, 1962. Laird, Dugan, Approaches to Training and Development , 2d ed. (revised), Reading, MS: Addison-Wesley Publishing Company, 1985. Lawson, Reed, Frustration: The Development of a Scientific Concept , New York: MacMillan, 1965. Leshin, Cynthia B., Pollock, Joellyn, and Reigeluth, Charles M., Instructional Design Strategies and Tactics , Englewood Cliffs, NJ: Educational Technology Publications, 1992. "Let's Get Down to Cases," Regular feature in issues of Supervisory Management

. Lippey, Gerald, and others, Computer Assisted Test Construction, Englewood Cliffs, NJ: Educational Technology Publications, 1974. Lippincott, H. F., Ph. D. (Lt Col, USAF, Retired). Lecture notes prepared for use in a Case Writing Laboratory presented at the Academic Instruct or School, Air University, Maxwell AFB, AL. Lovell-Troy, Larry, and Eickmann, Paul, Course Design for College Teachers , Englewood Cliffs, NJ: Educational Technology Publications, 1992. Lyons, Paul, Thirty-five Lesson Formats: A Sourcebook of Instructional Alternatives , Englewood Cliffs, NJ: Educational Technology Publications, 1992. Maccoby, E. E., and Jacklin, C. N., The Psychology of Sex Differences , Stanford: Stanford University Press, 1974. Macey, James R., "The Case Method: A Handbook for Teachers," Maxwell AFB, AL: Air University, 15 April 1980. Mager, Robert F., Goal Analysis , Belmont, CA: Fearon Pitman Publishers, 1972. Mager, Robert F., Developing Attitudes Toward Instruction , Palo Alto, CA: Fearon, 1968. Mager, Robert F., Measuring Instructional Intent or Got A Match?, Belmont, CA: Fearon Publishers, 1973. Mager, Robert F., Preparing Instructional Objectives . Atlanta: Center for Effective Performance, 1997.376 AFMAN36-2236 12 NOVEMBER 2003

Mager, Robert F., and Beach, Kenneth M., Jr., Developing Vocational Instruction , Belmont, CA: Fearon Publishers, 1967. Mager, Robert F., and Pipe, Peter, Analyzing Performance Problems or 'You Really Oughta Wanna ,' Belmont, CA: Fearon Publishers, 1970. Magoun, F. Alexander, The Teaching of Human Relations by the Case Demonstration Method, Boston: Beacon Press, 1959. Maidment, Robert, and Bronstein, Russell H., Simulation Games: Design and Implementation , Columbus, OH: Charles E. Merrill, 1973. Maier, Norman R. F., and others, The Role-Play Technique: A Handbook for Management and Leadership Practice , La Jolla, CA: University Associates, 1975. Main, S., and Gallagher, D., 1972 Student and Faculty Evaluations of the New Undergraduate Grading System: A Preliminary Report, Stanford, CA: Stanford University Planning Office. Making It Strange , Prepared by Synectics, Inc., New York: Harper & Row, 1968. Malone, T. W., and Lepper, M. R., "Making Lear ning Fun: A Taxonomy of Intrinsic Motivations for Learning," in Aptitude, Learning and Instruction: III, Cognitive and Affective Process Analysis , Hillsdale, NJ: Erlbaum, 1985. Markel, Susan Meyer, Good Frames and Bad: A Grammar of Frame Writing , 2d ed., New York: John Wiley & Sons, Inc., 1969. Martin, B., and Briggs, L. J., The Cognitive and Affective Domain , Englewood Cliffs, NJ: Educational Technology Publications, 1986. Maslow, A. H., Motivation and Personality, New York: Harper and Row, 1954. McAshan, H. H., The Goals Approach to Performance Objectives , Philadelphia: W.B. Sanders, 1974. McKeachie, W. J., "Motivation, Teachi ng methods, and College Learning," in Nebraska Symposium on Motivation . Edited by M. R. Jones. Lincoln: University of Nebraska Press, 1961. McKeachie, Wilbert J., Teaching Tips: A Guidebook for the Beginning College Teacher , 7th ed., Lexington, MA: D.C. Health, 1978. McKeachie, Wilbert J., Teaching Tips: A Guidebook for the Beginning College Teacher , 8th ed., Lexington, MA: D.C. Health and Company, 1986. McKeachie, Wilbert J., Teaching Tips: Strategies, Research, and Theory for College and University Teachers, Lexington, MA: D.C. Health, 1994. McKeachie, W. J., and Kulik, J. A., "Effective College Teaching," in Review of Research in Education
, Edited by F. N. Kerlinger, Washington: Am erican Educational Research Association, 1975. McNair, Malcolm P., ed., The Case Method at the Harvard Business School , New York: McGraw-Hill, 1954. McNett, Ian, "It Worked for Socrates," American Education , 15:33-38, November 1979. Means, Richard K., Methodology in Education , Columbus, OH: Charles E. Merrill, 1968. Media and Methods, Philadelphia: North Am erican Publishing Company, Monthly.AFMAN36-2236 12 NOVEMBER 2003 377

Mehrabian, Albert, "Communications Without Words," Psychology Today , 1:52-55, February 1968. Merrill, M. David, ed., Instructional Design: Readings , Englewood Cliffs, NJ: Prentice-Hall, 1971. Merrill M. David, and Tennyson, Robert D., Teaching Concepts: An Instructional Design Guide , Englewood Cliffs, NJ: Educational Technology Publications, 1977. Merrill, P. F., "Job and Analysis," in R. M. Gagne (Ed.) Instructional Technology: Foundations , Hillsdale, NJ: Lawrence Erlbaum Associates, 1987. Merry, Robert W., "Preparation to Teach a Case," in The Case Method at the Harvard Business School

. Edited by Malcolm P. McNair , New York: McGraw-Hill, 1954. Merry, Robert W., "The Usefulness of the Case Me thod for Training in Administration," Paper distributed by the Intercollegiate Case Clearing House, Boston, MA. Merry, Robert W., "Use of Case Material in the Cl assroom," in the Case Method at the Harvard Business School. Edited by Malcolm P. McNair, New York: McGraw-Hill, 1954. Mills, Glen E., Message Preparation Analysis and Structure , Indianapolis: Bobbs-Merrill, 1966. Milton, Ohmer and Associates, On College Teaching , San Francisco: Jossey-Bass, 1978. Montgomery, K. E., "How to Criticize Student Speeches," Speech Teacher , 6:200-204, September 1957. Moore, Carl M., Group Techniques for Idea Building , Beverly Hills, CA: Sage Publications, 1987, See Chapter 3. National Society for the Study of Education, The Psychology of Teaching Methods, 75 Yearbook, Part I
, Edited by N. L. Gage, Chicago: University of Chicago Press, 1976. Nitko, A. J., "Defining Criterion-Referenced Test," in R. A. Berk, ed., A Guide to Criterion-Referenced Test Construction, Baltimore: Johns Hopkins University Press, 1984, Chapter 1, A comprehensive review of the various meanings of criterion referencing. Noll, Victor H., Introduction to Educational Measurement , Boston: Houghton Mifflin Co., 1965. Nunnally, J. C., Educational Measurement and Evaluation , New York: McGraw-Hill, 1964. Osborn, A. F., Applied Imagination , New York: Charles Scribner's Sons, 1963. Osgood, C., Suci, J., and Tannenbaum, P., The Measurement of Meaning , Chicago: University of Chicago Press, 1957. Otto, Herbert A., Human Potentialities: The Challenge and the Promise , St. Louis: Warren H. Green, 1968. Page, E. B., "Teacher Comments and Student Perfo rmance: A Seventy-Four Classroom Experiment in School Motivation," Journal of Educational Psychology , 49:173-181, August 1958. Parnes, Sidney J., Creative Behavior Guidebook, New York: Charles Scribner's Sons, 1976. Parnes, S. J., Toward a Better Understanding of Brainstorming , Originally published in Adult Leadership (April 1959); reprinted in Parnes and Harding, Source Book for Creative Thinking , New York: Charles Scribner's Sons, 1962; and Biondi, The Creative Process , Buffalo: D.O.K., 1972. Parnes, Sidney J., and Harding, Harold F., eds., A Source Book for Creative Thinking , New York: Charles Scribner's Sons, 1976.378 AFMAN36-2236 12 NOVEMBER 2003

Pearson, Craig, and Marguggi, Joseph, Creativity and Using Learning Games , Palo Alto, CA: Learning Handbooks, 1975. Peterson, Gary T., The Learning Center: A Sphere for Non- traditional Approaches to Education, Hamden, CT: The Shoe String Press, 1975. Phye, G. D., and Andre, T., Cognitive Classroom Learning: Understanding, Thinking, and Problem Solving , Orlando: Academic Press, 1986. Popham, W. James, and Baker, Eva L., Establishing Instructional Goals, Englewood Cliffs, NJ: Prentice-Hall, 1970. Popham, W. James, and Baker, Eva L., Planning an Instructional Sequence, Englewood Cliffs, NJ: Prentice-Hall, 1970. Popham, W. James, and Baker, Eva L., Systematic Instruction, Englewood Cliffs, NJ: Prentice-Hall, 1970. Popham, W. James, and others, Instructional Objectives, Chicago: Rand McNally, 1969. (American Educational Research Association, monograph series on curriculum evaluation, No. 3.) Popham, W. James, Educational Evaluation , Englewood Cliffs, NJ: Prentice-Hall, 1975. Popham, W. James, An Evaluation Guidebook: A Se t of Practical Guidelines for the Educational Evaluator, Los Angeles: The Instructional Objectives Exchange, 1972. Popham, W. James, ed., Evaluation in Education , Berkeley: McCutchan, 1974. Popham, W. James, Modern Educational Measurement , Englewood Cliffs, NJ: Prentice-Hall, 1981. Popham, W. James, and Husek, T. R., "Implicat ions of Criterion-Referenced Measurement," Journal of Educational Measurement , 6:1-9, Spring 1969. Popham, W. James, and others, Criterion-Referenced Measurement: An Introduction , Englewood Cliffs, NJ: Educational Technology Publications, 1971. Powers, David A., and Russell, Tommy, "Interaction Rese arch: An Approach to the Investigation of College Teaching Methods," Educational Technology , 20:11-18, August 1980. Prince, G., The Practice of Creativity , Cambridge, MA: Synectics, 1969. Rahmlow, Harold F., and Woodley, Katheryn K., Objectives-Based Testing: A Guide to Effective Test Development , Englewood Cliffs, NJ: Educational Technology Publications, 1979. Reid, Loren D., "The Art of Criticism," in Teaching Speech by Loren D. Reid, 4th ed., New York: McGraw-Hill, 1971. Reid, Loren, Speaking Well , 3d ed., New York: McGraw-Hill, 1977. Reisner, Robert A., and Gagne, Robert M., Selecting Media for Instruction , Englewood Cliffs, NJ: Educational Technology Publications, 1983. Richey, R., The Theoretical and Conceptual Basis of Instructional Design , New York: Nicols, 1986. Roberts, Arthur D., ed., Educational Innovation: Alternatives in Curriculum and Instruction, Boston: Allyn & Bacon, 1975. Roid, G., and Haladyna, T. A., Technology for Test-Item Writing , New York: Academic Press, 1981. Romiszowski, A. J., Producing Instructional Systems , New York: Nichols Publishers, 1984.AFMAN36-2236 12 NOVEMBER 2003 379

Romiszowski, A. J., The Selection and Use of Instructional Media , New York: Nichols Publishers, 1988. Romiszowski, A. J., Designing Instructional Systems , London: Koran Page, 1981. Rosenthal, R., and Jacobson, L., Pygmalion in the Classroom , New York: Holt, Rinehart, and Winston, 1968. Royer, J. M., Cognitive Classroom Learning: Understanding Thinking, and Problem Solving, Orlando: Academic Press, Inc., 1986 Rushby, Nicholas John, Computers in the Teaching Process , New York: John Wiley and Sons, 1979. Sanders, Norris M., Classroom Questions: What Kinds?, New York: Harper & Row, 1966. Schmuck, Richard A., and Schmuck, Patricia A., Group Processes in the Classroom , Dubuque, IA: William C. Brown, 1971. Scholer, Lowell A., Test Construction: A Programmed Guide , Boston: Allyn & Bacon, 1972. Schrank, J., Teaching Human Beings: 101 Subversive Activities for the Classroom , Boston: Beacon Press: 1972. Schustereit, Roger C., "Team-Teaching and Academic Achievement," Improving College and University Teaching, 28:85-89, Spring 1980. Seels, Barbara, and Glasgow, Zita, Exercises in Instructional Design, Columbus, OH: Merrill Publishing Company, 1990. Shavelson, R. J., and others, "Self-Concept: Validation of 'Construct Interpretation'," Review of Educational Research , 1967. Shaw, Malcolm, E., and others, Role-Playing: A Practical Manual for Group Facilitators , San Diego, CA: University Associates, 1980. Sheffield, Edward F., ed., Teaching in the Universities , Montreal: McGill-Queen's Press, 1974. Shertzer, Bruce, and Stone, Shelley C., Fundamentals of Counseling , 2d ed., Boston: Houghton Mifflin, 1974. Simon, Sidney B., and others, Values Clarification , New York: Hart Publishing, 1972. Simpson, E. J., The Classification of Educational Objectives in the Domain, The Psychomotor Domain, Volume 3, Washington: Gryphon House, 1972. Skinner, B. F., The Technology of Teaching , New York: Appleton-Century-Crofts, 1968. Stanley, Julian C., Measurement in Today's Schools , Englewood Cliffs, NJ: Prentice-Hall, 1964. Statlings, W. M., and Smock, H. R., "The Pass- Fail Grading Option at a State University," Journal of Educational Measurement , 8:153-160, Fall 1971. Stenzel, Anne K., and Feeney, Helen M., Learning by the Case Method: Practical Approaches for the Community Leader, New York: Seabury Press, 1970. Stewart, Charles J., and Cash, William B., Interviewing: Principles and Practices , Dubuque, IA: W. C. Brown, 1974. Stolovitch, H., Case Study Method, Performance and Instruction , 29(9), 35-37, 1990.380 AFMAN36-2236 12 NOVEMBER 2003

Swezey, Robert W., and Pearlstein, Richard B., Guidbook for Developing Criterion-Referenced Tests
, Arlington, VA: US Army Research Institute for the Behavioral and Social Sciences, August 1975. Taber, Julian I., and others, A Guide to the Preparation of Programmed Instructional Material , Pittsburgh: University of Pittsburgh, 1962. Taylor, Jack W., How to Create New Ideas, Englewood Cliffs, NJ: Prentice Hall, 1961. Thorndike, Robert L., ed., Educational Measurement , 2d ed., Washington: American Council on Education, 1971. Thorndike, Robert L., and Hagan, Elizabeth, Measurement and Evaluation in Psychology and Education, New York: John Wiley & Sons, 1969. Tiemann, Philip W., and Markle, Susan M., Analyzing Instructional Content: A Guide to Instruction and Evaluation , Champaign, IL: Stripes Publishing, 1978. Travers, Cobert M. W., How to Make Achievement Tests , New York: The Odyssey Press, 1950. Travers, R. M., Educational Measurement , New York: MacMillan, 1955. Tubbs, Stewart L., and Moss, Sylvia, Human Communication , 3d ed., New York: Random House, 1980. Tuckman, Bruce Wayne, Evaluating Instructional Programs , Boston: Allyn & Bacon, 1979. Tyler, Leona E., The Work of the Counselor , 3d ed., New York: Appl eton-Century-Crofts, 1969. Tyler, Ralph W., Basic Principles of Curriculum and Instruction , Chicago: University of Chicago Press, 1949. University of Michigan Center for Research on Learning/Teaching, Memo to the Faculty, No. 60, April 1978. Walther, R. E., and Crowder, Norman, A Guide to Preparing Intrinsically Programmed Instructional Materials, (AMRL-TR-65-43) Wright-Patte rson AFB, OH: Behavioral Scie nces Laboratory, Air Force Systems Command, April 1965. Ward, B. A., Minicourse 15: Organizing Independent Learni ng, Intermediate Level, Teacher's Handbook, New York: MacMillan, 1973. Weiner, Bernard, Theories of Motivation: From Mechanism to Cognition , Chicago: Markham, 1972. Westmeyer, P., Effective Teaching in Adult and Higher Education , Springfield, IL: Charles C. Thomas Publishers, 1988. Whiting, C. S., "Forced Relationship Techniques," Have an Affair with Your Mind , Edited by A.M. Biondi, Buffalo: D.O.K., 1974. Williams, Robert O., "What Teaching Methods When?" Theory into Practice , 19:82-86, Spring 1980. Wilen, William W., ed. Questions, Questioning Techniques, and Effective Teaching, Washington, D.C.: NEA Professional Library, 1987. Wittich, Walter A., and Shuller, Charles F., Audio-Visual Materials , New York: Harper and Bros., 1957. Wood, Dorothy A., Test Construction: Developing and Interpretation of Achievment Tests, Columbus, OH: Charles E. Merrill Books, 1960. Wrightstone, J. W., and others, Evaluation in Modern Education , New York: American Book Co., 1956.AFMAN36-2236 12 NOVEMBER 2003 381

Abbreviations and Acronyms
ACSC— Air Command and Staff College
AIS—Academic Instructor School
AR— anticipated response
AU— Air University
AWC— Air War College
CRO— criterion-referenced objective
CRT— criterion-referenced test
d— deviation

d 2

— deviation squared
DI— differentiation index
D-P— demonstration-performance
EI— ease index
f— frequency FUQ— follow-up question
ISD— instructional systems development LOQ— lead-off question
LOL— level of learning
M— mean MEI— management effectiveness inspection
MP— main point
N— number of scores
NCO— noncommissioned officer
OJT— on-the-job training
OTJAG— Office of The Judge Advocate General
PD— prosecutorial discretion
POW— prisoner of war Q&A— question and answer
ROE—rules of engagement
S— standard deviation
SOB— sample of behavior
SOS— Squadron Officers School382 AFMAN36-2236 12 NOVEMBER 2003

SS— standard score
TOOTLIFEST—the objective of this lesson is for each student to... TQM—total quality management
UCMJ—Uniform Code of Military Justice
USAFE— United States Air Forces in Europe

Terms
Affective Domain— Major area of learning which deals wi th acquired attitudes, values, etc. Affective Sample of Behavior—A statement containing (1) a measurab le or observable action verb, (2) reflecting the lesson objective's level of learning, (3) covering the entire subject matter of the objective and (4) containing a personal adverb or adverbial phrase (one that describes the person versus one that describes the action). (See also Cognitive Sample of Behavior Elements and Sample of Behavior

. ) Analysis— Level of cognitive domain (Bloom, 1956) in whic h students are able to break down complex organizational structures into their component parts. Anticipated Responses— Answers the instructors expect students to give in reply to planned questions. Application— Level of cognitive domain (Bloom, 1956) in which students are able to use learned material in new and concrete situations. Attention Step— Segment of a lesson introduction in which an instructor gains the attention of the students and focuses upon the subject to be taught.
Behavioral Indicators— (See Sample of Behavior. ) Body—Major section of a lesson in which learning is developed through support material and various teaching exercises to achieve inst ructional objectives; preceded by an introduction and followed by a conclusion. Case Study— A teaching method in which students encounter a real-life or fictional situation under the guidance of an instructor in order to achieve an instructional objective. Characteristics (of a concept)— (See Critical Attributes. )
Characterization—Highest level of affective domain (Krathwohl, 1964) in which students integrate values or value systems into their own life style or philosophy of life. Clarification Support— Type of instructional mate rial used in the body of a lesson to develop learning and clarify ideas; may include definitions, examples, co mparisons, statistics, or testimony from experts and trustworthy sources.
Closure— The final segment of a lesson conclusion duri ng which instruction is appropriately ended. Cognitive Domain— A major area of learning which deals with acquiring knowledge (as opposed to attitudinal or manu al skill knowledge). Cognitive Sample of Behavior— A statement containing (1) a measurable, observable, reliable, and verifiable action verb, (2) reflecting the lesson objectiv e's level of learning and (3) covering significant aspects of the subject matter. (See Affective Sample of Behavior and Sample of Behavior. )AFMAN36-2236 12 NOVEMBER 2003 383

Comprehension— Level of the cognitive domain (Bloom, 19 56) in which students begin to develop understanding and are able to translate, interpre t, and extrapolate subject matter under study. Comprehension-Level Summary— Segment of a lesson at the comprehension level in which the instructor reviews and expands on ke y material and develops relationshi ps which lead to a generalization which is, or supports, the instructional objective. Comprehensiveness— A characteristic of evaluation which requi res the stated objectives of instruction be tested or rated. The quality of a lesson overview or summaries thatdescribes a complete preview or review of all teaching points within the lesson. Concept—A class of people, objects, events, ideas, sym bols, or actions grouped together on the basis of shared critical attributes or characteristics and are called the same name.
Conclusion—A major section of a lesson that follows an introduction and body. It should contain a summary, remotivation, and closure.
Condition Statement— The part of a criterion-referenced objective that describes the testing environment including those problems , materials, and supplies to be given (included) or specifically excluded from a measurement situation. Consistency—Describes the results of a reliable evalua tion instrument which remain similar given similar testing conditions (similar st udents, knowledge base, physical tes ting situation, etc.) over a period of several uses. (See Reliability. ) Counseling—A process characterized as a helping relationship between teacher and student directed toward improvement, change, or re inforcement of student behavior. Creativity— The imaginative recombination of known elements into something new and useful. Criterion Objective— A precise description of a student-centered learning outcome for a planned program of instruction that describes the performa nce and describes the conditions and standards for assessment through criterion-referenced testing.
Criterion-Referenced Test— Any test of carefully written measur able objectives to obtain data to compare student performance levels with that specifie d in the criterion-referenced objectives or SOBs of level-of-learning objectives.
Critical Attributes— Necessary characteristics for determining class membership in a concept.
Demonstration-Performance Method—A teaching method in which students observe and then practice a sequence of events designed to teach a procedure, technique, or operation. It co mbines oral explanation with the operation or handling of sy stems, equipment, or materials.
Demonstration Phase—A phase of the demonstration-performance teaching method during which the instructor shows students how to perform the skill to be learned.
Differentiation— A characteristic of evaluation that requires tests and rating instruments be capable of making distinctions between selected groups; usuallymasters or nonmasters of specific instructional objectives in criterion-referencedtesting or high and low overall te st performers in norm-referenced testing. Direct Question— An instructor-initiated question addressed to a particular student to elicit involvement, seek an opinion, or draw out support.384 AFMAN36-2236 12 NOVEMBER 2003

Domain of Learning— A broad classification of learning types. The three widely accepted domains used in this manual are the cognitive (thinking, understandi ng), affective (attitudes, values), and psychomotor (physical/mental skills). Educational Objective— (See Student-Centered Instructional Objective. )
Evaluation— The systematic process of measuring or observing and judging how well individuals, procedures, or programs have met educational objectives.
Evaluation Phase—A phase of the demonstration-performan ce teaching method during which the instructor conducts criterion-referenced testing to determine the extent to wh ich students have mastered the instructional objectives. Examples (in concept teaching)—People, objects, events, ideas, symbols, or actions which have all the critical attributes of a particular concept and can be correctly called by that concept name. Explanation Phase—A phase of the demonstration-perfor mance teaching method during which the instructor tells the students how to perform the skill to be learned. Extrapolation— A type of learning at the comprehension le vel (Bloom, 1956) in which students develop sufficient understanding to estimate trends or predict outcomes regard ing the subject matter under study. The trend estimate or prediction is based solely on the data given. Feedback for Learning— Information students receive from thei r instructor about their performance which will cause them to accept gui dance and take corrective action to attain the goals of the course. Followup Question— An instructor-initiated question designed to guide student responses to a previous question by rephrasing the original question to get the same response or by posing a new question that elicits a partial or more specific answer to the original question.
Formal Lecture—A structured and often rehearsed teaching lecture with no verbal participation by students.
Formative Evaluation— An evaluation of student progress towa rd instructional objectives during the learning experience that is not used to determine criterion- or norm-referenced assessments of student achievement. General-to-Specific Presentation— The process or pattern of outlining lesson main and subpoints so as to start with the subject of an objective to be taught and then applying this subject in specific instances that support original objective.
Generalization— The result of identifying an example of a concept by matching its critical attributes with those of the original concept.
Guided Discussion Method—A teaching method in which students participate in an instructor-controlled, interactive process of sharing knowledge and interpreting experiences in order to achieve an instructional objective. Hierarchy— The characteristic of a domain of learning that rank orders the levels of learning of which it is composed. (See Taxonomy of Educational Objectives and Domain of Learning. ) Higher Levels of Learning— Those levels of learning above the comprehension level (Bloom, 1956) that may be considered as the prac tical application of concepts and principles to complex, real problems.
Informal Lecture— An often conversational teaching lecture with considerable verbal interactionAFMAN36-2236 12 NOVEMBER 2003 385

between instructor and students in which the instructor asks questions of and answers questions from the audience.
Instructional Media— All forms of instructional aids that give audible or visual support in a learning environment. Instructional Objective— (See Student-Centered Instructional Objective. ) Instructional Systems Development—A deliberate, orderly, flexible process for planning, developing, conducting, and managing high-quality, stude nt-centered instructional programs. Interpretation— A type of learning at the comprehension le vel (Bloom, 1956) in which students develop and understand relationships among the various aspects of a communication and are able to perform such activities as making inferences, generalizing, and summarizing.
Introduction—Major section of a lesson designed to establish a common ground between the instructor and students, to capture and hold attention, to outline the lesson and relate it to the overall course, to point out benefits to the students, and to lead the students into the body of the lesson; usually contains attention, motivation, and overview steps. Knowledge— The lowest level of the cognitive domain (Blo om, 1956) in which students have the ability to recall or recognize material in esse ntially the same form as it was taught. Knowledge-Level Summary— A reiteration of key points of content in a knowledge-level lesson designed to help stude nts remember facts. Leadoff Question— An instructor-initiated question at the be ginning of a lesson or main point designed to generate discussion. Learning— A change in student behavior resulting from expe rience or insight. The behavior can be overt or covert and physical, inte llectual, or attitudinal. Learning Center— A learning environment specifically develope d to foster individualized instruction and which emphasizes employment of instructional me dia to augment textbooks, manuals, and teacher presentations.
Lecture— (See Teaching Lecture, Formal Lecture, and Informal Lecture. )
Lesson Plan— A teaching-learning plan which includes student-centered instructional objectives, detailed content outline, and significant details de scribing the instructional elements such as media, teaching method, and length of period. Level of Learning— The degree to which a student is expected to internalize (master) a mental subject, values, or ability to perform psychomotor skills. Level-of-Learning Objective— A student-centered instructional obj ective that states the intended subject matter for a planned program of learning activ ities and assigns a carefully defined taxonomy level or proficiency level to that subject matter to indicate the desired level of mastery.
Main Points— The primary, logical break out of subject matter to support an instructional objective. Mastery Learning— An approach to learning in which stude nts progress from learning experience to learning experience based on achievement of instructi onal objectives prescribed in the curriculum design rather than other factors such as age, effort, or time of year.
Measurement— The act of acquiring data in the educat ional environment without making value386 AFMAN36-2236 12 NOVEMBER 2003

judgments regarding the relative or absolute merits of those data.
Method of Instruction— A planned program of instruction with characteristics sufficiently different from other alternatives to be identified as a major vehicle for teaching subject matter. Motivation Step— The segment of a lesson introduction in whic h an instructor provides specific reasons why students need to learn whatever they are about to learn. Nonexamples (in concept teaching)—People, objects, events, ideas, symbols, or actions that lack one or more critical attributes of a particular concept and which should not be called by that concept name. A close-in nonexample is a nonexample that is missing onl y one critical attribute. For instance, a close-in nonexample of a chair is a stool.
Norm-Referenced Test—Any test designed to obtain data for rank ordering or comparing relative student performance. Objective— (See Student-Centered Instructional Objective. ) Objectivity— A characteristic of evaluation that requires measurement in an educational environment be correct, factual, and free from instructor bias. Organization— Level of the affective domain (Krathwohl, 1964) in which students compare, relate, and synthesize new values into their own value systems. Overhead Question— An instructor-initiated question to which the instructor expects an answer, directed to an entire group rather than to a specific student.
Overview— Segment of a lesson introduction in which the instructor provides a clear and concise explanation of the lesson objective, subject matter, and teaching method to be employed. Patterns (lesson plan organization)— The logical ways of organizing main or subpoints of a lesson; includes time space, problem-soluti on, pro-con, cause-effect, or topical.
Performance—Part of a criterion objective describing the observable student behavior (or the product of that behavior) acceptable to the instruct or as proof that learning has occurred. Performance-Supervision Phase— A phase of the demonstration-pe rformance teaching method during which students, under the supervision of the instructor, practice the skill they are learning.
Post-test—A test given to a student upon completion of a learning experience to measure achievement.
Pretest— A test given to students prior to entry into a learning environment to determine entry skills or knowledge; can be used to identify portions of the instruction the student can bypass. Principle— A casual relationship between two or more concepts; a rule, axiom, theorem, or algorithm. Proof Support— A type of instructional material used duri ng the body of a lesson that provides hard data or expert testimony in support of an assertion. Psychomotor Domain—A major area of learning that deals with acquiring the ability to perform discrete physical skills requiring dexterity, coordination, and muscular activity. Rating Scales— Any of a number of instruments upon which instructors record their assessments of student performance through a process of observation or measurement and judgment. Receiving— Lowest level of affective domain (Krathwohl, 1964) in which students become aware of and pay attention to so meone or something.AFMAN36-2236 12 NOVEMBER 2003 387

Relay Question— The teacher's response to a student-initiated question whereby the instructor redirects it to another student to answer. Reliability—A characteristic of evaluation that requires te sting instruments yield consistent results. (See Consistency. ) Remotivation Step— Segment of a lesson conclusion during which the instructor explains to them how they can use the information presented and challe nges the students to use what they have learned. Responding—A level of affective domain (Krathwohl, 196 4) in principle in which students act or comply with the instructor's expectations by performing an act and obtain satisfaction from it. Reverse Question—A teacher's response to a student-initiated question whereby the instructor redirects the question to the student who asked it. Rhetorical Question— An instructor-initiated overhead questi on directed to a group but expecting no

answer. Sample of Behavior— A statement of student behavior that, if performed correctly, indicates to the teacher that the students can perform a significant aspect of the lesson objective. A description of behaviors from which the teacher can draw to write test questions. (See Affective Sample of Behavior and Cognitive Sample of Behavior. ) Self-Concept—A relatively stable framework of beliefs about ourselves that helps us act in a consistent

manner. Specific-to-General Presentations— The process or pattern of outlini ng lesson main and subpoints so as to start by examining elements of the original subject then drawing a conclusion about the original subject of the objective. Spontaneous Question— An unplanned, instructor-initiated question used to seek clarification, probe for understanding, or to control the direc tion of the discussion; may be eith er a direct or overhead question.
Standards Statement—Part of a criterion objective that describes the qualitative and quantitative criteria against which student performance or the product of that performance will be measured to determine successful learning. Strategies (lesson plan organization)—Provides the designer's intent during implementation. Student-Centered Instructional Objective— A statement of the student's learning goal, identifying the level of learning and subject of th e lesson; the description of the components of a domain of learning. (See Domain of Learning. ) This manual focuses on three that describe the cognitive, affective, and psychomotor domains respectively that are hierarchical in nature. (See Hierarchy. ) Summary— Segment of a lesson conclusion during which th e instructor reiterates key points of lesson content (knowledge level) or review s and expands on key material and de velops relationships that lead to generalizations (comprehension level).
Summative Evaluation— An evaluation of student achievement of instructional objectives at the end of the learning experience used to measure and report the student's class standing or success in achieving the objectives with the emphasis being on assigning a grade. Support— A type of instructional material used during th e body of a lesson to clarify, characterize, or prove an assertion, claim, or idea. (See Clarification Supportand Proof Support. )388 AFMAN36-2236 12 NOVEMBER 2003

Synthesis— Level of cognitive domain (Bloom, 1956) in which students are able to put parts together to form new patterns or structures. Task Steps— The sequential, component steps in a larger task; represented by achievement of a criterion objective. Taxonomy of Educational Objectives— A systematic classification scheme for sorting learning outcomes into three broad categories (cognitive, af fective, and psychomotor) and rank ordering these outcomes in a developmental hierarchy from least complex to most complex. Teaching Interview—A learning experience in which an inst ructor questions a visiting expert and follows a highly structured plan which leads to achieving educational objectives. Teaching Lecture— A formal or informal presentation of info rmation, concepts, or principles by the teachers.
Transition— Statements used by the instructor to move from the introduction of a lesson to the body, between main points, between subpoints within each main point, and from the body to the conclusion of the lesson. These statements show a logical relati onship between the lesson segments they connect.
Translation— A type of learning (Bloom, 1956) at the comprehension level in which students demonstrate sufficient learning to grasp the meani ng of a concept, principle, or other communication. Validity— A characteristic of evaluation that requires tes ting instruments measure exactly what they were intended to measure. Valuing— Level of affective domain (Krathwohl, 1964) in which students accept, prefer, or commit themselves to an object or behavior because of its perceived worth or value; to appreciate.
Variable Attributes— Characteristics shared by some, but not all, members of a class of people, objects, events, ideas, or actions which are grouped together on the basis of shared critical attributes and called by the same concept name.AFMAN36-2236 12 NOVEMBER 2003 389

Attachment 2

EXAMPLE LESSON PLAN FOR FORMAL LECTURE METHOD

ACADEMIC INSTRUCTOR SCHOOL Maxwell Air Force Base, Alabama

PART I
COVER SHEET

LESSON TITLE: Early Developments in Military Aviation
RESOURCE PERSON:
TEACHING METHOD: Formal Lecture
REFERENCES: Instructor Handbook for The Air Force Today , AFROTC/AU/1974; US Air Power: Ascension to Prominence , AFROTC/AU/1974; Of Those Who Fly , AFROTC/AU/1972; Through the Eyes of Aviators , AFROTC/AU/1973.
AIDS/HANDOUTS/NOTETAKERS: Overview chart, overhead projector
STUDENT PREPARATION/READING ASSIGNMENT:
PRESENTATION TIME: 30-37 Minutes

PART IA
COGNITIVE OBJECTIVE: The objective of this lesson is for each student to know the significant developments in military aviation (1) prior to World War I, (2) during World War I, and (3) between World War I and World War II.
COGNITIVE SAMPLES OF BEHAVIOR: 1. a. Describe the origins of military aviation. b. Identify uses of military aircraft prior to World War I. c. List accomplishments of selected military aviators prior to World War I. 2. a. Select from a list military air functions in World War I. b. State the extent of US military air contributions in World War I. c. Identify significant accomplishments of aviators during World War I.
3. a. Name General Mitchell as Commander Ae ronautical Branch after World War I. b. Outline the major accomplishments of General Mitchell prior to his court-martial. c. State the primary mission of the Army Air Co rps Tactical School between World War I and II.390 AFMAN36-2236 12 NOVEMBER 2003

AFFECTIVE OBJECTIVE: The objective of this lesson is for each student to willingly receive the lecture on the significant developments in military avia tion prior to, during, and just after World War I.
AFFECTIVE SAMPLES OF BEHAVIOR: 1. Voluntarily takes notes during the lesson on si gnificant developments in military aviation. 2. Asks questions of the lecturer on the significant developments in military aviation.3. Encourages others to listen attentively during the lecture on significant developments in military aviation.
PART IB
ORGANIZATIONAL PATTERN: Chronological
STRATEGY: These students are just beginning to study military history, so I will present these developments chronologically rather than topically because the chronological pattern will gi ve them a better sense of the relative speed of these developments and how th ey built on each other. I've divided the lecture into three periods that reflect very si gnificant escalations and deescalati ons of exigencies brought on by external forces. This will show how need breeds more rapid developments than the natural progress of discovery through ongoing experimentation. I will proceed from the earlier period to the later period after the war. I have chosen the formal lecture because of th e anticipated size of the audience and the amount of new material I recently incorporated into the presenta tion not yet recounted in a single, easily accessible reading for the students.
LESSON OUTLINE: MP1. Significant developments in military aviation prior to World War I: a. First flight of heavier-than-air craft b. Military Aeronautical Division in Signal Corps c. First US military aircraft d. First US aviation combat operations e. US entry into World War I MP2. Significant developments in military aviation during World War I: a. European development b. S entry into the war c. S accomplishments and personalities d. t. Mihiel and Meuse-Argonne e. ublic recognition of air battles MP3. Significant developments in military aviation between World War I and World War II: a. General Billy Mitchell b. Air Corps Tactical School c. Moves toward a separate air service d. Advances in equipment and technologyAFMAN36-2236 12 NOVEMBER 2003 391

ACADEMIC INSTRUCTOR SCHOOL Maxwell Air Force Base, Alabama

PART II
TEACHING PLAN

Introduction
ATTENTION: (Rhetorical Questions) Who was the first Airman to be awarded the Medal of Honor? Did you know that Captain Barry Goldwater flew in the first flight of single engine fighters to cross the Atlantic? Did you know that Lieutenant Frank Luke had 18 kills in 18 days during World War I?
MOTIVATION: As ROTC cadets, each of you is considering a commitment to the Air Force. Knowing the history of our service (particularly of US mi litary aviation) can help you strengthen that commitment and develop an appreciation for the professional heritage we share.
OVERVIEW: (Chart paper) We will review selected events and personalities of early military aviation during three broad periods: 1. Prior to World War I (1907-1917) 2. During World War I (1917-1918) 3. Between the World Wars (1919-1939)
TRANSITION: Today in military aviation we have sophis ticated aircraft with complex weapons systems. The situation was quite different in the earliest periods of military aviation.

Body
MP1. Significant developments in military aviation prior to World War I. a. First flight of heavier-than-air craft (1) Wright brothers - 1903 (2) Discontinued activities due to patent problems - 1905-08 b. Military Aeronautical Division in Signal Corps (1) Prompted by inquiry from President Roosevelt (2) Included one officer, two enlisted personnel (3) Operations confidential c. First US military aircraft (1) Specifications developed - 1907: 40 mph, 1 hour flying time, 2 passengers, able to land without damage, "dismountable" (2) Three contracts awarded initially—two forfeited (3) Acceptance flight flown - 1909 (a) Pilot—Orville Wright392 AFMAN36-2236 12 NOVEMBER 2003

(b) Army observer—Lt Frank P. Lahm (4) No operational mission defined—planes used (a) For reconnaissance (b) As directed by commander d. First US aviation combat operations (1) General Pershing in Mexico - 1916 (2) Reconnaissance and communications dispatch (3) Eight aircraft taken—six destroyed and two returned as unsafe (4) Need for more and better equipment e. US entry into World War I - 1917 (1) 35 flying officers - 1,000 enlisted men
TRANSITION: Military aviation prior to World War I was characterized by indecision and confusion.No clear mission beyond reconnaissance was established. World War I brought with it the need for military aviation in various roles and both the mission and the force expanded greatly.
MP2. Significant developments in military aviation during World War I. a. European development (1) More rapid than US (2) Extensive early reconnaissance and artillery direction (3) Efforts to disrupt reconnaissanc e led to aerial combat and pursuit (4) Development of air superiority strategies and tactics b. US entry into the war—late in 1917 (1) No aviation involvement until spring 1918 (2) European equipment and tactics used (3) Primary roles were reconnaissance and pursuit c. US accomplishments and personalities (1) Lt Field Kindley (a) 148 Aero Squadron (b) First American credited with aircraft kill - 13 July 1918 (2) Lt Frank Luke (a) Flew combat mid August to 29 September 1919 (b) Characterized by Bill Luke as headstrong, undisciplined (c) Once landed beside burning balloon he ha d shot down to get confirming signature from ground witnesses (d) 18 September 1918 - five kills in 10 minutes (two balloons, 3 aircraft) (e) 29 September 1918 - three kills in last mission (18 in 18 days) (f) Killed in action—first Americ an to win Medal of HonorAFMAN36-2236 12 NOVEMBER 2003 393

(3) Capt Eddie Rickenbacker—leading American ace of World War I d. St. Mihiel and Meuse-Argonne (1) Control point for almost 1,500 aircraft—one-third American (2) Commanded by Gen Billy Mitchell (3) New concept of air supremacy under central control e. Public recognition of air battles (1) Exciting and colorful (2) Contrast to ground actions
TRANSITION: World War I gave US military aviation its first combat test and a clear mission. Acknowledgment of its full value, however, would have to be achieved during the years between the World Wars. MP3. Significant developments in military aviation between World War I and World War II. a. Gen Billy Mitchell (1) Director of Military Aeronautics - 1919 (2) Early believer in offensive air doctrine and equal status with Army and Navy (3) Air Service - 1920—combatant branch of Army (4) Demonstration of bombing effectiveness - 1921 (a) German battleship sunk (b) Navy interest in aviation followed (c) Proved defensive and offensive use of bombing (5) Courts-Martial - 1925 (a) Charges preferred by President Coolidge (b) Used as a forum by political and military enemies (c) Given 5-year suspension—chose to resign b. Air Corps Tactical School (1) 1920 - Langley; 1931 - Maxwell (2) Little experience on which to base teac hing concerned with developing air doctrine (a) Strategic bombing doctrin e developed—led to B-17 (b) Pursuit doctrine neglected—fighter e quipment and tactics not developed until 1943 (3) Responsible for almost all military aviati on strategy and tactics between World Wars— training ground for World War II leadership c. Moves toward a separate air service (1) Gen Billy Mitchell was early leader (2) No major changes made by Army and Navy indefining separate role for military aviation (3) No clear decision by Army and Navy for responsibility of coastal defense d. Advances in equipment and technology394 AFMAN36-2236 12 NOVEMBER 2003

(1) Lindbergh's cross-Atlantic flight (1927) gave impetus to military aviation—no limit to flying capabilities (2) Early refueling efforts (3) Instrument flight capability (4) First all-metal aircraft—Martin B-10 bomber (5) B-17 in 1937, higher and faster than pursuit aircraft, carried up to 5,000-lb bomb load
TRANSITION: Military aviation showed tremendous adva ncement from 1903 to the beginning of World War II. We have discussed only a few highlights of thatadvancement, but there are some events and individuals especially important to remember.

Conclusion
SUMMARY: 1. Significant developments in military aviation prior to World War I a. Wright Brothers - 1903
b. Problems in Mexico - 1906 c. Military Air in the Signal Corps - 1907 2. Significant developments in military aviation during World War I a. Europeans ahead of US in equipment and tactics b. Primary mission was reconnaissance and patrol/pursuit c. Secondary mission was bombing, interdiction, and communications d. Public recognition was large and positive 3. Significant developments in military aviation between World War I and World War II a. Gen Mitchell was early true believe r—frustrations and court martial b. Army Air Corps Tactical School (1) Developed strategic bombing doctrine (2) Training ground for World War II leadership (3) Responsible for development of B-17 c. No successful moves toward separate air service d. Progress in technology crowned by B-17
REMOTIVATION: Today we have looked at some highlights of military aviation in its early days. Many more fascinating events and individuals are included inthis time frame. I encourage you to read and learn as much as you can about them. Knowledge of and appr eciation for our professiona l heritage can help to make each of us more informed, dedicate d, and effective as Air Force professionals.
CLOSURE: Study of history can prevent us from having to "reinvent the wheel" with each generation, each job, or each change in personnel.AFMAN36-2236 12 NOVEMBER 2003 395

ACADEMIC INSTRUCTOR SCHOOL Maxwell Air Force Base, Alabama

PART III
TEST ITEMS
ITEM 1
LESSON OBJECTIVE: The objective of this lesson is for each student to know significant developments in military aviation during World War I.
SAMPLE OF BEHAVIOR: Select from a list military air functions in World War I. QUESTION: The two primary military air functions in World War I were patrol/pursuit and a. Courier. b. Interdiction. c. Reconnaissance.
RATIONALE: Reconnaissance (c) is correct—consistent withlecture content. The other choices (a and b) are incorrect because those functions were not attr ibutable to the years associated with World War I.
ITEM 2
LESSON OBJECTIVE: The objective of this lesson is for each student to know significant developments in military aviation during World War I.
SAMPLE OF BEHAVIOR: Identify significant accomplishments of selected aviators during World War I. QUESTION: Column A lists accomplishments in aviation. Column B lists World War I aviators. In the space to the left of each accomplishment, place the letter which matches the aviator responsible for that accomplishment. Each item in Column B may be used once or more than once.

KEY:
1. a
2. b
3. b
4. c

Column A—Accomplishments Column B—Aviators 1. First American credited with ai rcraft kill a. Lt Field Kindley 2. First American to win Medal of Honor b. Lt Frank Luke 3. Five kills in 10 minutes c. Capt Eddie Rickenbacker 4. Leading American ace of World War I396 AFMAN36-2236 12 NOVEMBER 2003

Attachment 3

EXAMPLE LESSON PLAN FOR INFORMAL LECTURE METHOD

ACADEMIC INSTRUCTOR SCHOOL Maxwell Air Force Base, Alabama

PART I
COVER SHEET
LESSON TITLE: Time Management
RESOURCE PERSON:
TEACHING METHOD: Informal Lecture
REFERENCES:
1. How to Find the Time You Need Through Better Time Management , Robert C. Dorney
2. How to Manage Your Time More Effectively With a Day-Timer, Charles R. Hobbs
3. How to Get Control of Your Time, Alan Laiken
AIDS/HANDOUTS/NOTETAKERS: N/A
STUDENT PREPARATION/READING ASSIGNMENT:
PRESENTATION TIME: 30-37 Minutes

PART IA
COGNITIVE OBJECTIVE: The objective of this lesson is for each student to comprehend the concept of time management.
COGNITIVE SAMPLES OF BEHAVIOR: 1. Define time management in one's own words. 2. Identify missing critical attributes in new nonexamples of time management. 3. Identify new examples/nonexa mples of time management. 4. Explain why a given attribute is critical to the concept of time management.
AFFECTIVE OBJECTIVE: The objective of this lesson is for each student to respond with interest to the concept of time management.
AFFECTIVE SAMPLES OF BEHAVIOR: 1. Openly participates in class discussion on the concept of time management.
2. Assists others in unde rstanding time management. 3. Volunteers information from personal background to support the comprehension of time management.AFMAN36-2236 12 NOVEMBER 2003 397

4. Displays positive, nonverbal communications during lesson on time management.

PART IB
ORGANIZATIONAL PATTERN: Topical
STRATEGY: Present this 30- to 37-minute lesson using the informal lecture method and the general-to-specific format to help the students logicallyfollow the flow of the lesson. A deductive approach will be helpful in leading the students to comprehe nd the lesson objective. Define and explain the concept of time management by applying its crit ical attributes and tying them together to present the total concept. The lesson begins with a role play scenario of a pers on just tasked to do more projects because he or she is efficient. After the scenario, as k the rhetorical question, “Have you ever said to yourself, ‘If only I had more time.... !’ or ‘There just aren’t enough hours in the day!’”? Inform the students of the need to listen to the lesson. Maybe a statement like “if you have felt this way, it’s important to understand how to manage your time so you don’t become frustrated at work.” Then, present an overview so the students will know the key ideas and the direction this lesson is going. Start the body by defining time management from a working definition based on personal experience dealing with time management issues. This will establish a foundation for the lesson and help the stude nts focus on the concept of time management. Then break down the critical attributes of time management by defining and s upporting each attribute. Begin by first defining and explaining analysis. Then explain and define what’s meant by adjustment, performance, and goals and how they work together to ensure proper time management. Write each attribute on the white board and explain using examples to incr ease student understanding. Ask questions during MP2, the critical attributes to ensure student comprehensi on of the material. In MP3, provide scenarios and ask the students to evaluate them based on the critical attr ibutes as either an example or nonexample of time management. If the students are able to distinguishbetween examples and nonexamples, you will feel confident they comprehend the conc ept of time management. Summarize the lesson by giving them the definition and critical attributes once more and reviewing the scenario s to ensure student understanding of the concept of time management. The lesson will then close with the proper remotivation, so the students will feel a need to retain and use the information. Finally, use an appropriate closure to end the lesson.
LESSON OUTLINE:
MP1. Definition of time management. MP2. Critical attributes: a. Analysis b. Adjustment c. Performance d. Goals MP3. Examples and nonexamples (scenarios): a. Bob, example b. Bill, nonexample c. Doyle, example398 AFMAN36-2236 12 NOVEMBER 2003

ACADEMIC INSTRUCTOR SCHOOL Maxwell Air Force Base, Alabama

PART II
TEACHING PLAN

Introduction
ATTENTION: Role play scenario of a person just tasked to do more projects because he or she is so efficient (videotape sequence ahead of time). After scen ario, ask rhetorical question, "Have you ever said to yourself, 'If only I had more time...!' or 'There just aren't enough hours in the day!'"?
MOTIVATION: Recall that Robert C. Dorney, the vice pr esident of a time management company said that time is a fixed commodity that can't be manufac tured or stored. Also rec ount that Peter Drucker, author and one of our country's foremost experts on the business scene has said, "Time is our scarcest resource and unless it is managed, nothing else can be managed." Explain the importance of our roles in protecting national resources and even life (at times). Stress the importance of continually doing the best in the time we have available. Personalize this importance to the individuals in the class according to their career fields showing how time management can help them finish PME on time or early and how they can stand out as the experts simply by competently doing what needs to be done when it needs to be done.
OVERVIEW: Before we can manage time properly, we mu st first understand exactly what time management is; that's exactly what we are going to do toda y. We are going to first define time management. From that definition we will see that four critical attributes must be present for time management to exist. Then, to ensure we fully understand this concept of time management, I will presen t some situations to you where you will need to determine if those situations are examples of time manage ment or not. This is an informal lecture, so feel free to ask questions. I will ask you some as well.
TRANSITION: We have already touched on the importanc e of time management. Let's see why it's important by showing what it is.

Body
MP1. Definition: Having researched extensively on the topic of time management, I found several excellent works. Two in particular seem to relate to our military environment more easily than others: Alan Laiken in his book, How to Get Control of Your Time, and Robert Dorney's, How to Find the Time You Need Through Better Time Management

. Even with these excellent books, I discovered that ne ither one had the perfect definition for our purpose. What I did find was that by combining the thoughts and words of these two individuals we could create a working definition that clearly applies to our environment.AFMAN36-2236 12 NOVEMBER 2003 399

Time management is: "Continually analyzing time and tasks and adjusting task requirements to most efficiently and effectively accomplish goals that have been established and prioritized."
TRANSITION: Having created our own working definition that accurately tells us what time management is, we need to break that definition down to see just why this represents time management.
MP2. Critical Attributes: (Reference definition) Looking at our definition "Continually analyzing time and tasks" we will refer to as analysis (mag card). "Adjusting task requirements" which means we must react to our analysis, we'll refer to as adjustment (mag card). "Most efficiently and effectively accomplish our goals" (the purpose of adjusting) or performance (mag card). These goals don't just fall out of the sky; therefore, we must work with "goals th at have been established and prioritized" goals (mag card). If all four of these critical attributes are present in a given situation, we have time manage ment. But it takes more time than this to really comprehend this idea, so let's analyze each of these attributes further: A. Goals I don't mean to confuse you, but we need to start here at the bottom with goals. Why do you think we should address the idea of establishing goals before any of these other items? AR: Must have direction/objective.
Provides focus.
Don't know what task must be done without them.
Communication—if the team doesn't know what the goal is, they may be working against
each other. A goal is the result or achievement toward which effo rt is directed. Establishing a goal, then, is making a conscious decision or having knowledge of the desire d outcome. What are some of your current personal and professional goals? AR: Particular work project/tasking.
School. Study for promotion. So you all have established goals—whether they are pe rsonal or have been established for you be someone else. But, as we all know, there are only so many hours in a day, week, month, etc. So we have to make choices. We have to prioritize or have priorities made for us. Th ere must be an establishment of precedence, what is needed first, second, third, etc. Although it appears at the end of our definition, if we are going to have effective time management it must start with establishing and prioritizing our goals.400 AFMAN36-2236 12 NOVEMBER 2003

B. Analysis Once we have this solid list of prioritized goals, we must continually analyze both the time and tasks involved with reaching those goals. How do you analyze a task?
AR: Break it down to see. What is required by the end user?
What resources are needed? Who is qualified to perform the task? So you must know what the expected outcome is and how you will be able to accomplish that task. On rare occasions the task has an unlimited time requirement, but usually time is a critical factor. Therefore, time management inherently requires you to also analyze your time. How much time will it take to accomplish task X, Y, Z? How much time do I have to work with? What other constraints from tasks or duties do I have to deal with? It's one thing to analyze your tasks and another to an alyze your time, but putting those two critical factors together is where it starts getting tricky. What are some situations you've been in where you had to analyze both your tasks and time?
AR: Job and school. Job and family/house. Two, three, and four tasks on the job that all needed to be done now

. C. Adjustment
Let's assume we have established a nd prioritized our goals and analyzed the time and task elements. That sounds good; however, you can have the best analysis in town, but unless you react and adjust to those elements you have not managed your time. The reason you need to continually analyze your time and tasks is that they are usually very fluid and dynamic —they are always changing. Priority #3 just became priority #1, task #1 needs a part that will not arrive fo r 2 more days, priority #4 was just modified and only requires a quick 30-minute fix, General X just walked in to check on his "baby" which until that minute was your priority #51, etc.... You cannot change the number of hours in a day, but effective time management requires you to adjust tasks based on your continual analysis of both the time and the tasks. Adjustment— it's a proactive response, not a reactive reaction. Why do all of this? If priority #3 just became priority#1, so what, I only had a day's worth of work left on it to finish—why not just finish it? If task #1 needs a part that will not arrive for 2 more days—why not just sit down and wait for it to show up? D. Performance Although we often get frustrated by changing priorities, the fact is that things change, situations (needed parts) change. To meet our goals , we and our customers need quality products and service in a timely manner—quality, quantity, efficiency, and effectiveness. We must perform our duties and get our job done by completing the form or report by getting the aircraft launched ensuring safe parameters.AFMAN36-2236 12 NOVEMBER 2003 401

I would wager that all of us at one time or another has had a true "want and need" for more time. Time management, something most of us do quite well but probably wish we could do even better. To do it better is to first truly understand "what" time management is.
INTERIM SUMMARY: If we fail at time management, one of the most likely reasons stems from the first critical attribute we spoke of, goals. Our goals must be established and prioritized, both the long-range goals and the short-term goals. We must continually analyze both the time and tasks involved in reaching those goals. It does no good to analyze unless you actually react and adjust the task requirements based on your analysis. Keep in mind, however, that no action or adjustment is sometimes the correct action or adjustment. The purpose of course brings it full circle, in that the accomplishment of our goals can be performed in the most efficient and effective manner.
TRANSITION: Having managed our time here effectively, we have just enough time to see whether you can now distinguish time management from time mismanagement.
MP3: Examples/Nonexamples Hand out scenarios and have students discuss why each one is or is not an example of time management (see attached).402 AFMAN36-2236 12 NOVEMBER 2003

Answers to scenarios: Scenario #1, Bob: This is an example. Although he got off to a slow start, Maj Bob did establish and prioritize goals. He now seems to be continually analyzing both the time and tasks involved and making appropriate adjustments to more efficiently and effectively accomplish those goals. Scenario #2, Bill: This is a nonexample. Although he acknowledged his shortcomings and took the class and analyzed his overall situation, Capt Bill di d not establish goals. If he does not establish goals he cannot possibly prioritize them. Without those esta blished and prioritized goals, all the analysis of time and tasks will serve no purpose because whatadjustments he would make would have no foundation, therefore eroding what efficiency and effe ctiveness may or may not have been there. Scenario #3, Doyle: This is an example. Mr. Doyle establishes and prioritizes goals along with his managers. Mr. Doyle continually reviews and resuspen ses (reacts) based upon his analysis of time and tasks. Company results indicate that things are running in an efficient and effective manner.

TRANSITION: Excellent. We have managed our time extremely well.

Conclusion
SUMMARY: We started this hour with a definition of time management, saying that it is "Continually analyzing time and tasks, and adjusting task requirements to most efficiently and effectively accomplishgoals that have been established and prioritized." From that definition we determined that for time management to exist we needed to continually analyze our time and tasks, react to our analysis by adjusting for the purpose, and performing efficiently and effectively those goals that have been established (our critical attributes). With that understanding, you were able to identify examples and nonexamples of time management. We saw that both Maj Bob and Mr Doyle were mana ging their time. While Capt Bill attempted to manage his time, he had never establishe d his goals—critical to our concept of time management.
REMOTIVATION: Now that you have a tool to be more effi cient and effective, you have an obligation to use it on a daily basis. It can only result in career success for you, your unit, and the Air Force.
CLOSURE: Even though time cannot be created or recovered, it can be lived to the fullest here and now by our using it correctly day by day, hour by hour.AFMAN36-2236 12 NOVEMBER 2003 403

SCENARIO 1: Bob is a major with 15 years in the Air Force. He is in charge of the Information Systems Division in his squadron and has four people working directly for him. He is also married and the father of 3 children, ages 12, 18, and 8. Bob notices that things are pretty hectic at work. He feels overburdened and notices a streak of rebellion in his children over the last 2 months. Bob has decided he is stretched to the limit and has no time for anything. He decides to take a time mana gement class. After the cl ass, he begins to take action. He establishes and prioritizes his top three l ong-range and top three short- term goals. He takes a survey of his time for 2 weeks. A pattern emerges. He discovers there are many things that can and should be delegated to his subordinates. There are many things that his division is doing that shouldn't be done and he decides to tactfully return the responsibility for those things to the division that should have responsibility. Bob discovers he has more free time to begin working on reports that were due 2 monthsago. He then takes his survey and breaks it down into three classifications: daily goals, career and company goals, and personal goals. Breaking the items on his list into: A - vital, B - important, C - some value, and D - complete waste, he finds that his daily company and career goals are coming into order. In the area of personal goals, he realizes he is not spending qual ity time with his children. He then reviews his goals in this area and prioritizes them. Bob decides that his usual Wednesday night poker game with the boys is time better spent with his children. In 2 months, he not ices the streak of rebellion that was so evident in his children is now gone. QUESTION: Why is this an example or nonexample of time management? SCENARIO 2: Bill is a captain with 12 years in the Air Force. He is in charge of the Disaster Preparedness Division for the wing. He has a master sergeant and a technical serg eant working directly for him. They are very sharp and Bill has things pretty easy because of these tw o talented coworkers. His boss knows this and decides to task him with the base voter registration drive and also the Combined Federal Campaign. Overnight, Bill has deadlines and reports due for these projects. He has also just found out his section will be assuming the disaster preparedness duties for another near by base because of budget cutbacks. Bill reads in the base newspaper about a time management class being offered. He decide s to take it. Afte r taking the class, Bill decides to put what he has learne d into practice. He starts by analyzing his time to find out where it is going. He then prioritizes each area. Bill decides afte r all this work he deserves a vacation. Things are coming together. QUESTION: Why is this an example or nonexample of time management? SCENARIO 3: Doyle is a happy-go-lucky fellow on the fast track athis company. He runs the budget division and has succeeded in cutting expenditures by 7 percent over the pa st year. He attributes this success to his semiannual meetings with managers at wh ich he reviews the company goals, division goals, and his goals. After this meeting, supervisors meet with him and with th eir own workers so that bosses can review individual goals with their subordinates. Doyle also sends out memos reviewing all divisional suspenses and prioritizing them for this level but encouraging the supervisor s to reprioritize those that pertain directly to themselves. As Christmas gifts, he gave each supervisor a day-timer for the office with training on how to record appointments, to-do lists, and suspenses. Anindependent consultant cam e in and confirmed that these day-timers are being used effectively by 80 percen t of the supervisors and that 65 percent of the subordinate budget analysts also started using them. Doyle intends to reward the success of his division by giving all staff members some time off. QUESTION: Why is this an example or nonexample of time management?404 AFMAN36-2236 12 NOVEMBER 2003

Attachment 4

EXAMPLE LESSON PLAN FOR GUIDED DISCUSSION METHOD PROBLEM-SOLUTION PATTERN

ACADEMIC INSTRUCTOR SCHOOL Maxwell Air Force Base, Alabama

PART I COVER SHEET

LESSON TITLE: Deployments Decrease Area Tensions RESOURCE PERSON: TEACHING METHOD: Guided Discussion REFERENCES: 1. Personal Experiences 2. Navy Times 3. http://www.quoteland.com/quotes/author/394.html 4. http://www.pbs.org/wgbh/amex/presid ents/nf/record/tr/trooseveltfp.html AIDS HANDOUTS/NOTETAKERS: STUDENT PREPARATION/READING ASSIGNMENT: PRESENTATION TIME: 30-37 Minutes

PART IA COGNITIVE OBJECTIVE: The objective of this lesson is for each student to comprehend that deployments decrease area tensions. COGNITIVE SAMPLES OF BEHAVIOR: 1. Describe how deployments decrease area tensions. 2. Explain why it is important to decrease area tensions. 3. Defend the statement that deploym ents decrease area tensions. 4. Predict the effect of area tensions when deployments occur. AFFECTIVE OBJECTIVE: The objective of this lesson is for each student to respond with interest to the principle that deploymen ts decrease area tensions. AFFECTIVE SAMPLES OF BEHAVIOR: 1. Voluntarily participates in class discussi on on how deployments decrease area tensions. 2. Displays positive, nonverbal communications duri ng lesson on how deployments decrease area tensions. 3. Shares relevant experiences and stories on how deployments decrease area tensions.AFMAN36-2236 12 NOVEMBER 2003 405

PART IB
ORGANIZATIONAL PATTERN: Problem/Solution STRATEGY: Present this lesson using the guided discussion method. Do this because most students will have either experienced themselves or had contact wi th others who have been on deployment. Also, most students will have knowledge of current world events and the United States' role in them. A problem/solution approach will be helpful for the students to achieve the cognitive lesson objective because it will provide a different perspective. This lesson is impor tant because most military members have or will experience deployment sometime in their careers or know someone who has deployed (or will). Sometimes, we are too close to the "front line" to see the big picture. This lesson will show that there is a reason we do the things we do (go on deployment). Begin the lesson by passing out copies of the latest "All the ships at sea" from Navy Times. Explain that this is where the Navy is currently on deployment and ask if anyone has been on deployment (if no one has, ask if they know of anybody who is or has been). Have students expound—where did they go, how long were they gone, what did they do while in the country to support their mission, what was their mission. Tell students why they need to participate in the lesson (see importance of lesson above). Then, present an overv iew of the lesson. Start by having a student define deployment so everyone is starting on the same "she et of music." Then, intr oduce the problem/solution approach to this topic. The problem is that increased area tensions threaten national security. The solution is that deployments decrease area tensions. The stude nts will discuss both the problem and the validity of the solution using the deductive approach. I will star t the discussion of the problem using a leadoff question. By using followup questions, I can make sure the students look at how increased area tensions threaten national security through war (or armed confli ct), how the closure of sea lanes (which I will define for them prior to the followup question) threatens national securi ty, and how nondemocratic aggression threatens national security. Once the stude nts have adequately disc ussed the problem, I will summarize using the inputs provided. Then I will use a leadoff question to start the students discussing how deployments decrease area tension. I will use follo wup questions to make sure the students look at how deployments decrease area tensions through our presence and through international training and cooperation. Once the students have adequately disc ussed the solution, I will summarize the problem again and then summarize the solution. I will remotivate by again emphasizing that all military personnel may be called upon to deploy and that it helps to unders tand that there is a purpose. I will conclude with the following quote from Theodore Roosevelt that sums up what a deployment is all about: "Speak softly and carry a big stick." During the discussion, I will use PowerPoint to display the main points.

LESSON OUTLINE: Main Points with Subpoints: MP1. Problem - Area tensions threaten national security. a. War (or armed conflict) threatens national security. b. Closure of sea lanes threatens national security. c. Nondemocratic aggression threatens national security. MP2. Solution - Deployments decrease area tensions. a. Presence decreases area tensions. b. International training decreases area tensions. c. International cooperation decreases area tensions.406 AFMAN36-2236 12 NOVEMBER 2003

ACADEMIC INSTRUCTOR SCHOOL Maxwell Air Force Base, Alabama

PART II

TEACHING PLAN

Introduction

ATTENTION: (Pass out copies of the latest "All the ships at sea" from Navy Times. Show a slide of the amphibious ship leaving on deployment.) Good mo rning/afternoon everyone, I am _________. What I am passing out is a copy of "All the ships at sea" as of (whatever date). It shows where all the Navy ships are currently. Notice that about one third of the ships are on deployment. I know the Air Force also goes on deployments. How many of you have experienced a scene si milar to this one (point to slide), not deploying on a ship like these men and women are, but le aving loved ones behind to deploy? Where did you go? What did you do while you were there? What was your unit's mission? (If no one has been on deployment, ask if anyone has gone and ask the same questions. If no one can do that either, then give your own experience.)
MOTIVATION: We all have seen on CNN troops being mobi lized to go on deployment or aircraft taking off and landing on a carrier for deployment. Have you asked yourself, "Why are these people there? What is the purpose of sending American soldiers, sailors , airmen, and marines on deployment?" Being in the military, it is very possible that some of us at one time or another will be tasked to deploy or at least know someone who has deployed. So metimes we can be too close to the "f ront lines" to really understand why we deploy. Today, we are going to look at some reasons why the United States military deploys.
OVERVIEW: During the next 35 minutes or so, we will discuss how increased area tensions threaten national security. I think we would all agree this is a problem. One of the most effective solutions the United States has found to this problem is thr ough deployments. We will discuss how deployments decrease area tensions. Before we move on, a good definition of a deployment (as used by the US military) is the sending of troops temporarily to an area of the world to maintain a US presence or the taking of personnel and equipment from the United States and transporting them to a "hot spot" to cool it down. This is going to be a guided discussion. I will ask a fe w questions, but I want you to discuss the topic at hand. I will take notes on what you say so I can summari ze your ideas. Please feel comfortable to express your opinion and share your experiences. Be courteous to each other and do not attack anyone because of a differing opinion.
TRANSITION: Now that we know where we are going, let us start with the problem—increased area tensions threaten national security.AFMAN36-2236 12 NOVEMBER 2003 407

Body
MP1: Increased area tensions threaten national security.

Instructor Activity Anticipated Responses

Leadoff question (LOQ): How do increased area tensions threaten national security? Political instability will cause tension which may lead to war or armed conflict

National assets may be threatened which will cause increased troop presence because of instability in the region

Resources may be cut off from us if tensions increase, which will probably lead to increased troop presence

Political unrest may cause terrorists to target US citizens which will force the United States to retaliate

a. Followup question (FUQ): How does war or armed conflict threaten national security? We could lose or end up in a stalemate, the government would lose credibility which would cause us to look vulnerable

Lose valuable assets, including people which will directly threaten national security

Might not have the right or enough assets which would put all pers onnel and national assets at risk

Concentrating in one area of the world may leave another "unprotected"; if this occurs, then someone or something will be vulnerable to attack

b. Setup for FUQ: Sea lanes are like highway lanes. They are where ships, both military and civilian, travel. There are some critical sea lanes in the world like the Straits of Taiwan and the Straits of Malaysia near Singapore. FUQ: How would the closure of sea lanes threaten national security? Resources would be cut off from us. If the resources are critical, we will have to take action to supply the necessary resources, which could include armed conflict

Cannot participate in world trade effectively, thus hurting our economy

National pride is injured, we do not like to be told we can't go somewhere408 AFMAN36-2236 12 NOVEMBER 2003

TRANSITION: From your discussion we see that increased area tensions threaten our national security. Let us review what you said.
INTERIM SUMMARY: Summarize student ideas that support the first main point using what they said and, if possible, the terms they used. Add new information from my anticipated responses if they weren't addressed. Restate the first main point, "Increased area tensions threaten our national security."
TRANSITION: The United States has found an effective way to reduce area tensions and thereby protect our national security—by deploying its military men and women.
MP2. Deployments decrease area tensions. c. Lead-in to final FUQ: You have hit on some

r eally good po i nts, let us look at one mor e

. FUQ: H ow does nondemoc r atic agg r ess i on threaten our national secur i ty? Embassy and other in-country personnel can be in danger. Personnel are part of national security

In-country assets are in danger. These assets could be valuable to the population or to the United States

We believe it is in our best interest for the entire world to be democratic, so we feel we have to do what we can to make everyone a democracy

We are closely tied economically to a lot of countries. If they are jeopardized, all resources or products provided are in jeopardy. Our economy is jeopardized

We have a role as the world's "police" to stop aggression. We would lose credibility if we did nothing

Instructor Activity Anticipated Responses LOQ: How do deployments decrease area tensions? Our presence has the effect of decreasing tensions because of military power

Training with other countries creates a common bond with other nations

Quick response time to diffuse tensions

Countries know we have things in place to take care of problems. This relieves tension or fear of attack

a. FUQ: How does our presence decrease area tensions? We have a reputation as being a superior military force to that of most countries that cause the tensionsAFMAN36-2236 12 NOVEMBER 2003 409

TRANSITION: As you have discussed, deployments are imperative to reducing area tensions. Now, let us take a look at what you have discussed today.

Conclusion
FINAL SUMMARY: Briefly review MP1 (increased area tensi ons threaten national security). Then take MP2 (lesson objective/solution) and summarize the student's comments and add any appropriate anticipated responses. State what they concluded—deployments decrease area tensions; therefore, deployments protect national security.

REMOVIVATION: It is always helpful in any task to know why you are doing it. Everyone wants toaccomplish meaningful things. All of us in the military can be tasked to deploy. It can be a planned, cyclical thing like it is in the Navy or it may be a short-notice thing like it is sometimes in the Air Force and theWe usually carry through with our "promises," so if a country "acts up" while forces are in the area, they know they are in trouble

Other countries may not want to feel like they have to rely on us to solve all the world's problems and may put pressure on the offending country to straighten up

b. Setup for FUQ: On navy deployments we more often than not train with other countries and I have been told the Air Force does also. FUQ: How does international training decrease area tensions? We train countries to take care of themselves so we don't always have to be there

If we train with countries on opposite sides of the fence, they may see that they are not so different after all and are able to work their differences out

Individuals in countries will get to know each other and as they go up in position may be able to work with each other

Interoperability if there is a problem

c. Setup for FUQ: On deployments, we are often performing the mission with other countries, like enforcing the no-fly zone over If there is a problem in the area, we work together to solve it. One country may have a novel approach Iraq with the British. FUQ: How does this international cooperation decrease area tensions? With the downsizing, we have fewer resources but must do more. It is easier to accomplish the mission if we have others to help

It is hard for us to go into an area and interfere if we do not have the backing of other countries (Desert Storm vs Vietnam)410 AFMAN36-2236 12 NOVEMBER 2003

Army. In either case, it is nice to know that what we do matters. As you have discussed today, deployments are needed to decrease area tensions and therefore safeguard national security. Even though you may not personally deploy, you will probably know so meone who does. Perhaps they will not know why they are going. With the information discussed today, you can educate your peers and give them a sense of purpose that perhaps they did not have before.

CLOSURE: I would like to leave you now with an African proverb our 26th President, Theodore Roosevelt, was fond of saying. Pres ident Roosevelt believed in exerci sing military muscle in any manner that would bolster the United States as a world pow er. This proverb sums up what we do on deployment. "Speak softly and carry a big stick."AFMAN36-2236 12 NOVEMBER 2003 411

ACADEMIC INSTRUCTOR SCHOOL Maxwell Air Force Base, Alabama

PART III
TEST ITEM

OBJECTIVE: The objective of this lesson is for each stude nt to comprehend that deployments decrease area tensions.
SAMPLE OF BEHAVIOR: Describe how deployments decrease area tensions. QUESTION: Which phrase BEST describes how deployments decrease area tensions? a. participating in international training exercises promotes teamwork. b. stopping nondemocratic aggression reduces stressful situations. c. maintaining freedom of the seas allows countries to freely operate.
RATIONALE: Option a is the correct answer. Participating in international training exercises is a part of many deployments and a way (answering the question "how") to decrease area tensions. By training with other countries, it shows unity to any possible aggresso rs. It also promotes interoperability in case there are problems. This rationale will be discussed by the st udents; if not, it will be brought up as a point by the instructor during the summary of MP2. Options b and c are incorrect because they answer the question of "why" we go on deployments—not "how" deployments decrease area tensions.412 AFMAN36-2236 12 NOVEMBER 2003

Attachment 5

EXAMPLES OF CASE METHOD OBJECTIVES WITH CASE ELEMENTS

ACADEMIC INSTRUCTOR SCHOOL Maxwell Air Force Base, Alabama

Example 1
OBJECTIVE: The objective of this lesson is for each stude nt to apply the ISD process to course development.
MAIN POINTS: 1. Analysis. 2. Design. 3. Development. 4. Implementation.
5. Evaluation. 6. "... apply the ISD process to course development."
BOARD ARRANGEMENT:

Example 2
OBJECTIVE: The objective of this lesson is for each student to apply the Carkuff counseling approach to given counseling situations.
MAIN POINTS: 1. (A) Attending skills. 2. (A) Responding skills. 3. (A) Initiating skills. 4. "... apply the Carkuff counseling approa ch to given counseling situations." Analysis Design Development Implementation Evaluation Application of ISDAFMAN36-2236 12 NOVEMBER 2003 413

BOARD ARRANGEMENT:

Example 3 OBJECTIVE: The objective of this lesson is to analyze the concept of professional ethics in a leadership situation.

MAIN POINTS: 1. (A) Analysis of cast of characters and their relationships with each other. 2. (A) Analysis of ethical considerations of each character. 3. (A) Analysis of the impact of professional ethics in a leadership situation. Board Arrangement: Attending Responding Initiating Application of Approach

Characters Ethics Impact414 AFMAN36-2236 12 NOVEMBER 2003

Attachment 6

EXAMPLE LESSON PLAN FOR CASE STUDY METHOD PROBLEM-SOLUTION PATTERN

ACADEMIC INSTRUCTOR SCHOOL Maxwell Air Force Base, Alabama

PART I
COVER SHEET
LESSON TITLE: Involved Leadership Promotes Subordinate Success
RESOURCE PERSON:
TEACHING METHOD: Case Study
AIDS/HANDOUTS/NOTETAKERS: Case Study: Money Troubles (Attached); Dry erase board.
STUDENT PREPARATION/READING ASSIGNMENT:
REFERENCES: Personal Experiences
PRESENTATION TIME:

PART IA
COGNITIVE OBJECTIVE: TOOTLIFEST comprehend that involved leadership promotes subordinate personal success.
COGNITIVE SAMPLES OF BEHAVIOR: 1. Describe how involved leadership promotes subordinate success. 2. Explain why involved leadership is important to promote subordinate success. 3. Predict how involved leadership will lead to subordinate success.
AFFECTIVE OBJECTIVE: TOOTLIFEST respond with interest to the principle that involved leadership promotes subordinate success.
AFFECTIVE SAMPLES OF BEHAVIOR: 1. Voluntarily participates in class discussion on how involved leadership promotes subordinate success.2. Displays positive, nonverbal communications during lesson on how involved leadership promotes subordinate success. 3. Shares relevant experiences and stories on how involved leadership promotes subordinate success.AFMAN36-2236 12 NOVEMBER 2003 415

PART IB
ORGANIZATIONAL PATTERN: Problem-Solution
STRATEGY: This lesson on comprehending that involved leadership promotes subordinate success will be presented using the case study method with th e problem/solution pattern because this allows the students the opportunity to fully explore the possible problems inherent in a management situation like this one. The students will also participate in formul ating possible solutions to the problems they have identified. Since there are various effective approaches to solving these kinds of problems, the case study method is ideal in that it provides an opportunity to tackle the problems from a number of different perspectives. I will begin by relating an experience I had with a sailor who was in a similar situation as the staff sergeant in this case. I will note that since th e students are or will be supervisors, they should be motivated to develop a caring approach. The student s should understand that the success of our subordinates stems not only from their professional efforts, but also from their personal efforts and we must do all we can to make sure they succeed. I am going to use the facts, assumptions, problems, and solutions (FAPS) model with this case. The facts, assumptions, problems, and solutions constitute the case elements (main points). The FAPS model is ideal since the pattern in the lesson is problem/solution. This model allows students to determine all the problem s before deciding on the solutions. I will go around the room and have different people read the case aloud wh ile the others follow along. This engages the visual and auditory senses increasing the ability to understand the material. I will ask the students to extract the facts from the case. After considering the facts, I will have the students identify logical assumptions based on what they have read and by drawing from their ow n experiences with similar situations. I will then summarize the assumptions. Next, I will have the stude nts determine the problems they see in the case,both with events that occurred and actions that should have been taken. Next, I will summarize the problems found by the students. The students must then form ulate solutions to the problems and pick the best solutions based on their experiences. I will summari ze the case by recapping the facts, assumptions, problems, and the solutions so that the students can clearl y see how crucial leader involvement is to furthering the success of subordinates. Finally, I will close by reminding the students that as supervisors and future supervisors it is important to take time to make sure our people, who are our greatest asset, are successful not only in their professional lives but also in thei r personal lives because the two are intertwined. I will close with an affective statement about how our invol vement as leaders promotes success in our subordinates.

LESSON OUTLINE:
MP1. Facts relevant to involved leader ship promoting subordinate success.
MP2. Assumptions relevant to involved lead ership promoting subordinate success.
MP3. Problems with involved leadership promoting subordinate success.
MP4 . Solutions to problems raised in involved leadership promoting subordinate success.416 AFMAN36-2236 12 NOVEMBER 2003

PART II

TEACHING PLAN

Introduction

ATTENTION: Good morning, I am __________, a Navy surface warfare officer. I have served aboard 2 ships and have supervised from between 10 and 40 people at a time, depending on my division. As a supervisor, I have had to deal with many personnel problems, both work related and personal. One of my more difficult problems dealt with one of my E-4s who was having mone tary difficulties. He was married to a woman whom he cared about very much who had five children by previous relationships. They had marital problems which did not surp rise anyone on the ship and he was seriously considering getting a divorce. He closed their joint checking account and told his wife about it. They separated and he left her in family housing while he moved aboard the ship. She continued to write checks on the closed account, cashing them at the base. She was smart about it b ecause she would write them for the maximum amount, knowing that when the base found out about the closed account, she would not be held accountable, he would. They had some other debts for furniture, stereo equipment, and a car. We received several letters of indebtedness from the base each time a bad ch eck was discovered. Our disbursing officer would give them to me instead of the commanding officer. We went to base legal to see if there was any way to hold her accountable for the money, but there was not. My E- 4 continued to get farther and farther behind in his debts. His pay was garnished to the point that he had less than $100 per payche ck. His work performance was less than what I knew he was capable of. Much to everyone's surprise, he was promoted to E-5. Being the eternal optimist, I predicted that the promotion w ould cause him to take better pride in his work and that his performance would improve. My chief, an E- 7 in the Navy, predicted that nothing would change and that his performance would cont inue its downward spiral. I am happy to report that I was right, the work performance did improve. Although when I left the ship he was still having some personal problems, he was not as despondent about them as he could have been. He appreciated the efforts my chief, the legal officer, and I had made to he lp him. Because I had faith in him and gave him encouragement, he was a great success as the workcenter supervisor.
MOTIVATION: As supervisors and potential supervisors, we are obviously concerned about a subordinate's work performance. But this is only part of thei r lives. They also have a personal life we should be concerned with. Sometimes problems with their home liv es will have devastating effects on their professional lives. They can even be potential career "enders."
OVERVIEW: Today we are going to look at a case somewhat similar to the situation I experienced. The problem for the SSgt we are going to read about shortly starts as a personal one, but is leading down the path toward professional disaster. As we read, I wantyou to think about the facts of the case, the assumptions we can make, what the problems are—both with what is done and perhaps what was not done, and finally, a solution for the lieutenant. (Pick someone to read the first two paragraphs, someone else for the second two, and someone else for the last two). As you may have guessed, this is a case study. I am going to ask you questions and you are going to provide input s. Before we continue, are there any questions about the case?
TRANSITION: First, let us take a look at the facts of the case.AFMAN36-2236 12 NOVEMBER 2003 417

Body
MP1. Facts relevant to SSgt James' success and failure and Lt Roberts' actions.

TRANSITION: We have picked out many pertinent facts in this case, let us quickly summarize them.
INTERIM SUMMARY: (From the whiteboard) Review the facts relevant to the situation.
TRANSITION: Now that we have looked at the facts as stated in the case, let us make some logical assumptions based on what we know in the case and perhaps drawing from our own experiences.
MP2. Assumptions relevant to SSgt James' success and failure. Instructor Activity Anticipated Responses

Leadoff question (LOQ): What facts are relevant to SSgt James' potential failure? Wife flew to previous duty station (overseas) and wants a divorce

Lt Roberts has received many dunning letters, including ones for airfare to previous duty station

SSgt James had told the first sergeant that he could pay for the trips to see his wife and children

SSgt James claims the indebtedness letters are misunderstandings and that he has paid the bills

Lt Roberts' secretary sa id that SSgt James had received many phone calls from a creditor SSgt James had disclaimed the debt for

a. Followup question (FUQ): What facts are present relating to SSgt James' past success? He has 12 years in the Air Force with a good record

He worked extra time to catch up on the unit

b. FUQ: What facts do we know about Lt Roberts? Counseled SSgt James when he got back from emergency leave

Counseled SSgt James every time a new dunning letter was received

Forwarded overdue notices to the military personnel flight for verification of "just debts," but no results yet418 AFMAN36-2236 12 NOVEMBER 2003

TRANSITION: We have made some very valid assump tions. Let us now quickly recap them.
INTERIM SUMMARY: (From whiteboard) Review the assumptions relevant to the situation.
TRANSITION: After looking at the facts and making some logical assumptions, we probably can see some problems. Let us pick those out.
MP3. Problems related to SSgt James and Lt Roberts. Instructor Activity Anticipated Responses

LOQ: What assumptions can we make about SSgt James? His work has suffered due to his personal stress

He is too embarrassed to ask for help

He has not been to any sort of professional counseling for his financial problems

He wants to reenlist

He has not paid the overdue bills

FUQ: What assumptions can we make His counseling method is not working about Lt Roberts' actions or inactions? He has not tried to ge t SSgt James to go to counseling

He does not trust that SSgt James has paid the bills he claims he did

He is torn about the recommendation for reenlistment

He has never dealt with a similar situation

He does not know who to turn to for help

Instructor Activity Anticipated Responses

LOQ: What problems are facing SSgt May not be recommended for reenlistment James? Cannot pay his bills, which will continue to worsen if he can't reenlist

Getting a divorce

Work performance is deterioratingAFMAN36-2236 12 NOVEMBER 2003 419

TRANSITION: We have come up with many problems in this situation. Let us summarize them.
INTERIM SUMMARY: (From whiteboard) Summarize the problems associated with the situation. (Make sure and connect the problems to SSgt James' overall failures.)
TRANSITION: There are several ways to look at solutions to the problems we identified, but let us focus on the positive ones.
MP4. Positive solutions for SSgt James and Lt Roberts.Providing for two households

a. FUQ: What problems does Lt Roberts face? Must make a recommendation on whether SSgt James should be allowed to reenlist

Spends a disproportionate amount of time with the one individual

He has not been involved in the case enough. Counseling is the first step; if the problem persists, he should take additional steps

Inexperience supervising personnel

Not sure he can trust SSgt James personally, but can professionally

Is losing productivity because of SSgt James' deteriorating work performance

b. FUQ: What are some problems in the No one provided Lt Roberts training organization for this to occur? First sergeant has not been involved

Leadership was not actively involved in subordinates' careers and personal lives

Other subordinates may be having similar problems

Instructor Activity Anticipated Responses

LOQ: What options are available to SSgt Seek financial counseling James? Get loan from Air Force Relief Society

Give Lt Roberts proof he is trying to pay bills

Seek divorce counseling

Get custody of children, so he won't spend money to go see them420 AFMAN36-2236 12 NOVEMBER 2003

TRANSITION: We have discussed a lot today, let us take a look at where we have been.

Conclusion
FINAL SUMMARY: (From whiteboard) Briefly summarize the facts, assumptions, and problems. Then spend more time reviewing the solutions the students came up with to SSgt James' potential failure and Lt Roberts' actions or inactions. All these solutions fa ll into one category—more involved leadership. I think it is safe to say that a more involve d leader promotes subordinate success.
REMOTIVATION: All of us are supervisors, either of perso nnel or of students in the classroom. We all want our subordinates and students to be successful in their jobs, but how many of us think about their personal success? The two go hand in hand, and as lead ers we must remember to do our best to get involved and make sure they are successful in both areas.
CLOSURE: In these days of drawdowns, decreased spendi ng, and realignments, we are all trying to use all of our assets to their fullest potential. I fully beli eve our most important asset is our people, regardless of whether I get laughed at for it or not. As leaders, it is important we remember our most precious asset— our people. Thank you. a. FUQ: What options are available to Lt Roberts? Recommend SSgt James be allowed to reenlist with stipulations that he attend financial counseling and provide proof he is trying to pay bills

Possible extension to SSgt James' enlistment to give him time to prove he can be responsible and pay bills

Ensure SSgt James has food and a roof over his head (take care of his well-being)

Refer SSgt James to the First Sergeant

Go to the first sergeant for guidance

b. FUQ: What could Lt Roberts have done to help SSgt James avoid the potential of his professional downfall? Been more involved. Instead of leaving it at counseling, he could have forced SSgt James to go to base financial counseling.

Followed up on progress of payments

Talked with creditors to see if there was an arrangement that could be made

Had first sergeant get more involved

Talked with the commander early to say what a good worker SSgt James was and asked for the commander's advice on how to deal with the financial problemsAFMAN36-2236 12 NOVEMBER 2003 421

MONEY TROUBLES Lt Roberts received a note from the squadron commande r asking him if he would recommend SSgt James for reenlistment. Attached to the note was a letter from a creditor who complained about SSgt James. SSgt James had been the 608th Radar Squadron's pers onnel specialist for 1 year. When SSgt James first arrived from an overseas location, he was a very cons cientious worker. The position he filled had been vacant for some time, and SSgt James had frequently worked during his lunch hour and after duty hours to catch up on the backlog of work. SSgt James was ma rried and had two children. Six months after his arrival, SSgt James had applied for emergency leave. He explained to the first sergeant that his wife had unexpectedly withdrawn most of thei r savings and had returned to their previous residence with the children and he needed to fly out there to attempt to save his marriage. SSgt James assured the first sergeant he had ample funds to cover round-trip airfare if he could not get "hops" to his last base of assignment. After SSgt James returned from leave, Lt Roberts had an informal chat with him. SSgt James explained his family situation and, at the time, was unsure whethe r or not a marriage rec onciliation was possible. Shortly thereafter, SSgt James told the lieutenant he was going to get a divorce. About 2 months after SSgt James had taken leave, Lt Roberts star ted receiving dunning letters from many of SSgt James' creditors. These re flected, among other things, airfares to SSgt James' previous location. When Lt Roberts counseled him on meeting his financ ial obligations, SSgt James claimed the bills were the result of his wife's move. He assured the lieutenant they were being paid and that he had retrieved his wife's credit cards so she could not charge anything else in his name. Over the next several months, more overdue notices arrived for nonpayment of the original bills in addition to several new ones. Each time an overdue notice arrived, Lt Roberts counseled SSgt James. Each time he was counseled SSgt James explained that the debt was a misunderstanding or that the bill was paid. On one occasion, he disclaimed the debt, but Lt R oberts' secretary said that SSgt James had received many calls from that particular creditor. The lieutenant had forwarded the overdue notices to the military personnel flight for verification as "just debts," but the results had not yet been returned. SSgt James had almost 12 years in the Air Force and, to this point, had a good record. Because SSgt James was due to reenlist soon, the commander wanted Lt Roberts' recommendation now.422 AFMAN36-2236 12 NOVEMBER 2003

TEACHING NOTE

TOOTLIFEST comprehend involved leader ship promotes subordinate success.

Facts relevant to... Assumptions relevant to... Problems with ... Solutions to ...

LOQ: What facts are relevant to SSgt James’ potential failure? LOQ: What assumptions can we make about SSgt James? LOQ: What problems are facing SSgt James? LOQ: What options are available to SSgt James? - May not be - Wife flew to previous recommended for - Seek financial duty station - His work has suff ered reenlistment counseling (overseas) and wants due to his personal a divorce stress - Can not pay his bills, - Get loan from Air which will continue Force Relief Society - Lt Roberts has - He is too to worsen if he can’t received many embarrassed to as k reenlist - Give Lt Roberts dunning letters, for help pr oof he is trying to including ones for - Getting a divorce pay bills airfare to previous - He has not been to duty station any sort of - Work performance is - Seek divorce professional deteri orating counseling - SSgt James had told counseling for his the first sergeant that financial problems - Providing for two - Get custody of he could pay off the households children so he won’t trips to see his wife - He wants to reenlist spend money to go and children FUQ: What see them - He has not paid the problems does Lt - SSgt James claims the overdue bills Roberts face? FUQ: What options indebtedness letters are available to Lt are misunderstandings FUQ: What- Must make a Roberts? and he has paid the assumptions can we recommendation on bills make about Lt whether SSgt James - Recommend SSgt Roberts’ actions orshould be allowed to James be allowed to - Lt Roberts’ secretary inactions? reenlist reenlist with said that SSgt James stipulations that he had received many - His counseli ng - Spends a attend financial phone calls from a method is not disproportionate counseling and creditor SSgt James working amount of time with provide proof he is had disclaimed the the one individual trying to pay bills debt for - He has not tried to get SSgt James to go - He has not been - Possible extension to FUQ: What facts areto counseling involved in the case SSgt James’ present relating to enough. Counseling enlistment to give SSgt James’ past - He does not trust that is the first step; if the him time to prove he success? SSgt James has paid problem persists, he can be responsible the bills he claims he should take and pay bills - He has 12 years in the did additional stepsAFMAN36-2236 12 NOVEMBER 2003 423

Air Force with a good record - He worked extra time - He is torn about the - Inexperience - Ensure SSgt James to catch up on the unit recommendation fo r supervising has food and a roof reenlistment personnel over his head (take FUQ: What facts do care of his wellwe know about Lt- He has never dealt - Not sure he can trust being) Roberts? with a similar SSgt James situation personally, but can - Refer SSgt James to - Counseled SSgt professionally the First Sergeant James when he got - He does not know back from emergency who to turn to for - Is losing productivity - Go to the First leave help because of SSgt Sergeant for James’ deteriorating guidance - Counseled SSgt work performance James every time a FUQ: What could Lt new dunning letter FUQ: What are some Roberts have done to was received problems in the help SSgt James organization for this avoid the potential of - Forwarded overdue to occur? his professional notices to the military downfall? personnel flight for - No one provided Lt verification of “just Roberts training - Even more involved. debts” ...but no Instead of leaving it results yet - First sergeant has not at counseling, he been involved could have forced SSgt James to go to - Leadership was not base financial actively involved in counseling subordinates’ careers and personal lives - Followed up on progress of payments - Other subordinates may be having - Talked with creditors similar problems to see if there was an arrangement that could be made - Had First Sergeant get more involved

- Talked with the commander early to say what a good worker SSgt James was and asked for the commander’s advice on how to deal with the financial problems424 AFMAN36-2236 12 NOVEMBER 2003

ACADEMIC INSTRUCTOR SCHOOL Maxwell Air Force Base, Alabama

PART III
TEST ITEM

OBJECTIVE: TOOTLIFEST comprehend that involved leadership promotes subordinate success.
SAMPLE OF BEHAVIOR: Describe how involved leadership promotes subordinate success. QUESTION: Which phrase BEST describes how involve d leaders promote subordinate success? a. By leaders encouraging personnel to study for advancement exams, subor dinates will strive to succeed. b. Leaders who follow counseling with other methods ensure all avenues are used to enhance the subordinate. c. When leaders praise personnel for doing good work, the subordinates are motivated to do more.
KEY: Option b is the correct answer. An involved leader will follow counseling with other methods if the counseling fails to bring about th e desired results. This will be br ought up in the case study discussion under solutions. Option a is incorrect. Although it is a way to promote subordinate success, it does not take much involvement on the part of a leader. Option c is incorrect. Praising personnel is a way to promote subordinate success, but like option a, it does nottake much involvement on the part of a leader.AFMAN36-2236 12 NOVEMBER 2003 425

Attachment 7

EXAMPLE LESSON PLAN FOR CASE STUDY METHOD TOPICAL PATTERN

ACADEMIC INSTRUCTOR SCHOOL Maxwell Air Force Base, Alabama

PART I
COVER SHEET

LESSON TITLE: Effective Leadership Enhanc es Subordinate Effectiveness
RESOURCE PERSON:
TEACHING METHOD: Case Study
REFERENCES: Battle Cry of Freedom by James M. McPherson
AIDS/HANDOUT/NOTETAKERS: Video clip from the movie "Gettysburg," whiteboard, mag cards, music from "Gettysburg" to se t the appropriate atmosphere
STUDENT PREPARATION/READING ASSIGNMENT:
PRESENTATION TIME: 30-37 Minutes

PART IA
COGNITIVE OBJECTIVE: TOOTLIFEST comprehend that effectiv e leadership enhances subordinate effectiveness.
COGNITIVE SAMPLES OF BEHAVIOR: 1. Explain why effective leadership enhances subordinate effectiveness. 2. Describe how effective leadership enhances subordinate effectiveness.
3. Predict the outcome on subordinate effectiven ess when effective leadership is practiced.
AFFECTIVE OBJECTIVE: TOOTLIFEST respond positively to effective leadership enhances subordinate effectiveness. 1. Voluntarily participates in discussion of how effect ive leadership enhances s ubordinate effectiveness. 2. Shares personal examples on how effective lead ership enhances subordinate effectiveness.3. Displays positive nonverbal communications during lesson on how effective leadership enhances subordinate effectiveness.426 AFMAN36-2236 12 NOVEMBER 2003

PART IB
ORGANIZATIONAL PATTERN: Topical
STRATEGY: In a small seminar environment, I will begi n the lesson with a short introduction about how effective leadership enhances subordinate effectiveness . I will carefully cover the rules of engagement so that the students are aware that their participation is critical to the success of the lesson. I will explain my role as a recorder and facilitator. I will emphasize the importance of competency, caring for people, and motivation, which are the key characteristics of a leader, and how these enhance subordinate effectiveness. The small seminar will allow maximum participation which is critical for comprehension of these important leadership concepts we have been st udying and how they relate to us on the job. I will use a 5 1/2-minute video clip from the movie "Gettysburg" which depicts Colonel Chamberlain's charge at the battle of Little Round Top. The video clip will be pr esented during the overview and sets the stage for examination of these concepts during the lesson. The case elements (main points) of competencyenhances subordinate effectiveness, care for people enhances subordinate effectiveness, and motivation enhances subordinate effectiveness will be used to analyze the case and can be presented in any order. Magnetic cards will be created for each of the case elements and placed on the enamel board prior to the beginning of the lesson. I will begin by asking questions about competencyenhancing subordinate effectiveness and record the inputs given by the stude nts. After all the inputs on the first case element have been recorded on the enamel board, I will summ arize those inputs in order to drive home the pertinence of competency enhancing subordinate effectivene ss. I will repeat this procedure with the remaining case elements . I will teach this lesson using a topical pattern since there is no problem-solving activity. I will ask questions targeted at the three sublevels of comprehension (translation, interpretation, and extrapolation) to reach the comprehension level. The students will provide the inputs necessary to provide a thorough look at competency, care for people, and motivation exhibited by Colonel Chamberlain; how these concepts enhanced his subordinates' effec tiveness; and how those dynamics shown by leaders enhanced subordinate effectiveness in their own careers. It will be apparent to the students after the final summary, in which all the pertinent information gene rated during the lesson is reviewed, that effective leadership enhances subor dinate effectiveness.

LESSON OUTLINE:
MP1. (C) competency enhances subordinate effectiveness.
MP2. (C) care for people enhances subordinate effectiveness.
MP3. (C) motivation enhances s ubordinate effectiveness.AFMAN36-2236 12 NOVEMBER 2003 427

ACADEMIC INSTRUCTOR SCHOOL Maxwell Air Force Base, Alabama

PART II
TEACHING PLAN

Introduction

ATTENTION: Imagine your subordinates in chemical suits- -hot, sweaty, tired, and wanting to go home. You know you can't send them home so what can you do to keep them focused on their jobs, motivated to do their best? What you can do is be an effective leader!!
MOTIVATION: We are all leaders whether it is as a formal leader or informal leader. We expect our subordinates or coworkers to follow our lead. We expect th em to give their complete effort to a task even if they are tired, hungry, etc. It could be the difference between life and death! We cannot get the most from our subordinates or coworkers unless we are ef fective leaders. Today, I will show you how.
OVERVIEW: Today we will look at how effective leadership enhances subordinate effectiveness. To dothis, we will use a case which is a clip from the movie "Gettysburg." After watching this clip, we will analyze the case by breaking it down into three main points. We will look at how competency enhances subordinate effectiveness, then how care for people e nhances subordinate effec tiveness, and finally how motivation enhances subordinate effectiveness. We w ill break this case apart by me asking questions and you providing support based on the case. Please keep in mind that we may have different opinions, so be polite and respectful to each other. Before we be gin, let me give you a working definition from my 14 years as an officer: Effective leadership: Someone able to produce the desired results while still focusing on the people and mission. Competency: Properly qualified, capable. Subordinate effectiveness: Someone doing a job to the best of his or her ability without complaining embodies excellence in all we do. Let me set up the clip. This movie is about the Civ il War. Colonel Chamberlain led the 20th Maine for the North (Union). He is now confronted by Confederate soldiers. Read (slowly) the passage from the book Battle Cry of Freedom , page 659, paragraph 2. "Posted at the far left of this brigade was the 20th Maine, commanded by Colonel Joshua L. Chamberlain. A year earlier Chamberlain had been a professor of rhetoric and modern la nguages at Bowdoin College.Taking leave, ostensibly to study in Europe, he joined the army instead and now found himself responsible for preventing the rebels from rolling up the Unio n left. The fighting professor and his down-easterners proved equal to the occasion. For nearly two hours they stood off repeated assaults by portions of several Confederate regiments along the rocky, woode d slope filled with smoke, noise, and terror. With more than a third of his men down and the remainde r out of ammunition—and with the Rebels forming another assault—Chamberlain was in a tight spot. But cool and quick-witted—perhaps a legacy of dealing with fractious students—he ordered his men to fix bayonets on their empty rifles and charge. With a yell,428 AFMAN36-2236 12 NOVEMBER 2003

these smoke-grimed Yanks lurched downhill against the surprised rebels. Exhausted by their uphill fighting following a 25-mile march that day to reach the battlefield, and shocked by th e audacity of this bayonet assault, the Alabamians surrendered to the jubila nt boys from Maine." Let us look at the video clip.
TRANSITION: Now that we have seen the case, we need to break it down into the case elements—Competence, Care for People, and Motiv ation—and see how each of these e nhance subordinate effectiveness. Body

INTERIM SUMMARY: From whiteboard, summarize student comments and show how competence relates to the overall objective by showing both Cham berlain's competence and how our being competent will enhance our subordi nates' effectiveness.
TRANSITION: Competence is just one element of leadership that can be used to enhance subordinate effectiveness. Another important element of leadership is care for people. Let's take a look at the ways Chamberlain's subordinates' effectiveness was e nhanced as a result of his care for people. MP1. Competency enhances subordinate effectiveness.

Questions Anticipated Responses

LOQ: How did Chamberlain display -Use of tactics (wheel movement) competence in this clip? -Knew the objective -Knew the overall situation

a. FUQ: How did this competence enhance -They followed him because they believed his subordinates' effectivene ss? he knew what he was doing -Troops didn't hesitate when given an unbelievable order -Believed his competence would keep them alive

b. FUQ: How would your demonstration of -Subordinates would want to follow the competence enhance your subordinates' example set effectiveness? -Subordinates' attention to detail would increaseAFMAN36-2236 12 NOVEMBER 2003 429

INTERIM SUMMARY: From whiteboard, summarize student comments and show how care for people relates to the overall objective by showing both Chamberlain's care for people and how our caring for our people will enhance our subordinates' effectiveness.
TRANSITION: We've looked at two vital elements of le adership—competence and care for people— and saw how both can enhance subordinate effectivene ss. Now let's look at the ways Chamberlain's subordinates' effectiveness was enhanced as a result of him motivating them. MP2. Care for people enhances subordinate effectiveness.

Questions Anticipated Responses

LOQ: How did Chamberlain display care for -Gave the prisoner water people in this clip? -Did not kill the enemy who tried to shoot him -Cared for the entire Union Army by not retreating off the hill

a. FUQ: How did his caring for people -His troops knew they would probably be enhance his subordinates' effectiveness? treated in the same manner if captured that Chamberlain treated the rebels -Would fight harder for him since they knew he had their best interest at heart -Knew he would not ask them to do anything he wouldn't do himself so they were more willing to fight for him

b. FUQ: How would your demonstration of -Subordinates tend to work harder when caring for people enhance your subordinates' they know they are cared for effectiveness? -Subordinates are happier and morale increases -Subordinates begin to care more about others430 AFMAN36-2236 12 NOVEMBER 2003

TRANSITION: We've looked at how three critical elements of leadership can enhance today. Now let's recap each of those and see how using them effectively will enhance subordinate effectiveness. MP3. Motivation enhances subordinate effectiveness.

Questions Anticipated Responses

LOQ: How did Chamberlain motivate his -E xplained what his plan was and why he men? was doing it -Decisiveness/confidence -Use of bugle and flag

a. FUQ: How did this motivation enhance -Troops had buy-in for plan since they his subordinates' effectiveness? understood why he was doing it -His confidence in the plan was infectious to his troops -Reached the troops affectively by having the colors up front where they could see them and used music (bugle) to get their energies up

b. FUQ: How would motivating your -They become more productive subordinates enhance their effectivene ss? -They become better team players -They enjoy their work moreAFMAN36-2236 12 NOVEMBER 2003 431

Conclusion
FINAL SUMMARY: From whiteboard, summarize student comm ents and show how motivation relates to the overall objective by showing how both Chambe rlain's use of motivation and how our motivating our troops will enhance their effectiveness. Briefly reiterate important points from MP2, then MP1. Show how if we use each of these leadership elements our selves we, too, will enhance our subordinates' effectiveness. Ensure an understanding that these three elements of leadership are related and have a direct link to the entire objective. Completely write out your fi nal summary and practice it as such, to ensure you include all important areas. During your actual summaries, personalize them to include student inputs. See note below.
NOTE: Summarize the information you drew from the case element of competence. This is just one of several characteristics of an effective leader. Then show how Chamberlain used competence to enhance his subordinates' effectiveness. Add additional exampl es of a competent leader from past lessons or from experience. Explain how these enhanced subordinate effectiveness. You have already summarized each case element individually and drawn conclusions that each one enhances subordinate effectiveness. Now show how the A term (Effective Leadership) relates to the B term (Subordinate Effectiveness) in the Chamberlain case. Having comprehended this principl e in the Chamberlain case, use your expertise to draw relationships, connections, and c onclusions and tie these to real-world examples that directly benefit your audience.
REMOTIVATION: The next time your subordinates have to wo rk long hours, whether to build a pallet, fly a mission, or type a document, remember you are the key to their effectiveness. You need to remember Colonel Chamberlain and how he knew the job, cared about his people, and encouraged them. Use these same traits to get your subor dinates to accomplish the mission. In th is time of doing more with less, it is highly likely we will work longer hours. Therefore, it is within your control whether subordinates work hard for you or complain to you.
CLOSURE: Put yourself back in che mical gear—how will you influenc e your people? Ken Blanchard, The Heart of a Leader , says "Influence is an art—and it begins with the heart of a leader. It's time to let your heart soar."432 AFMAN36-2236 12 NOVEMBER 2003

TEACHING NOTE

TOOTLIFEST comprehend effective leadership enhances subordinate effectiveness.

Competency enhances Care for people enhances Motivation enhances subordinate effectiveness subordinate effectiveness subordinate effectiveness
LOQ: How did Chamberlain LOQ: How did Chamberlain LOQ: How did Chamberlain display competence in this display Care for People in this motivate his men? clip? clip? - Use of tactics (wheel - Gave the pris oner water - Explained what his plan was movement) - Did not kill enemy who tried and why he was doing it - Knew the objective to shoot him - Decisiveness/confidence - Knew the overall situation - Cared for the entire Union - Use of bugle and flag Army by not retreating off the hill
FUQ: How did this FUQ: How did his caring for FUQ: How did this motivation competence enhance his people enhance his enhance his subordinates’ subordinates’ effectiveness? subordinates’ effectiveness? effectiveness? - They followed him because - His troops knew that they - Troops had buy-in for plan they believed he knew what he would probably be treated in since they understood the was doing the same manner if captured rationale for why he was doing - Troops didn’t hesitate when that Chamberlain treated the it given an unbelievable order rebels - His confidence in the plan was - Believed his competence - Would fight harder for him infectious to his troops would keep them alive since they knew he had their - Reached the troops affectively best interest at heart by having the colors up front - Knew he would not ask them where they could see them and to do anything that he used music (bugle) to get wouldn’t do himself so they their energies up were more willing to fight for him
FUQ: How would your FUQ: How would your FUQ: How would motivating demonstration of competence demonstration of caring for your subordinates enhance enhance your subordinates’ people enhance your their effectiveness? effectiveness? subordinates’ effectiveness?

- subordinates would want to - Subordinate s tend to work - become more productive follow the example set harder when they know they - become better team players - subordinates attention to detail are cared for - enjoy their work more would increase - Subordinates are happier and morale increases - Subordinates begin to care more about othersAFMAN36-2236 12 NOVEMBER 2003 433

ACADEMIC INSTRUCTOR SCHOOL Maxwell Air Force Base, Alabama

PART III
TEST ITEM

LESSON OBJECTIVE: TOOTLIFEST comprehend that effectiv e leadership enhances subordinate effectiveness.
SAMPLE OF BEHAVIOR: Describe how effective leadership enhances subordinate effectiveness. TEST QUESTION: Which of the following BEST describes how effective leadership enhances subordinate effectiveness? a. Capable leadership, which inspires subordinates and has compassion for them, will influence them to do their best. b. Leadership, which embodies integrity, inspires subordinates, and has compassion for them, will influence them to put forth some effort. c. Capable leadership sets the example and has co mpassion for subordinates day in and day out while on the job.
KEY: Option a is correct because the l eader is competent, motivates subordinates, and cares for his or her people all the while ensuring they are most effective. Option b is incorrect because integrity was not oneof the leadership elements taught in the lesson. Option c incorrect because it doesn't address how the subordinates were influenced.434 AFMAN36-2236 12 NOVEMBER 2003

Attachment 8

EXAMPLE LESSON PLAN FOR TEACHING INTERVIEW

ACADEMIC INSTRUCTOR SCHOOL Maxwell Air Force Base, Alabama

PART I
COVER SHEET

LESSON TITLE: Laws of Warfare
RESOURCE PERSON:
TEACHING METHOD: Teaching Interview
REFERENCES: AFPD 51-4, Compliance With the Law of Armed Conflict
AIDS AND HANDOUTS/NOTETAKERS: Overview Chart
STUDENT PREPARATION/READING ASSIGNMENT:
PRESENTATION TIME: 30-37 Minutes

PART IA
COGNITIVE OBJECTIVE: TOOTLIFEST comprehend that adherence to laws of warfare prevents conflicts from escalating to unc ontrollable death and destruction.
COGNITIVE SAMPLES OF BEHAVIOR: 1. Justify why adherence to laws or warfare prevents conflicts from escalating to uncontrollable death and destruction. 2. Predict the effects on conflict if the US did not adhere to the laws of warfare. 3. Estimate the consequences on conflict if the enemy did not adhere to the laws of warfare. 4. Given a scenario, explain how adherence to laws of warfare could prevent the conflict from escalating to uncontrollable death and destruction.
AFFECTIVE OBJECTIVE: TOOTLIFEST respond positively to the principle that adherence to laws of warfare prevents conflicts from escalating to uncontrollable death and destruction.
AFFECTIVE SAMPLES OF BEHAVIOR: 1. Unhesitatingly predicts the probable outcomes in war s ituations resulting from violations of the laws of

war. 2. Openly explains the rationale and benefits of the la ws of war using examples of actual past conflicts. 3. Actively tries to persuade others who mock the ut ility of the laws in preventing uncontrollable death and destruction.AFMAN36-2236 12 NOVEMBER 2003 435

PART IB
ORGANIZATIONAL PATTERN: Topical
STRATEGY: This lesson, adherence to laws of warfare pr events conflicts from escalating to uncontrollable death and destruction, is desi gned to last approximately 45 minut es. My primary purpose is not to study the laws of war in depth, but to sensitize the st udents to the very real a nd important benefits we derive from adhering to them. Consequently, we will s imply look at a sample of the laws and talk about their past and potential benefits during wartime. I’ve decided to use the teaching interview to heighten the affective impact of this lesson. I will also use the topi cal pattern to organize the main points. This is themost logical pattern to use. The inductive approach was selected to more easily understand this complicated subject. I've invited an interviewee who works intimately on questions related to the law of armed conflict (laws of war). The first main point is how adherence to requirements governing the status of parties prevents the indiscriminate killing of combatants , noncombatants, and civilians. First, the guest will talk about the requirements governing the status of combatants, noncombatants, and civilians and how these requirements protect human life during periods of conflict. Since this point deals with human life, it should be of great interest to the students and, conseque ntly, draw students' attention to the interview moreforcefully. Our focus will then shift to objects. The second main point is adherence to rules of aerial bombardment prevents the indiscriminate bombing of possi ble targets in an enemy nation. We'll develop rules of aerial bombardment so students can see why rules, valid targets, unacceptable targets, and location problems are such concerns in warfare. This is the second main point because students need to understand the impact of war on objects. This is important but not as important as the first main point. Both MP1 and MP2 need to be discussed before we can generalize to MP3. Once students understand how adherence to these requirements and rules prevents indiscriminate killing and bombing of target, I will question the interviewee on how the laws as a whole prevent conflicts from escalating. This is the third main point— adherence to laws of warfare conf licts from escalating to uncontrollable death and destruction. In looking at the laws as a whole, we will address the reasons for following the laws, the consequences of nonadherence, the effect of bombing cities and disobeying orde rs, and finally how adhering to the laws keeps the smallest war from resulting in the end of civilization. I take this line of questioning so the students can deduce the conclusion about the bene fits of following the law for their own good as well as the good of others. I will be asking all the ques tions of the guest during the body of the class; therefore, I will allow students to ask questions at the end, but before the conclusion of the lesson. This will allow the students an opportunity to eliminate any area of confusion as well as have the interview ee expand on any particulararea. After student questioning is completed, I will do a final summary and conclusion based on my questions during the body and their questions during the ques tion-and answer (Q & A) period. This final step will enhance the likelihood of the students identifying w ith this lesson. I want to make sure their concerns (as well as mine) are noted and emphasized. The only visual aids I will use are Powerpoint slides listing each main point on separate slides. When discussing MP1 make sure and have only the slide listing MP1 on the monitor. The same applies for each other main point. In the summary, use the slide with all the main points on it to show the generali zation of reaching the lesson objective.

LESSON OUTLINE
MP1. Adherence to requirements governing the status of parties prevents the indiscriminate killing of combatants, noncombatants, and civilians.436 AFMAN36-2236 12 NOVEMBER 2003

a. Combatants
b. Noncombatants
c. Civilians
MP2. Adherence to rules of aerial bombardment prevents the indiscriminate bombing of possible targets in an enemy nation. a. Rules dictate targets b. Valid targets c. Unacceptable targets d. Location problems

MP3. Adherence to laws of warfare prevent conflicts from escalating to uncontrollable death and destruction. a. Reasons for allowing laws of warfare b. Consequences of nonadherence c. Effect of bombing cities d. Disobeying orders
e. Civilized warfareAFMAN36-2236 12 NOVEMBER 2003 437

ACADEMIC INSTRUCTOR SCHOOL Maxwell Air Force Base, Alabama

PART II
TEACHING PLAN

Introduction

ATTENTION: Some war criminals say they believed it was their duty to kill everyone in their operational area, including women and children, because their superior desired it. The most common defense at Nuremburg was, "I was following orders." North Vietnam threatened to kill captured pilots because they were not combatants in a declared war.
MOTIVATION: It is obvious that crewmembers would have an interest in the laws of warfare—their lives could depend upon it. It's not so obvious for the rest of us support personnel. But I look at Operation DESERT STORM. How many surprise deployments were there? The bottom line is that it's too late to learn it at the time. We must be familiar with the basics in advance.
INTRODUCTION OF GUEST: Maj Nick Grebeldinger Education: Juris Doctorate Degree, Duquesne University LLM (International Law), George Wash ington University in Washington DC Graduate of Armed Forces Staff College Background: 3 years on Air Staff in International Law Division of OTJAG 2 years in USAFE as member of International Law Directorate 2 years at Jag School -Advised Tactical Air Command, Military Airlift Command, Air Force Logistics Command, and Air University on law of warfare training -Lectured on laws of warfare at AWC, ACSC, and SOS
OVERVIEW: 1. Requirements governing the status of parties 2. Rules of aerial bombardment 3. Importance of following the laws of warfare
DEFINE TERMS ANNOUNCE Q&A PERIOD
TRANSITION: Before entering the discussion, Maj Grebeldi nger, why have you channeled so much of your energies into this area of study?438 AFMAN36-2236 12 NOVEMBER 2003

Body
MP1. Adherence to requirements governing the status of parties prevents the indiscriminate killing of combatants, noncombatants, and civilians.

Questions Anticipated Responses
LOQ: Why can a squad of soldiers operating behind the lines in civilian clothes be shot as spies? Must be uniformed or else not protected

a. FUQ: What is the status of an escaping POW? He or she does not easily fit into the combatant/ noncombatant dichotomy

POW is entitled to attempt to escape and can only face administrative punishment (such as solitary confinement, bread and water) for attempting

However, the captors can use deadly force to prevent an escape, provided the POWs are warned beforehand (the wire before the barbed wire fence you see in WWII movies)

POWs cannot use deadly force to make their escape, nor can they steal items that would not be necessary for their escape (for example, they can steal clothes, money, or food, but not jewelry or personal items); in those cases, they would be tried under the legal system of the captor nation and sentenced to confinement in prison or death (in other words, they would no longer be treated as POWs)

They revert to combatant status if they reach their own lines or country
b. FUQ: How can a surr endering airman not Deceiving enemy resist when the enemy is approaching and the odds look better than when the airman surrendered? Could jeopardize future captives

c. FUQ: Do those in noncombatant status Medics, chaplains differ? Separate category

Afforded protection
d. FUQ: In what conditions could civilians Not unless uniformed fight?AFMAN36-2236 12 NOVEMBER 2003 439

INTERIM SUMMARY: Summarize comments as they relate to how these requirements prevent indiscriminate killing.
TRANSITION: We have seen how adherence to requirement s governing the status of parties prevents the indiscriminate killing of human beings. This acti on helps prevent conflicts from escalating. Now, let's turn from people to objects as we examine target selection under the rules of aerial bombardment.
MP2. Adherence to rules of aerial bombardment prevents the indiscriminate bombing of possible targets in an enemy nation.

INTERIM SUMMARY: Summarize comments as they relate to the prevention of indiscriminate bombing of targets.
TRANSITION: We have learned some basic rules and how they are applied to prevent the indiscriminate bombing of targets. Now let's find out what this means to us.
MP3. Adherence to laws of warfare prevents conf licts from escalating to uncontrollable death and destruction. Questions Anticipated Responses
LOQ: How do the rules of aerial bombardment affect the decision on targets? Rules dictate the targets that can be attacked

a. FUQ: What would be the situation of Valid target—if military targets are there targeting the capital of an enemy country with a nuclear weapon under the laws of warfare?
b. FUQ: Why can't our enemies target downtown Montgomery? Montgomery is not a military target, though Maxwell is
c. FUQ: What are some examples of targets that may be targeted and those that may not? Targets: military facilities, access roads, waterways, etc.

Nontargets: Civilia n population areas
d. FUQ: How could we attack a factory if it's Precision bombing or military necessity next to a hospital?

Questions Anticipated Responses
LOQ: Everyone knows the enemy doesn't Court-martial follow the laws of warfare. Why should we? War crime trial

You as a POW440 AFMAN36-2236 12 NOVEMBER 2003

TRANSITION: In adhering to the laws of a good teaching interview, I would like to now offer an opportunity for you (the students) to ask Ma j Grebeldinger any questions you may have. Q&A SESSION:
TRANSITION: Those were great questions. Let’s see what we have looked at today.

Conclusion
FINAL SUMMARY: Summarize relevant material from Q&A session
Summarize MP3
Review MP2
Review MP1 Restate lesson objective
REMOTIVATION: In time of peace it is hard to get excited about the laws of warfare. However, now is the time for us to prepare. If war comes, we will be too busy to take time. Our country cannot tolerate war criminals. You can't afford to be the war criminal or have one work for you.
CLOSURE: Thank you, Maj Grebeldinger a. FUQ: How can adhering to the laws of warfare preclude escalation to uncontrollable death and destruction? Play game or get "nuked"

b. FUQ: Why would it not be a good idea to bomb their cities to "bring them to their knees"? Sometimes unites enemy

c. FUQ: What if I am ordered to disobey the laws of warfare, such as attack a civilian target...what are my choices? Refuse, but be right

d. FUQ: How will adhering to the laws of warfare keep even the smallest war from resulting in the end of civilization, as we know it? "Civilized" warfareAFMAN36-2236 12 NOVEMBER 2003 441

ACADEMIC INSTRUCTOR SCHOOL Maxwell Air Force Base, Alabama

PART III
TEST ITEMS

ITEM 1
LESSON OBJECTIVE: TOOTLIFEST comprehend that adherence to laws of warfare prevents conflicts from escalating to uncontrollable death and destruction.
SAMPLE OF BEHAVIOR: Predict the effect on conflict if the US did not adhere to the laws of warfare. QUESTION: If the US elected NOT to adhere to the laws of war, what would be the MOST LIKELY effect? a. The US public would be behind us be cause "all is fair in love and war." b. There would be instant diplomatic, econom ic, and military reprisal by other nations.c. Our overseas trade would suffer because such action is not consistent with the principles of capitalism.
KEY: b
RATIONALE: Option b is correct because this answer covers all the areas of retribution open to other nations (consistent with te xt and discussion). Option b is incorrectbecause civilians could be killed under this rational. Option c is incorrect because overseas trade has nothing to do with the laws of war.
ITEM 2
LESSON OBJECTIVE: TOOTLIFEST comprehend that adherence to laws of warfare prevents conflicts from escalating to uncontrollable death and destruction.
SAMPLE OF BEHAVIOR: Justify why adhering to laws of warfare prevents conflicts from escalating to uncontrollable death and destruction. QUESTION: In 250 words or less, justify why adhering to laws of warfare prevents conflicts from escalating to uncontrollable death and destruction. Questi on is worth 40 points, and partial credit will be given. One point will be deducted for each spelling error.
KEY: A complete answer must include:

 Adherence to requirements governing the status of parties prevents the indiscriminate killing of combatants, noncombatants, and civilians. (10 points)

 Adherence to rules of aerial bombardment prevents the indiscriminate bombing of possible targets in an enemy nation. (10 points)

 Students should include discussion on enforcement pow er of the UCMJ (Court Martial), national and international law (war crime tria ls), and the expectation theory (what you would expect as a POW), escalating to uncontrollable death and destruction. (10 points)
NOTE: Partial credit will be given with a maximum possible 40 points.442 AFMAN36-2236 12 NOVEMBER 2003

Attachment 9

EXAMPLE LESSON PLAN FOR DEMONS TRATION-PERFORMANCE METHOD

ACADEMIC INSTRUCTOR SCHOOL Maxwell Air Force Base, Alabama

PART I COVER SHEET

LESSON TITLE: Tying a Trucker's Hitch Knot TEACHING METHOD: Demonstration-Performance REFERENCES: AETC Survival Training Guide ORGANIZATIONAL PATTERN: Sequential VISUAL AIDS/EQUIPMENT: Poster of task steps, rope for each student STUDENT PREPARATION/READING ASSIGNMENT: None

PART IA

CRITERION-OBJECTIVE:

TASK STEPS: 1. Secure one end of the rope with any known knot. 2. Tie slipknot 2 feet from secured end. 3. Place locking loop over the slipknot. 4. Route the rope around a second secure point or to an item to be secured. 5. Route a loop partially through the locked slipknot. 6. Route the free end of the rope through the loop. 7. Pull the free end of the rope tight through the loop. 8. Apply enough pressure to the free end to tighten and lock the hitch knot. 9. Reverse the pressure to free the hitch knot. PERFORMANCE CONDITIONS STANDARDS Tie a trucker's hitch knot 1. Given: Trucker's hitch knot must: a. Rope 1. Be tight b. Tie-off 2. Lock 2. Without: 3. Easily unlock a. Instructor assistance 4. Be completed within 4 b. Task list minutesAFMAN36-2236 12 NOVEMBER 2003 443

AFFECTIVE OBJECTIVE: TOOTLIFEST respond in a positive manner to tying a trucker's hitch knot. AFFECTIVE SAMPLES OF BEHAVIOR: 1. Reads additional information on howto tie a trucker’s hitch knot. 2. Helps others master the techniques needed to tie a trucker's hitch knot. 3. Asks questions about the procedure for tying a trucker's hitch knot.

PART IB ORGANIZATIONAL PATTERN: Sequential STRATEGY: The demonstration-performance method is usedin this lesson because I’m teaching a physical skill. For an attention step, I will use an example of living in tent city, and not being able to get the tent tight enough. As a motivator, I will explain that using a hitch knot helps to get those tents tight, as well as secure. In the overview, I will give details of what to expect in the lesson by telling the students I will explain and demonstrate how to tie a trucker’s hitch knot using the task steps. Then each student will have time to practice tying a trucker’s hitch knot under mysupervision. The last practice will be under evaluation conditions, which will prepare them for their eval uation. I will state the task steps and the standard on which they will be evaluated. I will transition into MP1, Explanation/Demonstration. I will demonstrate the skill while explaining each task step shown on a visual aid to the students. Students who lack knot-tying experience will not be able to really understa nd the meaning of each task step without a visual reference and demonstration. I will carefully and precis ely demonstrate the skill. This will ensure the students understand all task steps necessa ry to tie the trucker’s hitch knot. After I complete the explanation/ demonstration, I will give each student a length of rope and allow each student ample time to practice and ask questions. During MP2, Performance-Supervision, I will observe each student; if I notice anyone having difficulty, I will provide assistance so he or she wi ll be able to correctly tie the trucker’ hitch knot. I will allow ample time for practice. This will ensure each student’s ability to correctly tie the trucker’s hitch knot. When everyone seems comfortable with th e task, I will conduct the last practice under evaluation conditions by removing the task steps, providi ng no assistance, and allowing only 4 minutes to complete the task. This will give them a clear understa nding of what the evaluation will be like and should decrease the students’ anxieties. Once everyone ha s completed the task, I will provide an interim summary by reviewing the task steps again, reemphasizi ng any areas the students were having problems with. Again, I will explain the standards they need to meet so everyone is aware of what it takes to pass this evaluation. I will then begin MP3, the evaluation. At th e end of 4 minutes, I will evaluate each student’s performance, determining if the student mastered the skill or not according to the standard. I will use the performance evaluation sheet to the performance and provide positive feedback reemphasizing any weak areas demonstrated by the student. I will transition to the summary and review key areas in the Explanation/Demonstration for tying a trucker’s hitch knot. I w ill state key areas of their performance under my supervision. I will provide some positive point of the evaluation. I will remotivate the students by stating that the hitch knot can be used to tie the tent in a survival situation to manufacture shelter or as a tiedown when transporting items at work or at home. I will close by quoting the Boy Scout motto and by telling them that knowing this trucker’s hitch knot gives them just one more tool in their arsenal of preparation.

LESSON OUTLINE: MP1. Explanation/Demonstration444 AFMAN36-2236 12 NOVEMBER 2003

MP2. Performance-Supervision MP3. EvaluationAFMAN36-2236 12 NOVEMBER 2003 445

ACADEMIC INSTRUCTOR SCHOOL Maxwell Air Force Base, Alabama

PART II TEACHING PLAN

Introduction

ATTENTION: Picture this . . . you've been deployed to Bosn ia. You get to spend some time in a tent city. No matter how hard you try, you can't seem to be able to keep that tent upright and tight! If you only knew how to tie one of those COOL hitch knots they use at the survival school! MOTIVATION: If you have ever been in a situation where you needed a knot or hitch knot that could provide a GREAT deal of pressure and not slip on you, the trucker's hitch knot is for you. If you learn to tie a trucker's hitch knot, you will be able to tie down those tents that you may have the pleasure of living in, or you can use it to tie down equipment, supplies, or furniture. You can secure anything you need to from moving around in the wind or when moving it in a truck or on the back of a trailer. OVERVIEW: Today, I am going to teach you how to tie a trucker's hitch knot. First I'll explain the process to you while I demonstrate to you how it is tied. Once you have watched me tie the hitch knot, I will give each of you some rope and le t you practice tying it at least tw o times, with your first attempt under my supervision. If you have problems, I will let you ha ve several additional practi ces. Your last practice will be under evaluation conditions, which means no help from either the task list or me. You then will be evaluated on your ability to tie a trucker's hitch knot on your own. You must complete the hitch knot within 4 minutes. It has to be tight, and it must lock inplace. It will also have to release (unlock) easily and correctly to get a satisfactory completion. During this lesson right up to the evaluation, I'll be more than happy to answer questions and help you with this task in any way, so please ask questions if you have any. TRANSITION: Now that you know WHAT we are going to do... let's get to doing it!

Body MP1. EXPLANATION/DEMONSTRATION: Explain what a trucker's hitch knot is and what it can be used for. Make sure all students have a clear view of the demonstration, and demonstrate each step while explaining why each step occurs. Put up the poste r of the task steps, explaining each step as you demonstrate it. 1. Secure one end of the rope with any known knot. (anchor) Let the student know that any method of securing the line is acceptable as long as it remains se cure. This is so the base of the hitch knot remains secure but will not cause the student to spend unn ecessary time on the anchor. Use a table leg or some secure part of the classroom. 2. Tie slipknot 2 feet from secured end. (act as anchor for hitch knot) Demonstrate how and where to place the slipknot. Explain that the location will vary depending on the distance between the anchor and the second secure point. Be sure to explain that this must be done before attaching to the second anchor because slack is needed to tie the slipknot.446 AFMAN36-2236 12 NOVEMBER 2003

3. Place locking loop over the slipknot. (lock down the slipknot) The locking loop prevents the slipknot from slipping. The slipknot loop will be the anchor for the self-locking hitch knot. 4. Route the rope around a second secure point or to an item to be secured. (anchor) Explain that this is an anchor for the hitch knot—not for the rope. It is important to make this distinction because the rope is not actually tied off to the anchor, and you don't want to confuse the student. 5. Route a loop partially through the locked slipknot. (lock for hitch knot) This loop will act as the locking mechanism for the hitch knot. 6. Route the free end of the rope th rough the loop. (end of hitch knot) 7. Pull the free end of the rope ti ght through the loop. (hitch knot) Take the free end of the rope, feed it through the partial loop, and pull it tight. Th is completes the tying of the hitch knot. 8. Apply enough pressure to the free end to tighten and lock the hitch. (lock) When I pull the hitch knot tight, the partial loop and the free end of the rope should tighten down, locking it in place. This should lock down the hitch knot. Explain to the students to take care when cinching this hitch knot because excessive force may break the le gs on the table you are demonstrating on. 9. Reverse the pressure to free the hitch knot. (unlock) Reverse the direction of your force to unlock the hitch knot. Explain to the students that locking and unlocking the hitch knot is part of their evaluations. Locking the loop will allow the students to continue other work without worrying about the hitch knot slipping. Also explain that when the work is done and the hitch knot is in place to stay, they can place a locking knot of some type on the hitch knot for added safety. TRANSITION: Now that you have seen one tied, it is your turn to practice a bit. MP2. PERFORMANCE-SUPERVISION: Review task steps and standards. Go over the warning again about making the hitch knot too tight. Hand out rope. Continue to encourage the students to ask questions. Monitor all students so you can identify a nd correct any potential problems. Allow students to practice as much as they need. The final practice will be a practice evaluation, performed under exactly the same conditions as the actual evaluation. (Hide poster of task steps.) TRANSITION: I have demonstrated how to tie the trucker's hitch knot and you have tried it a couple of times. Before we begin the evaluation, I want to review the steps one more time. INTERIM SUMMARY: Before starting the evaluation, review the tasks once more and emphasize any problem areas. Review the standards. Make sure students are ready for the evaluation. TRANSITION: You all seem to have a grasp on how to tie it, so let's get ready for the evaluation. I'm sure you will all do just fine. MP3. EVALUATION: Review standards. Make sure the task lis t is not in use. Start the evaluation. Monitor each student. Record the results of each student's performance (see attached form) and make notes of any areas that need to be retaught due to student diffic ulty. Tactfully reveal any areas of difficulty if any students do not successfully complete the exercise. TRANSITION: I showed you how to tie a trucker's hitch knot, let you practice, and then tested you on it. Everyone did well. Now, let's take a look at how things went.

Conclusion SUMMARY: Review key points in explanation/demonstration, performance-supervision, and evaluation. Review the task steps, emphasizing any weak areas. State the standard, which they were evaluated against.AFMAN36-2236 12 NOVEMBER 2003 447

REMOTIVATION: Today, you have learned to tie a trucke r’s simple hitch knot. The simplicity, however, belies the usefulness of the hitch knot. With this knot, you can now tie those tents as tight as you desire. You can also use it to build a survival shelter should the need arise. In your work centers, if you need to transport items in the back of a truck or van, this hitch knot can be used as a tiedown. You can even use it to, dare I suggest it, tie up the kids! Just kidding, but you can see that this simple trucker’s hitch knot does have many uses. CLOSURE: The Boy Scouts have a motto, "Always be prep ared." In mastering this hitch knot, you have added one more tool to your inventory of preparedness.448 AFMAN36-2236 12 NOVEMBER 2003

ACADEMIC INSTRUCTOR SCHOOL Maxwell Air Force Base, Alabama

PART III EVALUATION INSTRUMENT

Evaluator: __________________________________

Place a check in each box in which the item was performed to standards.
EVALUATOR COMMENTS (weak areas, things to reteach, etc.): Student Student Student Student Student Student Student Standards #1 #2 #3 #4 #5 #6 #7 Trucker’s hitch knot
must: Be tight Lock Easily unlock Be completed within 4 minutesAFMAN36-2236 12 NOVEMBER 2003 449

Attachment 10

EXAMPLE LESSON PLAN FOR METHOD OF INSTRUCTION LEARNING GAME

ACADEMIC INSTRUCTOR SCHOOL Maxwell Air Force Base, Alabama

PART I
COVER SHEET

LESSON TITLE: Capture the Flag
RESOURCE PERSON:
GAME: Experiential
REFERENCES: AFI 36-1201, Discrimination Complaints
AIDS/HANDOUTS/NOTETAKERS: Game pieces, task sheets (to be cut into three strips), a red pen, USAF ISD Model handout and AI-763f, Designing Level of Learning Lessons, handout.
READING ASSIGNMENT: None
PRESENTATION TIME: 35 Minutes

PART IA
COGNITIVE OBJECTIVE: The objective of this lesson is for each student to comprehend that discrimination negatively impacts the workplace.
COGNITIVE SAMPLES OF BEHAVIOR: 1. Describe how discrimination negatively impacts the workplace. 2. Given a scenario, predict how discrimin ation negatively impacts the workplace. 3. Give examples of how discrimination negatively impacts the workplace.
AFFECTIVE OBJECTIVE: The objective of this lesson is for each student to respond with interest to the idea that discrimination negatively impacts the workplace.
AFFECTIVE SAMPLES OF BEHAVIOR: 1. Participates in discussion on how discri mination negatively impacts the workplace. 2. Gives meaningful exampl es from experience on how discrimination negatively impacts the workplace. 3. Assists others in understanding how discrimin ation negatively impacts the workplace.450 AFMAN36-2236 12 NOVEMBER 2003

PART 1B
ORGANIZATIONAL PATTERN: Topical
STRATEGY: The lesson length is 30-37 minutes. The organi zational pattern used for this game is sequential because of the step-by-step order of events . This experiential lesson is designed to be played by four to eight people. If there are extra persons in the se minar, appoint them as observers, task them to take notes on the group dynamics, and include their inputs as part of the debrief. This will keep them involved in the process and facilitate them meeting the lesson objective. The reason for the smaller group of players is to ensure clear, observable reactions from all the players. With more than eight people, some reactions may be masked or exaggerated and it could be difficult to control. This lesson will use the inductive approach because the students should have knowledge through experience on how discrimination negatively impacts the workplace. As such, introduce the game and review the rules (MP1). Execute the game (MP2). Discuss the game results (debrief), and last, draw a formal definition of discrimination and a final generalization as a conclusion to th e activity (MP3). Therefore, ensure you do not tell the students the game is to focus on discrimination since this conclu sion will be drawn during the debrief. Prior to the class, review the task sheets to ensure familiarity wi th them. The key to the game is the proper administration of the tasks to the participants. Each team will have three tasks to accomplish. Cut the two task sheets into the three strips required to i ssue them at the sequential times. The tasks are also listed in the body of the lesson to facilitate the instructor's control over the proceedings. One team will have to do the tasks under extremely constrained conditions (discriminatory). The other team will accomplish its tasks under conditions with more flexibility. Do not give the tasks out all at once. Each team will only work on one task at a time. After both teams have finished, or upon expiration of the time limit, the next task can be started. During the playing of the game be sensitive to student reactions to the scenarios because you will use this in the debrief. Record secretively if necessary. Following game pl ay, do a thorough debrief. Include any observer's comments. Discuss student reacti ons to tasks. Some reactions may come from the"winning" side. By bringing out the feelings and results of the discrimination during the game, the students should be able to see how it negatively impacts everyone concerned. Once the students have comprehensively discussed the game and drawn the conc lusion that discrimination was at work in the scenarios, you should then review the definition of di scrimination. This ensures everyone is fully aware of what constitutes discrimination and how the Air Force de fines it. As a conclusion to the debrief, ask students how discrimination negatively impacts their jobs , productivity, view of others, etc. This will develop the generalization stated in the lesson objecti ve that discrimination negatively impacts the workplace.

LESSON OUTLINE:
MP1. Introduce game and review rules.
MP2. Execute "Capture the Flag" game.
MP3. Game debrief, provide definiti on of discrimination, and discuss its negative impact on the workplace.AFMAN36-2236 12 NOVEMBER 2003 451

PART II

INTRODUCTION

ATTENTION: How many of you like to play games? Great! You are going to get a chance to play one today.
MOTIVATION: This little game is a lot of fun and shoul d help you eventually do your job better as a supervisor and a leader. Just to add a little competition, the winning team is going to get __________________. ( NOTE: There should be some type of appropriate reward for the winning team.It could be a jar of candy, etc. Initially give the prize only to the winners to emphasize the impact of discrimination. However, BOTH teams will receiv e the prize at the end of the lesson.)
OVERVIEW: Our path today is simple. First, we're going to play the game. This game will require you to accomplish some activities following the directions I give. Follow the directions exactly or you will be penalized. I will be recording scor ing information, so don't be concerned with any writing I may be doing. When we are done, we will discuss the actions that happened in the game.
TRANSITION: Now that you know what we will be doing, le t's go over the rules of the game and get started.
MP1. Introduce Game and Review Rules. The object of the game is to reach the summit of the hill and capture the flag. You will be divided into two teams of four people. (Can be two groups of three or four depending on class size. Appoint extra personnel as monitors/notetakers.) One team is the Red Team, the other is the Blue Team. There will be a designated leader in each team. Each team will be given instructions to complete a task. The instructions must be followed completely—"to the letter!" Each team will have similar, yet different (to some extent) rules of engagement. Do not concern yourse lf with what the other team is doing. Their actions have no bearing on your ability to accomplish your task. There are only three tasks to complete. I will give you the tasks one at a time. Completion of each ta sk moves your team up the hill. The team who reaches the top of the hill first captures the flag and will receive the prize. Are there any questions?
1. Select teams. 2. Appoint leaders. 3. When ready, hand out first instruction sheet to appropriate team.
TRANSITION: Okay, now that you know what we are going to do and the game rules, let's get started.
MP2. The Game:
Task Sheet 1
Red 1. As a team, draw and label pictures of 30 items needed to construct a temporary living shelter. Draw and label as many as you can in the specified time. You have 5 minutes to complete this task. Do not speak! You may not ask the instructor for any assistan ce. If any member on your team speaks, it will constitute an automatic failure.
Blue 1. Using a red pen, draw seven items you might ne ed to construct a temporary living shelter.452 AFMAN36-2236 12 NOVEMBER 2003

( NOTE: Have a red pen available for the Blue Team if they ask you for one. ) You may quietlydiscuss your selections among yourselves and provide assistance to the person drawing. List as many items as you can in the specified time limit. You have 5 minutes to complete this task. You mayask your instructor for help if necessary. As each team completes a task, move its pl ayer game piece up the hill closer to the flag

. Only the Blue Team will be able to complete all three tasks. Find some reason to fail the Red Team on one of the tasks

. Task Sheet 2
Red 2. Without the use of any written materials or outside assistance, draw a replica of the USAF ISD model naming all functions, phases, and other labe ls as specified by AFMAN 36-2234, Section B. You have 5 minutes to complete this task.
Blue 2. Draw a replica of the USAF ISD Model naming all functions, phases, and othe r labels as specified by AFMAN 36-2234, Section B. You mayuse handouts and manuals as provided by AIS. Do not speak loud enough for the Red Team to hear your answers. You have 7 minutes to complete this task. Have an ISD Model handout available for the Blue Team.
Task Sheet 3
Red 3. Without the use of any written materials, list the de sign elements in the design phase as taught in AIS lesson titled "Designing Leve l of Learning Lessons." You may discuss this quietly among yourselves. You have 2 minutes to complete this task.
Blue 3. Using AI-763f Handout, Designing Level of Learning Lessons, list the design elements taught in that lesson. You mayask the instructor for assistance if necessary. Do not speak loud enough for the Red Team to hear your answers. You have 3 minutes to complete this task. Have AI-763f handout available for the students if they ask for one.
TRANSITION: When the Blue Team reaches the top, decl are them the winners and award the prize. Then have the groups assemble together and do a debrief. MP3. Debrief/Summary:

Questions Anticipated Responses

LOQ: Red Team, how did you feel as you Unfair went through each task? Other unit was getting advantages

No apparent reason

a. FUQ: What kinds of thoughts were you thinking about:AFMAN36-2236 12 NOVEMBER 2003 453

Let's look at the definition of discrimination: Webster defines it as "treatment or distinction not based on individual merit in favor of or against a person, group, etc." AFI 36-2706, Military Equal Opportunity and Treatment Program, refers to unlawful discrimination and defines it as: "...discrimination on the basis of color, national origin, race, religion, or sex that is not otherwise authorized by law or regulation." (1) task 1? Couldn't talk, other team could, no apparent reason. Not fair

(2) task 2? Unfair, couldn't use outside materials and less time for no apparent reason

(3) task 3? Unfair, couldn't ask for help, other team could, for no apparent reason

b. FUQ: How did this affect your operations Made us less likely to try as a team? Demotivating

LOQ: Blue Team, how did the rules of Made us less likely to try hard engagement (ROE) affect your team's Knew we couldn't lose operations?" Embarrassed

a. FUQ: How did you feel about your Less than worthy or unearned achievements when it became obvious they Again, embarrassed were achieved under less than fair circumstances?

b. FUQ: Ask for observations from any other students in the group.

Questions Anticipated Responses

LOQ: How were the ROE of this game discriminatory? Gave favorable treatment to one group over the other and it was not based on merit

a. FUQ: What do you think would be the Low morale454 AFMAN36-2236 12 NOVEMBER 2003

TRANSITION: Well, that just about wraps up this exercise. Let's take a look at what we have done today. results in the work setting if these same type of Tension discriminatory practices were used? Dislike for the favored group

Low productivity

Have and have-nots

Unfair promotion or advancement

View that some are better than others at job

b. FUQ: Based on these expected results, what is the logical conclusion we can draw Discrimination has a negative impact on the work place from this exercise? (When the students come up with this conclusion, display the lesson objective.)AFMAN36-2236 12 NOVEMBER 2003 455

CONCLUSION
SUMMARY: Today we had a chance to play a game that so me would say was unfair. That is exactly what discrimination is—UNFAIR. It can either hold indivi duals or groups back or give recognition, advancement, advantage, profit, status, et c., to groups or individuals, not ba sed on their merit, but based on who they are. We saw how discriminatory practices not onl y affect the ability of a group to succeed, but also the negative impact it has on everyone involved. ( NOTE: Summarize observations students and you made during the debrief. Make a clear correlation between observations, te stimony, and the lesson objective.) Winners and losers alike are really both losers in the game of discrimination. The individual, the group, the job, the Air Force, and society all suffer in this loss. We cannot afford discrimination—the cost is too great.
REMOTIVATION: While discrimination isn't always as blatant as it was in this game, it will still have a negative impact on the attitudes and morale of military units and the individuals who make up the team. This exercise clearly demonstrated that fact. You have seen the effects yourselves and can understand the pain and embarrassment that can be caused.
CLOSURE: So, the next time you have an opportunity to in teract with someone, give him or her a fair shake. Also, if you think you are being discriminated against, respond positively to correct the situation. Discrimination is not fair to anyone; ther efore, since we are all winners here, both teams get the prize. (Give the Red Team the same prize the Blue Team received.)456 AFMAN36-2236 12 NOVEMBER 2003

TEAM TASKS

RED TEAM TASKS

Red 1. As a team, draw and label pictures of 30 items ne eded to construct a temporary living shelter. Draw and label as many as you can in the specified time. You have 5 minutes to complete this task. Do not speak! You may not ask the instructor for any assistance . If any member on your team speaks, it will constitute an automatic failure.

Red 2. Without the use of any written materials or outside assistance, draw a replica of the USAF ISD model naming all functions, phases, and other labels as specified by AFMAN 36-2234, Section B. You have 5 minutes to complete this task.

Red 3. Withoutthe use of any written materials, list the Design Elements in the design phase as taught in AIS lesson titled "Designing Level of Learning Lessons." You may discuss this quietly. You have 2 minutes to complete this task.

BLUE TEAM TASKS Blue 1. Using a red pen, draw seven items you might need to construct a temporary living shelter. You may quietly discuss your selections among yourselves and provide assistance to the person who is drawing. List as many items as you can in the specifie d time limit. You have 5 minutes to complete this task. You mayask your instructor for help if necessary.

Blue 2. Draw a replica of the USAF ISD Model naming all functions, phases, and othe r labels as specified by AFMAN 36-2234, Section B. You mayuse handouts and manuals as provided by AIS. Do not speak loud enough for the Red unit to hear your answers. You have 7 minutes to complete this task.

Blue 3. Using AI-763f Handout, Designing Level of Learning Lessons, list the design elements taught in that lesson. You may ask the instructor for assistance if ne cessary. Do not speak loud enough for the Red unit to hear your answers. You have 3 minutes to complete this task.AFMAN36-2236 12 NOVEMBER 2003 457

ACADEMIC INSTRUCTOR SCHOOL Maxwell Air Force Base, Alabama

PART III

TEST ITEM

LESSON OBJECTIVE: The objective of this lesson is for each student to comprehend that discrimination negatively impacts the workplace.
SAMPLE OF BEHAVIOR: Describe how discrimination negatively impacts the workplace. TEST QUESTION: Which of the following BEST describes how disc rimination negatively impacts the workplace? a. Decreases morale overall, but increases espritin various organizations within the whole unit. b. Produces hostility and frustration because it lowers morale and productivity. c. Decreases esprit throughout the unit; however, it raises productivity in certain sectors.
KEY: b
RATIONALE: Option b is the correct response. Discrimination creates hostility and frustration in the affected individuals and may cause them to resent co workers, thus adversely affecting the working relationship. Option a is incorrect because although it ma y decrease morale overall, it should not increase esprit anywhere in the unit. Option c is incorrect becaus e although it will decrease esprit, it will not raise productivity.458 AFMAN36-2236 12 NOVEMBER 2003

Attachment 11

FEEDBACK FOR STUDENT SPEAKING

ACADEMIC INSTRUCTOR SCHOOL Maxwell Air Force Base, Alabama
A11.1. Procedures for Giving Feedback. Make the presentation feedback session as effective as possible by adjusting to the communication transaction. For example, when giving a student feedback, consider as wide a range of variables as possible—the type of speech, the maturity of the student, the setting for the session, etc. What works in one situation may not work in another. Taking a transactional approach shows that you recognize the complexity of the feedback process. Specifically: A11.1.1. Adapt feedback to the student. Human bei ngs are sensitive to comments made about them— the person is always more important than the perfor mance. A word of praise or a word of disapprovalwill stay with the student for a long time. When reviewing a presentation, try to separate the performance from the individual. Instead of saying "what you did wrong was ...," say "what you might have done to make the presentation better is to ...," or instead of "what you shoul d have done is ...," say "the next time you speak you might ... ." A11.1.2. Comment on both content and delivery. Often an instructor directs attention almost exclusively to delivery (focusing on such things as eye c ontact, movement, gestures, or voice) or on content alone (focusing on such things as subject, main po ints, or supporting material). To help the student develop totally as a speaker, be sure to consider both aspects. A11.1.3. Encourage self-evaluation. Evaluation is often thought of as one-way communication where the instructor furnishes all the ideas—a kind of self-a ction. The best feedback is more of an interaction or a transaction. Instead of saying to a student, "you did this," or "you didn't se em to pay any attention to that," a better approach might be to ask, "what do you think of your perfor mance?" Or "what could you do to improve?" In this way, the student become s involved with the instructor. If the student does not respond, the instructor might ask if there is any particular problem the student would like addressed. Remember, the first goal is to he lp the student—not merely to give advice. A11.1.4. Follow criticism with suggestions for improvement . It doesn't help students to tell them their attention steps are ineffective or their organization is difficult to follow if a better approach is not suggested. Constructive feedback implies building, not tearing down. A11.1.5. Mention as many favorable factors as you honestly can. Candid approval opens the recipient's mind. Speaking in front of others is a terrifying experience for many people. Most people need affirmation for things they did well. If many negative factors and few positiv e ones are mentioned, the challenge to improve seems overwhelming. A11.1.6. End comments on a note of praise. Repeat any outstanding item in the performance. If it is difficult to find items to praise, describe the progr ess the student is making. Undeserved praise makes the instructor less credible with the class and the student being reviewed.
A11.2. Common Traps. Make the presentation feedback as effe ctive as possible by avoiding these common traps:AFMAN36-2236 12 NOVEMBER 2003 459

A11.2.1. Avoid abstract general approval (or disa pproval). Instructors who say only "that was a good presentation" are not doing their jobs. Adjectives su ch as "wonderful" or "interesting" may make the student feel good for the moment, but they provide no motivation for growth or improvement. By the same reasoning, abstract expressions of disapproval such as "terrible" or "bad" also fall short of the objective as useful feedback. A11.2.2. Avoid summarizing the presentation. After a well-organized student has finished, the instructor may be tempted to enumerate the mainpoints and summarize the support. While such comments may be interesting and show that the instruct or was listening, the result is simply a short speech on a longer one. Summary is not feedback. A11.2.3. Resist the impulse to extend the presenter's subject. A student's topic may be so arresting or controversial that you may want to take the floor yourself and present additional ideas. The student's train of thought may legitimately stir up associated ideas, but free association is not feedback. A11.2.4. Avoid calling attention to minor faults th at will correct themselves automatically. Suggestions to the student for improvement should help make the presentation better. For example, telling the student to speed up, move around more, quit playing with the paper clip, or stop being so nervous have limited usefulness. Keep trivialities in proportion. A11.2.5. Don't criticize something that can't be corrected. It does little good to criticize eye contact of a student who has crossed eyes or the accent of a person from another part of the country. If the problem is not correctable or the correction is not worth the effort and expense is required, then don't mention it.460 AFMAN36-2236 12 NOVEMBER 2003

Attachment 12

GUIDELINES FOR PROVIDING FEED BACK ON STUDENT WRITING
ACADEMIC INSTRUCTOR SCHOOL Maxwell Air Force Base, Alabama

A12.1. Always keep in mind the purpose in giving feed back—to improve the student's ability to write. No matter how much you would like to edit or rewrite the whole piece, don't do it. There just isn't time, and it is the student's job to revise from your clear clues.

A12.2. Strive for maximum objectivity. If you think something is wrong but you can't put your finger on the exact problem, don't bluff. Always try to be honest, fair, and unbiased.

A12.3. Don't arbitrarily impose your own personal style or preference on the student. The comment, "I like it better this way," is not adequate justificat ion for downgrading a student's work. If you can follow this comment with a reason or a reference to the Guide for AF Writing, you are on much firmer ground. If what has been written communicates, but you wouldn't have said it that way, let it be. Judge the writing by its merit alone. You can't measure the amount of effort or improvement.

A12.4. Don't give undue emphasis to mechanical deta ils such as punctuation, capitalization, and grammar. They are important, but less significant than th e quality of the thought. Most mechanical details are not vital to the communication process; they merely as sist in it. When usage is divided about mechanics or spelling, be flexible, but be sure the student has the same form throughout —encourage consistency.

A12.5. Remember that good writing is hard work and that writers have a justifiable pride in their efforts. Don't nit-pick, wordsmith, or find fault just to be fi nding fault. Your feedback must be constructive to be acceptable. All writers have personal and social needs. If writing is loaded with errors, try to comment on the most important ones and deal with the others at a later time.

A12.6. Approach the feedback job systematically. A12.6.1. If you can physically separate the parts of th e paper, do so. For ready reference, place the table of contents or overview on one side and the notes on another. A12.6.2. Read the title or subject alone and see if you understand it. What expectation does the title arouse in the reader? Does the title gi ve direction and a clue to content? A12.6.3. On your first reading, attempt no corrections. Instead: A12.6.3.1. Try to get a feel for the writer's focus and direction, the organization of main points, and the quality of the support. See if the conclusions or recommendations are justified. In other words, see if the writing as a whole adds up before you worry too much about the mechanics. Every time you spot a mechanical error, however, make a check in the margin. A12.6.3.2. Before you read the paper the second time, jot down your impression of the paper's overall quality apart from mechanical deficiencies. Does the writing communicate? If so, why? If not, what are the distracters?AFMAN36-2236 12 NOVEMBER 2003 461

A12.6.3.3. On the second reading, analyze why you made checkmarks and identify the errors in the margin. Rewrite one or two ite ms, if necessary, to show what you mean by the corrections, but don't get in the habit of a complete edit.

A12.7. In grading writing, use your pencil as carefully as a surgeon's scalpel. A12.7.1. Don't be sarcastic, cute, or cryptic. Don't try to impress the student; try to improve the performance. When marking out words, be careful you don't cross out something necessary to the writer's meaning. A12.7.2. Make your comments meaningful. For instance: A12.7.2.1. The comment "wordy" is not enough. Circle the unnecessary words. A12.7.2.2. The comment "awkward" is not helpful. Be specific. Tell the student, for example, the sentence is "awkward because of a dangl ing modifier or a misplaced adjective."A12.7.2.3. A question mark in the margin merely raises a question. If you find the sentence confusing or ambiguous, point out where the confusion exists by asking a specific question or by suggesting a way to eliminate the confusion.

A12.8. Don't be so concerned with style that you igno re the quality of examples or support material. Make sure, for instance, that the writer has trans itioned smoothly into each quotation, using phrases like the following: "As the Secretary of State wrote in Foreign Affairs in the spring of 1973... ," "In 1942, a professor at Cornell summarized the problem as... ."

A12.9. Use the standard proofreader's marks to avoid confusion. (See Figure A13.1.)
Figure A12.1. Standard Proofreader's Marks.