**Federal Acquisition Institute (FAI)**

**Pilot Evaluation of**

**Program Management Government Specific Course III**

**Final Report**

**Prepared for:**

**Federal Acquisition Institute**

**9820 Belvoir Road, Bldg. 291**

**Ft. Belvoir, VA 22060**

**And**

**US Office of Personnel Management**

**Training and Management Assistance Program**

**1900 E Street, NW**

**Room 1453**

**Washington, DC 20415-0001**

**Developed by:**

**PerformTech Inc.**

**810 King Street**

**Alexandria, VA 22314**

**Under OPM Contract Number: OPM-07-00005**

**Project Code: 3130**

**July 11, 2008**

810 King Street  Alexandria, VA  703.548.0320  Fax 703.548.2981

# 

###### Table of Contents

###### 

[Overview 3](#overview)

[Description of Program 3](#description-of-program)

[Program Goal 3](#program-goal)

[Training Modules 3](#training-modules)

[Analysis and Overall Findings 5](#analysis-and-overall-findings)

[Evaluation Form 5](#evaluation-form)

[Analysis 9](#analysis)

[Findings 9](#findings)

[Observations and Recommendations 11](#observations-and-recommendations)

[Structure of Course 11](#structure-of-course)

[Content of Course 12](#content-of-course)

[Presentation of Course 13](#presentation-of-course)

[Suggestions from Participants 14](#suggestions-from-participants)

[Appendix A: Agenda 18](#appendix-a-agenda)

[Appendix B: Evaluation Comments 19](#appendix-b-evaluation-comments)

##  

# Overview

This report provides an evaluation summary of the FAI Program Management
Government Specific Course III, conducted by the Federal Acquisitions
Institute (FAI). The training was targeted to program managers and it
was offered during the week of June 2-6, 2008.

## Description of Program 

The Program Management Government Specific Course III is an
instructor-led course which covers the senior/expert level of
contracting and acquisition in the federal government, leadership
techniques and skills, and an overview of some important processes used
for the allocation of resources and performance-based considerations. It
took place over the course of five days and was led by instructor
Michael O'Brochta, a consultant hired by FAI.

## Program Goal

The goal of the Program Management Government Specific Course III is to
increase the ability of participants to:

1.  Understand the federal acquisition process

2.  Use appropriate program management techniques to improve the success
    of their projects.

The long-term outcome desired is to make the participants better
informed about program management skills and procedures that are vital
when working in the federal government.

## Training Modules

## 

The training modules that comprise the Program Management Government
Specific Course III include:

Course Introduction

Module 1: Leadership/Professional Skills

Module 2: Agency Policy and Practices

Module 3: Basics of Contracting

Module 4: Pre-Solicitation Phase

Module 5: Perform Source Selection

Module 6: Contract Management

Module 7: Performance-Based Agreements

Module 8: Life Cycle Logistics (LCL) Management, Product Support and
Interoperability

Module 9: Business Financial Planning and Management

Course Review

The draft modules were reviewed extensively by a subject matter expert
in May 2008.

Each module consisted of some PowerPoint slides that were delivered as
an overhead presentation. Some modules include case studies and all
modules included some type of participant activity and review questions.

The participant guides were distributed in binders to the participants,
and the instructor guide was given to the instructor, Michael O'Brochta,
in a binder.

Course III on Program/Project Management Government Specific is the
third of three courses that was offered on this topic. The other two
courses were designed for entry level project managers and mid-level
project managers and new program managers in the federal government and
cover content that is less advanced and more appropriate for their level
of management.

# Analysis and Overall Findings

This section describes:

-   FAI -- Program Management Government Specific Course III participant
    evaluation form

-   How the analysis was conducted

-   A summary of the findings

## Evaluation Form 

The training evaluation instrument has two sections. See a sample of the
form below. (Amy, evaluation form should be presented first so the
reviewer can look at the form and then be able to read the specifics and
refer back to the form)

##### Section 1

#### 

Section 1 evaluates the training modules. The instructions read:

> **"**Please rate **each** training module based on **how useful** you
> think it will be in your job. Please **circle the number** that best
> shows how useful it will be, on a scale of 1 to 5, with **1 = Not
> Useful** and **5 = Very Useful**. If you did not attend the session
> for a module, please mark **N/A**. To further help us improve the FAI
> Program Management Training Program, please also **tell us why** you
> gave a particular rating."

For each module a space is provided for making comments about that
module. There are 9 modules to evaluate.

##### Section 2

Section 2 evaluates the training experience. The instructions read:

"Thinking about the training overall, please rate **each** topic below
based on how much you **agree or disagree** with the statement. Please
circle the number that best shows your opinion, on a scale of 1 to 5,
with **1 = Strongly Disagree** and **5 = Strongly Agree**. To further
help us improve the FAI Program Management Training Program, please also
**tell us why** you gave a particular rating."

There are six statements that participants are asked to rate.

1.  The objectives provided for each module were clear.

2.  The course materials were accurate.

3.  The course materials helped me to understand the course content.

4.  The activities/questions reinforced the learning.

5.  The audio-visual materials supported the learning.

6.  The content covered in this course is relevant to my job.

For each statement a space is provided for comments.

Finally, there is a place for additional comments.

Participant Evaluation Form

#### [Section 1]{.underline} Please rate each training module based on how useful you think it will be in your job. Please circle the number that best shows how useful it will be, on a scale of 1 to 5, with 1 = Not Useful and 5 = Very Useful. If you did not attend the session for a module, please mark N/A. To further help us improve the FAI Project Management Training Program, please also tell us why you gave a particular rating.

+-------------------------+------+------+------+------+------+------+
| **Category**            | **   |      |      |      |      |      |
|                         | Rati |      |      |      |      |      |
|                         | ng** |      |      |      |      |      |
+-------------------------+------+------+------+------+------+------+
|                         | *    |      |      |      | **   | *    |
|                         | *Not |      |      |      | Very | *Did |
|                         | Usef |      |      |      | Usef | Not  |
|                         | ul** |      |      |      | ul** | Atte |
|                         |      |      |      |      |      | nd** |
+-------------------------+------+------+------+------+------+------+
| ####                    | 1    | 2    | 3    | 4    | 5    | N/A  |
| ### Course Introduction |      |      |      |      |      |      |
+-------------------------+------+------+------+------+------+------+
| Comments:               |      |      |      |      |      |      |
+-------------------------+------+------+------+------+------+------+
| ##                      | 1    | 2    | 3    | 4    | 5    | N/A  |
| ##### Module 1: Leaders |      |      |      |      |      |      |
| hip/Professional Skills |      |      |      |      |      |      |
+-------------------------+------+------+------+------+------+------+
| Comments:               |      |      |      |      |      |      |
+-------------------------+------+------+------+------+------+------+
| **Module 2: Agency      | 1    | 2    | 3    | 4    | 5    | N/A  |
| Policy and Practices**  |      |      |      |      |      |      |
+-------------------------+------+------+------+------+------+------+
| Comments:               |      |      |      |      |      |      |
+-------------------------+------+------+------+------+------+------+
| **Module 3: Basics of   | 1    | 2    | 3    | 4    | 5    | N/A  |
| Contracting**           |      |      |      |      |      |      |
+-------------------------+------+------+------+------+------+------+
| Comments:               |      |      |      |      |      |      |
+-------------------------+------+------+------+------+------+------+
| **Module 4:             | 1    | 2    | 3    | 4    | 5    | N/A  |
| Pre-Solicitation        |      |      |      |      |      |      |
| Phase**                 |      |      |      |      |      |      |
+-------------------------+------+------+------+------+------+------+
| Comments:               |      |      |      |      |      |      |
+-------------------------+------+------+------+------+------+------+
| **Module 5: Perform     | 1    | 2    | 3    | 4    | 5    | N/A  |
| Source Selection**      |      |      |      |      |      |      |
+-------------------------+------+------+------+------+------+------+
| Comments:               |      |      |      |      |      |      |
+-------------------------+------+------+------+------+------+------+
| **Module 6: Contract    | 1    | 2    | 3    | 4    | 5    | N/A  |
| Management**            |      |      |      |      |      |      |
+-------------------------+------+------+------+------+------+------+
| Comments:               |      |      |      |      |      |      |
+-------------------------+------+------+------+------+------+------+
| **Module 7:             | 1    | 2    | 3    | 4    | 5    | N/A  |
| Performance-Based       |      |      |      |      |      |      |
| Agreements**            |      |      |      |      |      |      |
+-------------------------+------+------+------+------+------+------+
| Comments:               |      |      |      |      |      |      |
+-------------------------+------+------+------+------+------+------+
| **Module 8: Life Cycle  | 1    | 2    | 3    | 4    | 5    | N/A  |
| Logistics (LCL)         |      |      |      |      |      |      |
| Management, Product     |      |      |      |      |      |      |
| Support and             |      |      |      |      |      |      |
| Interoperability**      |      |      |      |      |      |      |
+-------------------------+------+------+------+------+------+------+
| Comments:               |      |      |      |      |      |      |
+-------------------------+------+------+------+------+------+------+
| **Module 9: Business    | 1    | 2    | 3    | 4    | 5    | N/A  |
| Financial Planning and  |      |      |      |      |      |      |
| Management**            |      |      |      |      |      |      |
+-------------------------+------+------+------+------+------+------+
| Comments:               |      |      |      |      |      |      |
+-------------------------+------+------+------+------+------+------+
| **Course Review**       | 1    | 2    | 3    | 4    | 5    | N/A  |
+-------------------------+------+------+------+------+------+------+
| Comments:               |      |      |      |      |      |      |
+-------------------------+------+------+------+------+------+------+

#### 

#### Section 2 Thinking about the workshop overall, please rate each topic below based on how much you agree or disagree with the statement. Please circle the number that best shows your opinion, on a scale of 1 to 5, with 1 = Strongly Disagree and 5 = Strongly Agree. To further help us improve the FAI Project Management Training Program, please also tell us why you gave a particular rating.

+----------------------------+-------+-------+-------+-------+--------+
| **Category**               | **Rat |       |       |       |        |
|                            | ing** |       |       |       |        |
+----------------------------+-------+-------+-------+-------+--------+
|                            | **Str |       |       |       |        |
|                            | ongly |       |       |       |        |
|                            | Stron |       |       |       |        |
|                            | gly** |       |       |       |        |
|                            |       |       |       |       |        |
|                            | **Dis |       |       |       |        |
|                            | agree |       |       |       |        |
|                            | Ag    |       |       |       |        |
|                            | ree** |       |       |       |        |
+----------------------------+-------+-------+-------+-------+--------+
| **The objectives provided  | 1     | 2     | 3     | 4     | 5      |
| for each module were       |       |       |       |       |        |
| clear.**                   |       |       |       |       |        |
+----------------------------+-------+-------+-------+-------+--------+
| Comments:                  |       |       |       |       |        |
+----------------------------+-------+-------+-------+-------+--------+
| **The module materials     | 1     | 2     | 3     | 4     | 5      |
| were accurate.**           |       |       |       |       |        |
+----------------------------+-------+-------+-------+-------+--------+
| Comments:                  |       |       |       |       |        |
+----------------------------+-------+-------+-------+-------+--------+
| **The course materials     | 1     | 2     | 3     | 4     | 5      |
| helped me to understand    |       |       |       |       |        |
| the course content.**      |       |       |       |       |        |
+----------------------------+-------+-------+-------+-------+--------+
| Comments:                  |       |       |       |       |        |
+----------------------------+-------+-------+-------+-------+--------+
| **The activities/questions | 1     | 2     | 3     | 4     | 5      |
| reinforced the learning.** |       |       |       |       |        |
+----------------------------+-------+-------+-------+-------+--------+
| Comments:                  |       |       |       |       |        |
+----------------------------+-------+-------+-------+-------+--------+
| **The audio-visual         | 1     | 2     | 3     | 4     | 5      |
| materials supported the    |       |       |       |       |        |
| learning.**                |       |       |       |       |        |
+----------------------------+-------+-------+-------+-------+--------+
| Comments:                  |       |       |       |       |        |
+----------------------------+-------+-------+-------+-------+--------+
| **The content covered in   | 1     | 2     | 3     | 4     | 5      |
| the course is relevant to  |       |       |       |       |        |
| my job.**                  |       |       |       |       |        |
+----------------------------+-------+-------+-------+-------+--------+
| Comments:                  |       |       |       |       |        |
+----------------------------+-------+-------+-------+-------+--------+

# 

Please use this space to give us any additional comments on the
workshop: \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
Optional: Name and e-mail address---

## Analysis

The data responses and comments for 16 attendees were compiled and
analyzed. There were 19 participants that attended the pilot.

## Findings

In general, the training was rated satisfactorily. Participants judged
it to be helpful and worthwhile, but also commented on missing content
that might be helpful along with additional resources and material that
was needed. The charts on the next pages show that the average data
scores for the 10 modules (Section 1) ranged from 2.6 to 3.5. There was
a much larger range in the average data scores for the Section 2
statements: 1.8 to 3.5.

Here is a representative sample of some of the comments made:

-   "Pulls in the importance of leadership in PM, which is lacking in
    other PM courses"

-   "Focus on real life experiences were helpful-learning from others"

-   "Case study was good. I enjoyed the case study and working with my
    table team. Mike was great and kept the class going at a level I
    could understand."

-   "Cross agency discussion was excellent"

-   "VCF was a good example and working in teams was good"

-   "Good Exercises. Focus on real life experiences were
    helpful-learning from others"

-   "Good take-away about need to always manage regardless of contract
    type"

-   "I enjoyed the case study and working with my table team. Mike was
    great and kept the class going at a level I could understand"

-   "Good info on Policy and Practices. Good take-away about need to
    always manage regardless of contract type "

-   "Mike did a great job as a facilitator and is certainly a PM SME. He
    inspired me to do more PM professional reading. I enjoyed every
    minute of the course."

The findings are discussed more fully in the next section: Observations
and Recommendations.

Module Ratings

  ----------------------------------- -----------------------------------
  **Module**                          **Average Rating (1-5)**

  Course Introduction                 **2.6**

  Module 1: Leadership/Professional   **3.4**
  Skills                              

  Module 2: Agency Policy and         **3.0**
  Practices                           

  Module 3: Basics of Contracting     **3.2**

  Module 4: Pre-Solicitation Phase    **3.3**

  Module 5: Perform Source Selection  **3.4**

  Module 6: Contract Management       **3.2**

  Module 7: Performance-Based         **3.5**
  Agreements                          

  Module 8: Life Cycle Logistics      **3.2**
  (LCL) Management, Product Support   
  and Interoperability                

  Module 9: Business Financial        **2.8**
  Planning and Management             
  ----------------------------------- -----------------------------------

**Statement Ratings**

  ----------------------------------- -----------------------------------
  **Statement**                       **Average Rating (1-5)**

  The objectives provided for each    **2.7**
  module were clear.                  

  The course materials were accurate. **1.8**

  The course materials helped me to   **2.0**
  understand the course content.      

  The activities/questions reinforced **3.4**
  the learning.                       

  The audio-visual materials          **2.5**
  supported the learning.             

  The content covered in the course   **3.5**
  is relevant to my job.              
  ----------------------------------- -----------------------------------

# Observations and Recommendations 

The table below provides observations and recommendations about:

-   The structure of the course

-   The content of the course

-   The presentation of the course

## Structure of Course

+-----------------------------------+-----------------------------------+
| **Observation**                   | **Recommendation**                |
+===================================+===================================+
| 1.  It is stated repeatedly that  | Restructure the order of the      |
|     the material is good but is   | modules to enhance the quality of |
|     not organized correctly.      | the training.                     |
+-----------------------------------+-----------------------------------+
| 2.  The pace of the course is too | Determine what content can be     |
|     slow.                         | eliminated to speed up the course |
|                                   | delivery.                         |
+-----------------------------------+-----------------------------------+
| 3.  Some of the objectives were   | Review the course content to      |
|     not achieved.                 | ensure that all objectives have   |
|                                   | been met. Modify content in any   |
|                                   | modules where objectives have not |
|                                   | been met.                         |
+-----------------------------------+-----------------------------------+

## Content of Course

+-----------------------------------+-----------------------------------+
| **Observation**                   | **Recommendation**                |
+===================================+===================================+
| 1.  Requests for additional       | Review these requests to          |
|     content and material were     | determine what additional content |
|     present throughout each       | and materials can be added to the |
|     module.                       | course.                           |
+-----------------------------------+-----------------------------------+
| 2.  We might want to add a        | Glossary has already been         |
|     glossary or a definition      | provided to FAI.                  |
|     page. A lot of participants   |                                   |
|     stated that they needed one   |                                   |
|     for this course because of    |                                   |
|     the large amount of terms.    |                                   |
+-----------------------------------+-----------------------------------+
| 3.  Some of the content could not | The participants were from a      |
|     be applied to the             | variety of government agencies.   |
|     participants' specific jobs.  | In some cases, specific job       |
|     This could be inevitable      | functions are different agency by |
|     considering the wide variety  | agency. The                       |
|     of responsibilities that each | instructor/facilitator will need  |
|     of the participants have in   | to be aware of the mix of         |
|     their job.                    | participants and work with them   |
|                                   | to identify differences and       |
|                                   | similarities.                     |
+-----------------------------------+-----------------------------------+
| 4.  It is stated that a lot of    | Review the content to determine   |
|     the material was too          | where basic information can be    |
|     'elementary' in nature and    | removed and replaced with more    |
|     not appropriate to the level  | advance program management        |
|     of the participants attending | content related to the particular |
|     the training.                 | topic.                            |
+-----------------------------------+-----------------------------------+

## Presentation of Course

  -----------------------------------------------------------------------
  **Observation**                     **Recommendation**
  ----------------------------------- -----------------------------------
  No additional feedback provided     

  -----------------------------------------------------------------------

# Suggestions from Participants

**Module 2 - Agency Policy and Practices:**

> Put all program management responsibilities onto one condensed sheet

**Module 3 - Pre-Solicitation Phase:**

> The role of a program manager in many agencies is very ambiguous (not
> precise)
>
> Bill Archibald's [View of Program Managers]{.underline} -- represent a
> threat to the status-quo
>
> View that CO's have of program and project managers -- think that
> sometimes PM's are just difficult -- CO's are able to use "bribes" to
> negotiate a fix to the contract
>
> Third activity doesn't fit in the module -- Does not focus on
> requirements -- module focuses on CO-PM relationship, activity focuses
> on the PM, IPT and PCO
>
> Discussed how program and project management works these days in the
> FBI
>
> Discussed integrating new technology to make information about
> program/project management more readily available
>
> Interoperability between Agents and PMs
>
> All PMs in the government are experiencing similar problems with
> information sharing, finishing projects that were started by other
> people, etc.
>
> Put Risk Analysis Methods in the Participant Guide
>
> Just make copies of the page that explains these analyses

**Module 5 - Perform Source Selection:**

> In technical be sure to stress Performance Measures
>
> Add project management to the management responsibilities
>
> Colors and adjectives are used to symbolize risk. Not numbers or
> letters
>
> No Slides were used - - -?

**Module 6 - Contract Management:**

> DoD database of projects (over 400) shows that "If CPI is .9 or worse
> after 15% of budget spent -- the CPI never improves (very important;
> already on a slide)
>
> EVM Co-I meetings are a good thing to mention to go to. Ask Michael
> for more information
>
> Almost this module was skipped because the students felt like the
> information was already covered
>
> The beginning was useful, but they all felt like the middle and end
> was unnecessary

**Module 7 - Performance-Based Agreements:**

> How do you document when changes to the contract need to be made.
> (i.e., if something must be done to fix something)
>
> Michael mentioned what program management program is used in the CIA
>
> Missing instruction in the activity: **We need to have the groups
> select a point in time and then answer the questions.**
>
> The wording of this activity threw some of the participants off (must
> be reworked)

**Module 8 - Life Cycle Logistics (LCL) Management, Product Support and
Interoperability:**

> All the participants are saying that they want everything in the
> Instructor Guide in the Participant Guide. In other words, they want a
> text book. Michael also thought that this is how the course should be
> run.
>
> We need a glossary of terms for Life-Cycle Logistics (there are
> different terms used by each agency; the students want a list of terms
> because it will help them to communicate with other agencies about
> Life-Cycle Logistics).
>
> Regarding the activity: For the instructions, we need to ask
> participants to choose a specific point in time during the life-cycle
> of the project

Material from Instructor Guide starting on page 8-31 should be given to
the participants

**Errors in Participant Guide that need fixing:**

> **Pg. 21**
>
> Slide 1-15 empowerment
>
> Second sub bullet -- the word "to" is missing
>
> **Page 23**
>
> Number 3
>
> Contact instead of compact
>
> Fix the last sentence, something is missing
>
> Number 4
>
> Second sentence
>
> "What versus that"
>
> Next to last sentence: should be "it's"
>
> What does the whole sentence "mean"?
>
> **Pg. 24**
>
> Item 9; second sentence
>
> "Servant" should be changed to "servant"
>
> Number 10
>
> "Caused a sense of loss" sense is missing

**Additional Comments**:

Flow of both Instructor Guide and Participant Guide should match without
any

CD-Rom in the back of the Participant Guide might be a good idea
(take-away material)

Online resources would be a good idea

"Decisiveness" wasn't covered enough

For each section, go into what is happening at each individual agency

> More clarification on what "government specific" means and exactly
> what will be covered during the class
>
> Specifics of contracting might not be necessary; At SSA they don't
> handle contracting the same way as the other agencies.
>
> More leadership, risk management, etc. (common areas across the entire
> government)
>
> The course should start with contracting reference material, going
> thru it to make sure that everyone understands the wording. Then you
> can go into what the differences between each agency is (open
> discussion).
>
> No title slides. No filler pages. If there isn't information on the
> page, don't give it to us
>
> Have someone at a higher level speak to the participants about what is
> expected from a program manager in each agency
>
> Participants stated they do not need slides and wish to have more
> blank pages in the back to take notes.
>
> It is important to make sure that the participants have the same base
> of knowledge of contracting after the course is over
>
> More information about recent IT, acquisition, etc. issues from
> different agencies -- specific lessons learned
>
> The slides are too small in the Participant Guide
>
> This course should have discussed A-123
>
> The participants want to be able to take the competencies and proof
> that they took the class and be able to have their boss accept them as
> having these skills.
>
> Need to make sure that everyone has the prerequisites to make sure
> that they know what to know, what to wear, where it is, etc.
>
> Greatest value was in listening to what other agencies do with
> contracting
>
> We need an educated assessment of how well the participant knows the
> material to make sure that they are covered on this competency
>
> The biggest benefit from this course was hearing what the problems
> other agencies have to deal with and how they actually deal with them
>
> The material for the class was at a lower level than necessary. It
> should be more interactive; focused more on discussion amongst the
> students and teacher. (more real-world examples)
>
> It is too low-level for the participants. They, for example, have all
> created RFPs on their own, so they don't need an activity where they
> write one.
>
> It was suggested that the Instructor Guide is used as the reference
> material (the prework was not used).
>
> It would be beneficial to state up front that the course is intended
> to be a forum of discussion.
>
> The expectations from the blueprint need to be discussed and covered,
> but things that aren't connected with those expectations
> (competencies) need to be omitted.
>
> Feedback session is really important
>
> Everyone involved in this course should put you into a community where
> they can continue to talk to each other and ask questions
>
> Perhaps some take home work can be prepared and provided for each day.

# Appendix A: Agenda

Program Management

Government Specific Course III

Course Agenda

+------------------------------+---------------------------------------+
| **Day 1**                    |                                       |
+------------------------------+---------------------------------------+
| **Morning**                  | Course Introduction                   |
|                              |                                       |
|                              | Professional/Leadership Skills        |
+------------------------------+---------------------------------------+
| **Afternoon**                | Professional/Leadership Skills        |
|                              | (cont'd.)                             |
|                              |                                       |
|                              | Agency Policy and Practices           |
+------------------------------+---------------------------------------+
| **Day 2**                    |                                       |
+------------------------------+---------------------------------------+
| **Morning**                  | Basics of Contracting                 |
+------------------------------+---------------------------------------+
| **Afternoon**                | Pre-solicitation Phase                |
|                              |                                       |
|                              | Perform Source Selection              |
+------------------------------+---------------------------------------+
| **Day 3**                    |                                       |
+------------------------------+---------------------------------------+
| **Morning**                  | Perform Source Selection (cont'd.)    |
|                              |                                       |
|                              | Contract Management                   |
+------------------------------+---------------------------------------+
| **Afternoon**                | Contract Management (cont'd.)         |
|                              |                                       |
|                              | Performance-Based Agreements          |
+------------------------------+---------------------------------------+
| **Day 4**                    |                                       |
+------------------------------+---------------------------------------+
| **Morning**                  | Performance-Based Agreements          |
|                              | (cont'd.)                             |
|                              |                                       |
|                              | LCL Management, Product Support       |
+------------------------------+---------------------------------------+
| **Afternoon**                | Business Financial Planning and       |
|                              | Management                            |
|                              |                                       |
|                              | Course Review                         |
+------------------------------+---------------------------------------+

# Appendix B: Evaluation Comments

**Course Introduction**

Would be more effective if everyone would have been more informed of
what the class expectation was for their agency

Vision not clear

**Module 1 -- Leadership/Professional Skills**

Too elementary. Either have more content or spend less time

Help to have more on servant leadership

**Module 2 - Agency Policy and Practices**

Help to more formally id best, worst agency practice/policies i.e. not
just theory or anecdotes

**Module 3 - Basics of Contracting**

More on how each agency deals with this (basics of contracting)

VCF was a good example and working in teams was good

**Module 4 - Pre-Solicitation Phase**

More on how each agency deals with this (Pre-Solicitation Phase)

**Module 5 -- Perform Source Selection**

More on how each agency deals with this (Performance Source Selection)

**Module 6 - Contract Management**

Would like more content/time spent on this (Contract Management)

**Module 7 - Performance-Based Agreements**

Would like more content/time spent on this (Performance-Based
Agreements)

Good illumination, but still unclear on difference with cost-plus award
fee which typically includes measures

**Module 8 - Life Cycle Logistics (LCL) Management, Product Support and
Interoperability**

Put entire life cycle in view

**Module 9 - Business Financial Planning and Management**

Not Covered

Overall course requires improvement

**The objectives provided for each module were clear.**

Needs work

**The course materials were accurate.**

Needs improvement on the course materials

Materials were a disaster; not used, not readable, etc.

Notebook provided was skimpy

Not in sync with instructor manual

Didn't exist

Limited material

Non-existent

**The course materials helped me to understand the course content.**

The course material provided needs updating to include information in
more detail.

Instructor was better than course materials.

Missed the mark on this -- instructor guided us through.

Proctors books/experiences were good.

Yes, however very limited; facilitator had excellent personal library.

Mike was great, materials not so much.

**The activities/questions reinforced the learning.**

Learning was a challenge - - value was contacts/partners.

**The audio-visual materials supported the learning.**

There were none.

Rarely used.

**The content covered in the course is relevant to my job.**

Yes, while I picked up new things, covering material I know in a
different perspective was good.

**Additional comments**

More structure of book/content.

Better time management

> Vision declaration day/would have been effective. Course content needs
> bolstering. Framework of delivery needs to be detailed.
>
> Good training module. However, content needs to be worked on. The
> facilitator 'made' the difference - - excellent presentation. I will
> recommend this training to my division.

Lots of material - - intense. Needs to add topics on PM methodologies.

##