# **MS Powerpoint Presentation: Section J, Evaluation**

**FY 2009 Technical Assistance Workshop**

**Graduate Assistance in Areas of National Need Program (GAANN)**

# 

# SLIDE 1 of 18

# GAANN Technical Assistance Workshop Section j. Evaluation

## Tim Topoleski

## UMBC Presidential Teaching Professor 2008-2011

## Professor and Graduate Program Director

## Department of Mechanical Engineering

## University of Maryland, Baltimore County

## 30 January 2009

\[two graphics on bottom left and right of page: UMBC an Honors
University in Maryland\]

# SLIDE 2

# Outline

## I. Introduction to the Theme

## II. The Requirements (GAANN) - seven items

## III. Examples from the proposal

## IV. Summary

##  

# SLIDE 3

# The Theme Two Thoughts: 1. Stick to the Script

## 2. Use what you Have / Do what you do.

# SLIDE 4 The Requirements 1. Seven points required by the Department of Education

## 2. No Creativity! (Stick to the Script)

# SLIDE 5 A Look at the "Magnificent Seven" What does the DOE want????

\(j\) Evaluation plan (15 points). The Secretary reviews each
application to determine the quality of the evaluation plan for the
project, including the extent to which the applicant\'s methods of
evaluation\--

# SLIDE 6

## ...including the extent to which the applicant\'s methods of evaluation\--

## 

## Relate to the specific goals and measurable objectives of the project;

## (2) Assess the effect of the project on the students receiving fellowships under this program, including the effect on persons of different racial and ethnic backgrounds, genders, and ages, and on persons with disabilities who are served by the project;

# 

# SLIDE 7

## (3) List both process and product evaluation questions for each project activity and outcome, including those of the management plan;

## (4) Describe both the process and product evaluation measures for each project activity and outcome;

## (5) Describe the data collection procedures, instruments, and schedules for effective data collection;

# SLIDE 8

## (6) Describe how the applicant will analyze and report the data so that it can make adjustments and improvements on a regular basis; and

## (7) Include a timeline chart that relates key evaluation processes and benchmarks to other project component processes and benchmarks.

# SLIDE 9

## Stick to the Script. You must address each of these items!

# 

## SLIDE 10

## (1) Relate to the specific goals and measurable objectives of the project;

## 

The GAANN Committee will use several methods to monitor the progress of
the fellows and to evaluate whether the specific goal of the project is
being met.

An evaluation at the end of each semester will review the fellows
academic performance, progress toward their Ph.D. degree, their
teaching, and overall productivity.

Results of the evaluations will be provided to the students and used for
program modification.

The evaluation process will also allow the Department to determine its
success in increasing the annual output of domestic Ph.D. students in
Mechanical Engineering to meet regional and national needs.

# SLIDE 11

\(2\) Assess the effect of the project on the students...including
students of different backgrounds...;

## The effect of this program on student's performance will be evaluated by: 

## a questionnaire

## student and advisor evaluations

## faculty who have contact with the students, 

## the GAANN committee as a whole. 

## 

## Additional input: 

## 

## UMBC Graduate School's peer-mentoring program *(have/do)*

## 

a member of the GAANN Committee and WISE group will also serve as a
critical point of contact for women students to solicit input on both
their academic progress and their comfort in the program *(have/do).*

## 

# SLIDE 12

\(3\) List both process and product evaluation questions for each
project activity and outcome, including those of the management plan;

A Table: (two examples from a list of 11 -- what do you have/do??)

## 

+---------------+-----------------+------------------+----------------+
| **Project     | **Evaluation    | **Measurement    | **Outcome**    |
| Activity**    | Questions**     | Tools**          |                |
|               |                 |                  |                |
|               |                 | **(*data to be   |                |
|               |                 | collected)***    |                |
+---------------+-----------------+------------------+----------------+
| 2\. Ensure    | Is the GPA,     | GPA and GRE      | Students       |
| the students  | GRE, relevant   | records,         | enrolled are   |
| are           | practical       | employment       | academically   |
| academically  | experience, and | history, letters | on par with    |
| talented      | other           | of               | current        |
|               | indicators for  | recommendations, | students.      |
|               | new Fellows on  | interview        |                |
|               | par with        | results          | [Data          |
|               | current         |                  | ]{.underline}: |
|               | students?       |                  | Student GPA's  |
+---------------+-----------------+------------------+----------------+
| 4\. Ensure    | Do the fellows  | Data from        | Student meet   |
| that those    | meet the        | standard         | federal        |
| recruited as  | requirements    | financial aid    | criteria for   |
| GAANN fellows | for federal     | forms collected  | economic need. |
| have          | need-based      | by the Office of |                |
| financial     | programs?       | Financial Aid.   | [Data          |
| need.         |                 |                  | ]{.underline}: |
|               |                 |                  | Financial      |
|               |                 |                  | needs          |
|               |                 |                  | analysis.      |
+---------------+-----------------+------------------+----------------+

## 

## SLIDE 13

\(4\) Describe both the process and product evaluation measures for each
project activity and outcome;

This is already included in the Table!

\*\* Note that many of the processes used to address points 3 and 4 are
things we already have/do, either in the Department, or in the Graduate
School, from [ABET]{.underline} (Accreditation Board for Engineering and
Technology) or [Middle States Accreditation]{.underline}.

SLIDE 14

\(5\) Describe the data collection procedures, instruments, and
schedules for effective data collection;

Column Three in the Table above describes the data collection
procedures... and Column Four briefly describes the types of data to be
gathered.

The Department of Mechanical Engineering and UMBC have extensive
database systems for keeping track of grades, research or teaching
appointments, fellowships, advancement to candidacy, progress toward
degree, and graduation dates.

Finally, the project director will conduct personal interviews with each
of the GAANN fellows upon completion of their program to determine their
overall view of the program, and their post-graduation positions.

# SLIDE 15

\(6\) Describe how the applicant will analyze and report the data so
that it can make adjustments and improvements on a regular basis;

Data on enrolled students will be collected and updated each semester by
the Department's Graduate Program Director.

The data will be used to monitor student progress, identify and address
potential student and program problems.

If a GAANN Fellow is not fulfilling their requirements, has not made
progress toward their degree milestones, the Committee will meet with
the GAANN Fellow and/or their Academic Advisors to determine a plan of
action.

## 

## *Much of this is have/do*

# SLIDE 16

\(7\) Include a timeline chart that relates key evaluation processes and
benchmarks to other project component processes and benchmarks.

## Some timeline highlights:

  --------------------- -------------------------------------------------
  Mid-August            Graduate Student Orientations

  Mid-November          Graduate Horizons (minority student recruiting
                        event)

  Early-December        Teaching course evaluation questionnaire
                        administered

  Mid-January           Fellows' semester's work evaluated

  Mid-February          Faculty observe GAANN fellows in class

  Late-February         Applications for admission reviewed for new GAANN
                        awards

  Late-February         GAANN Committee meets with fellows to discuss
                        progress
  --------------------- -------------------------------------------------

## 

SLIDE 17\
SUMMARY:\
1. Stick to the Script

On the Outside (1-7). (Don't risk losing points)

2\. Use what you Have / Do what you Do. (Creativity on the Inside --
Students' program)

SLIDE 18\
Thank You

\[two graphics on bottom left and right of page: UMBC, an Honors
University in Maryland\]

\###