# Presentation: 114559

**REASoN’s                                                         [**_**R**_**esearch, **_**E**_**ducation, and **_**A**_**pplications **_**So**_**lutions **_**N**_**etwork]**** **
**Metrics Planning And Reporting Working Group Education Subcommittee*** *
**November 15, 2006**
**Glen Schuster**
**U.S. Satellite Laboratory, Inc.**
** **
** **
** **

**MPRWG Subcommittee Charge:**
**To Design, Propose and Implement an Efficient        “new generation of metrics”*** *
*FOCUS:  *
*Customer Service *
*Improved Reporting*
***for the benefit of Education REASoN Projects and NASA Education Management in SMD***** **
**  ****.**

## NASA ESDWG Meeting   November 13-15, 2006

**The Survey  ...*****for REASoN Projects***
***What is it?***
**     **Bank of Questions used to derive Stakeholder feedback
- 		   -  *validated by Office of Management and Budget*
*	**      **-  to be used by PIs, project evaluators*
- 	      *-  multiple choice, short-answer, open-ended*
***What is involved?***
**     ****Design, Development, Justification, Implementation **
***Who?***
**      **
**	**

**The Survey                                                       -- *****background***** 5 minutes                                         -- *****updating***** 10 minutes**

## NASA ESDWG Meeting   November 13-15, 2006

** ****SURVEY:  **_**Benefits for NASA Program Managers**_
**Allows overall evaluation and comparison of REASoN grants.**
**Organizes many levels of feedback.**
**Provides data to analyze effectiveness of individual project activities and products.**
**Provides data to meet NASA reporting requirements.**
**Question bank requires only *****one *****OMB review.**
**Common toolkit leads to cost savings. **

**Education Subcommittee Activities**

*The Internet will measure participation and customer service*

## NASA ESDWG Meeting   November 13-15, 2006

** ****SURVEY:  **_**Benefits for Project Development Teams**_
**Provides *****real-time***** feedback to project development staff from surveys of teachers, students and other stakeholders.**
**Reduces evaluation costs; comprehensive support for a fixed price.**
**Offers survey and question customizations.**
**Allows for immediate and/or repeated feedback.**
**Automatically compiles reports and graphs.**
**Facilitates stakeholder confidentiality.**
**Demonstrates program effectiveness and identifies areas to improve.**

**Education Subcommittee Activities**

## NASA ESDWG Meeting   November 13-15, 2006

***What is involved?***
-   **Design and Development**.                                                                             	   QUESTION BANK for TEN STAKEHOLDERS.
- 	              SURVEY QUESTIONS
-   FORMAL EDUCATION
- 	1)  Students   2)  Teachers   3)  Administrators   4)  Parents
-   INFORMAL EDUCATION
- 	5)  Families/Individuals   6)  Virtual Users    7)  Community Groups
-   HIGHER EDUCATION
- 	8)  Faculty   9)  Administrators  10)  Students

**The SURVEY**

## NASA ESDWG Meeting   November 13-15, 2006

-   **Development**. QUESTION BANK for TEN STAKEHOLDERS.
-  Survey available at any time for any purpose
-  Question bank: Many multiple choice questions rank 1-5
- 	(very unimportant to very important)
-  10-50 questions for each of the ten  stakeholders
-  Various topics

**The SURVEY**

## NASA ESDWG Meeting   November 13-15, 2006

**Development [and Use]**
*Who?*
**PHASE I PROPOSAL:  $8500 ( ÷ 6 Edu REASoNs)**
**Policy One and USM to work with Committee/PIs to make Question bank**
**PHASE II:**** **
**Creation and Use of Survey Generator by ****U.S. Satellite**** (planned)**
-   -  Projects generate online surveys for their stakeholders
-   -  Emails go out to participants/launch survey
-   -  Collect responses, data available, including pie charts, tables

**Education Subcommittee Activities**

## NASA ESDWG Meeting   November 13-15, 2006

**The SURVEY**

-   **Development**.  SURVEY QUESTIONS.
- Example:   K-12 / Teacher / TOPIC      Answer from 1-5
-  Teacher (TOPIC:  Content):
- 	*I am able to apply my knowledge of Earth data interpretation with my  students for the *_*classroom activities*_*.*
-  Teacher (TOPIC: Methods of inquiry):
- 	*Constructivist activities provided background for students to better utilize *_*Earth science data.*_
-  Teacher (TOPIC: Professional development):
- 	*I would be open to teacher training online.*

**PHASE I**

## NASA ESDWG Meeting   November 13-15, 2006

-   **Development**.  SURVEY QUESTIONS.
- Example:   K-12 / Administrators / TOPIC      Answer from 1-5
-  Administrator (TOPIC:  Relevance and Content):
- 	*Reform efforts and projects.....Product meeting goals for achievement.*
-  Adminstrator (TOPIC: Content):
- 	*Effectiveness of lessons.....Viable to be included in curriculum*_*.*_
-  Administrator (TOPIC: Skills):
- 	*Application of important process skills.....Effective us of computers/Internet*

**The SURVEY**

## NASA ESDWG Meeting   November 13-15, 2006

-   **Development**.  SURVEY QUESTIONS.
- Example:   Informal Ed / Virtual Users / TOPIC      Answer from 1-5
-  Virtual User (TOPIC:  Product Use):
- 	*Product user-friendly.....useful in personal or professional role....*
-  Virtual User (TOPIC: Information Dissemination):
- 	*Change the way you view topics (environment, climate systems, ...)...Level of understanding of the topic*

**The SURVEY**

## NASA ESDWG Meeting   November 13-15, 2006

**The SURVEY**

-   **Development**.  SURVEY QUESTIONS.
- Additional Topics
-  Teacher (TOPIC:  Background):
- 	*Preparation and foundation to implement project...education and content...Material lends itself to improving content, sharpening skills*
-  Teacher (TOPIC: Instructional Practices):
- 	*Equipped with materials such as computers, probes...Use of dialogue to provoke contradiction to hypothesis.....Make relationships to content and real-life applications, etc...Change in practice—using Earth data*
-  Teacher (TOPIC: Best Practices):
- 	 *Education material contribute to:*
*     **Meeting a learning standard.....Developing a hands-on practice*
*     **Integrate science with reading......Integrate science with math*
*     **Teach data gathering....Connect to students’ prior learning*

## NASA ESDWG Meeting   November 13-15, 2006

-   **Justification**.  OMB REQUIREMENTS
-  Privacy Act –
-    Paper Reduction Act...
-  We submit paperwork to NASA, to submit to OMB

**The SURVEY**

## NASA ESDWG Meeting   November 13-15, 2006

-   **Challenges**.
- Survey developer’s experience is not Earth science
- Contributors way is “the way”
- Making the survey truly useable by all
- Simplicity !
- Support by NASA in development Phase II

**The SURVEY**

## NASA ESDWG Meeting   November 13-15, 2006

**Release of Survey to SMD community **
**	****	****January 1, 2007**
**     ***** Question bank available for review **
**     ***** Solicit feedback / propose additional questions...3-5 weeks**
**	****   *****  Changes made by February 2007**
**	****   *****  Submit to OMB in March 2007 **
**	****   *****  6-7 Weeks for Tool Development**
***    *******   Working for Years 4-5...***
***    *******   Model for other REASoNs***

**Discussion**

## NASA ESDWG Meeting   November 13-15, 2006

- _[gschuster@us-satellite.net](mailto:gschuster@us-satellite.net)_
*(914) 921-5920   x201*
*32 Elm Place  1**st** Floor*
*Rye, NY  10580*

** **

## Slide 17

## NASA Earth Science Education Team [PI’s/Leads] Meeting

** ****GOAL:  New Generation of Education METRICS****   **
-  **Address** NASA reporting requirements and efficiently provide _QUANTITATIVE_ and _QUALITATIVE_ data for REASoN projects and NASA SMD-Earth Education
-  **Serve** Education REASoN projects and ultimately NASA SMD Education
** ****Meeting NASA/PI Requirements:  **
**   ****What is needed?  *****Three Components***
-  Quantitative Metrics Requirements
-  Reporting Tool [NEEIS, Office of Technology] Re
-  Surveys in Education
** ****Scope of Work**
-  Teleconferences and Working Sessions, Meetings [3 + others]
-  Community Input and Buy-in for Future and Expanded Use by SMD
-  **PROPOSED WORK DISCUSSION -END**

**Overview**

## NASA Earth Science Education Team [PI’s/Leads] Meeting

** ****GOAL:  New Generation of Education METRICS   **
**	****	****ADDRESS NEEDS      SERVE NASA/PIs**
-  Quantitative Data Collection
- NASA Numbers on: the Users, the Stakeholders, the Customers
-   Qualitative Data Collection
- Events, Milestones, Accomplishments, Impact Metrics, Survey Data

**Overview**

## NASA Earth Science Education Team [PI’s/Leads] Meeting

** ****Meeting NASA/PI Requirements:  Three Components**
-  #1  Quantitative/Qualitative Data*
-   “Education #10”
-   #2  Qualitative Data
-    Reporting Tool
-   #3  Qualitative Data
-    Surveys
- 	    *possible overlap

**Overview**

## NASA Earth Science Education Team [PI’s/Leads] Meeting

** ****#1 Quantitative/Qualitative Data**
- “Education #10”  with input from P. Coble, M.Y. Wei
-  Specific Audience
-  Communication Method*
-  Existing Tools or New Tools [immersive, viewer, etc.]*
-  Type of Product
-  Presentations/Press/Publication*
-  Indicators of Success*

** **

## NASA Earth Science Education Team [PI’s/Leads] Meeting

** ****#1 Quantitative/Qualitative Data**
-  Specific Audience
-  K-12  -
-    **Teachers/Faculty****, Students, Administrators, Parents**
-  - Number of Teachers
-  - Which Subject/Discipline/Course Taught
-  - Which Grade Level
-  - Generalist/Specialist
-  - Demographics and/or Underrepresented Audiences
-  	   City, Rural, Special Education, Limited English Proficiency

** **

## NASA Earth Science Education Team [PI’s/Leads] Meeting

** ****#1 Quantitative/Qualitative Data**
-  Specific Audience
-  K-12  -
-    **Teachers/Faculty, ****Students****, Administrators, Parents**
-  - Number of Students
-  - Amount of time/days working with the Project
-  - Demographics and/or Underrepresented Audiences
-  	   City, Rural, Special Education, Limited English Proficiency

** **

## NASA Earth Science Education Team [PI’s/Leads] Meeting

** ****#1 Quantitative Data**
-  Specific Audience
-  K-12  -
-    **Teachers/Faculty, Students, ****Administrators, Parents**
-  - Numbers, based on interaction

** **

## NASA Earth Science Education Team [PI’s/Leads] Meeting

** ****#1 Quantitative Data**
-  Specific Audience
-  Post Secondary  -
-    **Faculty****, Students, Administrators**
-  - Numbers of Instructors, or Students, or Administrators
-  - Which Subject/Discipline/Course Taught
-  - Demographic Audience [HBCUs, HSIs, TCUs, OMIs, etc.]

** **

## NASA Earth Science Education Team [PI’s/Leads] Meeting

** ****#1 Quantitative Data**
-  Specific Audience
-  Post Secondary  -
-    **Faculty, ****Students****, Administrators**
-  - Numbers of Students
-  - Amount of time/days working with the Project
-  - Demographic Audience [HBCUs, HSIs, TCUs, OMIs, etc.]

** **

## NASA Earth Science Education Team [PI’s/Leads] Meeting

** ****#1 Quantitative Data**
-  Specific Audience
-  Informal Science  -
-    **Families/Individuals****, Virtual Users, Community Groups**
** ****- **Number of visitors
-  - Demographic and/or Underrepresented Audience [City, Rural, Special Ed., Limited English Proficiency, Other] including age breakdown

** **

## NASA Earth Science Education Team [PI’s/Leads] Meeting

** ****#1 Quantitative Data**
-  Specific Audience
-  Informal Science  -
-    **Families/Individuals, ****Virtual Users****, Community Groups**
** ****- **Educators or Public
-  - Numbers or Visitors
-  - Demographic and/or Underrepresented Audience [City, Rural, Special Ed., Limited English Proficiency, Other] including age breakdown

** **

## NASA Earth Science Education Team [PI’s/Leads] Meeting

** ****#1 Quantitative Data**
-  Specific Audience
-  Informal Science  -
-    **Families/Individuals, Virtual Users, ****Community Groups**
** ****- **Educators or Public
-  - Numbers or Visitors
-  - Demographic and/or Underrepresented Audience [City, Rural, Special Ed., Limited English Proficiency, Other] including age breakdown

** **

## NASA Earth Science Education Team [PI’s/Leads] Meeting

** ****#1 Quantitative/Qualitative Data**
**	**
-  Specific Audience
-  **Communication Method***
- 	   In-person     Active Internet
-      Passive Internet [web-site access]

** **

## NASA Earth Science Education Team [PI’s/Leads] Meeting

** ****#1 Quantitative/Qualitative Data**
-  Specific Audience
-  Communication Method*
-  **Existing Tools or New Tools**                        [immersive, viewer, etc.]*
-  **Type of Product**
- 	    Data      	Enhanced Data     Training (service)
- 	    Career Activities      Public Engagement
- 	    Educational Materials      Services/data access

** **

## NASA Earth Science Education Team [PI’s/Leads] Meeting

** ****#1 Quantitative Data**
-  Specific Audience
-  Communication Method*
-  Existing Tools or New Tools [immersive, viewer, etc.]*
-  Type of Product
-  **Presentations/Press/Publication***
** ****Indicators of Success***

**   **

## NASA Earth Science Education Team [PI’s/Leads] Meeting

** ****#1 Quantitative/Qualitative Data**
- Indicators of Success*
-   **Formal Education, Informal Education**

** **

## NASA Earth Science Education Team [PI’s/Leads] Meeting

** ****#1 Quantitative/Qualitative Data**
- Indicators of Success*
-   **Formal Education****, Informal Education**
-  Students
- 	 - Classroom or other artifacts
-    -  Pictures   *with NASA release forms*
-    -  Shared Work  *contests, online, portfolio*
*   *-  Other
-   Teachers/Faculty/Administrators
-    -  Evidence of lasting impact from Project/Product
- 	      Systemic change/Word-of-mouth/Survey results
- 	      More Earth science is received by stakeholders/Attitudes
-   Administrators
-    -  How do they support educator/How project demonstrates meeting the need?

** **

## NASA Earth Science Education Team [PI’s/Leads] Meeting

** ****#1 Quantitative/Qualitative Data**
- Indicators of Success*
-   **Formal Education, ****Informal Education**
-  Families/Virtual Users/Community Groups
- 	 - Level of engagement
-    -  Level of comfort [intuitive, frustration, ...]
-    -  Word-of-mouth communication
*   *-  Survey Results
-    -  More Earth science is received by stakeholders

** **

## NASA Earth Science Education Team [PI’s/Leads] Meeting

** ****#2 Qualitative Data**
**Reporting Tool**
***What?***
**     ****Highlight milestones, activities....**
**     ****90-day Highlights**
***How?***
**     ****NEEIS and/or Development of a simple tool**
***Why?***
**     ****Program management for NASA**
**	****   ****Marketing for Projects**
**	**

**Reporting Tool **