# Presentation: 551502

## Session 44: Verification Analysis: Understanding the Reports Generated by the ISIR Analysis Tool 

- David Rhodes, AIR
- Holly Langer-Evans, FSA

## Goals for this session

- Learn how to read the reports produced by the ISIR Analysis Tool
- Learn how to structure the analysis of your own data

## Overview of this session

- Placing analysis in context
- Understanding ISIR Analysis Tool reports
- Structuring your own analysis

## Placing Analysis in Context

## Why Analyze?

- Disprove theories and advance science
- Understand the past
- Predict the future
- Improve

## ISIR Analysis Tool Seeks to Improve

- Verification
- FAFSA instructions and guidance
- Accuracy of need-based aid awards

## Flowchart of Analysis When Seeking Improvement

**U**nderstand,
- sufficient?

- Yes

- No

**F**ormulate
- Improvements

**F**ormulate
- Questions

**D**o data Analysis

**A**pply Improvements

## UFFDA!

- Understand
- Formulate improvements
- Formulate questions
- Do data analysis
- Apply improvements

## Avoid uttering “Uffda” 

- Keep the goal of improving verification, FAFSA instruction, and accuracy of aid awards in mind
- Focus analysis on questions that enhance *useful *understanding
- Avoid perpetual analysis

## Understanding ISIR Analysis Tool Reports

## ISIR Analysis Tool Reports 

- Overviews
- Sample Summary Report
- Verification Summary Report
- Comparisons of changes in ISIR information
- Field Change Report
- Field Increment Report
- Pell Eligibility by Dependency Report
- Information on individual students
- Student Listing Report
- Student Detail Report

## Overviews

- Sample Summary
- What students are in my database?
- What was the aggregate impact of change on eligibility?
- Verification Summary
- Which records were selected for CPS verification?
- Which records were selected for discretionary verification?
- Which verified records experienced an increase or decrease to EFC?
- Which non-verified records experienced an increase or decrease to EFC?

## Overviews – Example Dashboard Results

## Field Change Report

**Which ISIR fields were most likely to be corrected?**
**Which corrections were most likely to be associated with a change to aid eligibility?**
**Which corrections were most likely to be selected by school and CPS verification criteria?**

## ExampleField Change Report

## Field Increment Report

- Are changes to a particular field concentrated within a particular value range of that or some other field?
- Are corrections to a particular field that matter concentrated within a particular value range of that or some other field?

## Example Field Increment Report

## Pell Eligibility by Dependency Report

- Are changes to the selected field related to Pell eligibility and dependency status?
- What percentage of students with changes to the selected field are verified by CPS or discretionary criteria?

## Example Pell Eligibility by Dependency Report

## Student Listing Report

- Which records experienced a change to aid eligibility and were NOT selected by either the CPS or school (discretionary) verification?
- Which records were selected for school (discretionary) verification and did not experience a change in aid eligibility?

## Example: Student Listing Report

## Student Detail Report

- What was the impact on this student's EFC?
- What ISIR information changed after the initial application?
- What was the primary reason for observed changes to EFC?

## Example Student Detail Report

## Ad-Hoc Reports

*WebFOCUS Report Assistant* allows users to create custom reports
- Answers other questions users may have

## Example Ad Hoc Report

## Structuring Your Own Analysis

## Important Questions to Answer BEFORE Structuring Your Analysis

- What ISIR information does your school verify?
- Which ISIR records have confirmed information?
- Analyze corrections NOT changes

## Structuring your own analysis

- Step #1: Define goal
- Step #2: Determine relevant records
- Step #3: Decide which reports to run
- Step #4: Run those reports
- Step #5: Apply improvements

## Step #1: What is the goal of your analysis?

- What do you want to improve?
- What is in your power to change?
- You may use the Tool to help identify possible areas for improvement, but meaningful analysis cannot begin until you identify an area for improvement

## Possible improvement-based goals

- Identify the most problematic ISIR fields to address with educational outreach efforts
- Target federal verification at the 30 percent of applicants most likely to experience an increase in EFC (loss of eligibility)
- Reduce the scope of discretionary verification to exclude records that do not experience a change in eligibility
- Expand the scope of discretionary verification to include records that experience changes in eligibility

## Step #2: Determine “relevant” records 

- Not all records may be relevant to your improvement
- Ways to focus reports
- Before uploading ISIR records into the Tool
- Filters
- Drill down

## Step #3: Decide which reports to run

- What do you need to know?
- What questions do you need to ask?
- Select the reports that provide answers

## Step #4: Run Those Reports

- Review results
- Run additional reports ONLY if understanding is not sufficient
- Keep analysis as simple and focused as possible

## Step #5: Apply Improvements 

- Make improvements in light of results
- Implement during 2005-06 or 2006-07 award year
- Monitor impact of improvement with future versions of the ISIR Analysis Tool

## Analytical Recipes

- Improve educational outreach
- Expand discretionary verification
- Reduce discretionary verification

## More about the ISIR Analysis Tool

- Session #2: Hands On Session
- PC Lab
- Web-based training on using the Ad Hoc Report  will be offered this Spring

## Questions?

## Technical Assistance

- We would appreciate your feedback and comments on this session and are available to answer questions you may have in the future.  We can be reached at:
- Email: _[drhodes@air.org](mailto:drhodes@air.org)_
- 	_[Holly.Langer-Evans@ed.gov](mailto:Holly.Langer-Evans@ed.gov)_