# Presentation: 113031

## <date/time>

**Notes:**
Opening Slide....No comments here.

## Session  14

**Tools For Schools – FSA Assessments and the Quality Analysis Tool Software**

- Session 21-

**Notes:**
Welcome to The FSA Assessments and the Quality Analysis Tool Session.

My name is Anne Tuccillo.  I work with the Performance & Accountability Improvement area of FSA.

With me today is Mike Cagle, my co-worker.

We are glad you are here today, and hope you go away with some new ideas!

## Tools For Schools

- 	This session provides an overview of the redesigned tools developed by FSA to help schools improve the delivery of student aid and customer service.  We will discuss the FSA Assessments and the Quality Analysis Tool (verification tool) available for all schools.

- Session 21-

**Notes:**
Read Slide.

## Session Goals

- Introduce FSA's Tools for Success
- Review the Quality Analysis Tool
- Review the FSA Title IV Assessment Tool

- Session 21-

## Quality Analysis Tool (Verification Tool)

- Part of the EDE Suite of Software
- Available to all schools in ‘02-’03
- Analyze FAFSA application/ISIR data
- Generate reports showing effect of data element changes on EFC/Pell Grant eligibility
- Improve/customize verification program
- Data to suggest future changes in federal verification selection criteria

- Session 21-

**Notes:**
Have you ever wondered how effectively verification edits target students in your applicant population?  The Quality Analysis Tool can support your verification efforts by telling you who your verification procedures is succeeding with – and who you are needlessly pestering.

Revised and available for any school to use in 2002-03, the tool will help schools analyze information reported on the FAFSA to determine the impact of changes on the EFC and Pell Grant eligibility.

Users will import ISIR records into the tool.  Through the use of queries and reports, users can obtain data about aid applicants at their school.  This data can be used to make changes to a school’s own institutional verification program.  Also, the Department can learn from what schools have discovered and make changes to the federal verification selection criteria.

## Locally Designed Verification

- Focuses on local sources of error
- Identifies changes to FAFSA fields that have an impact on awards
*What are the benefits of school-designed verification procedures?*

- Session 21-

**Notes:**
At a given school, this verification tool enables you to identify practices that have an impact on aid awards.  For common errors, students and parents can be advised and educated.

On a more practical level, when you are dealing with the impact the changes on applications have on EFC, awards can be adjusted so that deserving students who are missed can be helped.  At the same time, knowing the real impact of changes, a school will not have to burden students and families with unnecessary follow-up.  Schools will be able to see if the CPS edits are selecting the students at their school that are most likely to misreport or change applicant information.

FSA wants schools to be successful.  The Quality Analysis Tool can help schools refine verification procedures to permit schools to redirect resources from pushing paper to handling activities that make a difference.

## The Power of Data

- The Quality Analysis Tool allows schools to conduct in-depth analysis of applicant data
- A series of reports can be constructed using custom formats, selected fields and queries to extract specific information about your applicant population

- Session 21-

**Notes:**
In a few minutes you will learn more about the importance of the data generated by this verification tool.  If your office has not had the capability to conduct in depth analysis of your verification outcomes, then this tool can help.  If your school does have the resources to run complex reports and conduct analysis, then this tool can also help verify your information.

Using this product, schools will be able to conduct analysis using a series of reports to determine how well its verification procedures are working.  Schools should collect information capable of addressing the question they want answered by their analysis.  For example, do dependent students with family incomes below $25,000 have significant changes to applicant data, and if so, do the changes impact EFC and Pell Eligibility?  Using this approach, a school can stop spinning its wheels and develop verification practices that fit its particular population.

## Software Components

- Import
- Tools
- Export
- Process
- Entry
- Print

- Session 21-

**Notes:**
Similar in design to the rest of  the EDE Suite of software products, the software consists of many components.  Several listed are worth mentioning.

Import – This component is used to bring ISIR records into a the tool.  Schools can import records from an EDExpress database, or a flat ascii file.

Tools – This component provides functions for the users to set up their system, design verification edit profiles, design custom formats, manage user security and file management.
Export – This component enables users to export data from the tool to another software application such as SAS or SPSS
Process – This component enables users to enter data on multiple records at one time, and to validate records after importing.
Entry – This component enables a user to enter data  on a record by record basis.  Currently, only one field is available for manual entry, the institutional verification flag.  Users can update records manually, or by using the multiple entry function under the process menu.

Print – This component really is the meat of the tool.  Using a series of queries, custom formats, income ranges, efc ranges and field increments users can generate reports that provide information about  verification practices at their school.  Schools will use the data from the reports to analyze changes that occur to ISIR information during normal application, packaging, verifying, and awarding processes.

## Verification Profiles	

- Allow schools to setup verification profiles
- Queries: complex or simple
- Designed to let schools know who they are selecting, who they are missing, and who they are “pestering.”

- Session 21-

**Notes:**
A new feature added this year to the tool is the verification edit profiles.  Basically, the profiles are developed by constructing queries, either very simple or very complex.  A query is a request for certain records in a database that meet a specific condition or set of conditions.

A school would develop queries that are representative of its own verification selection criteria.  These criteria would then be applied to records in the database.  Those records that met the condition, would be flagged.  Using several new reports a school could learn what records met the criteria, what records did not, as well as the number of records selected by the CPS but not by the school.  From this information you can learn about who you are targeting for verification, who may be falling through the cracks, and who is being verified, when verification will not result in a change in outcome.

## Print

- Report Types
- 9 “analysis” reports
- Custom Report
- Query Management
- 3 Record Layout reports
- System Setup
- User Database

- Session 21-

**Notes:**
The Tool provides 9 analysis reports as well as a custom report feature, query management capabilities, record layout reports, as well as system set up and user database information.
I encourage you to come to the PC lab to take a look at the tool and the report features, as they are too numerous to go through in this short of a presentation.  I would however like to briefly describe the 3 new verification reports that were added to the 2002-3 version.
List Verification Flags by Student – This report provides a side by side comparison of CPS and institutional verification practices, e.g., what records were flagged by CPS, the institution, both, or not at all.
List Verification Selection Criteria by Student – This reports indicates the reasons each individual verified student was verified; permits each individual verification edit profile to be evaluated.

List Verification Selection Criteria by Code – This report lists student to whom a specific verification profile applies.  You can analyze the contribution that each verification profile has on the overall verification process at your school.
As I mentioned earlier, the tool can support your schools efforts by telling you who your verification processes is succeeding with and who you are needlessly pestering.

## Quality Analysis Tool

- ALL analysis is simply structured comparison
- The QA Tool provides a variety of frameworks (reports) to make comparisons
- The QA Tool allows the user a great deal of flexibility in tailoring these comparisons to suit their institution’s needs

- Session 21-

**Notes:**
No matter the how crucial or trivial the topic or how brilliant or dim the analyst, analysis is simply structured comparison.  The QA tool uses reports to present comparisons.  Complete flexibility is offered users in terms of  the ISIR data elements they choose to examine and which subsets of cases they choose to subject to examination. 

## Purpose of Comparing

- Descriptions of applicants who experienced _changes_ to EFC but were “_missed_” by verification
- Descriptions of applicants who had _no changes_ to EFC but were “_pestered_” by verification
- Descriptions of applicants who_ self-correct_ information outside the verification process

- Session 21-

**Notes:**
The QA Program provides regulatory relief for participating institutions from following federally proscribed verification edits.  Therefore, the QA Tool was developed to help QA participants monitor and improve their own verification programs.  Verification has twin evils: missing and pestering.  The QA Tool helps schools identify what income bands or other sub-populations are over-represented in the two problematic populations.  The tool is available to all schools and serves the same purpose of monitoring the effectiveness of verification – CPS edits – on any Title IV campus.  By using the QA Tools, schools can identify problematic cases in terms of their verification program, including the Federally proscribed edits. Changes to FAFSA information occurring independently of verification can also be examined.  It might not make sense to expend limited verification resources on cases who are likely to self-correct. 

## Who Benefits From Making Comparisons

- QA Program participants – fine tune effective institutional verification programs
- Non-QA Program institutions – evaluate the effectiveness of CPS edits on their campus
- FSA – fine tune CPS verification and evaluation of FAFSA

- Session 21-

**Notes:**
QA Tool designed to monitor effectiveness of any verification system on a given campus.  Everyone benefits.  

Any school can use this verification tool to evaluate their use of CPS edits.  Identify the types of students who are flagged by CPS yet experience no change in award.

FSA can use data to evaluate CPS verification flag present on ISIR data.   

## What We’ve Learned So Far

- Improve FAFSA instructions for and/or education efforts regarding estimating income and tax information
- Incorporate tax “estimators” into verification profiles
- Use - but don’t overly rely on student/family income level when deciding who to verify
- Consider collecting information on the cost of verification.

- Session 21-

**Notes:**
Read bullets.  
QA Tool data collected from QA schools who pilot tested the product last year played a role in shaping verification selection criteria for the 2002-03 award year.

Findings from data analysis performed on data submitted by QA schools was presented to FSA senior management as they worked on fine tuning the current verification selection criteria.  Program integrity is of the utmost importance, and we believe that this verification tool will play an important role at the school level for ensuring that aid is awarded accurately and to eligible students.  FSA wants to work in partnership with you to continue to strengthen the Title IV programs.  We hope that you will use the QA Tool evaluate the effectiveness of your own verification processes an d procedures.

At this point I am going to turn it over to Mike so he can talk about another Tool for School to use to strengthen program integrity and management of Title IV programs at the campus level.

## What are the FSA Self Assessments?

-    Tools to prevent and/or identify compliance issues, implement Management Enhancements for noted problems, and share Effective Practices

- Session 21-

**Notes:**
Read Slide

We will also show you the assessments available on the website and provide instructions regarding how to use the assessments.

## What are the FSA Assessments?

- Originally for Quality Assurance schools’ self-evaluation of Title IV operations
- Series of questions/comments outlining major requirements/policies needed to properly administer the Title IV programs (checklist, exercises, description boxes – who, what, when, where, why, how)
- Updated with hyperlinks to _Reg’s_ and the _Student Financial Aid Handbook_
- Addition of new, interactive assessments
- Now available for ALL schools

- Session 21-

**Notes:**
Read slide.

Emphasize the use for ALL schools.

Also, the Assessments are included in the program review guide so that program reviewers can also encourage schools to use the assessments.

Even if your school thinks they are in compliance, we have found that schools who use the assessments find improvements that can be made.  Remember, continuous improvement should be your goal!  A school must continuously evaluate procedures and fix any noted problems before these problems potentially lead to a large monetary liability for the school!

Another good time to use the assessments would be when your school changes staff.  You can pick different assessments to ensure things are working properly.

## What’s In It For Me?

- Assessments empower your school to:
- Self-awareness
- Implement Management Enhancements (Action Plans)
- Share Effective Practices
- Utilize Technical Assistance Resources
-  FREE!!!
- Help develop working relationship with ED
- Enlightenment

-  Can help avoid a program review

- Session 21-

**Notes:**
Read Slide.

## Mgt. Enhancements’ Benefits

**Allow you to take corrective action in order to resolve noted problems**
- Identify problem
- Required action
- Identify resolution team (offices, team leads...)
- Timeframe
- Reminder to update policy & procedure manual
**Link to the Management Enhancement Worksheet from each assessment**

- Session 21-

**Notes:**
Management Enhancements allow your school to develop a plan of action to “fix” any noted problems.

A Management Enhancement Form can be printed for easy use and follow-through.

A school should follow-up on management enhancements to ensure the corrective action is taken.

A direct link to a Management Enhancement Worksheet can be found throughout each assessment.

## What is the Effective Practice Database?

**A way to share practices that work well – “in running a school”**
**A continuously expanding record from which schools can benefit** (does not have to be Title IV related)
***To facilitate sharing:***** A direct link to share an idea in each assessment **
**Currently in testing mode (**Will be unveiled in the very near future)

- Session 21-

**Notes:**
We are very excited about the Effective Practice database.  

When FSA first became a PBO, many schools informed FSA that they would like to know the good things their colleagues are doing.  This allows schools to learn from each other.

This database allows schools to share an effective practice.  FSA will post these practices to the website so that schools can view them.

A direct link to an Effective Practice form is available in each assessment.

THE EFFECTIVE PRACTICE DATABASE IS CURRENTLY IN TEST MODE. WE PLAN TO ALLOW ALL SCHOOLS TO SHARE EFFECTIVE PRACTICES IN THE NEAR FUTURE.  WE WILL POST ANNOUNCEMENT TO IFAP, LIST SERVES, ETC WHEN THIS BECOMES AVAILABLE.. STAY TUNED...

## What are Technical Assistance Resources?

**A listing of all resources available to schools that need assistance**
- Contact information, phone numbers, email...
- Websites
- Search engines
- Technical/software support
***To facilitate use:***** Each assessment has a link to the Technical Assistance Resources for quick reference**

- Session 21-

**Notes:**
We have developed a quick reference for Technical Assistance Resources available for FSA services.  

Each assessment includes a direct link to the TA Resources for quick reference.  This resource list will be updated regularly!

## Where can I find the FSA Assessments?

- The assessments can be accessed at the following website:
***qaprogram.air.org***
***(“Tools for Schools”)***

- Session 21-

**Notes:**
Currently, the Assessments are available on the Quality Assurance Website.

There will also be links to these assessments from the SFA Portals and IFAP websites. 

REMEMBER, THESE ARE AVAILABLE FOR ALL SCHOOLS.  YOU DO NOT HAVE TO BE A QUALITY ASSURANCE SCHOOL TO ACCESS THE WEBSITE TO USE THE TOOLS!

## Tools For Schools

- Interactive assessments allow the school to complete self-evaluations online
- Non-Interactive assessments require download by schools
- All assessments provide hyperlinks to Regulations, Student Financial Aid Handbook
- All assessments have printing options

- Session 21-

**Notes:**
Remember, the website includes both interactive and non interactive assessments.

What is the difference between interactive and non interactive assessments?  Interactive assessments allow your school to complete on-line.

The non-interactive assessments require your school to download and complete “off-line.”  

Both types of assessments allow your school to view the assessment while on the web-site before downloading (for non interactive) or before completing on line (for interactive).  

## Interactive assessments

- If your school wants to complete the assessment online and save it, a password will be required.
**User Name = OPEID**
**Password = *****Create Your Own      *****If you create your own password, you will be asked to register, and provide your email address.**
**While online you will only have to enter this information once to move within other interactive assessments**
- If completing online, can email assessment through internet browser (file/send/page by email) and can print assessment with completed answers
- Schools can view the assessments in the tour mode without a password (unable to complete online).
- If you do not want to complete the assessment online, you may download the assessment.  Save it to your hard drive and complete it later.

- Session 21-

**Notes:**
Schools can view the assessments in the “tour mode” without having to enter a password.

If the school would like to complete the assessment online and save it, a password will be required.  You will be asked to enter a USER Name which will be your OPEID.  You will also be asked to enter a PASSWORD.  You may Create your own password.   If you create your own password, you will be asked to register and provide your email address. Why do we ask for an email address? Because more than one person from each school (OPE ID) may attempt to log in and create a password.  There will be only one password per institution (per OPE ID).  So, if a user from your school attempts to create a password under your school's OPE ID AFTER someone from your school has already created a password, the following error message will occur:  This OPE ID is registered to the individual below.  Please contact this person for the password to access the record.  The message will then provide the name, office, phone number and email address of the person who originally registered (as entered by this user).   ON THE NEXT SLIDE, YOU WILL SEE WHAT THE ERROR MESSAGE LOOKS LIKE.

If you do not want to complete the assessment online, you may download the assessment, save it to your hard drive and complete it later.

## Multiple User Error Message

- Session 21-

**Notes:**
Here is the error message that a user will receive if they attempt to log in under an OPE ID after someone else from the school has already created a password.

As you can see, the error message provides contact information for the user who originally created the password.

Therefore, it is extremely important for users to know that when they are prompted for a password, their contact information they enter is EXTREMELY important.

## Tools For Schools

- Session 21-

**Notes:**
The assessments page is your toolbox to help you with compliance.
What is a toolbox generally used for?  To store tools to “fix” things!! We like to call the assessments “Tools for schools.”
In the past, this use to be a paper product.  The assessment was one large document.  We have gone through a rather remarkable program redesign.  Things change, and the SFA Assessments have evolved and caught up with the electronic age!

The FSA Assessments are categorized in Four different areas:

Students
Schools
Managing Funds
Campus Needs

There are various assessments/modules within each category.

## Students

**** Interactive***
***Assessment***

- Session 21-

**Notes:**
Let’s Take a look at the modules in the “STUDENT” category.

PRESENTER:  Pull up the website and click on “Students.”.

These modules include Student Eligibility and Awarding Aid (non interactive) and SAP (interactive)

Click on one of the non-interactive assessments and show some of the features.  It may also be good to pull up the SAP assessment to show the features.

## Schools

**** Interactive***
***Assessment***

- Session 21-

**Notes:**
Now let’s take a look at the assessments under the SCHOOLS section.

PRESENTER:  Pull up the website and click on “Schools.”.
These assessments include:

Institutional Eligibility (non-interactive)
Consumer Information (interactive)
Recertification (interactive)

Pull up the recertification assessment and highlight some of the features.

## Managing Funds

**** Interactive***
***Assessment***

- Session 21-

**Notes:**
Now let’s take a look at the assessments under the MANAGING FUNDS section.

PRESENTER:  Pull up the website and click on “Managing Funds.”.
These assessments include:

General Fiscal Matters (non-interactive)
Disbursing Aid (non-interactive)
Reporting and Reconciling (non-interactive)
Perkins (four non-interactive assessments)
Financial Responsibility and Cash Management (interactive)
Return of Title IV Funds (interactive)

Pull up the Financial Responsibility and Cash Management assessment.  Highlight the features ESPECIALLY THE ACTIVITY...

## Campus Needs

- Session 21-

**Notes:**
Now let’s take a look at the assessments under the CAMPUS NEEDS section.

PRESENTER:  Pull up the website and click on “Campus Needs.”.
These assessments include:

Automation (non interactive)
Administrative Practices (non interactive)

Pull up the Administrative Practices assessment.  Highlight the features.

## Printing the Assessments

- All FSA Assessments can be downloaded.

- Session 21-

**Notes:**
All FSA Assessments can be downloaded which allows the user to download, print, and potentially complete the assessments within the ORIGINAL WORD DOCUMENTS.

## Printing the Assessments

**The “Print this Module” button at the bottom of each page of the Interactive Assessments will allow the user to print while the user has the assessment on the screen.**
**Both Interactive and Non-interactive assessments will have printable Word documents**
**All assessments have the similar printing features.**

- Session 21-

**Notes:**
If the user completes the assessment on-line (interactive assessments only), they can log in using the OPE ID, and a password, and then the work can be saved for future follow-up, accessible under the school’s OPEID #.
A user will also have the option to:
Print your work to date
E-mail your work to a colleague using Internet Explorer
E-mail your work to a colleague using Netscape
The “Print this Module” feature in the interactive assessments is really designed for users who are logged in and have entered data.  A user in “Tour Mode” may still be able to use this feature, but if they have answered any questions on-line, the answers may not print in the proper format.  I would NOT recommend using this feature unless the user is logged in.  If the user is in “Tour Mode”, it would be best to use the DOWNLOAD THIS ASSESSMENT TO YOUR COMPUTER button and download and print the Original Word Document.
The interactive and non-interactive assessments will have printable Word documents.  

Really, the main difference between the interactive and the non-interactive assessments printing features is that the interactive assessments have one additional printing feature.  You can print those assessments with your answers using the PRINT THIS MODULE button.  You can also save the information you enter, and forward the entered information to another user (if you are logged in).

The non Interactive assessments do not have the PRINT THIS MODULE button and you cannot forward information to another user.  The only choice for printing the non-interactive modules is by using the DOWNLOAD THIS ASSESSMENT TO YOUR COMPUTER button which will allow the user to print and/or save the original Word Document.

## Future Assessments

- The assessments will be updated annually
- More web-based assessments will be added
- Constant feedback needed from schools

- Session 21-

**Notes:**
What’s on the Horizon for future Assessments?

The current assessments will be updated annually.

We anticipate adding additional interactive assessments each year.

We are relying on feedback from the users to tell us how the assessments work for them, suggested improvements, and other assessments they would like to see added in the future.

WE ENCOURAGE YOU TO USE THE ASSESSMENTS REGULARLY AND PROVIDE US WITH YOUR COMMENTS AND FEEDBACK.

Remember, the new FSA Assessments are on our website:  QA Program@air.org.  And they may be accessed via IFAP.  Please take time to peruse through the assessments and provide us with your feedback.   We’d love to hear from you.

## In Summary

**The more you put into the assessments, the more you will get out of them**
**Don’t assume that everything you do is in compliance**
**Be willing to change**
**Administering the Title IV programs is an Institutional-Wide Effort**

- Session 21-

**Notes:**
These assessments contain much valuable information.  Like anything else you deal with, you must remember that the more you put into the assessments, the more you will get out of them.

Many schools go along day to day with the belief that everything they do is within compliance.  With these tools, you can test your belief and make improvements.  Remember, there is always room for improvement.  It is our hope that your school will take the time to test your school to ensure that you are in compliance.

You must be willing to change.  Change is never easy....
Administering the programs is an Institutional Wide Effort...Not just a Financial Aid Office Responsibility.
We would like to close with a story.

One day a young man visits his mother for a holiday dinner.  He notices his mother is cooking a ham and before she places the ham in the oven, she cuts off both ends of the ham.  The young man asks his mother “why do you cut off both ends of the ham before placing it in the oven?”  The mother replies:  “Ask your grandmother, that is how she always has done it.”  Later that day, the young man asks his grandmother why she cuts both ends off the ham.  The grandmother replies:  “that is the way my sister always did it, so ask Aunt Agnes.”  The young man forgets about it and a few weeks later he sees his Aunt Agnes.  He tell her about observing both his mother and Grandmother cutting off both sides of the ham before placing it in the oven.  
Aunt Agnes simply replies:  “That is the only way I could fit the Ham into the Baking Dish.”
The morale of the story:  Never assume business as usual.  If your school does things solely because the process is the way it is always done, dare to make a change.  There might be a better way! You might be missing the best part of the ham. We hope the assessments and the Quality Analysis Tool  will guide you through this process!

## Questions?

- We appreciate your feedback and comments via Email at:
***Anne.Tuccillo@ed.gov***
***Michael.Cagle@ed.gov***

- Session 21-

**Notes:**
WE ARE RELYING ON YOUR FEEDBACK.

We will be happy to answer any questions.

Thanks for your time today!