![](media/image1.jpeg){width="10.75in" height="1.125in"}

**Going Further**

*Quality Assurance and the ISIR Analysis Tool*

**Hands - On Session**

During this session we will work through the new School Verification
Analysis (Shared) Report. This report is designed to show what types of
students are the most likely to be verified with little consequence to
their aid eligibility and -- when using random sample data -- which
records not selected by current verification would experience a major
change if they were. We will follow the following work plan:

1.  Review the Ad hoc report writer Report Assistant

2.  Run the new School_Verfication_Analysis shared report

3.  Take a behind the scene tour of the new School_Verfication_Analysis
    report

4.  Analyze your own school's results

5.  Share results and discuss how to apply them

6.  If time permits -- get your feedback for improvement

This session assumes that all participants have uploaded sample data and
set their school verification flags.

Does any one need to apply for amnesty?

If any school has not loaded data we ask that they partner up with
someone else who has.

If a school has uploaded their data, but has not set the school
verification flags, we ask that they substitute the
CPS_Verification_Analysis report in all instructions referencing the
School_Verification_Analysis. The two reports are very similar to each
other. The only difference between the two reports is which verification
flag they use in the analysis.

Step 1: Review Report Assistant

**Action: Log into the Tool**

-   Log into the ISIR Analysis Tool by clicking on the ISIR Analysis
    Tool icon available on the desktop or launching Microsoft Explorer,
    going to favorites, and selecting ISIR Analysis Tool. These will
    link to the FAA Acess to CPS Online Home Page
    [[http://www.fafsa.ed.gov/FOTWWebApp/faa/faa.jsp]{.underline}](http://www.fafsa.ed.gov/FOTWWebApp/faa/faa.jsp)

-   From the menu bar -- Select **Analysis of Data.**

-   Select **continue.**

-   If the Security Alert box displays, select **yes.**

-   The Data Analysis page should display.

-   In the domain trean, click on the the box ![](media/image2.png) next
    to IA Tool folder.

**Action: Open Report Assistant**

To Access the Custom Report folder to create an Ad hoc report:

-   Click on the box ![](media/image3.png) next to **Reporting Objects
    folder** on the left toolbar.

-   Click on the ![](media/image4.png) next to **Custom Reports
    folder.**

-   Right click on **Ad Hoc.**

-   From the drop down menu, select **Report Assistant.**

-   Maximize the Report Assistant window.

    ![](media/image5.png){width="5.99375in"
    height="4.502083333333333in"}

## The Buttons

The seven buttons that run along the top to screen are available in all
tabs. Use them to:

-   **Save** -- save the current report settings

-   **Save as** -- save the current report settings under a new name

-   **Run** -- execute the report

-   **Run deferred** -- not currently supported

-   **Help** -- access the *off the shelf* Report Assistant online help
    documentation

-   **About** -- display technical information about the version of
    Report Assistant being used

-   **Quit** -- leave Report Assistant

## The Tabs

Use the four tabs below the buttons to:

-   **Field selection** -- select fields, define new fields, choose how
    to include fields in reports, and set display options

-   **Report headings** -- enter heading and footing text

-   **Selection criteria** -- limit the records included in the report

-   **Report options** -- select the format for report output (e.g.,
    HTML, Excel, etc.)

## The Field Selection Tab

Report Assistant opens in the **Field Selection** tab. This is
appropriate, as users will spend the vast majority of their time
authoring Ad hoc reports working under this tab. Let's just take a quick
guided tour. Feel free to "click along" on your own computer during this
"tour" or simply follow along with the instructor as we point out
various aspects of the Field Selection tab.

To select fields use the box labeled "**Available Fields**." Simply
highlight the field(s) you would like to include in your report. *Hint
-- to select multiple fields hold down the control key.*

You have choices in how you include the selected fields in a report.
Fields can be "Report Values" or be used to define "Column or Row
Groupings." See the annotated screen shot above. You place highlighted
fields in the desired box by clicking on the
![](media/image6.png){width="0.21875in" height="0.25in"} "yellow/teal,
plus/minus sign" icon above the desired box. If you want to group
records based on their value, place the selected field(s) in either
"**Sort by**" or "**Sort across**" windows. If you want the report to
reflect the values the individual or aggregated values of the selected
field in the box under **Sum and Print** area. You can add as many
fields as you want to the boxes.

Clicking **Sum** or **Print** above the report values box changes the
options available for displaying the values of select fields in this
box. Select **Print** when you want to the report to be a listing of all
individual student values for the selected fields and to select **Sum**
when want a report summarizing the values of multiple students in a
single statistic (e.g., average or sum). *Note that the default has
changed from Print to Sum.*

When you highlight a field in the report value box, you can make
formatting selections in the **Field Options** box. When Print is
selected the options are: Plain Field Value: Number of distinct values;
Percentage; and Total. When Sum is selected the options are: Plain Field
Value; Average Square; Average; Count; Count Distinct; Number of
distinct values; Show first in group; Show last in group; Maximum;
Minimum; Percentage; Count Percentage; Row Percentage; Sum; and Total.

Report Assistant has a function called **New Define Field** that allows
you to define new fields that reflect either the values of currently
available fields and/or the results of calculations. Once defined you
can use any newly defined field just as you would any available field,
(e.g. as an element in a report or selection criteria).

**Action: Define New Field**

To create a new field, click on **New Define Field** button.

-   Wait for the define Field Creator window to load.

-   In the **Field** area in the upper left-hand corner of the
    calculator window, enter a name for the new field. For this example
    enter **EFC_Change.** We will create a field that calculates the
    change between the initial EFC and paid-on EFC.

-   Click the ![](media/image7.png){width="0.25in"
    height="0.31180555555555556in"} (**List)** icon on the right side of
    the screen to get a listing of fields. Scroll down and double click
    on **EFC_P**. Notice this field is added to the calculator window.

-   On the **calculator pad, click the -** (minus) sign to the left of
    the number 4.

-   Under the **List** icon double click **EFC_I.**

-   On the **calculator pad,** click the **OK** to close the calculator
    pad. You should be back to the Field Selection tab.

-   Find the newly defined field in the **Available fields** window.
    They are listed alphabetically. Note the function symbol that
    indicates defined fields. Also note that when you highlight \-\-\--
    also the activated buttons that would allow you to **Edit define
    field** or **Delete define field.** Report Assistant can only
    display 200 fields at once. Clicking **Next** at bottom of the list,
    will display additional fields.

![](media/image8.png){width="5.9944444444444445in"
height="3.4895833333333335in"}

**Action: Add New Field to Report**

Follow these following instructions to add the new "EFC_Change" as a
report value.

-   Highlight EFC_Change in the Available Fields box.

-   Click the yellow/teal "plus/minus sign" icon above the report values
    box, just to the right of the sum/print options.

```{=html}
<!-- -->
```
-   Select the Print option above the report values box.

```{=html}
<!-- -->
```
-   In the report values box confirm that EFC_Change in the report
    values box is highlighted.

-   In the field options box confirm that the default "(Plain field
    value)" is selected.

![](media/image9.png){width="5.9944444444444445in"
height="4.354861111111111in"}

## The Report Headings Tab

Clicking on the **Report headings** tab allows you to enter information
to be displayed at the top (Heading) and bottom (Footing) of each page
of the report. We will not be adding any information on this tab today,
but we did want to show this tab breifly. You may "click along" with the
instructor or simply just watch the screen in front during this section
of the training.

After selecting the **Report headings** tab under **Page Heading** you
may enter any desired heading. On the bottom of screen, under **Page
Footing** contains by default "Report Sample Size: \<TOT.STUDCNTR." This
will display "Report Sample Size: #" with the number being the count of
students who satisfy any selection criteria imposed. You enter any
desired changes in the **Page Footing** box. There are a variety of
formatting options for the text in the Heading and Footing (e.g.,
justification, font, etc.).

**Action: Add Selection Criteria**

## The Selection Criteria Tab

There are two ways to add selection criteria. There are predefined
"available filters." You may also use any available field(s) to define
your own logical conditions. Let's look at an example of both.

-   Click the **Selection Criteria** tab at the top of the screen

-   In the **Available Filters** (bottom of screen), scroll down and
    double click on the desired filter. In this example double click the
    funnel icon next to **Dependent Students.**

-   In the **Available Fields** area, scroll down and double click on
    any field you want to include in a logical condition. For this
    example scroll down and double click on: **PAR_TAX_FIG_I**

-   To complete the "Where" statement, select the Field Operator Down
    button and select **EQUAL TO**. Click on **SELECT VALUES** and click
    on the **Constant** radio button and enter **2**. Select **OK**

![](media/image10.png){width="5.9944444444444445in"
height="4.354861111111111in"}

**Action: Select Report options**

## The Report Options Tab

To select the report format:

-   Prior to running the report, select the **Reports Options** tab.

-   Under the **Generate Report As** section, click on the down arrow
    and select. **EXL-97** **-- Excel 97**. (You may also select another
    version of Excel depending upon the version you have on your
    computer)

![](media/image11.png){width="5.1194444444444445in"
height="2.627083333333333in"}

**Action: Save the report and run**

To Save and Run the AD hoc Report:

-   Select the **Save** button. *[Helpful hint:]{.underline} Do not use
    spaces when naming your report. Use the underscore bar. In the
    description box enter the name **EFC_CHANGE_LISTING***

-   Generally we recommend you check the **Share Report** to make your
    AD Hoc report available for use by any QA school, but since this is
    just a training exercise leave that box empty.

-   After entering the name of the report, click **OK.**

-   On the tool bar, click **Run**. The Ad Hoc Report will display in
    Excel Format -- You may get a window asking you how to download the
    file being generated. Choose **Open** to launch the Excel document.

-   The report should be a column of numbers that reflect the change in
    EFC for the records who were dependent students and had estimated
    parental tax information.

-   You can now save this report as an Excel Workbook on the desktop.
    Within Excel click **File**, **SAVE AS**, and then enter a name such
    as **EFC_CHANGE_LISTING**. Under the **SAVE AS** section, click on
    the down arrow and select **Microsoft Excel Workbook** as the type.
    Click **SAVE**. You may receive a pop up box regarding formatting.
    If so, simply click **YES.**

-   Exit Excel by clicking on the red "X" in the top right-hand corner.
    This will bring you back to the Report Assistant.

**Action: Modify to display average change**

In addition to creating lists of data from the Tool, you can use Report
Assistant to present summary information. To demonstrate this, let's
modify the report we just finished and create a second report that
displays the average EFC change rather than a list of all of the
changes.

-   Click on the **Field Selection** tab.

-   Change the selection of **Print** to **Sum** above the report values
    box.

-   Confirm that **EFC_CHANGE** in the report values box is
    highlighted**.**

-   In the **Field Options** box, click on the arrow
    (![](media/image12.png){width="0.2916666666666667in"
    height="0.25in"}) under "Calculated as" and select **Average.**

-   Select the **Save As** button. Enter the name **EFC_CHANGE_AVG** in
    the description box.

-   After entering the name of the report, click **OK.**

-   On the tool bar, click **Run**. The Ad Hoc Report will display in
    Excel Format.

-   The report should be single value that reflects the average change
    in EFC for the records who were dependent students and had estimated
    parental tax information.

-   You can now save this report as an Excel Workbook on the desktop.
    Within Excel click **File**, **SAVE AS**, and then enter the
    **EFC_CHANGE_AVG**. Under the **SAVE AS** section, click on the down
    arrow and select **Microsoft Excel Workbook** as the type. Click
    **SAVE**. You may receive a pop up box regarding formatting. If so,
    simply click **YES.**

-   Close Excel.

-   To go back to the domain tree close out of the report we just ran by
    selecting the **X** in the to right hand corner of the report.

Any questions on the Report Assistant review?

Step 2: Run the new School Verification Analysis Report

We have created two new shared reports -- School_Verification_Analysis
and CPS_Verification_Analysis -- that we hope will help you see what
types of students your current verification procedures may be missing as
well as identify the types of students that your current selection
procedures may be burdening unnecessarily. We designed the reports to
minimize the amount of "programming" work you have to do. Other than the
verification selection criteria they use in defining the results, the
School and CPS versions are identical.

**Action: Access Shared Report**

From the domain tree of the ISIR Tool:

-   Click on **Shared Reports** folder.

-   Go to **+ Brigham Young University** and **select + Custom Reports**

-   Right Click on "**School Verification Analysis**" and select "**Save
    As My Report**" and select "**OK**."

    ![](media/image13.png){width="5.9944444444444445in"
    height="4.207638888888889in"}

These "reports" will produce counts of specific types of students by
whether or not they were selected for verification and whether or not
they experienced a "major change" in eligibility for need based aid. As
explained below you have the option of setting your own threshold as to
what change in EFC constitutes a "major" change.

Cutting and pasting the output of these reports into Excel templates we
have prepared will produce a series of 24 graphs that allow you to "see"
your results in a format that we hope helps you identify areas in
verification procedures to tweak. As explained in more detail below
these templates do assume that the information is in a specific order,
therefore it is essential that you not modify the content of the report.
Your data uploaded into ISIR Tool must contain at least some records
that are dependent students and (this sample year) some records not
normally selected for the verification. Again other than the labels they
apply to the output the two versions (School and CPS) of the two
templates are the same. During this training we will work exclusively on
the School verification version. If you are so inclined, you could
follow the exact same steps for the CPS verification version when you
get home.

**Action: Run Shared Report**

-   Go to **My Reports** and click on custom reports then left click on
    **School_Verification_Analysis**.

-   This will run the report and generate an Excel workbook.

-   If you get a warning box asking for instructions on how to handle
    the downloading of the file, click **OPEN.**

-   IMPORTANT -- BUT RARELY AN ISSUE -- make sure that that the EXCEL
    output contains all the combinations of SELECTED_BY_SCHOOL and
    MAJOR_CHANGE in the order indicated below. The vast majority of
    schools should have all their bases covered. Note there are 2 merged
    rows between the field name and the values. If any row is missing,
    simply use Excel to insert a row in the right location(s). You do
    not need to add any content to the row. A blank row will correctly
    position whatever data are there.

  ----------------------------------------- -----------------------------
  SELECTED_BY_SCHOOL                        MAJOR_CHANGE

                                            

  NO                                        NO

                                            YES

  YES                                       NO

                                            YES
  ----------------------------------------- -----------------------------

-   JUST AS IMPORTANT -- BUT EVEN MORE RARE -- You must have some
    dependent students in the data uploaded in the Tool for the
    information to be in the right location for the graphs produced by
    the templates. In the unlikely event that you have ONLY independent
    records uploaded and want to run the shared report and use the
    templates, you must highlight cells 1C to 1AC, Select insert, and
    choose columns. This will correct the placement of data when only
    independent records are present.

-   After confirming all data is in the position assumed by the
    template, left click in the box next to column A -- this will copy
    entire sheet.

-   Open the "School Verification Graphs" Excel document from the Desk
    Top.

-   Paste the information generated by the School_Verification_Analysis
    Report by placing the cursor on A1 in the "Paste Data Here" tab of
    the School Verification Graph Excel template, right click and select
    paste.

-   Click through the tabs of the bottom of the School Verification
    Graphs Excel Workbook to review your 24 graphs.

-   Save the Excel spreadsheet.

That's all there is to running the report. Are there any questions about
how to run the shared report and paste the results into the Excel
template before we take a brief tour in Report Assistant to understand
how the results were generated?

Step 3: Tour the new School Verification Analysis Report

Depending on how we are doing on time, we may truncate this "step"
during the training session, but we did want to let you know -- in at
least written form -- how the shared report produced its results. Also
note that there are two easy to use options that will allow you to
tailor the report to your school's needs.

**Action: Touring the School Verification Analysis Report**

-   Go to **My Reports** and click on **Custom Reports** then **right
    click on "School Verification Analysis"** and select **Report
    Assistant.** Report Assistant should open as indicated on the next
    page.

![](media/image14.png){width="5.9944444444444445in"
height="4.122916666666667in"}

![](media/image15.png){width="5.9944444444444445in"
height="4.5256944444444445in"}

-   Note the first two defined fields in the **Available Fields** box:
    AAA_COST_OF_ATTENDANCE and AAA_MAJOR_CHANGE_DEF. You have the option
    to customize these two fields.

-   The report uses the AAA_COST_OF_ATTENDANCE field as a selection
    criteria; selecting only students who have an EFC (initial or paid
    on) less than the specified value. By default this value is set to
    \$17,336 -- the national average for in-state public four-year
    students in 2007-08. To change this highlight AAA_COST_OF_ATTENDANCE
    and click on **the Edit Define Field** button. Change the value to
    reflect the average cost of attendance at your school or some other
    meaningful threshold, e.g., your school's EFC threshold for
    packaging grant aid or a maximum EFC for eligibility for a state
    scholarship. Click **OK.**

-   By default the report will define as a major change in aid
    eligibility any change to a Pell award or an absolute value change
    to EFC in excess of 400. If you would prefer a lower or higher EFC
    threshold, highlight AAA_MAJOR_CHANGE_DEF. Click on **the Edit
    Define Field** button. Change the value to any positive number you
    like. The higher you set your EFC threshold, the more exclusively
    your results will reflect only changes to Pell awards. Click **OK.**

-   The report uses **MODEL_I** (D for dependent students and I for
    independent students) to **SORT ACROSS**. This means that the report
    will display columns of all information first -- from left to right
    -- for dependent students and then display values for independent
    students. The order is determined alphabetically; D comes before I.
    *Warning if your database contains only independent students, the
    graphs in the template will be mislabeled.*

-   The report uses **SELECTED_BY_SCHOOL** (No and Yes) and
    **MAJOR_CHANGE** (No and Yes) to **SORT BY**. This means that the
    first row of data in the report will display information for records
    that were not selected by school verification and did not experience
    a major change. The second row will also reflect those students not
    selected by school verification; however these records will have
    experienced a major change. The third row of data will include
    students who were selected by the school but did not experience a
    major change. The final row will contain students that were selected
    by the school and did experience a major change. The order here is
    determined alphabetically -- No comes before Yes -- and the order of
    fields in the sort by box. *Warning if your database does not
    contain at least one record in each of the first three rows, the
    graphs in the template will be mislabeled.*

-   Review how SELECTED_BY_SCHOOL was defined. Highlight
    **SELECTED_BY_SCHOOL** in the Available Fields box. Click on **the
    Edit Define Field** button. Just look. To make sure you don't change
    anything click **Cancel.**

-   Review how MAJOR_CHANGE was defined. Highlight **MAJOR_CHANGE** in
    the Available Fields box. Click on **the Edit Define Field** button.
    Click **Cancel.**

-   Review how ABS_EFC_CHANGE was defined. Highlight **ABS_EFC_CHANGE**
    in the Available Fields box. Click on **the Edit Define Field**
    button. Click on the "Fx" function symbol on the far right and top
    of the define field window. Scroll down to "ABS" and hold the cursur
    over this function to display the description, "Returns the absolute
    value of its argument." Click **Cancel.**

-   Note all the entries in the Report Values boxes are sums.

-   Review how COUNTER was defined. Highlight **COUNTER** in the
    Available Fields box. Click on **the Edit Define Field** button.
    Note that it is simply 1 for every case, so the sum would be simply
    the total number of cases in a particular row and column of the
    report. Click **Cancel.**

-   All the other fields in the report are "conditional" 1's. That is,
    they are 1 if and only if the record satisfies the logical condition
    implied by the field's name. For example review how EST_FILER was
    defined. Highlight **EST_FILER** in the Available Fields box. Click
    on **the Edit Define Field** button. This field is 1 if "APP_FIG" is
    equal to '2'. What is APP_FIG? Click **Cancel.**

-   Review how APP_FIG was defined. Highlight **APP_FIG** in the
    Available Fields box; note how many other "APP's" there are. These
    follow the same logic. Click on **the Edit Define Field** button.
    Note that the formula assigns the applicable parent ISIR field if
    the student is dependent and the student field if not. Click
    **Cancel.**

-   Click on the **Report Heading** tab, review content. It is important
    to NOT add or delete any rows to the heading.

-   Click on the **Selection Criteria** tab, review content. Note how
    AAA_COST_OF_ATTENDANCE field is used.

-   Click on the **Report Option** tab. Note that Excel 97 is selected.
    We recommend leaving this option as is, even if you have a more
    recent version of Microsoft Office.

-   If you made any changes to AAA_COST_OF_ATTENDANCE or
    AAA_MAJOR_CHANGE_DEF be sure to save the report, by clicking the
    **Save** button.

-   If you want to stick with the AAA_COST_OF_ATTENDANCE and
    AAA_MAJOR_CHANGE_DEF defaults, click on the **Quit** button.

This completes the "tour" of the new, shared report. We are now ready to
analyze the results. Any questions before we do?

If you made a change to either of the modifiable fields and would like
analyze results reflecting those selections, you will need to rerun the
report and paste that information into the template. At this time any
school that wants to rerun the report, go back to the "Action: Run
Shared Report" heading above and rerun the report.

Step 4: Analyze your Results

During this section of the training we will be going through a series of
PowerPoint slides that mirror the illustrative graphs contained in this
training manual. After explaining how to analyze the results in general,
we will ask each of you to apply this lesson to your own results with
the help of an analytic worksheet. The analytic worksheet should not be
necessary once you get the hang of using the graphs, but we think the
worksheet helps develop that understanding.

There are two types of graphs contained in the "Graph" template. The
first type denoted by D# or I# tabs, graphs the percent of records in
the indicated category of students who:

-   Were selected for verification and experienced a major change in aid
    eligibility -- Black

-   Were selected for verification and did not experience a major change
    in aid eligibility -- Light Blue

-   Were not selected for verification and experienced a major change in
    aid eligibility -- Dark Blue

-   Were not selected for verification and did not experience a major
    change in aid eligibility -- White

![](media/image29.wmf)

We have designed this first type of graph to make it easy for you to
identify the types of students your current verification procedures may
be missing (dark blue) or may be selecting unnecessarily (light blue).
These are the graphs you should concentrate on. The goal of your
analysis is quite simple; "get rid of the blues" by modifying your
verification efforts to target (turn dark blue to black) or eliminate
records without a major change form your verification efforts (turn
light blue to white). For training purposes we keep our discussion of
these new graphs fairly simplistic. That is, we talk about how to
interpret the results in isolation. In practice we encourage you to use
these reports in combination with other information to apply more
nuanced modifications to your verification procedures.

If blue is "bad" than perfect results would be like TV in the 1950s,
just black and white. Note that the accuracy of your students' initial
application is the only thing that determines the percent your school
"should" verify.

It is probably not realistic to assume that any school would ever be
able to achieve 100 percent accuracy in all the verification selection
criteria. Don't be discouraged if a little blue remains.

The first step in analyzing these results is to identify categories of
students who are the most "blue." It is probably easier to do this
sequentially. First identify the types of students with the highest
percents not selected with a major change (dark blue) and then go back
through to identify which students had the highest percent of selected
without major change (light blue).

After you identify the students most likely to be missed by verification
(dark blue), you need to consider whether or not it makes sense to add
ALL students in the identified category to your verification criteria.
This depends, of course, on the total percentage of students of this
type who experienced a major change to aid eligibility. Consider the
percent of records that were selected for verification with a major
change (black). If there is high percent of major changes not being
selected and a high percent major changes that are being selected, it
might makes sense to go ahead and verify the entire group. See an
illustration of the sliding scale you could apply to your own results
below. When your results fall in the "maybe" region try to think of
other information you could use to distinguish the subset of students
that need verification from those that do not.

Likewise after you identify the students most likely to be selected
unnecessarily by verification (light blue), you need to consider whether
or not it makes sense to simply exclude ALL students in the identified
category from verification. This will depend on the total percentage of
students of this type who didn't experience a major change. An easy way
to see this in the graph is to consider the percent of records not
selected for verification without major change (white). If there is high
percent of no major changes not being selected and a high percent of no
major changes being selected, it might makes sense to go ahead exclude
the entire group. See an illustration of this below. When your results
fall in the "maybe" region try to think of other information you could
use to distinguish the subset of students that do not need verification
from those that do.

While we encourage you to focus on the graphs that display the
percentage breakdowns of verification selection and major change status,
you also need to consider the number of records your findings are based
on before you make changes to your verification criteria. The second
type of graph, denoted by D#N or I#N, are stacked bar charts that
represent records selected for verification are represented in black and
those not selected are indicated by white. Here is an example of this
type of graph.

![](media/image31.wmf){width="5.995138888888889in"
height="2.748611111111111in"}

Generally speaking the larger the numbers of records, the more
confidence you can place in the results. As a minimum, you should have
at least five applicable records that you selected for verification
before you use your results as a basis to reduce the scope of current
verification efforts. Likewise, you should have at least five applicable
records that were not selected for verification before you consider
expanding verification based on the corresponding finding. So if there
are four or fewer applicable records we recommend foregoing any
analysis. When there are 5 to 24 records, take you results with a grain
of salt.

**Action: Complete the analytic worksheets**

What we would like you to do now is to go through your results and
complete the two worksheets on the following pages.

First, go through tabs D2, D3, D4, D5, and D6 in the Excel template to
identify the **three** categories of dependent students with the largest
percentage of major changes who were not selected for school
verification (dark blue). We are skipping D1 because it contains summary
information for all dependent students. When you have identified these
three groups list them on the worksheet and fill in the percent not
selected with a major change (dark blue) and percent selected with a
major change (black). Next, refer to the D#N tab corresponding the D#
graph to find out how many records were not selected for verification.
For example if you found on tab D5 that dependent students whose parents
will not file a tax return were very likely to not be selected for
verification and experience a major change, refer to tab D5N to get the
number of "will not file" records not selected for verification. Leave
the final column blank until you get all the other data filled in on
both worksheets.

Do the same for independent students using tabs I2, I3, I4, I5, and I6
and the corresponding I# tabs.

Move on to the second worksheet and complete a similar process. Only
this time identify the **three** categories among dependent and
independent students who were the most likely to be selected for
verification without experiencing a major change to aid eligibility
(light blue). Record this percent and the percent not selected without a
major change (white) on the worksheet. Use the appropriate "N" tabs to
find and record the number of records selected for verification in the
relevant groups.

After completing the first four columns of the two worksheets apply the
results by answering the two questions. We provide *rule of thumb
guidance* to answering these questions. You should feel empowered to
draw your thresholds differently.

Should you change the way you treat ALL records in this category based
on the percents recorded in the second and third columns of the
worksheet?

-   *if sum of the two percents totals less than 40% then "No"*

-   *if the sum is between 40-70% then "Maybe"*

-   *if the sum is higher then 70% then "Yes"*

Based on record count, how confident are you?

-   *If less than 5, not at all -- in fact if this wasn't a training
    exercise we should even consider answering the first question*

-   *If 5 to 25 -- only a little confident, but the closer to 25 the
    better I feel*

-   *If more than 25, pretty confident, but this is still just one
    sample*

Take the next 20 minutes to complete on you own.

+--------------+--------+--------+-------+---------------------------+
| Highest      |        |        |       |                           |
| Percent Not  |        |        |       |                           |
| Selected     |        |        |       |                           |
| with Major   |        |        |       |                           |
| Change       |        |        |       |                           |
+--------------+--------+--------+-------+---------------------------+
| Type of      | P      | P      | \# of | Applying of results:      |
| Student      | ercent | ercent | re    |                           |
|              | Not    | Se     | cords | -   Would it be a good    |
|              | Se     | lected | not   |     idea to add student   |
|              | lected | Major  | sel   |     group to              |
|              | with   | Change | ected |     verification?         |
|              | Major  | (      |       |                           |
|              | Change | Black) |       | -   Based on record       |
|              | (Dark  |        |       |     count, how confident  |
|              | Blue)  |        |       |     are you?              |
+--------------+--------+--------+-------+---------------------------+
| Dependent    |        |        |       |                           |
| Students     |        |        |       |                           |
+--------------+--------+--------+-------+---------------------------+
|              |        |        |       |                           |
+--------------+--------+--------+-------+---------------------------+
|              |        |        |       |                           |
+--------------+--------+--------+-------+---------------------------+
|              |        |        |       |                           |
+--------------+--------+--------+-------+---------------------------+
| Independent  |        |        |       |                           |
| Students     |        |        |       |                           |
+--------------+--------+--------+-------+---------------------------+
|              |        |        |       |                           |
+--------------+--------+--------+-------+---------------------------+
|              |        |        |       |                           |
+--------------+--------+--------+-------+---------------------------+
|              |        |        |       |                           |
+--------------+--------+--------+-------+---------------------------+

+--------------+--------+--------+-------+---------------------------+
| Highest      |        |        |       |                           |
| Percent      |        |        |       |                           |
| Selected     |        |        |       |                           |
| without      |        |        |       |                           |
| Major Change |        |        |       |                           |
+--------------+--------+--------+-------+---------------------------+
| Type of      | P      | P      | \# of | Applying the results:     |
| Student      | ercent | ercent | re    |                           |
|              | Se     | Not    | cords | -   Would it be a good    |
|              | lected | Se     | sel   |     idea exclude student  |
|              | w      | lected | ected |     group to              |
|              | ithout | w      |       |     verification?         |
|              | Major  | ithout |       |                           |
|              | Change | Major  |       | -   Based on record       |
|              | (Light | Change |       |     count, how confident  |
|              | Blue)  | (      |       |     are you?              |
|              |        | Black) |       |                           |
+--------------+--------+--------+-------+---------------------------+
| Dependent    |        |        |       |                           |
| Students     |        |        |       |                           |
+--------------+--------+--------+-------+---------------------------+
|              |        |        |       |                           |
+--------------+--------+--------+-------+---------------------------+
|              |        |        |       |                           |
+--------------+--------+--------+-------+---------------------------+
|              |        |        |       |                           |
+--------------+--------+--------+-------+---------------------------+
| Independent  |        |        |       |                           |
| Students     |        |        |       |                           |
+--------------+--------+--------+-------+---------------------------+
|              |        |        |       |                           |
+--------------+--------+--------+-------+---------------------------+
|              |        |        |       |                           |
+--------------+--------+--------+-------+---------------------------+
|              |        |        |       |                           |
+--------------+--------+--------+-------+---------------------------+

Step 5: Share Results and Discuss How to Apply

Now that every one has completed the worksheets, please form groups of
no more than four schools. We want everyone to go over their "answers"
in a group setting. Please review the other schools in your groups work,
and ask questions if you don't understand or disagree with how they have
applied their results discuss. Remember be polite, we are all learning.

After "checking" everyone's work discuss:

-   Did any student categories appear on multiple schools lists?

-   Can schools see any connections to their selection criteria from
    these results?

-   Which one worksheet will school be able to fill in with 2009-10
    (non-sample) data?

-   Which threshold for applying results -- enough cases or high
    combined percent -- was the most difficult to meet?

20 minutes allotted for group discussion

If time permits

Step 6: Your Feedback

If there is time, we would like to get your reaction and ideas for
further enhancement of the analysis possible in the Tool. What else
would you like to see? Some ideas to get you thinking:

-   What additional filters would you want to have the option of
    applying to these shared reports?

-   What additional student groupings would you want to define in these
    shared reports?

-   Are there other measures beyond "major change to aid eligibility"
    would you want to define in reports similar to these shared reports?

We are looking forward to addressing your needs with the recently
purchased Active Reports add on. If possible we may be able demonstrate
how apply some of your ideas for further analysis by editing the
existing shared report templates or by manipulating the data differently
in Excel template.

If we run out of time or you think of something after the session ends,
please send any and all ideas for improving these shared reports and
templates to <David.Rhodes@ed.gov> and <Anne.Tuccillo@ed.gov>. We would
also welcome ideas for other additional analyses.