### B1 StudentAid.gov Mobile Device Usability Evaluation IDI Guide FINAL 11/ 27/1 2

Market Connections Inc. 1 

### Interview #_____ **FSA MOBILE DEVICE USABILITY STUDY** 

**INTERVIEW GUIDE FINAL VERSION**  **1 1** **/ 27** **/ 12**  **SAMPLE RECORD:** 1 BORROWER IN REPAYMENT

2 CURRENT COLLEGE /ADULT STUDENT 3 HIGH SCHOOL SENIOR 4 PARENT OF HS SENIOR **DEVICE USED:** A. ANDROID TABLET

B. ANDROID PHON E C. i PAD D. i PHONE 

### I. **INTRODUCTION** The U.S. Department of Education’s office of Federal Student Aid seeks to continuously improve its interactions with those who seek information and guidance on the FSA website regarding education financing issues and options
. To that end we are conducting a brief interview to assess our performance and provide you with an opportunity to share any ideas or feedback you may have to help us better serve you. The interview is conducted by Market Connections, a Virginia
- based pr ofessional market research company. Your responses will remain confidential to FSA; we hope this will allow you to feel free to be candid in your feedback to us. As mentioned, t he purpose of this interview is to talk about a website operated by the Department of Education’s office of Federal Student Aid, and get your impressions. We will soon go online and I’ll ask you to use your mobile device **[RECONFIRM DEVICE USED]** to review some information and complete a number of tasks. It is very important that you talk through out loud the steps you take while on the web site. Tell us what you are doing and why, and the feelings you have, including any problems you experience and also any areas where things are going particularly well. We are interested primar ily in your personal views on the content, terminology, the layout and detail of information, and how the task
- completion process unfolds for you, along with any other insights you may have. Again, don’t tell me what you think other people might say abo ut the site – I want to know what you think. This will take about 40 to 45 minutes. We are recording so we can prepare a report, and I have colleagues observing in another room. As part of the recording process, we have a few cameras set up so that we can see what you see on your device. To reiterate, your comments will be treated as confidential. The information gained from this study will be used to improve FSA’s website to make it more intuitive and user
-friendly. Do you have any questions ? 0:03

### B1 StudentAid.gov Mobile Device Usability Evaluation IDI Guide FINAL 11/ 27/1 2

Market Connections Inc. 2 **II. INITIAL IMPRESSIONS**  **OF THE SITE** When we invited you here, you told us you were: [ READ APPROPRIATE STATEMENT ]

**1 [**  **BORROWER IN REPAYMENT** **]** A n [undergrad/graduate] student with outstanding student loans . **2 [**  **CURRENT COLLEGE/ADULT STUDENT]** A curren t undergrad or graduate student. Are you a returning student repaying any student loans at this time? YES__ NO ___ **3 [**  **PARENT OF HIGH SCHOOL SENIOR**  **]** T he parent of a high school senior intending to enter a two - or four
- year college in the next year . **4 [HI**  **G H SCHOOL SENIOR]**  A h igh school [ senior/ junior] who intends to enter a two - or four
-year college in the [ next year/next few years]
. 1. Understanding your current situation, what topic areas or information would be most helpful or relevant to you personally if you were to visit a website titled “ Student Aid .gov”? What specifically would you want to look for?
The site you are about to review contains a variety of information
. Please type in studentaid.gov and take a few minutes now to get familiar with th e h ome page . Also, feel free to explore any of the links you see; you do not necessarily n eed to stay on this page. I’m interested in where you’d go next. Please remember to talk aloud through your actions so we can know what you’re thinking as you explore the homepage. [ **INTERVIEWER: ALLOW THE RESPONDENT TO EXPLORE THE HOMEPAGE FOR A FEW MINUTES.**  **]** 2. Thinking about the home page itself, please tell me how you would rate it on a scale of 1 to 10 in terms of the ease of finding information on your particular a reas of interest, where 1 is “poor” and 10 is “excellent.” Poor Excellent 1 2 3 4 5 6 7 8 9 10
a. Wh at makes you say that ?

b. So far, which features on the homepage have caught your attention? Please explain. [ **PROBE** : specific features like d, disliked] 0: 1 0

### B1 StudentAid.gov Mobile Device Usability Evaluation IDI Guide FINAL 11/ 27/1 2

Market Connections Inc. 3

c. To what degree does the homepage give you an idea of what to expect when looking through the site?

d. When you first assessed the homepage, did you carefully read the text, skim it, or mostly ignore the text portion ? i. READ CAREFULLY

_

___ SKIMMED ___ IGNORED___
3. **IF SKIMMED/IGNORED** : Please take few minutes to look over the text once more. **WAIT TWO MINUTES AS THEY READ, THEN ASK**  : thinking about the text you read, in addition to any other information on the page, to what degree do es the homepage give you an idea of what to expect on the site as a whole? How so?

e. Prior to our interview, had you ever visited this site previously? **IF YES:** i. F or what purpose? ii. Were you able to accomplish what you set out to do? f. Had you ever specifically visited this site using your mobile device? For what purpose? g. Are there any specific challenges or advantages to viewing and navigating this site with your mobile device versus on a PC or laptop? What exactly? 4. At this point, what suggestions, if any, do you have for improving th e homepage to meet your needs? a. **IF MEGA MENU/BLACK NAVIGATION BAR MENTIONED** **, PROBE BRIEFLY, THEN ASK** , “ Is there a nything else?”
5. Given what you have seen so far, are you more likely to want to continue looking through the site, or are you more likely to close the site?
a. What makes you say that? 0: 2 0

### B1 StudentAid.gov Mobile Device Usability Evaluation IDI Guide FINAL 11/ 27/1 2

Market Connections Inc. 4 **I I**  **I. USABILITY TASKS**  **TASKS: BORROWERS IN REPAYMENT**  0:2 0 1. Understanding that you are a student loan borrower in repayment, where would you go for informati on on this topic? Task 1: Review “ Understanding Repayment” contents 2. Please touch the heading, “ Understanding Repayment”. Studentaid.gov/repay
loans/understand
. Looking at this information, what would you touch first to learn more?
a. What do you think of the contents covered in this section? b. Is there too much information here to process, or not enough information to get a good sense of what you want to learn?
c. What about the amount of text to read? (Too much, too little, just right?) d. What are your feelings about the terminology used – do you feel it was written with you in mind?
e. What are your feelings about the layout and organization of this information? (PROBE: Does it flow in a logical way?) f. Is there anything you would change , or is it fine as is? 0:26 Task 2: Review Loan Forgiveness Options 3. Now look for information on loan forgiveness
. **[IF PARTICIPANT HAVING TROUBLE:** touch “ Repay Your Loans”. 4. What would you do next? **[ IF STILL HAVING DIFFICULTY**  : Touch “ Forgiveness, Cancellation and Discharge ” link. Then touch the link “Learn about forgiveness, cancellation or discharge” ] 5. On this page , after scrolling down a bit , you ’ll see a table . Take a moment to look it over. What is the purpose of this table? 6. Please tell me how easy or difficult it was to find th is information , using a scale of 1 to 10, where 1 is “very difficult and 10 is “very easy”
. Very difficult to find Very easy to find 1 2 3 4 5 6 7 8 9 10
7. What suggestions, if any, do you have for improving the table? [ **PROBE I F NECESSARY:** 

... in terms of the content, layout and design, terminology or any other factors ? ]

### B1 StudentAid.gov Mobile Device Usability Evaluation IDI Guide FINAL 11/ 27/1 2

Market Connections Inc. 5 Task 3 : Locate and use Monthly Loan Repayment Estimate Calculator 0: 31 8. Touch the Repay Your Loans menu . Locate and use the repayment calculator to estimate the monthly payment amount for the **Pay as You Earn** option . You may use fictional loan amounts for this exercise. [ **IF PARTICIPANT IS HAVING TROUBLE LOCATING**  **THE CALCULATOR:**  touch “ Understanding Repayment”, then touch “ Repayment Plans” and then scroll down to locate the **Pay as You Earn** option and the calculator for that option .]
a. Is this where you would have expected to find the calculator? i. **IF NOT:** What needs to change to make the calculator easier to find? 9. **IF NOT ALREADY ON THE “REPAYMENT PLANS” MENU**  **:** Now touch the “Repayment Plans” menu . Please locate a calculator. [ **IF PARTICIPANT IS STILL**  **HAVING TROUBLE LOCATING A CALCULATOR:**  Scroll down to the table. You’ll see some Quick Links for diffe rent types of repayment plans, such as Standard Plan, Pay as you Earn and others. Touch **the Pay as you Earn** link . ( **IF NECESSARY:** now scroll down to find the calculator to calculate your estimated loan payments for that repayment plan . You will see the “Calculate your Estimated loan Payments...” link) 10. Please tell me how you would rate the calculator itself on a scale of 1 to 10 where 1 is “not at all easy to use ” and 10 is “very easy to use.” Not at all easy to use Very easy to use 1 2 3 4 5 6 7 8 9 10
11. Please tell me how you would rate the calculator it self on a scale of 1 to 10 where 1 is “not at all easy to understand the results ” and 10 is “very easy to understand the results .” Not at all easy to understand Very easy to understand 1 2 3 4 5 6 7 8 9 10
a. What makes you say that? 12. What feelings , if any, d o you have in terms of locating the calculators? What exactly? 0: 40
a. If you were on your own to look for the calculator, realistically, would you have continued to search for it or would you have abandoned the search and moved on to somethin g else? What makes you say that? 13. What do you feel are the pros and/or cons of completing any of these tasks on your mobile device, versus doing so on a PC or laptop?

### B1 StudentAid.gov Mobile Device Usability Evaluation IDI Guide FINAL 11/ 27/1 2

Market Connections Inc. 6

a. Realistically, would you have performed any these tasks with your mobile device? Whi ch ones? Are there any you definitely would not have tried with your mobile device? 14. Now that you have reviewed several aspects of the studentaid.gov website, do you have any final advice for the Office of Federal Student Aid to consider, in order to make the site easier to use for those using a mobile device such as yours? **SKIP Q15**  **IF NO TIME** 15. Having seen/used the calculator, and become acquainted with the repayment plan options, what would be your next step? [ **PROBE** : How would you expect to go about makin g a payment?] That concludes our interview. Thanks very much for your input. It was very helpful! 0:45

### B1 StudentAid.gov Mobile Device Usability Evaluation IDI Guide FINAL 11/ 27/1 2

Market Connections Inc. 7 **TASKS: CURRENT COLLEGE/ADULT STUDENTS**  0:20 1. Understanding that you are a current college student straight out of high school, or a ret urning college student, you may or may not currently be repaying your student loans. In your case, a re you repay ing any student loans at this time? YES__ NO__ Task 1: Review “ Understanding Repayment” contents 2. Please touch the heading, “ Understanding Rep ayment”. Studentaid.gov/repay
loans/understand
. Looking at this information, what would you touch first to learn more?
a. What do you think of the contents covered in this section? b. Is there too much information here to process, or not enough information to get a good sense of what you want to learn?
c. What about the amount of text to read? (Too much, too little, just right?) d. What are your feelings about the terminology used – do you feel it was written with you in mind?
e. What are your feelings about t he layout and organization of this information? (PROBE: Does it flow in a logical way?) f. Is there anything you would change , or is it fine as is? 0:2 6 Task 2: Review Overview of Direct Loan and FFEL Program Repayment Plans table 3. Now look for information on ways to choose a repayment plan. **[IF PARTICIPANT HAVING TROUBLE:**  scroll down a bit and touch “Repayment Plans”. ] 4. What would you do next? **[ IF STILL HAVING DIFFICULTY:**  Touch “Get details about repayment plans” link.] 5. On this page you see a table, “ Overview of Direct Loan and FFEL Program Repayment Plans”. Take a moment to look it over. What is the purpose of this table? 6. Please tell me how you would rate the information in the table on a scale of 1 to 10 , where 1 is “not at all relevant” and 10 is “very relevant” to you . Not at all relevant Very relevant 1 2 3 4 5 6 7 8 9 10 7. Please tell me how you would rate the information in the table on a scale of 1 to 10 , where 1 is “ after reviewing the table I do not understand how to choose the best plan for me” and 10 is “ I clearly understand how to choose the best plan for me ”
. Do not understand how to choose Clearly understand how to choose

### B1 StudentAid.gov Mobile Device Usability Evaluation IDI Guide FINAL 11/ 27/1 2

Market Connections Inc. 8 1 2 3 4 5 6 7 8 9 10 Task 3: Locate and use Monthly Loan Repayment Estimate Calculator 0: 31 8. Touch the Repay Your Loans menu . Locate and use the repayment calculator to estimate the monthly payment amount. You may use fictional loan amounts for this exercise. [ **IF PARTICIPANT IS HAVING TROUBLE LOCATING A CALCULATOR:** touch “Understanding Repayment”, then touch “Repayment Plans” and then scroll down to locate the calculator.]
a. Is this where you would have expected to find the calculator? i. IF NOT: What needs to change to make the calculator easier to find? 9. **IF NOT ALREADY ON THE “REPAYMENT PLAN S”**  **MENU :**  Now touch the “Repayment Plans” menu . Please locate a calculator. [ **IF PARTICIPANT IS STILL HAVING TROUBLE LOCATING A CALCULATOR:**  Scroll down to the table . You’ll see some Quick Links for different types of repayment plans, such as Standard Plan , Graduated Repayment Plan and others. Touch any of those links, and then locate the link to calculate your estimated loan payments for that repayment plan. **(IF NECESSARY:** scroll down until you see “Calculate your Estimated loan Payments...” link) 10. Please tell me how you would rate the calculator itself on a scale of 1 to 10 where 1 is “not at all easy to use ” and 10 is “very easy to use.” Not at all easy to use Very easy to use 1 2 3 4 5 6 7 8 9 10 11. Please tell me how you would rate the calcula tor itself on a scale of 1 to 10 where 1 is “not at all easy to understand the results ” and 10 is “very easy to understand the results .” Not at all easy to understand Very easy to understand 1 2 3 4 5 6 7 8 9 10
a. What makes you say that? 12. What fe elings, if any, d o you have with regard to finding the calculator? What exactly?
a. If you were on your own to look for the calculator, realistically, would you have continued to search for it or would you have abandoned the search and moved on to someth ing else? What makes you say that? 13. What do you feel are the pros and/or cons of completing any of these tasks on your mobile device, versus doing so on a PC or laptop?
a. Realistically, would you have performed any these tasks with your mobile device? W hich ones? Are there any you definitely would not have tried with your mobile device? 0:4 3

### B1 StudentAid.gov Mobile Device Usability Evaluation IDI Guide FINAL 11/ 27/1 2

Market Connections Inc. 9 14. Now that you have reviewed several aspects of the studentaid.gov website, do you have any final advice for the Office of Federal Student Aid to consider in o rder to make the site easier to use for those using a mobile device such as yours? **SKIP Q15**  **IF NO TIME** 15. Having seen/used the calculator, and become acquainted with the repayment plan options, what would be your next step? [ **PROBE** : How would you expect to g o about making a payment?] That concludes our interview. Thanks very much for your input. It was very helpful! 0:45

### B1 StudentAid.gov Mobile Device Usability Evaluation IDI Guide FINAL 11/ 27/1 2

Market Connections Inc. 10 **TASKS: PARENTS OF HS SENIORS**  0: 2 0 1. Understanding that you are the parent of a high school senior planning to enter college in the next year, w here on the site would you go to understand about applying for student aid ? Task 1 : Review “FAFSA: Apply for Aid” contents 2. I want you to touch the menu , “FAFSA: Apply for Aid” ( studentaid.gov/fafsa ) . Looking at this information, what wou ld you touch first to learn more?
a. What do you think of the contents covered in this section? b. Is there too much information here to process, or not enough information to get a good sense of what you want to learn?
c. What about the amount of text to re ad? (Too much, too little, just right?) d. What are your feelings about the terminology used – do you feel it was written with you in mind?
e. What are your feelings about the layout and organization of this information? (PROBE: Does it flow in a logical way? ) f. Is there anything you would change , or is it fine as is? 0:25 Task 2 : Review FAFSA Overview video 3. Please touch the video link titled “FAFSA Overview” and either during or after the video let me know what you think. 4. Thinking about the video, please te ll me how you would rate it on a scale of 1 to 10 in terms of your ability to understand the information presented in the video , where 1 is “the video is not at all easy to understand ” and 10 is “very easy to understand .” Not at all easy to understand Very easy to understand 1 2 3 4 5 6 7 8 9 10
a. What makes you say that? **SKIP Q5 IF NO TIME** 5. Again thinking about the video, please tell me how you would rate it on a scale of 1 to 10 in terms of its effect on your confidence in being able to complete the student aid application process, where 1 is “ I am not at all confident ” and 10 is “ I am highly confident .” Not at all confident Highly confident 1 2 3 4 5 6 7 8 9 10
a. What makes you say that? 0: 40

### B1 StudentAid.gov Mobile Device Usability Evaluation IDI Guide FINAL 11/ 27/1 2

Market Connections Inc. 11 6. What are your feelings regarding the length of the video when viewing on your mobile device ? ( PROBE: Is it too long, too short, just right?) 7. Please tell me how you would rate the ease of finding this video on a scale of 1 to 10 , where 1 is “very difficult” and 10 is “very easy” to find
. V ery difficult to find Very easy to find 1 2 3 4 5 6 7 8 9 10
a. What makes you say that? 8. When using your mobile device, what circumstances would lead you to view videos of this type? 9. What do you feel are the pros and/or cons of watching this video on your mobile device, versus doing so on a PC or laptop? 0:33 Task 3 : Assess “Types of Aid” menu information 10. Let’s suppose you want some preliminary information on the types of aid available
. I want you to tell me out loud what types of informatio n you’re looking for and how you are going about finding it. 11. Thinking about th is section in general, please tell me how you would rate it on a scale of 1 to 10 in terms of the ease with which you can f ind what you are looking for, where 1 is the info rmation I want is “Very difficult to fi nd ” and 10 is “Very easy to fi nd ”
. Very difficult to f ind Very easy to understand 1 2 3 4 5 6 7 8 9 10
a. What makes you say that? b. What are your feelings about the number of steps required to complete the form? 12. Also t hinking about this section , please tell me how you would rate it on a scale of 1 to 10 in terms of your ability to understand the information presented , where 1 is “very difficult to understand ” and 10 is “very easy to understand .” Very difficul t to understand Very easy to understand 1 2 3 4 5 6 7 8 9 10
a. What makes you say that? 13. What do you feel are the pros and/or cons of completing any of these tasks on your mobile device, versus doing so on a PC or laptop?

### B1 StudentAid.gov Mobile Device Usability Evaluation IDI Guide FINAL 11/ 27/1 2

Market Connections Inc. 12

a. Realistically, would you hav e performed any these tasks with your mobile device? Which ones? Are there any you definitely would not have tried with your mobile device? 14. Now that you have reviewed several aspects of the studentaid.gov website, do you have any final advice for the Offi ce of Federal Student Aid to consider, in order to make the site easier to use for those using a mobile device such as yours? That concludes our interview. Thanks very much for your input. It was very helpful! 0:45

### B1 StudentAid.gov Mobile Device Usability Evaluation IDI Guide FINAL 11/ 27/1 2

Market Connections Inc. 13 **TASKS: HIGH SCHOOL SENIORS**  0: 2 0 2. Understanding that you are a high school [ senior/ junior] planning to enter college in the next [ year/few years] , where on the site would yo u go to understand more about college? Task 1: Review “ Prepare for College” contents 3. I want you to touch the heading, “ Prepare for College ” ( studentaid.gov/prepare
-for
college)
. Take a few minutes to look at this section . a. What do you think of the contents cover ed in this section ? b. Is there too much information here to process, or not enough information to get a good sense of what you want to learn?
c. What about the amount of text to read? (Too much, too little, just right?) d. What are your feelings about th e terminology used – do you feel it was written with you in mind?
e. What are your feelings about the layout and organization of this information? ( PROBE: Does it flow in a logical way? ) f. Is there anything you would change , or is it fine as is? 0:2 6 Task 2: Review “ Preview the Financial Aid Process” infographic 4. Please locate and touch the graphic titl ed “ Preview the Financial Aid Process” and let me know what you think. ( **IF INITIAL DIFFICULTY:** under what main menu heading would you expect to find that information? **(IF STILL HAVING DIFFICULTY , DESCRIBE AS NEEDED**  **:** Touch “Prepare for College”, then touch “C hecklists for Academic and Financial Preparation” link. Then scroll down to “Preview the Financial Aid Process”. This is the graphic I’d like you to review. Please take a few moments to look it over. 5. What do you feel are the pros and/or cons of fin ding this graphic on your mobile device, versus doing so on a PC or laptop? ( **PROBE:** how about finding this information in general?) 6. What do you feel are the pros and/or cons of viewing this graphic o n your mobile device, versus doing so on a PC or laptop ? 7. Thinking about the graphic, please tell me how you personally would rate it on a scale of 1 to 10 in terms of the impact it has, if any, on encouraging your decision to go to college , w here 1 is “no impact at all ” and 10 is “very strong impact” on yo u . No impact at all Very strong impact

### B1 StudentAid.gov Mobile Device Usability Evaluation IDI Guide FINAL 11/ 27/1 2

Market Connections Inc. 14 1 2 3 4 5 6 7 8 9 10
a. What makes you say that? i. (PROBE IF RATING OF 6
- 10) : Does the impact come mainly from the content, or the way the content is presented in the graphic, or both?) ii. (PROBE IF RATING OF 1
- 5) : What, if anything, would you suggest to increase the impact of this information? 0 : 3 2 Task 3: Review “Overview of the financial aid process” video 8. Of course, an other aspect of going to college involves how to finance your college education. With th at in mind, p lease go to the main menu heading labeled, “Types of Aid” and touch the video link titled “Overview of the Financial Aid Proces s” and either during or after the video, let me know what you think. 9. P lease tell me how you would rate the ease of finding the video on a scale of 1 to 10, where 1 is “very difficult to find ” and 10 is “very easy to find”
. Very difficult to find Very easy to find 1 2 3 4 5 6 7 8 9 10
a. What makes you say that? 10. Thinking about the video itself, please tell me h ow you would rate it on a scale of 1 to 10 in terms of your ability to understand the information presented in the video , where 1 is “not at all easy to understand ” and 10 is “very easy to understand.” Not at all easy to understand Very easy to u nderstand 1 2 3 4 5 6 7 8 9 10
a. What makes you say that? 11. Realistically, is this the type of information you would **a)** want to share and discuss together with your parents; **b)** watch on your own without your parents’ involvement; or **c)** forward to your paren ts to watch on their own without any other involvement on your part? **SKIP Q12 IF NO TIME** 12. Still thinking about the video, please tell me how you would rate it on a scale of 1 to 10 in terms of its effect on your confidence in being able to complete the student aid application process, where 1 is “I am not at all confident ” and 10 is “I am highly confident .” Not at all confident Highly confident 1 2 3 4 5 6 7 8 9 10
a. What makes you say that? 0:4 0

### B1 StudentAid.gov Mobile Device Usability Evaluation IDI Guide FINAL 11/ 27/1 2

Market Connections Inc. 15

b. Would your rating change if you knew your parents w ill be actively involved in complet ing the student aid application process? What would be your new rating? 13. Is this the type of in formation you would share with your friends? How exactly would you do that? 14. What do you feel are the pros and/or cons of completing any of these tasks on your mobile device, versus doing so on a PC or laptop?
a. Realistically, would you have performed any these tasks with your mobile device? Which ones? Are there any you definitely would not have tried with your mobile device? 15. Now that you have reviewed several aspects of the studentaid.gov website, do you have any final advice for the Office of Federal Student Aid to consider in order to make the site easier to use for those using a mobile device su ch as yours? That concludes our interview. Thanks very much for your input. It was very helpful! 0:45