Amanda Lockett

University of South Carolina, Columbia, SC, USA

Samaneh Borji

University of South Carolina, Columbia, SC, USA

# DIGITAL SEARCH AND HEALTH INSURANCE LITERACY IN THE U.S.

## Abstract

This research focuses on the impact of digital search on health information, primarily health insurance and health insurance literacy (HIL). Using Methods AGI, a Google search simulation software, this study examined the health insurance search of young adult university students, ages 18-25, and evaluated their literacy using the health insurance literacy measure (HILM). With the marketing of AI to mass audiences as a tool for search, this study incorporated AI into the search simulation and measured AI interaction and its impact (if any) on the HIL of the participants. By dividing participants into two groups, this study was able to compare whether those conducting the search for health insurance information utilized AI, if that use of AI was impacted by sponsored ads, and if the use of AI increased HIL.

## Introduction

The digitization of health information has transformed how individuals' access and understand healthcare resources, particularly in the context of health insurance. As artificial intelligence (AI) and large language models become increasingly integrated into search processes, there is a critical need to understand how these technologies impact health insurance literacy (HIL) among vulnerable populations. Young adults aged 18-25 represent a demographic particularly susceptible to low HIL rates, making them an ideal population for studying the intersection of digital search, AI assistance, and health insurance comprehension.

While previous research has established the relationship between internet skills and health information disparities (Jacobs et al., 2017), less attention has been paid to how emerging AI technologies might bridge or exacerbate these gaps. The introduction of AI into search processes presents both opportunities and challenges. Furthermore, comprehension of this shift in information technology is required. Thus, this research poses the following questions:

1. How does AI in search engines impact HIL levels in young adults 18-25?

## Background

While information and technology are at the center of healthcare innovation, health disparities continue to exist globally (Hoagland & Kipping, 2024). Disparities in healthcare outcomes and access have been partially attributed to disparities in health information literacy, with the internet transforming the sharing and retrieval of health information (Upadhyay et al., 2022). In 2017, 33%
of the US population used the internet to search for health information online, but the growing trend also saw an increase in health information disparity due to variables such as: internet skills, health status, family health history, race, age, education, etc. (Jacobs et al., 2017). In the US, health insurance and health insurance literacy (HIL) have historically been one of the overlooked, yet fundamental aspects of health information literacy. Quincy (2102) defines HIL as the “degree to which individuals have the knowledge, ability, and confidence to find and evaluate information about health plans, select the best plan for their own (or their family’s) financial and health circumstances, and use the plan once enrolled” (1). With the digitization of health insurance search, the disparity in health insurance literacy has increased exponentially, leaving most Americans unprepared to navigate and maximize the health insurance system (Nobles et al., 2016). This phenomenon is problematic as HIL is a strong indicator of healthcare access and outcomes based on the patient’s ability to evaluate information regarding health insurance plans, coverages and costs (Quincy, 2012). However, Paez et al. (2016) offer a health insurance literacy measure (HILM), which provides an opportunity to evaluate insurance information channels that claim to provide adequate access to health insurance information and resources.

Through the use of the HILM, this study explores the impact of current search engines and the inclusion of AI on HIL. The study demographic, undergraduate students (ages 18-25), possess particularly low HIL rates compared to other groups (Edwards et al., 2019; Nobles et al., 2018). Using a simulated Google search engine, this study examines the impact of both search bias and AI on the health insurance search process and the probability of users improving HIL. (Novin and Myers, 2017). AI has been identified as a potential solution to the complexity of health information through the provision of simplified, timely and personalized information (Volin & Ellmore, 2024). But there are major concerns about the risk of inaccuracy, biases, lack of explainability, and unsustainability (Giudici et al., 2024). This study addresses a shift in search and the criterion for successful search and evaluation after the introduction of new models and technologies. (Belkin, 2016).

## Study

This research adopts a mixed methods approach. 41 Participants conducted a search for health insurance information via a simulated Google search engine created in MethodsAGI.com. The software presented each participant with a task description to conduct a search for health insurance on behalf of a family member who could not conduct the search themselves. This task description included text boxes where participants could leave both notes and a final answer after they conducted their search [Figure 1]. Next, each participant was presented with the simulated search engine results page (SERP), which contained search results relating to health insurance [Figure 2]. The search results were selected by querying Google with different private browsers and filtering for the most frequent results and most common answer from the AI bot, which was displayed at the top right of the SERP. Each group was provided with the AI bot’s overview. This process allowed for a controlled SERP and AI bot. The original independent variable to be studied was the advertisements that appear on the SERP. The advertisements were randomly shown to about half of the participants to create two groups labeled as Group 1 sponsored (The SERP with advertisements) and Group 2 non-sponsored (The SERP with no advertisements). Site engagement was measured through quantitative and qualitative participant feedback. Data collection included: clicks, video recording, screen recording, and search queries, along with each participant’s notes and final answer. Participants were also asked to perform a cognitive walkthrough or to “think aloud” while conducting their search. Both groups (sponsored and non-sponsored) received 10
search engine results as sources for health insurance information. Sources were selected by the research team based on real time Google search engine results: Obama care, Sidecar Health, healthcare.gov, Healthinsurance.com, Blue Cross Blue Shield, AffordableHealthinsurance.com and NIH. Search results were ranked by the research team based on a consensus of most useful to least useful.

User ID: Enter User ID

Task: Imagine you have a family member who cannot conduct a Google search. I'd like you to conduct the search to help the family member understand health insurance. Write them a message in the final answer box.

START SESSION

Notes:

Final Answer:

<figure><img src="image_1.png" /><figcaption>**Figure 1.** Methods AGI prompt page</figcaption></figure>

**Sponsored**

<figure><img src="image_2.png" /></figure>

obamacare-registration.org

https://obamacare-registration.org/

Obamacare 2024 Open Enrollment - Open Enrollment is Available

Register today for Ob amacare affordable health insurance. Open enrollment is now available

**Sponsored**

<figure><img src="image_3.png" /></figure>

Sidecar Health

https://sidecarhealth.com/

Sidecar Health: Health insurance the way it should be

Looking for a new kind of health insurance? Sidecar Health offers transparent & affordable health insurance for individuals, and you can choose any doctor ...

**Sponsored**

<figure><img src="image_4.png" /></figure>

HealthCare.gov

https://finder.healthcare.gov/

Health Insurance Plan Finder - HealthCare.gov

This Plan Finder website is provided by the federal government to help you find private health plans available outside the Health Insurance Marketplace. We want ...

**Sponsored**

<figure><img src="image_5.png" /></figure>

Affordable Health Plans. Compare & Save. It's Easy

https://health1insurance.com/

Health1Insurance

Free 2024 health insurance quotes online in minutes with no obligation. Compare Affordable health plans and enroll online or over the phone.

<figure><img src="image_6.png" /></figure>

Blue Cross Blue Shield

<figure><img src="image_7.png" /></figure>

Health Insurance

I

Health insurance or medical insurance (also known as medical aid in South Africa) is a type of insurance that covers the whole or a part of the risk of a person incurring medical expenses.

As with other types of insurance, risk is shared among many individuals. By estimating the overall risk of health risk and health system expenses over the risk pool, an insurer can develop a routine finance structure, such as a monthly premium or payroll tax, to provide the money to pay for the health care benefits specified in the insurance agreement.

<caption>**Figure 2.** Methods AGI simulated Google search page (Group 1 sponsored)</caption>

## Data Analysis

To assess the HIL of participants, the HILM will be used (Paez et al, 2016). The HILM is a conceptual measure that includes four categories: knowledge, information seeking, document
literacy, and cognitive skills, which eventually measure the participants self-efficacy (Paez et al,
2016).

Health Insurance Literacy Measure - Categories and Subcategories:

**Knowledge:** insurance terms, types of healthcare services, insurance concepts, beneficiary rights

**Information Seeking:** formulation and articulate questions, locate health plan information,
evaluate credibility of information, navigate information sources

**Document Library:** read and follow written instructions, interpret summary of benefits and explanation of benefit documents, use schedules, complete health insurance forms

**Cognitive Skills:** assess preventative care needs and personal risks, project healthcare utilization, apply health plan benefits, calculate cost sharing, assess value weighing costs versus benefit of insurance relative to health service needs

With the HILM, this analysis will examine whether each participant utilized their health insurance information search/sources to address a particular HILM subcategory. Moreover, this analysis will examine if the participants used AI to assist in the interpretation/discovery of that HILM subcategory information. The information interaction will be identified through website visits or asked/declared aloud through the participant's cognitive walkthrough. By utilizing the HILM, our data findings can reveal whether MethodsAGI.com and the chosen health insurance sources facilitated HIL for the participants. The data will also reveal what aspects of the search engine impacted the HIL, including: AI usage, advertisements, and demographics.

Site visitation was recorded via Methods AGI analytics and screen recordings, which were used to record cognitive walk throughs. Previous data analyses examined the impact of advertisements on health insurance search. Both Group 1 (sponsored) and Group 2 (non-sponsored) utilized .gov sites the most. But after experiencing sponsored advertisements, Group 1 was more likely to utilize BCBS sites, information, and marketing verbiage [Figure 4 and 5]. However, regardless of advertisement both groups looked to AI at least once in search of information.

This new HILM data analysis will provide a more precise examination of the participant's HIL levels as well as their engagement with AI in relationship to the development of HIL.
<figure><img src="image_8.png" /><figcaption>Figure 4. Previous Methods AGI Analytics. BCBL (Websites 5 & 6) with majority first clicks</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>Figure 5. Previous Methods AGI data analytics longest time spent on websites 3 & 10 (.gov)</figcaption></figure>

## References

* Belkin, N. J. (2016). People, Interacting with Information1. *SIGIR Forum*, 49(2), 13–27.
https://doi.org/10.1145/2888422.2888424

* Edward, J., Wiggins, A., Young, M. H., & Rayens, M. K. (2019). Significant Disparities Exist in Consumer Health Insurance Literacy: Implications for Health Care Reform. *HLRP: Health Literacy Research and Practice*, 3(4). https://doi.org/10.3928/24748307-20190923-01
Giudici, P., Centurelli, M., & Turchetta, S. (2024). Artificial Intelligence risk measurement. *Expert Systems with Applications*, 235, 121220.
https://doi.org/10.1016/j.eswa.2023.121220

Hoagland, A., & Kipping, S. (2024). Challenges in Promoting Health Equity and Reducing Disparities in Access Across New and Established Technologies. *Canadian Journal of Cardiology*, 40(6), 1154–1167. https://doi.org/10.1016/j.cjca.2024.02.014

Jacobs, W., Amuta, A. O., & Jeon, K. C. (2017). Health information seeking in the digital age: An analysis of health information seeking behavior among US adults. *Cogent Social Sciences*, 3(1), 1302785. https://doi.org/10.1080/23311886.2017.1302785

Nobles, A. L., Curtis, B. A., Ngo, D. A., Vardell, E., & Holstege, C. P. (2019). Health insurance literacy: A mixed methods study of college students. *Journal of American College Health*, 67(5), 469–478. https://doi.org/10.1080/07448481.2018.1486844

Novin, A., & Meyers, E. (2017). Making Sense of Conflicting Science Information: Exploring Bias in the Search Engine Result Page. *Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval*, 175–184.
https://doi.org/10.1145/3020165.3020185

Novin, A., & Meyers, E. M. (2017). Four Biases in Interface Design Interactions. In A. Marcus & W. Wang (Eds.), *Design, User Experience, and Usability: Theory, Methodology, and Management* (pp. 163–173). Springer International Publishing.
https://doi.org/10.1007/978-3-319-58634-2_13

Paez, K. A., Mallery, C. J., Noel, H., Pugliese, C., McSorley, V. E., Lucado, J. L., & Ganachari, D. (2014). Development of the Health Insurance Literacy Measure (HILM): Conceptualizing and Measuring Consumer Ability to Choose and Use Private Health Insurance. *Journal of Health Communication*, 19(sup2), 225–239.
https://doi.org/10.1080/10810730.2014.936568

Quincy, L. (2012). Measuring health insurance literacy: A call to action. *Yonkers: Consumers Union*.

Upadhyay, S. S. N., Merrell, L. K., Temple, A., & Henry, D. S. (2022). Exploring the Impact of Instruction on College Students’ Health Insurance Literacy. *Journal of Community Health*, 47(4), 697–703. https://doi.org/10.1007/s10900-022-01096-2

Volin, B. & Ellmore, K (2024) Unlocking better health: Why health literacy needs a digital revolution. Managed Healthcare Executive. Retrieved, December 2, 2024