Blinded by Context: Unveiling the Halo Effect of MLLM in AI Hiring (2025)

Abstract: This study investigates the halo effect in AI-driven hiring evaluations using Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs). Through experiments with hypothetical job applications, we examined how these models’ evaluations are influenced by non-job-related information, including extracurricular activities and social media images. By analyzing models’ responses to Likert-scale questions across different competency dimensions, we found that AI models exhibit significant halo effects, particularly in image-based evaluations, while text-based assessments showed more resistance to bias. The findings demonstrate that supplementary multimodal information can substantially influence AI hiring decisions, highlighting potential risks in AI-based recruitment systems.

## introduction job recruitment processes have traditionally been the domain of human decision-makers, who evaluate candidates based on resumes, interviews, and other relevant information. with recent advancements in artificial intelligence, a growing movement has emerged to leverage ai to enhance the efficiency and objectivity of these evaluations. in particular, large language models (llms) and multimodal large language models (mllms) are increasingly being positioned as automated evaluators in tasks that demand structured judgment and decision-making. these models are now being deployed across a range of domains-including finance @xcite , law @xcite , peer review @xcite , and recruitment @xcite )-where they assess complex multimodal inputs and generate evaluative outputs that may influence consequential outcomes. however, the implementation of ai in these evaluative settings, particularly in recruitment, must be approached with caution due to inherent biases. as llms and mllms take on greater responsibilities, concerns about bias in their decision-making have intensified. research has identified various forms of bias, including gender bias @xcite , racial bias @xcite , and bias based on physical attributes @xcite . in the context of job hiring, one particularly significant cognitive bias is the halo effect-a phenomenon in which an overall impression of an individual influences judgments about unrelated attributes @xcite . this bias is well-documented in human decision-making, where positive impressions based on appearance, personality, or background often lead to inflated assessments of professional competence @xcite @xcite . while the halo effect has been extensively studied in human evaluations, its impact on ai-driven assessments remains largely unexplored. @xcite suggests that llms, due to their holistic information processing, may overvalue certain attributes based on irrelevant cues. some studies also indicate that llms may favor high-status authors or familiar individuals @xcite . as mllms become more prevalent in hiring processes, it is essential to examine whether multimodal inputs induce similar biases in ai-based evaluations. in this work, we present the first comprehensive study of the halo effect in ai-driven recruitment (figure 1 ). we designed a structured hiring framework covering multiple job roles and resumes of varying quality. to assess the impact of multimodal information, we built a dataset comprising when evaluated with the resume alone, candidate a is rated as moderately competent (score: 6.2). however, the addition of multimodal cues-such as a confident appearance or professionallooking photo-leads to a notably higher evaluation (score: 8.7). this illustrates a halo effect, where seemingly unrelated yet positively perceived information indirectly inflates judgments of competence and suitability. our main contributions are as follows: ## related work to better understand how biased judgments emerge in ai-assisted hiring, we review literature on 1) the halo effect in both human and machine evaluations, and 2) how llms and mllms function as evaluators, and the biases they may introduce. ## research design to empirically investigate how the halo effect manifests in ai-driven hiring, we designed a controlled job application and evaluation setup. ## experiment based on the previously defined job application setup and evaluation protocol, we ran experiments to examine how multimodal cues influence ai models' hiring judgments. ## results this section presents the experimental results on the halo effect induced by two types of supplementary information: text and image. ## multifaceted analysis of the halo effect to gain a deeper understanding of how the halo effect manifests in ai hiring, we examine its patterns across multiple dimensions-including scoring categories, demographic variations, and video-based contextual settings. ## conclusion in this study, we explored how various multimodal language models respond to job-irrelevant information across different contexts and input types, analyzing halo effects from multiple perspectives. additionally, the study examined the influence of demographic variations and the impact on specific score categories, providing a comprehensive understanding of bias and susceptibility in ai-driven evaluation systems. these findings emphasize the need for ongoing scrutiny and refinement of ai evaluation systems to ensure fairness and reliability in high-stakes decision-making contexts.