Theory and Data-Driven Competence Evaluation with Multimodal Machine Learning--A Chinese Competence Evaluation Multimodal Dataset  
Teli Xian  
Business School, Sichuan University, Chengdu 610065, China; [EMAIL_ADDRESS] (TX. ); [EMAIL_ADDRESS] (P.D.) 1  
Peiyuan Du  
Business School, Sichuan University, Chengdu 610065, China; [EMAIL_ADDRESS] (TX. ); [EMAIL_ADDRESS] (P.D.) 1  
Chengchengchengchengchengchengchengchengchengchengchengchengchengchengchengchengchengchengchengchengchengchengchengchengchengchengchengchengchenghengchengchengchenghengchengchengchenghengchengchenghengchengchenghengchengchenghengchengchengchenghengchenghengchengchenghengchengchenghengchengchenghengchengchenghengchenghengchenghengchenghengchengchenghengchengchenghengchengchenghengchengchengchenghengchengchenghengchengchengchengchengchengchengheng  
in intercultural, critical, creative, responsible, and autonomous manners [10]. For instance, in politics, highly competent candidates receive more support [11]. In marketing, competent salespersons earn more customer trust, thus leading to more extensive sales [12]. In an organization, a CEO who even \"looks competent\" is more likely to be selected and receive more salary compensation [13]. Given the importance of competence, researchers have conducted much research on competence frameworks [1; 14; 15; 16]. [PERSON] (1993) proposed that competence is related to three factors: \"intelligence, willpower, and courage\" (1993, p. 329) [14]. [PERSON] et al. (2007) built a classical competence framework including \"perceived ability, intelligence, skill, creativity, and efficacy\" (2007, p. 77) [15]. In subsequent studies, new dimensions for competence evaluation, such as independence, education, and effectiveness, have continued to be added [16; 17; 18]. Researchers have offered valuable insights into the concept of competence, and among the scales they developed, there are four main dimensions used as measures of competence: skill, efficiency, intelligence, and capability [1; 14; 19]. However, previous research has mainly focused on measuring competence based on individuals' self-perception or others' judgment using surveys, and the hard evidence of competence-related individual traits is quite limited. Moreover, existing studies have been disproportionately focused on a series of inner traits (e.g., intelligence and capability) related to competence. However, individual competence can actually be evaluated based on both inner and outer traits. For example, among inner traits, skill, intelligence, capability, and efficiency highlight individuals' previous studies and are thus correlated to competence evaluation [14; 16; 19; 20; 21; 22; 23; 24]. Among outer traits, facial and vocal cues play an important role in competence assessment [25; 26; 27; 28]. Given the above, not only inner competence traits (i.e., skill, intelligence, capability, and efficiency) but also outer traits convey rich social information about competence evaluation. Therefore, there is a chance to evaluate perceived competence directly through easily observed individual traits. To fill these gaps, we developed a two-step empirical study that leverages machine learning and computer vision methods and proposed a framework and a predictor of competence evaluation based on multimodal data. This study aims to address the following questions:  
1. What are the multimodal variables (i.e., outer and inner traits) that affect individual competence? 2. How do large-granular multimodal competence variables and small-granular multimodal competence features contribute to competence prediction? 3. How can cutting-edge deep learning models be utilized to make highly accurate predictions of competence by utilizing visual, vocal, and textual information? To answer these questions, we conducted two studies from knowledge-driven and data-driven perspectives. In the knowledge-driven study (study 1), we first extracted 10 competence-related traits (both outer and inner traits) from the previous literature.