# A Multimodal Affective Sensing Model for Constructing a Personality-Based Financial Advisor System
## 1 Introduction  
Automated analysis of human affective behavior is attracting increasing attention from researchers in psychology, computer science, linguistics, neuroscience, and other related disciplines. With the help of advanced artificial intelligence techniques, multimodal sensing systems using pattern recognition methods for human behavior analysis can operate the fusion of measurements from different sensor modalities [1]. Most of the research work focuses on the application of emotion recognition, such as [2]. This advance implies that the application of human-computer interaction (HCI) relies on knowledge about human emotional experience and about the relationship between emotional experience and emotional expression [3]. More recently, several multimodal sensing systems were directed at developing applications for personality trait recognition, such as for job interviews [4, 5], work stress tests [6], public speaking [7, 8], consumer behavior [9], and verbal rating systems [10]. These applications achieved good results in various fields in terms of analyzing personality traits using individual behavioral characteristics. Another application area that could benefit from an automatic understanding of an individual's personality traits is the wealth management industry. The wealth management industry faces a shift in the role of financial advisors. At the same time, automation technology spawned tools to make this transition more seamless. Using personality trait prediction, financial advisors can provide quality service to customers while using reasonable resources. An increasing amount of research suggests that emotional changes influence the investment process [11; 12], and that deeper personality traits also play an important role in the investment decision-making process [13]. Interestingly, most of the past research in finance used questionnaires to understand people's personality traits and used statistical methods to discover which specific personality traits are associated with, and influence, an individual's investment behavior. [14; 15; 16; 17; 18]. However, few studies chose to directly analyze changes in customer behavior applied to the financial sector. The personality characteristics of bank customers can be analyzed through personality computing. Therefore, in addition to the traditional use of questionnaires provided by banks to investigate customer risk tolerance, we can also use the developed system to collect additional information about customer interaction behavior, financial knowledge, consumption habits, and personality. An example of an automatic personality-recognition system workflow for prediction of a bank client's economic behavior is illustrated in Figure 1. As shown in Figure 1, by collecting the features of human external behavior signals, it is possible to quantify the performance of human behavior, analyze the personality characteristics of individuals, help financial advisors understand the real risk tolerance of customers, and then provide appropriate investment planning and direction. Therefore, in this study, we propose a personalized multimodal affective sensing system to understand the investment attributes of financial customers in wealth management. The goals of this study are listed as follows:  
1. To digitize the behavioral characteristics of individuals and explore how to develop appropriate investment plans according to the analyzed clients' personalities. 2. To assist financial advisors in helping clients with wealth management, we developed a multimodal personality-recognition system to collect subtle variations in client-generated conversational data. By analyzing the information on customer interaction behavior, we can help them adjust their investment plans in a timely manner.