###### Abstract  
With the deep integration of psychology and artificial intelligence technology and other related technologies, eye control technology has achieved certain results at the practical application level. However, it is found that the accuracy of the current single-modal eye control technology is still not high, which is mainly caused by the inaccurate eye movement detection caused by the high randomness of eye movements in the process of human-computer interaction. Therefore, this study will propose an intent recognition method that fuses facial expressions and eye movement information and expects to complete an eye control method based on the fusion of facial expression and eye movement information based on the multimodal intent recognition dataset, including facial expressions and eye movement information constructed in this study. Based on the self-attention fusion strategy, the fused features are calculated, and the multi-layer perceptron is used to classify the fused features, so as to realize the mutual attention between different features, and improve the accuracy of intention recognition by enhancing the weight of effective features in a targeted manner. In order to solve the problem of inaccurate eye movement detection, an improved YOLOv5 model was proposed, and the accuracy of the model detection was improved by adding two strategies: a small target layer and a CA attention mechanism. At the same time, the corresponding eye movement behavior discrimination algorithm was combined for each eye movement action to realize the output of eye behavior instructions. Finally, the experimental verification of the eye-computer interaction scheme combining the intention recognition model and the eye movement detection model showed that the accuracy of the eye-controlled manipulator to perform various tasks could reach more than 95 percent based on this scheme. eye-computer interaction; intent recognition; facial expression; attention mechanisms +
Footnote †: journal: Applied sciences  
but small expression amplitude or a lack of expression may cause the system to be unable to accurately recognize user intent [4]. Single-modal data may not provide enough information to understand complex user behavior; for example, in the absence of contextual information, a rapid eye movement may be misinterpreted as a spontaneous eye movement rather than an expression of intent [5]. Given the limitations of single-modal systems, researchers began experimenting with multiple-mode fusion data supplementation methods to improve the accuracy and reliability of interactions. By combining information from multiple modes, such as eye movements and facial expressions, a multimodal approach can capture the user's interaction intent more comprehensively. For example, [PERSON] and [PERSON] [6] used a hidden Markov model to design an intention prediction mechanism that can identify a driver's intention more quickly and accurately. [PERSON] et al. [4] used eye tracking technology instead of a mouse on the computer, and the interactor could choose different objects to interact with. In 2023, Apple's Vision Pro integrated eye tracking technology into the core interaction, using a multimodal interaction model of \"eye tracking + gesture + voice\", which raised the importance of eye movement interaction to an unprecedented level. This multimodal interaction mode uses the movement of the eyes to select the object of interaction and uses gestures to determine or switch, which significantly improves the naturalness and intuitiveness of human-computer interaction. However, multimodal fusion also brings new challenges, including hardware systems that require more efficient fusion algorithms and higher operating speeds [7].