A robot system comprising: an emotion recognizer including: a uni-modal preprocessor having a plurality of recognizers (521, 522, 523) for modals, the plurality of recognizers (521, 522, 523) being taught to recognize emotion information of a user contained in a plurality of uni-modal input data; and a multi-modal recognizer configured to merge output data of the plurality of recognizers (521, 522, 523) for each modal, the multi-modal recognizer being taught to recognize emotion information of the user contained in the merged data; and a robot including: a communication unit configured to transmit moving image data including data of the user to the emotion recognizer coupled to the robot, wherein the emotion recognizer is taught to recognize emotion information of the user included in the moving image data, wherein the robot is configured to receive a complex emotion recognition result from the emotion recognizer, the complex emotion recognition result including respective emotion recognition results of the plurality of recognizers (521, 522, 523) for each modal and an emotion recognition result of the multi-modal recognizer, and a sound output unit configured to utter a question for checking an emotion of the user by combining two or more emotion recognition results that do not match, when the complex emotion recognition result includes the two or more emotion recognition results that do not match. an emotion recognizer including: a uni-modal preprocessor having a plurality of recognizers (521, 522, 523) for modals, the plurality of recognizers (521, 522, 523) being taught to recognize emotion information of a user contained in a plurality of uni-modal input data; and a multi-modal recognizer configured to merge output data of the plurality of recognizers (521, 522, 523) for each modal, the multi-modal recognizer being taught to recognize emotion information of the user contained in the merged data; and a uni-modal preprocessor having a plurality of recognizers (521, 522, 523) for modals, the plurality of recognizers (521, 522, 523) being taught to recognize emotion information of a user contained in a plurality of uni-modal input data; and a multi-modal recognizer configured to merge output data of the plurality of recognizers (521, 522, 523) for each modal, the multi-modal recognizer being taught to recognize emotion information of the user contained in the merged data; and a robot including: a communication unit configured to transmit moving image data including data of the user to the emotion recognizer coupled to the robot, wherein the emotion recognizer is taught to recognize emotion information of the user included in the moving image data, wherein the robot is configured to receive a complex emotion recognition result from the emotion recognizer, the complex emotion recognition result including respective emotion recognition results of the plurality of recognizers (521, 522, 523) for each modal and an emotion recognition result of the multi-modal recognizer, and a communication unit configured to transmit moving image data including data of the user to the emotion recognizer coupled to the robot, wherein the emotion recognizer is taught to recognize emotion information of the user included in the moving image data, wherein the robot is configured to receive a complex emotion recognition result from the emotion recognizer, the complex emotion recognition result including respective emotion recognition results of the plurality of recognizers (521, 522, 523) for each modal and an emotion recognition result of the multi-modal recognizer, and a sound output unit configured to utter a question for checking an emotion of the user by combining two or more emotion recognition results that do not match, when the complex emotion recognition result includes the two or more emotion recognition results that do not match.