A user authentication device, comprising: a card reader configured to obtain a face image stored in a card when the card is recognized, wherein the face image is encoded in the card; an infrared light source configured to emit infrared light toward a user; an infrared camera configured to obtain an infrared image comprising a facial region of the user; a color camera configured to obtain a color image comprising the facial region of the user; a storage; and a controller configured to: store the obtained infrared image, the obtained color image and the obtained face image in the storage temporarily; perform a liveness test based on the obtained infrared image; when a result of the liveness test is determined to be successful, extract a first facial feature from the encoded face image obtained through the card reader, extract a second facial feature from at least one of the obtained color image and the obtained infrared image, and determine a result of face authentication based on a result of comparing the first facial feature and the second facial feature; and when the result of the face authentication is determined, perform a control to delete the obtained infrared image, the obtained color image and the obtained face image from the storage. a card reader configured to obtain a face image stored in a card when the card is recognized, wherein the face image is encoded in the card; an infrared light source configured to emit infrared light toward a user; an infrared camera configured to obtain an infrared image comprising a facial region of the user; a color camera configured to obtain a color image comprising the facial region of the user; a storage; and a controller configured to: store the obtained infrared image, the obtained color image and the obtained face image in the storage temporarily; perform a liveness test based on the obtained infrared image; when a result of the liveness test is determined to be successful, extract a first facial feature from the encoded face image obtained through the card reader, extract a second facial feature from at least one of the obtained color image and the obtained infrared image, and determine a result of face authentication based on a result of comparing the first facial feature and the second facial feature; and when the result of the face authentication is determined, perform a control to delete the obtained infrared image, the obtained color image and the obtained face image from the storage. store the obtained infrared image, the obtained color image and the obtained face image in the storage temporarily; perform a liveness test based on the obtained infrared image; when a result of the liveness test is determined to be successful, extract a first facial feature from the encoded face image obtained through the card reader, extract a second facial feature from at least one of the obtained color image and the obtained infrared image, and determine a result of face authentication based on a result of comparing the first facial feature and the second facial feature; and when the result of the face authentication is determined, perform a control to delete the obtained infrared image, the obtained color image and the obtained face image from the storage.