Electronic device and method for registering new user through authentication by registered user

An electronic device and method are disclosed herein. The electronic device includes a speaker, microphone, processor and memory storing instructions, which implement the method, including: determining whether registration of a first user is required based on a first voice signal obtained through a microphone of the electronic device, when registration of the first user is required, requesting authentication of the first user by a second user preregistered at the electronic device, and when information authenticating the first user by the second user is received, registering the first user based on the received information.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is based on and claims priority under 35 U.S.C. § 119 to Korean Patent Application No. 10-2018-0091772, filed on Aug. 7, 2018, in the Korean Intellectual Property Office, the disclosure of which is herein incorporated by reference in its entirety.

BACKGROUND

Field

Certain embodiments relate to a method and an electronic device for registering a new user through authentication by a registered user.

Description of Related Art

With the enhancement of digital technology, electronic devices capable of providing various speech recognition services based on speech recognition technology have advanced in complexity and utility. Thanks to the development of speech recognition technology, electronic devices can provide speech recognition services, allowing users to formulate inputs and commands for the electronic devices based on “Natural Language Understanding” (NLU). Speech recognition services may be related to technology relevant to, for example, speech recognition, artificial intelligence (AI), and internet of things (IoT).

SUMMARY

When a user registers himself/herself to have authority regarding an electronic device providing a speech recognition service, the user should follow a procedure of generating an account or registering an account by using a user terminal. In this case, the user is required to input information necessary for the account through the user terminal, which may cause inconvenience. In addition, when a user does not own a user terminal, there is a problem that the user cannot register to use an electronic device providing a speech recognition service.

Certain embodiments of the disclosure provide an electronic device and a method for registering a new user through authentication by a registered user.

According to certain embodiments of the disclosure, an electronic device includes: a speaker, a microphone, and at least one processor, and a memory storing programming instructions, the programming instructions executable by the at least one processor to cause the electronic device to: determine whether registration of a first user is required based on a first voice signal obtained through the microphone, when registration of the first user is required, requesting authentication of the first user by a second user preregistered at the electronic device, and when information authenticating the first user by the second user is received, registering the first user based on the received information.

In addition, according to certain embodiments of the disclosure, an operation method of an electronic device includes: determining whether registration of a first user is required based on a first voice signal obtained through a microphone of the electronic device, when registration of the first user is required, requesting authentication of the first user by a second user preregistered at the electronic device, and when information authenticating the first user by the second user is received, registering the first user based on the received information.

DETAILED DESCRIPTION

An electronic device (e.g., a user device, an artificial intelligence (AI) device, an Internet of Things (IoT) device) according to certain embodiments may be one of various types of electronic devices. The electronic devices may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance. According to an embodiment of the disclosure, the electronic devices are not limited to those described above.

FIG. 1illustrates an example of a network environment including an AI device, an AI server, an account server, and a user device according to certain embodiments.

Referring toFIG. 1, the network environment100includes the AI device101, the AI server102, the account server103, and the user device105.

The AI device101may perform functions of an AI agent or an AI assistant. The AI device101may include, for example, an AI speaker, but is not limited thereto. The AI device101may be implemented as various types of devices capable of performing the functions of the AI agent. For example, the AI device101in a home may be implemented through one of home appliances or may be implemented as a family hub.

The AI device101may be installed in a home to function as an AI assistant for a family, or may be installed in a public place to be used as a sharing device for a plurality of users. The functions or purpose of the AI device is not limited. For example, the AI device101may be installed in a private space such as a hotel room, and in this case, a user using the AI device101may continuously change.

The AI device101may register a user who wishes to use the AI device101. The AI device101may give a registered user authority to use the AI device101by registering the user. For example, an action of registering a user by the AI device101may include an action of generating an account of the user. The action of registering a user by the AI device101may be performed based on a conversational dialogue with the user based on a voice signal, and based on communication with a server (for example, at least one of the AI server102or the account server103). The AI device101may store information regarding an account of a registered user (for example, a profile) in a server (for example, at least one of the AI server102or the account server103) connected with the AI device101. However, this should not be considered as limiting, and the AI device101may directly store information regarding an account of a registered user in a memory of the AI device101.

The user registered to use the AI device101may have authority of a designated level or higher with respect to the AI device101. For example, the AI device101may provide a customized function to the registered user by using the information (for example, a profile, voiceprint information, a nickname, an appellation, etc.) regarding the account of the registered user. The function that the AI device101can provide to the registered user may be different from a function that the AI device101provides to an unregistered user. For example, the AI device101may provide weather information or time information to all of the registered user and the unregistered user. However, the AI device101may additionally provide the registered user with at least one of a function of informing a personal schedule of the registered user, a function of calling registered another user, a function of transmitting a message to registere another user, or a function of controlling an IoT device associated with the AI device101. The AI device101may not provide the functions which are additionally provided to the registered user as described above to the unregistered user.

The AI device101may register a plurality of users and may provide customized services to the registered users, respectively. For example, the AI server102(or the account server103) connected with the AI device101may store voiceprint information of users to register a user. The AI device101(or the AI server102) may identify (or recognize) the plurality of users by using the voiceprint information. Accordingly, the AI device101may receive a voice input such as “Let me know ‘my’ schedule today.” and may identify a user corresponding to the voice input among the registered users, and may provide a customized service to the user.

The user which will be described in the disclosure may include a first user and a second user. The first user may be a new user who is not registered to use the AI device101. In certain embodiments, the AI device101may perform actions of identifying the first user who is not registered, and of registering the first user. For example, the AI device101may perform an action of generating an account of the first user for use of the AI device101.

The second user may be a user who is already registered to use the AI device101. For example, an account of the second user may be generated and/or registered for use of the AI device101. Information regarding the account of the second user may be stored in a server (for example, at least one of the AI server102or the account server103) connected with the AI device101. The account of the second user for using the AI device101may be generated and/or registered through the user device105of the second user. The user device105of the second user may include, for example, a smartphone, a wearable device, a tablet PC, a computer device, or a portable multimedia device which is owned by the second user, but is not limited thereto.

In certain embodiments, the AI device101may request the already registered second user to authenticate the first user to register the first user. The AI device101may generate an account of the first user to register the first user, based on the first user being authenticated by the second user. The action of authenticating the first user by the second user may include at least one of an action of authenticating the first user by using a voice input of the second user obtained through a microphone of the AI device101(for example, an AI speaker), or an action of obtaining authentication information from the second user through the user device105of the second user.

The AI server102may be connected with the AI device101through a communication network, and may identify (recognize) utterance of a user or may identify (generate or select) a response corresponding to user's utterance to allow the AI device101to have conversational dialogues with the user.

For example, the AI server102may receive a user voice input from the AI device101through the communication network, and may change the user voice input to text data. The AI server102may identify (generator or select) information regarding an action (or operation) for performing a function based on the text data, or information regarding a parameter necessary for executing the action. The AI server102may generate a sequence of states for performing a task requested by a user. The task may be any action that can be provided by AI software (or application). The task may include, for example, searching a schedule, providing weather information, controlling an external electronic device (for example, an IoT device) associated with the AI device101, or calling an electronic device (for example, the user device105) registered at the AI device101. The actions of the AI server102as described above may be provided by an AI system. The AI system may be a rule-based system or a neural network-based system (for example, a feedforward neural network (FNN), a recurrent neural network (RNN)). Alternatively, the AI system may be a combination of the above-described systems or other AI systems. According to an embodiment, the task or the action or a parameter for executing the action may be selected from a pre-defined set or may be generated in real time in response to a user request.

The AI server102may include, for example, an automatic speech recognition (ASR) module, a natural language understanding (NLU) module, a natural language generator (NLG) module, or a text-to-speech (TTS) module. The AI server102may include a communication circuit, a memory, and a processor. The processor may drive the ASR module, the NLU module, the NLG module, and the TTS module by executing a command stored in the memory. The AI server102may exchange data (or information) with an external electronic device (for example, the AI device101, the account server103, the user device105) through the communication circuit.

According to an embodiment, the AI server102may convert a user input received from the AI device101into text data by using the ASR module. For example, the ASR module may include an utterance recognition module. The utterance recognition module may include an acoustic model and a language model. For example, the acoustic model may include information regarding a vocalization, and the language model may include unit phoneme information and information regarding a combination of unit phoneme information. The utterance recognition module may convert user utterance into text data by using the information regarding the vocalization and the information regarding the unit phoneme information. Information regarding the acoustic model and the language model may be stored in, for example, an automatic speech recognition database (ASR DB).

According to an embodiment, the AI server102may understand a user intent by performing syntactic analysis or sematic analysis by using the NLU module. The syntactic analysis may be performed by dividing a user input into grammatical units (for example, words, phrases, morphemes, etc.), and understanding what grammatical element each divided unit has. The semantic analysis may be performed by using sematic matching, rule matching, formula matching, or the like. Accordingly, the NLU module may obtain a domain and an intent corresponding to the user input, or a parameter (or slot) necessary for representing the intent.

According to an embodiment, the NLU module may understand the meaning of a word extracted from the user input by using linguistic characteristics (for example, grammatical elements) of the morpheme, phrase, or the like, and may determine the intent of the user by matching the understood meaning of the word with the domain and the intent.

According to an embodiment, the NLU module may determine the user's intent by using a natural language recognition database in which linguistic characteristics for understanding the intent of a user input are stored. According to another embodiment, the NLU module may determine the user's intent by using a personal language model (PLM). For example, the NLU module may determine the user's intent by using personal information (for example, information regarding a user account, a list of contacts, a music list). The PLM may be stored in the natural language recognition database.

According to an embodiment, the NLU module may determine an action for performing a function corresponding to the user input, or a parameter necessary for executing the action, based on the intent of the user input and the parameter. The AI server102may transmit information regarding the action to the AI device101or another external device (for example, an IoT server). For example, the IoT server may control an IoT device based on the information regarding the action being received.

According to an embodiment, the AI server102may change designated information into a text form by using the NLG module. The information changed to the text form may be a form of utterance of natural language. The designated information may be, for example, information regarding an additional input, information informing completion of the action corresponding to the user input, or information guiding an additional input of the user (for example, feedback information regarding the user input). The information changed to the text form may be transmitted to the AI device101or may be transmitted to the TTS module and may be changed to a voice form.

According to an embodiment, the AI server102may change information in a text form to information in a voice form by using the TTS module. The TTS module may receive information in a text form from the NLG module, may change the information in the text form to information in a voice form, and may transmit the information to the AI device101. The AI device101may output the information in the voice form to a speaker.

According to an embodiment, one or more of the ASR module, the NLU module, the NLG module, or the TTS module may be implemented as one module in the AI server102. Alternatively, one or more of the ASR module, the NLU module, the NLG module, or the TTS module may be implemented in the AI device101. Accordingly, at least some function of the AI server102may be performed in the AI device101.

The AI server102may perform voiceprint recognition with respect to a user's voice signal received from the AI device101. The AI server102may extract voiceprint information from the user's voice signal. The AI server102may store the voiceprint information in the database of the AI server102, or may temporarily store the voiceprint information. For example, the AI server102may store the voiceprint information of a user registered to use the AI device101in association with the account of the user. The AI server102may identify a user uttering a voice signal among one or more registered users, based on the database of the voiceprint information. For example, when the AI server102identifies the user uttering the voice signal as a new user who is not registered, the AI server102may temporarily store the voiceprint information of the voice signal.

The AI server102may store at least part of information (for example, a profile) regarding the account of a user registered for the AI device101. For example, the AI server102may store voiceprint information of registered users, a relationship between registered users, and appellations that registered users call each other (for example, nicknames), as the information (for example, the profile) regarding the account of the registered user. For example, when the AI device101is installed in a home, family members may be registered as users of the AI device101, and the AI server102may store a relationship or appellations of family members (for example, mother, son, nicknames such as “gwiyomi”, which is Korean for a “cutie”).

The account server103may store an account related to the user device105. For example, the account server103may store a last name, a first name, a birth date, and an email of the user of the user device105. The account server103may provide information regarding the account related to the user device105to the AI server102.

The information regarding the account of the user registered for the AI device101may be stored in the AI server102, or may be stored in the account server103. For example, a part of information (for example, a profile) regarding the account of a registered user may be stored in the AI server102, and the other part may be stored in the account server103. According to an embodiment, the AI server102and the account server103may be implemented as one device (for example, a server104). According to another embodiment, the AI server102and the account server103may be integrated into the AI device101. For example, the information regarding the account of the user registered for the AI device101may be stored in the AI device101. In certain embodiments, at least part of the functions of the AI server102and the account server103may be implemented through the AI device101.

FIG. 2illustrates an example of a functional configuration of the AI device101according to certain embodiments. The AI device101may be a device which performs the functions of an AI agent, and for example, may include an AI speaker.

Referring toFIG. 2, the AI device101may include a processor220, a communication module290, a memory230, and an audio module270. However, this should not be considered as limiting, and at least one of the above-described components may be omitted from the AI device101or one or more other components may be added.

The audio module270may convert a sound into an electric signal, and to the contrary, may convert an electric signal into a sound. The audio module270may include a microphone271as an input device, and may include a speaker272as an output device. The audio module270may obtain a sound from the outside through the microphone271. In certain embodiments, the audio module270may obtain a voice signal of a first user and a voice signal of a second user through the microphone271. The audio module270may convert an analogue audio signal corresponding to the obtained sound into a digital audio signal. The audio module270may perform various processing operations with respect to the digital audio signal. For example, the audio module270may perform a noise processing operation (for example, noise or echo attenuation) or a processing operation related to extraction of feature points with respect to one or more digital audio signals. In another example, the audio module270may transmit the digital audio signal to the processor220, such that the processor220can perform the above-described processing operations. In still another example, at least part of the above-described processing operations may be performed by the AI server102. In this case, the audio module270or the processor220may transmit the digital audio signal to the AI server102though the communication module290, such that the AI server102can perform the above-described processing operations.

The audio module270may convert a digital audio signal into an analogue audio signal. For example, the audio module270may convert a digital audio signal received from the AI server102(for example, a digital audio signal processed by the processor220) into an analogue audio signal. In certain embodiments, the digital audio signal received from the AI server102may correspond to a response of the AI device101to have conversational dialogues with the first user or the second user. The audio module270may output the converted analogue audio signal to the outside through the speaker272.

The AI device101may have conversational dialogue with the unregistered first user or the already registered second user based on a voice by inputting through the microphone271and outputting through the speaker272as described above.

The communication module290may establish a communication link between the AI device101and an external electronic device (for example, the AI server102, the account server103, or the user device105), and may perform communication through the established communication link. For example, the communication module290may transmit data corresponding to a user's voice signal obtained through the microphone271to the AI server102. The communication module290may receive, from the AI server102, data corresponding to a voice signal responding to the user's voice signal.

The processor220may control overall operations of the AI device101. The processor220may receive commands of the other components (for example, the audio module270, the communication module290, the memory230, or the like), and may interpret a received command, and may perform calculation or process data according to the interpreted command. The processor220may be implemented by software, may be implemented by hardware such as a chip, a circuitry, or the like, or may be implemented by a combination of software and hardware. The processor220may be one processor or may be a combination of a plurality of processors. A detailed operation of the processor220(or the AI device101) according to certain embodiments will be described in detail below with reference toFIGS. 3 to 10.

The memory230may refer to a set of one or more memories. The memory230may execute commands stored in the memory230based on signaling with the processor220. The memory230may store data and/or commands received or generated from the other components (for example, the processor220, the communication module290, the audio module270). According to a certain embodiment, the memory230may store at least part of information (for example, a profile) regarding the account of a user registered to use the AI device101. For example, when the AI server102(or the account server103) is integrated into the AI device101and is implemented through the AI device101, the memory230may store the information (for example, a profile) regarding the account of the user registered to use the AI device101in the form of a database.

FIG. 3illustrates an example of a signal flow among the AI device101, the AI server102, and the account server103according to certain embodiments. According to an embodiment, the AI server102and the account server103may be integrated into one server104.

Referring toFIG. 3, in operation301, the AI device101may obtain a voice signal through the microphone271. The voice signal may be speech uttered by a certain user (for example, the first user). According to an embodiment, the voice signal of the first user may also be an utterance or speech outputted from another electronic device (for example, another AI speaker).

In operation302, the AI device101may transmit data (for example, a digital audio signal) corresponding to the voice signal to the AI server102through the communication module290. The AI server102may receive the data corresponding to the voice signal that the AI device101obtained from the user.

In operation303, the AI server102may perform voiceprint recognition by using the received data. The voiceprint recognition may include identifying a voice corresponding to the voice signal. The AI server102may store voiceprint information for users already registered to use the AI device101. The AI server102may identify whether the user uttering the voice signal is a registered user or an unregistered user, based on attempting to match the voice signal with the stored voiceprint information.

In operation304, the AI server102may identify that a voiceprint corresponding to the voice signal is an unregistered voiceprint based on the voiceprint recognition. In other words, the AI server102may identify that the first user uttering the voice signal is an unregistered user. The AI server102may store (or temporarily store) the voiceprint information in response to it being identified that the voiceprint corresponding to the voice signal is an unregistered voiceprint.

In operation305, the AI server102may transmit an account generation signal to the AI device101, based on it being identified that the voiceprint corresponding to the voice signal is an unregistered voiceprint, such that the AI device101proceeds with an account generation process for the first user (who uttered the voice signal).

In some embodiments, the AI device101may be an AI speaker which is installed in a home. The AI server102may accumulate data or voiceprint information corresponding to the voice signal over a designated period (for example, 1 week) to identify whether the first user uttering the voice signal is a family member or a temporary guest. The AI server102may identify that the first user is a family member in response to the voiceprint of the first user being identified with a designated frequency, over a designated period, or a designated number of times or more. The AI server102may transmit the account generation signal for generating an account of the first user to the AI device101in response to the voiceprint of the first user being identified with the designated frequency, over the designated period, or the designated number of times or more.

According to an embodiment, the account generation signal may correspond to audio data (for example, a statement) for outputting through the speaker272in the AI device101. For example, the account generation signal may be audio data such as “You are new here. Who are you?” or “Do you want me to register?” The AI device101may perform a registering process of the first user based on audio data received from the AI server102thereafter.

According to another embodiment, the account generation signal may be a control signal for causing the AI device101to start the account generation process of the first user. The AI device101may output a designated voice signal to generate the account of the first user in response to the account generation signal (that is, the control signal) being received. For example, the voice signal to be outputted from the AI device101may be “You are new here. Who are you?” or “Do you want me to begin registration?”, and may be generated at the AI device101or may be pre-designated and stored.

In operation306, the AI device101may output the speech prompting generation of the account of the first user in response to the account generation signal being received from the AI server102. The speech outputted from the AI device101may be, for example, a statement inducing generation of the account of the first user (for example, “Do you want me to register?”), or a statement asking for information necessary for generation of the account of the first user (for example, “What is your name?”), but this should not be considered as limiting.

In operation307, the AI device101may receive a speech of the first user responding to the speech outputted for generation of the account of the user. However, this should not be considered as limiting. The AI device101may receive a voice signal of the second user as a response to the speech outputted for generation of the account of the first user. For example, in response to the question of the AI device101, “Do you want me to begin registration?”, the first user may say “Yes” or the already registered second user (for example, another family member) may say “Yes.”

In operation308, the AI device101may identify whether information required to generate the account of the first user is present. The information required to generate the account of the first user may include a minimal degree of information necessary for generating the account (or temporary account) of the first user, such as the last name, first name of the first user. When the information required to generate the account of the first user is not present (e.g., by a failure to obtain the information), the AI device101may return to operation306to output speech requesting information necessary for generation of the account of the first user as a voice signal for generating the account of the first user. For example, the AI device101may output speech such as “What is your name?” in operation306, and may receive a speech saying the name of the first user from the first user or the second user in operation307.

In operations306to308, conversational statements exchanged between the AI device101and the user (for example, the first user or the second user) may be displayed. According to an embodiment, in operations306to308, the AI device101may have statements with the user by outputting a designated voice signal based on data pre-stored in the AI device101. According to another embodiment, in operations306to308, the AI device101may have statements with the user based on communication with the AI server102. In this case, the AI device101may statements with the user by transmitting a user's voice signal to the AI server102and receiving audio data responding to the user's voice signal from the AI server102, and outputting the audio data through the speaker272.

In some embodiments, the conversational statements between the AI device101and the user (for example, the first user or the second user) in operations306to308may include the AI device101questioning about a relationship between the unregistered first user and the already registered second user. Based on a user's answer to the question, the AI device101may obtain information regarding the relationship between the unregistered first user and the already registered second user (for example, son, mother).

In addition, when a voice signal of the already registered second user is received during operations306to308, the AI device101may identify the second user existing in the same space as the first user. For example, the AI server102may receive data corresponding to the voice signal of the second user from the AI device101, may identify the already registered second user through voiceprint recognition, and may identify that the second user is in the same space as the first user.

When it is identified that the information (for example, the last name, first name) required to generate the account of the first user is present (e.g., obtained) in operation308, the AI device101may proceed to operation309to transmit the information required to generate the account of the first user to the AI server102. The AI server102may receive the information required to generate the account of the first user from the AI device101. The AI server102may transmit the received information (for example, the name of the first user) to the account server103to generate the account of the first user. For example, the AI server102may transmit data indicating the name of the first user and the voiceprint information of the first user which is stored (or temporarily stored) in the AI server102to the account server103, such that the account server103generates the account (or temporary account) of the first user. The account server103may receive the data indicating the name of the first user and the voiceprint information of the first user from the AI server102. According to an embodiment, the AI server102and the account server103may be integrated into one server104.

In operation310, the account server103may store the information pertinent to the first user, as received from the AI server102(or received from the AI device101). For example, the account server103may store the name of the first user and the voiceprint information of the first user to generate the account of the first user. The account generation process of the first user may be completed when the first user is authenticated through operations311to318.

In operation311, the account server103may request authentication of the first user to complete the generation of the account of the first user. The account server103may transmit a signal requesting authentication of the first user to the AI device101through the AI server102, such that the AI device101outputs a voice signal requesting authentication of the first user. For example, the account server103may transmit the signal requesting authentication of the first user to the AI server102in response to the information (for example, the name of the first user, the voiceprint information of the first user) required to generate the account of the first user being received from the AI server102. The AI server102may receive the signal requesting authentication of the first user from the account server103. In response to receiving the signal, the AI server102may transmit the signal requesting authentication of the first user to the AI device101. The AI device101may receive the signal requesting authentication of the first user from the AI server102.

According to an embodiment, the signal requesting authentication of the first user, received by the AI device101from the AI server102, may correspond to audio data (such as, for example, statement) for output through the speaker272at the AI device101. For example, the AI device101may output a voice signal corresponding to the audio data requesting authentication of the first user, received from the AI server102.

According to another embodiment, the signal requesting authentication of the first user may be a control signal for causing the AI device101to request authentication of the first user. The AI device101may output a designated voice signal to request authentication of the first user in response to the authentication request signal (that is, the control signal) being received.

In operation312, the AI device101may output the voice signal for authenticating the first user in response to the signal requesting authentication of the first user being received from the AI server102. The voice signal outputted from the AI device101to authenticate the first user may include a request for the already registered second user to authenticate the first user.

The voice signal outputted from the AI device101to authenticate the first user may be a statement inducing authentication of the first user (for example, “Please authenticate!”). However, this disclosure is not limited to this example, and the AI device101may output a designated statement to authenticate the first user. For example, the AI device101may output a designated statement asking about a relationship with the already registered second user, as a part of the authentication procedure of the first user. The AI device101may perform at least part of the authentication procedure of the first user based on an answer of the already registered second user being received with respect to the designated statement. For example, at least part of the authentication procedure of the first user may be performed, when it is identified that a voiceprint responding to the designated statement is the voiceprint of the already registered second user.

In certain embodiments, the AI device101may request the already registered second user to authenticate the first user to register the first user. The AI device101may output a voice or statement for requesting the already registered second user to authenticate the first user to authenticate the first user.

In some embodiments, the AI device101may output a voice signal guiding the already registered second user to utter a designated sentence to authenticate the first user. For example, the designated sentence may a passphrase which is pre-set by the second user. In another example, the designated sentence may be a sentence which is suggested by the AI device101.

In some other embodiments, the AI device101may output a voice signal guiding the already registered second user to perform authentication through the user device105of the second user to authenticate the first user. The authentication through the user device105of the second user may include authentication using biometric recognition (for example, face recognition, fingerprint recognition, iris recognition), authentication using a pattern, authentication using a PIN, authentication using a password, authentication using a message or email, or authentication provided by a certificate authority or a communication operator. A detailed operation of the authentication through the user device105of the second user will be described below with reference toFIG. 5.

In operation313, the AI device101may obtain a voice signal of the already registered second user in response to the voice signal for authenticating the first user. For example, the second user may utter the pre-set passphrase or may utter a sentence suggested by the AI device101in operation312. However, this should not be considered as limiting. For example, when the AI device101outputs a designated statement asking about the relationship with the already registered second user, as a part of the authentication procedure of the first user, the second user may answer about the relationship with the first user (for example, “This is my son” or “He calls me mother”). The AI device101(or the AI server102) may authenticate or register the first user based at least on obtaining the relationship between the first user and the second user.

In operation314, the AI device101may transmit data corresponding to the voice signal of the second user obtained to the AI server102through the communication module290. The AI server102may receive the data corresponding to the voice signal of the second user from the AI device101.

In operation315, the AI server102may perform voiceprint recognition by using the received data. The AI server102may store voiceprint information of users already registered to use the AI device101.

In operation316, the AI server102may identify that the user uttering the voice signal is already registered, based on the pre-stored voiceprint information. In addition, the AI server102may identify that the voice signal of the second user indicates that the second user has authenticated the first user. For example, the AI server102may identify that the voice signal of the second user matches utterance of the designated phrase (for example, a passphrase or a sentence suggested by the AI device101). The AI server102may authenticate the first user based on it being identified that the user uttering the voice signal matches the already registered second user and the voice signal matches utterance of the designated phrase. However, this should not be considered as limiting. The AI server102may authenticate the first user seamlessly simply by identifying that a voiceprint obtained during the statement with the AI device101is the voiceprint of the already registered second user.

In some embodiments in which the AI server102and the account server103are integrated into one server104, the server104may complete the authentication of the first user in operation316. For example, the server104may complete the authentication of the first user based on the voice signal of the second user being received from the AI device101in operation314, and may generate the account of the first user and may store information of the account of the first user (for example, name, voiceprint, relationship with the second user, etc.).

In some other embodiments in which the AI server102and the account server103are separated from each other, the AI server102may transmit information indicating the already registered second user to the account server103, based on it being identified that the voiceprint is the voiceprint of the already registered second user through voiceprint recognition. The account server103may receive the information indicating the already registered second user from the AI server102.

In operation318, the account server103may complete authentication of the first user based on the information indicating the second user. For example, the account server103may store information regarding accounts of registered users. The account server103may identify the already registered second user based on the information regarding the accounts of the registered users, and may complete the authentication of the first user based on the identification.

In operation319, the account server103may generate the account of the first user based on determining that the authentication of the first user is completed. The account server103may store information regarding the account of the first user (for example, the profile of the first user). The account server103may store, as the profile of the first user, the information regarding the first user (for example, the first name, last name, voiceprint information, relationship with the already registered second user) obtained through the dialogue between the AI device101and the first user (or second user) in operations306to308.

In certain embodiments, even when the first user is a child or an elderly person who does not have a personal device, the first user can be easily registered through dialogue with the AI device101by authenticating the new first user by the already registered second user. According to an embodiment, the AI device101(or the AI server102) may authenticate the first user based on the account of the already registered second user, based at least on obtaining the relationship between the already registered second user and the unregistered first user. For example, the AI device101(or the AI server102) may authenticate and register the first user based on it being identified that the first user is the son of the already registered second user.

In certain embodiments, there may be various aspects, forms, or ranges of accounts. For example, there may be a formal account which has all information necessary for the account. The information necessary for the account may include sex, name, birth date, or email of a user. The account of the already registered second user may be the above-described formal account. The account of the already registered second user may be generated through communication between the user device105of the second user and the account server103.

In another example, for a user having no email, there may be a temporary account having minimal information of the information necessary for the account. The temporary account may temporarily (for example, randomly) generate and store an email with respect to the user having no email. In addition, the minimal information of the information necessary for the account may be at least one of the sex, name, or birth date of the user. For example, when the first user is a child or an elderly person having no email, the account of the first user may be the above-described temporary account. For example, the account server103may temporarily (for example, randomly) generate an email of the first user, and may generate the account (that is, the temporary account) of the first user by using the generated temporary email and the minimal information of the first user (for example, the name of the first user).

In still another example, there may be an account of a profile concept. The account of the profile concept may be registered only for a specific device (for example, the AI device101, an AI speaker, a family hub, an IoT device, or the like) such that the account is not generally used and is used only for the specific device. For example, the account of the first user may be generated as the profile concept as described above.

In yet another example, there may be a sub account associated with a specific account. The sub account may be associated with an already registered formal account (for example, as a sub structure). For example, the account of the first user may be generated as a sub account of the account of the already registered second user.

As described above, various aspects of accounts may include at least one of a formal account, a temporary account, an account of a profile concept, or a sub account, but are not limited thereto. For example, the account of the first user may be generated in the form of at least one of the temporary account, the account of the profile concept, or the sub account, such that the first user having no email can be registered to use the AI device101.

FIG. 4illustrates an example of an operation of the AI device101according to certain embodiments. According to some embodiments, at least part of the operations of the AI server102and the account server103ofFIG. 3may be performed by the AI device101.

Referring toFIG. 4, in operation401, the AI device101may obtain a voice signal through the microphone271. The voice signal may be a voice signal uttered by a certain user (for example, the first user). For example, the voice signal uttered by the user may correspond to a statement451of the first user. The voice signal uttered by the user may be a plurality of statements including the statement451.

In operation402, the AI device101may identify that the registration of the first user is required based on the obtained voice signal.

In some embodiments, the AI device101may identify that the registration of the first user is required, based on it being identified that the first user uttering the obtained voice signal is an unregistered user. The AI device101may identify that the registration of the first user is required, based on it being identified that the first user uttering the voice signal is different from the already registered second user. For example, when the voice signal of the unregistered first user is obtained (or accumulated) a designated number of times or more, the AI device101may identify that the registration of the first user is required. When the voice signal of the unregistered first user is obtained over a designated period or longer, the AI device101may identify that the registration of the first user is required. When the voice signal of the unregistered first user is obtained a designated number of times or more (that is, with a designated frequency or more) over the designated period, the AI device101may identify that the registration of the first user is required. The AI device101may identify that the first user is a family member of the home where the AI device101is installed, based on the voice signal of the unregistered first user being obtained with the designated frequency, over the designated period, and/or the designated number of times or more.

In some other embodiments, the AI device101may identify that the registration of the first user is required, in response to a voice signal of the first user or a voice signal of the second user requesting the registration of the first user (for example, “Please register!”) being obtained.

In operation403, the AI device101may output a voice signal for registering the first user through the speaker272in response to it being identified that the registration of the first user is required. The voice signal may be a voice signal inducing the start of a registration process of the first (for example, “Do you want me to register?”). The voice signal may be a voice signal asking for information necessary for generation of an account of the first user (for example, a name, a relationship with the already registered second user). For example, the AI device101may output a voice signal asking the name of the first user. The AI device101may output a voice signal asking about a relationship between the first user and the already registered second user.

In operation404, the AI device101may obtain a voice signal from the first user or the second user through the microphone271. The voice signal may be a voice signal of the first user or the second user answering the question of the AI device101.

Operations403and404may be performed multiple times. For example, the AI device101may repeat operations403and404until minimal information necessary for generation of the account of the first user is obtained. The minimal information necessary for generation of the account of the first user may include at least one of the name of the first user or the relationship between the first user and the already registered second user. For example, operations403and404may correspond to statements452.

When the information necessary for generation of the account of the first user is obtained, the AI device101may request the already registered second user to authenticate the first user in operation405. For example, the AI device101may identify that the already registered second user exists in the same space as the first user, based on a voice signal of the second user being obtained in operations403and404. The AI device101may seamlessly identify the second user based on the voice signal of the already registered second user being obtained in operations403and404. The AI device101may identify the name of the second user (for example, “Lee Young-hee”), based on voiceprint recognition regarding the voice signal of the second user. The AI device101may request the second user to authenticate the first user based on the identification.

The operation of the AI device101requesting the second user to authenticate the first user may include an operation of outputting a voice signal inducing the second user to authenticate. The voice signal inducing the second user to authenticate may correspond to statements453, for example.

In some embodiments, the method for the second user to authenticate the first user may include, for example, authentication using a passphrase pre-set by the second user. For example, in operation405, the AI device101may output a voice signal saying “Lee Young-hee, please authenticate by the passphrase.” The AI device101may receive a voice signal of the second user uttering the designated passphrase in operation406-1.

In some other embodiments, the method for the second user to authenticate the first user may include, for example, authenticating by using a sentence suggested by the AI device101. For example, in operation405, the AI device101may output a voice signal saying “Lee Young-hee, please say ‘Hi bixby, I agree.’” The AI device101may receive a voice signal of the second user uttering the sentence suggested by the AI device101in operation406-1.

In some other embodiments, the method for the second user to authenticate the first user may include authenticating through the user device105of the second user. According to circumstances, authenticating through the user device105of the second user may be performed when authentication through the voice signal of the second user (operation406-1) fails. Alternatively, authentication through the user device105of the second user may be performed when a voice signal of the second user is not obtained a designated number of times or more with respect to the operation (405) of the AI device101requesting the second user to authenticate. The authentication through the user device105of the second user may include authentication using biometric recognition (for example, face recognition, fingerprint recognition, iris recognition), authentication using a pattern, authentication using a PIN, authentication using a password, authentication using a message or email, or authentication provided by a certificate authority or a communication operator. For example, in operation405, the AI device101may output a voice signal saying “Lee Young-hee, please check your smartphone.” The AI device101may receive authentication information from the user device105of the second user in operation406-2. A specific operation of authentication through the user device105of the second user will be described below with reference toFIG. 5.

In operation407, the AI device101may authenticate the first user in response to at least one of the voice signal of the second user being received in operation406-1or the authentication information being received from the user device105of the second user in operation406-2. The new first user is authenticated by the already registered second user, such that the first user can be easily registered through conversational dialogue with the AI device101even when the first user is a child or an elderly person having no personal device.

In operation408, the AI device101may register the first user to use the AI device101. The AI device101may generate an account of the first user. The AI device101may store information regarding the first user that is obtained through conversational dialogue with the first user or the second user as information of the account of the first user (for example, a profile). The information regarding the first user that is obtained through the conversational dialogue with the first user or the second user may include the name of the first user, the voiceprint information of the first user, the relationship between the first user and the second user, or the appellations that the first user and the second user call each other.

At least part of the operations of the AI device101illustrated inFIG. 4may be performed by the AI server102or the account server103.

FIG. 5illustrates an example of a signal flow among the AI device101, the AI server102, and the account server103, for authenticating the first user by using the device105of the second user according to certain embodiments.

Operations illustrated inFIG. 5may correspond to the operation of authenticating the first user after obtaining the information necessary for generation of the account of the first user inFIG. 3 or 4. The operations illustrated inFIG. 5may correspond to the method of authenticating the first user through the user device105of the second user among various methods of authenticating the first user by the second user.

Referring toFIG. 5, in operation501, the AI device101may obtain a voice signal of the second user. The voice signal of the second user may be obtained from conversational dialogue with the AI device101for registering the first user. For example, the voice signal of the second user may be obtained by an answer of the second user from the statements for registering the first user. Operation501may correspond to operation307ofFIG. 3or operation404ofFIG. 4.

In operation502, the AI device101may transmit data corresponding to the voice signal of the second user to the AI server102. The AI server102may receive the data corresponding to the voice signal of the second user.

In operation503, the AI server102may perform voiceprint recognition with respect to the data corresponding to the received voice signal. The AI server102may identify that the user uttering the voice signal is the already registered second user, based on the voiceprint recognition.

In operation504, the AI server102may transmit information regarding the identified second user to the account server103. The account server103may receive the information regarding the second user from the AI server102. The account server103may identify the account of the second user and may identify the user device105of the second user based on the reception.

In operation505, the account server103may transmit a signal requesting the identified user device105of the second user to authenticate, as an operation of requesting the second user to authenticate.

In operation506, the server104may transmit the signal requesting the second user to authenticate to the AI server102, such that the AI device101outputs a voice signal for authenticating the first user. For example, the account server103may transmit the signal requesting the second user to authenticate to the AI server102, and the AI server102may transmit audio data to the AI device101to request the second user to authenticate.

In operation507, the AI device101may output a voice signal for authenticating the first user in response to the authentication request signal being received from the AI server102(or the account server103). For example, the AI device101may output a voice signal saying “Lee Young-hee, please check your smartphone.”

In operation508, the second user device105may provide a user interface (UI) for authenticating the first user in response to the authentication request signal being received from the account server103. For example, the second user device105may display the UI for authenticating the first user through a display of the second user device105.

In operation509, the second user device105may authenticate the first user by the second user. For example, the second user device105may perform biometric recognition (for example, face recognition, fingerprint recognition, iris recognition) of the second user, may receive a pattern pre-designated by the second user, may perform authentication by using a PIN of the second user, may receive a password pre-designated by the second user, may perform authentication by using a message or an email of the second user, or may perform authentication provided by a certificate authority or a communication operator. However, this should not be considered as limiting.

In operation510, the second user device105may transmit information indicating the result of authentication by the second user to the account server103. The account server103may receive the information indicating the result of authentication by the second user from the second user device105.

In operation511, the account server103may complete the authentication process of the first user based on the reception.

In operation512, the account server105may generate the account of the first user based on the first user being normally authenticated by the second user. Operation512may correspond to operation319ofFIG. 3or operation408ofFIG. 4.

According to some embodiments, the AI server102and the account server103may be integrated into one server104. According to some embodiments, at least part of the operations of the AI server102and the account server103may be performed by the AI device101.

FIG. 6illustrates an example of an operation for requesting the second user to authenticate the first user in the AI device101according to whether the second user and the first user are in the same space according to certain embodiments. Operations ofFIG. 6may be performed by the processor220of the AI device101. At least part of the operations ofFIG. 6may be performed by the AI server102or the account server103.

Referring toFIG. 6, in operation601, the AI device101may output a voice signal for registering the first user, and may receive a response to the voice signal. Outputting and receiving the voice signal may indicate statements spoken or output between the AI device101and a certain user (for example, the first user or the second user) for registering the first user. Operation601may correspond to operation306to308ofFIG. 3. Operation601may correspond to operation403and404ofFIG. 4. Operation601may correspond to statements451and452ofFIG. 4.

In operation602, the AI device101may identify whether a voice signal of the already registered second user is received within a designated period. For example, the AI device101may identify whether a voice signal of the already registered second user is received during a designated number of statements or for a designated time (for example, for a few minutes). The identification may be performed based on communication with the AI server102. For example, the AI device101may transmit the obtained voice signal to the AI server102, and the AI server102may identify whether the voice signal is a voice signal of the already registered second user through voiceprint recognition.

Based on it being identified that the voice signal of the already registered second user is received within the designated period, the AI device101may identify that the already registered second user is in the same space as the first user. Accordingly, based on it being identified that the voice signal of the already registered second user is received within the designated period, the AI device101may authenticate the first user based on at least part of the voice signal of the second user in operation603. The operation of authenticating the first user based on at least part of the voice signal of the second user may correspond to the authentication methods described inFIGS. 3 to 5. The operation of authenticating the first user based on at least part of the voice signal of the second user may be authenticating the first user based at least part on conversational statements with the second user.

Based on it being identified that the voice signal of the second user is not received within the designated period, the AI device101may identify that the already registered second user is not in the same space as the first user. Accordingly, the AI device101may proceed to operation604to output a voice signal for specifying the second user to request authentication. For example, the AI device101may output a voice signal saying “Who do I ask to authenticate?” The AI device101may receive a response to the voice signal. For example, the AI device101may obtain a voice signal saying “Please ask my mom, Lee Young-hee, to authenticate.” from the first user. The AI device101may specify the second user to request authentication, based on the voice signal being obtained. For example, the AI server102or the account server103may identify the second user named “Lee Young-hee” among users registered with respect to the AI device101, and the second user device105.

In operation605, the AI device101(or the AI server102or the account server103) may request the specified second user device105to authenticate. For example, the AI device101may transmit a signal for requesting authentication of the first user to the second user device105which is identified based on the voice signal of the first user obtained. Operation605may correspond to operation505ofFIG. 5, for example.

In operation606, the AI device101may receive authentication information from the second user device105. The AI device (or the AI server102or the account server103) may receive information indicating the result of authentication by the second user from the second user device105. For example, the result of authentication received from the second user device105may include a result of authentication based on biometric recognition (for example, face recognition, fingerprint recognition, iris recognition) of the second user, a result of authentication based on a pattern, a result of authentication based on a PIN, a result of authentication based on a password, or a result of authentication based on a message or email of the second user. Operation606may correspond to operation510ofFIG. 5, for example. The AI device101(or the AI server102or the account server103) may authenticate the first user based on the result of authentication received from the second user device105.

In operation607, the AI device101may generate the account of the first user in response to the first user being authenticated in operation603or606.

FIG. 7illustrates an example of a signal flow among the AI device101, the AI server102, and the account server103, for performing an additional operation in relation with the registration of the first user according to certain embodiments. InFIG. 7, the AI server102and the account server103may be integrated into one server104. Operations illustrated inFIG. 7may be performed after the account of the first user is generated inFIGS. 3 to 6.

Referring toFIG. 7, in operation701, the server104(for example, the account server103) may generate the account of the first user. Operation701may correspond to operation319ofFIG. 3, operation408ofFIG. 4, operation512ofFIG. 5, or operation607ofFIG. 6.

In operation702, the server104may notify the AI device101that the generation of the account of the first user is completed. For example, the account server103may transmit a signal indicating that the generation of the account of the first user is completed to the AI device101through the AI server102. The AI device101may receive the signal indicating that the generation of the account of the first user is completed from the server104.

In operation703, the AI device101may output a voice signal for storing a contact regarding the account of the first user in an address book of the second user device105. For example, the AI device101may output a voice signal corresponding to audio data received from the AI server102. In another example, the AI device101may output a designated voice signal in response to the account generation completion signal of the first user being received. The voice signal may be a voice signal of the AI device101, saying “Shall I store in the address book of the family?” as a statement753. In some embodiments, the first user may not own a personal device (for example, a smartphone). In this case, the contact regarding the account of the first user may indicate identification information for calling the AI device101. In some other embodiments, when the first user owns a personal device (for example, a smartphone), the contact regarding the account of the first user may be a phone number of the personal device.

In operation704, the AI device101may obtain a voice signal of the second user in response to the voice signal. For example, the AI device101may obtain a voice signal of the second user, saying “Yes, Thanks.” as a statement754.

In operation705, the AI device101may transmit data corresponding to the obtained voice signal to the server104. For example, the AI device101may transmit data corresponding to the voice signal to the AI server102. The AI server102may transmit, to the account server103, a signal for storing the contact regarding the account of the first user in the address book of the second user device105, based on the identification of the voice signal.

In operation706, the server104(for example, the account server103) may store the contact regarding the account of the first user in the address book of the second user device105. For example, referring to a database770, the contact of the first user (that is, Kim Cheol-soo) may be stored in the address book of the second user (that is, Lee Young-hee). In addition, the contact of the second user (that is, Lee Young-hee) may be stored in the address book of the first user (that is, Kim Cheol-soo).

In some embodiments, when the first user does not own a personal device or information regarding a personal device of the first user is not registered, the server104may store identification information for calling the AI device101as a contact of the first user.

In some other embodiment, when information regarding a personal device of the first user is registered, the server104may automatically add the contact of the first user to the address book of the second user.

In operation707, the server104may transmit a signal for synchronizing the contact regarding the account of the first user with the second user device105to the second user device105. The second user device105receiving the synchronization signal may automatically store (update) the contact regarding the account of the first user in the book address of the second user device105.

FIG. 8illustrates an example of a screen related to the contact of the first user stored in the address book of the second user device105according to certain embodiments. The screen illustrated inFIG. 8may be a screen that is displayed on the user device105of the second user.

Referring toFIG. 8, the second user device105may display a screen810in response to the contact of the first user being registered at the second user device105(for example, operation707ofFIG. 7). The screen810may include a notification indicating that the account of the first user (that is, Kim Cheol-soo) is registered at the address book of the second user device105. For example, in response to a user input on an icon811for displaying the address book being received on the screen810, the second user device105may display a screen820converted from the screen810.

The screen820may be a screen displaying the address book related to the AI device101. The screen820may include items indicating contacts of users registered at the AI device101. The second user device105may display the screen820showing that the contact of the first user (that is, Kim Cheol-soo) newly registered at the AI device101is updated. In response to a user input on an item821indicating the contact of the first user newly registered at the AI device101being received on the screen820, the second user device105may display a screen830converted from the screen820.

The screen830may be a screen regarding the contact of the first user. The screen830may include an icon831for calling the first user and an icon832for exchanging a text-based message with the first user. The second user device105may call a device corresponding to the contact of the first user, in response to a user input on the icon831being received. The device corresponding to the contact of the first user may include, for example, the AI device101or a personal device (for example, a smartphone) of the first user.

In some embodiments, when the first user does not own a personal device or information regarding a personal device of the first user is not registered, identification information for calling the AI device101may be stored as the contact of the first user. In this case, the second user device105may call the AI device101in response to the user input on the icon831being received. Accordingly, the second user may talk to the first user existing in the same space as the AI device101.

In some other embodiments, a phone number of a personal device of the first user may be stored as the contact of the first user. In this case, the second user device105may call the personal device of the first user in response to the user input on the icon831being received.

The second user device105may display a screen840on which the second user exchanges message with the first user, in response to a user input on the icon832for exchanging messages with the first user being received on the screen830.

In some embodiments, when the identification information of the AI device101is stored as the contact of the first user, the second user device105may transmit a message to the AI device101based on a message input of the second user through the screen840. The AI device101may output a voice signal corresponding to the message received from the second user device105. The AI device101may obtain a voice signal of the first user responding to the voice signal. The voice signal of the first user may be converted into text data by the AI server102, for example. The text data may be transmitted to the second user device105and may be displayed through the screen840. By the above-described process, the second user may exchange messages with the first user through the AI device101.

FIG. 9illustrates an example of a signal flow among the AI device101, the AI server102, and the account server103, for performing additional other operations in relation to the registration of the first user according to certain embodiments. InFIG. 9, the AI server102and the account server103may be integrated into one server104. Operations illustrated inFIG. 9may be performed after the account of the first user is generated inFIGS. 3 to 6.

Referring toFIG. 9, in operation901, the AI device101may obtain a voice signal of the newly registered first user. The voice signal may be, for example, a voice signal of the first user saying “Bixby, it's my birthday tomorrow. Did you know that?” as a statement951.

In operation902, the AI device101may transmit data corresponding to the voice signal of the first user to the server104.

In operation903, the server104may identify the user corresponding to the voice signal. For example, the AI server102may identify that the user uttering the voice signal is the first user, based on voiceprint recognition.

In operation904, the server104(for example, the AI server102) may identify information to be added to the profile of the identified first user, based on semantic analysis of the voice signal. For example, the AI server102may identify the birthday of the first user from the statement951to add to the profile of the first user.

In operation905, the server104(for example, the account server103) may store the identified information in the profile of the identified first user. For example, the AI server102may transmit data indicating the birthday of the first user to the account server103. The account server103may receive the data indicating the birthday of the first user from the AI server102. The account server103may store the birthday of the first user in the account of the first user based on the reception.

In operation906, the server104(for example, the AI server102) may transmit designated response data responding to the voice signal (for example, the statement951) of the first user to the AI device101. The AI device101may receive the designated response data (for example, audio data) from the AI server102.

In operation907, the AI device101may output a voice signal corresponding to the response data in response to the response data being received from the AI server102. For example, the AI device101may output a voice signal saying “Cheol-soo, happy birthday!” as a statement957as a response to the user's voice signal.

As in the operations illustrated inFIG. 9, information may be added to the profile of the registered user through conversation between the registered user and the AI device101. Alternatively, the AI device101may add information to profiles of registered users by identifying conversational statements between the plurality of users registered. Information to be added to the profile of a user may include, for example, registered user's birthday, a TV content desired by the registered user, a You Tube content desired by the registered user, or nicknames (or appellations) that the plurality of registered users call one another. For example, when a TV, which is an IoT device, is operated in association with the AI device101, information such as a TV content desired by the user may be used to control the TV through the AI device101.

Certain embodiments are not limited thereto. For example, when the AI device101is installed in a hotel room, a guest staying the hotel room may be registered as a user for using the AI device101. In this case, the AI device101may output a voice signal asking the registered user about a period of use (for example, “How long are you going to stay here?”). The AI device101may set the period of use of the registered user based on a voice signal of the user (for example, “I will stay here for 3 days.”) received as a response to the voice signal. The AI device101may give the user authority to use the AI device101during the set period of use.

FIG. 10illustrates an example of a signal flow among the AI device101, the AI server102, an IoT device1000, and an IoT server1050, for registering the IoT device1000according to certain embodiments.

In certain embodiments, the IoT device (for example, an air cleaner, an air conditioner, a TV, etc.) may be registered in association with the AI device101. When the IoT device is registered at the AI device101, a user may control the registered IoT device through the AI device101based on a voice signal. InFIG. 10, an example of an operation for registering the IoT device through statements with the AI device101is illustrated.

Referring toFIG. 10, in operation1001, the IoT device1000may be activated. For example, the operation of the IoT device1000being activated may include an operation of the IoT device1000being initially turned on after being installed in a home.

In operation1002, the IoT device1000may broadcast its own identification information. The AI device101may receive the identification information broadcasted by the IoT device1000. For example, the AI device101may perform a hub function of the IoT device1000.

In operation1003, the AI device101may identify the IoT device1000. For example, the AI device101may identify the IoT device1000as a hub for the IoT device1000.

In operation1004, the AI device101may transmit the identification information of the IoT device1000to the IoT server1050. For example, the AI device101may transmit the identification information of the IoT device1000to the IoT server1050through the AI server102. However, this should not be considered as limiting, and the AI device101may directly transmit the identification information of the IoT device1000to the IoT server1050.

In operation1005, the IoT server1050may identify that the IoT device1000is an unregistered device.

In operation1006, the IoT server1050may transmit a signal for registering the IoT device1000to the AI device101based on the identification. For example, the IoT server1050may transmit a control signal for initiating a registration process of the IoT device1000to the AI server102. The AI server102may receive the control signal for initiating the registration process of the IoT device1000from the IoT server1050. The AI server102may transmit a control signal for initiating the registration process of the IoT device1000to the AI device101, or may transmit audio data for registering the IoT device1000to the AI device101, in response to the control signal being received.

In operation1007, the AI device101may output a voice signal in response to the signal for registering the IoT device1000being received from the AI server102(or the IoT server1050). In some embodiments, the AI device101may output a pre-designated voice signal in response to the control signal for initiating the registration process of the IoT device1000being received from the AI server102. In some other embodiments, the AI device101may output a voice signal corresponding to the audio data in response to the audio data for registering the IoT device1000being received from the AI server102. The voice signal outputted from the AI device101may be, for example, a voice signal saying “A new device has been searched. Do you want me to register?”

In operation1008, the AI device101may obtain a voice signal from a registered user. For example, the voice signal may be a voice signal indicating agreement on the registration of the IoT device1000. The voice signal may be a voice signal of the registered user, “Yes, please register!”

In operation1009, the AI device101may transmit data corresponding to the voice signal of the registered user to the AI server102. The AI server102may transmit data indicating intent to register the IoT device1000to the IoT server1050, based on sematic analysis of the voice signal. The IoT server1050may initiate the registration process of the IoT device1000in response to the data being received. The registration process of the IoT device1000may include operations of requesting a registered user to authenticate and of receiving authentication from the registered user.

However, this should not be considered as limiting. According to some embodiments, the registration process of the IoT device1000may be initiated by a voice signal of a registered user. For example, the registration process of the IoT device1000may be initiated in response to a voice signal of a registered user, saying “This is my TV, please register!” as a statement1058.

In operation1010, the IoT server1050may transmit information for authenticating the IoT device1000to the IoT device1000. The information may be, for example, a password or passphrase, and may be outputted or displayed through the IoT device1000.

In operation1011, the IoT server1050may transmit a signal requesting authentication of the IoT device1000to the AI device101. For example, the IoT server1050may transmit a signal requesting authentication of the IoT device1000to the AI server102. The AI server102may receive the signal requesting authentication of the IoT device1000from the IoT server1050. The AI server102may transmit a control signal requesting authentication of the IoT device1000to the AI device101, or may transmit audio data requesting authentication of the IoT device1000to the AI device101, in response to the signal being received. The AI device101may receive the control signal or the audio data from the AI server102.

In operation1012, the AI device101may output a voice signal for authenticating the IoT device1000in response to the reception. For example, the voice signal may be a voice signal of the AI device101saying “Please read the password appearing on the TV.” as a statement1062.

In operation1013, the IoT device1000may display the information for authenticating the IoT device1000in response to the information being received from the IoT server1050. For example, the IoT device1000may display or output the password or passphrase received from the IoT server1050.

In operation1014, the AI device101may receive a voice signal of the registered user. The voice signal may be a voice signal saying the password displayed on the IoT device1000, for example. The voice signal may correspond to a statement1064.

In operation1015, the AI device101may transmit data corresponding to the voice signal to the AI server102. The AI server102may identify that the registered user authenticates the IoT device1000, based on the data corresponding to the voice signal (for example, utterance of the password) being received. The AI server102may transmit data indicating that the IoT device1000is authenticated to the IoT server1050. The IoT server1050may receive the data indicating that the IoT device1000is authenticated from the AI server102.

In operation1016, the IoT server1050may authenticate the IoT device1000, based on the reception, and may register the IoT device1000. Thereafter, an operation of obtaining additional information regarding the IoT device1000may be performed.

In operation1017, the IoT server1050may request information regarding the IoT device1000from the AI device101through the AI server102. The information regarding the IoT device1000may include, for example, a place where the IoT device1000is installed, a user that the IoT server1050will give authority to use the IoT device1000, or an owner of the IoT device1000.

In operation1018, the AI device101may output a designated voice signal for collecting the information regarding the IoT device1000. The voice signal may include a question “Where is the TV installed?” or “Are you going to use it with your family?”

In operation1019, the AI device101may obtain a voice signal of the registered user in response to the voice signal. In operation1020, the AI device101may transmit data corresponding to the voice signal to the IoT server1050through the AI server102. In operation1021, the IoT server1050may store the information regarding the IoT device1000. For example, the IoT server1050may store a user that the IoT server1050will give the authority regarding the IoT device1000, as the information regarding the IoT device1000, and may give authority to control the IoT device1000specifically to the stored user.

Operations1017to1021may be repeated, and for example, may correspond to statements1068.

According to certain embodiments as described above, an electronic device (for example, the AI device101) may include: a speaker (for example, the speaker272); a microphone (for example, the microphone271); and at least one processor (for example, the processor220) configured to: determine whether registration of a first user is required based on a voice signal obtained through the microphone; based on it being identified that the registration of the first user is required, request a second user registered at the electronic device to authenticate the first user; and, based on the first user being authenticated by the second user, register the first user.

In certain embodiments, the voice signal obtained through the microphone may include a voice signal of the first user, and the at least one processor may identify that the registration of the first user is required, based on it being identified that the voice signal obtained through the microphone is different from a voice signal of the second user registered at the electronic device.

In certain embodiments, the at least one processor may output a voice signal requesting the second user to authenticate through the speaker in order to request the second user to authenticate the first user.

In certain embodiments, the at least one processor may authenticate the first user based on a voice signal indicating a designated phrase being obtained from the second user through the microphone.

In certain embodiments, the at least one processor may authenticate the first user based on authentication information being received from a device of the second user (for example, the user device105).

In certain embodiments, the at least one processor may authenticate the first user based at least part on a voice signal regarding a relationship between the first user and the second user being obtained, may generate an account of the first user to register the first user, and may store information regarding the relationship between the first user and the second user in association with the account of the first user.

In certain embodiments, the at least one processor may transmit a signal for requesting the second user to authenticate the first user to a device of the second user, based on it being identified that a voice signal of the second user is not obtained within a designated time from a time when the voice signal is obtained through the microphone.

In certain embodiments, the at least one processor may store identification information for accessing the electronic device as a contact of the first user, based on the first user being registered.

In certain embodiments, the at least one processor may, in response to an unregistered external device (for example, the IoT device1000) being identified, output a voice signal for registering the external device through the speaker, and may register the external device at the electronic device based at least on a voice signal of a user registered at the electronic device being received in response to the voice signal. For example, the at least one processor may transmit information for using for authentication of the external device to the external device, and may register the external device at the electronic device, based on a voice signal corresponding to the information being received from the user registered at the electronic device through the microphone.

According to certain embodiments of the disclosure, a new user can be simply registered by registering the new user through authentication by a registered user.

While specific embodiments have been described in the detailed description of the disclosure, it will be understood by those skilled in the art that various changes may be made therein without departing from the disclosure. Therefore, the scope of the disclosure should be defined not by the embodiments described above but by the appended claims or equivalents to the claims.