Patent ID: 12223956

DETAILED DESCRIPTION

Throughout this specification, like reference numerals will refer to like components. The present specification does not describe all elements of the embodiments, and descriptions about content being general in the technical art to which the disclosure belongs or overlapping content between the embodiments will be omitted. As used herein, the terms “part”, “portion”, “module”, “member” and “block” may be implemented as software or hardware, and according to embodiments, a plurality of “parts”, “portions”, “modules”, “members” or “blocks” may be implemented as a single component, or a single “part”, “portion”, “module”, “member” or “block” may include a plurality of components.

In the entire specification, it will be understood that when a certain part is referred to as being “connected” to another part, it can be directly or indirectly connected to the other part. When a part is indirectly connected to another part, it may be connected to the other part through a wireless communication network.

Also, it will be understood that when a certain part “includes” a certain component, the part does not exclude another component but can further include another component, unless the context clearly dictates otherwise.

In the entire specification, it will also be understood that when an element is referred to as being “on” or “over” another element, it can be directly on the other element or intervening elements may also be present.

It will be understood that the terms “first”, “second”, etc., may be used only to distinguish one component from another, and these components should not be limited by these terms.

It is to be understood that the singular forms “a,” “an,” and “the” include plural referents unless the context clearly dictates otherwise.

Reference numerals used in operations are provided for convenience of description, without describing the order of the operations, and the operations can be executed in a different order from the stated order unless a specific order is definitely specified in the context.

The disclosed display device and the method for controlling the same may reduce a user's inconvenience and easily execute an artificial intelligence conversion service by changing setting information for connecting to the artificial intelligence conversation service with an input button for performing a Push To Talk (PTT) function through the user's wake up word utterance.

An aspect of the disclosure relates to a display device capable of reducing a user's inconvenience and easily executing an artificial intelligence conversion service by changing setting information for connecting to the artificial intelligence conversation service with an input button for performing a Push To Talk (PTT) function through the user's wake up word utterance, and a method for controlling the display device.

Hereinafter, an operation principle and embodiments of the disclosure will be described with reference to the accompanying drawings.

FIG.1shows an outer appearance of a TV implemented as a display device.

Referring toFIG.1, a display device1may be implemented as a Television (TV)1a. The TV1amay include a display panel2, a housing3supporting the display panel1, and a support4.

The housing3may form an outer appearance of the TV1a, and install components for enabling the TV1ato display various images or perform various functions therein. The housing3and the display panel2may be integrated into one body, or the housing3may be configured as a combination of a front housing and a rear housing.

The support4may support the housing3. The support4may have various shapes according to a designer's selection, or may be omitted as necessary. The support4may be attached to or detached from the housing3as necessary.

The display panel2may be installed on a front side of the housing3and display various images to outside. More specifically, the display panel2may display at least one still image or moving image. The display panel2may be implemented by further using a separate component, such as a touch panel and the like, as necessary.

In the display panel2, a plurality of pixels may be formed, and an image displayed through the display panel2may be formed by a combination of light emitted from the plurality of pixels. For example, light emitted from the plurality of pixels may be combined like mosaic, thereby forming an image on a screen102.

An image displayed by the display panel2may include various user interfaces10. The user interfaces10may include icons and shapes that may represent various functions of the display device1.

Meanwhile, the display device1may execute various conversation services, such as Bixby, Alexa, Google Assistance, and the like. For this, the display device1may store Voice Assistance (VA) applications for executing the individual conversation services in a memory, and communicate with VA servers that execute the conversation services.

As shown inFIG.1, a user U may input an utterance voice (U1: Hi A) for executing at least one VA application among a plurality of VA applications to the TV1a. The TV1amay perform preprocessing on the utterance voice from the user U, and transmit a converted electrical signal to the plurality of VA applications. When a first VA application among the plurality of VA applications has set A to a wake up word, the first VA application may operate. The TV1amay output a voice message (D1: Yes, Hello, May I help you?) generated by the first VA application through a speaker. The user U may additionally input an utterance voice for operating the TV1a, and the TV1amay perform an operation desired by the user U based on a conversation policy and a control signal processed by the first VA application. Through the example, the TV1A may provide the user U with a conversation service.

The TV1amay display various setting information required for executing the conversation service with the user U, as a user interface. When the user U executes a Push To Talk (PTT) function through a button provided in a remote controller190(seeFIG.3), the TV1amay execute a preset VA application.

A typical display device executes, when a user U executes the PTT function, only a VA application for performing a preset conversation service. Also, the typical display device executes, in order to change the preset VA application, a user interface for enabling the user U to change setting information and allows a change of the setting information according to manual operation.

The display device1may solve the above-described problem by changing setting information designating a VA application among the plurality of VA applications by recognizing an utterance voice from a user U. A method for changing setting information in the display device1will be described in detail with reference to the following drawings, below.

FIG.2is a view for describing display devices according to various embodiments and a method for executing a conversation service.

The display device1may also be applied to various products20a,20band30shown inFIG.2, as well as the TV1adescribed as the embodiment ofFIG.1. More specifically, the display device1may be applied to a user terminal, such as a smart phone20aand a tablet PC20b, and an artificial intelligence speaker such as a sound bar30.

The user terminal20aand20bmay be implemented as a portable terminal capable of accessing a client server200that performs a conversation service through a network. More specifically, the user terminal20aand20bmay include all kinds of handheld-based wireless communication devices, such as Personal Communication System (PCS), Global System for Mobile communications (GSM), Personal Digital Cellular (PDC), Personal Handyphone System (PHS), Personal Digital Assistant (PDA), an International Mobile Telecommunication-2000 (IMT-2000) terminal, a Code Division Multiple Access-2000 (CDMA-2000) terminal, a W-Code Division Multiple Access (W-CDMA) terminal, a Wireless Broadband Internet (WiBro) terminal, and the like, and wearable devices, such as a watch, a ring, a bracelet, an anklet, a necklace, glasses, contact lenses, a head-mounted-device (HMD), and the like.

The sound bar30may be a fixed type, not a portable type, and may function as a console capable of controlling various electronic products through a home network connected thereto. Also, the sound bar30may include a display panel and a speaker capable of processing an utterance voice from a user U, and output a voice message in response to the utterance voice from the user U. When a user U inputs an utterance voice including a wake up word, the sound bar30may be converted from a standby mode to an active mode. The sound bar30may transmit a control command included in an additional utterance voice from the user U to an electronic product connected through a network, and enable the electronic product to process the control command from the user U.

As such, the display device1that may be applied to the TV1a, the user terminal20, and the sound bar30may be connected to external servers200a,200b, and200cthrough a network. The external servers200a,200b, and200cmay be servers for conversation services associated with the plurality of VA applications included in the display device1.

More specifically, a VA application may execute a conversation service through a conversation policy by using the memory included in the display device1. The VA application may output a voice message corresponding to an utterance voice from a user U through the conversation policy. However, the VA application may not store all of various conversation policies due to a limit of the memory installed therein. Also, to establish a conversation policy capable of continuing to be learned, the VA application may communicate with the external servers200a,200b, and200c, and continue to update the conversation policy. For example, the VA application may store a simple conversation policy such as an output of a voice message corresponding to a wake up word input in the display device1, and receive detailed conversation policies corresponding to various utterance voices input by a user U from the servers200a,200b, and200c.

The plurality of VA applications included in the display device1may communicate with the respective servers200a,200b, and200c. For example, when the display device1includes three VA applications, the display device1may be connected to three external servers200a,200b, and200c.

FIG.3is a control block diagram of a display device according to an embodiment.

Referring toFIG.3, the display device1may include a user input receiver110for receiving an input from a user U which is a user tactical input (a user input), a voice receiver120for receiving an utterance voice from the user U (a user voice input), a sensor130for sensing a gesture made by the user U, a processor140for performing preprocessing on the utterance voice, recognizing a wake up word from the processed utterance voice, and connecting to a VA application150corresponding to the recognized wake up word, a plurality of VA applications150for executing a conversation service described above with reference toFIGS.1and2, a communicator160for transmitting data required or processed by the processor140and the VA applications150to outside, an outputter170for outputting voice messages and images processed by the processor140and the VA applications150, and a memory180storing setting information and data required for operations of the processor140.

More specifically, the user input receiver110may include an input button for receiving various commands input by the user U. According to an example, when the display device1is provided in the user terminal20, the input button may be a button provided on an outer side. The input button may be used as a device for receiving a PTT function. When the user U pushes the input button for performing the PTT function, the display device1may connect to a preset first VA application151among the plurality of VA applications150.

Also, the user input receiver110may include a power button for turning on or off the display device1, a source selection button for selecting a source device from among a plurality of source devices, a sound control button for adjusting a sound volume to be output from the display device1, etc. The various buttons may be implemented by various input means, such as a push switch, a touch switch, a dial, a slide switch, a toggle switch, etc.

The voice receiver120, which is a device for receiving an utterance voice from a user U, may include a microphone for receiving sound, converting the sound into an electrical signal, and outputting the electrical signal. The voice receiver120may be configured with a plurality of microphones to effectively receive voices. For example, when the display device1is provided in the TV1a, the plurality of microphones may be positioned at different locations of the housing3.

The sensor130may be a hardware device for collecting various information based on which the display device1may operate. For example, when the display device1senses a user's gesture, the sensor130may include a camera. As another example, when the display device1is provided in a wearable device, etc., the display device1may include various sensors, such as an ultrasonic sensor, an IR sensor, an acceleration sensor, or a gyro sensor.

The VA application150may process an utterance voice from the user U, generate a voice message according to a conversation policy, and understand a control command included in the utterance voice from the user U. The VA application150may control the display device1to perform an operation according to the control command included in the utterance voice.

More specifically, the VA application150may include a speech recognition engine, and the speech recognition engine may apply a speech recognition algorithm to an input utterance voice to recognize a voice uttered by the user U. The VA application150may detect a start point and an end point from a voice signal to detect (End Point Detection (EPD)) an actual voice section included in the input voice, and generate a result of recognition in a form of an utterance through filtering. The utterance may be implemented by an acoustic model of modeling and comparing signalistic characteristics of a voice, and a language model of modeling a linguistic ordering relation of a word, a syllable, etc. corresponding to a recognition vocabulary.

The VA application150may perform natural language understanding through the utterance as the result of speech recognition. The natural language understanding may perform various analysis, such as morphological analysis, entity name recognition, speech act analysis, etc., on the utterance, and interpret an intention of the uttered voice. After the intention is interpreted, the VA application150may identify the intention to a conversation policy and generate a voice message stored in the conversation policy. In the generated voice message, an input word such as an entity name analyzed in the natural language understanding may be used.

The VA application150may update the conversion policy through the communicator160. More specifically, the VA application150may transmit a result of processing on the voice uttered by the user U to the server200. The server200may input various collected examples as input values to an artificial intelligence neural network. The server200may update the conversation policy through a result output based on the input values. The server200may transmit the updated conversation policy to the VA application150, and the VA application150may use the updated conversation policy in a future conversation with the user U.

Meanwhile, the VA application150may not necessarily update only a conversation policy through the server200. The VA application150may update an algorithm included in the above-described speech recognition engine through the server200, and perform a conversation service with the user U through various updates.

The VA application150may include a plurality of VA applications151,152, and153, and the VA applications151,152, and153may include different recognition algorithms, natural understandings, and conversation policies according to manufacturing companies. Also, the individual servers200a,200b, and200cprovided by the manufacturing companies may update conversation policies, etc. through different artificial intelligences. That is, the display device1may include the plurality of VA applications151,152, and153, and provide various conversation services to the user U.

The processor140may be a configuration for performing preprocessing on a user's utterance voice input to the VA application150.

More specifically, the voice receiver120may receive an utterance voice from a user U, and change the utterance voice to an electrical signal. The voice receiver120may transmit the electrical signal to the processor140, and the processor140may perform preprocessing, such as noise removal, echo canceling or beamforming, on the electrical signal before transmitting the electrical signal to the VA application150, such that the speech recognition engine is capable of recognizing the voice.

The noise removal may be a preprocessing process of removing various noises except for an utterance from a user U from among various sounds received from the voice receiver120. For example, the noise removal may be performed by removing a signal of a preset frequency band.

The echo canceling may be preprocessing of generating an opposite sound wave of a sound wave to be processed as a noise and causing destructive interference. The processor140may perform echo canceling by controlling the voice receiver120.

The beamforming may be a preprocessing process of performing, when the voice receiver120includes a plurality of microphones, synchronization such as time delay on a plurality of electrical signals generated by an utterance voice from a user U.

The processor140may perform various preprocessing on the voice input, in addition to the above-described preprocessing, and then transfer the utterance voice from the user U to the VA application150.

Meanwhile, the processor140according to the disclosed embodiment may extract a wake up word before transferring the preprocessed utterance voice to the VA application150. The processor140may recognize the extracted wake up word, and execute at least one VA application among the plurality of VA applications based on the recognized wake up word.

More specifically, the processor140may operate the first VA application151based on a user input corresponding to the PTT function. That is, the processor140may execute the first VA application151based on setting information for the first VA application151designated to be automatically executed by a user input received by the user input receiver110from among the plurality of VA applications stored in the memory180.

Meanwhile, the processor140may receive a wake up word for waking a second VA application152up in a process of preprocessing the utterance voice from the user U. The processor140may change the setting information from the first VA application151to the second VA application152based on the recognized wake up word. After the setting information changes, an input command for the PTT function may be again input. In this case, the processor140may transfer a preprocessed utterance voice to the second VA application152according to the input command. Thereby, the display device1may reduce inconvenience that a user U himself/herself has to change setting information through a user interface, and increase easiness in execution of an artificial intelligence conversation service.

The processor140may be provided as a chip for performing preprocessing on an utterance voice from a user U. The processor140may be sufficient as long as the processor140connects the VA application150to the voice receiver120. However, the processor140may be provided on a main board for controlling the overall operations of the display device1.

The communicator160may include a wired communication module161and a wireless communication module162.

The wired communication module161may be a module for enabling the display device1to receive image data from a source device through various kinds of image transmission cables. For example, when the display device1is provided as the TV1a, the wired communication module161may receive image data from a source device through a component (YPbPr/RGB) cable, a composite video blanking and sync (CVBS) cable, a High Definition Multimedia Interface (HDMI) cable, a Universal Serial Bus (USB) cable, an Ethernet (IEEE 802.3 technical standard) cable, etc.

The wireless communication module162may be a module for performing wireless communication with outside by using various wireless communication standards, and include wireless communication modules that supports various wireless communication methods, such as global System for Mobile Communication (GSM), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), universal mobile telecommunications system (UMTS), Time Division Multiple Access (TDMA), and Long Term Evolution (LTE), as well as a Wifi module and a Wireless broadband module.

When the display device1according to an embodiment is provided as the TV1a, the display device1may perform wireless communication with the remote controller190. The remote controller190may receive an utterance voice around a user U. The remote controller190may convert the utterance voice into a wireless signal, and transmit the wireless signal to the wireless communication module162.

The remote controller190may include various input buttons. The display device1may provide an input button for performing the PTT function in the remote controller190. That is, the remote controller190may change the user input received by input button to a wireless signal, and transmit the wireless signal to the wireless communication module162. The display device1may change setting information based on the wireless signal received by the communicator160, and transfer a user's utterance voice to the second VA application152.

The outputter170may include a sound outputter171that outputs a voice message and sound data generated by the VA application150, and an image outputter172that outputs image data generated by the VA application150according to a conversation policy, a user interface including setting information, and image data in a format of a frame representing an image signal received from a source device or outside.

The sound outputter171may include an audio amp (not shown) for amplifying sound, and a speaker (not shown) for acoustically outputting the amplified sound.

The sound outputter171may amplify a sound signal output through the audio amp. Also, the speaker may convert the sound signal amplified by the audio amp into sound (sound wave). For example, the speaker may include a thin film vibrating according to an electrical sound signal, and a sound wave may be generated by vibrations of the thin film.

The image outputter172may include the display panel2for visually displaying images, and a display driver (not shown) for driving the display panel2.

According to an embodiment, when the display device1is implemented as the TV1a, the individual pixels of the display panel2may receive electrical signals representing an image from the display driver, and output optical signals corresponding to the received electrical signals. The optical signals output from the plurality of pixels may be combined and accordingly, an image may be displayed on the display panel2. The display panel2may be implemented by various types of panels, such as a Liquid Crystal Display (LCD) Panel, a Light Emitting Diode (LED) Panel, or an Organic Light Emitting Diode (OLED) Panel, etc.

The memory180may store programs and data for controlling configurations included in the display device1, and store setting information.

The setting information may mean transmitting an utterance voice from a user U to the first VA application151according to reception of an input command corresponding to the PTT function. The setting information may be decided by a manufacturing company or a detailed selection by the user U, and stored in the memory180. In the disclosed embodiment, when the display device1recognizes a wake up word for the second VA application152from an utterance voice from a user U, the stored setting information may change.

Meanwhile, a VA application shown inFIG.2may be implemented as software that executes a conversation service. In the case in which VA applications are implemented as software, a plurality of VA applications may be stored as a program in the memory180. The processor140may perform preprocessing on an utterance voice, read a VA application corresponding to a wake up word from the memory180, then execute the VA application, and thereby implement a conversation service.

The memory180may be provided as a non-volatile memory, such as Read Only Memory (ROM) and a flash memory, for storing data for a long time, and a volatile memory, such as a Static Random Access Memory (S-RAM) and Dynamic Random Access Memory (D-RAM), for temporarily storing data. The memory180and the processor140may be implemented as separate chips or integrated into a single chip.

Meanwhile, the display device1may further include various configurations in addition to the above-described configurations, or may omit some of the above-described configurations. According to applied embodiments, the positions or connection relationships of the above-described configurations may change.

FIGS.4to6are views for describing operations of a display device implemented as a user terminal, according to another embodiment.

As described above with reference toFIG.2, the display device1may be provided in the smart phone20aamong user terminals. InFIGS.4to7, the display device1is assumed to be the smart phone20a.

Referring first toFIG.4, a user U may transmit an input for the PTT function to the smart phone20athrough at least one button110ain the user input receiver110.

The smart phone20amay execute the first VA application151according to preset setting information. The first VA application151may output a voice message based on a conversation policy corresponding to the input for the PTT function.

More specifically, the smart phone20amay output, as sound, a voice message, that is, “Hello, I am A” and “What should we do together today”, including a wake up word A corresponding to the first VA application151. Also, the smart phone20amay display “Hello, I am A” and “What should we do together today” on the display panel2.

After the voice message is output, the user U may input an additional utterance voice, and the first VA application151may understand an intention of the user U from the additional utterance voice and then perform a function based on the intention.

Referring toFIG.5, the user U may input an utterance voice without controlling the input button110a. For example, the utterance voice may include “Hi˜B” and “Connect to B”.

The smart phone20amay perform preprocessing on the utterance voice including “Hi˜B” and “Connect to B”. Also, the smart phone20amay transmit the preprocessed utterance voice to the plurality of VA applications150. When “Hi˜B” or a wake up word B is a condition for executing operation of the second VA application152, the second VA application152will operate. The second VA application152may output a voice message corresponding to the utterance voice from the user U according to a conversation policy. For example, the smart phone20amay output a voice message, that is, “Yes, Hello, May I help you?”, generated by the second VA application152.

Meanwhile, the smart phone20amay recognize the wake up word B in a preprocessing process, regardless of operation of the second VA application152. The smart phone20amay change setting information such that the second VA application152corresponding to the wake up word B is included in a connection application for the PTT function. That is, based on the utterance voice from the user U according to the embodiment ofFIG.5, the smart phone20amay change the first VA application151included in the setting information to the second VA application152.

To notify the user U that the setting information has changed, the smart phone20amay display a guide sentence “PTT automatically connects to B” on the display panel2without outputting a voice message.

Referring toFIG.6, after the setting information changes, the user U may execute the PTT function through the input button110a. The smart phone20amay operate the second VA application152based on an input command, and the second VA application152may output “Yes, I am B, May I help you?” based on a conversation policy.

Meanwhile, the voice message and guide sentence included inFIGS.4to6may be an example. That is, the display device1may output another voice message and another output sentence, instead of various methods described above.

FIG.7shows an example of a user interface related to activation/deactivation of setting information.

The display device1may display a user interface for selecting activation/deactivation of a function of automatically changing setting information. When the display device1is implemented as the smart phone20a, the smart phone20amay induce a selection by a user U by outputting a text D2of “Auto Connection Activation”, as shown inFIG.7.

When the user U selects On of Auto Connection Activation, the smart phone20amay automatically change setting information to correspond to a wake up word recognized from an utterance voice from the user U.

When the user U selects Off of Auto Connection Activation, the smart phone20amay not change setting information although a wake up word for the second VA application152is included in an utterance voice from the user U. In this case, when an input command for the PTT function is again input, the smart phone20amay execute the first VA application151.

Meanwhile, the embodiment shown inFIG.7is for convenience of description, and various user interfaces may be possible. For example, the text D2of “Auto Connection Activation” may also change.

FIG.8is a flowchart showing a method for controlling a display device according to an embodiment.

Referring toFIG.8, the display device1may store setting information for connecting an input by a user U to the first VA application151among the plurality of VA applications150(300).

The setting information may be stored in the memory180, and change according to an input by a user U.

The display device1may receive an utterance voice (310).

The display device1may receive the utterance voice through the sound receiver120such as a microphone. The voice receiver120may change the utterance voice into an electrical signal, and transfer the electrical signal to the processor140.

The display device1may perform preprocessing on the utterance voice (320).

More specifically, the processor140may perform preprocessing on the utterance voice based on at least one of noise removal, echo canceling, or beamforming.

The display device1may recognize a wake up word from the preprocessed utterance voice (330).

That is, the processor140may recognize the wake up word from the preprocessed utterance voice before transferring the preprocessed utterance voice to the plurality of VA applications150.

The display device1may identify the second VA application152corresponding to the recognized wake up word (340).

For example, when a wake up word B for the second VA application152is recognized from an utterance by the user U, the display device1may identify the second VA application152corresponding to the wake up word B.

The display device1may change the first VA application151included in the setting information to the second VA application152(350).

More specifically, the display device1may compare the second VA application152corresponding to the wake up word with the first VA application151included in the setting information. When the wake up word is recognized as ‘A’, the display device1may identify the first VA application151. Because the first VA application151included in the setting information is identical to the identified first VA application151, the display device1may not change the setting information. As another example, when a third VA application153is recognized, the display device1may change the stored setting information from the first VA application151to the third VA application153.

FIG.9is a flowchart for describing various methods for changing setting information in a display device.

Referring toFIG.9, the display device1may store setting information for connecting an input command to the first VA application151among the plurality of VA applications150(400).

The display device1may receive an utterance voice (410), and perform preprocessing on the utterance voice (420). Then, the display device1may recognize a wake up word for a preset time period (430).

The display device1may determine whether an utterance voice is input for the preset time period. For example, a user U may input an utterance voice including no wake up word. To accurately recognize whether a wake up word is included, the display device1may wait an input of another utterance voice for the preset time period.

After the preset time period elapses, the display device1may recognize a wake up word from an utterance voice input for the preset time period. When a wake up word for the first VA application151is included in the recognized utterance voice (YES in440), the display device1may execute a conversation service of the first VA application151.

That is, because a wake up word for the first VA application151included in the setting information is identical to the wake up word recognized from the utterance voice, the display device1may operate the first VA application151without changing the setting information.

When the wake up word for the first VA application151is not included in the recognized utterance voice (NO in440), the display device1may determine whether the recognized wake up word is identical to a wake up word for the second VA application152.

When the recognized wake up word does not include the wake up word for the second VA application152(NO in450), the display device1may wait to recognize an utterance voice (430).

When the recognized wake up word includes the wake up word for the second VA application152(YES in450), the display device1may determine the number of the wake up words included in the preprocessed utterance voice (460).

To prevent the setting information from changing by a one-time wrong input of the wake up word, the display device1may change the setting information when a preset number of the wake up words or more is input.

When the preset number of the wake up words or more is recognized (YES in460), the display device1may change the setting information to the second VA application152corresponding to the recognized wake up word (470), and execute a conversation service of the second VA application152to enable the second VA application152to process an utterance by the user U (480).

When the preset number of the wake up word or more is not recognized (NO in460), the display device1may execute the conversation service of the second VA application152without changing the setting information (470) to enable the second VA application152to process an utterance by the user U (480).

Thereby, the display device may reduce inconvenience of a user U and easily execute a conversion service by changing setting information for connecting to the conversation service with an input button for performing a PTT function through a voice of a wake up word recognized in advance.

Meanwhile, the methods according to various embodiments of the disclosure may be included in a machine-readable storage medium and provided.

The machine-readable storage medium may be provided in the form of a non-transitory storage medium, wherein the term ‘non-transitory storage medium’ simply means that the storage medium is a tangible device, and does not include a signal (e.g., an electromagnetic wave), but this term does not differentiate between where data is semi-permanently stored in the storage medium and where the data is temporarily stored in the storage medium. For example, a ‘non-transitory storage medium’ may include a buffer in which data is temporarily stored.

According to an embodiment, the methods according to various embodiments of the disclosure may be included and provided in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or be distributed (e.g., downloadable or uploadable) online via an application store (e.g., Play Store™) or between two user devices (e.g., smart phones) directly. When distributed online, at least part of the computer program product (e.g., a downloadable app) may be temporarily generated or at least temporarily stored in the machine-readable storage medium, such as a memory of the manufacturer's server, a server of the application store, or a relay server.