ELECTRONIC DEVICE FOR DISPLAYING VIRTUAL OBJECT AND OPERATION METHOD THEREOF

A first electronic device is provided. The electronic device includes memory, a communication module, a display, and at least one processor. The memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, display a first virtual object, identify an input to the first virtual object, transmit information about a first event corresponding to the input to the first virtual object and first identification information of the first electronic device to the server when the first virtual object is in the active state, receive, second identification information of one among multiple electronic devices connected to the server and information about a second event for the first virtual object, and execute the first event when the second identification information corresponds to the first identification information.

BACKGROUND

Field

The disclosure relates to an electronic device for displaying a virtual object and a method for operating the same.

Description of Related Art

Metaverse-related content is being actively provided. For example, augmented reality-based metaverse-related content in which additional information is superimposed on the real-world environment may be provided. As metaverse-related content is advanced, metaverse services may be accessed by multiple users.

An increasing number of various services and additional functions are being provided through wearable electronic devices such as augmented reality glasses (AR glasses), video see-through (VST) devices, and head-mounted display (HMD) devices. In order to increase the utility value of these electronic devices and satisfy the needs of different users, telecommunications service providers or electronic device manufacturers are competitively developing electronic devices to provide various features and differentiate themselves from other companies. As a result, various functions provided through wearable electronic devices are becoming increasingly advanced.

AR glasses or VR glasses may provide a realistic experience to a user by displaying virtual images while worn on the user's body. AR glasses or video see-through (VST) devices may replace the usability of a smartphone in various fields such as game entertainment, education, and social networking service (SNS). Users may be provided with content similar to reality through AR glasses or the video see-through (VST) devices, and may experience a sensation being in a virtual world through interactions.

SUMMARY

Aspects of the disclosure are to address at least the above-mentioned problems and/or disadvantages and to provide at least the advantages described below.

According to an embodiment, a first electronic device may include memory, a communication module, a display, and a processor.

According to an embodiment, the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, perform a connection to a server through the communication module.

According to an embodiment, the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, display a first virtual object through the display, based on information about the first virtual object received from the server.

According to an embodiment, the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, identify a first input to the first virtual object.

According to an embodiment, the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, based on the first input to the first virtual object, identify whether the first virtual object is in an active state.

According to an embodiment, the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, transmit information about a first event corresponding to the first input to the first virtual object and first identification information of the first electronic device to the server when the first virtual object is in the active state.

According to an embodiment, the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, receive, from the server, second identification information of one among a plurality of electronic devices connected to the server and information about a second event for the first virtual object.

According to an embodiment, the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, execute the first event when the second identification information corresponds to the first identification information.

According to an embodiment, a method for operating a first electronic device may include performing a connection to a server through a communication module included in the first electronic device.

According to an embodiment, the method for operating the first electronic device may include displaying a first virtual object through a display included in the first electronic device, based on information about the first virtual object received from the server.

According to an embodiment, the method for operating the first electronic device may include identifying a first input to the first virtual object.

According to an embodiment, the method for operating the first electronic device may include, based on the first input to the first virtual object, identifying whether the first virtual object is in an active state.

According to an embodiment, the method for operating the first electronic device may include transmitting a first event corresponding to a third input to the first virtual object and first identification information of the first electronic device to the server when the first virtual object is in the active state.

According to an embodiment, the method for operating the first electronic device may include receiving, from the server, second identification information of one among a plurality of electronic devices connected to the server and information about a second event for the first virtual object.

According to an embodiment, the method for operating the first electronic device may include executing the first event in case that the second identification information corresponds to the first identification information.

According to an embodiment, a non-transitory computer-readable recording medium may be configured to store at least one instruction to perform a connection to a server through a communication module included in a first electronic device.

According to an embodiment, the non-transitory computer-readable recording medium may be configured to store at least one instruction to display a first virtual object through a display included in the first electronic device, based on information about the first virtual object received from the server.

According to an embodiment, the non-transitory computer-readable recording medium may be configured to store at least one instruction to identify a first input to the first virtual object.

According to an embodiment, the non-transitory computer-readable recording medium may be configured to store at least one instruction to, based on the first input to the first virtual object, identify whether the first virtual object is in an active state.

According to an embodiment, the non-transitory computer-readable recording medium may be configured to store at least one instruction to transmit a first event corresponding to the first input to the first virtual object and first identification information of the first electronic device to the server when the first virtual object is in the active state.

According to an embodiment, the non-transitory computer-readable recording medium may be configured to store at least one instruction to receive, from the server, second identification information of one among a plurality of electronic devices connected to the server and information about a second event for the first virtual object.

According to an embodiment, the non-transitory computer-readable recording medium may be configured to store at least one instruction to execute the first event when the second identification information corresponds to the first identification information.

DETAILED DESCRIPTION

FIG.1is a block diagram illustrating an electronic device101in a network environment100according to an embodiment of the disclosure.

FIG.2is a perspective view illustrating the internal configuration of a wearable electronic device according to an embodiment of the disclosure.

Referring toFIG.2, a wearable electronic device200according to an embodiment of the disclosure may include at least one among a light output module211, a display member201, and a camera module250.

According to an embodiment of the disclosure, the light output module211may include a light source capable of outputting an image and a lens configured to guide the image to the display member201. According to an embodiment of the disclosure, the light output module211may include at least one of a liquid crystal display (LCD), a digital mirror device (DMD), a liquid crystal on silicon (LCoS), an organic light-emitting diode (OLED), or a micro light-emitting diode (micro-LED).

According to an embodiment of the disclosure, the display member201may include an optical waveguide (e.g., a wave guide). According to an embodiment of the disclosure, an image incident on the optical waveguide and output by the light output module211may be propagated within the optical waveguide and provided to a user. According to an embodiment of the disclosure, the optical waveguide may include at least one of at least one diffractive element (e.g., a diffractive optical element (DOE) or a holographic optical element (HOE)) or a reflective element (e.g., a reflective mirror). For example, the optical waveguide may use the at least one diffractive element or reflective element to guide the image output by the light output module211to a user's eye.

According to an embodiment of the disclosure, the camera module250may capture a still image and/or a moving image. According to an embodiment, the camera module250may be disposed in a lens frame and disposed around the display member201.

According to an embodiment of the disclosure, a first camera module251may capture and/or recognize a trajectory of the user's eyes (e.g., pupils or irises) or gaze. According to an embodiment of the disclosure, the first camera module251may periodically or aperiodically transmit information related to the trajectory of the user's eye or gaze (e.g., trajectory information) to a processor (e.g., the processor120inFIG.1).

According to an embodiment of the disclosure, a second camera module253may capture an external image.

According to an embodiment of the disclosure, a third camera module255may be used to detect and track a hand and to recognize the user's gesture (e.g., hand motion). The third camera module255according to an embodiment of the disclosure may be used for 3 degrees of freedom (3 DoF) or 6 DoF head tracking, location (space, environment) recognition, and/or movement recognition. According to an embodiment of the disclosure, the second camera module253may also be used to detect and track a hand and to recognize the user's gesture. According to an embodiment, at least one among the first camera module251and the third camera module255may be replaced by a sensor module (e.g., a LiDAR sensor). For example, the sensor module may include at least one of a vertical cavity surface emitting laser (VCSEL), an infrared sensor, and/or a photodiode.

FIGS.3A and3Billustrate the front and rear surfaces of a wearable electronic device according to an embodiment of the disclosure.

Referring toFIGS.3A and3B, in an embodiment, camera modules311,312,313,314,315, and316and/or a depth sensor317may be disposed on a first surface310of a housing so as to acquire information related to the surrounding environment of a wearable electronic device300.

In an embodiment, the camera modules311and312may acquire images related to the surrounding environment of the wearable electronic device.

In an embodiment, the camera modules313,314,315, and316may acquire images while the wearable electronic device is worn by the user. The camera modules313,314,315, and316may be used for hand detection, tracking, and recognition of a user's gesture (e.g., hand motion). The camera modules313,314,315, and316may be used for 3 DoF or 6 DoF head tracking, location (space, environment) recognition, and/or movement recognition. In an embodiment, camera modules311and312may be used for hand detection and tracking, and the user's gesture.

In an embodiment, the depth sensor317may be configured to transmit a signal and receive a signal reflected from a subject, and may be used to identify a distance to an object, such as time of flight (TOF). Instead of or in addition to the depth sensor317, the camera modules313,314,315, and316may identify the distance to the object.

According to an embodiment, camera modules325and326for face recognition and/or a display321(and/or a lens) may be disposed on a second surface320of the housing.

In an embodiment, the camera modules325and326for face recognition, adjacent to the display, may be used to recognize a user's face or may recognize and/or track both eyes of the user.

In an embodiment, the display321(and/or the lens) may be disposed on the second surface320of the wearable electronic device300. In an embodiment, the wearable electronic device300may not include camera modules315and316among the multiple camera modules313,314,315, and316. Although not shown inFIGS.3A and3B, the wearable electronic device300may further include at least one of the elements shown inFIG.2.

As described above, according to an embodiment, the wearable electronic device300may have a form factor to be worn on a user's head. The wearable electronic device300may further include a strap and/or a wearing member to be fixed on the user's body part. The wearable electronic device300, while worn on the user's head, may provide a user experience based on augmented reality, virtual reality, and/or mixed reality.

FIG.4Aillustrates an artificial reality providing system according to an embodiment of the disclosure.

Referring toFIG.4A, the artificial reality providing system may include at least one of a server400, a first electronic device401, at least one external electronic device421,422,423, or424, or an external server440.

According to an embodiment, the server400may produce data for representing artificial reality (e.g., at least one among augmented reality (an augmented reality environment) or virtual reality (a virtual reality environment)). In addition to augmented reality or virtual reality, the server400may provide content capable of enhancing a user's immersion, and such content may be referred to as content for the metaverse. The server400may include a processor410, a memory420, and/or a communication device407. The server400including the processor410, the memory420, and/or the communication device407is merely illustrative, and at least some of operations of the server400may be implemented by a cloud server. The server400may be implemented as a distributed server, and those skilled in the art will understand that there is no limitation on the implementation form of the server.

According to an embodiment, the processor410may execute commands (or instructions) included in a program (or an application) stored in the memory420. The processor410may include, for example, a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), a tensor processing unit (TPU), a digital signal processor (DSP), and an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), and/or a programmable logic device, but any means capable of executing a program (or an instruction or a command) may be included without limitation. The processor410may execute a program for artificial reality. The memory420may store a program for artificial reality. According to an embodiment, the memory420may include volatile memory and/or non-volatile memory, and may include, for example, but is not limited to, for example, a hard disk storage device, RAM, ROM, and/or flash memory which are illustrative. The program for artificial reality, as a program for a server, may cause, for example, the generation of data to represent the artificial reality, the provision of the generated data, the verification of user input, and/or the generation and provision of updated data to represent the artificial reality based on the verified user input, and may include commands (or instructions) corresponding to at least some of the operations performed by the server400of the disclosure. The communication device407may assist in establishing a communication channel between the server400and the first electronic device401through a network450and performing communication through the established communication channel. The communication device407may be, but is not limited to, a device capable of providing a wide area network (e.g., the Internet). The operations performed by the server400may be performed, for example, by the processor410, or by other hardware under the control of the processor410. The commands (or instructions) causing the operations to be performed by the server400may be stored in the memory420. The processor410, the memory420, and/or the communication device407may send and receive data through a bus408(or, a communication interface or a network) of the server400.

According to an embodiment, the first electronic device401may use the data for representing the artificial reality to perform at least one operation for representing the artificial reality, (e.g., which may include, but is not limited to, providing visual content (e.g., an image), providing auditory content (e.g., a voice), providing tactile content (e.g., a vibration), and/or providing olfactory content (e.g., an odor). A user who owns or wears the first electronic device401may experience artificial reality based on content provided by the first electronic device401. The first electronic device401may include at least one of a processor411, a memory412, an input/output device413, a display414, a sensor device415, a camera416, or a communication device417. The processor411may include, for example, CPU, GPU, NPU, TPU, DSP, ASIC, FPGA, and/or a programmable logic device, but any means capable of executing a program (or an instruction or a command) may be included without limitation. For example, the processor411may execute a program for artificial reality. The program for artificial reality, as a client program, may cause, for example, the reception of data for representing the artificial reality from the server400, the carrying-out of at least one operation for representing the artificial reality based on the received data (e.g., which may include, but is not limited to, providing visual content (e.g., an image), providing auditory content (e.g., a voice), providing tactile content (e.g., a vibration), and/or providing olfactory content (e.g., an odor)), the identification of user input, and/or the transmission of the user input (or a command corresponding to the user input) to the server, and performing the server400. In addition, the program for artificial reality may include commands (or instructions) corresponding to at least some of operations performed by the first electronic device401of the disclosure. According to an embodiment, the memory412may include volatile memory and/or non-volatile memory, and may include, but is not limited to, for example, a hard disk storage device, RAM, ROM, and/or flash memory which are illustrative.

According to an embodiment, the input/output device413may include a touch pad, a button, a mouse, a digital pen, and/or a microphone, but any device for receiving (or sensing) a user input may be included without limitation. For example, a touch screen panel, which is an example of the input/output device413, may be implemented integrally with the display414. The input/output device413may include a speaker, a haptic module, and/or a light-emitting module, but any device for outputting content related to artificial reality may be included without limitation.

According to an embodiment, the sensor device415may include a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, and a humidity sensor, and/or an illuminance sensor. According to an embodiment, the camera416may include one or more lenses, image sensors, image signal processors, or flashes.

According to an embodiment, the communication device417may support establishing a communication channel between the server400and the first electronic device401through the network450and performing communication through the established communication channel. The communication device417may be, but is not limited to, a device capable of providing a wide area network (e.g., the Internet). The communication device417may support wired and/or wireless communications. For example, the communication device417may support short-range communication (e.g., short-range communication such as Bluetooth, wireless fidelity (Wi-Fi) direct, or infrared data association (IrDA)). The communication device417may transmit and receive data to and from an external sensor431and/or an external controller433, based on short-range communication. For example, when the first electronic device401is implemented as a standalone type, the communication device417may support a function of wirelessly accessing the network450. The communication device417may support cellular communication, such as LTE, 5G, or 6G, and/or IEEE 802 series-based communication (which may be named Wi-fi, for example). The communication device117may be implemented to support wired communication, and the implementation method thereof is not limited. When the first electronic device401is implemented as a non-standalone type, the first electronic device401may communicate with the server400through a relay device connectable to the network450. In this case, the communication device417may support short-range communication such as Bluetooth, wireless fidelity (Wi-Fi) direct, or infrared data association (IrDA), and may use short-range communication to communicate with the server400through the relay device. The external sensor431may include a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR sensor, a biometric sensor, a temperature sensor, a humidity sensor, and/or an illuminance sensor.

The operations performed by the first electronic device401may be performed, for example, by the processor411, or by other hardware under the control of the processor411. Commands (or instructions) causing the operations to be performed by the first electronic device401may be stored in the memory412. The processor411, the memory412, the input/output device413, the display414, the sensor device415, the camera416, and/or the communication device417may transmit/receive data through a bus418(or a communication interface or a network) of the first electronic device401. The transmitting and receiving of data by the server400and the first electronic device401based on an application for artificial reality is merely illustrative, and those skilled in the art will understand that the server400and the first electronic device401may also transmit and receive at least some of data on a web basis.

According to an embodiment, the external sensor431may be, for example, a ring-type device, a bracelet-type device, or a head-mounted device, but the type of the external sensor431and/or the user's body part to which the external sensor431is attached are not limited. The external sensor431may provide sensed data to the first electronic device401based on short-range communication. The controller433may include, for example, a touch pad, a button, a mouse, a digital pen, and/or a microphone, but any device for receiving (or sensing) a user input may be included without limitation. The controller433may provide the acquired data to the first electronic device401based on short-range communication. In an embodiment, the controller433may further include at least one sensor in addition to the device for receiving a user input. In this case, the controller433may provide sensing data and/or data associated with the user input to the first electronic device401based on short-range communication.

According to an embodiment, the server400may transmit and receive data to and from the at least one external electronic device421,422,423, or424. The server400may transmit, to the first electronic device401, data for representing artificial reality updated and/or changed based on data from the at least one external electronic device421,422,423, or424. The first electronic device401may perform at least one operation for representing the artificial reality based on the data. Thus, when there are multiple users in one artificial reality, an artificial reality reflecting the behavior of one user may be provided to other users.

According to an embodiment, the external server440may transmit and receive data to and from the server400over the network450. The external server440may be, for example, a server that supports the same application (or the same artificial reality) as the server400. Alternatively, the external server440may be a server that supports a different application (or a different artificial reality) from the server400. In this case, the server400may convert the data from the external server440into the format of an application (or artificial reality) supported by the server400. The server400may transmit, to the first electronic device401, data for representing artificial reality reflecting the converted data. As described above, the server400may also interact with artificial reality that is different from artificial reality supported thereby, and this function may be referred to as a multiverse function.

According to an embodiment, the first electronic device401may be a head-mounted device (HMD) for supporting virtual reality, or a smartphone connectable to a structure that is fixable to the head. While wearing the HMD on the head or the structure to which the smartphone is connected on the head, the user may binocularly observe a left-eye image and a right-eye image for representing virtual reality displayed on the display414. Alternatively, the user may observe images for representing virtual reality displayed on the display414of the first electronic device401without wearing the first electronic device401on the head. In this case, the first electronic device401may be implemented as, but is not limited to, a smartphone, a tablet, a general-purpose computer, or a smart mirror.

According to an embodiment, the server400may produce data for representing at least one space of virtual reality (or a scene of viewing the space). For example, the server400may receive first user information (e.g., account information and/or authentication information of a first user) from the first electronic device401. Based on the first user information, the server400may perform a login procedure for the first user. The server400may identify a space corresponding to the first user in the virtual reality. For example, the server400may identify a space privately allocated to the first user. For example, the server400may identify a space corresponding to the first user's location from among opened spaces. For example, the server400may identify a space corresponding to the user's input.

There are no limitations on how the server400identifies the space corresponding to the first user's location. For example, the identified space may include at least one object and/or an avatar (or a character) corresponding to the user. When the viewpoint of a scene is a first-person viewpoint, data for representation may relate to the scene in which the identified space is viewed from the user's viewpoint. In some cases, the scene in which the identified space is viewed may not include an avatar (or a character) corresponding to the first user, or may include only a part (e.g., a hand, etc.) of the body, but is not limited thereto. Alternatively, a back view of the avatar may be included. When the viewpoint of the scene is third-person viewpoint, data for representation may relate to a scene in which a space including the avatar (or character) corresponding to the user is viewed from one direction.

According to an embodiment, the scene viewed from the user's viewpoint may include an avatar corresponding to another user. For example, a second user may access the server400by using the external electronic device422. The server400may produce data for representing artificial reality used by both the first user and the second user. For example, when both the first user and the second user are present in a particular space, the server400may produce data for representing artificial reality that is used by both the first user and the second user. For example, when the viewpoint of a scene is a first-person viewpoint, a scene for the first user may include at least a part of an avatar of the second user. For example, when the viewpoint of a scene is a third-person viewpoint, a scene for the first user may include at least a part of a first avatar (or a character) corresponding to the first user and/or at least a part of a second avatar (or a character) corresponding to the second user. In an embodiment, at least a part of a screen displayed on the first electronic device401may be provided to the server400. At least a part of the screen displayed on the first electronic device401(or an object corresponding to at least a part of the screen) may be disposed in a virtual reality space.

According to an embodiment, the server400may receive a user input and/or a command corresponding to the user input from the first electronic device401. For example, the first electronic device401may identify the user input through the input/output device413. For example, the first electronic device401may identify the user input through the built-in sensor device415. For example, the first electronic device401may acquire the user input from the external sensor431and/or the controller433connected through the communication device417. The processor411may identify, based on sensing data identified through the sensor device415, that movement information of the first electronic device401is the user input.

According to an embodiment, the first electronic device401may identify a command based on a user input. The command may include, for example, but is not limited to, movement within virtual reality, designation of an object within the virtual reality, manipulation of an object within the virtual reality, and/or interaction with another avatar. The first electronic device401may transmit the command to the server400. For example, the first electronic device401may transmit the user input to the server400without identifying the command based on the user input, in which case the server400may identify the command based on the user input.

According to an embodiment, based on the command, the server400may update the virtual reality space, or may change the virtual reality space to a different space. For example, when the command is the designation of an object, the space may be updated to reflect a function associated with the designated object. For example, when the command is manipulation of an object, the space may be updated to change the location of the object. For example, when the command is to perform an action on an avatar, the space may be updated so that the user's avatar performs a corresponding reaction. For example, when the command is an interaction with another avatar, the space may be updated so that the avatar performs a corresponding reaction. For example, when the command is movement, the space for display may be changed to a different space.

Those skilled in the art will understand that there are no limitations on updating and/or changing the virtual reality space based on the command. In addition to updating and/or changing visual content, the server400may provide auditory content, provide tactile content, and/or provide olfactory content. The server400may relay voice data and/or text for chatting between users. For example, the server400may update and/or change the space by using information about the association between the command and the update and/or change. For example, the server400may store an artificial intelligence model that receives a user input and/or a command as an input value and outputs the update and/or change of a space as an output value. The server400may perform updates and/or changes to the space based on the output of the artificial intelligence model. For example, the server400may store an artificial intelligence model that updates and/or changes a space based on the context of the space without any user input. The server400may use the artificial intelligence model to update and/or change a space based on the context of the space.

According to an embodiment, the server400may transmit data for representation of the updated space and/or data for representation of the changed space to the first electronic device401. The server400may transmit the data for the representation of the updated space and/or the data for the representation of the changed space to the external electronic device422corresponding to the second user. Accordingly, on the external electronic device422, virtual reality reflecting the space updated by the first user of the first electronic device401may be represented. Furthermore, based on information (e.g., a user input and/or a command) transmitted to the server400from the external electronic device422, the server400may update the space which the first user and the second user use (or are present in) together. The server400may transmit data for representing the updated space to the first electronic device401. The first electronic device401may represent the updated space, based on the received data.

As described above, the server400may share an updated space corresponding to one user with electronic devices of other users corresponding to the space. For example, time-series update and/or change of a space may be named a user experience. The server400and/or the first electronic device401may store at least one piece of data associated with the user experience in the memory420and/or412. For example, the server400may store at least one piece of data associated with the user experience on a per-user basis (e.g., per user account). For example, the server400and/or the first electronic device401may store, in the memory420and/or412, data for representing a point in time during the user experience. For ease of description, this may be described as performing a capture of the user experience.

The server400may store data associated with the user experience, and this may be referred to as life logging. The server400may additionally store data associated with a user. For example, the server400may receive at least one piece of sensing data from the first electronic device401and store the same in time series, or update a final value. Based on the at least one piece of sensing data, the server400may produce a user (e.g., an avatar) in virtual reality that corresponds to the user in the real world, and this may be referred to as a digital twin.

According to an embodiment, the first electronic device401may provide content for augmented reality that represents at least one visual object that may be superimposed on a real-world environment viewed by a particular user. Those skilled in the art will understand that at least some of the operations of the server400and/or the first electronic device401described in embodiments regarding virtual reality may also be performed by the server400and/or the first electronic device401described in embodiments regarding augmented reality, and vice versa. According to an embodiment, the first electronic device401may be a glasses-type electronic device for supporting augmented reality, a smart lens, or a smartphone capable of displaying a captured image in real time. A user, while wearing a glasses-type electronic device or a smart lens, may observe a visual object displayed on a transparent display (or a semi-transparent display) of the glasses-type electronic device or the smart lens along with the real-world environment. Alternatively, a user may observe an image captured by a smartphone and a visual object displayed to be superimposed on the image.

According to an embodiment, the first electronic device401may acquire a foreground image through the camera416(e.g., a camera oriented toward the front). The first electronic device401may transmit the foreground image, a part of the foreground image, or3D modelling data acquired based on the foreground image to the server400through the communication device417. The first electronic device401may identify the orientation of the first electronic device401based on a captured image and/or sensing data by the sensor device415. The first electronic device401may transmit data about the orientation of the first electronic device401through the communication device417. The first electronic device401may acquire an image of the user's eyes by using the camera416(e.g., a camera oriented toward the rear). The first electronic device401may identify the user's gaze based on the captured image of the eyes. The first electronic device401may transmit data about the user's gaze through the communication device417.

According to an embodiment, the server400may produce data for representing at least one visual object, which may be superimposed on the real-world environment as viewed by a particular user, as data for representing artificial reality. The server400may analyze, for example, data (e.g., data associated with the foreground image, the orientation of the first electronic device401, and/or the user's gaze) received from the first electronic device401, and may identify at least one visual object based on the result of the analysis. The server400may transmit data for representing the at least one visual object to the first electronic device401through the communication device407. The at least one visual object may be displayed, for example, by the display414of the first electronic device401, and the user may observe the at least one visual object superimposed on the real-world environment. For example, a visual object may have information and/or a shape that is associated with an object being placed in the real-world environment. In this case, the first electronic device401may display the visual object so that the visual object can be observed by the user, as in the case in which the visual object is located in the vicinity of the object being placed in the real environment.

According to an embodiment, the first electronic device401may identify a user input. For example, the user input may be identified through the input/output device413included in the first electronic device401and/or through the external sensor431and/or the controller433. The user input may cause, for example, the designation and/or manipulation of a displayed visual object. The first electronic device401may transmit the user input and/or a command corresponding to the user input to the server400. The server400may, based on the user input and/or the command corresponding to the user input, produce data for representing artificial reality. For example, the server400may identify that the user input is based on the designation and/or manipulation of a visual object and, in response thereto, may transform the visual object, move the visual object, and/or provide another visual object that corresponds to the function of the visual object, but there is no restriction on performing the operations. The server400may transmit, to the first electronic device401, the data for representing artificial reality produced based on the user input and/or the command corresponding to the user input. The first electronic device401may provide, based on the data for representing artificial reality, content associated with the artificial reality. As described above, the server400and/or the first electronic device401may provide a function enabling the user to interact with the visual object.

In an embodiment, the server400may produce an avatar (or a character) corresponding to another user as the data for representing artificial reality. The server400may transmit the avatar (or the character) corresponding to the other user to the first electronic device401. The first electronic device401may display the avatar (or the character) corresponding to the other user by using the received data for representing artificial reality. Accordingly, the user may observe the avatar (or the character) corresponding to the other user that is superimposed on the real-world environment. Accordingly, the user may experience the avatar (or the character) corresponding to the other user as being located in the real-world environment. The avatar (or the character) corresponding to the other user may be manipulated, for example, by a user input acquired from the external electronic device421,422,423, or424, and/or may be manipulated based on the artificial intelligence model stored in the server400. There is no limitation on how to manipulate the avatar (or the character). Based on the manipulation of the avatar (or the character), the server400may transmit data for representing the manipulated avatar (or character) to the electronic device401. The first electronic device401may express the manipulated avatar (or character), based on the received data, and accordingly, the user may experience the avatar (or character) corresponding to the other user as acting in the real-world environment.

As described above, the server400and/or the first electronic device401may store a user experience associated with augmented reality in the memory420and/or412. For example, the server400may store at least one piece of data, which is associated with the user experience associated with the augmented reality, on a per-user basis (e.g., per user account). For example, the server400and/or the first electronic device401may store, in the memory420and/or412, data for a representation of a point in time during the user experience associated with the augmented reality.

It is exemplary that the server400and the first electronic device401may produce data for representing artificial reality to represent the artificial reality. According to an embodiment, the first electronic device401may produce data for representing artificial reality, and/or may produce data for artificial realty based on data from external electronic device421,422,423, or424. In this case, the first electronic device401may produce the data for representing artificial reality without data from the server400.

FIG.4Billustrates an artificial reality providing system in which at least one electronic device is communicatively connected to a server according to an embodiment of the disclosure.

Referring toFIG.4B, according to an embodiment, a first electronic device401may connect to a server400through a communication device417(e.g., the communication device417inFIG.4A). According to an embodiment, a second electronic device402may be connected to the server400through a communication device of the second electronic device402, and a third electronic device403may be connected to the server400through a communication device of the third electronic device403. For example, the first electronic device401, the second electronic device402, and the third electronic device403may be connected to the server400based on various communication methods.

According to an embodiment, the first electronic device401, the second electronic device402, and the third electronic device403may be implemented as any one of a smartphone, augmented reality (AR) glasses, or a video see-through (VST) device.

According to an embodiment, a processor411(e.g., the processor411inFIG.4A) may display a first virtual object through a display414(e.g., the display414inFIG.4A).

According to an embodiment, the processor411may identify an input to the first virtual object. According to an embodiment, the input to the first virtual object may include an input by any one of touch, tap, multi-tap, long tap, drag, or drag and drop. However, this is an example, and embodiments of the disclosure may include various types of inputs.

According to an embodiment, the processor411may identify, based on the input to the first virtual object, whether the first virtual object is in an active state. According to an embodiment, the processor411may identify that the first virtual object is in an active state, based on not receiving event information corresponding to inputs to the first virtual object by multiple electronic devices (e.g., the second electronic device402and the third electronic device403) connected to the server400. According to an embodiment, executing an event for the first virtual object may imply that the first virtual object is selected based on the input to the first virtual object.

According to an embodiment, when the first virtual object is in an active state, the processor411may transmit, to the server400, information about a first event corresponding to the input to the first virtual object and first identification information of the first electronic device401. According to an embodiment, the information about the first event may imply an input to the first virtual object. According to an embodiment, the first identification information may imply unique identification (ID) information of the first electronic device401that is capable of distinguishing the first electronic device401from the remaining electronic devices, other than the first electronic device401, among multiple electronic devices connected to the server400.

According to an embodiment, the processor411may receive, from the server400, second identification information of one of the multiple electronic devices connected to the server400and information about a second event for the first virtual object. According to an embodiment, the information about the second event may imply the start of the input to the first virtual object or the end of the input to the first virtual object.

According to an embodiment, the processor411may identify whether the second identification information matches the first identification information.

According to an embodiment, when the second identification information matches the first identification information, the processor411may identify that the first electronic device401among the multiple electronic devices connected to the server400has first requested an input to the first virtual object.

According to an embodiment, the processor411may execute the first event when the second identification information matches the first identification information. According to an embodiment, the execution of the first event may imply that the first virtual object is selected based on the input to the first virtual object.

According to an embodiment, when the second identification information does not match the first identification information, the processor411may identify that one of the remaining multiple electronic devices connected to the server400, other than the first electronic device401, has been first requested an input to the first virtual object.

According to an embodiment, when it is identified that the second electronic device402requested an input to the first virtual object before the first electronic device401, the first electronic device401may identify, based on the information about the second event, whether the input to the first virtual object by the second electronic device402has ended. According to an embodiment, the information about the second event may include information about the end of the input to the first virtual object by the second electronic device402.

According to an embodiment, when it is identified that the input to the first virtual object by the second electronic device402has not ended the processor411may display the first virtual object in an inactive state through the display414. According to an embodiment, the inactive state may refer to a state in which the processor411does not execute an event for the first virtual object even when the input to the first virtual object is identified.

According to an embodiment, the processor411may display the first virtual object in an active state through the display414when it is identified that the input to the first virtual object by the second electronic device402has ended. According to an embodiment, when a reinput to the first virtual object is identified, the processor411may execute an event corresponding to the reinput to the first virtual object. For example, the event corresponding to the reinput may imply that the first virtual object is selected based on the reinput.

According to an embodiment, the processor411may visually differentiate the first virtual object depending on whether the first virtual object is in an active state and display the same through the display414. For example, the processor411may adjust a preconfigured reference brightness, reference shade, reference color, and/or reference transparency to display the first virtual object in an inactive state. However, this is an example, and in embodiments of the disclosure, the first virtual object may be displayed in an inactive state in a variety of ways.

According to an embodiment, the server400may store the identification information of the multiple electronic devices connected to the server400in a memory420(e.g., the memory420inFIG.4A) in the chronological order in which the identification information is received. For example, the memory420may be implemented as a queue. According to an embodiment, when it is identified that the second electronic device402among the multiple electronic devices connected to the server400has requested an input to the first virtual object before the first electronic device401, the processor411may display the first virtual object in a waiting state through the display414to indicate waiting for execution of the first event.

According to an embodiment, in the waiting state, the processor411may display second identification information of the second electronic device402, which requested an input to the first virtual object before the first electronic device401, around the first virtual object through the display414. According to an embodiment, when the end of the input to the first virtual object by the second electronic device402is identified, the processor411may display the first virtual object in an active state through the display414. For example, in the active state, the processor411may display the first virtual object with the second identification information deleted. According to an embodiment, when a reinput to the first virtual object is identified, the processor411may execute an event corresponding to the reinput to the first virtual object.

In addition, according to an embodiment, an event for the first virtual object may be simultaneously executed by multiple electronic devices connected to the server400. For example, the event for the first virtual object may refer to an event related to a writing action using a virtual pointer displayed through a display. For example, when the multiple electronic devices execute an event related to a writing action using the virtual pointer, a written content may be shared through displays of the multiple electronic devices.

According to an embodiment, the processor411may receive, from the server400, first multiple pieces of identification information of the multiple electronic devices. For example, the first multiple pieces of identification information may be identification information corresponding to multiple electronic devices in which an input to the first virtual object has been identified. According to an embodiment, the processor411may identify, from among the first multiple pieces of identification information, identification information which matches the first identification information of the first electronic device401. According to an embodiment, the processor411may execute the first event when the identification information matching the first identification information of the first electronic device401is identified from among the first multiple pieces of identification information.

According to an embodiment, the server400may receive information, indicating that the execution of the event for the first virtual object has ended, from at least one electronic device, among the multiple electronic devices, in which an input to the first virtual object has ended. According to an embodiment, when identification information matching the first identification information is not identified from among the first multiple pieces of identification information, the processor411may receive, from the server400, second multiple pieces of identification information of the multiple electronic devices, excluding the identification information of the at least one electronic device, among the multiple electronic devices, in which the input to the first virtual object has ended.

According to an embodiment, the processor411may receive, from the server400, information indicating whether the number of second multiple pieces of identification information is less than a predetermined number. According to an embodiment, when the number of second multiple pieces of identification information is less than the predetermined number, the processor411may display the first virtual object in an active state through the display414. According to an embodiment, the predetermined number may refer to the number of multiple electronic devices capable of simultaneously executing an event for the first virtual object.

According to an embodiment, when the number of second multiple pieces of identification information is greater than the predetermined number, the processor411may display the first virtual object in an inactive state through the display414.

Operations of the first electronic device401, described with reference to the drawings below, may be performed by the processor411. However, for ease of description, the operations performed by the processor411will be described as being performed by the first electronic device401.

The second electronic device402and the third electronic device403may perform the same operations as the first electronic device401described above. However, a description of the operations performed by the second electronic device402and the third electronic device403is redundant, and thus will be omitted. The first electronic device401, the second electronic device402, and the third electronic device403may perform the above-described operations of the server400. However, the description of the first electronic device401, the second electronic device402, and the third electronic device403performing the operations of the server400described above is redundant, and thus will be omitted.

FIG.5is a flowchart illustrating a method for operating a first electronic device according to an embodiment of the disclosure.

Referring toFIG.5, an embodiment describes a case where only one of multiple electronic devices connected to a server400may execute, based on an input to a first virtual object, an event for the first virtual object at a particular time.

According to an embodiment, in operation501, a first electronic device401(e.g., the first electronic device401inFIG.4A) may be connected to the server400(e.g., the server400inFIG.4A) through the communication device417(e.g., the communication device417inFIG.4A).

According to an embodiment, in operation503, the first electronic device401may display, based on information about the first virtual object received from the server400, the first virtual object through a display414(e.g., the display414inFIG.4A).

According to an embodiment, in operation505, the first electronic device401may identify an input to the first virtual object. According to an embodiment, the input to the first virtual object may include an input by any one of touching, tapping, multi-tapping, long-tapping, dragging, or dragging and dropping on the first virtual object. In addition, the input to the first virtual object may be implemented in various types of input.

According to an embodiment, in operation507, the first electronic device401may identify, based on the input to the first virtual object, whether the first virtual object is in an active state. According to an embodiment, the first electronic device401may identify that the first virtual object is in an active state, based on not receiving event information corresponding to inputs for the first virtual object by multiple electronic devices connected to the server400.

According to an embodiment, in operation509, the first electronic device401may transmit information about a first event corresponding to the input to the first virtual object and first identification information of the first electronic device401to the server400when the first virtual object is in an active state. According to an embodiment, the information about the first event may imply information corresponding to the input to the first virtual object. According to an embodiment, the first identification information may imply unique identification (ID) information of the first electronic device401that is capable of distinguishing the first electronic device401from the remaining electronic devices, other than the first electronic device401, among the multiple electronic devices connected to the server400.

According to an embodiment, in operation511, the first electronic device401may receive second identification information and information about a second event for the first virtual object. According to an embodiment, the second identification information may include identification information of any one of the multiple electronic devices connected to the server400. According to an embodiment, the information about the second event may imply an input to the first virtual object or the end of the input to the first virtual object.

According to an embodiment, in operation513, the first electronic device401may identify whether the second identification information matches the first identification information.

According to an embodiment, in operation515, the first electronic device401may execute the first event when the second identification information matches the first identification information. According to an embodiment, the execution of the first event may imply that the first virtual object is selected based on the input to the first virtual object.

According to an embodiment, when the second identification information does not match the first identification information, the first electronic device401may identify that a second electronic device402(e.g., the second electronic device402inFIG.4B) of the multiple electronic devices connected to the server400has requested an input to the first virtual object before the first electronic device401. According to an embodiment, the second identification information may refer to identification information of the second electronic device402. According to an embodiment, the second identification information may imply unique identification (ID) information of the second electronic device402that is capable of distinguishing the second electronic device402from the remaining electronic devices, other than the second electronic device402, among the multiple electronic devices connected to the server400.

According to an embodiment, when it is identified that the second electronic device402requested an input to the first virtual object before the first electronic device401, the first electronic device401may identify, based on the information about the second event, whether the input to the first virtual object by the second electronic device402has ended. According to an embodiment, the information about the second event may imply the end of the input to the first virtual object. According to an embodiment, the information about the second event may imply that the second event for the first virtual object is executed based on the end of the input to the first virtual object.

According to an embodiment, the first electronic device401may display the first virtual object in an inactive state through the display414when it is identified that the input to the first virtual object by the second electronic device402has not ended. According to an embodiment, the first electronic device401may ignore the input to the first virtual object when the first virtual object is in an inactive state.

According to an embodiment, when the second event information indicating that the input to the first virtual object by the second electronic device402has ended is received from the server400, the first electronic device401may identify that the input to the first virtual object by the second electronic device402has ended. According to an embodiment, when it is identified that the input to the first virtual object by the second electronic device402has ended, the first electronic device401may display the first virtual object in an active state through the display414. According to an embodiment, the first electronic device401may adjust reference brightness, reference shade, reference color, and/or reference transparency depending on whether the first virtual object is in an active state and may display the first virtual object through the display414. For example, the first electronic device401may reflect the reference brightness, the reference shade, the reference color, and/or the reference transparency to the first virtual object to display the first virtual object in an active state. The first electronic device401may adjust the reference brightness, the reference shade, the reference color, and/or the reference transparency to display the first virtual object in an inactive state. According to an embodiment, the operation in which the first electronic device401identifies whether the input to the first virtual object by the second electronic device402ended and displays the first virtual object in an inactive state or an active state will be described in detail with reference toFIG.6A.

Furthermore, according to an embodiment, the server400may store the identification information in a memory420(e.g., the memory420inFIG.4A) in the chronological order in which the identification information is received, based on the input to the first virtual object from the multiple electronic devices connected to the server400. For example, the memory420may be implemented as a queue. According to an embodiment, when it is identified that the second electronic device402among the multiple electronic devices connected to the server400has requested an input to the first virtual object before the first electronic device401, the first electronic device401may display the first virtual object in a waiting state through the display414to indicate the waiting for execution of the first event. According to an embodiment, the first electronic device401may display the first virtual object in an active state through the display414when the end of the input to the first virtual object by the second electronic device402is identified. For example, the server400may receive the identification information of the second electronic device402at a first time point, may receive the identification information of the first electronic device401at a second time point after the first time point, and may receive identification information of a third electronic device403(e.g., the third electronic device403inFIG.4A) at a third time point after the second time point. The first electronic device401and the third electronic device403may display the first virtual object in a waiting state. At this time, the first electronic device401and the third electronic device403may display second identification information of the second electronic device402around the first virtual object. When the end of the input to the first virtual object by the second electronic device402is identified, the first electronic device401may display the first virtual object in an active state through the display414. The third electronic device403may display the first virtual object in a waiting state through the display414until the end of the input to the first virtual object by the first electronic device401is identified. At this time, the third electronic device403may display, in the waiting state, the first identification information of the first electronic device401around the first virtual object. According to an embodiment, the operation of the first electronic device401displaying the first virtual object in a waiting state will be described in detail with reference toFIG.7A.

According to an embodiment, the event for the first virtual object may be executed simultaneously by multiple electronic devices connected to the server400. According to an embodiment, when identification information matching the first identification information of the first electronic device401is not identified from among first multiple pieces of identification information, the first electronic device401may receive, from the server400, second multiple pieces of identification information of the multiple electronic devices, excluding identification information of at least one electronic device, among the multiple electronic device, in which the input to the first virtual object has ended. According to an embodiment, the first electronic device401may display the first virtual object in an active state through the display414when the number of second multiple pieces of identification information is less than a predetermined number. According to an embodiment, the first electronic device401may display the first virtual object in an inactive state through the display414when the number of second multiple pieces of identification information is greater than the predetermined number. For example, the predetermined number may refer to the number of electronic devices that may simultaneously execute the event for the first virtual object. According to an embodiment, the operation of simultaneously executing the event for the first virtual object by the multiple electronic devices connected to the server400will be described in detail with reference toFIG.8A.

According to an embodiment, the above-described operations performed by the second electronic device402and the third electronic device403may be performed by the first electronic device401. However, the description of the first electronic device401performing the above-described operations of the second electronic device402and the third electronic device403is redundant and thus will be omitted.

The first electronic device401, the second electronic device402, and the third electronic device403may perform the above-described operations of the server400. However, the description of the first electronic device401, the second electronic device402, and the third electronic device403performing the above-described operations of the server400is redundant, and thus will be omitted.

FIG.6Ais a flowchart illustrating operations of a first electronic device, a server, and a second electronic device according to an embodiment of the disclosure.

The embodiment illustrated inFIG.6Adescribes a case where only one of multiple electronic devices connected to a server400(e.g., the server400inFIG.4B) is capable of executing an event for a first virtual object at a particular time.

According to an embodiment, in operation607, a first electronic device401(e.g., the first electronic device401inFIG.4A) may be communicatively connected to the server400(e.g., the server400inFIG.4B) through a communication device417(e.g., the communication device417inFIG.4A). According to an embodiment, in operation609, a second electronic device402(e.g., the second electronic device402inFIG.4B) may be communicatively connected to the server400through the communication device.

According to an embodiment, in operation611, the first electronic device401may identify an input to a first virtual object. According to an embodiment, the first electronic device401may identify, based on the input to the first virtual object, whether the first virtual object is in an active state. According to an embodiment, the first electronic device401may identify that the first virtual object is in an active state, based on not receiving second event information corresponding to an input to the first virtual object by the second electronic device402.

According to an embodiment, in operation613, the first electronic device401may transmit first identification information and first event information regarding the first virtual object to the server400, based on the first virtual object being in an active state. For example, first event information regarding the first virtual object may refer to information corresponding to the input to the first virtual object by the first electronic device401.

According to an embodiment, in operation615, the second electronic device402may identify an input to a first virtual object. According to an embodiment, the second electronic device402may identify, based on the input to the first virtual object, whether the first virtual object is in an active state. According to an embodiment, the second electronic device402may identify that the first virtual object is in an active state, based on not receiving the first event information corresponding to the input to the first virtual object by the first electronic device401connected to the server400.

According to an embodiment, in operation617, the second electronic device402may transmit, based on the first virtual object being in an active state, second identification information and second event information regarding the first virtual object to the server400. According to an embodiment, the second identification information may imply identification information that is different from the first identification information. For example, the second event information regarding the first virtual object may imply information corresponding to the input to the first virtual object by the second electronic device402.

According to an embodiment, in operation619, the first electronic device401and the second electronic device402may receive the first identification information and the first event information regarding the first virtual object from the server400. According to an embodiment, the server400may transmit first event information and first identification information, which have been received first, to the first electronic device401and the second electronic device402.

According to an embodiment, in operation621, the first electronic device401may identify that the identification information of the first electronic device401transmitted to the server400matches the identification information received from the server400. According to an embodiment, when the identification information received from the server400matches the identification information transmitted to the server400, it may be identified that the first electronic device401requested the input to the first virtual object before the second electronic device402connected to the server400.

According to an embodiment, in operation623, the second electronic device402may identify that the identification information of the second electronic device402transmitted to the server400does not match the first identification information received from the server400. According to an embodiment, when the second identification information does not match the first identification information, the second electronic device402may identify that the first electronic device401requested the input to the first virtual object before the second electronic device402.

According to an embodiment, in operation625, the second electronic device402may display the first virtual object in an inactive state through a display. According to an embodiment, the second electronic device402may not execute a second event based on identifying that the first electronic device401requested the input to the first virtual object before the second electronic device402. According to an embodiment, the inactive state may include a case where the input to the first virtual object is ignored and no event is executed. According to an embodiment, the second electronic device402may adjust reference brightness, reference shade, reference color, and/or reference transparency, which has been predetermined, to display the first virtual object in an inactive state. According to an embodiment, the execution of the second event may imply that the first virtual object is selected based on the input of the second electronic device402to the first virtual object.

According to an embodiment, in operation627, the first electronic device401may execute a first event, based on matching of the first identification information and the identification information received from the server400. According to an embodiment, the execution of the first event may imply that the first virtual object is selected based on the input of the first electronic device401to the first virtual object.

According to an embodiment, in operation629, the first electronic device401may transmit, to the server400, information indicating that the execution of the first event has ended. According to an embodiment, the information indicating that the execution of the first event has ended may imply information indicating that the input of the first electronic device401to the first object has ended and the first virtual object has been selected.

According to an embodiment, in operation631, the server400may transmit, to the second electronic device402, the information indicating that the execution of the first event by the first electronic device401has ended.

According to an embodiment, in operation633, the second electronic device402may identify the end of the input to the first virtual object by the first electronic device401, based on receiving, from the server400, the information indicating that the execution of the first event by the first electronic device401has ended.

According to an embodiment, in operation635, the second electronic device402may display the first virtual object in an active state through the display, based on identifying the end of the input to the first virtual object by the first electronic device401. According to an embodiment, the second electronic device402may display the first virtual object in an active state with reference brightness, reference shade, reference color, and/or reference transparency, which has been predetermined. According to an embodiment, when a reinput to the first virtual object is identified, the second electronic device402may execute an event corresponding to the reinput. In doing so, the first electronic device401may notify a user of whether the reinput for the execution of an event for the first virtual object is valid.

According to an embodiment, the above-described operations performed on the second electronic device402may also be performed by the first electronic device401. However, a description of the first electronic device401performing the above-described operations of the second electronic device402is redundant and will be omitted.

The first electronic device401and the second electronic device402may perform the above-described operations of the server400. However, a description of the first electronic device401and the second electronic device402performing the above-described operations of the server400is redundant and will be omitted.

FIG.6Bis a view in which a first electronic device and a second electronic device display a first virtual object in an active state through a display according to an embodiment of the disclosure.

Referring toFIG.6B, a first electronic device401(e.g., the first electronic device401inFIG.4B) may display a screen660through a display414(e.g., the display414inFIG.4A), and a second electronic device402(e.g., the second electronic device402inFIG.4B) may display a screen670through a display included in the second electronic device402.

According to an embodiment, the first electronic device401and the second electronic device402may be communicatively connected to a server400(e.g., the server400inFIG.4B).

According to an embodiment, the first electronic device401may display a first virtual object680in an active state through the display414, based on the fact that an input to the first virtual object by the second electronic device402connected to the server400is not identified.

According to an embodiment, the second electronic device402may display a first virtual object690in an active state through the display, based on the fact that an input to the first virtual object by the first electronic device401connected to the server400is not identified.

For example, the first electronic device401and the second electronic device402may display the first virtual object680and690in an active state with reference brightness, reference shade, reference color, and/or reference transparency, which has been predetermined.

FIG.6Cillustrates a screen661on which a first electronic device displays a first virtual object in an event-running state through a display, and a screen671on which a second electronic device displays a virtual object in an inactive state through a display according to an embodiment of the disclosure.

Referring toFIG.6C, according to an embodiment, a first electronic device401(e.g., the first electronic device401inFIG.4B) may display a virtual pointer601through a display414(e.g., the display414inFIG.4A). The virtual pointer601may move in response to movement of a user's body (e.g., the hand or the finger).

According to an embodiment, the first electronic device401may identify an input (e.g., touch) for a first virtual object by the virtual pointer601. According to an embodiment, the first electronic device401may identify, based on the input to the first virtual object, whether the first virtual object is in an active state. According to an embodiment, when it is identified that the first virtual object is in an active state, the first electronic device401may transmit, to a server400, first identification information of the first electronic device401and first event information corresponding to the input to the first virtual object. According to an embodiment, the server400may transmit first identification information and first event information corresponding to the input to the first virtual object, which have been received first, to the first electronic device401and the second electronic device402.

According to an embodiment, the first electronic device401may identify whether the identification information received from the server400matches the identification information of the first electronic device401. According to an embodiment, the first electronic device401may execute a first event based on the matching between the first identification information received from the server400and the identification information of the first electronic device401. For example, the execution of the first event may imply that the first virtual object has been selected. According to an embodiment, when the first event is executed, the first electronic device401may display a first virtual object681in an event-running state through the display414. For example, the first electronic device401may adjust reference brightness, a reference shade, reference color, and/or reference transparency, which has been predetermined, to display the first virtual object681in an event-running state. For example, the first electronic device401may display a shape (e.g., an ellipse, etc.) in the first virtual object having a reference value to display the first virtual object681in an event-running state.

According to an embodiment, when first identification information different from second identification information of the second electronic device402and first event information corresponding to an input to the first virtual object are received from the server400, the second electronic device402may identify that there is a request for the input to the first virtual object by the first electronic device401. According to an embodiment, the second electronic device402may display a first virtual object691in an inactive state when it is identified that there is an input request for the first virtual object by the first electronic device401. For example, the second electronic device402may adjust reference brightness, reference shade, reference color, and/or reference transparency, which has been predetermined, to display the first virtual object691in an inactive state.

FIG.6Dillustrates a screen662on which a first electronic device displays a virtual object in an inactive state through a display and a screen672on which a second electronic device displays a first virtual object in an event-running state through a display according to an embodiment of the disclosure.

Referring toFIG.6D, according to an embodiment, a second electronic device402(e.g., the second electronic device402inFIG.4B) may display a virtual pointer602on a display. The virtual pointer602may move in response to movement of a user's body (e.g., the hand or the finger).

According to an embodiment, the second electronic device402may identify an input (e.g., a touch) to the first virtual object by the virtual pointer602. According to an embodiment, the second electronic device402may identify, based on the input to the first virtual object, whether the first virtual object is in an active state. According to an embodiment, when it is identified that the first virtual object is in an active state, the second electronic device402may transmit, to a server400, second identification information of the second electronic device402and second event information corresponding to the input to the first virtual object. According to an embodiment, the server400may transmit second event information corresponding to the input to the first virtual object and second identification information, which have been received first, to a first electronic device401and the second electronic device402.

According to an embodiment, the second electronic device402may identify whether identification information received from the server400matches identification information of the second electronic device402. According to an embodiment, the second electronic device402may execute a second event based on the matching between the second identification information received from the server400and the identification information of the second electronic device402. For example, the execution of the second event may imply that the first virtual object is selected. According to an embodiment, when the second event is executed, the second electronic device402may display a first virtual object692in an event-running state through a display. For example, the second electronic device402may adjust reference brightness, reference shade, reference color, and/or reference transparency, which has been predetermined, to display the first virtual object692in an event-running. For example, the second electronic device402may display a shape (e.g., an ellipse, etc.) in the first virtual object having a reference value to display the first virtual object692in an event-running state.

According to an embodiment, when second identification information different from identification information of the first electronic device401and second event information corresponding to an input to the first virtual object are received from the server400, the first electronic device401may display the first virtual object in an inactive state. For example, the first electronic device401may adjust reference brightness, reference shade, reference color, and/or reference transparency, which has been predetermined, to display a first virtual object682in an inactive state.

FIG.7Ais a flowchart illustrating operations of a first electronic device, a server, and a second electronic device according to an embodiment of the disclosure.

Referring toFIG.7A, according to an embodiment, multiple electronic devices connected to a server400(e.g., the server400inFIG.4B) may be controlled such that only one of the multiple electronic devices executes an event for a first virtual object at a particular time point.

According to an embodiment, in operation707, a first electronic device401(e.g., the first electronic device401inFIG.4B) may be connected to the server400through a communication device417(e.g., the communication device417inFIG.4A). According to an embodiment, in operation709, a second electronic device402(e.g., the second electronic device402inFIG.4B) may be connected to the server400through the communication device.

According to an embodiment, in operation711, the first electronic device401may identify an input (e.g., touch) to a first virtual object. According to an embodiment, the first electronic device401may determine, based on the input to the first virtual object, whether the first virtual object is in an active state. According to an embodiment, the first electronic device401may identify that the first virtual object is in an active state, based on not receiving second event information corresponding to the input to the first virtual object by the second electronic device402connected to the server400.

According to an embodiment, in operation713, the first electronic device401may transmit first identification information and first event information regarding the first virtual object to the server400. According to an embodiment, the first electronic device401may transmit, based on the identification of the active state of the first virtual object, the first identification information and the first event information corresponding to the input to the first virtual object to the server400.

According to an embodiment, in operation715, the second electronic device402may identify an input (e.g., touch) to the first virtual object. According to an embodiment, the second electronic device402may identify, based on the input to the first virtual object, whether the first virtual object is in an active state. According to an embodiment, the second electronic device402may identify that the first virtual object is in an active state, based on not receiving the first event information corresponding to the input to the first virtual object by the first electronic device401connected to the server400.

According to an embodiment, in operation717, the second electronic device402may transmit second identification information and second event information regarding the first virtual object to the server400. According to an embodiment, the second electronic device402may transmit, based on the identification of the active state of the first virtual object, the second identification information and the second event information corresponding to the input (e.g., touch) for the first virtual object to the server400.

According to an embodiment, in operation719, the server400may store the identification information in chronological order of reception thereof. According to an embodiment, the server400may store the pieces of identification information in a memory420(e.g., the memory420inFIG.4A). For example, the memory420may be implemented as a queue. According to an embodiment, based on the first identification information being received earlier, the server400may store, in the queue, the first identification information received earlier and the second identification information received later. According to an embodiment, the server400may transmit first identification information and first event information, which have been received first, to the first electronic device401and the second electronic device402.

According to an embodiment, in operation721, the first electronic device401and the second electronic device402may receive the first identification information and the first event information regarding the first virtual object from the server400.

According to an embodiment, in operation723, the first electronic device401may compare the first identification information of the first electronic device401transmitted to the server400with the first identification information received from the server400, and may identify a match therebetween as a result of the comparison.

According to an embodiment, in operation725, the second electronic device402may compare the second identification information of the second electronic device402transmitted to the server400with the first identification information received from the server400, and may identify a mismatch therebetween as a result of the comparison.

According to an embodiment, in operation727, the server400may transmit, to the second electronic device402, information indicating that the first virtual object is in a waiting state. According to an embodiment, when it is determined that the first electronic device401, among the multiple electronic devices, has requested an input to the first virtual object before the second electronic device402, the server400may transmit, to the second electronic device402, the information indicating that the first virtual object is in a waiting state.

According to an embodiment, in operation729, the second electronic device402may display the first virtual object in a waiting state through a display. At this time, the second electronic device402may display the first identification information of the first electronic device401around the first virtual object. For example, the server400may receive the first identification information of the first electronic device401at a first time point, may receive the second identification information of the second electronic device402at a second time point after the first time point, and may receive third identification information of a third electronic device403(e.g., the third electronic device403inFIG.4B) at a third time point after the second time point. The server400may store the identification information in the memory420based on the order in which the identification information is received. For example, the memory420may be implemented as a queue. The second electronic device420and the third electronic device403each may display the first virtual object as a waiting state. The second electronic device401and the third electronic device403each may display the first identification information of the first electronic device401around the first virtual object. The second electronic device402may display the first virtual object as in an active state through the display when the end of the input to the first virtual object by the first electronic device401is identified. The third electronic device403may display the first virtual object in a waiting state through a display until the end of the input to the first virtual object by the second electronic device402is identified. In the standby state, the third electronic device403may display the second identification information of the second electronic device402around the first virtual object.

According to an embodiment, in operation731, the first electronic device401may execute a first event. According to an embodiment, the execution of the first event may imply that the first virtual object is selected.

According to an embodiment, in operation733, the first electronic device401may transmit, to the server400, information indicating that the execution of the first event has ended.

According to an embodiment, in operation735, the server400may transmit, to the second electronic device402, the information indicating that the execution of the first event has ended.

According to an embodiment, in operation736, the server400may delete the first identification information stored in the memory420. According to an embodiment, the server400may delete the first identification information stored in the memory420, based on receiving the information indicating that the execution of the first event has ended. WhileFIG.7describes operation736as being performed after operation735, according to another embodiment of the disclosure operation735and operation736may be performed simultaneously.

According to an embodiment, in operation737, the second electronic device402may identify that the end of the input to the first virtual object by the first electronic device401(or the end of the execution of the first event), based on the information indicating that the execution of the first event has ended.

According to an embodiment, in operation739, the second electronic device402may display the first virtual object in an active state through the display based on the end of input to the first virtual object by the first electronic device401. According to an embodiment, when a reinput to the first virtual object is identified, the second electronic device402may transmit, to the server400, the second identification information and event information corresponding to the reinput to the first virtual object. According to an embodiment, the server400may transmit the second identification information stored in the memory420to the second electronic device402. According to an embodiment, the second electronic device402may execute a second event.

According to an embodiment, the above-described operations performed by the second electronic device402may also be performed by the first electronic device401. However, a description of the first electronic device401performing the above-described operations of the second electronic device402is redundant and will be omitted.

The first electronic device401and the second electronic device402may perform the above-described operations of the server400. However, a description of the first electronic device401and the second electronic device402performing the above-described operations of the server400is redundant and will be omitted.

FIG.7Bis a view in which a first electronic device displays a first virtual object in a waiting state through a display according to an embodiment of the disclosure.

Referring toFIG.7B, a first electronic device401(e.g., the first electronic device401inFIG.4B) may display a screen760or761through a display414(e.g., the display414inFIG.4B).

According to an embodiment, a second electronic device402(e.g., the second electronic device402inFIG.4B) may display a first virtual object in an active state when it is identified that the first virtual object is in an active state. The second electronic device402may identify an input to the first virtual object by a user of the second electronic device402. Based on the input to the first virtual object, the second electronic device402may transmit event information about the first virtual object (e.g., a touch input to the first virtual object) and second identification information of the second electronic device402to a server400(e.g., the server400inFIG.4B).

According to an embodiment, after the second electronic device402transmits second event information and the second identification information to the server400, the first electronic device401(e.g., the first electronic device401inFIG.4B) may display the first virtual object in an active state when the first virtual object is determined to be in the active state. The first electronic device401may identify an input to the first virtual object by a user of the first electronic device401. Based on the input to the first virtual object, the first electronic device401may transmit event information about the first virtual object (e.g., a touch input to the first virtual object) and first identification information of the first electronic device401to the server400.

According to an embodiment, the server400may store the second identification information received at a first time point and the first identification information received at a second time point after the first time point in a memory420(e.g., the memory420inFIG.4A) in chronological order. For example, the memory420may be implemented as a queue. According to an embodiment, the server400may transmit second identification information and second event information, which have been received first, to the first electronic device401and the second electronic device402.

According to an embodiment, the first electronic device401may receive, from the server400, the second identification information that is different from the first identification information. The first electronic device401may receive, from the server400, information indicating that the first virtual object is in a waiting state.

According to an embodiment, the first electronic device401may display a first virtual object770in a waiting state. For example, the first electronic device401may display the second identification information around the first virtual object. However, this is an example, and in embodiments of the disclosure, the first virtual object in an active state and the first virtual object in a waiting state may be displayed to be distinguished from each other in various ways. In doing so, the first electronic device401may notify of the fact that another electronic device (e.g., the second electronic device402), among multiple electronic devices connected to the server400, is executing an event for the first virtual object before the first electronic device401. The first electronic device401may notify the user to wait for the execution of the event for the first virtual object. Furthermore, the first electronic device401may visually provide the user with information about the timing of a reinput to the first virtual object.

According to an embodiment, the first electronic device401may receive, from the server400, information indicating that an event (e.g., touch input) for the first virtual object by the second electronic device402has ended. The first electronic device401may display a first virtual object780in an active state, based on the end of the event for the first virtual object by the second electronic device402.

FIG.8Ais a flowchart illustrating operations of a first electronic device, a server, a second electronic device, and a third electronic device according to an embodiment of the disclosure.

Referring toFIG.8A, according to an embodiment, multiple electronic devices connected to a server400(e.g., the server400inFIG.4B) may be controlled such that a predetermined number of electronic devices, among the multiple electronic devices, may simultaneously execute an event for the first virtual object at a particular time point. For example, a predetermined number of electronic devices of the multiple electronic devices may execute an event related to a writing action by using virtual pointers displayed through respective displays thereof. For example, the execution of the event for the first virtual object may include writing using the virtual pointers. For example, when each of a predetermined number of electronic devices among multiple electronic devices executes an event related to a writing action by using a virtual pointer, the writing may be shared by displays of the predetermined number of electronic devices of the multiple electronic devices connected to the server400.

According to an embodiment, in operation803, a third electronic device403(e.g., the third electronic device403inFIG.4A) may be communicatively connected to the server400(e.g., the server400inFIG.4A) through a communication device. According to an embodiment, in operation805, a first electronic device401(e.g., the first electronic device401inFIG.4A) may be communicatively connected to the server400through the communication device417(e.g., the communication device417inFIG.4A). According to an embodiment, in operation807, a second electronic device402(e.g., the second electronic device402inFIG.4A) may be communicatively connected to the server400through a communication device.

According to an embodiment, in operation811, the second electronic device402may identify an input (e.g., touch) to a first virtual object. According to an embodiment, the second electronic device402may identify, based on the input to the first virtual object, whether the first virtual object is in an active state. According to an embodiment, the second electronic device402may identify that the first virtual object is in an active state, based on not receiving event information corresponding to an input to the first virtual object by other electronic devices connected to the server400.

According to an embodiment, in operation813, the second electronic device402may transmit second identification information and second event information regarding the first virtual object to the server400based on identifying that the first virtual object is in an active state. For example, the second event information may refer to information corresponding to an input to the first virtual object.

According to an embodiment, in operation815, the third electronic device403may identify an input (e.g., touch) to a first virtual object. According to an embodiment, the third electronic device403may identify, based on the input to the first virtual object, whether the first virtual object is in an active state. According to an embodiment, the third electronic device403may identify that the first virtual object is in an active state, based on not receiving event information corresponding to inputs to the first virtual object by other electronic devices connected to the server400.

According to an embodiment, in operation817, the third electronic device403may transmit third identification information and third event information regarding the first virtual object to the server400based on identifying that the first virtual object is in an active state. For example, the third event information may refer to information corresponding to an input to the first virtual object.

According to an embodiment, in operation819, the first electronic device401, the second electronic device402, and the third electronic device403may receive, from the server400, the second identification information and the second event information regarding the first virtual object and the third identification information and the third event information regarding the first virtual object. According to an embodiment, the server400may identify whether the number of received multiple pieces of identification information is less than or equal to a predetermined number (e.g., two). According to an embodiment, the predetermined number may refer to the number of electronic devices capable of simultaneously executing an event for the first virtual object. According to an embodiment, when it is determined that the number of received first multiple pieces of identification information is less than or equal to the predetermined number, the server400may transmit all of the received first multiple pieces of identification information to multiple electronic devices connected to the server400. According to an embodiment, when the number of received first multiple pieces of identification information exceeds the predetermined number, the server400may transmit second multiple pieces of identification information, excluding identification information (or pieces of identification information) of an electronic device (or electronic devices) corresponding to the exceeded number, to multiple electronic devices currently connected to the server400. For example, the server400may transmit the second identification information and the second event information and the third identification information and the third event information to the first electronic device401, the second electronic device402, and the third electronic device403, based on the number of the received multiple pieces of identification information (e.g., the second identification information and the third identification information) (e.g., two) being less than or equal to the predetermined number (e.g., two).

According to an embodiment, in operation823, the second electronic device402may identify that the multiple pieces of identification information received from the server400includes identification information that matches the second identification information.

According to an embodiment, in operation825, the third electronic device403may identify that the multiple pieces of identification information received from the server400includes identification information that matches the third identification information.

According to an embodiment, in operation827, the first electronic device401may display the first virtual object in an inactive state through the display414. According to an embodiment, the first electronic device401may receive, from the server400, information indicating that the number of pieces of identification information for which an event of the first virtual object is currently running is not less than a predetermined number. For example, the predetermined number may refer to the number of electronic devices capable of simultaneously executing an event for the first virtual object. According to an embodiment, the first electronic device401may identify, based on the above information, whether the number of received pieces of identification information is less than the predetermined number. According to an embodiment, the first electronic device401may display the first virtual object in an inactive state, based on identifying that the number of received pieces of identification information (e.g., two) is not less than the predetermined number (e.g., two). Alternatively, the first electronic device401may directly determine, based on the pieces of identification information received from the server400, whether the number of received pieces of identification information is less than the predetermined number.

According to an embodiment, in operation828, the second electronic device402may execute a second event for the first virtual object. According to an embodiment, the second electronic device402may execute the second event based on identifying identification information matching the second identification information from among the received multiple pieces of identification information. For example, when the second event is executed, the second electronic device402may perform an event related to a writing action using a virtual pointer displayed through the display.

According to an embodiment, in operation829, the second electronic device402may end the execution of the second event for the first virtual object. For example, the end of the execution of the second event may imply ending the event related to the writing action using the virtual pointer.

According to an embodiment, in operation830, the third electronic device403may execute a third event for the first virtual object. According to an embodiment, the third electronic device403may execute the third event based on identifying identification information matching the third identification information from among the received multiple pieces of identification information. For example, when the third event is executed, the third electronic device403may perform an event related to a writing action using a virtual pointer displayed through the display.

According to an embodiment, in operation831, the second electronic device402may transmit, to the server400, information indicating that the execution of the second event has ended.

According to an embodiment, in operation833, the server400may identify the end of the execution of the second event for the first virtual object by the second electronic device402.

According to an embodiment, in operation834, the server400may delete, from a memory420(e.g., the memory420inFIG.4A), the second event information and the second identification information of the second electronic device402that has ended the execution of the event for the first virtual object.

According to an embodiment, in operation835, the server400may transmit the third identification information and the third event for the first virtual object to the first electronic device401, the second electronic device402, and the third electronic device403.

According to an embodiment, in operation837, the first electronic device401may display the first virtual object in an active state through the display414based on identifying that the number of pieces of identification information received from the server400is less than the predetermined number (e.g., two). For example, the first electronic device401may receive, from the server400, information indicating that the number of pieces of identification information for which an event of the first virtual object is currently being executed is less than the predetermined number.

According to an embodiment, the first electronic device401may display the first virtual object in an inactive state through the display414based on identifying that the number of pieces of identification information received from the server400is greater than the predetermined number. For example, the first electronic device401may receive, from the server400, information indicating that the number of pieces of identification information for which an event of the first virtual object is currently being executed is greater than the predetermined number.

The first electronic device401, the second electronic device402, and the third electronic device403may perform the above-described operations of the server400. However, a description of the first electronic device401, the second electronic device402, and the third electronic device403performing the above-described operations of the server400is redundant and will be omitted.

FIG.8Bis a flowchart illustrating operations of a first electronic device, a server, a second electronic device, and a third electronic device according to an embodiment of the disclosure.

Referring toFIG.8B, a predetermined number of electronic devices among multiple electronic devices connected to a server400(e.g., the server400inFIG.4B) may execute an event for a first virtual object at a particular time point.

According to an embodiment, in operation841, a third electronic device403(e.g., the third electronic device403inFIG.4A) may be connected to a server400(e.g., the server400inFIG.4A) through a communication device. According to an embodiment, in operation843, a second electronic device402(e.g., the second electronic device402inFIG.4A) may be connected to the server400through a communication device. According to an embodiment, in operation845, a first electronic device401(e.g., the first electronic device401inFIG.4A) may be connected to the server400through a communication device417(e.g., the communication device417inFIG.4A).

According to an embodiment, in operation847, the first electronic device401may identify an input (e.g., a touch) to a first virtual object. According to an embodiment, the first electronic device401may identify, based on the input to the first virtual object, whether the first virtual object is in an active state. According to an embodiment, the first electronic device401may identify that the first virtual object is in an active state, based on not receiving event information corresponding to inputs to the first virtual object by other electronic devices connected to the server400.

According to an embodiment, in operation849, the first electronic device401may transmit first identification information and first event information regarding the first virtual object to the server400, based on identifying that the first virtual object is in an active state.

According to an embodiment, in operation851, the third electronic device403may identify an input to a first virtual object.

According to an embodiment, in operation853, the third electronic device403may transmit third identification information and third event information regarding the first virtual object to the server400.

According to an embodiment, in operation855, the first electronic device401, the second electronic device402, and the third electronic device403may receive, from the server400, the first identification information and the first event information regarding the first virtual object and the third identification information and the third event information regarding the first virtual object. For example, the first event information and the third event information may refer to information corresponding to an input to the first virtual object. According to an embodiment, the server400may identify whether the number of received multiple pieces of identification information is less than or equal to a predetermined number (e.g., two). According to an embodiment, the predetermined number may refer to the number of electronic devices capable of simultaneously executing an event for the first virtual object. According to an embodiment, when it is determined that the number of received first multiple pieces of identification information (e.g., the first identification information and the third identification information) is less than or equal to the predetermined number, the server400may transmit all of the received first multiple pieces of identification information to multiple electronic devices connected to the server400. For example, when the number of received first multiple pieces of identification information exceeds the predetermined number, second multiple pieces of identification information, excluding identification information (or pieces of identification information) of an electronic device (or electronic devices) corresponding to the exceeded number, may be transmitted to multiple connected electronic devices.

According to an embodiment, in operation857, the first electronic device401may compare the first identification information of the first electronic device401transmitted to the server400with the pieces of identification information received from the server400and identify that identification information matching the first identification information is included in the received pieces of identification information.

According to an embodiment, at operation859, the third electronic device403may compare the third identification information of the third electronic device403transmitted to the server400with the pieces of identification information received from the server400and identify that there is identification information matching the third identification information is included in the received pieces of identification information.

According to an embodiment, in operation861, the second electronic device402may display the first virtual object in an inactive state through a display. According to an embodiment, the second electronic device402may identify whether the number of received pieces of identification information is less than a predetermined number. For example, the predetermined number may refer to the number of electronic devices capable of simultaneously executing an event for the first virtual object. According to an embodiment, the first electronic device401may receive, from the server400, information indicating that the number of pieces of identification information for which an event for the first virtual object is currently executed is not less than the predetermined number. According to an embodiment, the second electronic device402may identify, based on the above information, that the number of received pieces of identification information is not less than the preset number. According to an embodiment, the second electronic device402may display the first virtual object in an inactive state, based on the identification that the number of received pieces of identification information (e.g., two) is not less than the predetermined number (e.g., two). Alternatively, the second electronic device402may directly determine, based on the pieces of identification information received from the server400, whether the number of received pieces of identification information is less than the predetermined number.

According to an embodiment, in operation863, the first electronic device401may execute a first event for the first virtual object. According to an embodiment, the first electronic device401may execute the first event based on identifying identification information matching the first identification information from among received multiple pieces of identification information. For example, when the first event is executed, the first electronic device401may perform an event related to a writing action using a virtual pointer displayed through a display.

According to an embodiment, in operation865, the first electronic device401may end the execution of the first event for the first virtual object. For example, ending the execution of the first event may imply ending the event related to the writing action using the virtual pointer.

According to an embodiment, in operation867, the third electronic device403may execute a third event for the first virtual object. According to an embodiment, the third electronic device403may execute the third event based on identifying identification information matching the third identification information from among the received multiple pieces of identification information. For example, when the third event is executed, the third electronic device403may perform an event related to a writing action using a virtual pointer displayed through a display.

According to an embodiment, in operation869, the first electronic device401may transmit, to the server400, information indicating that the execution of the first event has ended.

According to an embodiment, in operation871, the server400may identify that the execution of the first event for the first virtual object by the first electronic device401has ended.

According to an embodiment, in operation873, the server400may delete, from a memory420(e.g., the memory420inFIG.4A), the first event information and the first identification information of the first electronic device401that has ended the execution of the event for the first virtual object.

According to an embodiment, in operation875, the server400may transmit, to the first electronic device401, the second electronic device402, and the third electronic device403, the third identification information and third event information regarding the first virtual object of the third electronic device403for which the event is currently running.

According to an embodiment, in operation877, the first electronic device401may display the first virtual object in an active state through the display414, based on identifying that the number of pieces of identification information received from the server400is less than the predetermined number (e.g., two). According to an embodiment, the first electronic device401may receive, from the server400, information indicating that the number of pieces of identification information received by the server400is less than the predetermined number (e.g., two). According to an embodiment, the first electronic device401may directly determine, based on the identification information received from the server400, whether the number of received pieces of identification information is less than the predetermined number (e.g., two).

According to an embodiment, the first electronic device401may display the first virtual object in an inactive state through the display414, based on identifying that the number of pieces of identification information received from the server400is greater than the predetermined number.

According to an embodiment, the above-described operations performed by the second electronic device402and the third electronic device403may also be performed by the first electronic device401. However, a description of the first electronic device401performing the above-described operations of the second electronic device402and the third electronic device403is redundant and will be omitted.

The first electronic device401, the second electronic device402, and the third electronic device403may perform the above-described operations of the server400. However, a description of the first electronic device401, the second electronic device402, and the third electronic device403performing the above-described operations of the server400is redundant and will be omitted.

FIG.9Ais a view in which a first electronic device, a second electronic device, and a third electronic device display a first virtual object in an active state according to an embodiment of the disclosure.

Referring toFIG.9A, a server400(e.g., the server400inFIG.4A) may control multiple electronic devices, the number of which is predetermined, such that the multiple electronic devices may simultaneously execute events for the first virtual object. For example, the predetermined number may be two.

According to an embodiment, a first electronic device401(e.g., the first electronic device401inFIG.4B), a second electronic device402(e.g., the second electronic device402inFIG.4B), and a third electronic device403(e.g., the third electronic device403inFIG.4B) may be connected to a server400(e.g., the server400inFIG.4B).

According to an embodiment, the server400may identify that there is no input to the first virtual object, based on not receiving multiple pieces of event information and multiple pieces of identification information of the first electronic device401, the second electronic device402, and the third electronic device403.

According to an embodiment, the server400may identify that the number of multiple pieces of identification information received from multiple electronic devices connected to the server400(e.g., zero) is less than or equal to a predetermined number (e.g., two). Each of the first electronic device401, the second electronic device402, and the third electronic device403may receive, from the server400, information indicating that the number of multiple pieces of identification information received by the server400(e.g., zero) is less than or equal to the predetermined number (e.g., two). Based on the above information, the first electronic device401, the second electronic device402, and the third electronic device403may display first virtual objects910,920, and930in an active state through displays, respectively.

FIG.9Billustrates a state in which a first electronic device executes a first event for a first virtual object according to an embodiment of the disclosure.

Referring toFIG.9B, according to an embodiment, a first electronic device401(e.g., the first electronic device401inFIG.4B) may identify an input to a first virtual object. According to an embodiment, the first electronic device401may identify whether the first virtual object is in an active state. According to an embodiment, the first electronic device401may transmit, based on a first virtual object911being in an active state, first identification information and first event information regarding the first virtual object (e.g., information about touch input of the first electronic device401to the first virtual object) to a server400. According to an embodiment, the server400may identify whether the number of pieces of identification information received from at least one electronic device connected to the server400(e.g., one) is less than or equal to a predetermined number (e.g., two). When it is identified that the number of received pieces of identification information is less than or equal to the predetermined number (e.g., two), the server400may transmit the first event information and the first identification information of the first electronic device401to the first electronic device401, a second electronic device402(e.g., the second electronic device402inFIG.4B), and a third electronic device403(e.g., the third electronic device403inFIG.4B). For example, the server400may transmit the first identification information and the first event information to the first electronic device401, the second electronic device402, and the third electronic device403, based on the number of received pieces of identification information being one and the predetermined number being two.

According to an embodiment, the second electronic device402and the third electronic device403may display first virtual objects921and931in an active state, based on receiving, from the server400, information indicating that the number of electronic devices currently executing events for the first virtual objects is less than the predetermined number.

According to an embodiment, the first electronic device401may execute a first event based on a match between the first identification information and the identification information received from the server400. For example, when the first event is executed, the first electronic device401may perform an event related to a writing action using a virtual pointer901displayed on the display414. According to an embodiment, when the first electronic device401executes the event related to the writing action using the virtual pointer901, the first electronic device401may display a writing content951on the first virtual object911through the display414. According to an embodiment, the server400may transmit the writing content951to the second electronic device402and the third electronic device403connected to the server400. According to an embodiment, the second electronic device402and the third electronic device403may display, on virtual objects921and931, the writing content951based on the execution of the first event by the first electronic device401.

FIG.9Cillustrates a state in which a first electronic device and a second electronic device execute a first event for a first virtual object according to an embodiment of the disclosure.

Referring toFIG.9C, after the operations illustrated inFIG.9B, the second electronic device402(e.g., the second electronic device402inFIG.4B) may identify an input to a first virtual object922.

According to an embodiment, after the first electronic device401(e.g., the first electronic device401inFIG.4B) executes a first event for a first virtual object912, the second electronic device402may identify the input to the first virtual object922. According to an embodiment, the second electronic device402may transmit, based on the input to the first virtual object922, second identification information and second event information regarding the first virtual object922(e.g., information about touch input of the second electronic device402to the first virtual object922) to the server400. According to an embodiment, the server400may identify whether the number of pieces of identification information received by the server400, including identification information of an electronic device in which an event is currently running (e.g., two) is less than or equal to the predetermined number (e.g., two). When it is identified that the number of received pieces of identification information (e.g., two) is less than or equal to the predetermined number (e.g., two), the server400may transmit the second event information and the identification information of the second electronic device402and the first event information and the identification information of the first electronic device401currently executing the first event to the first electronic device401, the second electronic device402, and the third electronic device403.

According to an embodiment, the third electronic device403may receive information indicating that the number of multiple pieces of identification information corresponds to the predetermined number (e.g., two). According to an embodiment, the third electronic device403may display a first virtual object932in an inactive state, based on the number of multiple identification information not being less than the predetermined number (e.g., two).

According to an embodiment, the second electronic device402may execute a second event based on a match between the second identification information and the identification information received from the server400. For example, when the second event is executed, the second electronic device402may perform an event related to a writing action using a virtual pointer902displayed on a display. According to an embodiment, when the event related to the writing action using the virtual pointer902is executed, the second electronic device402may display a writing content961on the first virtual object922through the display. According to an embodiment, based on executing the event related to the writing action using the virtual pointer902by the second electronic device402, the first electronic device401may display the writing content961by the second electronic device402on the first virtual object912.

According to an embodiment, the third electronic device403may display the writing content961while displaying the first virtual object932in an inactive state.

FIG.9Dillustrates an operation in which, after execution of events by a first electronic device and a second electronic device, execution of a first event for a first virtual object by the first electronic device is ended and a second event for the first virtual object by a third electronic device is executed according to an embodiment of the disclosure.

Referring toFIG.9D, after events are executed by the first electronic device401(e.g., the first electronic device401inFIG.4B) and the second electronic device402(e.g., the second electronic device402inFIG.4B), the first electronic device401may end execution of a first event for a first virtual object923. For example, based on the termination of execution of an event for a first virtual object913, the first electronic device401may not display a virtual pointer to be displayed on the first virtual object.

According to an embodiment, the server400(e.g., the server400inFIG.4B) may transmit, based on ending execution of the event by the first electronic device401, second event information (e.g., event information corresponding to an input of the second electronic device402) and second identification information of the second electronic device402, which is currently executing an event for the first virtual object, to the first electronic device401, the second electronic device402, and the third electronic device403(e.g., the third electronic device403inFIG.4B).

According to an embodiment, the third electronic device403may receive, from the server400, information indicating that the number of pieces of identification information received by the server400(e.g., one) is less than a predetermined number (e.g., two), along with the second identification information and the second event information. According to an embodiment, the third electronic device403may display, based on the above information, a first virtual object933in an active state.

According to an embodiment, the third electronic device403may identify an input to the first virtual object. The third electronic device403may transmit, to the server400, third identification information and third event information regarding the first virtual object (e.g., information about an input of the third electronic device403to the first virtual object). According to an embodiment, the server400may identify whether the number of pieces of identification information received from at least one electronic device, including the second identification information of the second electronic device402in which an event is currently running (e.g., two) is less than or equal to the predetermined number (e.g., two). When it is identified that the number of received pieces of identification information (e.g., two) is less than or equal to the predetermined number (e.g., two), the server400may transmit the third event information and the third identification information of the third electronic device403and the second identification information and the second event information to the first electronic device401, the second electronic device402, and the third electronic device403. According to an embodiment, the third electronic device403may execute a third event based on identifying identification information, which matches the identification information of the third electronic device403, from among the multiple pieces of identification information received from the server400.

For example, when the third event is executed, the third electronic device403may perform an event related to a writing action using a virtual pointer903displayed on the display. According to an embodiment, when the event associated with the writing action is executed using the virtual pointer903, the third electronic device403may display a writing content971on the first virtual object933through the display. According to an embodiment, the server400may transmit the writing content971to the first electronic device401and the second electronic device402connected to the server400. According to an embodiment, the first electronic device401and the second electronic device402may display, on the virtual objects913and923, the writing content971based on the execution of the third event on the third electronic device403.

According to an embodiment, the first electronic device401may receive, from the server400, information indicating that the number of pieces of identification information received by the server400(e.g., two) is not less than the predetermined number (e.g., two). According to an embodiment, the first electronic device401may display, based on the above information, the first virtual object913in an inactive state. According to an embodiment, the first electronic device401may display the writing contents961and971on the first virtual object913through the display414while displaying the first virtual object913in the inactive state.

According to an embodiment of the disclosure, a first electronic device may include memory, a communication module, a display, and a processor.

According to an embodiment, wherein the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, perform a connection to a server through the communication module.

According to an embodiment, wherein the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, display a first virtual object through the display, based on information about the first virtual object received from the server.

According to an embodiment, wherein the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, identify a first input to the first virtual object.

According to an embodiment, wherein the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, based on the first input to the first virtual object, identify whether the first virtual object is in an active state.

According to an embodiment, wherein the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, transmit information about a first event corresponding to the first input to the first virtual object and first identification information of the first electronic device to the server when the first virtual object is in the active state.

According to an embodiment, wherein the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, receive, from the server, second identification information of one among a plurality of electronic devices connected to the server and information about a second event for the first virtual object.

According to an embodiment, wherein the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, execute the first event when the second identification information corresponds to the first identification information.

According to an embodiment, wherein the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, when the second identification information corresponds to the first identification information, identify that the first electronic device among the plurality of electronic devices connected to the server has first requested the first input to the first virtual object.

According to an embodiment, wherein the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, when the second identification information does not correspond to the first identification information, identify that a second electronic device among the plurality of electronic devices connected to the server has requested a second input to the first virtual object before the first electronic device.

According to an embodiment, wherein the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, identify, based on the information about the second event, whether the second input to the first virtual object by the second electronic device ends.

According to an embodiment, wherein the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, display the first virtual object in an inactive state through the display when identifying that the second input to the first virtual object by the second electronic device does not end.

According to an embodiment, wherein the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, display the first virtual object in an active state through the display when identifying that the second input to the first virtual object by the second electronic device ends.

According to an embodiment, wherein the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, when identifying that the second electronic device among the plurality of electronic devices connected to the server has requested a second input to the first virtual object before the first electronic device, display the first virtual object in a waiting state to indicate waiting for execution of the first event.

According to an embodiment, wherein the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, in the waiting state, display the second identification information of the second electronic device around the first virtual object.

According to an embodiment, wherein the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, receive, from the server, first multiple pieces of identification information of the plurality of electronic devices when an event related to the first virtual object is simultaneously executed by the plurality of electronic devices.

According to an embodiment, wherein the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, identify identification information corresponding to the first identification information from among the first plurality of pieces of identification information.

According to an embodiment, wherein the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, execute the first event when the identification information matching the first identification information is identified from among the first plurality of pieces of identification information.

According to an embodiment, when the identification information corresponding to the first identification information is not identified from among the first plurality of pieces of identification information, wherein the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, receive second plurality of pieces of identification information of the plurality of electronic devices, excluding identification information of at least one electronic device, in which a third input to the first virtual object has ended, among the plurality of electronic devices.

According to an embodiment, wherein the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, display the first virtual object in an active state through the display when the number of the second plurality of pieces of identification information is less than a predetermined number.

According to an embodiment, wherein the memory stores at least one instruction configured to cause, when executed by the processor, the first electronic device to, display the first virtual object in an inactive state through the display when the number of the second plurality of pieces of identification information is greater than the predetermined number.

According to an embodiment of the disclosure, a method for operating a first electronic device may include performing a connection to a server through a communication module included in the first electronic device.

According to an embodiment, the method for operating the first electronic device may include displaying a first virtual object through a display included in the first electronic device, based on information about the first virtual object received from the server.

According to an embodiment, the method for operating the first electronic device may include identifying a first input to the first virtual object.

According to an embodiment, the method for operating the first electronic device may include identifying, based on the first input to the first virtual object, whether the first virtual object is in an active state.

According to an embodiment, the method for operating the first electronic device may include transmitting a first event corresponding to the first input to the first virtual object and first identification information of the first electronic device to the server when the first virtual object is in the active state.

According to an embodiment, the method for operating the first electronic device may include receiving, from the server, second identification information of one among a plurality of electronic devices connected to the server and information about a second event for the first virtual object.

According to an embodiment, the method for operating the first electronic device may include executing the first event when the second identification information corresponds to the first identification information.

According to an embodiment, the method for operating the first electronic device may include identifying that the first electronic device among the plurality of electronic devices connected to the server has first requested the first input to the first virtual object, when the second identification information corresponds to the first identification information.

According to an embodiment, the method for operating the first electronic device may include identifying that a second electronic device among the plurality of electronic devices connected to the server has requested a second input to the first virtual object before the first electronic device, when the second identification information does not correspond to the first identification information.

According to an embodiment, the method for operating the first electronic device may include identifying, based on the information about the second event, whether the second input to the first virtual object by the second electronic device has ended.

According to an embodiment, the method for operating the first electronic device may include displaying the first virtual object in an inactive state through the display when the second input to the first virtual object by the second electronic device is identified not to have ended.

According to an embodiment, the method for operating the first electronic device may include displaying the first virtual object in an active state through the display when the second input to the first virtual object by the second electronic device is identified to have ended.

According to an embodiment, the method for operating the first electronic device may include displaying the first virtual object in a waiting state to indicate waiting for execution of the first event, when the second electronic device among the plurality of electronic devices connected to the server is identified to have requested a second input to the first virtual object before the first electronic device.

According to an embodiment, the method for operating the first electronic device may include displaying, in the waiting state, the second identification information of the second electronic device around the first virtual object.

According to an embodiment, the method for operating the first electronic device may include receiving, from the server, first plurality of pieces of identification information of the plurality of electronic devices when an event related to the first virtual object is simultaneously executed by the plurality of electronic devices.

According to an embodiment, the method for operating the first electronic device may include identifying identification information corresponding to the first identification information from among the first plurality of pieces of identification information.

According to an embodiment, the method for operating the first electronic device may include executing the first event when the identification information matching the first identification information is identified from among the first plurality of pieces of identification information.

According to an embodiment, the method for operating the first electronic device may include when the identification information corresponding to the first identification information is not identified from among the first plurality of pieces of identification information, receiving second plurality of pieces of identification information of the plurality of electronic devices, excluding identification information of at least one electronic device, in which a third input to the first virtual object ends, among the plurality of electronic devices.

According to an embodiment, the method for operating the first electronic device may include displaying the first virtual object in an active state through the display when a number of the second plurality of pieces of identification information is less than a predetermined number.

According to an embodiment, the method for operating the first electronic device may include displaying the first virtual object in an inactive state through the display when the number of the second plurality of pieces of identification information is greater than the predetermined number.

According to an embodiment of the disclosure, a non-transitory computer-readable recording medium may store at least one instruction to perform a connection to a server through a communication module included in a first electronic device.

According to an embodiment, the non-transitory computer-readable recording medium may store at least one instruction to display a first virtual object through a display included in the first electronic device, based on information about the first virtual object received from the server.

According to an embodiment, the non-transitory computer-readable recording medium may store at least one instruction to identify a first input to the first virtual object.

According to an embodiment, the non-transitory computer-readable recording medium may store at least one instruction to identify, based on the first input to the first virtual object, whether the first virtual object is in an active state.

According to an embodiment, the non-transitory computer-readable recording medium may store at least one instruction to transmit a first event corresponding to the input to the first virtual object and first identification information of the first electronic device to the server when the first virtual object is in the active state.

According to an embodiment, the non-transitory computer-readable recording medium may store at least one instruction to receive, from the server, second identification information of one among a plurality of electronic devices connected to the server and information about a second event for the first virtual object.

According to an embodiment, the non-transitory computer-readable recording medium may store at least one instruction to execute the first event when the second identification information matches the first identification information.

According to an embodiment, the non-transitory computer-readable recording medium may store at least one instruction to, when the second identification information corresponds to the first identification information, identify that the first electronic device among the plurality of electronic devices connected to the server has first requested the first input to the first virtual object.

According to an embodiment, the non-transitory computer-readable recording medium may store at least one instruction to, when the second identification information does not correspond to the first identification information, identify that a second electronic device among the plurality of electronic devices connected to the server has requested a second input to the first virtual object before the first electronic device.

According to an embodiment, the non-transitory computer-readable recording medium may store at least one instruction to, based on the information about the second event, identify whether the second input to the first virtual object by the second electronic device ends.

According to an embodiment, the non-transitory computer-readable recording medium may store at least one instruction to, display the first virtual object in an inactive state through the display when identifying that the second input to the first virtual object by the second electronic device does not end.

According to an embodiment, the non-transitory computer-readable recording medium may store at least one instruction to, display the first virtual object in an active state through the display when identifying that the second input to the first virtual object by the second electronic device ends.

According to an embodiment, the non-transitory computer-readable recording medium may store at least one instruction to, when identifying that the second electronic device among the plurality of electronic devices connected to the server requests a second input to the first virtual object before the first electronic device, display the first virtual object in a waiting state to indicate waiting for execution of the first event.

According to an embodiment, the non-transitory computer-readable recording medium may store at least one instruction to, in the waiting state, display the second identification information of the second electronic device around the first virtual object.

According to an embodiment, the non-transitory computer-readable recording medium may store at least one instruction to, receive, from the server, first plurality of pieces of identification information of the plurality of electronic devices when an event related to the first virtual object is simultaneously executed by the plurality of electronic devices.

According to an embodiment, the non-transitory computer-readable recording medium may store at least one instruction to, identify identification information corresponding to the first identification information from among the first plurality of pieces of identification information.

According to an embodiment, the non-transitory computer-readable recording medium may store at least one instruction to, execute the first event when the identification information corresponding to the first identification information is identified from among the first plurality of pieces of identification information.

According to an embodiment, the non-transitory computer-readable recording medium may store at least one instruction to, when the identification information corresponding to the first identification information is not identified from among the first plurality of pieces of identification information, receive second plurality of pieces of identification information of the plurality of electronic devices, excluding identification information of at least one electronic device, in which a third input to the first virtual object ends, among the plurality of electronic devices.

According to an embodiment, the non-transitory computer-readable recording medium may store at least one instruction to, display the first virtual object in an active state through the display when a number of the second plurality of pieces of identification information is less than a predetermined number.

According to an embodiment, the non-transitory computer-readable recording medium may store at least one instruction to, display the first virtual object in an inactive state through the display when the number of the second plurality of pieces of identification information is greater than the predetermined number.