Patent ID: 12216835

DETAILED DESCRIPTION

The present inventive concept will now be described more fully hereinafter with reference to the accompanying drawings, in which currently preferred variants of the inventive concept are shown. This inventive concept may, however, be implemented in many different forms and should not be construed as limited to the variants set forth herein; rather, these variants are provided for thoroughness and completeness, and fully convey the scope of the present inventive concept to the skilled person.

A computer implemented method for positioning and orienting a second electronic device in a virtual space to be shared with a first electronic device already positioned and oriented in the virtual space, as well as the first and second electronic device and system thereof will now be described with reference toFIG.1toFIG.4.

FIG.1is a flow chart illustrating the steps of the method100for positioning and orienting a second electronic device in a virtual space to be shared with a first electronic device already positioned and oriented in the virtual space. Below, the different steps is described in more detail. Even though illustrated in a specific order, the steps of the method100may be performed in any suitable order, in parallel, as well as multiple times.

A current position and orientation of the first electronic device in the virtual space is obtained S104.

A reference object displayed on a display of the first electronic device is imaged S108by a camera of the second electronic device. The reference object may be imaged S108by capturing image data of the reference object using the camera of the second electronic device. The image data may be a single image of the reference object. Alternatively, the image data may comprise multiple images, such as a video feed of the reference object.

The current position and orientation of the first electronic device in the virtual space may be the position and orientation of the first electronic device at a time of imaging the reference object. In other words, obtaining S104the current position and orientation of the first electronic device may be performed at the same time as imaging S108the reference object. This may be facilitated by synchronizing clocks of the first and second electronic device.

Known physical dimensions and appearance of the reference object as displayed on the display of the first electronic device is obtained S112.

By the term “known”, as in “known physical dimensions and appearance”, it is hereby meant that the physical dimensions and appearance is pre-determined. Put differently, the reference object has physical dimensions and appearance which is known in advance such that it can be compared to the apparent dimensions and appearance of the imaged reference object.

The appearance may for example comprise a shape and/or a pattern of the reference object. The appearance may comprise information pertaining to the orientation of the reference object. As an example, the appearance may comprise information about how the reference object is rotated.

A relative distance and orientation of the second electronic device to the first electronic device is determined S114based on the known physical dimensions and appearance of the reference object as displayed on the display of the first electronic device and apparent dimensions and appearance of the reference object as imaged by the camera of the second electronic device. Put differently, the relative distance and orientation of the second device to the first device may be determined S114using a relationship between the known physical dimensions and appearance of the reference object and the apparent dimensions and appearance of the imaged reference object. In other words, the relative distance and orientation of the second electronic device to the first electronic device can be determined S114by determining a transformation between the displayed reference object and the imaged reference object.

The position and orientation of the second electronic device in the virtual space is determined S116based on the current position and orientation of the first electronic device in the virtual space and the determined relative distance and orientation of the second electronic device to the first electronic device.

All of the above steps may be performed at the second electronic device. The second electronic device may be a portable electronic device. Alternatively, the second electronic device may be a stationary electronic device. The first electronic device may be a portable electronic device, such as a smartphone, a tablet, a laptop, a smart watch, smart glasses, augmented reality (AR) glasses, AR lenses or any other suitable device.

The reference object with the known physical dimensions and appearance may be displayed S106on a display of the first electronic device.

Information pertaining to position and orientation of the first electronic device in the virtual space may be transferred S102, from the first electronic device to the second electronic device. The act of transferring the information pertaining to position and orientation of the first electronic device in the virtual space may be part of the act of obtaining S104the current position and orientation of the first electronic device.

The term “pertaining to” is to be interpreted as that the information can be directly transferred from the first electronic device to the second electronic device, or as that a pointer to where the second electronic device can retrieve the information is transferred from the first electronic device to the second electronic device. The direct transfer and transfer via a pointer where to retrieve the information will be discussed in more detail directly below.

The act of obtaining S112the known physical dimensions and appearance of the reference object may comprise accessing a virtual space layer comprising the known physical dimensions and appearance of the reference object.

The act of obtaining S104the current position and orientation of the first electronic device in the virtual space may comprise accessing a virtual space layer comprising information pertaining to position and orientation of the first electronic device in the virtual space.

A pointer to the virtual space layer may be transferred S110from the first electronic device to the second electronic device. The act of transferring the pointer may be part of the act of obtaining S112the known physical dimensions and appearance of the reference object.

The act of obtaining S112known physical dimensions and appearance of the reference object as displayed on the display of the first electronic device may comprise displaying, on the display of the first electronic device, information pertaining to known physical dimensions and appearance of the reference object. The act of obtaining S112known physical dimensions and appearance of the reference object as displayed on the display of the first electronic device may further comprise acquiring at the second electronic device the information pertaining to physical dimensions and appearance of the reference object, using the camera of the second electronic device. This is an example of direct transfer of information from the first electronic device to the second electronic device.

The information pertaining to known physical dimensions and appearance of the reference object may be encoded as a machine-readable optical label. The act of displaying the information pertaining to known physical dimensions and appearance of the reference object may then comprise displaying the machine-readable optical label. The act of acquiring the information pertaining to physical dimensions and appearance of the reference object may then comprise decoding the machine-readable optical label. This is an example of direct transfer of information from the first electronic device to the second electronic device.

The machine-readable label may be a two-dimensional code, such as a QR code.

The act of obtaining S104current position and orientation of the first electronic device in the virtual space may comprise displaying, on the display of the first electronic device, information pertaining to position and orientation of the first electronic device in the virtual space. The act of obtaining S104current position and orientation of the first electronic device in the virtual space may further comprise acquiring at the second electronic device the information pertaining to position and orientation of the first electronic device in the virtual space, using the camera of the second electronic device. This is an example of direct transfer of information from the first electronic device to the second electronic device.

The information pertaining to current position and orientation of the first electronic device in the virtual space may be encoded as a machine-readable optical label. The act of displaying the information pertaining to current position and orientation of the first electronic device in the virtual space may then comprise displaying the machine-readable optical label. The act of acquiring the information pertaining to current position and orientation of the first electronic device in the virtual space may then comprise decoding the machine-readable optical label.

The machine-readable optical label may constitute the reference object. Put differently, the reference object may be provided within the machine-readable optical label.

FIG.2schematically illustrates a system200for sharing a virtual space between multiple devices. The system200allows an electronic device300to be positioned and oriented in the virtual space to be shared with a portable electronic device400already positioned and oriented in the virtual space. Using the wording of the method discussed above in connection with the method100, the portable electronic device400is to be seen as the first electronic device and the electronic device300is to be seen as the second electronic device. The system200comprises the electronic device300and the portable electronic device400. However, the system200may comprise more than one electronic device300and/or more than one portable electronic device400.

The portable electronic device400, which is further described in connection withFIG.4is positioned and oriented in the virtual space. As an example, the virtual space may be a virtual reality world. Alternatively, the virtual space may be a virtual part of an augmented reality world. The portable electronic device400may be described as a host device in the sense of hosting the virtual space to be shared with other devices. Alternatively, the portable electronic device400may be positioned and oriented in the virtual space hosted by another device. The another device hosting the virtual space may be a physical server. The another device hosting the virtual space may be a cloud implemented server. The another device hosting the virtual space may be another portable electronic device. As non-limiting examples, the portable electronic device400may be a smartphone, a tablet, a laptop, a smart watch, smart glasses, AR glasses, AR lenses or any other suitable device.

The electronic device300may be the second electronic device as described above in relation withFIG.1. The electronic device300is further described in connection withFIG.3. As non-limiting examples, the electronic device300may be a smartphone, a tablet, a laptop, a smart watch, smart glasses, AR glasses, AR lenses, a camera, or any other suitable device.

Even though the electronic device300and the portable electronic device400is herein illustrated as similar looking device, they may be different types of devices.

The electronic device300may be connected to a digital network, such as a mobile network (e.g. 3G, 4G, 5G, or the like) or a WiFi network. The portable electronic device400may be connected to a digital network, such as a mobile network (e.g. 3G, 4G, 5G, or the like) or a WiFi network. The electronic device300and the portable electronic device400may be communicatively connected to each other via wireless connection, such as a Bluetooth connection, via a WiFi Network or a mobile network (e.g. 3G, 4G, 5G, or the like).

The system200may further comprise one or more remote servers206and/or one or more databases208. The one or more remote servers206and/or databases208may for instance be provided in a cloud204as illustrated herein. The electronic device300and/or the portable electronic device400may be communicatively connected to the one or more remote servers206and/or one or more databases208. The one or more remote servers206may perform parts of the functions of the electronic device300and/or the portable electronic device400as described in connection withFIG.3andFIG.4. The one or more databases208may comprise at least a part of information described in connection withFIG.3andFIG.4. For example, the one or more databases208may comprise information pertaining to physical dimensions and appearance of a reference object202displayed on a display402of the portable electronic device400. It is hereby noted that the physical dimensions and appearance of the reference object illustrated herein should be regarded as a non-limiting example.

Upon having positioned and oriented the electronic device300in the virtual space of which the portable electronic device400is already positioned and oriented, the position and orientation of the respective electronic devices may be continuously transferred between the respective electronic devices in order to keep the respective electronic devices oriented and positioned.

FIG.3schematically illustrates the electronic device300as described in connection withFIG.2. The electronic device300comprises a camera302and circuitry304.

The circuitry304may be any type of circuitry304comprising a processing unit. The circuitry304may physically comprise one single circuitry device. Alternatively, the circuitry304may be distributed over several circuitry devices. The circuitry304may comprise a transceiver, a control circuit and a memory.

The transceiver may be configured to enable the circuitry304to communicate with other devices.

The control circuit may be configured to carry out overall control of functions and operations of the circuitry304. The control circuit may include a processor, such as a central processing unit (CPU), microcontroller, or microprocessor. The processor may be configured to execute program code stored in the memory, in order to carry out functions and operations of the circuitry304.

The memory may be one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, a random access memory (RAM), or another suitable device. In a typical arrangement, the memory may include a non-volatile memory for long term data storage and a volatile memory that functions as system memory for the control circuit. The memory may exchange data with the control circuit over a data bus. Accompanying control lines and an address bus between the memory and the control circuit also may be present.

Functions and operations of the circuitry304may be embodied in the form of executable logic routines (e.g., lines of code, software programs, etc.) that are stored on a non-transitory computer readable recording medium (e.g., the memory) of the circuitry304and are executed by the control circuit (e.g. using the processor). Furthermore, the functions and operations of the circuitry304may be a stand-alone software application or form a part of a software application that carries out additional tasks related to the circuitry. The described functions and operations may be considered a method that the corresponding device is configured to carry out. Such as the method discussed above in connection withFIG.1. Also, while the described functions and operations may be implemented in software, such functionality may as well be carried out via dedicated hardware or firmware, or some combination of hardware, firmware and/or software. The following functions may be stored on the non-transitory computer readable recording medium.

The circuitry304is configured to execute an information obtaining function306, configured to obtain current position and orientation of the portable electronic device400positioned and oriented in the virtual space.

The circuitry304is configured to execute an imaging function308, configured to image, by the camera302, a reference object202displayed on a display402of the portable electronic device400.

The information obtaining function308is further configured to obtain known physical dimensions and appearance of the reference object202as displayed on the display402of the portable electronic device400.

The circuitry304is further configured to execute an orientation function310. The orientation function310is configured to determine a relative distance and orientation of the electronic device300to the portable electronic device400based on the known physical dimensions and appearance of the reference object202as displayed on the display402of the portable electronic device400and apparent dimensions and appearance of the imaged reference object210as imaged by the camera302of the electronic device300. The orientation function310is configured to determine position and orientation of the electronic device300in the virtual space based on the current position and orientation of the portable electronic device400in the virtual space and the determined relative distance and orientation of the electronic device300to the portable electronic device400.

The electronic device300may acquire information from the portable electronic device400displayed on the display402of the portable electronic device400by using the camera302. The information may be acquired by capturing image data of the information displayed on the display402of the portable electronic device, or by scanning a machine-readable optical label displayed on the display402.

Alternatively, or in combination, the electronic device300may acquire information from the portable electronic device400via a wireless connection between the electronic device300and the portable electronic device400.

The circuitry304may further be configured to execute a decoding function312configured to decode the machine-readable optical label.

FIG.4schematically illustrates the portable electronic device400as described in connection withFIG.2andFIG.3. The portable electronic device400comprises a display402and circuitry404.

The circuitry404may be any type of circuitry404comprising a processing unit. The circuitry404may physically comprise one single circuitry device. Alternatively, the circuitry404may be distributed over several circuitry devices. The circuitry404may comprise a transceiver, a control circuit and a memory.

The transceiver may be configured to enable the circuitry404to communicate with other devices.

The control circuit may be configured to carry out overall control of functions and operations of the circuitry404. The control circuit may include a processor, such as a central processing unit (CPU), microcontroller, or microprocessor. The processor may be configured to execute program code stored in the memory, in order to carry out functions and operations of the circuitry404.

The memory may be one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, a random access memory (RAM), or another suitable device. In a typical arrangement, the memory may include a non-volatile memory for long term data storage and a volatile memory that functions as system memory for the control circuit. The memory may exchange data with the control circuit over a data bus. Accompanying control lines and an address bus between the memory and the control circuit also may be present.

Functions and operations of the circuitry404may be embodied in the form of executable logic routines (e.g., lines of code, software programs, etc.) that are stored on a non-transitory computer readable recording medium (e.g., the memory) of the circuitry404and are executed by the control circuit (e.g. using the processor). Furthermore, the functions and operations of the circuitry404may be a stand-alone software application or form a part of a software application that carries out additional tasks related to the circuitry. The described functions and operations may be considered a method that the corresponding device is configured to carry out. Such as the method discussed above in connection withFIG.1. Also, while the described functions and operations may be implemented in software, such functionality may as well be carried out via dedicated hardware or firmware, or some combination of hardware, firmware and/or software. The following functions may be stored on the non-transitory computer readable recording medium.

The circuitry404is configured to execute a display function406, configured to display, on the display402, the reference object202having known physical dimensions and appearance.

The circuitry404is further configured to execute a transferring function408, configured to transfer, to another electronic device, such as the electronic device300described in connection withFIGS.2and3, information pertaining to current position and orientation of the portable electronic device400in the virtual space. The transferring function408may transfer the information pertaining to current position and orientation of the portable electronic device400in the virtual space together with a timestamp of the current time.

The transferring function408may be configured to transfer a pointer to a virtual space layer to the electronic device300. The virtual space layer may comprise the known physical dimensions and appearance of the reference object202. The virtual space layer may further comprise information pertaining to position and orientation pf the portable electronic device400. The virtual space layer may further comprise information pertaining to virtual objects shown in the virtual space. The virtual space layer may be provided in the cloud204. Alternatively, the virtual space layer may be provided on the portable electronic device400. The pointer to the virtual space layer may comprise information of where to access the virtual space layer. The electronic device300may access the virtual space layer directly, without using the pointer.

The display function406may be configured to display the information pertaining to known physical dimensions and appearance of the reference object202and/or the information pertaining to position and orientation of the portable electronic device400in the virtual space. The circuitry404may be configured to execute an encoding function410, configured to encode the information pertaining to known physical dimensions and appearance of the reference object202and/or the information pertaining to position and orientation of the portable electronic device400in the virtual space as a machine-readable optical label. The display function406may be configured to display the encoded information as the machine-readable optical label. The machine-readable optical label may for example be a QR code.

The machine-readable optical label may constitute the reference object202. Put differently, the machine-readable optical label may also function as the reference object202. Alternatively, the machine-readable optical label may comprise the reference object202. The machine-readable optical code may further comprise the timestamp of which the current position and orientation of the portable electronic device400are associated with. The machine-readable optical code may comprise the pointer to the virtual space layer.

The portable electronic device400may function as an anchor between the virtual space and the real world since its orientation and position in both worlds are known.

As described above, the pointer to the virtual space layer may be transferred S1110from the first electronic device to the second electronic device. In other words, the second electronic device may obtain the pointer from the first electronic device. The pointer may indicate where to access the virtual space layer which is a place in the cloud where information pertaining to position and orientation of the portable electronic device in the virtual space is stored. The pointer may be obtained by the second electronic device by using the image of the reference object, not by the first electronic device transferring to the second electronic device.

The first electronic device may encode the pointer into the image of the reference object. In other words, information pertaining to the pointer is encoded into the image of the reference object as a machine-readable optical label by the first electronic device. The image would be a machine-readable optical label. For encoding, the first electronic device may further comprise encoding function configured to encode information into an image of an object. The encoding function may be configured to encode pointer into the image of the reference object.

As mentioned above, the first electronic device comprises the imaging function308. The information obtaining function is configured to obtain a pointer to where to access a virtual space layer being a place in the cloud where information pertaining to position and orientation of the portable electronic device400in the virtual space is stored. The virtual space layer may comprise information pertaining to known physical dimensions and appearance of the reference object.

The information obtaining function is also configured to obtain known physical dimensions and appearance of the reference object202as displayed on the display402of the portable electronic device400, and to obtain current position and orientation of the first electronic device300in the virtual space by accessing the virtual space layer comprising information pertaining to position and orientation of the first electronic device in the virtual space.

The second electronic device may obtain the pointer using the image of the reference object which the second electronic device imaged. The second electronic device may obtain the pointer by interpreting the image of the reference object which the second electronic device imaged. The second electronic device may decode the image of the reference object and extract the pointer from the image of the reference object. The pointer can be referred to the pointer information.

Additionally, variations to the disclosed variants can be understood and effected by the skilled person in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims.