Patent Description:
With development of science and technology, stereoscopic display technology has emerged to meet higher requirements of people for a visual effect of image display. The stereoscopic display technology may display a depth image to achieve a three-dimensional visual effect. Compared to a two-dimensional image, the depth image has a more detailed and vivid display effect and a larger data volume.

<CIT> discloses a system and method for stereoscopic video teleconferencing provides an immersive virtual meeting experience. Each participant is surrounded by at least two video cameras. The system senses the position of each of the participants and uses this information to select, for each participant, camera pairs to capture a stereo pair of video images of each of the other participants. The system is adapted to isolate the image of each participant from the background. The system transforms the stereo pair images to adjust the perspective for the interocular spacing of each participant. The processed stereo pairs of video images are transmitted to the respective participants. For each participant, the system assembles a stereo video display image of a virtual meeting room, combining the stereo pair images of each of the other participants appropriately sized and positioned. Three-dimensional (3D) virtual objects can also be displayed and manipulated by participants.

<CIT> discloses an eye-tracker and method can extract a desired area, amend multiple-user-gaze, and reduce the error in a horizontal direction. Therein, a first imaging unit (<NUM>) takes an image of an object in a direction at which a user gazes to obtain the image. A second imaging unit (<NUM>) obtains an image of both eyes of the user gazing at the object. An image processing unit detects the pupil area of both eyes by using the image of both eyes. The image processing unit maps the detected pupil area and the object image to extract the desired area of the object at which the user gazes. The second imaging unit comprises a plurality of cameras taking an image of both eyes of a user. A reflecting unit (<NUM>) takes an image of both eyes of a user in the second imaging unit.

<CIT> relates to creating a window-like experience during video communication scenarios. One example can include a spatial light deflector including a steerable array of light deflecting elements. The example can also include a wedge device positioned relative to the spatial light deflector. The example can also include a camera positioned at an end of the wedge device to receive the captured light.

It is therefore the object of the present invention to provide methods of capturing a depth image including a target object along a photographing direction determined by the relative position relationship between the client device's position and the user-specified display position.

To solve problems existing in the related art, an example of the present disclosure provides a method and an apparatus for controlling image display.

A first aspect of the present disclosure provides a method of controlling image display as defined in clam <NUM>.

A second aspect of the present disclosure provides a method of controlling image display as defined in clam <NUM>.

In particular, the method can further comprise features as described here, in the following detailed description or in the pending claims along with the method of controlling image display as described above.

A third aspect of the present disclosure provides an apparatus for controlling image display as defined in clam <NUM>.

In particular, the apparatus can further comprise features as described here, in the following detailed description or in the pending claims along with any of the methods of controlling image display as described above. A fourth aspect of the present disclosure provides an apparatus for controlling image display as defined in clam <NUM>.

In particular, the apparatus can further comprise features as described here, in the following detailed description or in the pending claims along with any of the methods of controlling image display or the apparatus for controlling image display as described above.

A fifth aspect of the present disclosure provides a computer readable storage medium as defined in clam <NUM>.

A sixth aspect of the present disclosure provides a computer readable storage medium as defined in clam <NUM>.

It is to be understood that the above general descriptions and the below detailed descriptions are merely exemplary and explanatory, and are not intended to limit the present disclosure.

The accompanying drawings herein, which are incorporated in and constitute a part of the present description, illustrate examples consistent with the present disclosure and serve to explain the principles of the present disclosure together with the description.

Exemplary embodiments will be described in detail herein with the examples thereof expressed in the drawings. When the following descriptions involve the drawings, like numerals in different drawings represent like or similar elements unless stated otherwise. Implementations described in the following examples do not represent all implementations consistent with the present disclosure. On the contrary, they are examples of an apparatus and a method consistent with some aspects of the present disclosure described in detail in the appended claims.

<FIG> is a schematic diagram illustrating an application scenario of a method of controlling image display according to an example of the present disclosure. In the application scenario shown in <FIG>, a client device <NUM> is in communication with a server <NUM>, and the server <NUM> is in communication with an image capture device <NUM>. A user may send a control instruction to the server <NUM> through the client device <NUM>, and the server <NUM> may obtain a depth image including an image of a target object <NUM> by controlling the image capture device <NUM> to perform image capture for the target object <NUM> according to the control instruction received from the client device <NUM>. The image capture device <NUM> may upload the captured depth image to the server <NUM>. The server <NUM> may extract the image of the target object (hereinafter referred to as a target object image) from the depth image and send the target object image to the client device <NUM> for display.

<FIG> is a flowchart illustrating a method of controlling image display executed by a server <NUM> according to an example of the present disclosure. <FIG> is a flowchart illustrating a method of controlling image display executed by a client device <NUM> according to an example of the present disclosure. With reference to <FIG>, as shown in <FIG> and <FIG>, the method of controlling image display includes at least the following acts as illustrated in blocks S201 to S204 and S301 to S304.

At block <NUM>, the server <NUM> receives the image capture instruction sent by the client device <NUM>. The image capture instruction includes a photographing direction, and the photographing direction is determined by the client device <NUM> according to a relative position relationship between a position of the client device <NUM> and a user-specified display position.

At block <NUM>, the server <NUM> controls the image capture device <NUM> to perform image capture according to the photographing direction, to obtain the depth image including the target object image.

At block S203, the server <NUM> extracts the target object image from the depth image.

At block <NUM>, the server <NUM> sends the target object image to the client device <NUM> so that the client device <NUM> displays the target object image at the display position.

At block S301, the client device <NUM> receives a control instruction including a user-specified display position.

At block S302, the client device <NUM> obtains a photographing direction that is used for photographing a target object and corresponds to the user-specified display position according to a corresponding relationship between the preset display position and the photographing direction. The photographing direction is determined according to a relative position relationship between the position of the client device <NUM> and the display position.

At block S303, the client device <NUM> sends an image capture instruction to the server <NUM> according to the photographing direction, so that the server <NUM> obtains a depth image including a target object image according to the image capture instruction and extracts the target object image from the depth image.

At block S304, the client device <NUM> receives the target object image sent by the server <NUM> and displays the target object image at the display position.

The order of the above acts is not limited. For example, the order does not need to follow the steps as illustrated. The acts corresponding to blocks S301-S303 may be performed before the act correspond to block S201.

In an example, the client device <NUM> may provide a display scenario for displaying the target object image. For example, the user may set a display position of the target object image in the display scenario through the client device <NUM>. In a case that the display scenario is an image captured for a place where the user is located, the display position is a position that is specified by the user to display the target object image in the display scenario, and corresponds to an actual position in the place where the user is located. For example, if the target object <NUM> is a speaker in a lecture hall, the target object image is a depth image (holographic image) of the speaker <NUM>, and the client device <NUM> is Augmented Reality (AR) glasses, when the user sits on a seat in another lecture hall and faces a lecture stage after wearing the AR glasses <NUM>, the AR glasses <NUM> may obtain an image of the lecture stage and its surrounding scenario as a display scenario of the above target object image. The user may specify the display position as a lecture stage position in the display scenario through the client device <NUM>, and the display position corresponds to the lecture stage position in the lecture hall where the user is located. Here, the user may set the display position of the depth image of the speaker <NUM> in the display scenario as the lecture stage position through the AR glasses <NUM>.

In one or more examples, the client device <NUM> may receive a control instruction of a user-specified display position. a first implementation, the client device <NUM> may provide a plurality of candidate positions in the above display scenario for user selection. When the user selects one of the candidate positions, the client device <NUM> is considered to have received the control instruction of the user-specified display position and determines the user-selected candidate position as the user-specified display position. In a second implementation, the client device <NUM> may provide the above display scenario and provide a rectangular box for encircling the display position on the display scenario, and the user may send the control instruction for specifying the display position to the client device <NUM> through the rectangular box and an "OK" button on the client device <NUM>. For example, after dragging the rectangular box to one position in the display scenario, the user may press the "OK" button on the client device <NUM> to determine the current position encircled by the rectangular box as the display position. In a practical application, the client device <NUM> may receive the control instruction of the user-specified display position in another manner, which is not limited herein.

In one or more examples, after receiving the control instruction of the user-specified display position, the client device <NUM> may obtain a photographing direction for photographing a target object by querying a pre-stored corresponding relationship between the display position and the photographing direction according to the user-specified display position. In the corresponding relationship between the display position and the photographing direction, the photographing direction is determined according to a relative position relationship between the position of the client device and the display position. That is, in the corresponding relationship between the display position and the photographing direction, the display position is in a one-to-one correspondence with the photographing direction, and a photographing direction corresponding to one display position is determined according to the relative position relationship between the display position and the position of the client device. In the above example, the photographing direction may be determined according to the relative position relationship between the position of the client device and an actual position that is in a place where the client device is located and corresponds to the user-specified display position.

Referring to Table <NUM>, the client device <NUM> pre-stores the corresponding relationship of the photographing direction D1 and the position <NUM> when the position <NUM> is used as the display position and the corresponding relationship of the photographing direction D2 and the position <NUM> when the position <NUM> is used as the display position. The photographing direction D1 is determined according to a relative position relationship between the position of the client device <NUM> and the position <NUM>. The photographing direction D2 is determined according to a relative position relationship between the position of the client device <NUM> and the position <NUM>.

In an illustrative scenario, as shown in <FIG>, a user sits on a seat <NUM> in a lecture hall <NUM>, and the position of the seat <NUM> corresponds to the above position of the client device <NUM>. The user may select one position from the candidate positions <NUM> and <NUM> as the display position. The position <NUM> is a lecture stage position, for example, the user may select the position <NUM> as the specified display position. After receiving a control instruction, the client device <NUM> may obtain the photographing direction D2 corresponding to the user-specified display position <NUM> by querying Table <NUM> according to the user-specified display position <NUM>.

In this example, the client device <NUM> may generate an image capture instruction according to the above photographing direction and send the image capture instruction to the server <NUM>, so that the server <NUM> may perform image capture by controlling an image capture device <NUM> according to the photographing direction. Therefore, the server <NUM> may obtain a depth image including a target object <NUM>, thereby extracting the target object image from the depth image.

In this example, after receiving the image capture instruction from the client device <NUM>, the server <NUM> may obtain the depth image including the target object <NUM> by controlling the image capture device <NUM> to perform image capture according to the photographing direction carried in the image capture instruction. In an example, the image capture device <NUM> may include at least one camera; the server <NUM> may pre-store a first corresponding relationship between a device identifier of the at least one camera and the photographing direction. The server <NUM> may obtain the device identifier of the camera corresponding to the photographing direction carried in the image capture instruction according to the first corresponding relationship, and obtain the depth image including the target object <NUM> by controlling the camera to perform image capture for the target object according to the obtained device identifier of the camera.

In an illustrative scenario, as shown in <FIG>, the image capture device <NUM> includes three cameras <NUM>, <NUM> and <NUM>. The cameras <NUM>, <NUM> and <NUM> may be binocular cameras for capturing depth images. The server <NUM> pre-stores corresponding relationships between device identifiers C01, C02 and C03 of the cameras <NUM>, <NUM> and <NUM> and photographing directions, for example, as shown in Table <NUM>. An image of a front side of a target object may be obtained by photographing in a photographing direction D1, an image of a left-front side of the target object may be obtained by photographing in a photographing direction D2, an image of a right side of the target object may be obtained by photographing in a photographing direction D3, an image of a left side of the target object may be obtained by photographing in a photographing direction D4, an image of a right-front side of the target object may be obtained by photographing in a photographing direction D5, and an image including the left, front and right sides of the target object may be obtained by photographing in a photographing direction D6.

For example, referring to <FIG>, if the user sits on the seat <NUM> in the lecture hall <NUM> and the user-specified display position is the position <NUM>, the client device <NUM> may determine the photographing direction D1 according to the relative position relationship between the seat <NUM> and the position <NUM>. The client device <NUM> may send the image capture instruction carrying the photographing direction D1 to the server <NUM>. The server <NUM> may obtain the device identifier C01 corresponding to the photographing direction D1 by querying Table <NUM> according to the photographing direction D1, and obtain one depth image including a target object image by controlling the camera <NUM> corresponding to the device identifier C01 to perform image capture.

In an example, the image capture device <NUM> may upload the captured depth image including the target object image to the server, and the server may extract the target object image from the depth image. In an example, in order to highlight or emphasize the target object image (other parts in the depth image may be blurred or neglected at the same time), the server <NUM> may obtain a corresponding image region in the depth image according to the photographing direction of the depth image and extract the target object image from the image region. Continuing the above example, the server <NUM> may obtain the image region corresponding to the photographing direction D1 in the depth image and extract the target object image from the image region. In this case, the target object image is equivalent to an image of the target object located at the lecture stage position <NUM> formed at eye ground of the user located at the position of the client device (i.e., the seat <NUM>), thereby increasing a spatial sense of the target object image.

In this example, the server <NUM> sends the obtained target object image to the client device <NUM>. After receiving the target object image from the server <NUM>, the client device <NUM> displays the target object image at the user-specified display position. In the above example, the client device <NUM> may obtain a synthetic image by synthesizing the received target object image and the display scenario according to the display position, and then display the synthetic image.

In this example, since the depth image including the target object image is captured from the photographing direction determined by the relative position relationship between the position of the client device and the user-specified display position of the target object, rather than capturing a <NUM>-degree omni-directional depth image for the target object, a data volume of the depth image is reduced. Further, the target object image extracted from the depth image is sent to the client device. In this case, a stereoscopic display effect of the target object is maintained and a data transmission amount of the image may also be reduced, thereby reducing a transmission cost of the image. Meanwhile, the photographing direction is determined according to the relative position relationship between the position of the client device and the user-specified display position of the target object, so that the target object image can reflect the relative position relationship between the display position and the position of the client device, thereby achieving a more vivid display effect, giving the user an immersive feeling, and improving a user experience.

In a practical application, at least two cameras with different photographing distances may exist in a same photographing direction of the target object. The client device may pre-store a corresponding relationship between the display position, the photographing direction and the photographing distance. The client device may obtain the photographing direction and the photographing distance both corresponding to the user-specified display position according to the corresponding relationship between the display position, the photographing direction and the photographing distance, and send an image capture instruction to the server according to the obtained photographing direction and the photographing distance. The server may also pre-store a corresponding relationship between the photographing direction, the photographing distance and a device identifier. The server may obtain the device identifier both corresponding to the photographing direction and the photographing distance carried in the image capture instruction according to the corresponding relationship between the photographing direction, the photographing distance and the device identifier, and control the camera corresponding to the obtained device identifier to perform image capture for the target object.

<FIG> are flowcharts illustrating a method of controlling image display executed by a server <NUM> according to another example of the present disclosure. Based on the example shown in <FIG>, the server <NUM> pre-stores a first corresponding relationship between a device identifier of the at least one camera and a corresponding photographing direction. As shown in <FIG>, the above block <NUM> includes the following blocks S501 and S502, and the above block <NUM> includes the following blocks S601 and <NUM>.

At block S501, the server <NUM> obtains at least two device identifiers corresponding to the photographing direction according to the first corresponding relationship.

At block S502, the server <NUM> controls a camera corresponding to each of the at least two device identifiers to perform image capture for the target object, to obtain at least two depth images each of which includes a target object image.

At block S601, the server <NUM> extracts at least two target object images respectively from each of the at least two depth images.

At this block, since one depth image includes one target object image, the server <NUM> may extract one corresponding target object image from each of the at least two depth images.

At block S602, the server <NUM> obtains a spliced target object image by performing splicing for the at least two target object images.

In this example, the server can obtain at least two device identifiers corresponding to the photographing direction according to the first corresponding relationship, and obtain at least two depth images each including a target object image by controlling a camera corresponding to each of the at least two device identifiers to perform image capture for the target object. For example, as shown in <FIG>, if the user sits on the seat <NUM> and the user-specified display position is the position <NUM>, the client device <NUM> determines the photographing direction D2 according to a relative position relationship between the seat <NUM> and the display position <NUM>, and the photographing direction carried in the image capture instruction sent from the client device <NUM> to the server <NUM> is D2. The server <NUM> can obtain device identifiers C01 and C03 by querying Table <NUM> according to the photographing direction D2, and obtain depth images P01 and P02 including target object images by controlling cameras <NUM> and <NUM> corresponding to the device identifiers C01 and C03 respectively to perform image capture. The depth image P01 is captured by the camera <NUM> corresponding to the device identifier C01, and the depth image P02 is captured by the camera <NUM> corresponding to the device identifier C03.

In this example, after obtaining at least two depth images including target object images, the server extracts the target object images from the at least two depth images, and then obtains the spliced target object image by performing splicing for the extracted at least two target object images. Continuing the above example, the server <NUM> extracts a target object image <NUM> from the depth image P01 and a target object image <NUM> from the depth image P02 respectively. Then, the server <NUM> obtains the spliced target object image by performing splicing for the target object image <NUM> and the target object image <NUM>.

In an example, the server <NUM> may obtain a target object image from a depth image including the target object image in a manner as follows: obtaining an image region corresponding to a photographing direction in the depth image according to the photographing direction adopted when the depth image is taken, and extracting the target object image from the image region of the depth image. Continuing the above example, the server <NUM> obtains the corresponding photographing directions D1 and D4 by querying Table <NUM> according to the device identifiers C01 and C03, that is, the photographing directions D1 and D4 corresponding to the depth image P01 and the depth image P02 respectively are obtained. The server <NUM> obtains an image region A1 corresponding to the photographing direction D1 in the depth image P01 and an image region A2 corresponding to the photographing direction D4 in the depth image P02, and then, extracts the target object image <NUM> from the image region A1 and the target object image <NUM> from the image region A2.

In this example, when at least two cameras are determined to perform image capture for the target object according to the user-specified display position and when at least two depth images including target object images are obtained, the target object images may be extracted from the at least two depth images according to the photographing direction. Then, the spliced target object image may be obtained for display at the display position by performing splicing for the at least two target object images, thereby extending a range of the user-specified display position and increasing practicability.

It is to be noted that, in a case that at least two cameras are known to be used to photograph the target object, the target object images are extracted respectively along photographing directions corresponding to a plurality of cameras by re-using Table <NUM> rather than extracted along one composite photographing direction corresponding to a plurality of cameras. Therefore, the image corresponding to the photographed target object can be extracted more accurately in this manner.

<FIG> is a flowchart illustrating a method of controlling image display executed by a server <NUM> according to still another example of the present disclosure. In this example, based on the example shown in <FIG>, referring to <FIG>, since a control instruction also includes an identifier of a target object, an image capture instruction also includes the identifier of the target object. The server <NUM> also pre-stores a second corresponding relationship between the identifier of the target object and a recognition model of the target object. The above block <NUM> also includes the following blocks S701 to S703.

At block <NUM>, the server <NUM> obtains a recognition model corresponding to the identifier of the target object according to the second corresponding relationship.

At block <NUM>, the server <NUM> recognizes the target object image from the depth image by performing image recognition for the depth image according to the recognition model.

At block S703, the server <NUM> extracts the target object image from the depth image.

In this example, the user may also specify a target object through a client device. For example, the client device may receive a control instruction of a user-specified display position and a user-specified target object, and the control instruction includes the user-specified display position and an identifier of the user-specified target object. The client device may determine a photographing direction according to the user-specified display position, and then, send an image capture instruction carrying the photographing direction and the identifier of the target object to the server.

In this example, the server <NUM> pre-stores a second corresponding relationship between the identifier of the target object and the recognition model. The server may obtain the recognition model corresponding to the identifier of the target object carried in the image capture instruction by querying the second corresponding relationship based on the identifier of the target object carried in the image capture instruction, recognize the target object image from the depth image by performing image recognition for the depth image according to the obtained recognition model, and then extract the target object image from the depth image. Therefore, the image of the user-specified target object may be extracted from the depth image, thereby effectively increasing pertinence of the target object image.

<FIG> is a flowchart illustrating a method of controlling image display executed by a server <NUM> according to yet another example of the present disclosure. In this example, the server <NUM> pre-stores a recognition model for recognizing a target object. Based on the example shown in <FIG>, referring to <FIG>, the above block <NUM> includes the following blocks S801 and <NUM>.

At block S801, the server <NUM> recognizes the target object image from the depth image by performing image recognition for the depth image according to the recognition model of the target object.

At block S802, the server <NUM> extracts the target object image from the depth image.

In this example, the server <NUM> pre-stores a recognition model for recognizing a target object as a default recognition model. After obtaining the depth image including the target object image, the server <NUM> may perform image recognition for the depth image according to the above default recognition model, identify the target object image in the depth image, and extract the identified target object image from the depth image. In this case, the user may only need to specify the display position of the target object image, thereby simplifying user operations and effectively improving convenience.

As shown in <FIG>, an apparatus <NUM> may be provided as a server. Referring to <FIG>, the apparatus <NUM> includes a processing component <NUM> and a memory resource represented by a memory <NUM>.

The processing component <NUM> may include one or more processors. The memory <NUM> is used to store instructions, for example, an application, executable by the processing component <NUM>. The application stored in the memory <NUM> may include one or more modules, and each module corresponds to a set of instructions.

Further, the processing component <NUM> is configured to execute instructions to: receive an image capture instruction sent by a client device, where the image capture instruction includes a photographing direction, and the photographing direction is determined by the client device according to a relative position relationship between a position of the client device and a user-specified display position; control an image capture device to perform image capture according to the photographing direction, to obtain a depth image including a target object image; extract the target object image from the depth image; and send the target object image to the client device so that the client device displays the target object image at the display position.

In an example, the memory <NUM> is configured to: pre-store a first corresponding relationship between a device identifier of at least one camera and a corresponding photographing direction. In this case, the processing component <NUM> obtains a device identifier corresponding to the photographing direction according to the first corresponding relationship stored in the memory <NUM>, and controls a camera corresponding to the device identifier to perform image capture for the target object, to obtain one depth image including a target object image.

In an example, the processing component <NUM> is configured to: obtain a corresponding image region in the depth image according to the photographing direction, and extract the target object image from the image region.

In an example, the memory <NUM> is configured to: pre-store a first corresponding relationship between a device identifier of at least one camera and a corresponding photographing direction. In this case, the processing component <NUM> is configured to: obtain at least two device identifiers corresponding to the photographing direction according to the photographing direction and the first corresponding relationship stored in the memory <NUM>; control a camera corresponding to the at least two device identifiers to perform image capture for the target object, to obtain at least two depth images each including a target object image; extract at least two target object images from the at least two depth images; and obtain a spliced target object image by performing splicing for the at least two target object images.

In an example, the processing component <NUM> is configured to: obtain corresponding image regions in the at least two depth images according to the photographing directions corresponding to the at least two depth images respectively, and extract at least two target object images from the corresponding image regions in the at least two depth images.

In an example, the memory <NUM> is configured to: pre-store a second corresponding relationship between an identifier of the target object and a recognition model of the target object. In this case, the processing component <NUM> is configured to: obtain the recognition model corresponding to the identifier according to the second corresponding relationship stored in the memory <NUM>; and recognize and extract the target object image from the depth image by performing image recognition for the depth image according to the recognition model.

In an example, the memory <NUM> is configured to: pre-store a recognition model for recognizing the target object. In this case, the processing component <NUM> is configured to: recognize and extract the target object image from the depth image by performing image recognition for the depth image according to the recognition model of the target object stored in the memory <NUM>.

The apparatus <NUM> may also include a power supply component <NUM>, a wired/wireless network interface <NUM> and an input/output (I/O) interface <NUM>. The power supply component <NUM> is configured to execute power supply management of the apparatus <NUM>. The network interface <NUM> is configured to connect the apparatus <NUM> to a network. The apparatus <NUM> may operate based on an operating system, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™ or the like, stored in the memory <NUM>.

In some examples of the present disclosure, a non-transitory computer readable storage medium including instructions, such as the memory <NUM> including instructions, is further provided. The instructions may be executed by the processing component <NUM> of the apparatus <NUM> to complete the above method. For example, the non-transitory computer readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk and an optical data storage device, etc..

As shown in <FIG>, an apparatus <NUM> may be AR glasses, a mobile phone, a computer, a tablet device, a medical device, a fitness device, a personal digital assistant and the like.

Referring to <FIG>, the apparatus <NUM> may include one or more of the following components: a processing component <NUM>, a memory <NUM>, a power supply component <NUM>, a multimedia component <NUM>, an audio component <NUM>, an input/output (I/O) interface <NUM>, a sensor component <NUM>, and a communication component <NUM>.

The memory <NUM> is used to store a computer program. The processing component <NUM> is used to execute the computer program stored in the memory <NUM> and configured to: receive a control instruction including a user-specified display position; obtain a photographing direction for photographing a target object according to the display position and a corresponding relationship between the display position and the photographing direction, where the photographing direction is determined according to a relative position relationship between a position of a client device and the display position; send an image capture instruction to a server according to the photographing direction, so that the server obtains a depth image including the target object image according to the image capture instruction and extract the target object image from the depth image; and receive the target object image sent by the server and display the target object image at the display position.

In an example, the control instruction also includes an identifier of a target object, and the image capture instruction also includes the identifier of the target object, so that the server can extract the target object image from the depth image according to the identifier of the target obj ect.

The processing component <NUM> usually controls overall operations of the apparatus <NUM>, such as operations related to display, a telephone call, data communication, a camera operation and a record operation. The processing component <NUM> may include one or more processors <NUM> for executing instructions to complete all or a part of blocks of the above method. Further, the processing component <NUM> may include one or more modules to facilitate interaction between the processing component <NUM> and another component. For example, the processing component <NUM> may include a multimedia module to facilitate the interaction between the multimedia component <NUM> and the processing component <NUM>.

The memory <NUM> is configured to store different types of data to support operations at the apparatus <NUM>. Examples of such data include instructions, contact data, phonebook data, messages, pictures, videos, and so on for any application or method that operates on the apparatus <NUM>. The memory <NUM> may be implemented by any type of volatile or non-volatile memory devices or a combination thereof, and the memory <NUM> may be a Static Random Access Memory (SRAM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), an Erasable Programmable Read-Only Memory (EPROM), a Programmable Read-Only Memory (PROM), a Read-Only Memory (ROM), a magnetic memory, a flash memory, a magnetic disk or an optical disk.

The multimedia component <NUM> includes a screen for providing an output interface between the apparatus <NUM> and a user. In some examples, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen for receiving an input signal from a user. The touch panel may include one or more touch sensors for sensing a touch, a slide and a gesture on the touch panel. The touch sensor may not only sense a boundary of a touching or sliding movement, but also detect duration and pressure related to the touching or sliding operation. In some examples, the multimedia component <NUM> may include a front camera and/or a rear camera. When the apparatus <NUM> is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each of the front camera and the rear camera may be a fixed optical lens system or be of a focal length and a capability of an optical zoom.

The audio component <NUM> is configured to output and/or input an audio signal. For example, the audio component <NUM> includes a microphone (MIC). When the apparatus <NUM> is in an operation mode, such as a call mode, a record mode and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal may be further stored in the memory <NUM> or sent via the communication component <NUM>. In some examples, the audio component <NUM> also includes a speaker for outputting an audio signal.

The I/O interface <NUM> provides an interface between the processing component <NUM> and a peripheral interface module. The above peripheral interface module may be a keyboard, a click wheel, a button, or the like. These buttons may include but not limited to, a home button, a volume button, a start button and a lock button.

The sensor component <NUM> includes one or more sensors for providing state assessments in different aspects for the apparatus <NUM>. For example, the sensor component <NUM> may detect an on/off state of the apparatus <NUM> and a relative location of components. For example, the components are a display and a keypad of the apparatus <NUM>. The sensor component <NUM> may also detect a position change of the apparatus <NUM> or a component of the apparatus <NUM>, presence or absence of a touch of a user on the apparatus <NUM>, an orientation or acceleration/deceleration of the apparatus <NUM>, and a temperature change of apparatus <NUM>. The sensor component <NUM> may include a proximity sensor for detecting the existence of a nearby object without any physical touch. The sensor component <NUM> may also include an optical sensor, such as a CMOS or CCD image sensor used in an imaging application. In some examples, the sensor component <NUM> may also include an acceleration sensor, a gyro sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component <NUM> is configured to facilitate wired or wireless communication between the apparatus <NUM> and other devices. The apparatus <NUM> may access a wireless network based on a communication standard, such as WiFi, <NUM> or <NUM>, or a combination thereof. In an example, the communication component <NUM> receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an example, the communication component <NUM> may also include a Near Field Communication (NFC) module for promoting short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

In an example, the apparatus <NUM> may be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic elements, for executing the method in any one of the above examples.

In an example, a non-transitory computer readable storage medium including instructions, such as the memory <NUM> including instructions, is also provided. The above instructions may be executed by the processor <NUM> of the apparatus <NUM> to complete the above method. For example, the non-transitory computer readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk and an optical data storage device, etc..

Optionally, the image capture device includes at least one camera, and the method further includes: pre-storing a first corresponding relationship between a device identifier corresponding to each of the at least one camera and a photographing direction. In this case, controlling the image capture device to perform image capture according to the photographing direction to obtain the depth image comprising the target object image includes: obtaining a device identifier corresponding to the photographing direction according to the first corresponding relationship; and controlling a camera corresponding to the device identifier to perform image capture for the target object, to obtain one depth image including a target object image.

Optionally, extracting the target object image from the depth image includes: obtaining a corresponding image region in the depth image according to the photographing direction; and extracting the target object image from the image region.

Optionally, the image capture device includes at least one camera, and the method further includes: pre-storing a first corresponding relationship between a device identifier corresponding to each of the at least one camera and a photographing direction. In this case, controlling the camera corresponding to each of the at least two device identifier to perform image capture for the target object, to obtain one depth image including a target object image includes: obtaining at least two device identifiers corresponding to the photographing direction according to the photographing direction and the first corresponding relationship; and controlling a camera corresponding to each of the at least two device identifiers to perform image capture for a target object, to obtain at least two depth images each of which includes target object images; and, extracting the target object images from the depth images includes: extracting at least two target object images respectively from each of the at least two depth images; and obtaining a spliced target object image by performing splicing for the at least two target object images.

Optionally, extracting the at least two target object images respectively from each of the at least two depth images includes: obtaining corresponding image regions in the at least two depth images according to the photographing direction corresponding to the at least two depth images respectively; and extracting the at least two target object images from the corresponding image regions in the at least two depth images.

In an example, the image capture instruction includes an identifier of a target object, and the method further includes: pre-storing a second corresponding relationship between the identifier of the target object and a recognition model of the target object. In this case, extracting the target object image from the depth image includes: obtaining the recognition model corresponding to the identifier according to the second corresponding relationship; and recognizing and extracting the target object image from the depth image by performing image recognition for the depth image according to the recognition model.

Optionally, the method further includes: pre-storing a recognition model for recognizing a target object. In this case, extracting the target object image from the depth image includes: recognizing and extracting the target object image from the depth image by performing image recognition for the depth image according to the recognition model of the target object.

Optionally, the control instruction further includes an identifier of the target object, and the image capture instruction further includes the identifier of the target object so that the server can extract the target object image from the depth image according to the identifier of the target object.

After considering the specification and practicing the present disclosure, the persons of skill in the prior art may easily conceive of other implementations of the present disclosure. The present disclosure is intended to include any variations, uses and adaptive changes of the present disclosure. These variations, uses and adaptive changes follow the general principle of the present disclosure and include common knowledge or conventional technical means in the prior art not disclosed in the present disclosure. The specification and examples herein are intended to be illustrative only and the real scope of the present disclosure are indicated by the following claims of the present disclosure.

Claim 1:
A method of controlling image display, the method being applied to a server, comprising:
receiving (S201) an image capture instruction from a client device,
wherein the image capture instruction comprises a photographing direction, and the photographing direction is determined by the client device according to a relative position relationship between a position of the client device and a user-specified display position, and
wherein the user-specified display position is a position, specified by a user through the client device, to display a target object image in a captured image of a place where the user is located, said captured image being obtained by the client device;
controlling (S202) an image capture device to perform image capture according to the photographing direction, to obtain a depth image comprising the target object image;
extracting (S203) the target object image from the depth image; and
sending (S204) the target object image to the client device such that the client device displays the target object image at the user-specified display position.