Communication methods and systems, electronic devices, servers, and readable storage media

The present disclosure provides a communication method, and an electronic device. The method includes: obtaining, by an electronic device, a plurality of 2D images and/or a plurality of depth maps for a current scene, the plurality of 2D images and/or the plurality of depth maps being aligned in time; and transmitting, by the electronic device, the plurality of 2D images and/or the plurality of depth maps to the server by means of wireless communication.

CROSS REFERENCE TO RELATED APPLICATION

This application claims priority to Chinese Patent Application No. 201810423518.4, filed May 6, 2018, the entire disclosure of which is incorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to the field of communications technologies, and more particularly, to a communication method, a communication system, an electronic device, a server, and a non-transitory computer-readable storage medium.

BACKGROUND

At present, 2D (2-dimensional or two-dimensional) video is widely used in people's lives and work, for example, 2D video playing, 2D video calling, 2D video conferencing. However, although 2D video may give people enough imagery, it does not contain depth information and cannot achieve stereoscopic rendering because images that make up 2D video are 2D.

SUMMARY

The communication method provided in a first aspect of the embodiments of the present disclosure, includes:

obtaining, by an electronic device, a plurality of 2D images and/or a plurality of depth maps for a current scene, the plurality of 2D images and/or the plurality of depth maps being aligned in time; and

transmitting, by the electronic device, the plurality of 2D images and/or the plurality of depth maps to the server by means of wireless communication.

The communication method provided in a second aspect of the embodiments of the present disclosure, includes:

receiving, from an electronic device, a plurality of 2D images and/or a plurality of depth maps, the plurality of 2D images and/or the plurality of depth maps being aligned in time;

generating a plurality of 3D images based on the plurality of 2D images and/or the plurality of depth maps; and

transmitting the plurality of 3D images to a terminal device.

The electronic device provided in a third aspect of the embodiments of the present disclosure includes: an image acquisition component, configured to obtain a plurality of two-dimensional (2D) images and/or a plurality of depth maps for a current scene, the plurality of 2D images and/or the plurality of depth maps being aligned in time; and a communication module, configured to transmit the plurality of 2D images and/or the plurality of depth maps to the server by means of wireless communication.

DETAILED DESCRIPTION

Referring toFIG. 1,FIG. 2,FIG. 11andFIG. 12, the present disclosure provides a communication method for an electronic device10. The electronic device10communicates with a server30by means of wireless communication. The method includes acts in the following blocks.

011: a plurality of 2D images and/or a plurality of depth maps, for a current scene, are obtained. The plurality of 2D images and/or the plurality of depth maps are aligned in time.

013: the plurality of 2D images and/or the plurality of depth maps are transmitted to the server30. The server30is configured to generate a plurality of 3D images based on the plurality of 2D images and/or the plurality of depth maps, and to transmit the plurality of 3D images to a terminal device20.

Referring toFIG. 2, the present disclosure further provides an electronic device10. The electronic device10communicates with the server30by means of wireless communication. The electronic device10includes an obtaining module111and a transmitting module113. The act in block011may be implemented by the obtaining module111. The act in block013may be implemented by the transmitting module113. That is, the obtaining module111may be configured to obtain the plurality of 2D images and/or the plurality of depth maps for a current scene. The transmitting module113may be configured to transmit the plurality of 2D images and/or the plurality of depth maps to the server30. The server30is configured to generate the plurality of 3D images based on the plurality of 2D images and/or the plurality of depth maps, and to transmit the plurality of 3D images to a terminal device20. In detail, the server30may generate the plurality of 3D images by the existing 3D technologies, such as data representing as (RGB (i.e., 2D image), depth (i.e., the depth image), T (i.e., time)) may be displayed by software such as meshlab, or holographic projection technologies, which is omitted herein.

Referring toFIG. 3, the present disclosure further provides an electronic device10. The electronic device10communicates with the server30by means of wireless communication. The electronic device10includes an image acquisition component121and a communication module123. The act in block011may be implemented by the image acquisition component121. The act in block013may be implemented by the communication module123. That is, the image acquisition component121may be configured to obtain the plurality of 2D images and/or the plurality of depth maps for a current scene. The communication module123may be configured to transmit the plurality of 2D images and/or the plurality of depth maps to the server30. The server30is configured to generate the plurality of 3D images based on the plurality of 2D images and/or the plurality of depth maps, and to transmit the plurality of 3D images to a terminal device20.

The electronic device10may be a mobile phone, a tablet computer, a notebook computer, a smart wearable device (a smart helmet, smart glasses, a smart bracelet, a smart watch, etc.), and the like. The terminal device20may also be a mobile phone, a tablet computer, a notebook computer, a smart wearable device (a smart helmet, smart glasses, a smart bracelet, a smart watch, etc.), and the like, and also be the other display device, such as a virtual-reality head-mounted display device, an augmented-reality display device, a holographic stereoscopic projection device, a television equipped with a 3D liquid crystal displayer. The server30may be a computer or the like having a strong capability of computing and processing and a large storage capacity.

The image acquisition component121may be front or rear.

In detail, the image acquisition component121may include a structured light projection module. The structured light projection module may include a structured light projector and an infrared camera. The structured light projector emits a laser pattern into a target space. The infrared camera captures a laser pattern modulated by the target object. The electronic device10further includes a processor122. The processor122calculates a depth map of the laser pattern by using an image matching algorithm. When the image acquisition component121includes the structured light projection module, the image acquisition component121also includes a visible light camera for acquiring a 2D image of the target space. The 2D image includes color information of each object in the target space. In this manner, after the image acquisition component121collects the plurality of 2D images and the plurality of depth maps having the one-to-one corresponding, the communication module123transmits the plurality of 2D images and the plurality of depth maps to the server30by means of wireless communication. The server30also includes a processor322(illustrated inFIG. 15). The processor322obtains a plurality of 3D images based on the plurality of 2D images and the plurality of depth maps. The server30transmits the plurality of 3D images to the terminal device20by means of wireless communication. The terminal device20performs display. The plurality of 3D images may form a 3D video. In detail, the plurality of 3D images is displayed at a predetermined frame rate in the terminal device20to form a 3D video. The predetermined frame rate is greater than or equal to a frame rate that can be resolved by the human eye (i.e. 24 3D images per second may be displayed). The predetermined frame rate may be 24 frames (images)/second, 30 frames (images)/second, 60 frames (images)/second, 120 frames (images)/second, and the like.

Alternatively, the image acquisition component121may include two cameras. The two cameras may both be visible light cameras, or the two cameras may be an infrared camera and a visible light camera. The two cameras have a fixed relative distance. Thus, a plurality of first 2D images and a plurality of second 2D images may be obtained by the binocular stereoscopic acquisition method. When the two cameras may both be the visible light cameras, the first 2D image and the second 2D image are 2D color images. When the two cameras are the infrared camera and the visible light camera, the first 2D image is a 2D infrared image, and the second 2D image is a 2D color image. In this manner, after the image acquisition component121collects the plurality of first 2D images and the plurality of second 2D images having the one-to-one corresponding, the communication module123transmits the plurality of first 2D images and the plurality of second 2D images to the server30by means of wireless communication. The processor322obtains the plurality of 3D images based on the plurality of received first 2D images and the plurality of received second 2D images having the one-to-one corresponding. The server30transmits the plurality of 3D images to the terminal device20by means of wireless communication. The terminal device20performs display. The plurality of 3D images may form a 3D video. In detail, the plurality of 3D images is displayed at a predetermined frame rate in the terminal device20to form a 3D video. The predetermined frame rate is greater than or equal to a frame rate that can be resolved by the human eye (i.e. 24 3D images per second may be displayed). The predetermined frame rate may be 24 frames (images)/second, 30 frames (images)/second, 60 frames (images)/second, 120 frames (images)/second, and the like.

Alternatively, the image acquisition component121may include a TOF sensor module. The TOF sensor module includes a laser projector and an infrared camera. The laser projector emits uniform light into a target space. The infrared camera receives the reflected light and records the time point of emitting the light and the time point of receiving the light. The processor122, based on a time difference between the time point of emitting the light and the time point of receiving the light, and the speed of light, calculates a depth pixel value corresponding to the object in the target space and combines the plurality of depth pixel values to obtain a depth map. When the image acquisition component121includes the TOF sensor module, the image acquisition component121also includes a visible light camera for capturing a 2D image of the target space. The 2D image includes color information of each object in the target space. In this manner, after the image acquisition component121collects the plurality of 2D images and the plurality of depth maps having the one-to-one corresponding, the communication module123transmits the plurality of 2D images and the plurality of depth maps to the server30by means of wireless communication. The server30also includes a processor322(illustrated inFIG. 15). The processor322obtains a plurality of 3D images based on the plurality of 2D images and the plurality of depth maps. The server30transmits the plurality of 3D images to the terminal device20by means of wireless communication. The terminal device20performs display. The plurality of 3D images may form a 3D video. In detail, the plurality of 3D images is displayed at a predetermined frame rate in the terminal device20to form a 3D video. The predetermined frame rate is greater than or equal to a frame rate that can be resolved by the human eye (i.e. 24 3D images per second may be displayed). The predetermined frame rate may be 24 frames (images)/second, 30 frames (images)/second, 60 frames (images)/second, 120 frames (images)/second, and the like.

It may be understood that current video calls, video conferences, and the like are limited to the 2D video transmission. The plurality of images that make up the 2D video are 2D, which do not contain depth information, and cannot render 3D stereoscopic effects.

The communication method and the electronic device10, provided in the embodiments of the present disclosure, collect the plurality of 2D images and the plurality of depth maps, and transmit the plurality of 2D images and the plurality of depth maps to the server30, such that the server30may convert the plurality of 2D images and the plurality of depth maps to the plurality of 3D images. The plurality of 3D images is transmitted by the server30to the terminal device20by means of wired communication and/or by means of wireless communication. In this way, on one hand, the 3D video transmission may be realized, such that a user may view stereoscopic 3D video effect at the terminal device20, and have a better experience; on the other hand, converting the plurality of 2D images and/or the plurality of depth maps to the plurality of 3D images may be realized at the server30. Since the server30has a powerful capability of computing and processing, it may shorten a period of converting the plurality of 2D images and/or the plurality of depth maps to the plurality of 3D images, without calculation at the electronic device10, thereby reducing an amount of data processing, and power consumption of the electronic device10, and reducing a time period required for image processing, and shortening the delay in video communication.

Referring toFIG. 4,FIG. 5,FIG. 11andFIG. 12, in some embodiments, the communication method provided in the embodiments of the present disclosure may be applied to a 3D video call and a 3D video conference, which may realize the interaction among the users. It is assumed that the electronic device10is a calling device, and the terminal device20is a called device. In this case, the act in block011may be follows: a plurality of calling 2D images and/or a plurality of calling depth maps are obtained for the calling scene; the act in block013may be follows: the plurality of calling 2D images and/or the plurality of calling depth maps are transmitted to the server30. The plurality of calling 2D images and/or the plurality of calling depth maps are aligned in time. Further, the method provided in the embodiments of the present disclosure further includes acts in the following blocks.

021: a plurality of called 2D images and/or a plurality of called depth maps, for the called scene, are obtained. The plurality of called 2D images and/or the plurality of called depth maps are aligned in time.

022: the plurality of called 2D images and/or the plurality of called depth maps are transmitted to the server30.

Referring toFIG. 5, in some embodiments, the terminal device20includes an obtaining sub-module211and a transmitting sub-module212. The act in block021may be implemented by the obtaining sub-module211. The act in block022may be implemented by the transmitting sub-module212. That is, the obtaining sub-module211is configured to obtain the plurality of called 2D images and/or the plurality of called depth maps, for the called scene. The transmitting sub-module212is configured to transmit the plurality of called 2D images and/or the plurality of called depth maps to the server30.

Referring toFIG. 6, in some embodiments, the terminal device20includes an image acquisition component221and a communication component223. The act in block021may be implemented by the image acquisition component221. The act in block022may be implemented by the communication component223. That is, the image acquisition component221is configured to obtain the plurality of called 2D images and/or the plurality of called depth maps, for the called scene. The communication component223is configured to transmit the plurality of called 2D images and/or the plurality of called depth maps to the server30.

The image acquisition component221may include a structured light projection module, dual cameras, or a TOF sensor module. The image acquisition component221may be front or rear.

In detail, the electronic device10is referred to as the reference device, such that the electronic device10is the calling device, and the use object of the electronic device10is the calling object; the terminal device20is the called device, and the use object of the terminal device20is the called object. The image acquisition component121of the electronic device10is responsible for collecting the plurality of calling 2D images and/or the plurality of calling depth maps of the target space in which the calling object is located, and the plurality of calling 2D images and/or the plurality of calling depth maps are transmitted to the server30by the communication module123. The processor321of the server30converts the plurality of calling 2D images and/or the plurality of calling depth maps to a plurality of calling 3D images, and the communication unit322of the server30transmits the plurality of calling 3D images to the terminal device20by means of wired communication and/or by means of wireless communication. After receiving by the terminal device20, the terminal device20plays the plurality of calling 3D images at a predetermined frame rate, such that the called object views the 3D video of the calling object. Similarly, at the end of the called device, the image acquisition component221of the terminal device20is responsible for collecting the plurality of called 2D images and/or the plurality of called depth maps of the target space in which the called object is located, and the communication component223transmits the plurality of called 2D images and/or the plurality of called depth maps to the server30. The processor321of the server30converts the plurality of called 2D images and/or the plurality of called depth maps to obtain the plurality of called 3D images, and the communication unit322of the server30transmits the plurality of called 3D images to the electronic device10by means of wireless communication. After receiving by the electronic device10, the electronic device10may display the plurality of called 3D images at a predetermined frame rate, such that the calling object may view the 3D video of the called object.

In this way, both the electronic device10and the terminal device20may transmit the plurality of 2D images and/or the plurality of depth maps, realizing the interaction of the 3D video between the electronic device10and the terminal device20, and the 3D video call, the 3D video conference, and the like. In addition, the calculation of the 3D image is performed by the server30, thereby reducing the data processing pressure on the electronic device10and the terminal device20, speeding up the image processing, and reducing the delay of the 3D video call or the 3D video conference. Therefore, the process of 3D video calling or 3D video conferencing is smoother and timelier, and the user experience is better.

Referring toFIG. 7,FIG. 8,FIG. 11andFIG. 12, in some embodiments, the communication method provided in the embodiments of the present disclosure may also be applied to a virtual-reality head-mounted display device, an augmented-reality display device, a holographic stereoscopic projection device, a display device equipped with a 3D liquid crystal displayer, or the like. At this time, the method provided in the embodiments of the present disclosure further includes acts in the following blocks.

023: the terminal device20displays the plurality of 3D images by means of holographic projection or 3D display.

Referring toFIG. 8, in some embodiments, the terminal device20includes a display sub-module213. The act in block023may be implemented by the display sub-module213. That is, the display sub-module213may be a light machine (an optical engine) using holographic projection or a display screen displaying the plurality of 3D images by means of 3D display.

Referring toFIG. 6, in some embodiments, the terminal device20further includes a processor222. The act in block023may be implemented by the processor222of the terminal device20. That is, the processor222may be configured to control the terminal device20to display the plurality of 3D images by means of holographic projection or 3D display.

In detail, the image acquisition component121of the electronic device10is responsible for collecting the plurality of 2D images and/or the plurality of depth maps for the target space in which the use object of the electronic device10is located. The communication module123of the electronic device10transmits the plurality of 2D images and/or the plurality of depth maps to the server30by means of wireless communication. The processor322of the server30converts the plurality of 2D images and/or the plurality of depth maps to the plurality of 3D images, and the communication unit322of the server30transmits the plurality of 3D images to the terminal device20by means of wired communication and/or by means of wireless communication. The terminal device20displays the plurality of 3D images to present a 3D display effect.

As such, when the electronic device10is not equipped with the display device of the display function, the plurality of 2D images and/or the plurality of depth maps may be transmitted to the server30by means of wireless communication, and the plurality of 3D images is formed by the server30, and then the plurality of 3D images is transmitted by the server30to the device that may display the 3D images, such that the user may view the stereoscopic rendering effect of the 3D video, which is beneficial to improve the user experience.

Referring toFIG. 9,FIG. 10,FIG. 11andFIG. 12, in some embodiments, the communication method provided in the embodiments of the present disclosure further includes acts in the following blocks.

012: the plurality of 2D images and/or the plurality of depth maps are compressed. The compressing may refer to standards published by the JPEG (Joint Photographic Experts Group).

The act in block013, i.e., the plurality of 2D images and/or the plurality of depth maps are transmitted to the server30, may include an act in the following blocks.

0131: a plurality of compressed 2D images and/or a plurality of compressed depth maps are transmitted to the server30.

Referring toFIG. 10, in some embodiments, the electronic device10further includes a compressing module112. The act in block012may be implemented by the compressing module112. The act in block0131may be implemented by the transmitting module113. That is, the compressing module112may be configured to compress the plurality of 2D images and/or the plurality of depth maps. The transmitting module113may be configured to transmit the plurality of compressed 2D images and/or the plurality of compressed depth maps to the server30.

Referring toFIG. 3, in some embodiments, the act in block012may be implemented by the processor122of the electronic device10. The act in block0131may be implemented by the communication module123. That is, the processor122of the electronic device10may be configured to compress the plurality of 2D images and/or the plurality of depth maps. The communication module123may be configured to transmit the plurality of compressed 2D images and/or the plurality of compressed depth maps to the server30.

It should be understood that a size of data packets including the plurality of 2D images and/or the plurality of depth maps is large, and it requires the wireless communication to have a higher transmission bandwidth when these data packets are transmitted (especially for an application scenario with higher requirements on low latency, such as a 3D video call). Due to the development of wireless communication technologies, current various wireless communication methods have not been able to achieve lossless transmission of data packets with the larger size. Therefore, it is necessary to compress the plurality of 2D images and/or the plurality of depth maps, and reduce the amount of the data packets including the plurality of 2D images and/or the plurality of depth maps, thereby realizing a higher-rate transmission on the data packet including the plurality of 2D images and/or the plurality of depth maps, and meeting the requirements of low latency in certain application scenarios.

Further, a compression ratio may be adaptively adjusted according to different application scenarios. In an actual application, the electronic device10determines the application scenario where the electronic device10locates firstly, and then adjusts the corresponding compression ratio according to the determined application scenario. For example, when the plurality of 2D images and/or the plurality of depth maps are transmitted for an application scenario of a video call, the accuracy of each 2D image and the corresponding depth map is not high. Therefore, the compression ratio may be appropriately increased correspondingly, such that the 3D video interaction may be realized on one hand, and the low delay requirement may also be satisfied on the other hand. When the plurality of 2D images and/or the plurality of depth maps are transmitted for holographic projection, the accuracy of each 2D image and the corresponding depth map is high, and accordingly, the compression ratio may be appropriately decreased, thereby meeting the high precision requirements of holographic projection scenes.

Similarly, for the terminal device20, when the terminal device20transmits the plurality of called 2D images and/or the plurality of called depth maps, the plurality of called 2D images and/or the plurality of called depth maps also may be compressed. The plurality of compressed called 2D images and/or the plurality of compressed called depth maps may be transmitted to the server30by means of wired communication and/or by means of wireless communication. The compression ratio may also be adaptively adjusted according to different application scenarios.

Referring toFIG. 11andFIG. 12, in some embodiments, the electronic device10transmits the plurality of 2D images and/or the plurality of depth maps to the server30by means of the sub-6G frequency band of 5G (the fifth-generation cellular mobile communication systems).

At this time, as illustrated inFIG. 11, a transmission route of the plurality of 2D images and/or the plurality of depth maps may be as follows. The electronic device10firstly transmits the plurality of 2D images and/or the plurality of depth maps to a first base station by means of the sub-6G frequency band of 5G. The first base station transmits the plurality of 2D images and/or the plurality of depth maps to a core network by means of wired communication. The core network transmits the plurality of 2D images and/or the plurality of depth maps to a second base station by means of wired communication. The second base station transmits the plurality of 2D images and/or the plurality of depth maps to the server30by means of wireless communication. After the server30receives the plurality of 2D images and/or the plurality of depth maps, the server3converts them to the plurality of 3D images. The wireless communication between the second base station and the server30may be at least one of: WIFI (Wireless Fidelity), 4G (the fourth-generation cellular mobile communication systems) and 5G.

Alternatively, as illustrated inFIG. 12, a transmission route of the plurality of 2D images and/or the plurality of depth maps may be as follows. The electronic device10firstly transmits the plurality of 2D images and/or the plurality of depth maps to a first base station by means of the sub-6G frequency band of 5G. The first base station transmits the plurality of 2D images and/or the plurality of depth maps to a core network by means of wired communication. The core network transmits the plurality of 2D images and/or the plurality of depth maps to the server30by means of wired communication. After the server30receives the plurality of 2D images and/or the plurality of depth maps, the server3convert them to the plurality of 3D images. At this time, the server30may be a server30independent of the core network, or may be a server30integrated in the core network. When the server30is integrated in the core network, the server30may be the rental server30provided by the operator.

The sub-6G frequency band of 5G includes a plurality of working frequency bands. For example, when the working frequency band is n78, the corresponding frequency range is 3.3 GHz to 3.8 GHz; when the working frequency band is n79, the corresponding frequency range is 4.4 GHz˜5.0 GHz; when the working frequency band is n77, the corresponding frequency range is 3.3 GHz˜4.2 GHz; when the working frequency band is n41, the corresponding frequency range is 2.496 GHz˜2.690 GHz; when the working frequency band is n8, the corresponding uplink frequency range is 880 MHz˜915 MHz, and the corresponding downlink frequency range is 915 MHz˜960 MHz; when the working frequency band is n3, the corresponding uplink frequency range is 1710 MHz˜1785 MHz, and the corresponding downlink frequency range is 1805 MHz˜1880 MHz; when the working frequency band is n80, the corresponding frequency range is 1710 MHz˜1785 MHz; and when the working frequency band is n81, the corresponding frequency range is 880 MHz˜915 MHz. The 5G sub-6 frequency band may improve the spectral efficiency of traditional frequency bands, and the data-rate expansion capacity is higher and the coverage is larger in the comparable frequency range. The wireless infrastructure of sub-6 frequency band will be widely deployed with a beamforming solution that will greatly extend network coverage and building penetration. In this way, by employing the 5G sub-6G frequency band to transmit the plurality of 2D images and/or the plurality of depth maps may meet the transmission rate requirements on one hand, and the user environment is less restrictive on the other hand, such that efficient transmission of the plurality of 2D images and/or the plurality of depth maps may be realized in most scenarios.

For example, the electronic device10collects the plurality of 2D images and the plurality of depth maps. The resolution of each 2D image is 1280×720, and the color resolution of each pixel is 12 bits. The resolution of each depth map is 1280×720, and the grayscale resolution of each pixel is 16 bits. The frame rate of the terminal device20for displaying the plurality of 3D images is 60 frames (i.e., the electronic device10transmits one-to-one corresponding 60 2D images and 60 depth map per second). The compression ratio is 102:1. The size of the header file corresponding to the data packet including the 2D image and the depth map corresponding to the 2D image is 108 bits. Therefore, a size of the data stream transmitted by the electronic device10is: [(1280*720*12+1280*720*16+108)*60]/(102/1)=14.48 Mbps. At present, the 5G sub-6G wireless transmission mode has an uplink rate of at least 230 Mbps and a downlink rate of at least 1300 Mbps. It may be seen that the 5G sub-6G wireless transmission mode may fully meet the high-efficiency transmission of 720P high-definition 3D video.

For another example, the electronic device10collects the plurality of 2D images and the plurality of depth maps. The resolution of each 2D image is 1920×1080, and the color resolution of each pixel is 12 bits. The resolution of each depth map is 1920×1080, and the grayscale resolution of each pixel is 16 bits. The frame rate of the terminal device20for displaying the plurality of 3D images is 60 frames (i.e., the electronic device10transmits one-to-one corresponding 60 2D images and 60 depth map per second). The compression ratio is 102:1. The size of the header file corresponding to the data packet including the 2D image and the depth map corresponding to the 2D image is 108 bits. Therefore, a size of the data stream transmitted by the electronic device10is: [(1920*1080*12+1920*1080*16+108)*60]/(102/1)=32.57 Mbps. At present, the 5G sub-6G wireless transmission mode has an uplink rate of at least 230 Mbps and a downlink rate of at least 1300 Mbps. It may be seen that the 5G sub-6G wireless transmission mode may fully meet the high-efficiency transmission of 1080P high-definition 3D video.

Referring toFIG. 11andFIG. 12, in some embodiments, the electronic device10transmits the plurality of 2D images and/or the plurality of depth maps to server30by means of millimeter waves.

At this time, as illustrated inFIG. 11, a transmission route of the plurality of 2D images and/or the plurality of depth maps may be as follows. The electronic device10firstly transmits the plurality of 2D images and/or the plurality of depth maps to a first base station by means of millimeter waves. The first base station transmits the plurality of 2D images and/or the plurality of depth maps to a core network by means of wired communication. The core network transmits the plurality of 2D images and/or the plurality of depth maps to a second base station by means of wired communication. The second base station transmits the plurality of 2D images and/or the plurality of depth maps to the server30by means of wireless communication. After the server30receives the plurality of 2D images and/or the plurality of depth maps, the server3convert them to the plurality of 3D images. The wireless communication between the second base station and the server30may be at least one of: WIFI, 4G and 5G.

Alternatively, as illustrated inFIG. 12, a transmission route of the plurality of 2D images and/or the plurality of depth maps may be as follows. The electronic device10firstly transmits the plurality of 2D images and/or the plurality of depth maps to a first base station by means of millimeter waves. The first base station transmits the plurality of 2D images and/or the plurality of depth maps to a core network by means of wired communication. The core network transmits the plurality of 2D images and/or the plurality of depth maps to the server30by means of wired communication. After the server30receives the plurality of 2D images and/or the plurality of depth maps, the server3convert them to the plurality of 3D images. At this time, the server30may be a server30independent of the core network, or may be a server30integrated in the core network. When the server30is integrated in the core network, the server30may be the rental server30provided by the operator.

The frequency band corresponding to the millimeter wave is 24.25 GHz to 52.6 GHz in 5G and 60 GHz of 802.11ad or 802.11ay in WIFI (Wireless Fidelity). The millimeter waves have a very large transmission bandwidth, which may greatly increase the wireless transmission rate. At present, based on Time Division Duplexing (TDD) standard 5G millimeter wave transmission mode, the peak of the uplink rate may reach 2000 Mbps, and the peak of the downlink rate may reach 2000 Mbps. Both the uplink rate and the downlink rate are measured when the uplink transmission exists only or when the downlink transmission exists only. Since the uplink data and downlink data in the time division duplex is transmitted in time-division intervals, considering the actual use, if the ratio of uplink and downlink is 50%, the uplink rate may also reach 1000 Mbps, and the downlink rate may also reach 1000 Mbps. It may be seen that the 5G millimeter wave wireless communication method may also efficiently transmit the plurality of 2D images, the plurality of depth maps and the 3D video.

Referring toFIG. 11andFIG. 12, in some embodiments, the electronic device10transmits the plurality of 2D images and/or the plurality of depth maps to server30by means of FDD-LTE.

At this time, as illustrated inFIG. 11, a transmission route of the plurality of 2D images and/or the plurality of depth maps may be as follows. The electronic device10firstly transmits the plurality of 2D images and/or the plurality of depth maps to a first base station by means of FDD-LTE. The first base station transmits the plurality of 2D images and/or the plurality of depth maps to a core network by means of wired communication. The core network transmits the plurality of 2D images and/or the plurality of depth maps to a second base station by means of wired communication. The second base station transmits the plurality of 2D images and/or the plurality of depth maps to the server30by means of wireless communication. After the server30receives the plurality of 2D images and/or the plurality of depth maps, the server3converts them to the plurality of 3D images. The wireless communication between the second base station and the server30may be at least one of: WIFI, 4G and 5G.

Alternatively, as illustrated inFIG. 12, a transmission route of the plurality of 2D images and/or the plurality of depth maps may be as follows. The electronic device10firstly transmits the plurality of 2D images and/or the plurality of depth maps to a first base station by means of FDD-LTE. The first base station transmits the plurality of 2D images and/or the plurality of depth maps to a core network by means of wired communication. The core network transmits the plurality of 2D images and/or the plurality of depth maps to the server30by means of wired communication. After the server30receives the plurality of 2D images and/or the plurality of depth maps, the server3converts them to the plurality of 3D images. At this time, the server30may be a server30independent of the core network, or may be a server30integrated in the core network. When the server30is integrated in the core network, the server30may be the rental server30provided by the operator.

FDD-LTE refers to the Frequency Division Duplexing (FDD) system in the fourth-generation mobile communication network (Long Term Evolution, LTE). In this system, the uplink data and the downlink data are simultaneously transmitted in different frequency bands. Therefore, the FDD-LTE wireless transmission mode has a strong data transmission capability. The FDD-LTE wireless transmission mode is more suitable for symmetric services. When supporting symmetric services, it may make full use of the uplink and downlink spectrum. For example, when FDD-LTE is used to transmit the plurality of 2D images, the plurality of depth maps and the plurality of 3D images during the 3D video call, the uplink data and the downlink data are simultaneously transmitted in the uplink channel and the downlink channel because the ratio of the uplink channel and the downlink channel of the FDD-LTE is 1:1. Therefore, it may simultaneously upload and download the 2D images, the depth maps and the 3D images with large data size in 3D video call. At present, based on the wireless communication method of the frequency division duplex system in the fourth-generation mobile communication network, the uplink rate is about 200 Mbps, and the downlink rate is about 1200 Mbps. Referring to the above-mentioned example of 32.57 Mbps of 1080P ultra clear 3D video, and of 14.48 Mbps of 720P HD 3D video, it may be seen that the FDD-LTE may fully meet the high-efficiency transmission of 1080P high-definition 3D video.

Referring toFIG. 11andFIG. 12, in some embodiments, the electronic device10transmits the plurality of 2D images and/or the plurality of depth maps to server30by means of TDD-LTE (Time Division Long Term Evolution).

At this time, as illustrated inFIG. 11, a transmission route of the plurality of 2D images and/or the plurality of depth maps may be as follows. The electronic device10firstly transmits the plurality of 2D images and/or the plurality of depth maps to a first base station by means of TDD-LTE. The first base station transmits the plurality of 2D images and/or the plurality of depth maps to a core network by means of wired communication. The core network transmits the plurality of 2D images and/or the plurality of depth maps to a second base station by means of wired communication. The second base station transmits the plurality of 2D images and/or the plurality of depth maps to the server30by means of wireless communication. After the server30receives the plurality of 2D images and/or the plurality of depth maps, the server3converts them to the plurality of 3D images. The wireless communication between the second base station and the server30may be at least one of: WIFI, 4G and 5G.

Alternatively, as illustrated inFIG. 12, a transmission route of the plurality of 2D images and/or the plurality of depth maps may be as follows. The electronic device10firstly transmits the plurality of 2D images and/or the plurality of depth maps to a first base station by means of TDD-LTE. The first base station transmits the plurality of 2D images and/or the plurality of depth maps to a core network by means of wired communication. The core network transmits the plurality of 2D images and/or the plurality of depth maps to the server30by means of wired communication. After the server30receives the plurality of 2D images and/or the plurality of depth maps, the server3converts them to the plurality of 3D images. At this time, the server30may be a server30independent of the core network, or may be a server30integrated in the core network. When the server30is integrated in the core network, the server30may be the rental server30provided by the operator.

TDD-LTE refers to Time Division Duplexing (FDD) standard in the fourth-generation mobile communication network (Long Term Evolution, LTE). In this system, the uplink data and the downlink data are transmitted on the same frequency band according to time cross allocation. The TDD-LTE wireless transmission mode has high flexibility, and the time slot ratio of uplink transmission and the time slot ratio of downlink transmission may be flexibly adjusted according to actual needs. The TDD-LTE wireless transmission mode is more suitable for asymmetric services. At present, based on the wireless communication method of time division duplex in the fourth-generation mobile communication network, the uplink rate is at least 24 Mbps, and the downlink rate is about 800 Mbps. Referring to the above-mentioned example of 14.48 Mbps of HD 3D video of 720P, the TDD-LTE wireless transmission mode may fully satisfy the efficient transmission of HD 3D video of 720P. Referring to the above-mentioned example of 32.57 Mbps of ultra-clear 3D video of 1080P, the uplink rate of the TDD-LTE wireless transmission mode may not meet the transmission rate requirement. However, due to the high flexibility of TDD-LTE, the time slot ratio of the uplink transmission and the time slot ratio of the downlink transmission may be flexibly adjusted. Therefore, in some scenarios, the TDD-LTE wireless transmission mode may also satisfy the transmission of ultra-clear 3D video of 1080P.

Referring toFIG. 11andFIG. 12, in some embodiments, the electronic device10transmits the plurality of 2D images and/or the plurality of depth maps to server30by means of WIFI.

At this time, as illustrated inFIG. 11, a transmission route of the plurality of 2D images and/or the plurality of depth maps may be as follows. The electronic device10firstly transmits the plurality of 2D images and/or the plurality of depth maps to a first base station by means of WIFI. The first base station transmits the plurality of 2D images and/or the plurality of depth maps to a core network by means of wired communication. The core network transmits the plurality of 2D images and/or the plurality of depth maps to a second base station by means of wired communication. The second base station transmits the plurality of 2D images and/or the plurality of depth maps to the server30by means of wireless communication. After the server30receives the plurality of 2D images and/or the plurality of depth maps, the server3converts them to obtain the plurality of 3D images. The wireless communication between the second base station and the server30may be at least one of: WIFI, 4G and 5G.

Alternatively, as illustrated inFIG. 12, a transmission route of the plurality of 2D images and/or the plurality of depth maps may be as follows. The electronic device10firstly transmits the plurality of 2D images and/or the plurality of depth maps to a first base station by means of WIFI. The first base station transmits the plurality of 2D images and/or the plurality of depth maps to a core network by means of wired communication. The core network transmits the plurality of 2D images and/or the plurality of depth maps to the server30by means of wired communication. After the server30receives the plurality of 2D images and/or the plurality of depth maps, the server3converts them to obtain the plurality of 3D images. At this time, the server30may be a server30independent of the core network, or may be a server30integrated in the core network. When the server30is integrated in the core network, the server30may be the rental server30provided by the operator.

The first base station refers to a wireless access point in WIFI communication. The second base station is also referred to a wireless access point in WIFI communication.

The application frequency bands of WIFI include 2G frequency band and 5G frequency band. The frequency range corresponding to the 2G frequency band is 2.402 GHz to 2.482 GHz, and the frequency range corresponding to the 5G frequency band is 5.150 GHz to 5.350 GHz, 5.470 GHz to 5.725 GHz, and 5.725 GHz to 5.850 GHz. The WIFI wireless communication mode adopts the time division duplex working mode. At present, in the 2G frequency band, the peak of the uplink rate of the WIFI wireless communication mode may reach 300 Mbps, and the peak of the downlink rate may reach 300 Mbps. Both the uplink rate and the downlink rate are measured when the uplink transmission exists only or when the downlink transmission exists only. Since the uplink data and downlink data in the time division duplex is transmitted in time-division intervals, considering the actual use, if the ratio of uplink and downlink is 50%, the uplink rate may also reach 150 Mbps, and the downlink rate may also reach 150 Mbps. In the 5G frequency band, the peak of the uplink rate of the WIFI wireless communication mode may reach 1732 Mbps, and the peak of the downlink rate may reach 1732 Mbps. Both the uplink rate and the downlink rate are measured when the uplink transmission exists only or when the downlink transmission exists only. Since the uplink data and downlink data in the time division duplex is transmitted in time-division intervals, considering the actual use, if the ratio of uplink and downlink is 50%, the uplink rate may also reach 866 Mbps, and the downlink rate may also reach 866 Mbps. It may be seen that the WIFI wireless communication method may also efficiently transmit the plurality of 2D images, the plurality of depth maps and the 3D video.

Referring toFIG. 11toFIG. 14, the present disclosure provides a communication method for the server30. The server30communicates with the electronic device10and the terminal device20by means of wireless communication. The method includes acts in the following block.

031: a plurality of 2D images and/or a plurality of depth maps from the electronic device10are received.

033: a plurality of 3D images is generated based on the plurality of 2D images and/or the plurality of depth maps.

034: the plurality of 3D images is transmitted to the terminal device20.

Referring toFIG. 14, the present disclosure further provides a server30. The server30communicates with the electronic device10and the terminal device20by means of wireless communication. The server30includes a receiving unit311, a processing unit313, and a transmitting unit314. The act in block031may be implemented by the receiving unit311. The act in block033may be implemented by the processing unit313. The act in block034may be implemented by the transmitting unit314. That is, the receiving unit311may be configured to receive a plurality of 2D images and/or a plurality of depth maps from the electronic device10. The processing unit313may be configured to generate a plurality of 3D images based on the plurality of 2D images and/or the plurality of depth maps. The transmitting unit314may be configured to transmit the plurality of 3D images to the terminal device20.

Referring toFIG. 15, the present disclosure further provides a server30. The server30communicates with the electronic device10and the terminal device20by means of wireless communication. The server30includes a communication unit322and a processor321. The act in block031and the act in block034may be implemented by the communication unit322. The act in block033may be implemented by the processor322. That is, the communication unit322may be configured to receive a plurality of 2D images and/or a plurality of depth maps from the electronic device10. The processor322may be configured to generate a plurality of 3D images based on the plurality of 2D images and/or the plurality of depth maps. The communication unit322may be further configured to transmit the plurality of 3D images to the terminal device20.

The plurality of 3D images may form a video. In detail, the plurality of 3D images is displayed at a predetermined frame rate in the terminal device20to form a 3D video. The predetermined frame rate is greater than or equal to a frame rate that can be resolved by the human eye (i.e. 24 3D images per second may be displayed). The predetermined frame rate may be 24 frames (images)/second, 30 frames (images)/second, 60 frames (images)/second, 120 frames (images)/second, and the like.

The electronic device10may be a mobile phone, a tablet computer, a notebook computer, a smart wearable device (a smart helmet, smart glasses, a smart bracelet, a smart watch, etc.), and the like. The terminal device20may also be a mobile phone, a tablet computer, a notebook computer, a smart wearable device (a smart helmet, smart glasses, a smart bracelet, a smart watch, etc.), and the like, and also be the other display device, such as a virtual-reality head-mounted display device, an augmented-reality display device, a holographic stereoscopic projection device, a television equipped with a 3D liquid crystal displayer. The server30may be a computer or the like having a strong capability of computing and processing and a large storage capacity.

It may be understood that current video calls, video conferences, and the like are limited to the 2D video transmission. The plurality of images that make up the 2D video are 2D, which do not contain depth information, and cannot render 3D stereoscopic effects.

The communication method and the server30, provided in the embodiments of the present disclosure, receive the plurality of 2D images and the plurality of depth maps from the electronic device10, converts the plurality of 2D images and the plurality of depth maps to the plurality of 3D images, and transmits the plurality of 3D images to the terminal device20by means of wired communication and/or by means of wireless communication. In this way, on one hand, the 3D video transmission may be realized, such that a user may view stereoscopic 3D video effect at the terminal device20, and have a better experience; on the other hand, converting the plurality of 2D images and/or the plurality of depth maps to the plurality of 3D images may be realized at the server30. Since the server30has a powerful capability of computing and processing, it may shorten a period of converting the plurality of 2D images and/or the plurality of depth maps to the plurality of 3D images, without the calculation of the electronic device10, thereby reducing an amount of data processing, and power consumption of the electronic device10, and reducing a time period required for image processing, and shortening the delay in video communication.

Referring toFIG. 11,FIG. 12andFIG. 16, in some embodiments, the act in block031, i.e., the plurality of 2D images and/or the plurality of depth maps from the electronic device10are received, may include an act in the following block.

0311: a plurality of compressed 2D images and/or a plurality of compressed depth maps from the electronic device10are received.

The communication method provided in the embodiments of the present disclosure may further include an act in the following block.

032: the plurality of compressed 2D images and/or the plurality of compressed depth maps are decompressed.

The act in block033, i.e., the plurality of 3D images is generated based on the plurality of 2D images and/or the plurality of depth maps, may include an act in the following block.

0333: the plurality of 3D images is generated based on a plurality of decompressed 2D images and/or a plurality of decompressed depth maps.

Referring toFIG. 17, in some embodiments, the server30also includes a compressing unit312. The act in block0311may be implemented by the receiving unit311. The act in block032may be implemented by the compressing unit312. The act in block0333may be implemented by the processing unit313. That is, the receiving unit311may be further configured to receive the plurality of compressed 2D images and/or the plurality of compressed depth maps from the electronic device10. The compressing unit312may be configured to decompress the plurality of compressed 2D images and/or the plurality of compressed depth maps. The processing unit313may be configured to generate the plurality of 3D images based on the plurality of decompressed 2D images and/or the plurality of decompressed depth maps.

Referring toFIG. 15, in some embodiments, the act in block0311may be implemented by the communication unit321. The act in block032and the act in block0333may be implemented by the processor322. That is, the communication unit321may be configured to receive the plurality of compressed 2D images and/or the plurality of compressed depth maps from the electronic device10. The processor322is further configured to decompress the plurality of compressed 2D images and/or the plurality of compressed depth maps, and to generate the plurality of 3D images based on the plurality of decompressed 2D images and/or the plurality of decompressed depth maps.

In detail, a size of data packets including the plurality of 2D images and/or the plurality of depth maps is large. Due to the development of wireless communication technologies, the electronic device10needs to compress the plurality of 2D images and/or the plurality of depth maps before performing data transmission. After the server30receives the plurality of compressed 2D images and/or the plurality of compressed depth maps, the server30needs to decompress the compressed data, and convert the plurality of decompressed 2D images and/or the plurality of decompressed depth maps to the plurality of 3D images.

Referring toFIG. 11andFIG. 12, in some embodiments, the server30receives the plurality of 2D images and/or the plurality of depth maps from the electronic device10by means of at least one wireless communication of: WIFI, 4G and 5G. That is, the communication mode between the server30and the electronic device10may include only one of WIFI, 4G or 5G, and may also include WIFI and 4G, WIFI and 5G, 4G and 5G, or also includes WIFI, 4G and 5G. The server30transmits the plurality of 3D images to the terminal device20by means of at least one wireless communication of WIFI, 4G and 5G, and/or by means of wired communication. That is, the communication mode between the server30and the electronic device10may include only one of WIFI, 4G, 5G and wired communication, and may also include any two or three of WIFI, 4G, 5G, and wired communication, and may also include WIFI, 4G, 5G and wired communication. When the server30and the terminal device20communicate by means of wired communication, the terminal device20may be a device that may communicate by means of wired connection, such as a notebook computer, a virtual-reality head-mounted display device, an augmented-reality display device, a holographic stereoscopic projection device.

In detail, the electronic device10may transmits the plurality of 2D images and/or the plurality of depth maps to the server30by means of the sub-6G frequency band of 5G. The server30transmits the plurality of 3D images to the terminal device20by means of wired communication, and/or by means of wireless communication.

At this time, as illustrated inFIG. 11, a transmission route of the plurality of 2D images and/or the plurality of depth maps may be as follows. The electronic device10firstly transmits the plurality of 2D images and/or the plurality of depth maps to a first base station by means of the sub-6G frequency band of 5G. The first base station transmits the plurality of 2D images and/or the plurality of depth maps to a core network by means of wired communication. The core network transmits the plurality of 2D images and/or the plurality of depth maps to a second base station by means of wired communication. The second base station transmits the plurality of 2D images and/or the plurality of depth maps to the server30by means of wireless communication (at least one of WIFI, 4G and 5G). After the server30receives the plurality of 2D images and/or the plurality of depth maps, the server3converts them to the plurality of 3D images. Subsequently, the server30transmits the plurality of 3D images to the terminal device20by means of wired communication and/or by means of wireless communication.

Alternatively, as illustrated inFIG. 12, a transmission route of the plurality of 2D images and/or the plurality of depth maps may be as follows. The electronic device10firstly transmits the plurality of 2D images and/or the plurality of depth maps to a first base station by means of the sub-6G frequency band of 5G. The first base station transmits the plurality of 2D images and/or the plurality of depth maps to a core network by means of wired communication. The core network transmits the plurality of 2D images and/or the plurality of depth maps to the server30by means of wired communication. After the server30receives the plurality of 2D images and/or the plurality of depth maps, the server3convert them to the plurality of 3D images. At this time, the server30may be a server30independent of the core network, or may be a server30integrated in the core network. A transmission route of the plurality of 3D images may be as follows. The server30may transmit the plurality of 3D images to the core network by means of wired communication. The core network may transmit the plurality of 3D images to the second base station by means of wired communication. The second base station may transmit the plurality of 3D images to the terminal device20by means of wireless communication (at least one of WIFI, 4G and 5G).

The 5G sub-6 frequency band may improve the spectral efficiency of traditional frequency bands, and the data-rate expansion capacity is higher and the coverage is larger in the comparable frequency range. The wireless infrastructure of sub-6 frequency band will be widely deployed with a beamforming solution that will greatly extend network coverage and building penetration. In this way, by employing the 5G sub-6G frequency band to transmit the plurality of 2D images and/or the plurality of depth maps may meet the transmission rate requirements on one hand, and the user environment is less restrictive on the other hand, such that efficient transmission of the plurality of 2D images and/or the plurality of depth maps may be realized in most scenarios.

Alternatively, the electronic device10may transmit the plurality of 2D images and/or the plurality of depth maps to the server30by means of millimeter wave, and the server30transmits the plurality of 3D images to the terminal device20by mean of wired communication and/or by means of wireless communication.

At this time, as illustrated inFIG. 11, a transmission route of the plurality of 2D images and/or the plurality of depth maps may be as follows. The electronic device10firstly transmits the plurality of 2D images and/or the plurality of depth maps to a first base station by means of millimeter waves. The first base station transmits the plurality of 2D images and/or the plurality of depth maps to a core network by means of wired communication. The core network transmits the plurality of 2D images and/or the plurality of depth maps to a second base station by mean of wired communication. The second base station transmits the plurality of 2D images and/or the plurality of depth maps to the server30by mean of wireless communication (at least one of WIFI, 4G and 5G). After the server30receives the plurality of 2D images and/or the plurality of depth maps, the server3convert them to the plurality of 3D images. Subsequently, the server30transmits the plurality of 3D images to the terminal device20by means of wired communication and/or by means of wireless communication.

Alternatively, as illustrated inFIG. 12, a transmission route of the plurality of 2D images and/or the plurality of depth maps may be as follows. The electronic device10firstly transmits the plurality of 2D images and/or the plurality of depth maps to a first base station by means of millimeter waves. The first base station transmits the plurality of 2D images and/or the plurality of depth maps to a core network by means of wired communication. The core network transmits the plurality of 2D images and/or the plurality of depth maps to the server30by means of wired communication. After the server30receives the plurality of 2D images and/or the plurality of depth maps, the server3convert them to the plurality of 3D images. At this time, the server30may be a server30independent of the core network, or may be a server30integrated in the core network. A transmission route of the plurality of 3D images may be as follows. The server30may transmit the plurality of 3D images to the core network by means of wired communication. The core network may transmit the plurality of 3D images to the second base station by means of wired communication. The second base station may transmit the plurality of 3D images to the terminal device20by means of wireless communication manner (at least one of WIFI, 4G and 5G).

The millimeter waves have a very large transmission bandwidth, which may greatly increase the wireless transmission rate, and efficiently transmit the plurality of 2D images, the plurality of depth maps and the 3D video.

Alternatively, the electronic device10may transmit the plurality of 2D images and/or the plurality of depth maps to the server30by means of FDD-LTE, and the server30transmits the plurality of 3D images to the terminal device20by means of FDD-LTE.

At this time, as illustrated inFIG. 11, a transmission route of the plurality of 2D images and/or the plurality of depth maps may be as follows. The electronic device10firstly transmits the plurality of 2D images and/or the plurality of depth maps to a first base station by means of FDD-LTE. The first base station transmits the plurality of 2D images and/or the plurality of depth maps to a core network by means of wired communication. The core network transmits the plurality of 2D images and/or the plurality of depth maps to a second base station by means of wired communication. The second base station transmits the plurality of 2D images and/or the plurality of depth maps to the server30by means of wireless communication (at least one of WIFI, 4G and 5G). After the server30receives the plurality of 2D images and/or the plurality of depth maps, the server3converts them to the plurality of 3D images. Subsequently, the server30transmits the plurality of 3D images to the terminal device20by means of wired communication and/or by means of wireless communication.

Alternatively, as illustrated inFIG. 12, a transmission route of the plurality of 2D images and/or the plurality of depth maps may be as follows. The electronic device10firstly transmits the plurality of 2D images and/or the plurality of depth maps to a first base station by means of FDD-LTE. The first base station transmits the plurality of 2D images and/or the plurality of depth maps to a core network by means of wired communication. The core network transmits the plurality of 2D images and/or the plurality of depth maps to the server30by means of wired communication. After the server30receives the plurality of 2D images and/or the plurality of depth maps, the server3converts them to the plurality of 3D images. At this time, the server30may be a server30independent of the core network, or may be a server30integrated in the core network. A transmission route of the plurality of 3D images may be as follows. The server30may transmit the plurality of 3D images to the core network by means of wired communication. The core network may transmit the plurality of 3D images to the second base station by means of wired communication. The second base station may transmit the plurality of 3D images to the terminal device20by means of wireless communication (at least one of WIFI, 4G and 5G).

In FDD-LTE system, the uplink data and the downlink data are simultaneously transmitted in different frequency bands. Therefore, the FDD-LTE wireless transmission mode has a strong data transmission capability. The FDD-LTE wireless transmission mode is more suitable for symmetric services. When supporting symmetric services, it may make full use of the uplink and downlink spectrum. For example, when FDD-LTE is used to transmit the plurality of 2D images, the plurality of depth maps and the plurality of 3D images during the 3D video call, the uplink data and the downlink data are simultaneously transmitted in the uplink channel and the downlink channel because the ratio of the uplink channel and the downlink channel of the FDD-LTE is 1:1. Therefore, it may simultaneously upload and download the 2D images, the depth maps and the 3D images with large data size in 3D video call, and satisfy the efficient transmission of the plurality of 2D images, the plurality of depth maps, and the plurality of 3D images.

Alternatively, the electronic device10may transmit the plurality of 2D images and/or the plurality of depth maps to the server30by means of TDD-LTE (Time Division Long Term Evolution), and the server30transmits the plurality of 3D images to the terminal device20by means of wired communication and/or by means of wireless communication.

At this time, as illustrated inFIG. 11, a transmission route of the plurality of 2D images and/or the plurality of depth maps may be as follows. The electronic device10firstly transmits the plurality of 2D images and/or the plurality of depth maps to a first base station by means of TDD-LTE. The first base station transmits the plurality of 2D images and/or the plurality of depth maps to a core network by means of wired communication. The core network transmits the plurality of 2D images and/or the plurality of depth maps to a second base station by means of wired communication. The second base station transmits the plurality of 2D images and/or the plurality of depth maps to the server30by means of wireless communication (at least one of WIFI, 4G and 5G). After the server30receives the plurality of 2D images and/or the plurality of depth maps, the server3converts them to the plurality of 3D images. Subsequently, the server30transmits the plurality of 3D images to the terminal device20by means of wired communication and/or by means of wireless communication.

Alternatively, as illustrated inFIG. 12, a transmission route of the plurality of 2D images and/or the plurality of depth maps may be as follows. The electronic device10firstly transmits the plurality of 2D images and/or the plurality of depth maps to a first base station by means of TDD-LTE. The first base station transmits the plurality of 2D images and/or the plurality of depth maps to a core network by means of wired communication. The core network transmits the plurality of 2D images and/or the plurality of depth maps to the server30by means of wired communication. After the server30receives the plurality of 2D images and/or the plurality of depth maps, the server3converts them to the plurality of 3D images. At this time, the server30may be a server30independent of the core network, or may be a server30integrated in the core network. A transmission route of the plurality of 3D images may be as follows. The server30may transmit the plurality of 3D images to the core network by means of wired communication. The core network may transmit the plurality of 3D images to the second base station by means of wired communication. The second base station may transmit the plurality of 3D images to the terminal device20by means of wireless communication (at least one of WIFI, 4G and 5G).

In the TDD-LTE system, the transmission of uplink data and downlink data is performed on the same frequency band according to time cross allocation. The wireless transmission mode of TDD-LTE has high flexibility, and the time slot ratio of uplink transmission and the time slot ratio of downlink transmission may be flexibly adjusted according to actual needs, thereby satisfying efficient transmission of the plurality of 2D images, the plurality of depth maps, and the plurality of 3D images.

Alternatively, the electronic device10may transmit the plurality of 2D images and/or the plurality of depth maps to the server30by means of WIFI, and the server30transmits the plurality of 3D images to the terminal device20by means of wired communication and/or by means of wireless communication.

At this time, as illustrated inFIG. 11, a transmission route of the plurality of 2D images and/or the plurality of depth maps may be as follows. The electronic device10firstly transmits the plurality of 2D images and/or the plurality of depth maps to a first base station by means of WIFI. The first base station transmits the plurality of 2D images and/or the plurality of depth maps to a core network by means of wired communication. The core network transmits the plurality of 2D images and/or the plurality of depth maps to a second base station by means of wired communication. The second base station transmits the plurality of 2D images and/or the plurality of depth maps to the server30by means of wireless communication (at least one of WIFI, 4G and 5G). After the server30receives the plurality of 2D images and/or the plurality of depth maps, the server3converts them to obtain the plurality of 3D images. Subsequently, the server30transmits the plurality of 3D images to the terminal device20by means of wired communication and/or by means of wireless communication.

Alternatively, as illustrated inFIG. 12, a transmission route of the plurality of 2D images and/or the plurality of depth maps may be as follows. The electronic device10firstly transmits the plurality of 2D images and/or the plurality of depth maps to a first base station by means of WIFI. The first base station transmits the plurality of 2D images and/or the plurality of depth maps to a core network by means of wired communication. The core network transmits the plurality of 2D images and/or the plurality of depth maps to the server30by means of wired communication. After the server30receives the plurality of 2D images and/or the plurality of depth maps, the server3converts them to obtain the plurality of 3D images. At this time, the server30may be a server30independent of the core network, or may be a server30integrated in the core network. A transmission route of the plurality of 3D images may be as follows. The server30may transmit the plurality of 3D images to the core network by means of wired communication. The core network may transmit the plurality of 3D images to the second base station by means of wired communication. The second base station may transmit the plurality of 3D images to the terminal device20by means of wireless communication (at least one of WIFI, 4G and 5G).

The uplink rate and the downlink rate of the WIFI wireless communication method are both high, which may satisfy the efficient transmission of the plurality of 2D images, the plurality of depth maps, and the plurality of 3D images.

Referring toFIG. 11,FIG. 12andFIG. 18, the present disclosure provides a communication method for a communication system100. The system100includes an electronic device10, a server30, and a terminal device20. The method includes acts in the following blocks.

041: the electronic device10obtains a plurality of 2D images and/or a plurality of depth maps for a current scene.

043: the electronic device10transmits the plurality of 2D images and/or the plurality of depth maps to the server30.

044: the server30receives the plurality of 2D images and/or the plurality of depth maps from the electronic device20.

046: the server30generates a plurality of 3D images based on the plurality of 2D images and/or the plurality of depth maps.

047: the server30transmits the plurality of 3D images to the terminal device20.

Referring toFIG. 2,FIG. 11,FIG. 12andFIG. 14, the present disclosure further provides a communication system100. The system100includes an electronic device10, a server30, and a terminal device20. The electronic device10may be the electronic device10of any of the foregoing embodiments, the server30may be the server30of any of the foregoing embodiments, the terminal device20may be the terminal device20of any of the foregoing embodiments, and details are not described herein.

Referring toFIG. 2andFIG. 14, in some embodiments, the act in block041may be implemented by the obtaining module111. The act in block043may be implemented by the transmitting module113. The act in block044may be implemented by the receiving unit311. The act in block046may be implemented by the processing unit313. The act in block047may be implemented by the transmitting unit314.

Referring toFIGS. 3 and 15, in some embodiments, the act in block041may be implemented by the image acquisition component121. The act in block043may be implemented by the communication module123. The act in block044and the act in block047may be implemented by the communication unit321. The act in block046may be implemented by the processor322.

Referring toFIG. 5andFIG. 19, in some embodiments, the communication method provided in the embodiments of the present disclosure may be applied to a 3D video call and a 3D video conference, which may realize the interaction among the users. It is assumed that the electronic device10is a calling device, and the terminal device20is a called device. In this case, the act in block041may be follows: the electronic device10obtains a plurality of calling 2D images and/or a plurality of calling depth maps for the calling scene; the act in block043may be follows: the electronic device10transmits the plurality of calling 2D images and/or the plurality of calling depth maps to the server30. Further, the communication method provided the embodiments of the present disclosure further includes acts in the following blocks.

051: the terminal device20obtains a plurality of called 2D images and/or a plurality of called depth maps, for the called scene.

052: the terminal device20transmits the plurality of called 2D images and/or the plurality of called depth maps to the server30.

Referring toFIG. 5, in some embodiments, the act in block051may be implemented by the obtaining sub-module211. The act in block052may be implemented by the transmitting sub-module212.

Referring toFIG. 6, in some embodiments, the act in block051may be implemented by the image acquisition component221. The act in block052may be implemented by the communication component223.

Referring toFIG. 5andFIG. 20, in some embodiments, the communication method provided in the embodiments of the present disclosure may also be applied to a virtual-reality head-mounted display device, an augmented-reality display device, a holographic stereoscopic projection device, a display device equipped with a 3D liquid crystal displayer, or the like. At this time, the communication method provided in the embodiments of the present disclosure further includes acts in the following blocks.

053: the terminal device20displays a plurality of 3D images by means of holographic projection or 3D display.

Referring toFIG. 5, in some embodiments, the act in block053may be implemented by the display sub-module213.

Referring toFIG. 6, in some embodiments, the terminal device20further includes a processor222. The act in block053may be implemented by the processor222of the terminal device20.

Referring toFIG. 2andFIG. 21, in some embodiments, the communication method provided in the embodiments of the present disclosure further includes acts in the following blocks.

042: the electronic device10compresses the plurality of 2D images and/or the plurality of depth maps.

The act in block043, i.e., the electronic device10transmits the plurality of 2D images and/or the plurality of depth maps to the server30, may include an act in the following blocks.

0431: the electronic device10transmits a plurality of compressed 2D images and/or a plurality of compressed depth maps to the server30.

Referring toFIG. 2, in some embodiments, the act in block042may be implemented by the compressing module112. The act in block0431may be implemented by the transmitting module113.

Referring toFIG. 3, in some embodiments, the act in block042may be implemented by the processor122of the electronic device10. The act in block0431may be implemented by the transmitting module123.

Referring toFIG. 11,FIG. 12andFIG. 21, in some embodiments, the act in block044, i.e., the server30receives the plurality of 2D images and/or the plurality of depth maps transmitted by the electronic device10, includes acts in the following block.

0441: the server30receives the plurality of compressed 2D images and/or the plurality of compressed depth maps from the electronic device10.

The communication method provided in the embodiments of the present disclosure may include an act in the following block.

045: the server30decompresses the plurality of compressed 2D images and/or the plurality of compressed depth maps.

The act in block046, i.e., the server30generates the plurality of 3D images based on the plurality of 2D images and/or the plurality of depth maps, may include an act in the following block.

0461: the server30generates the plurality of 3D images based on a plurality of decompressed 2D images and/or a plurality of decompressed depth maps.

Referring toFIG. 17, in some embodiments, the s act in block0441may be implemented by the receiving unit311. The act in block045may be implemented by the compressing unit312. The act in block0461may be implemented by the processing unit313.

Referring toFIG. 15, in some embodiments, the act in block0441may be implemented by the communication unit321. The act in block045and the act in block0461may be implemented by the processor322.

Referring toFIG. 11andFIG. 12, in some embodiments, the server30receives the plurality of 2D images and/or the plurality of depth maps transmitted by the electronic device10by means of at least one wireless communication of WIFI, 4G and 5G. The server30transmits the plurality of 3D images to the terminal device20by means of at least one wireless communication of WIFI, 4G and 5G.

With the communication method and the communication system100provided in the embodiments of the present disclosure, on one hand, the 3D video transmission may be realized, such that a user may view stereoscopic 3D video effect at the terminal device20, and have a better experience; on the other hand, converting the plurality of 2D images and/or the plurality of depth maps to the plurality of 3D images may be realized at the server30, without the calculation at the electronic device10, thereby reducing an amount of data processing, and power consumption of the electronic device10, and reducing a time period required for image processing, and shortening the delay in video communication.

Referring toFIG. 3,FIG. 6,FIG. 11andFIG. 12, the present disclosure further provides the non-transitory computer-readable storage medium including one or more computer executable instructions. When the one or more computer executable instructions are executed by one or more processors122/222/322, the one or more processors122/222/322, performs the communication method described in any of the above embodiments

For example, when the one or more computer executable instructions are executed by the one or more processors122, the one or more processors122may performs the following acts:

controlling the image acquisition component121to obtain a plurality of 2D images and/or a plurality of depth maps, for a current scene; and

controlling the communication module123to transmit the plurality of 2D images and/or the plurality of depth maps to the server30, such that the server30is configured to generate a plurality of 3D images based on the plurality of 2D images and/or the plurality of depth maps, and to transmit the plurality of 3D images to a terminal device20

For another example, when the one or more computer executable instructions are executed by the one or more processors222, the one or more processors222may performs the following acts:

controlling image acquisition component221to obtain a plurality of called 2D images and/or a plurality of called depth maps, for a called scene; and

controlling the communication component223to transmit the plurality of called 2D images and/or the plurality of depth maps to the server30.

For another example, when the one or more computer executable instructions are executed by the one or more processors322, the one or more processors322may performs the following acts:

controlling the communication unit322to receive a plurality of 2D images and/or a plurality of depth maps from the electronic device10;

controlling the processor322of the server10to generate a plurality of 3D images based on the plurality of 2D images and/or the plurality of depth maps; and

controlling the communication unit322to transmit the plurality of 3D images to the terminal device20.

For another example, when the one or more computer executable instructions are executed by the one or more processors122/222/322, the one or more processors122/222/322may performs the following acts:

controlling the image acquisition component121to obtain a plurality of 2D images and/or a plurality of depth maps, for a current scene;

controlling the communication module123to transmit the plurality of 2D images and/or the plurality of depth maps to the server30;

controlling the communication unit322to receive the plurality of 2D images and/or the plurality of depth maps from the electronic device10;

controlling the processor322of the server10to generate a plurality of 3D images based on the plurality of 2D images and/or the plurality of depth maps; and

controlling the communication unit322of the server10to transmit the plurality of 3D images to the terminal device20.

Any process or method described in a flow chart or described herein in other ways may be understood to include one or more modules, segments or portions of codes of executable instructions for achieving specific logical functions or steps in the process, and the scope of a preferred embodiment of the present disclosure includes other implementations, which should be understood by those skilled in the art.

The storage medium mentioned above may be read-only memories, magnetic disks or CD, etc. Although explanatory embodiments have been shown and described, it would be appreciated by those skilled in the art that the above embodiments cannot be construed to limit the present disclosure, and changes, alternatives, and modifications can be made in the embodiments without departing from spirit, principles and scope of the present disclosure.