Image slice transforming method and electronic device

The embodiments of the disclosure provide an image slice transforming method and an electronic device. The method includes: receiving an i-th image slice outputted by an image encoder, wherein the i-th image slice belongs to N image slices divided from an image frame, i is an index, and N is an integer; obtaining a slice header of the i-th image slice; transforming the i-th image slice into an i-th sub-frame by modifying the slice header of the i-th image slice.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The disclosure generally relates to an image transforming mechanism, in particular, to an image slice transforming method and an electronic device.

2. Description of Related Art

SeeFIG.1, which shows a general video streaming mechanism. As shown inFIG.1, general video streaming is frame based, in which the image frame can only be decoded after the encoder has outputted the encoded image frame. In this case, the subsequent process (e.g., rendering) would be blocked until the encoder has outputted the image frame, which makes the latency longer.

In some image processing specifications (e.g., H.264), although the encoder could output image slices of the image frame, the subsequent process still needs to wait until all image slices of the image frame have been outputted by the encoder, which might lead to longer latency.

SUMMARY OF THE INVENTION

Accordingly, the disclosure is directed to an image slice transforming method and an electronic device, which may be used to solve the above technical problem.

The embodiments of the disclosure provide an image slice transforming method, adapted to an electronic device. The method includes: receiving an i-th image slice outputted by an image encoder, wherein the i-th image slice belongs to N image slices divided from an image frame, i is an index, and N is an integer; obtaining a slice header of the i-th image slice; transforming the i-th image slice into an i-th sub-frame by modifying the slice header of the i-th image slice.

The embodiments of the disclosure provide an electronic device including a storage circuit and a processor. The storage circuit stores a program code. The processor is coupled to the non-transitory storage circuit and accesses the program code to perform: receiving an i-th image slice outputted by an image encoder, wherein the i-th image slice belongs to N image slices divided from an image frame, i is an index, and N is an integer; obtaining a slice header of the i-th image slice; transforming the i-th image slice into an i-th sub-frame by modifying the slice header of the i-th image slice.

DESCRIPTION OF THE EMBODIMENTS

SeeFIG.2, which shows a schematic diagram of an electronic device according to an exemplary embodiment of the disclosure. In some embodiments, the electronic device200may be any computer device and/or a smart device, and the electronic device200may belong to a video streaming system including a transmitter and a receiver. In one embodiment, the electronic device200may be the transmitter of the video streaming system. In this case, the electronic device200may generate image frames and use an image encoder to encode the image frames. Afterwards, the electronic device200may transmit the image frames to the receiver for the image frames to decode and show the image frames as visual contents.

In one embodiment, the electronic device200may be the receiver of the video streaming system. In this case, the electronic device200may receive the encoded image frames from the transmitter of the video streaming system, and the electronic device200may decode and show the image frames as visual contents.

In various embodiments, the mechanism of the transmitter and the receiver to encode/decode image frames may be performed based on any known image encoding/decoding algorithms. For better understanding the concept of the disclosure, H.264 would be used as an example of the image encoding/decoding algorithm used by the transmitter and the receiver, but the disclosure is not limited thereto.

InFIG.2, the electronic device200includes a storage circuit202and a processor204. The storage circuit202is one or a combination of a stationary or mobile random access memory (RAM), read-only memory (ROM), flash memory, hard disk, or any other similar device, and which records a program code and/or a plurality of modules that can be executed by the processor204.

The processor204may be coupled with the storage circuit202, and the processor204may be, for example, a graphic processing unit (GPU), a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Array (FPGAs) circuits, any other type of integrated circuit (IC), a state machine, and the like.

In the embodiments of the disclosure, the processor204may access the modules and/or the program codes stored in the storage circuit202to implement the image slice transforming method provided in the disclosure, which would be further discussed in the following.

SeeFIG.3, which shows a flow chart of the image slice transforming method according to an embodiment of the disclosure. The method of this embodiment may be executed by the electronic device200inFIG.2, and the details of each step inFIG.3will be described below with the components shown inFIG.2.

In one embodiment, an image frame may be divided into N image slices (N is an integer), and the image encoder may encode and output the image slices.

Accordingly, in step S310, the processor204receives an i-th image slice outputted by the image encoder, wherein i is an index.

In step S320, the processor204obtains a slice header of the i-th image slice. In one embodiment, since the image encoder is assumed to operate based on H.264, the i-th image slice may include a plurality of macro-blocks and a slice header. In one embodiment, the slice header of the i-th image slice may include a sequence parameter set and an initial macro-block index. Based on H.264, the initial macro-block index may be known as a variable named first_mb_in_slice, and the sequence parameter set may include a height parameter of the i-th image slice, which may be known as a variable named pic_height_in_map_units_minus1. The definitions of the sequence parameter set, the height parameter, and the initial macro-block index may be referred to the specification of H.264, which would not be provided herein.

In H.264, the receiver cannot perform decoding and/or rendering on any of the N image slices until all of the N image slices related to the image frame have been received. That is, the receiver one image slice cannot be individually decoded and/or rendered by the receiver, which increases the latency of video streaming. However, if one image slice may be modified to be regarded as an independent frame (which may be referred to as a sub-frame), the receiver may be allowed to decode/render the sub-frame individually.

Accordingly, in step S330, the processor204transforms the i-th image slice into an i-th subframe by modifying the slice header of the i-th image slice. In one embodiment, the processor204may modify the slice header of the i-th image slice by modifying the height parameter (i.e., pic_height_in_map_units_minus1) and the initial macro-block index (i.e., first_mb_in_slice) of the slice header of the i-th image slice.

In one embodiment, the processor204may modify the height parameter of the slice header based on a height of the i-th image slice and a size of the macro-blocks. In one embodiment, the processor204may divide the height of the i-th image slice with a height of one of the macro-blocks to obtain a quotient (referred to as Q). Next, the processor204may modify the height parameter of the slice header as Q−1.

For example, assuming that the height of the i-th image slice is 1600 pixels and the size of a macro-block is 16×16. In this case, the processor204may obtain a quotient of 100 by calculating 1600/16. Next, the processor204may modify the height parameter of the slice header as Q−1 (i.e., 99), but the disclosure is not limited thereto.

In one embodiment, the processor204may modify the initial macro-block index of the slice header as a predetermined index, which may be, for example, 0. In detail, based on H.264, the initial macro-block index is originally used to characterize the order of the first macro-block of the i-th image slice in the image frame. After modifying the initial macro-block index of the slice header of the i-th image slice to be 0, the i-th image slice may be regarded as an independent subframe. For better understanding the above concept,FIG.4would be used as an example.

SeeFIG.4, which shows a schematic diagram of modifying the initial macro-block indexes of slice headers according to an embodiment of the disclosure. InFIG.4, an image frame IM1may be divided into 4 image slices, i.e., image slice0, image slice1, image slice2, and image slice3, and each image slice may include 510 macro-blocks. For example, image slice0may include macro-block0to macro-block509of the image frame IM1, and image slice1may include macro-block510to macro-block1019of the image frame IM1. The order of the macro-blocks in image slice2and image slice3may be understood based on the above teachings, which would not be further provided.

Since the order of the first macro-block (i.e., macro-block0) in image slice0is 0, the original value indicated by the initial macro-block index in the slice header of image slice0may be 0. Since the order of the first macro-block (i.e., macro-block510) in image slice1is 510, the original value indicated by the initial macro-block index in the slice header of image slice1may be 510. Since the order of the first macro-block (i.e., macro-block1020) in image slice2is 1020, the original value indicated by the initial macro-block index in the slice header of image slice2may be 1020.

In the scenario ofFIG.4, when the processor204receives image slice0, the processor204may modify the initial macro-block index in the slice header of image slice0to be 0 and modify the height parameter to be the corresponding Q−1 such that image slice0with the modified header may be regarded as sub-frame0. When the processor204receives image slice1, the processor204may modify the initial macro-block index in the slice header of image slice1to be 0 and modify the height parameter to be the corresponding Q−1, such that image slice1with the modified header may be regarded as sub-frame1. When the processor204receives image slice2, the processor204may modify the initial macro-block index in the slice header of image slice2to be 0 and modify the height parameter to be the corresponding Q−1, such that image slice2with the modified header may be regarded as sub-frame2. When the processor204receives image slice3, the processor204may modify the initial macro-block index in the slice header of image slice3to be 0 and modify the height parameter to be the corresponding Q−1, such that image slice3with the modified header may be regarded as sub-frame3.

Based on the above teachings, the processor204may transform the i-th image slice into the i-th sub-frame by modifying the slice header of the i-th image slice. In one embodiment, since the length of a sub-frame header is specified to be a predetermined length in the specification of H.264, the processor204may determine whether the length of the modified slice header of the i-th image slice meets the predetermined length. In one embodiment, in response to determining that a length of the modified slice header of the i-th image slice is shorter than the predetermined length, the processor204may adjust the length of the modified slice header of the i-th image slice to meet the predetermined length by padding the modified slice header with specific data bits (e.g., 0s). In this case, the i-th image slice with the modified and padded slice header may be regarded as the i-th sub-frame, but the disclosure is not limited thereto.

In one embodiment, when the electronic device200is implemented as the transmitter of the video streaming system, the electronic device200may transmit the i-th subframe to the receiver of the video streaming system. Since the i-th image slice has been transformed into the i-th sub-frame, the receiver of the video streaming system may decode the i-th sub-frame and render the decoded i-th sub-frame right after receiving the i-th sub-frame. That is, the receiver does not have to wait for other sub-frames related to the image frame to start decoding and rendering the i-th sub-frame. Since the receiver may start decoding the i-th sub-frame earlier, the latency of the receiver may be reduced.

In another embodiment, when the electronic device200is implemented as the receiver of the video streaming system, the electronic device200may decode the i-th sub-frame and render the decoded i-th sub-frame right after transforming the i-th image slice into the i-th sub-frame. Accordingly, the latency of the electronic device200may be reduced.

In other embodiments, the processor204may perform other operations similar to those provided inFIG.3to other image slices of the N image slices to transform these image slices into the corresponding sub-frames. For example, after receiving an (i+1)-th image slice of the N image slices from the image encoder, the processor204may obtain a slice header of the (i+1)-th image slice and transform the (i+1)-th image slice into an (i+1)-th sub-frame by modifying the slice header of the (i+1)-th image slice, and the details thereof may be referred to the above teachings, which would not be repeated herein.

In one embodiment, when the electronic device200is implemented as the receiver of the video streaming system, the electronic device200may determine whether the N image slices have been transformed into the corresponding N sub-frames and each sub-frame have been decoded and rendered. If yes, the processor204may merge the rendered sub-frames as a visual content and provide the visual content.

For example, assuming that the electronic device200is a head-mounted display used to provide virtual reality contents to the wearer thereof. In this case, the electronic device200may show the merged sub-frames as the visual content for the wearer to see. Since the electronic device200may operate with a lower latency, the user experience of the wearer may be improved accordingly.

In summary, the embodiments of the disclosure may transform each image slice into the corresponding sub-frame by modifying the slice header thereof. Since each sub-frame may be individually decoded and/or rendered, the receiver of the video streaming system does not have to wait for other sub-frames related to the image frame. Accordingly, the receiver may start decoding each sub-frame earlier, and hence the latency of the receiver may be reduced, which may improve the user experience accordingly.