METHOD FOR MULTIVIEW PICTURE DATA ENCODING, METHOD FOR MULTIVIEW PICTURE DATA DECODING, AND MULTIVIEW PICTURE DATA DECODING DEVICE

A method for multiview picture data encoding comprising the steps of: performing extraction of features from multiview picture data to obtain a plurality of feature maps; performing stitching and/or transforming of the obtained plurality of feature maps to obtain at least one panoramic map of features; performing transforming of the multiview picture data to select a plurality of patches of view of the multiview picture data; encoding the at least one panoramic map of features; and encoding the plurality of patches of view.

TECHNICAL FIELD

The present invention relates to the technical field of compression and decompression of visual information. More specifically, the present invention relates to a method for multiview picture data encoding, a method for multiview picture data decoding, and a multiview picture data decoding device.

BACKGROUND

Coding is used in a wide range of applications which involve visual information such as pictures, for example, still pictures (such as still images) but also moving pictures such as picture streams and videos. Examples of such applications include transmission of still images over wired and wireless mobile networks, video transmission and/or video streaming over wired or wireless mobile networks, broadcasting digital television signals, real-time video conversations such as video-chats or video-conferencing over wired or wireless mobile networks and storing of images and videos on portable storage media such as DVD disks or Blue-ray disks.

Coding usually involves encoding and decoding. Encoding is the process of compressing and potentially also changing the format of the content of the picture. Encoding is important as it reduces the bandwidth needed for transmission of the picture over wired or wireless mobile networks. Decoding on the other hand is the process of decoding or uncompressing the encoded or compressed picture. Since encoding and decoding is applicable on different devices, standards for encoding and decoding called codecs have been developed. A codec is in general an algorithm for encoding and decoding of pictures.

Reducing the bandwidth needed for transmission of the pictures is particularly important when the picture is a so-called panoramic picture such as a still panoramic image or panoramic video due to, in general, the large size of the panoramic picture. Therefore, for example, a codec may be applied for encoding (compressing) the panoramic picture (for example the panoramic picture data) such that the bandwidth needed for transmission is reduced. In the same time, it is highly desirable that the quality of the encoded (compressed) panoramic picture is preserved as much as possible.

In general, the panoramic picture such as still panoramic picture (such as still panoramic image) but also moving panoramic picture such as panoramic picture stream and panoramic video may also be called or represent a panoramic view. In other words, a panoramic view is generally understood to represent a continues view in a plurality (at least two) of directions. For example, a panoramic view may be a 360° image or 360° video. Such 360° image or 360° video conveys the view of a whole panorama of a scene seen from a given point. The panoramic view may be just a 2D panoramic representation or a representation of an omnidirectional image or video obtained by mapping.

In general, the panoramic view is captured by multiple cameras each looking in a different direction. It is also possible to capture a panoramic view by using one camera which captures multiple views (view being understood in the sense of image or video), each view being captured with the camera looking in a different direction. Hence, a panoramic view may be seen as a multiview, since it is obtained based on several individual (input) views by applying suitable processing on the individual views.

For example, several (at least two) individual (input) views, such as several images or several videos are combined together into a panoramic view on the encoder side. The panoramic view is then encoded (compressed) and transmitted, normally in a form of a bitstream, to a decoding side for decoding as elaborated above.

At the decoding side, normally, feature extraction is applied for extracting features from the decoded panoramic view to reconstruct the panoramic view. However, the accuracy of feature extraction may depend strongly on the coding loss of the decoded panoramic view.

Therefore, there is a need to increase the quality of the reconstructed panoramic view on the decoding side.

SUMMARY

According to a first aspect of the present invention there is provided a method for multiview picture data encoding comprising the steps of:performing extraction of features from multiview picture data to obtain a plurality of feature maps;performing stitching and/or transforming of the obtained plurality of feature maps to obtain at least one panoramic map of features;performing transforming of the multiview picture data to select a plurality of patches of view of the multiview picture data;encoding the at least one panoramic map of features; andencoding the plurality of patches of view.

According to a second aspect of the present invention there is provided a method for multiview picture data decoding comprising the steps of:obtaining at least one encoded panoramic map of features;performing decoding of the obtained at least one encoded panoramic map of features;obtaining a plurality of encoded patches of view of a multiview picture data;performing decoding on the obtained plurality of encoded patches of view;performing extraction of features from the decoded plurality of patches of view to obtain a plurality of feature maps; andperforming matching of the obtained plurality of feature maps with said decoded panoramic map of features to obtain the position of each patch of view of the plurality of patches of view in a panoramic picture data.

According to a third aspect of the present invention there is provided a multiview picture data decoding device comprising a processor and an access to a memory to obtain code that instructs said processor during operation to:obtain at least one encoded panoramic map of features;perform decoding of the obtained at least one encoded panoramic map of features;obtain a plurality of encoded patches of view of a multiview picture data;perform decoding on the obtained plurality of encoded patches of view;perform extraction of features from the decoded plurality of patches of view to obtain a plurality of feature maps; andperform matching of the obtained plurality of feature maps with said decoded panoramic map of features to obtain the position of each patch of view of the plurality of patches of view in a panoramic picture data.

DETAILED DESCRIPTION

FIG.1Ashows a schematic view of a general use case as in the conventional arts as well as an environment for employing embodiments of the present invention. On the encoding side1there is arranged equipment100-1,100-2, such as data centres, servers, processing devices, data storages and the like that is arranged to store and process multiview picture data and generate one or more bitstreams by encoding the multiview picture data.

Generally, the term multiview picture data in the description here below refers to picture data relating to more than one view. In other words, multiview picture data comprises a plurality of individual views. The plurality of individual views may also be seen to represent a plurality of viewports or plurality of directions from a specific viewpoint. Each one of the individual views is and/or includes data that is, contains, indicates and/or can be processed to obtain an image, picture, a stream of pictures/images, a video, a movie and the like, wherein, in particular, a stream, a video or a movie may contain one or more images.

For simplicity, in the description here below, the term view is used in the sense of image or video. The image or the video may be monochromatic or colour image or video. Accordingly, multiview picture data may comprise a plurality of individual images or videos. Each individual view is captured by at least one image capturing unit (for example camera), each image capturing unit looking at a different direction outward from a viewpoint. It is also possible that each individual view is captured by a single image capturing unit, said image capturing unit looking in a different direction outward from a viewpoint when capturing each individual view.

By further processing such multiview picture data panoramic picture data on the decoding side may be obtained as elaborated further below. Panoramic picture data may be understood as data that is, contains, indicates and/or can be processed to obtain at least in part a (reconstructed) panoramic view. The panoramic view includes data that is, contains, indicates and/or can be processed to obtain a panoramic image, a panoramic picture, a stream of panoramic pictures/images, a panoramic video, a panoramic movie, and the like, wherein, in particular, a panoramic stream, panoramic video or a panoramic movie may contain one or more pictures. For simplicity, in the description here below the term panoramic view is used in the sense of panoramic image or panoramic video. The word reconstructed may be seen as indicating that the data is a reconstruction at least in part on the decoding side2of the corresponding data on the encoding side1.

Hence, a panoramic view may be seen as a multiview, since it is obtained based on several individual (input) views.

In general, panoramic view is a continuous view of a scene in at least two directions. The panoramic view may represent the scene in different manners, such as cylindrical, cubic, spherical and etc.

For example, the panoramic view may be a 360° image or 360° video. Such 360° image or 360° video conveys the view of a whole panorama of a scene seen from a given point. Panoramic view may also be just a 2D panoramic representation or a representation of an omnidirectional image or video obtained by any mapping.

On the encoding side1the one or more generated bitstreams are conveyed50via any suitable network and data communication infrastructure toward the decoding side2, where, for example, a mobile device200-1is arranged that receives the one or more bitstreams, decodes them and processes them to generate panoramic picture data which as elaborated above may be and/or contain and/or indicate and/or can be processed to obtain a (reconstructed) panoramic view for displaying it on a display200-2of the (target) mobile device200-1or are subjected to other processing on the mobile device200-1.

FIG.1Bshows a schematic view of the conventional configuration for encoding and decoding of multiview picture data andFIG.1Cshows schematically the pipeline for transmission of multiview picture data from an encoding side1to a decoding side2.

Multiview picture data10, which, as elaborated above, may comprise a plurality of individual views such as a plurality of individual images or videos, captured, for example, by a plurality of cameras are combined in one panoramic view28-1on the encoder side1. The plurality of individual views may also be called here below a plurality of input views. Combining may comprise for example stitching13together the plurality of individual views10in a stitcher13provided in the encoding side1to thereby generate a single panoramic view28-1. An encoder30provided in the encoded side1encodes the generated panoramic view28-1and the encoded panoramic view28-1is then transmitted50to the decoding side2normally in a form of one or more bitstreams.

On the decoding side2, there is provided a decoder60in which it is performed decoding of the received encoded panoramic view28-1to thereby obtain a decoded panoramic view28-2. A feature extractor25is further provided on the decoding side2, in which it is performed extraction of features (feature extraction) from the decoded panoramic view28-2to thereby obtain a panoramic map of features23. The extraction of features in the feature extractor25may involve for example Scale-Invariant Feature Transform (SIFT) keypoints extraction. Thus, a panoramic map of features23needs to be available on the decoding side2. The obtained panoramic map of features23is then used on the decoding side2to, at least partly reconstruct, the panoramic view28-2from the received encoded panoramic view on the decoding side2.

As elaborated above, the accuracy of feature extraction in the feature extractor25depends strongly on the coding loss of the decoded panoramic view28-2. Reduced accuracy of the step of feature extraction reduces in turn the accuracy and hence the quality of the at least partly reconstructed panoramic view.

Therefore, the present invention aims at increasing the quality of the at least partly reconstructed panoramic view on the decoding side2.

For this, the present invention proposes that the complete panoramic map of features is transmitted from the encoding side1to the decoding side2and further proposes building (or reconstructing) the panoramic view on the decoding side2from the received panoramic map of features and patches of view, as elaborated further below. Patch of view, as also elaborated here below, refers to a single (individual) view from the plurality of individual views, its fragment or combination of fragments. In other words, each patch of view, in the description here below is any one of an individual view, a part of an individual view or a combination of at least two parts of an individual view. Hence, according to the present invention the panoramic view does not need to be produced on the encoding side1, as elaborated above, in respect to the panoramic view28-1.

FIG.2Ashows schematically the configuration for multiview picture data encoding and multiview picture data decoding according to the embodiment of the present invention.FIG.2Bshows schematically a pipeline of transmission multiview picture data according to the embodiment of the present invention.

As elaborated above, multiview picture data10are obtained on the encoding side. As elaborated above, the multiview picture data10comprise a plurality of individual views. In this embodiment, each one of the individual views is captured by at least one image capturing unit, each image capturing unit looking in a different direction outward from a viewpoint. Accordingly, obtaining the multiview picture data10may be understood as receiving on the encoding side1the plurality of individual views from, for example, the corresponding image capturing units, and/or any other information processing, device and/or other encoding device.

In the encoding side1there is provided a feature extractor11in which it is performed extraction of features from the multiview picture data10to obtain a plurality of feature maps12. More specifically, in the feature extractor11it is performed extraction of features from each individual view of the multiview picture data10to thereby obtain at least one feature map12for each individual view. For simplicity, it may be considered that the number of feature maps12is equal to the number of individual views of the multiview picture data10.

In the feature extractor11the extraction of features is performed by applying a predetermined feature extraction method. The extracted features may be seen to represent small fragments in the corresponding individual view of the multiview picture data10. Each feature, in general, comprises a feature key point and a feature descriptor. The feature key point may represent the fragment 2D position. The feature descriptor may represent visual description of the fragment. The feature descriptor is generally represented as a vector, also called a feature vector.

The predetermined feature extraction method may result in the extraction of discrete features. For example, the feature extraction method may comprise any one of scale-invariant feature transform, SIFT, method, compact descriptors for video analysis, CDVA, method or compact descriptors for visual search, CDVS, method.

In other embodiment of the present invention the predetermined feature extraction method may also apply linear or non-linear filtering. For example, the feature extractor11may be a series of neural-network layers that extract features from the multiview picture data10through linear or non-linear operations. The series of neural-network layers may be trained based on a given data. The given data may be a set of images which have been annotated with what object classes are present in each image. The series of neural-network layers may automatically extract the most salient features with respect to each specific object class.

For example, in embodiments of the present invention, the predetermined feature extraction method may be, for example, the Scale-Invariant Feature Transform method as elaborated above and the performing of features extraction in the feature extractor11on the encoding side1may comprise for example calculation of SIFT keypoints.

In the encoding side1there is further provided a stitcher13in which there is performed stitching and/or transforming of the obtained plurality of feature maps12, extracted from the multiview picture data10, to obtain at least one panoramic map of features14. The panoramic map of features may be, for example cubic, cylindrical or spherical representation of the plurality of feature maps12. In the stitcher12the stitching and/or transforming may be performed, for example, based on overlapping features maps of the plurality of feature maps12extracted from the multiview picture data10. With transforming, for example, redundant elements and/or information may be removed. The particular way of stitching and/or transforming of the obtained plurality of feature maps12from the multiview picture data10to obtain at least one panoramic map of features14is not limiting to the present invention.

In the encoding side1there is further provides a transformer16in which it is performed transforming of the multiview picture data10to select a plurality of patches of view17of the multiview picture data10. For example, in the transformer16there is performed transformation of the multiview picture data (of the individual input views), by performing searching and cropping of overlapping regions based on the plurality of features maps12and the at least one panoramic map14to reduce redundant information and to thereby select the plurality of patches of view17. This is shown, for example inFIG.2B, with dashed arrows. One or more than one patch of view may be selected from each individual view. It is also possible that from some individual views no patch of view is selected. The way of selecting the plurality of patches of view17may be any suitable method. In other words, the present invention is not limited to any particular way of selecting the plurality of patches of view17.

As elaborated above, each patch of view is any one of an individual view of the multiview picture data10, a part of an individual view or a combination of at least two parts of an individual.

In the encoding side1there is further provided a first encoder15in which it is performed encoding the at least one panoramic map of features14.

In the encoding side1there is further provided a second encoder18in which it is performed encoding the plurality of patches of view17.

The encoding in the first encoder15may comprise performing compressing of the at least one panoramic map of features14. Similarly, the encoding in the second encoder18may comprise performing compressing of the plurality of patches of view17. In the following, the words encoding and compressing may be interchangeably used.

In the first encoder15and the second encoder18the encoding the at least one panoramic map of features14and the encoding the plurality of patches of view17are performed independently from each other.

The first encoder15and the second encoder18may also be placed in a single encoder, however, even when placed in a single encoder the encoding the at least one panoramic map of features14and encoding the plurality of patches of view17are performed independently from each other. For example such single encoder may have two input ports, one for the at least one panoramic map of features14and one for the plurality of patches of view17to thereby encode the at least one panoramic map of features14and the plurality of patches of view17independently from each other and may respectively have two output ports to output respectively the encoded at least one panoramic map of features14and the encoded plurality of patches of view17.

In addition, in the second encoder18, the encoding of the plurality of patches of view17may comprise encoding independently each one of the patches of view17.

The first encoder15which generates the encoded at least one panoramic map of features by performing encoding of the at least one panoramic map of features14may apply various encoding methods applicable for encoding the at least one panoramic map of features14. More specifically, the first encoder15may apply various encoding methods applicable for encoding in general pictures such as still images and/or videos. The first encoder15applying various encoding methods applicable for encoding in general still images and/or videos may comprise the first encoder15applying a predetermined encoding codec. Such encoding codec may comprise encoding codec for encoding images or videos such as any one of the Joint Photographic Experts Group, JPEG, JPEG 2000, JPEG XR etc., Portable Network Graphics, PNG, Advanced Video Coding, AVC (H.264), Audio Video Standard of China (AVS), High Efficiency Video Coding, HEVC (H.265), Versatile Video Coding, VVC (H.266) or AOMedia Video 1, AV1 codec. In general, the first encoder15may apply a lossy or lossless compression (encoding) of the at least one panoramic map of features14. The used specific encoding codec is not to be seen as limiting to the present invention.

Similarly, the second encoder18which generates the encoded plurality of patches of view by performing encoding to the plurality of patches of view17may apply any on the above-mentioned encoding codec. The first encoder15and the second encoder18may apply the same encoding codec but may also apply a different encoding codec. This is possible, since as elaborated above, in the first encoder15and the second encoder18the encoding the at least one panoramic map of features14and the encoding the plurality of patches of view17are performed independently from each other. Accordingly, it is possible to adjust (or control) the quality of the encoded at least one panoramic map of features and the encoded plurality of patches of view independently from each other. More specifically, the high quality of the panoramic map of features14can be preserved in this way using appropriate coding method.

The encoded or compressed at least one panoramic map of features, which in general may be represented as a bitstream, is outputted to a first transmitter50-1, for example any kind of communication interface configured to transmit the encoded at least one panoramic map of features14over a communication network to a decoding side2. The communication network may be any wired or wireless mobile network.

In other words, in the encoding side1there is further provided a first transmitter50-1for transmitting the encoded at least one panoramic map of features, normally as a bitstream, to the decoding side2for decoding.

Similarly, the encoded or compressed plurality of patches of view may be represented as a bitstream which is outputted to a second transmitter50-2, for example, any kind of communication interface configured to transmit the encoded plurality of patches of view17represented as a bitstream over a communication network. The communication network may be any wired or wireless mobile network.

In other words, in the encoding side1there is further provided a second transmitter50-2for transmitting the encoded plurality of patches of view, normally as a bitstream, to the decoding side2for decoding.

In the first transmitter50-1and the second transmitter50-2the transmitting the encoded at least one panoramic map of features to the decoding side2for decoding and transmitting the encoded plurality of patches of view to the decoding side for decoding are performed independently from each other.

The first transmitter50-1and the second transmitter50-2may be arranged in a single transmitter50, however, even when arranged in a single transmitter the transmitting the encoded at least one panoramic map of features to the decoding side2for decoding and transmitting the encoded plurality of patches of view to the decoding side for decoding are performed independently from each other. For example, such transmitter may comprise two input ports, one for the encoded at least one panoramic map of features to be fed in and one for the encoded plurality of patches of view to be fed in and may also comprise two output ports, one for the transmitting the encoded at least one panoramic map of features and one for transmitting the encoded plurality of patches of view, to thereby transmit the encoded at least one panoramic map of features and the encoded plurality of patches of view independently from each other.

In an implementation, a module may be used to multiplex the encoded at least one panoramic map of features and the encoded plurality of patches of view to form a single bitstream which is transmitted by a transmitter. In another implementation, the module may be within the transmitter.

In another implementation, the encoded at least one panoramic map of features and the encoded plurality of patches of view may be transmitted by a multiplex transmitter. In other words, the multiplex transmitter may be used to multiplex the encoded at least one panoramic map of features and the encoded plurality of patches of view to form a single bitstream.

In a complementary manner a module may be used in the decoding side2or between the encoding side1and the decoding side2to demultiplex the multiplexed encoded at least one panoramic map of features and the encoded plurality of patches of view to form two bitstreams which are provided for processing in the decoding side2.

At the decoding side2there is provided at least one communication interface configured to receive communication data conveying the encoded at least one panoramic map of features and the encoded plurality of patches of view over a communication network, which may be, as elaborated above, any wired or wireless mobile network. In other words, the communication interface is adapted to perform communication over a wired or a wireless mobile network. The at least one communication interface is configured to receive (or obtain) independently the encoded at least one panoramic map of features and the encoded plurality of patches of view. For example, the at least one communication interface may comprise two input ports and two output ports. One set of input port and output port is used for receiving and outputting to a first decoder21provided in the decoding side2the encoded at least one panoramic map of features and one set of input port and output port is used for receiving and outputting to a second decoder22provided in the decoding side2the encoded plurality of patches of view.

Accordingly, in the decoding side2there is provided a first decoder21in which there is performed obtaining the at least one encoded panoramic map of features and decoding (or decompressing) the obtained at least one encoded panoramic map of features to thereby generate a decoded (or decompressed) at least one panoramic map of features23. In the present description the words decoding and decompressing may be interchangeably used.

Further, accordingly, in the decoding side2there is provided a second decoder22in which there is performed obtaining the plurality of encoded patches of view of the multiview picture data10and performing decoding (or decompressing) on the obtained plurality of encoded patches of view to thereby obtain a decoded (or decompressed) plurality of patches of view24.

In the decoding side there is further provided a feature extractor25in which there is performed extraction of features (feature extraction) from the decoded plurality of patches of view24to obtain a plurality of feature maps26. Similar to the feature extractor11provided in the encoding side, in the feature extractor25provided in the decoding side2the extraction of features is performed by applying a predetermined feature extraction method. The predetermined feature extraction method may be any one of the predetermined feature extraction methods elaborated with respect to the feature extractor11on the encoding side1or may be other feature extraction method chosen according to the specific needs, such as computation power, accepted latency and etc.

In the decoding side2there is further provided a matcher27in which there is performed matching of the obtained plurality of feature maps26with the decoded panoramic map of features23to obtain the position of each patch of view of the plurality of patches of view in a panoramic picture data29. For the process of matching any suitable matching method may be used. In other words, the present invention is not limited to a particular matching method.

In the decoding side2there is further provided a stitcher28. The decoded plurality of patches of view24is also fed from the second decoder22into the stitcher28in which there is performed stitching of the decoded plurality of patches of view24to obtain the panoramic picture data29based on the obtained position of each patch of view in the matcher27. In other words, information for the obtained position of each patch of view from the plurality of patches of view24is fed from the matcher27in the stitcher28which uses this information to respectively stitch the decoded plurality of patches of view24fed from the second decoder22to thereby obtain (or reconstruct) panoramic picture data29.

As elaborated above, panoramic picture data29may be understood as data that is, contains, indicates and/or can be processed to obtain at least in part a (reconstructed) panoramic view. The panoramic view includes data that is, contains, indicates and/or can be processed to obtain a panoramic image, a panoramic picture, a stream of panoramic pictures/images, a panoramic video, a panoramic movie, and the like, wherein, in particular, a panoramic stream, panoramic video or a panoramic movie may contain one or more pictures. For simplicity, in the description here below the term panoramic view is used in the sense of panoramic image or panoramic video.

This obtained panoramic picture data29may be output from the stitcher28for further processing in the decoding side2, for example for a display on a display200-2of the mobile device200-1elaborated with respect toFIG.1Aabove or other processing. The obtained panoramic picture data29may be at least a partly reconstructed panoramic view.

In this way, according to the present invention, the reconstruction of the panoramic view on the decoding side2is performed using the decoded panoramic map of features23and the decoded plurality of patches of view24. Therefore, the information about location and transformation of each patch of view of the plurality of patches of view24in the obtained panoramic picture data29is concluded from the matching between the decoded panoramic map of features23and features of the plurality of patches of view24.

Because the encoding the panoramic map of features14and encoding the plurality of patches of views17are performed independently from each other, the quality of both can be adjusted independently as elaborated above. Especially, the high quality of the encoded panoramic map of features14can be preserved, using appropriate coding method. Since the decoded panoramic map of features23whose high quality can be preserved in this way is used for obtaining (reconstructing or generating) the panoramic picture data29, the quality of the obtained (reconstructed) panoramic picture data29and hence the quality of the at least in part reconstructed panoramic view is also increased.

FIG.3Ashows a schematic view of a general device embodiment for the encoding side1according to an embodiment of the present invention. An encoding device80comprises processing resources81, a memory access82as well as a communication interface83. The mentioned memory access82may store code or may have access to code that instructs the processing resources81to perform the one or more steps of any method embodiment of the present invention a as described and explained in conjunction with the present disclosure.

Specifically, the code may instruct the processing resources81to perform extraction of features from multiview picture data10to obtain a plurality of feature maps12; to perform stitching and/or transforming of the obtained plurality of feature maps12to obtain at least one panoramic map of features14; perform transforming of the multiview picture data10to select a plurality of patches of view17of the multiview picture data; encode the at least one panoramic map of features14; and encode the plurality of patches of view17.

The processing resources81may be embodied by one or more processing units, such as a central processing unit (CPU), or may also be provided by means of distributed and/or shared processing capabilities, such as present in a datacentre or in the form of so-called cloud computing.

The memory access82which can be embodied by local memory may include but not limited to, hard disk drive(s) (HDD), solid state drive(s) (SSD), random access memory (RAM), FLASH memory. Likewise, also distributed and/or shared memory storage may apply such as datacentre and/or cloud memory storage.

The communication interface83may be adapted for receiving data conveying the multiview picture data10as well as for transmitting communication data conveying the encoded at least one panoramic map of features and the plurality of encoded patches of view over a communication network. The communication network may be a wired or a wireless mobile network.

FIG.3Bshows a schematic view of a general device embodiment for the decoding side2according to an embodiment of the present invention. A decoding device90comprises processing resources91, a memory access92as well as a communication interface93. The mentioned memory access92may store code or may have access to code that instructs the processing resources91to perform the one or more steps of any method embodiment of the present invention an as described and explained in conjunction with the present disclosure. The communication interface93may be adapted for receiving communication data conveying the encoded at least one panoramic map of features and the plurality of encoded patches of view over a network. The network may be a wired network or a wireless mobile network. The communication interface93may in addition be adapted for transmitting communication data conveying the above-elaborated panoramic picture data29.

Further, the device90may comprise a display unit94that can receive display data from the processing resources91so as display content in line with the display data. The display data may be based on the panoramic picture data29elaborated above. The device90can generally be a computer, a personal computer, a tablet computer, a notebook computer, a smartphone, a mobile phone, a video player, a tv set top box, a receiver, etc. as they are as such known in the arts.

Specifically, the code may instruct the processing resources91to obtain at least one encoded panoramic map of features; perform decoding of the obtained at least one encoded panoramic map of features; obtain a plurality of encoded patches of view of a multiview picture data; perform decoding on the obtained plurality of encoded patches of view; perform extraction of features from the decoded plurality of patches of view to obtain a plurality of feature maps; perform matching of the obtained plurality of feature maps with said decoded panoramic map of features to obtain the position of each patch of view of the plurality of patches of view in a panoramic picture data.

FIG.4Ashows a flowchart of general method embodiment of the present invention that refers to encoding multiview video data. Specifically, the embodiment provides a method for multiview video data encoding comprising the steps of: performing extraction of features (S11) from multiview picture data10to obtain a plurality of feature maps; performing stitching and/or transforming (S12) of the obtained plurality of feature maps to obtain at least one panoramic map of features14; performing transforming (S13) of the multiview picture data to select a plurality of patches of view17of the multiview picture data; encoding (S14) the at least one panoramic map of features14; and encoding (S15) the plurality of patches of view17.

FIG.4Bshows a flowchart of a general method embodiment of the present invention which relates to decoding of multiview data10. More specifically the embodiment provides a method multiview video data decoding comprising the steps of: obtaining (S21) at least one encoded panoramic map of features; performing decoding (S22) of the obtained at least one encoded panoramic map of features; obtaining (S23) a plurality of encoded patches of view of a multiview picture data; performing decoding (S24) on the obtained plurality of encoded patches of view; performing extraction (S25) of features from the decoded plurality of patches of view24to obtain a plurality of feature maps26; and performing matching (S26) of the obtained plurality of feature maps26with said decoded panoramic map of features23to obtain the position of each patch of view of the plurality of patches of view in a panoramic picture data29.

In summary, according to the embodiments of the present invention there is provided a transmission of (complete) panoramic map of features14from an encoding side1to a decoding side2and building the panoramic picture data29on the decoding side2form the received and decoded panoramic map of features23and received and decoded patches of view24. Hence, a panoramic view does not need to be produced on the encoding side1as elaborated in respect toFIG.1BandFIG.1C. In other words, there is no need for stitching the panoramic view28-1on the encoding side1and encoding the stitched panoramic view. Since according to the present invention, the encoding of the at least one panoramic map of features14and encoding of the plurality of patches of views17are independent from each other, the quality of both can be adjusted independently from each other. In particular, the high quality of the at least one panoramic map of features can be preserved using appropriate coding method.

In general, the skilled person will understand that the exact method for encoding of multiview picture data10can be chosen according to the available computing power, acceptable latency etc.

Although detailed embodiments have been described, these only serve to provide a better understanding of the invention defined by the independent claims and are not to be seen as limiting.

LIST OF REFERENCE SIGNS

1encoding side2decoding side100-1,100-2equipment on encoding side200-1equipment on decoding side200-2display of equipment on decoding side10multiview picture data11feature extractor on encoding side12plurality of feature maps on encoding side13stitcher on encoding side14panoramic map of features on encoding side15first encoder16transformer17patches of view on encoding side18second encoder21first decoder22second decoder23panoramic map of features on decoding side24patches of view on decoding side25feature extractor on decoding side26plurality of feature maps on decoding side27matcher on decoding side28stitcher on decoding side29reconstructed panoramic view/panoramic picture data28-1panoramic view on encoding side28-2decoded panoramic view30encoder50transmitting, transmitter50-1first transmitter50-2second transmitter60decoder