Method and device for managing multimedia data

A method for managing multimedia data by a contents server comprises the steps of: generating one or more chunks from an input image; generating chunk information of the input image including a list of the generated chunks; performing an identity check among chunks on the basis of the chunk information of the input image and chunk information stored in the contents server; and updating the stored chunk information according to a result of the identity check, wherein the chunks are a meaningful scene unit in an image.

TECHNICAL FIELD

The present disclosure relates to a method and device for managing overlapping multimedia content in a network.

BACKGROUND ART

As for dramas, movies, and sports images, many clipped images, such as their trailers or highlight images related to their original images exist on the Internet. When it comes to uploading various images on the Internet sites, such as Facebook, Youtube, etc., a lot of users often edit a single original image and then upload the results. As such, the images edited from the same original image and uploaded by the user are provided by a plurality of contents providers, and thus a lot of similar images are provided in a network. In the meantime, analysis of patterns of users using image content in a network shows that it is more likely for the users to watch drama series, teaser trailers, sports game highlights with a lot of similar images in similar time zones.

Many images provided in a network are determined as the same images to the naked eye of a human, but the images often have different binary data in reality. As such, images looking the same to the naked eye but having different binary data are called “logically identical” images.

The reasons why the logically identical images are generated are as follows:

Firstly, the existing image encoding scheme is designed to maximize compression ratio in an image. Secondly, even if multiple uses upload the same images, operators such as Facebook, Youtube may perform self encoding on the images uploaded by the users depending on their own circumstances, and repetition of the encoding may cause the logically identical images. Thirdly, in a case of performing image encoding according to a multi-thread based image encoding scheme for loss compression, a phenomenon of thread synchronization between threads that encode respective frames that make up an image occurs, and the thread synchronization phenomenon causes the logically identical images because even the same images for encoding have different binary information. Fourthly, in a case of an encoding scheme to insert an I frame, a reference for image decoding, at regular intervals, in image editing, the position of the I frame in an original image and the position of the I frame in an edited image may be different. If the position of the I frame is changed, information about other frames that refer to the I frame is changed and binary information of the respective images is changed, thereby causing logically identical images.

DISCLOSURE

Technical Problem

The present disclosure provides a method and apparatus for reducing network load by deleting logically identical images.

The present disclosure also provides a method and apparatus, by which image data is divided into chunks, chunk information of an image is generated, and chunk information is updated by reflecting chunk information of a new image.

The present disclosure also provides a method and apparatus, by which a content server provides image data to a client via a cache server located in an Internet service provider.

The present disclosure also provides a method and apparatus, by which a cache server provides an image requested by a client through combination to the client.

Technical Solution

In accordance with an aspect of the present disclosure, a method for managing multimedia data in a content server is provided. The method includes generating at least one chunk from an input image, generating chunk information of the input image including a list of the generated chunks, performing sameness check among chunks based on the chunk information of the input image and chunk information stored in the content server, and updating the stored chunk information based on the result of the sameness check, wherein the chunk is a meaningful scene unit in an image.

In accordance with another aspect of the present disclosure, a content server for managing multimedia data is provided. The content server includes an encoder for generating at least one chunk from an input image, and a controller for generating chunk information of the input image including a list of the generated chunks, performing sameness check among chunks based on the chunk information of the input image and chunk information stored in the content server, and updating the stored chunk information based on the result of the sameness check, wherein the chunk is a meaningful scene unit in an image.

MODE FOR INVENTION

Descriptions of some well-known technologies that possibly obscure the invention will be omitted, if necessary. Embodiments of the present disclosure will now be described with reference to accompanying drawings.

While the embodiments as will be described below are separated for convenience of explanation, two or more embodiments may be performed by being combined within a non-conflicting range.

Further, terms, as will be mentioned later, are defined by taking functionalities of embodiments of the present disclosure into account, but may vary depending on certain practices or intentions of users or operators. Accordingly, the definition of the terms should be made based on the descriptions throughout this specification.

It will be understood that, although the terms first, second, third, etc., may be used herein to describe various elements, components, regions, layers and/or sections, these elements, components, regions, layers and/or sections should not be limited by these terms. These terms are only used to distinguish one element, component, region, layer or section from another region, layer or section. Thus, a first element, component, region, layer or section discussed below could be termed a second element, component, region, layer or section without departing from the teachings of the present disclosure. Descriptions shall be understood as to include any and all combinations of one or more of the associated listed items when the items are described by using the conjunctive term “˜ and/or ˜”, or the like.

The terminology as used herein is only used for describing particular embodiments and not intended to limit the present disclosure. It is to be understood that the singular forms “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

Various embodiments of the present disclosure will now be described with reference to accompanying drawings. Like numbers refer to like elements throughout. Detailed description of well-known functionalities and configurations will be omitted to avoid unnecessarily obscuring the present disclosure. Descriptions of some well-known technologies that possibly obscure the disclosure will be omitted, if necessary.

Prior to a detailed description of the present disclosure, basic concepts of embodiments of the present disclosure will be briefly described.

The present disclosure is to reduce network traffic caused by the existence of a lot of logically overlapping images in the network. In the present disclosure, a content server divides a new image input to the content server into chunks, each being a meaningful scene unit, and generates chunk information about the chunks that constitute the image. It then compares the information of the chunks with chunk information already stored in the content server (called “recipe”) to perform sameness check among chunks.

If the result of checking reveals that the generated chunk is similar to the stored chunk, previous chunk information included in the recipe may be updated to be the latest chunk information about the corresponding chunk, and the corresponding chunk may be stored. However, in some instances, for the similar chunk, the chunk information may not be updated and the chunk may not be stored.

If the result of checking reveals that the generated chunk is the same as the stored chunk, previous chunk information included in the recipe is updated to the latest chunk information. Since the generated chunk is the same as the already stored chunk, the newly generated chunk is deleted. Although the same chunks as the stored chunks are deleted, corresponding chunk information of the chunks is updated to the latest one.

If the result of checking reveals that the generated chunk is a new chunk not stored, the new chunk is stored and the chunk information of the corresponding chunk is included in the recipe.

In the present disclosure, a client (or terminal) may be provided through a cash server in a network. The cash server requests and receives chunk information about a requested image from the content server, determines a chunk that is not stored in the cash server using the received cash information, and requests and receives the chunk not stored and header information of the image from the content server. After that, using the chunk received from the content server and the chunk stored in the cache server, the corresponding image is composed and the result is sent to the client.

Although embodiments of the present disclosure are described based on image data, what is provided in the present disclosure may be applied not only to the image data but also to multimedia data that could be logically overlapping content in the network.

Embodiments of the present disclosure will now be described in detail.

FIG. 1shows an image content transmission and reception system, according to embodiments of the present disclosure.

A content server101of the present disclosure may compose binary data of logically identical images to be the same when encoding image data, and delete a logically overlapping image. Specifically, configuration and operation of the content server101will be described later in connection withFIG. 2.

Referring toFIG. 1, (A) is an occasion when the content server101and a normal terminal105without its own cache provide image content through an image distribution system102in a typical network. (B) is an occasion when the content server101provides image content to a normal terminal106without its own cache through a cache server103. (C) is an occasion when the content server101provides image content to a terminal107with its own cache.

In the case of (A), the content server101deletes a logically overlapping image. Accordingly, using the content server101as proposed in the present disclosure reduces traffic of image data in the network as compared to an occasion when a content server using an existing scheme is used to provide image content through the image distribution system102.

In the case of (B), to provide an image to the terminal106, the cache server103in the present disclosure requests and receives chunks that are not owned by the cache server103among parts that constitute the image, and combines the received chunks and chunks it has. Accordingly, through the operation of the content server101and the operation of the cache server103, traffic in the network may be reduced. For reference, the term “chunk” is a unit into which an image is divided in the present disclosure, which will be described below in connection withFIG. 2.

In the case of (C), since the terminal107includes a cache that operates in the same or similar way to that of the cache server103, traffic of the image data received by the terminal107may be reduced.

FIG. 2is a block diagram of the content server101, according to an embodiment of the present disclosure.

The content sever101includes an encoder201, a controller203, and a storage205.

The encoder201encodes an image uploaded to the content server101and divides the image. For reference, in an embodiment of the present disclosure, the encoder201may apply encoding of a cache-friendly encoding library scheme modified from an H.264 encoding scheme to make logically identical images include the same binary data.

In the present disclosure, an image to be divided may be divided into scene units having particular meanings. For this, for example, a certain function such as a scene-detection function may be used to divide the image into meaningful scene units. For example, as for a singer's concert image, a scene of the singer singing, a scene of audience, etc., may be examples of meaningful scenes. The phrase “meaningful scene unit” as herein used will be referred to as the “chunk”. Dividing an image into chunks in the present disclosure is distinguished from the existing encoding scheme that divides an image into predetermined time units or predetermined size units. The chunk is configured to include at least one group of picture (GOP), and these chunks may be combined into an image.

The controller203generates chunk information about the generated chunk, and store the chunks and chunk information in the storage205.

The chunk information will now be described.

The chunk information includes perceptual hash (phash) values for the respective chunks. The phash is a 64-bit long value, and used as an identifier (ID) of the chunk. For reference, phash is a value generated based on a DCT coefficient of a still image, and checking similarity among phash values may be used to check similarity of the chunks. Specifically, a hamming distance between the phash values may be calculated to determine whether the phash values are similar to each other. For example, phash values with the hamming distance equal to or less than 4 may be determined to be similar. Accordingly, if measurements of hamming distance between phash values determine that the phash values are similar, it may be determined that the chunks corresponding to the phash values are similar.

The chunk information may include not only the phash value but also at least one of resolution, frame rate, frame length, etc., of the chunk. In the present disclosure, the chunk information of the chunk may be used in determining sameness or similarity between chunks.

FIG. 3shows an example of chunk information about chunks, according to the present disclosure.

Assuming thatFIG. 3is chunk information of image A, the first column represents the position of frame at which the chunk begins and the second column represents the value of pHash, i.e., the ID of the chunk.

InFIG. 3, the image A includes 6 chunks from (a1) to (a6). It represents that chunk (a1) begins at frame 1 and includes frames up to frame 285, and the ID of the chunk (a1) is “(m1)89000061174B2901”. Likewise, it represents that chunk (a2) begins at frame 286 and includes frames up to frame 378, and the ID of the chunk (a2) is “(m2)CDDD696527965211”. In the same way, the position at which the corresponding chunk begins and the chunk ID are represented for chunks (a3) to (a6). The controller203stores the chunk information created in this way in the storage205. Information of the chunk information stored in the storage205will be referred to as a “recipe”. That is, the recipe includes information about chunks that constitute the image A.

Furthermore, the controller203may generate chunk information about chunks generated by the encoder201according to the aforementioned method, update chunk information of the recipe by checking sameness among chunks based on the chunk information included in the recipe stored in the storage205, and store necessary chunks in the storage205. Upon reception of a request for a particular image from a client, it may combine chunks to generate a requested image and provide the image to the client. In the present disclosure, however, besides the generated image being provided directly to the client, the image may be provided to the client via a cache server located in an Internet Service Provider (ISP). An embodiment of providing an image through a cache server to a client will be described later in connection withFIGS. 5 and 6.

As an example of the operation as described above, after the chunk information about the image A is stored in the storage205, if an image A′ logically identical to the image A is uploaded to the content server101, the encoder201may divide the image A′ into at least one chunk. The controller203may generate chunk information of the image A′, compare the chunk information of the image A′ with the stored chunk information (i.e., recipe) of the image A to check sameness between chunks of the image A and chunks of the image A′, update the recipe stored in the storage205according to the checking result and store required chunks.

FIG. 4shows an example of updating a recipe by examining sameness on chunks, according to an embodiment of the present disclosure.

Reference numeral401represents chunk information of the image A shown inFIG. 3, reference numeral403represents chunk information of the image A′,405represents chunk information updated according to a result of checking sameness between the chunks of the image A′ and the chunks already stored in the storage based on the chunk information of the image A′ and the chunk information of the image A.

The chunk information401of the image A includes m1, m2, m3, m4, m5, m6, and the chunk information403of the image A′ includes m1′, m2, m6, m7. For reference, it is indicated that m1′ included in the chunk information403of the image A′ is similar to m1 included in the chunk information402of the image A′. As described above, if a hamming distance between two chunk ID values is equal to or less than a threshold, e.g.,4, it may be determined that the two are similar.

Comparing the ID values of the two chunks, m1′ included in the chunk information403of the image A′ is similar to m1 included in the chunk information401of the image A, and m2, m6 included in the chunk information403of the image A′ also exist in the chunk information401of the image A. In the meantime, m7 included in the chunk information403of the image A′ is a chunk ID of a new chunk not included in the chunk information401of the image A.

In this situation, a recipe405, the chunk information stored in the storage205, may be updated as follows:

In the first case where a chunk generated by the encoder201is similar to the chunk stored in the storage205, the controller203may update chunk information of the chunk included in the recipe to latest chunk information and store the generated chunk in the storage205. In an example ofFIG. 4, since m1′ is similar to m1, it may be seen that m has been updated to m′ in the recipe405. Furthermore, a chunk with the value of m′ is stored in the storage205.

AlthoughFIG. 4shows that the recipe405is updated with m′, chunk information for a similar chunk may not be updated but the previously stored chunk information may remain in some instances. Similar chunk IDs means, for example, two chunk images are very similar, such as chunks m1 and m1′ obtained by adding a subtitle to m1. Accordingly, in some cases, m1 may not have to be updated to m1′ in the recipe405.

In the second case where a chunk generated by the encoder201is the same as the chunk stored in the storage205, the controller203updates previous chunk information included in the recipe to the latest chunk information. However, since the chunk generated by the encoder201is the same as one already stored in the storage205, the chunk generated by the encoder201is deleted. In other words, the same chunks as those stored are deleted, but the chunk information of the corresponding chunks is updated to the latest one.

In the example, m2 and m6 included in the chunk information403of the image A′ are included in the chunk information401of the image A. In this case, m2 and m6 are updated in the recipe405, and newly generated chunks corresponding to the newly generated m2 and m6 are deleted in the encoder201.

In the third case where a chunk generated by the encoder201is a new chunk not stored in the storage205, the new chunk is stored in the storage205and chunk information of the chunk is newly included in the recipe.

In the example, since m7 included in the chunk information403of the image A′ is an ID of a new chunk not included in the chunk information401of the image A, m7 is newly included in the recipe405and a chunk corresponding to m7 is stored in the storage205. For reference, a frame start position of m7 may be determined using frame length from the frame start position1347of m6 to the chunk included in the chunk information. InFIG. 4, the frame start position of m7 is shown to be1501.

InFIG. 4, it was described that sameness of chunks are checked by comparison of chunk IDs included in the chunk information. However, the sameness check among chunks may be performed by comparing not only chunk IDs, but also other information, e.g., at least one of resolutions, frame rates, frame lengths.

FIG. 5shows a procedure of obtaining image data by a client from a content server via a cache server, according to an embodiment of the present disclosure.

In the embodiment ofFIG. 5, a client does not obtain an image directly from a content server but via a cache server503. The cache server503is located in e.g., an Internet Service Provider (ISP) and operates with a content server501according to a preset protocol, thereby reducing overlapping traffic between the content server501and the cache server503and enhancing quality of experience (QoE) of a corresponding image by transmitting image data more quickly to the client505.

In step511, the client505transmits an image request message to the cache server503to request the cache server503to provide a desired image. In step513, the cache server503receives the image request message and requests the content server501not for the corresponding image but for chunk information about the corresponding image. This is for the cache server503to determine which chunks constitute the corresponding image. Upon reception of the request for the chunk information, the content server501transmits the chunk information about the corresponding image to the cache server, in step515. How the content server501composes and updates the chunk information was described in connection withFIGS. 3 and 4.

In step517, the cache server503analyzes the received chunk information to check whether the chunks constituting the corresponding image are included in the cache server503. Such checking may be performed in the same way as the checking of sameness among chunks in connection withFIG. 4. In step519, the cache server503requests chunks that are not owned by the cache server503and header of the corresponding image based on the checking result of step517. In step521, the content server509sends the requested chunk and header information of the corresponding image to the cache server503. In step523, the cache server503composes an image by combining the received chunks and chunks it has, and the header of the corresponding image. The received chunks are stored in the cache server503to be used when other users request the image. After that, in step525, the composed image is transmitted to the client505.

For reference, in relation to configuration of the cache server503, although not shown inFIG. 5for convenience sake, a controller may control the entire operation, a transceiver may transmit or receive signals to or from an external entity, and a cache storage may store chunks and chunk information.

FIG. 6shows an example of an image providing scheme as described inFIG. 5, according to an embodiment of the present disclosure.

Steps601to617ofFIG. 6proceed in the same order of steps511to525ofFIG. 5. It, however, shows an example of providing an image to the client505with examples of actual chunks added to the image.

In step601, the client505requests the cache server503to provide a desired image, and in step603, the cache server503requests the content server501for chunk information about the image. In step605, the content server501transmits chunk information about the image stored in its storage205(i.e., a recipe) to the cache server503.

In step607, the cache server503analyzes the received chunk information to check whether the chunks constituting the corresponding image are included in the cache storage504in the cache server503. Assume herein that the image is comprised of a chunk 1 and a chunk 2.FIG. 6shows that chunk 42, chunk 2, and chunk 45 are included in the cache storage504. The cache storage504includes chunk 2 but no chunk 1 among chunks required to compose the image. Accordingly, the cache server503requests header and chunk 1 of the image from the content server501in steps608and609. For reference, the header is information corresponding to Moove part. Upon reception of the request, the content server501transmitted the header and chunk 1 to the cache server503in steps610and611. The cache server503combines the header, chunk 1, chunk 2 received in step613to generate a combined image615, and transmits the combined image to the client in step617. The cache server503stores the received chunk 1 in the cache storage504.

FIG. 7is a flowchart illustrating operation of a content server, according to an embodiment of the present disclosure.

In step701, a content server divides a newly input image into chunks, and in step703, generates chunk information of the image. In step705, checking sameness among chunks is performed by comparing already stored chunk information (i.e., recipe) and the chunk information generated in step703. In step705, subsequent operation is performed according to the sameness checking result. Detailed operation is the same as what is described in connection withFIG. 2.

Specifically, in the first case where the generated chunk is similar to the stored chunk, previous chunk information included in the recipe may be updated to be the latest chunk information about the corresponding chunk, and the corresponding chunk may be stored. In some instances, however, the chunk information of a similar chunk may not be updated but the previously stored chunk information may remain the same.

In the second case where the generated chunk is the same as the stored chunk, previous chunk information included in the recipe is updated to the latest chunk information. Since the generated chunk is the same as the already stored chunk, the newly generated chunk is deleted from the encoder201. Although the same chunks as the stored chunks are deleted, corresponding chunk information about the chunks is updated to the latest one.

In the third case where the generated chunk is a new chunk not stored, the new chunk is stored and the chunk information about the chunk is newly included in the recipe.

FIG. 8is a flowchart illustrating operation of a cache server, according to an embodiment of the present disclosure.

In step801, upon reception of a request for an image from the client, in step803, it requests and receives chunk information from the content server. In step805, sameness check is performed by comparing the received chunk information and chunk stored in the cache server. In step807, based on the checking result, it requests and receives chunks not stored in the cache server and header of the corresponding image from the content server. After that, in step809, it combines the stored chunks related to the image, the chunks received from the content server, and header information to compose an image, and in step811, transmits the composed image to the client.

Particular aspects of the present disclosure may be implemented as computer-readable codes in a computer-readable recording medium. The computer-readable recording medium is any data storage device that may store data readable to computer systems. For example, the computer-readable recording medium may include Read Only Memories (ROMs), Random Access Memories (RAMs), CD-ROMs, magnetic tapes, floppy discs, optical data storage devices, and carrier waves. The computer-readable recording medium may be distributed by computer systems connected over a network, and thus the computer-readable codes may be stored and executed in distributed ways. Furthermore, functional programs, codes, and code segments for achieving various embodiments of the present disclosure may be readily interpreted by skilled programmers in the art to which embodiments of the present disclosure are applied.

It will be appreciated that the embodiments of the present disclosure may be implemented in a form of hardware, software, or a combination of hardware and software. The software may be stored as program instructions or computer readable codes executable on the processor on a computer-readable medium. Examples of the computer readable recording medium include magnetic storage media (e.g., ROM, floppy disks, hard disks, etc.), and optical recording media (e.g., CD-ROMs, or DVDs). The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. This media can be read by the computer, stored in the memory, and executed by the processor. The method in accordance with an embodiment of the present disclosure may be implemented by a computer or portable terminal including a controller and a memory, and the memory may be an example of the computer readable recording medium suitable for storing a program or programs having instructions that implement the embodiments of the present disclosure.

The present disclosure may be implemented by a program having codes for embodying the apparatus and method described in claims, the program being stored in a machine (or computer) readable storage medium. The program may be electronically carried on any medium, such as communication signals transferred via wired or wireless connection, and the present disclosure suitably includes its equivalent.

The electronic device in accordance with the embodiments of the present disclosure may receive and store the program from a program provider connected thereto via cable or wirelessly. The program provider may include a memory for storing programs having instructions to perform the embodiments of the present disclosure, information necessary for the embodiments of the present disclosure, etc., a communication unit for wired/wirelessly communicating with mobile devices, and a controller for sending the program to the mobile devices on request or automatically.