Handling of video segments in a video stream

There is provided mechanisms for determining a time offset for a video segment of a video stream using metadata. The metadata comprises time information of at least one of a start time and an end time of the video segment. A method is performed by a video network node. The method comprises extracting a first video part and a second video part from the video stream. Each of the first video part and the second video part comprises a common video segment. The method comprises identifying a sequence of video frames in the first video part that represents the common video segment. The method comprises determining the time offset based on a time difference between an end-point frame of the Identify sequence of video identified sequence of video frames and the time information in the metadata.

This application is a 35 U.S.C. § 371 national phase filing of International Application No. PCT/EP2016/082847, filed Dec. 29, 2016, the disclosure of which is incorporated herein by reference in its entirety.

TECHNICAL FIELD

Embodiments presented herein relate to a method, a video network node, a computer program, and a computer program product for determining a time offset for a video segment of a video stream using metadata.

BACKGROUND

Communications systems, for example implementing functionality of a content delivery network (CDN), can be used to serve content, such as video streams, to end-users with high availability and high performance. In some scenarios, additional content, such as advertisements, are inserted at one or more places in the video stream before it is delivered to the end-users.

In general terms, advertisement insertion concerns the insertion of new advertisement segments into video streams, and advertisement replacement concerns the replacement of existing advertisement segments in video streams with new advertisement segments. Advertisement segments are commonly grouped together into consecutive sequences of advertisements, each such sequence being denoted an “advertisement break”. A television (TV) program may have a pre-roll advertisement break (comprising a sequence of advertisements before program start), any number of mid-roll advertisement breaks (each comprising a sequence of advertisements in the middle of the program), and a post-roll advertisement break (a sequence of advertisements after the end of the program). Pay TV operators usually sell advertisement slots for a certain time window. Two examples are called C3 and C7. For a C3 time window, for example, advertisements slots are sold for 3 days, and between the time the TV program was aired and until 3 days afterwards the advertisements must not be replaced. However, after the time period of 3 days, advertisement slots sold under the C3 contract may be replaced with new advertisements.

The act of inserting advertisements at the beginning and/or end of advertisement breaks, and/or replacing existing advertisements with new advertisements require accuracy in identifying the first and last frame of the advertisement break. Without this accuracy, advertisement insertion and advertisement replacement may result in a disruptive, choppy, or jagged appearance of the video stream to the viewer. To get a smooth advertisement insertion and advertisement replacement, the exact boundaries of the advertisement break within the video stream needs to be known.

TV operators have metadata regarding which advertisements where inserted to the video stream, at what start and end times each advertisement is found in the stream, and what is the duration of each advertisement. Such metadata can be stored in log files.

One mechanism for advertisement insertion and advertisement replacement could thus be to use the metadata as is, which describes approximately the start and end times of ad-breaks. However, it could be that the metadata of the log file is not well synchronized with the video stream, thus resulting in new advertisements being inserted in the middle of an existing advertisement, or replacing parts of TV programs and a prefix or a suffix of an existing advertisements with new advertisements instead of accurately replacing existing advertisements within an advertisement break with new advertisements.

Although advertisements have been mentioned as an example where a video segment (as defined by a single advertisement or an entire advertisement break) is to be replaced or removed from a video stream, there are also other examples where a video segment is to be replaced or removed from a video stream.

In view of the above, there is thus a need for an improved handling of video segments in a video stream.

SUMMARY

An object of embodiments of the invention herein is to provide mechanisms for accurately identifying a video segment in a video stream.

According to a first aspect of the invention there is presented a method for determining a time offset for a video segment of a video stream using metadata. The metadata comprises time information of at least one of a start time and an end time of the video segment. The method is performed by a video network node. The method comprises extracting a first video part and a second video part from the video stream. Each of the first video part and the second video part comprises a common video segment. The method comprises identifying a sequence of video frames in the first video part that represents the common video segment. The method comprises determining the time offset based on a time difference between an end-point frame of the identified sequence of video frames and the time information in the metadata.

According to a second aspect of the invention there is presented a video network node for determining a time offset for a video segment of a video stream using metadata. The metadata comprises time information of at least one of a start time and an end time of the video segment. The video network node comprises processing circuitry. The processing circuitry is configured to cause the video network node to extract a first video part and a second video part from the video stream. Each of the first video part and the second video part comprises a common video segment. The processing circuitry is configured to cause the video network node to identify a sequence of video frames in the first video part that represents the common video segment. The processing circuitry is configured to cause the video network node to determine the time offset based on a time difference between an end-point frame of the identified sequence of video frames and the time information in the metadata.

According to a third aspect of the invention there is a video network node for determining a time offset for a video segment of a video stream using metadata. The metadata comprises time information of at least one of a start time and an end time of the video segment. The video network node comprises processing circuitry and a storage medium. The storage medium stores instructions that, when executed by the processing circuitry, cause the video network node to perform operations, or steps. The operations, or steps, cause the video network node to extract a first video part and a second video part from the video stream. Each of the first video part and the second video part comprises a common video segment. The operations, or steps, cause the video network node to identify a sequence of video frames in the first video part that represents the common video segment. The operations, or steps, cause the video network node to determine the time offset based on a time difference between an end-point frame of the identified sequence of video frames and the time information in the metadata.

According to a fourth aspect of the invention there is presented a video network node for determining a time offset for a video segment of a video stream using metadata. The metadata comprises time information of at least one of a start time and an end time of the video segment. The video network node comprises an extract module configured to extract a first video part and a second video part from the video stream. Each of the first video part and the second video part comprises a common video segment. The video network node comprises an identify module configured to identify a sequence of video frames in the first video part that represents the common video segment. The video network node comprises a determine module configured to determine the time offset based on a time difference between an end-point frame of the identified sequence of video frames and the time information in the metadata.

According to a fifth aspect of the invention there is presented a computer program for determining a time offset for a video segment of a video stream using metadata, the computer program comprising computer program code which, when run on a video network node, causes the video network node to perform a method according to the first aspect.

According to a sixth aspect of the invention there is presented a computer program product comprising a computer program according to the fifth aspect and a computer readable storage medium on which the computer program is stored. The computer readable storage medium could be a non-transitory computer readable storage medium.

Advantageously the embodiments of the invention enable accurate identification of the video segment in the video stream. In turn, this enables efficient handling of video segments in the video stream.

Advantageously the embodiments of the invention provide an accurate identification of the first and last frames of the video segment.

Advantageously the embodiments of the invention need a comparatively small search window to accurately find the first and last frames of the video segment.

Advantageously the embodiments of the invention enable, with the use of the metadata, to identify the video segment even when the content of the video segment appears for the first time in the video stream.

Advantageously the embodiments of the invention enable accurate determination of the time offset in scenarios where the time offset is caused by transcoding, re-encoding, or other processing operations occurring before the video stream is played out at a client node.

Advantageously the embodiments of the invention enable efficient separation of the video segment from the video stream such that the video segment can be replaced or removed.

It is to be noted that any feature of the first, second, third, fourth, fifth and sixth aspects of the invention may be applied to any other aspect, wherever appropriate. Likewise, any advantage of the first aspect of the invention may equally apply to the second, third, fourth, fifth and/or sixth aspect of the invention, respectively, and vice versa. Other objectives, features and advantages of the enclosed embodiments will be apparent from the following detailed disclosure, from the attached dependent claims as well as from the drawings.

DETAILED DESCRIPTION

FIG. 1is a schematic diagram illustrating a communications system100where embodiments presented herein can be applied. The communications system100could implement the functionality of a content delivery network and comprises a video streamer node110, a video network node1200, a manipulator node130(optional), a client node140, and content databases150,160acting as video servers streaming and serving Uniform Resource Locators (URLs) of the video segments to the client node140. The video streamer node110issues metadata120(for example provided in a log file) which specifies advertisement breaks within a video stream. The metadata120could describe a unique identity for every advertisement and the approximated start time and end times of each advertisement. In this respect the start time and end times of each advertisement as given by the metadata120could differ from only by a single video frame (corresponding to a duration in time of a fraction of a second) from the true start time and end times of each advertisement, to several video frames (corresponding to a duration in time of more than a second) from the true start time and end times of each advertisement. Further, the start time and end times of each advertisement could be indicated by the insertion of cue-tones in the video stream, which indicate the exact position of the ad-breaks. However, not all video streams have cue-tones inserted.

The client node140is configured to request a manifest170from the video network node1200upon playout of the video stream. In response to the request the video network node1200returns a manipulated manifest170which contains segments of the video stream from the original Content Origin database150.

The video network node1200is configured to remove segments of old advertisements, and to insert segments of new advertisements with pointers, such as URLs, pointing to the Alternative Content Origin database160(instead of to the original advertisement segments in the Content Origin database150). The decisions of where the advertisements are, that is, the decisions of which video segments to remove and where to insert the video segments of the new advertisements, are made based on the information supplied to the video network node1200by the metadata120. For example, the metadata may be supplied by the operator in terms of starting times of the original advertisements when the video streamer node110inserts the advertisements into the video stream for the first time.

The video network node1200is configured, for example, to replace old advertisements within a recording of the video stream with new advertisements. The video network node1200relies on accurate metadata120describing where the existing advertisements are found.

The embodiments disclosed herein therefore relate to mechanisms for determining a time offset for a video segment of a video stream using metadata120. The time offset results from the start time and end times of each advertisement as given by the metadata120not being accurate. In order to obtain such mechanisms there is provided a video network node1200, a method performed by the video network node1200, a computer program product comprising code, for example in the form of a computer program, that when run on a video network node1200, causes the video network node1200to perform the method.

FIGS. 3 and 4are flowcharts illustrating embodiments of methods for determining a time offset for a video segment230′ of a video stream200using metadata120. The methods are performed by the video network node1200. The methods are advantageously provided as computer programs1420.

Reference is now made toFIG. 3illustrating a method for determining a time offset for a video segment230′ of a video stream200using metadata120as performed by the video network node1200according to an embodiment. Parallel reference is made toFIG. 2

The video network node1200obtains as input metadata120and an approximate start and/or end time of a video segment230′.FIG. 2at (a) and (b) schematically illustrates a video stream200.FIG. 2at (a) shows that metadata120points out a starting point of video segment230′. That is, the metadata120comprises time information of at least one of a start time and an end time of the video segment230′.FIG. 2at (b) illustrates the true location of the video segment230. This location differs by a time offset t0 from the location of the video segment230′ as given by the metadata120inFIG. 2at (a). The video network node1200is configured to download parts of the video stream200in order to find the exact start time and/or end time of the video segment230′ using the downloaded parts together with the metadata120.

S102: The video network node1200extracts a first video part210and a second video part220from the video stream200, each of which comprising a common video segment230,240. That is, the first video part210and the second video part220are extracted such that they both comprise a common video segment230,240representing content occurring in both the first video part210and the second video part220.

In the illustrative example ofFIG. 2the first video part210has a duration t3 and the second video part220has a duration t5, and the common video segment230,240has a duration t2 in the first video part210and a duration t4 in the second video part220. Further, the common video segment230,240starts a time offset Δt=t1 from the start of the first video part210.

S106: The video network node1200identifies a sequence of video frames in the first video part210that represents the common video segment230,240. That is, the identified sequence of video frames occurs somewhere in the first video part210and is thus a sub-part of the first video part210.

S108: The video network node1200determines the time offset t0 based on a time difference between an end-point frame of the identified sequence of video frames and the time information in the metadata.

Here, the end-point frame could be either the first frame of the identified sequence of video frames or the last frame of the identified sequence of video frames. That is, in an embodiment the end-point frame of the sequence of video frames is a first occurring frame of the sequence of video frames, and the end-point frame constitutes the beginning of the video segment. In an alternative embodiment the end-point frame of the sequence of video frames is a last occurring frame of the sequence of video frames, and the end-point frame constitutes the ending of the video segment.

The common video segment230,240could be identical to the video segment230′. Hence, in such embodiments the first video part210and the second video part220both comprise the content of the video segment (i.e., the content of the video segment230′ is identical to the content of the video segments230and240). The end-point frame of the identified sequence is thus identical to an end-point frame of the video segment230′. This is the case in the illustrative example ofFIG. 2.

However, it could be that neither the first video part210nor the second video part220comprises the video segment230′. In such scenarios it can be assumed that there is a known time difference between the sequence of video frames in the first video part210and the video segment230′ such that the video network node1200can identify an end-point frame of the video segment230′ by adding (or subtracting) this know time difference to/from the end-point frame of the identified sequence in order to determine the time offset t0.

Embodiments relating to further details of determining the time offset t0 for the video segment230′ of the video stream200using the metadata120as performed by the video network node1200will now be disclosed.

Reference is now made toFIG. 4illustrating methods for determining the time offset to for the video segment230′ of the video stream200using the metadata120as performed by the video network node1200according to further embodiments. It is assumed that steps S102, S106, S108are performed as described above with reference toFIG. 3and a thus repeated description thereof is therefore omitted.

There may be different ways to extract the first video part210and the second video part220from the video stream200. As disclosed above, the first video part210and the second video part220are extracted such that they both comprise a common video segment230,240. Further, according to the metadata120the start time and stop time of the video segment230′ is known. Hence, in scenarios where the common video segment230,240is identical to the video segment230′ first video part210and the second video part220could be selected to at least comprise content corresponding to the video segment230′. The first video part210and the second video part220could thus be extracted by downloading the video stream200from approximate start time−Δt and until approximate end time+Δt. The value of Δt is taken to be large enough to contain the maximum approximation error of the metadata. In view of the above, the value of Δt could correspond to a single video frame (corresponding to a duration in time of a fraction of a second) up to several video frames (corresponding to a duration in time of more than a second).

There may be different ways to perform the identifying in step S106. Embodiments relating thereto will now be described in turn.

As disclosed above, the metadata120comprises time information of at least one of a start time and an end time of the video segment230′. In one embodiment the metadata120comprises information of a time duration of the video segment230′.

The sequence of video frames could then in above step S106be identified such that it has a time duration equal to the time duration of the video segment.

The sequence of video frames could in step S106be identified using a similarity measure. Particularly, according to an embodiment the video network node1200is configured to perform step S106aas part of step S106in order to identify the sequence of video frames:

S106a: The video network node1200identifies, in the first video part210, a first sequence of video frames that is similar to a second sequence of video frames in the second video part220. A condition for this first sequence of video frames is that it has a time duration equal to the time duration of the video segment (as given by the metadata120).

As disclosed above, the common video segment230,240could be identical to the video segment230′. Hence, since the common video segment230,240is part of the first video part210the first sequence of video frames as identified in step S106acould be identical to the video segment230′.

However, as also disclosed above, it could be that neither the first video part210nor the second video part220comprise the video segment230′. In such scenarios the first sequence of video frames as identified in step S106acould be adjacent the video segment230′ or even further separated from the video segment230′, again assuming that there is a known time difference between the sequence of video frames in the first video part210and the video segment230′.

There could be different ways to identify first sequence of video frames in step S106a. According to an embodiment an image similarity measure is determined for all combinations (or a subset thereof) of video frames between the first video part210and the second video part220. Hence, according to an embodiment the video network node1200is configured to perform step S106aas part of step S106in order to identify the sequence of video frames:

S106b: The video network node1200determines that the first sequence of video frames (as identified in step s106a) in the first video part210is similar to the second sequence of video frames in the second video part220using an image similarity measure between video frames in the first video part210and video frames in the second video part220.

There could be different examples of image similarity measures that could be applied in the determination in step S106b. Either the image similarity measure is determined using the video frames of the first video part210and the second video part220as is, or the image similarity measure is determined using processed video frames of the first video part210and the second video part220. One way to process the video frames is to subject the video frames to similarity hashing. According to an embodiment the image similarity measure is thus determined using similarity hashes of video frames in the first video part210and similarity hashes of video frames in the second video part220. There are different ways to determine the similarity hashes (that is, to perform similarity hashing on the video frames). One type of similarity hashing is perceptual hashing, in which perceptually similar images obtain similar hash values with small distance between them. In general terms, perceptual hashing is the use of an algorithm that produces a snippet, or fingerprint, of various forms of multimedia. Perceptual hash functions are analogous if features are similar, whereas cryptographic hashing relies on the avalanche effect of a small change in input value creating a drastic change in output value. Further aspects of the similarity hashing will be described below with reference toFIG. 5.

FIG. 5is a schematic illustration of similarity hashing according to an embodiment. Input as defined by the first video part210and the second video part220are decoded by a decoder510(possibly using down-sampling as in step S104to reduce the frame rate) to produce respective sequences of frames520a,520b(denoted Frames1 and Frames2 inFIG. 5). The video frames520a,520bare then subjected to similarity hashing530, producing respective image hashes540a,540b(denoted Hashes1 and Hashes2 inFIG. 5). Each frame is thus represented by its own image hash.

Every image hash of a frame of the first video part210could be compared with every image hash of a frame of the second video part220. Alternatively, only a selected subset of the image hashes of the first video part210are compared to the same selected subset of image hashes of the second video part220. The higher the similarity measure, the more similar two frames are. Denote by S(i,j) the image similarity score between the i:th frame of the first video part210and the j:th frame of the second video part220. S(i,j) is determined by comparing the image hash of frame i with the image hash of frame j using an appropriate distance measure (e.g. dot-product).

FIG. 6is a schematic illustration of a similarity matrix600according to an embodiment.FIG. 6shows the similarity matrix600which holds at position (i,j) the similarity score S(i,j). In the illustrative example ofFIG. 6, darker entries in the similarity matrix600represent higher similarity score and lighter entries in the similarity matrix600represent lower similarity score. The maximum entry per row in the similarity matrix600can be stored in a first vector630afor the first video part210and the maximum entry per column in the similarity matrix600can be stored in a second vector630bfor the second video part220. The similarity matrix600can be interpreted as a heat-map. A search can be made for the diagonal610in the similarity matrix600with the maximum similarity score. The position of this diagonal610yields the time value of step S106c(by multiplying the number frames skipped from the main diagonal of the similarity matrix600in order to reach the diagonal610with the frame rate of the first video part210). Further aspects of searching for the diagonal610in the similarity matrix600will be disclosed below with reference toFIG. 11.

The image similarity measure is maximized when the first sequence of video frames and the second sequence of video frames match each other. Hence, according to an embodiment the video network node1200is configured to perform step S106cas part of step S106:

S106c: The video network node1200determines, in relation to a first occurring frame of the first video part210, a time value that maximizes the image similarity measure. The time offset t0 is then determined based on the time value.

If the common video segment230,240is identical to the video segment230′, then the time offset t0 is identical to the time value determined in step S106c. Otherwise, the known time difference between the sequence of video frames in the first video part210and the video segment230′ needs to be added to the time value determined in step S106cto yield the time offset t0.

The image similarity measure could in step S106bbe determined to comprise a sequence of image similarity values. It could be that the sequence of image similarity values comprises isolated high image similarity values. Such isolated high image similarity values could be removed from the image similarity measure when determining the time value in step S106c. That is, isolated high values620in the similarity matrix600could be removed before searching for the diagonal610in order to reduce the possibility of false positives. Thus, elements representing isolated high image similarity values could be removed from the matrix when determining the time value. This enables isolated high image similarity values to be removed from the image similarity measure.

The similarity matrix600does not necessarily need to be a square matrix; it will be a rectangular (non-square) matrix in case the first video part210and the second video part220do not result in the same number of image hashes (for example by the first video part210and the second video part220not containing the same number of frames).

In order to reduce the execution time of at least above steps S106and S108the first video part210and/or the second video part220could be down-sampled before steps S106and S108are performed. Hence, according to an embodiment, the video network node1200is configured to perform step S104before steps S106and S108:

S104: The video network node1200down-samples at least one of the first video part210and the second video part220before identifying the sequence of video frames in step S106.

Down-sampling generally refers to reducing the frame rate of the first video part210and/or the second video part220, such as using only every k:th frame, where k>1 is an integer, or any other subset of frames. However, this does not exclude that, additionally or alternatively, the resolution of the individual frames could be reduced.

An approximation of the time offset t0 could then be found using the thus down-sampled at least one of the first video part210and the second video part220. Hence, steps S104, S106, and S108could be iteratively performed at least two times. That is, step S106of identifying the sequence of video frames could be repeated for a new first video part and a new second video part. The new first video part and the new second video part are determined based on the sequence of video frames identified using the down-sampled at least one of the first video part and the second video part. For example, the new first video part and the new second video part could selected based on the time value determined in step s106cthat maximizes the image similarity measure. That is, a first approximation of the time offset t0 could be found using a down-sampled first video part210and a down-sampled second video part220in an initial search window, and a second, refined, approximation of the time offset t0 could be found using a down-sampled first video part210and a down-sampled second video part220in a refined search window, where the refined search window is selected based on the time value determined in step s106cthat maximizes the image similarity measure in the initial search window.

There could be different actions for the video network node1200to perform upon having determined the time offset t0 in step S108.

According to some aspects the video network node1200removes at least part of the video segment230′, for example to replace it with a new video segment. Hence, according to an embodiment, the video network node1200is configured to perform step S110a:

S110a: The video network node1200removes at least part of the video segment230′ from the video stream200using the end-point frame of the identified sequence of video frames as reference.

It could be that the video network node1200removes the entire video segment230′, or even that the video network node1200removes more than just the video segment230′, such as the video segment230′ and an adjacent video segment or the video segment230′ and another video segment separated from the video segment230′ by a known time difference. This could be in a case where the video segment230′ is a first video segment of a composite video segment, and, for example, where the first video part210comprises the composite video segment. The video network node1200could, for example, be configured to analyze the manifest170for the video stream200that the client node140requests, and to remove only the video segment corresponding to an advertisement break, thus allowing the replacement of the one or more of the advertisements of the advertisement break with a video segment corresponding to one or more new advertisements in a precise, frame-accurate manner, even when the metadata120is inaccurate.

According to some aspects the video network node1200does not perform any manipulation of the video stream200, such as removal or replacement of the video segment230′, but instead informs the manifest manipulator node130of the determined time offset t0 (for the manifest manipulator130to perform such manipulation). Hence, according to an embodiment, the video network node1200is configured to perform step S110b:

S110b: The video network node1200provides information of the time offset t0 to a manifest manipulator node130.

Further aspects of determining the time offset t0 for the video segment230′ of the video stream200using the metadata120as performed by the video network node1200and applicable to any of the above embodiments will now be described.

FIG. 7gives an illustrative example of metadata120. In the illustrative example metadata inFIG. 7there are 3 advertisement breaks, denoted Ad-break1, Ad-break2, and Ad-break3. Ad-break1 starts with advertisement Ad-3801and ends with advertisement Ad-3807; ad-break2 starts with advertisement Ad-3805and ends with advertisement Ad-3811; ad-break3 starts with advertisement Ad-3809and ends with advertisement ad-3810. As can be seen inFIG. 7, Ad-break2 comprises a segment denoted Ad-3805that occurs also in Ad-break1. Ad-3805in Ad-break2 is adjacent Ad-3808which does not occur in Ad-break1. By using embodiments disclosed herein pairs of ad-breaks could be found such that the first advertisement of the first ad-break appears somewhere within the second ad-break. For example, Ad-3805is the first advertisement in ad-break2 and it appears somewhere within ad-break1 (as its fifth advertisement), so the pair (ad-break2, ad-break1) has this property that the first advertisement of the first ad-break in the pair appears somewhere within the second ad-break of the pair. Also, Ad-3809appears as the first advertisement in ad-break3 and somewhere within ad-break2 (it is the third advertisement in ad-break2) so (ad-break3, ad-break2) is also a pair of advertisement breaks which has this property that the first advertisement of the first ad-break in the pair appears somewhere within the second ad-break of the pair. Hence, the exact start time and/or end time for Ad-3805in Ad-break2 (or Ad-break1) could be found using embodiments disclosed herein, and similar for Ad-3810. This is illustrated inFIG. 8.

FIG. 8is a schematic diagram illustrating a part100′ of the communications system inFIG. 1.FIG. 8schematically illustrates a video network node1200taking as input the metadata120(only part of the metadata ofFIG. 7is shown) and the video stream200from the video streamer node110as input and produces as output to a database810an accurate start time and end time of Ad-break2. In the database data representing the identifier of Ad-break2, the start time of Ad-break2 as given by the metadata, the end time of Ad-break2 as given by the metadata, the determined accurate start time of Ad-break2 as determined by the video network node1200, and the determined accurate end time of Ad-break2 as determined by the video network node1200is stored.

FIG. 9is a flowchart of a particular embodiment for determining the time offset t0 for the video segment230′ of the video stream200using the metadata120as performed by the video network node1200based on at least some of the above disclosed embodiments.

S201: The video network node1200receives a request from a client node140to playout the video stream200starting at time t.

S202: The video network node1200checks if the time t is close to an advertisement break. If no, step S203is entered, and if yes, step S204is entered.

S203: The video network node1200enables playout of the requested video stream200starting at time t at the client node140.

S204: The video network node1200checks if t is already stored in a database of fixed times (Already-Fixed-Times-DB). If no, step S205is entered, and if yes, step S207is entered.

S205: The video network node1200determines an initial start time t′ from the time t and Δt (see above for a definition of Δt).

S206: The video network node1200determines the exact start and end time of the advertisement break. The variable t′ is fixed to represent the exact start time of the advertisement break and stored in Already-Fixed-Times-DB together with t.

S207: The video network node1200retrieves the exact start time t′ from the Already-Fixed-Times-DB using t.

S208: The video network node1200enables playout of the requested video stream200from time t to time t′ at the client node140.

S209: The video network node1200replaces the original advertisement with a new advertisement to be played out at the client node140starting at time t′.

FIG. 10is a flowchart of a particular embodiment for determining the time difference based on at least some of the above disclosed embodiments.

S301: The video network node1200extracts a first video part (denoted video1) and a second video part (denoted video2) from the video stream, each of which comprising a common video segment230,240.

S302: The video network node1200checks if the first video part is shorter than the second video part. If yes, step S303is entered, and else step S304is entered.

S303: The video network node1200replaces the annotation of the first video part and the second video part with each other such that the first video part is longer than the second video part.

S304: The video network node1200identifies the first seconds, Y_Preff, of the first video part and denotes this part of the first video part as Prefix1.

S305: The video network node1200searches for Prefix1 in the second video part using an image similarity measure, e.g., as described with reference toFIG. 6.

S306: The video network node1200checks if a matching part in the second video part is found. If yes, step S307is entered, and if no, step S308is entered.

S307: The video network node1200outputs the time value that maximizes the image similarity measure in step S305.

S308: The video network node1200identifies the last seconds, Y_Suff, of the first video part and denotes this part of the first video part as Suffix1.

S309: The video network node1200searches for Suffix1 in the second video part using an image similarity measure, e.g., as described with reference toFIG. 6.

S310: The video network node1200outputs the time value that maximizes the image similarity measure in step S309.

FIG. 11is a flowchart of an embodiment for searching for the diagonal610in the similarity matrix600.

Let X represent the expected number of frames of the video segment230′. Further, assume that the video segment230′ has a time duration d as given by the metadata120. Further, let r represent the frame rate. That is, the first video part210and the second video part220are sampled to have a frame rate r. Then X=r·d. The video segment230′ is expected to represent a common video segment230,240with a length of X frames in both the first video part210and the second video part220.

S401: The video network node1200searches the first vector630for the next sequence consecutive entries of (approximately) length X of high similarities (i.e., a sequence of length X whose total similarity score is above a threshold).

S402: The video network node1200searches for a diagonal610starting at the row indicated by the first entry in the sequence found in step S401.

S404: The video network node1200determines that the video segment230′ was not found, and hence that no advertisement break was found. Step S401is entered once again.

S405: The video network node1200determines that the video segment230′ was found, and hence that an advertisement break was found.

S406: The video network node1200outputs the start and stop times of the video segment230′.

Although some of the examples presented herein relate to advertisements have been mentioned as an example where a video segment (as defined by a single advertisement or an entire advertisement break) is to be replaced or removed from a video stream, the herein disclosed embodiments are not limited to handling of advertisements; rather the herein disclosed embodiments are applicable to any examples where a particular video segment is to be accurately identified in a video stream.

FIG. 12schematically illustrates, in terms of a number of functional units, the components of a video network node1200according to an embodiment. Processing circuitry1210is provided using any combination of one or more of a suitable central processing unit (CPU), multiprocessor, microcontroller, digital signal processor (DSP), etc., capable of executing software instructions stored in a computer program product1410(as inFIG. 14), e.g. in the form of a storage medium1230. The processing circuitry1210may further be provided as at least one application specific integrated circuit (ASIC), or field programmable gate array (FPGA).

Particularly, the processing circuitry1210is configured to cause the video network node1200to perform a set of operations, or steps, S102-S110b, S201-S209, S301-S310, S401-S406, as disclosed above. For example, the storage medium1230may store the set of operations, and the processing circuitry1210may be configured to retrieve the set of operations from the storage medium1230to cause the video network node1200to perform the set of operations. The set of operations may be provided as a set of executable instructions.

Thus the processing circuitry1210is thereby arranged to execute methods as disclosed herein. The storage medium1230may also comprise persistent storage, which, for example, can be any single one or combination of magnetic memory, optical memory, solid state memory or even remotely mounted memory. The video network node1200may further comprise a communications interface1220at least configured for communications with other entities and devices. As such the communications interface1220may comprise one or more transmitters and receivers, comprising analogue and digital components. The processing circuitry1210controls the general operation of the video network node1200e.g. by sending data and control signals to the communications interface1220and the storage medium1230, by receiving data and reports from the communications interface1220, and by retrieving data and instructions from the storage medium1230. Other components, as well as the related functionality, of the video network node1200are omitted in order not to obscure the concepts presented herein.

FIG. 13schematically illustrates, in terms of a number of functional modules, the components of a video network node1200according to an embodiment. The video network node1200ofFIG. 13comprises a number of functional modules; an extract module1210aconfigured to perform step S102, an identify module1210cconfigured to perform step S106, and a determine module1210iconfigured to perform step S108. The video network node1200ofFIG. 13may further comprise a number of optional functional modules, such as any of a down-sample module1210bconfigured to perform step S104, an identify module1210dconfigured to perform step S106a, a determine module1210econfigured to perform step S106b, a determine module1210fconfigured to perform step S106c, a remove module1210hconfigured to perform step S110a, and a provide module1210iconfigured to perform step S110b. In general terms, each functional module1210a-1210imay in one embodiment be implemented only in hardware and in another embodiment with the help of software, i.e., the latter embodiment having computer program instructions stored on the storage medium1230which when run on the processing circuitry1210makes the video network node1200perform the corresponding steps mentioned above in conjunction withFIG. 13. It should also be mentioned that even though the modules correspond to parts of a computer program, they do not need to be separate modules therein, but the way in which they are implemented in software is dependent on the programming language used. Preferably, one or more or all functional modules1210a-1210imay be implemented by the processing circuitry1210, possibly in cooperation with the communications interface1220and/or the storage medium1230. The processing circuitry1210may thus be configured to from the storage medium1230fetch instructions as provided by a functional module1210a-1210iand to execute these instructions, thereby performing any steps as disclosed herein.

The video network node1200may be provided as a standalone device or as a part of at least one further device. For example, the video network node1200may be provided in the manifest manipulator node130. Alternatively, functionality of the video network node1200may be distributed between at least two devices, or nodes. These at least two nodes, or devices, may either be part of the same network part or may be spread between at least two such network parts.

Thus, a first portion of the instructions performed by the video network node1200may be executed in a first device, and a second portion of the of the instructions performed by the video network node1200may be executed in a second device; the herein disclosed embodiments are not limited to any particular number of devices on which the instructions performed by the video network node1200may be executed.

Hence, the methods according to the herein disclosed embodiments are suitable to be performed by a video network node1200residing in a cloud computational environment. Therefore, although a single processing circuitry1210is illustrated inFIG. 12the processing circuitry1210may be distributed among a plurality of devices, or nodes. The same applies to the functional modules1210a-1210iofFIG. 13and the computer program1420ofFIG. 14(see below).

FIG. 14shows one example of a computer program product1410comprising computer readable storage medium1430. On this computer readable storage medium1430, a computer program1420can be stored, which computer program1420can cause the processing circuitry1210and thereto operatively coupled entities and devices, such as the communications interface1220and the storage medium1230, to execute methods according to embodiments described herein. The computer program1420and/or computer program product1410may thus provide means for performing any steps as herein disclosed.