Patent Description:
Historically web filters have been used to filter pornography on the Internet by blocking websites based on URL blacklists and lexical analysis. These techniques are becoming less effective as the Internet is becoming more dynamic. They also are ineffective at addressing pornographic videos which can be found on media sharing sites that would be considered legitimate by a blacklist filter. Lexical analysis is also ineffective as online videos often have little text associated with them that would indicated the videos contents are pornographic.

One solution to this problem could be to intercept the video stream and analyse the content of the video. However video analysis is time consuming and delaying the video while it is being analysed can be detrimental to the user experience and cause connection problems. Alternatively allowing the video to play during analysis allows the client device to download video data into its buffer much faster than standard playback. Therefore, blocking the video after analysis is completed is ineffective as the video will continue to play on the device until there is no more data in its buffer.

Document <CIT> details a transmission delay system for removing undesired content from an input media stream. The undesired content is excised, and logged and an alert is provided that an excision event has occurred.

Document <CIT> details a system for filtering pornographic movies. The filtering system includes an image fetching device and a pornographic picture discerning engine. The image fetching device receives an input of the image data stream and controls an output of the image data stream to a playing device. The image fetching device further includes a fetching unit, a duplicate unit, and an intercepting unit. The fetching unit is used to fetch the image data of the image data stream. The duplicate unit is used to copy the fetched image data as a duplicate image data, and export the duplicate image data to the pornographic picture discerning engine to perform the discerning process. When the duplicate image data is discerned by the pornographic picture discerning engine that the pornographic picture is included, the pornographic picture discerning engine transmits a control signal to the intercepting unit.

The intercepting unit immediately intercepts the rest part of image data in the image data stream.

Accordingly, there is a desire for a video stream transmission system which facilitates the analysis and filtering of a video stream from a server to a client device.

Document <CIT> describes a technique for controlling the streaming of content through a network by changing the rate at which content chunks are sent from an intermediate network device in order to cause a client to request content chunks that are encoded at a different bit rate.

It is an object of the present invention to provide a video stream transmission system which overcomes the disadvantages of the prior art, or at least provides a useful alternative.

According to a first aspect of the invention there is provided a method of transmitting a video stream relating to a video, including:.

The step of interception may occur at a proxy device between the transmission from the server to the user device. Alternatively, the step of interception may occur at the user device.

The throttling may be caused by introducing a pause into onward transmission. The pause may be introduced before transmission of every block of the video stream. The size of the block may be determined by, at least, the video size and the video length. The size of the block (t) may be calculated by the following formula: <MAT> where.

The length of the delay (d) may be determined by the video length.

During initial interception of the video stream an initial block of a specified size of the video stream may be transmitted onto the video client. The specified size of the initial block may be at least the size of the header of the video.

The step of analysing content may include analysing the content for pornographic content. The action in step (d) of the method may include blocking of the onward transmission of the video stream if the pornographic content as a result of the analysis is determined to exceed a specified threshold.

The method may further include the step of requesting a second transmission of the video from the server; wherein the analysis may be performed on the content of the video within the second transmission.

The method may further include the steps of requesting a plurality of non-contiguous chunks across the video and receiving the plurality of chunks of the video streamed from the server; wherein the content within the video is analysed by analysing the plurality of chunks.

The method may further include the steps of extracting one or more frames from each chunk of the plurality of chunks and analysing all of the extracted frames in accordance with a pornographic content recognition engine to generate a pornographic content rating for the video.

According to a further aspect of the invention there is provided a system for transmission of a video stream relating to a video, including:.

Other aspects of the invention are described within the claims.

The present invention provides a video stream transmission method and system which facilitates the analysis and filtering of the video stream during transmission from a server to a client.

The inventors have discovered that the problems existing in the prior art can be solved by intercepting the video stream and limiting the flow of the data to the requesting device while video analysis is performed on the visual data. By throttling the data the device's buffer remains only a few seconds ahead of real time playback. If video analysis returns a positive result the video can be blocked and as the device's buffer is empty the video will stop playing shortly thereafter.

In addition the invention can dynamically increase or decrease the throttle based on feedback from the video analysis system.

In <FIG>, a video stream transmission system in accordance with an embodiment of the invention is shown.

A server <NUM> is shown. The server <NUM> is configured to deliver video streams in response to requests from clients <NUM>. For example, the server <NUM> may be a web server receptive to HTTP/HTTPS requests.

A first communications apparatus <NUM> is shown. The communications apparatus <NUM> is configured to intercept video streams from the server <NUM> to a client <NUM>.

A first processor <NUM> is shown. The first processor <NUM> is configured to receive the intercepted video stream and to throttle onward transmission of the video stream to a video client <NUM>.

The video client <NUM> is executing on a user device. The video client <NUM> may be, for example, a web browser, a plugin within a web browser such as Adobe Flash Player or Microsoft Silverlight, or a dedicated video application such as Quicktime.

The user device may be a general purpose computing device such as a desktop computer, a laptop, a tablet computer, or a smartphone. Alternatively the user device may be a dedicated video player such as a network-enabled Blu-ray/DVD player, a network-enabled media player such as an Apple TV or Roku, or a Smart TV, or the user device may be a multifunction device such as a games console.

The first processor <NUM> may request that the content of the video stream is analysed by a second processor <NUM>. The requests may be made by storing the request within a database accessible to both processors <NUM> and <NUM>. The database may be collocated with one or other processors <NUM> and <NUM>, or the database may be remotely located from both processors <NUM> and <NUM>. In the latter case, the processors <NUM> and <NUM> may communicate with the database via a network or combination of networks, such as the Internet.

In one embodiment, a plurality of first processors <NUM> may communicate via the database with one or a plurality of second processors <NUM>.

The second processor <NUM> may be configured for analysing the content of the video stream either directly from the first transmission or from a requested second transmission of the video stream. Analysis of the content of the video stream is performed by static image analysis of one or more frames of the video stream.

The first <NUM> and/or second <NUM> processor may exist within the user device itself or may be deployed in a physically separate proxy device. The proxy device may be collocated with the user device, for example, at a router or wireless access point, or may be remotely located at an ISP utilised by the user device or virtually within or across a cloud-based architecture. It will be appreciated that other physical deployments can be envisaged.

A second communications apparatus <NUM> is shown. In one embodiment, the second communications apparatus <NUM> is configured to request a second transmission of the video stream from the server <NUM> for the second processor <NUM> to analyse.

The first <NUM> and second <NUM> communications apparatus may be the same physical apparatus.

The first <NUM> and second <NUM> processor may be the same physical processor.

The server <NUM>, the communications apparatuses <NUM> and <NUM>, and the video client <NUM> on the user device may communicate across a network or plurality of interconnected networks such as the Internet.

The first processor <NUM> may transmit onward the video stream to the video client <NUM> within the user device when the processor <NUM> exists within the user device, or across a communications channel, a network or plurality of interconnected networks where the first processor <NUM> is deployed in a physically separate device.

In <FIG>, a method of video stream transmission in accordance with an embodiment of the invention will be described.

In step <NUM>, a video stream transmission from a server to a video client on a user device is intercepted.

The video stream may have been requested by the video client from the server, for example, by a HTTP/HTTPS request.

In one embodiment, all requests from the user device to servers across the Internet or a network may be intercepted and the nature of the request may be determined. Requests that relate to video streams may trigger interception of data transmitted from servers in response to the requests.

In step <NUM>, the transmission is throttled for onward transmission to the video client.

The transmission may be throttled by pausing before transmitting blocks of the video. The length of the pause may be, preferably, preconfigured or calculated from one or more factors of the video such as the size and length of the video and/or the size of the blocks. The size of the blocks may be, less preferably, preconfigured or calculated from one or more factors of the video such as the size and length of the video and/or the length of the pause. For example, the pause and/or block size could be configured such that playback of the video at the video client does not slow down, but also such that the video client does not locally buffer a significant quantity of the video.

In one embodiment, before throttling of onward transmission of the video, an initial block of the video stream may be transmitted to the video client. The size of the initial block may be of a size to ensure that the entire header of the video is transmitted to the video client. Certain types of video clients may drop the connection if the entire header is not received quickly during a video stream. Transmitting an initial block may, therefore, prevent these video clients from dropping the connection. The size of the header of a video may not be uniform. Therefore, in one embodiment, the size of the initial block may be predefined to a size sufficiently large to cover, at least, most videos. Alternatively, the size of video headers tend to be correlated to video size, therefore, the size of the initial block may be dynamically calculated based upon the size of the video.

In one embodiment, before throttling the video stream, metadata relating to the video is first searched within a database of prior analysed videos. If the video has already been analysed, the process may skip directly to step <NUM> and the action may be performed on the basis of the prior analysis. The metadata may be the URI (Uniform Resource Identifier) for the video.

In step <NUM>, content of the video is analysed.

The transmission is throttled during analysis of the content of the video.

The video may be analysed for pornographic content.

The content may be extracted from the received video stream. More preferably, a request is made for a second transmission of the video. This may provide an advantage of enabling faster transmission of the video if the request is made from an apparatus with a faster connection to the server or if the request indicates that the video is to be transmitted faster - for example if a lower quality version of the video could be transmitted. The request may relate to transmission of chunks of the video. The chunks may be non-contiguous and may be from across the length of the video. Non-contiguous chuck transmission may provide an advantage by permitting assessment of the entire video quickly.

In step <NUM>, as a result of that analysis, an action is performed in relation to the onward transmission of the video stream.

For example, if the content of the video is deemed by the analysis to exceed a specific pornographic threshold, the throttling may be increased or the transmission may be blocked, and if the content is deemed by the analysis not to exceed a specific pornographic threshold, the throttling may be decreased or removed entirely.

In one embodiment, the video stream may be replaced by another video stream. For example, a video stream indicating that the content is being blocked.

In <FIG>, an embodiment of the invention will be described.

A Video Throttling (VT) Proxy Server <NUM> listens on a port for web requests to web servers accessible via the Internet <NUM> or a network from a browser <NUM> or device. The proxy server <NUM> intercepts the responses <NUM> to each request to identify whether they contain video data. If no video data is identified the response is sent to the requesting browser <NUM> or device. If the response <NUM> contains video data then the VT proxy <NUM> performs a look up of the video URI in a URI database <NUM>. If the URI is not present in the database <NUM> then it will be added and the status set to Waiting To Be Scanned. If the URI is present in the database <NUM> and the status is Scanning Complete it will check the result and take action based upon the result. If the URI is present in the database <NUM> and the status is Scanning In Progress then the VT Proxy <NUM> will start throttling the data.

The VT Proxy <NUM> sends an initial <NUM> bytes of data to the requesting browser <NUM> or device to ensure the request does not time-out. The number of initial bytes is configurable. For example, the number of initial bytes may be configured to ensure that the header for the video (including metadata information), which is typically streamed before the video content, is entirely transmitted to the browser <NUM>.

The VT Proxy <NUM> then polls the database <NUM> periodically for the video length. This information is added to the database <NUM> by a Video Rating System <NUM>. When the information is available the VT Proxy <NUM> then calculates the throttling rate which is the number of bytes to be sent after a specified pause. Preferably the specified pause is <NUM> millisecond. In one embodiment, the pause is configured as small as supported by the hardware and operating system.

The throttling rate may be calculated using the following formula: <MAT> where:.

The value of d may calculated as a percentage of the video length. This percentage is configurable.

For example, where the delay percentage is <NUM>% and the video is of length <NUM> seconds, the video size is <NUM> thousand Kilobytes, and the pause is <NUM> millisecond, there will be a delay of <NUM> seconds and the throttling rate will be <NUM> bytes.

In an alternative configuration, the throttling rate is predefined and the pause is calculated, for example, using the following formula: <MAT>.

For example, given the original example, where the throttling rate is <NUM> bytes, the pause will be <NUM> milliseconds.

Once the value of t is calculated the VT Proxy <NUM> release t bytes to the requesting browser <NUM> or device, it then checks the database <NUM> for a result and waits p milliseconds before it sends another t bytes. It repeats this process until a result is present. The VT Proxy <NUM> can modify its behaviour based on the result for example it could increase throttling, decrease throttling or terminate the connection.

Increasing the throttle effectively pauses the video until the throttle is decrease or the connection terminated. This happens because there is no data in the client's video buffer to support playback.

Decreasing the throttle will effectively allow the video to play as normal and the client's video buffer to be filled.

Terminating the connection will effectively block the video as no more data will be sent to the requesting browser <NUM> or device. As the video buffer only contains data a few seconds ahead of playback the video will stop playing shortly after the connection is terminated.

The Video Rating (VR) System <NUM> has a dedicated thread to check the database <NUM> for video URI's with the status Waiting To Be Scanned. When a video URI with the above status is identified it changes the status to Scanning in Progress and starts downloading <NUM> the video on a separate thread. If the video is greater than 480p then to improve performance the VR System <NUM> will attempt to download the video in 480p; if it is available or the lowest resolution above it. Once the video header is received it decodes the data using a video decoder and updates the database <NUM> with the video length in seconds (LengthSeconds), file size and frame rate (FR). The VR System <NUM> also uses the header information to calculate parameters for the scanning logic.

The VR System <NUM> utilises the following parameters for the scanning logic:.

From the video header the VR System <NUM> retrieves the total size of the video in seconds which is LengthSeconds.

The SSF defines the gap in seconds between each frame extraction; LengthSeconds/SSF will give the total number of frames to extract (TFE). If LengthSeconds/SSF is greater than MaxFS then the TFE for scanning will equal MaxFS or if LengthSeconds/SSF is less than MinFS then the TFE for scanning will equal MinFS. If either of these are true then a new SSF is calculated as LengthSeconds/MaxFS or LengthSeconds/MinFS.

The VR System <NUM> has two separate modes for downloading the video data; Sequential mode and Intermittent mode.

In Sequential mode the VR System <NUM> downloads the entire video from beginning to end like a video player. This is done if the video is very short in length or the video does not support range header requests. The advantage of this mode is that it produces a minimal number of corrupt frames however the disadvantage is that it takes longer to download the data as all the data is required.

In Intermittent mode the VR System <NUM> only downloads chunks <NUM> of data at various locations along the video length based on the total number of frames to extract. The advantage of this mode is that only a fraction of the data needs to be downloaded which significantly improves the performance however the disadvantage of this mode is a higher number of corrupt frames are produced.

In Intermitted mode the following formula is used to calculate the location of each frame that is to be extracted: <MAT> <MAT>.

The BSF is the number of bytes to skip between each frame that is to be downloaded.

In Intermittent mode the following formula is used to calculate the size of each data chunk to be download: <MAT>.

The Data Chunk Size is the number of bytes to be downloaded before and after each frame that is to be extracted.

The VR System <NUM> then downloads each chunk <NUM> of data using a "range" parameter in the http request. Once it has the required chunks <NUM> of data it decodes the frames from each chunk <NUM> using a video decoder. The VR System <NUM> then selects one or more frames <NUM> from each chunk <NUM> and passes it to the pornographic content recognition engine for analysis. When all the frames have been analysed the pornographic content recognition engine returns a result as to whether the video is pornographic or not. The result is written to the database <NUM> and the status for the URI is changed to Scanning Complete.

Embodiments of the invention may be deployed in a variety of deployment configurations including cloud, hybrid and on premise. It will be appreciated that embodiments of the invention may be implemented within software, hardware, or a combination of both.

An application program interface may be provided to facilitate interoperation by user/client devices or other systems with embodiments of the invention.

The cloud deployment allows the VT Proxy <NUM>, VR System <NUM> and database <NUM> to be deployed in the cloud as a Software as a Service (SaaS) solution. Client devices are configured to browse via the cloud based proxy and no client side software is required.

The hybrid deployment allows for the VT Proxy <NUM> to be deployed on the client device or within the local network while the VR System <NUM> and database <NUM> are deployed in the cloud. The client device is configured to use the local instance of the VT Proxy <NUM> which communicates with the database <NUM> hosted in the cloud. It is also possible to deploy the VR System <NUM> locally and only have the database <NUM> deployed in the cloud.

The on premise deployment allow for the VT Proxy <NUM>, VR System <NUM> and database <NUM> to be deployed on the device or within the local network.

An example of an embodiment of the invention in use will now be described.

A minor connects to the Internet on a PC protected by the invention. He browses to a popular video hosting site and searches recently uploaded video for something that could be pornographic. The traditional web filter allows him to start playing the video as the site is considered legitimate.

The VT Proxy <NUM> identifies the response contains video data. It looks up the URI in the database <NUM> and it is not present therefore it adds the URI to the database <NUM> with a status of Waiting To Be Scanned. The VT Proxy <NUM> releases an initial <NUM> bytes of video data to the browser <NUM> to maintain the connection.

The VR System <NUM> downloads the video header and writes the following details to the database <NUM>: <MAT>.

It also obtains the following information from the video header: <MAT> <MAT> <MAT>.

The VT Proxy <NUM> obtains the video length from the database <NUM> and calculates the (t) throttling rate based on a delay (d) of <NUM> seconds (which is <NUM>% of the video length): <MAT>.

The value of p is <NUM> millisecond therefore the throttling rate is <NUM> bytes. The VT Proxy <NUM> delivers <NUM> bytes to the browser <NUM> and then pauses for <NUM> millisecond and then sends another <NUM> bytes; it continues to do this until a result is present in the database <NUM>.

The video starts to play on the browser <NUM>; the beginning of the video is not pornographic. So far the user's experience is unaffected.

The VR System <NUM> extracts the defined parameters for the scanning logic: <MAT> <MAT> <MAT> <MAT>.

The VR System <NUM> calculates the total number of frames to be extracted (TFE): <MAT>.

The VR System <NUM> checks that <NUM> is less than <NUM> (MaxFS) and greater than <NUM>(MinFS) which it is so the TFE = <NUM>.

The VR System <NUM> calculates the frame size: <MAT>.

The VR System <NUM> calculates the bytes to skip per frame: <MAT>.

The following data positions are identified.

The VR System <NUM> calculates the size of data chunk to be downloaded before and after each data position: <MAT>.

The VR System <NUM> then downloads each chunk <NUM> of data using a "range" request in the http request. Once it has the required chunks <NUM> of data it decodes the frames from each chunk using a video decoder. In this case one frame <NUM> is selected from each chunk <NUM> and passed to a pornographic content recognition engine for analysis.

The pornographic recognition engine returns a result that indicates the video is pornographic. The VR System <NUM> writes this result to the database <NUM>.

The VT Proxy <NUM> detects and extracts the result from the database <NUM><NUM> seconds into the playback of the <NUM> second long video and terminates the connection. The playback continues until <NUM> seconds at which point the video stop playing as the video buffer is empty. Therefore, the user is protected from being exposed to age restrictive material.

The user refreshes his browser <NUM> and tries to play the video again. The VT Proxy <NUM> identifies the video data and looks up the URI in the database <NUM>. The URI is present with a result that indicates the video is pornographic and the VT Proxy <NUM> terminates the connection.

If the video had initially returned a result of non-pornographic the VT Proxy <NUM> would have decreased the throttle so that the video would have downloaded as normal and the user's experience would have remained unaffected.

Potential advantages of some embodiments of the present invention include that video streams can be analysed during transmission from a server to a video client without the video client dropping the video stream and/or within interruption to video playback. Further potential advantages include that the video content may be analysed based upon content rather than predefined blacklists or lexical analysis of metadata, video streams may be filtered for receipt by any device, user experience of non-pornographic video streaming may be unaffected, and video content analysis may be faster.

Claim 1:
A method of transmitting a video stream relating to a video, including:
a) intercepting a video stream from a server to a video client on a user device (<NUM>);
b) throttling onwards transmission of the video stream to the video client (<NUM>) such that a buffer for the video stream on the video client remains close to real time playback;
c) analysing content within the video of the video stream (<NUM>); and
d) performing an action in relation to the onward transmission to the video client as a result of the analysis of the content (<NUM>), wherein the action includes blocking onward transmission of the video stream;
wherein the throttling continues during the analysis of the content and ceases when the analysis is complete and wherein the analysis utilises a static image analysis of at least some frames within the video.