Controlling the transmission of a video data stream over a network to a network user device

A device and method for controlling the transmission of a video data stream over a network to a network user device involve storing video data segments in a data storage entity, receiving a query from at least one network user device, determining current parameters of the network user device, changing the video data segments stored in the data storage entity, and transmitting the changed video data segments to the network user device in response to the query. The video data segments stored in the data storage entity are changed by changing the bit rate of the video data segments by transcoding each video data segment in real-time. It becomes possible to control the transmission of a video data stream over a network to a user device, as well as enhancing speed, simplicity and cost effectiveness, with minimal losses of quality and maximum user satisfaction.

FIELD OF THE INVENTION

The present disclosure relates to delivering data; in particular, to a controller and to the way of controlling video data streaming via a network to a network user device.

BACKGROUND OF THE INVENTION

A significant part of the data transferred via mobile networks is video traffic in the streaming mode. The most modern streaming technology is dynamic adaptive streaming over HTTP. This technology is used, for example, by HTTP Live Streaming (Apple), Smooth Streaming (Microsoft), HTTP Dynamic Streaming (Adobe) and DASH standard. In this technology, the server stores several copies of a video file, each one coded with different parameters, for example, different resolutions and bit rates. The higher the bit rate is, the faster radio channel you need to transfer the video and the higher the quality of individual frames will be. The copies are split into independent segments synchronized by time. This allows you to rewind forward without loading all the segments, and also to switch between different bit rates when necessary.

From the client's point of view, video quality is perceived as good not only when its bit rate is high but also when there are no delays (also known as stallings) in playing. These delays appear when the client's input buffer becomes empty, i.e. the client is watching the video segments quicker than they are coming via the radio channel. When using a mobile network, one of the reasons for that is that the base station to which the client is connected has a limited time-and-frequency resource. This resource, in its turn, can be exhausted when it is congested by a large number of clients connected to the same base station. Because of the congestion, the transfer rate in the radio channel becomes lower for the most clients than the bit rates available on the server, leading to delays in playing.

From the prior art, we know ways to control video data flow via a network. For example, European patent 1872536 which is the closest analogue of the applied invention describes a way to control the transfer of data packages from the server to a client when the server switches between different versions of streaming, depending on the state of the client's buffer. The same multimedia contents, for example, a particular video sequence is preliminarily coded with various bit rates and stored on the server. However, this method requires storing a huge number of versions for each data flow. Moreover, even having a lot of versions does not guarantee that for every particular case the best possible version will always be available.

Hence it appears that there is a need for a new technical solution about the control over network streaming of video data to a user device which would provide high speed, simplicity, efficiency, minimum loss in quality and maximum satisfaction of the users.

SUMMARY OF THE INVENTION

The objective of the present invention is creating a controller along with a way of controlling video data transfer which would make it possible to change the amount of video data transferred over a radio channel from the base station to a client, in order to optimize the transfer.

The said problem is solved by a special method of controlling the network data stream to a user device. The method is storing video segments in a data storage, receiving a request from at least one network user device, determining the current parameters of this device, transforming the stored video segments on the base of these parameters, and sending the transformed segments to the user device as a response to the request. According to this method, the stored video segments are transformed by changing the bit rate via transcoding every segment in real time.

The technical effect obtained by the invention is a smaller average number of delays (stallings) when playing video, and smaller load on the provider's core and transit network. The effect is provided by changing the amount of video data by their segment-by-segment real-time transcoding into bit rates lower than the bit rates of the top-quality copies stored on the server.

In one of the possible embodiments, video segments are saved in the data storage when the first request for these video data has been received from a network user device.

In one of the preferable embodiments, the stage of saving video segments in the data storage includes the stage of caching these segments.

In another embodiment, the current parameters of a network user device include at least one of such parameters as the current capacity of the device channel, the state of the user's buffer and the tariff.

In another preferable embodiment, the stored video segments are transcoded at the hardware level.

In another embodiment, the stored video segments are transcoded with changing their resolution.

In another preferable embodiment, when filling up the data storage, the video segments saved in it are selected according to the requests for saved video data from the network user devices.

In another embodiment, the selection is based on the frequency of requests for particular video data from the network user devices. Videos with the highest request frequency have a priority.

In another embodiment, the selection is based on removing videos whose last received request is the oldest one.

In one more embodiment video segments additionally contain audio or/and text data.

The said problem is also solved by proposing a controller responsible for the network transfer of video data flow to a user device. The controller contains at least one data storage suitable for storing video segments, and at least one controlling block. The controlling block of the controller is able to receive a request from at least one network user device and determine the current parameters of at least one network user device. The video segments stored in the data storage can be altered by changing their bit rates via transcoding every segment in real time according to the current parameters of the user device determined by at least one controller. Besides, the controller contains a network interface which can send the altered video segments to at least one network user device.

In another embodiment, at least one controlling block contains at least one scheduler.

In another embodiment, the scheduler provides the possibility to choose a bit rate for sending video segments to the indicated user device as a response to a request from at least one user device according to the determined current parameters of the network user device.

In another embodiment, the video streaming controller is placed in a network which contains at least one user device, at least one base station, at least one gateway and at least one content server.

In another embodiment, the controller is designed in such a way that no user device, no basic station, no gateway and no content server receive any information about the presence of this controller in the network.

The said problem is also solved by a device which contains a proxy block capable of receiving requests from at least one network user device, a caching block to store video segments, and a scheduler which can determine the current parameters of at least one network user device. Besides, the device contains a transcoding block which can alter the video segments stored in the caching block via changing their bit rate and/or their resolution by means of transcoding each segment in real time according to the current parameters of a user device, and a network interface which can send the altered video segments to at least one network user device.

In one more embodiment, the proxy block can work at the network level.

These and other aspects of the invention will be clear from the subsequent description of the preferable embodiments, drawings and summary of invention. Details of one or several examples of the implementation of the invention are shown on the additional drawings and in the subsequent description. Other distinctive features, problems and advantages of the inventions will be evident from the description, drawings and claims.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1illustrates the logical structure of the network which consists of two parts. The first part is a transit network1(E-UTRAN) responsible for the radio access. It consists of the base stations4(eNodeB) and user devices3(UE, user equipment). The second part is a core network2(EPC, Evolved Packet Core) responsible for providing the required quality of the service (QpS), mobility control, authentication, billing, etc. Logical nodes of these networks use standardized protocols for communication. The controller5, according to the present invention, is placed between the first and second parts. The controller5works on so-called interface S1.

The conventional centralized placement of the proxy server with a cache in the core network decreases only load on the Internet gateway, being unable to unload the transit network. The proposed solution uses a distributed placement of the proxy server inside the transit network, making it possible to decrease the load on the transit network channels. The controller, according to the present invention, is placed on the edge of the transit network, diminishing the load on the transit channels. Such a placement is possible due to the use of a proxy block. Besides, this placement in the logical hierarchy of the network allows the operator to position the controller5at various levels of the physical network: from a base station4to the points aggregating the traffic from tens of base stations4. In addition to that, the hardware and software platform of the controller5is scalable by the number of processing modules and data storage systems, making it possible for the operators to use this invention in their networks in the best way in terms of efficiency and low costs.

In a mobile network, data are transmitted inside a tunnel. Packages whose non-restrictive formats are shown inFIG. 7are sent from the user device3to the base station4, then transmitted via a tunnel connection to the serving gateway6where they are taken out from the tunnel and sent to the core network2. Tunneling provides the required quality of service for various types of traffic from different clients by the transit network. The tunnel is controlled by the protocol S1-AP, while the transmission inside tunnel is controlled by S1-U. The proxy block acts as a transparent proxy server, i.e. the proxy function is implemented for both the tunnel itself and the connection inside it. Thus, the optimization and video traffic caching are transparent for both the user device3and content server8with the video data; and also for the base station4and the serving gateway6, without significant changes for the headers of the transmitted packages. This allows us to simplify the integration of the controller5into the existing network, as we do not need to change the configuration of the user device3, the base station4and the gateway6.

Another advantage of such a placement is a smaller delay of the transmitting channel due to the use of a proxy server. The video data are transferred by the Transmission Control Protocol (TCP) which guarantees reliable delivery. In the framework of this protocol, the content server chooses the rate to send the data to the client in accordance with the rate of the compound logical channel, i.e. the channel consisting of a radio subchannel and a subchannel which includes some parts of the transit network, the core network and Internet. These two subchannels have different characteristics, therefore the rate of the compound logical channel is basically determined by the rate of the channel with worse characteristics (which is the radio channel), affecting the speed of the TCP connection, Proxy server makes it possible to provide two different logical connections for each subchannel, with independently chosen TCP parameters. This can significantly reduce the delays in delivering the video data.

In the packet network, the users data are transmitted via channels for a stream of IF packets with a certain quality of service. The connection of a user device3with an external Internet server8is called a end-to-end bearer. It consists of several channels established between the logical nodes in the transit network1(E-UTRAN) and the core network2(EPC) by protocols like S1-U, TCP, HTTP, etc.

The set of protocols S1 consists of two parts: S1-U and S1-AP. The protocol S1-U directly provides the transfer of S1 tunnel stream via the IF network, while the protocol S1-AP provides that the channel E-RAB is established between a user device3and the serving gateway6; in particular, the tunnel S1 (a component of the channel E-RAB) is established between the base station4and the serving gateway6. S1 tunnel stream is a stream of IF packets between the base station4and the serving gateway6in which IF packates of the user device3are tunneled.

The IP tunnel of the protocol S1-U is implemented by the protocol GTP-U and also by UDP transport protocol. The end points (input/output) of the tunnel are the base station4and the serving gateway6(i.e. the added IF header contains the IF addresses of the base station4and the serving gateway6).

FIG. 2shows the scheme of interaction between the blocks of controller5. According to the present invention, the controller5contains a proxy block11, a scheduler13, a transcoding block14, a caching block12and network interfaces23which can send video data to a user device3.

The proxy block11whose functioning is illustrated inFIG. 3works at the network level, intercepting and analyzing all the packets passing through it. In other words, it consecutively expands the tunnel level (IP1) between the base station4and the serving gateway6and then analyzes the embedded packet in order to determine its type (TCP, UDP). Thus, the work at the network level allows the proxy block11to have an access to packetes not addressed to it. Neither the content server8, nor the user device3, nor the base station4, nor the serving gateway6receive any information about the interference or even existence of the proxy block11, because after the headers of the packets are processed, the proxy block adapts them in a special way imitating their response from the content server8to a user device3. UDP packets are sent without any change, but TCP packages are processed in the following way. When a request is received from at least one network user device3, the proxy block determines the current parameters of this device3. When a connection is established between the user device3and the content server8, the proxy block breaks the tunnel between the base station4and the serving gateway6. To do that, the proxy block scans all the passing packets determining their types. For UDP packets containing TCP packets embedded into them, the headers are saved in the memory with Correspondence tables: IF address+user device port and all the data of the tunnels from the base station4to the serving gateway6and back, numbers of tunnels teID such as IF+base station port, IF+serving gateway port, numbers of tunnels teID, IF+content server port, and others. The tunnel numbers are intercepted front the analysis of the SCTP traffic between the base station4and the mobility management entity (MME)9before the tunnel is established. To analyze this traffic, an additional syntactical packet analyzer is used. After the headers are saved, data are extracted out of the packets. If the data contain a request for video data from the user3to the content server8, this request is passed for the service to the scheduler13. If the scheduler replies with a refusal, the request is passed to the content server8on behalf of the user3. If the scheduler returns the requested video and the bit rates calculated for its transfer, the proxy block11forms packets out of the video data, adapting the headers on the base of the saved tables in the way to imitate the response from the content server8to the user3.

If the data contain a file to pass to the user3, it is sent to the scheduler13to be passed to the caching block and transcoded if necessary. The scheduler replies with the information about the transfer bit rate and the transcoded video. After that, the proxy block11forms packets out of the video data, adapting the headers on the base of the saved tables in the way to imitate the response from the content server8to the user3.

The caching bock12whose functioning is illustrated inFIG. 4includes a data storage (for example, a solid-state drive, a hard disk drive, their possible combinations, etc.), storing the popularity information received from the proxy server and the most popular videos in low and high resolution. When information arrives about a newly loaded video, the caching block12makes a decision about saving it (in particular, in the cache) on the base on its own video popularity statistics, meta-data from the content server8, the video age, its popularity in the network and the amount of links to it on HTML pages. When a request from the scheduler13is received, the caching block12refreshes the popularity statistics and checks whether such a video is contained in the cache. If it is, the caching block12puts the video into the queue to be sent to the scheduler13. If after the first request for this video it is not found in the cache, the caching block12returns a refusal to the scheduler13. If a video is received from the scheduler13, the caching block analyzes the popularity statistics for this video and makes a decision about saving it.

The scheduler13whose functioning is illustrated inFIG. 5estimates the current capacity of the channel of the user device3: in particular, the state of the network, the buffers of the supported user devices3, their tariffs, the resource necessary for the next transfer, and the load of the transcoding block14. After that, it makes a decision about the bit rate for the given user, in particular, if the channel and the transcoding block14have some free resources, the scheduler13decides to send the video data in the mode required by the user. If the resources of the channel and transcoding block14are not enough, they are distributed according to the priority of the user device. User devices with low priority get only residual resources. According to the calculated resource, the block14transcodes the video data. After the transfer rate is chosen, depending on the request from the proxy block11for data search or for transfer, the scheduler can request for video data from the caching block12with required parameters, if a request has come from a user3. If the data are found, they will be passed to the proxy block11after the transcoding, if necessary. If the video is not found, the scheduler will return a refusal. If a video is received from the content server8, then after possible transcoding it will be passed to the proxy block, along with the information about the required rate of its transfer to the user3. The transcoding block14changes the bit rate or the resolution of the stream of the video segments saved in the caching block12or received by the network, by means of transcoding every video segment in real time by demand from the scheduler13according to the current parameters of the user device3. If the transcoding block14has free capacity, it preliminarily transcodes the most popular videos into lower bit rates, so that if the network is overloaded, a part of its load will move to the caching block12.

According to the present invention, when the scheduler13is overloaded, it starts to prioritize the video streams sent to the user devices3in accordance to the users tariffs (for example, Gold, Silver and Bronze). Due to the prioritization, the time-and-frequency resource is redistributed among the users, i.e. the channel rates grow for some users and drop for others. According to the protocols of dynamic adaptive streaming, the users start to request their next video segments in accordance with the channel rates of their devices3, determined by the scheduler13. For some users, the channel rate can turn out to be lower than the minimum available bit rate on the content server8. In this case, for such users, the transcoding block14starts transcoding the requested video data in real time, providing the quality which would match the current rates of the channels, This minimizes the general number of buffer-emptyings for the user devices, whatever their tariffs are.

Such a speed is possible only when using hardware transcoding which is a unique feature of the present invention. It also can provide a much higher performance with a lower consumed power, as compared to the solutions using software transcoding. When there is no congestion, hardware transcoding also allows us to rapidly recode the video data in the caching block12into lower bit rates in order to prepare the caching block12for an congestion of the base station4. In this case, when the congestion has already occured and the radio channel capacity is not enough, the video data will be sent to the user devices from the caching block12.

The transcoding block14is built on the base of a multi-processor hardware implementation of a video codec, providing real-time transcoding of several video streams. At a request from the scheduler13, the transcoding block14starts transcoding every video segment from the caching block12, consequently passing the coded fragments to the proxy block11to pack them in packets and send to the user device3. In the case when the video was not found in the caching block12, transcoding is impossible without caching: for example, if the requested video is not popular. Specifically, the scheduler13sends the video data from the proxy block11to the transcoding block14which performs their segment-to-segment transcoding in real time and returns the transcoded segments back to the scheduler13, without any considerable delay in the transfer process. In order to successfully perform all its functions, the transcoding block14can support such stream data formats as MP2-TS, MP4, WebM, and others.

Delays can be possibly added to the transfer process if the scheduler13, when redistributing the resources between the user devices3at the scheduling stage, makes a decision about delaying the traffic for the finite recipient of the video. In this case, the functioning of the transcoding block14is defined as follows: the transcoded segments can temporarily, until their dispatch, be accumulated in the caching block12, or the transcoding can start each time before the next video data package is sent, and the delay time can be used for transcoding video segments for other user devices3with a higher priority or with a lower satisfaction.

The present invention uses hardware codecs for transcoding, for example, the codec of h.264 format (QuickSync technology) used in modern processor architectures, in particular, Intel processors (Sandy Bridge, Ivy Bridge, for example Intel Xeon E3-1275) and others, Media SDK library which uses QuickSync technology can transcode up to 8 video streams in HD or up to 16 video streams from HD to SD, on just one computational module. The controller5can include several computational modules, a load balancer distributing jobs between them. This allows us to linearly scale the performance of the transcoding block14, adding new computational modules.

High speed of the transcoding leads to a higher efficiency of caching. The possibility appears to rapidly obtain several versions of the same cached video with different bit rates and resolutions.

Besides, quick transcoding makes it easier to rapidly react to the congestion of the base stations4, performing real-time transformation of the video data sent to the user devices3whose radio channel rate is lower than the minimum bit rate of the video on the content server8.

According to the present invention, the transcoding block14can support various stream formats suitable for video data. In particular, data in a certain stream format can store not only video segments but also text (for example, subtitles) and audio data (for example, a soundtrack). Before the transcoding starts, the transcoding block14splits the video data of a particular stream format into several data streams. After the transcoding is finished, it assembles them together again.

The present invention supports intellectual caching algorithms of predictive and proactive types. A proactive algorithm uses statistical information about users' requests to the content provides, and not just the information about the requests passing through a particular controller5but also the information from other controllers5. This ensures a high efficiency of getting into the caching block12even for the devices on the edge of the network.

A predictive algorithm uses various meta-information about the popularity of a certain content from the sites of the content servers8, using traffic analysis methods to accumulate statistics on possible downloads of the contents. For example, a caching block supporting this algorithm accumulates the statistics about the amount of requests for particular videos on social networks, web pages, etc, passing through a particular server. This makes it possible to preload the data which will probably be requested by the users in the near future.

When the free room in a data storage is filled up with saved video data, there comes the necessity to clear it again. On the base of statistical analysis of the requests, the caching block12can pick out and store the most popular video files out of the transferred data, and also make a selection in order to remove the least popular ones from the storage. The present invention supports such caching policies as LFU and LRU which can be used either together or separately. In other embodiments, the present invention provides joint or separate usage of other caching policies, LFU policy gives preference to videos with the highest request frequency. LRU policy gives preference to videos requested recently; i.e. it is based on removing the videos the last request to which was the oldest. Thus, the present invention assumes a frequent change of the subset of popular videos, using a hybrid solution which combines both these policies.

In order to provide high efficiency, the caching block12supports both parallel requests from several processes of the proxy block11and requests from several proxy blocks11placed on several computational modules. One caching block12on a separate hardware device can support several proxy blocks11working with different basic stations4. In one of the embodiments, the proxy block, the transcoding block and the scheduler can be united in a controlling block21(FIG. 6).FIG. 6shows a structural scheme of the controller5according to the present invention, which contains at least one controlling block21; at least one data storage22capable of storing video segments and network interfaces23to connect the controller5with the transit network1and the core network2in order to send the video segments to the user device3.

The foregoing description serves as an example and should not be considered limiting. Experts will understand variations and modifications of the open examples which do not deviate from the gist of the present invention.