Patent Description:
A trade-off has therefore existed between providing high resolution video at low latencies. Some streaming technologies are capable of dynamically increasing or decreasing video bit rates in order to maintain latency. For example, HTTP live streaming (HLS) downloads short clips of a video to a client. The client receiving a clip maintains a buffer that gives it time to attempt to download a higher bit rate version of a segment. If successful, the client plays out the higher bit rate video, but if not, it will cancel the download in favor of a lower bit rate segment. Unfortunately, the buffering involved creates too much delay for the purpose of real-time streaming and two-way communications.

Some explicitly real-time services such as voice and video calling services utilize connectionless and unreliable protocols to send and receive data. An example is the user datagram protocol (UDP), which allows a sender to transmit packets to a receiver without regard for whether or in what order the packets arrive at their destination. Such protocols can provide a low latency experience, although jitter and packet loss may occur.

A client receiving video over UDP may provide feedback to the sender to ensure that the playback is smooth. The sending client may then adjust the bit rate up or down to combat any of the ill effects of such streaming. Some of these streamers can send to multiple clients at the same time, which is not scalable beyond a limited number of clients. <CIT> relates to a packet data communication network that is probed with stuffed data inside the ordinary media streams to determine whether the network is capable of handling a higher bitrate before performing adaptive streaming to switch from a lower bitrate to a higher bitrate. The probing with stuffed data avoids having to transmit actual streaming data from a lower bitrate to a higher bitrate first, determining that the network cannot handle the higher bitrate, and causing the server to switch back to the lower bitrate while providing varying video quality to the user. <NPL>, relates to an algorithm for managing the quality level of a live video stream using a standard feedback mechanism to adapt throughput to varying network conditions. The algorithm uses redundancy to both protect traffic from losses but also as a form of safety margin to both predict available throughput and isolate random fluctuations. <CIT> relates to a system and method for streaming data. A data stream is encoded at a first bitrate and a plurality of first data blocks is transmitted to a receiver. Each of the first data blocks include a first source packet corresponding to the encoded data stream and a first repair packet. A plurality of second data blocks is transmitted to the receiver for a first predetermined period of time. Each of the second data blocks include a second source packet corresponding to the encoded data stream, a second repair packet, and a probing packet.

The object of the invention is to provide enhanced video stream bit rate adaption to varying transmission quality conditions.

Technology is disclosed herein for optimizing the streaming of video to end points while maintaining low latency. In an implementation, a streaming service receives video data for distribution to a plurality of end points. For each end point, the service streams the video data at a given bit rate to the end point. While the video is being streamed, the service sends test data to the end point at an additional bit rate. The service also increases the additional bit rate of the test data until a threshold decline in quality of the video data occurs or until a total bit rate of the video data and the test data reaches a next available bit rate for the video data. The service conditionally switches from streaming the video data at the given bit rate to streaming the video at the next available bit rate if the total bit rate has reached the next available bit rate. In this manner, the bit rate of the video may be safely increased without jeopardizing a low-latency experience.

This Overview is provided to introduce a selection of concepts in a simplified form that are further described below in the Technical Disclosure. It may be understood that this Overview is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

Technology is disclosed herein to enhance video streaming. In various implementations, a streaming service receives video data to be streamed to multiple end points. The service encodes the video at different bit rates such that it may switch between the streams as-needed to provide an optimal streaming experience. However, to ascertain whether a particular bit rate is supported, the service first sends test data to a given end point while also streaming video to the end point at an initial bit rate.

As the video is streamed, the service incrementally increases the test data until a point at which the bit rate of the test data and the bit rate of the original stream meet or exceed the next available bit rate of the other versions of the video. If the quality of the original stream has not degraded at this point (or has not degraded too much), the streaming service switches to the video stream that was encoded at the next available bit rate. If the quality has declined a threshold amount while the test data was being increased, then the service maintains the present bit rate at which the video is streamed.

The quality of the streaming may be evaluated using one or more of a number of metrics, such as latency, packet loss, and jitter. An element within the streaming service may make such a determination, e.g. an edge server or the like, based on metrics fed back to it by a client on the receiving end of the video stream. However, the quality of the streaming may also be evaluated by the client application. The client may thus report to the service that the quality has been maintained or degraded or, optionally, the client may even request the higher bit rate stream (assuming the quality was sufficient as the test data was increased).

A decrease in the bit rate of the video is also possible. The streaming service may switch to a version of the video encoded at a lesser bit rate if, for example, network conditions (latency, bit rate, jitter, etc.) indicate a decline in quality irrespective of any test data. That is, the quality of the streaming may decline even without any test data being set. The service may responsively switch to one of the versions of the video that has been encoded at a lesser bit rate than the present version being sent.

In some implementations, the streaming service need only encode the video at the various bit rates once. Then, different versions of the video at the different bit rates are distributed within the service to the servers that ultimately transmit the video to end points. In this manner, the servers responsible for sending the video to the end points need not be burdened with the processing overhead required to encode the video at the various bit rates.

Various technical effects may be appreciated from the foregoing and following discussion, including the ability to increase video bit rates while mitigating detrimental effects on latency. This may be especially beneficial in the context of experiences that are sensitive to delay, such as "live" gaming experiences with two-way interaction between a gamer and his or her fans. In another example, a user may wish to transmit video to many users via a social network. In both cases, low latency allows users to communicate bi-directionally in a real-time or near real-time fashion while still enjoying the benefits of a high-resolution video stream.

<FIG> illustrates operational environment <NUM> in an implementation. Operational environment <NUM> includes streaming service <NUM> and end points <NUM>, <NUM>, <NUM>, and <NUM>. Streaming service <NUM> may be implemented in one or more data centers (physical or virtual), represented by data center <NUM>, and on one or more computing systems, of which computing system <NUM> in <FIG> is representative.

Streaming service <NUM> receives video (and optionally other content) from one or more end points and transmits the video (and optionally other content) to one or more other end points for user consumption. End point <NUM> is representative of the various computing devices that may stream video to streaming service <NUM>, while end points <NUM>, <NUM>, and <NUM> are representative of the computing devices to which streaming service <NUM> may stream video data. Examples include, but are not limited to, laptop and desktop computers, tablets, mobile phones, wearable devices, entertainment devices, gaming consoles, other server computers, Internet of Things (IoT) devices, or any other type of device. Streaming service <NUM> communicates with end points <NUM>, <NUM>, <NUM>, and <NUM> over one or more communication networks (e.g. the Internet), combination of networks, or variations thereof. In implementations according to the claimed invention, streaming service <NUM> at least includes an encoder, a relay server and an edge server. networks, or variations thereof. In implementations according to the claimed invention, streaming service <NUM> at least includes an encoder, a relay server and an edge server.

Streaming service <NUM> employs a streaming process <NUM> when receiving, encoding, sending, and switching between video streams. Referring to <FIG>, streaming process <NUM> may be implemented in program instructions in the context of any of the software applications, modules, components, or other such programming elements deployed in the various elements of streaming service <NUM>, such servers, switches, and routers. The program instructions direct the underlying physical or virtual computing system or systems to operate as follows, referring parenthetically to the steps in <FIG> in the context of operational environment <NUM> in <FIG>.

To begin, end point <NUM> transmits video data <NUM> to streaming service <NUM>. Streaming service <NUM> receives the video stream and encodes the video data in multiple streams, each at a different bit rate (step <NUM>). That is, one video feed is received at a given bit rate from end point <NUM> and is decoded to extract the video data being streamed. The video data is then re-encoded multiple times to produce multiple versions of the original video stream. Each of the multiple versions is encoded at a different bit rate relative to each other of the multiple versions of the video stream.

Next, streaming service <NUM> sends one or more of the video streams to end point <NUM>, end point <NUM>, and end point <NUM> (step <NUM>). The one or more video streams are sent via a connectionless and unreliable transport protocol such as UDP, a custom or proprietary protocol, or the like.

From the perspective of a given end point, only one stream is sent to the end point, although the same stream may be sent to more than one of the end points. End points <NUM>, <NUM>, and <NUM> receive the video streams and play them out for users <NUM>, <NUM>, and <NUM> respectively. The end users may optionally desire to chat with or otherwise engage with user <NUM>. Accordingly, return content <NUM> may be sent to streaming service <NUM> to be forwarded to end point <NUM>, although it may bypass streaming service <NUM> entirely. This may occur in the context of, for example, a gaming session hosted by user <NUM> that is "broadcast" from end point <NUM> to other participating end points, e.g. end points <NUM>, <NUM>, and <NUM>.

Each stream that is sent to a given end point is streamed at an initial bit rate (step <NUM>) from streaming service <NUM> to the end point. This may be the same bit rate at which the video is received from end point <NUM>. However, it may also be a different one of the other encoded streams - and thus - one of the other bit rates. The video is streamed to the end points and each end point decodes and plays-out the video.

For a given video stream and end point, streaming service <NUM> proceeds to send test data at an additional bit rate to the end point at substantially the same time as the video stream (step <NUM>). The test data may be "noise" that may be ignored and discarded by the end point. In some cases, the test data may be a duplicate of the video stream, which may even increase the robustness of the transmission if some of the packets of the video stream are lost. In some implementations, the duplicate data may be sent with a flag set to indicate that it is a duplicate of the video stream. The flag may be set in a header portion of the packets that comprise the video stream, for instance. Such a flag or other such indication may allow the client playing back the video to discard or otherwise ignore duplicates.

Streaming service <NUM> monitors the quality of the video stream while (or after) the test data is being sent to ascertain whether a higher bandwidth version of the video could be supported. The service ascertains whether a threshold decline in quality has occurred in response to the test data (step <NUM>). The threshold decline may be defined in terms of one or more quality metrics, such as an increase in latency, an increase in packet loss, an increase in jitter, or the like. Streaming service <NUM> may obtain such metrics from clients on the end points that track and report the metrics. In some implementations, the clients monitor for the threshold decline in quality and report the decline to streaming service <NUM>, rather than (or in addition to) reporting the individual metrics.

If a threshold quality decline has occurred, then streaming service <NUM> determines to continue streaming the video at its present bit rate. The service may optionally continue to test the bandwidth, but may also cease with the additional data, at least for a period of time. However, if the quality of the video stream has not suffered a threshold decline, then the streaming service <NUM> determines if the total bit rate (the sum of the additional bit rate and the present bit rate) equals, exceeds, or otherwise nears the next available bit rate at which the video data has been encoded (step <NUM>). For instance, assume that the video has been encoded at <NUM> mb/s, <NUM> mb/s, and <NUM> mb/s, and is presently being streamed at <NUM> mb/s. In this case, streaming service <NUM> would determine whether the sum of the test data and the present bit rate (<NUM> mb/s) meets, exceeds, or is otherwise close to <NUM> mb/s.

If so, then streaming service <NUM> switches the stream to the video data encoded at the next available bit rate (step <NUM>). However, if there remains room between the present bit rate and the next available bit rate, then streaming service increases the bit rate of the test data (step <NUM>) and continues to evaluate the quality of the stream. The bit rate of the test data may thus be incrementally increased up until a point that the quality declines precipitously or the next available bit rate is reached, at which point the stream may be switched to the stream encoded at next available bit rate.

<FIG> illustrates operational environment <NUM> in another implementation according to the invention. Operational environment <NUM> includes streaming service <NUM>, which provides video streaming services to end points in the context of gaming sessions, social network activity, video conferences, and the like. In general, an end point sends a video stream to streaming service <NUM>, which is then decoded and encoded into multiple video streams having different bit rates. Streaming service <NUM> sends the various video streams to the end points and switches between lower and higher bit rates when doing so to optimize the user experience.

More particularly, streaming service <NUM> includes various elements that function together to provide streaming services to end points. Encoder <NUM> receives video streams from end points, decodes them, and re-encodes them into multiple streams having different bit rates relative to each other. Relay servers <NUM> and <NUM> each receive copies of the streams and send them to edge servers <NUM>, <NUM>, <NUM>, and <NUM>. In this implementation, edge server <NUM> sends video traffic to end points <NUM>; edge server <NUM> sends video traffic to end points <NUM>; edge serer <NUM> sends video traffic to end points <NUM>; and edge server <NUM> sends traffic to end points <NUM>.

Each edge server implements a streaming process (e.g. streaming process <NUM>) that governs which of the multiple streams is sent to any one end point. The streaming process may be implemented in program instructions in the context of any of the software applications, modules, components, or other such programming elements deployed in the various elements of each edge server, of which computing system <NUM> in <FIG> is representative.

As shown, each edge server receives multiple versions of the video to be streamed, each encoded at different bit rates relative to each other as indicated by legend <NUM>. One version at a time is then streamed to any individual end point. Test data is also transmitted to the end point, allowing the edge server to ascertain whether a higher bit rate version of the video stream could be supported. If so, the edge server switches to the higher bit rate version, but if not, the edge server continues to stream at the present bit rate. In some cases, the edge server may step down the bit rate if the quality of the streaming drops, which may occur separately and independent from sending the test data.

<FIG> illustrates an operational scenario in an implementation with respect to operational environment <NUM> in <FIG>. In operation, end point <NUM> streams video of a local experience to encoder <NUM>. The local experience may be, for example, a gaming experience being play out on end point <NUM>, a live video feed of a scene captured by end point <NUM> or a peripheral device, a video conference, or any other content that may be captured in video.

Assuming that end point <NUM> had encoded the video, encoder <NUM> proceeds to decode the video and then re-encode it at multiple bit rates. Encoder <NUM> sends the multiple versions of the video to relay server <NUM>, which then relays the video one or more edge servers, including edge server <NUM> in this example scenario.

Edge server <NUM> streams out the video at one of the bit rates to end point 311A and also streams out the video at the same or a different bit rate to end point 311B. It is assumed for exemplary purposes that the video is streamed out at <NUM> mb/s initially. End points 311A and 311B receive their respective video streams and play them out to their users. One or more of the users may optionally engage with a user associated with end point <NUM> by way of user input, e.g. chat messages, which may be exchanged with end point <NUM> either directly or indirectly in furtherance of the gaming, social network, or other such experience.

Edge server <NUM> employs streaming process <NUM> to gauge the robustness of the network link(s) between it and the end points, so as to increase the bit rate of the respective video streams, if feasible. In furtherance of this goal, edge server <NUM> sends test data to end point 311A in increasing increments. End point 311A measures the performance of the video stream as the test data is received and provides feedback to edge server <NUM>. The feedback may indicate, for example, the latency of the video, packet loss, and/or jitter. Other metrics in place of - or in addition to - those metrics may also be included in the feedback, such as an indication of key frame loss, perceptual quality of the video, and the like.

Edge server <NUM> receives the feedback and determines whether a quality of the video stream satisfies a quality measure. The quality measure may be related to any one or more of the performance metrics in the feedback. For example, the quality measure may be a minimum acceptable amount of latency, packet loss, or jitter, or a composite value derived from any two or more of the metrics. The test data is incremented after each set of feedback is received until the quality fails to satisfy the minimums or until the present bit rate and the test bit rate together reach the next available bit rate. It is assumed for exemplary purposes that the next available bit rate is reached and thus the <NUM> mb/s video feed is streamed to end point 311A.

The same process is carried out by edge server <NUM> with respect to end point 311B. The process may occur with respect to end point 311B at substantially the same time as with end point 311A, even though they are shown as happening sequentially in <FIG>.

Edge server <NUM> edge server <NUM> sends test data to end point 311B in increasing increments. End point 311B measures the performance of the video stream as the test data is received and provides feedback to edge server <NUM>. Edge server <NUM> receives the feedback and determines whether a quality of the video stream satisfies a quality measure. The test data is incremented after each set of feedback is received until the quality fails to satisfy the minimums or until the present bit rate and the test bit rate together reach the next available bit rate. It is assumed for exemplary purposes that the next available bit rate is reached and thus the <NUM> mb/s video feed is streamed to end point 311B.

Edge server <NUM> may continue to employ streaming process <NUM> with respect to one or both of end point 311A and 311B since the <NUM> mb/s bit rate version of the video remains. Thus, edge server <NUM> may continue to send test data at incrementally increased rates, receive feedback, and switch to the <NUM> mb/s feed if allowed based on the quality response of the <NUM> mb/s video feed to the increasing amounts of test data.

<FIG> illustrates the same scenario as <FIG>, but with end point 311A making the determination whether to upgrade to a higher bit rate video feed. In operation, end point <NUM> streams video of a local experience to encoder <NUM>. The local experience may be, for example, a gaming experience being play out on end point <NUM>, a live video feed of a scene captured by end point <NUM> or a peripheral device, a video conference, or any other content that may be captured in video.

Encoder <NUM> proceeds to decode the video and then re-encode it at multiple bit rates. Encoder <NUM> sends the multiple versions of the video to relay server <NUM>, which then relays the video one or more edge servers, including edge server <NUM> in this example scenario.

End point 311A next requests test data from edge server <NUM>. Alternatively, edge server <NUM> may be programmed to automatically and/or periodically send test data to end point 311A. End point 311A receives the test data and evaluates the quality response of the initial video feed to the test data. If the quality of the video feed remains satisfactory, end point 311A sends a request to edge server <NUM> to increase the bit rate of the test data.

Edge server <NUM> complies with the request and sends an increased amount of test data to end point 311A. End point 311A again receives the test data and evaluates the quality response of the initial video feed. It is assumed that the quality remains high and so end point 311A again requests an incremental increase in the test data.

Edge server <NUM> sends more test data at a further-increased bit rate. End point 311A receives the increased test data and, in response to the quality of the video feed continuing to remain satisfactory, requests edge server <NUM> to switch to the next available bit rate. Accordingly, edge server <NUM> sends the <NUM> mb/s version of the video to end point 311A.

It may be appreciated that end point 311A could continue with such a process since the <NUM> mb/s version of the video remains. In addition, end point 311B could also employ the same process locally with respect to the video feed it receives.

<FIG> illustrate various operational scenarios to better explain some aspects of the video streaming technology disclosed herein. In <FIG>, graph <NUM> includes two y-axes (bit rate and quality) graphed against the x-axis (time). On the left y-axis, the encoded bit rate of a video stream sent from an encoder to an end point in operational environment <NUM> is illustrated. On the right y-axis, the quality of the video stream is illustrated in terms of an increase in quality and a decrease in quality. Finally, three lines are graphed: the bit rate of the main video stream; the bit rate of the test data being sent; and the relative quality of the main video stream (which indicates if the quality is increasing, decreasing, or remains the same).

As shown, the bit rate of the video stream is initially <NUM> mb/s and its quality is considered high. As time progresses, test data is introduced at <NUM> mb/s. The edge server streaming the video and sending the data monitors how the quality of the stream reacts to the test data. Here, the quality is maintained and so the edge server totals the additional bit rate and the present bit rate of the stream. Since the sum is <NUM> mb/s, which is less than the next available bit rate (<NUM> mb/s), the edge server increments the bit rate of the test data to <NUM>½ mb/s in this example, although the increments may be smaller or larger.

Once the test data is increased, the edge server gauges the quality response to the added traffic. Here, the quality remains strong. In addition, the sum of the additional bit rate and the present bit rate is still less than the next available bit rate. Accordingly, the edge server increments the additional bit rate to <NUM> mb/s. At this point, the quality still has not declined and, since the additional bit rate plus the present bit rate equals the next available bit rate (<NUM> mb/s), the edge server switches the stream to the version encoded at <NUM> mb/s. The higher bit rate video is thus streamed to the end point for the duration or optionally until its quality declines to a point that triggers a step down to a lesser bit rate.

In <FIG>, graph <NUM> includes the same x and y axes. The bit rate of the video stream is initially <NUM> mb/s and its quality is high. As time progresses, test data is introduced at <NUM> mb/s. The edge server streaming the video and sending the data monitors how the quality of the stream reacts to the test data. In this example, the quality is maintained and so the edge server sums the additional bit rate and the present bit rate of the stream. Since the sum is <NUM> mb/s, which is less than the next available bit rate (<NUM> mb/s), the edge server increments the bit rate of the test data to <NUM>½ mb/s in this example, although the increments may be smaller or larger.

Having increased the test data, the edge server monitors the quality of the video stream to the increase. It is assumed for exemplary purposes that the quality declines beyond a threshold amount. The quality decline indicates that a higher bit rate cannot be supported. The test data is stopped, and the video stream is maintained at its initial bit rate for the duration or optionally until another decline in quality precipitates a step down in its bit rate. The test data may be re-introduced again at a later time to gauge the ability of the network(s) connecting the edge server to the end point to support a higher bit rate.

Graph <NUM> in <FIG> also contains the same x and y axes for time, bit rate, and quality as in <FIG>. In operation, an edge server transmits a video stream at a first bit rate (<NUM> mb/s) to an end point. The edge server proceeds to send test data at. <NUM> mb/s to the end point while monitoring the quality of the original video stream. As no decline occurs in the quality, and since the sum of the initial bit rate and the additional bit rate (<NUM> mb/s) does not reach the next available bit rate (<NUM> mb/s), the edge server steps up the test data.

As the test data increases to <NUM> mb/s, the quality of the original video signal declines. However, the decline is not sufficient enough to stop the incremental increases to the test data. Rather, the quality is judged to be acceptable. Accordingly, the bit rate of the test data is increased again to <NUM> mb/s, bringing the total bit rate to <NUM> mb/s.

At this point, the sum of the bit rates has reached the next available bit rate and the edge server responsively switches the stream to the <NUM> mb/s feed. The edge server also continues to send test data to ascertain whether the end point and/or the network links there between can accommodate the next available bit rate of <NUM> mb/s. It is assumed for exemplary purposes that the quality remains satisfactory as the additional bit rate is increased from <NUM> mb/s to <NUM> mb/s, at which point the edge server switches the video stream to the data encoded at <NUM> mb/s.

<FIG> illustrates computing system <NUM>, which is representative of any system or collection of systems in which the various applications, services, scenarios, and processes disclosed herein may be implemented. Examples of computing system <NUM> include, but are not limited to, server computers, rack servers, web servers, cloud computing platforms, and data center equipment, as well as any other type of physical or virtual server machine, container, and any variation or combination thereof. Other examples may include smart phones, laptop computers, tablet computers, desktop computers, hybrid computers, gaming machines, virtual reality devices, smart televisions, smart watches and other wearable devices, as well as any variation or combination thereof.

Computing system <NUM> may be implemented as a single apparatus, system, or device or may be implemented in a distributed manner as multiple apparatuses, systems, or devices. Computing system <NUM> includes, but is not limited to, processing system <NUM>, storage system <NUM>, software <NUM>, communication interface system <NUM>, and user interface system <NUM>. Processing system <NUM> is operatively coupled with storage system <NUM>, communication interface system <NUM>, and user interface system <NUM>.

Processing system <NUM> loads and executes software <NUM> from storage system <NUM>. Software <NUM> includes streaming process <NUM>, which is representative of the processes discussed with respect to the preceding <FIG>, including streaming process <NUM>. When executed by processing system <NUM> to enhance video streaming, software <NUM> directs processing system <NUM> to operate as described herein for at least the various processes, operational scenarios, and sequences discussed in the foregoing implementations. Computing system <NUM> may optionally include additional devices, features, or functionality not discussed for purposes of brevity.

Referring still to <FIG>, processing system <NUM> may comprise a microprocessor and other circuitry that retrieves and executes software <NUM> from storage system <NUM>. Processing system <NUM> may be implemented within a single processing device but may also be distributed across multiple processing devices or sub-systems that cooperate in executing program instructions. Examples of processing system <NUM> include general purpose central processing units, application specific processors, and logic devices, as well as any other type of processing device, combinations, or variations thereof.

Storage system <NUM> may comprise any computer readable storage media readable by processing system <NUM> and capable of storing software <NUM>. Storage system <NUM> may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. Examples of storage media include random access memory, read only memory, magnetic disks, optical disks, flash memory, virtual memory and non-virtual memory, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or other suitable storage media, except for propagated signals. In no case is the computer readable storage media a propagated signal.

Software <NUM> may be implemented in program instructions and among other functions may, when executed by processing system <NUM>, direct processing system <NUM> to operate as described with respect to the various operational scenarios, sequences, and processes illustrated herein. For example, software <NUM> may include program instructions for implementing streaming process <NUM>.

In particular, the program instructions may include various components or modules that cooperate or otherwise interact to carry out the various processes and operational scenarios described herein. The various components or modules may be embodied in compiled or interpreted instructions, or in some other variation or combination of instructions. The various components or modules may be executed in a synchronous or asynchronous manner, serially or in parallel, in a single threaded environment or multi-threaded, or in accordance with any other suitable execution paradigm, variation, or combination thereof. Software <NUM> may include additional processes, programs, or components, such as operating system software, virtual machine software, or other application software, in addition to or that include streaming process <NUM>. Software <NUM> may also comprise firmware or some other form of machine-readable processing instructions executable by processing system <NUM>.

In general, software <NUM> may, when loaded into processing system <NUM> and executed, transform a suitable apparatus, system, or device (of which computing system <NUM> is representative) overall from a general-purpose computing system into a specialpurpose computing system customized to stream video. Indeed, encoding software <NUM> on storage system <NUM> may transform the physical structure of storage system <NUM>. The specific transformation of the physical structure may depend on various factors in different implementations of this description. Examples of such factors may include, but are not limited to, the technology used to implement the storage media of storage system <NUM> and whether the computer-storage media are characterized as primary or secondary storage, as well as other factors.

For example, if the computer readable storage media are implemented as semiconductor-based memory, software <NUM> may transform the physical state of the semiconductor memory when the program instructions are encoded therein, such as by transforming the state of transistors, capacitors, or other discrete circuit elements constituting the semiconductor memory. A similar transformation may occur with respect to magnetic or optical media. Other transformations of physical media are possible without departing from the scope of the present description, with the foregoing examples provided only to facilitate the present discussion.

Communication interface system <NUM> may include communication connections and devices that allow for communication with other computing systems (not shown) over communication networks (not shown). Examples of connections and devices that together allow for inter-system communication may include network interface cards, antennas, power amplifiers, RF circuitry, transceivers, and other communication circuitry. The connections and devices may communicate over communication media to exchange communications with other computing systems or networks of systems, such as metal, glass, air, or any other suitable communication media. The aforementioned media, connections, and devices are well known and need not be discussed at length here.

User interface system <NUM> is optional and may include a keyboard, a mouse, a voice input device, a touch input device for receiving a touch gesture from a user, a motion input device for detecting non-touch gestures and other motions by a user, and other comparable input devices and associated processing elements capable of receiving user input from a user. Output devices such as a display, speakers, haptic devices, and other types of output devices may also be included in user interface system <NUM>. In some cases, the input and output devices may be combined in a single device, such as a display capable of displaying images and receiving touch gestures. The aforementioned user input and output devices are well known in the art and need not be discussed at length here.

User interface system <NUM> may also include associated user interface software executable by processing system <NUM> in support of the various user input and output devices discussed above. Separately or in conjunction with each other and other hardware and software elements, the user interface software and user interface devices may support a graphical user interface, a natural user interface, or any other type of user interface.

Communication between computing system <NUM> and other computing systems (not shown), may occur over a communication network or networks and in accordance with various communication protocols, combinations of protocols, or variations thereof. Examples include intranets, internets, the Internet, local area networks, wide area networks, wireless networks, wired networks, virtual networks, software defined networks, data center buses, computing backplanes, or any other type of network, combination of network, or variation thereof. The aforementioned communication networks and protocols are well known and need not be discussed at length here.

The functional block diagrams, operational scenarios and sequences, and flow diagrams provided in the Figures are representative of exemplary systems, environments, and methodologies for performing novel aspects of the disclosure. While, for purposes of simplicity of explanation, methods included herein may be in the form of a functional diagram, operational scenario or sequence, or flow diagram, and may be described as a series of acts, it is to be understood and appreciated that the methods are not limited by the order of acts, as some acts may, in accordance therewith, occur in a different order and/or concurrently with other acts from that shown and described herein. For example, those skilled in the art will understand and appreciate that a method could alternatively be represented as a series of interrelated states or events, such as in a state diagram. Moreover, not all acts illustrated in a methodology may be required for a novel implementation.

Claim 1:
A method of operating a streaming service (<NUM>), the method comprising:
in an encoder (<NUM>):
receiving encoded video data as a video stream at an initial bit rate from an end point (<NUM>) for distribution to a plurality of other end points (311A, 311B);
decoding the encoded video data and re-encoding the video data at a plurality of different bit rates; and
sending the video data at the plurality of different bit rates simultaneously to a relay server (<NUM>);
in the relay server, sending the video data at the plurality of different bit rates simultaneously to an edge server (<NUM>); and
in the edge server:
streaming the video data at a given bit rate to an end point of the plurality of other end points;
sending test data comprising a duplicate of at least a portion of the video data to the end point at an additional bit rate while streaming the video data at the given bit rate;
increasing the additional bit rate of the test data until a total bit rate of the video data and the test data reaches a next available bit rate of the plurality of different bit rates, or until a quality of the video data fails to satisfy a quality measure; and
conditionally switching from streaming the video data to the end point at the given bit rate to streaming the video at the next available bit rate if the total bit rate has reached the next available bit rate.