Patent Description:
The following discussion generally delivery of digital video content over a wireless network. More particularly, the following discussion relates to systems, processes and devices to improve the delivery of streaming video content to a client device while the client device is moving based upon expected geographic location of the device.

Streaming media can present numerous technical challenges, particularly when the client device that is receiving the media stream is a mobile device such as a telephone, tablet, portable computer or the like. One challenge that can arise is that as the device moves during the streaming session, different locations can have substantially different network conditions. Typical media players adapt their content requests based solely upon network conditions. When the network is good, the streaming is of very high quality and when the network is bad, the service reduces the quality based upon available bandwidth.

But as a user passes through a tunnel or under an overpass, for example, network service can be temporarily interrupted. Even if service continues, available bandwidth can vary substantially from location to location. Rural areas, for example, often have less network coverage than more metropolitan areas. Conversely, densely populated areas may have less available bandwidth, particularly at certain times of the day. These variations in network availability based upon time and/or location can interrupt streaming videos during playback, or otherwise have adverse effects upon the user experience.

<CIT> describes a scheme for optimizing segment sizes for an adaptive bitrate (ABR) streaming client engaged in a current ABR streaming session. In one implementation, a determination is made whether a wireless UE device executing the ABR streaming client is approaching a radio white spot area. If so, a video buffer of the ABR client is configured to preload a fixed number of segments having an adjusted size depending on the duration of the radio white spot area. The preloaded segments may comprise lower quality video segments, and as the wireless UE device exits the radio white spot area, the segment size and/or bitrates may be restored depending on the signal quality.

<CIT> describes a technology for performing dynamic adaptive streaming over hypertext transfer protocol (DASH). A planned route may be selected for a mobile device. Wireless channel information may be received for the planned route from a channel information database (CID). Geographical locations along the planned route where wireless network channel conditions are below a defined threshold may be determined based on the wireless channel information. Additional segments of a media file may be requested from a media server prior to entering the determined locations along the planned route.

<CIT> describes initiating actions for mobile devices prior to a mobile user entering problem zones. A position and a route of a user are identified. A database is accessed to retrieve data identifying problem zones in a communications network. The position and route of the user are compared to the retrieve data identifying problem zones. A determination is made to identify when the user is approaching a problem zones. An action is performed to compensate for unfavorable network conditions of the problem zone when the user enters the problem zone. <CIT> describes that a mobile device can obtain wireless network signal strength map data that indicates, for various nearby geographical regions, the wireless network signal strength in each such region. The mobile device can transmit that data to a vehicular navigation system responsible for automatically selecting a high-quality route of vehicular travel between a specified source and destination. The system can take the wireless network signal map data into account when selecting that route. When selecting from among multiple different routes of vehicular travel between a specified source and destination, the system may employ an algorithm that considers wireless network signal strengths along those routes, in addition to the other factors. Consequently, the system can select a longer route having better signal strength over a shorter route having worse signal strength. The system can present the selected route within a set of suggested routes, potentially along with reasons for each route's suggestion.

It is therefore desirable to create systems, devices and automated processes that can improve the delivery of streaming media content as the mobile device moves from geographic location to location.

Various embodiments relate to different automated processes, computing systems, devices and other aspects of a video streaming system that delivers digital media content, particularly video content, to a telephone, laptop, tablet or other mobile device. In various embodiments, the media stream is requested from a network server by a mobile device for delivery using adaptive streaming techniques.

The following detailed description is intended to provide several examples that will illustrate the broader concepts that are set forth herein, but it is not intended to limit the invention or the application and uses of the invention. Furthermore, there is no intention to be bound by any theory presented in the preceding background or the following detailed description.

Various embodiments improve media streaming by adapting requests for content in anticipation of the user's future location. If a user (or, more specifically, the user's phone or other device) is expected to pass through a lower-bandwidth location, then requests for media content are pre-emptively obtained beforehand at places and times that the available bandwidth is more favorable. Additionally, requests for content may be delayed during lower bandwidth conditions if the device is expected to soon encounter more bandwidth. Content requests can be optimized (or at least improved) based upon the expected bandwidth that will be encountered during the upcoming movement.

Yhe phone or other mobile device stores received signal strength indications (RSSI) or similar indicia of signal quality as the device moves during the day. The location tracking may be performed with reference to GPS coordinates, for example, or any other available reference. Signal strength/quality may be recorded locally and/or remotely as the device moves from place to place. Moreover, the movements may be identified to recognize regular movements (e.g., commutes, frequent travel routes, etc.) of the device. As the user moves through a recognized or repeated movement, the previously-stored signal quality data can be consulted to optimize (or at least improve) content delivery. Content can be retrieved while the signal is stronger, for example, to fill a cache that can be depleted when the signal is weaker. Adaptive streaming techniques based upon segments of varying quality can be exploited, for example, to pre-request higher quality segments while bandwidth allows. Several examples of basic adaptive streaming techniques are described in <CIT>, although other techniques, processes and systems could be equivalently used.

<FIG> illustrates one example wherein a media player or other device <NUM> travels from point "A" to point "B" along a route that encounters points L0, L1, L2, L3,. , LN to an end at point "B". In this example, the device (or a network service in communication with the device) can build a map of the bandwidth, signal strength and/or other indicia of video quality as the device travels along the path <NUM>. From L0 to LN. Of course <FIG> illustrates one example; equivalent embodiments may consider any number of points along any path, with points being regularly or irregularly spaced as desired.

Device <NUM> suitably stores the quality measurements that are obtained at the various points along path <NUM> in a database or other structure <NUM>. In the example shown in <FIG>, structure <NUM> shows a two-dimensional structure that records measurements taken at different times for various locations. Example structure <NUM> also records one or more measurement times, as well as an average value for signal strength encountered at each point on path <NUM>. Other embodiments could be structured in other ways, with additional or alternate data recorded, or with less data stored (e.g., only average values, without retaining individual measurements). Still other embodiments may transmit measurement data to a network service <NUM>, if desired, for cloud-based computations and storage of data. Computations performed by server <NUM> may include path recognition, data averaging or other smoothing, and/or other computations as desired. In some implementations, data <NUM> collected by multiple devices <NUM> may be aggregated into a shared database <NUM> maintained on a cloud-based storage <NUM> to permit "crowdsourced" data that may be more complete and/or accurate than data <NUM> recorded by any single device <NUM>.

Data <NUM> (and/or aggregated data <NUM>) may be processed locally by device <NUM> or remotely by service <NUM>. In some embodiments, device <NUM> processes location data <NUM> to optimize (or at least improve) fetching of streaming media content or the like based upon expected future locations of the device <NUM>.

If, for example, the video streaming quality at location L3 is always (or at least usually) better than at location L4, then area L4 can be considered low signal area. To provide a smooth video experience, the video segments can be pre-fetched at better quality before the device enters the area of lower signal quality. If quality content is pre-fetched while the signal permits, then the user can treated to a more consistent user experience even while traversing areas of low signal quality. In a further example, suppose that the video streaming quality at L2 is regularly observed to be lower than at location L3. In the intermediate area following L2, then, the quality keeps improving. Instead of lowering the quality and prefetching low quality video segment while in area L2, then, the device <NUM> could instead wait to reach location L3 before downloading higher quality segments. Many additional examples could be formulated.

Various embodiments therefore build a map of streaming quality with respect to geographic location (see, e.g., data <NUM> in <FIG>). As the device <NUM> passes the same (or nearby) locations at different times, signal quality measurements recorded in data <NUM> can be averaged or otherwise combined for better accuracy over time. The data <NUM> may also record time of day, day of week, seasonal data, etc. if desired to further improve predictability (e.g., some locations may be slower at certain dates/times than at others). Using this map and the probable commute (or other expected route) for the user, decisions can be made to decide when to pre-fetch or delay fetching of video segments.

Data map <NUM> can be built by a streaming app executing on the client device <NUM> and/or by service <NUM> based on the data <NUM> collected by users and can be built using data from third party sources. This data, when used in conjunction with user movement patterns, can be used to determine when to prefetch or delay fetching of video segments as the user moves along a predictable path <NUM>. If a user regularly follows the same commute path, for example, the concepts described herein may be used to pre-fetch data (or post-fetch data) to avoid fetching while the user is in poor signal quality areas (e.g., areas near tunnels, underpasses, natural or human-made valleys, areas with poor wireless coverage, etc.). Similar techniques can be used when the device is moving across or within a building and/or as the device crosses low quality locations within a building or campus (elevator lifts, basements, shielded locations, etc.). These concepts may be particularly useful for transit riders, commuters, students, working professionals and many others.

Device <NUM> may represent any sort of media player, mobile phone, tablet, portable computer or other computing device as desired. The device <NUM> illustrated in <FIG> suitably includes a processor <NUM> and associated memory <NUM> that stores digital instructions executed by processor <NUM>, as desired. Various embodiments will include a radio frequency (RF) interface <NUM> that includes a receive signal strength indicator (RSSI) <NUM>. Interface <NUM> may provide wireless communications via antenna <NUM> or the like for cellular telephony, wireless local area or personal area networks (e.g., IEEE <NUM> networks), point-to-point communications (e.g., Bluetooth or other IEEE <NUM> communications), and/or any other wireless communications as desired. In various embodiments, RSSI readings indicating the received strength of cellular telephony signals are used to populate date <NUM>, as desired.

As illustrated in <FIG>, device <NUM> also includes a global positioning system (GPS) or other location detection interface <NUM> that determines the geographic position of the device <NUM> using, for example, signals received from a satellite cluster via antenna <NUM>. Note that although antennas <NUM> and <NUM> are shown separately in <FIG>, many practical embodiments will use an equivalent hybrid antenna that receives different frequencies for position determination and communications. <FIG> also shows an interface <NUM> to a display <NUM> that can be used to present streaming video, user interfaces and other graphical displays. Typical devices <NUM> will also include interfaces to a touchscreen, keyboard, microphone and/or the like to receive user inputs.

Media player devices <NUM> will typically include a non-transitory data storage <NUM> such as a magnetic, optical or solid state drive that can be used to store cached media content, data <NUM> and/or any other data as desired. Storage <NUM> is illustrated separately from memory <NUM> in this example, but in practice a common solid state or other storage could be used to store data <NUM>, media content, and/or digital program instructions for execution by processor <NUM>, as desired.

In practice, then, device <NUM> suitably records signal strength data at various locations as data <NUM>. The signal strength data may be collected by RF interface <NUM> and associated with position data from location interface <NUM> in any manner for storage in local storage <NUM>, remote storage <NUM>, and/or elsewhere as desired. The various functions of device <NUM> generally occur under the control of processor <NUM>, which suitably executes digital instructions stored in memory <NUM>, storage <NUM> and/or elsewhere as appropriate.

<FIG> shows an example process <NUM> that could be automatically performed by processor <NUM> or the like to improve the delivery of streaming media content while the device <NUM> is moved from place to place, e.g., along path <NUM>. Process <NUM> may be implemented in digital instructions stored in memory <NUM> and/or storage <NUM>, as desired, for execution by processor <NUM>.

As noted above, device <NUM> suitably gathers and stores location and signal strength data <NUM> for subsequent use (function <NUM>). In some implementations, device <NUM> collects location and signal data as frequently as possible to maximize the scope of data <NUM>. In various embodiments, collected data is shared with a remote service <NUM> via network <NUM>, and/or location data obtained from other devices <NUM> is retrieved from remote service <NUM> via network <NUM> as needed. Information may be retrieved on a periodic or aperiodic basis, as desired. Other embodiments may send requests for signal measurements from service <NUM> to obtain information from anticipated future locations, as desired.

Path determination may occur as needed (function <NUM>). In the example of <FIG>, path determination <NUM> is performed in response to a user request to stream a media program (function <NUM>), or in response to the user activating a media player application that provides streaming media. This is helpful if the process <NUM> instructions are coded into the media player application. Other embodiments, however, may perform path prediction <NUM> outside of the media player function, as desired.

Near term movement may be predicted in any number of different ways. In various embodiments, paths can be predicted based upon repeated behaviors (e.g., commutes or other regular movements), based upon expected movement at certain times of the day or week, based upon entering or leaving a geofence or other location determination (e.g., recognize that the user is leaving home or work), based upon recent movements, based upon an expected return to a known location (home or work) at certain times, and/or any number of other ways. Paths may be defined based upon historical observations, or the like.

If the user requests a video stream during expected movement, the stream is requested and delivered as expected (function <NUM>). Video may be obtained from a video-on-demand (VOD) service, a remote storage digital video recorder (RSDVR), from a traditional service on network <NUM>, and/or from any other source.

If the viewer requests a media stream to be delivered during expected movement of device <NUM> (function <NUM>), then any "dead zones" or areas of anticipated poor signal quality are identified (function <NUM>). When subsequent position reports indicate that the device is approaching a dead zone (function <NUM>), then further analysis can be performed to reduce or avoid any adverse effects of the dead zone. If no dead zones are identified or encountered, then adaptive streaming can continue as normal (function <NUM>).

If the device <NUM> is approaching a dead zone (function <NUM>), however, then the device <NUM> determines whether enough time remains to pre-fetch higher quality media segments while sufficient bandwidth remains (function <NUM>). If enough time remains, then upcoming segments of the media stream can be pre-fetched at higher quality and cached in storage <NUM> or elsewhere for later playback (function <NUM>). The particular segments that are needed will typically be identified based upon playback time of the media stream in comparison to the expected duration of the dead zone during the device's movement. That is, if the device <NUM> anticipates a two minute dead zone, then at least two minutes of programming can be pre-fetched (assuming time allows before entering the dead zone).

If there is not sufficient time to pre-fetch needed segments (function <NUM>), then the device <NUM> determines whether enough content is currently in the buffer to traverse the dead zone without additional content (function <NUM>). If so, then payback can continue and segment requests may be delayed until later to make more efficient use of the limited bandwidth (function <NUM>). Otherwise, various embodiments may reduce the playback rate (function <NUM>) to stretch the currently-available content, or take other remedial actions as necessary to maintain the media stream, even at a lower quality. Other embodiments may simply pause playback and/or notify the viewer that better quality playback will resume when network conditions permit.

As noted above, the concepts set forth herein are very compatible with adaptive media streaming techniques in which "streamlets" or "segments" of relatively fixed duration are selected by the player from multiple copies of different quality available from a media server on network <NUM>. If then-current conditions are insufficient to support playback at the current quality (function <NUM>), then the media player adapts its requests sent to the server so that lower quality segments are obtained (function <NUM>). Conversely, if network conditions improve, then better quality segments can be requested, as appropriate.

Many of the examples described herein describe systems and processes that could be executed locally by the phone, tablet or other device to improve media segment requests made by that device in subsequent media streaming. Equivalent embodiments could process route prediction and/or signal quality prediction using a "cloud based" or similar network server, as desired. In various embodiments, the location/quality data is gathered from one or more devices (e.g., crowdsourced) to facilitate even better prediction of signal strength along an upcoming route. Predicted signal strength data and/or a plan of segment requests could be forwarded from the network service to the relevant device to reduce the computational load on the device, as desired.

<FIG> shows an example process <NUM> that could be executed by a mobile device <NUM> and/or by a network server <NUM> to plan and deliver a media stream based upon an expected travel path of the mobile device. To that end, process <NUM> could be performed locally by processor <NUM> of the device <NUM>, or remotely as part of a "cloud-based service" provided by one or more processors of network server <NUM>. In some embodiments, the various functions shown in <FIG> could be divided between device <NUM> and service <NUM>, with appropriate messages passed via network <NUM> to facilitate cooperative processing as desired. Still other embodiments may provide alternate, additional or different functions that could be organized and executed in any other equivalent manner.

Process <NUM> is initiated in any manner (function <NUM>). As noted above, location prediction may be performed within a media player application executing on device <NUM>, which would typically indicate that the process is initiated when the application is opened, or otherwise made active. Other embodiments may be initiated when a media stream is requested by the user. Still other embodiments may run as daemons or other background processes for more universal execution even when the media player application may not be active.

As noted above, the expected path <NUM> of device <NUM> may be determined in any manner (function <NUM>), and expected quality measurements can be obtained for various points along the path <NUM>. In various embodiments, signal quality is predicted based upon any combination of local data <NUM> stored on device <NUM> and remote data <NUM> stored on the cloud service database <NUM>. In some embodiments, it may be particularly helpful to recognize "dead zones" so that sudden drops in video quality can be avoided. In other embodiments, it may be desirable to plan for some or all of the journey so that locations of greatest signal quality/bandwidth can be exploited for maximum segment requests, while segment requests are reduced or even suspended while the device <NUM> is located in areas with lower signal quality/bandwidth. Segments of the media stream are then requested by the media device <NUM> according to the plan (function <NUM>).

In various embodiments, it may be helpful to monitor actual segment delivery (function <NUM>) in comparison to the plan in case updates are needed. If signal quality differs from the expected values, for example, segment requests may be delayed, or if signal quality improves, then perhaps video quality can be improved, or segment request timing may be altered to make use of the newly-available bandwidth.

Still further embodiments may monitor location data collected by device <NUM> to track any variation from the expected location that may necessitate changes to the video streaming plan. If the user takes a different route as part of a morning or evening commute, for example, then different dead zones may occur. If the device <NUM> is found to be in an unknown location (or a location in which no previous signal strength data <NUM> is available locally), then device <NUM> may query service <NUM> to request signal strength data in some embodiments. The requested data may then be used to create a new segment request plan, as desired.

Claim 1:
A mobile device having a processor (<NUM>), memory (<NUM>) and input/output interfaces (<NUM>), wherein the processor is configured to execute instructions stored in the memory to:
measure, by the mobile device, a signal quality of a wireless signal encountered at various locations as the mobile device visits the various locations;
store, in the memory (<NUM>) of the mobile device, data indicative of the signal quality of the wireless signal encountered at the various locations;
determine (<NUM>), by the processor, an expected path of the mobile device;
retrieve (<NUM>) from the memory (<NUM>), by the processor, the data indicative of the signal quality of the wireless signal encountered at each of a plurality of the various locations along the expected path of the mobile device;
process, using the processor, the data to identify one or more locations along the expected path of the mobile device having reduced signal quality; and
adapt a schedule of requests for segments of a video stream based upon the processing to thereby reduce data delivery while the device is near the one or more locations along the expected path of the mobile device having reduced signal quality;
wherein received segments of the video stream are stored in a buffer of the mobile device and adapt comprises:
compare an expected duration of a dead zone of reduced signal quality to the playback time of the media stream and determine if sufficient time remains to pre-fetch one or more segments of the media stream before the mobile device enters the dead zone,
if sufficient time remains, pre-fetch the one or more segments for playback through the expected duration of the dead zone,
otherwise determine if the buffer contains enough of the media stream to maintain playback through the expected duration of the dead zone,
if the buffer contains enough of the media stream, maintain playback through the expected duration of the dead zone,
otherwise adapt a playback rate of the media stream while the mobile device traverses the dead zone.