Self-organized storage nodes for distributed delivery network

A distributed delivery network for capacity enhancement of a communication link shared by multiple communication devices for network access service. The distributed delivery network may include one or more distributed storage devices, some of which may include at least one rotating disk storage device, a network interface, and one or more environmental sensors. Each distributed storage device may monitor data from the environmental sensor(s) and transition between an active state where messages are stored in or retrieved from the storage device, and a standby state where access is suppressed. The distributed storage devices may self-organize control operations for the distributed delivery network including message storage and retrieval and redundancy of messages, which may be determined by frequency of requests for the messages.

BACKGROUND

Field of the Disclosure

The present disclosure relates to broadband communications in general, and in particular, to a distributed delivery network for providing capacity enhancement in broadband communications networks.

Relevant Background

As the use of communications and networking continues to grow around the world, users are demanding a high-quality broadband experience even in remote areas or mobile environments. The ability to provide a high-quality broadband experience in these environments presents many challenges.

SUMMARY

Methods, systems, and devices are described for providing capacity enhancement in broadband communication networks. In some examples, the effective capacity of a communication link between a network and multiple communication devices is enhanced by using a distributed delivery network to store messages for subsequent delivery to the communication devices. The communication devices may receive network access service via a network gateway device that communicates over the shared communication link. The distributed delivery network may include one or more distributed storage devices, which may be connected with the network gateway device via a local network.

In some examples, each distributed storage device includes at least one rotating disk storage device, a network interface for communicating with the network gateway device, and one or more environmental sensors. Each distributed storage device may include a control circuit to manage operation of the distributed storage device. The control circuit may monitor data from the environmental sensor(s) and transition the distributed storage device between an active state where messages are stored in or retrieved from the rotating disk storage device, and a standby state where access of the rotating disk storage device is suppressed to protect the rotating disk storage device.

The distributed delivery network may provide control of message flow and message redundancy for messages requested by the communication devices via the network gateway device. For example, redundancy of messages on the distributed delivery network may be determined by frequency of request of the messages. The distributed storage devices may self-organize control operations for the distributed delivery network. In some examples, one distributed storage device may operate as a leader device for message flow in the distributed delivery network. The leader device may monitor the operational states of distributed storage devices and maintain an index of messages stored in the distributed storage devices for retrieval. The leader device may also monitor message flow via the network gateway device and manage redundancy for messages stored in the distributed delivery network. The leader device may be statically defined or may be dynamically selected based on a predetermined or pseudo-random selection scheme.

A communications system including a network gateway device and a plurality of distributed storage devices coupled to the network gateway device is described. In some aspects, a distributed storage device of the plurality of distributed storage devices includes a housing, a rotating disk storage device within the housing, a network interface to support communication with the network gateway device, a sensor to produce a sensor signal indicating a characteristic of an environment external to the housing, and a control circuit to detect an adverse operating condition for the rotating disk storage device based on the sensor signal, and to transition the distributed storage device from an active state to a standby state upon detection of the adverse operating condition. When in the active state, the distributed storage device may enable access of the rotating disk storage device to store messages received via the network interface, to retrieve the stored messages in response to requests received via the network interface, and to provide a given stored message of the stored messages for redundancy of storage within other distributed storage devices of the plurality of distributed storage devices based on a frequency of requests for the given stored message. When in the standby state, the distributed storage device may prevent access to the rotating disk storage device.

In some examples, upon transitioning from the active state to the standby state, the distributed storage device transmits an indication that the distributed storage device is unavailable. The indication that the distributed storage device is unavailable may be, for example, broadcast over a storage area network coupled with the network gateway device and the plurality of distributed storage devices. In some examples, the transition of the distributed storage device from the active state to the standby state is performed independently of an operational state of at least one other distributed storage device of the plurality of distributed storage devices.

According to some aspects, when in the standby state the distributed storage device configures the rotating disk storage device in a protected state. The protected state may include, for example, one or more of a parked media state, a deactivated state, or an unpowered state.

According to some aspects, when in the standby state the control circuit may detect a safe operating condition for the rotating disk storage device based on the sensor signal and transition the distributed storage device from the standby state to the active state upon detection of the safe operating condition.

In some examples of the communication system, the control circuit may determine that the distributed storage device is operating as a leader device for the plurality of distributed storage devices. The control circuit may, for example, determine that the distributed storage device is operating as the leader device for the plurality of distributed storage devices based on a result of a consensus election.

In some examples, the control circuit may, while the distributed storage device is operating as the leader device, determine a change for an index of the stored messages across the plurality of distributed storage devices, and broadcast the change for the index to the plurality of distributed storage devices.

In some examples, the control circuit may determine that the distributed storage device is no longer operating as the leader device for the plurality of distributed storage devices. The control circuit may monitor broadcast messages for indications of changes to the index an update the index upon based on received indications of changes to the index.

In some examples, the control circuit may, while the distributed storage device is operating as the leader device, monitor for periodic status messages from the plurality of distributed storage devices, determine that a second distributed storage device from the plurality of distributed storage devices is unavailable based upon not receiving a periodic status message from the second distributed storage device, and flag one or more messages stored in the second distributed storage device as unavailable in the index.

In some examples, the control circuit may, while the distributed storage device is operating as the leader device, receive a request for a first message, determine that the first message is stored in a first distributed storage device of the plurality of distributed storage devices, retrieve the first message from the first distributed storage device, and provide the first message in response to the request.

In some examples, the control circuit may, while the distributed storage device is operating as the leader device, receive a request for a first message, determine that the first message is stored in a first distributed storage device of the plurality of distributed storage devices, and provide a proxy address of the first message at the first distributed storage device in response to the request.

In some examples, the control circuit may, while the distributed storage device is operating as the leader device, determine an amount of redundancy for storing a the given stored message in the plurality of distributed storage devices based on the frequency of requests, and forward the given stored message for storing in one or more distributed storage devices of the plurality of distributed storage devices based on the determined amount of redundancy.

According to some aspects, the plurality of distributed storage devices includes a first set of distributed storage devices having a first reliability characteristic and a second set of distributed storage devices having a second, lower reliability characteristic. In some examples, the control circuit may, while the distributed storage device is operating as the leader device, determine that the frequency of requests for the given stored message exceeds a threshold, and forward the given stored message for storing in a distributed storage device of the first set of distributed storage devices.

In some examples, the control circuit may receive information related to an impending adverse operating condition for the rotating disk storage device and transition the distributed storage device from the active state to the standby state based on the impending adverse operating condition.

In some examples, the control circuit may detect a failure of the rotating disk storage device and broadcast an indication that the distributed storage device is unavailable based on the detected failure.

According to some aspects, the control circuit may perform a periodic diagnostic scan on the rotating disk storage device. In some examples, the control circuit may determine one or more storage locations of the rotating disk storage device failing the periodic diagnostic scan, identify one or more messages stored in the one or more storage locations, broadcast an indication that the one or more messages are unavailable, and flag the one or more storage locations as unavailable for further use.

According to some aspects, the network interface comprises any of a wired networking interface, a wireless networking interface, or combinations thereof.

According to some aspects, the distributed storage device includes a fitting mounted to the housing for coupling with an external bracket, and a release lever coupled with the fitting to release the fitting from the external bracket.

According to some aspects, the network gateway device provides network access service for a mobile environment via a shared wireless communication link. The mobile environment may be, for example, an aircraft, and the plurality of distributed storage devices may be located on the aircraft. The shared wireless communication link may be a satellite communications link.

According to some aspects the sensor may include any of an inertial measurement sensor, a gyroscope, a temperature sensor, or combinations thereof.

A method for enhancing capacity of a communication system is described. The method may include providing a distributed storage device of a plurality of distributed storage devices coupled to a network gateway device in the communications system. The method may include operating the distributed storage device in an active state, the operating including providing access of a rotating disk storage device to store messages received via a network interface, retrieving the stored messages in response to requests received via the network interface, and providing a given stored message of the stored messages for redundancy of storage within other distributed storage devices of the plurality of distributed storage devices based on a frequency of requests for the given stored message. The method may include detecting an adverse operating condition for the rotating disk storage device of the distributed storage device based on a sensor signal from a sensor of the distributed storage device indicating a characteristic of an environment of the distributed storage device, transitioning the distributed storage device from the active state to a standby state upon detection of the adverse operating condition, and preventing, in the standby state, access to the rotating disk storage device.

According to some aspects, the method includes transmitting, upon transitioning from the active state to the standby state, an indication that the distributed storage device is unavailable. The method may include configuring, in the standby state, the rotating disk storage device in a protected state.

According to some aspects, the method includes detecting, in the standby state, a safe operating condition for the rotating disk storage device based on the sensor signal, transitioning the distributed storage device from the standby state to the active state upon detection of the safe operating condition.

According to some aspects, the method includes determining that the distributed storage device is operating as a leader device for the plurality of distributed storage devices. The determining that the distributed storage device is operating as the leader device for the plurality of distributed storage devices may be based on a result of a consensus election.

According to some aspects, the method includes, when the distributed storage device is operating as a leader device for the plurality of distributed storage devices, determining a change for an index of the stored messages across the plurality of distributed storage devices, and broadcasting the change for the index to the plurality of distributed storage devices.

According to some aspects, the method includes determining that the distributed storage device is no longer operating as the leader device for the plurality of distributed storage devices, monitoring broadcast messages for indications of changes to the index, and updating the index upon based on received indications of changes to the index.

According to some aspects, the method includes, when the distributed storage device is operating as a leader device for the plurality of distributed storage devices, monitoring for periodic status messages from the plurality of distributed storage devices, determining that a second distributed storage device from the plurality of distributed storage devices is unavailable based upon not receiving a periodic status message from the second distributed storage device, and flagging one or more messages stored in the second distributed storage device as unavailable in the index.

According to some aspects, the method includes, when the distributed storage device is operating as a leader device for the plurality of distributed storage devices, receiving a request for a first message, determining that the first message is stored in a first distributed storage device, retrieving the first message from the first distributed storage device, and provide the first message in response to the request.

According to some aspects, the method includes, when the distributed storage device is operating as a leader device for the plurality of distributed storage devices, receiving a request for a first message, determining that the first message is stored in a first distributed storage device, and providing an address of the first message at the first distributed storage device in response to the request.

According to some aspects, the method includes, when the distributed storage device is operating as a leader device for the plurality of distributed storage devices, determining an amount of redundancy for storing the given stored message in the plurality of distributed storage devices based on the frequency of requests, and forwarding the given stored message for storing in one or more distributed storage devices of the plurality of distributed storage devices based on the determined amount of redundancy.

According to some aspects, the method includes receiving information related to an impending adverse operating condition for the rotating disk storage device and transitioning the distributed storage device from the active state to the standby state based on the impending adverse operating condition.

According to some aspects, the method includes detecting a failure of the rotating disk storage device, and broadcasting an indication that the distributed storage device is unavailable based on the detected failure.

According to some aspects, the method includes performing a periodic diagnostic scan on the rotating disk storage device. In some examples, the method includes determining one or more storage locations of the rotating disk storage device failing the periodic diagnostic scan, identifying one or more messages stored in the one or more storage locations, broadcasting an indication that the one or more messages are unavailable, and flagging the one or more storage locations as unavailable for further use.

Further scope of the applicability of the described systems, methods, apparatuses, or computer-readable media will become apparent from the following detailed description, claims, and drawings. The detailed description and specific examples are given by way of illustration only, since various changes and modifications within the scope of the description will become apparent to those skilled in the art.

DETAILED DESCRIPTION

The described feature generally relate to capacity enhancement of a shared communication link using a distributed delivery network. The distributed delivery network may include multiple distributed storage devices, which may each include a rotating disk storage device. The distributed delivery network may self-organize for managing message flow and redundancy while distributed storage devices may enter and leave the distributed delivery network at any time.

This description provides examples, and is not intended to limit the scope, applicability or configuration of embodiments of the principles described herein. Rather, the ensuing description will provide those skilled in the art with an enabling description for implementing embodiments of the principles described herein. Various changes may be made in the function and arrangement of elements.

Thus, various embodiments may omit, substitute, or add various procedures or components as appropriate. For instance, it should be appreciated that the methods may be performed in an order different than that described, and that various steps may be added, omitted or combined. Also, aspects and elements described with respect to certain embodiments may be combined in various other embodiments. It should also be appreciated that the following systems, methods, devices, and software may individually or collectively be components of a larger system, wherein other procedures may take precedence over or otherwise modify their application.

The rise in consumption of network multimedia content such as systems providing audio and video on demand (AVOD) and use of audio and video sharing services (e.g., YouTube, Vine, etc.) has dramatically increased consumer bandwidth usage. In addition, users increasingly expect a high-quality broadband experience while travelling. For example, there is a growing demand for network access service during air travel, and the desire to be able to access network multimedia content on airplanes and in other mobile environments has created challenges in providing sufficient bandwidth for users over bandwidth-limited wireless networks.

In various mobile environments, multiple users may be provided network access service via a shared communication link. For example, users may connect their communication devices to a wireless local area network (WLAN), which routes data communications to other networks (e.g., the Internet) via the shared communication link. The shared communication link may be a wireless link (e.g., cellular link, satellite communications link, etc.), and may have limited communication bandwidth. For example, while the resources (e.g., frequency, time, etc.) of a wireless network may be flexibly applied for multiple users, the overall capacity of the network may be limited, and congestion can occur when users request more resources at a particular time than the network can support. Increasing bandwidth for wireless communications systems is expensive and sometimes additional usable spectrum is unavailable. Therefore, providing a high-quality network access experience at a reasonable cost in mobile environments presents many challenges.

FIG. 1shows an example communications environment100for capacity enhancement of network access service in accordance with various aspects of the disclosure. The communications environment100includes a network gateway device150-athat provides network access service to multiple communication devices115in a mobile environment105via a shared communication link110-a. The communication devices115may be mobile devices (e.g., smartphones, laptops, tablets, netbooks, and the like) and may communicate with the network gateway device150-avia communication links125. Communication links125may be, for example, part of a local network such as a WLAN supported by an access point165-a.

The shared communication link110-amay be a wireless communication link between the network gateway device150-alocated on the mobile environment105and one or more networks120, which may include any suitable public or private networks and may be connected to other communications networks such as the Internet, telephony networks (e.g., Public Switched Telephone Network (PSTN), etc.), and the like. The information rate for servicing the communication devices115may depend on the number of communication devices115in the mobile environment105and types of traffic requested by the users, and may vary over time. For example, the network traffic provided over the shared communication link110-amay include a mix of web traffic, streaming traffic, and other types of traffic. The types of network traffic that generally consume the most communication resources are file sharing and video communications (e.g., video conferencing, video streaming, etc.) with some estimates indicating that as much as 50% of all Internet traffic is video streaming traffic.

The probability distribution of requested content tends to be non-uniform. For example, the same streaming traffic may be requested by multiple users at different points in time, or may be shared across multiple users if they request broadcast or multicast traffic. In one specific example, the percentage of unique messages of one popular video library requested as a function of message requests is approximately log-linear, with 20% of the unique messages accounting for approximately 85% of the message requests. Similarly, content such as “viral” videos may be repeatedly requested by many different users within a span of days or even hours.

The capacity of shared communication link110-amay be flexible to meet the information rate demands of the communication devices115. However, the information rate provided by shared communication link110-amay have limits given by system bandwidth, hardware limitations (e.g., power constraints), or overall system capacity. For example, communications environment100may be part of a communications system, which may have a limited overall information rate capacity. In addition, the incremental cost for the information rate provided over shared communication link110-amay be relatively high. While it may be possible to restrict some types of traffic (e.g., high definition video, etc.), this may degrade the network access experience for various users. These and other issues of providing network access service in mobile environments typically result in high cost of service, a reduced quality user experience, or both.

One way to increase the effective information rate of the shared communication link110-ais to store some content locally to the mobile environment105. For example, when a communication device115makes requests for a set of messages from network120via the shared communication link110-a, the messages can also be stored in a storage system connected to network gateway device150-a. If the same set of messages are then later requested by other communication devices115, the stored messages can be provided from the local storage instead of being re-sent over the shared communication link110-a. The amount of effective information rate enhancement provided by locally stored content depends on the amount of storage, the reliability of the storage system, and the probability distribution of requested messages. To provide a significant effective increase in information rate (e.g., 2× or higher) provided to the communication devices115, the local storage capacity required may be large. For example, to cover a substantial percentage of streaming traffic (e.g., 10-50%), storage capacities in the 10-200 Terabyte (TB) range may be necessary.

Different types of storage media may have different trade-offs between storage capacity and reliability. For example, a rotating disk storage device (commonly known as a hard disk drive (HDD)) uses a rotating magnetic media platter with a read and write head that operates over the magnetic surface to detect and modify magnetization of the magnetic medium on the platter. Rotating disk storage devices have relatively high storage capacity while providing data access in any desired order (e.g., random access). A solid state drive (SSD) is a data storage device that uses non-volatile solid state (e.g., NAND flash memory, magnetoresistive random-access memory (MRAM), etc.) to persistently store data (maintain data when powered off). Compared with HDDs, SSDs are typically more resistant to physical shock, run silently, have lower access time, and less latency. In addition, SSDs may be more reliable than HDDs in some environments. For example, because HDDs have read/write heads that float above spinning platters, HDDs may be susceptible to shock and vibration, as well as other mechanical failure mechanisms. Because SSDs have no moving parts, mechanical failure rates for SSDs are generally low. However, SSDs may have a limited number of write cycles (or erase cycles), which may affect life-time of the drive in certain applications.

Higher cost is a substantial drawback to the use of SSDs for larger capacity storage applications. For example, consumer-grade SSDs may be approximately 10 times more expensive per unit of storage than consumer-grade HDDs. Although prices for SSDs are declining, the consumer price for SSDs may currently be in the range of $500-$1500 per TB of storage (depending on drive storage size) compared to approximately $40-80 per TB for HDDs. Additionally, the available storage capacity in a single HDD device is generally larger than that available for a single SSD device. For example, the largest available 3.5″ consumer HDDs may be in the 8-12 TB range, while the largest available SSDs, although a smaller form factor, may be in the 1-2 TB range. Thus, many more SSD devices may be necessary to provide a given total storage size.

To create a storage system using multiple drive devices, data virtualization techniques such as redundant array of independent disks (RAID) may be used to combine multiple drive devices into a single logical unit that stores data using redundancy techniques. A commonly used RAID technique is data mirroring where a set of data is written to two or more drive devices, and the data is available as long as at least one drive device is functional. Another RAID technique for use with more than two drive devices is striping with distributed parity information. Striping with distributed parity information may provide higher effective storage capacity for a given total storage size but is only tolerant of failure of a single drive device. Fault tolerance may be increased by increasing the parity information (e.g., double parity information for fault tolerance of up to two failed drives, etc.), at the expense of effective capacity.

In some cases, the mobile environment105may be subject to substantial shock and vibration, as well as other extreme environmental conditions. For example, the mobile environment105may be an aircraft, which may suffer from vibration due to engines and hydraulic systems, and may encounter turbulence that results in significant gravitational forces (g-forces) seen by components on the aircraft. Other mobile environments such as ships and trains may also see significant shock or vibration occur intermittently. Providing a robust and cost-effective local storage solution for increasing effective information rate for a shared communication link in these types of mobile environments thus presents many challenges.

In embodiments, the communications environment100is configured to increase the effective capacity between the network120and the mobile devices115by using a distributed delivery network130-ato store messages for subsequent delivery to communication devices115. The distributed delivery network130-aincludes one or more distributed storage devices140-a, which may be connected with the network gateway device150-avia a local network link155-a(e.g., storage area network (SAN), etc.). Each distributed storage device140-amay include at least one rotating disk storage device (e.g., HDD, etc.), a network interface (e.g., Ethernet, WLAN, etc.) for communicating with the network gateway device150-a, one or more environmental sensors such as an inertial measurement sensor (e.g., accelerometer, etc.), a gyroscope, a temperature sensor, and the like. Each distributed storage device140-amay include a control circuit to manage operation of the distributed storage device140-awithin the distributed delivery network130-a. The control circuit may monitor data from the environmental sensor(s) and transition the distributed storage device140-abetween an active state where messages are stored in or retrieved from the rotating disk storage device, and a standby state where access of the rotating disk storage device is suppressed to protect the rotating disk storage device. Operational state transitions may be performed independently and autonomously by each distributed storage device140-aaccording to the local environmental conditions experienced by the distributed storage device140-a, which may be different than environmental conditions experienced by other distributed storage devices140-a.

The distributed delivery network130-amay provide control of message flow and message redundancy for messages requested by the communication devices115via the network gateway device150-a. For example, redundancy of messages on distributed delivery network130-amay be determined according to a storage policy that is based on frequency of request of the messages. The distributed storage devices140-amay self-organize control operations for distributed delivery network130-a. In some examples, one distributed storage device140-amay operate as a leader device for message flow in the distributed delivery network130-a. The leader device may monitor the operational states of distributed storage devices140-aand maintain an index of messages stored in the distributed storage devices140-afor retrieval. The leader device may also monitor message flow via the network gateway device150-aand manage redundancy for messages stored in the distributed delivery network130-a. The leader device may be statically defined or may be dynamically selected based on a predetermined or pseudo-random selection scheme.

FIG. 2shows an example communications environment200for capacity enhancement of network access service in accordance with various aspects of the disclosure. The communications environment200includes a network gateway device150-bthat provides network access service to multiple communication devices (not shown) within mobile environment105via a shared communication link110-b. The shared communication link110-bmay be a wireless communication link between the network gateway device150-blocated on the mobile environment105-band networks120(e.g., private networks, the Internet, etc.). Communications over the shared communication link110-bmay be performed by a modem220, which may be integrated with the network gateway device150-b, or may be a separate component, in some examples.

In communications environment200, network gateway device150-bis connected to multiple wireless access points (WAPs)165-b(e.g., WAP165-b-1,165-b-2,165-b-m, etc.), which provide connection points for the communication devices to connect to network gateway device150-band request content via shared communication link110-b. WAPs165-bmay be distributed about the mobile environment105-b, and may provide traffic switching or routing functionality (e.g., as part of a WLAN extended service set (ESS), etc.).

As illustrated inFIG. 2, communications environment200includes distributed delivery network130-bto increase the effective capacity of shared communication link110-b. Distributed delivery network130-bincludes multiple distributed storage devices140-b(e.g.,140-b-1,140-b-2,140-b-n, etc.), which each may include the components of distributed storage devices140-adiscussed with reference toFIG. 1. Each distributed storage device140-bmay operate as a separate logical storage node on distributed delivery network130-b. The distributed storage devices140-bmay be distributed in different locations throughout the mobile environment105-b, and may communicate with network gateway device150-bvia one or more WAPs165-b. The distributed storage devices140-bmay also communicate directly with each other (e.g., peer-to-peer) for selecting a leader device and managing message flow in distributed delivery network130-b.

FIG. 3shows an example communications environment300for capacity enhancement of network access service in accordance with various aspects of the disclosure. The communications environment300includes a network gateway device150-cthat provides network access service to multiple communication devices (not shown) within mobile environment105-cvia a shared communication link110-c. The shared communication link110-cmay be a wireless communication link between the network gateway device150-clocated on the mobile environment105-cand networks120(e.g., private networks, the Internet, etc.). Communications over the shared communication link110-cmay be performed by a modem220-a, which may be integrated with the network gateway device150-c, or may be a separate component, in some examples. In communications environment300, network gateway device150-cis connected to multiple wireless access points (WAPs)165-c(e.g., WAP165-c-1,165-c-2,165-c-m, etc.), which may be the WAPs165-bdiscussed above with reference toFIG. 2.

As illustrated inFIG. 3, communications environment300includes distributed delivery network130-cto increase the effective capacity of shared communication link110-c. Distributed delivery network130-cincludes multiple distributed storage devices140-c(e.g.,140-c-1,140-c-2,140-c-n, etc.), which each may include the components of distributed storage devices140discussed with reference toFIG. 1 or 2. Each distributed storage device140-cmay operate as a separate logical storage node on distributed delivery network130-c. The distributed storage devices140-cmay be distributed in different locations throughout the mobile environment105-c, and may communicate with network gateway device150-bvia network155-c, which may be a wired network. In some examples, network155-cmay be a storage area network (SAN) (e.g., Ethernet, Fibre Channel, etc.). The distributed storage devices140-cmay also communicate directly with each other (e.g., peer-to-peer) via network155-cfor selecting a leader device and managing message flow in distributed delivery network130-c.

In some examples of communications environments100,200or300, messages may be pre-loaded or pushed to a distributed delivery network130from one or more content providers. For example, a content provider may be allocated a portion of a distributed delivery network130for pre-loading with highly-requested content. Distributed delivery network130may determine redundancy for the messages in the pre-loaded content. For example, distributed delivery network130may store messages that are more frequently requested (e.g., based on historical information of the content provider, etc.) with higher redundancy. In some examples, the pre-loaded or pushed content may be updated based on changes in frequency of requests for the content. For example, the distributed delivery network130may determine that certain content of the pushed content is seldom requested in the mobile environment105, and may adapt the portion of the content of the content provider that is pre-loaded based on the requests for content in the mobile environment105. In some examples, updating of the pre-loaded or pushed content may be performed during times of relatively low network activity (e.g., when an aircraft is not in flight, at night, etc.).

In some examples of communications environments100,200or300, distributed delivery networks130-bor130-cmay include one or more high-reliability or shock and vibration tolerant storage nodes. For example, a subset of distributed storage devices140may be configured to include solid-state storage media (e.g., SSD, etc.). A distributed storage device140with high-reliability or shock and vibration tolerant storage may operate as the leader device (e.g., manage message flow, manage redundancy of messages, maintain the index of storage messages for the distributed delivery networks130, etc.). In some embodiments, the high-reliability or shock and vibration tolerant device may be used to store the most requested messages (e.g., with or without redundancy in the other distributed storage devices140). In some examples, the network gateway device150may include the high-reliability storage device and may operate as the leader device for the distributed delivery network130. Additionally or alternatively, distributed storage devices140may include a combination of a rotating disk storage device and solid-state storage (e.g., hybrid drive, etc.). The solid-state storage may be used for maintaining the index of stored messages, or for the most highly requested content, in some cases.

In some examples of communications environments100,200or300, messages may be pre-fetched from a rotating disk storage device of distributed storage devices140and buffered in solid-state storage (e.g., RAM memory, SSD, etc.). For example, streaming traffic may be organized as an ordered list of messages such that when a streaming media file is requested, the messages that make up the file will generally be transferred in order (e.g., subject to packet flow and acknowledgement, etc.). Thus, for streaming media programming, distributed delivery network130can predict that the next messages in the ordered list for the streaming media will likely be requested (e.g., unless the user pauses or stops the stream, etc.). Turbulence or other events that cause shock may typically last from a few seconds to a few minutes. Therefore, distributed delivery network130may pre-fetch messages corresponding to several minutes or more of a requested streaming media file from the rotating disk storage device of a distributed storage device140and store the messages in the solid-state storage (e.g., in a high reliability node of the distributed delivery network, the solid state storage of a hybrid drive, etc.). If a turbulence event occurs, the distributed delivery network130can continue to provide the buffered messages even when the rotating disk storage device is disabled for protection. While the streaming media player at the user device may have its own buffer, the distributed delivery network130may pre-fetch a longer time-window for the streaming media programming, up to and including the full streaming media file. The distributed delivery network130may maintain the pre-fetched messages for a predetermined amount of time, or until an event occurs (e.g., the solid-state storage buffer is full with active message flows, the aircraft lands, etc.).

FIG. 4shows a diagram of an example distributed storage device140-dfor a distributed delivery network in accordance with various aspects of the disclosure. Distributed storage device140-dincludes rotating disk storage device (e.g., HDD, etc.)410, control circuit415, environmental sensor420, and network interface425. The components of distributed storage device140-dmay be included within a physical housing440that connects to a power bus450and network signaling455(e.g., wired or wireless, etc.). Each of these components may be in communication with each other.

The control circuit415may manage operation of the distributed storage device140-dfor operation in a distributed delivery network. For example, the control circuit415may manage operational states of the distributed storage device140-d, monitor sensor data from the environmental sensor420, and manage communication with a network gateway device, other distributed storage devices, or other devices (e.g., communication devices115ofFIG. 1) via network interface425. Network interface425may implement wired network interfaces (e.g., Ethernet, Fibre Channel, etc.) and/or wireless network interfaces (e.g., IEEE 802.11 compliant interfaces, etc.).

Rotating disk storage devices may have maximum operating specifications for vibration and shock, which are typically specified in acceleration within a given frequency range (vibration) or over a given time period (shock). Environmental sensor420may include an inertial measurement sensor (e.g., accelerometer, etc.), which may indicate when the distributed storage device140-dis experiencing high levels of vibration or shock. The control circuit415may filter the accelerometer data to determine the acceleration, velocity, or displacement, and may compare these values with one or more thresholds to determine if the vibration or shock experienced by the distributed storage device140-dexceeds operating thresholds for the rotating disk storage device410, which may be considered adverse environmental conditions for the rotating disk storage device410. Additionally or alternatively, environmental sensor420may include other sensors such as a gyroscope or a temperature sensor. The control circuit415may monitor data from these sensors to determine if environmental conditions (e.g., orientation, temperature, etc.) exceed operational tolerances. The control circuit415may transition the distributed storage device140-dfrom an active state to a standby state based on detection of adverse operating conditions. Additionally or alternatively, the control circuit415may receive information related to an impending adverse operating condition for the rotating disk storage device410and may transition the distributed storage device from the active state to the standby state based on the impending adverse operating condition. For example, the control circuit415may receive information from an aircraft data bus, which may include weather information indicating that impending turbulence is in the flight path. Additionally or alternatively, distributed storage devices may communicate local environmental conditions from different locations within the aircraft, which may indicate that vibrations or turbulence is increasing and an adverse environmental condition is impending.

FIG. 5shows an example operational state diagram500for a distributed storage device140in accordance with various aspects of the disclosure. For example, operational state diagram500may illustrate operational states for distributed storage device140-dofFIG. 4. The operational states of the distributed storage device140-dmay include an initial power on state505, an active state510, a standby state515, an error state520, and an exit state525. Transitions between states may be managed by the control circuit415. The distributed storage device140-dmay start in the power on state505when powered on. From the power on state505, the distributed storage device140-dmay transition to the active state and announce entry of the distributed storage device140-dfor operation on a distributed delivery network such as distributed delivery networks130ofFIGS. 1-3.

In the active state510, access to the rotating disk storage device410is enabled. For example, the distributed storage device140-dmay receive messages for storage via the network interface425in the active state and store the messages in the rotating disk storage device410. The distributed storage device140-dmay also receive requests for stored messages in the active state and may retrieve and deliver the stored messages in response to the requests.

In the active state, the control circuit415may monitor the sensor data from environmental sensor420and determine when environmental conditions for the distributed storage device140-dare adverse for operation of the rotating disk storage device410. If, while in the active state510, the control circuit415detects an adverse operating condition for the rotating disk storage device420, the control circuit415may transition the distributed storage device140-dto the standby state515. The control circuit415may also announce (e.g., broadcast, etc.) that the distributed storage device140-dis entering the standby state515and is no longer available for storing or delivering messages.

When the distributed storage device140-dis in the standby state515, access to the rotating disk storage device410is disabled and the rotating disk storage device410may be configured in a protected state. For example, the rotating disk storage device410may be configured in a parked media or deactivated state, or may be unpowered. In some examples, the distributed storage device140-dmay include a disk buffer, which may be solid state memory (e.g., RAM, etc.). When entering the standby state515, the distributed storage device140-dmay continue to deliver buffered messages based on requests received in the active state510. Additionally or alternatively, if the distributed storage device140-dincludes a hybrid drive, the distributed storage device140-dmay continue to allow access to the solid-state storage of the hybrid drive in the standby state515.

If an exception (e.g., recoverable error, etc.) occurs while in the active state510or the standby state515, the distributed storage device140-dtransitions to the error state520, where error recovery may be attempted. If error recovery in the error state520is successful, the distributed storage device140-dtransitions back to the standby state515. If error recovery in the error state520is unsuccessful, the distributed storage device140-dtransitions to the exit state525. Upon transitioning from the active state510to the error state520, the distributed storage device140-dmay announce that the distributed storage device140-dis no longer available for storing or delivering messages. The distributed storage device140-dmay also transition to the error state520if an unrecoverable error occurs while in the active state510or the standby state515. In this instance, the distributed storage device140-dmay also announce that the distributed storage device140-dis no longer available for storing or delivering messages.

In the standby state515, the control circuit415may continue to monitor the sensor data from environmental sensor420and determine when environmental conditions for the distributed storage device140-dare safe for operation. When safe operating conditions for the rotating disk storage device410are detected, the control circuit415may transition the distributed storage device140-dback to the active state510. The control circuit415may also announce (e.g., broadcast, etc.) that the distributed storage device140-dis entering the active state510and is again available for storing or delivering messages.

Returning toFIG. 4, the distributed storage device140-dmay include a power supply405, which may be coupled with a power bus450of a mobile environment, and may convert a supply voltage of the power bus450to internal voltages used by components of the distributed storage device140-d. The power supply405may also provide robustness to power surges and the capability of replacing the distributed storage device140-dwithout disrupting the power bus450or damaging the rotating disk storage device410.

The housing440may be configured to orient the rotating disk storage device in a horizontal orientation as installed, and may include vibration dampening bushings between the housing440and brackets of the mobile environment105. The housing may be configured with fittings that couple with the brackets and a release lever to release the fitting from the bracket. The power supply405may be configured to be connected to the power bus450while the power bus450is live (e.g., hot swappable).

As discussed above, the distributed storage devices140of distributed delivery networks130may operate according to a leader/follower topology for message flow management where one distributed storage device140acts as a leader device and the other distributed storage devices140act as follower devices. The leader device may monitor the operational states of follower devices and maintain a device state table that indicates which distributed storage devices140are in the active state and available for storing and retrieving messages, and which distributed storage devices140are unavailable (e.g., in the standby state or having left the distributed delivery network). The leader device may also manage storage of messages in the distributed storage devices140(e.g., manage redundancy of messages, etc.) and maintain an index of messages stored in the distributed storage devices140for retrieval. The leader device may broadcast changes in the device state table and message index, and follower devices may each maintain a separate copy of the device state table and message index. Thus, if a follower device becomes the leader (e.g., based on consensus election or if the leader enters the standby state, etc.), the new leader will have updated copies of the device state table and message index.

FIG. 6shows a flow diagram600of an example message flow for capacity enhancement of network access service in accordance with various aspects of the disclosure. Flow diagram600illustrates message flow between a network gateway device150-d, distributed storage devices140-d-1and140-d-nof distributed delivery network130-d, and communication devices115. While network gateway device150-dmay provide network access service for a number of communication devices115, only communication devices115-aand115-bare shown in flow diagram600for clarity. Flow diagram600may illustrate, for example, message flow in the communications environments100,200, or300ofFIGS. 1-3.

Communication device115-amay send a request605for message A. Message A may be, for example, a file or object that is part of video streaming programming (e.g., HTTP live streaming (HLS) file, etc.) requested from a content server of a network (e.g., network120ofFIGS. 1-3). The network gateway device150-dmay send a query610to distributed delivery network130-dto see if message A has been stored in the distributed delivery network130-d.

Distributed storage device140-d-1may currently be acting as the leader device for distributed delivery network130-d. In some examples, distributed storage device140-d-1may identify itself as the leader device to network gateway device150-dand network gateway device150-dmay communicate query610directly to distributed storage device140-d-1. Alternatively, network gateway device150-dmay broadcast query610to distributed delivery network130-d(e.g., over a SAN, etc.), and the leader device (e.g., distributed storage device140-d-1) may respond for the distributed delivery network130-d.

Distributed storage device140-d-1may determine (e.g., based on the message index for distributed delivery network130-d) that message A is not stored in distributed delivery network130-d. Distributed storage device140-d-1may send an indication615to network gateway device150-dthat message A is not stored in distributed delivery network130-d. Network gateway device150-dmay then retrieve message A over a shared communications link (e.g., shared communication links110of communications environments100,200, or300ofFIGS. 1-3) at620, and may deliver message A to communications device115-aat625.

Network gateway device150-dmay also send message A to distributed delivery network130-dat630. Acting as the leader device for distributed delivery network130-d, distributed storage device140-d-1may determine if message A should be stored in distributed delivery network130-d. If distributed storage device140-d-1determines that message A should be stored at635, distributed storage device140-d-1may send message A to distributed storage device140-d-nat640and may update the message index accordingly. Determining whether message A should be stored at635may be based on a message type for message A (e.g., determining that message A is a type of message for which a local copy can be delivered in response to a subsequent request, etc.).

Subsequently, communication device115-bmay send a request645for message A. The network gateway device150-dmay send a query650to distributed delivery network130-dto see if message A has been stored in the distributed delivery network130-d. At655, distributed storage device140-d-1may locate message A in distributed storage device140-d-n. In some cases, a content provider may also be involved in determining that a message is stored in the distributed delivery network130-d. For example, where a content provider has pre-loaded or pushed content to the distributed delivery network130-d, the content provider may send an index to the content in response to the request from the communication device115-b, and distributed storage device140-d-1may locate the content based on the content provider index.

Distributed storage device140-d-1may determine that distributed storage device140-d-nis available for message retrieval (e.g., in the active state, etc.) and request message A from distributed storage device140-d-nat660. Message A may be received from distributed storage device140-d-nat665. Distributed storage device140-d-1may provide message A to network gateway device150-dat670, and network gateway device150-dmay provide message A to communication device115-bat675, which may appear to communication device115-bas having come from the content server from which message A was requested.

Distributed storage device140-d-1may also determine redundancy for message A at655. Determining redundancy for message A at655may include determining how many copies of message A should be stored in distributed delivery network130-dand where to store the copies. Message redundancy may be performed according to a storage policy that is based on frequency of requests for message A. For example, distributed storage device140-d-1(as the leader device) may compare the number of requests (e.g., over a given time, etc.) for message A against one or more thresholds to determine the redundancy of storage of message A in distributed delivery network130-d. For example, for messages that have been requested only once, a single copy of the message may be stored in distributed delivery network130-d(e.g., no redundancy). For messages that have been requested a first number of times greater than a first threshold, two instances of the message may be stored in two different locations (e.g., on different distributed storage devices140-d, etc.) in distributed delivery network130-d. Additional thresholds may defined and associated with additional levels of redundancy, up to and including storing the message on each distributed storage device140-d.

FIG. 7shows a flow diagram700of an example message flow for capacity enhancement of network access service in accordance with various aspects of the disclosure. Flow diagram700illustrates message flow between a network gateway device150-e, distributed storage devices140-e-1and140-e-nof distributed delivery network130-e, and communication devices115. While network gateway device150-emay provide network access service for a number of communication devices115, only communication devices115-cand115-dare shown in flow diagram700for clarity. Flow diagram700may illustrate, for example, message flow in the communications environments100,200, or300ofFIGS. 1-3.

Block705of flow diagram700may include message flow actions corresponding to steps605to640of flow diagram600ofFIG. 6. For example, block705may include one or more requests for message A from communication devices115, and message A may be stored at least in distributed storage device140-e-n.

Subsequently, communication device115-dmay send a request745for message A. The network gateway device150-emay send a query750to distributed delivery network130-eto see if message A has been stored in the distributed delivery network130-e. At755, distributed storage device140-e-1(e.g., acting as leader device) may locate message A in distributed storage device140-e-nat755. Distributed storage device140-e-1may also determine redundancy for message A at755, as described above with reference to block655ofFIG. 6.

Distributed storage device140-e-1may determine that distributed storage device140-e-nis available for message retrieval (e.g., in the active state, etc.) and may provide a proxy address A′760for message A at distributed storage device140-e-n. Network gateway device150-dmay send a redirect message765for message A to communication device115-d, which may then request message A from distributed storage device140-e-nusing the proxy address A′ at770. Distributed storage device140-e-nmay provide message A to communication device115-dat775.

WhileFIGS. 6 and 7illustrate one of the distributed storage devices140acting as the leader device for distributed delivery networks130, the network gateway device150may instead act as the leader device. For example, the network gateway device150may monitor the operational states of the distributed storage devices140, manage storage and redundancy of messages in the distributed storage devices140and maintain an index of messages stored in the distributed storage devices140for retrieval.

As discussed above, distributed storage devices140may broadcast status messages to the other distributed storage devices140in the distributed delivery network130. For example, distributed storage devices140may announce entry into (e.g., upon power-on or error recovery) and exit from (e.g., unrecoverable error conditions, etc.) the distributed delivery network130and may periodically broadcast a status or “heartbeat” message that confirms the device is operational. Distributed storage devices140may also broadcast other information such as bad sectors, environmental conditions, and the like.

The leader device of a distributed delivery network (e.g., the distributed delivery networks130ofFIG. 1-3, 6 or 7) may maintain a device state table with current operational states of all distributed storage devices140.FIG. 8shows a diagram of an example device state table800in accordance with various aspects of the disclosure. Device state table800includes entries805for each distributed storage device810in a distributed delivery network (e.g., the distributed delivery networks130ofFIG. 1-3, 6 or 7). For each device810, the device state table800includes device status information820and may include other information830such as bad sectors for each device.

The leader device may receive broadcast status information (e.g., status messages, heartbeat messages, etc.) from each distributed storage device140and update device state table800accordingly. The leader may then (e.g., periodically, upon update, etc.) broadcast the values from device state table800, which may be mirrored in the follower devices in case the leader becomes unavailable and one of the follower devices becomes the leader.

As discussed above, the leader device may be selected in a variety of ways. For example, leader priority may be statically defined, or the leader may dynamically selected using peer-to-peer communication between distributed storage devices140. In some examples, the leader device is elected by a consensus operation of the distributed storage devices140currently in the active state.

FIG. 9shows an example dynamic leader selection state diagram900for a distributed delivery network in accordance with various aspects of the disclosure. Dynamic leader selection state diagram900may be used, for example, by a distributed storage device140operating on the distributed delivery networks130ofFIG. 1-3, 6 or 7.

At initial startup, the distributed storage device140may be initialized as a follower905. Election cycles may be performed on a periodic basis. Upon the next election cycle, the follower905may be a candidate910for leader. The election may be conducted according to a consensus algorithm (e.g., Paxos, Multi-Paxos, Raft, etc.). If the candidate910is elected by the consensus algorithm, the candidate will be the leader915until the leader915discovers a new leader and returns to being a follower905. If the leader becomes unavailable (e.g., transitions to the standby state, etc.) the leader may force an election, removing itself from being a candidate910. In some examples, the consensus algorithm may result in a statistical sharing of the operations associated with being the leader device. Because the leader device may see heavier use than other devices, multiplexing the leader device operations may reduce failure rates for the components of the distributed storage devices140(e.g., HDD failure rates, etc.).

FIG. 10illustrates a block diagram1000of a control circuit415-afor a distributed storage device in accordance with various aspects of the disclosure. The control circuit415-amay illustrate, for example, aspects of control circuit415ofFIG. 4. The control circuit415-aincludes device state manager1005, sensor monitor1010, communications manager1015, and disk controller1020. Each of these components may be in communication with each other, directly or indirectly.

The sensor monitor1010may perform functions associated with receiving sensor data and processing the sensor data (e.g., filtering, comparing sensor data with thresholds, etc.). The sensor monitor1010may provide information to the device state manager1005indicating whether the environmental conditions are safe or adverse for a rotating disk storage device of the distributed storage device.

Device state manager1005may manage operational states for the distributed storage device. For example, the device state manager1005may receive information from the sensor monitor1010relating to the current environmental conditions of the distributed storage device. If the sensor monitor1010indicates that the current environmental conditions are adverse for operation of a rotating disk storage device of the distributed storage device, the device state manager1005may transition the distributed storage device from an active state to a standby state. The device state manager1005may send a notification of the state transition (e.g., via communications manager1015). The device state manager1005may perform other operations for device state transitions described above with reference toFIG. 5.

The disk controller1020may perform operations associated with storing and retrieving messages from the rotating disk storage device. Additionally, the disk controller1020may perform operations for securing the rotating disk storage device if the distributed storage device enters the standby state because of adverse environmental conditions. For example, when the distributed storage device enters the standby state, the disk controller1020may configure the rotating disk storage device in a protected state such as a parked media state, a deactivated state, or an unpowered state.

FIG. 11illustrates a block diagram1100of a control circuit415-bfor a distributed storage device in accordance with various aspects of the disclosure. The control circuit415-bmay illustrate, for example, aspects of control circuits415ofFIG. 4orFIG. 10. The control circuit415-bincludes device state manager1005-a, sensor monitor1010-a, communications manager1015-a, disk controller1020-a, follower controller1125, message index manager1130, and leader controller1135. Each of these components may be in communication with each other, directly or indirectly. Device state manager1005-a, sensor monitor1010-a, communications manager1015-a, and disk controller1020-amay perform the functions of device state manager1005, sensor monitor1010, communications manager1015, disk controller1020ofFIG. 10, respectively.

Follower controller1125may control functions related to device state table and message index updates when the distributed storage device is operating as a follower device in the distributed delivery network. For example, follower controller1125may receive updates to the device state table and message index and update the corresponding entries (e.g., via message index manager1130, etc.). Follower controller1125may also manage participation of the distributed storage device in the dynamic selection of leader devices (e.g., consensus election process as described above with reference toFIG. 9, etc.).

Leader controller1135may control functions related message flow and maintaining the operational state table and message index when the distributed storage device is operating as the leader device in the distributed delivery network. For example, leader controller1135may include message flow manager1145, which may receive requests for messages (e.g., from a network gateway device150, etc.), determine if the requested messages are stored in the distributed delivery network (e.g., based on the message index, etc.), and retrieve and deliver stored messages in response to the requests. Additionally or alternatively, message flow manager1145may forward a proxy address of the stored messages for redirection to the stored messages.

Leader controller1135may also monitor for periodic status messages from distributed storage devices of the distributed delivery network and update the operational state table if any of the distributed storage devices have not sent the periodic status messages or have otherwise indicated themselves as unavailable (e.g., transition to standby mode, etc.). Additionally, leader controller1135may update the message index to indicate that instances of messages stored at unavailable storage nodes are unavailable.

Leader controller1135may include redundancy manager1140, which may manage redundancy for messages in distributed delivery network. For example, if a number of requests for a given stored message exceeds a threshold, redundancy manager1140may increase redundancy for the message. Leader controller1135may forward the message to a different distributed storage device for redundant storage of the message. Redundancy manager1140may determine redundancy levels using multiple thresholds based on the non-uniform probability density of messages. For example, redundancy manager1140may determine a storage policy for redundancy of messages by optimizing an information rate provided by the distributed delivery network based on the non-uniform probability density of messages and parameters of the distributed delivery network (e.g., number of distributed storage devices, total storage, reliability metrics of the distributed storage devices, etc.). Alternatively, redundancy manager1140may determine the storage policy as a normalized curve fit (e.g., linear, log-linear, polynomial) of the non-uniform probability density of messages or by using heuristic approaches providing a quasi-optimal information rate (e.g., gradient ascent, monte carlo, simulating annealing, etc.).

FIG. 12shows a simplified diagram of an example satellite communications system1200which may implement capacity enhancement using a distributed delivery network in accordance with various aspects of the disclosure. The satellite communication system1200includes a satellite1205(or multiple satellites1205), a ground station1215, a ground station antenna system1210, and a network gateway device150-f. In operation, the satellite communication system1200provides network access service via the network gateway device150-fto multiple communication devices115. For example, the satellite communication system1200may provide for two-way communications between the network gateway device150-fand a network120via the satellite1205and the ground station1215. The communication devices115may be connected to the network gateway device150-fvia one or more access points165-f(e.g., WAPs165, etc.).

The satellite or satellites1205may include any suitable type of communication satellite. In some examples, some or all of the satellites may be in geostationary orbits. In other examples, any appropriate orbit (e.g., medium earth orbit (MEO), low earth orbit (LEO), etc.) for satellite1205may be used. In one embodiment, the satellite1205operates in a multi-beam mode, transmitting a number (e.g., typically 20-150, etc.) of spot beams each directed at a different region of the earth. This can allow coverage of a relatively large geographical area and frequency re-use within the covered area. Spot beams for communication with subscribers may be called service beams while spot beams for communication with gateways such as ground station1215may be called feeder beams. In embodiments, the service beams are fixed location spot beams, meaning that the angular beam width and coverage area for each service beam does not intentionally vary with time.

The ground station1215sends and receives signals to and from the satellite1205via communication link1240using the ground station antenna system1210. The ground station antenna system1210may be two-way capable and designed with adequate transmit power and receive sensitivity to communicate reliably with the satellite1205. The ground station1215is connected to the one or more networks120, which may be the networks discussed with reference toFIGS. 1-3.

The network gateway device150-fmay use an antenna1240to communicate signals with the satellite1205via the communication link110-d. The antenna1240may be mounted to an elevation and azimuth gimbal which points the antenna1240(e.g., actively tracking) at satellite1205. The satellite communications system1200may operate in the International Telecommunications Union (ITU) Ku, K, or Ka-bands, for example from 17.7 to 21.2 Giga-Hertz (GHz). Alternatively, satellite communications system1200may operate in other frequency bands such as C-band, X-band, S-band, L-band, and the like. As illustrated inFIG. 12, network gateway device150-fand antenna1240are mounted on an aircraft1260. However, network gateway device150-fand antenna1240may be used in other applications besides onboard the aircraft1260, such as onboard boats, vehicles, or a stationary location where network access service is desired (e.g., a business, a school, etc.), in some cases.

The satellite communications system1200may also include one or more subscriber terminals1230, which may also be provided network access service via satellite1205and ground station1215. Each subscriber terminal1230is located within at least one service beam and is capable of two-way communication with the satellite1205via an antenna1225. Each subscriber terminal1230may be connected with (e.g., may provide network access service for) one or more customer devices1235(e.g., desktop computers, laptops, set-top boxes, smartphones, tablets, Internet-enabled televisions, and the like). These customer devices1235may also be known as customer premises equipment (CPE).

Typically, satellite communications system1200may have a fixed capacity based on a system bandwidth and a number of spot beams for frequency re-use. While that capacity may be allocated to subscriber terminals1230and mobile devices115in a variety of ways, it may not always be possible to provide a high data rate to all subscriber terminals1230and mobile devices115at the same time via signals transmitted over the satellite links110-dand1250.

In embodiments, the satellite system1200is configured to provide a high-quality network access experience to mobile devices115by increasing the capacity of the communications system1200using a distributed delivery network130-fas described above. The distributed delivery network130-fmay include multiple distributed storage devices140(e.g., distributed storage devices1404-1,140-f-2,140-f-n, etc.) that store messages requested by mobile devices115for delivery to other mobile devices in response to subsequent requests. The distributed storage devices140may be connected to the network gateway device150-fand each other via a network155-f(e.g., SAN155-f, etc.). The distributed delivery network130-fmay perform the functions described above for distributed delivery networks130ofFIG. 1-3, 6 or 7, which are not repeated here for the sake of brevity.

As discussed above, distributed storage devices140may be distributed throughout a mobile environment such as aircraft1260.FIG. 13shows a diagram1300illustrating example locations for installation of distributed storage devices140on an aircraft1260-ain accordance with various aspects of the disclosure. In diagram1300, distributed storage devices140-g-1and140-g-2are located in the roof of the cabin of the aircraft1260-a, above the interior paneling, which is generally removable for access. Distributed storage devices140-g-1and140-g-2may be located near WAP165-g, and may be connected to a network gateway device150(not shown) on aircraft1260-avia wired or wireless connections (e.g., via WAP165-g, etc.). For simplicity diagram1300illustrates only two distributed storage devices140-g, however, it should be understood that additional distributed storage devices140-gmay be distributed in other locations throughout aircraft1260-a.

In some cases, locating distributed storage devices140in various locations throughout aircraft1260-amay provide for higher tolerance of environmental conditions such as turbulence because different portions of aircraft1260-amay experience turbulence in different manners. For example, as an aircraft experiences shock, pitch, or yaw movement, different portions of the aircraft have different trajectories. In addition, an aircraft fuselage may experience flex, which may result in different levels of shock or vibration at different locations within the aircraft cabin. By distributing the distributed storage devices140-gthroughout the aircraft1260-awhere the local environmental conditions can vary significantly, the capacity enhancement provided by the network of distributed storage devices140-gcan be relatively robust. In other words, although the capacity enhancement can change over time as individual distributed storage devices140-gautonomously transition between active and standby states, the variance in local environmental conditions reduces the likelihood that all or most of them will transition into the standby state at the same time.

FIG. 14shows a diagram1400illustrating alternative example locations for installation of distributed storage devices140on an aircraft1260-bin accordance with various aspects of the disclosure. In diagram1400, distributed storage device140-his located underneath seat1410in the cabin of aircraft1260-b. Distributed storage device140-hmay be connected to a network gateway device150(not shown) on aircraft1260-bvia wired or wireless connections (e.g., via a WAP165, etc.). For simplicity, diagram1400illustrates only one distributed storage devices140-h, however, it should be understood that additional distributed storage devices140-gmay be distributed in similar locations throughout aircraft1260-b. In addition, a system may use the illustrated locations for distributed storage devices140of diagrams1300and1400in combination, as well as additional locations within an aircraft1260.

In some embodiments, distributed storage devices140include quick-release features to allow for replacement of non-functional distributed storage device140(e.g., due to failure of the rotating disk storage device, etc.). For example, the distributed storage devices140may include a housing with a fitting mounted to the housing for coupling with an external bracket mounted to the aircraft. A release lever may be coupled with the fitting to release the fitting from the external bracket. The distributed delivery network130may indicate (e.g., via a network gateway device150, etc.) distributed storage devices140that are to be replaced by sending messages to a service center. A technician may retrieve the information and replace the distributed storage devices140when servicing the aircraft. The non-functional distributed storage devices140may be refurbished (e.g., by installing a new rotating disk storage device, etc.) for future use.

FIG. 15shows a flowchart diagram of an example method1500for capacity enhancement of network access service in accordance with various aspects of the disclosure.

At block1505of method1500, a distributed storage device of a communications system may be provided. The communications system may include, for example, multiple distributed storage devices coupled to a network gateway device. For example, the distributed storage devices140may be implemented in a distributed delivery network130as described above with reference toFIG. 1-3, 6, 7 or 12.

At block1510, the distributed storage device may detect whether an adverse operating condition for a rotating disk storage device of the distributed storage device is present based on a sensor signal from a sensor of the distributed storage device indicating a characteristic of an environment of the distributed storage device. If an adverse operating condition is not present (e.g., the operating conditions are determined to be safe for the rotating disk storage device), the distributed storage device may stay in or transition to the active state1515.

In the active state1515, the distributed storage device may provide access to the rotating disk storage device for operations of the distributed delivery network. For example, at block1520a message may be received for storage in the distributed storage device and the distributed storage device may store the message at block1525. The distributed storage device may receive a request for the stored message at block1530and may retrieve and provide the message in response to the request at block1535. The request for the message received at block1530may, for example, be for redundancy of storage within other distributed storage devices of the distributed delivery network (e.g., based on a request from a leader device or the network gateway device, based on a frequency of requests for the given stored message, etc.).

The distributed storage device may continue to monitor the sensor signal in the active state1515at block1540, and determine whether adverse operating conditions exist at block1510. Based on detection of an adverse operating condition at block1510, the distributed storage device may transition to the standby state1545. Upon transitioning to the standby state1545, the distributed storage device may transmit (e.g., broadcast, etc.) an indication that the distributed storage device is unavailable to other devices of the distributed delivery network.

In the standby state1545, the distributed storage device may prevent access to the rotating disk storage device at block1550. For example, the distributed storage device may configure the rotating disk storage device in a protected state (e.g., parked media state, deactivated, unpowered, etc.). In the standby state1545, the distributed storage device may continue to monitor the sensor signal at block1555, and determine whether the adverse operating conditions still exist at block1510.

FIG. 16shows a flowchart diagram of an example method1600for capacity enhancement of network access service in accordance with various aspects of the disclosure. The method may be performed, for example, by a distributed storage device140as described above with reference toFIGS. 1-15.

At block1605, an election may determine a leader for a distributed delivery network. The consensus election may be performed periodically, be event driven (e.g., the leader becomes unavailable, etc.), and may be conducted according to a consensus algorithm (e.g., Paxos, Multi-Paxos, Raft, etc.).

If, at block1610, the distributed storage device140determines that it is the leader device as a result of the election at block1605, the distributed storage device140may perform functions associated with the leader device at blocks1615-1655. For example, the leader device may identify a new message for storage in the distributed delivery network at block1615and store the message in one of the distributed storage devices140at block1620. The leader device may receive a request for a stored message at block1625, and may locate the requested stored message at block1630. The leader device may retrieve the message, or may provide a proxy address for the requested stored message at block1630, in some cases.

At block1635, the leader device may determine message redundancy for the message. Determining redundancy for the message may include determining how many copies of the message should be stored in the distributed delivery network130and where to store the copies. Determining the redundancy may be based on a frequency of requests for the message, and may include multiple different levels of redundancy corresponding to multiple request thresholds as discussed above. The leader device may send the stored message to additional distributed storage devices based on the determined redundancy level.

After storing the message in one or more distributed storage devices at blocks1620or1635, the leader device may update the message index at block1650. The leader device may broadcast updates to the message index to the distributed storage devices for maintenance of back-up copies of the message index.

The leader device may monitor the status of other distributed storage devices at block1640. For example, the leader device may monitor for status messages from the follower devices and update an operational state table including entries and status conditions (e.g., active or standby state, etc.) of the other distributed storage devices. The leader device may also track other information such as bad sectors for each of the distributed storage devices. The leader device may broadcast the operational state table to the distributed storage devices for maintenance of back-up copies of the operational state table. The leader device may determine that a new election cycle is occurring at block1655and return to block1610to determine if it is still the leader device.

If, at block1610, the distributed storage device is not the leader device for the current election cycle, the distributed storage device may perform the functions of a follower device in blocks1660-1665. For example, the follower device may receive broadcast information at block1660including updates for the message index and operational state table, and may maintain a copy of the message index and operational state table. The follower device may determine that a new election cycle is occurring at block1665and return to block1610to determine if it will become the leader device. As a follower device, the distributed storage device may continue to perform the functions for sensor monitoring, state management, message storage, and message retrieval described with respect toFIG. 15.

Also, it is noted that the embodiments may be described as a process which is depicted as a flow diagram or block diagram. Although each may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be rearranged. A process may have additional steps not included in the figure. Processors may perform the necessary tasks. Features implementing functions may also be physically located at various positions, including being distributed such that portions of functions are implemented at different physical locations.

Furthermore, embodiments may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or combinations thereof. When implemented in software, firmware, middleware or microcode, the program code or code segments to perform the necessary tasks may be stored in a computer-readable medium such as a storage medium. A computer-readable medium may include, for example, RAM, ROM, EEPROM, flash memory, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code means in the form of instructions or data structures and that can be accessed by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor.

Having described several embodiments, it will be recognized by those of skill in the art that various modifications, alternative constructions, and equivalents may be used without departing from the spirit of the principles described herein. For example, the above elements may merely be a component of a larger system, wherein other rules may take precedence over or otherwise modify the application of the principles described herein. Also, a number of steps may be undertaken before, during, or after the above elements are considered. Accordingly, the above description should not be taken as limiting the scope of the invention.