Pass-through proxy for managing service connections

Techniques for a Predictive Connection Manager Service (PCMS) to predict when client applications will send service requests to backend services, and proactively establishes connections, caches data, or takes other actions, to reduce latencies between receipt of and response to these service requests. The PCMS analyzes historical usage data for the client applications to identify usage patterns, and uses those usage patterns to proactively scale resources to handle service requests. The PCMS can be implemented as a pass-through proxy for client applications to reduce frictions for managing how users interact with backend services. For instance, the PCMS can install client-side drivers such that updates or patches for the drivers need only be installed on the PCMS rather than on each client device. Further, the PCMS provides interfaces through which users can develop custom drivers for backend services, and also manages software drivers for different service provider networks, thus offering multi-provider connectors.

BACKGROUND

Service providers offer computing infrastructures and services that fulfill users' computing-service needs without the users having to invest in and maintain the computing infrastructure required to implement these services. These service providers maintain networks of managed computing resources and functionality to implement various types of scalable, on-demand services to provide to users. As users in the workforce continue to work remotely, these service providers continue to receive large amounts of service requests from users that peak at certain times of the day. For instance, users often log onto their computers and access their client applications at similar times of the day, such as in the mornings when users start work, around midday when users log back onto their computers after lunch, and so forth. Due to the large amounts of service requests received, service provider networks can experience difficulties in handling all of the requests during these peak times, which can result in poor user experience.

DETAILED DESCRIPTION

Reducing latencies between receiving and responding to service requests of client applications, and minimizing frictions in communications between client applications and backend services, are key concerns of service provider networks. This disclosure describes techniques for implementing a Predictive Connection Manager Service (PCMS) that predicts when client applications will send service requests to backend services, and proactively establishes connections, caches data, or takes other actions to reduce latencies between receipt of and response to these service requests. The PCMS analyzes historical usage data for the client applications to identify usage patterns, such as an application establishing a connection to a backend service at a same time each morning, and uses those usage patterns to proactively scale resources to handle service requests. This can help the PCMS reduce bottlenecks caused from receiving large amounts of service requests during periods of time, which results in more efficient utilization of the PCMS resources and decreased latencies in responding to service requests.

The PCMS can be implemented as a distributed, pass-through proxy for the client applications in order to reduce frictions for managing how users interact with backend services. For instance, the PCMS can install client-side drivers such that updates or patches for the drivers need only be installed on the PCMS rather than on each client device. Further, the PCMS provides interfaces through which users can develop custom drivers for backend services, and also manages software drivers for different service provider networks, thus offering multi-provider connectors. In some examples, to perform the techniques described herein, the PCMS may be positioned between client applications running on client devices and backend services and serve as a middleman, or proxy, by handling communications between the client applications and the backend services.

The disclosed PCMS addresses concerns around high latencies and inefficient usage of computing resources caused by high volumes of service requests by predicting service request and proactively responding to the requests prior to receipt. To predict when services requests are going to be received from client applications, the PCMS collects historical-usage data that represents historical connections previously established between client applications and backend services. The historical-usage data may indicate times at which service requests were received, client applications from which the service requests were received, backend services to which the services requests were sent, the types of the service requests (e.g., data requests, types and/or numbers of connections requested, etc.), and/or other data. The PCMS can analyze the historical-usage data to identify usage patterns of client applications with respect to sending service requests during a period of time. The PCMS may generate predictive models (e.g., machine-learning (ML) models, heuristic- or rule-based models, etc.) that represent these usage patterns and may be used to predict when client applications are likely to send service requests, and potentially predict other request information, such as types of service requests, numbers of service requests, and so forth.

Using the predictive models, the PCMS can take proactive actions prior to the predicted times at which service requests are likely to be received from client applications. For instance, the PCMS can proactively establish connections with backend services such that the connections are more quickly available for use by the client applications after sending service requests. Additionally, the PCMS may predict how many connections are likely to be requested by client applications, scale up resources needed to establish the connections, and proactively establish the predicted number of connections prior to the predicted times. As another example, the PCMS may use the predictive models predict different data (e.g., reports, files, etc.) client applications are likely to request at particular times. The PCMS may proactively (e.g., prior to the predicted times) begin to retrieve the data from the appropriate backend services and cache the data in data caches of the PCMS such that the data can be quickly provided to the client applications upon request.

In some instances, client applications may desire and attempt to establish many connections with a particular backend service, which results in many service requests. Additionally, or alternatively, malicious entities can use bots to flood servers of backend services with service requests in order to overwhelm the servers. Traditionally, backend services have not been able to limit how many service requests are received from entities, such as client applications or malicious bots. To solve this issue, the PCMS may implement rate limiting on client applications by restricting the number of requests the client applications can send to backend services in a given period of time. That is, the PCMS may assign rate limits (e.g., 10 requests per second, 100 requests per second, etc.) to client applications based on, for example, application type and act as a gateway to limit the number of service requests forwarded from the client applications to the backend services.

The PCMS may be implemented as a distributed, pass-through proxy for the client applications in order to reduce frictions for managing how users interact with backend services. Generally, the PCMS may behave as a pass-through proxy such that it masquerades as the backend services for which it is proxying, and responds to the client applications as if the backend services were responding. Traditionally, client applications or devices have installed client-side drivers that are used to communicate with backend services. In some instances, each backend service may have their own respective drivers, resulting in client applications or devices having to install and manage many drivers. Thus, when an update or patch is released for a driver (e.g., to remedy a zero-day vulnerability), all client applications or devices that use the driver must update their respective drivers. To alleviate this burden, the PCMS may install the client-side drivers such that updates or patches for the drivers need only be installed on the PCMS rather than on each client device.

Additionally, client applications often desire to establish connections with, and receive data from, services that have been developed by developers of client applications (e.g., third party services, enterprise services, non-cloud-based services, etc.). To accommodate this, the PCMS provides interfaces through which users can develop custom drivers for these backend services. Further, client applications and/or devices often desire to communicate with backend services that are provided, managed, and/or hosted by different service providers (e.g., different cloud-based platforms). Thus, in some examples the PCMS may also install and manage software drivers for different service provider networks, thus offering multi-provider connectors.

Along with managing service requests and connections on behalf of client applications, the PCMS may further provide various metrics to client applications and users thereof. For instance, the PCMS may monitor connections to backend services to determine that a service is becoming, or has become, unhealthy in some respect (e.g., limited access, oversubscription of computing resources, etc.). The PCMS may notify the client application and user of the health of the service, and potentially take various actions, such as establishing a new connection to a healthier backend node running the service. The PCMS may further provide metrics to give users a snapshot view of active connections versus stale/unused connections such that the users may determine how to manage connections more efficiently.

According to the techniques described herein, the PCMS may reduce latencies between receiving and responding to service requests of client applications, and improve the utilization of computing resources used to handle service requests. Additionally, the PCMS may reduce frictions for managing how users interact with backend services. For instance, the PCMS can install client-side drivers such that updates or patches for the drivers need only be installed on the PCMS rather than on each client device. Accordingly, the PCMS may perform techniques that solve problems rooted in computer technology, such as increasing efficiencies around handling service requests from client applications, which in turn improves the utilization of computing resources used to handle service requests, and reduces the amount of time taken to handle service requests.

The techniques described herein are with reference to a service provider network, such as a cloud provider network or platform. However, the techniques are equally applicable to any network and in any environment. For example, the PCMS may perform the techniques described herein for on-premises networks and/or other environments.

FIG.1illustrates a system-architecture diagram of an example environment100in which a predictive connection-manager service (PCMS)102predicts when client devices104are going to request data from, and connections to, backend services108or service provider networks106, and proactively retrieves the data and establishes the connections to reduce latency. Additionally, the PCMS102behaves as a pass-through proxy in order to simplify how client devices104interact with service provider networks106.

The service provider networks102may be operated and/or managed by respective service providers and may provide various services to users to fulfil their computing resource needs, such as cloud-based computing resources and services. For instance, users may operate client devices104in order to register for use of the computing resources and/or backend services108of the service provider networks106. The service provider networks106may include, or be associated with, the PCMS102that predicts when client applications114will send service requests to backend services106, and proactively establishes connections, caches data, or takes other actions to reduce latencies between receipt of and response to these service requests.

The client devices104may be any type of computing device, such as user devices (e.g., smart phones, tablets, wearable devices, personal computers, etc.), Internet of Things (IoT) device (e.g., sensors, smart appliances, medical sensors, security devices, etc.), network devices (e.g., routers, access points, servers, etc.), and/or any other type of device that communicates with service provider networks106and backend services108. The client devices104may run one or more applications114A,114B, and114N (where “N” is any integer of 1 or greater). The applications114may be any type of application executed on client devices104, such as a web browser, a software application, and/or any type of client-side application114. The client devices104may be operated by users that desire to interact with backend services108, such as by establishing network connections (e.g., secure tunnels, application programming interface (API) calls, command line interfaces, etc.) with the backend services, and/or obtaining various types of data from the backend services108. The service provider networks106may be separate networks of computing resources and backend services108, and may be managed or operated by different service providers. In some instances, the service provider networks106may include one or more of cloud-provider platforms/networks, cloud-based software providers, customer relationship management (CRM) service networks, on-premises networks, enterprise networks, and so forth. The backend services108may generally comprise any type of service that performs operations with or on behalf of client devices104, such as performing service operations, data operations, and so forth.

Traditionally, the applications114would send service requests to the service provider networks106in order to interact or communicate with the backend services108. However, various issues arise around this traditional platform, as noted above, such as latency in handling surges of service requests, inefficient scaling of resources to handle the service requests, and so forth.

The PCMS102may be included in, or deployed according to, a distributed architecture112that can be called by the client applications114. For instance, the PCMS102may be deployed as clusters of containers running on servers of data centers across different regions or subnetworks of the service provider networks106. However, the PCMS102may run in any network or at any location, such as in the backbone infrastructure of private and/or public networks. However, the PCMS102may be a centralized service as well, and may be implemented using any technology (e.g., virtual machines, bare metal servers, containers, processes, serverless functions, etc.). Thus, the PCMS102may be either centralized or distributed, and be supported by one or more computing devices. In some embodiments, the PCMS102may run, partially or entirely, as an application114on the client devices104themselves.

Rather than having the applications114send service requests directly to the service provider networks106and associated backend services108, the PCMS102may serve as a proxy that facilitates communications between the applications114of the client devices104and the backend services108of the service provider networks106.

To predict when services requests are going to be received from client applications114, the PCMS102collects historical-usage data that represents historical connections previously established between the client applications114and backend services108. The historical-usage data may indicate times at which service requests were received, client applications114from which the service requests were received, backend services108to which the services requests were sent, and types of the service requests (e.g., data requests, types and/or numbers of connections requested, etc.). The PCMS102can analyze the historical-usage data to identify usage patterns of client applications114with respect to sending service requests during a period of time. The PCMS102may generate predictive models118(e.g., machine-learning (ML) models, heuristic- or rule-based models, etc.) that represent these usage patterns and may be used to predict when client applications114are likely to send service requests, and potentially predict other request information, such as types of service requests, numbers of service requests, and so forth.

Using the predictive models118, a prediction component116of the PCMS102can take proactive actions prior to the predicted times at which service requests are likely to be received from client applications114. For instance, the prediction component116can proactively establish connections124with backend services108such that the connections124are more quickly available for use by the client applications114after sending service requests. Additionally, the prediction component116may predict how many connections124are likely to be requested by client applications114, scale up resources needed to establish the connections124, and proactively establish the predicted number of connections124prior to the predicted times.

In some instances, the prediction component116may use the predictive models118predict different data (e.g., reports, files, etc.) client applications114are likely to request at particular times. The prediction component116may proactively (e.g., prior to the predicted times) begin to retrieve the data from the appropriate backend services108and cache the data in a data cache120of the PCMS102such that the data can be quickly provided to the client applications114upon request.

In some instances, client applications114may desire and attempt to establish many connections124with a particular backend service108, which results in many service requests. Additionally, or alternatively, malicious entities can use bots to flood servers of backend services108with service requests in order to overwhelm the servers. Traditionally, backend services108have not been able to limit how many service requests are received from entities, such as client applications114or malicious bots. To solve this issue, the PCMS102may implement rate limiting on client applications114by restricting the number of requests the client applications114can send to backend services108in a given period of time. That is, the PCMS102may assign rate limits (e.g., 10 requests per second, 100 requests per second, etc.) to client applications114based on, for example, application type and act as a gateway to limit the number of service requests forwarded from the client applications114to the backend services108.

As noted above, the PCMS102may be implemented as a pass-through proxy for the client applications in order to reduce frictions for managing how users interact with backend services108. Generally, the PCMS102may behave as a pass-through proxy such that it masquerades as the backend services108for which it is proxying, and responds to the client applications114as if the backend services108were responding. Traditionally, client applications114or devices104have installed client-side drivers that are used to communicate with backend services108. In some instances, each backend service108may have their own respective drivers, resulting in client applications114or devices104having to install and manage many drivers. Thus, when an update or patch is released for a driver (e.g., to remedy a zero-day vulnerability), all client applications114that use the driver must update their respective drivers. To alleviate this burden, the PCMS102may install the client-side drivers122such that updates or patches for the drivers122need only be installed on the PCMS102rather than on each client device104. The PCMS102may use the drivers122to communicate with the backend services108appearing to the backend services108as if the PCMS102is the client applications114themselves. Generally, the drivers122may comprise software or code that represent instructions or permissions that instructor inform the PCMS102how to communicate with the backend services108and/or the applications114. The PCMS102may store client-side drivers122that enable the PCMS102to communicate with the backend services108as if it were client applications114. Additionally, or alternatively, the PCMS102may include service-side drivers122that enable the PCMS102to communicate with the client applications114as if they were talking to the backend services108themselves.

Additionally, client applications114often desire to establish connections124with, and receive data from, services108that have been developed by developers of client applications114(e.g., third party services, enterprise services, non-cloud-based services, etc.). To accommodate this, the PCMS102provides interfaces through which users can design custom drivers122for these backend services108. Further, client applications114and/or devices104often desire to communicate with backend services108that are provided, managed, and/or hosted by different service providers106(e.g., different cloud-based platforms). Thus, in some examples the PCMS102may also install and manage software drivers122for the different service provider networks106, thus offering multi-provider connectors.

In some examples, one or more of the service provider networks106may be or comprise a cloud provider network. A cloud provider network (sometimes referred to simply as a “cloud”) refers to a pool of network-accessible computing resources (such as compute, storage, and networking resources, applications, and services), which may be virtualized or bare-metal. The cloud can provide convenient, on-demand network access to a shared pool of configurable computing resources that can be programmatically provisioned and released in response to user commands. These resources can be dynamically provisioned and reconfigured to adjust to variable load. Cloud computing can thus be considered as both the applications delivered as services over a publicly accessible network (e.g., the Internet, a cellular communication network) and the hardware and software in cloud provider data centers that provide those services.

A cloud provider network106can be formed as a number of regions, where a region is a separate geographical area in which the cloud provider clusters data centers. Each region can include two or more availability zones connected to one another via a private high-speed network, for example a fiber communication connection. An availability zone (also known as an availability domain, or simply a “zone”) refers to an isolated failure domain including one or more data center facilities with separate power, separate networking, and separate cooling from those in another availability zone. A data center refers to a physical building or enclosure that houses and provides power and cooling to servers of the cloud provider network. Preferably, availability zones within a region are positioned far enough away from one other that the same natural disaster should not take more than one availability zone offline at the same time. Users can connect to availability zones of the cloud provider network via a publicly accessible network (e.g., the Internet, a cellular communication network) by way of a transit center (TC). TCs can be considered as the primary backbone locations linking customers to the cloud provider network, and may be collocated at other network provider facilities (e.g., Internet service providers, telecommunications providers) and securely connected (e.g., via a VPN or direct connection) to the availability zones. Each region can operate two or more TCs for redundancy. Regions are connected to a global network which includes private networking infrastructure (e.g., fiber connections controlled by the cloud provider) connecting each region to at least one other region. The cloud provider network may deliver content from points of presence outside of, but networked with, these regions by way of edge locations and regional edge cache servers. This compartmentalization and geographic distribution of computing hardware enables the cloud provider network to provide low-latency resource access to customers on a global scale with a high degree of fault tolerance and stability.

With cloud computing, instead of buying, owning, and maintaining their own data centers and servers, organizations can acquire technology such as compute power, storage, databases, and other services on an as-needed basis. The cloud provider network106may provide on-demand, scalable computing services to users through a network, for example allowing users to have at their disposal scalable “virtual computing devices” via their use of the compute servers and block store servers. These virtual computing devices have attributes of a personal computing device including hardware (various types of processors, local memory, random access memory (“RAM”), hard-disk and/or solid state drive (“SSD”) storage), a choice of operating systems, networking capabilities, and pre-loaded application software. Each virtual computing device may also virtualize its console input and output (“I/O”) (e.g., keyboard, display, and mouse). This virtualization allows users to connect to their virtual computing device using a computer application such as a browser, application programming interface, software development kit, or the like, in order to configure and use their virtual computing device just as they would a personal computing device. Unlike personal computing devices, which possess a fixed quantity of hardware resources available to the user, the hardware associated with the virtual computing devices can be scaled up or down depending upon the resources the user requires. Users302choose to deploy their virtual computing systems to provide network-based services for their own use and/or for use by their users or client.

The cloud provider network106may implement various computing resources or services, which may include a virtual compute service, data processing service(s) (e.g., map reduce, data flow, and/or other large scale data processing techniques), data storage services (e.g., object storage services, block-based storage services, or data warehouse storage services) and/or any other type of network based services (which may include various other types of storage, processing, analysis, communication, event handling, visualization, and security services not illustrated). The resources required to support the operations of such services (e.g., compute and storage resources) may be provisioned in an account associated with the cloud provider, in contrast to resources requested by users of the cloud provider network106, which may be provisioned in user accounts.

Generally, the PCMS102, and components thereof, may comprise computing devices, systems, or other hardware along with software, firmware, and/or other logic that is supported one computing device, or across more computing devices in a network, such as portions of one or more of the service provider networks106. Additionally, the PCMS102may comprise a system of other devices. The techniques described herein are generally described with respect to a service provider network106, such as a cloud provider network or platform. However, the techniques are generally applicable for any network, such as on-premises networks, hybrid networks, and so forth.

FIG.2Aillustrates a component diagram200of example components of a PCMS102that proactively establishes network connections124and/or caches data on behalf of client devices104, and behaves as a pass-through proxy in order to simplify how client devices104interact with service provider networks106.

As illustrated, the PCMS102may include one or more hardware processors202(processors), one or more devices, configured to execute one or more stored instructions. The processor(s)202may comprise one or more cores. Further, the PCMS102may include one or more network interfaces204configured to provide communications between the PCMS102and other devices, such as the client devices104in a client tier, the backend services108in the service tier, and/or other systems or devices in and/or remote from the service provider networks106. The network interfaces204may include devices configured to couple to various networks, such as the network(s)110, which may include one or more of personal area networks (PANs), wired and wireless local area networks (LANs), wired and wireless wide area networks (WANs), and so forth.

The PCMS102may also include computer-readable media206that stores various executable components (e.g., software-based components, firmware-based components, etc.). In addition to various components discussed inFIG.1, the computer-readable-media206may further store components to implement functionality described herein. While not illustrated, the computer-readable media206may store one or more operating systems utilized to control the operation of the one or more devices that comprise the service provider network106. According to one embodiment, the operating system comprises the LINUX operating system. According to another embodiment, the operating system(s) comprise the WINDOWS SERVER operating system from MICROSOFT Corporation of Redmond, Washington. According to further embodiments, the operating system(s) can comprise the UNIX operating system or one of its variants. It should be appreciated that other operating systems can also be utilized.

Additionally, the PCMS102may include a data store, or storage208which may comprise one, or multiple, repositories or other storage locations for persistently storing and managing collections of data such as databases, simple files, binary, and/or any other data. The storage208may include one or more storage locations that may be managed by one or more database management systems.

The computer-readable media206may store portions, or components, of the PCMS102described herein. For instance, the computer-readable media206may store and/or execute a metrics analyzer210that monitors and tracks various metrics228associated with the service requests and connections of the client applications114. For instance, the metrics analyzer210may track numbers of connection requests sent from individual client applications114, number of active and stale connections124for applications114(e.g., used versus unused connections), scaling needs of connections124and/or applications112, historical usage data for the client applications114, and so forth.

The computer-readable media206may store and/or execute interfaces through which users can interact with and query the PCMS102. The interfaces212may include APIs, command line interfaces, text/code editors, and/or other interfaces through which users may communicate with the PCMS102and develop custom drivers122.

The computer-readable media206may store and/or execute a notification component216configured to provide users associated with client applications114and/or backend services108with information around connections124, such as health values230and/or metrics228. For instance, the notification component216may send alerts to users associated with client applications114and/or backend services108to indicate that connections124and/or nodes of the services108are unhealthy, or moving towards being unhealthy.

Further, the computer-readable media206may store and/or execute a monitoring component218. The monitoring component218may perform health checks using health values230for backend services108with which the client applications114communicate. If any time backend service108is unhealthy, then PCMS102will refresh the connection pool to initiate connections to same backend service108running elsewhere in another location. If there are no other healthy backend service108instances, then the notification component216may notify the client application114that backend service108is unhealthy.

The computer-readable media206may store and/or execute a metrics component220configured to generate metrics228, which are all the configuration details and historical usage data226. The metrics component220may log the times when there are no active connections124(in case of busy application114). In case of non-busy application, the metrics component220logs only the times where are active connections124.

The computer-readable media206may store and/or execute an access control component222configured to authenticate users, applications114, and/or client devices104. For instance, the access control component222may perform role-based access control (RBAC) from the client tier to middle tier, and/or middle tier to service tier, by restricting access based on roles associated with the users, applications114, and/or client devices104. Additionally, or alternatively, the access control component222may perform fine-grained access control to authenticate the users by using a complete set of permissions to determine access control. The computer-readable media206may store and/or execute a connections component224configured to create connections124with the backend services108and/or client devices/applications.

The backend services108may comprise various types of services, such as serverless function services, long-term storage services, data warehouse services, database services, and/or custom services created or hosted on behalf of users.

To utilize the services provided by the PCMS102, the users may register for an account with the PCMS102. For instance, users may utilize a user device to interact with an identity and access management (IAM) component (not illustrated) that allows the users to create user accounts with the PCMS102. Generally, the IAM component may enable the users to manage their network infrastructures remotely, and view information provided by PCMS102. Generally, the different user accounts can assume different roles, or sets or permissions/credentials, that allow network users to perform different actions, and be restricted from performing some actions.

The computer-readable media206may be used to store and retrieve information, such as program modules, data structures, or other data. It should be appreciated by those skilled in the art that computer-readable storage media is any available media that provides for the non-transitory storage of data and that can be accessed by the PCMS102. In some examples, the operations performed by the PCMS102, and or any components included therein, may be supported by one or more server devices. Stated otherwise, some or all of the operations performed by the PCMS102, and or any components included therein, may be performed by one or more computer devices operating in a distributed arrangement.

FIGS.2B and2Cillustrate respective examples of client devices104that run one or more instances of a predictive connection manager246locally on the client devices104. In each ofFIGS.2B and2C, the client device104each include one or more hardware processors232(processors), one or more devices, configured to execute one or more stored instructions. The processor(s)232may comprise one or more cores. Further, the PCMS102may include one or more network interfaces234configured to provide communications between the client devices104and other devices, such as the PCMS102and/or other systems or devices in and/or remote from the service provider network102. The network interfaces234may include devices configured to couple to various networks, such as the network(s)110, which may include one or more of personal area networks (PANs), wired and wireless local area networks (LANs), wired and wireless wide area networks (WANs), and so forth.

The one or more network interfaces234such as a wireless or Wi-Fi network communications interface, an Ethernet communications interface, a cellular network communications interface, a Bluetooth communications interface, etc., for communications over various types of networks, including wide-area network, local-area networks, private networks, public networks etc. In the case of a wireless communications interfaces, such interfaces may include radio transceivers and associated control circuits and logic for implementing appropriate communication protocols.

For instance, the network interface(s)234may include a personal area network (PAN) component to enable communications over one or more short-range wireless communication channels. For instance, the PAN component may enable communications compliant with at least one of the following standards IEEE 802.15.4 (ZigBee), IEEE 802.15.1 (Bluetooth), IEEE 802.11 (WiFi), or any other PAN communication protocol. Furthermore, each of the network interface(s)234may include a wide area network (WAN) component to enable communication over a wide area network. The networks may represent an array of wired networks, wireless networks, such as WiFi, or combinations thereof.

The client devices104may comprise any type of portable and/or fixed device and include one or more input devices236and output devices238. The input devices236may include a keyboard, keypad, lights, mouse, touch screen, joystick, control buttons, etc. The output devices238may include a display, a light element (e.g., LED), a vibrator to create haptic sensations, or the like. In some implementations, one or more loudspeakers may function as output devices238to output audio sounds.

As shown, the client devices may each include CRM240that stores and/or executes various components, such as an OS242and one or more applications114.FIG.2Billustrates an example where the application(s)114have an application stack244in which the predictive connection-manager (PCM)246is inserted. The PCM246generally is the same as, or included similar or the same functionality, as PCMS102. That is, rather than running the functionality of PCMS102in a data center or other location, the PCM246may be run on the client device104and perform functionality of the PCMS102as part of the execution of the application stack244of applications114. Thus, techniques such as predictively/proactively establishing connections and/or retrieving data may be performed locally on the client device104and as part of execution of the application stack244. In this way, the PCM246may be proactively interacting with backend services108on behalf of the applications114and at least partially prior to the user making explicit requests for data or connections.

FIG.2Cillustrates a similar example where the PCM246is running in a container248on a client device104, rather than in the application stack244. However, the PCM246similarly executes in the container248to perform functionality similar to, or the same as the PCMS102. In this way, part or all of the logic and components of the PCMS102may be on the client devices104themselves such that some, or all, of the PCMS102may not be needed as a proxy (in some examples).

FIG.3Aillustrates a flow diagram300of an example process for a PCMS102to proactively establish network connections124on behalf of client devices108prior to the client devices108requesting the connections.

At “1,” a user302may login to use an application114, such as by providing a username and password, and/or by any other authentication means. The PCMS102may use the login to the application as a trigger, in some instances, to proactively perform an operation. For instance, the PCMS102may wait to perform authenticated operations until the user302has logged in and authenticated themselves.

At “2,” the PCMS102may determine to proactively establish connection(s)124with the backend service108. For instance, the prediction component116may use the predictive models118to determine that the user302of the application114is likely to request the connection124with the particular backend service108. In some instances, the predictive models118represent a prior usage pattern of the user302and/or client application114that indicates the connection124is generally requested by the user302and/or client application114at, or near, a current time. In some instances, the PCMS102may act as a proxy such that the PCMS102establishes connections124between itself and the backend service108, and the client applications114communicate with the backend service108via the PCMS102. In other examples, the connections124may be a pool of connection objects that are created and maintained by the PCMS102. A connection object may represent a unique session with the backend service108, and is generally equivalent to the actual network connection124to the backend service108.

At “3,” the PCMS102may maintain the connection(s)124on behalf of the client application114such that the connection(s)124are established prior to the client application114actually requesting the connection124(e.g., responsive to the log in).

At “4,” the user302may provide input via the client device104to the application114that indicates a request for an operation. This request may include a request for the connection124to a particular backend service108.

At “5A,” because the connection124has been established, or is already in the process of being established, the PCMS102may respond to the application114with an indication of the connection124(e.g., connection details). In this way, the PCMS102may start creating a predicted connection124in response to the user302logging into the application114such that the connection124is prepared, or nearly prepared, once the user302requests an operation that includes establishing the connection124. Step 5A illustrates an example where the client application114communicates with the backend service108via the PCMS102that is acting as a proxy. However, step 5B illustrates an example where the PCMS102provides the client application114with one or more connection objects to the backend service108, and the client application114communicates directly with the backend service108using the connection object(s).

FIG.3Billustrates a flow diagram of an example process302for a client device104to establish a connection124to a backend service108where the PCMS102did not predict the connection124, and the client device104notifies the PCMS102of the connection124to update a predictive model118of the PCMS102.

At “1,” a user302may login to use an application114, such as by providing a username and password, and/or by any other authentication means. At “2,” the user may perform a user operation by interacting with the application114that results in at least one of a request to communicate with a backend service108, a request to obtain data from a backend service108, and so forth. In this case, however, the login and/or associated operation may not be predicted by the PCMS102as the behavior or login is not part of a normal usage pattern by the user and/or application114.

At “3,” the client application114may reach out to the appropriate backend service108and establish a connection124based on the user operation, and receive a response to the user operation at “4” from the backend service108and using the connection124. At “5,” the client application114may indicate the connection124to the PCMS102by providing information around the connection that was established by the application114, such as an indication of which application114established the connection124, the user operation performed, the time of day, a number of connections124established, and/or other data associated with the connection124. The PCMS102may then update the predictive models118based on the connection124made by the client application114such that the PCMS102may predict the user logging into the applications114and performing the operation at a future time, and take proactive actions (e.g., establishing the connection124in response to the log in).

FIG.4Aillustrates a flow diagram400of an example process for a PCMS102to proactively retrieve and cache data on behalf of client devices104prior to the client devices104requesting the data.

At “1,” a user302may login to use an application114, such as by providing a username and password, and/or by any other authentication means. The PCMS102may use the login to the application as a trigger, in some instances, to proactively perform an operation. For instance, the PCMS102may wait to perform authenticated operations until the user302has logged in and authenticated themselves.

At “2,” the PCMS102may determine to proactively fetch predicted data402from a backend service108. For instance, the prediction component116may use the predictive models118to determine that the user302of the application114is likely to request the predicted data402from a particular backend service108. In some instances, the predictive models118represent a prior usage pattern of the user302and/or client application114that indicates the predicted data402is generally requested by the user302and/or client application114at, or near, a current time.

At “3,” the PCMS102may fetch and cache the predicted data402on behalf of the client application114such that the predicted data402is cached prior to the client application114actually requesting the predicted data402(e.g., responsive to the log in). Additionally, or alternatively, the PCMS102may create connection objects usable to obtain the predicted data402from the backend service108.

At “4,” the user302may provide input via the client device104to the application114that indicates a request for a particular operation, which may include a request for the PCMS102to retrieve the predicted data402.

At “5A,” because the predicted data402has been retrieved and cached in the data cache120, or is already in the process of being retrieved and cached in the data cache120, the PCMS102may respond to the application114with the predicted data402. In this way, the PCMS102may start retrieving data that a user302is likely to request via an application114in response to the user302logging into the application114such that the predicted data402is retrieved and cached120, or nearly cached120, once the user302requests the particular data402.

Additionally, or as an alternative to step 5A, at “5B” the PCMS102may provide the connection object(s) to the client application114, and the client application114may use the connection object to obtain the predicted data402from the backend service108itself.

FIG.4Billustrates a flow diagram of an example process404for a client device104to retrieve data from a backend service108where the PCMS102did not predict the data retrieval, and the client device104notifies the PCMS102of the data retrieval to update a predictive model118of the PCMS102.

At “1,” a user302may login to use an application114, such as by providing a username and password, and/or by any other authentication means. At “2,” the user may perform a user operation by interacting with the application114that results in at least one of a request to communicate with a backend service108, a request to obtain data from a backend service108, and so forth. In this case, however, the login and/or associated operation may not be predicted by the PCMS102as the behavior or login is not part of a normal usage pattern by the user and/or application114.

At “3,” the client application114may reach out to the appropriate backend service108and establish a connection124based on the user operation, and retrieve the data in response to the user operation at “4” from the backend service108and using the connection124. At “5,” the client application114may indicate the connection124and data retrieval to the PCMS102by providing information around the data that was retrieved by the application114, such as an indication of which application114retrieved the data, the user operation performed, the time of day, and/or other data associated with the connection124and/or data retrieval. The PCMS102may then update the predictive models118based on the data retrieved by the client application114such that the PCMS102may predict the user logging into the applications114and performing the operation at a future time, and take proactive actions (e.g., caching the data and/or creating connection objects in response to the login).

FIG.5illustrates a system-architecture diagram of an example environment500in which a PCMS102uses historical-usage data226to create predictive models118that represent usage patterns of client applications114requesting data from, or connections to, backend services108.

The PCMS102may receive, obtain, monitor, or otherwise obtain historical usage data226indicating historical connections124previously established between client applications114and backend services108. The historical-usage data226may indicate times at which service requests were received, client applications114from which the service requests were received, backend services108to which the services requests were sent, and types of the service requests (e.g., data requests, types and/or numbers of connections requested, etc.). The metrics analyzer210can analyze the historical-usage data226to identify usage patterns of client applications114with respect to sending service requests during a period of time. The metrics analyzer210may generate predictive models118(e.g., machine-learning (ML) models, heuristic- or rule-based models, etc.) that represent these usage patterns and may be used to predict when client applications114are likely to send service requests, and potentially predict other request information, such as types of service requests, numbers of service requests, and so forth.

The prediction component116may then provide the cache component214and connection component224with access to the predictive models, and/or instruct the cache component214and connection component224to predictively cache data and/or establish connections prior to predicted times indicated by the predicted models118.

FIG.6illustrates a system-architecture diagram of an example environment600in which PCMS102allows users302to build or develop custom drivers606to interface with various services108, and also patch service drivers610on a PCMS such that the client devices104are not required to patch their service drivers610.

As shown, the PCMS102may expose interfaces212, such as CLIs, APIs, UIs, and/or other interfaces212through which users302of client devices104may build or develop customer driver604(e.g., drivers606A-606N). The custom drivers606may be used by the PCMS102to communicate with custom backend services108with which the PCMS102would otherwise be unable to communicate.

The PCMS102may further include service drivers608A for backend services108A, and service drivers608B for backend services108B. These service drivers610A and610B may enable client applications114to communicate with multiple service provider networks106hosting backend services108through the PCMS102such that the PCMS102may be a multi-provider connector. As shown, a patch component612may access patch storage614to identify new patches610that are to be installed on the different drivers606,608A, and/or608B.

FIG.7illustrates a graphical user interface illustrating metrics228that are presented to a user302of client applications114where the metrics indicate information about connections124established by the PCMS102and on behalf of the client applications114.

The user interface702may illustrate connection metrics228for connections128associated with a user302, an application114, and/or a device104. One illustration of metrics may be connection requests704where the number of connection requests received from a particular client application114(or user and/or device) over a period of time. Further, the user interface702illustrates active/stale connection706where active connections are connections124that are being used by client applications114and backend services108, and stale connections are connections that are unused for more than a threshold period of time. using these metrics706, a user302can see whether or not they have too many stale connections at different parts of the day, and that they can scale down the stale connections during those times of the day.

FIGS.8-12illustrate flow diagrams of example methods800-1200that illustrate aspects of the functions performed at least partly by the service provider network106as described in this disclosure. The logical operations described herein with respect toFIGS.8-12may be implemented (1) as a sequence of computer-implemented acts or program modules running on a computing system and/or (2) as interconnected machine logic circuits or circuit modules within the computing system.

FIG.8illustrates a flow diagram of an example method800for a PCMS102to generate and use a predictive model118to determine a time at which a client application114is likely to request connections124to a backend service108and a number of the connections124, and to establish the number of connections124with the backend service108on behalf of the client application114prior to the time.

At802, the PCMS102may receive historical usage data226that represents historical connections established between a client application114on a client device104and a backend service108. For instance, the PCMS may receive historical usage data226such as data indicating times at which service requests were sent from client applications114, client applications114from which the service requests were received, backend services108to which the services requests were sent, the types of the service requests (e.g., data requests, types and/or numbers of connections requested, etc.), and/or other data.

At804, the PCMS102may analyze the historical usage data226to generate a predictive model118that models a usage pattern according to which the client application114established the historical connections. For instance, the PCMS102can analyze the historical-usage data226to identify usage patterns of client applications114with respect to sending service requests during a period of time. The PCMS102may generate predictive models118(e.g., machine-learning (ML) models, heuristic- or rule-based models, etc.) that represent these usage patterns and may be used to predict when client applications114are likely to send service requests, and potentially predict other request information, such as types of service requests, numbers of service requests, and so forth.

At806, the PCMS102may, using the predictive model, predict (i) a time at which the client application114is likely to request connections124to the backend service108, and (ii) a number of connections124to the backend service108that the client application114is likely to request at the time.

At808, the PCMS102may establish, prior to the time, the number of connections with the backend service108. In some instances, the PCMS102may act as a proxy such that the PCMS102establishes connections124between itself and the backend service108, and the client applications114communicate with the backend service108via the PCMS102. In other examples, the number of connections124may be a pool of connection objects that are created and maintained by the PCMS102. A connection object may represent a unique session with the backend service108, and is generally equivalent to the actual network connection124to the backend service108.

In some instances, the PCMS may establish the number of connections in response to detecting an event, such as a user logging into the client application114. For instance, the PCMS102may predict that the user is likely to request the connections at a particular time, and in some instances, may wait until the user logs into the client application114. In response to the user logging into the application114, the PCMS102may then proactively establish the connections (or receive data) prior to the user submitting a request or query.

At810, the PCMS102may provide the client application with access to use the number of connections with the backend service at least by the time. For instance, the connections124may be maintained in a pool such that, when the client application114connects to the proxy PCMS102, the connections124are available for use. As another example, the PCMS102may provide the client applications114with access to the connection objects such that the client applications114can use the connection objects to the backend service108.

FIG.9illustrates a flow diagram of an example method900for a PCMS102to determine a time at which a client application114is likely to request a connection124to a backend service108, and to establish the connection124with the backend service108on behalf of the client application114prior to the time.

At902, the PCMS102may receive historical data that represents historical connections previously established between a client application of a client device and a backend service of a service provider network. For instance, the PCMS may receive historical usage data226such as data indicating times at which service requests were sent from client applications114, client applications114from which the service requests were received, backend services108to which the services requests were sent, the types of the service requests (e.g., data requests, types and/or numbers of connections requested, etc.), and/or other data.

At904, the PCMS102may determine, using the historical data, a time at which the client application114is likely to request a connection124to the backend service108. For instance, the PCMS102can analyze the historical-usage data226to identify usage patterns of client applications114with respect to sending service requests during a period of time. The PCMS102may use these usage patterns to predict when client applications114are likely to send service requests, and potentially predict other request information, such as types of service requests, numbers of service requests, and so forth.

At906, the PCMS102may establish, prior to the time, the connection124between the connection-manager system and the backend service. In some instances, the PCMS102may act as a proxy such that the PCMS102establishes a connection124between itself and the backend service108, and the client applications114communicate with the backend service108via the PCMS102. In other examples, the connection124may be a connection object that is created by the PCMS102. A connection object may represent a unique session with the backend service108, and is generally equivalent to the actual network connection124to the backend service108. The PCMS102may indicate the connection object to the client application114for use by the client application114such that the client application114reaches out to the backend service108directly.

At908, the PCMS102may provide the client application with access to use the connection with the backend service at least by the time. For instance, the connection124may be maintained such that, when the client application114connects to the proxy PCMS102, the connection124is available for use. As another example, the PCMS102may provide the client application114with access to the connection object such that the client application114can use the connection object to connect directly to the backend service108.

FIG.10illustrates a flow diagram of an example method1000for a PCMS102to determine a time at which a client application114is likely to request a particular data from a backend service108, and to retrieve and cache the particular data on behalf of the client application prior to the time.

At1002, the PCMS102may receive historical data that represents historical connections previously established between a client application of a client device and a backend service of a service provider network. For instance, the PCMS may receive historical usage data226such as data indicating times at which data requests were sent from client applications114, client applications114from which the data requests were received, backend services108to which the services requests were sent, the types of the data requested, and/or other data.

At1004, the PCMS102may determine, using the historical data, a time at which the client application is likely to request particular data from the backend service. For instance, the PCMS102can analyze the historical-usage data226to identify usage patterns of client applications114with respect to sending data requests during a period of time. The PCMS102may use these usage patterns to predict when client applications114are likely to send data requests, and potentially predict other request information, such as types of data requests, numbers of data requests, and so forth.

At1006, the PCMS102may obtain, prior to the time, the particular data402from the backend service108. At1008, the PCMS102may cache the particular data402in a data cache120of the connection-manager service102.

At1010, the PCMS102may provide the client application114with access to the particular data402at least by the time. For instance, the particular data402may be cached in the data cache120and the particular data402may be made accessible to the client application114in response to receiving a request.

FIG.11illustrates a flow diagram of an example method1100for a PCMS102to install a software driver608to communicate with a backend service108, update the software driver308using a software patch610, and using the updated software driver to establish connections with the backend service108on behalf of a client application114.

At1102, the PCMS102may receive input data from the client devices indicating a backend service of the service provider network with which each of the client devices desire to communicate. At1104, the PCMS102may install a software driver that is usable to communicate with the backend service.

In some instances, the PCMS102may provide a user of the connection-manager system with access to an interface602through which the user develops a software driver associated with a backend service102. The PCMS102may then install the software driver that is usable to communicate with the backend service on behalf of the user.

At1106, the PCMS102may receive a software patch for updating the software driver. In some instances, the software driver608was installed with a zero day vulnerability in a library associated with the software driver608, and the software patch608remedies the zero day vulnerability in the library.

At1108, the PCMS102may use the software patch610to update the software driver608to result in an updated software driver608. In this way, each client application114does not need to be individually updated with the software patch610.

At1110, the PCMS102may establish connections124with the backend service108on behalf of the client devices104and at least partly using the updated software driver608such that the client devices104are not required to update respective software drivers608using the software patch610.

FIG.12illustrates a flow diagram of an example method1200for a PCMS102to install software drivers608to communicate with backend services108of at least two service provider networks102, and acting as a proxy and using the drivers608, establishing connections with the at least two service provider networks.

At1202, the PCMS102may receive a first indication of a first backend service of a first service provider network108with which first client devices desire to communicate. At1204, the PCMS102may install a first driver608that is usable to communicate with the first backend service108. At1206, the PCMS102may receive a second indication of a second backend service of a second service provider network with which second client devices desire to communicate. At1208, the PCMS102may install a second driver that is usable to communicate with the second backend service. The first and second backend services108may be managed or supported by different service providers (e.g., multi-cloud environment, hybrid-cloud environment, etc.).

At1210, the PCMS102may establish first connections with the first backend service on behalf of the first client devices using the first driver. At1212, the PCMS102may establish second connections with the second backend service on behalf of the second client devices using the second driver. In this way, application(s)114are able to communicate with different service provider networks through the same PCMS102proxy mechanism.

FIG.13is a system and network diagram1300that shows an illustrative operating environment that includes data centers1304in one or more regions1306of a service provider network106that can be configured to implement aspects of the functionality described herein. The service provider network106can provide computing resources, like VM instances and storage, on a permanent or an as-needed basis. Among other types of functionality, the computing resources provided by the service provider network106may be utilized to implement the various services described above. As also discussed above, the computing resources provided by the service provider network106can include various types of computing resources, such as data processing resources like VM instances, data storage resources, networking resources, data communication resources, network services, and the like.

The computing resources provided by the service provider network106may be enabled in one embodiment by one or more data centers1304A-1304N (which might be referred to herein singularly as “a data center1304” or in the plural as “the data centers1304”). The data centers1304are facilities utilized to house and operate computer systems and associated components. The data centers1304typically include redundant and backup power, communications, cooling, and security systems. The data centers1304can also be located in geographically disparate locations, or regions1308. One illustrative embodiment for a data center1304that can be utilized to implement the technologies disclosed herein will be described below with regard toFIG.14.

The users, such as administrators, of the client devices108that utilize the service provider network106may access the computing resources provided by the service provider network106over any wired and/or wireless network(s)110, which can be a wide area communication network (“WAN”), such as the Internet, an intranet or an Internet service provider (“ISP”) network or a combination of such networks. For example, and without limitation, a user device operated by a user302of the service provider network106may be utilized to access the service provider network106by way of the network(s)110. It should be appreciated that a local-area network (“LAN”), the Internet, or any other networking topology known in the art that connects the data centers1304to remote clients and other users can be utilized. It should also be appreciated that combinations of such networks can also be utilized.

FIG.14is a computing system diagram1400that illustrates one configuration for a data center1304that implements aspects of the technologies disclosed herein. The example data center1304shown inFIG.14includes several server computers1402A-1402F (which might be referred to herein singularly as “a server computer1402” or in the plural as “the server computers1402”) for providing computing resources1404A-1404E. In some examples, the resources1404and/or server computers1402may include, be included in, or correspond to, the computing devices described herein.

The server computers1402can be standard tower, rack-mount, or blade server computers configured appropriately for providing the computing resources described herein (illustrated inFIG.14as the computing resources1404A-1404E). As mentioned above, the computing resources provided by the service provider network106can be data processing resources such as VM instances or hardware computing systems, database clusters, computing clusters, storage clusters, data storage resources, database resources, networking resources, and others. Some of the servers1402can also be configured to execute a resource manager1406capable of instantiating and/or managing the computing resources. In the case of VM instances, for example, the resource manager1406can be a hypervisor or another type of program configured to enable the execution of multiple VM instances on a single server computer1402. Server computers1402in the data center1304can also be configured to provide network services and other types of services.

In the example data center1304shown inFIG.14, an appropriate LAN1408is also utilized to interconnect the server computers1402A-1402F. It should be appreciated that the configuration and network topology described herein has been greatly simplified and that many more computing systems, software components, networks, and networking devices can be utilized to interconnect the various computing systems disclosed herein and to provide the functionality described above. Appropriate load balancing devices or other types of network infrastructure components can also be utilized for balancing a load between each of the data centers1404A-1404N, between each of the server computers1402A-1402F in each data center1304, and, potentially, between computing resources in each of the server computers1402. It should be appreciated that the configuration of the data center1304described with reference toFIG.14is merely illustrative and that other implementations can be utilized.

FIG.15shows an example computer architecture for a computer1500capable of executing program components for implementing the functionality described above. The computer architecture shown inFIG.15illustrates a conventional server computer, workstation, desktop computer, laptop, tablet, network appliance, e-reader, smartphone, or other computing device, and can be utilized to execute any of the software components presented herein.

The computer1500includes a baseboard1502, or “motherboard,” which is a printed circuit board to which a multitude of components or devices can be connected by way of a system bus or other electrical communication paths. In one illustrative configuration, one or more central processing units (“CPUs”)1504operate in conjunction with a chipset1506. The CPUs1504can be standard programmable processors that perform arithmetic and logical operations necessary for the operation of the computer1500.

The chipset1506provides an interface between the CPUs1504and the remainder of the components and devices on the baseboard1502. The chipset1506can provide an interface to a RAM1508, used as the main memory in the computer1500. The chipset1506can further provide an interface to a computer-readable storage medium such as a read-only memory (“ROM”)1510or non-volatile RAM (“NVRAM”) for storing basic routines that help to startup the computer1500and to transfer information between the various components and devices. The ROM1510or NVRAM can also store other software components necessary for the operation of the computer1500in accordance with the configurations described herein.

The computer1500can operate in a networked environment using logical connections to remote computing devices and computer systems through a network, such as the network1508. The chipset1506can include functionality for providing network connectivity through a network interface controller (NIC)1512, such as a gigabit Ethernet adapter. The NIC1512is capable of connecting the computer1500to other computing devices over the network1508(or140). It should be appreciated that multiple NICs1512can be present in the computer1500, connecting the computer to other types of networks and remote computer systems.

In addition to the storage1514described above, the computer1500can have access to other computer-readable storage media to store and retrieve information, such as program modules, data structures, or other data. It should be appreciated by those skilled in the art that computer-readable storage media is any available media that provides for the non-transitory storage of data and that can be accessed by the computer1500. In some examples, the operations performed by the service provider network106, and or any components included therein, may be supported by one or more devices similar to computer1500. Stated otherwise, some or all of the operations performed by the service provider network106, and or any components included therein, may be performed by one or more computer devices1500operating in a network-based arrangement.

The storage1514can store an operating system utilized to control the operation of the computer1500. According to one embodiment, the operating system comprises the LINUX operating system. According to another embodiment, the operating system comprises the WINDOWS SERVER operating system from MICROSOFT Corporation of Redmond, Washington. According to further embodiments, the operating system can comprise the UNIX operating system or one of its variants. It should be appreciated that other operating systems can also be utilized. The storage1514can store other system or application programs and data utilized by the computer1500.

In one embodiment, the storage1514, RAM1508, ROM1510, and/or other computer-readable storage media may be encoded with computer-executable instructions which, when loaded into the computer1500, transform the computer from a general-purpose computing system into a special-purpose computer capable of implementing the embodiments described herein. These computer-executable instructions transform the computer1500by specifying how the CPUs1504transition between states, as described above. According to one embodiment, the computer1500has access to computer-readable storage media storing computer-executable instructions which, when executed by the computer1500, perform the various techniques described above. The computer1500can also include computer-readable storage media having instructions stored thereupon for performing any of the other computer-implemented operations described herein.