Executing user-defined code in response to determining that resources expected to be utilized comply with resource restrictions

Systems and methods are described for determining a location in an on-demand code execution environment to execute user-specified code. The on-demand code execution environment may include many points of presence (POPs), some of which have limited computing resources. An execution profile for a set of user-specified code can be determined that indicates the resources likely to be used during execution of the code. Each POP of the environment may compare that execution profile to resource restrictions of the POP, to determine whether execution of the code should be permitted. In some instances, where execution of the code should not be permitted at a given POP, an alternative POP may be selected to execute the code.

BACKGROUND

To facilitate increased utilization of data center resources, virtualization technologies allow a single physical computing device to host one or more instances of virtual machines that appear and operate as independent computing devices to users of a data center. With virtualization, the single physical computing device can create, maintain, delete, or otherwise manage virtual machines in a dynamic manner. In turn, users can request computer resources from a data center, including single computing devices or a configuration of networked computing devices, and be provided with varying numbers of virtual machine resources.

In some scenarios, virtual machine instances may be configured according to a number of virtual machine instance types to provide specific functionality. For example, various computing devices may be associated with different combinations of operating systems or operating system configurations, virtualized hardware resources and software applications to enable a computing device to provide different desired functionalities, or to provide similar functionalities more efficiently. These virtual machine instance type configurations are often contained within a device image, which includes static data containing the software (e.g., the OS and applications together with their configuration and data files, etc.) that the virtual machine will run once started. The device image is typically stored on the disk used to create or initialize the instance. Thus, a computing device may process the device image in order to implement the desired software configuration.

DETAILED DESCRIPTION

Generally described, aspects of the present disclosure relate to executing user-defined code within a low latency, on-demand code execution environment, as well as managing the computing devices within the code execution environment on which the code is executed. The on-demand code execution environment may operate as part of a system of rapidly provisioned and released computing resources, often referred to as a “cloud computing environment.” Specifically, the code execution environment may include one or more computing devices, virtual or non-virtual, that are “pre-warmed” (e.g., booted into an operating system and executing a complete or substantially complete runtime environment) and configured to enable execution of user-defined code, such that the code may be executed rapidly without initializing the virtual machine instance. Each set of code on the on-demand code execution environment may define a “task,” and implement specific functionality corresponding to that task when executed on the on-demand code execution environment. Individual implementations of the task on the on-demand code execution environment may be referred to as an “execution” of the task. By defining tasks on the on-demand code execution environment and selectively executing those tasks, users may implement complex functionalities at high speed and low latency, without being required to deploy, configure, or manage the computing devices on which the tasks are executed. The on-demand code execution environment, in turn, may execute tasks of multiple users simultaneously, thus allowing efficient use of computing resources of those devices. To ensure the security and privacy of user information, the on-demand code execution environment may generally ensure that tasks of each user are executed on distinct computing devices (which may be virtual computing devices), thus reducing the chances that a task executing on behalf of a first user could interfere with or gain information regarding execution of a task on behalf of a second user. However, dedicating a computing device to a single user can negatively impact the performance of the on-demand code execution environment, by increasing the required computing resources of that environment and reducing the flexibility of the environment in allocating execution of tasks. Accordingly, aspects of the present disclosure enable the on-demand code execution environment to execute multiple tasks on the same computing device (virtual or non-virtual), while maintaining the security and data privacy of each task execution. Specifically, the present disclosure enables the generation of a risk profile for a task, which indicates a set of computing resources that the task is likely to utilize during execution, and potentially the expected level of use of those resources. By utilizing risk profiles for multiple tasks, the on-demand code execution environment can determine a risk associated with allowing two tasks of unrelated users to execute on the same device. For example, where the risk profiles of the two tasks indicate that each task utilizes different computing resources (e.g., a first task utilizes a persistent storage device, while a second task does not), the risk of executing both tasks on the same computing device can be considered low. Alternatively, where the risk profiles of two tasks indicate that each task requires access to the same computing resource (e.g., both tasks require simultaneous access to the same storage device), the risk of executing both tasks on the same computing device can be considered relatively higher. Thus, by utilizing risk profiles, the on-demand code execution environment can selectively assign execution of tasks of multiple users to the same computing device while maintaining security and privacy of those executions, thus increasing the efficiency and performance of the on-demand code execution system.

In some instances, an on-demand code execution environment may operate as a distributed system, in which multiple points of presence (POPs) implement instances of the on-demand code execution environment. As used herein, a POP is intended to refer to any collection of related computing devices utilized to implement functionality on behalf of one or many providers. POPs are generally associated with a specific geographic location in which the computing devices implementing the POP are located, or with a region serviced by the POP. For example, a datacenter or a collection of computing devices within a datacenter may form a POP. An on-demand code execution environment may utilize multiple POPs that are geographically diverse, to enable users in a variety of geographic locations to quickly transmit and receive information from the on-demand code execution environment. In some instances, the POPs may also implement other services, such as content delivery network (CDN) services, data storage services, data processing services, etc. For the purposes of the present disclosure, these other services will generally be referred to herein as “auxiliary services.” Implementation of auxiliary services and instances of the on-demand code execution environment on the same POP may be beneficial, for example, to enable tasks executed on the on-demand code execution environment to quickly interact with the auxiliary services. However, implementation of auxiliary services on a POP may also limit the amount of computing resources available to implement an instance of the on-demand code execution environment. For example, a POP implementing an edge server of a CDN service may have relatively little available computing resources (e.g., in the form of disk space, processor time, memory, etc.) with which to execute tasks. These resources may be even further depleted by attempting to execute those tasks within a distinct computing device, such as a virtual computing device, that does not implement functionalities of the CDN service. Accordingly, aspects of the present disclosure can enable, in some circumstances, the execution of tasks on devices of a POP that also implement an auxiliary service. Specifically, the present disclosure enables a risk profile for the task to be compared with computing resources utilized by the auxiliary service, to determine the risk that execution of the task would interfere with or compromise the security or privacy of the auxiliary service. In instances where such a risk is low, the task can be executed on the same computing device that implements the auxiliary service, thus enabling high efficiency utilization of that computing device, and enabling low latency task execution with respect to data made available by that auxiliary service (e.g., because such execution occurs on the same device implementing the auxiliary service, and no transmission outside the device need occur to execute the task).

Where an on-demand code execution environment utilizes multiple POPs, each implementing an instance of the on-demand code execution environment, it may also be beneficial to share or transfer assignments of task executions between those POPs. For example, where a first POP implements an auxiliary service that limits the computing resources available for on-demand code execution, and where a request is received at that POP to execute a task that would exceed or unduly tax those limited computing resources, it would be beneficial to allow the first POP to reroute the request to a second POP associated with greater available computing resources. Moreover, it would be beneficial to enable such rerouting before execution of the task begins, to reduce inefficiencies or errors related to partial execution of tasks. Accordingly, aspects of the present disclose also enable generation of an execution profile for a task, indicative of the level of computing resources likely to be utilized during execution of a task. Similarly to the risk profile described above, an execution profile may be generated by either or both static analysis of the code corresponding to a task or analysis of historical executions of the task. In some instances, the risk profile and execution profile may be combined into a single profile for the task. In other instances, both profiles may be maintained separately. On receiving a request to execute a task, a POP may compare the execution profile of the task to a set of restrictions for the POP (which may reflect, for example, a permitted level of computing resources to be utilized during execution of a task, or a current level of computing resources available at the POP for execution of a task). If the execution profile of the task satisfies the restrictions for the POP, the POP may execute the first task locally. If the execution profile of the task does not satisfy the restrictions for the POP, the POP may decline to execute the task locally, and may notify a user of the error. In some instances, the POP may also reroute the request to execute the task to an alternative POP (e.g., associated with lower restrictions). Thus, by utilization of an execution profile, each POP associated with an on-demand code execution environment may determine, prior to execution of a task, whether execution of the task should be permitted at that POP, ensuring efficient utilization of resources at each POP.

As used herein, the term “virtual machine instance” is intended to refer to an execution of software or other executable code that emulates hardware to provide an environment or platform on which software may execute (an “execution environment”). Virtual machine instances are generally executed by hardware devices, which may differ from the physical hardware emulated by the virtual machine instance. For example, a virtual machine may emulate a first type of processor and memory while being executed on a second type of processor and memory. Thus, virtual machines can be utilized to execute software intended for a first execution environment (e.g., a first operating system) on a physical device that is executing a second execution environment (e.g., a second operating system). In some instances, hardware emulated by a virtual machine instance may be the same or similar to hardware of an underlying device. For example, a device with a first type of processor may implement a plurality of virtual machine instances, each emulating an instance of that first type of processor. Thus, virtual machine instances can be used to divide a device into a number of logical sub-devices (each referred to as a “virtual machine instance”). While virtual machine instances can generally provide a level of abstraction away from the hardware of an underlying physical device, this abstraction is not required. For example, assume a device implements a plurality of virtual machine instances, each of which emulates hardware identical to that provided by the device. Under such a scenario, each virtual machine instance may allow a software application to execute code on the underlying hardware without translation, while maintaining a logical separation between software applications running on other virtual machine instances. This process, which is generally referred to as “native execution,” may be utilized to increase the speed or performance of virtual machine instances. Other techniques that allow direct utilization of underlying hardware, such as hardware pass-through techniques, may be used, as well.

While a virtual machine executing an operating system is described herein as one example of an execution environment, other execution environments are also possible. For example, tasks or other processes may be executed within a software “container,” which provides a runtime environment without itself providing virtualization of hardware. Containers may be implemented within virtual machines to provide additional security, or may be run outside of a virtual machine instance.

The execution of tasks on the on-demand code execution environment will now be discussed. Specifically, to execute tasks, the on-demand code execution environment described herein may maintain a pool of pre-initialized virtual machine instances that are ready for use as soon as a user request is received. Due to the pre-initialized nature of these virtual machines, delay (sometimes referred to as latency) associated with executing the user code (e.g., instance and language runtime startup time) can be significantly reduced, often to sub-100 millisecond levels. Illustratively, the on-demand code execution environment may maintain a pool of virtual machine instances on one or more physical computing devices, where each virtual machine instance has one or more software components (e.g., operating systems, language runtimes, libraries, etc.) loaded thereon. When the on-demand code execution environment receives a request to execute the program code of a user, which specifies one or more computing constraints for executing the program code of the user, the on-demand code execution environment may select a virtual machine instance for executing the program code of the user based on the one or more computing constraints specified by the request and cause the program code of the user to be executed on the selected virtual machine instance. The program codes can be executed in isolated containers that are created on the virtual machine instances. Since the virtual machine instances in the pool have already been booted and loaded with particular operating systems and language runtimes by the time the requests are received, the delay associated with finding compute capacity that can handle the requests (e.g., by executing the user code in one or more containers created on the virtual machine instances) is significantly reduced.

The on-demand code execution environment may include a virtual machine instance manager configured to receive user code (threads, programs, etc., composed in any of a variety of programming languages) and execute the code in a highly scalable, low latency manner, without requiring user configuration of a virtual machine instance. Specifically, the virtual machine instance manager can, prior to receiving the user code and prior to receiving any information from a user regarding any particular virtual machine instance configuration, create and configure virtual machine instances according to a predetermined set of configurations, each corresponding to any one or more of a variety of run-time environments. Thereafter, the virtual machine instance manager receives user-initiated requests to execute code, and identify a pre-configured virtual machine instance to execute the code based on configuration information associated with the request. The virtual machine instance manager can further allocate the identified virtual machine instance to execute the user's code at least partly by creating and configuring containers inside the allocated virtual machine instance. Various embodiments for implementing a virtual machine instance manager and executing user code on virtual machine instances is described in more detail in U.S. patent application Ser. No. 14/502,648, entitled “PROGRAMMATIC EVENT DETECTION AND MESSAGE GENERATION FOR REQUESTS TO EXECUTE PROGRAM CODE” and filed Sep. 30, 2014 (“the '648 Application), the entirety of which is hereby incorporated by reference.

The foregoing aspects and many of the attendant advantages of this disclosure will become more readily appreciated as the same become better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings.

FIG. 1is a block diagram of an illustrative operating environment100in which a plurality of POPs105may implement an on-demand code execution environment110, as well as auxiliary services106, based on communication with user devices102. While the user devices102and POPs105are shown as grouped withinFIG. 1, the user devices102and POPs105may be geographically distant, and independently owned or operated. For example, the user devices102could represent a multitude of users in various global, continental, or regional locations accessing the POPs105. Further, the POPs105may be globally, continentally, or regionally disparate, in order to provide a wide geographical presence for the on-demand code execution environment110and/or the auxiliary services106. Accordingly, the groupings of user devices102and POPs105withinFIG. 1is intended to represent a logical, rather than physical, grouping. The POPs105ofFIG. 1are illustratively shown as implementing both auxiliary services106and instances of the on-demand code execution environment110. However, the operating environment100may additionally or alternatively include POPs that execute only auxiliary services106or only an instance of the on-demand code execution environment110.

As shown inFIG. 1, to implement an auxiliary service106, a POP105can include a server107communicatively coupled to a service data store108. The server107and service data store108may operate in conjunction to implement functionalities of the auxiliary service106. For example, where the auxiliary service106is an edge server for a CDN, the server107and service data store108may operate to cache distributed content (e.g., as provided by a user of the auxiliary service106), and respond to requests from end users for such cached content. As a further example, where the auxiliary service106is a database system, the server107and service data store108may operate to facilitate and manage interactions with a database. In general, auxiliary services106may include any network-based service or data source. Auxiliary services106may be associated with operation of the on-demand code execution environment110, e.g., to provide billing or logging services to the on-demand code execution environment. In some instances, auxiliary services106actively transmit information, such as API calls or other task-triggering information, to the on-demand code execution environment110. In other instances, auxiliary services106may be passive, such that data is made available for access by the on-demand code execution environment110. For example, components of the on-demand code execution environment110may periodically poll such passive data sources, and trigger execution of tasks within the on-demand code execution environment110based on the data provided. Operation of various auxiliary services106, including CDN networks, database services, data storage services, and data processing services, are known within the art, and therefore will not be described herein. While a simplified view of auxiliary services106is shown inFIG. 1(e.g., including a single server107and service data store108), the POP105may implement an auxiliary service106by use of any number of computing or storage devices, which may not be shown inFIG. 1. In some instances, computing or storage devices associated with an auxiliary service106may also be utilized to implement functionalities of the on-demand code execution environment110. For example, virtual machine instances150of the on-demand code execution environment110(which are described in more detail below) may be implemented by the server107. In some instances, the on-demand code execution environment110may execute tasks directly on the server107(e.g., without use of a virtual machine instance150).

The on-demand code execution environment110includes a frontend120, worker manager130, instance pool140, and data stores160collectively configured to enable users (via user devices102) to submit computer executable instructions (also referred to herein as “code” or “user code”) to the on-demand code execution environment110for execution as a “task.” As will be described in more detail below, the frontend120can facilitate interactions between the on-demand code execution environment110with user devices102, auxiliary services106, and/or other computing devices (not shown inFIG. 1) via a network104. These interactions may include, for example, submission of code, which may be stored within the data stores160, or transmission of requests to execute code, which may be communicated to the worker manager130for assignment to and execution by a virtual machine instance150within the instance pool140. The on-demand code execution environment110is depicted inFIG. 1as operating in a distributed computing environment including several computer systems that are interconnected using one or more computer networks (not shown inFIG. 1). The on-demand code execution environment110could also operate within a computing environment having a fewer or greater number of devices than are illustrated inFIG. 1. Thus, the depiction of the on-demand code execution environment110inFIG. 1should be taken as illustrative and not limiting to the present disclosure. For example, the on-demand code execution environment110or various constituents thereof could implement various Web services components, hosted or “cloud” computing environments, and/or peer to peer network configurations to implement at least a portion of the processes described herein. Further, the on-demand code execution environment110may be implemented directly in hardware or software executed by hardware devices and may, for instance, include one or more physical or virtual servers implemented on physical computer hardware configured to execute computer executable instructions for performing various features that will be described herein. The one or more servers may be geographically dispersed or geographically co-located, for instance, in one or more POPs105.

The network104may include any wired network, wireless network, or combination thereof. For example, the network104may be a personal area network, local area network, wide area network, over-the-air broadcast network (e.g., for radio or television), cable network, satellite network, cellular telephone network, or combination thereof. As a further example, the network104may be a publicly accessible network of linked networks, possibly operated by various distinct parties, such as the Internet. In some embodiments, the network104may be a private or semi-private network, such as a corporate or university intranet. The network104may include one or more wireless networks, such as a Global System for Mobile Communications (GSM) network, a Code Division Multiple Access (CDMA) network, a Long Term Evolution (LTE) network, or any other type of wireless network. The network104can use protocols and components for communicating via the Internet or any of the other aforementioned types of networks. For example, the protocols used by the network104may include Hypertext Transfer Protocol (HTTP), HTTP Secure (HTTPS), Message Queue Telemetry Transport (MQTT), Constrained Application Protocol (CoAP), and the like. Protocols and components for communicating via the Internet or any of the other aforementioned types of communication networks are well known to those skilled in the art and, thus, are not described in more detail herein.

Various example user devices102are shown in communication with the on-demand code execution environment110, including a desktop computer, laptop, and a mobile phone, each provided by way of illustration. In general, the user devices102can be any computing device such as a desktop, laptop or tablet computer, personal computer, wearable computer, server, personal digital assistant (PDA), hybrid PDA/mobile phone, mobile phone, electronic book reader, set-top box, voice command device, camera, digital media player, and the like. The on-demand code execution environment110may provide the user devices102with one or more user interfaces, command-line interfaces (CLI), application programming interfaces (API), and/or other programmatic interfaces for generating and uploading user-executable code, invoking the user-provided code (e.g., submitting a request to execute the user codes on the on-demand code execution environment110), scheduling event-based jobs or timed jobs, tracking the user-provided code, and/or viewing other logging or monitoring information related to their requests and/or user codes. Although one or more embodiments may be described herein as using a user interface, it should be appreciated that such embodiments may, additionally or alternatively, use any CLIs, APIs, or other programmatic interfaces.

InFIG. 1, users, by way of user devices102, may interact with the on-demand code execution environment110to provide executable code, and establish rules or logic defining when and how such code should be executed on the on-demand code execution environment110. For example, a user may wish to run a piece of code in connection with a web or mobile application that the user has developed. One way of running the code would be to acquire virtual machine instances from service providers who provide infrastructure as a service, configure the virtual machine instances to suit the user's needs, and use the configured virtual machine instances to run the code. In order to avoid the complexity of this process, the user may alternatively provide the code to the on-demand code execution environment110, and request that the on-demand code execution environment110execute the code using one or more pre-established virtual machine instances. The on-demand code execution environment110can handle the acquisition and configuration of compute capacity (e.g., containers, instances, etc., which are described in greater detail below) based on the code execution request, and execute the code using the compute capacity. The on-demand code execution environment110may automatically scale up and down based on the volume, thereby relieving the user from the burden of having to worry about over-utilization (e.g., acquiring too little computing resources and suffering performance issues) or under-utilization (e.g., acquiring more computing resources than necessary to run the codes, and thus overpaying).

To enable interaction with the on-demand code execution environment110, the environment110includes a frontend120, which enables interaction with the on-demand code execution environment110. In an illustrative embodiment, the frontend120serves as a “front door” to the other services provided by the on-demand code execution environment110, enabling users (via user devices102) to provide, request execution of, and view results of computer executable code. The frontend120may provide user devices102with the ability to upload or otherwise communicate user-specified code to the on-demand code execution environment110, and to thereafter request execution of that code. To facilitate storage of code, the on-demand code execution environment110includes one or more data stores160, which may correspond to any persistent or substantially persistent data storage, such as a hard drive (HDD), a solid state drive (SDD), network attached storage (NAS), a tape drive, or any combination thereof. In one embodiment, the request interfaces communicates with external computing devices (e.g., user devices102, auxiliary services106, etc.) via a graphical user interface (GUI), CLI, or API. The frontend120processes the requests and makes sure that the requests are properly authorized. For example, the frontend120may determine whether the user associated with the request is authorized to access the user code specified in the request.

The user code as used herein may refer to any program code (e.g., a program, routine, subroutine, thread, etc.) written in a specific program language. In the present disclosure, the terms “code,” “user code,” and “program code,” may be used interchangeably. Such user code may be executed to achieve a specific function, for example, in connection with a particular web application or mobile application developed by the user. As noted above, individual collections of user code (e.g., to achieve a specific function) are referred to herein as “tasks,” while specific executions of that code are referred to as “task executions” or simply “executions.” Tasks may be written, by way of non-limiting example, in JavaScript (e.g., node.js), Java, Python, and/or Ruby (and/or another programming language). Requests to execute a task may generally be referred to as “calls” to that task. Such calls may include the user code (or the location thereof) to be executed and one or more arguments to be used for executing the user code. For example, a call may provide the user code of a task along with the request to execute the task. In another example, a call may identify a previously uploaded task by its name or an identifier. In yet another example, code corresponding to a task may be included in a call for the task, as well as being uploaded in a separate location (e.g., storage of an auxiliary service106or a storage system internal to the on-demand code execution environment110) prior to the request being received by the on-demand code execution environment110. The on-demand code execution environment110may vary its execution strategy for a task based on where the code of the task is available at the time a call for the task is processed.

The frontend120may receive calls to execute tasks in response to Hypertext Transfer Protocol Secure (HTTPS) requests from a user. Also, any information (e.g., headers and parameters) included in the HTTPS request may also be processed and utilized when executing a task. As discussed above, any other protocols, including, for example, HTTP, MQTT, and CoAP, may be used to transfer the message containing a task call to the frontend120.

A call to execute a task may specify one or more third-party libraries (including native libraries) to be used along with the user code corresponding to the task. In one embodiment, the call may provide to the on-demand code execution environment110a ZIP file containing the user code and any libraries (and/or identifications of storage locations thereof) corresponding to the task requested for execution. In some embodiments, the call includes metadata that indicates the program code of the task to be executed, the language in which the program code is written, the user associated with the call, and/or the computing resources (e.g., memory, etc.) to be reserved for executing the program code. For example, the program code of a task may be provided with the call, previously uploaded by the user, provided by the on-demand code execution environment110(e.g., standard routines), and/or provided by third parties. In some embodiments, such resource-level constraints (e.g., how much memory is to be allocated for executing a particular user code) are specified for the particular task, and may not vary over each execution of the task. In such cases, the on-demand code execution environment110may have access to such resource-level constraints before each individual call is received, and the individual call may not specify such resource-level constraints. In some embodiments, the call may specify other constraints such as permission data that indicates what kind of permissions or authorities that the call invokes to execute the task. Such permission data may be used by the on-demand code execution environment110to access private resources (e.g., on a private network).

In some embodiments, a call may specify the behavior that should be adopted for handling the call. In such embodiments, the call may include an indicator for enabling one or more execution modes in which to execute the task referenced in the call. For example, the call may include a flag or a header for indicating whether the task should be executed in a debug mode in which the debugging and/or logging output that may be generated in connection with the execution of the task is provided back to the user (e.g., via a console user interface). In such an example, the on-demand code execution environment110may inspect the call and look for the flag or the header, and if it is present, the on-demand code execution environment110may modify the behavior (e.g., logging facilities) of the container in which the task is executed, and cause the output data to be provided back to the user. In some embodiments, the behavior/mode indicators are added to the call by the user interface provided to the user by the on-demand code execution environment110. Other features such as source code profiling, remote debugging, etc. may also be enabled or disabled based on the indication provided in a call.

Tasks may executed at the on-demand code execution environment110based on explicit calls from user devices102(e.g., as received at a request interface of the frontend120). Alternatively or additionally, tasks may be executed at the on-demand code execution environment110based on data retrieved from one or more auxiliary services106. Illustratively, the frontend120may periodically transmit a request to one or more user-specified auxiliary services106to retrieve any newly available data (e.g., social network “posts,” news articles, etc.), and to determine whether that data corresponds to a user-established trigger to execute a task on the on-demand code execution environment110. Illustratively, each trigger may specify one or more criteria for execution of a task, including but not limited to whether new data is available at the auxiliary services106, the type or content of the data, or timing information corresponding to the data. In addition to tasks executed based on explicit user calls and data from auxiliary services106, the on-demand code execution environment110may in some instances operate to execute tasks independently. For example, the on-demand code execution environment110may operate (based on instructions from a user) to execute a task at each of a number of specified time intervals (e.g., every 10 minutes).

The frontend120can further include an output interface configured to output information regarding the execution of tasks on the on-demand code execution environment110. Illustratively, the output interface may transmit data regarding task executions (e.g., results of a task, errors related to the task execution, or details of the task execution, such as total time required to complete the execution, total data processed via the execution, etc.) to the user devices102or to auxiliary services106, which may include, for example, billing or logging services. The output interface may further enable transmission of data, such as service calls, to auxiliary services106. For example, the output interface may be utilized during execution of a task to transmit an API request to an external service106(e.g., to store data generated during execution of the task).

Requests to execute a task may be allocated to the various instances of the on-demand code execution environment110executing on the POPs105in a variety of manners. In one embodiment, a request may be transmitted directly to a specific POP105, thereby requesting execution of a task on the instance of the on-demand code execution environment110associated with that specific POP105. In another embodiment, network routing techniques may be utilized to provide load balancing between the POPs105, or to route requests to a closest POP105. A variety of such network routing techniques, such as anycast routing and DNS-based load balancing, are known within the art, and therefore will not be discussed in detail herein.

In accordance with embodiments of the present disclosure, the frontend120of an instance of the on-demand code execution environment executing on a given POP105may analyze any received requests for execution of tasks to determine whether the task may be executed at the POP105associated with the frontend120. For example, each instance of the on-demand code execution environment110executed by a POP105may be associated with usage restrictions, such as specific types or amounts of computing resources that tasks executing on the POP105may utilize. Illustratively, where a POP105includes relatively few, low-powered computing devices and/or implements an auxiliary service106that utilizes a high amount of computing resources, the resources available to execute tasks on the POP105may be limited. In some instances, a POP105may simply lack a computing resource required by a task. For example, where a task corresponds to executable code that is intended to run on a discrete graphical processing unit (GPU), the POP105may lack such a GPU, and therefore be unable to execute the task. As yet another example, a POP105may be capable of providing a task execution with access to a computing resource, but it may be inefficient to do so. For example, where execution of a task requires access to a specific auxiliary service106that is provided by a remote POP105, and where the communication capabilities between the initial POP105(on which execution of the task is requested) and the remote POP105are limited, the frontend120of the initial POP105may determine that the task should not be permitted to execute on the initial POP105(and may instead be executed on the remote POP105, or another POP105with sufficient communication capabilities to the remote POP105).

To avoid instances in which a POP105is unable to execute a task due to lack of sufficient computing resources, or where use of such computing resources would be inefficient, the frontend120can be configured to compare an execution profile of the task to a set of resource restrictions associated with the POP105. Generally described, execution profiles can indicate a set of computing resources that are likely to be utilized during execution of a task, or an amount of each computing resource likely to be utilized during such executions. The computing resources reflected in an execution profile may include local computing resources, such as processing power (e.g., of a central processing unit [CPU], GPU, etc.), memory (e.g., random access memory [RAM], persistent data storage, etc.], or bandwidth (e.g., network bandwidth, internal bus utilization, etc.). In addition, the computing resources reflected in an execution profile may include external resources, such as specific auxiliary services106(e.g., an individual database service).

The frontend120may compare an execution profile of a task to a set of resource restrictions associated with the POP105, to determine whether execution of a task on the POP105would comply with those usage restrictions. Generally described, resource restrictions for a POP105may indicate levels of computing resources that may be used during execution of a task at the POP105. In instances where a given computing resource is not available at the POP105, the resource restrictions may indicate that no use of that computing resource is permitted during execution of a task at the POP105. Accordingly, when a request to execute a task is received at the frontend120, the frontend120may compare the execution profile of the task to a set of resource restrictions for the POP105to determine whether the task should be permitted to execute on the POP105. In the instance that the task should be permitted to execute, the frontend120can distribute execution of the task to the worker manager130, which in turn interacts with virtual machine instances150to cause execution of the task. In the instance that the frontend120determines that the execution profile of the task does not comply with the set of resource restrictions, the frontend120can decline to service the request, and may notify a user associated with the request that the request has been declined. In some instances, the frontend120may also reroute the request to an alternative POP105, such as a POP105with lower resource restrictions. In one embodiment, at least one POP105may be associated with no specific resource restrictions, and thus may enable any task permitted by the on-demand code execution environment to be executed at that POP105. Such a POP105may serve as a “default” POP105, such that any task that fails to satisfy the resource restrictions of a first POP105is routed to the default POP105. Where such a default POP105is associated with no resource restrictions, the frontend120of that POP105may not be required to implement the functionalities described herein with respect to execution profiles.

On verifying that a task satisfies an execution profile of a POP105, the frontend120can distribute the request to execute a task to a worker manager130, which can assign tasks to virtual machine instances150for execution. In the example illustrated inFIG. 1, the worker manager130manages the instance pool140, which is a group (sometimes referred to as a pool) of virtual machine instances150that are utilized to execute tasks. As shown inFIG. 1, instances150may have operating systems (OS)152, language runtimes154, and containers156. Containers156are logical units created within a computing device using the resources available on that device, and may be utilized to isolate execution of a task from other processes (e.g., task executions) occurring on the device. For example, in order to service a request to execute a task, the worker manager130may, based on information specified in the request, create a new container156within a virtual machine instance150within which to execute the task. In other instances, the worker manager130may locate an existing container156in one of the instances150in the instance pool140and assign the container150to handle the execution of the task. Containers156may be implemented, for example, as Linux containers. The containers156may have individual copies of the OSes152, the runtimes154, and user code158corresponding to various tasks assigned to be executed within the container156.

While the instance pool140is shown inFIG. 1as a single grouping of virtual machine instances150, some embodiments of the present application may separate virtual machine instances150that are actively assigned to execute tasks from those virtual machine instances150that are not actively assigned to execute tasks. For example, those virtual machine instances150actively assigned to execute tasks may be grouped into an “active pool,” while those virtual machine instances150not actively assigned to execute tasks may be placed within a “warming pool.” Those virtual machine instances150within the warming pool may be pre-initialized with an operating system, language runtimes, or other software required to enable rapid execution of tasks in response to user requests. Further details regarding active pools and warming pools are described in greater detail within the '648 application, incorporated by reference above (e.g., at FIG. 1 of the '648 application).

On receiving a request to execute a task, the worker manager130may locate a virtual machine instance150within the instance pool140that has available capacity to execute the task, as described in more detail below. The worker manager130may further create a container156within the virtual machine instance150, and provision the container156with any software required to execute the task (e.g., an operating system152, runtime154, or code). For example, a container156is shown inFIG. 1provisioned with operating system152B, runtime154B, and a set of code158. The operating system152B and runtime154B may be the same as the operating system152A and runtime154A utilized by the virtual machine instance150, or may be different. After provisioning the container156with the requisite software for a task, the worker manager130can cause the virtual machine instance156to execute the task on behalf of a user.

In some instances, each virtual machine instance150may be utilized only to execute tasks associated with a single user, or group of associated users. Such allocation may provide a high amount of security, by greatly reducing the likelihood that a task execution on behalf of a first user would interfere with or gain unauthorized access to data of a second user. However, such dedicated allocation may increase the amount of computing resources required to execute tasks, as each virtual machine may be associated with “overhead” costs in terms of computing resources. Accordingly, embodiments of the present disclosure can enable the worker manager130to utilize a single virtual machine instance150to execute tasks on behalf of multiple, unrelated users, while still maintaining the security and data privacy of each user. Each task may be executed within a distinct container156. However, the security and data privacy provided by use of containers156may not be sufficient. Thus, the worker manager130may additionally or alternatively utilize risk profiles associated with each task in order to determine when two tasks of different users may be executed by the virtual machine instance150.

Generally described, risk profiles can reflect the potential ability of a task to interfere with or intercept data from other tasks or processes, thus compromising the security and privacy of those other tasks and processes. In one embodiment, the risk profile can reflect the specific user code corresponding to a task, such as functions or libraries included within the code, as described in more detail below. In another embodiment, the risk profile can reflect the information described above with respect to an execution profile, including the set of computing resources expected to be utilized to execute the task, or the amount of such computing resources. In still more embodiments, a risk profile may reflect a set of permissions available to a task, which indicate actions that the task is allowed to take on the on-demand code execution environment110.

In some instances, a risk profile may indicate a specific level of risk for a task (e.g., a numerical value within a range), and the worker manager130may be configured to assign execution of a task to a virtual machine instance150such that a total level of risk for all tasks or processes on the virtual machine instance does not exceed a specific threshold. In other instances, a risk profile may indicate categories of risk for a task, each of which may be associated with a specific characteristic of the task. Categories of risk may include, for example, attributes of the code associated with the task (e.g., programming language used, types of function calls included, libraries loaded), computing resources utilized by the task (e.g., local computing resources, such as disk drives, external computing resources, etc.), or permissions utilized by a task (e.g., as individual permissions or groups of related permissions). For example, permissions related to disk drive access may define a first risk category, while permissions related to a given auxiliary service106may define a second risk category, etc. In such instances, the worker manager130may be configured to assign execution of a task to a virtual machine instance such that the total risk level for a given risk category of all tasks or process on the virtual machine instance150does not exceed a threshold level (which may be the same level for all risk categories, or set on a per category basis). Illustratively, the use of risk categories may enable the worker manager130to assign multiple high-risk tasks to the same virtual machine instance150, so long as those tasks do not pose the same type of risk (and thus are unlikely to interfere with or gain unauthorized access to other tasks or processes). In some embodiments, thresholds for a given risk category may be set to a minimum level, such that any task associated with a non-zero risk level for a given risk category blocks another task with a non-zero risk level in that risk category from being assigned to the same virtual machine instance150.

Accordingly, when a request to execute a task is received at the on-demand code execution environment110and processed by the frontend120, the worker manager130may inspect the risk profile of the task, and compare that risk profile to the risk profiles of other tasks currently assigned to the various virtual machine instances150. The worker manager130can thereafter assign execution of the task to a specific virtual machine instance150, as described in more detail below. The virtual machine instance150, in turn, may execute the task to implement desired functionality of the user.

In some embodiments, the on-demand code execution environment110is adapted to begin execution of a task shortly after it is received (e.g., by the frontend120). A time period can be determined as the difference in time between initiating execution of the task (e.g., in a container on a virtual machine instance associated with the user) and receiving a call to execute the task (e.g., received by a frontend). The on-demand code execution environment110is adapted to begin execution of a task within a time period that is less than a predetermined duration. In one embodiment, the predetermined duration is 500 ms. In another embodiment, the predetermined duration is 300 ms. In another embodiment, the predetermined duration is 100 ms. In another embodiment, the predetermined duration is 50 ms. In another embodiment, the predetermined duration is 10 ms. In another embodiment, the predetermined duration may be any value chosen from the range of 10 ms to 500 ms. In some embodiments, the on-demand code execution environment110is adapted to begin execution of a task within a time period that is less than a predetermined duration if one or more conditions are satisfied. For example, the one or more conditions may include any one of: (1) the user code of the task is loaded on a container in the active pool140at the time the request is received; (2) the user code of the task is stored in the code cache of an instance in the instance pool140at the time the call to the task is received; (3) the instance pool140contains an instance assigned to execute a task associated with the user associated with the call at the time the call is received; or (4) the instance pool140has capacity to handle the call at the time the call is received.

After the task has been executed, the worker manager130may tear down the container156used to execute the task to free up the resources it occupied to be used for other containers156in the instance150. Alternatively, the worker manager130may keep the container156running to use it to service additional calls from the same user. For example, if another call associated with the same task that has already been loaded in the container156, the call can be assigned to the same container156, thereby eliminating the delay associated with creating a new container156and loading the code of the task in the container156. In some embodiments, the worker manager130may tear down the instance150in which the container156used to execute the task was created. Alternatively, the worker manager140may keep the instance150running to use it to service additional calls. The determination of whether to keep the container156and/or the instance150running after the task is done executing may be based on a threshold time, average call volume, and/or other operating conditions. For example, after a threshold time has passed (e.g., 5 minutes, 30 minutes, 1 hour, 24 hours, 30 days, etc.) without any activity (e.g., task execution), the container156and/or the virtual machine instance150is shutdown (e.g., deleted, terminated, etc.), and resources allocated thereto are released. In some embodiments, the threshold time passed before a container156is torn down is shorter than the threshold time passed before an instance150is torn down.

The on-demand code execution environment110may in some instances provide data to one or more of the auxiliary services106as it services incoming calls. For example, the frontend120may communicate with the monitoring/logging/billing services included within the auxiliary services106. The monitoring/logging/billing services may include: a monitoring service for managing monitoring information received from the on-demand code execution environment110, such as statuses of containers and instances on the on-demand code execution environment110; a logging service for managing logging information received from the on-demand code execution environment110, such as activities performed by containers and instances on the on-demand code execution environment110; and a billing service for generating billing information associated with executing user code on the on-demand code execution environment110(e.g., based on the monitoring information and/or the logging information managed by the monitoring service and the logging service). In addition to the system-level activities that may be performed by the monitoring/logging/billing services (e.g., on behalf of the on-demand code execution environment110), the monitoring/logging/billing services may provide application-level services on behalf of the tasks executed on the on-demand code execution environment110. For example, the monitoring/logging/billing services may monitor and/or log various inputs, outputs, or other data and parameters on behalf of the tasks being executed on the on-demand code execution environment110.

In some embodiments, the worker manager130may perform health checks on the instances and containers managed by the worker manager130(e.g., those in the instance pool140). For example, the health checks performed by the worker manager130may include determining whether the instances150and the containers156managed by the worker manager130have any issues of (1) misconfigured networking and/or startup configuration, (2) exhausted memory, (3) corrupted file system, (4) incompatible kernel, and/or any other problems that may impair the performance of the instances and the containers. In one embodiment, the worker manager130performs the health checks periodically (e.g., every 5 minutes, every 30 minutes, every hour, every 24 hours, etc.). In some embodiments, the frequency of the health checks may be adjusted automatically based on the result of the health checks. In other embodiments, the frequency of the health checks may be adjusted based on user requests.

While the worker manager130is described above as assigning execution of a task to a virtual machine instance150, in some embodiments, tasks may be assigned for execution on non-virtual devices, such as the server107. Execution of tasks directly on an underlying physical device may reduce the amount of computing resources necessary to execute such tasks, and enable such tasks to execute more quickly. However, execution of tasks directly on an underlying physical device may also increase the risk that such a task will interfere with or gain unauthorized access to data on the underlying physical device (e.g., generated by processes implementing the auxiliary service106A). In such instances, those processes may also be associated with risk profiles, and the worker manager130may be configured to determine that a total risk associated with the server107(or with specific risk categories) would not exceed a threshold value, should a task be added to the server. Alternatively or additionally, underlying physical devices may be associated with maximum risk levels, such that a task may not execute on the device if the risk level of the task (or of a specific category) exceeds a threshold amount.

The illustration of the various components within the on-demand code execution environment110is logical in nature and one or more of the components can be implemented by a single computing device or multiple computing devices. For example, each of the frontend120, the worker manager140, and the virtual machine instances150can be implemented in a common physical computing device, or across multiple physical computing devices. Moreover, any one or more of the frontend120, the worker manager140, and the virtual machine instances150can be implemented on one or more physical computing devices also implementing functionality corresponding to the auxiliary service106. In some embodiments, the on-demand code execution environment110may comprise multiple frontends, multiple warming pool managers, and/or multiple worker managers. One skilled in the art will appreciate that the on-demand code execution environment110may comprise any number of virtual machine instances implemented using any number of physical computing devices.

FIG. 2depicts a general architecture of a computing system (referenced as worker manager130) that manages the virtual machine instances150in the on-demand code execution environment110. The general architecture of the worker manager130depicted inFIG. 2includes an arrangement of computer hardware and software modules that may be used to implement aspects of the present disclosure. The hardware modules may be implemented with physical electronic devices, as discussed in greater detail below. The worker manager130may include many more (or fewer) elements than those shown inFIG. 2. It is not necessary, however, that all of these generally conventional elements be shown in order to provide an enabling disclosure. Additionally, the general architecture illustrated inFIG. 2may be used to implement one or more of the other components illustrated inFIG. 1. As illustrated, the worker manager130includes a processing unit190, a network interface192, a computer readable medium drive194, and an input/output device interface196, all of which may communicate with one another by way of a communication bus. The network interface192may provide connectivity to one or more networks or computing systems. The processing unit190may thus receive information and instructions from other computing systems or services via the network104. The processing unit190may also communicate to and from memory180and further provide output information for an optional display (not shown) via the input/output device interface196. The input/output device interface196may also accept input from an optional input device (not shown).

The memory180may contain computer program instructions (grouped as modules in some embodiments) that the processing unit190executes in order to implement one or more aspects of the present disclosure. The memory180generally includes RAM, ROM and/or other persistent, auxiliary or non-transitory computer readable media. The memory180may store an operating system184that provides computer program instructions for use by the processing unit190in the general administration and operation of the worker manager140. The memory180may further include computer program instructions and other information for implementing aspects of the present disclosure. For example, in one embodiment, the memory180includes a user interface unit182that generates user interfaces (and/or instructions therefor) for display upon a computing device, e.g., via a navigation and/or browsing interface such as a browser or application installed on the computing device. In addition, the memory180may include and/or communicate with one or more data repositories (not shown), for example, to access user program codes and/or libraries.

In addition to and/or in combination with the user interface unit182, the memory180may include an instance allocation unit186and a user code execution unit188that may be executed by the processing unit190. In one embodiment, the user interface unit182, instance allocation unit186, and user code execution unit188individually or collectively implement various aspects of the present disclosure, e.g., finding compute capacity (e.g., a container156or virtual machine instance150) to be used for executing user code, causing the user code to be loaded and executed on the container156, etc. as described further below.

The instance allocation unit186finds the compute capacity to be used for servicing a call to execute a task. For example, the instance allocation unit186identifies a virtual machine instance150and/or a container156that satisfies any constraints specified by the call and assigns the identified virtual machine instance150and/or container156to execute the called task. The instance allocation unit186may perform such identification based on the programming language in which the user code corresponding to the task is written. For example, if the user code is written in Python, and the instance allocation unit186may find a virtual machine instance150having the Python runtime pre-loaded thereon and assign the virtual machine instance150to execute task. In another example, if the program code specified in the call of the user is already loaded on an existing container156or on another virtual machine instance150assigned to the user (e.g., in the instance pool140ofFIG. 1), the instance allocation unit186may cause the called task to be executed in the container156or in a new container156on the virtual machine instance150. In some embodiments, if the virtual machine instance has multiple language runtimes loaded thereon, the instance allocation unit186may create a new container156on the virtual machine instance150and load the appropriate language runtime on the container156based on the computing constraints specified in the call.

In addition, prior to assigning a virtual machine instance150to execute a task, the instance allocation unit186can verify that the risk profile of the task enables that task to be assigned to the virtual machine instance150. In one embodiment, the instance allocation unit186can verify that, should the task be assigned to a virtual machine instance150, a total risk level of tasks assigned to that virtual machine instance150would not exceed a threshold amount (which may be established by an administrator of the on-demand code execution environment). Such at total risk level may be determined, for example, by summing the risk levels associated with each task (e.g., as reflected in a risk profile of the task). In another embodiment, the instance allocation unit186can verify that, should the task be assigned to a virtual machine instance150, a total risk level of tasks assigned to that virtual machine instance150would not exceed a threshold amount form any specific category of risk.

In the instance that an appropriate virtual machine instance150is located to execute the task, the task may be assigned to the virtual machine instance150for execution. In the instance that no appropriate virtual machine instance150is located, execution of the task may be delayed by the on-demand code execution environment110, or an error may be generated for reporting to a user associated with the task. In some instances, where no appropriate virtual machine instance150is located on a specific POP105, the instance allocation unit186may cause a request to execute the task to be transmitted to an alternative POP105hosting an instance of the on-demand code execution environment110.

After assignment of a task to a virtual machine instance150, the user code execution unit188can manage execution of the program code corresponding to the task. Specifically, the user code execution unit188can instruct the virtual machine instance150to generate a container156in which to execute the code, or to select a pre-existing container (e.g., associated with the same user) in which to execute the code. The user code execution unit188can further cause the code to be downloaded into that container on the virtual machine instance150, and instruct the virtual machine instance150to execute that code.

While the instance allocation unit186and the user code execution unit188are shown inFIG. 2as part of the worker manager140, in other embodiments, all or a portion of the instance allocation unit186and the user code execution unit188may be implemented by other components of the on-demand code execution environment110and/or another computing device. For example, in certain embodiments of the present disclosure, another computing device in communication with the on-demand code execution environment110may include several modules or components that operate similarly to the modules and components illustrated as part of the worker manager140.

In some embodiments, the worker manager130may further include components other than those illustrated inFIG. 2. For example, the memory180may further include a container manager for managing creation, preparation, and configuration of containers156within virtual machine instances150.

While the computing device ofFIG. 2is described as implementing a worker manager130, the same or a similar computing device may additionally or alternatively be utilized to implement other components of the on-demand code execution environment110. For example, such a computing device may be utilized, independently or in conjunction with other components (e.g., data stores) to implement the frontend120ofFIG. 1. The software or computer-executable instructions placed within the memory180may be modified to enable execution of the functions described herein with respect to the task profiler120.

With reference toFIGS. 3A-3C, illustrative interactions are depicted for servicing a request to execute a task on an instance of the on-demand code execution environment110, shown inFIGS. 3A-3Cas executing on POP105A. Specifically,FIG. 3Adepicts illustrative interactions for receiving a call to execute a task, and for verifying that the execution profile of the task satisfies the resource restrictions associated with the POP105A.FIG. 3Bdepicts illustrative interactions for allocating the task to a virtual machine instance150within the instance pool140based on a risk profile of the task.FIG. 3Cdepicts illustrative interactions for executing a task at a virtual machine instance within the instance pool140, and for utilizing information gathered during execution of the task to update execution and risk profiles for the task.

The interactions ofFIG. 3Abegin at (1), where a user device102A transmits a call to execute a task to the frontend120associated with a POP105A. Illustratively, interaction (1) may include transmission of an API call from the user device102to the frontend120. While shown as transmitted by a user device102A, task calls may also be transmitted from other sources, such as auxiliary services106(implemented on the same or different POP105). In some instances, a task call may be received at the frontend120based on actions by the on-demand code execution environment110itself. For example, a task call may be received at the frontend120based on execution of a task on the on-demand code execution environment110(e.g., at an instance of the environment110implemented by the POP105A or by another POP105), or based on the on-demand code execution environment110detecting the addition of new information (e.g., the posting of a new message to a social networking service, the uploading of a file, etc.) at an auxiliary service106. In some instances, tasks may be generated automatically by the on-demand code execution environment110at intervals specified by a user.

Thereafter, at (2), the frontend120obtains an execution profile for the called task. Illustratively, where the call to the task includes code of the task itself, the frontend120may conduct an analysis of the task to generate the execution profile. Where the code of a task has been previously provided to the on-demand code execution environment110, the frontend120may retrieve a previously created execution profile for the task (e.g., created when the code was provided to the on-demand code execution environment110). The previously created execution profile may be stored at the frontend120itself, or within the data stores160.

The execution profile of a task may be generated by the frontend120(or other component of the on-demand code execution environment110) based on static analysis of user code corresponding to the task, dynamic analysis of executions of the task, or both. Generally described, static analysis includes the analysis of a task without requiring execution of the task. For example, on submission of user code corresponding to a task, the frontend120may analyze the code to determine any computing resources expected to be utilized during execution of the code. Illustratively, the frontend120may identify any functions or calls to specific computing resources (e.g., API calls to specific auxiliary services106, functions that utilize a specific local computing resources, such as a disk drive, or libraries enabling interaction with specific resources, such as a GPU) included within the code, as well as any libraries or APIs referenced by the code, and determine that execution of the code is likely to utilize those resources. As a further example, the frontend120may parse user code corresponding to a task (e.g., utilizing an automated theorem proving algorithm) to determine whether execution of the code would actually utilize resources referenced or implicated by the code, or may inspect permissions requested for a task, to determine what resources those permissions implicate. As yet another example, the frontend120may compare code corresponding to the task to other forms of code with known profiles (e.g., an execution or risk profile) to determine a predicted profile for the task. Additionally or alternatively, the execution profile of a task may be generated or revised based on “dynamic analysis,” which includes information regarding one or more executions of the task. For example, at each execution of a task, the on-demand code execution environment110may record the computing resources utilized during that execution, or the levels of such computing resources used. The on-demand code execution environment110(e.g., via the frontend120) may thereafter compile historical usage information, and provide statistical information regarding the likely computing resources used (or levels of computing resources used) during execution of a task. For example, an execution profile may indicate that, on average, a single execution of a given task utilizes a given amount of RAM, processing power (or cycles), disk read/writes, etc. The execution profile may also indicate additional statistical information, such as a variance in the levels of computing resources used. In some instances, dynamic profiling may further include modifying code corresponding to a task to cause execution of a library that tracks, traces, or otherwise monitors calls to resources made during execution of the code. For example, dynamic profiling may include the addition of a library to the code of a task that periodically reports calls to resource made during execution of the code to the frontend120or another component of the on-demand code execution environment110. Calls to resources made during execution of code corresponding to a task may also be accomplished without requiring modification of code, such as by using operating-system level tracing (e.g., as provided by the Linux seccomp facility). In some instances, both static and dynamic profiling may be utilized to determine the computing resource use of a task execution. Illustratively, an execution profile may be initially created based on static analysis, and continuously updated based on dynamic analysis.

Returning to the interactions ofFIG. 3A, at (3), the frontend120can verify that the execution profile of the called task complies with a set of resource restrictions associated with the POP105A. In one embodiment, the set of resource restrictions may be static, such that no task may utilize more than a given amount of each computing resource indicated within the set of resource restrictions. For example, resource restrictions may specify that a task must be expected to execute within a threshold period of time, to utilize less than a threshold amount of memory, etc. In another embodiment, the set of resource restrictions may be dynamic, such that the amount of a given computing resource that may be utilized during execution of a task is based at least in part on current operation of the POP105A or an instance of the on-demand code execution environment110implemented on the POP105A. For example, resource restrictions may specify that a task utilize no more than 1% of the available RAM of the POP105A, or that a task must execute within a threshold period of time that varies based on the level of activity of the POP105A. In yet another embodiment, resource restrictions may include both static and dynamic restrictions (e.g., such that a task must comply with the strictest restriction for any given computing resource within the set of restrictions). Accordingly, the frontend120, at (3), can compare the computing resources indicated in the execution profile of a task to the resource restrictions associated with the POP105A to determine whether execution of the task should be permitted at the POP105. In the instance that the execution profile of a task does not comply with the resource restrictions of the POP105A (e.g., because execution of the task is expected to utilize computing resources not available at the POP105A or utilize too high a level of one or more computing resources), the frontend120may delay execution of the task at the POP110(e.g., to allow resource restrictions to be reduced, where such restrictions are based on activity of the POP105A), or may generate an error for reporting to a user associated with the task. In some instances, where the execution profile of the task fails to comply with the resource restrictions of the POP105A, the frontend120may cause a request to execute the task to be transmitted to an alternative POP105hosting an instance of the on-demand code execution environment110.

For the purposes ofFIG. 3A, it will be assumed that the execution profile of the called task satisfies the resource restrictions of the POP105A, and thus is permitted to be assigned to a virtual machine instance150for execution. Accordingly, at (4), the frontend120distributes the task to the worker manager130, which can facilitate assignment of the task to a virtual machine instance150.

The interactions described above with respect toFIG. 3Athen continue withinFIG. 3B, where the worker manager130obtains a risk profile for the task. Illustratively, where no prior risk profile of a task is available to the worker manager130, the worker manager130may conduct an analysis of the task to generate the risk profile. Where a risk profile has been previously generated for the task, the worker manager130may retrieve a previously created risk profile for the task (e.g., created when the code was provided to the on-demand code execution environment110). The previously created execution profile may be stored at the worker manager130itself, or within the data stores160.

In one embodiment, the risk profile for a task may be generated by static analysis of the code corresponding to the task, or metadata regarding the task itself. For example, some types of user code, such as those that make direct system calls or implement other high-risk functions, may be associated with higher risk than other types of user code. In some instances, user code written in different programming languages may be associated with different levels of risk. Accordingly, the on-demand code execution environment110(e.g., via the worker manager130) may analyze the code corresponding to a task when that code is provided by a user, and assign a risk level to the code that is incorporated into a risk profile. Illustratively, the on-demand code execution environment110may analyze a set of user-submitted code to identify functions calls made or libraries referenced therein, and assign a risk level to the code associated with the highest risk level function call. In some embodiments, a task may be assigned different categories of risk level based on specific attributes of the code. For example, specific functions, libraries, or attributes of a code may be associated with a given risk level within a category of risk. The on-demand code execution environment110may utilize additional static profiling techniques, such as those described above, to generate a risk profile for a task. For example, the on-demand code execution environment110may parse user code corresponding to a task (e.g., utilizing an automated theorem proving algorithm) to determine whether execution of the code would actually utilize functions or libraries referenced or implicated by the code, or may utilize fingerprinting to assign generate a risk profile for the task based on a known risk profile of a similar set of code.

In another embodiment, a risk profile may include information similar to that included within the execution profile of a task, such as the computing resources likely to be used to execute a task, or the likely level of use of such computing resources. The computing resources reflected in a risk profile may include local computing resources, such as processing power (e.g., of a central processing unit [CPU], graphical processing unit [GPU], etc.), memory (e.g., random access memory [RAM], persistent data storage, etc.], or bandwidth (e.g., network bandwidth, internal bus utilization, etc.). In addition, the computing resources reflected in a risk profile may include external resources, such as specific auxiliary services106(e.g., an individual database service). As described in more detail below, the worker manager130may compare the risk profile of a task awaiting assignment to a virtual machine140to the risk profiles of other processes or tasks currently assigned or being executed by that virtual machine140, to determine whether assignment of the task awaiting execution is permissible. Illustratively, one or more computing resources may be considered a “blocking resource,” such that utilization of that resource by a first task or process on a virtual machine instance150blocks the assignment of an additional task that also requires access to that resource. For example, a specific auxiliary service106, such as a database service, may be considered a blocking resource, such that only a single task assigned for execution on any given virtual machine instance150may access that specific auxiliary service106. As a further example, specific local computing resources, such as disk drives (either individually or in combination), may be considered blocking resources. Because only a single task is enabled to utilize the blocking resource, the risk of a security or privacy breach associated with that specific resource is substantially reduced. In some instances, any utilization of a blocking resource may be sufficient to bar assignment of other tasks requiring access to that resource to a given virtual machine. In other instances, a threshold level of use of a blocking resource (which may be set by an operator of the on-demand code execution environment) may be required to bar assignment of other tasks requiring access to that resource to a given virtual machine.

In still more embodiments, a risk profile of a task may reflect permissions of the task, which reflect the ability of a task, when executed, to take specific actions on the on-demand code execution environment110. Illustratively, permissions may be associated with a task by a user or entity that has submitted code of the task to the on-demand code execution environment110. The permissions of a task may include, for example, the ability of the task to access specific auxiliary services106or local computing resources, or to execute specific functions or classes of functions. Permissions may generally reflect the functionalities expected of that task (e.g., whether or not the permissions are actually utilized). Permissions may be associated with a specific risk level (either overall or in a given category of risk), such that a task that has permission to undertake a specific action (e.g., access to a specific auxiliary service106) is associated with a specific risk level. In one embodiment, the risk level of a task (overall or in a given category) may be based at least in part on a sum of the risk levels associated with each permission (overall or in a given category) given to that task. In another embodiment, specific permissions may be implemented as “blocking” permissions, such that a single task provided that permission on a virtual machine instance150prevents other task with that permission to be assigned to the virtual machine instance150.

In addition or alternatively to generation of a risk profile based on static analysis, a risk profile for a task may be determined by “dynamic analysis” (e.g., based at least in part on prior executions of the task). For example, the on-demand code execution environment110may be configured to monitor executions of a task to determine the specific computing resources (or amounts of those computing resources) used during past executions, or to determine the permissions actually utilized in executing the task (as opposed to simply those requested by a user that submitted the task). In some embodiments, a risk profile may be created via both static and dynamic analysis. For example, an initial risk profile of a task may be created based on static analysis, and modified after subsequent executions, based on dynamic analysis. For example, an initial risk level assigned to a task via static analysis may be reduced where a particular high risk function reference in the code of the task has never actually been called, despite a large number of prior executions. Similarly, a risk profile may be modified to reflect that while a task has the potential to utilize specific computing resources or permissions, it rarely or never has. Thus, historical information may be utilized by the on-demand code execution environment to further increase the accuracy of a risk profile. In some embodiments, such as where dynamic analysis has not yet occurred, executions of the task may be limited or restricted on the on-demand code execution environment110. For example, the on-demand code execution environment110may “isolate” the executions, such as by permitting the code to be executed only on a virtual machine instance that is not executing any other task.

Returning to the interactions ofFIG. 3B, at (6), the worker manager130selects a virtual machine instance150from the instance pool140to assign to execute the task based on the risk profile of the task. In one embodiment, the worker manager130may select a virtual machine150, such that the total risk level associated with the virtual machine150would not exceed a threshold level after assignment to execute the task. For example, assume that virtual machines instances140are required (e.g., by an administrator of the on-demand code execution environment110) to maintain a total risk level of lower than 100, and that the called task referenced inFIG. 3Bhas a risk level of 10. Under such assumptions, the worker manager130may assign execution of the task to any virtual machine instance150with a current risk level (e.g., as the sum total of all current tasks assigned to, or processes executing on, the virtual machine instance150) of lower than 90. In another embodiment, the worker manager130may select a virtual machine150, such that a risk level associated with each of a set of risk categories would not exceed a threshold level on an individual virtual machine instance150after assignment to execute the task. For example, where the risk profile of the called task indicates a risk only in a specific category, the worker manager130may determine that, should the task be assigned to a given virtual machine instance150, a total risk level within that category for the virtual machine instance150would not exceed a threshold value. Threshold values for categories may be set collectively (e.g., such that all categories are associated with the same threshold risk level) or individually (e.g., such that individual categories are associated with different threshold risk levels). Threshold risk levels for one or more categories of risk may be set to a minimum value, such that any task assigned to (or process executing on) a virtual machine instance with a non-zero risk level in that category blocks other tasks associated with a non-zero risk level in that category from being executed.

In some instances, multiple virtual machine instances150may be available to be assigned to a task based on the risk profile of the task. The worker manager130may be configured to select between such multiple instances150according to a default preference set by an operator of the on-demand code execution environment110. For example, the worker manager130may be configured to prefer to heavily group tasks, such that a task is assigned to the available virtual machine instance150with the most current tasks, which may allow greater latitude in further assignment of tasks. Alternatively, the worker manager130may be configured to prefer to spread out assignment of tasks, such that a task is assigned to the available virtual machine instance150with the least current tasks. Still further, the worker manager130may be configured to attempt to minimize risk, such that the task is assigned to the available virtual machine instance150that results in the lower total risk (or risk in one or more risk categories). While assignment of tasks is described above with respect to risk profiles, the worker manager130may additionally consider other factors in assigning a task to a virtual machine instance150, such as the operating system152or runtimes154loaded on the virtual machine instance150. Examples of these additional factors are provided in more detail within the '648 application, incorporated by reference above (e.g., at FIG. 4 of the '648 application).

After selecting a virtual machine instance150to which to assign execution of the task, the worker manager130, at (7), assigns that virtual machine instance150to execute the task. Accordingly, the interactions ofFIG. 3Benable the on-demand code execution environment110to execute multiple tasks of unrelated users on the same virtual machine instance150, thus reducing the total computing resources needed to execute those multiple tasks. Moreover, because tasks may be grouped onto virtual machine instances150by use of a risk profile, the security and data privacy of those tasks can be ensured, despite the lack of a dedicated virtual machine instance on which to execute each task.

With reference toFIG. 3C, the illustrative interactions described above are continued. Specifically,FIG. 3Cdepicts interactions for executing a task on a virtual machine instance150, and utilizing the results of that execution to generate or modify either or both a risk profile or execution profile of the task.

The interactions ofFIG. 3Cbegin at (8), where the previously called task is executed on a virtual machine instance within the instance pool140. Specifically, the virtual machine instance150assigned to execute the task can obtain code corresponding to the task (e.g., from the data stores160), load any software (e.g., runtimes154) required by that code, and execute the code, in order to implement functionality corresponding to the task.

Thereafter, at (9), the virtual machine instance150can report information regarding the task execution to the frontend120. Illustratively, the information can include data utilized to update or generate a risk or execution profile for the task, including but not limited to functions called by code of the task, computing resources (either local or external) utilized during execution of the task, and levels of that resource usage. At (10), the frontend120can utilize the execution details to generate or update an execution profile or risk profile for the task (e.g., as described above with respect to generation of risk or execution profiles based on dynamic analysis of task executions). Thus, the risk and/or execution profile of a task can be continuously or periodically updated utilizing information regarding actual executions of the task on the on-demand code execution environment110. While interaction (10) is shown inFIG. 3Cas implemented on at the frontend120, in some instances, the interaction may be partially or wholly implemented on another component of the on-demand code execution environment110, such as the worker manager130.

In some embodiments, the various POPs105implementing instances of the on-demand code execution environment110may be configured to share risk and execution profiles among one another. Accordingly, in such instances, after updating a risk or execution profile for a task, the frontend120may report the updated risk and/or execution profiles (or the modifications made to the respective profiles) to other POPs105, at (11). Alterations or updates to risk and/or execution profiles may be synchronized among the POPs105according to any of a variety of known synchronization protocols. For example, in some instances the POPs105may transmit risk and/or execution profiles of a task via peer-to-peer communication protocols. In other instances, the POPs105may transmit risk and/or execution profiles of a task via a central authority, such as a specific POP105. In other embodiments, each POP105may maintain a local risk and/or execution profile for a task, and thus, interaction (11) may be omitted.

While reference is made herein to two distinct profiles—an execution profile and a risk profile-embodiments of the present application may in some instances combine the information of both profiles into a single profile (e.g., a “task profile”). Thus, some embodiments of the present application may utilize execution and risk profiles interchangeably. In other instances, the information of a first profile, such as the execution profile, may be incorporated into and form a subset of a second profile, such as the risk profile.

In addition, while reference is made above with respect to execution of a task on a virtual machine instance150, some embodiments of the present disclosure may enable the execution of tasks on non-virtual devices, such as the server107associated with the auxiliary service106A (which may also be a host device on which the virtual machine instances150are executed). Accordingly, any functionality described herein with respect to a virtual machine instance150made herein could also be applied to non-virtual, physical computing devices. In embodiments where such a device also implements other functionalities, such as functionalities of an auxiliary service106, those functionalities may also be associated with risk profiles, and the total risk level of the device may include a sum of risk levels for all tasks as well as of the other functionalities. Alternatively, the threshold risk levels of a device may be reduced or set to reflect the other functionalities implemented on the device, and the limited risk tolerance for executing tasks on such a device.

With reference toFIG. 4, a block diagram depicting an illustrative routine400for determining an execution location of a task within an on-demand code execution environment110will be described. The routine400may be implemented, for example, by the frontend120ofFIG. 1, alone or in conjunction with other components of the on-demand code execution environment110, such as the worker manager130.

The routine400begins at block402, where the on-demand code execution environment110obtains a task call, requesting execution of a task on the on-demand code execution environment110. For the purposes ofFIG. 4, it will be assumed that the task call is received at an individual POP105of a plurality of POPs105implementing instances of the on-demand code execution environment110. As described above, the task call may include an API call or other information transmitted to the POP105by a user device102or auxiliary service106, or may be generated at the POP105(e.g., based on instructions to periodically call the task, to call the task in response to information obtained from an auxiliary service106, etc.).

At block404, the on-demand code execution environment110loads an execution profile of the task, which indicates a set of computing resources expected to be utilized during execution of the task, and potentially a level of expected use of those computing resources. As noted above, the execution profile may be generated based on static analysis of the task, or dynamic analysis regarding prior executions of the task on the on-demand code execution environment110. In instances where no prior execution profile for a task is available to the on-demand code execution environment110, the on-demand code execution environment110(e.g., via the frontend120) may generate an execution profile for the task during implementation of block404.

At block406, the on-demand code execution environment110determines whether the execution profile of the task satisfies a set of resource restrictions associated with the POP105at which the request was received. The resource restrictions may indicate a set of computing resources available at the POP105, as well as a permitted usage of those computing resources (e.g., a permitted execution time, permitted RAM usage, etc.). Thus, by comparing an execution profile of the task to the resource restrictions of the POP105, the on-demand code execution environment110may determine, prior to attempting to execute the task, whether execution of the task is permitted at the POP105.

In the instance that execution of the task is not permitted at the POP105, the routine400continues to block408, where the request to execute the task is distributed to an alternative POP105. Illustratively, where the task call initially requested execution of a task at a POP105with relatively low computing resources, the task may be distributed to an alternative POP105with relatively high computing resources. In some instances, a frontend120handling the task call, during implementation of block408, may be configured to determine an alternative POP105on which execution of the task would be permitted, according to the execution profile of the task and the resource restrictions of the alternative POP105. In other instances, the frontend120may be configured to distribute any task not permitted to execute on the current POP105to a specified alternative POP105(e.g., a “default” POP105). In still other instances, the frontend120may be configured to distribute the task to an alternative POP105via a variety of load balancing techniques, such as round robin distribution, random distribution, etc. Load balancing techniques are known in the art and thus will not be described in detail herein. The routine400can then end at block418(and an additional implementation of the routine400may begin at the alternative POP105).

In the instance that execution of the task is permitted at the POP105, the routine400continues to block410, where a risk profile of the task is loaded. As described above, the risk profile indicates a risk level associated with executing the task, such as a risk that execution of the task will reduce the security or data privacy of other tasks or processes executing on the on-demand code execution environment110. In instances where no prior risk profile for a task is available to the on-demand code execution environment110, the on-demand code execution environment110(e.g., via the frontend120or a worker manger130) may generate a risk profile for the task during implementation of block410.

At block412, the on-demand code execution environment110determines whether a computing device, which may include virtual machine instances150or non-virtual computing devices, is available to be assigned to execute the task based on the task's risk profile. As described above, a computing device may be determined to be available to execute a task when the assignment of that task to the device does not cause the total risk level of the device to exceed a threshold value (e.g., as a whole or with respect to specific categories of risk). Accordingly, at block412, the on-demand code execution environment110can determine whether the risk profile of the called task allows that task to be assigned to an available device. While not shown inFIG. 4, implementation of block412may additionally include determining that any available device satisfies pre-requisites for execution of the task (e.g., required operating systems, runtimes, etc.), or that the available device can be loaded with such pre-requisites.

If no device is available to be assigned to the task (e.g., because assignment of the task would cause the device to exceed a threshold risk level), the routine400continues to block414, where the on-demand code execution environment110determines whether the task should be retried. Illustratively, the on-demand code execution environment110may select to retry tasks where the inability to assign the task is based on other tasks being executed by the on-demand code execution environment110, which may complete in the near future. In such instances, the on-demand code execution environment110may attempt to re-try the task at a future time. For example, the on-demand code execution environment110may place the task on a queue of tasks awaiting assignment to a device, or may impose a delay before reattempting to assign the task to a device. The on-demand code execution environment110may select not to retry tasks where it would not be possible to assign the task to a device (e.g., because the task has a risk level that could never permit it to be assigned to an instance), or where other criteria for abandoning assignment of a task have been met (e.g., a threshold number of prior attempts to assign the task, a threshold time elapsing since the task was called, etc.). In the instance that on-demand code execution environment110selects to retry assignment of the task, the routine400returns to block412. In the instance that the on-demand code execution environment110selects not to retry the task, the routine400ends at block418. In some instances, where the on-demand code execution environment110selects not to retry the task, an error notification may be sent to a user associated with the task.

In the instance that a device is located during implementation of block412that is available to be assigned to execute the called task, the routine400continues at block416, where the task is executed by the device. Illustratively, execution of the task may include loading of code corresponding to the task, as well as other data required to execute the task (e.g., runtimes), and executing that code.

At block417, the on-demand code execution environment110can utilize data regarding the execution to modify the execution profile of the task, the risk profile of the task, or both. For example, the on-demand code execution environment110may utilize logging data to determine the function calls made during execution of a task, or the computing resources used during execution of the task, and may update the execution and/or risk profiles in accordance with the dynamic analysis mechanisms described above. Thereafter, the routine400may end at block418.

The routine400may be altered based on the requirements of the on-demand code execution environment110. For example, in some embodiments of the present disclosure various functionalities described with respect to the routine400may be implemented in parallel, or as separate routines. For example, blocks404through408may be implemented as a first routine (e.g., executing by a frontend120on receiving a call to a task), blocks412and414may be implemented as a second routine (e.g., executed by a worker manager130, potentially in parallel with blocks404through408), and blocks416and417may be implemented as one or more additional routines. Division of the routine400into multiple routines may advantageously increase the speed of various functionalities of the routine400, for example, where the on-demand code execution environment110utilizes parallel processing techniques. In some embodiments, one or more portions of the routine400may be carried out by other systems or devices, including systems or devices external to the on-demand code execution environment, which provide information to the on-demand code execution environment110.

All of the methods and processes described above may be embodied in, and fully automated via, software code modules executed by one or more general purpose computers or processors. The code modules may be stored in any type of non-transitory computer-readable medium or other computer storage device. Some or all of the methods may alternatively be embodied in specialized computer hardware.

Unless otherwise explicitly stated, articles such as ‘a’ or ‘an’ should generally be interpreted to include one or more described items. Accordingly, phrases such as “a device configured to” are intended to include one or more recited devices. Such one or more recited devices can also be collectively configured to carry out the stated recitations. For example, “a processor configured to carry out recitations A, B and C” can include a first processor configured to carry out recitation A working in conjunction with a second processor configured to carry out recitations B and C.