APPLICATION DEPLOYMENT

The present disclosure generally relates deploying an application. Some techniques described herein occur during compile time while executable code is being generated from source code. In one example, the executable code causes different operations in an application to be assigned to different compute systems such that particular operations are required to be executed on particular compute systems. The executable code may further include bridges that assist data being transmitted between different compute systems, the bridges generated during compile time. In another example, the executable code causes data to be sent to a recording service during execution of an application. The recording service, though not included in the source code before compile time, is configured to receive copies of data transmitted on a compute system including the recording service. The recording service may also be configured to receive metadata corresponding to operations executed on the compute system.

BACKGROUND

Execution of an application is sometimes split between different compute systems. Ensuring that different parts of the application are executed on the correct compute systems is sometimes difficult. Accordingly, there is a need to better deploy an application on multiple compute systems.

SUMMARY

Current techniques for deploying an application on multiple compute systems are generally ineffective and/or inefficient. This disclosure provides more effective and/or efficient techniques for deploying an application. Such techniques optionally complement or replace other methods for deploying an application.

Some techniques described herein occur during compile time while executable code is being generated from source code. In one example, the executable code causes different operations in an application to be assigned to different compute systems such that particular operations are required to be executed on particular compute systems. The executable code is generated based on definitions identified using a hardware model, such as a definition that includes an identifier of a compute system, types of interfaces, and/or memory associated with the compute system. The executable code may further include bridges that assist data being transmitted between different compute systems, the bridges generated during compile time. In another example, the executable code causes data to be sent to a recording service during execution of an application. The recording service, though not included in the source code before compile time, is configured to receive copies of data transmitted on a compute system including the recording service. Different recording services may be added to different compute systems such that a respective recording service is specific to a respective compute system. The recording service may also be configured to receive metadata corresponding to operations executed on the compute system.

The disclosure herein often describes an application represented by a graph with nodes and edges to illustrates techniques. It should be understood that other types of architectures for an application may be used with techniques described herein to improve current techniques.

DESCRIPTION OF EMBODIMENTS

Some techniques described herein occur during compile time while executable code is being generated from source code. In one example, the executable code causes different operations in an application to be assigned to different compute systems such that particular operations are required to be executed on particular compute systems. The executable code is generated based on definitions identified using a hardware model, such as a definition that includes an identifier of a compute system, types of interfaces, and/or memory associated with the compute system. The executable code may further include bridges that assist data being transmitted between different compute systems, the bridges generated during compile time. In another example, the executable code causes data to be sent to a recording service during execution of an application. The recording service, though not in included in the source code before compile time, is configured to receive copies of data transmitted on a compute system including the recording service. Different recording services may be added to different compute systems such that a respective recording service is specific to a respective compute system. The recording service may also be configured to receive metadata corresponding to operations executed on the compute system.

Although the following description uses terms “first,” “second,” etc. to describe various elements, these elements should not be limited by the terms. In some examples, these terms are used to distinguish one element from another. For example, a first device could be termed a second device, and, similarly, a second device could be termed a first device, without departing from the scope of the various described embodiments. In some examples, the first device and the second device are two separate references to the same device. In some embodiments, the first device and the second device are both devices, but they are not the same device or the same type of device.

Turning now toFIG.1, a block diagram of compute system100is depicted. Compute system100is a non-limiting example of a compute system that may be used to perform functionality described herein. It should be recognized that other computer architectures of a compute system may be used to perform functionality described herein.

In the illustrated example, compute system100includes processor subsystem110coupled (e.g., wired or wirelessly) to memory120(e.g., a system memory) and I/O interface130via interconnect150(e.g., a system bus, one or more memory locations, or other communication channel for connecting multiple components of compute system100). In addition, I/O interface130is coupled (e.g., wired or wirelessly) to I/O device140. In some examples, I/O interface130is included with I/O device140such that the two are a single component. It should be recognized that there may be one or more I/O interfaces, with each I/O interface coupled to one or more I/O devices. In some examples, multiple instances of processor subsystem110may be coupled to interconnect150.

Compute system100may be any of various types of devices, including, but not limited to, a system on a chip, a server system, a personal computer system (e.g., an iPhone, iPad, or MacBook), a sensor, or the like. In some examples, compute system100is included with or coupled to a physical component for the purpose of modifying the physical component in response to an instruction (e.g., compute system100receives an instruction to modify a physical component and, in response to the instruction, causes the physical component to be modified (e.g., through an actuator)). Examples of such physical components include an acceleration control, a break, a gear box, a motor, a pump, a refrigeration system, a suspension system, a steering control, a vacuum system, and a valve. As used herein, a sensor includes one or more hardware components that detect information about a physical environment in proximity to (e.g., surrounding) the sensor. In some examples, a hardware component of a sensor includes a sensing component (e.g., an image sensor or temperature sensor), a transmitting component (e.g., a laser or radio transmitter), a receiving component (e.g., a laser or radio receiver), or any combination thereof. Examples of sensors include an angle sensor, a chemical sensor, a brake pressure sensor, a contact sensor, a non-contact sensor, an electrical sensor, a flow sensor, a force sensor, a gas sensor, a humidity sensor, an image sensor (e.g., a camera), an inertial measurement unit, a leak sensor, a level sensor, a light detection and ranging system, a metal sensor, a motion sensor, a particle sensor, a photoelectric sensor, a position sensor (e.g., a global positioning system), a precipitation sensor, a pressure sensor, a proximity sensor, a radio detection and ranging system, a radiation sensor, a speed sensor (e.g., measures the speed of an object), a temperature sensor, a time-of-flight sensor, a torque sensor, and an ultrasonic sensor. Although a single compute system is shown inFIG.1, compute system100may also be implemented as two or more compute systems operating together.

In some examples, processor subsystem110includes one or more processors or processing units configured to execute program instructions to perform functionality described herein. For example, processor subsystem110may execute an operating system, a middleware system, one or more applications, or any combination thereof.

In some examples, the operating system manages resources of compute system100. Examples of types of operating systems covered herein include batch operating systems (e.g., Multiple Virtual Storage (MVS)), time-sharing operating systems (e.g., Unix), distributed operating systems (e.g., Advanced Interactive eXecutive (AIX), network operating systems (e.g., Microsoft Windows Server), and real-time operating systems (e.g., QNX). In some examples, the operating system includes various procedures, sets of instructions, software components, and/or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, or the like) and for facilitating communication between various hardware and software components. In some examples, the operating system uses a priority-based scheduler that assigns a priority to different tasks that are to be executed by processor subsystem110. In such examples, the priority assigned to a task is used to identify a next task to execute. In some examples, the priority-based scheduler identifies a next task to execute when a previous task finishes executing (e.g., the highest priority task runs to completion unless another higher priority task is made ready).

In some examples, the middleware system provides one or more services and/or capabilities to applications (e.g., the one or more applications running on processor subsystem110) outside of what is offered by the operating system (e.g., data management, application services, messaging, authentication, API management, or the like). In some examples, the middleware system is designed for a heterogeneous computer cluster, to provide hardware abstraction, low-level device control, implementation of commonly used functionality, message-passing between processes, package management, or any combination thereof. Examples of middleware systems include Lightweight Communications and Marshalling (LCM), PX4, Robot Operating System (ROS), and ZeroMQ. In some examples, the middleware system represents processes and/or operations using a graph architecture, where processing takes place in nodes that may receive, post, and multiplex sensor data, control, state, planning, actuator, and other messages. In such examples, an application (e.g., an application executing on processor subsystem110as described above) may be defined using the graph architecture such that different operations of the application are included with different nodes in the graph architecture.

In some examples, a message sent from a first node in a graph architecture to a second node in the graph architecture is performed using a publish-subscribe model, where the first node publishes data on a channel in which the second node is able to subscribe. In such examples, the first node may store data in memory (e.g., memory120or some local memory of processor subsystem110) and notify the second node that the data has been stored in the memory. In some examples, the first node notifies the second node that the data has been stored in the memory by sending a pointer (e.g., a memory pointer, such as an identification of a memory location) to the second node so that the second node can access the data from where the first node stored the data. In some examples, the first node would send the data directly to the second node so that the second node would not need to access a memory based on data received from the first node.

Memory120may include a computer readable medium (e.g., non-transitory or transitory computer readable medium) usable to store program instructions executable by processor subsystem110to cause compute system100to perform various operations described herein. For example, memory120may store program instructions to implement the functionality associated with the flow described inFIGS.4and/or5.

Memory120may be implemented using different physical, non-transitory memory media, such as hard disk storage, floppy disk storage, removable disk storage, flash memory, random access memory (RAM-SRAM, EDO RAM, SDRAM, DDR SDRAM, RAMBUS RAM, or the like), read only memory (PROM, EEPROM, or the like), or the like. Memory in compute system100is not limited to primary storage such as memory120. Rather, compute system100may also include other forms of storage such as cache memory in processor subsystem110and secondary storage on I/O device140(e.g., a hard drive, storage array, etc.). In some examples, these other forms of storage may also store program instructions executable by processor subsystem110to perform operations described herein. In some examples, processor subsystem110(or each processor within processor subsystem110) contains a cache or other form of on-board memory.

I/O interface130may be any of various types of interfaces configured to couple to and communicate with other devices. In some examples, I/O interface130includes a bridge chip (e.g., Southbridge) from a front-side bus to one or more back-side buses. I/O interface130may be coupled to one or more I/O devices (e.g., I/O device140) via one or more corresponding buses or other interfaces. Examples of I/O devices include storage devices (hard drive, optical drive, removable flash drive, storage array, SAN, or their associated controller), network interface devices (e.g., to a local or wide-area network), sensor devices (e.g., camera, radar, LiDAR, ultrasonic sensor, GPS, inertial measurement device, or the like), and auditory or visual output devices (e.g., speaker, light, screen, projector, or the like). In some examples, compute system100is coupled to a network via a network interface device (e.g., configured to communicate over WiFi, Bluetooth, Ethernet, or the like).

FIG.2depicts a block diagram of device200with interconnected subsystems. In the illustrated example, device200includes three different subsystems (i.e., first subsystem210, second subsystem220, and third subsystem230) coupled (e.g., wired or wirelessly) to each other. An example of a possible computer architecture of a subsystem as included inFIG.2is described inFIG.1(i.e., compute system100). Although three subsystems are shown inFIG.2, device200may include more or fewer subsystems.

In some examples, some subsystems are not connected to other subsystem (e.g., first subsystem210may be connected to second subsystem220and third subsystem230but second subsystem220may not be connected to third subsystem230). In some examples, some subsystems are connected via one or more wires while other subsystems are wirelessly connected. In some examples, one or more subsystems are wirelessly connected to one or more compute systems outside of device200, such as a server system. In such examples, the subsystem may be configured to communicate wirelessly to the one or more compute systems outside of device200.

In some examples, device200includes a housing that fully or partially encloses subsystems210-230. Examples of device200include a home-appliance device (e.g., a refrigerator or an air conditioning system), a robot (e.g., a robotic arm or a robotic vacuum), and a vehicle. In some examples, device200is configured to navigate device200(with or without direct user input) in a physical environment.

In some examples, one or more subsystems of device200are used to control, manage, and/or receive data from one or more other subsystems of device200and/or one or more compute systems remote from device200. For example, first subsystem210and second subsystem220may each be a camera that is capturing images for third subsystem230to use to make a decision. In some examples, at least a portion of device200functions as a distributed compute system. For example, a task may be split into different portions, where a first portion is executed by first subsystem210and a second portion is executed by second subsystem220.

Attention is now directed towards techniques for deploying an application using an example of a graph application. The graph application uses a graph to model at least a portion of the application. The graph includes nodes to represent one or more operations and edges to represent channels for data to be sent between different nodes. It should be understood that other types of architectures for a software application are within scope of this disclosure and may benefit from techniques described herein. For example, an object oriented architecture with different objects communicating with each other may be used for deploying the application.

As used herein, source code (sometimes referred to as assembly, uncompiled, or programming code) includes code written by a programmer (sometimes referred to as a developer) using a programming language which is comprehensible to appropriately trained persons and is not capable of directly being used to give one or more instructions to a processor. As used herein, object code (sometimes referred to as executable code) includes machine code (e.g., code readable by a processor when loaded into memory and used directly by the processor to execute one or more instructions) and/or byte code (e.g., code readable by an interpreter during runtime and used by the interpreter to execute one or more instructions by a processor). As used herein, compile time is the time at which source code is converted into object code. In some examples, such conversion may be performed by a compiler and/or an assembler. As used herein, runtime is the time at which one or more instructions of object code are executing by a processor. In some examples, such execution may be performed by a program loader in response to a request to execute the object code. As used herein, a process is an instance of a computer program that is being executed by an operating system on one or more processors. As used herein, a node is an abstraction for one or more operations (e.g., software instructions) of a graph application. As used herein, an edge is an abstraction for sending and/or receiving data by a node. In some examples, an edge is a channel for which data is sent and/or received by a node, the channel used for sending and/or receiving data at different times. As used herein, “sending” data from one node to another node may refer to either actually sending the data to the other node or storing the data such that the other node may access the data (such as through a publication and subscription model). An example of actually sending the data to the other node includes addressing a message including the data to the other node. An example of storing the data includes notifying the other node that the data has been stored (and optionally notifying where the data has been stored through, for example, a pointer to a memory location) and the other node accessing the data where it has been stored. In another example, storing the data may not include notifying the other node that the data has been stored and instead the other node requests (e.g., at a fixed or variable rate) to access a known location for the data.

FIG.3is a block diagram illustrating different components of a graph application. The different components of the graph application include logical graph310, hardware model320, deployment model330, and deployment graph340. As further discussed below, logical graph310describes what the graph application is doing, hardware model320describes information about different compute systems, deployment model330assigns particular operations of the graph application to compute systems (or components of compute systems), and deployment graph340indicates the execution architecture of the graph application.

In some examples, source code of the graph application includes logical graph310and, optionally, hardware model320, deployment model330, or any combination thereof. For example, the source code may only include logical graph310. In such an example, hardware model and/or deployment model330may be generated during compile time based on logical graph310. For another example, the source code may include logical graph310and deployment model330. In such an example, deployment graph340may be generated during compile time based on logical graph310and deployment model330. It should be recognized that the graph application may include more or fewer components, including the combination of some components described herein.

As mentioned above, logical graph310describes what the graph application is doing. In some examples, logical graph310includes at least one node (each node representing one or more operations of the graph application) and zero or more edges (each edge connecting to at least one node representing data being sent or received by the node). For example, logical graph310may include a first node that is connected to a second node via an edge directed toward the second node, indicating that the first node is configured to send data to the second node. In some examples, logical graph310further includes where particular data is stored. In such examples, a location were the particular data is stored may correspond to an abstract location that is later connected to a compute system using a hardware model.

As illustrated inFIG.3, logical graph310is an input to deployment model330and deployment graph340. This indicates that deployment model330and deployment graph340may use information from logical graph310. It should be recognized that logical graph310may be an input to deployment model330and not deployment graph340.

In some examples, logical graph310is included in source code, before compile time. An example of source code for a logical graph with two camera nodes that are connected to a detector node is shown in the next 5 paragraphs.

The source code above includes two instantiations of a camera node (i.e., c1 and c2). The instantiations point to source code that define one or more operations of the camera node, such as an operation to capture an image and an operation to send the image to another node. The source code above also includes an instantiation of a detector node (i.e., d1). In some examples, the detector node corresponds to one or more operations associated with detecting a feature in a pair of images, such as an operation to receive an image from a camera node and an operation to determine differences between images. The source code above also uses interfaces of the nodes to indicate how the nodes are connected to each other through edges. In particular, both c1 and c2 include a respective “output” interface, and d1 includes a “c1_input” interface and a “c2 input” interface. The interfaces are then connected to each other to create an edge using an arrow (i.e., “->”). For example, “c1.output->d1.c1 input” corresponds to the “output” interface of c1 sending data to the “c1_input interface” of d1.

Referring back toFIG.3, the block diagram includes hardware model320. As mentioned above, hardware model320describes information about different compute systems. In some examples, such information includes a definition for each subsystem (e.g., first subsystem210) in a device (e.g., device200). Examples of information in a definition include a number of processors within a processor subsystem (e.g., processor subsystem110), a number of processor subsystems within a compute system (e.g., compute system100), different types of memory associated with a compute system (e.g., memory120or memory of processor subsystem110), an amount of different types of memory associated with a compute system, different types of I/O devices (e.g., I/O device140) within a compute system, different I/O interfaces (e.g., I/O interface130) within a compute system, a number of subsystems within a device (e.g., device200), identifying information of any component described in this list (e.g., a universally unique identifier (UUID), a media access control (MAC) address, or an Internet Protocol (IP) address), an identification of a particular interrupt (e.g., a software or hardware interrupt), and any other information related to a hardware component.

In some examples, hardware model320is included in source code, before compile time, based on an understanding of a system for which the graph application will be deployed. In other examples, hardware model320is automatically (e.g., without a programmer manually adding hardware model320to the source code) included, either before or while compiling, in the source code by a process based on an analysis of the system for which the graph application will be deployed. For example, the process may query the system to identify information about the system to add to a hardware model.

An example of source code defining a hardware model is shown in the next 17 paragraphs.

The source code defining the hardware model above includes two processors (i.e., cpu_0 and cpu_1) that are included in a subsystem (i.e., ss_1). Both processors each include two identifiers: a MAC address and an IP address. It should be recognized that one or more of the processors might include more or fewer identifiers. cpu_0 also includes memory in the form of L1 cache (i.e., l1_0). cpu_1 does not include any memory, indicating that cpu_1 does not have access to the L1 cache of cpu_0. ss1 includes identifiers to both CPUs that are included with the subsystem (i.e., cpu_0 and cpu_1). ss1 also includes three different types of memory, including L2 cache (i.e., l2_0), random access memory (i.e., ram_0), and non-volatile memory (i.e., rom_0). By ss1 including the different types of memory, the source code indicates that both processors have access to all three different types of memory, unlike with respect to the L1 cache of cpu_0. ss_1 also includes one I/O device in the form of a Bluetooth radio (i.e., bt_0). It should be recognized that other information and limitations of hardware may be defined in the hardware model, such as limitations on memory (e.g., local memory, write-only memory, read-only memory, read-write memory, or the like).

Similar to logical graph310, hardware model320is depicted as an input to deployment model330and deployment graph340. This indicates that deployment model330and deployment graph340may use information from hardware model320. It should be recognized that hardware model320may be an input to deployment model330and not deployment graph340.

As mentioned above, deployment model330assigns particular operations of the graph application to compute systems (or components of compute systems). Such assignments refer to establishing that particular compute systems will execute particular operations. In some examples, deployment model330includes assignments of nodes from logical graph310to compute systems from hardware model320. For example, a node may represent an operation and be assigned to a compute system such that the compute system should execute the operation rather than a different compute system, allowing to plan where different operations are executing before execution.

In some examples, deployment model330further assigns data to be stored in particular memory locations, such as data being sent for an edge. Such assignments may refer to establishing that a node will store data (1) at a particular location so that other nodes may know where to access the data or (2) on a different subsystem from where the node is executing so that the data is located on a subsystem for which it will be used for a calculation or determination.

In some examples, deployment model330defines one or more bridges for communicating outside of a compute system. For example, a bridge may be added to each compute system sending to or receiving from outside of the compute system. In such an example, a bridge on a first compute system receives data from a first node to send to a second node on another compute system. In some examples, the bridge packages the data in a particular format for transmission between the compute systems. The bridge then sends the packaged data to a bridge on the other compute system so that the bridge on the other compute system may send the data to the second node. For another example, a bridge may be added to a single compute system that handles sending data from the compute system directly to a node on another compute system, without going through a bridge on the other compute system.

In some examples, a bridge is defined for a compute system when there is an edge between the compute system and another compute system. Such bridges are often included with deployment model330before compiling when there are one or more constraints for the bridge that would not be identifiable by logic graph310, hardware model320, deployment model330, or any combination thereof. For example, a node for which a bridge is needed may not be included in logic graph310, hardware model320, or deployment model330. In such an example, the bridge may not be generated when deployment graph340is generated and would need to be included beforehand, such as in the source code that is being compiled. In some examples, different types of bridges that send data using particular protocols may be added to deployment model330, such as bridges sending data according to Transmission Control Protocol (TCP), User Datagram Protocol (UDP), or Controller Area Network (CAN).

In some examples, deployment model330is included in source code, before compile time, based on an understanding of a system for which the graph application will be deployed. In other examples, deployment model330is automatically (e.g., without a programmer manually adding deployment model330to the source code) included, either before or while compiling, in the source code by a software program based on an analysis of logical graph310and hardware model320. In some examples, deployment model330includes information from hardware model320such that the source code does not include hardware model320. An example of source code for a deployment model is shown in next 27 paragraphs.

The source code for the deployment model above indicates a single process to be created on each CPU (i.e., cpu_0 and cpu_1). In some examples, a process includes multiple threads of execution that execute instructions concurrently. While the source code only includes a single process on each CPU, it should be recognized that more processes may be executing on a single CPU. The source code also indicates that the camera nodes (i.e., c1 and c2) are to be executed on a first process (i.e., p1) and the detector node (i.e., d1) are to be executed on a second process (i.e., p2). The source code also defines that the input of c1 will be stored in RAM of ss1 (i.e., ram_0), the output of c1 at c1.output will be stored in the L1 cache of cpu_0 (i.e., l1_0), and the output of c2 at c2.output will also be stored in the L1 cache of cpu_0 (i.e., l1_0). Although not included in the source code above, definitions of where the input and/or output of an edge is to be put into memory may further define a size that is expected (e.g., 10 MB). The code above does not define where other data will be stored, indicating that the compiler, one or more processors, and/or operating system associated with c1, c2, and d1 may choose where to store data. It should be recognized that c1.input, c1.output, and c2.output may not be defined where the data will be stored such that the compiler, one or more processors, and/or operating system associated with c1 and c2 may choose where to store the outputs.

As illustrated inFIG.3, deployment model330is an input to deployment graph340. This indicates that deployment graph340may use information from deployment model330.

As mentioned above, deployment graph340indicates the execution architecture of the graph application. In some examples, deployment graph340includes one or more nodes, each node representing one or more operations performed during execution of the graph application, and zero or more edges, each edge representing a channel for data to be sent between different nodes. As depicted, deployment graph340is generated using logical graph310, hardware model320, and deployment model330. It should be recognized that deployment graph340may be generated using some other combination of components (e.g., logical graph310and hardware model320, logical graph320and deployment model330, or just deployment model330).

In some examples, deployment graph340indicates assignments of nodes to compute systems (e.g., compute system100) and is used during runtime when executing the graph application to ensure nodes are executed on proper compute systems. In some examples, deployment graph340includes additional components that were not defined for the graph application before compiling, such components facilitating execution of the graph application on the proper compute systems. Examples of such additional components include instruction sets corresponding to different compute systems, code to check information about a particular compute system, bridges between different compute systems, timer nodes, recording services, and additional edges to facilitate other additional components.

In some examples, deployment graph340includes object code that is executable by a compute system (e.g., compute system100), either directly by a processor subsystem (e.g., processor subsystem110) or through an interpreter during runtime. In such examples, the deployment graph340may include executable code that is configured to require particular nodes to be executed by particular compute systems. Operations corresponding to such executable code may have not been defined before compiling, such that there was no requirement that particular nodes be executed by particular compute systems before compiling.

In some examples, the executable code requires such execution by using an instruction set specific to a particular compute system. In such examples, generating deployment graph340includes, for each compute system, identifying an instruction set for the compute system, identifying one or more nodes to be executed by the compute system, and generating executable code (e.g., byte, machine, or object code) according to the instruction set for the one or more nodes.

In other examples, the executable code requires such execution by including executable code to check one or more characteristics of a compute system that the executable code is executing. The one or more characteristics may include any information included in hardware model310, such as identifying information associated with the compute system, a particular type of I/O device, or a particular amount of memory accessible by the compute system. In some examples, the executable code to check the one or more characteristics may perform the check and only continue to execute code associated with different nodes when the check is successful.

In some examples, one or more bridges are generated for deployment graph340while compiling the graph application. Such generation may be performed for an edge when information is known with respect to compute systems on both sides of the edge, such as an incoming and outgoing format of data. Similar to as described above with respect to deployment model330, a bridge may be added to each compute system sending to or receiving from outside of the compute system. In such an example, a bridge on a first compute system receives data from a first node to send to a second node on another compute system. In some examples, the bridge packages the data in a particular format for transmission between the compute systems. The bridge then sends the packaged data to a bridge on the other compute system so that the bridge on the other compute system may send the data to the second node. For another example, a bridge may be added to a single compute system that handles sending data from the compute system directly to a node on another compute system, without going through a bridge on the other compute system.

In some examples, different types of bridges are automatically added between different nodes based on capabilities of each compute system. For example, it may be determined that a pair of compute systems, or a particular operation, supports a particular type of bridge and that type of bridge should be used, such as bridges sending data according to Transmission Control Protocol (TCP), User Datagram Protocol (UDP), or Controller Area Network (CAN).

In some examples, one or more timer nodes are generated for deployment graph340while compiling the graph application. Such generation may be performed for a node when the node requires a current time during operation (e.g., code associated with the node includes an indication that time is needed by the node during execution). For example, a node may include an operation that captures an image and sends the image with a timestamp of when the image was captured. Instead of the node calling a function to receive a current time from an operating system or compute system, the node will receive a current time (e.g., an activation or scheduled time) at execution from a timer node and use the received time as the timestamp. Such generation may also be performed for a node when the node is required to operate at some time interval (e.g., code associated with the node includes an indication that the node should be executed at some time interval). For example, a node may include an operation that executes every 5 seconds. In such an example, a timer node may be added to begin execution of the node every 5 seconds, providing the node a time corresponding to (e.g., an activation or scheduled time) when 5 seconds would have passed even if there was some time required to begin the execution and a current time is after the time corresponding to when 5 seconds would have passed.

In some examples, the time provided to a node is different from a time provided by a hardware component of a compute system. For example, the compute system may include a hardware component that keeps time for the compute system. The hardware component may provide a current time to a process managing time for the graph application, such as in response to a request by the process for the current time. The process may then use time provided by the hardware component to determine when to provide messages with a current time to nodes of the graph application. In some examples, the process manages a global graph-level clock that all nodes in a graph (e.g., a timer node) read current time from. The goal of the global graph-level clock is to abstract the hardware/platform time in order to allow, in some examples, for time to be simulated and/or achieve determinism and reproducibility regardless where playback runs. When executing offline, the process manages the global graph-level clock and simulates time for all nodes in the graph during playback and simulation using it. The process may then use time provided by the hardware component to determine when to provide messages with a current time to nodes of the graph application.

In some examples, the hardware component keeping time for the compute system is synchronized with hardware components of one or more other compute systems that are included in a system executing the graph application. In such examples, synchronized time may allow for reproducible playback by providing a single clock for captured events. Such synchronization may occur according to any known time synchronization mechanism (e.g., clock-sampling mutual network synchronization, Network Time Protocol (NTP), or Precision Time Protocol (PTP)) in response to execution of a graph application, boot up of a compute system, or a compute system connects (or reconnects) to a network (e.g., a physical network, such as ethernet) including another compute system, a compute system joins or rejoins the system.

In some examples, one or more recording services are generated for deployment graph340while compiling the graph application. Such generation may cause a recording service to be added to a compute system during runtime360(e.g., after compiling or executing the graph application). In some examples, the recording service is added to a compute system that is not executing a node of the graph application. In other examples, the recording service is added to a compute system executing a node of the graph application. In other examples, a recording service is added to each compute system that is executing a node of the graph application. In such examples, a recording service on a compute system may not be aware of another recording service on another compute system. Instead, each recording service may only be aware of what occurs on their respective compute system.

In some examples, a recording service on a compute system is configured to receive data sent on edges originating on the compute system. For example, there may be an edge added to connect every edge originating on the compute system to the recording service. In some examples, the added edges for existing edges are generally recording content of data sent along the edges. In addition to edges added for existing edges, edges may also be added between each node on the compute system to the recording service. In some examples, the added edges for nodes are generally recording metadata about data sent on edges and execution of nodes. The added edges for existing edges and/or for nodes may be optional for some nodes or edges, such as when the recording service is configured to not record data for the nodes or edges. Data may not be recording to reduce overhead in a system by reducing the number of recording channels or data being recorded in the system, for example, when particular nodes or edges are not needed for playback or it is determined to re-create such data offline. To illustrate, a simple graph includes two nodes connected together: A and B; [A]->[B]. As node A is publishing data, content of the data is recorded via an edge added to the edge between A and B while the metadata about node A is recorded via an edge added to node A. The recording of the content of the data from node A to node B may not be needed if only node A is being played back or both nodes A and B together. However, the recording of the content of the data from node A to node B is needed if playing back node B without playing back node A.

In some examples, a recording service sends received data to an ingestion process to gather data from different compute systems together. In other examples, a recording service temporarily stores (e.g., in local storage or a buffer, such as a ring buffer) received data and does not send to an ingestion process until the recording service determines that the received data should be maintained. When the recording service determines that the received data should be maintained, the recording service may send the received data to the ingestion process. In some examples, the determination to send the received data to the ingestion process is based on whether the recording service (or another process) determines that an event occurred that needs to be recorded. In such examples, different types of events may cause the recording service to send different amounts of data to the ingestion process. For examples, a first type of event may cause the recording service to send data corresponding to the last 5 minutes to the recording service while a second type of event may cause the recording service to send data corresponding to the last 10 minutes.

In some examples, the ingestion process is executing on a compute system that is not executing a node of the graph application. In some examples, the ingestion process receives data from each recording service associated with the graph application. In such examples, the ingestion process creates a package of the data for further processing, such as for playback. The package may include data sent to different nodes (e.g., obtained by a recording service and sent to the ingestion process) and metadata corresponding to execution of the nodes (e.g., sent to a recording service and further sent to the ingestion process).

In some examples, the package is used to playback execution of at least a portion of the graph application, such as to simulate a previous execution for purposes of training a model, viewing what occurred, debugging an issue, or any other purpose that requires re-executing operations (e.g., generating intermediate data that was not recorded originally due to hardware constraints). In some examples, observed activities in the package (including time) are abstracted to be events. For example, data being stored, triggers being sent/received, nodes executing, and time are all made into events that are processed in time order during playback. In such an example, time is simulated, such as by using a timer node to send the simulated time to nodes. In some examples, this allows for deterministic playback regardless of where the playback is running (on the same or different hardware than the events were recorded using). In such examples, because time is simulated, playback may run faster than real time. The timer nodes may be managed by a process using a global clock that ensures all nodes receive current simulated times and timer nodes on different compute systems are synchronized. In some examples, the process using the global clock determines a current simulated time based on graph events and sends all timer nodes in the graph the current simulated time for sending to other nodes. In such examples, the process may schedule timer nodes to execute and send a current simulated time to different nodes to activate the respective nodes. By using simulated time to create a simulated timeline of graph events, different execution models and/or scenarios that were not necessarily recorded are able to be executed by adding (e.g., synthetically generated events), changing, and/or removing graph events.

In some examples, where the ingestion process receives data from one or more recording services without the recording services determining that an event occurred, the ingestion process may determine whether the event occurred before packaging the data for further processing. In such examples, the ingestion process may temporarily store the data in a buffer (e.g., a ring buffer) until a threshold time elapses or a decision is made to maintain the data. Similar to above, the ingestion process may store different amounts of data based on a type of event detected.

In some examples, a list of nodes and/or edges associated with a recording service is provided to a process during execution of the graph application. The process is then configured to establish channels to have data associated with the nodes and/or edges sent to the recording service. In some examples, the same process or another process is configured during execution to create an identifier (e.g., a universally unique identifier (UUID)) for execution of the graph application and provide the identifier to each recording service. In such examples, the identifier is used by each recording service to record data received so as to associate the data with a particular execution of the graph application. In this way, multiple graph applications may be executing at the same time and storing data with different identifiers to differentiate between different graph applications.

In some examples, a recording service is configured to obtain data related to execution of the graph application. The data may be used for playback of the execution of the graph application on the same or different set of compute systems. In some examples, the data is received by the recording service in a message that also includes a timestamp (e.g., the time when an event occurred) and an identity of an event (e.g., an identifier of an edge or an operation that generated the event). An example of a timestamp includes a current time as tracked by a hardware component of a compute system rather than a time received from a timer node. This means that the recording service will be receiving a time that is potentially different from a time that a node is using to operate (i.e., the time received from the timer node). Another example of a timestamp includes a time received from a timer node.

Examples of data obtained by a recording service include (1) any data stored, accessed, and/or received by a node of the graph application and (2) metadata (sometimes referred to as execution metadata) indicating state and/or operations occurring on a node of the graph application during execution. In some examples, the metadata is used to restore the state of a node at any arbitrary time during playback. In such examples, the node outputs a persisted state message that includes information needed to restore its state. The persisted state message is written to a data edge, which is captured via a data channel of the recording service. In some examples, the node can be triggered to output the persisted state message at some time interval using the timer node. The persisted state message is then used offline by a process to restore the node state if playback was requested to start at an arbitrary time of the package.

In some examples, a recording service receives data stored, accessed, and/or received by a node of the graph application by subscribing to edges included in the graph application. For example, a node of the graph application may be subscribed to receive data from another node in the graph application. In such an example, the recording service may subscribe to receive the same data from the other node. In some examples, such a subscription may require the other node to send the data to both the node and the recording service. In other examples, such a subscription may require the other node to send a notification to both the node and recording service so that each of the node and the recording service may independently access the data based on information provided in the notification. In such examples, the message received by the recording service may include (1) a location within memory to access the data and (2) an identifier used to identify a channel and an identifier used to identify a specific message.

In some examples, a recording service receives metadata from a node by subscribing to a node in the graph application. In such examples, a process associated with the graph application (e.g., other than the recording service) identifies nodes executing on a compute system and establishes a single metadata channel from each node to the recording service. A metadata channel may be configured to send data related to execution of a respective node to the recording service.

An example of metadata includes a node launch event. The node launch event indicates that a respective node has been initiated (sometimes referred to as launched) and is executing on a compute system. In some examples, the node launch event is sent (1) when the node begins executing for the first time, (2) when the node has been restarted, (3) or any combination thereof. The node launch event allows for playback to identify when to launch a node.

Another example of metadata includes a trigger event. The trigger event indicates when a trigger (sometimes referred to as a notification) has been sent or received by a node. In some examples, the trigger event is sent (1) when a sending node (e.g., a timer node or a node executing one or more operations of the graph application) sends one or more triggers to one or more receiving nodes, (2) when a receiving node is executing and acknowledges that the receiving node has received a trigger from a sending node (e.g., a timer node or a node executing one or more operations of the graph application), (3) when a receiving node transitions from a sleep mode to an active mode and acknowledges that the receiving node has received one or more triggers from one or more sending nodes (e.g., a timer node or a node executing one or more operations of the graph application) (in some examples, there is a different trigger event for each sending node), or (4) any combination thereof. In some examples, the trigger event includes a number of triggers sent and/or received. For example, a sending node may publish three different messages to one or more subscribers of a data channel, and the sending node will send a trigger event with count of three indicating that three different messages are available on that data channel. For another example, a receiving node may begin executing and have received three triggers since the receiving node last executed or checked whether the receiving node has received a trigger. In such an example, receiving node will send a trigger event indicating a count of three triggers have been received. The trigger event allows for playback to identify communications that are sent across different nodes during execution.

Another example of metadata includes a data range event. The data range event indicates an amount or range of data that is available for a node to read. In some examples, the data range event is sent when a node is about to execute an operation that operates on data (e.g., in response to determining to execute an operation based on the data). The data range event allows for playback to identify a state known by a node with respect to data that needs to be read when the node begins execution.

Another example of metadata includes a read event. The read event indicates that a node has initiated or finished reading data. In some examples, the read event is sent when a node executes a command to read or when the node receives an indication that a read has finished. In some examples, the read event does not include the data being read. Instead, the read event includes an indication of (1) an identifier used to identify an edge and a memory location corresponding to the data being read, (2) an identifier used to identify an edge and an identifier used to identify a specific message (e.g., a message including the data or a message sending the read request), (3) a message including the data, or (4) any combination thereof. The indication can then be used to obtain the data at a later time to be used for playback. In some examples, the read event includes an indication of a status of the read, such as that the read has been initiated or that the read was successful or unsuccessful. The read event allows for playback to identify when and what to read without having to include the data being read at the time of sending the read event.

Another example of metadata includes a write event. The write event indicates that a node has initiated or finished writing data. In some examples, the write event is sent when a node executes a command to write or when the node receives an indication that a write has finished. In some examples, the write event does not include the data being written. Instead, the write event includes an indication of (1) an identifier used to identify an edge and a memory location corresponding to the data, (2) an identifier used to identify an edge and an identifier used to identify a specific message (e.g., a message including the data or a message sending the write request), (3) a message including the data, or (4) any combination thereof. The indication can then be used to obtain the data at a later time to be used for playback. In some examples, the write event also includes an indication of a status of the write, such as that the write has been initiated or that the write was successful or unsuccessful. The write event allows for playback to identify when and what to write without having to include the data being written at the time of sending the write event.

Another example of metadata includes a function event. The function event indicates that a function (sometimes referred to as an operation) has initiated or finished execution. In some examples, the function event is sent (1) each time an operation is executed, (2) when an indication is received that the operation has finished executing, (3) when a subsequent operation begins executing, indicating that the operation has finished executing, or (4) any combination thereof. The function event allows for playback to identify when to launch an operation and/or when to wait for a certain operation to terminate before processing the next event and advancing simulated time. For example, a function event indicating when an operation has launched may be used during playback to determine when to launch the operation, and a function event indicating when the operation has finished may be used during playback to determine when to wait for the operation to finish. Tracking both of these times allows for playback to simulate execution duration of an operation accurately, impacting when output of the operation is visible to other nodes.

In some examples, recording services on different compute systems may be subscribed to all nodes on their respective compute systems such that recording services do not receive metadata from nodes executing on different compute systems.

Referring toFIG.3, deployment graph340is used during runtime360. In particular, deployment graph340may be generated while compiling the graph application and the resulting executable code is executed on one or more compute systems during runtime360. In some examples, deployment graph340requires that portions of the resulting executable code be executed on particular compute systems during runtime360as further discussed in the figures below. For example, a process (e.g., a node part of the graph application, such as the recording service, or a separate process) may execute on a compute system and establish one or more edges for a recording service. In such an example, the process may use a list of nodes and edges associated with one or more compute systems to establish data channels for each edge and metadata channels for each node. The data channels may duplicate data being sent for the graph application, such as message content sent between nodes, timing information sent by timer nodes, and the like. In some examples, the recording service may access the data without the data channel needing to duplicate the data. The metadata channels may be used by nodes to send metadata corresponding to execution of nodes (e.g., triggers sent and received and other information metadata discussed herein), to the recording service.

FIGS.4-6are block diagrams illustrating different components of a graph application being used during runtime. The block diagrams are intended to merely be examples. It should be recognized that alternative configurations may be used.

FIG.4is a block diagram illustrating logical graph400for a graph application. Logical graph400includes multiple nodes (e.g., first node410, second node420, third node430, fourth node440, fifth node450, sixth node460, and seventh node470). In some examples, a node corresponds to one or more operations that are logically grouped together into a single operational unit.

Some of the nodes in logical graph400are connected via an edge to another node in logical graph400. An edge corresponds to data being sent from one node to another, shown using a line connecting two different nodes (e.g., the line between first node410and third node430). In some examples, an edge is directed such that it indicates the direction of data, such as first node410is sending data to third node430through edge412.

Some of the edges in logical graph400are only connected to one node, illustrated by an arrow starting from a location not corresponding to a node and connecting to a node (e.g., edge408and edge418). Such origin-less edges indicates that data is coming into or out of a node from or to a location with one or more operations that are not known by the application. An example of such a location is a third-party sensor and/or a remote server that is configured to send data to a node within the graph application. For example, the graph application may require that a node identify a current weather but the graph application may not include code for sensing the current weather. Instead, the graph application may use an application programming interface (API) and/or other method for obtaining the current weather from another compute system to receive an identification of the current weather.

As depicted inFIG.4, first node410receives data through edge408. First node410may execute one or more operations based on data received through edge408and send data to third node430through edge412. In some examples, first node410receives data at different times through edge408and sends data at different times through edge412. Second node420receives data through edge418. Second node420may execute one or more operations based on data received through edge418and send data to third node430through edge422. Third node430may execute one or more operations based on data received through edge412and edge422and send data to fourth node440through edge432. Fourth node440may execute one or more operations based on data received through edge432and send data to (1) fifth node450through edge442, (2) sixth node460through edge444, and (3) seventh node470through edge446. In some examples, the data sent from fourth node440to each other node may be the same or different. For example, fourth node440may store data at a particular location and send a trigger indicating that the data has been stored to each of fifth node450, sixth node460, and seventh node470. While not illustrated inFIG.4, it should be recognized that an operation executed by a node may cause data to be sent to a node not included in logical graph400and/or stored in memory not included in logical graph400.

FIG.5is a block diagram illustrating deployment graph500for a graph application. In some examples, deployment graph500is generated while compiling the graph application, such that deployment graph500was not defined before compiling the graph application.

Deployment graph500is based on logical graph400, in that nodes and edges defined in logical graph400are included in deployment graph500. Deployment graph500includes three different subsystems (i.e., first subsystem510, second subsystem520, and third subsystem530). In some examples, each of the subsystems may be defined in a hardware model, as described above.

As depicted inFIG.5, first subsystem510includes first node410and second node420. This indicates that operations performed by each of first node410and second node420are executed by one or more processors included in first subsystem510. In some examples, such operations are performed concurrently by multiple processors and/or through whatever scheme is enforced by an operating system executing on first subsystem510. Edge408and edge418are illustrated as beginning outside of first subsystem510. This indicates that data received through edge408and edge418is from outside of first subsystem510. In some examples, an origin-less edge may begin inside of first subsystem510, indicating that data is being produced by a process executing on first subsystem510that is not defined in deployment model500. As mentioned with respect toFIG.4, first node410sends data to third node430through edge412.FIG.5depicts that edge412begins within first subsystem510and ends within second subsystem520, indicating that data will be transferred from one subsystem to another. Similarly, edge420begins within first subsystem510and ends within second subsystem520, indicating that data will be transferred from one subsystem to another.

Second subsystem520includes third node420. This indicates that operations performed by third node430are executed by one or more processors included in second subsystem520. In some examples, such operations are performed concurrently by multiple processors and/or through whatever scheme is enforced by an operating system executing on second subsystem520(potentially a different operating system from the operating system executing on first subsystem510). As mentioned with respect toFIG.4, third node430sends data to fourth node440through edge432.FIG.5depicts that edge432begins within second subsystem520and ends within third subsystem530, indicating that data will be transferred from one subsystem to another.

Third subsystem530includes fourth node440, fifth node450, sixth node460, and seventh node470. This indicates that operations performed by each of fourth node440, fifth node450, sixth node460, and seventh node470are executed by one or more processors included in first subsystem510. In some examples, such operations are performed concurrently by multiple processors and/or through whatever scheme is enforced by an operating system executing on third subsystem530(potentially a different operating system from the operating system executing on first subsystem510and/or second subsystem520). As mentioned with respect toFIG.4, fourth node430sends data to fifth node450through edge442, sixth node460through edge444, and seventh node470through edge446.FIG.5depicts that edge442, edge444, and edge446begin and end within third subsystem530, indicating that data will not be sent to another subsystem. Based on discussion above, it should be recognized that data associated with edge442, edge444, or edge446may be stored in memory associated with fourth node440, fifth node450, sixth node460, or seventh node470and accessed by a node requiring the data.

FIG.6is a block diagram illustrating deployment graph600with bridges and recording services for a graph application. In some examples, deployment graph600is generated while compiling the graph application, such that deployment graph500was not defined before compiling the graph application.

Deployment graph600is based on logical graph400, in that nodes and edges defined in logical graph400are included in deployment graph600. In some examples, deployment graph600is an alternative or further processed version of deployment graph500as depicted inFIG.5. The description below forFIG.6will add to the description ofFIG.5, intending to add to what was included in deployment graph500.

As depicted inFIG.6, first subsystem510further includes recording service610. In some examples, recording service610was not defined in the graph application before compiling the graph application. In such examples, recording service610is configured to receive data corresponding to each edge associated with first subsystem510. For example, recording service610is configured to receive a copy of data provided to first node410through edge408. In some examples, generating and sending the copy is performed by an I/O device and/or interface (e.g., I/O interface130or I/O device140) or an operating system of first subsystem510. In such examples, the I/O device and/or interface or the operating system receives data from a device external to first subsystem510and sends the data to both first node410and recording service610. In other examples, generating and sending the copy is performed by first node410. In such examples, the first node410receives data from a device external to first subsystem510and sends the data to recording service610(e.g., when first subsystem510is not performing another operation, such as after waiting until finished performing operations based on data received through edge408). Similar operations may be performed with data with respect to second node420and edge418. For another example, recording service610is configured to receive a copy of data provided to third node430through edge412. In some examples, generating and sending the copy is performed by an I/O device and/or interface (e.g., I/O interface130or I/O device140) or the operating system of first subsystem510. In such examples, the I/O device and/or interface or the operating system receives data from first node410and sends the data to both third node430and recording service610. In other examples, generating and sending the copy is performed by first node410. In such examples, the first node410send data to both third node430and recording service610(e.g., sent to recording service610when first subsystem510is not performing another operation, such as after waiting until finished performing operations based on data received through edge408). In some examples, recording service610subscribes to a data channel corresponding to edge412such that recording service610is notified when first node410publishes to the data channel. In other examples, no copy is made but the recording service has direct access to the data via edge408. Similar operations may be performed with respect to second node420and edge422.

Recording service610is also configured to receive data (e.g., metadata, as discussed above) corresponding to operations performed by each node of first subsystem510. In some examples, such data is sent by first node410or the operating system of first subsystem510when first subsystem510is not performing another operation, such as after waiting until finished performing operations based on data received through edge408or edge418. Similar operations may be performed by second node420.

In some examples, edges to recording service610are configured to be created during execution. In such examples, a process (e.g., recording service610or another process executing on first subsystem510) identifies a list of nodes executing on first subsystem510and a list of edges associated with first subsystem510. The process, using the list of nodes, add a channel (sometimes referred to as a data channel) for each node to connect to recording service610to ensure that operations being performed by nodes within first subsystem510are reported to recording service610. The process, using the list of edges, adds a channel for each edge to connect to recording service610to ensure that data (or pointers to data) that is being passed within first subsystem510are provided to recording service610.

FIG.6illustrates that similar operations discussed above with respect to recording service610occur with respect to separate recording services on subsystem520(e.g., recording service620) and subsystem530(e.g., recording service630). In some examples, data received by a node on a subsystem from another node on another subsystem is not sent to a recording service of the system. For example, data corresponding to edge412is not sent to recording service620, as illustrated inFIG.6. In other examples, data received by a node on a subsystem from another node on another subsystem is sent to a recording service of the system. For example, data corresponding to edge412may be sent to recording service620(similar to as described above with respect to edge408and recording service610), as not illustrated inFIG.6.

FIG.6also illustrates that each recording service is configured to send data out of their respective subsystem, such as recording service610through edge612. In other examples, the recording service may be configured to save the data to local storage via an application programming interface (API) instead of sending data through an edge. Such data may correspond to data received by recording service610(e.g., data associated with edge408, first node410, and/or edge412). In some examples, such data is temporarily stored in local storage or a buffer (e.g., a ring buffer) on first subsystem510and only sent via edge612when it is determined that an event occurred. The buffer may be configured to store data that comes in and store such data for a period of time, such as using a first-in-first-out mechanism that removes the oldest data first when new data arrives and additional space is needed.

In some examples, the determination that an event occurred may be made by recording service610, a node on first subsystem510(e.g., first node410or second node420), another node on another subsystem (e.g., third node430or recording service620), or a process associated with the graph application that is executing on one of the subsystems (e.g., first subsystem510, second subsystem520, or third subsystem530), or a process executing on a remote device (e.g., a notification system that is wirelessly communicating with first subsystem510, second subsystem520, and/or third subsystem530. In such examples, the determination may include determining a type of the event and, based on the type of event, determining an amount of data to send via edge612. For example, different types of events may cause different amounts of data to be sent via edge612, such as data corresponding to 5 mins in the past is sent for a first type of event and data corresponding to 10 mins in the past is sent for a second type of event.

In some examples, data sent via edge612includes an identifier (e.g., UUID) associated with a particular execution of the graph application. For example, at the start of execution of the graph application, recording service610may be provided the identifier to be used for sending data associated with the execution outside of first subsystem510.

FIG.6further depicts that second subsystem520further includes recording service620and third subsystem530includes recording service630. It should be recognized that recording service620and recording service630operate similar to recording service610described above.

An example of a detector node receiving images from two camera nodes will now be discussed usingFIG.6. It should be recognized that this example is used to show how deployment graph600may be used in runtime and what data is being sent to a recording service. The example will only refer to operations associated with second subsystem520. In the example, first node410and second node420are camera nodes that are each configured to store an image in memory of second subsystem520and notify third node430that the images have been stored. The third node430is a detector node that is configured to activate in response to receiving triggers from both first node410and second node420indicating that each has stored an image. The third node, after activating, is configured to determine whether there is an error with the images and notify fourth node440whether there is an issue.

The example starts with execution of a graph application, including executing third node430and recording service620on second subsystem520. After initiating execution of recording service620, third node430is identified and metadata channel624is established between third node430and recording service620. In addition, edge432is identified and data channel626is established between third node430and recording service620for edge432. In some examples, recording service620identifies and establishes metadata channel624and data channel626. In other examples, a different process (other than recording service620) identifies and establishes metadata channel624and data channel626.

After initiating execution of third node430, third node430generates a node launch event and sends the node launch event to recording service620through metadata channel624. The node launch event indicates that third node430began execution at a particular time. The third node430, after sending the node launch event, then transitions to a sleep mode to wait to receive triggers from both first node410and second node420.

After third node transitions to the sleep mode, first node410stores a first image in memory of second subsystem520. Storing the first image includes sending the first image through edge412and bridge612to the memory of second subsystem520and notifying third node430via a trigger that the first image has been stored in the memory. Based on receiving only the one notification, third node430transitions into an active mode but does not attempt to obtain the first image or do any operations using the first image. Instead, third node430generates and sends a trigger event with a count of one and transitions back to a sleep mode. Then, third node430continues to wait to receive a notification from second node420. The storage of the first image may not cause any data to be sent to recording service620via metadata channel624. Instead, data (e.g., the first image) may be sent to recording service620via a data channel corresponding to edge412(e.g., a copy of edge412on second subsystem520, similar to edge608) that is connected to recording service620(not illustrated inFIG.6).

At a later time, second node420stores a second image in the memory of second subsystem520. Storing the second image includes sending the second image through edge522and bridge614to the memory of second subsystem520and notifying third node430via a trigger that the second image has been stored in the memory.

In response to being notified of both the first image and the second image, third node420transitions from the sleep mode to the active mode and identifies (e.g., based on the notifications) data available (e.g., a range of memory locations) for third node430to access, including data associated with the first image and the second image. In response to identifying the available data, third node430generates a data range event and sends the data range event to recording service620through metadata channel624. The data range event indicates that third node430has identified the available data at a particular time.

In addition to the data range event (either before, at the same time or after sending the data range event), third node430identifies a trigger received from node420. In response to identifying the trigger, third node430generates a trigger event and sends the trigger event to recording service620through metadata channel624. The trigger event indicates that third node430has identified the count of triggers received at a particular time (e.g., third node430has received two triggers since last being in the active mode).

After sending the trigger event and/or the data range event, third node430executes a start function. In response to executing the start function, third node430generates a first function event and sends the first function event to recording service620through metadata channel624. The first function event indicates that third node430began executing the start function at a particular time.

As part of the first function, third node430sends a first instruction to obtain the first image. In addition to the first instruction, third node430generates a first read event and sends the first read event to recording service620through metadata channel624. The first read event indicates that third node430is requesting to read the first image at a particular time.

In addition to the first instruction (either at the same time or after sending the first instruction), third node430sends a second instruction to obtain the second image. In addition to the second instruction, third node430generates a second read event and sends the second read event to recording service620through metadata channel624. The second read event indicates that third node430is requesting to read the second image at a particular time.

After sending the first instruction, third node430determines that third node430has finished reading the first image. In response to this determining, third node430generates a third read event and sends the third read event to recording service620through metadata channel624. The third read event indicates that third node430has finished reading the first image at a particular time. Third node430stores the first image in memory.

After sending the second instruction, third node430determines that third node430has finished reading the second image. In response to this determining, third node430generates a fourth read event and sends the fourth read event to recording service620through metadata channel624. The fourth read event indicates that third node430has finished reading the second image at a particular time. Third node430stores the second image in memory.

After executing the start function, third node430executes a comparison function to determine whether there is an issue when comparing the first image in memory with the second image in memory. In response to executing the comparison function, third node430generates a third function event and sends the third function event to recording service620through metadata channel624. The third function event indicates that third node430began executing the comparison function at a particular time.

As part of the comparison function, third node430initiates writing a result of whether there is an issue to the memory of second subsystem520. In response to initiating the writing, third node420generates a first write event and sends the first write event to recording service620through metadata channel624. The first write event indicates that third node430is requesting to write the result at a particular time.

After initiating the writing, third node430determines that the result has finished being written to the memory. In response to this determining, third node430generates a second write event and sends the second write event to recording service620through metadata channel624. The second write event indicates that third node430has finished writing the result at a particular time.

In addition to generating the second write event (either at the same time or after the second write event), third node430generates and sends (1) a first trigger to fourth node440through edge432and bridge622and, optionally, (2) a second trigger to recording service620through metadata channel624(in some examples, the second trigger is not sent to recording service620and instead recording service620runs periodically). The first and second trigger each indicate that third node430has stored the result. Based on sending the trigger, third node420generates a trigger event and sends the trigger event to recording service620through metadata channel624. The trigger event indicates that third node430sent a single trigger at a particular time.

Before or after sending the trigger, third node430completes the comparison function. In response to completing the comparison function, third node430generates a fourth function event and sends the fourth function event to recording service620through metadata channel624. The fourth function event indicates that third node430has completed the comparison function at a particular time.

After sending the fourth function event, third node430transitions to the sleep mode to wait for additional images to be received. Similar functions as described above are performed as third node430receives additional images from first node410through edge412and second node420through edge422, including reporting information about execution of third node430using metadata channel624and sending additional triggers and data to node440.

FIG.7is a block diagram illustrating deployment graph700of first subsystem510with timer node710for a graph application. In some examples, deployment graph700is generated while compiling the graph application, such that deployment graph700was not defined before compiling the graph application.

Deployment graph700is based on logical graph400, in that nodes and edges defined in logical graph400are included in deployment graph700. In some examples, deployment graph600is an alternative or further processed version of deployment graph600as depicted inFIG.6. The description below forFIG.7will add to the description ofFIG.6, intending to add to what was included in deployment graph600.

As depicted inFIG.7, first subsystem510further includes timer node710and real-clock720. In some examples, timer node710was not defined in the graph application before compiling the graph application and real-time clock720was included in first subsystem510regardless of the graph application (e.g., real-time clock720executes independent of the graph application). In such examples, timer node710is configured to receive a current time from real-time clock720and determine when to notify a node of the graph application of the current time. In some examples, timer node710determines when to notify based on definitions of different nodes. For example, first node410may be defined to require a current time as input when executing. In such an example, timer node710identifies this requirement and determines to send the current time to first node410when first node410is executing. In some examples, first node410may require a current time at a set interval (e.g., every 5 seconds). In such examples, timer node710provides the current time according to the set interval. While timer node710is illustrated as sending a current time to both first node410and second node420, it should be recognized that in some examples timer node710may send to only one of the nodes or send to each node at a different rate.

FIG.7depicts that timer node710sends data to recording service610. In some examples, the data is a copy of a current time sent to another node of the graph application, such as first node410and/or second node420. In such examples, timer node710duplicates a message sent to a node executing on first subsystem510and sends the duplicate message to recording service610. The duplicate message will include the time sent to the node along with a current time received from real-time clock720. In other examples, the message does not have to be duplicated for both receiving nodes to read it. In some examples, the time sent to the node and the current time received from real-time clock720are different because timer node710was configured to send the time to the node at a set interval and timer node710did not send the time at that time. In such examples, a time corresponding to the set interval will be sent to the node and the recording service610as data in the message and the current time received from real-time clock720will be sent to the recording service610(and not the node) via a metadata event with the time corresponding to the set interval. This allows us to ensure that nodes of the graph application receive are provided a consistent view of a current time that is based on what the node is expecting instead of an actual current time that might be delayed based on operation of first subsystem510.

FIG.8is a block diagram illustrating conversion of composite message800into a current version. In some examples, such conversion is performed while a graph application is executing (i.e., during runtime), either in real time or through playback of an execution of the graph application.

In some examples, the conversion is performed when a node is intended to receive a message that is in accordance with a version of the message that is not current (i.e., a previous version of the message). For example, a message may include a field that has evolved over time. The field might have originally been an integer and later transformed into a double. In such an example, the field of the message would be converted from an integer to a double before a node uses the message. In some examples, the conversion is performed before the node receives the message by a process executing outside of the node. In other examples, the conversion is performed by the node in response to receiving the message.

As depicted inFIG.8, composite message800includes two different objects (i.e., first object V1810and second object V1820). Each object within composite message800is intended to represent one or more fields, such as a primitive type or a message with a message. In some examples, a process determines that first object V1810is a previous version. In such examples, the process identifies a first conversion function to convert first object V1810to a subsequent version and applies the first conversion function to first object V1810, resulting in first object V2830. After applying the first conversion function, the process determines that first object V2830is a previous version. In such examples, the process identifies a second conversion function to convert first object V2830to a subsequent version and applies the second conversion function to first object V2830, resulting in first object V3840. After applying the second conversion function, the process determines that first object V3840is a current version and thus no further conversion is needed for first object V3840. The process (or another process), either concurrently with or after operations described above with respect to the first object, determines that second object V1820is a previous version. In such examples, the process identifies a third conversion function to convert second object V1820to a subsequent version and applies the third conversion function to second object V1820, resulting in second object V2860. After applying the third conversion function, the process determines that second object V2860is a current version and thus no further conversion is needed for second object V2860. After determining that both first object V3840and second object V2860are a current version, the process packages the object within updated message850and provides updated message850to the node intended to receive composite message800to be used during execution of the node.

In some examples, the conversion functions described above are defined by a developer, such as when updating an object to a new version. In such examples, the developer indicates that a current version of the object is to be frozen and an updated version is to be added. After indicting the current version is to be frozen, the developer uploads the updated version and adds a function to convert from the current version to the updated version.

FIG.9is a flow diagram illustrating method900for assigning logical nodes to hardware devices at compile time. Some operations in method900are, optionally, combined, the orders of some operations are, optionally, changed, and some operations are, optionally, omitted. In some examples, method900is performed by a compute system (e.g., compute system100) as part of compiling a software program (e.g., an application, a graph application, or an application including a flow defined using a graph) (in some examples, the following steps occur in response to a request to compile the software program; in some examples, the software program is configured to execute on a real-time operative system; in some examples, compiling the software program includes receiving source code (e.g., human-readable code) and generating executable code (e.g., machine code); in some examples, the software program executes based on a defined flow between different software components; in some examples, the software program includes a plurality of software components (e.g., logical nodes) and one or more defined communication channels (e.g., edges) between different software components).

At910, method900includes identifying a graph (e.g., a logical graph or a deployment graph) (in some examples, the graph includes a plurality of logical nodes connected via one or more edges) defining flow of data for the software program, wherein the graph includes a first node, a second node, and a first edge, and wherein the graph indicates that the first edge is connected to the first node and the second node.

At920, method900includes identifying, using the graph, an interface (e.g., a port, an input and/or output (I/O) location, a publication location, an I/O point, an I/O identifier, a network I/O location, a message I/O location, a communication channel, a data channel, a publication channel, a uni-directional communication channel, a bi-directional communication channel, a data channel, a uni-directional data channel, or a bi-directional data channel) of the first node, wherein the first node is configured to use the interface to send (e.g., publish) data for the first edge (in some examples, the first node is configured to publish data on the first edge; in some examples, the port includes an address or identifier used to subscribe to data published on the first edge; in some examples, the port includes a queue that the first node uses to publish data corresponding to the first edge).

At930, method900includes generating executable code (e.g., a deployment graph or machine code) for the software program, wherein: the executable code is configured, when executed, to establish a recording channel (e.g., an edge, a publish/subscribe channel) between the interface and a recording service (e.g., a recording node) (in some examples, the executable code is configured to establish the recording channel by adding an edge to a graph (e.g., the graph or a different graph, such as a deployment graph) between the first node and the recording service; in some examples, the recording service is represented as another node in the graph; in some examples, the recording channel is established in the executable code such that the recording service does not need to subscribe to the port when executing; in some examples, when the interface is referred to as a channel (e.g., a communication channel, a data channel, a publication channel, a uni-directional communication channel, a bi-directional communication channel, a data channel, a uni-directional data channel, or a bi-directional data channel), the executable code is configured, when executed, to cause the recording service to receive data via the channel), establishing the recording channel causes the recording service to receive data sent via (e.g., at, on, or by) the interface (in some examples, subscribing to data published on the port causes the recording service to be notified when the first node publishes data on the first edge), and the recording channel is not defined (e.g., declared) in the software program before compiling the software program (in some examples, the recording channel is added to the software program while compiling the software program).

In some examples, the first hardware definition includes a first identifier (e.g., a MAC or IP address) of the first hardware device, and wherein the first executable code includes an operation that checks (in some examples, the check occurs when the first executable code begins executing) that the first executable code is executing on a hardware device identified by the first identifier (in some examples, the operation causes the first executable code to output an error when the executable code executes on a hardware device that is not identified by the first identifier; in some examples, the operation causes the first executable code to fail when the executable attempts to execute on a hardware device that is not identified by the first identifier).

In some examples, the first hardware definition includes a type of communication technology supported (examples of types of communication technologies include radio frequency transmission, infrared transmission, microwave transmission, lightwave transmission, radar communication, satellite communication, cellular communication, WiFi, Bluetooth, ZigBee, and WiMAX) or an amount of memory (e.g., available memory) (in some examples, the first hardware definition includes both the type of communication support and the amount of memory), and wherein the first executable includes an operation that checks that the first executable code is executing on a hardware device (1) that supports the type of communication technology (in some examples, the operation causes the first executable code to output an error when the executable code executes on a hardware device that does not support the type communication technology; in some examples, the operation causes the first executable code to fail when the executable attempts to execute on a hardware device that does not support the type communication technology) or (2) that includes the amount of memory (in some examples, the operation causes the first executable code to output an error when the executable code executes on a hardware device that does not include the amount of memory; in some examples, the operation causes the first executable code to fail when the executable attempts to execute on a hardware device that does not include the amount of memory).

In some examples, the first hardware definition includes a definition of memory included with a hardware device, wherein the definition of memory includes an identification of another hardware device that is able to access the memory (in some examples, the memory is shared memory that is accessible to multiple hardware devices), and wherein the first executable includes an operation that checks that the first executable code is executing on a hardware device that includes memory accessibly by a third hardware device (in some examples, the hardware model is generated by a program in communication with the first hardware device and the second hardware device; in some examples, the program is executing on the first hardware device and/or the second hardware device; in some examples, the program sends a request to the first hardware device for information to generate the first hardware definition; in some examples, the operation causes the first executable code to output an error when the executable code executes on a hardware device that does not include memory accessibly by the third hardware device; in some examples, the operation causes the first executable code to fail when the executable attempts to execute on a hardware device that does not include memory accessibly by the third hardware device).

In some examples, the first executable code includes a first interface (e.g., a port, an input and/or output (I/O) location, a publication location, an I/O point, an I/O identifier, a network I/O location, a message I/O location, a communication channel, a data channel, a publication channel, a uni-directional communication channel, a bi-directional communication channel, a data channel, a uni-directional data channel, or a bi-directional data channel) configured to communicate via a first channel between the first hardware device and the second hardware device, the second executable code includes a second interface (e.g., a port, an input and/or output (I/O) location, a publication location, an I/O point, an I/O identifier, a network I/O location, a message I/O location, a communication channel, a data channel, a publication channel, a uni-directional communication channel, a bi-directional communication channel, a data channel, a uni-directional data channel, or a bi-directional data channel) configured to communicate via the first channel between the second hardware device and the first hardware device, and the second interface is different from the first interface.

In some examples, in accordance with a determination that one or more first criteria are met (in some examples, the one or more first criteria includes a criterion based on the first hardware device and the second hardware device), the first interface is configured to support sending messages according to a first communication protocol (examples of a communication protocol include Transmission Control Protocol, User Datagram Protocol, and Controller Area Network Protocol); and in accordance with a determination that one or more second criteria are met (in some examples, the one or more second criteria includes a criterion based on the first hardware device and the second hardware device), the first interface is configured to support sending messages according to a second communication protocol different from the first communication protocol (in some examples, the first executable code includes two different types of interfaces for a single hardware device).

In some examples, the first interface and the second interface are defined based on the logical graph and the hardware model, and wherein the first interface and the second interface were not defined before compiling the software program.

In some examples, the first executable code includes a third interface configured to communicate via a second channel between the first hardware device and a third hardware device, and wherein the third interface was defined before compiling the software program (in some examples, the third interface was defined before compiling the software program because the third hard device is not defined in the hardware model; in some examples, the third hardware device is not included in the hardware model).

Note that details of the processes described below with respect to method1000(i.e.,FIG.10) are also applicable in an analogous manner to method900ofFIG.9. For example, method900optionally includes one or more of the characteristics of the various methods described below with reference to method1000. For example, the executable code from method900may be configured, when executed, to establish a recording channel between an interface of a logical node and a recording service from method1000.

FIG.10is a flow diagram illustrating method1000for assigning logical nodes to hardware devices at compile time. Some operations in method1000are, optionally, combined, the orders of some operations are, optionally, changed, and some operations are, optionally, omitted. In some examples, method1000is performed by a compute system (e.g., compute system100) as part of compiling a software program (e.g., an application, a graph application, or an application including a flow defined using a graph) (in some examples, the following steps occur in response to a request to compile the software program; in some examples, the software program is configured to execute on a real-time operative system; in some examples, compiling the software program includes receiving source code (e.g., human-readable code) and generating executable code (e.g., machine code); in some examples, the software program executes based on a defined flow between different software components; in some examples, the software program includes a plurality of software components (e.g., logical nodes) and one or more defined communication channels (e.g., edges) between different software components).

At1010, method1000includes identifying a graph (e.g., a logical graph or a deployment graph) (in some examples, the graph includes a plurality of logical nodes connected via one or more edges) defining flow of data for the software program, wherein the graph includes a first node, a second node, and a first edge, and wherein the graph indicates that the first edge is connected to the first node and the second node.

At1020, method1000includes identifying, using the graph, an interface (e.g., a port, an input and/or output (I/O) location, a publication location, an I/O point, an I/O identifier, a network I/O location, a message I/O location, a communication channel, a data channel, a publication channel, a uni-directional communication channel, a bi-directional communication channel, a data channel, a uni-directional data channel, or a bi-directional data channel) of the first node, wherein the first node is configured to use the interface to send (e.g., publish) data for the first edge (in some examples, the first node is configured to publish data on the first edge; in some examples, the port includes an address or identifier used to subscribe to data published on the first edge; in some examples, the port includes a queue that the first node uses to publish data corresponding to the first edge).

At1030, method1000includes generating executable code (e.g., a deployment graph or machine code) for the software program, wherein: the executable code is configured, when executed, to establish a recording channel (e.g., an edge, a publish/subscribe channel) between the interface and a recording service (e.g., a recording node) (in some examples, the executable code is configured to establish the recording channel by adding an edge to a graph (e.g., the graph or a different graph, such as a deployment graph) between the first node and the recording service; in some examples, the recording service is represented as another node in the graph; in some examples, the recording channel is established in the executable code such that the recording service does not need to subscribe to the port when executing; in some examples, when the interface is referred to as a channel (e.g., a communication channel, a data channel, a publication channel, a uni-directional communication channel, a bi-directional communication channel, a data channel, a uni-directional data channel, or a bi-directional data channel), the executable code is configured, when executed, to cause the recording service to receive data via the channel), establishing the recording channel causes the recording service to receive data sent via (e.g., at, on, or by) the interface (in some examples, subscribing to data published on the port causes the recording service to be notified when the first node publishes data on the first edge), and the recording channel is not defined (e.g., declared) in the software program before compiling the software program (in some examples, the recording channel is added to the software program while compiling the software program).

In some examples, the recording service is not defined (e.g., declared) in the software program before compiling the software program (in some examples, the recording service is added to the software program while compiling the software program).

In some examples, the executable code includes: first executable code for a first hardware device, wherein the first executable code is configured to execute the recording service on the first hardware device, and second executable code for a second hardware device, wherein the second executable code is configured to execute a second recording service on the second hardware device, wherein the second hardware device is different from the first hardware device, and wherein the second recording service is different from the recording service (in some examples, a different recording service is added to each different hardware device such that each different hardware device includes a respective recording service (in some examples, each different hardware device includes only a single recording service); in some examples, the recording service does not communicate with the second recording service and vice versa).

In some examples, the executable code is configured, when executed, to establish a second recording channel (e.g., a metadata recording channel) between the first node and the recording service, the second recording channel is different from the recording channel, and establishing the second recording channel causes the recording service to receive execution metadata of the first node (in some examples, all nodes in the graph application are identified and different channels are established for each node; in some examples, only a single metadata channel is established for each node).

In some examples, the execution metadata includes metadata indicating when the first node launches, metadata indicating one or more triggers have been sent or received by the first node, or metadata indicating when a function of the first node starts or finishes executing.

In some examples, the executable code is configured, when executed, to establish a third recording channel (e.g., a data recording channel) between a second interface of the first node and the recording service, establishing the third recording channel causes the recording service to receive data sent via the second interface, the third recording channel is different from the recording channel, and the second interface is different from the interface.

In some examples, the recording channel is established by the recording service sending a request to subscribe to a publication of the first node, wherein the request includes an identification of the first edge (in some examples, the identification is an address; in some examples, the identification is a combination of an identifier for the first node and an identifier for the interface; in some examples, providing).

Note that details of the processes described above with respect to method900(i.e.,FIG.9) are also applicable in an analogous manner to method1000ofFIG.10. For example, method1000optionally includes one or more of the characteristics of the various methods described above with reference to method900. For example, the executable code from method1000may be configured to establish a different recording service on each hardware device from method900.