SYSTEMS AND METHODS FOR VISUALIZING MACHINE INTELLIGENCE

A system to deploy virtual sensors to a machine learning project and translate data of the machine learning project is provided. The system can deploy, for a machine learning project, a plurality of virtual sensors at a first location of a plurality of locations to detect metadata of a data source of the machine learning project, at a second location of the plurality of locations to detect deployment information of a model trained for the machine learning project, and at a third location of the plurality of locations to detect learning session information for creation of the model. The system can collect, via the plurality of virtual sensors, data for the machine learning project. The system can translate, for render on a computing system, the data collected via the plurality of virtual sensors into a plurality of graphics.

TECHNICAL FIELD

The present implementations relate generally to machine learning and graphic user interfaces.

BACKGROUND

A machine learning system can train models based on training data sets. The machine learning system can implement training techniques that determine values for various parameters or weights of the models. The models can execute on inference data sets to make model decisions, predictions, other inferences based on the various values for the parameters or weights. However, the process of designing and deploying a model using machine learning can be complex and difficult for a user to understand or visualize, which can result in errors or inaccuracies in the as-built model.

SUMMARY

Systems and methods of the technical solution can visualize machine intelligence. A user may have difficulty in comprehending a machine learning project due to the expansive and complex nature of the machine learning project. Furthermore, a system that creates visualizations of the machine learning project may not include an efficient method for tracking the state of the machine learning project. To solve these and other technical problems, the system described herein can generate a graphic representation of the machine learning project. The system can generate a graphic representation in a visual language, e.g., in accordance with a defined set of rules based on data streamed from virtual sensors. The visual language can allow for a user to view the graphic representation of the machine learning project and readily understand the current configuration of the machine learning project without requiring the user to perform extensive review of code, low level model design, or technical details of the machine learning project. The system can deploy a virtual sensor to the machine learning project that streams data describing the machine learning project. The data can be streamed in real time or in a low latency manner. Based on this data streaming, graphic representations of the machine learning project can be determined based on the streamed data so that the visual appearance of the graphic representation matches a current configuration of the machine learning project.

An aspect of this technical solution is directed to a system. The system can include a data processing system including one or more processors, coupled with memory, to deploy, for a machine learning project, virtual sensors at a first location of a locations to detect metadata of a data source of the machine learning project, at a second location of the locations to detect deployment information of a model trained for the machine learning project, and at a third location of the locations to detect learning session information for creation of the model. The data processing system can collect, via virtual sensors deployed at the locations, data for the machine learning project. The data processing system can translate, for render on a computing system, the data collected via the virtual sensors into graphics including a graphic representing the metadata of the data source, a graphic representing the deployment of the model, and a graphic representing the learning session.

A virtual sensor of the virtual sensors can monitor values of a data element of the machine learning project. The virtual sensor can stream the values of the data element to the data processing system.

A virtual sensor of the virtual sensors includes a web-hook that monitors a data element of the machine learning project.

The data processing system can apply a visualization rule to the data collected based on a virtual sensor of the virtual sensors, identify a visual appearance of one or more of the graphics based on the visualization rule and the data, and generate the one or more of the graphics to include the visual appearance.

The data processing system can receive, from the computing system, a selection of the graphic representing the learning session and provide, for render on the computing system, entities within the graphic representing the learning session, the entities representing components of the learning session.

The data processing system can determine, based on the data, a health level of a component of the machine learning project, compare the health level to a threshold, and update a visual appearance of at least one of the graphics or connections between the graphics responsive to the health level satisfying the threshold.

The data processing system can determine, based on the data, a health level of a connection of connections between the graphics, compare the health level to a threshold, generate an update to the machine learning project, and modify a visual appearance of the connection.

The learning session can design and train the model of the machine learning project based on a machine learning problem received from the computing system.

The data processing system can generate data causing the computing system to display a time control element, receive a selection of the time control element from the computing system, and animate at least one of the graphics or connections between the graphics based on a historical record of states of the machine learning project at points in time.

The data processing system can animate at least one of the graphics or the connections by adding, removing, or adjusting entities of the graphics based on the historical record.

The graphic representing the metadata of the data source can include a first spherical portion including a metadata entity representing metadata of the data source of the machine learning project. The graphic representing the deployment of the model can be a second spherical portion including a deployment entity representing the deployment of the model trained for the machine learning project.

The data processing system can draw, based on the data, a first connection between the metadata entity and the graphic representing the learning session and a second connection between the graphic representing the learning session and the deployment entity. The first connection and the second connection can indicate that the learning session uses data of the data source to produce the deployment of the model.

The data processing system can generate, based on the data, a third spherical portion, the third spherical portion including a decision entity indicating a decision produced by the deployment of the model of the machine learning project and draw, based on the data, a connection between the deployment entity and the decision entity. The connection indicates that the deployment of the model of the machine learning project produces the decision.

The data processing system can generate the first spherical portion to be a semi-sphere, generate, based on the data, a third spherical portion, the third spherical portion including a decision entity indicating a decision produced by the deployment of the model of the machine learning project, and generate the second spherical portion and the third spherical portion to be quarter-spheres.

An aspect of this technical solution is directed to a method. The method can include deploying, by a data processing system including one or more processors, coupled with memory, for a machine learning project, a virtual sensors at a first location of a locations to detect metadata of a data source of the machine learning project, at a second location of the locations to detect deployment information of a model trained for the machine learning project, and at a third location of the locations to detect learning session information for creation of the model. The method can include collecting, by the data processing system, via the virtual sensors deployed at the locations, data for the machine learning project. The method can include translating, by the data processing system, for render on a computing system, the data collected via the virtual sensors into graphics including a graphic representing the metadata of the data source, a graphic representing the deployment of the model, and a graphic representing the learning session.

A virtual sensor of the virtual sensors monitors values of a data element of the machine learning project and streams the values of the data element to the data processing system.

A virtual sensor of the virtual sensors includes a web-hook that monitors a data element of the machine learning project.

The method can include applying, by the data processing system, a visualization rule to the data collected based on a virtual sensor of the virtual sensors, identifying, by the data processing system, a visual appearance of one or more of the graphics based on the visualization rule and the data, and generating, by the data processing system, the one or more of the graphics to include the visual appearance.

An aspect of this technical solution is directed to a computer readable medium. The computer readable medium can store instructions thereon, that, when executed by one or more processors, cause the one or more processors to deploy, for a machine learning project, virtual sensors at a first location of locations to detect metadata of a data source of the machine learning project, at a second location of the locations to detect deployment information of a model trained for the machine learning project, and at a third location of the locations to detect learning session information for creation of the model. The instructions can cause the one or more processors to collect, via the virtual sensors deployed at the locations, data for the machine learning project. The instructions cause the one or more processors to translate, for render on a computing system, the data collected via the virtual sensors into graphics including a graphic representing the metadata of the data source, a graphic representing the deployment of the model, and a graphic representing the learning session.

A virtual sensor of the virtual sensors can monitor values of a data element of the machine learning project. The virtual sensor of the virtual sensors can stream the values of the data element to the one or more processors.

DETAILED DESCRIPTION

The present implementations will now be described in detail with reference to the drawings, which are provided as illustrative examples of the implementations so as to enable those skilled in the art to practice the implementations and alternatives apparent to those skilled in the art. Notably, the figures and examples below are not meant to limit the scope of the present implementations to a single implementation, but other implementations are possible by way of interchange of some or all of the described or illustrated elements. Moreover, where certain elements of the present implementations can be partially or fully implemented using known components, only those portions of such known components that are necessary for an understanding of the present implementations will be described, and detailed descriptions of other portions of such known components will be omitted so as not to obscure the present implementations. Implementations described as being implemented in software should not be limited thereto, but can include implementations implemented in hardware, or combinations of software and hardware, and vice-versa, as will be apparent to those skilled in the art, unless otherwise specified herein. In the present specification, an implementation showing a singular component should not be considered limiting; rather, the present disclosure is intended to encompass other implementations including a plurality of the same component, and vice-versa, unless explicitly stated otherwise herein. Moreover, applicants do not intend for any term in the specification or claims to be ascribed an uncommon or special meaning unless explicitly set forth as such. Further, the present implementations encompass present and future known equivalents to the known components referred to herein by way of illustration.

This disclosure is generally directed to systems and methods for visualizing a machine learning project. A machine learning project can be a single-objective or multi-objective project that designs, constructs, and deploys multiple models to achieve one or multiple different machine learning goals. The machine learning project can further be designed and managed by one or multiple different users. A user may have difficulty in comprehending the machine learning project due to the expansive and complex nature of the machine learning project. Furthermore, a system that creates visualizations of the machine learning project may not include an efficient method for tracking updates that are made to the machine learning project or the status changes of the machine learning project. Furthermore, these changes to the machine learning project can happen rapidly in real-time and the visualization may not appropriately track the current state of the machine learning project.

To solve these and other technical problems, the system described herein can generate a graphic representation of the machine learning project. The graphic representation can provide both a high level view of the machine learning project and a low level view of various components of the machine learning project. The system can generate a graphic representation in a visual language, e.g., in accordance with a defined set of rules. The visual language can allow for a user to view the graphic representation of the machine learning project and readily understand the current configuration of the machine learning project without requiring the user to perform extensive review of code, low level model design, or technical details of the machine learning project. The graphic can represent high level components of the machine learning project, e.g., metadata of data sources of the machine learning project, learning sessions that design and construct models of the machine learning projects, deployments of the models, and decisions of the models. Within the same graphic, lower level components can be displayed, e.g., individual metadata entities of individual data sources, the various components and pipelines that make up the learning sessions, individual deployments of models and their associated decisions. The lower level components can be included as graphics within graphics representing the higher level components. This allows a user to view a single graphic and zoom in or out to view various levels of the machine learning project without requiring the user to navigate to different user interfaces or screens.

Furthermore, the system can solve technical data collection problems for visualizing the machine learning project by deploying virtual sensors to the machine learning project. The system can deploy a virtual sensor, which can be a portion of code that monitors a data element of the project, to various points in the machine learning project. The virtual sensors can be deployed when the machine learning project is created or over time as new components are added to the machine learning project. The virtual sensors can be web-hooks or web sockets that stream data from the machine learning project to the system. The data can be streamed in real time or in a low latency manner. Based on this data streaming, the system can render the graphic representation of the machine learning project based on the streamed data so that the visual appearance of the graphic representation matches a current configuration of the machine learning project.

FIG.1includes an example system100including at least one data processing system102that can generate at least one graphic representation104of at least one machine learning project106. The data processing system102can communicate with at least one computing system108. The data processing system102can transmit data to the computing system108, for example, at least one user interface110. The data processing system102can receive data from the computing system108, for example, user interactions with the user interface110. The data processing system102can generate data that causes the computing system108to display the user interface110. The data processing system102can generate data that renders graphic elements, e.g., the graphic representation104, in the user interface110.

The computing system108can be or include a data processing system. Examples of data processing systems are described atFIG.18. The computing system108can be a laptop computer, a tablet computer, a smartphone, a desktop computer. The computing system108can include at least one output device, e.g., light emitting diode (LED) displays, organic light emitting diode (OLED) displays, quantum dot LED display (QLED), a liquid crystal display (LCD), etc. The computing system108can display the user interface110on the output devices for a user or users to view. The computing system108can include input devices. For example, the input devices can include a touch-screen, a keyboard, a mouse, a microphone. The input devices can receive input from a user or users.

The data processing system102can include various software modules, components, functions, or other computing elements. The data processing system102can include various data elements that are stored by the data processing system102. For example, the data processing system102can include a machine learning project106. The machine learning project106can be defined by the computing system108. For example, a user or multiple users can create or modify the machine learning project106via the computing system108. For example, a user or multiple can provide input via the user interface110that defines or otherwise creates the machine learning project106. The computing system108can provide a data problem to be solved or multiple data problems to be solved simultaneously or sequentially. The computing system108can provide at least one data set for training machine learning models or algorithms of the machine learning project106. The computing system108can provide data or data sets to be used to generate inferences via the trained machine learning models or algorithms.

The data processing system102can include a machine learning engine112. The machine learning engine112can execute at least one learning session116. The machine learning engine112can include a learning core114that implements and runs the learning session116of the machine learning project106. The learning session116can construct pipelines for the machine learning project106based on data provided by the computing system108. For example, the pipelines can be learning pipelines, training pipelines, and/or inference pipelines. The learning session116can perform a search process. The search process can identify a description of a predictive model (e.g., a blueprint, an architecture, a previously trained model) that solves a machine learning problem provided by the computing system108. The machine learning problem can be a problem to predict, determine, or infer a particular target value of a data set. The learning session116can identify or construct a predictive model in a graph format to solve the machine learning problem. The graph can be a DAG that includes nodes representing actions or data and edges between the nodes that represent the order that the actions are performed in. The learning session116can design, construct, or train one or multiple models and track the performance of the models over time. The learning session116can deploy a highest or high performing model or models (e.g., models that have error rates less than a value or accuracy levels greater than a particular level).

The learning core114can execute the learning session116to design or train the model based on data of at least one data source116. The data source116can be an external data source to the data processing system102or an internal data source to the data processing system102. The machine learning project106can store at least one data source metadata118that describes the data source116. The data source metadata118can describe the type of data that the data source1126provides. For example, the data source metadata118can indicate that the data is category based data, binary data, value data, label data, date data, image data. The data source metadata118can indicate that the data is derived data or raw data.

The learning session116can produce at least one model deployment120. The model deployment120can be a model designed and trained by the learning session116. The model deployment120can be a model selected from a group of models that is a highest performing model of the group (e.g., is the most accurate model or has the least error rate). The model deployment120can be a deployed model that executes on the machine learning engine112. The model deployment120can generate decisions based on data of a data stream or data of a static data source. The model deployment120can generate decisions based on data of the data source116. For example, the model deployment120can execute on telemetry data collected from Internet of Things (IoT) devices, web-activity data collected via cookies or tracking pixels, product sales data collected from a product sales platform.

The machine learning project106can include at least one model decision122. The model decision122can be a decision of the model deployment120. The model decision122can be a product order decision, a web content delivery decision, an insurance premium decision. The model decision122can be provided to various applications internal or external to the data processing system102. An application can perform actions based on the model decision122. For example, the application can operate to make purchases of merchandise to fill inventory, generate navigation route suggestions for driving a vehicle, alert a user of predicted security threats in surveillance camera data, etc. The model decision122can be provided to the computing system108via the user interface110by an application or the data processing system102.

The data processing system102includes a back-end system124and a front-end system126. The back-end system124can collect and process data for generating the graphic representation104. The front-end system126can generate the graphic representation104based on the data collected by the back-end system124. The back-end system124and the front-end system126can communicate via application programming interfaces (APIs), publish-subscribe channels, or any other communication protocol. The front-end system126can generate the graphic representation104by running a two dimensional or three dimensional graphics engine. The front-end system126can render the graphic representation104to provide a representation of the machine learning project106based on the data received from the back-end system124. The front-end system126can run a graphics engine such as the UNREAL ENGINE, UNITY, CRYENGINE.

The graphic representation104can include at least one three dimensional or two dimensional graphic element that represents components of the machine learning project106. The graphic representation104can generate the graphic representation104to appear as a brain-like structure. The brain-like structure can have a brain-like appearance. For example, the brain-like structure can include two hemispheres and a core between the two hemispheres. The brain-like structure can include at least one semi-sphere, quarter-sphere, eighth-sphere, or any other spherical or sphere shaped portion. The graphic representation104can include a graphic representation of the data source metadata118. The graphic representation104of the data source metadata118can include a cube, a prism, a sphere, a hemisphere, a quarter-sphere, a spherical portion. The graphic representation104of the data source metadata118can include entities that represent metadata of individual data sources. For example, a spherical portion can include multiple smaller points or other two dimensional or three dimensional objects that represent the metadata of the individual data sources (e.g., blocks, cubes, rectangles, rectangular solids, stars, diamonds, free form shapes).

The graphic representation104can include at least one graphic representation of the learning core114or the learning session116. The graphic representation104of the learning core114or the learning session116can include a cube, a prism, a sphere, a hemisphere, a quarter-sphere, an eighth-sphere, a spherical portion. The graphic representation104can include a first sphere that represents the learning core114. The first sphere can be located in between two semi-spheres. The first sphere can be centered between the two semi-spheres. The graphic representation104can include at least one second sphere that represents the learning session116. The at least one second sphere can be located in between the two semi-spheres. The at least one second sphere can be offset from the first sphere.

The graphic representation104can include a graphic representation of the model deployment120. The graphic representation104of the model deployment120can include a cube, a prism, a sphere, a hemisphere, a quarter-sphere, an eighth-sphere, a spherical portion. The graphic representation104of the model deployment120can include entities that represent individual model deployments of models learned or trained by the learning session116. For example, a spherical portion can include multiple smaller points or other two dimensional or three dimensional objects that represent the individual model deployments (e.g., blocks, cubes, rectangles, rectangular solids, stars, diamonds, free form shapes). The spherical portion can be a quarter-sphere that is located opposite a semi-sphere representing the data source metadata118.

The graphic representation104can include a graphic representation of the model decision122. The graphic representation104of the model decision122can include a cube, a prism, a sphere, a hemisphere, a quarter-sphere, an eighth-sphere, a spherical portion. The graphic representation104of the model decision122can include entities that represent individual model decisions of deployments of models learned or trained by the learning core114(e.g., blocks, cubes, rectangles, rectangular solids, stars, diamonds, free form shapes). The individual model decisions can be decisions, inferences, predictions, or categorizations, that the model deployment120generates. The graphic representation104can include a spherical portion that includes multiple smaller points or other two dimensional or three dimensional objects (e.g., blocks, cubes, rectangles, rectangular solids, stars, diamonds, free form shapes) that represent the individual model decisions122. The spherical portion can be a quarter-sphere that the front-end system126locates opposite a semi-sphere representing the data source metadata118in the graphic representation104. The front-end system126can locate the quarter-sphere next to a quarter-sphere representing the model deployment120in the graphic representation104.

The back-end system124can include at least one virtual sensor128. The virtual sensor128can be a component of the back-end system124that is stored by the back-end system124. At least a portion of the virtual sensor128can be deployed in the machine learning project106or the machine learning engine112. The back-end system124can deploy the virtual sensor128to the machine learning project106when the machine learning project106is first created. The back-end system124can deploy the virtual sensor128to the machine learning project106as updates are made to the machine learning project106. For example, if a new learning session116is implemented, the back-end system124can deploy the virtual sensor128to the new learning session116to monitor data of the new learning session116. One virtual sensor128deployed by the back-end system124can monitor the creation of new data elements in the machine learning project106. Based on an indication of the creation of new data elements in the machine learning project106received from the one virtual sensor128, the back-end system124can deploy additional virtual sensors128to monitor the new data elements.

The virtual sensor128can collect data for various data points, data registers, data elements, data locations, data pointers of the machine learning project106. The virtual sensor128can be a web-hook or web socket. The virtual sensor128can read, retrieve, or monitor data values or data elements of the machine learning project106or the machine learning engine112. The virtual sensors128can be stored at various locations within the machine learning project106or the machine learning engine112that allow the virtual sensors128to monitor and detect information associated with the machine learning project106(e.g., the data source metadata118, the learning session116, the model deployment120, the model decision122, the learning core114). The virtual sensors128can identify changes to the data values or data elements that indicate changes to the data source metadata118, the learning session116, the model deployment120, the model decision122, or the learning core114. The changes to the machine learning project106can be streamed as one or more data streams, e.g., event streams, data streams, value streams, to the back-end system124. The back-end system124can provide the data streams to the front-end system126.

The front-end system126can modify the appearance of the graphic representation104based on the data of the data streams. For example, the front-end system126can identify new, deleted, or modified elements of the data source metadata118, the learning sessions116, the model deployments120, or the model decisions122based on the data streams. The front-end system126can translate the data received from the back-end system124into a visual language that the front-end system126uses to generate the graphic representation104or causes the graphic representation104to be rendered by the computing system108. For example, the front-end system126can include a set of rules that define the visual language for the appearance of the graphic representation104. The front-end system126can apply the data received from the virtual sensor128to the set of rules to determine whether to update, modify, or add to the visual appearance of the graphic representation104. Applying the set of rules can translate values of the data received from the back-end system124into visual representations in the graphic representation104. The front-end system126can add new graphics, delete existing graphics, or modify existing graphics within the graphic representation104to represent the changes. For example, the front-end system126can add a new metadata entity to represent a new data source or delete the metadata entity responsive to detecting that the data source is deleted or removed from the machine learning project106. Furthermore, the front-end system126can draw lines connecting the entities representing relationships between the various components of the machine learning project106. As the relationships are added, removed, for modified, the front-end system126can modify the lines drawn in the graphic representation104.

FIG.2illustrates an example of the graphic representation104of the machine learning project106. The graphic representation104can include various three dimensional graphic elements that describe the machine learning project106. For example, the graphic representation104can include at least one data source graphic202that represents the data source116. The data source graphic202can be a three dimensional solid. For example, the data source graphic202can be a cylinder, a prism, a sphere, or any other shape. The data source graphic202can include entities within the three dimensional solid that represent individual data sources106. For example, the entities can be two dimensional or three dimensional shapes or solids, e.g., squares, cubes, rectangles, stars, prisms, diamonds.

The graphic representation104can include at least one connection drawn to connect entities of the data source graphic202to entities of an external storage service graphic204. The external storage service graphic204can represent a data storage service that stores the data sources104. For example, the external storage service can be a database service of a computing system such as a server, a desktop computer, a cloud system. The external storage service graphic204can be a three dimensional solid, for example, a cylinder, a prism, a sphere, or any other shape. The external storage service graphic204can include entities within the three dimensional solid that represent individual data storage services. For example, the entities can be two dimensional or three dimensional shapes or solids, e.g., squares, cubes, rectangles, stars, prisms, diamonds. The graphic representation104can include connection lines that connect the entities of the external storage service graphic204with entities of the data source graphic202. The lines can indicate relationships between the entities, e.g., one entity represents a data storage service that is related to a second entity representing a data source that the data storage service manages. The graphic representation104can include a line that connects the first entity with the second entity.

The graphic representation104can include at least one data source metadata graphic206. The data source metadata graphic206can represent the data source metadata118. The data source metadata graphic206include a three dimensional solid, for example, a cylinder, a prism, a sphere, or any other shape. The data source metadata graphic206can include entities that represent individual data source metadata206. The entities representing the individual data source metadata206can include two dimensional or three dimensional shapes or solids, e.g., squares, cubes, rectangles, stars, prisms, diamonds. The entities representing the individual data source metadata206can be connected to entities of the data source graphic202via connectors drawn between the entities of the data source metadata206and the entities of the data source graphic202. For example, one particular metadata entity may represent metadata for one particular data source entity. The graphic representation104can include a line drawn between the two entities to represent the relationship between the two entities.

The graphic representation104can include at least one learning core graphic208. The learning core graphic208can represent the learning core114. The learning core graphic208can represent a capability (e.g., an ability, a software application or function, the availability of) of the machine learning project106to run or execute the learning core114to perform learning sessions116to create model deployments120. The learning core graphic208can be a three dimensional solid, for example, a cylinder, a prism, a sphere, or any other shape. The learning core graphic208can be surrounded by at least one learning session graphic210. The learning session graphic210can represent the learning session116. The learning session graphic210can be a three dimensional solid, for example, a cylinder, a prism, a sphere, or any other shape. The learning session210can be connected to entities of the data source metadata graphic206. For example, one or multiple entities that represent data source metadata for data sources that the learning session uses to performing learning or training can be connected to the learning session graphic210via connectors.

The graphic representation104can include at least one model deployment graphic212and at least one decision graphic214. The model deployment graphic212can represent the model deployment120. The decision graphic214can represent the model decision122. The model deployment graphic212and the model decision graphic214include be a three dimensional solid, for example, a cylinder, a prism, a sphere, or any other shape. The model deployment graphic212and the decision graphic214can be combined into a single shape, e.g., quarter-spheres combined into a semi-sphere. The semi-sphere can be located opposite another semi-sphere, e.g., the data source metadata graphic206. The model deployment graphic212can include entities that represent individual model deployments120. The entities can be two dimensional or three dimensional shapes or solids, e.g., squares, cubes, rectangles, stars, prisms, diamonds. The entities of the model deployment graphic212can be connected to the learning session graphic210representing the learning session that created the model deployment. For example, an entity representing a model deployment can be connected via a line between the entity and the learning session graphic210that represents the learning session that produced the model deployment.

The graphic representation104can include at least one internal application graphic216representing an internal application (e.g., internal to the data processing system102or the machine learning project106) that runs operations based on the model decisions122. The graphic representation104can include at least one hosted application graphic218representing a hosted application (e.g., external to the data processing system102or the machine learning project106hosted on a remote server or computer system) that runs operations based on the model decisions122. The internal application graphic216or the hosted application graphic218can include two dimensional or three dimensional shapes or solids, e.g., squares, cubes, rectangles, stars, prisms, diamonds. The internal application graphic216or the hosted application graphic218can be connected to the model deployment graphic212or the decision graphic214via lines. For example, the lines can indicate that a particular application consumes or operates on outputs of a particular model deployment or operates on particular decisions. The lines can connect the internal application graphic216or the hosted application graphic218to a particular entity representing a particular model deployment in the model deployment graphic212or a particular entity representing a decision of a model deployment of the decision graphic214.

The graphic representation104can include at least one internal compute service graphic220that represents computational resources (e.g., processors, graphics processing units, application specific integrated circuits, memory, etc.) internal to the data processing system102that execute the machine learning project106. The graphic representation104can include at least one external compute service graphic222that represents computational resources (e.g., processors, graphics processing units, application specific integrated circuits, memory) external to the data processing system102that can be included in one or more remote servers or computing systems that execute the machine learning project106. The graphic representation104includes a compute interface graphic224that represents a piece of software or code that serves as an interface between the machine learning project106and the internal or external compute services represented by the internal compute service graphic220or the external compute service graphic222. The compute interface graphic224can include two dimensional or three dimensional shapes or solids, e.g., squares, cubes, rectangles, stars, prisms, diamonds, cylinders.

The graphic representation104can include at least one hosted storage service graphic226. The hosted storage service graphic226can represent a hosted storage service that stores the data source116. The hosted storage service can be a storage service located remote from the data processing system102. For example, the hosted storage service can be hosted on an external server or computer system. The hosted storage service represented by the hosted storage service graphic226can interface with the machine learning project106via a storage interface. The graphic representation104can include at least one storage interface graphic228. The storage interface graphic228can represent the storage interface. The storage interface can be an application or code that interfaces the data source116with the data processing system102. The storage interface graphic228can include two dimensional or three dimensional shapes or solids, e.g., squares, cubes, rectangles, stars, prisms, diamonds, cylinders.

The graphic representation104can include at least one external user application graphic230and at least one hosted user application graphic232. The external user application graphic230can represent a user application that is external to the machine learning project106. The application can be an application that allows the computing system108to interact with the machine learning project106, e.g., provide design inputs or modify the machine learning project106. The hosted user application graphic232can represent a user application that is hosted on an external server or computing system. The user application can be an application that allows the computing system108to interact with the machine learning project106, e.g., provide design inputs or modify the machine learning project106. The graphic representation104includes an API interface graphic234. The API interface graphic234can represent an API which the external or hosted user applications represented by the external user application graphic230or the hosted user application graphic232communicate with to interface and interact with the machine learning project106.

FIG.3illustrates example graphic representations of the data source metadata graphic206representing the data source metadata118of the machine learning project106.FIG.3includes at least one graphic302. The graphic302can include at least one spherical portion (e.g., a semi-sphere). The graphic302illustrates an example of the data source metadata graphic206without any entities or connections between entities. A user can provide an input via the computing system108to turn on or off the entities or connections. For example, a user can interact with a single entity via the computing system108. Responsive to the interaction, the front-end system126can hide or de-emphasis (e.g., make transparent) other entities or connections that are separate from the entity that the user interacted with. The front-end system126can emphasis the entity that the user interacted with or emphasis connections of the entity with other entities, e.g., change the entity or connections to a darker or vibrant color).

The graphic302can illustrate an example of the data source metadata graphic206at a first point in time when the machine learning project106is early in its development. Before a user provides data sources116to the machine learning project106, the front-end system126can render the graphic302to not include any entities to illustrate that the user has not added any data sources116to the machine learning project106. At a later point in time, as a user adds the data sources116to the machine learning project106, the front-end system126can add entities to the data source metadata graphic206to represent the added data source metadata118, e.g., create the graphic304. The front-end system126can draw connections between the entities of the graphic304and other graphics, e.g., the data source graphic202or the learning session graphic210. For example, the front-end system126can draw a connection between an entity representing a particular data source of the data source graphic202and an entity of the graphic representing metadata for the particular data source. The front-end system126can draw a connection between the entity representing the particular data source and a particular learning session graphic210that represents a learning session that consumes the data of the particular data source.

FIG.4illustrates an example of a graphic representation400representing deployments and decisions of the machine learning project106.FIG.4includes a graphic400that can be a spherical portion (e.g., a semi-sphere) that includes the model deployment graphic212and the decision graphic214. The graphic400can include the model deployment graphic212as a first quarter-sphere and the decision graphic214as a second quarter-sphere. The graphic400can include no entities or connections as shown in at least one graphic402. The graphic400can include entities and connections as shown in at least one graphic404.

The graphic403provides an example of the graphic400when no entities or connections are displayed. For example, a user can provide an input via the computing system108to hide the entities or connections. At a particular point in time, before a learning session116has finished creating a model deployment120and no model decision122has been made, the front-end system126can render the graphic402to illustrate that the machine learning project106does not yet have any model deployments120or model decisions122. As the learning sessions116are instantiated by the machine learning engine112and model deployments120are generated, the front-end system126can add entities to the graphic402, creating the graphic404, which includes entities to represent model deployments120and model decisions122of the model deployments120.

The graphic404can include connections between the entities of the graphic404. For example, a first entity that represents a particular model deployment120can be connected via a line drawn by the front-end system126to a second entity that represents a model decision122made by the particular model deployment120. This example connection can be included within the spherical portion of the graphic404. Furthermore, the front-end system126can draw connections between the entities of graphic404and other graphics. For example, an application represented by the internal application graphics216or the hosted application graphics218can consume a particular model decision represented by an entity within the graphic404. The front-end system126can draw a line between the entity and the internal application graphics216or the hosted application graphics218to represent that the applications consume the model decision represented by the entity.

FIG.5illustrates an example of the graphic representation104of the machine learning project106. The graphic representation104can include the data source metadata graphic206as a semi-sphere. The graphic representation104can include the model deployment graphic212as a quarter-sphere located opposite the semi-sphere. The graphic representation104can include the decision graphic214as another quarter-sphere located opposite the semi-sphere of the data source metadata graphic206. The quarter-spheres can be located together forming a second semi-sphere opposite the semi-sphere of the data source metadata graphic206. The graphic representation104can include the learning core graphic208as a sphere located between the two semi-spheres.

The data source metadata graphic206can include at least one entity. The data source metadata graphic206can include an entity502representing metadata of a particular data source. The graphic representation104includes a graphic504representing a particular learning session116. The particular learning session116can consume the data of the data source. To represent this relationship, the graphic representation104can include a connection506drawn between the entity502and the graphic504. The particular learning session116can use the data of the data source to perform a learning session where a training or inference pipeline is produced. The training pipeline can produce a particular model deployment120, e.g., an inference pipeline. The particular model deployment120can be represented as an entity508within the model deployment graphic212. To represent that the learning session116produces the particular model deployment120, the graphic representation104can include a connection510between the graphic504and the entity508. The particular model deployment120can create outputs based on input data fed into the particular model deployment120. The outputs can be model decisions (e.g., predictions, inferences, categorizations). The model decision122of the particular model deployment120can be represented as entity512included within the decision graphic214. To represent that the particular model deployment120produces the model decision, the graphic104can include a connection513between the entity508and the entity512.

The back-end system124can monitor the health of the machine learning project106. For example, the back-end system124can determine a health level of data of the data source116, of a learning session116, of a model deployment120, or a model decision122. For example, the back-end system124can generate health scores for the various components of the machine learning project106and compare the health scores to a threshold. Responsive to identifying that the health scores satisfy the threshold (e.g., the health scores are greater than the threshold), the back-end system124can identify an issue and cause the front-end system126to modify the appearance of the graphic representation104to indicate the issue.

For example, the back-end system124can identify a health level of a model deployment120by comparing model decisions122of the model deployment120to truth data. The back-end system124can identify error levels with the model decisions122. Responsive to the error levels satisfying a threshold, e.g., being greater than a particular level, the front-end system126can modify the appearance of the graphic representation104to represent the health issue. For example, if the health level of a model deployment represented by an entity satisfies a threshold, a connection516between a graphic518that represents a learning session that produces the model deployment represented by the entity514can be highlighted (e.g., bolded, colored red, animated to flash). The learning session represented by the graphic518can consume data of a data source. The metadata of the data source can be represented as entity520and the consumption of the data by the learning session can be represented by the connection522between the entity520and the graphic518. The connection516can be highlighted by changing a color of the connection516(e.g., changing the color from black to red) or changing a thickness of the connection516(e.g., increasing the thickness).

The machine learning engine112can respond to the health issue of the model deployment. For example, the machine learning engine112can instantiate a new learning session to replace the learning session represented by the graphic518to improve the performance of the model deployment represented by the entity514. The back-end system124can identify the instantiation of the new learning session116and can modify the graphic representation104to include a new graphic532to represent the new leaning session116. The graphic representation104can include a connection524between the entity520and the graphic532indicating that the learning session represented by the graphic518consumes data of a data source which has its metadata represented by entity520. The learning session represented532can identify that additional data is needed for the model deployment514. The graphic representation104can draw a connection526between the graphic532and an entity528representing metadata of the additional data. The graphic representation104can draw a connection530between the graphic532and the entity514representing that the learning session represented by the graphic518is producing the model deployment represented by the entity514. Responsive to the new learning session being instantiated and replacing the learning session represented by the graphic518, the front-end system126can remove the connection516, the graphic518, and the connection522from the graphic representation104.

FIG.6illustrates an example of the learning session graphic210of the machine learning project of106. The learning session graphic210can include various graphic components (e.g., sub-components or sub-entities) representing components of the learning session116. The graphic components can be two dimensional or three dimensional graphic components, e.g., spheres, alpha-numeric letters, cubes, DAGs. For example, learning session116can include at least one learning strategy represented by at least one graphic602. The learning strategy can include a set of instructions to perform meta-learning. The set of instructions can indicate steps to identifying template learning pipelines, adjustments to learning pipelines, constructing inference pipelines. The set of instructions can indicate steps for identifying the appropriate models or model types used in a previous project that are applicable for another project that the learning session is implemented for.

The learning session116includes at least one pipe constructor represented by graphic604. The pipeline constructor can be a piece of code that builds pipelines of the learning session600. For example, the pipeline constructor can build pipelines for training or inference, e.g., DAGs that represent the steps of training of a model or generating inferences by the model. The pipelines can be represented as at least one graphic610in the learning session graphic210. The learning session116can include pointer to a repository of experiments. The experiments can be experiments that test whether certain model architectures or training methodologies that solve various machine learning problems. The pointer can be represented as an element606. Knowledge retrieved from the repository via the pointer can be included within the learning session116and represented as at least one graphic608. The learning session116can include various tasks that can be represented as at least one graphic612.

FIGS.7-8illustrates an example of a method700of performing a learning service. The data processing system102can store instructions on memory devices that cause at least one processor of the data processing system102to perform the method700. Any computing system or device described herein can execute the method700. The method700includes an ACT702of sending problem metadata to a learning core. The method700includes an ACT704of labeling components according to their importance to the problem. The method700includes an ACT706of sending instructions to solve the problem to a learning session. The method700includes an ACT708of instantiating components inside a new learning session based on the instructions. The method700includes an ACT710of creating learning pipelines. The method700includes an ACT712of producing training pipelines with the learning pipelines. The method700includes an ACT714of training on data with the learning pipelines to produce inference pipelines. The method700includes an ACT716of storing results of inference pipelines in an experience repository. The method700includes an ACT718of repeating at least one of ACTS710-716to continue a learning process. The method700includes an ACT720of sending experience data from the learning session to the learning core.

The method700can be performed by the data processing system102, e.g., by the machine learning project106(e.g., the learning session116) or the machine learning engine112(e.g., the learning core114). As the data processing system102performs the method700, the virtual sensor128can monitor the performance of the method700and generate data indicating changes made to learning, training, inferences, or model deployment. The front-end system126can modify the graphic representation104as the data processing system102performs the method700to align the graphic representation104with the current actions being performed by the data processing system102. A visual representation of the learning session116can be modified by the front-end system126as illustrated by the changes to the graphics752-762.

At ACT702, the method700can include sending, by the data processing system102, problem metadata to the learning core114. The problem metadata can describe a machine learning problem. For example, the problem metadata can indicate a variable or parameter of a data set that a machine learning model is intended to predict or determine. The problem metadata can be generated by a user via the computing system108and provided to the learning core114by the data processing system102.

At ACT704, the method700can including labeling, by the data processing system102, components available for a learning session according to their importance to the problem indicated by the problem metadata. For example, the learning core114can analyze a repository of records of other machine learning projects, the training algorithms used in the other projects, the model architectures used in the other projects to identify components that are important or relevant for solving the problem. The learning core114can identify learning strategies, pipeline constructors, pipelines, tasks, or knowledge that is important or relevant for solving the problem. The learning core114can rank the various components according to their important or relevance for solving the problem and select a set of high ranking components (e.g., a set of the highest ranking components). The front-end system126can generate the graphic752including various components. The front-end system126can highlight the components in the graphic752that the learning core114selects as being important to solving the machine learning problem.

At ACT706, the method700can include sending, by the data processing system102, instructions to solve the problem to a learning session. For example, the learning core114can send the instructions to the machine learning engine112or the machine learning project106to instantiate a learning session116including the components identified at ACT704. At ACT708, the method700can include instantiating, by the data processing system102, components inside a new learning session116based on the instructions received at ACT706. The data processing system102can instantiate the learning session116in the machine learning project106. The data processing system102can cause the learning session116to include the components identified at the ACT704. Responsive to the learning session116being instantiated, the front-end system126can generate the graphic754. The graphic754can be a new graphic representing the instantiated learning session116. The graphic754can be a modified version of the graphic752. For example, the front-end system126can add or remove components from the graphic752to create the graphic754. The graphic754can include a reorganization of the components identified at ACT704.

At ACT708, the method700can include creating, by the data processing system102, learning pipelines. The learning pipelines can be DAGs. The learning pipelines can lay out a set of steps for selecting model architectures, training the model architectures, comparing the performance of the various model architectures, and selecting a model for implementation. The learning pipeline can further include training pipelines. The training pipelines can indicate the steps of identifying parameter values for training a machine learning model. Furthermore, the pipelines can include inference pipelines. The inference pipelines can be executions of the machine learning model to generate an output, e.g., a decision, a prediction, and inference. The front-end system126can update the graphic754to include components representing the various pipelines created at step710.

At ACT712, the method700can include producing, by the data processing system102, training pipelines with the learning pipelines created at the ACT710. The learning pipelines, when executed, can create training pipelines for training particular models. For example, the learning pipelines can identify a model architecture to be trained that can solve the machine learning problem. The learning pipeline can produce a training pipeline that trains an instance of the model architecture. The learning pipeline can identify hyper-parameters for the training pipeline, training algorithms for the training pipeline, configurations of the training algorithms. The front-end system126can create new components within the graphic758to represent the training pipelines that the data processing system102produces.

At ACT714, the method700can include training, by the data processing system102, based on data with the training pipelines to produce inference pipelines. The data processing system102can produce an inference pipeline for each training pipeline. The inference pipeline can be a set of steps that execute a machine learning model trained according to the training pipeline. The front-end system126can create new components within the graphic760to represent the inference pipelines that the data processing system102produces.

At ACT716, the method700can include storing, by the data processing system102, results of the inference pipelines in an experience repository. The experience repository can include data that indicates the training steps taken to produce inference pipelines or train models. The experience repository can indicate the inference pipelines or trained models. The results can indicate the performance of the various training methodologies and model architectures. The front-end system126can draw lines in the graphic762between the inference pipelines and a component of the graphic762representing a link to the experience repository to indicate that the inference pipelines have been stored in the experience repository.

At ACT718, the method700can include repeating, by the data processing system102, one or more steps of the method700to continue learning. The repeated learning can build off of the knowledge stored in the experience repository at ACT716. For example, the data processing system102can identify that certain types or structures of machine learning models that produced high performing results (e.g., results that meet a threshold.). At ACT720, the method700can include sending, by the data processing system102, the experience data from the learning session116to the learning core114. The data processing system102can send the experience data to the learning core114responsive to completing a learning process, e.g., completing steps702-720. The learning core114can use the experience data for perform future learning sessions. The experience data can be the results store din the experience repository at ACT716.

FIG.9illustrates an example of a DAG900of the learning session116. The DAG900can include be a learning pipeline, a training pipeline, or an inference pipeline. The DAG900can be displayed as a graphical component or set of components within a graphic representation such as the graphic600. The DAG900can be a zoomed in version of the graphic610. The DAG900can include data elements902representing data that is processed or derived. The DAG900can include actions904. The actions can be computational actions such as computing functions, computing tasks, calculations, etc. that are performed on the data elements902.

FIG.10illustrates an example of a user interface element1000indicating data for the learning sessions116of the machine learning project106. The user interface element1000can be displayed within the user interface110. The user interface element1000can be generated by the front-end system126to represent the status or description of the learning sessions116. The front-end system126can collect data from the machine learning project106indicating a number of the learning sessions116, a number of models produced by the learning sessions116, a number of active learning sessions116, a number of finished learning session116, and a number of versions. The front-end system126can cause the collected data to be included within the user interface element1000.

FIG.11illustrates an example of a graphic representation1100of raw metadata entities of the data source metadata graphic206. The graphic representation1100can be rendered on the computing system108responsive to a user interacting with the data source metadata graphic206. For example, a user can interact with the data source metadata graphic206and an expanded or zoomed in version of the data source metadata graphic206can be rendered including the graphic representation1100. The graphic representation1100can be a grid of elements or entities in two dimensions or three dimensions, rows of elements, or a free form shape of elements (e.g., the elements can be located in pseudo-random locations).

Each element of graphic representation1100can be named and can include a visual representation. The visual representation can include a color, shape, or size. The visual representation can be set based on the data type of represented by the element. For example, the element can represent categorical data, numeric data, a date, text, an image. The visual appearance can further indicate whether the element represents a high value (e.g., a highest value in a set), a low value (e.g., a lowest value in a set), an indication of a target value that the machine learning project106is determining or a key value that has a significant impact on the determinations of the machine learning project106.

FIG.12illustrates an example of a graphic representation1200of derived metadata entities of the data source metadata graphic206. The graphic representation1200can be rendered on the computing system108responsive to a user interacting with the data source metadata graphic206. For example, a user can interact with the data source metadata graphic206and an expanded or zoomed in version of the data source metadata graphic206can be rendered including the graphic representation1200. The graphic representation1200can be a grid of elements in two dimensions or three dimensions, rows of elements, or a free form shape of elements (e.g., the elements can be located in pseudo-random locations). The individual elements of the graphic representation1200can include a visual representation representing the type of data or other features of the data, for example, indicating whether the data element is categorical data, numeric data, a date, text, an image, etc. The visual appearance can further indicate whether the element represents a high value (e.g., a highest value in a set), a low value (e.g., a lowest value in a set), an indication of a target value that the machine learning project106is determining or a key value that has a significant impact on the determinations of the machine learning project106.

FIG.13illustrates an example of a user interface element1300for controlling the graphic representation of the machine learning project106. The user interface element1300can be included within the user interface110and displayed by the computing system108. The user interface1300includes control elements1302-1312. The control elements1302-1312can include a disable selection element1302. Responsive to interacting with the disable selection element1302, the ability to select components of the graphic representation104can be disabled. This can allow a user to pan, zoom, or tilt the graphic representation104on a display of the computing system108without accidently making selections of the components of the graphic representation104.

The control element1304can be a time control element that allows a user to move forward, backwards in time, or pause the passage of time relative to the graphic representation104. For example, the front-end system126can store a historical record of the graphic representation104that a user can rewind and play back on the computing system108. Responsive to interacting with the control element1304, an element1600discussed for example atFIG.16can be displayed in the user interface110. The control element1306can filter components of the graphic representation104by name. For example, the front-end system126can filter entities of the data source metadata graphic206based on based on the name of each entity.

The control element1308can filter components of the graphic representation104by purpose. For example, the front-end system126can filter entities of the data source metadata graphic206based on purpose each entity, e.g., whether the entity represents data being predicted, used for training, used for making an inference. The control element1310can filter components of the graphic representation104by type. For example, the front-end system126can filter entities of the data source metadata graphic206based on type each entity, e.g., whether the entity represents data of various types including audio, numeric, categorical, text, date, date duration, image, category, geospatial. The control element1312can filter components of the graphic representation104by value. For example, the front-end system126can filter entities of the data source metadata graphic206based on the value each entity, e.g., filter out entities representing data values below a threshold, above a threshold, within thresholds.

FIG.14illustrates an example of a user interface element1400representing data sources. The graphic representation1400can be rendered on the computing system108responsive to a user interacting with the data source graphic202. For example, a user can interact with the data source graphic202and an expanded or zoomed in version of the data source graphic202can be rendered including the graphic representation1400. The graphic representation1400can be a grid of elements in two dimensions or three dimensions, rows of elements, or a free form shape of elements (e.g., the elements can be located in pseudo-random locations).

FIG.15illustrates an example of a user interface element1500including data describing the data sources. The user interface element1500can be displayed within the user interface110. The user interface element1500can be generated by the front-end system126to represent data of a particular data source represented as an entity within the data source graphic202. The front-end system126can collect data from the machine learning project106indicating a name of a data source, a number of rows in the data source, a number of columns of the data source, and a data type detected (e.g., audio, numeric, categorical, text, date, date duration, image, categorical). The front-end system126can cause the collected data to be included within the user interface element1500.

FIG.16is an example of a user interface element1600for controlling time in an animation of the graphic representation104of the machine learning project106is shown. The front-end system126can record data received from the back-end system124describing changes of the machine learning project106at various points in time. The front-end system126can store a historical record cataloging the data source metadata118, the learning sessions116, the model deployments120, or the model decisions122of the machine learning project. The historical record can catalog a configuration of the elements or indicate the point in time that the component was instantiated. Based on user input to the user interface element1600, the front-end system126can animate the graphic representation104to illustrate changes to the machine learning project106over time, e.g., illustrating components being added, removed, adjusted, new connections drawn, removed, adjusted, etc. The user interface element1600can include a play or pause button, a fast forward button, a rewind button, a skip ahead button, a skip back button, etc.

FIG.17is an example method1700of generating the graphic representation104of the machine learning project106. The data processing system102can store instructions on memory devices that cause at least one processor of the data processing system102to perform the method1700. Any computing system or device described herein can execute the method1700. The method1700includes an ACT1702of deploying virtual sensors to a machine learning project. The method includes an ACT1704of collecting data of the machine learning project with the virtual sensors. The method includes an ACT1706of translating the data into a graphic that represents the machine learning project.

At ACT1702, the method1700includes deploying, the data processing system102, virtual sensors128to the machine learning project106. The back-end system124can deploy the virtual sensors128throughout the machine learning project106, e.g., at least one first virtual sensor128to monitor the data source metadata118, at least one second virtual sensor128to monitor the learning session116, at least one third virtual sensor128to monitor the model deployment120, and at least one fourth virtual sensor128to monitor the model decision122. The virtual sensors128can be located at various points within the machine learning project106. For example, the virtual sensors128can be stored within code modules, functions, data elements, data storage elements. Each virtual sensor128can be stored at a location where the virtual sensor128can monitor a component of the machine learning project106. For example, the first virtual sensor128that monitors the data source metadata118can be located at a first point in the machine learning project106that accesses and detects information regarding the data source metadata118. The second virtual sensor128can be located at a second point in the machine learning project106that accesses and detects information regarding the learning session116. The third virtual sensor128can be located at a third location that accesses and detects information regarding the model deployment120. The fourth virtual sensor128can be located at a fourth location that accesses and detects information regarding the model decision122.

The back-end system124can deploy the virtual sensors128to the machine learning project106by causing the virtual sensors128to be stored at various locations in the machine learning project106. Each of the virtual sensors128can monitor a data point, data element, status, or configuration of the machine learning project106. The virtual sensors128can detect the generation of new components of the machine learning project106, the deletion of existing components of the machine learning project106, a reconfiguration of components of the machine learning project106. For example, one of the virtual sensors128can identify that new learning session116is implemented. Another virtual sensor128can identify that the data source metadata118was used by a first learning session116but has been changed to be provided to a second learning session116.

At ACT1704, the method1700includes collecting, by the data processing system102, data of the machine learning project106with the virtual sensor128. The virtual sensors128can generate data streams indicating periodic values of data points or data elements, indications of changes of value, indications of new data elements being created or existing data elements being removed. The data streams can be event streams. The data streams can indicate real-time or near real-time data. The back-end system124can aggregate or collect the data of the virtual sensors128and store the data in one or more databases or data storage elements. The back-end system124can provide the data to the front-end system126.

At ACT1706, the method1700includes translating, by the data processing system102, the collected data into at least one graphic that represents the project. The front-end system126can apply a visual language to the collected data to translate the data into visual representations of the machine learning project104. The visual language can include a set of rules indicating the shapes, sizes, or locations of the various graphic components that make up the graphic representation104. The front-end system126can apply the rule sets to the data and generate the graphics based on the application of the data to the rule sets. For example, a rule can indicate that when a new learning session116is implemented in the machine learning project106, the front-end system126should render a sphere in the graphic representation104to represent the new learning session116. Another rule can indicate that if data source is used by the learning session, a connection should be drawn by the front-end system126between an entity representing metadata of the data source and a graphic representing the learning session. The graphic representation104of the machine learning project106can be rendered by the computing system108.

FIG.18is a block diagram of an example of the data processing system102. The data processing system102can include or be general-purpose computers, network appliances, mobile devices, servers, cloud computing systems, or other electronic systems. The data processing system102can include at least one processor1802, at least one memory1804, at least one storage device1806, and at least one input/output device1808. The processor1802, the memory1804, the storage device1806, and the input/output device1808can be interconnected, for example, using at least one system bus1810. The processor1802can process instructions for execution within the data processing system102. The processor1802can include a single-threaded processor. The processor1802can include a multi-threaded processor. The processor1802can process instructions stored in the memory1804or on the storage device1806.

The memory1804can store information within the data processing system102. The memory1804can include a non-transitory computer-readable medium. The memory1804can include a volatile memory unit. The memory1804can include a non-volatile memory unit. The storage device1806can provide mass storage for the data processing system102. The storage device1806can include a non-transitory computer-readable medium. The storage device1806can include a hard disk device, an optical disk device, a solid-date drive, a flash drive, or some other large capacity storage device. The storage device1806can store long-term data (e.g., database data, file system data, etc.). At least one input/output device1808can perform input/output operations for the data processing system102. The input/output device1808can include one or more of a network interface devices, e.g., an Ethernet card, a serial communication device, e.g., an RS-232 port, and/or a wireless interface device, e.g., a Wi-Fi card (e.g., an 802.11 card), a 3G wireless modem, a 4G wireless modem, or a 5G wireless modem. In some implementations, the input/output device1808can include driver devices configured to receive input data and send output data to other input/output devices, e.g., keyboard, printer and display devices1812. The input/output devices1812can include smartphones, laptops, tablets, desktop computers, printers, speakers, microphones, or other devices.

FIGS.19-34depict display screens or portions thereof with example graphical user interfaces. The outermost broken lines inFIGS.19-34illustrate the display screen or portion thereof. The graphical user interfaces may be generated, provided, and/or otherwise included with one or more embodiments described herein. Various modifications to the depicted graphical user interfaces are contemplated, such as certain of the depicted elements of one graphical user interface may be added to another graphical user interface, one or more depicted elements of certain graphical user interfaces may be removed, and/or other modifications (e.g., various graphical user interfaces may be linked together as a sequence of images to form an animated graphical user interface sequence). Further, the depicted graphical user interfaces may include various colors, color combinations, and/or other visual elements (e.g., textures, patterns, etc.) to illustrate contrasts in appearance. Moreover, modifications of the values of any depicted numbers, words, and letters is contemplated with such changes intended to fall within the scope of the disclosure. Thus, the depicted graphical user interfaces may have a variety of different appearances.

Further, unless otherwise noted, the use of the words “approximate,” “about,” “around,” “substantially,” etc., mean plus or minus ten percent.