A system for executing variations of a machine learning experiment graph (108; 204), the system comprising: one or more processors; and one or more computer-readable media having stored thereon instructions that are executable by the one or more processors to configure the computer system to execute a graph, including instructions that are executable to configure the computer system to perform at least the following: providing a user interface comprising an experiment canvas; receiving user input defining a graph by placing nodes on the canvas and connecting them by edges, wherein each node may have one or more input ports and one or more output elements, and wherein the graph includes nodes having input ports connected to multiple inputs; receiving a user input to execute the graph; identifying that the graph includes at least two nodes (106-1, 106-2) having input ports that are coupled to multiple alternative inputs, wherein a node that is designed with an input port for a single input is implemented with a multiple-input port that is connected to multiple different alternative inputs; identifying permutations of inputs (204-1, 204-2) where each permutation uses exactly one of the alternative inputs for each of the at least two nodes having input ports coupled to multiple alternative inputs; executing the experiment variation according to each of the identified permutations; generating a metric for each execution of the graph for each of the permutations; and providing, on the user interface, the user with a summary of the generated metrics, the instructions further including instructions that are executable to configure the computer system to perform alerting a user that an input port of a node that would normally have a single input connected to it has a plurality of inputs connected to it, and prompting the user to confirm that the user wishes for different permutations of the graph to be executed using the different inputs. one or more processors; and one or more computer-readable media having stored thereon instructions that are executable by the one or more processors to configure the computer system to execute a graph, including instructions that are executable to configure the computer system to perform at least the following: providing a user interface comprising an experiment canvas; receiving user input defining a graph by placing nodes on the canvas and connecting them by edges, wherein each node may have one or more input ports and one or more output elements, and wherein the graph includes nodes having input ports connected to multiple inputs; receiving a user input to execute the graph; identifying that the graph includes at least two nodes (106-1, 106-2) having input ports that are coupled to multiple alternative inputs, wherein a node that is designed with an input port for a single input is implemented with a multiple-input port that is connected to multiple different alternative inputs; identifying permutations of inputs (204-1, 204-2) where each permutation uses exactly one of the alternative inputs for each of the at least two nodes having input ports coupled to multiple alternative inputs; executing the experiment variation according to each of the identified permutations; generating a metric for each execution of the graph for each of the permutations; and providing, on the user interface, the user with a summary of the generated metrics, the instructions further including instructions that are executable to configure the computer system to perform alerting a user that an input port of a node that would normally have a single input connected to it has a plurality of inputs connected to it, and prompting the user to confirm that the user wishes for different permutations of the graph to be executed using the different inputs. providing a user interface comprising an experiment canvas; receiving user input defining a graph by placing nodes on the canvas and connecting them by edges, wherein each node may have one or more input ports and one or more output elements, and wherein the graph includes nodes having input ports connected to multiple inputs; receiving a user input to execute the graph; identifying that the graph includes at least two nodes (106-1, 106-2) having input ports that are coupled to multiple alternative inputs, wherein a node that is designed with an input port for a single input is implemented with a multiple-input port that is connected to multiple different alternative inputs; identifying permutations of inputs (204-1, 204-2) where each permutation uses exactly one of the alternative inputs for each of the at least two nodes having input ports coupled to multiple alternative inputs; executing the experiment variation according to each of the identified permutations; generating a metric for each execution of the graph for each of the permutations; and providing, on the user interface, the user with a summary of the generated metrics, the instructions further including instructions that are executable to configure the computer system to perform alerting a user that an input port of a node that would normally have a single input connected to it has a plurality of inputs connected to it, and prompting the user to confirm that the user wishes for different permutations of the graph to be executed using the different inputs.