A system, comprising: a metadata generator configured to: receive a dataflow plan comprising data operations in a dataflow pipeline, wherein the dataflow plan indicates one or more data sources, one or more dataflow transformations and one or more data sinks, wherein the data operations comprise at least one of extracting input data from the one or more data sources, transforming the input data into transformed data, and loading the input data or the transformed data into the one or more data sinks; and generate metadata for the received dataflow plan comprising information about each of the data operations, the one or more data sources and the one or more data sinks; a weight generator configured to: generate a feature set comprising a plurality of features from the received dataflow plan, wherein each feature of the plurality of features corresponds to one of a type of a dataflow transformation, a number of join operations, a type of a write operation, a type of a data source, and a type of a data sink; and create a weighted dataflow plan from the received dataflow plan by generating a feature weight set for the feature set based on the metadata using a trained machine learning, ML, model and associating a weight with each of the data operations based on the generated feature weight set; a max flow detector configured to: determine a maximum dataflow for the weighted dataflow plan; and a mapper configured to: select at least one of a runtime environment or one or more runtime resources to execute the received dataflow plan based at least on the maximum dataflow. a metadata generator configured to: receive a dataflow plan comprising data operations in a dataflow pipeline, wherein the dataflow plan indicates one or more data sources, one or more dataflow transformations and one or more data sinks, wherein the data operations comprise at least one of extracting input data from the one or more data sources, transforming the input data into transformed data, and loading the input data or the transformed data into the one or more data sinks; and generate metadata for the received dataflow plan comprising information about each of the data operations, the one or more data sources and the one or more data sinks; receive a dataflow plan comprising data operations in a dataflow pipeline, wherein the dataflow plan indicates one or more data sources, one or more dataflow transformations and one or more data sinks, wherein the data operations comprise at least one of extracting input data from the one or more data sources, transforming the input data into transformed data, and loading the input data or the transformed data into the one or more data sinks; and generate metadata for the received dataflow plan comprising information about each of the data operations, the one or more data sources and the one or more data sinks; a weight generator configured to: generate a feature set comprising a plurality of features from the received dataflow plan, wherein each feature of the plurality of features corresponds to one of a type of a dataflow transformation, a number of join operations, a type of a write operation, a type of a data source, and a type of a data sink; and create a weighted dataflow plan from the received dataflow plan by generating a feature weight set for the feature set based on the metadata using a trained machine learning, ML, model and associating a weight with each of the data operations based on the generated feature weight set; generate a feature set comprising a plurality of features from the received dataflow plan, wherein each feature of the plurality of features corresponds to one of a type of a dataflow transformation, a number of join operations, a type of a write operation, a type of a data source, and a type of a data sink; and create a weighted dataflow plan from the received dataflow plan by generating a feature weight set for the feature set based on the metadata using a trained machine learning, ML, model and associating a weight with each of the data operations based on the generated feature weight set; a max flow detector configured to: determine a maximum dataflow for the weighted dataflow plan; and a mapper configured to: select at least one of a runtime environment or one or more runtime resources to execute the received dataflow plan based at least on the maximum dataflow.