Repository: alteryx/evalml (python)

Issue:
Code generation for pipelines
Add the ability for a pipeline to generate code (as a string) which can be copy-pasted to retrain and recreate the pipeline.

Initially, we should only support python code. However, in the future we could generate code for other languages (c, java, etc.). We should keep that in mind as we design the APIs.

Some third-party modeling libraries will support this easily and some won't. Let's focus on the easier ones first.

Example: suppose we've defined a binary classification pipeline `LogisticRegressionPipeline` which uses missing value imputation, one-hot encoding and then a logreg estimator. We'd add a method which generates something like the following code as a string:
```python
from evalml import ... # fill in with the right names
def train_pipeline(X, y):
    class CodeGeneratedPipeline(BinaryClassificationPipeline):
        component_graph = [Imputer, OneHotEncoder, LogisticRegressionClassifier]
        custom_name = 'Logistic Regression Pipeline'
    parameters = {'Imputer': {'numeric_impute_strategy': 'mean', ...}, ...}
    pipeline = CodeGeneratedPipeline(parameters=parameters)
    pipeline.fit(X, y)
    return pipeline
```

Pull request:
Code Generation for Pipelines and Components
fix #1028 

Design Doc [here](https://alteryx.quip.com/WABbAD1QHEE1/Code-Generation-Design-Document)

Component docs [here](https://evalml.alteryx.com/en/bc_code_gen/user_guide/components.html#Generate-Component-Code)
Pipeline docs [here](https://evalml.alteryx.com/en/bc_code_gen/user_guide/pipelines.html#Generate-Code)
Api docs [here](https://evalml.alteryx.com/en/bc_code_gen/api_reference.html#pipeline-utils) and [here](https://evalml.alteryx.com/en/bc_code_gen/generated/evalml.pipelines.components.utils.generate_component_code.html#evalml.pipelines.components.utils.generate_component_code)

Requirements:
- [x] Functions to generate pipeline or component code as a string so users can copy and use it elsewhere
- [x] Documentation for usage
- [x] Support for custom components

Patch:
diff --git a/docs/source/api_reference.rst b/docs/source/api_reference.rst
index de52f922..b0956ba2 100644
--- a/docs/source/api_reference.rst
+++ b/docs/source/api_reference.rst
@@ -125,6 +125,7 @@ Pipeline Utils
 
     make_pipeline
     make_pipeline_from_components
+    generate_pipeline_code
 
 
 .. currentmodule:: evalml.pipelines.components
@@ -155,6 +156,7 @@ Component Utils
 
     allowed_model_families
     get_estimators
+    generate_component_code
 
 
 .. currentmodule:: evalml.pipelines.components
diff --git a/docs/source/release_notes.rst b/docs/source/release_notes.rst
index bdd6d853..42c9156d 100644
--- a/docs/source/release_notes.rst
+++ b/docs/source/release_notes.rst
@@ -10,6 +10,8 @@ Release Notes
         * Added parameter to ``OneHotEncoder`` to enable filtering for features to encode for :pr:`1249`
         * Added percent-better-than-baseline for all objectives to automl.results :pr:`1244`
         * Added ``HighVarianceCVDataCheck`` and replaced synonymous warning in ``AutoMLSearch`` :pr:`1254`
+        * Added `PCA Transformer` component for dimensionality reduction :pr:`1270`
+        * Added ``generate_pipeline_code`` and ``generate_component_code`` to allow for code generation given a pipeline or component instance :pr:`1306`
         * Added ``PCA Transformer`` component for dimensionality reduction :pr:`1270`
         * Updated ``AutoMLSearch`` to support ``Woodwork`` data structures :pr:`1299`
         * Added cv_folds to ``ClassImbalanceDataCheck`` and added this check to ``DefaultDataChecks`` :pr:`1333`
diff --git a/docs/source/user_guide/components.ipynb b/docs/source/user_guide/components.ipynb
index 21b49f1a..078126c2 100644
--- a/docs/source/user_guide/components.ipynb
+++ b/docs/source/user_guide/components.ipynb
@@ -439,6 +439,103 @@
     "        return self._component_obj.coef_"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Generate Component Code\n",
+    "\n",
+    "Once you have a component defined in EvalML, you can generate string Python code to recreate this component, which can then be saved and run elsewhere with EvalML. `generate_component_code` requires a component instance as the input. This method works for custom components as well, although it won't return the code required to define the custom component. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from evalml.pipelines.components import LogisticRegressionClassifier\n",
+    "from evalml.pipelines.components.utils import generate_component_code\n",
+    "\n",
+    "lr = LogisticRegressionClassifier(C=5)\n",
+    "code = generate_component_code(lr)\n",
+    "print(code)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# this string can then be copy and pasted into a separate window and executed as python code\n",
+    "exec(code)\n",
+    "logisticRegressionClassifier"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# custom component\n",
+    "from evalml.pipelines.components import Transformer\n",
+    "import pandas as pd\n",
+    "from evalml.pipelines.components.utils import generate_component_code\n",
+    "\n",
+    "class MyDropNullColumns(Transformer):\n",
+    "    \"\"\"Transformer to drop features whose percentage of NaN values exceeds a specified threshold\"\"\"\n",
+    "    name = \"My Drop Null Columns Transformer\"\n",
+    "    hyperparameter_ranges = {}\n",
+    "\n",
+    "    def __init__(self, pct_null_threshold=1.0, random_state=0, **kwargs):\n",
+    "        \"\"\"Initalizes an transformer to drop features whose percentage of NaN values exceeds a specified threshold.\n",
+    "\n",
+    "        Arguments:\n",
+    "            pct_null_threshold(float): The percentage of NaN values in an input feature to drop.\n",
+    "                Must be a value between [0, 1] inclusive. If equal to 0.0, will drop columns with any null values.\n",
+    "                If equal to 1.0, will drop columns with all null values. Defaults to 0.95.\n",
+    "        \"\"\"\n",
+    "        if pct_null_threshold < 0 or pct_null_threshold > 1:\n",
+    "            raise ValueError(\"pct_null_threshold must be a float between 0 and 1, inclusive.\")\n",
+    "        parameters = {\"pct_null_threshold\": pct_null_threshold}\n",
+    "        parameters.update(kwargs)\n",
+    "\n",
+    "        self._cols_to_drop = None\n",
+    "        super().__init__(parameters=parameters,\n",
+    "                         component_obj=None,\n",
+    "                         random_state=random_state)\n",
+    "\n",
+    "    def fit(self, X, y=None):\n",
+    "        pct_null_threshold = self.parameters[\"pct_null_threshold\"]\n",
+    "        if not isinstance(X, pd.DataFrame):\n",
+    "            X = pd.DataFrame(X)\n",
+    "        percent_null = X.isnull().mean()\n",
+    "        if pct_null_threshold == 0.0:\n",
+    "            null_cols = percent_null[percent_null > 0]\n",
+    "        else:\n",
+    "            null_cols = percent_null[percent_null >= pct_null_threshold]\n",
+    "        self._cols_to_drop = list(null_cols.index)\n",
+    "        return self\n",
+    "\n",
+    "    def transform(self, X, y=None):\n",
+    "        \"\"\"Transforms data X by dropping columns that exceed the threshold of null values.\n",
+    "        Arguments:\n",
+    "            X (pd.DataFrame): Data to transform\n",
+    "            y (pd.Series, optional): Targets\n",
+    "        Returns:\n",
+    "            pd.DataFrame: Transformed X\n",
+    "        \"\"\"\n",
+    "\n",
+    "        if not isinstance(X, pd.DataFrame):\n",
+    "            X = pd.DataFrame(X)\n",
+    "        return X.drop(columns=self._cols_to_drop, axis=1)\n",
+    "    \n",
+    "myDropNull = MyDropNullColumns()\n",
+    "print(generate_component_code(myDropNull))"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -472,4 +569,4 @@
  },
  "nbformat": 4,
  "nbformat_minor": 4
-}
\ No newline at end of file
+}
diff --git a/docs/source/user_guide/pipelines.ipynb b/docs/source/user_guide/pipelines.ipynb
index 5978388a..8195a1fa 100644
--- a/docs/source/user_guide/pipelines.ipynb
+++ b/docs/source/user_guide/pipelines.ipynb
@@ -317,6 +317,102 @@
    "source": [
     "pipeline.input_feature_names"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Generate Code\n",
+    "\n",
+    "Once you have a pipeline defined in EvalML, you can generate string Python code to recreate this pipeline, which can then be saved and run elsewhere with EvalML. `generate_pipeline_code` requires a pipeline instance as the input. It can also handle custom components, but it won't return the code required to define the component."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from evalml.pipelines.utils import generate_pipeline_code\n",
+    "from evalml.pipelines import MulticlassClassificationPipeline\n",
+    "import pandas as pd\n",
+    "\n",
+    "class MyDropNullColumns(Transformer):\n",
+    "    \"\"\"Transformer to drop features whose percentage of NaN values exceeds a specified threshold\"\"\"\n",
+    "    name = \"My Drop Null Columns Transformer\"\n",
+    "    hyperparameter_ranges = {}\n",
+    "\n",
+    "    def __init__(self, pct_null_threshold=1.0, random_state=0, **kwargs):\n",
+    "        \"\"\"Initalizes an transformer to drop features whose percentage of NaN values exceeds a specified threshold.\n",
+    "\n",
+    "        Arguments:\n",
+    "            pct_null_threshold(float): The percentage of NaN values in an input feature to drop.\n",
+    "                Must be a value between [0, 1] inclusive. If equal to 0.0, will drop columns with any null values.\n",
+    "                If equal to 1.0, will drop columns with all null values. Defaults to 0.95.\n",
+    "        \"\"\"\n",
+    "        if pct_null_threshold < 0 or pct_null_threshold > 1:\n",
+    "            raise ValueError(\"pct_null_threshold must be a float between 0 and 1, inclusive.\")\n",
+    "        parameters = {\"pct_null_threshold\": pct_null_threshold}\n",
+    "        parameters.update(kwargs)\n",
+    "\n",
+    "        self._cols_to_drop = None\n",
+    "        super().__init__(parameters=parameters,\n",
+    "                         component_obj=None,\n",
+    "                         random_state=random_state)\n",
+    "\n",
+    "    def fit(self, X, y=None):\n",
+    "        pct_null_threshold = self.parameters[\"pct_null_threshold\"]\n",
+    "        if not isinstance(X, pd.DataFrame):\n",
+    "            X = pd.DataFrame(X)\n",
+    "        percent_null = X.isnull().mean()\n",
+    "        if pct_null_threshold == 0.0:\n",
+    "            null_cols = percent_null[percent_null > 0]\n",
+    "        else:\n",
+    "            null_cols = percent_null[percent_null >= pct_null_threshold]\n",
+    "        self._cols_to_drop = list(null_cols.index)\n",
+    "        return self\n",
+    "\n",
+    "    def transform(self, X, y=None):\n",
+    "        \"\"\"Transforms data X by dropping columns that exceed the threshold of null values.\n",
+    "        Arguments:\n",
+    "            X (pd.DataFrame): Data to transform\n",
+    "            y (pd.Series, optional): Targets\n",
+    "        Returns:\n",
+    "            pd.DataFrame: Transformed X\n",
+    "        \"\"\"\n",
+    "\n",
+    "        if not isinstance(X, pd.DataFrame):\n",
+    "            X = pd.DataFrame(X)\n",
+    "        return X.drop(columns=self._cols_to_drop, axis=1)\n",
+    "\n",
+    "class CustomPipeline(MulticlassClassificationPipeline):\n",
+    "    name = \"Custom Pipeline\"\n",
+    "    component_graph = ['Imputer', MyDropNullColumns, 'One Hot Encoder', 'Random Forest Classifier']\n",
+    "    custom_hyperparameters={\n",
+    "        \"Imputer\": {\n",
+    "            \"numeric_impute_strategy\": ['mean', 'median']\n",
+    "        }\n",
+    "    }\n",
+    "\n",
+    "pipeline_instance = CustomPipeline(parameters={\"Imputer\": {\"numeric_impute_strategy\": \"median\"}})\n",
+    "code = generate_pipeline_code(pipeline_instance)\n",
+    "print(code)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# This string can then be pasted into a separate window and run, although since the pipeline has custom component `MyDropNullColumns`, \n",
+    "#      the code for that component must also be included\n",
+    "\n",
+    "from evalml.demos import load_fraud\n",
+    "X, y = load_fraud()\n",
+    "exec(code)\n",
+    "pipeline.fit(X, y)"
+   ]
   }
  ],
  "metadata": {
@@ -340,4 +436,4 @@
  },
  "nbformat": 4,
  "nbformat_minor": 4
-}
\ No newline at end of file
+}
diff --git a/evalml/pipelines/components/utils.py b/evalml/pipelines/components/utils.py
index 71d9d137..396fad02 100644
--- a/evalml/pipelines/components/utils.py
+++ b/evalml/pipelines/components/utils.py
@@ -217,3 +217,33 @@ def scikit_learn_wrapped_estimator(evalml_obj):
         elif evalml_obj.supported_problem_types == [ProblemTypes.BINARY, ProblemTypes.MULTICLASS]:
             return WrappedSKClassifier(evalml_obj)
     raise ValueError("Could not wrap EvalML object in scikit-learn wrapper.")
+
+
+def generate_component_code(element):
+    """Creates and returns a string that contains the Python imports and code required for running the EvalML component.
+
+    Arguments:
+        element (component instance): The instance of the component to generate string Python code for
+
+    Returns:
+        String representation of Python code that can be run separately in order to recreate the component instance.
+        Does not include code for custom component implementation.
+    """
+    # hold the imports needed and add code to end
+    code_strings = []
+    base_string = ""
+
+    if not isinstance(element, ComponentBase):
+        raise ValueError("Element must be a component instance, received {}".format(type(element)))
+
+    if element.__class__ in all_components():
+        code_strings.append("from {} import {}\n".format(element.__class__.__module__, element.__class__.__name__))
+    component_parameters = element.parameters
+    name = element.name[0].lower() + element.name[1:].replace(' ', '')
+    base_string += "{0} = {1}(**{2})" \
+                   .format(name,
+                           element.__class__.__name__,
+                           component_parameters)
+
+    code_strings.append(base_string)
+    return "\n".join(code_strings)
diff --git a/evalml/pipelines/utils.py b/evalml/pipelines/utils.py
index 84062c5a..b44c5a66 100644
--- a/evalml/pipelines/utils.py
+++ b/evalml/pipelines/utils.py
@@ -7,6 +7,7 @@ from .multiclass_classification_pipeline import (
 from .regression_pipeline import RegressionPipeline
 
 from evalml.model_family import ModelFamily
+from evalml.pipelines import PipelineBase
 from evalml.pipelines.components import (  # noqa: F401
     CatBoostClassifier,
     CatBoostRegressor,
@@ -21,7 +22,7 @@ from evalml.pipelines.components import (  # noqa: F401
     StackedEnsembleRegressor,
     StandardScaler
 )
-from evalml.pipelines.components.utils import get_estimators
+from evalml.pipelines.components.utils import all_components, get_estimators
 from evalml.problem_types import ProblemTypes, handle_problem_types
 from evalml.utils import get_logger
 from evalml.utils.gen_utils import categorical_dtypes, datetime_dtypes
@@ -147,6 +148,49 @@ def make_pipeline_from_components(component_instances, problem_type, custom_name
     return TemplatedPipeline({c.name: c.parameters for c in component_instances})
 
 
+def generate_pipeline_code(element):
+    """Creates and returns a string that contains the Python imports and code required for running the EvalML pipeline.
+
+    Arguments:
+        element (pipeline instance): The instance of the pipeline to generate string Python code for
+
+    Returns:
+        String representation of Python code that can be run separately in order to recreate the pipeline instance.
+        Does not include code for custom component implementation
+    """
+    # hold the imports needed and add code to end
+    code_strings = []
+    if not isinstance(element, PipelineBase):
+        raise ValueError("Element must be a pipeline instance, received {}".format(type(element)))
+
+    component_graph_string = ', '.join([com.__class__.__name__ if com.__class__ not in all_components() else "'{}'".format(com.name) for com in element.component_graph])
+    import_strings = [com.__class__.__name__ for com in element.component_graph if com.__class__ in all_components()]
+    if import_strings:
+        code_strings.append("from evalml.pipelines.components import (\n\t{}\n)".format(",\n\t".join(import_strings)))
+    code_strings.append("from {} import {}".format(element.__class__.__bases__[0].__module__, element.__class__.__bases__[0].__name__))
+    # check for other attributes associated with pipeline (ie name, custom_hyperparameters)
+    pipeline_list = []
+    for k, v in sorted(list(filter(lambda item: item[0][0] != '_', element.__class__.__dict__.items())), key=lambda x: x[0]):
+        if k == 'component_graph':
+            continue
+        pipeline_list += ["{} = '{}'".format(k, v)] if isinstance(v, str) else ["{} = {}".format(k, v)]
+
+    pipeline_string = "\t" + "\n\t".join(pipeline_list) + "\n" if len(pipeline_list) else ""
+    # create the base string for the pipeline
+    base_string = "\nclass {0}({1}):\n" \
+                  "\tcomponent_graph = [{2}]\n" \
+                  "{3}" \
+                  "\nparameters = {4}\n" \
+                  "pipeline = {0}(parameters)" \
+                  .format(element.__class__.__name__,
+                          element.__class__.__bases__[0].__name__,
+                          component_graph_string,
+                          pipeline_string,
+                          element.parameters)
+    code_strings.append(base_string)
+    return "\n".join(code_strings)
+
+
 def _make_stacked_ensemble_pipeline(input_pipelines, problem_type):
     """
     Creates a pipeline with a stacked ensemble estimator.


Test patch:
diff --git a/evalml/tests/component_tests/test_components.py b/evalml/tests/component_tests/test_components.py
index 22d24f8e..47b246d8 100644
--- a/evalml/tests/component_tests/test_components.py
+++ b/evalml/tests/component_tests/test_components.py
@@ -58,7 +58,8 @@ from evalml.pipelines.components.utils import (
     _all_estimators,
     _all_estimators_used_in_search,
     _all_transformers,
-    all_components
+    all_components,
+    generate_component_code
 )
 from evalml.pipelines.utils import make_pipeline_from_components
 from evalml.problem_types import ProblemTypes
@@ -1023,3 +1024,50 @@ def test_categorical_hyperparameters(X_y_binary, categorical):
 
     assert MockComponent(agg_type="mean").fit(X, y)
     assert MockComponent(agg_type="moat", category="blue").fit(X, y)
+
+
+def test_generate_code_errors():
+    with pytest.raises(ValueError, match="Element must be a component instance"):
+        generate_component_code(make_pipeline_from_components([RandomForestClassifier()], ProblemTypes.BINARY))
+
+    with pytest.raises(ValueError, match="Element must be a component instance"):
+        generate_component_code(LinearRegressor)
+
+    with pytest.raises(ValueError, match="Element must be a component instance"):
+        generate_component_code(Imputer)
+
+    with pytest.raises(ValueError, match="Element must be a component instance"):
+        generate_component_code(ComponentBase)
+
+
+def test_generate_code():
+    expected_code = "from evalml.pipelines.components.estimators.classifiers.logistic_regression import LogisticRegressionClassifier" \
+                    "\n\nlogisticRegressionClassifier = LogisticRegressionClassifier(**{'penalty': 'l2', 'C': 1.0, 'n_jobs': -1, 'multi_class': 'auto', 'solver': 'lbfgs'})"
+    component_code = generate_component_code(LogisticRegressionClassifier())
+    assert component_code == expected_code
+
+    expected_code = "from evalml.pipelines.components.estimators.regressors.et_regressor import ExtraTreesRegressor" \
+                    "\n\nextraTreesRegressor = ExtraTreesRegressor(**{'n_estimators': 50, 'max_features': 'auto', 'max_depth': 6, 'min_samples_split': 2, 'min_weight_fraction_leaf': 0.0, 'n_jobs': -1})"
+    component_code = generate_component_code(ExtraTreesRegressor(n_estimators=50))
+    assert component_code == expected_code
+
+    expected_code = "from evalml.pipelines.components.transformers.imputers.imputer import Imputer" \
+                    "\n\nimputer = Imputer(**{'categorical_impute_strategy': 'most_frequent', 'numeric_impute_strategy': 'mean', 'categorical_fill_value': None, 'numeric_fill_value': None})"
+    component_code = generate_component_code(Imputer())
+    assert component_code == expected_code
+
+
+def test_generate_code_custom(test_classes):
+    MockComponent, MockEstimator, MockTransformer = test_classes
+
+    expected_code = "mockComponent = MockComponent(**{})"
+    component_code = generate_component_code(MockComponent())
+    assert component_code == expected_code
+
+    expected_code = "mockEstimator = MockEstimator(**{})"
+    component_code = generate_component_code(MockEstimator())
+    assert component_code == expected_code
+
+    expected_code = "mockTransformer = MockTransformer(**{})"
+    component_code = generate_component_code(MockTransformer())
+    assert component_code == expected_code
diff --git a/evalml/tests/pipeline_tests/test_pipelines.py b/evalml/tests/pipeline_tests/test_pipelines.py
index daa13fc0..cc219070 100644
--- a/evalml/tests/pipeline_tests/test_pipelines.py
+++ b/evalml/tests/pipeline_tests/test_pipelines.py
@@ -42,6 +42,7 @@ from evalml.pipelines.components.utils import (
     allowed_model_families
 )
 from evalml.pipelines.utils import (
+    generate_pipeline_code,
     get_estimators,
     make_pipeline,
     make_pipeline_from_components
@@ -1420,3 +1421,166 @@ def test_pipeline_repr(pipeline_class):
     pipeline_with_nan_parameters = MockPipeline(parameters={'Imputer': {'numeric_fill_value': float('nan'), 'categorical_fill_value': np.nan}})
     expected_repr = f"MockPipeline(parameters={{'Imputer':{{'categorical_impute_strategy': 'most_frequent', 'numeric_impute_strategy': 'mean', 'categorical_fill_value': np.nan, 'numeric_fill_value': np.nan}}, '{final_estimator}':{{'n_estimators': 100, 'max_depth': 6, 'n_jobs': -1}},}})"
     assert repr(pipeline_with_nan_parameters) == expected_repr
+
+
+def test_generate_code_pipeline_errors():
+    class MockBinaryPipeline(BinaryClassificationPipeline):
+        name = "Mock Binary Pipeline"
+        component_graph = ['Imputer', 'Random Forest Classifier']
+
+    class MockMulticlassPipeline(MulticlassClassificationPipeline):
+        name = "Mock Multiclass Pipeline"
+        component_graph = ['Imputer', 'Random Forest Classifier']
+
+    class MockRegressionPipeline(RegressionPipeline):
+        name = "Mock Regression Pipeline"
+        component_graph = ['Imputer', 'Random Forest Regressor']
+
+    with pytest.raises(ValueError, match="Element must be a pipeline instance"):
+        generate_pipeline_code(MockBinaryPipeline)
+
+    with pytest.raises(ValueError, match="Element must be a pipeline instance"):
+        generate_pipeline_code(MockMulticlassPipeline)
+
+    with pytest.raises(ValueError, match="Element must be a pipeline instance"):
+        generate_pipeline_code(MockRegressionPipeline)
+
+    with pytest.raises(ValueError, match="Element must be a pipeline instance"):
+        generate_pipeline_code([Imputer])
+
+    with pytest.raises(ValueError, match="Element must be a pipeline instance"):
+        generate_pipeline_code([Imputer, LogisticRegressionClassifier])
+
+    with pytest.raises(ValueError, match="Element must be a pipeline instance"):
+        generate_pipeline_code([Imputer(), LogisticRegressionClassifier()])
+
+
+def test_generate_code_pipeline():
+    class MockBinaryPipeline(BinaryClassificationPipeline):
+        component_graph = ['Imputer', 'Random Forest Classifier']
+        custom_hyperparameters = {
+            "Imputer": {
+                "numeric_impute_strategy": "most_frequent"
+            }
+        }
+
+    class MockRegressionPipeline(RegressionPipeline):
+        name = "Mock Regression Pipeline"
+        component_graph = ['Imputer', 'Random Forest Regressor']
+
+    mock_binary_pipeline = MockBinaryPipeline({})
+    expected_code = "from evalml.pipelines.components import (" \
+                    "\n\tImputer," \
+                    "\n\tRandomForestClassifier\n)" \
+                    "\nfrom evalml.pipelines.binary_classification_pipeline import BinaryClassificationPipeline\n" \
+                    "\nclass MockBinaryPipeline(BinaryClassificationPipeline):" \
+                    "\n\tcomponent_graph = ['Imputer', 'Random Forest Classifier']" \
+                    "\n\tcustom_hyperparameters = {'Imputer': {'numeric_impute_strategy': 'most_frequent'}}\n" \
+                    "\nparameters = {'Imputer': {'categorical_impute_strategy': 'most_frequent', 'numeric_impute_strategy': 'mean', 'categorical_fill_value': None, 'numeric_fill_value': None}, 'Random Forest Classifier': {'n_estimators': 100, 'max_depth': 6, 'n_jobs': -1}}" \
+                    "\npipeline = MockBinaryPipeline(parameters)"
+    pipeline_code = generate_pipeline_code(mock_binary_pipeline)
+    assert expected_code == pipeline_code
+
+    mock_regression_pipeline = MockRegressionPipeline({})
+    expected_code = "from evalml.pipelines.components import (" \
+                    "\n\tImputer," \
+                    "\n\tRandomForestRegressor\n)" \
+                    "\nfrom evalml.pipelines.regression_pipeline import RegressionPipeline\n" \
+                    "\nclass MockRegressionPipeline(RegressionPipeline):" \
+                    "\n\tcomponent_graph = ['Imputer', 'Random Forest Regressor']" \
+                    "\n\tname = 'Mock Regression Pipeline'\n" \
+                    "\nparameters = {'Imputer': {'categorical_impute_strategy': 'most_frequent', 'numeric_impute_strategy': 'mean', 'categorical_fill_value': None, 'numeric_fill_value': None}, 'Random Forest Regressor': {'n_estimators': 100, 'max_depth': 6, 'n_jobs': -1}}" \
+                    "\npipeline = MockRegressionPipeline(parameters)"
+    pipeline = generate_pipeline_code(mock_regression_pipeline)
+    assert pipeline == expected_code
+
+    mock_regression_pipeline_params = MockRegressionPipeline({"Imputer": {"numeric_impute_strategy": "most_frequent"}, "Random Forest Regressor": {"n_estimators": 50}})
+    expected_code_params = "from evalml.pipelines.components import (" \
+                           "\n\tImputer," \
+                           "\n\tRandomForestRegressor\n)" \
+                           "\nfrom evalml.pipelines.regression_pipeline import RegressionPipeline\n" \
+                           "\nclass MockRegressionPipeline(RegressionPipeline):" \
+                           "\n\tcomponent_graph = ['Imputer', 'Random Forest Regressor']" \
+                           "\n\tname = 'Mock Regression Pipeline'\n" \
+                           "\nparameters = {'Imputer': {'categorical_impute_strategy': 'most_frequent', 'numeric_impute_strategy': 'most_frequent', 'categorical_fill_value': None, 'numeric_fill_value': None}, 'Random Forest Regressor': {'n_estimators': 50, 'max_depth': 6, 'n_jobs': -1}}" \
+                           "\npipeline = MockRegressionPipeline(parameters)"
+    pipeline = generate_pipeline_code(mock_regression_pipeline_params)
+    assert pipeline == expected_code_params
+
+
+def test_generate_code_pipeline_custom():
+    class CustomTransformer(Transformer):
+        name = "My Custom Transformer"
+        hyperparameter_ranges = {}
+
+        def __init__(self, random_state=0):
+            parameters = {}
+
+            super().__init__(parameters=parameters,
+                             component_obj=None,
+                             random_state=random_state)
+
+    class CustomEstimator(Estimator):
+        name = "My Custom Estimator"
+        hyperparameter_ranges = {}
+        supported_problem_types = [ProblemTypes.BINARY, ProblemTypes.MULTICLASS]
+        model_family = ModelFamily.NONE
+
+        def __init__(self, random_state=0):
+            parameters = {}
+
+            super().__init__(parameters=parameters,
+                             component_obj=None,
+                             random_state=random_state)
+
+    class MockBinaryPipelineTransformer(BinaryClassificationPipeline):
+        name = "Mock Binary Pipeline with Transformer"
+        component_graph = [CustomTransformer, 'Random Forest Classifier']
+
+    class MockBinaryPipelineEstimator(BinaryClassificationPipeline):
+        name = "Mock Binary Pipeline with Estimator"
+        component_graph = ['Imputer', CustomEstimator]
+        custom_hyperparameters = {
+            'Imputer': {
+                'numeric_impute_strategy': 'most_frequent'
+            }
+        }
+
+    class MockAllCustom(BinaryClassificationPipeline):
+        name = "Mock All Custom Pipeline"
+        component_graph = [CustomTransformer, CustomEstimator]
+
+    mockBinaryTransformer = MockBinaryPipelineTransformer({})
+    expected_code = "from evalml.pipelines.components import (" \
+                    "\n\tRandomForestClassifier\n)" \
+                    "\nfrom evalml.pipelines.binary_classification_pipeline import BinaryClassificationPipeline\n" \
+                    "\nclass MockBinaryPipelineTransformer(BinaryClassificationPipeline):" \
+                    "\n\tcomponent_graph = [CustomTransformer, 'Random Forest Classifier']" \
+                    "\n\tname = 'Mock Binary Pipeline with Transformer'\n" \
+                    "\nparameters = {'Random Forest Classifier': {'n_estimators': 100, 'max_depth': 6, 'n_jobs': -1}}" \
+                    "\npipeline = MockBinaryPipelineTransformer(parameters)"
+    pipeline = generate_pipeline_code(mockBinaryTransformer)
+    assert pipeline == expected_code
+
+    mockBinaryPipeline = MockBinaryPipelineEstimator({})
+    expected_code = "from evalml.pipelines.components import (" \
+                    "\n\tImputer\n)" \
+                    "\nfrom evalml.pipelines.binary_classification_pipeline import BinaryClassificationPipeline\n" \
+                    "\nclass MockBinaryPipelineEstimator(BinaryClassificationPipeline):" \
+                    "\n\tcomponent_graph = ['Imputer', CustomEstimator]" \
+                    "\n\tcustom_hyperparameters = {'Imputer': {'numeric_impute_strategy': 'most_frequent'}}" \
+                    "\n\tname = 'Mock Binary Pipeline with Estimator'\n" \
+                    "\nparameters = {'Imputer': {'categorical_impute_strategy': 'most_frequent', 'numeric_impute_strategy': 'mean', 'categorical_fill_value': None, 'numeric_fill_value': None}}" \
+                    "\npipeline = MockBinaryPipelineEstimator(parameters)"
+    pipeline = generate_pipeline_code(mockBinaryPipeline)
+    assert pipeline == expected_code
+
+    mockAllCustom = MockAllCustom({})
+    expected_code = "from evalml.pipelines.binary_classification_pipeline import BinaryClassificationPipeline\n" \
+                    "\nclass MockAllCustom(BinaryClassificationPipeline):" \
+                    "\n\tcomponent_graph = [CustomTransformer, CustomEstimator]" \
+                    "\n\tname = 'Mock All Custom Pipeline'\n" \
+                    "\nparameters = {}\n" \
+                    "pipeline = MockAllCustom(parameters)"
+    pipeline = generate_pipeline_code(mockAllCustom)
+    assert pipeline == expected_code