opened
kamalmemon
username_0
According to [documentation](https://www.kubeflow.org/docs/pipelines/sdk/pipelines-metrics/) in order to enable metrics, your program must write out a file named `/mlpipeline-metrics.json` and this [stackoverflow answer](https://stackoverflow.com/a/60742184/3711575) says that you have to add the `file_outputs` in `dsl.ContainerOp` method. 

However, while creating a component, I am unable to do so when using `create_component_from_func` method. I have added `mlpipeline_metrics: OutputPath('mlpipeline-metrics')` in my component args but to no avail. 

Here is the full component:

`from kfp.components import InputPath, OutputPath, create_component_from_func

def xgboost_train_eval(
    processed_data_path: InputPath('CSV'),
    model_path: OutputPath('XGBoostModel'),
    model_config_path: OutputPath('XGBoostModelConfig'),
    mlpipeline_metrics: OutputPath('mlpipeline-metrics'),

    label_column: str = 'install',
    num_iterations: int = 10,
    booster_params: dict = None,

    # Train-test split ratio
    test_size: float = 0.25,

    # Booster parameters
    objective: str = 'binary:logistic',
    booster: str = 'gbtree',
    learning_rate: float = 0.3,
    min_split_loss: float = 0,
    max_depth: int = 6,
    eval_metric: str = 'logloss',
    
):
 
    import json
    import pandas as pd
    import xgboost as xgb
    from sklearn.model_selection import train_test_split
    from sklearn.metrics import roc_auc_score, log_loss
    from tensorflow.python.lib.io import file_io

    # Read csv in pandas dataframe
    df = pd.read_csv(
        processed_data_path,
    )

    X = df.drop(columns=[label_column])
    y = df.loc[:, label_column]

    # Train-test split
    X_train, X_test, y_train, y_test = train_test_split(X, y,
                                                        test_size=0.25,
                                                        stratify=y)

    # Convert train/test datasets into the optimized DMatrix data structure
    dtrain = xgb.DMatrix(X_train, label=y_train)
    dtest = xgb.DMatrix(X_test, label=y_test)

    # Set default booster parameters if not passed as arguments
    booster_params = booster_params or {}
    booster_params.setdefault('objective', objective)
    booster_params.setdefault('booster', booster)
    booster_params.setdefault('learning_rate', learning_rate)
    booster_params.setdefault('min_split_loss', min_split_loss)
    booster_params.setdefault('max_depth', max_depth)
    booster_params.setdefault('eval_metric', eval_metric)

    print('Training model... this might take a while')
    # Train model
    model = xgb.train(
        params=booster_params,
        dtrain=dtrain,
        num_boost_round=num_iterations,
        evals=[(dtrain, "train")],
        verbose_eval=2
    )

    # Saving the model in binary format
    model.save_model(model_path)

    # Saving model config as JSON string
    model_config_str = model.save_config()
    with open(model_config_path, 'w') as model_config_file:
        model_config_file.write(model_config_str)

    # predict test data probabilites
    y_proba = model.predict(dtest)
    
    # evaluate model on test data
    loss = log_loss(y_test, y_proba)
    roc_auc = roc_auc_score(y_test, y_proba)

    # Exporting eval metrics for KF pipeline
    metrics = {
        'metrics': [
            {
                'name': 'roc-auc-score',
                'numberValue':  roc_auc
            }, 
            {
                'name': 'log-loss',
                'numberValue':  loss
            }
        ]
    }
    
    with file_io.FileIO('/mlpipeline-metrics.json', 'w') as f:
        json.dump(metrics, f)


if __name__ == '__main__':
    create_component_from_func(
        xgboost_train_eval,
        output_component_file='component.yaml',
        base_image='python:3.7',
        packages_to_install=[
            'xgboost==1.1.1',
            'pandas==1.0.5',
            'scikit_learn==0.23.2',
            'tensorflow==1.15.0',
            'gcsfs==0.7.0'
        ]
    )
`
metric export from create_component_from_func
issue
closed
kamalmemon
username_0
issue
created
kamalmemon
username_0
I resolved this issue by adding `mlpipeline_metrics_path: OutputPath()` instead of `mlpipeline_metrics: OutputPath('mlpipeline-metrics')` in my component arguments. And then instead of writing to `with file_io.FileIO('/mlpipeline-metrics.json', 'w') as f:` I used the metrics output path as `with file_io.FileIO(mlpipeline_metrics_path, 'w') as f:`.

This approach worked and now I can see the metrics in the pipeline UI.
comment