Repository: kedro-org/kedro

Issue #148: [KED-1155] Getting error in Kedro Spaceflight tutorial while using tags in pipeline definition
Created: 2019-10-29T02:25:43Z
Description
Completed all steps successfully but getting an error in running pipeline using tags:
https://kedro.readthedocs.io/en/latest/03_tutorial/04_create_pipelines.html#using-tags
when tags are applied, "kedro run" or "kedro run --tag=ds_tag" gives below error:
TypeError: __init__() got an unexpected keyword argument 'tags'

Context
without tags in pipeline definition, the application runs successfully as below:
$ kedro run
2019-10-29 07:42:04,749 - root - INFO - ** Kedro project kedro-tutorial
fatal: not a git repository (or any of the parent directories): .git
2019-10-29 07:42:04,761 - kedro.versioning.journal - WARNING - Unable to git describe /home/ajit/ajit-code/zzztemp/kedro/kedro-site/02-tutorial/kedro-tutorial
2019-10-29 07:42:04,782 - kedro.io.data_catalog - INFO - Loading data from `companies` (CSVLocalDataSet)...
2019-10-29 07:42:04,846 - kedro.pipeline.node - INFO - Running node: preprocess1: preprocess_companies([companies]) -> [preprocessed_companies]
2019-10-29 07:42:04,936 - kedro.io.data_catalog - INFO - Saving data to `preprocessed_companies` (CSVLocalDataSet)...
2019-10-29 07:42:05,482 - kedro.runner.sequential_runner - INFO - Completed 1 out of 6 tasks
2019-10-29 07:42:05,482 - kedro.io.data_catalog - INFO - Loading data from `shuttles` (ExcelLocalDataSet)...
2019-10-29 07:42:23,180 - kedro.pipeline.node - INFO - Running node: preprocess2: preprocess_shuttles([shuttles]) -> [preprocessed_shuttles]
2019-10-29 07:42:23,303 - kedro.io.data_catalog - INFO - Saving data to `preprocessed_shuttles` (CSVLocalDataSet)...
2019-10-29 07:42:24,418 - kedro.runner.sequential_runner - INFO - Completed 2 out of 6 tasks
2019-10-29 07:42:24,418 - kedro.io.data_catalog - INFO - Loading data from `preprocessed_shuttles` (CSVLocalDataSet)...
2019-10-29 07:42:24,601 - kedro.io.data_catalog - INFO - Loading data from `preprocessed_companies` (CSVLocalDataSet)...
2019-10-29 07:42:24,659 - kedro.io.data_catalog - INFO - Loading data from `reviews` (CSVLocalDataSet)...
2019-10-29 07:42:24,753 - kedro.pipeline.node - INFO - Running node: master_table: create_master_table([preprocessed_companies,preprocessed_shuttles,reviews]) -> [master_table]
2019-10-29 07:42:27,359 - kedro.io.data_catalog - INFO - Saving data to `master_table` (CSVLocalDataSet)...
2019-10-29 07:42:49,238 - kedro.runner.sequential_runner - INFO - Completed 3 out of 6 tasks
2019-10-29 07:42:49,238 - kedro.io.data_catalog - INFO - Loading data from `master_table` (CSVLocalDataSet)...
2019-10-29 07:42:52,026 - kedro.io.data_catalog - INFO - Loading data from `parameters` (MemoryDataSet)...
2019-10-29 07:42:52,027 - kedro.pipeline.node - INFO - Running node: split_data([master_table,parameters]) -> [X_test,X_train,y_test,y_train]
2019-10-29 07:42:52,765 - kedro.io.data_catalog - INFO - Saving data to `X_train` (MemoryDataSet)...
2019-10-29 07:42:52,829 - kedro.io.data_catalog - INFO - Saving data to `X_test` (MemoryDataSet)...
2019-10-29 07:42:52,852 - kedro.io.data_catalog - INFO - Saving data to `y_train` (MemoryDataSet)...
2019-10-29 07:42:52,854 - kedro.io.data_catalog - INFO - Saving data to `y_test` (MemoryDataSet)...
2019-10-29 07:42:52,925 - kedro.runner.sequential_runner - INFO - Completed 4 out of 6 tasks
2019-10-29 07:42:52,925 - kedro.io.data_catalog - INFO - Loading data from `X_train` (MemoryDataSet)...
2019-10-29 07:42:52,990 - kedro.io.data_catalog - INFO - Loading data from `y_train` (MemoryDataSet)...
2019-10-29 07:42:52,991 - kedro.pipeline.node - INFO - Running node: train_model([X_train,y_train]) -> [regressor]
2019-10-29 07:42:53,426 - kedro.io.data_catalog - INFO - Saving data to `regressor` (PickleLocalDataSet)...
2019-10-29 07:42:53,606 - kedro.runner.sequential_runner - INFO - Completed 5 out of 6 tasks
2019-10-29 07:42:53,606 - kedro.io.data_catalog - INFO - Loading data from `regressor` (PickleLocalDataSet)...
2019-10-29 07:42:53,607 - kedro.io.data_catalog - INFO - Loading data from `X_test` (MemoryDataSet)...
2019-10-29 07:42:53,624 - kedro.io.data_catalog - INFO - Loading data from `y_test` (MemoryDataSet)...
2019-10-29 07:42:53,625 - kedro.pipeline.node - INFO - Running node: evaluate_model([X_test,regressor,y_test]) -> None
2019-10-29 07:42:53,704 - kedro_tutorial.nodes.price_prediction - INFO - Model has a coefficient R^2 of 0.456.
2019-10-29 07:42:53,760 - kedro.runner.sequential_runner - INFO - Completed 6 out of 6 tasks
2019-10-29 07:42:53,761 - kedro.runner.sequential_runner - INFO - Pipeline execution completed successfully.


Steps to Reproduce
But if tags are placed in the pipeline, and the project is run "kedro run" or "kedro run --tag=ds_tag", I am getting an error as below:
    ds_pipeline = Pipeline(
        [
            node(
                split_data,
                ["master_table", "parameters"],
                ["X_train", "X_test", "y_train", "y_test"],
            ),
            node(train_model, ["X_train", "y_train"], "regressor"),
            node(evaluate_model, ["regressor", "X_test", "y_test"], None),
        ],
        tags=["ds_tag"],
    )

Expected Result
The project should run successfully or only pipeline with specified tag should run.
Actual Result
After using tags in the pipeline, the below error is occurring:
$ kedro run
2019-10-29 07:46:47,501 - root - INFO - ** Kedro project kedro-tutorial
Traceback (most recent call last):
  File "/home/ajit/miniconda3/envs/kedro/bin/kedro", line 10, in <module>
    sys.exit(main())
  File "/home/ajit/miniconda3/envs/kedro/lib/python3.7/site-packages/kedro/cli/cli.py", line 621, in main
    ("Project specific commands", project_groups),
  File "/home/ajit/miniconda3/envs/kedro/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/ajit/miniconda3/envs/kedro/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/ajit/miniconda3/envs/kedro/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/ajit/miniconda3/envs/kedro/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ajit/miniconda3/envs/kedro/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/ajit/ajit-code/zzztemp/kedro/kedro-site/02-tutorial/kedro-tutorial/kedro_cli.py", line 196, in run
    pipeline_name=pipeline,
  File "/home/ajit/miniconda3/envs/kedro/lib/python3.7/site-packages/kedro/context/context.py", line 413, in run
    pipeline = self._get_pipeline(name=pipeline_name)
  File "/home/ajit/miniconda3/envs/kedro/lib/python3.7/site-packages/kedro/context/context.py", line 154, in _get_pipeline
    pipelines = self._get_pipelines()
  File "/home/ajit/ajit-code/zzztemp/kedro/kedro-site/02-tutorial/kedro-tutorial/src/kedro_tutorial/run.py", line 51, in _get_pipelines
    return create_pipelines()
  File "/home/ajit/ajit-code/zzztemp/kedro/kedro-site/02-tutorial/kedro-tutorial/src/kedro_tutorial/pipeline.py", line 124, in create_pipelines
    tags=["ds_tag"],
TypeError: __init__() got an unexpected keyword argument 'tags'

Your Environment
Include as many relevant details about the environment in which you experienced the bug:

Kedro version used (pip show kedro or kedro -V): 0.15.3
Python version used (python -V): 3.7.3
Operating system and version: ubuntu 18.04.03 LTS

Duplicate issue #152: [KED-1170] `tag` argument in `node` constructor is incorrect in User Guide > Nodes > Tagging Nodes
Created: 2019-10-29T19:21:10Z
Description
In the Tagging Nodes section of the User Guide in the kedro docs, the following example is given:
node(func=add, inputs=["a", "b"], outputs="sum", name="adding_a_and_b", tag="node_tag")

However, if you look at the documentation for the class node, there is no tag argument in the constructor. The correct argument is tags and it only accepts a list. The corrected example should read:
node(func=add, inputs=["a", "b"], outputs="sum", name="adding_a_and_b", tags=["node_tag"])

Context
This error in the documentation makes it difficult to figure out how to use tags effectively with nodes. Overall, I think a consolidated explanation of tagging nodes and tagging pipelines would be helpful.
Your Environment
Include as many relevant details about the environment in which you experienced the bug:

Kedro version used (pip show kedro or kedro -V): 0.15.1 (also tested on 0.15.3)
Python version used (python -V): 3.6.9
Operating system and version: MacOS 10.14.6

Closing comment (2019-10-29T19:23:03Z): Perhaps related to #148.