Repository: kubeflow/pipelines

Issue:
Constructing dsl.ContainerOp instance directly in user code doesn't always raise the warning.
### What steps did you take:
Compile the following pipeline
```
import kfp


def _sample_component_op(
    name: str, run_id: str, sample_metric_value: float
) -> kfp.dsl.ContainerOp:
    arguments = [run_id, sample_metric_value]
    return kfp.dsl.ContainerOp(
        name=name,
        image='library/hello-world:latest',
        arguments=arguments,
    )


@kfp.dsl.pipeline(name='minimul')
def sample_pipeline(
    run_id: str = kfp.dsl.RUN_ID_PLACEHOLDER,
    sample_metric_value: float = 0.5,
):
    """Execute main procedure of the pipeline."""
    _sample_component_op("component", run_id, sample_metric_value)
```

### What happened:
No warning raised on using `ContainerOp` directly.

### What did you expect to happen:
Seeing the following warning:
```
/usr/local/lib/python3.7/dist-packages/kfp/dsl/_container_op.py:1039: FutureWarning: Please create reusable components instead of constructing ContainerOp instances directly. Reusable components are shareable, portable and have compatibility and support guarantees. Please see the documentation: https://www.kubeflow.org/docs/pipelines/sdk/component-development/#writing-your-component-definition-file The components can be created manually (or, in case of python, using kfp.components.create_component_from_func or func_to_container_op) and then loaded using kfp.components.load_component_from_file, load_component_from_uri or load_component_from_text: https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.components.html#kfp.components.load_component_from_file
  category=FutureWarning,
```

KFP SDK version: 1.4.0


### Anything else you would like to add:
For comparison, compiling the following example raises the warning as expected.
```
import kfp

@kfp.dsl.pipeline(name='hello-world-pipeline')
def sample_pipeline():
  kfp.dsl.ContainerOp(
      name='hello-world',
      image='library/hello-world:latest',
      arguments=[],
  )
```

/kind bug
/area sdk




Discussion:
/assign @chensun 
/cc @Ark-kun 
Hmm.
Interesting. We had this issue before and @munagekar fixed it by changing the warning type from `DeprecationWarning` to `FutureWarning`. https://github.com/kubeflow/pipelines/pull/4658

It looks like that warning is still not show to the users even when we see it in tests.
This is really weird. Unlike `DeprecationWarning`, `FutureWarning` should always be shown: https://docs.python.org/3/library/warnings.html#default-warning-filter

I can reproduce the behavior. I also see that when the warning is shown it's only shown once (in a Jupyter session).
> I also see that when the warning is shown it's only shown once (in a Jupyter session).

This is the usual behavior with python warnings. Warning in a loop get called once only.

> This is really weird. Unlike DeprecationWarning, FutureWarning should always be shown

https://docs.python.org/3/library/warnings.html#default-warning-filter
This is correct, however this is not a python issue.

I have investigated this further, the reason the compiler doesn't produce the warning is because the code in the first case does not reach the warning. The attached screenshot explains it further.

https://github.com/kubeflow/pipelines/blob/a12e88d1da57b897a3a25b5c44540fc7d3c9a40e/sdk/python/kfp/dsl/_container_op.py#L1104-L1105

The core issue is that `__eq__` in `kfp.dsl._pipeline_param.PipelineParam` is incorrectly returning NamedTuple. `__eq__ `should return either True or False. A named tuple is assumed to be True.
https://github.com/kubeflow/pipelines/blob/67afca4938c35e6b8be29e61df3883af11554220/sdk/python/kfp/dsl/_pipeline_param.py#L227-L228

![Screen Shot 2021-03-13 at 2 11 15](https://user-images.githubusercontent.com/10258799/110974425-db784e00-83a1-11eb-9201-7e9420070c61.png)




Resolving patch:
diff --git a/sdk/python/kfp/dsl/_container_op.py b/sdk/python/kfp/dsl/_container_op.py
--- a/sdk/python/kfp/dsl/_container_op.py
+++ b/sdk/python/kfp/dsl/_container_op.py
@@ -15,7 +15,7 @@
 import inspect
 import re
 import warnings
-from typing import Any, Dict, List, TypeVar, Union, Callable, Optional, Sequence
+from typing import Any, Dict, Iterable, List, TypeVar, Union, Callable, Optional, Sequence
 
 from kubernetes.client import V1Toleration, V1Affinity
 from kubernetes.client.models import (V1Container, V1EnvVar, V1EnvFromSource,
@@ -32,6 +32,9 @@
 T = TypeVar('T')
 # type alias: either a string or a list of string
 StringOrStringList = Union[str, List[str]]
+ContainerOpArgument = Union[str, int, float, bool,
+                            _pipeline_param.PipelineParam]
+ArgumentOrArguments = Union[ContainerOpArgument, List]
 
 ALLOWED_RETRY_POLICIES = (
     'Always',
@@ -1086,7 +1089,7 @@ def __init__(
       name: str,
       image: str,
       command: Optional[StringOrStringList] = None,
-      arguments: Optional[StringOrStringList] = None,
+      arguments: Optional[ArgumentOrArguments] = None,
       init_containers: Optional[List[UserContainer]] = None,
       sidecars: Optional[List[Sidecar]] = None,
       container_kwargs: Optional[Dict] = None,
@@ -1101,19 +1104,6 @@ def __init__(
                      sidecars=sidecars,
                      is_exit_handler=is_exit_handler)
 
-    if (not ContainerOp._DISABLE_REUSABLE_COMPONENT_WARNING) and (
-        '--component_launcher_class_path' not in (arguments or [])):
-      # The warning is suppressed for pipelines created using the TFX SDK.
-      warnings.warn(
-          'Please create reusable components instead of constructing ContainerOp instances directly.'
-          ' Reusable components are shareable, portable and have compatibility and support guarantees.'
-          ' Please see the documentation: https://www.kubeflow.org/docs/pipelines/sdk/component-development/#writing-your-component-definition-file'
-          ' The components can be created manually (or, in case of python, using kfp.components.create_component_from_func or func_to_container_op)'
-          ' and then loaded using kfp.components.load_component_from_file, load_component_from_uri or load_component_from_text: '
-          'https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.components.html#kfp.components.load_component_from_file',
-          category=FutureWarning,
-      )
-
     self.attrs_with_pipelineparams = BaseOp.attrs_with_pipelineparams + [
         '_container', 'artifact_arguments', '_parameter_arguments'
     ]  #Copying the BaseOp class variable!
@@ -1147,6 +1137,19 @@ def resolve_artifact_argument(artarg):
     command = as_string_list(command)
     arguments = as_string_list(arguments)
 
+    if (not ContainerOp._DISABLE_REUSABLE_COMPONENT_WARNING) and (
+        '--component_launcher_class_path' not in (arguments or [])):
+      # The warning is suppressed for pipelines created using the TFX SDK.
+      warnings.warn(
+          'Please create reusable components instead of constructing ContainerOp instances directly.'
+          ' Reusable components are shareable, portable and have compatibility and support guarantees.'
+          ' Please see the documentation: https://www.kubeflow.org/docs/pipelines/sdk/component-development/#writing-your-component-definition-file'
+          ' The components can be created manually (or, in case of python, using kfp.components.create_component_from_func or func_to_container_op)'
+          ' and then loaded using kfp.components.load_component_from_file, load_component_from_uri or load_component_from_text: '
+          'https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.components.html#kfp.components.load_component_from_file',
+          category=FutureWarning,
+      )
+
     # `container` prop in `io.argoproj.workflow.v1alpha1.Template`
     container_kwargs = container_kwargs or {}
     self._container = Container(image=image,


Test patch:
diff --git a/sdk/python/tests/dsl/component_bridge_tests.py b/sdk/python/tests/dsl/component_bridge_tests.py
--- a/sdk/python/tests/dsl/component_bridge_tests.py
+++ b/sdk/python/tests/dsl/component_bridge_tests.py
@@ -20,7 +20,7 @@
 from pathlib import Path
 from kfp.components import load_component_from_text, create_component_from_func
 from kfp.dsl.types import InconsistentTypeException
-
+from kfp.dsl import PipelineParam
 
 class TestComponentBridge(unittest.TestCase):
     # Alternatively, we could use kfp.dsl.Pipleine().__enter__ and __exit__
@@ -244,6 +244,10 @@ def test_reusable_component_warnings(self):
         with self.assertWarnsRegex(FutureWarning, expected_regex='reusable'):
             kfp.dsl.ContainerOp(name='name', image='image')
 
+        with self.assertWarnsRegex(FutureWarning, expected_regex='reusable'):
+            kfp.dsl.ContainerOp(name='name', image='image', arguments=[PipelineParam('param1'), PipelineParam('param2')])
+
+
     def test_prevent_passing_container_op_as_argument(self):
         component_text = textwrap.dedent('''\
             inputs: