internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: kubeflow/pipelines

Issue:
Upgrade cloudpickle to > 2.0.0
Hello,

One of the features of pipelines is Step Caching (https://www.kubeflow.org/docs/components/pipelines/caching/) to avoid running the costly computations again and again. 

The key for caching is:

```
message CacheKey {
  map<string, ArtifactNameList> inputArtifactNames = 1;
  map<string, Value> inputParameters = 2;
  map<string, RuntimeArtifact> outputArtifactsSpec = 3;
  map<string, string> outputParametersSpec=4;
  ContainerSpec containerSpec=5;
}
```

When using the option `use_code_pickling` from https://github.com/kubeflow/pipelines/blob/74c7773ca40decfd0d4ed40dc93a6af591bbc190/sdk/python/kfp/components/_python_op.py#L516

the pickle of the function gets embedded in the `ContainerSpec` (and hence becomes part of the key). 

So far, all good. 

However, the pickle is generated with `cloudpickle` which leads to non deterministic pickles every time you run the pipeline. As you can imagine, this makes caching feature useless because it will invalidate the cache every time it is run.

This non determinism was removed from `cloudpickle` with the following commit:
https://github.com/cloudpipe/cloudpickle/pull/428 and released as part of `2.0.0` release:
https://github.com/cloudpipe/cloudpickle/releases/tag/v2.0.0

Currently, `kfp` has bounded cloudpickle to less than v2.0.0 here:
https://github.com/kubeflow/pipelines/blob/74c7773ca40decfd0d4ed40dc93a6af591bbc190/sdk/python/setup.py#L37


Would it be possible to make a new `kfp` release with upgraded cloudpickle? Without this cloudpickle version, step caching is currently impossible to use (or at the mercy of dictionary insertion order of cloudpickle). 

Thanks!


Discussion:

Resolving patch:
diff --git a/sdk/python/setup.py b/sdk/python/setup.py
--- a/sdk/python/setup.py
+++ b/sdk/python/setup.py
@@ -34,7 +34,7 @@
     'google-api-python-client>=1.7.8,<2',
     'google-auth>=1.6.1,<2',
     'requests-toolbelt>=0.8.0,<1',
-    'cloudpickle>=1.3.0,<2',
+    'cloudpickle>=2.0.0,<3',
     # Update the upper version whenever a new major version of the
     # kfp-server-api package is released.
     # Update the lower version when kfp sdk depends on new apis/fields in


Test patch:
Repository: kubeflow/pipelines

Issue:
[sdk] superfluous isinstance check in local run
Hello,

I am trying to run a kubeflow pipeline locally, and am currently running into issue due to this check:

https://github.com/kubeflow/pipelines/blob/4abc4fd1874f7937a193d31dbbe650618c88ca95/sdk/python/kfp/_local_client.py#L475

My code (roughly) looks like this:

```py
@container_op
def upload(..) -> list:
   ...

@container_op
def download(path: String()):
   ...

with ParallelFor(upload.output) as path:
    download(path)
```

The code works on a remote k8 run, but fails locally when run with LocalClient due to the extra layer of serialization because of the above check.

It changes path from `gs://...` to `"gs://..."`. (Note the extra pair of quotations). 

What's the purpose of that check? Isn't that always going to succeed since everything is an instance of `object` in python anyway. 
[sdk] superfluous isinstance check in local run
Hello,

I am trying to run a kubeflow pipeline locally, and am currently running into issue due to this check:

https://github.com/kubeflow/pipelines/blob/4abc4fd1874f7937a193d31dbbe650618c88ca95/sdk/python/kfp/_local_client.py#L475

My code (roughly) looks like this:

```py
@container_op
def upload(..) -> list:
   ...

@container_op
def download(path: String()):
   ...

with ParallelFor(upload.output) as path:
    download(path)
```

The code works on a remote k8 run, but fails locally when run with LocalClient due to the extra layer of serialization because of the above check.

It changes path from `gs://...` to `"gs://..."`. (Note the extra pair of quotations). 

What's the purpose of that check? Isn't that always going to succeed since everything is an instance of `object` in python anyway. 


Discussion:
cc @lynnmatrix 
@feizerl Thanks for this issue, and you are right, the check will always be true. 
What i really want to check is whether the item is an instance of primitive type, if not, convert the item to string so that it can be used in command line.

cc @lynnmatrix 
@feizerl Thanks for this issue, and you are right, the check will always be true. 
What i really want to check is whether the item is an instance of primitive type, if not, convert the item to string so that it can be used in command line.


Resolving patch:
diff --git a/sdk/python/kfp/_local_client.py b/sdk/python/kfp/_local_client.py
--- a/sdk/python/kfp/_local_client.py
+++ b/sdk/python/kfp/_local_client.py
@@ -454,7 +454,7 @@ def _run_group(
                 with open(_altered_list_file, "r") as f:
                     _param_values = json.load(f)
                 for index, _param_value in enumerate(_param_values):
-                    if isinstance(_param_values, object):
+                    if isinstance(_param_values, (dict, list)):
                         _param_value = json.dumps(_param_value)
                     stack[_loop_args.pattern] = _param_value
                     loop_run_name = "{run_name}/{loop_index}".format(


Test patch:
Repository: kubeflow/pipelines

Issue:
[sdk] superfluous isinstance check in local run
Hello,

I am trying to run a kubeflow pipeline locally, and am currently running into issue due to this check:

https://github.com/kubeflow/pipelines/blob/4abc4fd1874f7937a193d31dbbe650618c88ca95/sdk/python/kfp/_local_client.py#L475

My code (roughly) looks like this:

```py
@container_op
def upload(..) -> list:
   ...

@container_op
def download(path: String()):
   ...

with ParallelFor(upload.output) as path:
    download(path)
```

The code works on a remote k8 run, but fails locally when run with LocalClient due to the extra layer of serialization because of the above check.

It changes path from `gs://...` to `"gs://..."`. (Note the extra pair of quotations). 

What's the purpose of that check? Isn't that always going to succeed since everything is an instance of `object` in python anyway. 


Discussion:
cc @lynnmatrix 
@feizerl Thanks for this issue, and you are right, the check will always be true. 
What i really want to check is whether the item is an instance of primitive type, if not, convert the item to string so that it can be used in command line.


Resolving patch:
diff --git a/sdk/python/kfp/_local_client.py b/sdk/python/kfp/_local_client.py
--- a/sdk/python/kfp/_local_client.py
+++ b/sdk/python/kfp/_local_client.py
@@ -472,7 +472,7 @@ def _run_group(
                 with open(_altered_list_file, "r") as f:
                     _param_values = json.load(f)
                 for index, _param_value in enumerate(_param_values):
-                    if isinstance(_param_values, object):
+                    if isinstance(_param_values, (dict, list)):
                         _param_value = json.dumps(_param_value)
                     stack[_loop_args.pattern] = _param_value
                     loop_run_name = "{run_name}/{loop_index}".format(


Test patch:
Repository: kubeflow/pipelines

Issue:
[sdk] Update absl-py requirement to accommodate future TF versions
### Environment

*  KFP version:
N/A

*  KFP SDK version: 
1.8.10 (latest)

*  All dependencies version:
N/A


### Steps to reproduce

I believe that Tensorflow is frequently used with KFP and TF and should be able to be installed together. TF updated their absl-py requirement to `>=1.0.0` recently. [setup.py in TF](https://github.com/tensorflow/tensorflow/blob/c93e4757491a4abb7aa990e18f2d321e4a935a03/tensorflow/tools/pip_package/setup.py#L77). The next TF version(probably, 2.8.0) cannot be installed with the recent version kfp, because kfp is pinned to `<=0.11`. TF-nightly was already hit by this issue.

```
$ pip install tf-nightly==2.9.0.dev20220116 kfp==1.8.10
...
ERROR: Cannot install kfp==1.8.10 and tf-nightly==2.9.0.dev20220116 because these package versions have conflicting dependencies.

The conflict is caused by:
    tf-nightly 2.9.0.dev20220116 depends on absl-py>=1.0.0
    kfp 1.8.10 depends on absl-py<=0.11 and >=0.9

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies
```

### Expected result

I think that the version range of absl should become `absl-py>=0.9,<2.0.0` if possible. Absl-py doesn't have [any breaking change recently ([Change log](https://github.com/abseil/abseil-py/blob/main/CHANGELOG.md)) and it should be safe to upgrade. We are doing [a similar upgrade in TFX](https://github.com/tensorflow/tfx/pull/4573).

It would be great if you can do a patch release with the fix if possible. Because we are already seeing failures in integration tests that utilizing TF-nightly and kfp.

### Materials and Reference
N/A

---

<!-- Don't delete message below to encourage users to support your issue! -->
Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.



Discussion:

Resolving patch:
diff --git a/sdk/python/setup.py b/sdk/python/setup.py
--- a/sdk/python/setup.py
+++ b/sdk/python/setup.py
@@ -23,7 +23,7 @@
 # NOTICE, after any updates to the following, ./requirements.in should be updated
 # accordingly.
 REQUIRES = [
-    'absl-py>=0.9,<=0.11',
+    'absl-py>=0.9,<2',
     'PyYAML>=5.3,<6',
     # `Blob.from_string` was introduced in google-cloud-storage 1.20.0
     # https://github.com/googleapis/python-storage/blob/master/CHANGELOG.md#1200


Test patch:
Repository: kubeflow/pipelines

Issue:
[bug] TFJob launcher pipeline task fails when `delete_finished_tfjob` flag is `True`
https://github.com/kubeflow/pipelines/blob/127dab4d4671849d596c455f4619ea807d22f6ea/components/kubeflow/launcher/component.yaml#L19

When `delete_finished_tfjob` flag is `True`, TFJob launcher task fails with error below.

```
Traceback (most recent call last):
  File "/ml/launch_tfjob.py", line 136, in <module>
    main()
  File "/ml/launch_tfjob.py", line 133, in main
    tfjob.delete(args.name, args.namespace)
  File "/ml/launch_crd.py", line 115, in delete
    body)
TypeError: delete_namespaced_custom_object() takes exactly 6 arguments (7 given)
```

I think it's some kind of kubernetes client SDK version issue in `nikenano/launchernew:latest` container image.


Discussion:
I found this PR in `kubeflow/training-operator` repo about similar problem.
https://github.com/kubeflow/training-operator/pull/1281

I think those changes above should be applied here and also to `nikenano/launchernew` container image.
https://github.com/kubeflow/pipelines/blob/127dab4d4671849d596c455f4619ea807d22f6ea/components/kubeflow/common/launch_crd.py#L117-L123

Resolving patch:
diff --git a/components/kubeflow/common/launch_crd.py b/components/kubeflow/common/launch_crd.py
--- a/components/kubeflow/common/launch_crd.py
+++ b/components/kubeflow/common/launch_crd.py
@@ -115,12 +115,12 @@ def delete(self, name, namespace):
       logger.info("Deleteing %s/%s %s in namespace %s.",
         self.group, self.plural, name, namespace)
       api_response = self.client.delete_namespaced_custom_object(
-        self.group,
-        self.version,
-        namespace,
-        self.plural,
-        name,
-        body)
+        group=self.group,
+        version=self.version,
+        namespace=namespace,
+        plural=self.plural,
+        name=name,
+        body=body)
       logger.info("Deleted %s/%s %s in namespace %s.",
         self.group, self.plural, name, namespace)
       return api_response


Test patch:
Repository: kubeflow/pipelines

Issue:
[SDK] AttributeError: 'ComponentStore' object has no attribute 'uri_search_template'
Hi, I am trying to load some prebuit gcp kubeflow components using `kfp.components.ComponentStore`. However I am getting this error:

```
line 180, in _load_component_spec_in_component_ref
    if self.uri_search_template:
AttributeError: 'ComponentStore' object has no attribute 'uri_search_template'
```

when at this line of code:
`mlengine_train_op = component_store.load_component('ml_engine/train')`

### Environment

*  How did you deploy Kubeflow Pipelines (KFP)?
I am running this locally.

*  KFP version: 
```
kfp                             1.8.10
kfp-pipeline-spec               0.1.13
kfp-server-api                  1.7.1
```


### Steps to reproduce
```
import kfp
from kfp.components import func_to_container_op

COMPONENT_URL_SEARCH_PREFIX = "https://raw.githubusercontent.com/kubeflow/pipelines/1.7.1/components/gcp/"

component_store = kfp.components.ComponentStore(
    local_search_paths=None, url_search_prefixes=[COMPONENT_URL_SEARCH_PREFIX])

mlengine_train_op = component_store.load_component('ml_engine/train')
mlengine_deploy_op = component_store.load_component('ml_engine/deploy')
```

---

<!-- Don't delete message below to encourage users to support your issue! -->
Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.



Discussion:
I solved the issue. It looks like there is a bug if you don't set up `uri_search_template` when creating the ComponentStore. 
I solved it by providing it as parameter, it doesn't really matter what you pass to the constructor really, but it is required because in the code there is the following
```
if self.url_search_prefixes: 
```
but if you don't pass something that attribute doesn't get created in the constructor.
I fixed it like this:
```
component_store = kfp.components.ComponentStore(
    local_search_paths=["local_search_path"], url_search_prefixes=[COMPONENT_URL_SEARCH_PREFIX]), uri_search_template="{name}/"
```

I hope it helps.
I would suggest addressing this by either make the error more clear or a better check on `uri_search_template` attribute.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Resolving patch:
diff --git a/sdk/python/kfp/components/_component_store.py b/sdk/python/kfp/components/_component_store.py
--- a/sdk/python/kfp/components/_component_store.py
+++ b/sdk/python/kfp/components/_component_store.py
@@ -45,8 +45,7 @@ def __init__(self,
             uri_search_template: A URI template for components, which may include {name}, {digest} and {tag} variables.
         """
         self.local_search_paths = local_search_paths or ['.']
-        if uri_search_template:
-            self.uri_search_template = URITemplate(uri_search_template)
+        self.uri_search_template = URITemplate(uri_search_template) if uri_search_template else None
         self.url_search_prefixes = url_search_prefixes or []
         self._auth = auth
 


Test patch:
Repository: kubeflow/pipelines

Issue:
[sdk] Can't compile component that uses PipelineTaskFinalStatus: Enum value for ParameterTypeEnum must be an int, but got <class 'NoneType'> None.
### Environment

*  KFP version: 2.0.0b8
<!-- For more information, see an overview of KFP installation options: https://www.kubeflow.org/docs/pipelines/installation/overview/. -->
*  KFP SDK version: 2.0.0b8
<!-- Specify the version of Kubeflow Pipelines that you are using. The version number appears in the left side navigation of user interface.
To find the version number, See version number shows on bottom of KFP UI left sidenav. -->
*  All dependencies version: 2.0.0b8
<!-- Specify the output of the following shell command: $pip list | grep kfp -->


### Steps to reproduce

When compiling a component that uses `PipelineTaskFinalStatus`, only the given parameter type is special cased during type checking, leading to an exception:

```python
from kfp.dsl import component, PipelineTaskFinalStatus
from kfp.compiler import Compiler

@component
def test(status: PipelineTaskFinalStatus):
    pass

Compiler().compile(test, package_path='test.yaml')  # TypeError: Enum value for ParameterTypeEnum must be an int, but got <class 'NoneType'> None.
```


### Expected result

It should be possible to compile components that use `PipelineTaskFinalStatus` to yaml.


### Materials and Reference

<!-- Help us debug this issue by providing resources such as: sample code, background context, or links to references. -->

---

<!-- Don't delete message below to encourage users to support your issue! -->
Impacted by this bug? Give it a 👍. 


Discussion:

Resolving patch:
diff --git a/sdk/python/kfp/components/types/type_utils.py b/sdk/python/kfp/components/types/type_utils.py
--- a/sdk/python/kfp/components/types/type_utils.py
+++ b/sdk/python/kfp/components/types/type_utils.py
@@ -131,7 +131,9 @@ def get_parameter_type(
     Raises:
       AttributeError: if type_name is not a string type.
     """
-
+    # Special handling for PipelineTaskFinalStatus, treat it as Dict type.
+    if is_task_final_status_type(param_type):
+        param_type = 'dict'
     if type(param_type) == type:
         type_name = param_type.__name__
     elif isinstance(param_type, dict):
@@ -194,10 +196,6 @@ def verify_type_compatibility(
         InconsistentTypeException if types are incompatible and TYPE_CHECK==True.
     """
 
-    # Special handling for PipelineTaskFinalStatus, treat it as Dict type.
-    if is_task_final_status_type(given_type):
-        given_type = 'Dict'
-
     types_are_compatible = False
     is_parameter = is_parameter_type(str(given_type))
 


Test patch:
Repository: kubeflow/pipelines

Issue:
[sdk] client wait_for_run_completion function has an extra wait iteration
### Environment

*  KFP version:
Version 2.0.0-beta.2
*  KFP SDK version: 
KFP SDK v2.0.0b16
*  All dependencies version:
<!-- Specify the output of the following shell command: $pip list | grep kfp -->


### Steps to reproduce
The state of the pipeline is updated and still have to sleep for 1 iteration even though the state is completed.

https://github.com/kubeflow/pipelines/blob/96947e6fb971178e5a67b407b0d0ecb7192db9d4/sdk/python/kfp/client/client.py#L1387-L1393

### Expected result

The wait_for_run_completion should check the state right after retrieving it instead of sleep and validate in the next iteration.


### Materials and Reference

<!-- Help us debug this issue by providing resources such as: sample code, background context, or links to references. -->

---

<!-- Don't delete message below to encourage users to support your issue! -->
Impacted by this bug? Give it a 👍. 


Discussion:

Resolving patch:
diff --git a/sdk/python/kfp/client/client.py b/sdk/python/kfp/client/client.py
--- a/sdk/python/kfp/client/client.py
+++ b/sdk/python/kfp/client/client.py
@@ -1370,8 +1370,8 @@ def wait_for_run_completion(
         if isinstance(timeout, datetime.timedelta):
             timeout = timeout.total_seconds()
         is_valid_token = False
-        while (state is None or state.lower()
-               not in ['succeeded', 'failed', 'skipped', 'error']):
+        finish_states = ['succeeded', 'failed', 'skipped', 'error']
+        while True:
             try:
                 get_run_response = self._run_api.get_run(run_id=run_id)
                 is_valid_token = True
@@ -1390,8 +1390,9 @@ def wait_for_run_completion(
             logging.info('Waiting for the job to complete...')
             if elapsed_time > timeout:
                 raise TimeoutError('Run timeout')
+            if state is not None and state.lower() in finish_states:
+                return get_run_response
             time.sleep(sleep_duration)
-        return get_run_response
 
     def upload_pipeline(
         self,


Test patch:
Repository: kubeflow/pipelines

Issue:
chore(sdk): Improve message printed to users when IPython or SHH is used for OOB authentication
              nit: This message should be updated as well to include the notebook case.

_Originally posted by @chensun in https://github.com/kubeflow/pipelines/pull/10094#discussion_r1359146375_
            


Discussion:

Resolving patch:
diff --git a/sdk/python/kfp/client/auth.py b/sdk/python/kfp/client/auth.py
--- a/sdk/python/kfp/client/auth.py
+++ b/sdk/python/kfp/client/auth.py
@@ -264,9 +264,10 @@ def get_auth_code(client_id: str) -> Tuple[str, str]:
     if ('SSH_CONNECTION' in os.environ) or ('SSH_CLIENT'
                                             in os.environ) or is_ipython():
         try:
-            print((
-                'SSH connection detected. Please follow the instructions below.'
-                ' Otherwise, press CTRL+C if you are not connected via SSH.'))
+            print(('SSH connection or IPython shell detected. Please follow the'
+                   ' instructions below. Otherwise, press CTRL+C if you are not'
+                   ' connected via SSH and not using IPython (e.g. Jupyter'
+                   ' Notebook).'))
             authorization_response = get_auth_response_ssh(host, port, auth_url)
         except KeyboardInterrupt:
             authorization_response = None


Test patch:
Repository: Rapptz/discord.py

Issue:
Error in edit_profile for user accounts
`NameError: name 'data' is not defined`
data isn't defined anymore after the HTTP update:
https://github.com/Rapptz/discord.py/blob/async/discord/client.py#L1503
https://github.com/Rapptz/discord.py/commit/1fba1b06faca31d07c9296b2badabfe22f173001#diff-5bab608907a27a2180166115e6299cefL1547



Discussion:

Resolving patch:
diff --git a/discord/client.py b/discord/client.py
--- a/discord/client.py
+++ b/discord/client.py
@@ -1498,7 +1498,7 @@ def edit_profile(self, password=None, **fields):
             if 'new_password' in fields:
                 args['new_password'] = fields['new_password']
 
-        yield from self.http.edit_profile(**args)
+        data = yield from self.http.edit_profile(**args)
         if not_bot_account:
             self.email = data['email']
             if 'token' in data:


Test patch:
Repository: Rapptz/discord.py

Issue:
Cannot connect to voice channels
Running `await voice_channel.connect()` raises
`AttributeError: 'LP_EncoderStruct' object has no attribute 'value'`

Relevant portions of the traceback ([and the line itself](https://github.com/Rapptz/discord.py/blob/rewrite/discord/opus.py#L52)):
```
  File ".../lib/python3.6/site-packages/discord/abc.py", line 985, in connect
    voice = VoiceClient(state=state, timeout=timeout, channel=self)
  File ".../lib/python3.6/site-packages/discord/voice_client.py", line 109, in __init__
    self.encoder = opus.Encoder()
  File ".../lib/python3.6/site-packages/discord/opus.py", line 225, in __init__
    self._state = self._create_state()
  File ".../lib/python3.6/site-packages/discord/opus.py", line 239, in _create_state
    return _lib.opus_encoder_create(self.SAMPLING_RATE, self.CHANNELS, self.application, ctypes.byref(ret))
  File ".../lib/python3.6/site-packages/discord/opus.py", line 52, in _err_ne
    if result.value != 0:
AttributeError: 'LP_EncoderStruct' object has no attribute 'value'
```
I have opus 1.2.1 installed on 64-bit Linux, and it is loaded according to `discord.opus.is_loaded()`.

Any clue as to what might be the issue?


Discussion:

Resolving patch:
diff --git a/discord/opus.py b/discord/opus.py
--- a/discord/opus.py
+++ b/discord/opus.py
@@ -49,9 +49,10 @@ def _err_lt(result, func, args):
     return result
 
 def _err_ne(result, func, args):
-    if result.value != 0:
+    ret = args[-1]._obj
+    if ret.value != 0:
         log.info('error has happened in {0.__name__}'.format(func))
-        raise OpusError(result.value)
+        raise OpusError(ret.value)
     return result
 
 # A list of exported functions.


Test patch: