internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: kedro-org/kedro

Issue:
Make Kedro instantiate datasets from `kedro-datasets` with higher priority than `kedro.extras.datasets`
https://github.com/kedro-org/kedro/blob/1b1558952c059eea5636d9ccf9a883f9cf4ef643/kedro/io/core.py#L346


Discussion:

Resolving patch:
diff --git a/kedro/io/core.py b/kedro/io/core.py
--- a/kedro/io/core.py
+++ b/kedro/io/core.py
@@ -347,7 +347,7 @@ class Version(namedtuple("Version", ["load", "save"])):
     "intermediate data sets where possible to avoid this warning."
 )
 
-_DEFAULT_PACKAGES = ["kedro.io.", "kedro.extras.datasets.", ""]
+_DEFAULT_PACKAGES = ["kedro.io.", "kedro_datasets.", "kedro.extras.datasets.", ""]
 
 
 def parse_dataset_definition(


Test patch:
diff --git a/tests/io/test_data_catalog.py b/tests/io/test_data_catalog.py
--- a/tests/io/test_data_catalog.py
+++ b/tests/io/test_data_catalog.py
@@ -19,7 +19,13 @@
     LambdaDataSet,
     MemoryDataSet,
 )
-from kedro.io.core import VERSION_FORMAT, Version, generate_timestamp
+from kedro.io.core import (
+    _DEFAULT_PACKAGES,
+    VERSION_FORMAT,
+    Version,
+    generate_timestamp,
+    parse_dataset_definition,
+)
 
 
 @pytest.fixture
@@ -373,6 +379,20 @@ def test_config_relative_import(self, sane_config):
         with pytest.raises(DataSetError, match=re.escape(pattern)):
             DataCatalog.from_config(**sane_config)
 
+    def test_config_import_kedro_datasets(self, sane_config, mocker):
+        """Test kedro.extras.datasets default path to the dataset class"""
+        # Spy _load_obj because kedro_datasets is not installed and we can't import it.
+
+        import kedro.io.core  # pylint: disable=import-outside-toplevel
+
+        spy = mocker.spy(kedro.io.core, "_load_obj")
+        parse_dataset_definition(sane_config["catalog"]["boats"])
+        for prefix, call_args in zip(_DEFAULT_PACKAGES, spy.call_args_list):
+            # In Python 3.7 call_args.args is not available thus we access the call
+            # arguments with less meaningful index.
+            # The 1st index returns a tuple, the 2nd index return the name of module.
+            assert call_args[0][0] == f"{prefix}pandas.CSVDataSet"
+
     def test_config_import_extras(self, sane_config):
         """Test kedro.extras.datasets default path to the dataset class"""
         sane_config["catalog"]["boats"]["type"] = "pandas.CSVDataSet"
Repository: kedro-org/kedro

Issue:
Document how to read and write a multi-sheet excel
## Introduction

>A high-level, short overview of the problem(s) you are designing a solution for.

We have the implementation, but it is not easy to point to a documented example:

![image](https://user-images.githubusercontent.com/35801847/178237457-9152ffa0-fbb1-48ca-bf70-d5e2c89323ed.png)

## Background

> Provide the reader with the context surrounding the problem(s) you are trying to solve.

This is a common feature and something Kedro can already do.



Discussion:
Add the example here: https://kedro.readthedocs.io/en/stable/data/data_catalog.html#use-the-data-catalog-with-the-yaml-api

Resolving patch:
diff --git a/kedro/extras/datasets/pandas/excel_dataset.py b/kedro/extras/datasets/pandas/excel_dataset.py
--- a/kedro/extras/datasets/pandas/excel_dataset.py
+++ b/kedro/extras/datasets/pandas/excel_dataset.py
@@ -65,6 +65,36 @@ class ExcelDataSet(
         >>> reloaded = data_set.load()
         >>> assert data.equals(reloaded)
 
+    Note: To save a multi-sheet excel file, no special ``save_args`` are required.
+    Instead, return a dictionary of ``Dict[str, pd.DataFrame]`` where the string
+    keys are your sheet names.
+
+    Example adding a catalog entry for multi-sheet excel file with the ``YAML API``:
+
+    .. code-block:: yaml
+
+        >>> trains:
+        >>>   type: pandas.ExcelDataSet
+        >>>   filepath: data/02_intermediate/company/trains.xlsx
+        >>>   load_args:
+        >>>     sheet_name: [Sheet1, Sheet2, Sheet3]
+
+    Example multi-sheet excel file using Python API:
+    ::
+
+        >>> from kedro.extras.datasets.pandas import ExcelDataSet
+        >>> import pandas as pd
+        >>>
+        >>> dataframe = pd.DataFrame({'col1': [1, 2], 'col2': [4, 5],
+        >>>                      'col3': [5, 6]})
+        >>> another_dataframe = pd.DataFrame({"x": [10, 20], "y": ["hello", "world"]})
+        >>> multiframe = {"Sheet1": dataframe, "Sheet2": another_dataframe}
+        >>> data_set = ExcelDataSet(filepath="test.xlsx", load_args = {"sheet_name": None})
+        >>> data_set.save(multiframe)
+        >>> reloaded = data_set.load()
+        >>> assert multiframe["Sheet1"].equals(reloaded["Sheet1"])
+        >>> assert multiframe["Sheet2"].equals(reloaded["Sheet2"])
+
     """
 
     DEFAULT_LOAD_ARGS = {"engine": "openpyxl"}


Test patch:
Repository: kedro-org/kedro

Issue:
Fix case in docs
As mentioned in https://github.com/kedro-org/kedro/pull/1760#pullrequestreview-1069581386_

Change `excel`(lowercase) to `Excel`(uppercase).


Discussion:

Resolving patch:
diff --git a/kedro/extras/datasets/pandas/excel_dataset.py b/kedro/extras/datasets/pandas/excel_dataset.py
--- a/kedro/extras/datasets/pandas/excel_dataset.py
+++ b/kedro/extras/datasets/pandas/excel_dataset.py
@@ -65,11 +65,11 @@ class ExcelDataSet(
         >>> reloaded = data_set.load()
         >>> assert data.equals(reloaded)
 
-    Note: To save a multi-sheet excel file, no special ``save_args`` are required.
+    Note: To save a multi-sheet Excel file, no special ``save_args`` are required.
     Instead, return a dictionary of ``Dict[str, pd.DataFrame]`` where the string
     keys are your sheet names.
 
-    Example adding a catalog entry for multi-sheet excel file with the ``YAML API``:
+    Example adding a catalog entry for multi-sheet Excel file with the ``YAML API``:
 
     .. code-block:: yaml
 
@@ -79,7 +79,7 @@ class ExcelDataSet(
         >>>   load_args:
         >>>     sheet_name: [Sheet1, Sheet2, Sheet3]
 
-    Example multi-sheet excel file using Python API:
+    Example multi-sheet Excel file using Python API:
     ::
 
         >>> from kedro.extras.datasets.pandas import ExcelDataSet
@@ -119,7 +119,7 @@ def __init__(
                 `s3://`. If prefix is not provided, `file` protocol (local filesystem) will be used.
                 The prefix should be any protocol supported by ``fsspec``.
                 Note: `http(s)` doesn't support versioning.
-            engine: The engine used to write to excel files. The default
+            engine: The engine used to write to Excel files. The default
                 engine is 'openpyxl'.
             load_args: Pandas options for loading Excel files.
                 Here you can find all available arguments:


Test patch:
Repository: kedro-org/kedro

Issue:
tracking.JSONDataSet documentation and docstring are not accurate
Here is the source code for `tracking.JSONDataSet`: https://kedro.readthedocs.io/en/latest/_modules/kedro/extras/datasets/tracking/json_dataset.html#JSONDataSet

The `_load` method has been overriden to raise a `DataSetError` when called.

The documentation does not reflect that well. What has to be fixed:

- The `tracking.JSONDataSet` documentation and API ref should mention that this dataset cannot be loaded (see https://kedro.readthedocs.io/en/latest/kedro.extras.datasets.tracking.JSONDataSet.html)
- The `tracking.JSONDataSet` class docstring should not call `data_set.load()` (this code snippet that was probably copied from the `json.JSONDataSet`)


Discussion:
Thanks for flagging this @clotildeguinard ! We'd be happy accept a PR for this if you have time 🙂, but will add it to our backlog as well. 

Resolving patch:
diff --git a/kedro/extras/datasets/tracking/json_dataset.py b/kedro/extras/datasets/tracking/json_dataset.py
--- a/kedro/extras/datasets/tracking/json_dataset.py
+++ b/kedro/extras/datasets/tracking/json_dataset.py
@@ -12,7 +12,7 @@ class JSONDataSet(JDS):
     """``JSONDataSet`` saves data to a JSON file using an underlying
     filesystem (e.g.: local, S3, GCS). It uses native json to handle the JSON file.
     The ``JSONDataSet`` is part of Kedro Experiment Tracking.
-    The dataset is versioned by default.
+    The dataset is write-only and it is versioned by default.
 
         Example:
         ::
@@ -24,8 +24,6 @@ class JSONDataSet(JDS):
         >>> # data_set = JSONDataSet(filepath="gcs://bucket/test.json")
         >>> data_set = JSONDataSet(filepath="test.json")
         >>> data_set.save(data)
-        >>> reloaded = data_set.load()
-        >>> assert data == reloaded
 
     """
 
diff --git a/kedro/extras/datasets/tracking/metrics_dataset.py b/kedro/extras/datasets/tracking/metrics_dataset.py
--- a/kedro/extras/datasets/tracking/metrics_dataset.py
+++ b/kedro/extras/datasets/tracking/metrics_dataset.py
@@ -12,9 +12,9 @@
 
 class MetricsDataSet(JSONDataSet):
     """``MetricsDataSet`` saves data to a JSON file using an underlying
-    filesystem (e.g.: local, S3, GCS). It uses native json to handle the JSON file.
-    The ``MetricsDataSet`` is part of Kedro Experiment Tracking. The dataset is versioned by default
-    and only takes metrics of numeric values.
+    filesystem (e.g.: local, S3, GCS). It uses native json to handle the JSON file. The
+    ``MetricsDataSet`` is part of Kedro Experiment Tracking. The dataset is write-only,
+    it is versioned by default and only takes metrics of numeric values.
 
         Example:
         ::
@@ -26,8 +26,6 @@ class MetricsDataSet(JSONDataSet):
         >>> # data_set = MetricsDataSet(filepath="gcs://bucket/test.json")
         >>> data_set = MetricsDataSet(filepath="test.json")
         >>> data_set.save(data)
-        >>> reloaded = data_set.load()
-        >>> assert data == reloaded
 
     """
 


Test patch:
Repository: kedro-org/kedro

Issue:
pickle.PickleDataSet docstring examples are incorrect
## Description
Kind of a small issue but the "advanced" example in the [pickle.PickleDataSet API docs](https://kedro.readthedocs.io/en/stable/kedro.extras.datasets.pickle.PickleDataSet.html) is wrong.
`compression` is not a valid [`joblib.dump`](https://joblib.readthedocs.io/en/latest/generated/joblib.dump.html) parameter (it should simply be `compress`) and [`joblib.load`](https://joblib.readthedocs.io/en/latest/generated/joblib.load.html) does not require a `compression` kwarg at all since it can automagically discover the correct compression algorithm used.


## Context
Even if it's a trivial issue I stumbled upon it and I hope to fix it so that future users will not have to go the joblib docs to find the problem.


## Possible Alternatives
I'a m working on a trivial fix, I'm going to open a PR as soon as possible.



Discussion:

Resolving patch:
diff --git a/kedro/extras/datasets/pickle/pickle_dataset.py b/kedro/extras/datasets/pickle/pickle_dataset.py
--- a/kedro/extras/datasets/pickle/pickle_dataset.py
+++ b/kedro/extras/datasets/pickle/pickle_dataset.py
@@ -42,9 +42,7 @@ class PickleDataSet(AbstractVersionedDataSet[Any, Any]):
         >>>   backend: joblib
         >>>   credentials: s3_credentials
         >>>   save_args:
-        >>>     compression: lz4
-        >>>   load_args:
-        >>>     compression: lz4
+        >>>     compress: lz4
 
     Example using Python API:
     ::


Test patch:
Repository: kedro-org/kedro

Issue:
Remove/change log level for "Credentials not found in your Kedro project config"
## Description
When getting the credentials in the KedroContext, we specifically look for the `credentials` file. If a project doesn't make use of this file a warning log will be thrown saying "Credentials not found in your Kedro project config". Not all users use the credentials file and so this warning isn't applicable. 

## Context
We're adding functionality to allow users to have credentials in environment variables (which is currently also possible with the `TemplatedConfigLoader`), which makes this error irrelevant. 



Discussion:

Resolving patch:
diff --git a/kedro/framework/context/context.py b/kedro/framework/context/context.py
--- a/kedro/framework/context/context.py
+++ b/kedro/framework/context/context.py
@@ -1,5 +1,6 @@
 """This module provides context for Kedro project."""
 
+import logging
 from copy import deepcopy
 from pathlib import Path, PurePosixPath, PureWindowsPath
 from typing import Any, Dict, Optional, Union
@@ -336,7 +337,9 @@ def _get_config_credentials(self) -> Dict[str, Any]:
         try:
             conf_creds = self.config_loader["credentials"]
         except MissingConfigException as exc:
-            warn(f"Credentials not found in your Kedro project config.\n{str(exc)}")
+            logging.getLogger(__name__).debug(
+                "Credentials not found in your Kedro project config.\n %s", str(exc)
+            )
             conf_creds = {}
         return conf_creds
 


Test patch:
diff --git a/tests/framework/context/test_context.py b/tests/framework/context/test_context.py
--- a/tests/framework/context/test_context.py
+++ b/tests/framework/context/test_context.py
@@ -1,5 +1,6 @@
 import configparser
 import json
+import logging
 import re
 import textwrap
 from pathlib import Path, PurePath, PurePosixPath, PureWindowsPath
@@ -297,15 +298,20 @@ def test_missing_parameters(self, tmp_path, dummy_context):
         with pytest.warns(UserWarning, match=re.escape(pattern)):
             _ = dummy_context.catalog
 
-    def test_missing_credentials(self, dummy_context):
+    def test_missing_credentials(self, dummy_context, caplog):
+        caplog.set_level(logging.DEBUG, logger="kedro")
+
         env_credentials = (
             dummy_context.project_path / "conf" / "local" / "credentials.yml"
         )
         env_credentials.unlink()
 
-        pattern = "Credentials not found in your Kedro project config."
-        with pytest.warns(UserWarning, match=re.escape(pattern)):
-            _ = dummy_context.catalog
+        _ = dummy_context.catalog
+
+        # check the logs
+        log_messages = [record.getMessage() for record in caplog.records]
+        expected_msg = "Credentials not found in your Kedro project config."
+        assert any(expected_msg in log_message for log_message in log_messages)
 
 
 @pytest.mark.parametrize(
Repository: kedro-org/kedro

Issue:
Change autogeneration of node names to not contain commas
## Description
Change how node names are generated to not contain commas. Currently we stringify a node's definition and set that as the node name if no name has been set explicitly: https://github.com/kedro-org/kedro/blob/main/kedro/pipeline/node.py#L239

Change is so that the generated name will contain the function name and output, where the outputs are separated by `_` or `-` instead of `,`.

So `two_inputs([A0,B0]) -> [C1,C2]` would become `two_inputs -> [C1-C2]`

## Context
https://github.com/kedro-org/kedro/issues/1828


Discussion:

Resolving patch:
diff --git a/kedro/pipeline/node.py b/kedro/pipeline/node.py
--- a/kedro/pipeline/node.py
+++ b/kedro/pipeline/node.py
@@ -159,7 +159,7 @@ def __hash__(self):
 
     def __str__(self):
         def _set_to_str(xset):
-            return f"[{','.join(xset)}]"
+            return f"[{';'.join(xset)}]"
 
         out_str = _set_to_str(self.outputs) if self._outputs else "None"
         in_str = _set_to_str(self.inputs) if self._inputs else "None"


Test patch:
diff --git a/tests/pipeline/test_node.py b/tests/pipeline/test_node.py
--- a/tests/pipeline/test_node.py
+++ b/tests/pipeline/test_node.py
@@ -266,13 +266,13 @@ def duplicate_output_list_node():
         (
             duplicate_output_dict_node,
             r"Failed to create node identity"
-            r"\(\[A\]\) -> \[A,A\] due to "
+            r"\(\[A\]\) -> \[A;A\] due to "
             r"duplicate output\(s\) {\'A\'}.",
         ),
         (
             duplicate_output_list_node,
             r"Failed to create node identity"
-            r"\(\[A\]\) -> \[A,A\] due to "
+            r"\(\[A\]\) -> \[A;A\] due to "
             r"duplicate output\(s\) {\'A\'}.",
         ),
     ],
diff --git a/tests/pipeline/test_pipeline.py b/tests/pipeline/test_pipeline.py
--- a/tests/pipeline/test_pipeline.py
+++ b/tests/pipeline/test_pipeline.py
@@ -620,7 +620,7 @@ def test_full(self, str_node_inputs_list):
             "#### Pipeline execution order ####",
             "Inputs: input1, input2",
             "",
-            "node1: biconcat([input1,input2]) -> [input3]",
+            "node1: biconcat([input1;input2]) -> [input3]",
             "node2: identity([input3]) -> [input4]",
             "",
             "Outputs: input4",
Repository: kedro-org/kedro

Issue:
Release Kedro `0.18.4`
### Depends on:
- Dataset issues
- Spaceflights tutorial documentation
- Open PRs related to datasets:
   - [x] https://github.com/kedro-org/kedro/pull/2082
   - [x] https://github.com/kedro-org/kedro/pull/1746
   - [x] https://github.com/kedro-org/kedro/pull/1992
   - [x] https://github.com/kedro-org/kedro/pull/1865
   - [x] https://github.com/kedro-org/kedro/pull/1312
   - [x] https://github.com/kedro-org/kedro/pull/1844
   - [x] https://github.com/kedro-org/kedro/pull/1962
   - [x] https://github.com/kedro-org/kedro/pull/1964
   - [x] https://github.com/kedro-org/kedro/pull/1931
   - [x] https://github.com/kedro-org/kedro/pull/1587

For the above PRs: if it's nearly finished, but the author isn't responding, we as a team can take over and finish the PR. If the PR still needs a lot of work and the author isn't responding, I suggest we close it and ask them to re-open in the new `kedro-datasets` repo. 



Discussion:
Adding a couple more:
* Create a `kedro-datasets` 1.0.0 version
* Update starters to pin at `kedro-datasets`~=1.0
* Pin `kedro~=0.18.4` for `kedro-datasets`
As discussed with @jmholzer, the release process should go like this due to the dependencies.
* Release kedro 0.18.4
* Release kedro-datasets 1.0.0 - Pin Kedro~=0.18.4
* Release Starters, pin kedro-datasets~=1.0 and kedro=0.18.4

And before that we should finish this ticket. If we want to have the release out this sprint, I think we should clean up the datasets ticket and finish #1711 by Wednesday to minimize the difference between kedro-datasets and kedro. Then we will have roughly 2-3 days to work on the releases above.
- [ ] #1711 

@merelcht 

Resolving patch:
diff --git a/kedro/__init__.py b/kedro/__init__.py
--- a/kedro/__init__.py
+++ b/kedro/__init__.py
@@ -3,7 +3,7 @@
 configuration and pipeline assembly.
 """
 
-__version__ = "0.18.3"
+__version__ = "0.18.4"
 
 
 import logging


Test patch:
Repository: kedro-org/kedro

Issue:
Release Kedro `0.18.5`
### Description

Release Kedro `0.18.5` which will contain lots of new features for configuration. The release depends on the following tickets to be finished:

- [x] BLOCKER: https://github.com/kedro-org/kedro/issues/2255
- [x] #1909 (Docs)
- [x] #2148 
- [x] #2170
- [x] #2225 

Initially we wanted to include the below issues as well, but the implementation turned out to be trickier than expected, so we'll take more time to investigate a solution and won't let it block the release.
- [x] #2146 
- [x] #2212 



Discussion:

Resolving patch:
diff --git a/kedro/__init__.py b/kedro/__init__.py
--- a/kedro/__init__.py
+++ b/kedro/__init__.py
@@ -3,7 +3,7 @@
 configuration and pipeline assembly.
 """
 
-__version__ = "0.18.4"
+__version__ = "0.18.5"
 
 
 import logging


Test patch:
Repository: kedro-org/kedro

Issue:
Activate all built-in resolvers for `OmegaConfigLoader` except `oc.env`
## Description
This feature would allow users to make use of the [built-in resolvers of `omegaconf`](https://omegaconf.readthedocs.io/en/2.3_branch/custom_resolvers.html#built-in-resolvers)
Currently all `omegaconf` built-in resolvers are turned of by default for `OmegaConfigLoader`, except `oc.env` which is activated when credentials are being loaded, and cleared after eagerly resolving the credentials from environment variables.

We would still like to keep `oc.env` deactivated by default but users can overwrite/activate it using `CONFIG_LOADER_ARGS` in their `settings.py` following the completion of #2622 

Also update the docs where needed.

## Context
Follow up task from the tech design discussion #2407 (24/05)




Discussion:

Resolving patch:
diff --git a/kedro/config/omegaconf_config.py b/kedro/config/omegaconf_config.py
--- a/kedro/config/omegaconf_config.py
+++ b/kedro/config/omegaconf_config.py
@@ -109,9 +109,8 @@ def __init__(
         }
         self.config_patterns.update(config_patterns or {})
 
-        # In the first iteration of the OmegaConfigLoader we'll keep the resolver turned-off.
-        # It's easier to introduce them step by step, but removing them would be a breaking change.
-        self._clear_omegaconf_resolvers()
+        # Deactivate oc.env built-in resolver for OmegaConf
+        OmegaConf.clear_resolver("oc.env")
 
         file_mimetype, _ = mimetypes.guess_type(conf_source)
         if file_mimetype == "application/x-tar":
@@ -345,14 +344,3 @@ def _resolve_environment_variables(config: dict[str, Any]) -> None:
             OmegaConf.clear_resolver("oc.env")
         else:
             OmegaConf.resolve(config)
-
-    @staticmethod
-    def _clear_omegaconf_resolvers():
-        """Clear the built-in OmegaConf resolvers."""
-        OmegaConf.clear_resolver("oc.env")
-        OmegaConf.clear_resolver("oc.create")
-        OmegaConf.clear_resolver("oc.deprecated")
-        OmegaConf.clear_resolver("oc.decode")
-        OmegaConf.clear_resolver("oc.select")
-        OmegaConf.clear_resolver("oc.dict.keys")
-        OmegaConf.clear_resolver("oc.dict.values")


Test patch: