Repository: kedro-org/kedro

Issue:
Permission denied: 'git' is raised when running a node in apache/airflow Docker image
## Description
An Airflow DAG created from Kedro pipeline using Kedro-Airflow plugin is running inside Kubernetes, using official apache/airflow image, fails with an error `PermissionError: [Errno 13] Permission denied: 'git'`.

## Context
I'd like to run a Kedro pipeline in Airflow. 
I've used Kedro-Airflow plugin to generate an Airflow DAG from Kedro pipeline. My Airflow instance is working in Kubernetes cluster with official apache/airflow image.

When I run the DAG's task, it fails with an error during KedroSession `create` method: `PermissionError: [Errno 13] Permission denied: 'git'`.

I know that this happens because Airflow image doesn't have an installed `git`.
What really surprised me is the type of the error and that it is not covered.
I searched for similar cases and found an issue for `subprocess.check_output` method here https://github.com/python/cpython/issues/69667. It says that the function raises `PermissionError` instead of `FileNotFoundError` (which is covered inside `_describe_git` function in Kedro session) when different users are used. 
This is the case of Airflow image, where commands are executed under airflow user.

## Steps to Reproduce
Pre required: installed docker or another tool to create and run docker images.
1. Create a generic Kedro project and create a simple Kedro pipeline. It should have at least one node. It doesn't need to do anything.
    Name the root package as `my_kedro_dag` and name the node as `my_kedro_node`.
2. Generate Airflow DAG from the pipeline using [Kedro-Airflow plugin](https://github.com/kedro-org/kedro-plugins/tree/main/kedro-airflow).
3. Create a new docker image, based on [apache/airflow image](https://hub.docker.com/r/apache/airflow). Copy there the Kedro project with the generated DAG into default DAG folder - `/opt/airflow/dags`. Also install the project's python package inside the image. Name the image as `airflow-with-kedro-pipeline`.
4. Run the docker image under airflow user with the command `docker run --user 50000 -it airflow-with-kedro-pipeline airflow tasks run my_kedro_dag my_kedro_node my_run_id123`.

## Expected Result
Pipeline finished without errors.

## Actual Result
Traceeback of the error:
```
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.8/site-packages/ds_tools/airflow/operators/kedro.py", line 38, in execute
    with KedroSession.create(
  File "/home/airflow/.local/lib/python3.8/site-packages/kedro/framework/session/session.py", line 152, in create
    **_describe_git(session._project_path),
  File "/home/airflow/.local/lib/python3.8/site-packages/kedro/framework/session/session.py", line 34, in _describe_git
    res = subprocess.check_output(
  File "/usr/local/lib/python3.8/subprocess.py", line 415, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/local/lib/python3.8/subprocess.py", line 493, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/local/lib/python3.8/subprocess.py", line 858, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/local/lib/python3.8/subprocess.py", line 1704, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
PermissionError: [Errno 13] Permission denied: 'git'
```

## Your Environment

* Kedro version used (`pip show kedro` or `kedro -V`): 0.18.2
* Python version used (`python -V`): 3.8
* Operating system and version: Debian GNU/Linux 11 (bullseye)
* Airflow image: apache/airflow:2.3.4-python3.8



Discussion:
Quick question: Should we remove the dependency on git altogether? In production, the fewer dependencies the better…
Thanks for this report @vilozio and sorry no one seems to have handled it yet. It's a very valid point. And also a very valid question @MatthiasRoels. `git` is only used here to save a commit has to the session store for possible use in experiment tracking. It's certainly not a major feature, and `_describe_git` should never be allowed to crash the kedro process.

For now let's just change https://github.com/kedro-org/kedro/blob/471153d5fa853bba02256f32884904373a7b846b/kedro/framework/session/session.py#L49 to catch all `Exception` so that we have no chance of missing one. 

It's exactly the same problem as https://github.com/kedro-org/kedro-viz/issues/1291 and we should solve them both at the same time (and soon, it's very easy to solve).

Resolving patch:
diff --git a/kedro/framework/session/session.py b/kedro/framework/session/session.py
--- a/kedro/framework/session/session.py
+++ b/kedro/framework/session/session.py
@@ -46,8 +46,10 @@ def _describe_git(project_path: Path) -> Dict[str, Dict[str, Any]]:
         git_data["dirty"] = bool(git_status_res.decode().strip())
 
     # `subprocess.check_output()` raises `NotADirectoryError` on Windows
-    except (subprocess.CalledProcessError, FileNotFoundError, NotADirectoryError):
-        logging.getLogger(__name__).debug("Unable to git describe %s", project_path)
+    except Exception:  # pylint: disable=broad-except
+        logger = logging.getLogger(__name__)
+        logger.debug("Unable to git describe %s", project_path)
+        logger.debug(traceback.format_exc())
         return {}
 
     return {"git": git_data}
@@ -164,7 +166,6 @@ def create(  # pylint: disable=too-many-arguments
             "package_name": session._package_name,
             "project_path": session._project_path,
             "session_id": session.session_id,
-            **_describe_git(session._project_path),
         }
 
         ctx = click.get_current_context(silent=True)
@@ -187,8 +188,11 @@ def create(  # pylint: disable=too-many-arguments
 
         session._store.update(session_data)
 
-        # we need a ConfigLoader registered in order to be able to set up logging
+        # We need ConfigLoader and env to setup logging correctly
         session._setup_logging()
+        session_data.update(**_describe_git(session._project_path))
+        session._store.update(session_data)
+
         return session
 
     def _get_logging_config(self) -> Dict[str, Any]:


Test patch:
diff --git a/tests/framework/session/test_session.py b/tests/framework/session/test_session.py
--- a/tests/framework/session/test_session.py
+++ b/tests/framework/session/test_session.py
@@ -531,13 +531,13 @@ def test_git_describe_error(
         session = KedroSession.create(mock_package_name, fake_project)
         assert "git" not in session.store
 
-        expected_log_messages = [f"Unable to git describe {fake_project}"]
+        expected_log_message = f"Unable to git describe {fake_project}"
         actual_log_messages = [
             rec.getMessage()
             for rec in caplog.records
             if rec.name == SESSION_LOGGER_NAME and rec.levelno == logging.DEBUG
         ]
-        assert actual_log_messages == expected_log_messages
+        assert expected_log_message in actual_log_messages
 
     def test_get_username_error(self, fake_project, mock_package_name, mocker, caplog):
         """Test that username information is not added to the session store