Repository: apache/airflow

Issue:
Display DAGs dependencies  in CLI
### Description

Recently, we added [a new DAG dependencies view](https://github.com/apache/airflow/pull/13199) to the webserver. It would be helpful if a similar diagram could also be displayed/generated using the CLI. For now, only [one DAG can be displayed](http://airflow.apache.org/docs/apache-airflow/stable/usage-cli.html#exporting-dag-structure-as-an-image) using CLI.
![image](https://user-images.githubusercontent.com/12058428/137945580-6a33f919-2648-4980-bd2c-b40cfcacc9fe.png)

If anyone is interested, I will be happy to help with the review.

### Use case/motivation

* Keep parity between CLI and web serwer.
* Enable the writing of scripts that use these diagrams, e.g. for attaching in the documentation.

### Related issues

https://github.com/apache/airflow/pull/13199/files

### Are you willing to submit a PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)



Discussion:
Hi, 
I would like to contribute. However I'm new to this and would need some direction.
Thanks
Here is code that display DAG:
https://github.com/apache/airflow/blob/9ad8d2e49fc3394f58c6b92e40e9d84ce37fb9e1/airflow/cli/commands/dag_command.py#L187-L228
Here is PR that add DAG dependencies view to UI:
https://github.com/apache/airflow/pull/13199
You have to modify this code so that it displays DAG dependencies, not tasks dependencies.
> Here is code that display DAG:
> 
> https://github.com/apache/airflow/blob/9ad8d2e49fc3394f58c6b92e40e9d84ce37fb9e1/airflow/cli/commands/dag_command.py#L187-L228
> 
> 
> Here is PR that add DAG dependencies view to UI:
> #13199
> You have to modify this code so that it displays DAG dependencies, not tasks dependencies.

Thanks! I will look into this.
can I take this on please.
@subkanthi  Assgined.
Observation: the UI retrieves the dag dependencies of all the DAG's
`SELECT serialized_dag.dag_id AS serialized_dag_dag_id, json_extract(serialized_dag.data, %s) AS json_extract_1 
FROM serialized_dag`

But for the CLI, dag dependencies need to be filtered for the specific DAG passed by the user.
@subkanthi  Why do you want to filter dependencies based on DAG ID? I think it makes sense to see all the dependencies in one diagram.
> @subkanthi Why do you want to filter dependencies based on DAG ID? I think it makes sense to see all the dependencies in one diagram.

Hi @mik-laj , its good you actually looked at this comment, the current dag show takes the DAG id, so I assumed for the dag dependencies we need to show that specific DAG. Maybe its nice to have one for specific DAG id , and it no arguments are passed, show the dag dependencies for all DAGs.

`airflow dags show example_external_task_marker_parent --save output.jpg`

Resolving patch:
diff --git a/airflow/cli/cli_parser.py b/airflow/cli/cli_parser.py
--- a/airflow/cli/cli_parser.py
+++ b/airflow/cli/cli_parser.py
@@ -919,6 +919,33 @@ class GroupCommand(NamedTuple):
             ARG_IMGCAT,
         ),
     ),
+    ActionCommand(
+        name='show-dependencies',
+        help="Displays DAGs with their dependencies",
+        description=(
+            "The --imgcat option only works in iTerm.\n"
+            "\n"
+            "For more information, see: https://www.iterm2.com/documentation-images.html\n"
+            "\n"
+            "The --save option saves the result to the indicated file.\n"
+            "\n"
+            "The file format is determined by the file extension. "
+            "For more information about supported "
+            "format, see: https://www.graphviz.org/doc/info/output.html\n"
+            "\n"
+            "If you want to create a PNG file then you should execute the following command:\n"
+            "airflow dags show-dependencies --save output.png\n"
+            "\n"
+            "If you want to create a DOT file then you should execute the following command:\n"
+            "airflow dags show-dependencies --save output.dot\n"
+        ),
+        func=lazy_load_command('airflow.cli.commands.dag_command.dag_dependencies_show'),
+        args=(
+            ARG_SUBDIR,
+            ARG_SAVE,
+            ARG_IMGCAT,
+        ),
+    ),
     ActionCommand(
         name='backfill',
         help="Run subsections of a DAG for a specified date range",
diff --git a/airflow/cli/commands/dag_command.py b/airflow/cli/commands/dag_command.py
--- a/airflow/cli/commands/dag_command.py
+++ b/airflow/cli/commands/dag_command.py
@@ -46,7 +46,7 @@
     sigint_handler,
     suppress_logs_and_warning,
 )
-from airflow.utils.dot_renderer import render_dag
+from airflow.utils.dot_renderer import render_dag, render_dag_dependencies
 from airflow.utils.session import create_session, provide_session
 from airflow.utils.state import State
 
@@ -185,6 +185,25 @@ def set_is_paused(is_paused, args):
     print(f"Dag: {args.dag_id}, paused: {is_paused}")
 
 
+def dag_dependencies_show(args):
+    """Displays DAG dependencies, save to file or show as imgcat image"""
+    dot = render_dag_dependencies(SerializedDagModel.get_dag_dependencies())
+    filename = args.save
+    imgcat = args.imgcat
+
+    if filename and imgcat:
+        raise SystemExit(
+            "Option --save and --imgcat are mutually exclusive. "
+            "Please remove one option to execute the command.",
+        )
+    elif filename:
+        _save_dot_to_file(dot, filename)
+    elif imgcat:
+        _display_dot_via_imgcat(dot)
+    else:
+        print(dot.source)
+
+
 def dag_show(args):
     """Displays DAG or saves it's graphic representation to the file"""
     dag = get_dag(args.subdir, args.dag_id)
diff --git a/airflow/utils/dot_renderer.py b/airflow/utils/dot_renderer.py
--- a/airflow/utils/dot_renderer.py
+++ b/airflow/utils/dot_renderer.py
@@ -25,6 +25,7 @@
 from airflow.models.baseoperator import BaseOperator
 from airflow.models.dag import DAG
 from airflow.models.taskmixin import TaskMixin
+from airflow.serialization.serialized_objects import DagDependency
 from airflow.utils.state import State
 from airflow.utils.task_group import TaskGroup
 from airflow.www.views import dag_edges
@@ -128,6 +129,33 @@ def _draw_nodes(node: TaskMixin, parent_graph: graphviz.Digraph, states_by_task_
                 _draw_task_group(node, sub, states_by_task_id)
 
 
+def render_dag_dependencies(deps: Optional[Dict[str, List['DagDependency']]]) -> graphviz.Digraph:
+    """
+    Renders the DAG dependency to the DOT object.
+
+    :param deps: List of DAG dependencies
+    :type deps: Optional[List[DagDependency]]
+    :return: Graphviz object
+    :rtype: graphviz.Digraph
+    """
+    dot = graphviz.Digraph(graph_attr={"rankdir": "LR"})
+
+    for dag, dependencies in deps.items():
+        for dep in dependencies:
+            with dot.subgraph(
+                name=dag,
+                graph_attr={
+                    "rankdir": "LR",
+                    "labelloc": "t",
+                    "label": dag,
+                },
+            ) as dep_subgraph:
+                dep_subgraph.edge(dep.source, dep.dependency_id)
+                dep_subgraph.edge(dep.dependency_id, dep.target)
+
+    return dot
+
+
 def render_dag(dag: DAG, tis: Optional[List[TaskInstance]] = None) -> graphviz.Digraph:
     """
     Renders the DAG object to the DOT object.


Test patch:
diff --git a/tests/cli/commands/test_dag_command.py b/tests/cli/commands/test_dag_command.py
--- a/tests/cli/commands/test_dag_command.py
+++ b/tests/cli/commands/test_dag_command.py
@@ -190,6 +190,25 @@ def test_backfill_fails_without_loading_dags(self, mock_get_dag):
 
         mock_get_dag.assert_not_called()
 
+    def test_show_dag_dependencies_print(self):
+        with contextlib.redirect_stdout(io.StringIO()) as temp_stdout:
+            dag_command.dag_dependencies_show(self.parser.parse_args(['dags', 'show-dependencies']))
+        out = temp_stdout.getvalue()
+        assert "digraph" in out
+        assert "graph [rankdir=LR]" in out
+
+    @mock.patch("airflow.cli.commands.dag_command.render_dag_dependencies")
+    def test_show_dag_dependencies_save(self, mock_render_dag_dependencies):
+        with contextlib.redirect_stdout(io.StringIO()) as temp_stdout:
+            dag_command.dag_dependencies_show(
+                self.parser.parse_args(['dags', 'show-dependencies', '--save', 'output.png'])
+            )
+        out = temp_stdout.getvalue()
+        mock_render_dag_dependencies.return_value.render.assert_called_once_with(
+            cleanup=True, filename='output', format='png'
+        )
+        assert "File output.png saved" in out
+
     def test_show_dag_print(self):
         with contextlib.redirect_stdout(io.StringIO()) as temp_stdout:
             dag_command.dag_show(self.parser.parse_args(['dags', 'show', 'example_bash_operator']))
@@ -199,7 +218,7 @@ def test_show_dag_print(self):
         assert "runme_2 -> run_after_loop" in out
 
     @mock.patch("airflow.cli.commands.dag_command.render_dag")
-    def test_show_dag_dave(self, mock_render_dag):
+    def test_show_dag_save(self, mock_render_dag):
         with contextlib.redirect_stdout(io.StringIO()) as temp_stdout:
             dag_command.dag_show(
                 self.parser.parse_args(['dags', 'show', 'example_bash_operator', '--save', 'awesome.png'])
diff --git a/tests/utils/test_dot_renderer.py b/tests/utils/test_dot_renderer.py
--- a/tests/utils/test_dot_renderer.py
+++ b/tests/utils/test_dot_renderer.py
@@ -22,6 +22,7 @@
 from airflow.operators.bash import BashOperator
 from airflow.operators.dummy import DummyOperator
 from airflow.operators.python import PythonOperator
+from airflow.serialization.serialized_objects import DagDependency
 from airflow.utils import dot_renderer, timezone
 from airflow.utils.state import State
 from airflow.utils.task_group import TaskGroup
@@ -37,6 +38,27 @@ def setup_class(self):
     def teardown_method(self):
         clear_db_dags()
 
+    def test_should_render_dag_dependencies(self):
+        dag_dep_1 = DagDependency(
+            source='dag_one', target='dag_two', dependency_type='Sensor', dependency_id='task_1'
+        )
+        dag_dep_2 = DagDependency(
+            source='dag_two', target='dag_three', dependency_type='Sensor', dependency_id='task_2'
+        )
+
+        dag_dependency_list = []
+        dag_dependency_list.append(dag_dep_1)
+        dag_dependency_list.append(dag_dep_2)
+
+        dag_dependency_dict = {}
+        dag_dependency_dict['dag_one'] = dag_dependency_list
+        dot = dot_renderer.render_dag_dependencies(dag_dependency_dict)
+
+        assert "dag_one -> task_1" in dot.source
+        assert "task_1 -> dag_two" in dot.source
+        assert "dag_two -> task_2" in dot.source
+        assert "task_2 -> dag_three" in dot.source
+
     def test_should_render_dag(self):
         with DAG(dag_id="DAG_ID") as dag:
             task_1 = BashOperator(start_date=START_DATE, task_id="first", bash_command="echo 1")