Repository: apache/airflow

Issue:
TriggerDagRunOperator to configure the run_id of the triggered dag
<!--

Welcome to Apache Airflow!  For a smooth issue process, try to answer the following questions.
Don't worry if they're not all applicable; just try to include what you can :-)

If you need to include code snippets or logs, please put them in fenced code
blocks.  If they're super-long, please use the details tag like
<details><summary>super-long log</summary> lots of stuff </details>

Please delete these comment blocks before submitting the issue.

-->

**Description**

Airflow 1.10 support providing a run_id to TriggerDagRunOperator using [DagRunOrder](https://github.com/apache/airflow/blob/v1-10-stable/airflow/operators/dagrun_operator.py#L30-L33) object that will be returned after calling [TriggerDagRunOperator#python_callable](https://github.com/apache/airflow/blob/v1-10-stable/airflow/operators/dagrun_operator.py#L88-L95). With https://github.com/apache/airflow/pull/6317 (Airflow 2.0), this behavior changed and one could not provide run_id anymore to the triggered dag, which is very odd to say the least.

<!-- A short description of your feature -->

**Use case / motivation**

I want to be able to provide run_id for the TriggerDagRunOperator. In my case, I use the TriggerDagRunOperator to trigger 30K dag_run daily and will be so annoying to see them having unidentifiable run_id.

<!-- What do you want to happen?

Rather than telling us how you might implement this solution, try to take a
step back and describe what you are trying to achieve.

-->

After discussing with @ashb
**Suggested fixes can be**
 * provide a templated run_id as a parameter to TriggerDagRunOperator.
 * restore the old behavior of Airflow 1.10, where DagRunOrder holds the run_id and dag_run_conf of the triggered dag
 * create a new sub-class of TriggerDagRunOperator to fully customize it

**Are you willing to submit a PR?**

<!--- We accept contributions! -->

**Related Issues**

<!-- Is there currently another issue associated with this? -->



Discussion:
/cc @BasPH 

Resolving patch:
diff --git a/airflow/operators/trigger_dagrun.py b/airflow/operators/trigger_dagrun.py
--- a/airflow/operators/trigger_dagrun.py
+++ b/airflow/operators/trigger_dagrun.py
@@ -47,11 +47,14 @@ class TriggerDagRunOperator(BaseOperator):
     """
     Triggers a DAG run for a specified ``dag_id``
 
-    :param trigger_dag_id: the dag_id to trigger (templated)
+    :param trigger_dag_id: The dag_id to trigger (templated).
     :type trigger_dag_id: str
-    :param conf: Configuration for the DAG run
+    :param trigger_run_id: The run ID to use for the triggered DAG run (templated).
+        If not provided, a run ID will be automatically generated.
+    :type trigger_run_id: str
+    :param conf: Configuration for the DAG run.
     :type conf: dict
-    :param execution_date: Execution date for the dag (templated)
+    :param execution_date: Execution date for the dag (templated).
     :type execution_date: str or datetime.datetime
     :param reset_dag_run: Whether or not clear existing dag run if already exists.
         This is useful when backfill or rerun an existing dag run.
@@ -63,13 +66,13 @@ class TriggerDagRunOperator(BaseOperator):
     :param poke_interval: Poke interval to check dag run status when wait_for_completion=True.
         (default: 60)
     :type poke_interval: int
-    :param allowed_states: list of allowed states, default is ``['success']``
+    :param allowed_states: List of allowed states, default is ``['success']``.
     :type allowed_states: list
-    :param failed_states: list of failed or dis-allowed states, default is ``None``
+    :param failed_states: List of failed or dis-allowed states, default is ``None``.
     :type failed_states: list
     """
 
-    template_fields = ("trigger_dag_id", "execution_date", "conf")
+    template_fields = ("trigger_dag_id", "trigger_run_id", "execution_date", "conf")
     template_fields_renderers = {"conf": "py"}
     ui_color = "#ffefeb"
 
@@ -82,6 +85,7 @@ def __init__(
         self,
         *,
         trigger_dag_id: str,
+        trigger_run_id: Optional[str] = None,
         conf: Optional[Dict] = None,
         execution_date: Optional[Union[str, datetime.datetime]] = None,
         reset_dag_run: bool = False,
@@ -93,6 +97,7 @@ def __init__(
     ) -> None:
         super().__init__(**kwargs)
         self.trigger_dag_id = trigger_dag_id
+        self.trigger_run_id = trigger_run_id
         self.conf = conf
         self.reset_dag_run = reset_dag_run
         self.wait_for_completion = wait_for_completion
@@ -122,10 +127,12 @@ def execute(self, context: Dict):
         else:
             execution_date = timezone.utcnow()
 
-        run_id = DagRun.generate_run_id(DagRunType.MANUAL, execution_date)
+        if self.trigger_run_id:
+            run_id = self.trigger_run_id
+        else:
+            run_id = DagRun.generate_run_id(DagRunType.MANUAL, execution_date)
+
         try:
-            # Ignore MyPy type for self.execution_date
-            # because it doesn't pick up the timezone.parse() for strings
             dag_run = trigger_dag(
                 dag_id=self.trigger_dag_id,
                 run_id=run_id,
@@ -133,7 +140,6 @@ def execute(self, context: Dict):
                 execution_date=self.execution_date,
                 replace_microseconds=False,
             )
-
         except DagRunAlreadyExists as e:
             if self.reset_dag_run:
                 self.log.info("Clearing %s on %s", self.trigger_dag_id, self.execution_date)
@@ -145,11 +151,8 @@ def execute(self, context: Dict):
                     raise DagNotFound(f"Dag id {self.trigger_dag_id} not found in DagModel")
 
                 dag_bag = DagBag(dag_folder=dag_model.fileloc, read_dags_from_db=True)
-
                 dag = dag_bag.get_dag(self.trigger_dag_id)
-
                 dag.clear(start_date=self.execution_date, end_date=self.execution_date)
-
                 dag_run = DagRun.find(dag_id=dag.dag_id, run_id=run_id)[0]
             else:
                 raise e


Test patch:
diff --git a/tests/operators/test_trigger_dagrun.py b/tests/operators/test_trigger_dagrun.py
--- a/tests/operators/test_trigger_dagrun.py
+++ b/tests/operators/test_trigger_dagrun.py
@@ -86,6 +86,20 @@ def test_trigger_dagrun(self):
             assert len(dagruns) == 1
             assert dagruns[0].external_trigger
 
+    def test_trigger_dagrun_custom_run_id(self):
+        task = TriggerDagRunOperator(
+            task_id="test_task",
+            trigger_dag_id=TRIGGERED_DAG_ID,
+            trigger_run_id="custom_run_id",
+            dag=self.dag,
+        )
+        task.run(start_date=DEFAULT_DATE, end_date=DEFAULT_DATE)
+
+        with create_session() as session:
+            dagruns = session.query(DagRun).filter(DagRun.dag_id == TRIGGERED_DAG_ID).all()
+            assert len(dagruns) == 1
+            assert dagruns[0].run_id == "custom_run_id"
+
     def test_trigger_dagrun_with_execution_date(self):
         """Test TriggerDagRunOperator with custom execution_date."""
         utc_now = timezone.utcnow()