Repository: mage-ai/mage-ai
Pull request: [dy] Add all parent block variables to the add on block
# Description
<!-- Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context.
List any dependencies that are required for this change.
-->

Add parent block global vars to the add on block if they don't interfere with the existing add on block global vars.

# How Has This Been Tested?
<!-- Please describe the tests that you ran to verify your changes.
Provide instructions so we can reproduce.
-->

- [x] Tested with callback block
- [x] Unit test


# Checklist
- [x] The PR is tagged with proper labels (bug, enhancement, feature, documentation)
- [x] I have performed a self-review of my own code
- [x] I have added unit tests that prove my fix is effective or that my feature works
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] If new documentation has been added, relative paths have been added to the appropriate section of `docs/mint.json`

cc: @wangxiaoyou1993 
<!-- Optionally mention someone to let them know about this pull request -->


Problem statement:
Inherit block variables inside callback block
I'm using block variables to check certain conditions on a callback that runs in that block, but the variable defined in a parent block it's not accessible by callback.

It would be helpful to check parent block run context to take some actions in callback block.

For instance, I would need access `'configuration': {'table_name': 'dcategoria'}` from callback block, but this variable is configure on parent block.

At this moment, this is what happens:

Parent block kwargs:
```
{
  "env": "dev",
  "execution_date": "2024-03-01T16:06:00.924202",
  "interval_end_datetime": "2024-03-02T16:06:00.924202",
  "interval_seconds": null,
  "interval_start_datetime": "2024-03-01T16:06:00.924202",
  "interval_start_datetime_previous": null,
  "event": {},
  "logger": "Logger load_dcategoria_test (INFO)",
  "configuration": {
    "table_name": "dcategoria"
  },
  "context": {},
  "pipeline_uuid": "inteligencia_mercado_main",
  "block_uuid": "load_dcategoria"
}

```

Callback block kwargs:

```
{
  "level": "INFO",
  "logger_name": "load_dcategoria_test",
  "message": {
    "env": "dev",
    "execution_date": "2024-03-01T16:06:00.924202",
    "interval_end_datetime": "2024-03-02T16:06:00.924202",
    "interval_seconds": null,
    "interval_start_datetime": "2024-03-01T16:06:00.924202",
    "interval_start_datetime_previous": null,
    "event": {},
    "pipeline_uuid": "inteligencia_mercado_main",
    "block_uuid": "trigger_existing_table",
    "pipeline_run": null,
    "parent_block_uuid": "load_dcategoria"
  }
}

```
As can be seen `'configuration': {'table_name': 'dcategoria'}` is only available in parent block. In this case, I would like to use it to trigger a pipeline (conditional callback).

Hints from issue discussion:

Commit message: [dy] Add a callback block unit test

Patch to review:
diff --git a/mage_ai/data_preparation/models/block/__init__.py b/mage_ai/data_preparation/models/block/__init__.py
index a9fe97a88921..b9a3fe7a41fc 100644
--- a/mage_ai/data_preparation/models/block/__init__.py
+++ b/mage_ai/data_preparation/models/block/__init__.py
@@ -3658,23 +3658,23 @@ def _create_global_vars(
             global_vars['dynamic_block_index'] = dynamic_block_index
 
         if parent_block:
+            global_vars = merge_dict(parent_block.global_vars, global_vars)
             global_vars['parent_block_uuid'] = parent_block.uuid
 
-        if parent_block and \
-                parent_block.pipeline and \
-                PipelineType.INTEGRATION == parent_block.pipeline.type:
-
-            template_runtime_configuration = parent_block.template_runtime_configuration
-            index = template_runtime_configuration.get('index', None)
-            is_last_block_run = template_runtime_configuration.get('is_last_block_run', False)
-            selected_streams = template_runtime_configuration.get('selected_streams', [])
-            stream = selected_streams[0] if len(selected_streams) >= 1 else None
-            destination_table = template_runtime_configuration.get('destination_table', stream)
-
-            global_vars['index'] = index
-            global_vars['is_last_block_run'] = is_last_block_run
-            global_vars['stream'] = stream
-            global_vars['destination_table'] = destination_table
+            if parent_block.pipeline and \
+                    PipelineType.INTEGRATION == parent_block.pipeline.type:
+
+                template_runtime_configuration = parent_block.template_runtime_configuration
+                index = template_runtime_configuration.get('index', None)
+                is_last_block_run = template_runtime_configuration.get('is_last_block_run', False)
+                selected_streams = template_runtime_configuration.get('selected_streams', [])
+                stream = selected_streams[0] if len(selected_streams) >= 1 else None
+                destination_table = template_runtime_configuration.get('destination_table', stream)
+
+                global_vars['index'] = index
+                global_vars['is_last_block_run'] = is_last_block_run
+                global_vars['stream'] = stream
+                global_vars['destination_table'] = destination_table
 
         return global_vars
 
diff --git a/mage_ai/tests/data_preparation/models/test_block.py b/mage_ai/tests/data_preparation/models/test_block.py
index ceaa3debe840..bd1be85ee082 100644
--- a/mage_ai/tests/data_preparation/models/test_block.py
+++ b/mage_ai/tests/data_preparation/models/test_block.py
@@ -7,14 +7,17 @@
 from pandas.testing import assert_frame_equal
 
 # from mage_ai.data_cleaner.column_types.constants import ColumnType
-from mage_ai.data_preparation.models.block import Block, BlockType
+from mage_ai.data_preparation.models.block import Block, BlockType, CallbackBlock
 from mage_ai.data_preparation.models.block.errors import HasDownstreamDependencies
 from mage_ai.data_preparation.models.pipeline import Pipeline
 from mage_ai.data_preparation.repo_manager import get_repo_config
 from mage_ai.data_preparation.variable_manager import VariableManager
 from mage_ai.shared.path_fixer import add_root_repo_path_to_relative_path
 from mage_ai.tests.base_test import DBTestCase
-from mage_ai.tests.factory import create_integration_pipeline_with_blocks
+from mage_ai.tests.factory import (
+    create_integration_pipeline_with_blocks,
+    create_pipeline,
+)
 from mage_ai.tests.shared.mixins import ProjectPlatformMixin
 
 
@@ -763,3 +766,37 @@ def test_file_path(self):
                 )
 
                 self.assertEqual(block.file_path, add_root_repo_path_to_relative_path(path))
+
+
+class CallbackBlockTest(DBTestCase):
+    def setUp(self):
+        self.pipeline = create_pipeline('callback_pipeline', self.repo_path)
+
+    def tearDown(self):
+        self.pipeline.delete()
+
+    def test_create_global_vars_from_parent_block(self):
+        parent_block = Block.create(
+            'test_data_loader',
+            'data_loader',
+            self.repo_path,
+            pipeline=self.pipeline,
+        )
+        callback_block = CallbackBlock.create(parent_block.name)
+        self.pipeline.add_block(callback_block)
+        parent_block = parent_block.update(
+            dict(callback_blocks=[callback_block.uuid])
+        )
+        parent_block.global_vars = dict(
+            configuration=dict(table_name='load_data_table')
+        )
+
+        global_vars = dict(
+            random_var=1,
+        )
+        new_vars = callback_block._create_global_vars(
+            global_vars,
+            parent_block=parent_block,
+        )
+
+        self.assertIsNotNone(new_vars.get('configuration'))


Code review comments:
diff_hunk: @@ -3658,23 +3658,23 @@ def _create_global_vars(
             global_vars['dynamic_block_index'] = dynamic_block_index
 
         if parent_block:
+            global_vars = merge_dict(parent_block.global_vars, global_vars)
path: mage_ai/data_preparation/models/block/__init__.py
text: @user1:
Is the parent_block.global_vars always set before calling this method? It's currently set here https://github.com/mage-ai/mage-ai/blob/master/mage_ai/data_preparation/models/block/__init__.py#L3091
So it should only be set after parent block is executed.

Have you tested with conditional block?


@author:
oh good point. i only tested it with callback blocks. i'll fix that