Repository: dagster-io/dagster (Python)

Issue:
Error "The instance is not available to load partitions" when launching backfill on a graph with MultiPartitions
### Dagster version

dagster, version 1.5.4

### What's the issue?

I have:
* 2 dynamic partitions (images and ai_models) and a MultiPartition which uses these 2 dynamic partitions;
* 4 assets:
  * First two are partitioned by image partition. One of these assets is `observable_source_asset` ;
  * Third is `observable_source_asset`  partitioned by ai_model partition;
  * The last asset is partitioned by the MultiPartition.

When launching a backfill on any (non `observable_source_asset`) I'm getting an error `The instance is not available to load partitions`. Debugging showed that [this call](https://github.com/dagster-io/dagster/blob/master/python_modules/dagster/dagster/_core/definitions/partition_mapping.py#L451)  doesn't provide `dynamic_partitions_store`. Changing this call from
```
b_dimension_partitions_def_by_name[dim_name].get_partition_keys()
```
to
```
b_dimension_partitions_def_by_name[dim_name].get_partition_keys(dynamic_partitions_store=dynamic_partitions_store)
```
fixed the issue. Although I'm not sure if the partition store is not being passed intentionally.

Issue appears only when the MutliPartition is being used in the graph. Excluding the asset which uses this partition also fixes it.

### What did you expect to happen?

I expect the backfill to be launched.

### How to reproduce?

_No response_

### Deployment type

Local

### Deployment details

_No response_

### Additional information

Full stack trace:

```
dagster._check.CheckError: Failure condition: The instance is not available to load partitions. You may be seeing this error when using dynamic partitions with a version of dagster-webserver or dagster-cloud that is older than 1.1.18.
  File "/home/vladimir/miniconda3/envs/ml-pipeline/lib/python3.8/site-packages/dagster_graphql/implementation/utils.py", line 125, in _fn
    return fn(*args, **kwargs)
  File "/home/vladimir/miniconda3/envs/ml-pipeline/lib/python3.8/site-packages/dagster_graphql/implementation/utils.py", line 56, in _fn
    result = fn(self, graphene_info, *args, **kwargs)
  File "/home/vladimir/miniconda3/envs/ml-pipeline/lib/python3.8/site-packages/dagster_graphql/schema/roots/mutation.py", line 317, in mutate
    return create_and_launch_partition_backfill(graphene_info, backfillParams)
  File "/home/vladimir/miniconda3/envs/ml-pipeline/lib/python3.8/site-packages/dagster_graphql/implementation/execution/backfill.py", line 188, in create_and_launch_partition_backfill
    backfill = PartitionBackfill.from_asset_partitions(
  File "/home/vladimir/miniconda3/envs/ml-pipeline/lib/python3.8/site-packages/dagster/_core/execution/backfill.py", line 351, in from_asset_partitions
    serialized_asset_backfill_data=AssetBackfillData.from_asset_partitions(
  File "/home/vladimir/miniconda3/envs/ml-pipeline/lib/python3.8/site-packages/dagster/_core/execution/asset_backfill.py", line 460, in from_asset_partitions
    target_subset |= asset_graph.bfs_filter_subsets(
  File "/home/vladimir/miniconda3/envs/ml-pipeline/lib/python3.8/site-packages/dagster/_core/definitions/asset_graph.py", line 605, in bfs_filter_subsets
    partition_mapping.get_downstream_partitions_for_partitions(
  File "/home/vladimir/miniconda3/envs/ml-pipeline/lib/python3.8/site-packages/dagster/_core/definitions/partition_mapping.py", line 512, in get_downstream_partitions_for_partitions
    result = self._get_dependency_partitions_subset(
  File "/home/vladimir/miniconda3/envs/ml-pipeline/lib/python3.8/site-packages/dagster/_core/definitions/partition_mapping.py", line 449, in _get_dependency_partitions_subset
    *[
  File "/home/vladimir/miniconda3/envs/ml-pipeline/lib/python3.8/site-packages/dagster/_core/definitions/partition_mapping.py", line 450, in <listcomp>
    b_dimension_partitions_def_by_name[dim_name].get_partition_keys(
  File "/home/vladimir/miniconda3/envs/ml-pipeline/lib/python3.8/site-packages/dagster/_core/definitions/partition.py", line 538, in get_partition_keys
    check.failed(
  File "/home/vladimir/miniconda3/envs/ml-pipeline/lib/python3.8/site-packages/dagster/_check/__init__.py", line 1590, in failed
    raise CheckError(f"Failure condition: {desc}")
```

### Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.


Discussion:
@clairelin135  is this something you could look into? 
@vladhc Thanks for reporting -- https://github.com/dagster-io/dagster/pull/17397 will fix



Patch:
diff --git a/python_modules/dagster/dagster/_core/definitions/partition_mapping.py b/python_modules/dagster/dagster/_core/definitions/partition_mapping.py
index 63de4e7f645ee..b3c174d375f38 100644
--- a/python_modules/dagster/dagster/_core/definitions/partition_mapping.py
+++ b/python_modules/dagster/dagster/_core/definitions/partition_mapping.py
@@ -448,7 +448,9 @@ def _get_dependency_partitions_subset(
                     ]
                 ),
                 *[
-                    b_dimension_partitions_def_by_name[dim_name].get_partition_keys()
+                    b_dimension_partitions_def_by_name[dim_name].get_partition_keys(
+                        dynamic_partitions_store=dynamic_partitions_store, current_time=current_time
+                    )
                     for dim_name in unmapped_b_dim_names
                 ],
             ):


Test patch:
diff --git a/python_modules/dagster/dagster_tests/asset_defs_tests/partition_mapping_tests/test_multipartition_partition_mapping.py b/python_modules/dagster/dagster_tests/asset_defs_tests/partition_mapping_tests/test_multipartition_partition_mapping.py
index adba9982d7d01..beeb5ab3b4163 100644
--- a/python_modules/dagster/dagster_tests/asset_defs_tests/partition_mapping_tests/test_multipartition_partition_mapping.py
+++ b/python_modules/dagster/dagster_tests/asset_defs_tests/partition_mapping_tests/test_multipartition_partition_mapping.py
@@ -5,6 +5,7 @@
     AllPartitionMapping,
     AssetExecutionContext,
     AssetKey,
+    DagsterInstance,
     DailyPartitionsDefinition,
     DynamicPartitionsDefinition,
     IdentityPartitionMapping,
@@ -901,3 +902,25 @@ def multi_asset_2(context: AssetExecutionContext):
         AssetKey("asset_1"): asset_1_partition_mapping,
         AssetKey("asset_2"): asset_2_partition_mapping,
     }
+
+
+def test_dynamic_dimension_multipartition_mapping():
+    instance = DagsterInstance.ephemeral()
+
+    foo = DynamicPartitionsDefinition(name="foo")
+    foo_bar = MultiPartitionsDefinition(
+        {
+            "foo": foo,
+            "bar": DynamicPartitionsDefinition(name="bar"),
+        }
+    )
+
+    instance.add_dynamic_partitions("foo", ["a", "b", "c"])
+    instance.add_dynamic_partitions("bar", ["1", "2"])
+
+    result = MultiToSingleDimensionPartitionMapping().get_upstream_mapped_partitions_result_for_partitions(
+        downstream_partitions_subset=foo.empty_subset().with_partition_keys(["a"]),
+        upstream_partitions_def=foo_bar,
+        dynamic_partitions_store=instance,
+    )
+    assert result.partitions_subset == foo_bar.empty_subset().with_partition_keys(["2|a", "1|a"])