Repository: metabase/metabase
Pull request #56826: [VISUALIZER] Support Dashboard Duplication
Author: login: tsplude
Labels: name: Reporting/Dashboards
name: .Team/DashViz
description: DEPRECATED  Please use .Team/UXWest instead
name: .Address With Project
Created: 2025-04-17T04:49:29Z | Merged: 2025-04-30T22:36:53Z

Closes VIZ-616

Changes include:
  - Support dashboard duplication for dashboards containing visualizer cards
    - Visualizer cards have unique viz settings which can contain embedded card id references
    - We need to find those references during duplication and update them to the new card ids


Diff:
diff --git a/src/metabase/api/dashboard.clj b/src/metabase/api/dashboard.clj
index c2c2e6ab3773..f2ca8b770873 100644
--- a/src/metabase/api/dashboard.clj
+++ b/src/metabase/api/dashboard.clj
@@ -4,6 +4,7 @@
    [clojure.core.cache :as cache]
    [clojure.core.memoize :as memoize]
    [clojure.set :as set]
+   [clojure.string :as str]
    [medley.core :as m]
    [metabase.actions.core :as actions]
    [metabase.analytics.core :as analytics]
@@ -405,6 +406,27 @@
                                                     (dissoc :id :entity_id :created_at :updated_at))))]
     (zipmap (map :id existing-tabs) new-tab-ids)))
 
+(defn- update-colvalmap-setting
+  "Visualizer dashcards have unique visualization settings which embed column id remapping metadata
+  This function iterates through the `:columnValueMapping` viz setting and updates referenced card ids
+
+  col->val-source can look like:
+  {:COLUMN_2 [{:sourceId 'card:<OLD_CARD_ID>', :originalName 'sum', :name 'COLUMN_2'}], ...}"
+  [col->val-source id->new-card]
+  (let [update-cvm-item (fn [item]
+                          (if-let [source-id (:sourceId item)]
+                            (if-let [[_ card-id] (and (string? source-id)
+                                                      (re-find #"^card:(\d+)$" source-id))]
+                              (if-let [new-card (get id->new-card (Long/parseLong card-id))]
+                                (assoc item :sourceId (str "card:" (:id new-card)))
+                                item)
+                              item)
+                            item))
+        update-cvm      (fn [cvm]
+                          (when (map? cvm)
+                            (update-vals cvm #(mapv update-cvm-item %))))]
+    (update-cvm col->val-source)))
+
 (defn update-cards-for-copy
   "Update dashcards in a dashboard for copying.
   If the dashboard has tabs, fix up the tab ids in dashcards to point to the new tabs.
@@ -449,7 +471,9 @@
                                          (keep (fn [card]
                                                  (when-let [id' (new-id (:id card))]
                                                    (assoc card :id id')))
-                                               series)))))))
+                                               series)))
+                    (m/update-existing-in [:visualization_settings :visualization :columnValuesMapping]
+                                          update-colvalmap-setting id->new-card)))))
           dashcards)))
 
 (api.macros/defendpoint :post "/:from-dashboard-id/copy"
diff --git a/test/metabase/api/dashboard_test.clj b/test/metabase/api/dashboard_test.clj
index 8b7c737e73f9..2c8eac062497 100644
--- a/test/metabase/api/dashboard_test.clj
+++ b/test/metabase/api/dashboard_test.clj
@@ -48,6 +48,34 @@
 
 (set! *warn-on-reflection* true)
 
+(deftest update-colvalmap-setting-test
+  (testing "update-colvalmap-setting function with regex matching"
+    (let [id->new-card {123 {:id 456}
+                        789 {:id 987}}
+          col->val-source {:COLUMN_1 [{:sourceId "card:123" :originalName "sum" :name "COLUMN_1"}]
+                           :COLUMN_2 [{:sourceId "card:789" :originalName "count" :name "COLUMN_2"}]
+                           :COLUMN_3 [{:sourceId "card:999" :originalName "avg" :name "COLUMN_3"}]
+                           :COLUMN_4 [{:sourceId "not-a-card" :originalName "x" :name "COLUMN_4"}]
+                           :COLUMN_5 [{:sourceId "card:abc" :originalName "invalid" :name "COLUMN_5"}]
+                           :COLUMN_6 [{:name "No source ID"}]}
+          result (#'api.dashboard/update-colvalmap-setting col->val-source id->new-card)]
+
+      (testing "should update valid card IDs that exist in the map"
+        (is (= "card:456" (-> result :COLUMN_1 first :sourceId)))
+        (is (= "card:987" (-> result :COLUMN_2 first :sourceId))))
+
+      (testing "should not modify card IDs that don't exist in the map"
+        (is (= "card:999" (-> result :COLUMN_3 first :sourceId))))
+
+      (testing "should not modify non-card sourceIds"
+        (is (= "not-a-card" (-> result :COLUMN_4 first :sourceId))))
+
+      (testing "should not modify invalid card IDs (non-numeric)"
+        (is (= "card:abc" (-> result :COLUMN_5 first :sourceId))))
+
+      (testing "should handle items without sourceId"
+        (is (= {:name "No source ID"} (-> result :COLUMN_6 first)))))))
+
 (use-fixtures
   :once
   (fixtures/initialize :test-users))


Reviews:
user: login: noahmoss
body: Functionally looks fine but two readability suggestions, and I think this would be worth at least a quick unit test to exercise the parsing logic?
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-04-25T20:25:32Z
user: login: noahmoss
body: Stamping to unblock but looks like there's a compilation error in the tests
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-29T13:14:56Z

Review comments:
diff_hunk: @@ -405,6 +406,30 @@
                                                     (dissoc :id :entity_id :created_at :updated_at))))]
     (zipmap (map :id existing-tabs) new-tab-ids)))
 
+(defn- update-colvalmap-setting
+  "Visualizer dashcards have unique visualization settings which embed column id remapping metadata
+  This function iterates through the `:columnValueMapping` viz setting and updates referenced card ids
+
+  col->val-source can look like:
+  {:COLUMN_2 [{:sourceId 'card:<OLD_CARD_ID>', :originalName 'sum', :name 'COLUMN_2'}], ...}"
+  [col->val-source id->new-card]
+  (let [update-cvm-item (fn [item]
+                          (if-let [source-id (:sourceId item)]
+                            (if (and (string? source-id) (str/starts-with? source-id "card:"))
+                              (let [old-id (Long/parseLong (subs source-id 5))]
+                                (if-let [new-card (and old-id (get id->new-card old-id))]
+                                  (assoc item :sourceId (str "card:" (:id new-card)))
+                                  item))
+                              item)
+                            item))
path: src/metabase/api/dashboard.clj
user: login: noahmoss
body: I think it'd be a bit more readable to use a regex here with a capturing group rather than `starts-with?` and `subs`. Something like `(re-find #"^card:(\d+)$" source-id)`
created_at: 2025-04-25T20:24:04Z
author_association: MEMBER
diff_hunk: @@ -405,6 +406,30 @@
                                                     (dissoc :id :entity_id :created_at :updated_at))))]
     (zipmap (map :id existing-tabs) new-tab-ids)))
 
+(defn- update-colvalmap-setting
+  "Visualizer dashcards have unique visualization settings which embed column id remapping metadata
+  This function iterates through the `:columnValueMapping` viz setting and updates referenced card ids
+
+  col->val-source can look like:
+  {:COLUMN_2 [{:sourceId 'card:<OLD_CARD_ID>', :originalName 'sum', :name 'COLUMN_2'}], ...}"
+  [col->val-source id->new-card]
+  (let [update-cvm-item (fn [item]
+                          (if-let [source-id (:sourceId item)]
+                            (if (and (string? source-id) (str/starts-with? source-id "card:"))
+                              (let [old-id (Long/parseLong (subs source-id 5))]
+                                (if-let [new-card (and old-id (get id->new-card old-id))]
+                                  (assoc item :sourceId (str "card:" (:id new-card)))
+                                  item))
+                              item)
+                            item))
+        update-cvm      (fn [cvm]
+                          (when (map? cvm)
+                            (into {}
+                                  (map (fn [[col-key values-vec]]
+                                         [col-key (mapv update-cvm-item values-vec)])
+                                       cvm))))]
path: src/metabase/api/dashboard.clj
user: login: noahmoss
body: can you use `update-vals`?
created_at: 2025-04-25T20:24:40Z
author_association: MEMBER

PR comments:
user: login: trunk-io[bot]
created_at: 2025-04-17T05:25:03Z
body: <!-- Trunk Test Analytics -->


[![Static Badge](https://raster.shields.io/badge/124828_tests-38851837-purple)](https://app.trunk.io/metabase/flaky-tests/pr/56826?repo=metabase/metabase&commitHash=3885183703bd1c0b5c7aac317a0d3b94c58d7898&utm_source=gh-comment) &ensp; [![Static Badge](https://raster.shields.io/badge/34-failed-crimson)](https://app.trunk.io/metabase/flaky-tests/pr/56826?repo=metabase/metabase&commitHash=3885183703bd1c0b5c7aac317a0d3b94c58d7898&utm_source=gh-comment&runConclusion=FAILURE) &ensp; [![Static Badge](https://raster.shields.io/badge/3-flaky-red)](https://app.trunk.io/metabase/flaky-tests/pr/56826?repo=metabase/metabase&commitHash=3885183703bd1c0b5c7aac317a0d3b94c58d7898&utm_source=gh-comment&runConclusion=FAILURE) &ensp; [![Static Badge](https://raster.shields.io/badge/0-quarantined-yellow)](https://app.trunk.io/metabase/flaky-tests/pr/56826?repo=metabase/metabase&commitHash=3885183703bd1c0b5c7aac317a0d3b94c58d7898&utm_source=gh-comment&runConclusion=QUARANTINED)


| Failed Test | Failure Summary | Logs |
| --- | --- | --- |
| <code>attachments-test</code> | The expected email message does not match the actual message sent. | [Logs ↗︎](https://app.trunk.io/metabase/flaky-tests/pr/56826?repo=metabase/metabase&commitHash=3885183703bd1c0b5c7aac317a0d3b94c58d7898&utm_source=gh-comment&runConclusion=FAILURE&runId=0f23a75b-6487-5e9d-b3ef-6586affb690e) |
| <code>render-funnel-visualizer</code> | Error occurred while rendering funnel charts, likely due to an undefined value. | [Logs ↗︎](https://app.trunk.io/metabase/flaky-tests/pr/56826?repo=metabase/metabase&commitHash=3885183703bd1c0b5c7aac317a0d3b94c58d7898&utm_source=gh-comment&runConclusion=FAILURE&runId=0f6d0aae-fe6e-5da4-be9c-7983630b2bd3) |
| <code>attachments-test</code> | The email message was not sent as expected, with attachments and inline content. | [Logs ↗︎](https://app.trunk.io/metabase/flaky-tests/pr/56826?repo=metabase/metabase&commitHash=3885183703bd1c0b5c7aac317a0d3b94c58d7898&utm_source=gh-comment&runConclusion=FAILURE&runId=1792838b-c2f8-5cef-89b5-1d5a6155c6a7) |
| <code>render-funnel-visualizer</code> | Visualizer funnel charts render failed due to TypeError: Cannot read property 'Symbol(Symbol.iterator)' of undefined. | [Logs ↗︎](https://app.trunk.io/metabase/flaky-tests/pr/56826?repo=metabase/metabase&commitHash=3885183703bd1c0b5c7aac317a0d3b94c58d7898&utm_source=gh-comment&runConclusion=FAILURE&runId=1bf0d0c1-0c0c-5b7b-9a92-6d40e1903bbc) |

[... and 30 more](https://app.trunk.io/metabase/flaky-tests/pr/56826?repo=metabase/metabase&commitHash=3885183703bd1c0b5c7aac317a0d3b94c58d7898&utm_source=gh-comment&runConclusion=FAILURE)


| Flaky Test | Failure Summary | Logs |
| --- | --- | --- |
| <code>query-generator-test</code> | The expected join clause did not match the actual join clause, with the difference being the 'fields' option. | [Logs ↗︎](https://app.trunk.io/metabase/flaky-tests/pr/56826?repo=metabase/metabase&commitHash=3885183703bd1c0b5c7aac317a0d3b94c58d7898&utm_source=gh-comment&runConclusion=FAILURE&runId=0cf5ef89-801a-5968-99b0-2d0bce3514ce) |
| <code>identity-hash-test</code> | The expected hash did not match the actual hash of a DashboardCard. | [Logs ↗︎](https://app.trunk.io/metabase/flaky-tests/pr/56826?repo=metabase/metabase&commitHash=3885183703bd1c0b5c7aac317a0d3b94c58d7898&utm_source=gh-comment&runConclusion=FAILURE&runId=42ab41df-dd9e-5c35-8273-a5f7e6bdf4f6) |
| <code>scenarios > question > download should remember the selected format across page reloads</code> | Expected '.xlsx' to be present in the list of elements, but timed out after 4000ms. | [Logs ↗︎](https://app.trunk.io/metabase/flaky-tests/pr/56826?repo=metabase/metabase&commitHash=3885183703bd1c0b5c7aac317a0d3b94c58d7898&utm_source=gh-comment&runConclusion=FAILURE&runId=bfdfd455-a9c0-5f4d-9eec-e10549c1d154) |
<sub>

[View Full Report ↗︎](https://app.trunk.io/metabase/flaky-tests/pr/56826?repo=metabase/metabase&commitHash=3885183703bd1c0b5c7aac317a0d3b94c58d7898&utm_source=gh-comment) ⋅ [Docs](https://docs.trunk.io/flaky-tests?utm_source=gh-comment)

</sub>
author_association: NONE
user: login: github-actions[bot]
created_at: 2025-04-30T19:13:31Z
body: 
  ## e2e tests failed on `3885183703bd1c0b5c7aac317a0d3b94c58d7898-1`

  [e2e test run](https://github.com/metabase/metabase/actions/runs/14762236089?pr=56826)

  File | Test Name
  ---- | ---------
  `downloads.cy.spec.js` | scenarios > question > download > should remember the selected format across page reloads

author_association: CONTRIBUTOR