Repository: metabase/metabase
Pull request #55345: Editable table components in dashboards
Author: login: crisptrutski
Created: 2025-03-18T15:19:36Z | Merged: 2025-03-18T22:07:08Z

Add support for editable table components within dashboards, with dashboard filter support.

The idea here is that the frontend can build a simple template, a bit like a text card, knowing nothing about our implementation details, and doesn't require touching MLib or requesting any additional metadata. The backend will then lazily construct something more complex, whose shape "just works" for executing queries, attaching filters, applying filters, and updating things when the filters change.

Basically saving us a ton of time and a ton of extra surface area. The returned card is just that... a `:model/Card`. I will still retains the "template" data though, and has a distinct "display" type so we know to render the new component. The template can be updated, and the underlying card will get rebuilt.

The major downside with this approach is that it adds more overloading to an already heavily overloaded primitive. I still believe this is the right approach versus duplicating a huge amount of APIs and functionality, or making all that code polymorphic. If we continue down this road though we'll need to improve the encapsulation of a "card" and add more validation and consistency checks. 

Given how directly we're able to deliver advanced new functionality in the short term, and how naturally many longer term features will fall out of things, I strongly believe in taking this route for now. For now it's hiding the "cards" as best it can, using them as an internal detail, but we might want to consider first class things like "editable models" for the join / aggregation cases. Joins provide a really nice way to segregate column permissions in many basic use cases.

An awkward thing with this "just in time" creation is that the frontend won't know how to attach dashboard filters until after first saving any new table dashcards. Not huge to handle, but not very obvious. We could look at adding a new API for returning this metadata immediately when the template card is created, or perhaps this can even use what's returned by from `/table/:id/data`.


Diff:
diff --git a/src/metabase/api/dashboard.clj b/src/metabase/api/dashboard.clj
index c2c2e6ab3773..12e4c47753f4 100644
--- a/src/metabase/api/dashboard.clj
+++ b/src/metabase/api/dashboard.clj
@@ -4,6 +4,7 @@
    [clojure.core.cache :as cache]
    [clojure.core.memoize :as memoize]
    [clojure.set :as set]
+   [clojure.walk :as walk]
    [medley.core :as m]
    [metabase.actions.core :as actions]
    [metabase.analytics.core :as analytics]
@@ -846,6 +847,54 @@
 
 ;;;;;;;;;;;;;;;;;;;;; End functions to handle broken subscriptions
 
+(defn create-editable-table-card!
+  "The implementation details of an editable table dashcard.
+  A quick solution, and a very extensible one, but perhaps overloading a core primitive too much, and could be leaky."
+  [dashboard-id dashcard table-id]
+  (let [tab-id  (:dashboard_tab_id dashcard)
+        table   (t2/select-one [:model/Table :db_id :name] table-id)
+        ;; To keep things simple for now, we don't make any attempt to deduplicate or reuse these cards.
+        ;; This lets us dashboard cards, which helps keep them hidden and have their lifecycle handled implicitly.
+        ;; We need to watch out for other ways they could "leak", for example, in search.
+        ;; We've handled search for now by filtering on :display, but that's quite opinionated and coupled.
+        ;; ;;
+        ;; It does mean that we could create a lot of garbage, but it shouldn't be a big problem in our PoC.
+        ;; We could also change from archiving to deleting them when they are orphaned.
+        card    {:dashboard_id           dashboard-id
+                 :dashboard_tab_id       tab-id
+                 :collection_position    nil
+                 :dataset_query          {:database (:db_id table)
+                                          :query    {:source-table table-id}
+                                          :type     :query}
+                 :description            nil
+                 :display                "table-editable"
+                 :name                   (:name table)
+                 :result_metadata        nil
+                 :type                   "question"
+                 ;; Redundant with :display, but just in case it's useful. Revisit once FE is built.
+                 :visualization_settings {:editable? true}}
+        ;; We always create a new card, rather than risk pockets of stale state, e.g., when we've changed the table.
+        ;; This also saves us checking that the existing card is a dashboard card, has the right display type, etc.
+        card-id (:id (card/create-card! card @api/*current-user* true false))]
+    (-> dashcard
+        ;; This is a downside to creating a new card. If we find more pockets like this we should pivot to reusing
+        ;; the existing card.
+        (u/update-if-exists :parameter_mappings #(walk/postwalk (fn [x] (if (:card_id x) (assoc x :card_id card-id) x)) %))
+        (assoc :dashboard_id dashboard-id
+               :card_id card-id))))
+
+(defn- init-editable-table-cards!
+  "This method insulated the FE from knowing anything about the implementation details of editable-table dashcards.
+  Given some 'template' we return the fully realized dashcards, with their internal dependencies already in the db.
+  The template data is preserved, allowing the frontend end to further manipulate it in future."
+  [dashboard-id new-dashcards]
+  (for [dashcard new-dashcards]
+    ;; I was expecting dashcards to have some type, but it seems they're duck typed?
+    ;; We probably want to look into this more and have them more clearly differentiated.
+    (if-let [table-id (get-in dashcard [:visualization_settings :table_id])]
+      (create-editable-table-card! dashboard-id dashcard table-id)
+      dashcard)))
+
 (defn- update-dashboard
   "Updates a Dashboard. Designed to be reused by PUT /api/dashboard/:id and PUT /api/dashboard/:id/cards"
   [id {:keys [dashcards tabs parameters] :as dash-updates}]
@@ -927,6 +976,7 @@
                                                                       (if-let [real-tab-id (get old->new-tab-id (:dashboard_tab_id card))]
                                                                         (assoc card :dashboard_tab_id real-tab-id)
                                                                         card))))
+                   new-dashcards                             (init-editable-table-cards! id new-dashcards)
                    dashcards-changes-stats                   (do-update-dashcards! hydrated-current-dash current-dashcards new-dashcards)]
                (reset! changes-stats
                        (merge
diff --git a/src/metabase/models/card.clj b/src/metabase/models/card.clj
index c15ccc1fb726..14c632047ec6 100644
--- a/src/metabase/models/card.clj
+++ b/src/metabase/models/card.clj
@@ -1327,7 +1327,12 @@
    :bookmark     [:model/CardBookmark [:and
                                        [:= :bookmark.card_id :this.id]
                                        [:= :bookmark.user_id :current_user/id]]]
-   :where        [:= :collection.namespace nil]
+   :where        [:and
+                  [:= :collection.namespace nil]
+                  [:or
+                   ;; Leaving the door open to editable models (e.g., joining read-only and editable columns)
+                   [:not= :this.display "table-editable"]
+                   [:= :this.dashboard_id nil]]]
    :joins        {:collection [:model/Collection [:= :collection.id :this.collection_id]]
                   :r          [:model/Revision [:and
                                                 [:= :r.model_id :this.id]
diff --git a/test/metabase/api/dashboard_test.clj b/test/metabase/api/dashboard_test.clj
index 0f76eca3759a..798b16cd37ca 100644
--- a/test/metabase/api/dashboard_test.clj
+++ b/test/metabase/api/dashboard_test.clj
@@ -1788,6 +1788,32 @@
                                                              {:dashcards [new-dashcard-info]
                                                               :tabs      []}))}))))))
 
+(deftest e2e-create-editable-table-card
+  (testing "PUT /api/dashboard/:id with a placeholder for an editable table.\n"
+    (mt/test-helpers-set-global-values!
+      (mt/with-temp
+        [:model/Dashboard               {dashboard-id :id}  {}]
+        (let [table-id (mt/id :venues)
+              resp (mt/user-http-request :rasta :put 200 (format "dashboard/%d" dashboard-id)
+                                         {:tabs      [{:id   -1
+                                                       :name "New tab"}]
+                                          :dashcards [{:id               -1
+                                                       :size_x           1
+                                                       :size_y           1
+                                                       :col              3
+                                                       :row              3
+                                                       :dashboard_tab_id -1
+                                                       :visualization_settings {:table_id table-id}}]})]
+          (testing "the dashcard gets turned into something richer, that supports filtering."
+            (is (=? [{:id                     (mt/malli=? [:fn pos-int?])
+                      :size_x                 1
+                      :size_y                 1
+                      :col                    3
+                      :row                    3
+                      :card_id                int?
+                      :visualization_settings {:table_id table-id}}]
+                    (:dashcards resp)))))))))
+
 (deftest e2e-update-dashboard-cards-and-tabs-test
   (testing "PUT /api/dashboard/:id with updating dashboard and create/update/delete of dashcards and tabs in a single req"
     (mt/test-helpers-set-global-values!


Reviews:
user: login: piranha
body: So for FE to use this it should just set `visualization_settings.table_id` when creating a card? Sounds fine by me, let's see what Denis thinks.
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-18T15:43:25Z
user: login: crisptrutski
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-18T21:26:18Z
user: login: crisptrutski
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-18T21:28:23Z

Review comments:
diff_hunk: @@ -846,6 +847,54 @@
 
 ;;;;;;;;;;;;;;;;;;;;; End functions to handle broken subscriptions
 
+(defn create-editable-table-card!
+  "The implementation details of an editable table dashcard.
+  A quick solution, and a very extensible one, but perhaps overloading a core primitive too much, and could be leaky."
+  [dashboard-id dashcard table-id]
+  (let [tab-id  (:dashboard_tab_id dashcard)
+        table   (t2/select-one [:model/Table :db_id :name] table-id)
+        ;; To keep things simple for now, we don't make any attempt to deduplicate or reuse these cards.
+        ;; This lets us dashboard cards, which helps keep them hidden and have their lifecycle handled implicitly.
+        ;; We need to watch out for other ways they could "leak", for example, in search.
+        ;; We've handled search for now by filtering on :display, but that's quite opinionated and coupled.
+        ;; ;;
+        ;; It does mean that we could create a lot of garbage, but it shouldn't be a big problem in our PoC.
+        ;; We could also change from archiving to deleting them when they are orphaned.
+        card    {:dashboard_id           dashboard-id
+                 :dashboard_tab_id       tab-id
+                 :collection_position    nil
+                 :dataset_query          {:database (:db_id table)
+                                          :query    {:source-table table-id}
+                                          :type     :query}
+                 :description            nil
+                 :display                "table-editable"
path: src/metabase/api/dashboard.clj
user: login: crisptrutski
body: I was surprised that this still results in the FE rendering a table, and it doesn't even log a warning!

This was both confusing and a bit scary.
created_at: 2025-03-18T21:26:18Z
author_association: CONTRIBUTOR
diff_hunk: @@ -846,6 +847,54 @@
 
 ;;;;;;;;;;;;;;;;;;;;; End functions to handle broken subscriptions
 
+(defn create-editable-table-card!
path: src/metabase/api/dashboard.clj
user: login: crisptrutski
body: Wasn't sure about whether to move this into the dashcard model namespace. Let's see.
created_at: 2025-03-18T21:28:22Z
author_association: CONTRIBUTOR

PR comments:
user: login: trunk-io[bot]
created_at: 2025-03-18T16:10:03Z
body: <!-- Trunk Test Analytics -->


[![Static Badge](https://img.shields.io/badge/96960_tests-2fc1e564-purple)](https://app.trunk.io/metabase/flaky-tests/pr/55345?repo=metabase/metabase&commitHash=2fc1e564f14b12752c42e42811d3acd8db3ea47b&utm_source=gh-comment) &ensp; [![Static Badge](https://img.shields.io/badge/0-failed-crimson)](https://app.trunk.io/metabase/flaky-tests/pr/55345?repo=metabase/metabase&commitHash=2fc1e564f14b12752c42e42811d3acd8db3ea47b&utm_source=gh-comment&runConclusion=FAILURE) &ensp; [![Static Badge](https://img.shields.io/badge/4-flaky-red)](https://app.trunk.io/metabase/flaky-tests/pr/55345?repo=metabase/metabase&commitHash=2fc1e564f14b12752c42e42811d3acd8db3ea47b&utm_source=gh-comment&runConclusion=FAILURE) &ensp; [![Static Badge](https://img.shields.io/badge/0-quarantined-yellow)](https://app.trunk.io/metabase/flaky-tests/pr/55345?repo=metabase/metabase&commitHash=2fc1e564f14b12752c42e42811d3acd8db3ea47b&utm_source=gh-comment&runConclusion=QUARANTINED)


| Flaky Test | Failure Summary | Logs |
| --- | --- | --- |
| <code>identity-hash-test</code> | The calculated hash of a dashboard does not match the expected hash. | [Logs ↗︎](https://app.trunk.io/metabase/flaky-tests/pr/55345?repo=metabase/metabase&commitHash=2fc1e564f14b12752c42e42811d3acd8db3ea47b&utm_source=gh-comment&runConclusion=FAILURE&runId=e0e9c274-deb4-553f-8357-41b2e4c50917) |
| <code>scenarios > visualizations > line/bar chart > tooltips should be enterable and scollable to view all rows in long tooltips (metabase\#53586) (metab...</code> | The test timed out while trying to find an element with data-testid "echarts-tooltip". | [Logs ↗︎](https://app.trunk.io/metabase/flaky-tests/pr/55345?repo=metabase/metabase&commitHash=2fc1e564f14b12752c42e42811d3acd8db3ea47b&utm_source=gh-comment&runConclusion=FAILURE&runId=f0b42483-e636-52c2-ac3d-b98b578ec99a) |
| <code>unsubscribe-receive-email-test</code> | The unsubscribe confirmation email was sent with incorrect body format. | [Logs ↗︎](https://app.trunk.io/metabase/flaky-tests/pr/55345?repo=metabase/metabase&commitHash=2fc1e564f14b12752c42e42811d3acd8db3ea47b&utm_source=gh-comment&runConclusion=FAILURE&runId=df776a1b-bea7-566f-a5b1-4ef3bd680019) |
| <code>e2e-storage-ingestion-test</code> | The expected and actual values are not equal, indicating a mismatch in the data. | [Logs ↗︎](https://app.trunk.io/metabase/flaky-tests/pr/55345?repo=metabase/metabase&commitHash=2fc1e564f14b12752c42e42811d3acd8db3ea47b&utm_source=gh-comment&runConclusion=FAILURE&runId=38d9163d-0e90-5a62-bf32-d6be66cf815c) |
<sub>

[View Full Report ↗︎](https://app.trunk.io/metabase/flaky-tests/pr/55345?repo=metabase/metabase&commitHash=2fc1e564f14b12752c42e42811d3acd8db3ea47b&utm_source=gh-comment) ⋅ [Docs](https://docs.trunk.io/flaky-tests?utm_source=gh-comment)

</sub>
author_association: NONE
user: login: deniskaber
created_at: 2025-03-18T16:52:59Z
body: Yep, I like the idea. Let me play with this code to understand if we can use it as is
author_association: CONTRIBUTOR
user: login: crisptrutski
created_at: 2025-03-18T21:23:21Z
body: Yeah, basically put the config in the same place that text cards put their text, although I'm not sure it's the right place.

If this approach works out well we can look into improving and formalizing the dashcard schema more.
author_association: CONTRIBUTOR
user: login: crisptrutski
created_at: 2025-03-18T21:43:31Z
body: Something important that didn't make it into this PR is to put up some guard rails around the editable grid doing unexpected things used with a unsupported question. This is non-blocking so I didn't think it was worth rushing.

You should be able to get into this state easily, because we don't let you reconfigure questions to use the editable grid visualization, and we only create editable cards with trivial `{:source-table table-id}` queries. Unlike a "simple table" view, the editable dashboard grid component could avoid linking directly to the card page, but a mischievous user could still navigate their manually using the card id, and then edit the query as they please.

What we want in the short term is for the grid to just say "sorry you can't edit this" if the query does anything beyond listing a single table. Then down the line we can relax it to support more and more things, and relax the editing restriction to be done on a column by column basis.

1. Sorry, you can't edit this column because it's calculated.
2. Sorry, you can't edit this column because its from a join and we don't have the full PK for its corresponding row.
3. Sorry, you can't edit this column because it corresponds to multiple rows, because there's an aggregation.
4. etc

Restricting which columns are editable will be table stakes with respect to permissions also, so it makes sense that the BE annotates each column with its status.

For the `table/:id/data` API I ended up using "reducing function function transformer" to avoid touching any QP code, but that was pretty obtuse. Now that the rules of engagement are different on our feature branch, I'm thinking we'll want to do our "decoration" of the metadata with some new QP middleware, but I need to get my head around that.

PS. This makes me think of another low hanging defensive wall we could put up - not allowing navigation to editable questions, just don't let them get to the query builder.

Cut an issue: https://linear.app/metabase/issue/WRK-171/prevent-unexpected-behavior-when-trying-to-edit-complex-questions
author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-03-18T21:44:21Z
body: 
  ## e2e tests failed on `2fc1e564f14b12752c42e42811d3acd8db3ea47b-1`

  [e2e test run](https://github.com/metabase/metabase/actions/runs/13933836573?pr=55345)

  File | Test Name
  ---- | ---------
  `line-bar-tooltips.cy.spec.js` | (_flaky_) scenarios > visualizations > line/bar chart > tooltips > should be enterable and scollable to view all rows in long tooltips (metabase#53586) (metabase#48347)

author_association: CONTRIBUTOR