Repository: gradio-app/gradio
Pull request: allow data frame to change number of columns
Closes #1198. Closes #1430.

This PR changes the preprocess signature for Dataframe so we have access to the current headers rather than only the initial ones. I'm not sure if there is an impact outside of what I've tested, there may also be some test failures that I may not understand.

Problem statement:
Dataframe with headers throws error when new column is added
### Is there an existing issue for this?

- [X] I have searched the existing issues

### Describe the bug

If a `gr.Dataframe` is created with defined `headers`, then when a new column is created, it throws an error, since the backend tries to apply the original headers to the new dataframe which is of a different size

### Reproduction

Minimum example:

```py
gr.Interface(lambda x:x, 
             gr.Dataframe(headers=["a", "b", "c"]), 
             "dataframe").launch()
```

### Screenshot

_No response_

### Logs

_No response_

### System Info

```shell
gradio==2.9b25
```


### Severity

critical

Dynamic Dataset Size for Dataframe Component
is there any way we can dynamically handle consuming of the dataframe in the background without limiting users to upload a certain size? 

Like when we create the dataframe component we have to give a list to headers which should match the number of columns in a dataset in a CSV/TSV file you're dragging to UI. This is not good for below reasons:
- one thing in tabular workflows is that we have so many feature engineering going on, we add modify datasets by adding one hot encoded features or eliminate features according to shap or importance, it would be reaaaalllly essential to have such feature.
- not so many people know about the number of columns of their dataset in a CSV file beforehand.

maybe ping @pngwn  for this 🙂
thought: We can change the size through UI, so it should be fine for CSV/TSV file read to have.

Hints from issue discussion:
Yesssss let's do this
I think this is a duplicate of #1198

Commit message: fix tests finally

Patch to review:
diff --git a/demo/blocks_plug/run.py b/demo/blocks_plug/run.py
index 97684fa61b..198b5ee8fe 100644
--- a/demo/blocks_plug/run.py
+++ b/demo/blocks_plug/run.py
@@ -1,36 +1,19 @@
 import gradio as gr
 
+import gradio as gr
+import pandas as pd
+import numpy as np
 
-def change_tab():
-    return gr.Tabs.update(selected=2)
-
-
-identity_demo, input_demo, output_demo = gr.Blocks(), gr.Blocks(), gr.Blocks()
-
-with identity_demo:
-    gr.Interface(lambda x: x, "text", "text")
 
-with input_demo:
-    t = gr.Textbox(label="Enter your text here")
-    with gr.Row():
-        btn = gr.Button("Submit")
-        clr = gr.Button("Clear")
-    clr.click(lambda x: "", t, t)
+def return_empty(x):
+    return np.atleast_1d(x.astype(float).mean(axis=0))
 
-with output_demo:
-    gr.Textbox("This is a static output")
 
-with gr.Blocks() as demo:
-    gr.Markdown("Three demos in one!")
-    with gr.Tabs(selected=1) as tabs:
-        with gr.TabItem("Text Identity", id=0):
-            identity_demo.render()
-        with gr.TabItem("Text Input", id=1):
-            input_demo.render()
-        with gr.TabItem("Text Static", id=2):
-            output_demo.render()
-    btn = gr.Button("Change tab")
-    btn.click(inputs=None, outputs=tabs, fn=change_tab)
+demo = gr.Interface(
+    return_empty,
+    gr.Dataframe(type="numpy", headers=["a"]),
+    gr.Dataframe(headers=["mean"]),
+).launch()
 
 if __name__ == "__main__":
     demo.launch()
diff --git a/gradio/components.py b/gradio/components.py
index 1c5afffe26..9b8f1f565f 100644
--- a/gradio/components.py
+++ b/gradio/components.py
@@ -16,7 +16,7 @@
 import warnings
 from copy import deepcopy
 from types import ModuleType
-from typing import Any, Callable, Dict, List, Optional, Tuple
+from typing import Any, Callable, Dict, List, Optional, Tuple, TypedDict
 
 import matplotlib.figure
 import numpy as np
@@ -2389,7 +2389,11 @@ def style(
         )
 
 
-@document()
+class DataframeData(TypedDict):
+    headers: List[str]
+    data: List[List[str | int | bool]]
+
+
 class Dataframe(Changeable, IOComponent):
     """
     Accepts or displays 2D input through a spreadsheet-like component for dataframes.
@@ -2521,26 +2525,22 @@ def update(
         }
         return IOComponent.add_interactive_to_config(updated_config, interactive)
 
-    def preprocess(
-        self, x: List[List[str | Number | bool]]
-    ) -> pd.DataFrame | np.ndarray | List[List[str | float | bool]]:
+    def preprocess(self, x: DataframeData):
         """
         Parameters:
-            x: 2D array of str, numeric, or bool data
+        x (Dict[headers: List[str], data: List[List[str | int | bool]]]): 2D array of str, numeric, or bool data
         Returns:
             Dataframe in requested format
         """
         if self.type == "pandas":
-            if self.headers:
-                return pd.DataFrame(x, columns=self.headers)
+            if x.get("headers") is not None:
+                return pd.DataFrame(x["data"], columns=x.get("headers"))
             else:
-                return pd.DataFrame(x)
-        if self.col_count[0] == 1:
-            x = [row[0] for row in x]
+                return pd.DataFrame(x["data"])
         if self.type == "numpy":
-            return np.array(x)
+            return np.array(x["data"])
         elif self.type == "array":
-            return x
+            return x["data"]
         else:
             raise ValueError(
                 "Unknown type: "
@@ -2587,8 +2587,6 @@ def postprocess(self, y: str | pd.DataFrame | np.ndarray | List[List[str | float
         if isinstance(y, (np.ndarray, list)):
             if isinstance(y, np.ndarray):
                 y = y.tolist()
-            if len(y) == 0 or not isinstance(y[0], list):
-                y = [y]
             return {
                 "data": Dataframe.__process_markdown(y, self.datatype),
             }
diff --git a/test/test_components.py b/test/test_components.py
index 1a1d81d916..a586cd09d6 100644
--- a/test/test_components.py
+++ b/test/test_components.py
@@ -999,7 +999,10 @@ def test_component_functions(self):
         """
         Preprocess, serialize, save_flagged, restore_flagged, generate_sample, get_config
         """
-        x_data = [["Tim", 12, False], ["Jan", 24, True]]
+        x_data = {
+            "data": [["Tim", 12, False], ["Jan", 24, True]],
+            "headers": ["Name", "Age", "Member"],
+        }
         dataframe_input = gr.Dataframe(headers=["Name", "Age", "Member"])
         output = dataframe_input.preprocess(x_data)
         self.assertEqual(output["Age"][1], 24)
@@ -1046,7 +1049,7 @@ def test_component_functions(self):
         )
         dataframe_input = gr.Dataframe()
         output = dataframe_input.preprocess(x_data)
-        self.assertEqual(output[1][1], 24)
+        self.assertEqual(output.iloc[1]["Age"], 24)
         with self.assertRaises(ValueError):
             wrong_type = gr.Dataframe(type="unknown")
             wrong_type.preprocess(x_data)
@@ -1120,13 +1123,13 @@ def test_in_interface_as_input(self):
         """
         Interface, process,
         """
-        x_data = [[1, 2, 3], [4, 5, 6]]
+        x_data = {"data": [[1, 2, 3], [4, 5, 6]]}
         iface = gr.Interface(np.max, "numpy", "number")
         self.assertEqual(iface.process([x_data]), [6])
-        x_data = [["Tim"], ["Jon"], ["Sal"]]
+        x_data = {"data": [["Tim"], ["Jon"], ["Sal"]]}
 
         def get_last(my_list):
-            return my_list[-1]
+            return my_list[-1][-1]
 
         iface = gr.Interface(get_last, "list", "text")
         self.assertEqual(iface.process([x_data]), ["Sal"])
@@ -1140,7 +1143,10 @@ def check_odd(array):
             return array % 2 == 0
 
         iface = gr.Interface(check_odd, "numpy", "numpy")
-        self.assertEqual(iface.process([[2, 3, 4]])[0], {"data": [[True, False, True]]})
+        print(iface.process([{"data": [[2, 3, 4]]}]))
+        self.assertEqual(
+            iface.process([{"data": [[2, 3, 4]]}])[0], {"data": [[True, False, True]]}
+        )
 
 
 class TestVideo(unittest.TestCase):
@@ -1356,14 +1362,16 @@ def test_in_interface_as_output(self):
         """
         timeseries_output = gr.Timeseries(x="time", y=["retail", "food", "other"])
         iface = gr.Interface(lambda x: x, "dataframe", timeseries_output)
-        df = pd.DataFrame(
-            {
-                "time": [1, 2, 3, 4],
-                "retail": [1, 2, 3, 2],
-                "food": [1, 2, 3, 2],
-                "other": [1, 2, 4, 2],
-            }
-        )
+        df = {
+            "data": pd.DataFrame(
+                {
+                    "time": [1, 2, 3, 4],
+                    "retail": [1, 2, 3, 2],
+                    "food": [1, 2, 3, 2],
+                    "other": [1, 2, 4, 2],
+                }
+            )
+        }
         self.assertEqual(
             iface.process([df]),
             [
@@ -1585,6 +1593,7 @@ def test_in_interface(self):
         """
 
         def get_avg_age_per_gender(data):
+            print(data)
             return {
                 "M": int(data[data["gender"] == "M"].mean()),
                 "F": int(data[data["gender"] == "F"].mean()),
@@ -1603,7 +1612,10 @@ def get_avg_age_per_gender(data):
             ["O", 20],
             ["F", 30],
         ]
-        self.assertDictEqual(iface.process([y_data])[0], {"M": 35, "F": 25, "O": 20})
+        self.assertDictEqual(
+            iface.process([{"data": y_data, "headers": ["gender", "age"]}])[0],
+            {"M": 35, "F": 25, "O": 20},
+        )
 
 
 class TestHTML(unittest.TestCase):
diff --git a/ui/packages/app/src/components/DataFrame/DataFrame.svelte b/ui/packages/app/src/components/DataFrame/DataFrame.svelte
index 9f4f87c7fa..0d3509d937 100644
--- a/ui/packages/app/src/components/DataFrame/DataFrame.svelte
+++ b/ui/packages/app/src/components/DataFrame/DataFrame.svelte
@@ -22,18 +22,6 @@
 	export let wrap: boolean;
 	export let datatype: Datatype | Array<Datatype>;
 
-	$: {
-		if (value && !Array.isArray(value)) {
-			if (Array.isArray(value.headers)) headers = value.headers;
-			value =
-				value.data.length === 0 ? [Array(headers.length).fill("")] : value.data;
-		} else if (value === null) {
-			value = [Array(headers.length).fill("")];
-		} else {
-			value = value;
-		}
-	}
-
 	const dispatch = createEventDispatcher();
 
 	export let loading_status: LoadingStatus;
diff --git a/ui/packages/table/src/Table.svelte b/ui/packages/table/src/Table.svelte
index cd91052be2..a2245b9855 100644
--- a/ui/packages/table/src/Table.svelte
+++ b/ui/packages/table/src/Table.svelte
@@ -22,7 +22,23 @@
 	export let style: Styles = {};
 	export let wrap: boolean = false;
 
-	const dispatch = createEventDispatcher<{ change: typeof values }>();
+	$: {
+		if (values && !Array.isArray(values)) {
+			if (Array.isArray(values.headers)) headers = values.headers;
+			values =
+				values.data.length === 0
+					? [Array(headers.length).fill("")]
+					: values.data;
+		} else if (values === null) {
+			values = [Array(headers.length).fill("")];
+		} else {
+			values = values;
+		}
+	}
+
+	const dispatch = createEventDispatcher<{
+		change: { data: typeof values; headers: typeof headers };
+	}>();
 
 	let editing: boolean | string = false;
 	let selected: boolean | string = false;
@@ -114,10 +130,10 @@
 	let old_val: undefined | Array<Array<string | number>> = undefined;
 
 	$: _headers &&
-		dispatch(
-			"change",
-			data.map((r) => r.map(({ value }) => value))
-		);
+		dispatch("change", {
+			data: data.map((r) => r.map(({ value }) => value)),
+			headers: _headers.map((h) => h.value)
+		});
 
 	function get_sort_status(
 		name: string,


Code review comments:
diff_hunk: @@ -1140,7 +1143,10 @@ def check_odd(array):
             return array % 2 == 0
 
         iface = gr.Interface(check_odd, "numpy", "numpy")
-        self.assertEqual(iface.process([[2, 3, 4]])[0], {"data": [[True, False, True]]})
+        print(iface.process([{"data": [[2, 3, 4]]}]))
path: test/test_components.py
text: @user1:
nit: Think we should remove this print statement prior to merge

@author:
ops. I am very forgetful about print/log/debug statements, so please comment these when you see them. I'm trying to be better at cleaning them up before committing but I still miss them sometimes.

diff_hunk: @@ -1,36 +1,19 @@
 import gradio as gr
 
+import gradio as gr
path: demo/blocks_plug/run.py
text: @user1:
nit: We're re-importing gradio here and we're not using pandas

@user1:
Actually, should we move this to another demo? I think `blocks_plug` is a nice demo for how to set named tabs and don't want to lose it unless there's a good reason for it.

@author:
Yeah something weird happened after a merge and I'm not sure what i did when resolving the conflict. I'll check this against main to see what is what and create a new demo for the tab changing if necessary.

@author:
reinstated, no idea what happened.