Package: textual-fastdatatable
textual-fastdatatableA performance-focused reimplementation of Textual's DataTable widget, with a pluggable data storage backend.Textual's built-in DataTable widget is beautiful and powerful, but it can be slow to load large datasets.Here are some benchmarks on my relatively weak laptop. For each benchmark, we initialize a Textual App that
loads a dataset from a parquet file and mounts a data table; it then scrolls around the table
(10 pagedowns and 15 right arrows).For the built-in table and the others marked "from Records", the data is loaded into memory before the timer
is started; for the "Arrow from Parquet" back-end, the timer is started immediately.The times in each column represent the time to the first paint of the table, and the time after scrolling
is completed (we wait until the table is fully rendered after each scroll):RecordsBuilt-In DataTableFastDataTable (Arrow from Parquet)FastDataTable (Arrow from Records)FastDataTable (Numpy from Records)lap_times_100.parquet0.019s /   1.716s0.012s /   1.724s0.011s /   1.700s0.011s /   1.688slap_times_1000.parquet0.103s /   1.931s0.011s /   1.859s0.011s /   1.799s0.015s /   1.848slap_times_10000.parquet0.977s /   2.824s0.013s /   1.834s0.016s /   1.812s0.078s /   1.869slap_times_100000.parquet11.773s /  13.770s0.025s /   1.790s0.156s /   1.824s0.567s /   2.347slap_times_538121.parquet62.960s /  65.760s0.077s /   1.803s0.379s /   2.234s3.324s /   5.031swide_10000.parquet5.110s /  10.539s0.024s /   3.373s0.042s /   3.278s0.369s /   3.461swide_100000.parquet51.144s /  56.604s0.054s /   3.294s0.429s /   3.642s3.628s /   6.732sNB:FastDataTable currently does not support rows with a height of more than one line. See below for
more limitations, relative to the built-in DataTable.Installationpipinstalltextual-fastdatatableUsageIf you already have data in Apache Arrow or another common table format:fromtextual_fastdatatableimportDataTabledata_table=DataTable(data=my_data)The currently supported types are:AutoBackendType=Union[pa.Table,pa.RecordBatch,Path,# to parquet onlystr,# path to parquet onlySequence[Iterable[Any]],Mapping[str,Sequence[Any]],]To override the column labels and widths supplied by the backend:fromtextual_fastdatatableimportDataTabledata_table=DataTable(data=my_data,column_labels=["Supports","[red]Console[/]","Markup!"],column_widths=[10,5,None])You can also pass in abackendmanually (if you want more control or want to plug in your own).fromtextual_fastdatatableimportArrowBackend,DataTable,create_backendbackend=create_backend(my_data)backend=ArrowBackend(my_arrow_table)# from python dictionary in the form key: col_valuesbackend=ArrowBackend.from_pydict({"col one":[1,2,3,4],"col two":["a","b","c","d"],})# from a list of tuples or another sequence of iterablesbackend=ArrowBackend.from_records([("col one","col two"),(1,"a"),(2,"b"),(3,"c"),(4,"d"),])# from a path to a Parquet file:backend=ArrowBackend.from_parquet("path/to/file.parquet")Limitations and CaveatsTheDataTabledoes not currently support rows with a height of more than one line. Only the first line of each row will be displayed.TheDataTabledoes not currently support row labels.TheArrowBackendis optimized to be fast for large, immutable datasets. Mutating the data,
especially adding or removing rows, may be slow.TheArrowBackendcannot be initialized without data, however, the DataTable can (either with or withoutcolumn_labels).TheArrowBackendcannot store arbitrary Python objects or Rich Renderables as values. It may widen types to strings unnecessarily.Additional FeaturesCopying Data from the Tablectrl+cwill post a SelectionCopied message with a list of tuples of the values selected by the cursor. To use, initialize withcursor_type=rangefrom an app that does NOT inherit bindings.fromtextual.appimportApp,ComposeResultfromtextual_fastdatatableimportArrowBackend,DataTableclassTableApp(App,inherit_bindings=False):BINDINGS=[("ctrl+q","quit","Quit")]defcompose(self)->ComposeResult:backend=ArrowBackend.from_parquet("./tests/data/lap_times_538121.parquet")yieldDataTable(backend=backend,cursor_type="range")if__name__=="__main__":app=TableApp()app.run()Truncating long valuesTheDataTablewill automatically calculate column widths; if you set amax_column_content_widthat initialization, it will truncate any long values at that width; the full value will be visible on hover in a tooltip (and the full value will always be copied to the clipboard).