Repository: innobi/pantab (language: python)

Issue:
Add support for writing INT8 values
From a conversation with @jorwoods it would be nice to allow users to _write_ these values. On read, these will come back as INT16 values since the Hyper API does not support 8 bit integers directly


Discussion / hints:
From a development perspective I think we just need to add `case NANOARROW_TYPE_INT8`: here:

https://github.com/innobi/pantab/blob/51a9d9b9d9cf649d837bdf4f227134b655c82329/src/pantab/writer.cpp#L21

and here:

https://github.com/innobi/pantab/blob/51a9d9b9d9cf649d837bdf4f227134b655c82329/src/pantab/writer.cpp#L359

And adding tests of course

Fix patch:
diff --git a/newsfragments/280.feat b/newsfragments/280.feat
new file mode 100644
index 00000000..0bfc0517
--- /dev/null
+++ b/newsfragments/280.feat
@@ -0,0 +1,3 @@
+Adds support for writing DataFrames/Tables that have 8 bit integers. Hyper does
+not have an 8 bit integer data type, so it will be converted to a 16 bit integer
+upon writing.
diff --git a/src/pantab/writer.cpp b/src/pantab/writer.cpp
index c88b2124..58f592a8 100644
--- a/src/pantab/writer.cpp
+++ b/src/pantab/writer.cpp
@@ -18,6 +18,7 @@ static auto GetHyperTypeFromArrowSchema(struct ArrowSchema *schema,
   }
 
   switch (schema_view.type) {
+  case NANOARROW_TYPE_INT8:
   case NANOARROW_TYPE_INT16:
     return hyperapi::SqlType::smallInt();
   case NANOARROW_TYPE_INT32:
@@ -356,6 +357,7 @@ static auto MakeInsertHelper(std::shared_ptr<hyperapi::Inserter> inserter,
   }
 
   switch (schema_view.type) {
+  case NANOARROW_TYPE_INT8:
   case NANOARROW_TYPE_INT16:
     return std::make_unique<IntegralInsertHelper<int16_t>>(
         inserter, chunk, schema, error, column_position);


Test patch:
diff --git a/tests/test_writer.py b/tests/test_writer.py
index 221eebd2..2ade4076 100644
--- a/tests/test_writer.py
+++ b/tests/test_writer.py
@@ -348,3 +348,22 @@ def test_write_date_bug(tmp_hyper):
         [tab_api.Date(2024, 1, 1)],
         [tab_api.Date(2024, 1, 31)],
     ]
+
+
+def test_eight_bit_int(tmp_hyper):
+    frame = pd.DataFrame(list(range(10)), columns=["nums"]).astype("int8")
+
+    pt.frame_to_hyper(frame, tmp_hyper, table="test")
+
+    with tab_api.HyperProcess(
+        tab_api.Telemetry.DO_NOT_SEND_USAGE_DATA_TO_TABLEAU
+    ) as hyper:
+        with tab_api.Connection(
+            hyper.endpoint, tmp_hyper, tab_api.CreateMode.CREATE_IF_NOT_EXISTS
+        ) as connection:
+            table_def = connection.catalog.get_table_definition(
+                tab_api.TableName("test")
+            )
+            num_col = table_def.get_column_by_name("nums")
+            assert num_col is not None
+            assert num_col.type == tab_api.SqlType.small_int()