Repository: mathesar-foundation/mathesar

Issue:
ID sequence isn't being created for tables
## Description
<!-- A clear and concise description of what the bug is. -->
I cannot add a new record to a new table because the ID does not have a default value. I set up this table by:
1. Importing [library_books.tsv.zip](https://github.com/centerofci/mathesar/files/10273401/library_books.tsv.zip) as "Items"
2. Pulling "Author First Name" and "Author First Name" into an "Authors" table
3. Moving "Author Website" to "Authors" table
4. Pulling "Publishers" into a "Publishers" table
5. Pulling "Title", "Publication Year", "ISBN", "Authors_id", and "Publishers_id" into a "Books" table

## Expected behavior
<!-- A clear and concise description of what you expected to happen. -->
I should be able to add a new record!

## To Reproduce
<!-- How can we recreate this bug? Please try to provide a Minimal, Complete, and Verifiable (http://stackoverflow.com/help/mcve) example if code-related. -->
~Try adding a new record on https://library.mathesar.org/mathesar_tables/4/tables/34/~

I needed to nuke this table since it was blocking my work. To reproduce, follow the steps under Description.

## Additional context
<!-- Add any other context about the problem or screenshots here. -->
This seems related to
- #2116 
- #1356 



Discussion:
Also, after creating a Checkouts table manually, when I tried to run:

`psql -d mathesar -f 02_add_checkouts.sql`

I got

`ERROR:  relation "Library Management.Checkouts_id_seq" does not exist`


As for the second error, that's expected (and unavoidable). It's not possible to combine manually-created tables with the auto-loading SQL, since the names of (in this case) sequences may not be the same and we need to be able to modify those in the auto-loading SQL. In this case, it likely failed when it tried to reset the sequence so that ID numbers of checkouts would work.
The first error (the sequences being incorrect after splitting) is difficult, and I can't believe we've missed it for this long.

It seems that when we split a table, the created new table ("Authors") has a primary key column associated with a sequence to generate integer IDs. The insert logic that adds records to the extracted table doesn't increment the sequence at all, and there's no logic to update the sequence's current value. Thus, when you try to insert another record into such a table, you have a clash since the sequence generates a 'new' id of `1`, which already exists in the table.

We need to add logic to set the sequence associated with the primary key to the maximum value of the table (or max + 1, depending what params SQLAlchemy supports)

Resolving patch:
diff --git a/db/tables/operations/split.py b/db/tables/operations/split.py
--- a/db/tables/operations/split.py
+++ b/db/tables/operations/split.py
@@ -89,6 +89,8 @@ def extract_columns_from_table(old_table_oid, extracted_column_attnums, extracte
             fk_column_name,
         )
         conn.execute(split_ins)
+        update_pk_sequence_to_latest(conn, engine, extracted_table)
+
         remainder_table_oid = get_oid_from_table(remainder_table_with_fk_column.name, schema, engine)
         deletion_column_data = [
             {'attnum': column_attnum, 'delete': True}
@@ -96,3 +98,28 @@ def extract_columns_from_table(old_table_oid, extracted_column_attnums, extracte
         ]
         batch_alter_table_drop_columns(remainder_table_oid, deletion_column_data, conn, engine)
     return extracted_table, remainder_table_with_fk_column, fk_column_name
+
+
+def update_pk_sequence_to_latest(conn, engine, extracted_table):
+    _preparer = engine.dialect.identifier_preparer
+    quoted_table_name = _preparer.quote(extracted_table.schema) + "." + _preparer.quote(extracted_table.name)
+    update_pk_sequence_stmt = func.setval(
+        # `pg_get_serial_sequence needs a string of the Table name
+        func.pg_get_serial_sequence(
+            quoted_table_name,
+            extracted_table.c[constants.ID].name
+        ),
+        # If the table can be empty, start from 1 instead of using Null
+        func.coalesce(
+            func.max(extracted_table.c[constants.ID]) + 1,
+            1
+        ),
+        # Set the sequence to use the last value of the sequence
+        # Setting is_called field to false, meaning that the next nextval will not advance the sequence before returning a value.
+        # We need to do it as our default coalesce value is 1 instead of 0
+        # Refer the postgres docs https://www.postgresql.org/docs/current/functions-sequence.html
+        False
+    )
+    conn.execute(
+        select(update_pk_sequence_stmt)
+    )


Test patch:
diff --git a/db/tests/tables/operations/test_split.py b/db/tests/tables/operations/test_split.py
--- a/db/tests/tables/operations/test_split.py
+++ b/db/tests/tables/operations/test_split.py
@@ -3,6 +3,7 @@
 from db import constants
 from db.columns.defaults import DEFAULT_COLUMNS
 from db.columns.operations.select import get_columns_attnum_from_names
+from db.records.operations.insert import insert_record_or_records
 from db.tables.operations.select import get_oid_from_table, reflect_table
 from db.tables.operations.split import extract_columns_from_table
 from db.metadata import get_empty_metadata
@@ -141,3 +142,10 @@ def test_extract_columns_leaves_correct_data(engine_with_roster, roster_table_na
     with engine.begin() as conn:
         actual_tuples = conn.execute(actual_tuple_sel).fetchall()
     assert sorted(expect_tuples) == sorted(actual_tuples)
+
+
+def test_extract_columns_insert_new_record(engine_with_roster, extracted_remainder_roster, roster_extracted_cols, roster_fkey_col):
+    engine, schema = engine_with_roster
+    extracted, remainder, _, _ = extracted_remainder_roster
+    # Verify if we are able to add new records to the Table. Refer https://github.com/centerofci/mathesar/issues/2144
+    insert_record_or_records(extracted, engine, {"Teacher": "Miyagi", "Teacher Email": "miyagi@karatekid.com"})