WIP - unifying some redundant code in the duplicate check generation.
Fixing imports so they are accessible from tests as well as the web application, fixing tests that were not compatible with recent changes.
Standardize sql insert and query handling of input data (that is quoting input or returning an unquoted NULL where needed)
Adding escape function for input to SQL queries. This should not be needed once prepared statements are used.
Adding a threshold for number of duplicate records that prompts a rejection of a sheet before anything is added to the database.
Adding comment to review how the column splitter is handling data.
Integer validator and small interface change TODO review initialization path for passing down system configuration
Handling of new sheet format, described in comment for columns_to_push_into_children field in the record processor. This allows for duplicating data in children, or if also ignoring main record insertion for a line of data to generate a series of child records to link to one existing parent.

TODO - Some commented out debugging stuff left to remove.
Implemented cross column validation. Added max error threshold for sheet.

External data is now passed into the record processor in two forms, one for insertion and one for use in cross column validation.

Cleaned up insertion into upload history to use system MySQL utility functions.

Moved static functions into MySQL_Utilities.

Implemented get_last_val for repeated output, add_values_to_assoc_array now functions as expected.
Proper handling of updated final records.
Fixed issue with repeated columns introduced by the changes to push columns from main records into children.

Allow records to be re-submitted even after the have been added to the final dataset.
I think this is wrong, need to review it.
delete final record when a re-upload results in an error.