Repository: agrc/palletjack (language: python)

Issue:
Make sure columns are valid dict keys
When geocoding a DF with street column `Physical Address Street` and zone `Physical Address Zip Code`, the `geocode_dataframe` method raised a KeyError. When I changed the source column names to `street` and `zone`, it worked. Need to check what's going on in the `_asdict()` call to make sure the column names are valid dict keys.
Handle datetime in load.py
FeatureServiceUpdater currently can't handle datetime fields.
Check for empty datetime strings
~`arcgis` can't convert a `pd.NA` value to a datetime value when converting a sdf to feature set. Ideas:~

~- Add another check ala nullable int shapely checker~
~- Convert these to empty strings via fillna? Would empty strings convert to the proper feature set data?~
~- Don't allow the nullable `string` type when checking types for `esriFieldTypeDate`- forcing user to not use this type~

~Which would be easiest for user?~

AGOL can't parse an empty string `''` to a date field.

- It can handle null values in the pandas nullable `string` dtype
- it can handle `None` in an `object` dtype
- it can handle `NaT` in a `datetime[ns]` dtype
Check for projection match, error out if not
Currently naively assumes same projection. Need to reproject data. Also, may need to check transformation to ensure no large shifts, etc.
Handle Postgres connection when running in Cloud Run
This was all working great on my local machine but failed to connect to the database when I pushed it as a cloud function. This PR branches the engine creation method based on the runtime environment.

Ref: https://cloud.google.com/sql/docs/postgres/connect-functions


Discussion / hints:

Fix patch:
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
new file mode 100644
index 0000000..9cf478f
--- /dev/null
+++ b/.github/workflows/release.yml
@@ -0,0 +1,38 @@
+name: Release Events
+on:
+  release:
+    types: [published]
+
+env:
+  GH_TOKEN: ${{ github.token }}
+
+jobs:
+  publish:
+    name: Build and Publish
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: ⬇️ Set up code
+        uses: actions/checkout@v3
+
+      - name: 🐍 Set up Python
+        uses: actions/setup-python@v3
+        with:
+          python-version: 3.9
+
+      - name: 👊 Bump version
+        run: |
+          TAG=$(gh release view --json tagName --jq ".tagName")
+          sed -i "s/__version__ = .*$/__version__ = '$TAG'/" src/palletjack/version.py
+
+      - name: 🛞️ Install wheel
+        run: pip install wheel
+
+      - name: 📦 Build package
+        run: python setup.py sdist bdist_wheel
+
+      - name: 🚀 Publish to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1
+        with:
+          user: __token__
+          password: ${{ secrets.PYPI_API_TOKEN }}
diff --git a/.gitignore b/.gitignore
index 0b864a9..cbd2736 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,4 +1,7 @@
 
+#: testing boilerplate
+_boilerplate.py
+
 #: Secrets file
 secrets.py
 known_hosts
diff --git a/.vscode/settings.json b/.vscode/settings.json
index ed906ef..5ee5d8f 100644
--- a/.vscode/settings.json
+++ b/.vscode/settings.json
@@ -1,33 +1,48 @@
 {
   "cSpell.words": [
     "AGRC",
+    "authed",
+    "auths",
+    "ceildiv",
     "cloudsql",
     "Codecov",
+    "concatted",
+    "Concatting",
     "Dataframe",
+    "DataFrame",
     "drivername",
     "Dtype",
     "featurelayer",
     "featureset",
     "fsupdater",
+    "geocheck",
+    "Geocoded",
     "geodataframe",
+    "geojsons",
     "geopandas",
     "gsheet",
+    "HASARCPY",
     "instafail",
     "isort",
     "knownhosts",
-    "lowres",
-    "naturalearth",
+    "minval",
     "Overwriter",
     "palletjack",
     "PARENTOBJECTID",
     "PGSQL",
+    "Polyline",
     "psycopg",
     "pylint",
     "pytest",
     "reclassifier",
+    "sedf",
     "sftploader",
-    "sqlalchemy",
+    "subsetted",
+    "subsetting",
+    "trycount",
     "unclassed",
+    "upserting",
+    "upserts",
     "yapf"
   ],
   "coverage-gutters.highlightdark": "rgb(61, 153, 112, .05)",
diff --git a/README.md b/README.md
index a66e989..e60629c 100644
--- a/README.md
+++ b/README.md
@@ -3,57 +3,68 @@
 ![Build Status](https://github.com/agrc/palletjack/workflows/Build%20and%20Test/badge.svg)
 [![codecov](https://codecov.io/gh/agrc/palletjack/branch/main/graph/badge.svg)](https://codecov.io/gh/agrc/palletjack)
 
-A library of classes for automatically updating AGOL feature services with data from external sources. Client apps (often called 'skids') can reuse these classes for common use cases. These classes handle different parts of the Extract and Load steps in the ETL process.
+A library of classes and methods for automatically updating AGOL feature services with data from several different types of external sources. Client apps (sometimes called 'skids') can reuse these classes for common use cases. The code modules are oriented around each step in the extract, transform, and load process.
 
-`palletjack` works with pandas DataFrames (either regular for tabular data or Esri's spatially-enabled dataframes for spatial data). Most methods either return a dataframe or use a dataframe for their source data.
+`palletjack` works with pandas DataFrames (either regular for tabular data or Esri's spatially-enabled dataframes for spatial data). The extract and transform methods return dataframes and the load methods consume dataframes as their source data.
 
-See `docs/api.md` for documentation on the available classes and methods, logging, and errors.
-
-See `docs/examples.py` for (bare-bones) example code implementing the various classes/methods. You can also search our GitHub organization for "skid" repositories that use palletjack.
+The [documentation](https://agrc.github.io/palletjack/palletjack) includes a user guide along with an API description of the available classes and methods.
 
 Pallet jack: [forklift's](https://www.github.com/agrc/forklift) little brother.
 
+## Dependencies
+
+`palletjack` relies on the dependencies listed in `setup.py`. These are all available on PyPI and can be installed in most environments, including Google Cloud Functions.
+
+The `arcgis` library does all the heavy lifting for spatial data. If the `arcpy` library is not available (such as in a cloud function), it relies on `shapely` for its geometry engine.
+
 ## Installation
 
 1. Activate your application's environment
 1. `pip install ugrc-palletjack`
 
-## Dependencies
+## Quick start
 
-`palletjack` relies on `setup.py` to install `pandas`, `numpy`, `pysftp`, and `arcgis`. `FeatureServiceInlineUpdater.update_existing_features_in_feature_service_with_arcpy()` also relies on having arcpy installed through either ArcGIS Pro or ArcGIS Enterprise.
-
-## Usage
-
-1. `import palletjack`
-1. Instantiate one or more of the classes as needed.
-1. Call the methods on your instantiated objects to perform the specific action desired.
+1. Import the desired modules
+1. Use a class in `extract` to load a dataframe from an external source
+1. Transform your dataframe as desired with helper methods from `transform`
+1. Use the dataframe to update a hosted feature service using the methods in `load`
 
    ```python
-   loader = palletjack.SFTPLoader(secrets, download_dir)
-   files_downloaded = loader.download_sftp_files(sftp_folder=secrets.SFTP_FOLDER)
-   dataframe = loader.read_csv_into_dataframe('data.csv', secrets.DATA_TYPES)
-
-   updater = FeatureServiceInlineUpdater(gis, dataframe, secrets.KEY_COLUMN)
-   rows_updated = updater.update_existing_features_in_hosted_feature_layer(
-      secrets.FEATURE_LAYER_ITEMID, list(secrets.DATA_TYPES.keys())
-    )
+   from palletjack import extract, transform, load
+
+   #: Load the data from a Google Sheet
+   gsheet_extractor = extract.GSheetLoader(path_to_service_account_json)
+   sheet_df = gsheet_extractor.load_specific_worksheet_into_dataframe(sheet_id, 'title of desired sheet', by_title=True)
+
+   #: Convert the data to points using lat/long fields, clean for uploading
+   spatial_df = pd.DataFrame.spatial.from_xy(input_df, x_column='longitude', y_column='latitude')
+   renamed_df = transform.DataCleaning.rename_dataframe_columns_for_agol(spatial_df)
+   cleaned_df = transform.DataCleaning.switch_to_nullable_int(renamed_df, ['an_int_field_with_null_values'])
+
+   #: Truncate the existing feature service data and load the new data
+   gis = arcgis.gis.GIS('my_agol_org_url', 'username', 'super-duper-secure-password')
+   updates = palletjack.load.FeatureServiceUpdater.truncate_and_load_features(
+      gis, 'feature_service_item_id', cleaned_df, r'c:\directory\to\save\truncated\data\in\case\of\error'
+   )
    ```
 
 ## Development
 
-1. Create a conda environment with arcpy, arcgis
-   - `conda create -n palletjack`
+1. Create a conda environment with Python 3.9
+   - `conda create -n palletjack python=3.9`
    - `activate palletjack`
-   - `conda install arcgis arcpy -c esri`
 1. Clone the repo
-1. Install in dev mode
+1. Install in dev mode with development dependencies
    - `pip install -e .[tests]`
 
-### Updating pypi
+### Troubleshooting Weird Append Errors
+
+If a `FeatureLayer.append()` call (within a load.FeatureServiceUpdater method) fails with an "Unknown Error: 500" error or something like that, you can query the results to get more info. The debug log will include the HTTP GET call, something like the following:
+`https://services1.arcgis.com:443 POST /<unique string>/arcgis/rest/services/<feature layer name>/FeatureServer/<layer id>/append/jobs/<job guid>?f=json token=<crazy long token string>`
+
+You can use this and a token from an AGOL tab to build a new job status url. To get the token, log into AGOL in a browser and open a private hosted feature layer item. Click the layer, and then open the developer console. With the Network tab of the console open, click on the "View" link for the service URL. You should see a document in the list whose name includes "?token=<really long token string>". Copy the name and then copy out the token string.
+
+Now that you've got the token string, you can build the status query:
+`https://services1.arcgis.com/<unique string>/arcgis/rest/services/<feature layer name>/FeatureServer/<layer id>/append/jobs/<job guid>?f=json&<token from agol>`
 
-1. Delete everything in dist/
-1. Make sure you've updated the version number in setup.py
-1. Recreate the wheels:
-   - python setup.py sdist bdist_wheel
-1. Re-upload the new files
-   - twine upload dist/*
+Calling this URL in a browser should return a message that will hopefully give you more info as to why it failed.
diff --git a/docs/README.md b/docs/README.md
new file mode 100644
index 0000000..cbea4bd
--- /dev/null
+++ b/docs/README.md
@@ -0,0 +1,98 @@
+# Overview
+
+`palletjack` is a library of modules for updating data in ArcGIS Online (AGOL) based on information from external data. It handles the repetitive parts of the extract, transform, and load stages of the ETL process, allowing the user to focus on the custom transform steps unique to every project.
+
+As a library, palletjack is not meant to be used by itself. Rather, its classes and methods should be used by other apps written for specific use cases. These other apps are referred to as the "client" apps, and internally we often refer to them as "skids".
+
+pandas dataframes are the main unifying data structure between the different steps. The client loads data from an external source into a dataframe and then modifies the dataframe according to their business needs. Once the dataframe is ready to go, it is used to update the hosted feature service in AGOL.
+
+## Organization
+
+The individual modules within palletjack each handle their own step of the ETL process. Each module contains classes for accomplishing its task organized by source, operation, or destination. There may be multiple, similar methods in a class depending on exactly how you want to perform a given step—you probably won't use all the available classes and methods in every application. The publicly-exposed methods usually call several private methods to keep functions small and testable.
+
+Classes in `extract` handle the extract stage, pulling data in from external sources. Tabular data (csvs, Google Sheets) are loaded into dataframes, while non-tabular data for attachments are just downloaded to the specified locations. You'll instantiate the desired class with the basic connection info and then call the appropriate method on the resulting object to extract the data.
+
+There are a handful of classes in `transform` with methods for cleaning and preparing your dataframes for upload to AGOL. You may also need to modify your data to fit your specific business needs: calculating fields, renaming fields, performing quality checks, etc. Some classes only have static methods can be called directly without needing to instantiate the class.
+
+Once your dataframe is looking pretty, the `load` module will help you update a hosted feature service with your new data. The `FeatureServiceUpdater` class contains several class methods that handle the instantiation process for you, allowing you to make a single method call. The other classes in `load` require you to instantiate the class yourself.
+
+While many parts of the classes' functionality are hidden in private methods, commonly-used code is exposed publicly in the `utils` module. You will probably not need any of the methods provided, but they may be useful for other projects. This is palletjack's junk drawer.
+
+## Data Considerations
+
+Under the hood, palletjack uses the `arcgis.features.FeatureLayer.append()` method to upload data. To eliminate the dependency on `arcpy` (and thus ArcGIS Pro/Enterprise), it converts and uploads the data as a geojson. This conversion process introduces several constraints and gotchas on the format of the data. palletjack tests for all the known gotchas and raises an error if the data needs extra work before uploading. In addition, AGOL imposes its own set of constraints.
+
+### Field Names
+
+The column names in your dataframes should match the field names in AGOL one-to-one, with the exception of AGOL's auto-generated fields (shape length/area, editor tracking, etc). You can use `transform.DataCleaning.rename_dataframe_columns_for_agol()` to handle the majority of field name formatting, but you'll want to double check the results.
+
+### Field Types
+
+The upload process is very particular about data types and missing data. `utils.FieldChecker.check_live_and_new_field_types_match()` contains a mapping of dataframe dtypes to Esri field types. They generally follow what you would expect. However, because pandas (currently) handles missing data by with `np.nan` by default, you may have integer data assigned a float dtype. In addition, some sources render missing data as an empty string, creating an object dtype. Finally, datetimes must be in UTC and stored in the non-timezone-aware `datetime64[ns]` dtype.
+
+`transform.DataCleaning` has methods to help convert your data to these dtypes. In addition, it's a good practice to use pandas' nullable dtypes via [`pd.DataFrame.convert_dtypes()`](https://pandas.pydata.org/pandas-docs/dev/reference/api/pandas.DataFrame.convert_dtypes.html) (see also the section on [nullable ints](https://pandas.pydata.org/pandas-docs/dev/user_guide/integer_na.html)).
+
+### Geometries
+
+palletjack uses Esri's spatially-enabled dataframes for handling geometries. However, there's no reason you couldn't use geodataframes for other parts of the process and convert them to spatially-enabled dataframes for use in the `load` methods.
+
+Because the upload process uses geojsons, you **MUST** project your dataframe to WGS84 (wkid 4326) (the upload stage will error out if it isn't). If your hosted feature service is in a different projection, AGOL will automatically project it back as part of the upload process.
+
+**WARNING**: The reprojection process may introduce spatial shifts/topological errors due to projection differences. We highly recommend your target hosted feature services use WGS84 from the beginning, and that you choose an appropriate transformation when projecting your data.
+
+### OBJECTID and Join Keys
+
+If you want to update existing data without truncating and loading, you will need a join key between the incoming new data and the existing AGOL data. Do not use OBJECTID for this field; it may change at any time. Instead, use your own custom field that you have complete control over. You will perform the join manually in the transform step with pandas by loading the live AGOL data into a dataframe, joining the new data into the live data, and then passing the resulting dataframe to `palletjack.load.FeatureServiceUpdater.update_features`. This method uses the live data's OBJECTID to apply the edits to the proper rows.
+
+## Error handling
+
+The client is responsible for handling errors and warnings that arise during the process. palletjack will raise its own errors when something occurs that would keep the process from continuing or when one of its internal data checks fails. It will also captured and [chain](https://docs.python.org/3/tutorial/errors.html#exception-chaining) errors from the underlying libraries to include additional, context-specific messages for the user. It will raise warnings when something happens that the client should be aware of but will not keep the process from completing.
+
+## Logging
+
+palletjack takes full advantage of python's built-in [`logging`](https://docs.python.org/3/howto/logging.html#advanced-logging-tutorial) library to perform its feedback and reporting. It employs a [hierarchical structure](https://stackoverflow.com/a/50751987), creating module-level loggers for each module and then each class in a module creates their own child logger. This allows rapid identification of where log events are occurring.
+
+### Accessing palletjack logs
+
+The client can get a reference to the palletjack logger and add their handlers, formatters, etc to it alongside its own logger:
+
+```python
+myapp_logger = logging.getLogger('my_app')
+myapp_logger.setLevel(logging.INFO)
+palletjack_logger = logging.getLogger('palletjack')
+palletjack_logger.setLevel(logging.INFO)
+#: set up handlers and formatters
+#: ...
+myapp_logger.addHandler(log_handler)
+palletjack_logger.addHandler(log_handler)
+```
+
+### Log levels
+
+- `logging.DEBUG` includes verbose debug info that should allow you to manually (possibly programmatically) undo or redo any operation.
+- `logging.INFO` includes standard runtime progress reports and result information.
+- `logging.WARNING` includes negative results from checks or other situations that the user should be aware of.
+
+## Updating from v2 to v3
+
+palletjack v3 has several changes from the previous version that users will need to consider when updating existing clients. Version 3 is designed to align with each step in the ETL process and to better follow the single-responsibility principle.
+
+### Namespace Changes
+
+The largest change is that the namespace has been refactored to match the ETL steps. `loaders.py` has been changed to `extract.py` and `updaters.py` has been changed to `load.py`. This eliminates the confusion created by "loaders" being used in the ETL "extract" stage.
+
+As a corollary to this, clients now import each module rather than palletjack exposing the classes directly. The recommended import is `from palletjack import extract, transform, load, utils` (omitting unused modules as necessary).
+
+Version 3 also introduces the use of class methods to take care of object instantiation for the client. These are used the most in `palletjack.load.FeatureServiceUpdater`, where the client just calls the relevant methods.
+
+### One Step at a Time
+
+As part of the refactor, each method generally only tries to do one thing: extract one piece of data, clean one common data error, or perform one load operation.
+
+Previous versions focused on being able to call a single method to do everything. This quickly got unwieldly and led to a lot of complexity trying to match the complexity of real-world data.
+
+Version 3 load methods expect the client to have already cleaned the data and made it ready for uploading. Many of the cleaning steps that were coupled to the load methods in version 2 have been refactored to the `palletjack.transform` module.
+
+### .append instead of .edit_features
+
+Under the hood, version 3 has completely replaced `FeatureLayer.edit_features()` with `FeatureLayer.append()` based on the recommendation in the ArcGIS API for Python docs. This has a couple ramifications for the client. First, in order to avoid the arcpy dependency, all data are converted to geojson for upload. This requires the client to project the dataframes to WGS84/wkid 4326 prior to updating the feature service. Secondly, the client must separate out add, update, and delete operations into individual method calls.
diff --git a/docs/api.md b/docs/api.md
deleted file mode 100644
index 1e90c8a..0000000
--- a/docs/api.md
+++ /dev/null
@@ -1,164 +0,0 @@
-# Overview
-
-`palletjack` is a container package with multiple classes for updating data in AGOL based on information from external data. It handles the repetitive parts of the "extract" and "load" stages of the ETL process, allowing the user to focus on the custom transform steps unique to every project.
-
-`palletjack` is not meant to be used by itself. Rather, its classes should be instantiated and used by other apps written for specific use cases. These other apps are referred to as "skids" or "client" apps.
-
-Each class handles one step of the process and has public methods for accomplishing its task. There may be multiple, similar methods depending on exactly how you want to accomplish the task, or different methods for different types of data input or output - you probably won't use all the available methods in every application. These public methods usually call several private methods to keep functions small and testable.
-
-pandas dataframes are the main unifying data structure between the different steps. "Loaders" handle the extract stage, pulling data in from external sources. Tabular data (csvs, Google Sheets) are loaded into dataframes, while non-tabular data are just downloaded to the specified locations.
-
-Feature-service related "updaters" handle the ETL load stage for data headed to AGOL. After a skid transforms the dataframes it got from the loaders, the updaters use the dataframe as the input for modifying AGOL data.
-
-Some loaders and updaters don't nicely fit in these patterns, but they still match the general idea of "downloading data" from an external source and then "updating/modifying" something on the AGOl side.
-
-## Loaders
-
-### SFTPLoader
-
-The `SFTPLoader` class allows you to pull data from an SFTP share and load it into a pandas dataframe.
-
-The initializer takes five arguments:
-
-1. `host`: sftp hostname or IP address
-1. `username`: sftp username
-1. `password`: sftp password
-1. `knownhosts_file`: path to a [knownhosts file](https://stackoverflow.com/questions/38939454/verify-host-key-with-pysftp) containing the public key of your sftp server
-1. `download_dir`: the local directory to save the files in
-
-#### Methods
-
-- `download_sftp_folder_contents(self, sftp_folder='upload')`
-  - Downloads all the files in `sftp_folder` to `self.download_dir`. Returns the number of files downloaded.
-- `download_sftp_single_file(self, filename, sftp_folder='upload')`
-  - Downloads `filename` from `sftp_folder` to `self.download_dir`. Returns `Path` object to the downloaded file.
-- `read_csv_into_dataframe(self, filename, column_types=None)`
-  - Reads `filename` from `self.download_dir` into a pandas dataframe, using optional `column_types` to define the column names and types. Returns the dataframe.
-  - `column_types` is a dict of format `{'column_name': <type>}`, where `<type>` is `np.float64`, `str`, etc.
-
-### GSheetLoader
-
-`GSheetLoader` allows you to import some or all of the worksheets within a Google Sheets document into a pandas dataframe.
-
-The initializer requires either the path to a service account .json file that has access to the sheet in question or a `google.auth.credentials.Credentials` object. The service account and file may need to be created by the GCP gurus. The Calling `google.auth.default()` in a Cloud Function environment will give you a tuple of the `Credentials` object of the current project and the project id. You can use this `Credentials` object to authorize pygsheets for the same account as the Cloud Function.
-
-#### Methods
-
-- `load_specific_worksheet_into_dataframe(self, sheet_id, worksheet, by_title=False)`
-  - Load a single worksheet from sheet_id into a dataframe, specified either with a 0-based index or the worksheet title.
-- `load_all_worksheets_into_dataframes(self, sheet_id)`
-  - Loads all the worksheets in sheet_id into a dictionary of dataframes, where the keys are the worksheet titles.
-- `combine_worksheets_into_single_dataframe(self, worksheet_dfs)`
-  - Attempts to merge all the worksheets in a dictionary (from `load_all_worksheets_into_dataframes()`) into a single dataframe, adding a 'worksheet' column to identify the worksheet each row came from/belongs to. Will raise an error if all the dataframes don't have the same columns.
-
-### GoogleDriveDownloader
-
-`GoogleDriveDownloader` provides methods to download any non-html file (ie, Content-Type != text/html) Google Drive file from it's sharing link. The files may be publicly shared or shared with a service account.
-
-The initializer sets the output directory for saving the files. The `out_dir` attribute can be modified at any time to change this destination.
-
-This class has two similar sets of methods. The `*_using_api` methods authenticate to the Google API using either a service account file or a `google.auth.credentials.Credentials` object and downloads using the API. If at all possible, use these methods to have the least likelihood of errors. They support publicly shared files along with files just shared to the service account.
-
-The other methods just use an anonymous HTTP request, which requires the files to be shared publicly. Google will block you from downloading after a certain amount of time/requests. A pause has been added to overcome this, but I've yet to find a good value for it.
-
-#### Methods
-
-- `download_file_from_google_drive_using_api(self, gsheets_client, sharing_link, join_id)`
-  - Download a file to the out_dir set on the instantiated object using the Google API. `gsheets_client` is the authenticated Client object from `pygsheets.authorize()`. `sharing_link` should be in the form `https://drive.google.com/file/d/big_long_id/etc`. `join_id` is used for logging purposes to identify which attachment is being worked on. Will log an error if the URL doesn't match this pattern or it can't extract the unique id from the sharing URL. Will also log an error if the header's Content-Type is text/html, which usually indicates the HTTP response was an error message instead of the file.
-- `download_attachments_from_dataframe_using_api(self, credentials, dataframe, sharing_link_column, join_id_column, output_path_column)`
-  - Calls `download_file_from_google_drive_using_api` for every row in `dataframe`, saving the full path of the resulting file in `output_path_column`. Uses `credentials` to authenticate to the Google API using `pygsheets.authorize()`
-- `download_file_from_google_drive(self, sharing_link, join_id, pause=0.)`
-  - Download a file to the out_dir set on the instantiated object using an anonymous HTTP request. `sharing_link` should be in the form `https://drive.google.com/file/d/big_long_id/etc`. `join_id` is used for logging purposes to identify which attachment is being worked on. `pause` specifies a sleep in seconds before downloading to try to get around Google's blocking. Will log an error if the URL doesn't match this pattern or it can't extract the unique id from the sharing URL. Will also log an error if the header's Content-Type is text/html, which usually indicates the HTTP response was an error message instead of the file.
-- `download_attachments_from_dataframe(self, dataframe, sharing_link_column, join_id_column, output_path_column)`
-  - Calls `download_file_from_google_drive` for every row in `dataframe`, saving the full path of the resulting file in `output_path_column`
-
-## Updaters
-
-### FeatureServiceInlineUpdater
-
-Updates the attributes of existing features in an AGOL Hosted Feature Service using either the `arcpy` or `arcgis` libraries. The `arcgis` library is required for interacting with AGOL.
-
-The initializer has three required arguments and one optional argument:
-
-1. `gis`: An `arcgis.gis.GIS` object representing your AGOL organization.
-1. `dataframe`: A pandas dataframe containing the new data where each row is a separate record.
-1. `index_column`: The index column that is present in both the existing and new data for joining the datasets.
-1. `field_mapping`(optional): A dictionary of existing field names to new field names for matching the new data to the existing data in AGOL.
-
-#### Methods
-
-- `update_existing_features_in_feature_service_with_arcpy(self, feature_service_url, fields)`
-  - Update `fields` in `feature_service_url` with data from `self.new_dataframe()` using `arcpy.da.UpdateCursor`. Requires arcpy to be installed. Returns the number of rows updated.
-- `update_existing_features_in_hosted_feature_layer(self, feature_layer_itemid, fields)`
-  - update `fields` in `self.gis`'s `feature_layer_itemid` item with data from `self.new_dataframe()` using `arcgis.feature.FeatureLayer.edit_features()`. Returns the number of rows successfully updated. If any updates fail, it tries to roll back all successful updates.
-- `upsert_new_data_in_hosted_feature_layer(self, feature_service_item_id, layer_index=0)`
-  - UPdate existing data and inSERT new data into feature service referenced by `feature_service_item_id` (at `layer_index` within the feature service) with data from `self.new_dataframe()` using `arcgis.features.FeatureLayer.append()`. The new data must not include any fields not present in the live data. The index column in the hosted feature service defined by `self.index_column` must be marked as 'Unique' in ArcGIS Online.
-    - You can't set the field as unique if it was specified as the Display Field when it was uploaded from Pro (layer properties -> Display -> Display Field). You can change this value and overwrite the feature service to change the display field. There may be a way to do it without re-writing the service, but I've not found it yet.
-    - I've had trouble specifying string fields as unique. You may have better luck using a numeric field as `self.index_column`.
-
-### FeatureServiceOverwriter
-
-Completely overwrite a feature service rather than updating it feature-by-feature.
-
-The initializer requires the `arcgis.gis.GIS` object representing your organization.
-
-#### Methods
-
-- `truncate_and_load_feature_service(self, feature_service_item_id, new_dataframe, failsafe_dir, layer_index=0)`
-  - Attempts to delete existing data from a feature layer and add new data from a spatially-enabled dataframe. First attempts to truncate existing data and loads it in memory as a dataframe. Then renames new data column names to conform to AGOL scheme (spaces, special chars changed to '_'). Finally attempts to append new data to now-empty feature layer using `arcgis.features.FeatureLayer.append()`. If the new data append fails, it attempts to re-upload the previous data from the in-memory dataframe. If this fails, it attempts to failsafe by writing the old data to disk as a json file.
-
-### FeatureServiceAttachmentsUpdater
-
-Updates the attachments on a hosted feature service based on a dataframe containing two columns: a join key present in the live data (the dataframe column name must match the feature service field name) and the file to attach to the feature. While AGOL supports multiple attachments, this only accepts a single file as input.
-
-If a matching feature in AGOl doesn't have an attachment, the file referred to by the dataframe will be uploaded. If it does have an attachment, it checks the existing filename with the referenced file. If they are different, the file from the dataframe will be updated. If they are the same, nothing happens.
-
-The initializer requires the `arcgis.gis.GIS` object representing your organization.
-
-#### Methods
-
-- `update_attachments(self, feature_layer_itemid, attachment_join_field, attachment_path_field, attachments_df, layer_number=0)`
-  - Updates the attachments on layer `layer_number` of `feature_layer_itemid`. Requires a dataframe containing one row for each feature to have it's attachments updated. This dataframe must have an `attachment_join_field` containing the unique key linking the live data to the dataframe and an `attachment_path_field` containing the path to the new attachment.
-
-### ColorRampReclassifier
-
-Manually edits a webmap's JSON definition to change a layer's color ramp values based on a simple unclassed scheme similar to AGOL's unclassed ramp. The minimum value is the dataset minimum, the max is the mean value plus one standard deviation.
-
-The initializer only takes two arguments:
-
-1. `webmap_item`: The `arcgis.gis.Item` object of the webmap to update.
-1. `gis`: The `arcgis.gis.GIS` object of the AGOL organization.
-
-#### Methods
-
-- `update_color_ramp_values(self, layer_name, column_name, stops=5)`
-  - Updates the color ramp stop values for `column_name` of `layer_name` in `self.webmap_item` using `stops` number of stops. Returns a boolean indicating success (passed through from `self.webmap_item.update()` call).
-
-## Logging
-
-`palletjack` takes full advantage of python's built-in `logging` library to perform its feedback and reporting. Each class creates it's own child logger under the main `palletjack` logger, becoming `palletjack.SFTPLoader`, etc.
-
-### Accessing `palletjack` logs
-
-Client programs can get a reference to the `palletjack` logger and add their handlers, formatters, etc to it alongside the client's logger:
-
-```python
-myapp_logger = logging.getLogger('my_app')
-myapp_logger.setLevel(secrets.LOG_LEVEL)
-palletjack_logger = logging.getLogger('palletjack')
-palletjack_logger.setLevel(secrets.LOG_LEVEL)
-#: set up handlers and formatters
-#: ...
-myapp_logger.addHandler(log_handler)
-palletjack_logger.addHandler(log_handler)
-```
-
-### Log levels
-
-- `logging.DEBUG` includes verbose debug info that should allow you to manually (possibly programmatically) undo or redo any operation.
-- `logging.INFO` includes standard runtime progress reports and result information.
-- `logging.WARNING` includes negative results from checks or other situations that the user should be aware of.
-
-## Exception Handling
-
-Some exceptions are captured and [chained](https://docs.python.org/3/tutorial/errors.html#exception-chaining) to include additional, context-specific messages for the user. The client is responsible for handling all re-raised exceptions and any other errors.
diff --git a/docs/examples.py b/docs/examples.py
index b847553..c1c25e6 100644
--- a/docs/examples.py
+++ b/docs/examples.py
@@ -2,11 +2,10 @@
 from tempfile import TemporaryDirectory
 
 import arcgis
+import pandas as pd
+from arcgis.features import GeoAccessor, GeoSeriesAccessor
 
-from palletjack import (
-    ColorRampReclassifier, FeatureServiceAttachmentsUpdater, FeatureServiceInlineUpdater, GoogleDriveDownloader,
-    GSheetLoader, SFTPLoader
-)
+from palletjack import extract, load, transform, utils
 
 
 def load_google_sheet_then_download_and_update_attachments():
@@ -26,12 +25,12 @@ def load_google_sheet_then_download_and_update_attachments():
     feature_layer_itemid = 'agol_item_id'
 
     #: Use a GSheetLoader to load a google sheet into a dictionary of dataframes
-    gsheetloader = GSheetLoader(service_account_json)
+    gsheetloader = extract.GSheetLoader(service_account_json)
     worksheets = gsheetloader.load_all_worksheets_into_dataframes(sheet_id)
 
     #: Use a GoogleDriveDownloader to download all the pictures from a single worksheet dataframe
     uorg_2021_dataframe = worksheets['2021']
-    downloader = GoogleDriveDownloader(out_dir)
+    downloader = extract.GoogleDriveDownloader(out_dir)
     downloader.download_attachments_from_dataframe(
         uorg_2021_dataframe, attachment_column, attachments_join_field, 'full_file_path'
     )
@@ -50,7 +49,7 @@ def load_google_sheet_then_download_and_update_attachments():
     gis = arcgis.gis.GIS(agol_org, agol_user, agol_password)
 
     #: Create our attachment updater and update attachments using the attachments dataframe
-    attachment_updater = FeatureServiceAttachmentsUpdater(gis)
+    attachment_updater = load.FeatureServiceAttachmentsUpdater(gis)
     attachment_updater.update_attachments(
         feature_layer_itemid, attachments_join_field, 'full_file_path', attachments_dataframe
     )
@@ -94,20 +93,25 @@ def download_from_sftp_update_agol_reclassify_map():
     gis = arcgis.gis.GIS(agol_org, agol_user, agol_password)
     webmap_item = gis.content.get(map_item_id)  # pylint:disable=no-member
 
-    #: Load the latest data from FTP
-    sftp_loader = SFTPLoader(sftp_host, sftp_username, sftp_password, local_knownhosts_path, tempdir_path)
+    #:Extract the latest data from FTP
+    sftp_loader = extract.SFTPLoader(sftp_host, sftp_username, sftp_password, local_knownhosts_path, tempdir_path)
     number_of_files_downloaded = sftp_loader.download_sftp_folder_contents(sftp_folder=sftp_folder)
     dataframe = sftp_loader.read_csv_into_dataframe(sftp_filename, csv_datatypes)
 
+    #: Load the live data and merge the updates
+    live_df = pd.DataFrame.spatial.from_featurelayer(
+        arcgis.Features.FeatureLayer.fromitem(gis.content.get(feature_layer_itemid))
+    )
+    update_df = transform.FeatureServiceMerging.update_live_data_with_new_data(live_df, dataframe, join_key_column)
+
     #: Update the AGOL data
-    erap_updater = FeatureServiceInlineUpdater(gis, dataframe, join_key_column)
-    number_of_rows_updated = erap_updater.update_existing_features_in_hosted_feature_layer(
-        feature_layer_itemid, list(csv_datatypes.keys())
+    number_of_rows_updated = load.FeatureServiceUpdater.update_features(
+        gis, feature_layer_itemid, update_df, update_geometry=False
     )
 
     #: Reclassify the break values on the webmap's color ramp
-    erap_reclassifier = ColorRampReclassifier(webmap_item, gis)
-    success = erap_reclassifier.update_color_ramp_values(map_layer_name, classification_field)
+    reclassifier = load.ColorRampReclassifier(webmap_item, gis)
+    success = reclassifier.update_color_ramp_values(map_layer_name, classification_field)
 
     #: update returns either True or False to denote success or failure
     reclassifier_result = 'Success'
diff --git a/pylintrc b/pylintrc
index fa32c44..f595654 100644
--- a/pylintrc
+++ b/pylintrc
@@ -1,6 +1,6 @@
 [MASTER]
 load-plugins=pylint_quotes
 max-line-length=120
-disable=bad-continuation,broad-except
+disable=broad-except
 ignore-patterns=test_.*?py
 generated-members=arcpy.da.SearchCursor,arcpy.da.UpdateCursor,arcpy.da.Describe,arcpy.env.scratchFolder,arcpy.env.scratchGDB
diff --git a/setup.py b/setup.py
index 7e02b9a..894ed91 100644
--- a/setup.py
+++ b/setup.py
@@ -4,28 +4,32 @@
 setup.py
 A module that installs palletjack as a module
 """
-from glob import glob
-from os.path import basename, splitext
+from pathlib import Path
 
 from setuptools import find_packages, setup
 
+version = {}
+with open('src/palletjack/version.py', encoding='utf-8') as fp:
+    exec(fp.read(), version)
+
 setup(
     name='ugrc-palletjack',
-    version='2.7.3',
-    license='MIT',
+    version=version['__version__'],
     description='Updating AGOL feature services with data from SFTP shares.',
+    long_description=(Path(__file__).parent / 'README.md').read_text(),
+    long_description_content_type='text/markdown',
     author='Jake Adams, UGRC',
     author_email='jdadams@utah.gov',
     url='https://github.com/agrc/palletjack',
     packages=find_packages('src'),
     package_dir={'': 'src'},
-    py_modules=[splitext(basename(path))[0] for path in glob('src/*.py')],
     include_package_data=True,
     zip_safe=True,
     classifiers=[
         # complete classifier list: http://pypi.python.org/pypi?%3Aaction=list_classifiers
         'Development Status :: 5 - Production/Stable',
         'Intended Audience :: Developers',
+        'License :: OSI Approved :: MIT License',
         'Topic :: Utilities',
     ],
     project_urls={
@@ -34,26 +38,26 @@
     keywords=['gis'],
     install_requires=[
         'pysftp==0.2.9',
-        'arcgis==2.0.*',
+        'arcgis==2.1.*',
         'pygsheets==2.0.*',
         'geopandas==0.12.*',
         'SQLAlchemy==1.4.*',
         'pg8000==1.29.*',
         'psycopg2-binary==2.9.*',
-        'numpy==1.23.*',  #: Pinned to fix "module 'numpy' has no attribute 'str'" error
+        'numpy==1.23.*',  #: Pinned to 1.23.* to fix "module 'numpy' has no attribute 'str'" error
     ],
     extras_require={
         'tests': [
             'pylint-quotes==0.2.*',
-            'pylint==2.14.*',
-            'pytest-cov==3.0.*',
+            'pylint==2.15.*',
+            'pytest-cov==4.0.*',
             'pytest-instafail==0.4.*',
-            'pytest-isort==3.0.*',
-            'pytest-pylint==0.18.*',
+            'pytest-isort==3.1.*',
+            'pytest-pylint==0.19.*',
             'pytest-watch==4.2.*',
             'pytest==7.*',
             'yapf==0.32.*',
-            'pytest-mock==3.8.*',
+            'pytest-mock==3.10.*',
         ]
     },
     setup_requires=[
diff --git a/src/palletjack/__init__.py b/src/palletjack/__init__.py
index b20a2bf..1809628 100644
--- a/src/palletjack/__init__.py
+++ b/src/palletjack/__init__.py
@@ -1,8 +1,7 @@
-"""A library for updating AGOL feature services with data from SFTP sources
+"""A library for extracting tabular/spatial data from many different sources, transforming it to meet ArcGIS Online hosted feature service requirements, and loading it into existing feature services.
+
+.. include:: ../../docs/README.md
 """
 
-from . import transform
-from .loaders import GoogleDriveDownloader, GSheetLoader, PostgresLoader, SFTPLoader
-from .updaters import (
-    ColorRampReclassifier, FeatureServiceAttachmentsUpdater, FeatureServiceInlineUpdater, FeatureServiceOverwriter
-)
+from . import extract, load, transform, utils
+from .errors import IntFieldAsFloatError, TimezoneAwareDatetimeError
diff --git a/src/palletjack/errors.py b/src/palletjack/errors.py
new file mode 100644
index 0000000..3c1e327
--- /dev/null
+++ b/src/palletjack/errors.py
@@ -0,0 +1,10 @@
+"""Errors specific to palletjack
+"""
+
+
+class IntFieldAsFloatError(Exception):
+    """A field expected to be integer is float instead"""
+
+
+class TimezoneAwareDatetimeError(Exception):
+    """A datetime field uses a timezone-aware dtype"""
diff --git a/src/palletjack/loaders.py b/src/palletjack/extract.py
similarity index 84%
rename from src/palletjack/loaders.py
rename to src/palletjack/extract.py
index 111d07e..bc07796 100644
--- a/src/palletjack/loaders.py
+++ b/src/palletjack/extract.py
@@ -1,4 +1,7 @@
-"""Classes for loading data from various sources into a pandas dataframe
+"""Extract tabular/spatial data from various sources into a pandas dataframe.
+
+Each different type of source has its own class. Each class may have multiple methods available for different loading
+operations or techniques.
 """
 
 import logging
@@ -16,15 +19,26 @@
 import sqlalchemy
 from googleapiclient.http import MediaIoBaseDownload
 
-from . import utils
+from palletjack import utils
 
 logger = logging.getLogger(__name__)
 
 
 class GSheetLoader:
-    """Loads data from a Google Sheets spreadsheet into a pandas data frame"""
+    """Loads data from a Google Sheets spreadsheet into a pandas data frame
+
+    Requires either the path to a service account .json file that has access to the sheet in question or a
+    `google.auth.credentials.Credentials` object. Calling `google.auth.default()` in a Google Cloud Function will give
+    you a tuple of a `Credentials` object and the project id. You can use this `Credentials` object to authorize
+    pygsheets as the same account the Cloud Function is running under.
+    """
 
     def __init__(self, credentials):
+        """
+        Args:
+            credentials (str or google.auth.credentials.Credentials): Path to the service file OR credentials object
+                obtained from google.auth.default() within a cloud function.
+        """
         self._class_logger = logging.getLogger(__name__).getChild(self.__class__.__name__)
         self.gsheets_client = utils.authorize_pygsheets(credentials)
 
@@ -76,18 +90,18 @@ def combine_worksheets_into_single_dataframe(self, worksheet_dfs):
 
         Raises:
             ValueError: If all the worksheets in worksheets_dfs don't have the same column index, it raises an error
-            and bombs out.
+                and bombs out.
 
         Returns:
             pd.DataFrame: A single combined data frame with a new 'worksheet' column identifying the worksheet the row
-            came from. The row index is the original row numbers and is probably not unique.
+                came from. The row index is the original row numbers and is probably not unique.
         """
 
         dataframes = list(worksheet_dfs.values())
 
         #: Make sure all the dataframes have the same columns
         if not all([set(dataframes[0].columns) == set(df.columns) for df in dataframes]):
-            raise ValueError('Columns do not match; cannot create mutli-index dataframe')
+            raise ValueError('Columns do not match; cannot create multi-index dataframe')
 
         self._class_logger.debug('Concatting worksheet dataframes %s into a single dataframe', worksheet_dfs.keys())
         concatted_df = pd.concat(dataframes, keys=worksheet_dfs.keys(), names=['worksheet', 'row'])
@@ -95,10 +109,19 @@ def combine_worksheets_into_single_dataframe(self, worksheet_dfs):
 
 
 class GoogleDriveDownloader:
-    """Downloads images from publicly-shared Google Drive links (https://drive.google.com/file/d/big_long_id/etc)
+    """Provides methods to download any non-html file (ie, Content-Type != text/html) Google Drive file from it's
+    sharing link (of the form `https://drive.google.com/file/d/big_long_id/etc`). The files may be publicly shared or shared with a service account.
+
+    This class has two similar sets of methods. The `*_using_api` methods authenticate to the Google API using either a
+    service account file or a `google.auth.credentials.Credentials` object and downloads using the API. The
+    `*_using_api` methods are the most robust and should be used whenever possible.
     """
 
     def __init__(self, out_dir):
+        """
+        Args:
+            out_dir (str or Path): Directory to save downloaded files. Can be reassigned later to change the directory.
+        """
         self._class_logger = logging.getLogger(__name__).getChild(self.__class__.__name__)
         self._class_logger.debug('Initializing GoogleDriveDownloader')
         self._class_logger.debug('Output directory: %s', out_dir)
@@ -128,11 +151,11 @@ def _get_http_response(self, file_id, base_url='https://docs.google.com/uc?expor
         Args:
             file_id (str): The Google-created unique id for the file
             base_url (str, optional): The base URL for the GET call. Defaults to 'https://docs.google.com/uc?
-                                      export=download'.
+                export=download'.
 
         Raises:
             RuntimeError: If Content-Type is text/html, we can't get the file, either because it doesn't exist or isn't
-                          publicly shared.
+                publicly shared.
 
         Returns:
             request.response: The requests response object.
@@ -195,6 +218,13 @@ def _get_file_id_from_sharing_link(self, sharing_link):
     def download_file_from_google_drive(self, sharing_link, join_id, pause=0.):
         """Download a publicly-shared image from Google Drive using it's sharing link
 
+        Uses an anonymous HTTP request with support for sleeping between downloads to try to get around Google's
+        blocking (I haven't found a good value yet).
+
+        Logs a warning if the URL doesn't match the proper pattern or it can't extract the unique id from the sharing
+        URL. Will also log a warning if the header's Content-Type is text/html, which usually indicates the HTTP
+        response was an error message instead of the file.
+
         Args:
             sharing_link (str): The publicly-shared link to the image.
             join_id (str or int): The unique key for the row (used for reporting)
@@ -229,7 +259,7 @@ def _get_request_and_filename_from_drive_api(self, client, file_id):
         """Get the request object and filename from a Google drive file_id. Will try to determine extension if missing.
 
         Args:
-            client (pygsheets.Client): Authenticated client object from pyghseets
+            client (pygsheets.Client): Authenticated client object from pygsheets
             file_id (str): The Google fileId to be downloaded
 
         Returns:
@@ -267,7 +297,13 @@ def _save_get_media_content(self, request, out_file_path):
         out_file_path.write_bytes(in_memory.getbuffer())
 
     def download_file_from_google_drive_using_api(self, gsheets_client, sharing_link, join_id):
-        """Download a file from Google Drive using an authenticated client and the Google API via pygsheets
+        """ Download a file using the Google API via pygsheets authentication.
+
+        Requires a pygsheets client object that handles authentication.
+
+        Logs a warning if the URL doesn't match the proper pattern or it can't extract the unique id from the sharing
+        URL. Will also log a warning if the header's Content-Type is text/html, which usually indicates the HTTP
+        response was an error message instead of the file.
 
         Args:
             gsheets_client (pygsheets.Client): The authenticated client object from pygsheets
@@ -303,8 +339,8 @@ def download_attachments_from_dataframe(self, dataframe, sharing_link_column, jo
             dataframe (pd.DataFrame): Input dataframe with required columns
             sharing_link_column (str): Column holding the Google sharing link
             join_id_column (str): Column holding a unique key (for reporting purposes)
-            output_path_column (str): Column for the resulting path; will be added if it doesn't existing in the
-            dataframe
+            output_path_column (str): Column for the resulting path; will be added if it doesn't exist in the
+                dataframe
 
         Returns:
             pd.DataFrame: Input dataframe with output path info
@@ -328,7 +364,7 @@ def download_attachments_from_dataframe_using_api(
             sharing_link_column (str): Column holding the Google sharing link
             join_id_column (str): Column holding a unique key (for reporting purposes)
             output_path_column (str): Column for the resulting path; will be added if it doesn't existing in the
-            dataframe
+                dataframe
 
         Returns:
             pd.DataFrame: Input dataframe with output path info
@@ -348,6 +384,15 @@ class SFTPLoader:
     """
 
     def __init__(self, host, username, password, knownhosts_file, download_dir):
+        """
+        Args:
+            host (str): The SFTP host to connect to
+            username (str): SFTP username
+            password (str): SFTP password
+            knownhosts_file (str): Path to a known_hosts file for pysftp.CnOpts. Can be generated via ssh-keyscan.
+            download_dir (str or Path): Directory to save downloaded files
+        """
+
         self.host = host
         self.username = username
         self.password = password
@@ -438,8 +483,16 @@ class PostgresLoader:
     """Loads data from a Postgres/PostGIS database into a pandas data frame"""
 
     def __init__(self, host, database, username, password, port=5432):
-        self._class_logger = logging.getLogger(__name__).getChild(self.__class__.__name__)
+        """
+        Args:
+            host (str): Postgres server host name
+            database (str): Database to connect to
+            username (str): Database user
+            password (str): Database password
+            port (int, optional): Database port. Defaults to 5432.
+        """
 
+        self._class_logger = logging.getLogger(__name__).getChild(self.__class__.__name__)
         if os.environ.get('FUNCTION_TARGET') is not None:  #: this is an env var specific to cloud functions
             self._class_logger.info('running in GCF, using unix socket')
             self.engine = sqlalchemy.create_engine(
@@ -482,9 +535,11 @@ def read_table_into_dataframe(self, table_name, index_column, crs, spatial_colum
         dataframe = gpd.read_postgis(
             f'select * from {table_name}', self.engine, index_col=index_column, crs=crs, geom_col=spatial_column
         )
-        dataframe = dataframe.to_crs(epsg=3857)
 
-        spatial_dataframe = pd.DataFrame.spatial.from_geodataframe(dataframe)
+        spatial_dataframe = pd.DataFrame.spatial.from_geodataframe(dataframe, column_name=spatial_column)
+        for column in spatial_dataframe.select_dtypes(include=['datetime64[ns, UTC]']):
+            self._class_logger.debug('Converting column `%s` to ISO string format', column)
+            spatial_dataframe[column] = spatial_dataframe[column].apply(pd.Timestamp.isoformat)
 
         self._class_logger.debug('Dataframe shape: %s', spatial_dataframe.shape)
         if len(spatial_dataframe.index) == 0:
diff --git a/src/palletjack/load.py b/src/palletjack/load.py
new file mode 100644
index 0000000..b507d07
--- /dev/null
+++ b/src/palletjack/load.py
@@ -0,0 +1,820 @@
+"""Modify existing ArcGIS Online content (mostly hosted feature services). Contains classes for updating hosted feature
+service data, modifying the attachments on a hosted feature service, or modifying map symbology.
+"""
+
+import json
+import logging
+import sys
+import warnings
+from datetime import datetime
+from pathlib import Path
+
+import arcgis
+import numpy as np
+import pandas as pd
+from arcgis.features import GeoAccessor, GeoSeriesAccessor
+
+from palletjack import utils
+
+logger = logging.getLogger(__name__)
+
+
+class FeatureServiceUpdater:
+    """Update an AGOL Feature Service with data from a pandas DataFrame.
+
+    Contains four class methods that can be called directly without needing to instantiate an object: add_features,
+    remove_features, update_features, and truncate_and_load_features.
+
+    It is the client's responsibility to separate out the new data into these different steps. If the extract/transform
+    stages result in seperate groups of records that need to be added, deleted, and updated, the client must call the
+    three different methods with dataframes containing only the respective records for each operation.
+
+    Because the update process uploads the data as geojson, all input geometries must be in WGS84 (wkid 4326). Input
+    dataframes can be projected using dataframe.spatial.project(4326). ArcGIS Online will then project the uploaded
+    data to match the hosted feature service's projection.
+    """
+
+    @classmethod
+    def add_features(cls, gis, feature_service_itemid, dataframe, layer_index=0):
+        """Adds new features to existing hosted feature layer from new dataframe.
+
+        The new dataframe must have a 'SHAPE' column containing geometries of the same type as the live data. The
+        dataframe must have a WGS84 (wkid 4326) projection. New OBJECTIDs will be automatically generated.
+
+        The new dataframe's columns and data must match the existing data's fields (with the exception of generated
+        fields like shape area and length) in name, type, and allowable length. Live fields that are not nullable and
+        don't have a default value must have a value in the new data; missing data in these fields will raise an error.
+
+        Args:
+            gis (arcgis.gis.GIS): GIS item for AGOL org
+            features_service_item_id (str): itemid for service to update
+            dataframe (pd.DataFrame.spatial): Spatially enabled dataframe of data to be added
+            layer_index (int): Index of layer within service to update. Defaults to 0.
+
+        Raises:
+            ValueError: If the new field and existing fields don't match, the SHAPE field is missing or has an
+                incompatible type, the new data contains null fields, the new data exceeds the existing field
+                lengths, or a specified field is missing from either new or live data.
+
+        Returns:
+            int: Number of features added
+        """
+        updater = cls(gis, feature_service_itemid, dataframe, fields=list(dataframe.columns), layer_index=layer_index)
+        return updater._add_new_data_to_hosted_feature_layer()
+
+    @classmethod
+    def remove_features(cls, gis, feature_service_itemid, delete_oids, layer_index=0):
+        """Deletes features from a hosted feature layer based on comma-separated string of Object IDs
+
+        This is a wrapper around the arcgis.FeatureLayer.delete_features method that adds some sanity checking. The
+        delete operation is rolled-back if any of the features fail to delete using (rollback_on_failure=True). This
+        function will raise a RuntimeError as well after delete_features() returns if any of them fail.
+
+        The sanity checks will raise errors or warnings as appropriate if any of them fail.
+
+        Args:
+            delete_oids (list[int]): List of OIDs to delete
+
+        Raises:
+            ValueError: If delete_string can't be split on `,`
+            TypeError: If any of the items in delete_string can't be cast to ints
+            ValueError: If delete_string is empty
+            UserWarning: If any of the Object IDs in delete_string don't exist in the live data
+            RuntimeError: If any of the OIDs fail to delete
+
+        Returns:
+            int: The number of features deleted
+        """
+
+        updater = cls(gis, feature_service_itemid, layer_index=layer_index)
+        return updater._delete_data_from_hosted_feature_layer(delete_oids)
+
+    @classmethod
+    def update_features(cls, gis, feature_service_itemid, dataframe, layer_index=0, update_geometry=True):
+        """Updates existing features within a hosted feature layer using OBJECTID as the join field.
+
+        The new dataframe's columns and data must match the existing data's fields (with the exception of generated
+        fields like shape area and length) in name, type, and allowable length. Live fields that are not nullable and
+        don't have a default value must have a value in the new data; missing data in these fields will raise an error.
+
+        Uses the OBJECTID field to determine which features should be updated by the underlying FeatureLayer.append()
+        method. The most robust way to do this is to load the live data as a dataframe, subset it down to the desired
+        rows, make your edits based on a separate join id, and then pass that dataframe to this method.
+
+        The new data can have either attributes and geometries or only attributes based on the update_geometry flag. A
+        combination of updates from a source with both attributes & geometries and a source with attributes-only must
+        be done with two separate calls. The geometries must be provided in a SHAPE column, be the same type as the
+        live data, and have a WGS84 (wkid 4326) projection.
+
+        Args:
+            gis (arcgis.gis.GIS): GIS item for AGOL org
+            features_service_item_id (str): itemid for service to update
+            dataframe (pd.DataFrame.spatial): Spatially enabled dataframe of data to be updated
+            layer_index (int): Index of layer within service to update. Defaults to 0.
+            update_geometry (bool): Whether to update attributes and geometry (True) or just attributes (False).
+                Defaults to False.
+
+        Raises:
+            ValueError: If the new field and existing fields don't match, the SHAPE field is missing or has an
+                incompatible type, the new data contains null fields, the new data exceeds the existing field
+                lengths, or a specified field is missing from either new or live data.
+
+        Returns:
+            int: Number of features updated
+        """
+
+        updater = cls(
+            gis,
+            feature_service_itemid,
+            dataframe,
+            fields=list(dataframe.columns),
+            layer_index=layer_index,
+        )
+        return updater._update_hosted_feature_layer(update_geometry)
+
+    @classmethod
+    def truncate_and_load_features(cls, gis, feature_service_itemid, dataframe, failsafe_dir='', layer_index=0):
+        """Overwrite a hosted feature layer by truncating and loading the new data
+
+        When the existing dataset is truncated, a copy is kept in memory as a spatially-enabled dataframe. If the new
+        data fail to load, this copy is reloaded. If the reload fails, the copy is written to failsafe_dir with the
+        filename {todays_date}.json (2022-12-31.json).
+
+        The new dataframe must have a 'SHAPE' column containing geometries of the same type as the live data. The
+        dataframe must have a WGS84 (wkid 4326) projection. New OBJECTIDs will be automatically generated.
+
+        The new dataframe's columns and data must match the existing data's fields (with the exception of generated
+        fields like shape area and length) in name, type, and allowable length. Live fields that are not nullable and
+        don't have a default value must have a value in the new data; missing data in these fields will raise an error.
+
+        Args:
+            gis (arcgis.gis.GIS): GIS item for AGOL org
+            feature_service_itemid (str): itemid for service to update
+            dataframe (pd.DataFrame.spatial): Spatially enabled dataframe of new data to be loaded
+            failsafe_dir (str, optional): Directory to save original data in case of complete failure. If left blank,
+            existing data won't be saved. Defaults to ''
+            layer_index (int, optional): Index of layer within service to update. Defaults to 0.
+
+        Returns:
+            int: Number of features loaded
+        """
+
+        updater = cls(
+            gis,
+            feature_service_itemid,
+            dataframe,
+            fields=list(dataframe.columns),
+            failsafe_dir=failsafe_dir,
+            layer_index=layer_index
+        )
+        return updater._truncate_and_load_data()
+
+    def __init__(self, gis, feature_service_itemid, dataframe=None, fields=None, failsafe_dir=None, layer_index=0):
+        self._class_logger = logging.getLogger(__name__).getChild(self.__class__.__name__)
+        self.feature_service_itemid = feature_service_itemid
+        self.feature_layer = arcgis.features.FeatureLayer.fromitem(gis.content.get(feature_service_itemid))
+        if dataframe is not None:
+            self.new_dataframe = dataframe
+            if 'SHAPE' in self.new_dataframe.columns:
+                self.new_dataframe.spatial.set_geometry('SHAPE')
+        if fields is not None:
+            self.fields = list(set(fields) - {'Shape_Area', 'Shape_Length'})  #: We don't use these auto-gen fields
+        self.failsafe_dir = failsafe_dir if failsafe_dir else None
+        self.layer_index = layer_index
+
+    def _add_new_data_to_hosted_feature_layer(self) -> int:
+        """Adds new features to existing hosted feature layer. Uses fields from new dataframe.
+
+        Raises:
+            ValueError: If the new field and existing fields don't match, the new data contains null fields,
+                the new data exceeds the existing field lengths, or a specified field is missing from either
+                new or live data.
+
+        Returns:
+            int: Number of features added
+        """
+
+        self._class_logger.info(
+            'Adding items to layer `%s` in itemid `%s` in-place', self.layer_index, self.feature_service_itemid
+        )
+        self._class_logger.debug('Using fields %s', self.fields)
+
+        #: Field checks to prevent various AGOL errors
+        utils.FieldChecker.check_fields(self.feature_layer.properties, self.new_dataframe, self.fields, add_oid=False)
+
+        #: Upsert
+        append_count = self._upload_data(
+            self.feature_layer,
+            self.new_dataframe,
+            upsert=False,
+        )
+        return append_count
+
+    def _delete_data_from_hosted_feature_layer(self, delete_oids) -> int:
+        """Deletes features from a hosted feature layer based on comma-separated string of OIDS
+
+        Args:
+            delete_oids (list[int]): List of OIDs to delete
+
+        Raises:
+            RuntimeError: If any of the OIDs fail to delete
+
+        Returns:
+            int: The number of features deleted
+        """
+
+        self._class_logger.info(
+            'Deleting features from layer `%s` in itemid `%s`', self.layer_index, self.feature_service_itemid
+        )
+        self._class_logger.debug('Delete string: %s', delete_oids)
+
+        #: Verify delete list
+        # oid_list = utils.DeleteUtils.check_delete_oids_are_comma_separated(delete_oids)
+        oid_numeric = utils.DeleteUtils.check_delete_oids_are_ints(delete_oids)
+        utils.DeleteUtils.check_for_empty_oid_list(oid_numeric, delete_oids)
+        delete_string = ','.join([str(oid) for oid in oid_numeric])
+        num_missing_oids = utils.DeleteUtils.check_delete_oids_are_in_live_data(
+            delete_string, oid_numeric, self.feature_layer
+        )
+
+        #: Note: apparently not all services support rollback: https://developers.arcgis.com/rest/services-reference/enterprise/delete-features.htm
+        deletes = utils.retry(
+            self.feature_layer.delete_features,
+            deletes=delete_string,
+            rollback_on_failure=True,
+        )
+
+        failed_deletes = [result['objectId'] for result in deletes['deleteResults'] if not result['success']]
+        if failed_deletes:
+            raise RuntimeError(f'The following Object IDs failed to delete: {failed_deletes}')
+
+        #: The REST API still returns success: True on missing OIDs, so we have to track this ourselves
+        actual_delete_count = len(deletes['deleteResults']) - num_missing_oids
+
+        return actual_delete_count
+
+    def _update_hosted_feature_layer(self, update_geometry) -> int:
+        """Updates existing features within a hosted feature layer using OBJECTID as the join field
+
+        Raises:
+            ValueError: If the new field and existing fields don't match, the new data contains null fields,
+                the new data exceeds the existing field lengths, or a specified field is missing from either
+                new or live data.
+
+        Returns:
+            int: Number of features updated
+        """
+
+        self._class_logger.info(
+            'Updating layer `%s` in itemid `%s` in-place', self.layer_index, self.feature_service_itemid
+        )
+        self._class_logger.debug('Updating fields %s', self.fields)
+
+        #: Add null geometries if update_geometry==False so that we can create a featureset from the dataframe
+        #: (geometries will be ignored by upsert call)
+        if not update_geometry:
+            self._class_logger.debug('Attribute-only update; inserting null geometries')
+            self.new_dataframe['SHAPE'] = utils.get_null_geometries(self.feature_layer.properties)
+
+        #: Field checks to prevent various AGOL errors
+        utils.FieldChecker.check_fields(self.feature_layer.properties, self.new_dataframe, self.fields, add_oid=True)
+
+        #: Upsert
+        append_count = self._upload_data(
+            self.feature_layer,
+            self.new_dataframe,
+            upsert=True,
+            upsert_matching_field='OBJECTID',
+            append_fields=self.fields,  #: Apparently this works if append_fields is all the fields, but not a subset?
+            update_geometry=update_geometry
+        )
+        return append_count
+
+    #: TODO: rename this method? not everything is an upsert
+    def _upload_data(self, target_featurelayer, dataframe, **append_kwargs):
+        """UPdate and inSERT data into live dataset with featurelayer.append()
+
+        Note: The call to to_featureset() in this method will add new OBJECTIDs to the new data if they aren't already present.
+
+        Args:
+            target_featurelayer (arcgis.features.FeatureLayer): Live dataset
+            dataframe (pd.DataFrame): Spatially-enabled dataframe containing new data. Column names must match live
+                data. New data must have a valid shape column
+            append_kwargs (keyword arguments): Arguments to be passed to .append()
+
+        Raises:
+            RuntimeError: If append fails; will attempt to rollback appends that worked.
+
+        Returns:
+            dict: Messages returned from append operation
+        """
+
+        try:
+            if append_kwargs['upsert'] \
+                and (
+                        append_kwargs['upsert_matching_field'] not in append_kwargs['append_fields']
+                        or
+                        append_kwargs['upsert_matching_field'] not in dataframe.columns
+                    ):
+                raise ValueError(
+                    f'Upsert matching field {append_kwargs["upsert_matching_field"]} not found in either append fields or existing fields.'
+                )
+        except KeyError:
+            pass
+
+        #: FIXME: With chunking, rollback could leave the data in an inconsistent state if a chunk fails midway through the process. All the chunks before that won't be rolled back, so some of the data will be new and some will be old. I suppose this will be ok for upserts, where the data will just get updated again. However, adds will probably just create duplicates of the chunks that aren't rolled back if we try to re-do the whole add op again. Do we create a list of dataframe row indices that were in the successful chunks?
+
+        running_append_total = 0
+        geojsons = utils.Chunking.build_upload_json(dataframe, target_featurelayer.properties.fields, 20_000_000)
+        self._class_logger.info('Appending/upserting data in %s chunk(s)', len(geojsons))
+        chunk_sizes = []
+        for chunk, geojson in enumerate(geojsons, 1):
+            try:
+                chunk_start = datetime.now()
+                chunk_size = sys.getsizeof(geojson.encode('utf-16'))
+                chunk_sizes.append(chunk_size)
+                self._class_logger.debug('Uploading chunk %s of %s, %s bytes', chunk, len(geojsons), chunk_size)
+                result, messages = utils.retry(
+                    target_featurelayer.append,
+                    upload_format='geojson',
+                    edits=geojson,
+                    return_messages=True,
+                    rollback=True,
+                    **append_kwargs
+                )
+                self._class_logger.debug(messages)
+                self._class_logger.debug('Chunk time: %s', datetime.now() - chunk_start)
+            except Exception:
+                self._class_logger.debug(pd.Series(chunk_sizes).describe())
+                self._class_logger.error('Append failed, feature service may be dirty due to append chunking.')
+                raise
+            if not result:
+                self._class_logger.debug(pd.Series(chunk_sizes).describe())
+                raise RuntimeError(
+                    f'Failed to append data at chunk {chunk} of {len(geojsons)}. Append operation should have been rolled back.'
+                )
+
+            running_append_total += messages['recordCount']
+
+        self._class_logger.debug(pd.Series(chunk_sizes).describe())
+        return running_append_total
+
+    def _truncate_and_load_data(self):
+        """Overwrite a layer by truncating and loading new data
+
+        Raises:
+            RuntimeError: If loading fails and reloading of old data fails (old data will be written to disk)
+
+        Returns:
+            int: Number of features loaded
+        """
+
+        self._class_logger.info(
+            'Truncating and loading layer `%s` in itemid `%s`', self.layer_index, self.feature_service_itemid
+        )
+        start = datetime.now()
+
+        #: Save the data to disk if failsafe dir provided
+        #: TODO: return path programmatically so client can catch exception and try to reload automatically?
+        if self.failsafe_dir:
+            self._class_logger.info('Saving existing data to %s', self.failsafe_dir)
+            saved_layer_path = utils.save_feature_layer_to_json(self.feature_layer, self.failsafe_dir)
+
+        #: Field checks to prevent various AGOL errors
+        utils.FieldChecker.check_fields(self.feature_layer.properties, self.new_dataframe, self.fields, add_oid=False)
+
+        self._class_logger.info('Truncating existing features...')
+        self._truncate_existing_data()
+
+        try:
+            self._class_logger.info('Loading new data...')
+            append_count = self._upload_data(self.feature_layer, self.new_dataframe, upsert=False)
+            self._class_logger.debug('Total truncate and load time: %s', datetime.now() - start)
+        except Exception:
+            if self.failsafe_dir:
+                self._class_logger.error(
+                    'Append failed, feature service may be dirty due to append chunking. Data saved to %s',
+                    saved_layer_path
+                )
+                raise
+            self._class_logger.error(
+                'Append failed, feature service may be dirty due to append chunking. Old data not saved (no failsafe dir set)'
+            )
+            raise
+
+        return append_count
+
+    def _truncate_existing_data(self):
+        """Remove all existing features from the live dataset
+
+        Raises:
+            RuntimeError: If the truncate fails
+
+        Returns:
+            pd.DataFrame: The feature layer's data as a spatially-enabled dataframe prior to truncating
+        """
+
+        self._class_logger.debug('Truncating...')
+        truncate_result = utils.retry(self.feature_layer.manager.truncate, asynchronous=True, wait=True)
+        self._class_logger.debug(truncate_result)
+        if truncate_result['status'] != 'Completed':
+            raise RuntimeError(
+                f'Failed to truncate existing data from layer id {self.layer_index} in itemid {self.feature_service_itemid}'
+            )
+
+
+class FeatureServiceAttachmentsUpdater:
+    """Add or overwrite attachments in a feature service using a dataframe of the desired "new" attachments.
+
+    Updates the attachments based on a dataframe containing two columns: a join key present in the live data (the
+    dataframe column name must match the feature service field name) and the path of the file to attach to the feature.
+    While AGOL supports multiple attachments, this only accepts a single file as input.
+
+    If a matching feature in AGOl doesn't have an attachment, the file referred to by the dataframe will be uploaded.
+    If it does have an attachment, it checks the existing filename with the referenced file. If they are different, the
+    file from the dataframe will be updated. If they are the same, nothing happens.
+    """
+
+    def __init__(self, gis):
+        """
+        Args:
+            gis (arcgis.gis.GIS): The AGOL organization's gis object
+        """
+
+        self.gis = gis
+        self._class_logger = logging.getLogger(__name__).getChild(self.__class__.__name__)
+        self.failed_dict = {}
+
+    def _get_live_oid_and_guid_from_join_field_values(self, live_features_as_df, attachment_join_field, attachments_df):
+        """Get the live Object ID and guid from the live features for only the features in attachments_df
+
+        Args:
+            live_features_as_df (pd.DataFrame): Spatial dataframe of all the feature layer's live data from AGOL
+            attachment_join_field (str): Column in attachments_df to use as a join key with live data
+            attachments_df (pd.DataFrame): New attachment data, including the join key and a path to the "new"
+                attachment
+
+        Returns:
+            pd.DataFrame: Attachments dataframe with corresponding live OIDs and GUIDs.
+        """
+
+        self._class_logger.debug('Using %s as the join field between live and new data', attachment_join_field)
+        subset_df = live_features_as_df.reindex(columns=['OBJECTID', 'GlobalID', attachment_join_field])
+        merged_df = subset_df.merge(attachments_df, on=attachment_join_field, how='inner')
+        self._class_logger.debug('%s features common to both live and new data', len(merged_df.index))
+
+        return merged_df
+
+    def _get_current_attachment_info_by_oid(self, live_data_subset_df):
+        """Merge the live attachment data using the ObjectID as the join
+
+        Args:
+            live_data_subset_df (pd.DataFrame): Live data with 'OBJECTID', 'GlobalID', and new attachment data
+
+        Returns:
+            pd.DataFrame: Live and new attachment data in one dataframe
+        """
+
+        live_attachments_df = pd.DataFrame(self.feature_layer.attachments.search())
+        live_attachments_subset_df = live_attachments_df.reindex(columns=['PARENTOBJECTID', 'NAME', 'ID'])
+        merged_df = live_data_subset_df.merge(
+            live_attachments_subset_df, left_on='OBJECTID', right_on='PARENTOBJECTID', how='left'
+        )
+        #: Cast ID field to nullable int to avoid conversion to float for np.nans
+        merged_df['ID'] = merged_df['ID'].astype('Int64')
+
+        return merged_df
+
+    def _create_attachment_action_df(self, attachment_eval_df, attachment_path_field):
+        """Create a dataframe containing the action needed for each feature resulting from the attachment join.
+
+        If the live feature doesn't have an attachment, add the attachment. If it does, compare the file names and only
+        attach if they are different. Otherwise, leave null.
+
+        Args:
+            attachment_eval_df (pd.DataFrame): DataFrame of live attachment data, subsetted to features that matched
+                the join key in the new attachments
+            attachment_path_field (str): The column that holds the attachment path
+
+        Returns:
+            pd.DataFrame: attachment_eval_df with 'operation' and 'new_filename' columns added
+        """
+
+        #: Get the file name from the full path
+        attachment_eval_df['new_filename'] = attachment_eval_df[attachment_path_field].apply(
+            lambda path: Path(path).name
+        )
+
+        #: Overwrite if different names, add if no existing name, do nothing if names are the same
+        attachment_eval_df['operation'] = np.nan
+        attachment_eval_df.loc[attachment_eval_df['NAME'] != attachment_eval_df['new_filename'],
+                               'operation'] = 'overwrite'
+        attachment_eval_df.loc[attachment_eval_df['NAME'].isna(), 'operation'] = 'add'
+
+        value_counts = attachment_eval_df['operation'].value_counts(dropna=False)
+        for operation in ['add', 'overwrite', np.nan]:
+            if operation not in value_counts:
+                value_counts[operation] = 0
+        self._class_logger.debug(
+            'Calculated attachment operations: adds: %s, overwrites: %s, none: %s', value_counts['add'],
+            value_counts['overwrite'], value_counts[np.nan]
+        )
+
+        return attachment_eval_df
+
+    def _add_attachments_by_oid(self, attachment_action_df, attachment_path_field):
+        """Add attachments using the feature's OID based on the 'operation' field of the dataframe
+
+        Args:
+            attachment_action_df (pd.DataFrame): A dataframe containing 'operation', 'OBJECTID', and
+                attachment_path_field columns
+            attachment_path_field (str): The column that holds the attachment path
+
+        Returns:
+            int: The number of features that successfully have attachments added.
+        """
+
+        adds_dict = attachment_action_df[attachment_action_df['operation'] == 'add'].to_dict(orient='index')
+        adds_count = 0
+
+        for row in adds_dict.values():
+            target_oid = row['OBJECTID']
+            filepath = row[attachment_path_field]
+
+            self._class_logger.debug('Add %s to OID %s', filepath, target_oid)
+            try:
+                result = self.feature_layer.attachments.add(target_oid, filepath)
+            except Exception:
+                self._class_logger.error('AGOL error while adding %s to OID %s', filepath, target_oid, exc_info=True)
+                self.failed_dict[target_oid] = ('add', filepath)
+                continue
+
+            self._class_logger.debug('%s', result)
+            if not result['addAttachmentResult']['success']:
+                warnings.warn(f'Failed to attach {filepath} to OID {target_oid}')
+                self.failed_dict[target_oid] = ('add', filepath)
+                continue
+
+            adds_count += 1
+
+        return adds_count
+
+    def _overwrite_attachments_by_oid(self, attachment_action_df, attachment_path_field):
+        """Overwrite attachments using the feature's OID based on the 'operation' field of the dataframe
+
+        Args:
+            attachment_action_df (pd.DataFrame): A dataframe containing 'operation', 'OBJECTID', 'ID', 'NAME', and
+                attachment_path_field columns
+            attachment_path_field (str): The column that holds the attachment path
+
+        Returns:
+            int: The number of features that successfully have their attachments overwritten.
+        """
+
+        overwrites_dict = attachment_action_df[attachment_action_df['operation'] == 'overwrite'].to_dict(orient='index')
+        overwrites_count = 0
+
+        for row in overwrites_dict.values():
+            target_oid = row['OBJECTID']
+            filepath = row[attachment_path_field]
+            attachment_id = row['ID']
+            old_name = row['NAME']
+
+            self._class_logger.debug(
+                'Overwriting %s (attachment ID %s) on OID %s with %s', old_name, attachment_id, target_oid, filepath
+            )
+            try:
+                result = self.feature_layer.attachments.update(target_oid, attachment_id, filepath)
+            except Exception:
+                self._class_logger.error(
+                    'AGOL error while overwriting %s (attachment ID %s) on OID %s with %s',
+                    old_name,
+                    attachment_id,
+                    target_oid,
+                    filepath,
+                    exc_info=True
+                )
+                self.failed_dict[target_oid] = ('update', filepath)
+                continue
+
+            if not result['updateAttachmentResult']['success']:
+                warnings.warn(
+                    f'Failed to update {old_name}, attachment ID {attachment_id}, on OID {target_oid} with {filepath}'
+                )
+                self.failed_dict[target_oid] = ('update', filepath)
+                continue
+
+            overwrites_count += 1
+
+        return overwrites_count
+
+    @staticmethod
+    def _check_attachment_dataframe_for_invalid_column_names(attachment_dataframe, invalid_names):
+        invalid_names_index = pd.Index(invalid_names)
+        intersection = attachment_dataframe.columns.intersection(invalid_names_index)
+        if not intersection.empty:
+            raise RuntimeError(f'Attachment dataframe contains the following invalid names: {list(intersection)}')
+
+    def update_attachments(
+        self, feature_layer_itemid, attachment_join_field, attachment_path_field, attachments_df, layer_number=0
+    ):
+        """Update a feature layer's attachments based on info from a dataframe of desired attachment file names
+
+        Depends on a dataframe populated with a join key for the live data and the downloaded or locally-available
+        attachments. If the name of the "new" attachment is the same as an existing attachment for that feature, it is
+        not updated. If it is different or there isn't an existing attachment, the "new" attachment is attached to that
+        feature.
+
+        Args:
+            feature_layer_itemid (str): The AGOL Item ID of the feature layer to update
+            attachment_join_field (str): The field containing the join key between the attachments dataframe and the
+                live data
+            attachment_path_field (str): The field containing the desired attachment file path
+            attachments_df (pd.DataFrame): A dataframe of desired attachments, including a join key and the local path
+                to the attachment
+            layer_number (int, optional): The layer within the Item ID to update. Defaults to 0.
+
+        Returns:
+            (int, int): Tuple of counts of successful overwrites and adds.
+        """
+
+        self._class_logger.info('Updating attachments...')
+        #: These names are present in the live attachment data downloaded from AGOL. Because we merge the dataframes
+        #: later, we need to make sure they're not the same. There may be better ways of handling this that allows the
+        #: client names to be preserved, but for now force them to fix this.
+        self._check_attachment_dataframe_for_invalid_column_names(
+            attachments_df, invalid_names=['OBJECTID', 'PARENTOBJECTID', 'NAME', 'ID']
+        )
+        self._class_logger.debug('Using layer %s from item ID %s', layer_number, feature_layer_itemid)
+        self.feature_layer = self.gis.content.get(feature_layer_itemid).layers[layer_number]
+        live_features_as_df = pd.DataFrame.spatial.from_layer(self.feature_layer)
+        live_data_subset_df = self._get_live_oid_and_guid_from_join_field_values(
+            live_features_as_df, attachment_join_field, attachments_df
+        )
+        #: TODO: Make sure layer supports attachments so we don't get an arcgis error.
+        #: Check out the feature layer .properties and FeatureLayerManager.add_to_definition to check/enable?
+        attachment_eval_df = self._get_current_attachment_info_by_oid(live_data_subset_df)
+        attachment_action_df = self._create_attachment_action_df(attachment_eval_df, attachment_path_field)
+
+        overwrites_count = self._overwrite_attachments_by_oid(attachment_action_df, attachment_path_field)
+        adds_count = self._add_attachments_by_oid(attachment_action_df, attachment_path_field)
+        self._class_logger.info('%s attachments added, %s attachments overwritten', adds_count, overwrites_count)
+
+        return overwrites_count, adds_count
+
+    @staticmethod
+    def build_attachments_dataframe(input_dataframe, join_column, attachment_column, out_dir):
+        """Create an attachments dataframe by subsetting down to just the two fields and dropping any rows
+           with null/empty attachments
+
+        Args:
+            input_dataframe (pd.DataFrame): Input data containing at least the join and attachment filename columns
+            join_column (str): Unique key joining attachments to live data
+            attachment_column (str): Filename for each attachment
+            out_dir (str or Path): Output directory, will be used to build full path to attachment
+
+        Returns:
+            pd.DataFrame: Dataframe with join key, attachment name, and full attachment paths
+        """
+
+        input_dataframe[attachment_column].replace('', np.nan, inplace=True)  #: pandas doesn't see empty strings as NAs
+        attachments_dataframe = input_dataframe[[join_column, attachment_column]] \
+                                            .copy().dropna(subset=[attachment_column])
+        #: Create the full path by prepending the output directory using .apply and a lambda function
+        attachments_dataframe['full_file_path'] = attachments_dataframe[attachment_column] \
+                                                    .apply(lambda filename: str(Path(out_dir, filename)))
+
+        return attachments_dataframe
+
+
+class ColorRampReclassifier:
+    """Updates the interval ranges on a webmap's layer's classification renderer based on the layer's current data.
+
+    Manually edits the JSON definition to change a layer's color ramp values based on a simple unclassed scheme similar
+    to AGOL's unclassed ramp. The minimum value is the dataset minimum, the max is the mean value plus one standard
+    deviation.
+    """
+
+    def __init__(self, webmap_item, gis):
+        """
+        Args:
+            webmap_item (arcgis.mapping.WebMap): The webmap item in the AGOL organization
+            gis (arcgis.gis.GIS): The AGOL organization as a gis object
+        """
+
+        self.webmap_item = webmap_item
+        self.gis = gis
+        self._class_logger = logging.getLogger(__name__).getChild(self.__class__.__name__)
+
+    def _get_layer_dataframe(self, layer_name, feature_layer_number=0):
+        """Create a dataframe from layer_name in self.webmap_item
+
+        Args:
+            layer_name (str): The exact name of the layer
+            feature_layer_number (int): The number of the layer with the feature service to update. Defaults to 0.
+
+        Returns:
+            spatially-enabled data frame: The layer's data, including geometries.
+        """
+
+        self._class_logger.info('Getting dataframe from `%s` on `%s`', layer_name, self.webmap_item.title)
+        webmap_object = arcgis.mapping.WebMap(self.webmap_item)
+        layer = webmap_object.get_layer(title=layer_name)
+        feature_layer = self.gis.content.get(layer['itemId'])
+        layer_dataframe = pd.DataFrame.spatial.from_layer(feature_layer.layers[feature_layer_number])
+
+        return layer_dataframe
+
+    def _get_layer_id(self, layer_name):
+        """Get the ID number of layer_name in self.webmap_item
+
+        Args:
+            layer_name (str): The exact name of the layer
+
+        Raises:
+            ValueError: If the layer is not found in the webmap
+
+        Returns:
+            int: The index (0-based) of the the layer in the web map
+        """
+
+        data = self.webmap_item.get_data()
+        for layer_id, layer in enumerate(data['operationalLayers']):
+            if layer['title'] == layer_name:
+                self._class_logger.debug('Layer `%s` has id `%s`', layer_name, layer_id)
+                return layer_id
+
+        #: If we haven't matched the title and returned a valid id, raise an error.
+        raise ValueError(f'Could not find "{layer_name}" in {self.webmap_item.title}')
+
+    @staticmethod
+    def _calculate_new_stops(dataframe, column, stops):
+        """Calculate new stop values for an AGOL color ramp using what appears to be AGOL's method for unclassed ramps.
+
+        Args:
+            dataframe (pd.DataFrame): Data being classified
+            column (str): Column to classify
+            stops (int, optional): Number of stops to create.
+
+        Returns:
+            List: New stops cast as ints
+        """
+
+        if column not in dataframe.columns:
+            raise ValueError(f'Column `{column}` not in dataframe')
+        minval = dataframe[column].min()
+        mean = dataframe[column].mean()
+        std_dev = dataframe[column].std()
+        upper = mean + std_dev  #: AGOL's default upper value for unclassed ramps seems to be mean + 1 std dev
+
+        new_stops = np.linspace(minval, upper, stops)
+        new_stops_ints = [int(stop) for stop in new_stops]
+
+        return new_stops_ints
+
+    def _update_stop_values(self, layer_number, new_stops):
+        """Update the stop values of an (un)classified polygon renderer in an AGOL Webmap
+
+        Args:
+            layer_number (int): The index for the layer to be updated
+            new_stops (List): New values for the existing stops
+
+        Returns:
+            Bool: Success or failure of update operation
+        """
+
+        #: Get short reference to the stops dictionary from the webmap's data json
+        data = self.webmap_item.get_data()
+        renderer = data['operationalLayers'][layer_number]['layerDefinition']['drawingInfo']['renderer']
+        stops = renderer['visualVariables'][0]['stops']
+
+        #: Overwrite the value, update the webmap item
+        for stop, new_value in zip(stops, new_stops):
+            stop['value'] = new_value
+        self._class_logger.info(
+            'Updating stop values on layer number `%s` in `%s`', layer_number, self.webmap_item.title
+        )
+        result = self.webmap_item.update(item_properties={'text': json.dumps(data)})
+        self._class_logger.debug('Update result: %s', result)
+
+        return result
+
+    def update_color_ramp_values(self, layer_name, column_name, stops=5):
+        """Update the color ramp ranges for layer_name in self.webmap_item.
+
+        Does not alter colors or introduce additional stops; only overwrites the values for existing breaks.
+
+        Args:
+            layer_name (str): The exact name of the layer to be updated
+            column_name (str): The name of the attribute being displayed as an (un)classified range
+            stops (int, optional): The number of stops to calculate. Must match existing stops. Defaults to 5.
+
+        Returns:
+            Bool: Success or failure of update operation
+        """
+        layer_id = self._get_layer_id(layer_name)
+        dataframe = self._get_layer_dataframe(layer_name)
+        new_stops = self._calculate_new_stops(dataframe, column_name, stops)
+        result = self._update_stop_values(layer_id, new_stops)
+
+        return result
diff --git a/src/palletjack/transform.py b/src/palletjack/transform.py
index d75cce7..05d76dd 100644
--- a/src/palletjack/transform.py
+++ b/src/palletjack/transform.py
@@ -1,26 +1,40 @@
-"""transform.py: Processes in the Transfomation step of ETL
+"""Transform pandas dataframes in preparation for loading to AGOL.
 """
 import logging
 import warnings
 from datetime import datetime
 
+import arcgis
 import pandas as pd
 from arcgis import GeoAccessor, GeoSeriesAccessor
 
-from . import utils
+from palletjack import utils
 
 module_logger = logging.getLogger(__name__)
 
 
 class APIGeocoder:
-    """Geocode using the UGRC Web API Geocoder.
+    """Geocode a dataframe using the UGRC Web API Geocoder.
 
-    Instantiate an APIGeocoder object with an api key from developer.mapserv.utah.gov
+    Instantiate an APIGeocoder object with an api key from developer.mapserv.utah.gov. It will attempt to validate the
+    API key. If validation fails, it will raise one of the following errors:
+
+    - RuntimeError: If there was a network or other error
+    - ValueError: If the key is invalid
+    - UserWarning: If the API responds with some other abnormal result
+
+    The individual geocoding steps are exposed in the `palletjack.utils.Geocoding` class in the utils module for use in
+    other settings.
     """
 
     def __init__(self, api_key):
+        """
+        Args:
+            api_key (str): API key obtained from developer.mapserv.utah.gov
+        """
         self.api_key = api_key
         self._class_logger = logging.getLogger(__name__).getChild(self.__class__.__name__)
+        utils.Geocoding.validate_api_key(self.api_key)
 
     def geocode_dataframe(self, dataframe, street_col, zone_col, wkid, rate_limits=(0.015, 0.03), **api_args):
         """Geocode a pandas dataframe into a spatially-enabled dataframe
@@ -33,7 +47,7 @@ def geocode_dataframe(self, dataframe, street_col, zone_col, wkid, rate_limits=(
             zone_col (str): The column containing either the zip code or the city name
             wkid (int): The projection to return the x/y points in
             rate_limits(Tuple <float>): A lower and upper bound in seconds for pausing between API calls. Defaults to
-            (0.015, 0.03)
+                (0.015, 0.03)
             **api_args (dict): Keyword arguments to be passed as parameters in the API GET call.
 
         Returns:
@@ -46,28 +60,27 @@ def geocode_dataframe(self, dataframe, street_col, zone_col, wkid, rate_limits=(
         if dataframe.empty:
             warnings.warn('No records to geocode (empty dataframe)', RuntimeWarning)
 
-        self._class_logger.debug('Renaming columns to conform to ArcGIS limitations, if necessary...')
-        dataframe.rename(columns=utils.rename_columns_for_agol(dataframe.columns), inplace=True)
-
         dataframe_length = len(dataframe.index)
         reporting_interval = utils.calc_modulus_for_reporting_interval(dataframe_length)
         self._class_logger.info('Geocoding %s rows...', dataframe_length)
 
+        street_col_index = dataframe.columns.get_loc(street_col)
+        zone_col_index = dataframe.columns.get_loc(zone_col)
         new_rows = []
         for i, row in enumerate(dataframe.itertuples(index=False)):
             if i % reporting_interval == 0:
                 self._class_logger.info('Geocoding row %s of %s, %s%%', i, dataframe_length, i / dataframe_length * 100)
             row_dict = row._asdict()
-            results = utils.geocode_addr(
-                row_dict[street_col],
-                row_dict[zone_col],
+            results = utils.Geocoding.geocode_addr(
+                row[street_col_index],
+                row[zone_col_index],
                 self.api_key,
                 rate_limits,
                 spatialReference=str(wkid),
                 **api_args
             )
             self._class_logger.debug(
-                '%s of %s: %s, %s = %s', i, dataframe_length, row_dict[street_col], row_dict[zone_col], results
+                '%s of %s: %s, %s = %s', i, dataframe_length, row[street_col_index], row[zone_col_index], results
             )
             row_dict['x'], row_dict['y'], row_dict['score'], row_dict['matchAddress'] = results
             new_rows.append(row_dict)
@@ -81,3 +94,160 @@ def geocode_dataframe(self, dataframe, street_col, zone_col, wkid, rate_limits=(
         except ZeroDivisionError:
             warnings.warn('Empty spatial dataframe after geocoding', RuntimeWarning)
         return spatial_dataframe
+
+
+class FeatureServiceMerging:
+    """Get the live dataframe from a feature service and update it from another dataframe
+    """
+
+    @staticmethod
+    def update_live_data_with_new_data(live_dataframe, new_dataframe, join_column):
+        """Update a dataframe with data from another
+
+        Args:
+            live_dataframe (pd.DataFrame): The dataframe containing info to be updated
+            new_dataframe (pd.DataFrame): Dataframe containing source info to use in the update
+            join_column (str): The column with unique IDs to be used as a key between the two dataframes
+
+        Raises:
+            ValueError: If the join_column is missing from either live or new data
+            RuntimeWarning: If there are rows in the new data that are not found in the live data; these will not be
+                added to the live dataframe.
+
+        Returns:
+            pd.DataFrame: The updated dataframe, with data types converted via .convert_dtypes()
+        """
+
+        try:
+            live_dataframe.set_index(join_column, inplace=True)
+            new_dataframe.set_index(join_column, inplace=True)
+        except KeyError as error:
+            raise ValueError('Join column not found in live or new dataframes') from error
+
+        indicator_dataframe = live_dataframe.merge(new_dataframe, on=join_column, how='outer', indicator=True)
+        new_only_dataframe = indicator_dataframe[indicator_dataframe['_merge'] == 'right_only']
+        if not new_only_dataframe.empty:
+            keys_not_found = list(new_only_dataframe.index)
+            warnings.warn(
+                f'The following keys from the new data were not found in the existing dataset: {keys_not_found}',
+                RuntimeWarning
+            )
+
+        live_dataframe.update(new_dataframe)
+        return (live_dataframe.reset_index().convert_dtypes())
+
+    @staticmethod
+    def get_live_dataframe(gis, feature_service_itemid, layer_index=0):
+        """Get a spatially-enabled dataframe representation of a hosted feature layer
+
+        Args:
+            gis (arcgis.gis.GIS): GIS object of the desired organization
+            feature_service_itemid (str): itemid in the gis of the desired hosted feature service
+            layer_index (int, optional): Index of the desired layer within the hosted feature service. Defaults to 0.
+
+        Raises:
+            RuntimeError: If it fails to load the data
+
+        Returns:
+            pd.DataFrame.spatial: Spatially-enabled dataframe representation of the hosted feature layer
+        """
+
+        try:
+            feature_layer = arcgis.features.FeatureLayer.fromitem(
+                gis.content.get(feature_service_itemid), layer_id=layer_index
+            )
+            live_dataframe = feature_layer.query(as_df=True)
+        except Exception as error:
+            raise RuntimeError('Failed to load live dataframe') from error
+
+        return live_dataframe
+
+
+class DataCleaning:
+    """Static methods for cleaning dataframes prior to uploading to AGOL
+    """
+
+    @staticmethod
+    def switch_to_nullable_int(dataframe, fields_that_should_be_ints):
+        """Convert specified fields to panda's nullable Int64 type to preserve int to EsriFieldTypeInteger mapping
+
+        Args:
+            dataframe (pd.DataFrame): Input dataframe with columns to be converted
+            fields_that_should_be_ints (list[str]): List of column names to be converted
+
+        Raises:
+            TypeError: If any of the conversions fail. Often caused by values that aren't int-castable floats (ie. x.0)
+                or np.nans.
+
+        Returns:
+            pd.DataFrame: Input dataframe with columns converted to nullable Int64
+        """
+
+        int_dict = {field: 'Int64' for field in fields_that_should_be_ints}
+        try:
+            retyped = dataframe.astype(int_dict)
+        except TypeError as error:
+            raise TypeError(
+                'Cannot convert one or more fields to nullable ints. Check for non-int/non-np.nan values.'
+            ) from error
+        return retyped
+
+    @staticmethod
+    def switch_to_float(dataframe, fields_that_should_be_floats):
+        """Convert specified fields to float, converting empty strings to None first as required
+
+        Args:
+            dataframe (pd.DataFrame): Input dataframe with columns to be converted
+            fields_that_should_be_floats (list[str]): List of column names to be converted
+
+        Raises:
+            TypeError: If any of the conversions fail. Often caused by values that aren't castable to floats
+                (non-empty, non-numeric strings, etc)
+
+        Returns:
+            pd.DataFrame: Input dataframe with columns converted to float
+        """
+
+        float_dict = {field: 'float' for field in fields_that_should_be_floats}
+        empty_string_dict = {field: {'': None} for field in fields_that_should_be_floats}
+        try:
+            no_empty_strings = dataframe.replace(empty_string_dict)
+            retyped = no_empty_strings.astype(float_dict)
+        except TypeError as error:
+            raise TypeError(
+                'Cannot convert one or more fields to floats. Check for non-float/non-null values.'
+            ) from error
+        return retyped
+
+    @staticmethod
+    def switch_to_datetime(dataframe, date_fields, **to_datetime_kwargs):
+        """Convert specified fields to datetime dtypes to ensure proper date formatting for AGOL
+
+        Args:
+            dataframe (pd.DataFrame): The source dataframe
+            date_fields (List[int]): The fields to convert to datetime
+            **to_datetime_kwargs (keyword arguments, optional): Arguments to pass through to pd.to_datetime
+
+        Returns:
+            pd.DataFrame: The source dataframe with converted fields.
+        """
+
+        for field in date_fields:
+            dataframe[field] = pd.to_datetime(dataframe[field], **to_datetime_kwargs).dt.tz_localize(None)
+
+        return dataframe
+
+    @staticmethod
+    def rename_dataframe_columns_for_agol(dataframe):
+        """Rename all the columns in a dataframe to valid AGOL column names
+
+        Args:
+            dataframe (pd.DataFrame): Dataframe to be renamed
+
+        Returns:
+            pd.DataFrame: Input dataframe with renamed columns
+        """
+
+        rename_dict = utils.rename_columns_for_agol(dataframe.columns)
+        renamed_dataframe = dataframe.rename(columns=rename_dict)
+        return renamed_dataframe
diff --git a/src/palletjack/updaters.py b/src/palletjack/updaters.py
deleted file mode 100644
index f0eb582..0000000
--- a/src/palletjack/updaters.py
+++ /dev/null
@@ -1,821 +0,0 @@
-"""Objects for bringing data from SFTP-hosted CSVs into AGOL Feature Services
-"""
-
-import datetime
-import json
-import logging
-import warnings
-from pathlib import Path
-
-import arcgis
-import numpy as np
-import pandas as pd
-from arcgis.features import GeoAccessor, GeoSeriesAccessor
-
-from . import utils
-
-logger = logging.getLogger(__name__)
-
-
-class FeatureServiceInlineUpdater:
-    """Updates an AGOL Feature Service with data from a pandas DataFrame
-    """
-
-    def __init__(self, gis, dataframe, index_column, field_mapping=None):
-        self.gis = gis
-        if field_mapping:
-            dataframe = utils.rename_fields(dataframe, field_mapping)
-        self.new_dataframe = dataframe.rename(columns=utils.rename_columns_for_agol(dataframe.columns))
-        self.index_column = utils.rename_columns_for_agol([index_column])[index_column]
-        try:
-            self.new_data_as_dict = self.new_dataframe.set_index(self.index_column).to_dict('index')
-        except KeyError as error:
-            raise KeyError(f'Index column {index_column} not found in dataframe columns') from error
-        self._class_logger = logging.getLogger(__name__).getChild(self.__class__.__name__)
-
-    def update_existing_features_in_feature_service_with_arcpy(self, feature_service_url, fields):
-        """Update a feature service in-place with data from pandas data frame using arcpy's UpdateCursor
-
-        Args:
-            feature_service_url (str): URL to feature service
-            fields (list): Names of fields to update
-        """
-
-        #: Put this here to enable the package to install/run without arcpy installed if desired
-        try:
-            import arcpy  # pylint: disable=import-outside-toplevel
-        except ImportError as error:
-            raise ImportError('Failure importing arcpy. ArcGIS Pro must be installed.') from error
-
-        self._class_logger.info('Updating `%s` in-place', feature_service_url)
-        self._class_logger.debug('Updating fields %s', fields)
-        rows_updated = 0
-        with arcpy.da.UpdateCursor(feature_service_url, fields) as update_cursor:
-            for row in update_cursor:
-                self._class_logger.debug('Evaluating row: %s', row)
-                key = row[0]
-                if key in self.new_data_as_dict:
-                    row[1:] = list(self.new_data_as_dict[key].values())
-                    self._class_logger.debug('Updating row: %s', row)
-                    try:
-                        update_cursor.updateRow(row)
-                        rows_updated += 1
-                    except RuntimeError as error:
-                        if 'The value type is incompatible with the field type' in str(error):
-                            raise ValueError('Field type mismatch between dataframe and feature service') from error
-        self._class_logger.info('%s rows updated', rows_updated)
-
-        return rows_updated
-
-    def _validate_working_fields_in_live_and_new_dataframes(self, live_dataframe, fields):
-        live_fields = set(live_dataframe.columns)
-        new_fields = set(self.new_dataframe.columns)
-        working_fields = set(fields)
-
-        live_dif = working_fields - live_fields
-        new_dif = working_fields - new_fields
-
-        if live_dif or new_dif:
-            raise RuntimeError(
-                f'Field mismatch between defined fields and either new or live data.\nFields not in live data: '
-                f'{live_dif}\nFields not in new data: {new_dif}'
-            )
-
-    def _get_common_rows(self, live_dataframe) -> pd.DataFrame:
-        """Create a dataframe containing only the rows common to both the existing live dataset and the new updates
-
-        Args:
-            live_dataframe (pd.Dataframe): The existing, live dataframe created from an AGOL hosted feature layer.
-
-        Returns:
-            pd.DataFrame: A new dataframe containing the common rows created by .merge()ing.
-        """
-
-        joined_dataframe = live_dataframe.merge(self.new_dataframe, on=[self.index_column], how='outer', indicator=True)
-        subset_dataframe = joined_dataframe[joined_dataframe['_merge'] == 'both'].copy()
-
-        not_found_dataframe = joined_dataframe[joined_dataframe['_merge'] == 'right_only'].copy()
-        if not not_found_dataframe.empty:
-            keys_not_found = list(not_found_dataframe[self.index_column])
-            self._class_logger.warning(
-                'The following keys from the new data were not found in the existing dataset: %s', keys_not_found
-            )
-
-        return subset_dataframe
-
-    def _clean_dataframe_columns(self, dataframe, fields) -> pd.DataFrame:
-        """Delete superfluous fields from dataframe that will be used for the .edit() operation
-
-        Removes or renames the '_x', '_y' fields that are created by dataframe.merge() as appropriate based on the
-        fields provided.
-
-        Args:
-            dataframe (pd.Dataframe): Dataframe to be cleaned
-            fields (list[str]): The fields to keep; all others will be deleted
-
-        Returns:
-            pd.DataFrame: A dataframe with only our desired columns.
-        """
-
-        #: rename specified fields, delete any other _x/_y fields (in effect, just keep the specified fields...)
-        #: TODO: There may be a simpler way to do this- maybe .reindex(fields)?
-        rename_dict = {f'{f}_y': f for f in fields}
-        renamed_dataframe = dataframe.rename(columns=rename_dict).copy()
-        fields_to_delete = [f for f in renamed_dataframe.columns if f.endswith(('_x', '_y'))]
-        fields_to_delete.append('_merge')
-        self._class_logger.debug('Deleting joined dataframe fields: %s', fields_to_delete)
-        return renamed_dataframe.drop(labels=fields_to_delete, axis='columns', errors='ignore').copy()
-
-    def _get_old_and_new_values(self, live_dict, object_ids):
-        """Create a dictionary of the old (existing, live) and new values based on a list of object ids
-
-        Args:
-            live_dict (dict): The old (existing, live) data, key is an arbitrary index, content is another dict keyed
-            by the field names.
-            object_ids (list[int]): The object ids to include in the new dictionary
-
-        Returns:
-            dict: {object_id:
-                    {'old_values': {data as dict from live_dict},
-                     'new_values': {data as dict from self.new_dataframe}
-                    },
-                  ...,
-                  }
-        """
-        oid_and_key_lookup = {
-            row['OBJECTID']: row[self.index_column] for _, row in live_dict.items() if row['OBJECTID'] in object_ids
-        }
-        old_values_by_oid = {row['OBJECTID']: row for _, row in live_dict.items() if row['OBJECTID'] in object_ids}
-        new_data_as_dict_preserve_key = self.new_dataframe.set_index(self.index_column, drop=False).to_dict('index')
-        new_values_by_oid = {
-            object_id: new_data_as_dict_preserve_key[key] for object_id, key in oid_and_key_lookup.items()
-        }
-
-        combined_values_by_oid = {
-            object_id: {
-                'old_values': old_values_by_oid[object_id],
-                'new_values': new_values_by_oid[object_id]
-            } for object_id in oid_and_key_lookup
-        }
-
-        return combined_values_by_oid
-
-    def _parse_results(self, results_dict, live_dataframe) -> int:
-        """Integrate .edit() results with data for reporting purposes. Relies on _get_old_and_new_values().
-
-        Args:
-            results_dict (dict): AGOL response as a python dict (raw output from .edit_features(). Defined in
-            https://developers.arcgis.com/rest/services-reference/enterprise/apply-edits-feature-service-layer-.htm,
-            where `true`/`false` are python True/False.
-            live_dataframe (pd.DataFrame): Existing/live dataframe created from hosted feature layer.
-
-        Returns:
-            int: Number of edits successfully applied. If any failed, rollback will cause this to be set to 0.
-        """
-
-        live_dataframe_as_dict = live_dataframe.to_dict('index')
-        update_results = results_dict['updateResults']
-        if not update_results:
-            self._class_logger.info('No update results returned; no updates attempted')
-            return 0
-        update_successes = [result['objectId'] for result in update_results if result['success']]
-        update_failures = [result['objectId'] for result in update_results if not result['success']]
-        rows_updated = len(update_successes)
-        if update_successes:
-            self._class_logger.info('%s rows successfully updated:', len(update_successes))
-            for _, data in self._get_old_and_new_values(live_dataframe_as_dict, update_successes).items():
-                self._class_logger.debug('Existing data: %s', data['old_values'])
-                self._class_logger.debug('New data: %s', data['new_values'])
-        if update_failures:
-            self._class_logger.warning(
-                'The following %s updates failed. As a result, all successful updates should have been rolled back.',
-                len(update_failures)
-            )
-            for _, data in self._get_old_and_new_values(live_dataframe_as_dict, update_failures).items():
-                self._class_logger.warning('Existing data: %s', data['old_values'])
-                self._class_logger.warning('New data: %s', data['new_values'])
-            rows_updated = 0
-
-        return rows_updated
-
-    def update_existing_features_in_hosted_feature_layer(self, feature_layer_itemid, fields) -> int:
-        """Update existing features with new attribute data in the defined fields using arcgis instead of arcpy.
-
-        Relies on new data from self.new_dataframe. Uses the ArcGIS API for Python's .edit_features() method on a
-        hosted feature layer item. May be fragile for large datasets, per .edit_features() documentation. Can't get
-        .append() working properly.
-
-        Args:
-            feature_layer_itemid (str): The AGOL item id of the hosted feature layer to update.
-            fields (list[str]): The field names in the feature layer to update.
-
-        Returns:
-            int: Number of features successfully updated (any failures will cause rollback and should return 0)
-        """
-
-        self._class_logger.info('Updating itemid `%s` in-place', feature_layer_itemid)
-        self._class_logger.debug('Updating fields %s', fields)
-        feature_layer_item = self.gis.content.get(feature_layer_itemid)
-        feature_layer = arcgis.features.FeatureLayer.fromitem(feature_layer_item)
-        live_dataframe = pd.DataFrame.spatial.from_layer(feature_layer)
-        self._validate_working_fields_in_live_and_new_dataframes(live_dataframe, fields)
-        subset_dataframe = self._get_common_rows(live_dataframe)
-        if subset_dataframe.empty:
-            self._class_logger.warning(
-                'No matching rows between live dataset and new dataset based on field `%s`', self.index_column
-            )
-            return 0
-        cleaned_dataframe = self._clean_dataframe_columns(subset_dataframe, fields)
-        results = feature_layer.edit_features(
-            updates=cleaned_dataframe.spatial.to_featureset(), rollback_on_failure=True
-        )
-        log_fields = ['OBJECTID']
-        log_fields.extend(fields)
-        number_of_rows_updated = self._parse_results(results, live_dataframe[log_fields])
-        return number_of_rows_updated
-
-    def _upsert_new_data(self, target_featurelayer, dataframe, feature_service_item_id, layer_index):
-        """UPdate and inSERT new data to live dataset with featurelayer.append()
-
-        Args:
-            target_featurelayer (arcgis.features.FeatureLayer): Live dataset
-            dataframe (pd.DataFrame): Spatially-enabled dataframe containing new data. Column names should match live
-            data. New data must have a valid shape column
-            feature_service_item_id (int): Layer ID within the main feature service
-            layer_index (str): AGOL item ID for the main feature service
-
-        Raises:
-            RuntimeError: If append fails; will attempt to rollback appends that worked.
-
-        Returns:
-            dict: Messages returned from append operation
-        """
-
-        feature_set = utils.fix_numeric_empty_strings(
-            dataframe.spatial.to_featureset(), target_featurelayer.properties.fields
-        )
-        result, messages = utils.retry(
-            target_featurelayer.append,
-            upload_format='geojson',
-            edits=feature_set.to_geojson,
-            upsert=True,
-            return_messages=True,
-            rollback=True,
-            upsert_matching_field=self.index_column
-        )
-
-        self._class_logger.debug(messages)
-        if not result:
-            raise RuntimeError(
-                f'Failed to append data to layer id {layer_index} in itemid {feature_service_item_id}. Append should'
-                ' have been rolled back.'
-            )
-
-        return messages
-
-    def upsert_new_data_in_hosted_feature_layer(self, feature_service_item_id, layer_index=0):
-        """UPdate existing data and inSERT new data to a hosted feature layer from a spatially-enabled data frame
-
-        Relies on the FeatureServiceInlineUpdater's self.new_dataframe for the new data and self.index_column to define
-        the common column used to match new and existing data. The field specified by self.index_column must have a
-        "unique constraint" in the target feature layer (ie, it must be indexed and be unique).
-
-        Args:
-            feature_service_item_id (arcgis.features.FeatureLayer): The target feature layer
-            layer_index (int, optional): The layer id within the target feature layer. Defaults to 0.
-
-        Raises:
-            RuntimeError: If append fails; will attempt to rollback appends that worked.
-            RuntimeError: If new data contains a field not present in the live data
-            Warning: If live data contains a field not present in the new data
-            RuntimeError: If index_column is not in featurelayer's fields
-            RuntimeError: If the field is not unique (or if it's indexed but not unique)
-
-        Returns:
-            int: The number of records updated
-        """
-
-        target_featurelayer = arcgis.features.FeatureLayer.fromitem(
-            self.gis.content.get(feature_service_item_id), layer_id=layer_index
-        )
-        #: temp fix until Esri fixes empty series as NaN bug
-        fixed_dataframe = utils.replace_nan_series_with_empty_strings(self.new_dataframe)
-        utils.check_fields_match(target_featurelayer, fixed_dataframe)
-        utils.check_index_column_in_feature_layer(target_featurelayer, self.index_column)
-        utils.check_field_set_to_unique(target_featurelayer, self.index_column)
-        self._class_logger.info('Loading new data...')
-        messages = self._upsert_new_data(target_featurelayer, fixed_dataframe, feature_service_item_id, layer_index)
-
-        return messages['recordCount']
-
-
-class FeatureServiceAttachmentsUpdater:
-    """Add or overwrite attachments in a feature service using a dataframe of the desired "new" attachments. Uses a
-    join field present in both live data and the attachments dataframe to match attachments with live data.
-    """
-
-    def __init__(self, gis):
-        self.gis = gis
-        self._class_logger = logging.getLogger(__name__).getChild(self.__class__.__name__)
-        self.failed_dict = {}
-
-    def _get_live_oid_and_guid_from_join_field_values(self, live_features_as_df, attachment_join_field, attachments_df):
-        """Get the live Object ID and guid from the live features for only the features in attachments_df
-
-        Args:
-            live_features_as_df (pd.DataFrame): Spatial dataframe of all the feature layer's live data from AGOL
-            attachment_join_field (str): Column in attachments_df to use as a join key with live data
-            attachments_df (pd.DataFrame): New attachment data, including the join key and a path to the "new"
-                                           attachment
-
-        Returns:
-            pd.DataFrame: Attachments dataframe with corresponding live OIDs and GUIDs.
-        """
-
-        self._class_logger.debug('Using %s as the join field between live and new data', attachment_join_field)
-        subset_df = live_features_as_df.reindex(columns=['OBJECTID', 'GlobalID', attachment_join_field])
-        merged_df = subset_df.merge(attachments_df, on=attachment_join_field, how='inner')
-        self._class_logger.debug('%s features common to both live and new data', len(merged_df.index))
-
-        return merged_df
-
-    def _get_current_attachment_info_by_oid(self, live_data_subset_df):
-        """Merge the live attachment data using the ObjectID as the join
-
-        Args:
-            live_data_subset_df (pd.DataFrame): Live data with 'OBJECTID', 'GlobalID', and new attachment data
-
-        Returns:
-            pd.DataFrame: Live and new attachment data in one dataframe
-        """
-
-        live_attachments_df = pd.DataFrame(self.feature_layer.attachments.search())
-        live_attachments_subset_df = live_attachments_df.reindex(columns=['PARENTOBJECTID', 'NAME', 'ID'])
-        merged_df = live_data_subset_df.merge(
-            live_attachments_subset_df, left_on='OBJECTID', right_on='PARENTOBJECTID', how='left'
-        )
-        #: Cast ID field to nullable int to avoid conversion to float for np.nans
-        merged_df['ID'] = merged_df['ID'].astype('Int64')
-
-        return merged_df
-
-    def _create_attachment_action_df(self, attachment_eval_df, attachment_path_field):
-        """Create a dataframe containing the action needed for each feature resulting from the attachment join.
-
-        If the live feature doesn't have an attachment, add the attachment. If it does, compare the file names and only
-        attach if they are different. Otherwise, leave null.
-
-        Args:
-            attachment_eval_df (pd.DataFrame): DataFrame of live attachment data, subsetted to features that matched
-                                               the join key in the new attachments
-            attachment_path_field (str): The column that holds the attachment path
-
-        Returns:
-            pd.DataFrame: attachment_eval_df with 'operation' and 'new_filename' columns added
-        """
-
-        #: Get the file name from the full path
-        attachment_eval_df['new_filename'] = attachment_eval_df[attachment_path_field].apply(
-            lambda path: Path(path).name
-        )
-
-        #: Overwrite if different names, add if no existing name, do nothing if names are the same
-        attachment_eval_df['operation'] = np.nan
-        attachment_eval_df.loc[attachment_eval_df['NAME'] != attachment_eval_df['new_filename'],
-                               'operation'] = 'overwrite'
-        attachment_eval_df.loc[attachment_eval_df['NAME'].isna(), 'operation'] = 'add'
-
-        value_counts = attachment_eval_df['operation'].value_counts(dropna=False)
-        for operation in ['add', 'overwrite', np.nan]:
-            if operation not in value_counts:
-                value_counts[operation] = 0
-        self._class_logger.debug(
-            'Calculated attachment operations: adds: %s, overwrites: %s, none: %s', value_counts['add'],
-            value_counts['overwrite'], value_counts[np.nan]
-        )
-
-        return attachment_eval_df
-
-    def _add_attachments_by_oid(self, attachment_action_df, attachment_path_field):
-        """Add attachments using the feature's OID based on the 'operation' field of the dataframe
-
-        Args:
-            attachment_action_df (pd.DataFrame): A dataframe containing 'operation', 'OBJECTID', and
-                                                 attachment_path_field columns
-            attachment_path_field (str): The column that holds the attachment path
-
-        Returns:
-            int: The number of features that successfully have attachments added.
-        """
-
-        adds_dict = attachment_action_df[attachment_action_df['operation'] == 'add'].to_dict(orient='index')
-        adds_count = 0
-
-        for row in adds_dict.values():
-            target_oid = row['OBJECTID']
-            filepath = row[attachment_path_field]
-
-            self._class_logger.debug('Add %s to OID %s', filepath, target_oid)
-            try:
-                result = self.feature_layer.attachments.add(target_oid, filepath)
-            except Exception:
-                self._class_logger.error('AGOL error while adding %s to OID %s', filepath, target_oid, exc_info=True)
-                self.failed_dict[target_oid] = ('add', filepath)
-                continue
-
-            self._class_logger.debug('%s', result)
-            if not result['addAttachmentResult']['success']:
-                warnings.warn(f'Failed to attach {filepath} to OID {target_oid}')
-                self.failed_dict[target_oid] = ('add', filepath)
-                continue
-
-            adds_count += 1
-
-        return adds_count
-
-    def _overwrite_attachments_by_oid(self, attachment_action_df, attachment_path_field):
-        """Overwrite attachments using the feature's OID based on the 'operation' field of the dataframe
-
-        Args:
-            attachment_action_df (pd.DataFrame): A dataframe containing 'operation', 'OBJECTID', 'ID', 'NAME', and
-                                                 attachment_path_field columns
-            attachment_path_field (str): The column that holds the attachment path
-
-        Returns:
-            int: The number of features that successfully have their attachments overwritten.
-        """
-
-        overwrites_dict = attachment_action_df[attachment_action_df['operation'] == 'overwrite'].to_dict(orient='index')
-        overwrites_count = 0
-
-        for row in overwrites_dict.values():
-            target_oid = row['OBJECTID']
-            filepath = row[attachment_path_field]
-            attachment_id = row['ID']
-            old_name = row['NAME']
-
-            self._class_logger.debug(
-                'Overwriting %s (attachment ID %s) on OID %s with %s', old_name, attachment_id, target_oid, filepath
-            )
-            try:
-                result = self.feature_layer.attachments.update(target_oid, attachment_id, filepath)
-            except Exception:
-                self._class_logger.error(
-                    'AGOL error while overwriting %s (attachment ID %s) on OID %s with %s',
-                    old_name,
-                    attachment_id,
-                    target_oid,
-                    filepath,
-                    exc_info=True
-                )
-                self.failed_dict[target_oid] = ('update', filepath)
-                continue
-
-            if not result['updateAttachmentResult']['success']:
-                warnings.warn(
-                    f'Failed to update {old_name}, attachment ID {attachment_id}, on OID {target_oid} with {filepath}'
-                )
-                self.failed_dict[target_oid] = ('update', filepath)
-                continue
-
-            overwrites_count += 1
-
-        return overwrites_count
-
-    @staticmethod
-    def _check_attachment_dataframe_for_invalid_column_names(attachment_dataframe, invalid_names):
-        invalid_names_index = pd.Index(invalid_names)
-        intersection = attachment_dataframe.columns.intersection(invalid_names_index)
-        if not intersection.empty:
-            raise RuntimeError(f'Attachment dataframe contains the following invalid names: {list(intersection)}')
-
-    def update_attachments(
-        self, feature_layer_itemid, attachment_join_field, attachment_path_field, attachments_df, layer_number=0
-    ):
-        """Update a feature layer's attachments based on info from a dataframe of desired attachment file names
-
-        Depends on a dataframe populated with a join key for the live data and the downloaded or locally-available
-        attachments. If the name of the "new" attachment is the same as an existing attachment for that feature, it is
-        not updated. If it is different or there isn't an existing attachment, the "new" attachment is attached to that
-        feature.
-
-        Args:
-            feature_layer_itemid (str): The AGOL Item ID of the feature layer to update
-            attachment_join_field (str): The field containing the join key between the attachments dataframe and the
-                                         live data
-            attachment_path_field (str): The field containing the desired attachment file path
-            attachments_df (pd.DataFrame): A dataframe of desired attachments, including a join key and the local path
-                                           to the attachment
-            layer_number (int, optional): The layer within the Item ID to update. Defaults to 0.
-
-        Returns:
-            (int, int): Tuple of counts of successful overwrites and adds.
-        """
-
-        self._class_logger.info('Updating attachments...')
-        #: These names are present in the live attachment data downloaded from AGOL. Because we merge the dataframes
-        #: later, we need to make sure they're not the same. There may be better ways of handling this that allows the
-        #: client names to be preserved, but for now force them to fix this.
-        self._check_attachment_dataframe_for_invalid_column_names(
-            attachments_df, invalid_names=['OBJECTID', 'PARENTOBJECTID', 'NAME', 'ID']
-        )
-        self._class_logger.debug('Using layer %s from item ID %s', layer_number, feature_layer_itemid)
-        self.feature_layer = self.gis.content.get(feature_layer_itemid).layers[layer_number]
-        live_features_as_df = pd.DataFrame.spatial.from_layer(self.feature_layer)
-        live_data_subset_df = self._get_live_oid_and_guid_from_join_field_values(
-            live_features_as_df, attachment_join_field, attachments_df
-        )
-        #: TODO: Make sure layer supports attachments so we don't get an arcgis error.
-        #: Check out the feature layer .properties and FeatureLayerManager.add_to_definition to check/enable?
-        attachment_eval_df = self._get_current_attachment_info_by_oid(live_data_subset_df)
-        attachment_action_df = self._create_attachment_action_df(attachment_eval_df, attachment_path_field)
-
-        overwrites_count = self._overwrite_attachments_by_oid(attachment_action_df, attachment_path_field)
-        adds_count = self._add_attachments_by_oid(attachment_action_df, attachment_path_field)
-        self._class_logger.info('%s attachments added, %s attachments overwritten', adds_count, overwrites_count)
-
-        return overwrites_count, adds_count
-
-    @staticmethod
-    def build_attachments_dataframe(input_dataframe, join_column, attachment_column, out_dir):
-        """Create an attachments dataframe by subsetting down to just the two fields and dropping any rows
-           with null/empty attachments
-
-        Args:
-            input_dataframe (pd.DataFrame): Input data containing at least the join and attachment filename columns
-            join_column (str): Unique key joining attachments to live data
-            attachment_column (str): Filename for each attachment
-            out_dir (str or Path): Output directory, will be used to build full path to attachment
-
-        Returns:
-            pd.DataFrame: Dataframe with join key, attachment name, and full attachment paths
-        """
-
-        input_dataframe[attachment_column].replace('', np.nan, inplace=True)  #: pandas doesn't see empty strings as NAs
-        attachments_dataframe = input_dataframe[[join_column, attachment_column]] \
-                                            .copy().dropna(subset=[attachment_column])
-        #: Create the full path by prepending the output directory using .apply and a lambda function
-        attachments_dataframe['full_file_path'] = attachments_dataframe[attachment_column] \
-                                                    .apply(lambda filename: str(Path(out_dir, filename)))
-
-        return attachments_dataframe
-
-
-class FeatureServiceOverwriter:
-    """Overwrites an AGOL Feature Service with data from a spatially-enabled DataFrame.
-    """
-
-    def __init__(self, gis):
-        self.gis = gis
-        self._class_logger = logging.getLogger(__name__).getChild(self.__class__.__name__)
-
-    def _truncate_existing_data(self, featurelayer, layer_index, itemid):
-        """Remove all existing features from live dataset
-
-        Args:
-            featurelayer (arcgis.features.FeatureLayer): Live data
-            layer_index (int): Layer id within the main feature service
-            itemid (str): AGOL item ID for the main feature service
-
-        Raises:
-            RuntimeError: If the truncate fails
-
-        Returns:
-            pd.DataFrame: The feature layer's data prior to truncating
-        """
-
-        old_data = featurelayer.query(as_df=True)
-        truncate_result = utils.retry(featurelayer.manager.truncate, asynchronous=True, wait=True)
-        self._class_logger.debug(truncate_result)
-        if truncate_result['status'] != 'Completed':
-            raise RuntimeError(f'Failed to truncate existing data from layer id {layer_index} in itemid {itemid}')
-        return old_data
-
-    def _append_new_data(self, target_featurelayer, dataframe, feature_service_item_id, layer_index):
-        """Add new data to live dataset
-
-        Args:
-            target_featurelayer (arcgis.features.FeatureLayer): Live dataset; should be empty
-            dataframe (pd.DataFrame): Spatially-enabled dataframe containing new data. Column names should match live
-            data
-            feature_service_item_id (int): Layer ID within the main feature service
-            layer_index (str): AGOL item ID for the main feature service
-
-        Raises:
-            RuntimeError: If append fails; will attempt to rollback appends that worked.
-
-        Returns:
-            dict: Messages returned from append operation
-        """
-
-        geojson = dataframe.spatial.to_featureset().to_geojson
-        result, messages = utils.retry(
-            target_featurelayer.append,
-            upload_format='geojson',
-            edits=geojson,
-            #:TODO figure out a way to preserve GUIDs?
-            upsert=False,
-            return_messages=True,
-            rollback=True,
-        )
-
-        self._class_logger.debug(messages)
-        if not result:
-            raise RuntimeError(
-                f'Failed to append data to layer id {layer_index} in itemid {feature_service_item_id}. Append should'
-                ' have been rolled back.'
-            )
-
-        return messages
-
-    def _save_truncated_data(self, dataframe, directory):
-        """Save the pre-truncate dataframe to directory for safety
-
-        Args:
-            dataframe (pd.DataFrame): The data extracted from the feature layer prior to truncating
-            directory (str or Path): The directory to save the data to.
-
-        Returns:
-            Path: The full path to the output file, named with today's date.
-        """
-
-        out_path = Path(directory, f'old_data_{datetime.date.today()}.json')
-        out_path.write_text(dataframe.spatial.to_featureset().to_json)
-        return out_path
-
-    def truncate_and_load_feature_service(self, feature_service_item_id, new_dataframe, failsafe_dir, layer_index=0):
-        """Attempt to delete existing data from a feature layer and add new data from a spatially-enabled dataframe.
-
-        First attempts to truncate existing data. Then renames new data column names to conform to AGOL scheme (spaces,
-        special chars changed to '_'). Finally attempts to append new data to now-empty feature layer. If the new data
-        append fails, it attempts to re-upload the previous data from the in-memory dataframe. If this fails, it
-        attempts to failsafe by writing the old data to disk as a json file.
-
-        Args:
-            feature_service_item_id (str): AGOL item ID for feature service to truncate and load
-            new_dataframe (pd.DataFrame): Spatially-enabled dataframe containing new data. Must not contain columns
-            that do not exist in the live data. Geometry type must match existing data.
-            layer_index (int, optional): ID for the feature service layer to be truncated and loaded. Defaults to 0.
-
-        Raises:
-            RuntimeError: If new data contains a field not present in the live data
-            RuntimeError: If the truncate fails
-            RuntimeError: If append fails; will attempt to rollback appends that worked
-            RuntimeError: If rollback fails; will attempt to write old live data to disk as json
-
-        Returns:
-            int: The number of new records added after deleting all existing records.
-        """
-
-        target_featurelayer = arcgis.features.FeatureLayer.fromitem(
-            self.gis.content.get(feature_service_item_id), layer_id=layer_index
-        )
-        self._class_logger.info('Truncating existing data...')
-        old_dataframe = self._truncate_existing_data(target_featurelayer, layer_index, feature_service_item_id)
-        try:
-            cleaned_dataframe = new_dataframe.rename(columns=utils.rename_columns_for_agol(new_dataframe.columns))
-            #: temp fix until Esri fixes empty series as NaN bug
-            fixed_dataframe = utils.replace_nan_series_with_empty_strings(cleaned_dataframe)
-            utils.check_fields_match(target_featurelayer, fixed_dataframe)
-            self._class_logger.info('Loading new data...')
-
-            messages = self._append_new_data(target_featurelayer, fixed_dataframe, feature_service_item_id, layer_index)
-        except Exception:
-            try:
-                self._class_logger.info('Append failed; attempting to re-load truncated data...')
-                messages = self._append_new_data(
-                    target_featurelayer, old_dataframe, feature_service_item_id, layer_index
-                )
-                self._class_logger.info('%s features reloaded', messages['recordCount'])
-            except Exception as inner_error:
-                failsafe_path = self._save_truncated_data(old_dataframe, failsafe_dir)
-                raise RuntimeError(
-                    f'Failed to re-add truncated data after failed append; data saved to {failsafe_path}'
-                ) from inner_error
-            finally:
-                raise
-
-        return messages['recordCount']
-
-
-class ColorRampReclassifier:
-    """Updates the interval ranges on a layer's classification renderer based on the layer's current data.
-    """
-
-    def __init__(self, webmap_item, gis):
-        self.webmap_item = webmap_item
-        self.gis = gis
-        self._class_logger = logging.getLogger(__name__).getChild(self.__class__.__name__)
-
-    def _get_layer_dataframe(self, layer_name, feature_layer_number=0):
-        """Create a dataframe from layer_name in self.webmap_item
-
-        Args:
-            layer_name (str): The exact name of the layer
-            feature_layer_number (int): The number of the layer with the feature service to update. Defaults to 0.
-
-        Returns:
-            spatially-enabled data frame: The layer's data, including geometries.
-        """
-
-        self._class_logger.info('Getting dataframe from `%s` on `%s`', layer_name, self.webmap_item.title)
-        webmap_object = arcgis.mapping.WebMap(self.webmap_item)
-        layer = webmap_object.get_layer(title=layer_name)
-        feature_layer = self.gis.content.get(layer['itemId'])
-        layer_dataframe = pd.DataFrame.spatial.from_layer(feature_layer.layers[feature_layer_number])
-
-        return layer_dataframe
-
-    def _get_layer_id(self, layer_name):
-        """Get the ID number of layer_name in self.webmap_item
-
-        Args:
-            layer_name (str): The exact name of the layer
-
-        Raises:
-            ValueError: If the layer is not found in the webmap
-
-        Returns:
-            int: The index (0-based) of the the layer in the web map
-        """
-
-        data = self.webmap_item.get_data()
-        for layer_id, layer in enumerate(data['operationalLayers']):
-            if layer['title'] == layer_name:
-                self._class_logger.debug('Layer `%s` has id `%s`', layer_name, layer_id)
-                return layer_id
-
-        #: If we haven't matched the title and returned a valid id, raise an error.
-        raise ValueError(f'Could not find "{layer_name}" in {self.webmap_item.title}')
-
-    @staticmethod
-    def _calculate_new_stops(dataframe, column, stops):
-        """Calculate new stop values for an AGOL color ramp using what appears to be AGOL's method for unclassed ramps.
-
-        Args:
-            dataframe (pd.DataFrame): Data being classified
-            column (str): Column to classify
-            stops (int, optional): Number of stops to create.
-
-        Returns:
-            List: New stops cast as ints
-        """
-
-        if column not in dataframe.columns:
-            raise ValueError(f'Column `{column}` not in dataframe')
-        minval = dataframe[column].min()
-        mean = dataframe[column].mean()
-        std_dev = dataframe[column].std()
-        upper = mean + std_dev  #: AGOL's default upper value for unclassed ramps seems to be mean + 1 std dev
-
-        new_stops = np.linspace(minval, upper, stops)
-        new_stops_ints = [int(stop) for stop in new_stops]
-
-        return new_stops_ints
-
-    def _update_stop_values(self, layer_number, new_stops):
-        """Update the stop values of an (un)classified polygon renderer in an AGOL Webmap
-
-        Args:
-            layer_number (int): The index for the layer to be updated
-            new_stops (List): New values for the existing stops
-
-        Returns:
-            Bool: Success or failure of update operation
-        """
-
-        #: Get short reference to the stops dictionary from the webmap's data json
-        data = self.webmap_item.get_data()
-        renderer = data['operationalLayers'][layer_number]['layerDefinition']['drawingInfo']['renderer']
-        stops = renderer['visualVariables'][0]['stops']
-
-        #: Overwrite the value, update the webmap item
-        for stop, new_value in zip(stops, new_stops):
-            stop['value'] = new_value
-        self._class_logger.info(
-            'Updating stop values on layer number `%s` in `%s`', layer_number, self.webmap_item.title
-        )
-        result = self.webmap_item.update(item_properties={'text': json.dumps(data)})
-        self._class_logger.debug('Update result: %s', result)
-
-        return result
-
-    def update_color_ramp_values(self, layer_name, column_name, stops=5):
-        """Update the color ramp ranges for layer_name in self.webmap_item.
-
-        Does not alter colors or introduce additional stops; only overwrites the values for existing breaks.
-
-        Args:
-            layer_name (str): The exact name of the layer to be updated
-            column_name (str): The name of the attribute being displayed as an (un)classified range
-            stops (int, optional): The number of stops to calculate. Must match existing stops. Defaults to 5.
-
-        Returns:
-            Bool: Success or failure of update operation
-        """
-        layer_id = self._get_layer_id(layer_name)
-        dataframe = self._get_layer_dataframe(layer_name)
-        new_stops = self._calculate_new_stops(dataframe, column_name, stops)
-        result = self._update_stop_values(layer_id, new_stops)
-
-        return result
diff --git a/src/palletjack/utils.py b/src/palletjack/utils.py
index 4b88c52..9d64039 100644
--- a/src/palletjack/utils.py
+++ b/src/palletjack/utils.py
@@ -1,13 +1,24 @@
+"""Utility classes and methods that are used internally throughout palletjack. Many are exposed publicly in case they are useful elsewhere in a client's code.
+"""
+
+import datetime
+import importlib
 import logging
 import random
 import re
+import sys
+import warnings
 from math import floor
+from pathlib import Path
 from time import sleep
 
+import arcgis
 import pandas as pd
 import pygsheets
 import requests
 
+from palletjack.errors import IntFieldAsFloatError, TimezoneAwareDatetimeError
+
 module_logger = logging.getLogger(__name__)
 
 
@@ -22,7 +33,7 @@ def retry(worker_method, *args, **kwargs):
         worker_method (callable): The name of the method to be retried (minus the calling parens)
 
     Raises:
-        error: The final error the causes worker_method to fail after 3 retries
+        error: The final error that causes worker_method to fail after 3 retries
 
     Returns:
         various: The value(s) returned by worked_method
@@ -76,26 +87,7 @@ def rename_columns_for_agol(columns):
     return rename_dict
 
 
-def replace_nan_series_with_empty_strings(dataframe):
-    """Fill all completely empty series with empty strings ('')
-
-    As of arcgis 2.0.1. to_featureset() doesn't handle completely empty series properly (it relies on str;
-    https://github.com/Esri/arcgis-python-api/issues/1281), so we convert to empty strings for the time being.
-
-    Args:
-        dataframe (pd.DataFrame): Data to clean/fix
-
-    Returns:
-        pd.DataFrame: The cleaned data
-    """
-
-    for column in dataframe:
-        if dataframe[column].isnull().all():
-            module_logger.debug('Column %s is empty; replacing np.nans with empty strings', column)
-            dataframe[column].fillna(value='', inplace=True)
-    return dataframe
-
-
+#: Unused?
 def check_fields_match(featurelayer, new_dataframe):
     """Make sure new data doesn't have any extra fields, warn if it doesn't contain all live fields
 
@@ -126,6 +118,7 @@ def check_fields_match(featurelayer, new_dataframe):
         )
 
 
+#: Unused?
 def check_index_column_in_feature_layer(featurelayer, index_column):
     """Ensure index_column is present for any future operations
 
@@ -142,6 +135,7 @@ def check_index_column_in_feature_layer(featurelayer, index_column):
         raise RuntimeError(f'Index column {index_column} not found in feature layer fields {featurelayer_fields}')
 
 
+#: unused?
 def rename_fields(dataframe, field_mapping):
     """Rename fields based on field_mapping
 
@@ -182,6 +176,7 @@ def build_sql_in_list(series):
         return f'({", ".join(quoted_values)})'
 
 
+#: Unused in v3, but keeping for "unique constraint" info.
 def check_field_set_to_unique(featurelayer, field_name):
     """Makes sure field_name has a "unique constraint" in AGOL, which allows it to be used for .append upserts
 
@@ -202,88 +197,123 @@ def check_field_set_to_unique(featurelayer, field_name):
                 raise RuntimeError(f'{field_name} does not have a "unique constraint" set within the feature layer')
 
 
-def geocode_addr(street, zone, api_key, rate_limits, **api_args):
-    """Geocode an address through the UGRC Web API geocoder
+class Geocoding:
+    """Methods for geocoding an address
+    """
 
-    Invalid results are returned with an x,y of 0,0, a score of 0.0, and a match address of 'No Match'
+    @staticmethod
+    def geocode_addr(street, zone, api_key, rate_limits, **api_args):
+        """Geocode an address through the UGRC Web API geocoder
 
-    Args:
-        street (str): The street address
-        zone (str): The zip code or city
-        api_key (str): API key obtained from developer.mapserv.utah.gov
-        rate_limits(Tuple <float>): A lower and upper bound in seconds for pausing between API calls. Defaults to
-        (0.015, 0.03)
-        **api_args (dict): Keyword arguments to be passed as parameters in the API GET call. The API key will be added
-        to this dict.
+        Invalid results are returned with an x,y of 0,0, a score of 0.0, and a match address of 'No Match'
 
-    Returns:
-        tuple[int]: The match's x coordinate, y coordinate, score, and match address
-    """
+        Args:
+            street (str): The street address
+            zone (str): The zip code or city
+            api_key (str): API key obtained from developer.mapserv.utah.gov
+            rate_limits (Tuple <float>): A lower and upper bound in seconds for pausing between API calls. Defaults to
+                (0.015, 0.03)
+            **api_args (dict): Keyword arguments to be passed as parameters in the API GET call. The API key will be
+                added to this dict.
 
-    sleep(random.uniform(rate_limits[0], rate_limits[1]))
-    url = f'https://api.mapserv.utah.gov/api/v1/geocode/{street}/{zone}'
-    api_args['apiKey'] = api_key
+        Returns:
+            tuple[int]: The match's x coordinate, y coordinate, score, and match address
+        """
 
-    try:
-        geocode_result_dict = retry(_geocode_api_call, url, api_args)
-    except Exception as error:
-        module_logger.error(error)
-        return (0, 0, 0., 'No API response')
+        sleep(random.uniform(rate_limits[0], rate_limits[1]))
+        url = f'https://api.mapserv.utah.gov/api/v1/geocode/{street}/{zone}'
+        api_args['apiKey'] = api_key
 
-    return (
-        geocode_result_dict['location']['x'],
-        geocode_result_dict['location']['y'],
-        geocode_result_dict['score'],
-        geocode_result_dict['matchAddress'],
-    )
+        try:
+            geocode_result_dict = retry(Geocoding._geocode_api_call, url, api_args)
+        except Exception as error:
+            module_logger.error(error)
+            return (0, 0, 0., 'No API response')
+
+        return (
+            geocode_result_dict['location']['x'],
+            geocode_result_dict['location']['y'],
+            geocode_result_dict['score'],
+            geocode_result_dict['matchAddress'],
+        )
 
+    @staticmethod
+    def _geocode_api_call(url, api_args):
+        """Makes a requests.get call to the geocoding API.
 
-def _geocode_api_call(url, api_args):
-    """Makes a requests.get call to the geocoding API.
+        Meant to be called through a retry wrapper so that the RuntimeErrors get tried again a couple times before
+            finally raising the error.
 
-    Meant to be called through a retry wrapper so that the RuntimeErrors get tried again a couple times before finally
-    raising the error.
+        Args:
+            url (str): Base url for GET request
+            api_args (dict): Dictionary of URL parameters
 
-    Args:
-        url (str): Base url for GET request
-        api_args (dict): Dictionary of URL parameters
+        Raises:
+            RuntimeError: If the server does not return response and request.get returns a falsy object.
+            RuntimeError: If the server returns a status code other than 200 or 404
 
-    Raises:
-        RuntimeError: If the server does not return response and request.get returns a falsy object.
-        RuntimeError: If the server returns a status code other than 200 or 404
+        Returns:
+            dict: The 'results' dictionary of the response json (location, score, and matchAddress)
+        """
 
-    Returns:
-        dict: The 'results' dictionary of the response json (location, score, and matchAddress)
-    """
+        response = requests.get(url, params=api_args)
 
-    response = requests.get(url, params=api_args)
-
-    #: The server times out and doesn't respond
-    if response is None:
-        module_logger.debug('GET call did not return a response')
-        raise RuntimeError('No response from GET; request timeout?')
-
-    #: The point does geocode
-    if response.status_code == 200:
-        return response.json()['result']
-
-    #: The point doesn't geocode
-    if response.status_code == 404:
-        return {
-            'location': {
-                'x': 0,
-                'y': 0
-            },
-            'score': 0.,
-            'matchAddress': 'No Match',
-        }
+        #: The server times out and doesn't respond
+        if response is None:
+            module_logger.debug('GET call did not return a response')
+            raise RuntimeError('No response from GET; request timeout?')
+
+        #: The point does geocode
+        if response.status_code == 200:
+            return response.json()['result']
+
+        #: The point doesn't geocode
+        if response.status_code == 404:
+            return {
+                'location': {
+                    'x': 0,
+                    'y': 0
+                },
+                'score': 0.,
+                'matchAddress': 'No Match',
+            }
+
+        #: If we haven't returned, raise an error to trigger _retry
+        raise RuntimeError(f'Did not receive a valid geocoding response; status code: {response.status_code}')
+
+    @staticmethod
+    def validate_api_key(api_key):
+        """Check to see if a Web API key is valid by geocoding a single, known address point
+
+        Args:
+            api_key (str): API Key
+
+        Raises:
+            RuntimeError: If there was a network or other error attempting to geocode the known point
+            ValueError: If the API responds with an invalid key message
+            UserWarning: If the API responds with some other abnormal result
+        """
 
-    #: If we haven't returned, raise an error to trigger _retry
-    raise RuntimeError(f'Did not receive a valid geocoding response; status code: {response.status_code}')
+        url = 'https://api.mapserv.utah.gov/api/v1/geocode/326 east south temple street/slc'
+
+        try:
+            response = retry(requests.get, url=url, params={'apiKey': api_key})
+        except Exception as error:
+            raise RuntimeError(
+                'Could not determine key validity; check your API key and/or network connection'
+            ) from error
+        response_json = response.json()
+        if response_json['status'] == 200:
+            return
+        if response_json['status'] == 400 and 'Invalid API key' in response_json['message']:
+            raise ValueError(f'API key validation failed: {response_json["message"]}')
+
+        warnings.warn(f'Unhandled API key validation response {response_json["status"]}: {response_json["message"]}')
 
 
 def calc_modulus_for_reporting_interval(n, split_value=500):
-    """Calculate a number that can be used as a modulus for splitting n up into 10 or 20 intervals, depending on split_value.
+    """Calculate a number that can be used as a modulus for splitting n up into 10 or 20 intervals, depending on
+    split_value.
 
     Args:
         n (int): The number to divide into intervals
@@ -302,40 +332,15 @@ def calc_modulus_for_reporting_interval(n, split_value=500):
     return floor(n / 20)
 
 
-def validate_api_key(api_key):
-    """Check to see if a Web API key is valid by geocoding a single, known address point
-
-    Args:
-        api_key (str): API Key
-
-    Returns:
-        str: One of three messages:
-                'valid': The key is valid
-                A string containing 'Invalid API key': Response from the server if the API key is invalid
-                'Could not determine key validity; check your API key and/or network connection': If there was some
-                network or response error.
-    """
-
-    url = 'https://api.mapserv.utah.gov/api/v1/geocode/326 east south temple street/slc'
-
-    try:
-        response = retry(requests.get, url=url, params={'apiKey': api_key})
-        response_json = response.json()
-        if response_json['status'] == 200:
-            return 'valid'
-        if response_json['status'] == 400 and 'Invalid API key' in response_json['message']:
-            return response_json['message']
-    except Exception as error:
-        module_logger.debug(error)
-        # return 'Could not determine key validity; check your API key and/or network connection'
-    return 'Could not determine key validity; check your API key and/or network connection'
-
-
 def authorize_pygsheets(credentials):
     """Authenticate pygsheets using either a service file or google.auth.credentials.Credentials object.
 
-    Tries first to load credentials as a service account file; if this fails with a FileNotFoundError, tries to pass
-    credentials directly as a custom_credential.
+    Requires either the path to a service account .json file that has access to the files in question or  a `google.
+    auth.credentials.Credentials` object. Calling `google.auth.default()` in a Google Cloud Function will give you a
+    tuple of a `Credentials` object and the project id. You can use this `Credentials` object to authorize pygsheets as
+    the same account the Cloud Function is running under.
+
+    Tries first to load credentials from file; if this fails tries credentials directly as a custom_credential.
 
     Args:
         credentials (str or google.auth.credentials.Credentials): Path to the service file OR credentials object
@@ -350,7 +355,7 @@ def authorize_pygsheets(credentials):
 
     try:
         return pygsheets.authorize(service_file=credentials)
-    except FileNotFoundError as err:
+    except (FileNotFoundError, TypeError) as err:
         module_logger.debug(err)
         module_logger.debug('Credentials file not found, trying as environment variable')
     try:
@@ -359,9 +364,554 @@ def authorize_pygsheets(credentials):
         raise RuntimeError('Could not authenticate to Google API') from err
 
 
+def save_feature_layer_to_json(feature_layer, directory):
+    """Save a feature_layer to directory for safety as {layer name}_{todays date}.json
+
+    Args:
+        feature_layer (arcgis.features.FeatureLayer): The FeatureLayer object to save to disk.
+        directory (str or Path): The directory to save the data to.
+
+    Returns:
+        Path: The full path to the output file, named with the layer name and today's date.
+    """
+
+    module_logger.debug('Downloading existing data...')
+    dataframe = feature_layer.query().sdf
+
+    if dataframe.empty:
+        return f'No data to save in feature layer {feature_layer.properties.name}'
+
+    out_path = Path(directory, f'{feature_layer.properties.name}_{datetime.date.today()}.json')
+    module_logger.debug('Saving existing data to %s', out_path)
+    out_path.write_text(dataframe.spatial.to_featureset().to_json, encoding='utf-8')
+
+    return out_path
+
+
+class FieldChecker:
+    """Check the fields of a new dataframe against live data. Each method will raise errors if its checks fail.
+    Provides the check_fields class method to run all the checks in one call with having to create an object.
+    """
+
+    @classmethod
+    def check_fields(cls, live_data_properties, new_dataframe, fields, add_oid):
+        """Run all the field checks, raising errors and warnings where they fail.
+
+        Check individual method docstrings for details and specific errors raised.
+
+        Args:
+            live_data_properties (dict): FeatureLayer.properties of live data
+            new_dataframe (pd.DataFrame): New data to be checked
+            fields (List[str]): Fields to check
+            add_oid (bool): Add OBJECTID to fields if its not already present (for operations that are dependent on
+                OBJECTID, such as upsert)
+        """
+
+        field_checker = cls(live_data_properties, new_dataframe)
+        field_checker.check_fields_present(fields, add_oid=add_oid)
+        field_checker.check_live_and_new_field_types_match(fields)
+        field_checker.check_for_non_null_fields(fields)
+        field_checker.check_field_length(fields)
+        field_checker.check_srs_wgs84()
+        field_checker.check_nullable_ints_shapely()
+
+    def __init__(self, live_data_properties, new_dataframe):
+        """
+        Args:
+            live_data_properties (dict): FeatureLayer.properties of live data
+            new_dataframe (pd.DataFrame): New data to be checked
+        """
+
+        self.live_data_properties = live_data_properties
+        self.fields_dataframe = pd.DataFrame(live_data_properties.fields)
+        self.new_dataframe = new_dataframe
+
+    def check_live_and_new_field_types_match(self, fields):
+        """Raise an error if the field types of the live and new data don't match.
+
+        Uses a dictionary mapping Esri field types to pandas dtypes. If 'SHAPE' is included in the fields, it calls
+        _check_geometry_types to verify the spatial types are compatible.
+
+        Args:
+            fields (List[str]): Fields to be updated
+
+        Raises:
+            ValueError: If the field types or spatial types are incompatible, the new data has multiple geometry types,
+                or the new data is not a valid spatially-enabled dataframe.
+            NotImplementedError: If the live data has a field that has not yet been mapped to a pandas dtype.
+        """
+
+        #: Converting dtypes to str and comparing seems to be the only way to break out into shorts and longs, singles
+        #: and doubles. Otherwise, checking subclass is probably more pythonic.
+        short_ints = ['uint8', 'uint16', 'int8', 'int16']
+        long_ints = ['int', 'uint32', 'uint64', 'int32', 'int64']
+
+        #: Leaving the commented types here for future implementation if necessary
+        esri_to_pandas_types_mapping = {
+            'esriFieldTypeInteger': ['int'] + short_ints + long_ints,
+            'esriFieldTypeSmallInteger': short_ints,
+            'esriFieldTypeDouble': ['float', 'float32', 'float64'],
+            'esriFieldTypeSingle': ['float32'],
+            'esriFieldTypeString': ['str', 'object', 'string'],
+            'esriFieldTypeDate': ['datetime64[ns]'],
+            'esriFieldTypeGeometry': ['geometry'],
+            'esriFieldTypeOID': ['int'] + short_ints + long_ints,
+            #  'esriFieldTypeBlob': [],
+            'esriFieldTypeGlobalID': ['str', 'object', 'string'],
+            #  'esriFieldTypeRaster': [],
+            'esriFieldTypeGUID': ['str', 'object', 'string'],
+            #  'esriFieldTypeXML': [],
+        }
+
+        #: geometry checking gets its own function
+        if 'SHAPE' in fields:
+            self._check_geometry_types()
+            fields.remove('SHAPE')
+
+        fields_to_check = self.fields_dataframe[self.fields_dataframe['name'].isin(fields)].set_index('name')
+
+        invalid_fields = []
+        int_fields_as_floats = []
+        datetime_fields_with_timezone = []
+        for field in fields:
+            #: check against the str.lower to catch normal dtypes (int64) and the new, pd.NA-aware dtypes (Int64)
+            new_dtype = str(self.new_dataframe[field].dtype).lower()
+            live_type = fields_to_check.loc[field, 'type']
+
+            try:
+                if new_dtype not in esri_to_pandas_types_mapping[live_type]:
+                    invalid_fields.append((field, live_type, str(self.new_dataframe[field].dtype)))
+                if new_dtype in ['float', 'float32', 'float64'
+                                ] and live_type in ['esriFieldTypeInteger', 'esriFieldTypeSmallInteger']:
+                    int_fields_as_floats.append(field)
+                if 'datetime64' in new_dtype and new_dtype != 'datetime64[ns]' and live_type == 'esriFieldTypeDate':
+                    datetime_fields_with_timezone.append(field)
+            except KeyError:
+                # pylint: disable-next=raise-missing-from
+                raise NotImplementedError(f'Live field "{field}" type "{live_type}" not yet mapped to a pandas dtype')
+
+        if invalid_fields:
+            if int_fields_as_floats:
+                raise IntFieldAsFloatError(
+                    f'Field type incompatibilities (field, live type, new type): {invalid_fields}\n' \
+                    'Check the following int fields for null/np.nan values and convert to panda\'s nullable int '\
+                    f'dtype: {", ".join(int_fields_as_floats)}'
+                )
+            if datetime_fields_with_timezone:
+                raise TimezoneAwareDatetimeError(
+                    f'Field type incompatibilities (field, live type, new type): {invalid_fields}\n' \
+                    'Check the following datetime fields for timezone aware dtypes values and convert to '\
+                    'timezone-naive dtypes using pd.to_datetime(df[\'field\']).dt.tz_localize(None): '\
+                    f'{", ".join(datetime_fields_with_timezone)}'
+                )
+            raise ValueError(f'Field type incompatibilities (field, live type, new type): {invalid_fields}')
+
+    def _check_geometry_types(self):
+        """Raise an error if the live and new data geometry types are incompatible.
+
+        Args:
+            live_data_properties (dict): FeatureLayer.properties of live data
+            new_dataframe (pd.DataFrame): New data to be added/updated
+
+        Raises:
+            ValueError: If the new data is not a valid spatially-enabled dataframe, has multiple geometry types, or has
+                a geometry type that doesn't match the live data.
+        """
+
+        esri_to_sedf_geometry_mapping = {
+            'esriGeometryPoint': 'point',
+            'esriGeometryMultipoint': 'multipoint',
+            'esriGeometryPolyline': 'polyline',
+            'esriGeometryPolygon': 'polygon',
+            'esriGeometryEnvelope': 'envelope',
+        }
+
+        if 'SHAPE' not in self.new_dataframe.columns:
+            raise ValueError('New dataframe does not have a SHAPE column')
+
+        live_geometry_type = self.live_data_properties.geometryType
+        new_geometry_types = self.new_dataframe.spatial.geometry_type
+        if len(new_geometry_types) > 1:
+            raise ValueError('New dataframe has multiple geometry types')
+        if esri_to_sedf_geometry_mapping[live_geometry_type] != new_geometry_types[0].lower():
+            raise ValueError(
+                f'New dataframe geometry type "{new_geometry_types[0]}" incompatible with live geometry type "{live_geometry_type}"'
+            )
+
+    def check_for_non_null_fields(self, fields):
+        """Raise an error if the new data contains nulls in a field that the live data says is not nullable.
+
+        If this error occurs, the client should use pandas fillna() method to replace NaNs/Nones with empty strings or
+        appropriate nodata values.
+
+        Args:
+            fields (List[str]): Fields to check
+
+        Raises:
+            ValueError: If the new data contains nulls in a field that the live data says is not nullable and doesn't
+                have a default value.
+        """
+
+        columns_with_nulls = self.new_dataframe.columns[self.new_dataframe.isna().any()].tolist()
+        # fields_dataframe = pd.DataFrame(self.live_data_properties['fields'])
+        non_nullable_live_columns = self.fields_dataframe[
+            ~(self.fields_dataframe['nullable']) &
+            ~(self.fields_dataframe['defaultValue'].astype(bool))]['name'].tolist()
+
+        columns_to_check = [column for column in columns_with_nulls if column in fields]
+
+        #: If none of the columns have nulls, we don't need to check further
+        if not columns_to_check:
+            return
+
+        problem_fields = []
+        for column in columns_to_check:
+            if column in non_nullable_live_columns:
+                problem_fields.append(column)
+
+        if problem_fields:
+            raise ValueError(
+                f'The following fields cannot have null values in the live data but one or more nulls exist in the new data: {", ".join(problem_fields)}'
+            )
+
+    def check_field_length(self, fields):
+        """Raise an error if a new data string value is longer than allowed in the live data.
+
+        Args:
+            fields (List[str]): Fields to check
+
+        Raises:
+            ValueError: If the string fields in the new data contain a value longer than the corresponding field in the
+                live data allows.
+        """
+
+        if 'length' not in self.fields_dataframe.columns:
+            module_logger.debug('No fields with length property')
+            return
+
+        length_limited_fields = self.fields_dataframe[
+            (self.fields_dataframe['type'].isin(['esriFieldTypeString', 'esriFieldTypeGlobalID'])) &
+            (self.fields_dataframe['length'].astype(bool))]
+
+        columns_to_check = length_limited_fields[length_limited_fields['name'].isin(fields)]
+
+        for field, live_max_length in columns_to_check[['name', 'length']].to_records(index=False):
+            new_data_lengths = self.new_dataframe[field].str.len()
+            new_max_length = new_data_lengths.max()
+            if new_max_length > live_max_length:
+                raise ValueError(
+                    f'Row {new_data_lengths.argmax()}, column {field} in new data exceeds the live data max length of {live_max_length}'
+                )
+
+    def check_fields_present(self, fields, add_oid):
+        """Raise an error if the fields to be operated on aren't present in either the live or new data.
+
+        Args:
+            fields (List[str]): The fields to be operated on.
+            add_oid (bool): Add OBJECTID to fields if its not already present (for operations that are dependent on
+                OBJECTID, such as upsert)
+
+        Raises:
+            RuntimeError: If any of fields are not in live or new data.
+        """
+
+        live_fields = set(self.fields_dataframe['name'])
+        new_fields = set(self.new_dataframe.columns)
+        working_fields = set(fields)
+        working_fields.discard('SHAPE')  #: The fields from the feature layer properties don't include the SHAPE field.
+        if add_oid:
+            working_fields.add('OBJECTID')
+
+        live_dif = working_fields - live_fields
+        new_dif = working_fields - new_fields
+
+        error_message = []
+        if live_dif:
+            error_message.append(f'Fields missing in live data: {", ".join(live_dif)}')
+        if new_dif:
+            error_message.append(f'Fields missing in new data: {", ".join(new_dif)}')
+
+        if error_message:
+            raise RuntimeError('. '.join(error_message))
+
+    def check_srs_wgs84(self):
+        """Raise an error if the new spatial reference system isn't WGS84 as required by geojson.
+
+        Raises:
+            ValueError: If the new SRS value can't be cast to an int (please log an issue if this occurs)
+            ValueError: If the new SRS value isn't 4326.
+        """
+
+        #: If we modify a spatial data frame, sometimes the .sr.wkid property/dictionary becomes {0:number} instead
+        #: of {'wkid': number}
+        try:
+            new_srs = self.new_dataframe.spatial.sr.wkid
+        except AttributeError:
+            new_srs = self.new_dataframe.spatial.sr[0]
+
+        try:
+            new_srs = int(new_srs)
+        except ValueError as error:
+            raise ValueError('Could not cast new SRS to int') from error
+        if new_srs != 4326:
+            raise ValueError(
+                f'New dataframe SRS {new_srs} is not wkid 4326. Reproject with appropriate transformation.'
+            )
+
+    def check_nullable_ints_shapely(self):
+        """Raise a warning if null values occur within nullable integer fields of the dataframe
+
+        Apparently due to a convention within shapely, any null values in an integer field are converted to 0.
+
+        Raises:
+            UserWarning: If we're using shapely instead of arcpy, the new dataframe uses nullable int dtypes, and there
+                is one or more pd.NA values within a nullable int column.
+        """
+
+        #: Only occurs if client is using shapely instead of arcpy
+        if importlib.util.find_spec('arcpy'):
+            return
+
+        nullable_ints = {'Int8', 'Int16', 'Int32', 'Int64', 'UInt8', 'UInt16', 'UInt32', 'UInt64'}
+        nullable_int_columns = [
+            column for column in self.new_dataframe.columns if str(self.new_dataframe[column].dtype) in nullable_ints
+        ]
+        columns_with_nulls = [column for column in nullable_int_columns if self.new_dataframe[column].isnull().any()]
+
+        if columns_with_nulls:
+            warnings.warn(
+                'The following columns have null values that will be replaced by 0 due to shapely conventions: '\
+                    f'{", ".join(columns_with_nulls)}'
+            )
+
+
+def get_null_geometries(feature_layer_properties):
+    """Generate placeholder geometries near 0, 0 with type based on provided feature layer properties dictionary.
+
+    Args:
+        feature_layer_properties (dict): .properties from a feature layer item, contains 'geometryType' key
+
+    Raises:
+        NotImplementedError: If we get a geometryType we haven't implemented a null-geometry generator for
+
+    Returns:
+        arcgis.geometry.Geometry: A geometry object of the corresponding type centered around null island.
+    """
+
+    # esri_to_sedf_geometry_mapping = {
+    #     'esriGeometryPoint': 'point',
+    #     'esriGeometryMultipoint': 'multipoint',
+    #     'esriGeometryPolyline': 'polyline',
+    #     'esriGeometryPolygon': 'polygon',
+    #     'esriGeometryEnvelope': 'envelope',
+    # }
+
+    live_geometry_type = feature_layer_properties.geometryType
+
+    if live_geometry_type == 'esriGeometryPoint':
+        return arcgis.geometry.Point({'x': 0, 'y': 0, 'spatialReference': {'wkid': 4326}}).JSON
+
+    if live_geometry_type == 'esriGeometryPolyline':
+        return arcgis.geometry.Polyline({
+            'paths': [[[0, 0], [.1, .1], [.2, .2]]],
+            'spatialReference': {
+                'wkid': 4326
+            }
+        }).JSON
+
+    if live_geometry_type == 'esriGeometryPolygon':
+        return arcgis.geometry.Polygon({
+            'rings': [[[0, .1], [.1, .1], [.1, 0], [0, 0]]],
+            'spatialReference': {
+                'wkid': 4326
+            }
+        }).JSON
+
+    raise NotImplementedError(f'Null value generator for live geometry type {live_geometry_type} not yet implemented')
+
+
+class DeleteUtils:
+    """Verify Object IDs used for delete operations
+    """
+
+    @staticmethod
+    def check_delete_oids_are_ints(oid_list):
+        """Raise an error if a list of strings can't be parsed as ints
+
+        Args:
+            oid_list (list[int]): List of Object IDs to delete
+
+        Raises:
+            TypeError: If any of the items in oid_list can't be cast to ints
+
+        Returns:
+            list[int]: oid_list converted to ints
+        """
+
+        numeric_oids = []
+        bad_oids = []
+        for oid in oid_list:
+            try:
+                numeric_oids.append(int(oid))
+            except ValueError:
+                bad_oids.append(oid)
+        if bad_oids:
+            raise TypeError(f'Couldn\'t convert OBJECTID(s) `{bad_oids}` to integer')
+        return numeric_oids
+
+    @staticmethod
+    def check_for_empty_oid_list(oid_list, numeric_oids):
+        """Raise an error if the parsed Object ID list is empty
+
+        Args:
+            oid_list (list[int]): The original list of Object IDs to delete
+            numeric_oids (list[int]): The cast-to-int Object IDs
+
+        Raises:
+            ValueError: If numeric_oids is empty
+        """
+
+        if not numeric_oids:
+            raise ValueError(f'No OBJECTIDs found in {oid_list}')
+
+    @staticmethod
+    def check_delete_oids_are_in_live_data(oid_string, numeric_oids, feature_layer):
+        """Warn if a delete Object ID doesn't exist in the live data, return number missing
+
+        Args:
+            oid_string (str): Comma-separated string of delete Object IDs
+            numeric_oids (list[int]): The parsed and cast-to-int Object IDs
+            feature_layer (arcgis.features.FeatureLayer): Live FeatureLayer item
+
+        Raises:
+            UserWarning: If any of the Object IDs in numeric_oids don't exist in the live data.
+
+        Returns:
+            int: Number of Object IDs missing from live data
+        """
+
+        query_results = feature_layer.query(object_ids=oid_string, return_ids_only=True)
+        query_oids = query_results['objectIds']
+        oids_not_in_layer = set(numeric_oids) - set(query_oids)
+
+        if oids_not_in_layer:
+            warnings.warn(f'OBJECTIDs {oids_not_in_layer} were not found in the live data')
+
+        return len(oids_not_in_layer)
+
+
+class Chunking:
+    """Divide a dataframe into chunks to satisfy upload size requirements for append operation.
+    """
+
+    @staticmethod
+    def _ceildiv(num, denom):
+        """Perform ceiling division: 5/4 = 2
+
+        Args:
+            num (int or float): Numerator
+            denom (int or float): Denominator
+
+        Returns:
+            int: Ceiling divisor
+        """
+
+        return -(num // -denom)
+
+    @staticmethod
+    def _chunk_dataframe(dataframe, chunk_size):
+        """Divide up a dataframe into a list of dataframes of chunk_size rows
+
+        The DataFrames are returned in a list. Elements [:-1] are as large as possible for the number of chunks needed,
+        while the last gets however many rows of the dataframe are left over. eg, a 10-row dataframe broken into 3
+        chunks would result in dataframes with 3, 3, and 1 rows.
+
+        Args:
+            dataframe (pd.DataFrame): Input DataFrame
+            chunk_size (int): The max number of rows for each sub dataframe
+
+        Returns:
+            list[pd.DataFrame]: A list of dataframes with at most chunk_size rows per dataframe
+        """
+
+        df_length = len(dataframe)
+
+        starts = range(0, df_length, chunk_size)
+        ends = [start + chunk_size if start + chunk_size < df_length else df_length for start in starts]
+        list_of_dataframes = [dataframe.iloc[start:end] for start, end in zip(starts, ends)]
+
+        return list_of_dataframes
+
+    @staticmethod
+    def build_upload_json(dataframe, feature_layer_fields, max_bytes=100_000_000):
+        """Create list of geojson strings of spatially-enabled DataFrame, divided into chunks if it exceeds max_bytes
+
+        Recursively chunks dataframe to ensure no one chunk is larger than max_bytes. Converts all empty strings in
+        nullable numeric fields in feature sets created from individual chunks to None prior to converting to geojson to
+        ensure the field stays numeric.
+
+        Args:
+            dataframe (pd.DataFrame.spatial): Spatially-enabled dataframe to be converted to geojson
+            feature_layer_fields: All the fields from the feature layer (feature_layer.properties.fields)
+            max_bytes (int, optional): Maximum size in bytes any one geojson string can be. Defaults to 100000000 (AGOL
+                text uploads are limited to 100 MB?)
+
+        Returns:
+            list[str]: A list of the dataframe chunks converted to geojson
+        """
+
+        geojson_size = sys.getsizeof(dataframe.spatial.to_featureset().to_geojson.encode('utf-16'))
+        module_logger.debug('Initial file size: %s', geojson_size)
+
+        chunked_dataframes = Chunking._recursive_dataframe_chunking(dataframe, max_bytes)
+
+        chunked_geojsons = [
+            fix_numeric_empty_strings(chunk.spatial.to_featureset(), feature_layer_fields).to_geojson
+            for chunk in chunked_dataframes
+        ]
+
+        return chunked_geojsons
+
+    @staticmethod
+    def _recursive_dataframe_chunking(dataframe, max_bytes):
+        """Break a dataframe into chunks such that their utf-16 encoded geojson sizes don't exceed max_bytes
+
+        Divides the dataframe into chunks based on the geojson representation's utf-16-encoded size by calculating the
+        number of chunks of size > max_bytes needed for the entire file size. It uses this number of chunks to chunk the
+        dataframe based on rows. Because there can be variability in geojson file size due to attribute lengths
+        (especially line and polygon geometry sizes), it uses recursion to again chunk each smaller dataframe if needed.
+
+        The chunks should (but not definitely proven to) maintain the sequential order of the features of the original
+        dataframe. Suppose an initial 10 rows gives us chunks for rows [1, 2, 3], [4, 5, 6], [7, 8, 9], [10]. However,
+        the second chunk [4, 5, 6] turns out to be too large, so it gets divided into [4, 5] and [6]. The resulting
+        list of chunks should be [1, 2, 3], [4, 5], [6], [7, 8, 9], [10]
+
+        Args:
+            dataframe (pd.DataFrame.spatial): A spatially-enabled dataframe to divide
+            max_bytes (int): The max utf-16 encoded geojson size for any one chunk
+        """
+
+        #: Calculate number of chunks needed and the guesstimate max number of rows to achieve that size
+        geojson_size = sys.getsizeof(dataframe.spatial.to_featureset().to_geojson.encode('utf-16'))
+        chunks_needed = Chunking._ceildiv(geojson_size, max_bytes)
+        max_rows = Chunking._ceildiv(len(dataframe), chunks_needed)
+
+        #: Chunk the dataframe and then check if the resulting chunks are now within the proper size, calling again on
+        #: the offending chunks if not
+        list_of_dataframes = Chunking._chunk_dataframe(dataframe, max_rows)
+        return_dataframes = []  #: Holds result of valid and recursive chunks
+        for chunk_dataframe in list_of_dataframes:
+            chunk_geojson_size = sys.getsizeof(chunk_dataframe.spatial.to_featureset().to_geojson.encode('utf-16'))
+            if chunk_geojson_size > max_bytes:
+                return_dataframes.extend(Chunking._recursive_dataframe_chunking(chunk_dataframe, max_bytes))
+            else:
+                return_dataframes.append(chunk_dataframe)
+
+        return return_dataframes
+
+
 def fix_numeric_empty_strings(feature_set, feature_layer_fields):
     """Replace empty strings with None for numeric fields that allow nulls
-
     Args:
         feature_set (arcgis.features.FeatureSet): Feature set to clean
         fields (Dict): fields from feature layer
@@ -370,7 +920,7 @@ def fix_numeric_empty_strings(feature_set, feature_layer_fields):
     fields_to_fix = {
         field['name']
         for field in feature_layer_fields
-        if field['type'] in ['esriFieldTypeDouble', 'esriFieldTypeInteger'] and field['nullable']
+        if field['type'] in ['esriFieldTypeDouble', 'esriFieldTypeInteger', 'esriFieldTypeDate'] and field['nullable']
     }
     fields_to_fix -= {'Shape__Length', 'Shape__Area'}
 
diff --git a/src/palletjack/version.py b/src/palletjack/version.py
new file mode 100644
index 0000000..219a65c
--- /dev/null
+++ b/src/palletjack/version.py
@@ -0,0 +1,6 @@
+"""A single source of truth for the version in a programatically-accessible variable.
+This must only include a single line: __version__ = '0.0.0'
+"""
+
+#: This will be automatically set by the publish action when it is uploaded to pypi; only change this for local dev
+__version__ = '0.0.0'


Test patch:
diff --git a/src/palletjack/testing.py b/src/palletjack/testing.py
deleted file mode 100644
index 17ea71e..0000000
--- a/src/palletjack/testing.py
+++ /dev/null
@@ -1,7 +0,0 @@
-from palletjack import GoogleDriveDownloader
-
-out_dir = r'c:/temp/google_python_tests/'
-downloader = GoogleDriveDownloader(out_dir)
-outfile = downloader.download_file_from_google_drive(
-    'https://drive.google.com/file/d/1YtJhQMcPSd2udmxlKnxEyeeYtUkAkSxt/view?usp=sharing', 'test'
-)
diff --git a/tests/mock_arcpy.py b/tests/mock_arcpy.py
index 1460af1..d4456c0 100644
--- a/tests/mock_arcpy.py
+++ b/tests/mock_arcpy.py
@@ -1,8 +1,6 @@
 import sys
-import types
 from unittest.mock import Mock
 
 module_name = 'arcpy'
-arcpy = types.ModuleType(module_name)
+arcpy = Mock(name=module_name)
 sys.modules[module_name] = arcpy
-arcpy.da = Mock(name=module_name + '.da')
diff --git a/tests/test_loaders.py b/tests/test_extract.py
similarity index 72%
rename from tests/test_loaders.py
rename to tests/test_extract.py
index 6303cdf..c168685 100644
--- a/tests/test_loaders.py
+++ b/tests/test_extract.py
@@ -8,7 +8,7 @@
 import pytest
 from pandas import testing as tm
 
-import palletjack
+from palletjack import extract
 
 
 class TestGSheetsLoader:
@@ -22,7 +22,7 @@ def test_load_specific_worksheet_into_dataframe_by_id(self, mocker):
         gsheet_loader_mock = mocker.Mock()
         gsheet_loader_mock.gsheets_client = client_mock
 
-        palletjack.GSheetLoader.load_specific_worksheet_into_dataframe(gsheet_loader_mock, 'foobar', 5)
+        extract.GSheetLoader.load_specific_worksheet_into_dataframe(gsheet_loader_mock, 'foobar', 5)
 
         sheet_mock.worksheet.assert_called_once_with('index', 5)
         sheet_mock.worksheet_by_title.assert_not_called()
@@ -36,9 +36,7 @@ def test_load_specific_worksheet_into_dataframe_by_title(self, mocker):
         gsheet_loader_mock = mocker.Mock()
         gsheet_loader_mock.gsheets_client = client_mock
 
-        palletjack.GSheetLoader.load_specific_worksheet_into_dataframe(
-            gsheet_loader_mock, 'foobar', '2015', by_title=True
-        )
+        extract.GSheetLoader.load_specific_worksheet_into_dataframe(gsheet_loader_mock, 'foobar', '2015', by_title=True)
 
         sheet_mock.worksheet.assert_not_called
         sheet_mock.worksheet_by_title.assert_called_once_with('2015')
@@ -57,7 +55,7 @@ def test_load_all_worksheets_into_dataframes_single_worksheet(self, mocker):
         gsheet_loader_mock = mocker.Mock()
         gsheet_loader_mock.gsheets_client = client_mock
 
-        df_dict = palletjack.GSheetLoader.load_all_worksheets_into_dataframes(gsheet_loader_mock, 'foobar')
+        df_dict = extract.GSheetLoader.load_all_worksheets_into_dataframes(gsheet_loader_mock, 'foobar')
 
         test_dict = {'ws1': 'df1'}
 
@@ -81,7 +79,7 @@ def test_load_all_worksheets_into_dataframes_multiple_worksheets(self, mocker):
         gsheet_loader_mock = mocker.Mock()
         gsheet_loader_mock.gsheets_client = client_mock
 
-        df_dict = palletjack.GSheetLoader.load_all_worksheets_into_dataframes(gsheet_loader_mock, 'foobar')
+        df_dict = extract.GSheetLoader.load_all_worksheets_into_dataframes(gsheet_loader_mock, 'foobar')
 
         test_dict = {'ws1': 'df1', 'ws2': 'df2'}
 
@@ -100,7 +98,7 @@ def test_combine_worksheets_into_single_dataframe_combines_properly(self, mocker
 
         df_dict = {'df1': df1, 'df2': df2}
 
-        combined_df = palletjack.GSheetLoader.combine_worksheets_into_single_dataframe(mocker.Mock(), df_dict)
+        combined_df = extract.GSheetLoader.combine_worksheets_into_single_dataframe(mocker.Mock(), df_dict)
 
         test_df = pd.DataFrame({
             'worksheet': ['df1', 'df1', 'df2', 'df2'],
@@ -125,8 +123,8 @@ def test_combine_worksheets_into_single_dataframe_raises_error_on_mismatched_col
 
         df_dict = {'df1': df1, 'df2': df2}
 
-        with pytest.raises(ValueError, match='Columns do not match; cannot create mutli-index dataframe'):
-            combined_df = palletjack.GSheetLoader.combine_worksheets_into_single_dataframe(mocker.Mock(), df_dict)
+        with pytest.raises(ValueError, match='Columns do not match; cannot create multi-index dataframe'):
+            combined_df = extract.GSheetLoader.combine_worksheets_into_single_dataframe(mocker.Mock(), df_dict)
 
 
 class TestGoogleDriveDownloader:
@@ -138,7 +136,7 @@ def test_get_filename_from_response_works_normally(self, mocker):
             'Content-Disposition': 'attachment;filename="file.name";filename*=UTF-8\'\'file.name'
         }
 
-        filename = palletjack.GoogleDriveDownloader._get_filename_from_response(response)
+        filename = extract.GoogleDriveDownloader._get_filename_from_response(response)
 
         assert filename == 'file.name'
 
@@ -147,7 +145,7 @@ def test_get_filename_from_response_raises_error_if_not_found(self, mocker):
         response.headers = {'foo': 'bar', 'Content-Disposition': 'attachment'}
 
         with pytest.raises(ValueError, match='filename not found in response header'):
-            palletjack.GoogleDriveDownloader._get_filename_from_response(response)
+            extract.GoogleDriveDownloader._get_filename_from_response(response)
 
     def test_get_http_response_works_normally(self, mocker):
         response_mock = mocker.Mock()
@@ -156,7 +154,7 @@ def test_get_http_response_works_normally(self, mocker):
         get_mock.return_value = response_mock
         mocker.patch('requests.get', new=get_mock)
 
-        palletjack.GoogleDriveDownloader._get_http_response(mocker.Mock(), 'foo_file_id')
+        extract.GoogleDriveDownloader._get_http_response(mocker.Mock(), 'foo_file_id')
 
         get_mock.assert_called_with(
             'https://docs.google.com/uc?export=download', params={'id': 'foo_file_id'}, stream=True
@@ -170,7 +168,7 @@ def test_get_http_response_raises_error_on_text_response(self, mocker):
         mocker.patch('requests.get', new=get_mock)
 
         with pytest.raises(RuntimeError) as error:
-            palletjack.GoogleDriveDownloader._get_http_response(mocker.Mock(), 'foo_file_id')
+            extract.GoogleDriveDownloader._get_http_response(mocker.Mock(), 'foo_file_id')
         assert 'Cannot access foo_file_id (is it publicly shared?). Response header in log.' in str(error.value)
 
     def test_save_response_content_skips_empty_chunks(self, mocker):
@@ -181,20 +179,20 @@ def test_save_response_content_skips_empty_chunks(self, mocker):
         open_mock = mocker.mock_open()
         mocker.patch('pathlib.Path.open', open_mock)
 
-        palletjack.GoogleDriveDownloader._save_response_content(response_mock, Path('/foo/bar'), chunk_size=1)
+        extract.GoogleDriveDownloader._save_response_content(response_mock, Path('/foo/bar'), chunk_size=1)
 
         assert open_mock().write.call_args_list[0][0] == (b'\x01',)
         assert open_mock().write.call_args_list[1][0] == (b'\x02',)
 
     def test_download_file_from_google_drive_creates_filename(self, mocker):
 
-        mocker.patch.object(palletjack.GoogleDriveDownloader, '_get_file_id_from_sharing_link')
-        mocker.patch.object(palletjack.GoogleDriveDownloader, '_get_http_response', return_value='response')
-        mocker.patch.object(palletjack.GoogleDriveDownloader, '_get_filename_from_response', return_value='baz.png')
+        mocker.patch.object(extract.GoogleDriveDownloader, '_get_file_id_from_sharing_link')
+        mocker.patch.object(extract.GoogleDriveDownloader, '_get_http_response', return_value='response')
+        mocker.patch.object(extract.GoogleDriveDownloader, '_get_filename_from_response', return_value='baz.png')
         save_mock = mocker.Mock()
-        mocker.patch.object(palletjack.GoogleDriveDownloader, '_save_response_content', save_mock)
+        mocker.patch.object(extract.GoogleDriveDownloader, '_save_response_content', save_mock)
 
-        downloader = palletjack.GoogleDriveDownloader('/foo/bar')
+        downloader = extract.GoogleDriveDownloader('/foo/bar')
 
         downloader.download_file_from_google_drive('1234', 42)
 
@@ -203,11 +201,11 @@ def test_download_file_from_google_drive_creates_filename(self, mocker):
     def test_download_file_from_google_drive_handles_empty_string_link(self, mocker, caplog):
 
         file_id_mock = mocker.Mock()
-        mocker.patch.object(palletjack.GoogleDriveDownloader, '_get_file_id_from_sharing_link', file_id_mock)
+        mocker.patch.object(extract.GoogleDriveDownloader, '_get_file_id_from_sharing_link', file_id_mock)
 
-        downloader = palletjack.GoogleDriveDownloader('/foo/bar')
+        downloader = extract.GoogleDriveDownloader('/foo/bar')
 
-        caplog.set_level(logging.DEBUG, logger='palletjack.loaders.GoogleDriveDownloader')
+        caplog.set_level(logging.DEBUG, logger='palletjack.extract.GoogleDriveDownloader')
         caplog.clear()
         result = downloader.download_file_from_google_drive('', 42)
 
@@ -218,11 +216,11 @@ def test_download_file_from_google_drive_handles_empty_string_link(self, mocker,
     def test_download_file_from_google_drive_handles_None_link(self, mocker, caplog):
 
         file_id_mock = mocker.Mock()
-        mocker.patch.object(palletjack.GoogleDriveDownloader, '_get_file_id_from_sharing_link', file_id_mock)
+        mocker.patch.object(extract.GoogleDriveDownloader, '_get_file_id_from_sharing_link', file_id_mock)
 
-        downloader = palletjack.GoogleDriveDownloader('/foo/bar')
+        downloader = extract.GoogleDriveDownloader('/foo/bar')
 
-        caplog.set_level(logging.DEBUG, logger='palletjack.loaders.GoogleDriveDownloader')
+        caplog.set_level(logging.DEBUG, logger='palletjack.extract.GoogleDriveDownloader')
         caplog.clear()
         result = downloader.download_file_from_google_drive(None, 42)
 
@@ -232,18 +230,16 @@ def test_download_file_from_google_drive_handles_None_link(self, mocker, caplog)
 
     def test_download_file_from_google_drive_handles_download_error(self, mocker, caplog):
 
-        mocker.patch.object(
-            palletjack.GoogleDriveDownloader, '_get_file_id_from_sharing_link', return_value='google_id'
-        )
-        mocker.patch.object(palletjack.GoogleDriveDownloader, '_get_http_response', side_effect=Exception('Boom'))
+        mocker.patch.object(extract.GoogleDriveDownloader, '_get_file_id_from_sharing_link', return_value='google_id')
+        mocker.patch.object(extract.GoogleDriveDownloader, '_get_http_response', side_effect=Exception('Boom'))
         filename_mock = mocker.Mock()
-        mocker.patch.object(palletjack.GoogleDriveDownloader, '_get_filename_from_response', new=filename_mock)
+        mocker.patch.object(extract.GoogleDriveDownloader, '_get_filename_from_response', new=filename_mock)
         # save_mock = mocker.Mock()
-        # mocker.patch.object(palletjack.GoogleDriveDownloader, '_save_response_content', save_mock)
+        # mocker.patch.object(extract.GoogleDriveDownloader, '_save_response_content', save_mock)
 
-        downloader = palletjack.GoogleDriveDownloader('/foo/bar')
+        downloader = extract.GoogleDriveDownloader('/foo/bar')
 
-        caplog.set_level(logging.DEBUG, logger='palletjack.loaders.GoogleDriveDownloader')
+        caplog.set_level(logging.DEBUG, logger='palletjack.extract.GoogleDriveDownloader')
         caplog.clear()
         result = downloader.download_file_from_google_drive('1234', 42)
 
@@ -257,13 +253,13 @@ def test_download_file_from_google_drive_handles_download_error(self, mocker, ca
         assert result is None
 
     def test_download_file_from_google_drive_pauses_appropriately(self, mocker):
-        sleep_mock = mocker.patch.object(palletjack.loaders, 'sleep')
-        mocker.patch.object(palletjack.GoogleDriveDownloader, '_get_file_id_from_sharing_link')
-        mocker.patch.object(palletjack.GoogleDriveDownloader, '_get_http_response', return_value='response')
-        mocker.patch.object(palletjack.GoogleDriveDownloader, '_get_filename_from_response', return_value='baz.png')
-        mocker.patch.object(palletjack.GoogleDriveDownloader, '_save_response_content')
+        sleep_mock = mocker.patch.object(extract, 'sleep')
+        mocker.patch.object(extract.GoogleDriveDownloader, '_get_file_id_from_sharing_link')
+        mocker.patch.object(extract.GoogleDriveDownloader, '_get_http_response', return_value='response')
+        mocker.patch.object(extract.GoogleDriveDownloader, '_get_filename_from_response', return_value='baz.png')
+        mocker.patch.object(extract.GoogleDriveDownloader, '_save_response_content')
 
-        downloader = palletjack.GoogleDriveDownloader('/foo/bar')
+        downloader = extract.GoogleDriveDownloader('/foo/bar')
 
         downloader.download_file_from_google_drive('1234', 42, 1.5)
 
@@ -272,13 +268,13 @@ def test_download_file_from_google_drive_pauses_appropriately(self, mocker):
 
     def test_download_attachments_from_dataframe_sleeps_specified_time(self, mocker):
 
-        sleep_mock = mocker.patch.object(palletjack.loaders, 'sleep')
-        mocker.patch.object(palletjack.GoogleDriveDownloader, '_get_file_id_from_sharing_link')
-        mocker.patch.object(palletjack.GoogleDriveDownloader, '_get_http_response', return_value='response')
-        mocker.patch.object(palletjack.GoogleDriveDownloader, '_get_filename_from_response', return_value='baz.png')
-        mocker.patch.object(palletjack.GoogleDriveDownloader, '_save_response_content')
+        sleep_mock = mocker.patch.object(extract, 'sleep')
+        mocker.patch.object(extract.GoogleDriveDownloader, '_get_file_id_from_sharing_link')
+        mocker.patch.object(extract.GoogleDriveDownloader, '_get_http_response', return_value='response')
+        mocker.patch.object(extract.GoogleDriveDownloader, '_get_filename_from_response', return_value='baz.png')
+        mocker.patch.object(extract.GoogleDriveDownloader, '_save_response_content')
 
-        downloader = palletjack.GoogleDriveDownloader('/foo/bar')
+        downloader = extract.GoogleDriveDownloader('/foo/bar')
 
         sheet_dataframe = pd.DataFrame({
             'join_id': [1],
@@ -300,7 +296,7 @@ def test_download_attachments_from_dataframe_handles_multiple_rows(self, mocker)
             'link': ['a', 'b'],
         })
 
-        downloaded_df = palletjack.GoogleDriveDownloader.download_attachments_from_dataframe(
+        downloaded_df = extract.GoogleDriveDownloader.download_attachments_from_dataframe(
             downloader_mock, sheet_dataframe, 'link', 'join_id', 'path'
         )
 
@@ -314,7 +310,7 @@ def test_download_attachments_from_dataframe_handles_multiple_rows(self, mocker)
 
     def test_get_file_id_from_sharing_link_extracts_group_file_link(self, mocker):
 
-        downloader = palletjack.GoogleDriveDownloader('foo')
+        downloader = extract.GoogleDriveDownloader('foo')
         test_link = 'https://drive.google.com/file/d/abcdefghijklm_opqrstuvwxy/view?usp=sharing'
 
         file_id = downloader._get_file_id_from_sharing_link(test_link)
@@ -323,7 +319,7 @@ def test_get_file_id_from_sharing_link_extracts_group_file_link(self, mocker):
 
     def test_get_file_id_from_sharing_link_extracts_group_file_link_with_dashes(self, mocker):
 
-        downloader = palletjack.GoogleDriveDownloader('foo')
+        downloader = extract.GoogleDriveDownloader('foo')
         test_link = 'https://drive.google.com/file/d/abcdefghijklm-opqrstuvwxy/view?usp=sharing'
 
         file_id = downloader._get_file_id_from_sharing_link(test_link)
@@ -332,7 +328,7 @@ def test_get_file_id_from_sharing_link_extracts_group_file_link_with_dashes(self
 
     def test_get_file_id_from_sharing_link_extracts_group_open_link(self, mocker):
 
-        downloader = palletjack.GoogleDriveDownloader('foo')
+        downloader = extract.GoogleDriveDownloader('foo')
         test_link = 'https://drive.google.com/open?id=abcdefghijklm_opqrstuvwxy'
 
         file_id = downloader._get_file_id_from_sharing_link(test_link)
@@ -341,7 +337,7 @@ def test_get_file_id_from_sharing_link_extracts_group_open_link(self, mocker):
 
     def test_get_file_id_from_sharing_link_raises_error_on_failed_match(self, mocker):
 
-        downloader = palletjack.GoogleDriveDownloader('foo')
+        downloader = extract.GoogleDriveDownloader('foo')
         test_link = 'bad_link'
 
         with pytest.raises(RuntimeError, match='Regex could not match sharing link bad_link'):
@@ -355,7 +351,7 @@ def test_get_request_and_filename_from_drive_api_returns_both(self, mocker):
         client.drive.service.files.return_value.get_media.return_value = 'request'
         client.drive.service.files.return_value.get.return_value.execute.return_value = {'name': 'foo.bar'}
 
-        request, filename = palletjack.GoogleDriveDownloader._get_request_and_filename_from_drive_api(
+        request, filename = extract.GoogleDriveDownloader._get_request_and_filename_from_drive_api(
             mocker.Mock(), client, 'bar'
         )
 
@@ -370,7 +366,7 @@ def test_get_request_and_filename_from_drive_api_adds_extension(self, mocker):
             'mimeType': 'image/jpeg'
         }
 
-        request, filename = palletjack.GoogleDriveDownloader._get_request_and_filename_from_drive_api(
+        request, filename = extract.GoogleDriveDownloader._get_request_and_filename_from_drive_api(
             mocker.Mock(), client, 'bar'
         )
 
@@ -384,7 +380,7 @@ def test_get_request_and_filename_from_drive_api_warns_on_missing_mimeType(self,
             'name': 'foo',
         }
 
-        downloader = palletjack.GoogleDriveDownloader('/foo/bar')
+        downloader = extract.GoogleDriveDownloader('/foo/bar')
 
         caplog.set_level(logging.DEBUG, logger='palletjack.loaders.GoogleDriveDownloader')
         caplog.clear()
@@ -403,7 +399,7 @@ def test_get_request_and_filename_from_drive_api_warns_on_cant_find_extension(se
             'mimeType': 'bee/boo'
         }
 
-        downloader = palletjack.GoogleDriveDownloader('/foo/bar')
+        downloader = extract.GoogleDriveDownloader('/foo/bar')
 
         caplog.set_level(logging.DEBUG, logger='palletjack.loaders.GoogleDriveDownloader')
         caplog.clear()
@@ -416,25 +412,25 @@ def test_get_request_and_filename_from_drive_api_warns_on_cant_find_extension(se
         assert filename == 'foo'
 
     def test_save_get_media_content_works_multiple_chunks(self, mocker):
-        mocker.patch.object(palletjack.loaders, 'BytesIO')
-        downloader_mock = mocker.patch.object(palletjack.loaders, 'MediaIoBaseDownload')
+        mocker.patch.object(extract, 'BytesIO')
+        downloader_mock = mocker.patch.object(extract, 'MediaIoBaseDownload')
         downloader_mock.return_value.next_chunk.side_effect = [(1, False), (2, True)]
 
-        palletjack.GoogleDriveDownloader._save_get_media_content(mocker.Mock(), 'foo', mocker.Mock())
+        extract.GoogleDriveDownloader._save_get_media_content(mocker.Mock(), 'foo', mocker.Mock())
 
         assert downloader_mock.return_value.next_chunk.call_count == 2
 
     def test_download_file_from_google_drive_using_api_creates_filename(self, mocker):
 
-        mocker.patch.object(palletjack.GoogleDriveDownloader, '_get_file_id_from_sharing_link')
+        mocker.patch.object(extract.GoogleDriveDownloader, '_get_file_id_from_sharing_link')
         mocker.patch.object(
-            palletjack.GoogleDriveDownloader,
+            extract.GoogleDriveDownloader,
             '_get_request_and_filename_from_drive_api',
             return_value=('response', 'baz.png')
         )
-        save_mock = mocker.patch.object(palletjack.GoogleDriveDownloader, '_save_get_media_content')
+        save_mock = mocker.patch.object(extract.GoogleDriveDownloader, '_save_get_media_content')
 
-        downloader = palletjack.GoogleDriveDownloader('/foo/bar')
+        downloader = extract.GoogleDriveDownloader('/foo/bar')
 
         downloader.download_file_from_google_drive_using_api(mocker.Mock(), '1234', 42)
 
@@ -442,11 +438,11 @@ def test_download_file_from_google_drive_using_api_creates_filename(self, mocker
 
     def test_download_file_from_google_drive_using_api_handles_empty_string_link(self, mocker, caplog):
 
-        file_id_mock = mocker.patch.object(palletjack.GoogleDriveDownloader, '_get_file_id_from_sharing_link')
+        file_id_mock = mocker.patch.object(extract.GoogleDriveDownloader, '_get_file_id_from_sharing_link')
 
-        downloader = palletjack.GoogleDriveDownloader('/foo/bar')
+        downloader = extract.GoogleDriveDownloader('/foo/bar')
 
-        caplog.set_level(logging.DEBUG, logger='palletjack.loaders.GoogleDriveDownloader')
+        caplog.set_level(logging.DEBUG, logger='palletjack.extract.GoogleDriveDownloader')
         caplog.clear()
         result = downloader.download_file_from_google_drive_using_api(mocker.Mock(), '', 42)
 
@@ -456,11 +452,11 @@ def test_download_file_from_google_drive_using_api_handles_empty_string_link(sel
 
     def test_download_file_from_google_drive_using_api_handles_None_link(self, mocker, caplog):
 
-        file_id_mock = mocker.patch.object(palletjack.GoogleDriveDownloader, '_get_file_id_from_sharing_link')
+        file_id_mock = mocker.patch.object(extract.GoogleDriveDownloader, '_get_file_id_from_sharing_link')
 
-        downloader = palletjack.GoogleDriveDownloader('/foo/bar')
+        downloader = extract.GoogleDriveDownloader('/foo/bar')
 
-        caplog.set_level(logging.DEBUG, logger='palletjack.loaders.GoogleDriveDownloader')
+        caplog.set_level(logging.DEBUG, logger='palletjack.extract.GoogleDriveDownloader')
         caplog.clear()
         result = downloader.download_file_from_google_drive_using_api(mocker.Mock(), None, 42)
 
@@ -470,19 +466,17 @@ def test_download_file_from_google_drive_using_api_handles_None_link(self, mocke
 
     def test_download_file_from_google_drive_using_api_handles_download_error(self, mocker, caplog):
 
+        mocker.patch.object(extract.GoogleDriveDownloader, '_get_file_id_from_sharing_link', return_value='google_id')
+        mocker.patch.object(extract.utils, 'sleep')
         mocker.patch.object(
-            palletjack.GoogleDriveDownloader, '_get_file_id_from_sharing_link', return_value='google_id'
-        )
-        mocker.patch.object(palletjack.utils, 'sleep')
-        mocker.patch.object(
-            palletjack.GoogleDriveDownloader, '_get_request_and_filename_from_drive_api', side_effect=Exception('Boom')
+            extract.GoogleDriveDownloader, '_get_request_and_filename_from_drive_api', side_effect=Exception('Boom')
         )
 
-        saver_mock = mocker.patch.object(palletjack.GoogleDriveDownloader, '_save_get_media_content')
+        saver_mock = mocker.patch.object(extract.GoogleDriveDownloader, '_save_get_media_content')
 
-        downloader = palletjack.GoogleDriveDownloader('/foo/bar')
+        downloader = extract.GoogleDriveDownloader('/foo/bar')
 
-        caplog.set_level(logging.DEBUG, logger='palletjack.loaders.GoogleDriveDownloader')
+        caplog.set_level(logging.DEBUG, logger='palletjack.extract.GoogleDriveDownloader')
         caplog.clear()
         result = downloader.download_file_from_google_drive_using_api(mocker.Mock(), '1234', 42)
 
@@ -497,7 +491,7 @@ def test_download_file_from_google_drive_using_api_handles_download_error(self,
 
     def test_download_attachments_from_dataframe_using_api_handles_multiple_rows(self, mocker):
 
-        mocker.patch.object(palletjack.utils, 'authorize_pygsheets')
+        mocker.patch.object(extract.utils, 'authorize_pygsheets')
 
         downloader_mock = mocker.Mock()
         downloader_mock.download_file_from_google_drive_using_api.side_effect = ['foo/bar.png', 'baz/boo.png']
@@ -507,7 +501,7 @@ def test_download_attachments_from_dataframe_using_api_handles_multiple_rows(sel
             'link': ['a', 'b'],
         })
 
-        downloaded_df = palletjack.GoogleDriveDownloader.download_attachments_from_dataframe_using_api(
+        downloaded_df = extract.GoogleDriveDownloader.download_attachments_from_dataframe_using_api(
             downloader_mock, 'service_file', sheet_dataframe, 'link', 'join_id', 'path'
         )
 
@@ -541,7 +535,7 @@ def test_download_sftp_folder_contents_uses_right_credentials(self, mocker):
         cnopts_mock.side_effect = lambda knownhosts: knownhosts
         mocker.patch('pysftp.CnOpts', new=cnopts_mock)
 
-        palletjack.SFTPLoader.download_sftp_folder_contents(sftploader_mock)
+        extract.SFTPLoader.download_sftp_folder_contents(sftploader_mock)
 
         context_manager_mock.assert_called_with(
             'sftp_host', username='username', password='password', cnopts='knownhosts_file'
@@ -564,7 +558,7 @@ def test_download_sftp_single_file_uses_right_credentials(self, mocker):
         cnopts_mock.side_effect = lambda knownhosts: knownhosts
         mocker.patch('pysftp.CnOpts', new=cnopts_mock)
 
-        palletjack.SFTPLoader.download_sftp_single_file(sftploader_mock, 'filename', 'upload')
+        extract.SFTPLoader.download_sftp_single_file(sftploader_mock, 'filename', 'upload')
 
         context_manager_mock.assert_called_with(
             'sftp_host', username='username', password='password', cnopts='knownhosts_file', default_path='upload'
@@ -580,7 +574,7 @@ def test_read_csv_into_dataframe_with_column_names(self, mocker):
 
         column_types = {'bar': np.float64}
 
-        palletjack.SFTPLoader.read_csv_into_dataframe(sftploader_mock, 'baz', column_types)
+        extract.SFTPLoader.read_csv_into_dataframe(sftploader_mock, 'baz', column_types)
 
         pd_mock.assert_called_with(Path('foo', 'baz'), names=['bar'], dtype=column_types)
 
@@ -592,7 +586,7 @@ def test_read_csv_into_dataframe_no_column_names(self, mocker):
         sftploader_mock = mocker.Mock()
         sftploader_mock.download_dir = 'foo'
 
-        palletjack.SFTPLoader.read_csv_into_dataframe(sftploader_mock, 'baz')
+        extract.SFTPLoader.read_csv_into_dataframe(sftploader_mock, 'baz')
 
         pd_mock.assert_called_with(Path('foo', 'baz'), names=None, dtype=None)
 
@@ -605,7 +599,7 @@ def test_get_postgres_connection(self, mocker):
             gpd, 'read_postgis', return_value=gpd.read_file(gpd.datasets.get_path('naturalearth_lowres'))
         )
 
-        loader = palletjack.PostgresLoader('host', 'app', 'user', 'password')
+        loader = extract.PostgresLoader('host', 'app', 'user', 'password')
         dataframe = loader.read_table_into_dataframe('table', 'name', '4326', 'geometry')
 
         assert dataframe.spatial.geometry_type == ['polygon']
diff --git a/tests/test_load.py b/tests/test_load.py
new file mode 100644
index 0000000..96dea81
--- /dev/null
+++ b/tests/test_load.py
@@ -0,0 +1,1193 @@
+import datetime
+import json
+import logging
+import re
+import sys
+import urllib
+from pathlib import Path
+
+import numpy as np
+import pandas as pd
+import pandas.testing as tm
+import pytest
+from arcgis import GeoAccessor, GeoSeriesAccessor
+
+from palletjack import load
+
+
+class TestFeatureServiceUpdaterInit:
+
+    def test_init_calls_get_feature_layer(self, mocker):
+        new_dataframe = pd.DataFrame(columns=['Foo field', 'Bar'])
+        arcgis_mock = mocker.patch('palletjack.load.arcgis')
+
+        updater = load.FeatureServiceUpdater(mocker.Mock(), 'itemid', new_dataframe)
+
+        arcgis_mock.features.FeatureLayer.fromitem.assert_called_once()
+
+    #: WIP: removing column name for agol renamer (#36), more __init__ tests
+    def test_init_removes_shape_fields(self, mocker):
+        new_dataframe = pd.DataFrame(columns=['Foo field', 'Bar', 'Shape_Length', 'Shape_Area'])
+        mocker.patch('palletjack.load.arcgis')
+
+        updater = load.FeatureServiceUpdater(mocker.Mock(), 'itemid', new_dataframe, fields=new_dataframe.columns)
+
+        assert set(updater.fields) == {'Foo field', 'Bar'}
+
+    def test_init_resets_geometry_and_sets_failsafe_dir(self, mocker):
+        mocker.patch('palletjack.load.arcgis')
+        df_mock = mocker.Mock()
+        df_mock.columns = ['Foo field', 'Bar', 'SHAPE']
+
+        updater = load.FeatureServiceUpdater(mocker.Mock(), 'itemid', df_mock, failsafe_dir=r'c:\foo')
+
+        df_mock.spatial.set_geometry.assert_called_once_with('SHAPE')
+        assert updater.failsafe_dir == r'c:\foo'
+
+
+class TestUpdateLayer:
+
+    def test_update_hosted_feature_layer_calls_upsert_correctly(self, mocker):
+        new_dataframe = pd.DataFrame({
+            'foo': [1, 2],
+            'bar': [3, 4],
+            'OBJECTID': [11, 12],
+            'SHAPE': ['s1', 's2'],
+        })
+        updater_mock = mocker.Mock()
+        updater_mock.feature_service_itemid = 'foo123'
+        fl_mock = mocker.Mock()
+        updater_mock.feature_layer = fl_mock
+        updater_mock.new_dataframe = new_dataframe
+        updater_mock.fields = list(new_dataframe.columns)
+        updater_mock.join_column = 'OBJECTID'
+        updater_mock.layer_index = 0
+
+        updater_mock._upload_data.return_value = {'recordCount': 1}
+
+        field_checker_mock = mocker.patch('palletjack.utils.FieldChecker')
+
+        load.FeatureServiceUpdater._update_hosted_feature_layer(updater_mock, update_geometry=True)
+
+        updater_mock._upload_data.assert_called_once_with(
+            fl_mock,
+            new_dataframe,
+            upsert=True,
+            upsert_matching_field='OBJECTID',
+            append_fields=['foo', 'bar', 'OBJECTID', 'SHAPE'],
+            update_geometry=True
+        )
+
+    def test_update_hosted_feature_layer_calls_field_checkers(self, mocker):
+        new_dataframe = pd.DataFrame({
+            'foo': [1, 2],
+            'bar': [3, 4],
+            'OBJECTID': [11, 12],
+        })
+        updater_mock = mocker.Mock()
+        updater_mock.feature_service_itemid = 'foo123'
+        updater_mock.feature_layer.properties.fields = {'a': [1], 'b': [2]}
+        updater_mock.new_dataframe = new_dataframe
+        updater_mock.fields = list(new_dataframe.columns)
+        updater_mock.join_column = 'OBJECTID'
+        updater_mock.layer_index = 0
+
+        updater_mock._upload_data.return_value = {'recordCount': 1}
+
+        mocker.patch.multiple(
+            'palletjack.utils.FieldChecker',
+            check_live_and_new_field_types_match=mocker.DEFAULT,
+            check_for_non_null_fields=mocker.DEFAULT,
+            check_field_length=mocker.DEFAULT,
+            check_fields_present=mocker.DEFAULT,
+            check_srs_wgs84=mocker.DEFAULT,
+            check_nullable_ints_shapely=mocker.DEFAULT,
+        )
+
+        load.FeatureServiceUpdater._update_hosted_feature_layer(updater_mock, update_geometry=True)
+
+        load.utils.FieldChecker.check_live_and_new_field_types_match.assert_called_once_with(['foo', 'bar', 'OBJECTID'])
+        load.utils.FieldChecker.check_for_non_null_fields.assert_called_once_with(['foo', 'bar', 'OBJECTID'])
+        load.utils.FieldChecker.check_field_length.assert_called_once_with(['foo', 'bar', 'OBJECTID'])
+        load.utils.FieldChecker.check_fields_present.assert_called_once_with(['foo', 'bar', 'OBJECTID'], add_oid=True)
+        load.utils.FieldChecker.check_srs_wgs84.assert_called_once()
+        load.utils.FieldChecker.check_nullable_ints_shapely.assert_called_once()
+
+    def test_update_hosted_feature_layer_no_geometry_calls_null_geometry_generator(self, mocker):
+        new_dataframe = pd.DataFrame({
+            'foo': [1, 2],
+            'bar': [3, 4],
+            'OBJECTID': [11, 12],
+        })
+        updater_mock = mocker.Mock()
+        updater_mock.feature_service_itemid = 'foo123'
+        updater_mock.feature_layer = mocker.Mock()
+        updater_mock.feature_layer.properties = {'geometryType': 'esriGeometryPoint'}
+        updater_mock.new_dataframe = new_dataframe
+        # updater_mock.fields = ['foo', 'bar']
+        updater_mock.join_column = 'OBJECTID'
+        updater_mock.layer_index = 0
+
+        updater_mock._upload_data.return_value = {'recordCount': 1}
+
+        field_checker_mock = mocker.patch('palletjack.utils.FieldChecker')
+        null_generator_mock = mocker.patch('palletjack.utils.get_null_geometries', return_value='Nullo')
+
+        load.FeatureServiceUpdater._update_hosted_feature_layer(updater_mock, update_geometry=False)
+
+        null_generator_mock.assert_called_once_with({'geometryType': 'esriGeometryPoint'})
+
+    def test_update_hosted_feature_layer_no_geometry_calls_upsert_correctly(self, mocker):
+        new_dataframe = pd.DataFrame({
+            'foo': [1, 2],
+            'bar': [3, 4],
+            'OBJECTID': [11, 12],
+        })
+        updater_mock = mocker.Mock()
+        updater_mock.feature_service_itemid = 'foo123'
+        updater_mock.feature_layer = mocker.Mock()
+        updater_mock.feature_layer.properties = {'geometryType': 'esriGeometryPoint'}
+        updater_mock.new_dataframe = new_dataframe
+        updater_mock.fields = list(new_dataframe.columns)
+        updater_mock.join_column = 'OBJECTID'
+        updater_mock.layer_index = 0
+
+        updater_mock._upload_data.return_value = {'recordCount': 1}
+
+        field_checker_mock = mocker.patch('palletjack.utils.FieldChecker')
+        null_generator_mock = mocker.patch('palletjack.utils.get_null_geometries', return_value='Nullo')
+
+        load.FeatureServiceUpdater._update_hosted_feature_layer(updater_mock, update_geometry=False)
+
+        updater_mock._upload_data.assert_called_once_with(
+            updater_mock.feature_layer,
+            new_dataframe,
+            upsert=True,
+            upsert_matching_field='OBJECTID',
+            append_fields=['foo', 'bar', 'OBJECTID'],
+            update_geometry=False
+        )
+
+
+class TestAddToLayer:
+
+    def test_add_new_data_to_hosted_feature_layer_calls_upsert(self, mocker):
+        new_dataframe = pd.DataFrame({
+            'foo': [1, 2],
+            'bar': [3, 4],
+            # 'OBJECTID': [11, 12],
+        })
+        updater_mock = mocker.Mock()
+        updater_mock.feature_service_itemid = 'foo123'
+        updater_mock.feature_layer = mocker.Mock()
+        updater_mock.new_dataframe = new_dataframe
+        updater_mock.fields = ['foo', 'bar']
+        # updater_mock.join_column = 'OBJECTID'
+        updater_mock.layer_index = 0
+
+        updater_mock._upload_data.return_value = {'recordCount': 1}
+
+        field_checker_mock = mocker.patch('palletjack.utils.FieldChecker')
+
+        load.FeatureServiceUpdater._add_new_data_to_hosted_feature_layer(updater_mock)
+
+        updater_mock._upload_data.assert_called_once_with(
+            updater_mock.feature_layer,
+            new_dataframe,
+            upsert=False,
+        )
+
+    def test_add_new_data_to_hosted_feature_layer_calls_field_checkers(self, mocker):
+        new_dataframe = pd.DataFrame({
+            'foo': [1, 2],
+            'bar': [3, 4],
+            # 'OBJECTID': [11, 12],
+        })
+        updater_mock = mocker.Mock()
+        updater_mock.feature_service_itemid = 'foo123'
+        updater_mock.feature_layer.properties.fields = {'a': [1], 'b': [2]}
+        updater_mock.new_dataframe = new_dataframe
+        updater_mock.fields = list(new_dataframe.columns)
+        updater_mock.layer_index = 0
+
+        updater_mock._upload_data.return_value = {'recordCount': 1}
+
+        mocker.patch.multiple(
+            'palletjack.utils.FieldChecker',
+            check_live_and_new_field_types_match=mocker.DEFAULT,
+            check_for_non_null_fields=mocker.DEFAULT,
+            check_field_length=mocker.DEFAULT,
+            check_fields_present=mocker.DEFAULT,
+            check_srs_wgs84=mocker.DEFAULT,
+            check_nullable_ints_shapely=mocker.DEFAULT,
+        )
+        load.FeatureServiceUpdater._add_new_data_to_hosted_feature_layer(updater_mock)
+
+        load.utils.FieldChecker.check_live_and_new_field_types_match.assert_called_once_with(['foo', 'bar'])
+        load.utils.FieldChecker.check_for_non_null_fields.assert_called_once_with(['foo', 'bar'])
+        load.utils.FieldChecker.check_field_length.assert_called_once_with(['foo', 'bar'])
+        load.utils.FieldChecker.check_fields_present.assert_called_once_with(['foo', 'bar'], add_oid=False)
+        load.utils.FieldChecker.check_srs_wgs84.assert_called_once()
+        load.utils.FieldChecker.check_nullable_ints_shapely.assert_called_once()
+
+
+class TestDeleteFromLayer:
+
+    def test_delete_data_from_hosted_feature_layer_returns_number_of_deleted_features(self, mocker):
+        updater_mock = mocker.Mock()
+        updater_mock.feature_service_itemid = 'foo123'
+        updater_mock.layer_index = 0
+        updater_mock.feature_layer.delete_features.return_value = {
+            'deleteResults': [
+                {'objectId': 11, 'uniqueId': 11, 'globalId': None, 'success': True},
+                {'objectId': 17, 'uniqueId': 17, 'globalId': None, 'success': True},
+            ]
+        }  # yapf: disable
+
+        delete_utils_mock = mocker.patch('palletjack.utils.DeleteUtils')
+        delete_utils_mock.check_delete_oids_are_in_live_data.return_value = 0
+
+        deleted = load.FeatureServiceUpdater._delete_data_from_hosted_feature_layer(updater_mock, [11, 17])
+
+        assert deleted == 2
+
+    def test_delete_data_from_hosted_feature_layer_raises_on_failed_delete(self, mocker):
+        updater_mock = mocker.Mock()
+        updater_mock.feature_service_itemid = 'foo123'
+        updater_mock.layer_index = 0
+        updater_mock.feature_layer.delete_features.return_value = {
+            'deleteResults': [
+                {'objectId': 11, 'uniqueId': 11, 'globalId': None, 'success': True},
+                {'objectId': 17, 'uniqueId': 17, 'globalId': None, 'success': False},
+            ]
+        }  # yapf: disable
+
+        mocker.patch('palletjack.utils.DeleteUtils')
+
+        with pytest.raises(RuntimeError, match=re.escape('The following Object IDs failed to delete: [17]')):
+            deleted = load.FeatureServiceUpdater._delete_data_from_hosted_feature_layer(updater_mock, [11, 17])
+
+    def test_delete_data_from_hosted_feature_layer_runs_proper_check_sequence(self, mocker):
+        updater_mock = mocker.Mock()
+        updater_mock.feature_service_itemid = 'foo123'
+        updater_mock.layer_index = 0
+        updater_mock.feature_layer.query.return_value = {'objectIdFieldName': 'OBJECTID', 'objectIds': [11, 17]}
+        updater_mock.feature_layer.delete_features.return_value = {
+            'deleteResults': [
+                {'objectId': 11, 'uniqueId': 11, 'globalId': None, 'success': True},
+            ]
+        }  # yapf: disable
+
+        deleted = load.FeatureServiceUpdater._delete_data_from_hosted_feature_layer(updater_mock, [11])
+
+        assert deleted == 1
+
+
+class TestTruncateAndLoadLayer:
+
+    def test_truncate_existing_data_normal(self, mocker):
+        updater_mock = mocker.Mock()
+        updater_mock.feature_service_itemid = 'foo123'
+        updater_mock.layer_index = 0
+
+        updater_mock.feature_layer.manager.truncate.return_value = {
+            'submissionTime': 123,
+            'lastUpdatedTime': 124,
+            'status': 'Completed',
+        }
+
+        load.FeatureServiceUpdater._truncate_existing_data(updater_mock)
+
+    def test_truncate_existing_raises_error_on_failure(self, mocker):
+        updater_mock = mocker.Mock()
+        updater_mock.feature_service_itemid = 'foo123'
+        updater_mock.layer_index = 0
+        updater_mock.feature_layer.manager.truncate.return_value = {
+            'submissionTime': 123,
+            'lastUpdatedTime': 124,
+            'status': 'Foo',
+        }
+
+        with pytest.raises(RuntimeError, match='Failed to truncate existing data from layer id 0 in itemid foo123'):
+            load.FeatureServiceUpdater._truncate_existing_data(updater_mock)
+
+    def test_truncate_existing_retries_on_HTTPError(self, mocker):
+        updater_mock = mocker.Mock()
+        updater_mock.feature_service_itemid = 'foo123'
+        updater_mock.layer_index = 0
+        updater_mock.feature_layer.manager.truncate.side_effect = [
+            urllib.error.HTTPError('url', 'code', 'msg', 'hdrs', 'fp'), {
+                'submissionTime': 123,
+                'lastUpdatedTime': 124,
+                'status': 'Completed',
+            }
+        ]
+
+        load.FeatureServiceUpdater._truncate_existing_data(updater_mock)
+
+    def test_truncate_and_load_feature_service_normal(self, mocker):
+        updater_mock = mocker.Mock()
+        updater_mock.feature_service_itemid = 'foo123'
+        updater_mock.layer_index = 0
+        updater_mock.failsafe_dir = ''
+
+        updater_mock.new_dataframe = pd.DataFrame(columns=['Foo', 'Bar'])
+
+        mocker.patch('palletjack.utils.FieldChecker')
+
+        updater_mock._upload_data.return_value = 42
+
+        uploaded_features = load.FeatureServiceUpdater._truncate_and_load_data(updater_mock)
+
+        assert uploaded_features == 42
+
+    def test_truncate_and_load_append_fails_failsafe_dir_present(self, mocker, caplog):
+        caplog.set_level(logging.DEBUG)
+
+        updater_mock = mocker.Mock()
+        updater_mock._class_logger = logging.getLogger('mock logger')
+        updater_mock.feature_service_itemid = 'foo123'
+        updater_mock.layer_index = 0
+        updater_mock.new_dataframe = pd.DataFrame(columns=['Foo', 'Bar'])
+        updater_mock.failsafe_dir = 'foo'
+        updater_mock.feature_layer.feature_layer.properties.name = 'foolayer'
+
+        mocker.patch('palletjack.utils.FieldChecker')
+        mocker.patch('palletjack.utils.sleep')
+        mocker.patch('palletjack.utils.save_feature_layer_to_json', return_value='bar_path')
+
+        updater_mock._upload_data.side_effect = RuntimeError(
+            'Failed to append data. Append operation should have been rolled back.'
+        )
+
+        with pytest.raises(RuntimeError, match='Failed to append data. Append operation should have been rolled back.'):
+            uploaded_features = load.FeatureServiceUpdater._truncate_and_load_data(updater_mock)
+
+        assert updater_mock._upload_data.call_args_list[0].args == (
+            updater_mock.feature_layer, updater_mock.new_dataframe
+        )
+        assert updater_mock._upload_data.call_args_list[0].kwargs == {'upsert': False}
+
+        assert f'Append failed, feature service may be dirty due to append chunking. Data saved to bar_path' in caplog.text
+
+    def test_truncate_and_load_append_fails_failsafe_dir_absent(self, mocker, caplog):
+        caplog.set_level(logging.DEBUG)
+
+        updater_mock = mocker.Mock()
+        updater_mock._class_logger = logging.getLogger('mock logger')
+        updater_mock.feature_service_itemid = 'foo123'
+        updater_mock.layer_index = 0
+        updater_mock.new_dataframe = pd.DataFrame(columns=['Foo', 'Bar'])
+        updater_mock.failsafe_dir = ''
+
+        mocker.patch('palletjack.utils.FieldChecker')
+        mocker.patch('palletjack.utils.sleep')
+        mocker.patch('palletjack.utils.save_feature_layer_to_json', return_value='bar_path')
+
+        updater_mock._upload_data.side_effect = RuntimeError(
+            'Failed to append data. Append operation should have been rolled back.'
+        )
+
+        with pytest.raises(RuntimeError, match='Failed to append data. Append operation should have been rolled back.'):
+            uploaded_features = load.FeatureServiceUpdater._truncate_and_load_data(updater_mock)
+
+        assert updater_mock._upload_data.call_args_list[0].args == (
+            updater_mock.feature_layer, updater_mock.new_dataframe
+        )
+        assert updater_mock._upload_data.call_args_list[0].kwargs == {'upsert': False}
+
+        assert f'Append failed, feature service may be dirty due to append chunking. Old data not saved (no failsafe dir set)' in caplog.text
+
+    def test_truncate_and_load_calls_field_checkers(self, mocker, caplog):
+
+        caplog.set_level(logging.DEBUG)
+
+        updater_mock = mocker.Mock()
+        updater_mock._class_logger = logging.getLogger('mock logger')
+        updater_mock.feature_service_itemid = 'foo123'
+        updater_mock.feature_layer.properties.fields = {'a': [1], 'b': [2]}
+        updater_mock.layer_index = 0
+        updater_mock.fields = ['Foo', 'Bar']
+        updater_mock.failsafe_dir = ''
+
+        mocker.patch('palletjack.utils.sleep')
+        mocker.patch.multiple(
+            'palletjack.utils.FieldChecker',
+            check_live_and_new_field_types_match=mocker.DEFAULT,
+            check_for_non_null_fields=mocker.DEFAULT,
+            check_field_length=mocker.DEFAULT,
+            check_fields_present=mocker.DEFAULT,
+            check_srs_wgs84=mocker.DEFAULT,
+            check_nullable_ints_shapely=mocker.DEFAULT,
+        )
+
+        updater_mock._upload_data.return_value = 42
+
+        uploaded_features = load.FeatureServiceUpdater._truncate_and_load_data(updater_mock)
+
+        load.utils.FieldChecker.check_live_and_new_field_types_match.assert_called_once_with(['Foo', 'Bar'])
+        load.utils.FieldChecker.check_for_non_null_fields.assert_called_once_with(['Foo', 'Bar'])
+        load.utils.FieldChecker.check_field_length.assert_called_once_with(['Foo', 'Bar'])
+        load.utils.FieldChecker.check_fields_present.assert_called_once_with(['Foo', 'Bar'], add_oid=False)
+        load.utils.FieldChecker.check_srs_wgs84.assert_called_once()
+        load.utils.FieldChecker.check_nullable_ints_shapely.assert_called_once()
+
+        assert uploaded_features == 42
+
+    def test_truncate_and_load_doesnt_raise_on_empty_column(self, mocker, caplog):
+
+        caplog.set_level(logging.DEBUG)
+
+        new_dataframe = pd.DataFrame({
+            'id': [0, 1, 2],
+            'floats': [np.nan, np.nan, np.nan],
+            'x': [21, 22, 23],
+            'y': [31, 32, 33]
+        })
+        spatial_df = pd.DataFrame.spatial.from_xy(new_dataframe, 'x', 'y')
+
+        updater_mock = mocker.Mock()
+        updater_mock.new_dataframe = spatial_df
+        updater_mock._class_logger = logging.getLogger('mock logger')
+        updater_mock.feature_service_itemid = 'foo123'
+        updater_mock.layer_index = 0
+        updater_mock.fields = ['id', 'floats', 'x', 'y']
+        updater_mock.failsafe_dir = ''
+
+        updater_mock.feature_layer.properties.fields = [
+            {
+                'name': 'id',
+                'type': 'esriFieldTypeInteger',
+                'nullable': True,
+                'defaultValue': None,
+            },
+            {
+                'name': 'floats',
+                'type': 'esriFieldTypeDouble',
+                'nullable': True,
+                'defaultValue': None,
+            },
+            {
+                'name': 'x',
+                'type': 'esriFieldTypeInteger',
+                'nullable': True,
+                'defaultValue': None,
+            },
+            {
+                'name': 'y',
+                'type': 'esriFieldTypeInteger',
+                'nullable': True,
+                'defaultValue': None,
+            },
+        ]
+        updater_mock.feature_layer.properties.geometryType = 'esriGeometryPoint'
+        updater_mock.feature_layer.properties.extent.spatialReference.latestWkid = 4326
+
+        mocker.patch('palletjack.utils.sleep')
+
+        updater_mock._upload_data.return_value = {'recordCount': 42}
+
+        #: Should not raise
+        uploaded_features = load.FeatureServiceUpdater._truncate_and_load_data(updater_mock)
+
+
+class TestAttachments:
+
+    def test_create_attachment_action_df_adds_for_blank_existing_name(self, mocker):
+        input_df = pd.DataFrame({
+            'NAME': [np.nan],
+            'new_path': ['bee/foo.png'],
+        })
+
+        ops_df = load.FeatureServiceAttachmentsUpdater._create_attachment_action_df(mocker.Mock(), input_df, 'new_path')
+
+        test_df = pd.DataFrame({
+            'NAME': [np.nan],
+            'new_path': ['bee/foo.png'],
+            'new_filename': ['foo.png'],
+            'operation': ['add'],
+        })
+
+        tm.assert_frame_equal(ops_df, test_df)
+
+    def test_create_attachment_action_df_overwrites_for_different_existing_name(self, mocker):
+        input_df = pd.DataFrame({
+            'NAME': ['bar.png'],
+            'new_path': ['bee/foo.png'],
+        })
+
+        ops_df = load.FeatureServiceAttachmentsUpdater._create_attachment_action_df(mocker.Mock(), input_df, 'new_path')
+
+        test_df = pd.DataFrame({
+            'NAME': ['bar.png'],
+            'new_path': ['bee/foo.png'],
+            'new_filename': ['foo.png'],
+            'operation': ['overwrite'],
+        })
+
+        tm.assert_frame_equal(ops_df, test_df)
+
+    def test_create_attachment_action_df_does_nothing_for_same_name(self, mocker):
+        input_df = pd.DataFrame({
+            'NAME': ['foo.png'],
+            'new_path': ['bee/foo.png'],
+        })
+
+        ops_df = load.FeatureServiceAttachmentsUpdater._create_attachment_action_df(mocker.Mock(), input_df, 'new_path')
+
+        test_df = pd.DataFrame({
+            'NAME': ['foo.png'],
+            'new_path': ['bee/foo.png'],
+            'new_filename': ['foo.png'],
+            'operation': [np.nan],
+        })
+        test_df['operation'] = test_df['operation'].astype(object)
+
+        tm.assert_frame_equal(ops_df, test_df)
+
+    def test_create_attachment_action_df_does_all_three_ops(self, mocker):
+        input_df = pd.DataFrame({
+            'NAME': ['foo.png', 'bar.png', np.nan],
+            'new_path': ['bee/foo.png', 'bee/baz.png', 'bee/bin.png'],
+        })
+
+        ops_df = load.FeatureServiceAttachmentsUpdater._create_attachment_action_df(mocker.Mock(), input_df, 'new_path')
+
+        test_df = pd.DataFrame({
+            'NAME': ['foo.png', 'bar.png', np.nan],
+            'new_path': ['bee/foo.png', 'bee/baz.png', 'bee/bin.png'],
+            'new_filename': ['foo.png', 'baz.png', 'bin.png'],
+            'operation': [np.nan, 'overwrite', 'add'],
+        })
+
+        tm.assert_frame_equal(ops_df, test_df)
+
+    def test_create_attachment_action_df_do_nothing_after_others(self, mocker):
+        input_df = pd.DataFrame({
+            'NAME': ['bar.png', np.nan, 'foo.png'],
+            'new_path': ['bee/baz.png', 'bee/bin.png', 'bee/foo.png'],
+        })
+
+        ops_df = load.FeatureServiceAttachmentsUpdater._create_attachment_action_df(mocker.Mock(), input_df, 'new_path')
+
+        test_df = pd.DataFrame({
+            'NAME': ['bar.png', np.nan, 'foo.png'],
+            'new_path': ['bee/baz.png', 'bee/bin.png', 'bee/foo.png'],
+            'new_filename': ['baz.png', 'bin.png', 'foo.png'],
+            'operation': ['overwrite', 'add', np.nan],
+        })
+
+        tm.assert_frame_equal(ops_df, test_df)
+
+    def test_get_live_data_from_join_field_values_only_gets_matching_data(self, mocker):
+        live_features_df = pd.DataFrame({
+            'OBJECTID': [1, 2, 3],
+            'GlobalID': ['guid1', 'guid2', 'guid3'],
+            'attachment_key': [11, 12, 13],
+            'deleted': ['a', 'b', 'c'],
+        })
+
+        attachments_df = pd.DataFrame({
+            'attachment_key': [12, 13],
+            'attachments': ['foo', 'bar'],
+        })
+
+        live_data_subset = load.FeatureServiceAttachmentsUpdater._get_live_oid_and_guid_from_join_field_values(
+            mocker.Mock(), live_features_df, 'attachment_key', attachments_df
+        )
+
+        test_df = pd.DataFrame({
+            'OBJECTID': [2, 3],
+            'GlobalID': ['guid2', 'guid3'],
+            'attachment_key': [12, 13],
+            'attachments': ['foo', 'bar'],
+        })
+
+        tm.assert_frame_equal(live_data_subset, test_df)
+
+    def test_get_current_attachment_info_by_oid_includes_nans_for_features_wo_attachments(self, mocker):
+
+        live_attachments = [
+            {
+                'PARENTOBJECTID': 1,
+                'PARENTGLOBALID': 'parentguid1',
+                'ID': 111,
+                'NAME': 'foo.png',
+                'CONTENTTYPE': 'image/png',
+                'SIZE': 42,
+                'KEYWORDS': '',
+                'IMAGE_PREVIEW': 'preview1',
+                'GLOBALID': 'guid1',
+                'DOWNLOAD_URL': 'url1'
+            },
+            {
+                'PARENTOBJECTID': 2,
+                'PARENTGLOBALID': 'parentguid2',
+                'ID': 222,
+                'NAME': 'bar.png',
+                'CONTENTTYPE': 'image/png',
+                'SIZE': 42,
+                'KEYWORDS': '',
+                'IMAGE_PREVIEW': 'preview2',
+                'GLOBALID': 'guid2',
+                'DOWNLOAD_URL': 'url2'
+            },
+        ]
+
+        updater_mock = mocker.Mock()
+        updater_mock.feature_layer.attachments.search.return_value = live_attachments
+
+        live_data_subset_df = pd.DataFrame({
+            'OBJECTID': [1, 2, 3],
+            'GlobalID': ['guid1', 'guid2', 'guid3'],
+            'attachment_key': [11, 12, 13],
+            'attachments': ['fee', 'ber', 'boo'],
+        })
+
+        current_attachments_df = load.FeatureServiceAttachmentsUpdater._get_current_attachment_info_by_oid(
+            updater_mock, live_data_subset_df
+        )
+
+        test_df = pd.DataFrame({
+            'OBJECTID': [1, 2, 3],
+            'GlobalID': ['guid1', 'guid2', 'guid3'],
+            'attachment_key': [11, 12, 13],
+            'attachments': ['fee', 'ber', 'boo'],
+            'PARENTOBJECTID': [1., 2., np.nan],
+            'NAME': ['foo.png', 'bar.png', np.nan],
+            'ID': [111, 222, pd.NA],
+        })
+        test_df['ID'] = test_df['ID'].astype('Int64')
+
+        tm.assert_frame_equal(current_attachments_df, test_df)
+
+    def test_check_attachment_dataframe_for_invalid_column_names_doesnt_raise_with_valid_names(self, mocker):
+        dataframe = pd.DataFrame(columns=['foo', 'bar'])
+        invalid_names = ['baz', 'boo']
+        load.FeatureServiceAttachmentsUpdater._check_attachment_dataframe_for_invalid_column_names(
+            dataframe, invalid_names
+        )
+
+    def test_check_attachment_dataframe_for_invalid_column_names_raises_with_one_invalid(self, mocker):
+        dataframe = pd.DataFrame(columns=['foo', 'bar'])
+        invalid_names = ['foo', 'boo']
+        with pytest.raises(RuntimeError) as exc_info:
+            load.FeatureServiceAttachmentsUpdater._check_attachment_dataframe_for_invalid_column_names(
+                dataframe, invalid_names
+            )
+        assert exc_info.value.args[0] == 'Attachment dataframe contains the following invalid names: [\'foo\']'
+
+    def test_check_attachment_dataframe_for_invalid_column_names_raises_with_all_invalid(self, mocker):
+        dataframe = pd.DataFrame(columns=['foo', 'bar'])
+        invalid_names = ['foo', 'bar']
+        with pytest.raises(RuntimeError) as exc_info:
+            load.FeatureServiceAttachmentsUpdater._check_attachment_dataframe_for_invalid_column_names(
+                dataframe, invalid_names
+            )
+        assert exc_info.value.args[0] == 'Attachment dataframe contains the following invalid names: [\'foo\', \'bar\']'
+
+    def test_add_attachments_by_oid_adds_and_doesnt_warn(self, mocker):
+        action_df = pd.DataFrame({
+            'OBJECTID': [1, 2],
+            'operation': ['add', 'add'],
+            'path': ['path1', 'path2'],
+        })
+
+        result_dict = [
+            {
+                'addAttachmentResult': {
+                    'success': True
+                }
+            },
+            {
+                'addAttachmentResult': {
+                    'success': True
+                }
+            },
+        ]
+
+        updater_mock = mocker.Mock()
+        updater_mock.feature_layer.attachments.add.side_effect = result_dict
+
+        with pytest.warns(None) as warning:
+            count = load.FeatureServiceAttachmentsUpdater._add_attachments_by_oid(updater_mock, action_df, 'path')
+
+        assert count == 2
+        assert not warning
+
+    def test_add_attachments_by_oid_warns_on_failure_and_doesnt_count_that_one_and_continues(self, mocker):
+        action_df = pd.DataFrame({
+            'OBJECTID': [1, 2, 3],
+            'operation': ['add', 'add', 'add'],
+            'path': ['path1', 'path2', 'path3'],
+        })
+
+        result_dict = [
+            {
+                'addAttachmentResult': {
+                    'success': True
+                }
+            },
+            {
+                'addAttachmentResult': {
+                    'success': False
+                }
+            },
+            {
+                'addAttachmentResult': {
+                    'success': True
+                }
+            },
+        ]
+
+        feature_layer_mock = mocker.Mock()
+        feature_layer_mock.attachments.add.side_effect = result_dict
+
+        updater = load.FeatureServiceAttachmentsUpdater(mocker.Mock())
+        updater.feature_layer = feature_layer_mock
+
+        with pytest.warns(UserWarning, match='Failed to attach path2 to OID 2'):
+            count = updater._add_attachments_by_oid(action_df, 'path')
+
+        assert count == 2
+        assert updater.failed_dict == {2: ('add', 'path2')}
+
+    def test_add_attachments_by_oid_handles_internal_agol_errors(self, mocker, caplog):
+        action_df = pd.DataFrame({
+            'OBJECTID': [1, 2],
+            'operation': ['add', 'add'],
+            'path': ['path1', 'path2'],
+        })
+
+        feature_layer_mock = mocker.Mock()
+        feature_layer_mock.attachments.add.side_effect = [
+            RuntimeError('foo'),
+            {
+                'addAttachmentResult': {
+                    'success': True
+                }
+            },
+        ]
+
+        updater = load.FeatureServiceAttachmentsUpdater(mocker.Mock())
+        updater.feature_layer = feature_layer_mock
+
+        count = updater._add_attachments_by_oid(action_df, 'path')
+        assert count == 1
+        assert 'AGOL error while adding path1 to OID 1' in caplog.text
+        assert 'foo' in caplog.text
+        assert updater.failed_dict == {1: ('add', 'path1')}
+
+    def test_add_attachments_by_oid_skips_overwrite_and_nan(self, mocker):
+        action_df = pd.DataFrame({
+            'OBJECTID': [1, 2, 3],
+            'operation': ['add', 'overwrite', np.nan],
+            'path': ['path1', 'path2', 'path3'],
+        })
+
+        result_dict = [
+            {
+                'addAttachmentResult': {
+                    'success': True
+                }
+            },
+        ]
+
+        updater_mock = mocker.Mock()
+        updater_mock.feature_layer.attachments.add.side_effect = result_dict
+
+        count = load.FeatureServiceAttachmentsUpdater._add_attachments_by_oid(updater_mock, action_df, 'path')
+
+        assert updater_mock.feature_layer.attachments.add.call_count == 1
+        assert count == 1
+
+    def test_overwrite_attachments_by_oid_overwrites_and_doesnt_warn(self, mocker):
+        action_df = pd.DataFrame({
+            'OBJECTID': [1, 2],
+            'operation': ['overwrite', 'overwrite'],
+            'path': ['path1', 'path2'],
+            'ID': ['existing1', 'existing2'],
+            'NAME': ['oldname1', 'oldname2'],
+        })
+
+        result_dict = [
+            {
+                'updateAttachmentResult': {
+                    'success': True
+                }
+            },
+            {
+                'updateAttachmentResult': {
+                    'success': True
+                }
+            },
+        ]
+
+        updater_mock = mocker.Mock()
+        updater_mock.feature_layer.attachments.update.side_effect = result_dict
+
+        with pytest.warns(None) as warning:
+            count = load.FeatureServiceAttachmentsUpdater._overwrite_attachments_by_oid(updater_mock, action_df, 'path')
+
+        assert count == 2
+        assert not warning
+
+    def test_overwrite_attachments_by_oid_warns_on_failure_and_doesnt_count_that_one_and_continues(self, mocker):
+        action_df = pd.DataFrame({
+            'OBJECTID': [1, 2, 3],
+            'operation': ['overwrite', 'overwrite', 'overwrite'],
+            'path': ['path1', 'path2', 'path3'],
+            'ID': [11, 22, 33],
+            'NAME': ['oldname1', 'oldname2', 'oldname3'],
+        })
+
+        result_dict = [
+            {
+                'updateAttachmentResult': {
+                    'success': True
+                }
+            },
+            {
+                'updateAttachmentResult': {
+                    'success': False
+                }
+            },
+            {
+                'updateAttachmentResult': {
+                    'success': True
+                }
+            },
+        ]
+
+        feature_layer_mock = mocker.Mock()
+        feature_layer_mock.attachments.update.side_effect = result_dict
+
+        updater = load.FeatureServiceAttachmentsUpdater(mocker.Mock())
+        updater.feature_layer = feature_layer_mock
+
+        with pytest.warns(UserWarning, match='Failed to update oldname2, attachment ID 22, on OID 2 with path2'):
+            count = updater._overwrite_attachments_by_oid(action_df, 'path')
+
+        assert count == 2
+        assert updater.failed_dict == {2: ('update', 'path2')}
+
+    def test_overwrite_attachments_by_oid_handles_internal_agol_errors(self, mocker, caplog):
+        action_df = pd.DataFrame({
+            'OBJECTID': [1, 2],
+            'operation': ['overwrite', 'overwrite'],
+            'path': ['path1', 'path2'],
+            'ID': [11, 22],
+            'NAME': ['oldname1', 'oldname2'],
+        })
+
+        feature_layer_mock = mocker.Mock()
+        feature_layer_mock.attachments.update.side_effect = [
+            RuntimeError('foo'),
+            {
+                'updateAttachmentResult': {
+                    'success': True
+                }
+            },
+        ]
+
+        updater = load.FeatureServiceAttachmentsUpdater(mocker.Mock())
+        updater.feature_layer = feature_layer_mock
+
+        count = updater._overwrite_attachments_by_oid(action_df, 'path')
+        assert count == 1
+        assert 'AGOL error while overwriting oldname1 (attachment ID 11) on OID 1 with path1' in caplog.text
+        assert 'foo' in caplog.text
+        assert updater.failed_dict == {1: ('update', 'path1')}
+
+    def test_overwrite_attachments_by_oid_skips_add_and_nan(self, mocker):
+        action_df = pd.DataFrame({
+            'OBJECTID': [1, 2, 3],
+            'operation': ['add', 'overwrite', np.nan],
+            'path': ['path1', 'path2', 'path3'],
+            'ID': [np.nan, 'existing2', 'existing3'],
+            'NAME': ['oldname1', 'oldname2', 'oldname3'],
+        })
+
+        result_dict = [
+            {
+                'updateAttachmentResult': {
+                    'success': True
+                }
+            },
+        ]
+
+        updater_mock = mocker.Mock()
+        updater_mock.feature_layer.attachments.update.side_effect = result_dict
+
+        count = load.FeatureServiceAttachmentsUpdater._overwrite_attachments_by_oid(updater_mock, action_df, 'path')
+
+        assert updater_mock.feature_layer.attachments.update.call_count == 1
+        assert count == 1
+
+    def test_create_attachments_dataframe_subsets_and_crafts_paths_properly(self, mocker):
+        input_df = pd.DataFrame({
+            'join': [1, 2, 3],
+            'pic': ['foo.png', 'bar.png', 'baz.png'],
+            'data': [11., 12., 13.],
+        })
+
+        attachment_df = load.FeatureServiceAttachmentsUpdater.build_attachments_dataframe(
+            input_df, 'join', 'pic', '/foo/bar'
+        )
+
+        test_df = pd.DataFrame({
+            'join': [1, 2, 3],
+            'pic': ['foo.png', 'bar.png', 'baz.png'],
+            'full_file_path': [Path('/foo/bar/foo.png'),
+                               Path('/foo/bar/bar.png'),
+                               Path('/foo/bar/baz.png')]
+        })
+
+        #: Column of path objects won't test equal in assert_frame_equal, so we make lists of their str representations
+        #: and compare those separately from the rest of the dataframe
+        other_fields = ['join', 'pic']
+        tm.assert_frame_equal(attachment_df[other_fields], test_df[other_fields])
+        assert [str(path) for path in attachment_df['full_file_path']
+               ] == [str(path) for path in test_df['full_file_path']]
+
+    def test_create_attachments_dataframe_drops_missing_attachments(self, mocker):
+        input_df = pd.DataFrame({
+            'join': [1, 2, 3],
+            'pic': ['foo.png', None, ''],
+        })
+
+        attachment_df = load.FeatureServiceAttachmentsUpdater.build_attachments_dataframe(
+            input_df, 'join', 'pic', '/foo/bar'
+        )
+
+        test_df = pd.DataFrame({
+            'join': [1],
+            'pic': ['foo.png'],
+            'full_file_path': [Path('/foo/bar/foo.png')],
+        })
+
+        #: Column of path objects won't test equal in assert_frame_equal, so we make lists of their str representations
+        #: and compare those separately from the rest of the dataframe
+        other_fields = ['join', 'pic']
+        tm.assert_frame_equal(attachment_df[other_fields], test_df[other_fields])
+        assert [str(path) for path in attachment_df['full_file_path']
+               ] == [str(path) for path in test_df['full_file_path']]
+
+
+class TestUploadData:
+
+    def test_upload_data_calls_append_with_proper_args(self, mocker):
+        mock_fl = mocker.Mock()
+        mock_fl.append.return_value = (True, {'recordCount': 42})
+        mocker.patch('palletjack.utils.sleep')
+        mocker.patch('palletjack.utils.Chunking.build_upload_json', autospec=True, return_value=['json'])
+
+        load.FeatureServiceUpdater._upload_data(mocker.Mock(), mock_fl, mocker.Mock(), upsert=True)
+
+        mock_fl.append.assert_called_once_with(
+            upload_format='geojson', edits='json', upsert=True, rollback=True, return_messages=True
+        )
+
+    def test_upload_data_calls_append_for_multiple_chunks(self, mocker):
+        mock_fl = mocker.Mock()
+        mock_fl.append.return_value = (True, {'recordCount': 42})
+        mocker.patch('palletjack.utils.sleep')
+        mocker.patch('palletjack.utils.Chunking.build_upload_json', autospec=True, return_value=['json1', 'json2'])
+
+        load.FeatureServiceUpdater._upload_data(mocker.Mock(), mock_fl, mocker.Mock(), upsert=True)
+
+        assert mock_fl.append.call_count == 2
+        assert mock_fl.append.call_args_list == [
+            mocker.call(upload_format='geojson', edits='json1', upsert=True, rollback=True, return_messages=True),
+            mocker.call(upload_format='geojson', edits='json2', upsert=True, rollback=True, return_messages=True),
+        ]
+
+    def test_upload_data_retries_on_exception(self, mocker):
+        mock_fl = mocker.Mock()
+        mock_fl.append.side_effect = [Exception, (True, {'recordCount': 42})]
+        mocker.patch('palletjack.utils.Chunking.build_upload_json', autospec=True, return_value=['json'])
+        mocker.patch('palletjack.utils.sleep')
+
+        load.FeatureServiceUpdater._upload_data(mocker.Mock(), mock_fl, mocker.Mock(), upsert=True)
+
+        assert mock_fl.append.call_count == 2
+
+    def test_upload_data_raises_on_False_result(self, mocker):
+        mock_fl = mocker.Mock()
+        mock_fl.append.return_value = (False, {'message': 'foo'})
+        mocker.patch('palletjack.utils.Chunking.build_upload_json', autospec=True, return_value=['json'])
+        mocker.patch('palletjack.utils.sleep')
+
+        with pytest.raises(RuntimeError) as exc_info:
+            load.FeatureServiceUpdater._upload_data(mocker.Mock(), mock_fl, mocker.Mock(), upsert=True)
+
+        assert exc_info.value.args[
+            0] == 'Failed to append data at chunk 1 of 1. Append operation should have been rolled back.'
+
+    def test_upload_data_raises_on_upsert_field_not_in_append_fields(self, mocker):
+        append_kwargs = {'upsert_matching_field': 'foo', 'append_fields': ['bar', 'baz'], 'upsert': True}
+        with pytest.raises(ValueError) as exc_info:
+            load.FeatureServiceUpdater._upload_data(mocker.Mock(), mocker.Mock(), mocker.Mock(), **append_kwargs)
+
+        assert exc_info.value.args[0
+                                  ] == 'Upsert matching field foo not found in either append fields or existing fields.'
+
+    def test_upload_data_raises_on_upsert_field_not_in_dataframe_columns(self, mocker):
+        append_kwargs = {'upsert_matching_field': 'foo', 'append_fields': ['foo', 'bar'], 'upsert': True}
+        df = pd.DataFrame(columns=['bar', 'baz'])
+        with pytest.raises(ValueError) as exc_info:
+            load.FeatureServiceUpdater._upload_data(mocker.Mock(), mocker.Mock(), df, **append_kwargs)
+
+        assert exc_info.value.args[0
+                                  ] == 'Upsert matching field foo not found in either append fields or existing fields.'
+
+    def test_upload_data_raises_on_upsert_field_not_in_dataframe_columns_and_append_fields(self, mocker):
+        append_kwargs = {'upsert_matching_field': 'foo', 'append_fields': ['bar', 'baz'], 'upsert': True}
+        df = pd.DataFrame(columns=['bar', 'baz'])
+        with pytest.raises(ValueError) as exc_info:
+            load.FeatureServiceUpdater._upload_data(mocker.Mock(), mocker.Mock(), df, **append_kwargs)
+
+        assert exc_info.value.args[0
+                                  ] == 'Upsert matching field foo not found in either append fields or existing fields.'
+
+
+class TestColorRampReclassifier:
+
+    def test_get_layer_id_returns_match_single_layer(self, mocker):
+        layers = {
+            'operationalLayers': [
+                {
+                    'title': 'foo'
+                },
+            ],
+        }
+        get_data_mock = mocker.Mock(return_value=layers)
+        webmap_item_mock = mocker.Mock()
+        webmap_item_mock.get_data = get_data_mock
+        reclassifier = load.ColorRampReclassifier(webmap_item_mock, 'gis')
+
+        layer_id = reclassifier._get_layer_id('foo')
+        assert layer_id == 0
+
+    def test_get_layer_id_returns_match_many_layers(self, mocker):
+        layers = {
+            'operationalLayers': [
+                {
+                    'title': 'foo'
+                },
+                {
+                    'title': 'bar'
+                },
+                {
+                    'title': 'baz'
+                },
+            ],
+        }
+        get_data_mock = mocker.Mock(return_value=layers)
+        webmap_item_mock = mocker.Mock()
+        webmap_item_mock.get_data = get_data_mock
+        reclassifier = load.ColorRampReclassifier(webmap_item_mock, 'gis')
+
+        layer_id = reclassifier._get_layer_id('bar')
+        assert layer_id == 1
+
+    def test_get_layer_id_returns_first_match(self, mocker):
+        layers = {
+            'operationalLayers': [
+                {
+                    'title': 'foo'
+                },
+                {
+                    'title': 'bar'
+                },
+                {
+                    'title': 'bar'
+                },
+            ],
+        }
+        get_data_mock = mocker.Mock(return_value=layers)
+        webmap_item_mock = mocker.Mock()
+        webmap_item_mock.get_data = get_data_mock
+        reclassifier = load.ColorRampReclassifier(webmap_item_mock, 'gis')
+
+        layer_id = reclassifier._get_layer_id('bar')
+        assert layer_id == 1
+
+    def test_get_layer_id_raises_error_when_not_found(self, mocker):
+        layers = {
+            'operationalLayers': [
+                {
+                    'title': 'bar'
+                },
+            ],
+        }
+        get_data_mock = mocker.Mock(return_value=layers)
+        webmap_item_mock = mocker.Mock()
+        webmap_item_mock.title = 'test map'
+        webmap_item_mock.get_data = get_data_mock
+        reclassifier = load.ColorRampReclassifier(webmap_item_mock, 'gis')
+
+        with pytest.raises(ValueError) as error_info:
+            layer_id = reclassifier._get_layer_id('foo')
+
+        assert 'Could not find "foo" in test map' in str(error_info.value)
+
+    def test_calculate_new_stops_with_manual_numbers(self):
+        dataframe = pd.DataFrame({'numbers': [100, 300, 500, 700, 900]})
+
+        stops = load.ColorRampReclassifier._calculate_new_stops(dataframe, 'numbers', 5)
+
+        assert stops == [100, 279, 458, 637, 816]
+
+    def test_calculate_new_stops_mismatched_column_raises_error(self):
+        dataframe = pd.DataFrame({'numbers': [100, 300, 500, 700, 900]})
+
+        with pytest.raises(ValueError) as error_info:
+            stops = load.ColorRampReclassifier._calculate_new_stops(dataframe, 'foo', 5)
+            assert 'Column `foo` not in dataframe`' in str(error_info)
+
+    def test_update_stops_values(self, mocker):
+        # renderer = data['operationalLayers'][layer_number]['layerDefinition']['drawingInfo']['renderer']
+        # stops = renderer['visualVariables'][0]['stops']
+
+        data = {
+            'operationalLayers': [{
+                'layerDefinition': {
+                    'drawingInfo': {
+                        'renderer': {
+                            'visualVariables': [{
+                                'stops': [{
+                                    'value': 0
+                                }, {
+                                    'value': 1
+                                }, {
+                                    'value': 2
+                                }, {
+                                    'value': 3
+                                }]
+                            }]
+                        }
+                    }
+                }
+            }],
+        }
+        get_data_mock = mocker.Mock(return_value=data)
+        webmap_item_mock = mocker.Mock()
+        webmap_item_mock.get_data = get_data_mock
+        update_mock = mocker.Mock()
+        webmap_item_mock.update = update_mock
+        reclassifier = load.ColorRampReclassifier(webmap_item_mock, 'gis')
+
+        reclassifier._update_stop_values(0, [100, 200, 300, 400])
+
+        data['operationalLayers'][0]['layerDefinition']['drawingInfo']['renderer']['visualVariables'][0]['stops'] = [{
+            'value': 100
+        }, {
+            'value': 200
+        }, {
+            'value': 300
+        }, {
+            'value': 400
+        }]
+
+        assert update_mock.called_with(item_properties={'text': json.dumps(data)})
diff --git a/tests/test_transform.py b/tests/test_transform.py
index a639ebe..5bdb5aa 100644
--- a/tests/test_transform.py
+++ b/tests/test_transform.py
@@ -1,5 +1,7 @@
 import logging
+import re
 
+import numpy as np
 import pandas as pd
 import pytest
 from pandas import testing as tm
@@ -10,13 +12,15 @@
 class TestAPIGeocoder:
 
     def test_geocode_dataframe_calls_with_right_args(self, mocker):
-        mocker.patch.object(palletjack.transform, 'utils', autospec=True)
-        palletjack.transform.utils.rename_columns_for_agol.return_value = {}
-        palletjack.transform.utils.geocode_addr.side_effect = [
+
+        utils_mock = mocker.patch('palletjack.transform.utils', autospec=True)
+        utils_mock.Geocoding.validate_api_key.return_value = 'valid'
+        utils_mock.rename_columns_for_agol.return_value = {}
+        utils_mock.Geocoding.geocode_addr.side_effect = [
             (123, 456, 100., 'foo_addr'),
             (789, 101, 100., 'bar_addr'),
         ]
-        mocker.patch.object(pd.DataFrame.spatial, 'from_xy')
+        mocker.patch('palletjack.transform.pd.DataFrame.spatial')
 
         geocoder = palletjack.transform.APIGeocoder('foo')
 
@@ -27,19 +31,45 @@ def test_geocode_dataframe_calls_with_right_args(self, mocker):
 
         geocoder.geocode_dataframe(test_df, 'street', 'zip', 3857)
 
-        assert palletjack.transform.utils.geocode_addr.call_args.args == (
+        assert utils_mock.Geocoding.geocode_addr.call_args.args == (
+            '4501 S Constitution Blvd', '84129', 'foo', (0.015, 0.03)
+        )
+        assert utils_mock.Geocoding.geocode_addr.call_args.kwargs == {'spatialReference': '3857'}
+
+    def test_geocode_dataframe_handles_street_zone_fields_with_invalid_python_names(self, mocker):
+
+        utils_mock = mocker.patch('palletjack.transform.utils', autospec=True)
+        utils_mock.Geocoding.validate_api_key.return_value = 'valid'
+        utils_mock.rename_columns_for_agol.return_value = {}
+        utils_mock.Geocoding.geocode_addr.side_effect = [
+            (123, 456, 100., 'foo_addr'),
+            (789, 101, 100., 'bar_addr'),
+        ]
+        mocker.patch('palletjack.transform.pd.DataFrame.spatial')
+
+        geocoder = palletjack.transform.APIGeocoder('foo')
+
+        test_df = pd.DataFrame({
+            'Physical Address Street': ['4315 S 2700 W', '4501 S Constitution Blvd'],
+            'Physical Address Zip Code': ['84129', '84129'],
+        })
+
+        geocoder.geocode_dataframe(test_df, 'Physical Address Street', 'Physical Address Zip Code', 3857)
+
+        assert utils_mock.Geocoding.geocode_addr.call_args.args == (
             '4501 S Constitution Blvd', '84129', 'foo', (0.015, 0.03)
         )
-        assert palletjack.transform.utils.geocode_addr.call_args.kwargs == {'spatialReference': '3857'}
+        assert utils_mock.Geocoding.geocode_addr.call_args.kwargs == {'spatialReference': '3857'}
 
     def test_geocode_dataframe_passes_kwargs_through_to_util_method(self, mocker):
-        mocker.patch.object(palletjack.transform, 'utils', autospec=True)
-        palletjack.transform.utils.rename_columns_for_agol.return_value = {}
-        palletjack.transform.utils.geocode_addr.side_effect = [
+        utils_mock = mocker.patch('palletjack.transform.utils', autospec=True)
+        utils_mock.Geocoding.validate_api_key.return_value = 'valid'
+        utils_mock.rename_columns_for_agol.return_value = {}
+        utils_mock.Geocoding.geocode_addr.side_effect = [
             (123, 456, 100., 'foo_addr'),
             (789, 101, 100., 'bar_addr'),
         ]
-        mocker.patch.object(pd.DataFrame.spatial, 'from_xy')
+        mocker.patch('palletjack.transform.pd.DataFrame.spatial')
 
         geocoder = palletjack.transform.APIGeocoder('foo')
 
@@ -50,18 +80,16 @@ def test_geocode_dataframe_passes_kwargs_through_to_util_method(self, mocker):
 
         geocoder.geocode_dataframe(test_df, 'street', 'zip', 3857, acceptScore=80)
 
-        assert palletjack.transform.utils.geocode_addr.call_args.args == (
+        assert utils_mock.Geocoding.geocode_addr.call_args.args == (
             '4501 S Constitution Blvd', '84129', 'foo', (0.015, 0.03)
         )
-        assert palletjack.transform.utils.geocode_addr.call_args.kwargs == {
-            'spatialReference': '3857',
-            'acceptScore': 80
-        }
+        assert utils_mock.Geocoding.geocode_addr.call_args.kwargs == {'spatialReference': '3857', 'acceptScore': 80}
 
     def test_geocode_dataframe_builds_output_dataframe(self, mocker, caplog):
         caplog.set_level(logging.DEBUG)
-        mocker.patch('palletjack.utils.requests', autospec=True)
+        requests_mock = mocker.patch('palletjack.utils.requests', autospec=True)
         mocker.patch('palletjack.utils.sleep')
+        mocker.patch('palletjack.utils.Geocoding.validate_api_key', return_value='valid')
 
         first_response = mocker.Mock()
         first_response.json.return_value = {
@@ -91,16 +119,9 @@ def test_geocode_dataframe_builds_output_dataframe(self, mocker, caplog):
         }
         second_response.status_code = 200
 
-        palletjack.utils.requests.get.side_effect = [first_response, second_response]
-
-        #: Get patch from_xy to just return the dataframe it was passed so we don't have to create a spatial one for
-        #: testing
-        mocker.patch.object(pd.DataFrame.spatial, 'from_xy')
-
-        def _mock_from_xy(dataframe, x, y, sr=None):
-            return dataframe
+        requests_mock.get.side_effect = [first_response, second_response]
 
-        pd.DataFrame.spatial.from_xy.side_effect = _mock_from_xy
+        mocker.patch('palletjack.transform.pd.DataFrame.spatial.from_xy', side_effect=lambda df, x, y, sr: df)
 
         geocoder = palletjack.transform.APIGeocoder('foo')
 
@@ -124,8 +145,9 @@ def _mock_from_xy(dataframe, x, y, sr=None):
 
     def test_geocode_dataframe_continues_after_failed_row(self, mocker, caplog):
         caplog.set_level(logging.DEBUG)
-        mocker.patch('palletjack.utils.requests', autospec=True)
+        requests_mock = mocker.patch('palletjack.utils.requests', autospec=True)
         mocker.patch('palletjack.utils.sleep')
+        mocker.patch('palletjack.utils.Geocoding.validate_api_key', return_value='valid')
 
         bad_response = mocker.Mock()
         bad_response.status_code = 500
@@ -144,18 +166,9 @@ def test_geocode_dataframe_continues_after_failed_row(self, mocker, caplog):
         }
         good_response.status_code = 200
 
-        palletjack.utils.requests.get.side_effect = [
-            bad_response, bad_response, bad_response, bad_response, good_response
-        ]
-
-        #: Get patch from_xy to just return the dataframe it was passed so we don't have to create a spatial one for
-        #: testing
-        mocker.patch.object(pd.DataFrame.spatial, 'from_xy')
-
-        def _mock_from_xy(dataframe, x, y, sr=None):
-            return dataframe
+        requests_mock.get.side_effect = [bad_response, bad_response, bad_response, bad_response, good_response]
 
-        pd.DataFrame.spatial.from_xy.side_effect = _mock_from_xy
+        mocker.patch('palletjack.transform.pd.DataFrame.spatial.from_xy', side_effect=lambda df, x, y, sr: df)
 
         geocoder = palletjack.transform.APIGeocoder('foo')
 
@@ -181,12 +194,9 @@ def _mock_from_xy(dataframe, x, y, sr=None):
         assert palletjack.utils.sleep.call_count == 5
 
     def test_geocode_dataframe_warns_on_empty_input(self, mocker):
-        mocker.patch.object(palletjack.transform, 'utils', autospec=True)
-        # palletjack.transform.utils.geocode_addr.side_effect = [
-        #     (123, 456, 100., 'foo_addr'),
-        #     (789, 101, 100., 'bar_addr'),
-        # ]
-        mocker.patch.object(pd.DataFrame.spatial, 'from_xy')
+        utils_mock = mocker.patch('palletjack.transform.utils', autospec=True)
+        utils_mock.Geocoding.validate_api_key.return_value = 'valid'
+        mocker.patch('palletjack.transform.pd.DataFrame.spatial.from_xy')
 
         geocoder = palletjack.transform.APIGeocoder('foo')
 
@@ -197,15 +207,15 @@ def test_geocode_dataframe_warns_on_empty_input(self, mocker):
         assert record[0].message.args[0] == 'No records to geocode (empty dataframe)'
 
     def test_geocode_dataframe_warns_on_empty_output(self, mocker):
-        mocker.patch.object(palletjack.transform, 'utils', autospec=True)
-        palletjack.transform.utils.rename_columns_for_agol.return_value = {}
-        palletjack.transform.utils.geocode_addr.side_effect = [
+        utils_mock = mocker.patch('palletjack.transform.utils', autospec=True)
+        utils_mock.Geocoding.validate_api_key.return_value = 'valid'
+        utils_mock.rename_columns_for_agol.return_value = {}
+        utils_mock.Geocoding.geocode_addr.side_effect = [
             (123, 456, 100., 'foo_addr'),
             (789, 101, 100., 'bar_addr'),
         ]
-        mocker.patch.object(
-            pd.DataFrame.spatial,
-            'from_xy',
+        mocker.patch(
+            'palletjack.transform.pd.DataFrame.spatial.from_xy',
             return_value=pd.DataFrame(columns=['street', 'zip', 'x', 'y', 'score', 'matchAddress'])
         )
 
@@ -220,35 +230,281 @@ def test_geocode_dataframe_warns_on_empty_output(self, mocker):
             geocoder.geocode_dataframe(test_df, 'street', 'zip', 3857)
         assert record[0].message.args[0] == 'Empty spatial dataframe after geocoding'
 
-    def test_geocode_dataframe_handles_weird_column_names(self, mocker):
-        mocker.patch('palletjack.utils.requests', autospec=True)
-        mocker.patch('palletjack.utils.sleep')
+    def test_geocode_dataframe_raises_on_invalid_key(self, mocker):
+        req_mock = mocker.patch('palletjack.utils.requests', autospec=True)
+        response_mock = mocker.Mock()
+        response_mock.json.return_value = {'status': 400, 'message': 'Invalid API key'}
+        req_mock.get.return_value = response_mock
 
-        good_response = mocker.Mock()
-        good_response.json.return_value = {
-            'status': 200,
-            'result': {
-                'location': {
-                    'x': 123,
-                    'y': 456
-                },
-                'score': 100.,
-                'matchAddress': 'bar'
-            }
-        }
-        good_response.status_code = 200
-        palletjack.utils.requests.get.return_value = good_response
+        with pytest.raises(ValueError):
+            palletjack.transform.APIGeocoder('foo')
 
-        #: Get patch from_xy to just return the dataframe it was passed so we don't have to create a spatial one for
-        #: testing
-        mocker.patch.object(pd.DataFrame.spatial, 'from_xy')
+    # : TODO: test api validate responses in __init__
 
-        def _mock_from_xy(dataframe, x, y, sr=None):
-            return dataframe
 
-        pd.DataFrame.spatial.from_xy.side_effect = _mock_from_xy
-        geocoder = palletjack.transform.APIGeocoder('foo')
-        input_df = pd.DataFrame({
+class TestFeatureServiceMerging:
+
+    def test_update_live_data_with_new_data_works_normally(self):
+        new_dataframe = pd.DataFrame({
+            'col1': [10, 20, 30],
+            'col2': [40, 50, 60],
+            'key': ['a', 'b', 'c'],
+        })
+        live_dataframe = pd.DataFrame({
+            'col1': [1, 2, 3],
+            'col2': [4, 5, 6],
+            'key': ['a', 'b', 'c'],
+        })
+
+        joined = palletjack.transform.FeatureServiceMerging.update_live_data_with_new_data(
+            live_dataframe, new_dataframe, 'key'
+        )
+
+        expected = pd.DataFrame({
+            'key': ['a', 'b', 'c'],
+            'col1': [10, 20, 30],
+            'col2': [40, 50, 60],
+        }).convert_dtypes()
+
+        pd.testing.assert_frame_equal(joined, expected, check_like=True)
+
+    def test_update_live_data_with_new_data_only_updates_common_rows(self):
+
+        new_dataframe = pd.DataFrame({
+            'col1': [20, 30],
+            'col2': [50, 60],
+            'key': ['b', 'c'],
+        })
+        live_dataframe = pd.DataFrame({
+            'col1': [1, 2, 3],
+            'col2': [4, 5, 6],
+            'key': ['a', 'b', 'c'],
+        })
+
+        joined = palletjack.transform.FeatureServiceMerging.update_live_data_with_new_data(
+            live_dataframe, new_dataframe, 'key'
+        )
+
+        expected = pd.DataFrame({
+            'col1': [1, 20, 30],
+            'col2': [4, 50, 60],
+            'key': ['a', 'b', 'c'],
+        }).convert_dtypes()
+
+        pd.testing.assert_frame_equal(joined, expected, check_like=True)
+
+    def test_update_live_data_with_new_data_warns_on_missing_keys_and_handles_ints(self):
+        new_dataframe = pd.DataFrame({
+            'col1': [10, 20, 30, 80],
+            'col2': [40, 50, 60, 70],
+            'key': ['a', 'b', 'c', 'd'],
+        })
+        live_dataframe = pd.DataFrame({
+            'col1': [1, 2, 3],
+            'col2': [4, 5, 6],
+            'key': ['a', 'b', 'c'],
+        })
+
+        with pytest.warns(RuntimeWarning) as warning:
+            joined = palletjack.transform.FeatureServiceMerging.update_live_data_with_new_data(
+                live_dataframe, new_dataframe, 'key'
+            )
+
+        expected = pd.DataFrame({
+            'key': ['a', 'b', 'c'],
+            'col1': [10, 20, 30],
+            'col2': [40, 50, 60],
+        }).convert_dtypes()
+
+        pd.testing.assert_frame_equal(joined, expected, check_like=True)
+        assert 'The following keys from the new data were not found in the existing dataset: [\'d\']' in \
+            warning[0].message.args[0]
+
+    def test_get_live_dataframe_calls_each_method(self, mocker):
+        mock_arcgis = mocker.patch('palletjack.transform.arcgis')
+        mock_gis = mocker.Mock()
+
+        palletjack.transform.FeatureServiceMerging.get_live_dataframe(mock_gis, 'itemid', 42)
+
+        mock_gis.content.get.assert_called_once_with('itemid')
+        mock_arcgis.features.FeatureLayer.fromitem.assert_called_once_with(
+            mock_gis.content.get.return_value, layer_id=42
+        )
+        mock_arcgis.features.FeatureLayer.fromitem.return_value.query.assert_called_once_with(as_df=True)
+
+    def test_get_live_dataframe_raises_error_on_get_error(self, mocker):
+        mock_arcgis = mocker.patch('palletjack.transform.arcgis')
+        mock_gis = mocker.Mock()
+        mock_gis.content.get.side_effect = ValueError('get error')
+
+        with pytest.raises(RuntimeError) as error:
+            palletjack.transform.FeatureServiceMerging.get_live_dataframe(mock_gis, 'itemid')
+
+        assert 'Failed to load live dataframe' in str(error.value)
+        assert isinstance(error.value.__cause__, ValueError)
+        assert error.value.__cause__.args[0] == 'get error'
+
+    def test_get_live_dataframe_raises_error_on_fromitem_error(self, mocker):
+        mock_arcgis = mocker.patch('palletjack.transform.arcgis')
+        mock_gis = mocker.Mock()
+        mock_arcgis.features.FeatureLayer.fromitem.side_effect = ValueError('fromitem error')
+
+        with pytest.raises(RuntimeError) as error:
+            palletjack.transform.FeatureServiceMerging.get_live_dataframe(mock_gis, 'itemid')
+
+        assert 'Failed to load live dataframe' in str(error.value)
+        assert isinstance(error.value.__cause__, ValueError)
+        assert error.value.__cause__.args[0] == 'fromitem error'
+
+    def test_get_live_dataframe_raises_error_on_query_error(self, mocker):
+        mock_arcgis = mocker.patch('palletjack.transform.arcgis')
+        mock_gis = mocker.Mock()
+        mock_arcgis.features.FeatureLayer.fromitem.return_value.query.side_effect = ValueError('query error')
+
+        with pytest.raises(RuntimeError) as error:
+            palletjack.transform.FeatureServiceMerging.get_live_dataframe(mock_gis, 'itemid')
+
+        assert 'Failed to load live dataframe' in str(error.value)
+        assert isinstance(error.value.__cause__, ValueError)
+        assert error.value.__cause__.args[0] == 'query error'
+
+
+class TestNullableIntFixing:
+
+    def test_switch_to_nullable_int_casts_float_field_with_nan(self):
+        df = pd.DataFrame({
+            'a': [1, 2, np.nan],
+        })
+
+        retyped_df = palletjack.transform.DataCleaning.switch_to_nullable_int(df, ['a'])
+
+        test_df = pd.DataFrame([1, 2, pd.NA], columns=['a'], dtype='Int64')
+
+        tm.assert_frame_equal(retyped_df, test_df)
+
+    def test_switch_to_nullable_int_doesnt_change_other_float_field(self):
+        df = pd.DataFrame({
+            'a': [1, 2, np.nan],
+            'b': [1.1, 1.2, 1.3],
+        })
+
+        retyped_df = palletjack.transform.DataCleaning.switch_to_nullable_int(df, ['a'])
+
+        test_df = pd.DataFrame({
+            'a': pd.Series([1, 2, pd.NA], dtype='Int64'),
+            'b': pd.Series([1.1, 1.2, 1.3], dtype='float64')
+        })
+
+        tm.assert_frame_equal(retyped_df, test_df)
+
+    def test_switch_to_nullable_int_raises_on_true_float_field(self):
+        df = pd.DataFrame({
+            'a': [1, 2, np.nan],
+            'b': [1.1, 1.2, 1.3],
+        })
+
+        with pytest.raises(
+            TypeError,
+            match=re.escape('Cannot convert one or more fields to nullable ints. Check for non-int/non-np.nan values.')
+        ):
+            retyped_df = palletjack.transform.DataCleaning.switch_to_nullable_int(df, ['a', 'b'])
+
+
+class TestFloatFixing:
+
+    def test_switch_to_float_all_string_values(self):
+        df = pd.DataFrame({
+            'a': ['1', '2', '3'],
+        })
+
+        retyped_df = palletjack.transform.DataCleaning.switch_to_float(df, ['a'])
+
+        test_df = pd.DataFrame([1., 2., 3.], columns=['a'], dtype='float')
+
+        tm.assert_frame_equal(retyped_df, test_df)
+
+    def test_switch_to_float_string_values_with_nan(self):
+        df = pd.DataFrame({
+            'a': ['1', '2', np.nan],
+        })
+
+        retyped_df = palletjack.transform.DataCleaning.switch_to_float(df, ['a'])
+
+        test_df = pd.DataFrame([1., 2., np.nan], columns=['a'], dtype='float')
+
+        tm.assert_frame_equal(retyped_df, test_df)
+
+    def test_switch_to_float_string_values_with_empty_string(self):
+        df = pd.DataFrame({
+            'a': ['1', '2', ''],
+        })
+
+        retyped_df = palletjack.transform.DataCleaning.switch_to_float(df, ['a'])
+
+        test_df = pd.DataFrame([1., 2., np.nan], columns=['a'], dtype='float')
+
+        tm.assert_frame_equal(retyped_df, test_df)
+
+
+class TestDatetimeSwitching:
+
+    def test_switch_to_datetime_handles_multiple_fields(self):
+        df = pd.DataFrame({
+            'a': ['2020-04-29T01:09:29+00:00', '2020-04-29T01:09:29+00:00'],
+            'b': ['2015-09-02T23:08:12+00:00', '2015-09-02T23:08:12+00:00']
+        })
+
+        retyped_df = palletjack.transform.DataCleaning.switch_to_datetime(df, ['a', 'b'])
+
+        test_df = pd.DataFrame({
+            'a': [pd.Timestamp('2020-04-29 01:09:29')] * 2,
+            'b': [pd.Timestamp('2015-09-02 23:08:12')] * 2
+        })
+
+        tm.assert_frame_equal(retyped_df, test_df)
+        assert retyped_df['a'].dtype == np.dtype('datetime64[ns]')
+        assert retyped_df['b'].dtype == np.dtype('datetime64[ns]')
+
+    def test_switch_to_datetime_handles_both_empty_string_and_None(self):
+        df = pd.DataFrame({
+            'a': ['', '2020-04-29T01:09:29+00:00'],
+            'b': [None, '2015-09-02T23:08:12+00:00'],
+        })
+
+        retyped_df = palletjack.transform.DataCleaning.switch_to_datetime(df, ['a', 'b'])
+
+        test_df = pd.DataFrame({
+            'a': [pd.Timestamp(None), pd.Timestamp('2020-04-29 01:09:29')],
+            'b': [pd.Timestamp(None), pd.Timestamp('2015-09-02 23:08:12')]
+        })
+
+        tm.assert_frame_equal(retyped_df, test_df)
+        assert retyped_df['a'].dtype == np.dtype('datetime64[ns]')
+        assert retyped_df['b'].dtype == np.dtype('datetime64[ns]')
+
+    def test_switch_to_datetime_handles_empty_column(self):
+        df = pd.DataFrame({
+            'a': [None, None],
+            'b': ['2015-09-02T23:08:12+00:00', '2015-09-02T23:08:12+00:00'],
+        })
+
+        retyped_df = palletjack.transform.DataCleaning.switch_to_datetime(df, ['a', 'b'])
+
+        test_df = pd.DataFrame({
+            'a': [pd.Timestamp(None), pd.Timestamp(None)],
+            'b': [pd.Timestamp('2015-09-02 23:08:12'),
+                  pd.Timestamp('2015-09-02 23:08:12')]
+        })
+
+        tm.assert_frame_equal(retyped_df, test_df)
+        assert retyped_df['a'].dtype == np.dtype('datetime64[ns]')
+        assert retyped_df['b'].dtype == np.dtype('datetime64[ns]')
+
+
+class TestDataFrameColumnRenaming:
+
+    def test_rename_dataframe_columns_for_agol(self, mocker):
+        df = pd.DataFrame({
             'street': ['4315 S 2700 W'],
             'zip': ['84129'],
             'col_with_underscores': ['foo'],
@@ -260,12 +516,18 @@ def _mock_from_xy(dataframe, x, y, sr=None):
             'includes!mark': ['brie'],
         })
 
-        geocoded_df = geocoder.geocode_dataframe(input_df, 'street', 'zip', 26912)
+        renamed_df = palletjack.transform.DataCleaning.rename_dataframe_columns_for_agol(df)
 
-        out_cols = [
-            'street', 'zip', 'col_with_underscores', 'Col_With_Spaces', 'starts_with_underscore_',
-            'starts_with_number1', 'starts_with_number_and_underscore1_', 'includes_1_number', 'includes_mark', 'x',
-            'y', 'score', 'matchAddress'
-        ]
+        test_df = pd.DataFrame({
+            'street': ['4315 S 2700 W'],
+            'zip': ['84129'],
+            'col_with_underscores': ['foo'],
+            'Col_With_Spaces': ['bar'],
+            'starts_with_underscore_': ['baz'],
+            'starts_with_number1': ['eggs'],
+            'starts_with_number_and_underscore1_': ['fried'],
+            'includes_1_number': ['ham'],
+            'includes_mark': ['brie'],
+        })
 
-        assert list(geocoded_df.columns) == out_cols
+        tm.assert_frame_equal(renamed_df, test_df)
diff --git a/tests/test_updaters.py b/tests/test_updaters.py
deleted file mode 100644
index c7e4143..0000000
--- a/tests/test_updaters.py
+++ /dev/null
@@ -1,2102 +0,0 @@
-import builtins
-import json
-import logging
-import sys
-import urllib
-from pathlib import Path
-
-import arcgis
-import numpy as np
-import pandas as pd
-import pandas.testing as tm
-import pytest
-import requests
-from arcgis.features import GeoAccessor, GeoSeriesAccessor
-from mock_arcpy import arcpy
-from pandas.api.types import CategoricalDtype
-
-import palletjack
-
-
-@pytest.fixture
-def combined_values():
-    values = {
-        1: {
-            'old_values': {
-                'objectId': 1,
-                'data': 'foo',
-            },
-            'new_values': {
-                'key': 'a',
-                'data': 'FOO'
-            }
-        },
-        2: {
-            'old_values': {
-                'objectId': 2,
-                'data': 'bar',
-            },
-            'new_values': {
-                'key': 'b',
-                'data': 'BAR'
-            }
-        }
-    }
-    return values
-
-
-@pytest.fixture
-def hide_available_pkg(monkeypatch):
-    """Mocks import to throw an ImportError (for error handling testing purposes) for a package that does, in fact, exist"""
-    import_orig = builtins.__import__
-
-    def mocked_import(name, *args, **kwargs):
-        if name == 'arcpy':
-            raise ImportError()
-        return import_orig(name, *args, **kwargs)
-
-    monkeypatch.setattr(builtins, '__import__', mocked_import)
-
-
-class TestFeatureServiceInlineUpdater:
-
-    def test_update_existing_features_in_feature_service_with_arcpy(self, mocker):
-        #: We create a mock that will be returned by UpdateCursor's mock's __enter__, thus becoming our context manager.
-        #: We then set it's __iter__.return_value to define the data we want it to iterate over.
-        cursor_mock = mocker.MagicMock()
-        cursor_mock.__iter__.return_value = [
-            ['12345', '42', 123.45, '12/25/2021'],
-            ['67890', '18', 67.89, '12/25/2021'],
-        ]
-        context_manager_mock = mocker.MagicMock()
-        context_manager_mock.return_value.__enter__.return_value = cursor_mock
-        mocker.patch('arcpy.da.UpdateCursor', new=context_manager_mock)
-
-        fsupdater_mock = mocker.Mock()
-        fsupdater_mock.new_data_as_dict = {'12345': {'Count': '57', 'Amount': 100.00, 'Date': '1/1/2022'}}
-
-        palletjack.FeatureServiceInlineUpdater.update_existing_features_in_feature_service_with_arcpy(
-            fsupdater_mock, 'foo', ['ZipCode', 'Count', 'Amount', 'Date']
-        )
-
-        cursor_mock.updateRow.assert_called_with(['12345', '57', 100.00, '1/1/2022'])
-        cursor_mock.updateRow.assert_called_once()
-
-    @pytest.mark.usefixtures('hide_available_pkg')
-    def test_update_existing_features_in_feature_service_with_arcpy_reports_error_on_import_failure(self, mocker):
-
-        with pytest.raises(ImportError, match='Failure importing arcpy. ArcGIS Pro must be installed.'):
-            palletjack.FeatureServiceInlineUpdater.update_existing_features_in_feature_service_with_arcpy(
-                mocker.Mock(), 'foo', ['ZipCode', 'Count', 'Amount', 'Date']
-            )
-
-    def test_clean_dataframe_columns_renames_and_drops_columns(self, mocker):
-        class_mock = mocker.Mock()
-        fields = ['foo', 'bar']
-        dataframe = pd.DataFrame(columns=['foo_x', 'bar_x', 'baz', 'foo_y', 'bar_y'])
-
-        renamed = palletjack.FeatureServiceInlineUpdater._clean_dataframe_columns(class_mock, dataframe, fields)
-
-        assert list(renamed.columns) == ['baz', 'foo', 'bar']
-
-    def test_clean_dataframe_columns_handles_field_not_found_in_dataframe(self, mocker):
-        class_mock = mocker.Mock()
-        fields = ['foo', 'bar', 'buz']
-        dataframe = pd.DataFrame(columns=['foo_x', 'bar_x', 'baz', 'foo_y', 'bar_y'])
-
-        renamed = palletjack.FeatureServiceInlineUpdater._clean_dataframe_columns(class_mock, dataframe, fields)
-
-        assert list(renamed.columns) == ['baz', 'foo', 'bar']
-
-    def test_clean_dataframe_columns_deletes_merge_field(self, mocker):
-        class_mock = mocker.Mock()
-        fields = ['foo', 'bar', 'buz']
-        dataframe = pd.DataFrame(columns=['foo_x', 'bar_x', 'baz', 'foo_y', 'bar_y', '_merge'])
-
-        renamed = palletjack.FeatureServiceInlineUpdater._clean_dataframe_columns(class_mock, dataframe, fields)
-
-        assert list(renamed.columns) == ['baz', 'foo', 'bar']
-
-    def test_get_common_rows_joins_properly_all_rows_in_both(self, mocker):
-        class_mock = mocker.Mock()
-        class_mock.index_column = 'key'
-        class_mock.new_dataframe = pd.DataFrame({'col1': [10, 20, 30], 'col2': [40, 50, 60], 'key': ['a', 'b', 'c']})
-        live_dataframe = pd.DataFrame({'col1': [1, 2, 3], 'col2': [4, 5, 6], 'key': ['a', 'b', 'c']})
-
-        joined = palletjack.FeatureServiceInlineUpdater._get_common_rows(class_mock, live_dataframe)
-
-        merge_type = CategoricalDtype(categories=['left_only', 'right_only', 'both'], ordered=False)
-        expected = pd.DataFrame({
-            'col1_x': [1, 2, 3],
-            'col2_x': [4, 5, 6],
-            'key': ['a', 'b', 'c'],
-            'col1_y': [10, 20, 30],
-            'col2_y': [40, 50, 60],
-            '_merge': ['both', 'both', 'both']
-        })
-        expected['_merge'] = expected['_merge'].astype(merge_type)
-
-        pd.testing.assert_frame_equal(joined, expected)
-
-    def test_get_common_rows_subsets_properly_new_has_fewer_rows(self, mocker):
-
-        class_mock = mocker.Mock()
-        class_mock.index_column = 'key'
-        class_mock.new_dataframe = pd.DataFrame({'col1': [20, 30], 'col2': [50, 60], 'key': ['b', 'c']})
-        live_dataframe = pd.DataFrame({'col1': [1, 2, 3], 'col2': [4, 5, 6], 'key': ['a', 'b', 'c']})
-
-        joined = palletjack.FeatureServiceInlineUpdater._get_common_rows(class_mock, live_dataframe)
-
-        merge_type = CategoricalDtype(categories=['left_only', 'right_only', 'both'], ordered=False)
-        expected = pd.DataFrame({
-            'col1_x': [2, 3],
-            'col2_x': [5, 6],
-            'key': ['b', 'c'],
-            'col1_y': [20, 30],
-            'col2_y': [50, 60],
-            '_merge': ['both', 'both']
-        },
-                                index=[1, 2])
-        expected['_merge'] = expected['_merge'].astype(merge_type)
-
-        pd.testing.assert_frame_equal(joined, expected, check_dtype=False)
-
-    def test_get_common_rows_logs_warning_for_rows_not_in_existing_dataset(self, mocker, caplog):
-
-        class_mock = mocker.Mock()
-        class_mock.index_column = 'key'
-        class_mock.new_dataframe = pd.DataFrame({
-            'col1': [10, 20, 30, 80],
-            'col2': [40, 50, 60, 70],
-            'key': ['a', 'b', 'c', 'd']
-        })
-        class_mock._class_logger = logging.getLogger('root')
-        live_dataframe = pd.DataFrame({'col1': [1, 2, 3], 'col2': [4, 5, 6], 'key': ['a', 'b', 'c']})
-
-        joined = palletjack.FeatureServiceInlineUpdater._get_common_rows(class_mock, live_dataframe)
-
-        merge_type = CategoricalDtype(categories=['left_only', 'right_only', 'both'], ordered=False)
-        expected = pd.DataFrame({
-            'col1_x': [1, 2, 3],
-            'col2_x': [4, 5, 6],
-            'key': ['a', 'b', 'c'],
-            'col1_y': [10, 20, 30],
-            'col2_y': [40, 50, 60],
-            '_merge': ['both', 'both', 'both']
-        },
-                                index=[0, 1, 2])
-        expected['_merge'] = expected['_merge'].astype(merge_type)
-
-        pd.testing.assert_frame_equal(joined, expected, check_dtype=False)
-        assert 'The following keys from the new data were not found in the existing dataset: [\'d\']' in caplog.text
-
-
-class TestFeatureServiceInlineUpdaterResultParsing:
-
-    def test_parse_results_returns_correct_number_of_updated_rows(self, mocker, combined_values):
-        class_mock = mocker.Mock()
-        class_mock._get_old_and_new_values.return_value = (combined_values)
-        results_dict = {
-            'addResults': [],
-            'updateResults': [
-                {
-                    'objectId': 1,
-                    'success': True
-                },
-                {
-                    'objectId': 2,
-                    'success': True
-                },
-            ],
-            'deleteResults': [],
-        }
-        live_dataframe = pd.DataFrame({
-            'objectId': [1, 2],
-            'data': ['foo', 'bar'],
-        })
-
-        rows_updated = palletjack.FeatureServiceInlineUpdater._parse_results(class_mock, results_dict, live_dataframe)
-
-        assert rows_updated == 2
-
-    def test_parse_results_returns_0_if_both_successful_and_failed_result(self, mocker, combined_values):
-        class_mock = mocker.Mock()
-        class_mock._get_old_and_new_values.return_value = (combined_values)
-        results_dict = {
-            'addResults': [],
-            'updateResults': [
-                {
-                    'objectId': 1,
-                    'success': True
-                },
-                {
-                    'objectId': 2,
-                    'success': False
-                },
-            ],
-            'deleteResults': [],
-        }
-        live_dataframe = pd.DataFrame({
-            'objectId': [1, 2],
-            'data': ['foo', 'bar'],
-        })
-
-        rows_updated = palletjack.FeatureServiceInlineUpdater._parse_results(class_mock, results_dict, live_dataframe)
-
-        assert rows_updated == 0
-
-    def test_parse_results_logs_success_info(self, mocker, caplog, combined_values):
-        class_mock = mocker.Mock()
-        class_mock._class_logger = logging.getLogger('root')
-        # combined_values = {
-        #     1: {
-        #         'old_values': {
-        #             'objectId': 1,
-        #             'data': 'foo',
-        #         },
-        #         'new_values': {
-        #             'key': 'a',
-        #             'data': 'FOO'
-        #         }
-        #     },
-        #     2: {
-        #         'old_values': {
-        #             'objectId': 2,
-        #             'data': 'bar',
-        #         },
-        #         'new_values': {
-        #             'key': 'b',
-        #             'data': 'BAR'
-        #         }
-        #     }
-        # }
-        class_mock._get_old_and_new_values.return_value = combined_values
-        results_dict = {
-            'addResults': [],
-            'updateResults': [
-                {
-                    'objectId': 1,
-                    'success': True
-                },
-                {
-                    'objectId': 2,
-                    'success': True
-                },
-            ],
-            'deleteResults': [],
-        }
-        live_dataframe = pd.DataFrame({
-            'objectId': [1, 2],
-            'data': ['foo', 'bar'],
-        })
-
-        with caplog.at_level(logging.INFO):
-            rows_updated = palletjack.FeatureServiceInlineUpdater._parse_results(
-                class_mock, results_dict, live_dataframe
-            )
-
-            assert '2 rows successfully updated' in caplog.text
-            assert "Existing data: {'objectId': 1, 'data': 'foo'}" not in caplog.text
-            assert "New data: {'key': 'a', 'data': 'FOO'}" not in caplog.text
-            assert "Existing data: {'objectId': 2, 'data': 'bar'}" not in caplog.text
-            assert "New data: {'key': 'b', 'data': 'BAR'}" not in caplog.text
-
-    def test_parse_results_logs_success_debug(self, mocker, caplog, combined_values):
-        class_mock = mocker.Mock()
-        class_mock._class_logger = logging.getLogger('root')
-        # combined_values = {
-        #     1: {
-        #         'old_values': {
-        #             'objectId': 1,
-        #             'data': 'foo',
-        #         },
-        #         'new_values': {
-        #             'key': 'a',
-        #             'data': 'FOO'
-        #         }
-        #     },
-        #     2: {
-        #         'old_values': {
-        #             'objectId': 2,
-        #             'data': 'bar',
-        #         },
-        #         'new_values': {
-        #             'key': 'b',
-        #             'data': 'BAR'
-        #         }
-        #     }
-        # }
-        class_mock._get_old_and_new_values.return_value = combined_values
-        results_dict = {
-            'addResults': [],
-            'updateResults': [
-                {
-                    'objectId': 1,
-                    'success': True
-                },
-                {
-                    'objectId': 2,
-                    'success': True
-                },
-            ],
-            'deleteResults': [],
-        }
-        live_dataframe = pd.DataFrame({
-            'objectId': [1, 2],
-            'data': ['foo', 'bar'],
-        })
-
-        with caplog.at_level(logging.DEBUG):
-            rows_updated = palletjack.FeatureServiceInlineUpdater._parse_results(
-                class_mock, results_dict, live_dataframe
-            )
-
-            assert '2 rows successfully updated' in caplog.text
-            assert "Existing data: {'objectId': 1, 'data': 'foo'}" in caplog.text
-            assert "New data: {'key': 'a', 'data': 'FOO'}" in caplog.text
-            assert "Existing data: {'objectId': 2, 'data': 'bar'}" in caplog.text
-            assert "New data: {'key': 'b', 'data': 'BAR'}" in caplog.text
-
-    def test_parse_results_logs_failures_at_warning(self, mocker, caplog):
-        class_mock = mocker.Mock()
-        class_mock._class_logger = logging.getLogger('root')
-        combined_values = [{
-            1: {
-                'old_values': {
-                    'objectId': 1,
-                    'data': 'foo',
-                },
-                'new_values': {
-                    'key': 'a',
-                    'data': 'FOO'
-                }
-            },
-        }, {
-            2: {
-                'old_values': {
-                    'objectId': 2,
-                    'data': 'bar',
-                },
-                'new_values': {
-                    'key': 'b',
-                    'data': 'BAR'
-                }
-            }
-        }]
-        class_mock._get_old_and_new_values.side_effect = combined_values
-        results_dict = {
-            'addResults': [],
-            'updateResults': [
-                {
-                    'objectId': 1,
-                    'success': True
-                },
-                {
-                    'objectId': 2,
-                    'success': False
-                },
-            ],
-            'deleteResults': [],
-        }
-        live_dataframe = pd.DataFrame({
-            'objectId': [1, 2],
-            'data': ['foo', 'bar'],
-        })
-
-        with caplog.at_level(logging.WARNING):
-            rows_updated = palletjack.FeatureServiceInlineUpdater._parse_results(
-                class_mock, results_dict, live_dataframe
-            )
-
-            assert caplog.records[
-                0
-            ].message == 'The following 1 updates failed. As a result, all successful updates should have been rolled back.'
-            assert caplog.records[0].levelname == 'WARNING'
-
-            assert caplog.records[1].message == "Existing data: {'objectId': 2, 'data': 'bar'}"
-            assert caplog.records[1].levelname == 'WARNING'
-
-            assert caplog.records[2].message == "New data: {'key': 'b', 'data': 'BAR'}"
-            assert caplog.records[2].levelname == 'WARNING'
-
-    def test_parse_results_doesnt_log_any_success_on_all_failed(self, mocker, caplog):
-        class_mock = mocker.Mock()
-        class_mock._class_logger = logging.getLogger('root')
-        combined_values = {
-            1: {
-                'old_values': {
-                    'objectId': 1,
-                    'data': 'foo',
-                },
-                'new_values': {
-                    'key': 'a',
-                    'data': 'FOO'
-                }
-            },
-            2: {
-                'old_values': {
-                    'objectId': 2,
-                    'data': 'bar',
-                },
-                'new_values': {
-                    'key': 'b',
-                    'data': 'BAR'
-                }
-            }
-        }
-        class_mock._get_old_and_new_values.return_value = combined_values
-        results_dict = {
-            'addResults': [],
-            'updateResults': [
-                {
-                    'objectId': 1,
-                    'success': False
-                },
-                {
-                    'objectId': 2,
-                    'success': False
-                },
-            ],
-            'deleteResults': [],
-        }
-        live_dataframe = pd.DataFrame({
-            'objectId': [1, 2],
-            'data': ['foo', 'bar'],
-        })
-
-        with caplog.at_level(logging.INFO):
-            rows_updated = palletjack.FeatureServiceInlineUpdater._parse_results(
-                class_mock, results_dict, live_dataframe
-            )
-
-            assert 'rows successfully updated' not in caplog.text
-
-    def test_parse_results_logs_successes_before_failure(self, mocker, caplog):
-        class_mock = mocker.Mock()
-        class_mock._class_logger = logging.getLogger('root')
-        combined_values = [{
-            1: {
-                'old_values': {
-                    'objectId': 1,
-                    'data': 'foo',
-                },
-                'new_values': {
-                    'key': 'a',
-                    'data': 'FOO'
-                }
-            },
-        }, {
-            2: {
-                'old_values': {
-                    'objectId': 2,
-                    'data': 'bar',
-                },
-                'new_values': {
-                    'key': 'b',
-                    'data': 'BAR'
-                }
-            }
-        }]
-        class_mock._get_old_and_new_values.side_effect = combined_values
-        results_dict = {
-            'addResults': [],
-            'updateResults': [
-                {
-                    'objectId': 1,
-                    'success': True
-                },
-                {
-                    'objectId': 2,
-                    'success': False
-                },
-            ],
-            'deleteResults': [],
-        }
-        live_dataframe = pd.DataFrame({
-            'objectId': [1, 2],
-            'data': ['foo', 'bar'],
-        })
-
-        with caplog.at_level(logging.DEBUG):
-            rows_updated = palletjack.FeatureServiceInlineUpdater._parse_results(
-                class_mock, results_dict, live_dataframe
-            )
-            assert caplog.records[0].message == '1 rows successfully updated:'
-            assert caplog.records[0].levelname == 'INFO'
-            assert caplog.records[1].message == "Existing data: {'objectId': 1, 'data': 'foo'}"
-            assert caplog.records[1].levelname == 'DEBUG'
-            assert caplog.records[2].message == "New data: {'key': 'a', 'data': 'FOO'}"
-            assert caplog.records[2].levelname == 'DEBUG'
-            assert caplog.records[
-                3
-            ].message == 'The following 1 updates failed. As a result, all successful updates should have been rolled back.'
-            assert caplog.records[3].levelname == 'WARNING'
-            assert caplog.records[4].message == "Existing data: {'objectId': 2, 'data': 'bar'}"
-            assert caplog.records[4].levelname == 'WARNING'
-            assert caplog.records[5].message == "New data: {'key': 'b', 'data': 'BAR'}"
-            assert caplog.records[5].levelname == 'WARNING'
-
-    def test_parse_results_returns_0_when_results_are_empty(self, mocker, caplog):
-        class_mock = mocker.Mock()
-        results_dict = {
-            'addResults': [],
-            'updateResults': [],
-            'deleteResults': [],
-        }
-        live_dataframe = pd.DataFrame({
-            'objectId': [1, 2],
-            'data': ['foo', 'bar'],
-        })
-
-        with caplog.at_level(logging.INFO):
-            rows_updated = palletjack.FeatureServiceInlineUpdater._parse_results(
-                class_mock, results_dict, live_dataframe
-            )
-
-            assert rows_updated == 0
-
-    def test_parse_results_logs_appropriately_when_results_are_empty(self, mocker, caplog):
-        class_mock = mocker.Mock()
-        class_mock._class_logger = logging.getLogger('root')
-        results_dict = {
-            'addResults': [],
-            'updateResults': [],
-            'deleteResults': [],
-        }
-        live_dataframe = pd.DataFrame({
-            'objectId': [1, 2],
-            'data': ['foo', 'bar'],
-        })
-
-        with caplog.at_level(logging.INFO):
-            rows_updated = palletjack.FeatureServiceInlineUpdater._parse_results(
-                class_mock, results_dict, live_dataframe
-            )
-
-            assert 'No update results returned; no updates attempted' in caplog.text
-            assert 'rows successfully updated' not in caplog.text
-            assert 'updates failed.' not in caplog.text
-
-    def test_get_old_and_new_values_with_correct_data(self, mocker):
-        class_mock = mocker.Mock()
-        class_mock.new_dataframe = pd.DataFrame.from_dict(
-            orient='index',
-            data={
-                0: {
-                    'foo': '42',
-                    'bar': 32,
-                    'key': 'a'
-                },
-                1: {
-                    'foo': '56',
-                    'bar': 8,
-                    'key': 'b'
-                },
-            },
-        )
-
-        class_mock.index_column = 'key'
-        live_dict = {
-            1: {
-                'OBJECTID': 2,
-                'key': 'a',
-                'foo': '42000',
-                'bar': 32000
-            },
-            2: {
-                'OBJECTID': 10,
-                'key': 'b',
-                'foo': '56000',
-                'bar': 8000
-            }
-        }
-        oids = [10, 2]
-
-        combined_data = palletjack.FeatureServiceInlineUpdater._get_old_and_new_values(class_mock, live_dict, oids)
-
-        assert combined_data == {
-            2: {
-                'old_values': {
-                    'OBJECTID': 2,
-                    'key': 'a',
-                    'foo': '42000',
-                    'bar': 32000
-                },
-                'new_values': {
-                    'foo': '42',
-                    'bar': 32,
-                    'key': 'a',
-                }
-            },
-            10: {
-                'old_values': {
-                    'OBJECTID': 10,
-                    'key': 'b',
-                    'foo': '56000',
-                    'bar': 8000
-                },
-                'new_values': {
-                    'foo': '56',
-                    'bar': 8,
-                    'key': 'b'
-                }
-            }
-        }
-
-    def test_get_old_and_new_values_only_includes_existing_data_that_match_passed_objectids(self, mocker):
-        class_mock = mocker.Mock()
-        class_mock.new_dataframe = pd.DataFrame.from_dict(
-            orient='index',
-            data={
-                0: {
-                    'foo': '42',
-                    'bar': 32,
-                    'key': 'a'
-                },
-                1: {
-                    'foo': '56',
-                    'bar': 8,
-                    'key': 'b'
-                },
-            },
-        )
-
-        class_mock.index_column = 'key'
-        live_dict = {
-            1: {
-                'OBJECTID': 2,
-                'key': 'a',
-                'foo': '42000',
-                'bar': 32000
-            },
-            2: {
-                'OBJECTID': 10,
-                'key': 'b',
-                'foo': '56000',
-                'bar': 8000
-            },
-            3: {
-                'OBJECTID': 42,
-                'key': 'c',
-                'foo': '-10',
-                'bar': -88
-            }
-        }
-        oids = [10, 2]
-
-        combined_data = palletjack.FeatureServiceInlineUpdater._get_old_and_new_values(class_mock, live_dict, oids)
-
-        assert combined_data == {
-            2: {
-                'old_values': {
-                    'OBJECTID': 2,
-                    'key': 'a',
-                    'foo': '42000',
-                    'bar': 32000
-                },
-                'new_values': {
-                    'foo': '42',
-                    'bar': 32,
-                    'key': 'a',
-                }
-            },
-            10: {
-                'old_values': {
-                    'OBJECTID': 10,
-                    'key': 'b',
-                    'foo': '56000',
-                    'bar': 8000
-                },
-                'new_values': {
-                    'foo': '56',
-                    'bar': 8,
-                    'key': 'b'
-                }
-            }
-        }
-
-    def test_get_old_and_new_values_only_returns_new_data_that_match_passed_objectids(self, mocker):
-        class_mock = mocker.Mock()
-        class_mock.new_dataframe = pd.DataFrame.from_dict(
-            orient='index',
-            data={
-                0: {
-                    'foo': '42',
-                    'bar': 32,
-                    'key': 'a'
-                },
-                1: {
-                    'foo': '56',
-                    'bar': 8,
-                    'key': 'b'
-                },
-                2: {
-                    'foo': '88',
-                    'bar': 64,
-                    'key': 'z'
-                },
-            },
-        )
-
-        class_mock.index_column = 'key'
-        live_dict = {
-            1: {
-                'OBJECTID': 2,
-                'key': 'a',
-                'foo': '42000',
-                'bar': 32000
-            },
-            2: {
-                'OBJECTID': 10,
-                'key': 'b',
-                'foo': '56000',
-                'bar': 8000
-            }
-        }
-        oids = [10, 2]
-
-        combined_data = palletjack.FeatureServiceInlineUpdater._get_old_and_new_values(class_mock, live_dict, oids)
-
-        assert combined_data == {
-            2: {
-                'old_values': {
-                    'OBJECTID': 2,
-                    'key': 'a',
-                    'foo': '42000',
-                    'bar': 32000
-                },
-                'new_values': {
-                    'foo': '42',
-                    'bar': 32,
-                    'key': 'a',
-                }
-            },
-            10: {
-                'old_values': {
-                    'OBJECTID': 10,
-                    'key': 'b',
-                    'foo': '56000',
-                    'bar': 8000
-                },
-                'new_values': {
-                    'foo': '56',
-                    'bar': 8,
-                    'key': 'b'
-                }
-            }
-        }
-
-
-class TestFeatureServiceInlineUpdaterFieldValidation:
-
-    def test_validate_working_fields_in_live_and_new_dataframes_doesnt_raise_when_matching(self, mocker):
-        live_df = pd.DataFrame(columns=['field1', 'field2', 'field3'])
-        new_df = pd.DataFrame(columns=['field1', 'field2', 'field3'])
-        fields = ['field1', 'field2', 'field3']
-
-        updater_mock = mocker.Mock()
-        updater_mock.new_dataframe = new_df
-
-        #: This shouldn't raise an exception and so the test should pass
-        palletjack.FeatureServiceInlineUpdater._validate_working_fields_in_live_and_new_dataframes(
-            updater_mock, live_df, fields
-        )
-
-    def test_validate_working_fields_in_live_and_new_dataframes_raises_not_in_live(self, mocker):
-        live_df = pd.DataFrame(columns=['field1', 'field2'])
-        new_df = pd.DataFrame(columns=['field1', 'field2', 'field3'])
-        fields = ['field1', 'field2', 'field3']
-
-        updater_mock = mocker.Mock()
-        updater_mock.new_dataframe = new_df
-
-        with pytest.raises(RuntimeError) as exc_info:
-            palletjack.FeatureServiceInlineUpdater._validate_working_fields_in_live_and_new_dataframes(
-                updater_mock, live_df, fields
-            )
-        assert exc_info.value.args[
-            0
-        ] == 'Field mismatch between defined fields and either new or live data.\nFields not in live data: {\'field3\'}\nFields not in new data: set()'
-
-    def test_validate_working_fields_in_live_and_new_dataframes_raises_not_in_new(self, mocker):
-        live_df = pd.DataFrame(columns=['field1', 'field2', 'field3'])
-        new_df = pd.DataFrame(columns=['field1', 'field2'])
-        fields = ['field1', 'field2', 'field3']
-
-        updater_mock = mocker.Mock()
-        updater_mock.new_dataframe = new_df
-
-        with pytest.raises(RuntimeError) as exc_info:
-            palletjack.FeatureServiceInlineUpdater._validate_working_fields_in_live_and_new_dataframes(
-                updater_mock, live_df, fields
-            )
-        assert exc_info.value.args[
-            0
-        ] == 'Field mismatch between defined fields and either new or live data.\nFields not in live data: set()\nFields not in new data: {\'field3\'}'
-
-    def test_validate_working_fields_in_live_and_new_dataframes_raises_not_in_both(self, mocker):
-        live_df = pd.DataFrame(columns=['field1', 'field2'])
-        new_df = pd.DataFrame(columns=['field1', 'field2'])
-        fields = ['field1', 'field2', 'field3']
-
-        updater_mock = mocker.Mock()
-        updater_mock.new_dataframe = new_df
-
-        with pytest.raises(RuntimeError) as exc_info:
-            palletjack.FeatureServiceInlineUpdater._validate_working_fields_in_live_and_new_dataframes(
-                updater_mock, live_df, fields
-            )
-        assert exc_info.value.args[
-            0
-        ] == 'Field mismatch between defined fields and either new or live data.\nFields not in live data: {\'field3\'}\nFields not in new data: {\'field3\'}'
-
-
-class TestFeatureServiceInlineUpdaterIntegrated:
-
-    def test_update_existing_features_in_hosted_feature_layer_no_matching_rows_returns_0(self, mocker, caplog):
-        pd_mock = mocker.Mock()
-        pd_mock.return_value = pd.DataFrame({
-            'OBJECTID': [1, 2],
-            'data': ['foo', 'bar'],
-            'key': ['a', 'b'],
-        })
-        mocker.patch.object(pd.DataFrame.spatial, 'from_layer', new=pd_mock)
-        mocker.patch.object(arcgis.features.FeatureLayer, 'fromitem')
-
-        new_dataframe = pd.DataFrame({
-            'data': ['FOO', 'BAR'],
-            'key': ['c', 'd'],
-        })
-        gis_mock = mocker.Mock()
-        updater = palletjack.FeatureServiceInlineUpdater(gis_mock, new_dataframe, 'key')
-
-        updated_rows = updater.update_existing_features_in_hosted_feature_layer('1234', ['data', 'key'])
-
-        assert updated_rows == 0
-
-    def test_update_existing_features_in_hosted_feature_layer_no_matching_rows_properly_logs(self, mocker, caplog):
-        pd_mock = mocker.Mock()
-        pd_mock.return_value = pd.DataFrame({
-            'OBJECTID': [1, 2],
-            'data': ['foo', 'bar'],
-            'key': ['a', 'b'],
-        })
-        mocker.patch.object(pd.DataFrame.spatial, 'from_layer', new=pd_mock)
-        mocker.patch.object(arcgis.features.FeatureLayer, 'fromitem')
-
-        new_dataframe = pd.DataFrame({
-            'data': ['FOO', 'BAR'],
-            'key': ['c', 'd'],
-        })
-        gis_mock = mocker.Mock()
-        updater = palletjack.FeatureServiceInlineUpdater(gis_mock, new_dataframe, 'key')
-
-        updated_rows = updater.update_existing_features_in_hosted_feature_layer('1234', ['data', 'key'])
-
-        assert 'No matching rows between live dataset and new dataset based on field `key`' in caplog.text
-        assert "The following keys from the new data were not found in the existing dataset: ['c', 'd']" in caplog.text
-
-    def test_update_existing_features_in_hosted_feature_layer_all_matching_returns_2(self, mocker, caplog):
-        pd_mock = mocker.Mock()
-        pd_mock.return_value = pd.DataFrame({
-            'OBJECTID': [1, 2],
-            'data': ['foo', 'bar'],
-            'key': ['a', 'b'],
-        })
-        mocker.patch.object(pd.DataFrame.spatial, 'from_layer', new=pd_mock)
-
-        new_dataframe = pd.DataFrame({
-            'data': ['FOO', 'BAR'],
-            'key': ['a', 'b'],
-        })
-        gis_mock = mocker.Mock()
-        fromitem_function_mock = mocker.Mock()
-        fromitem_function_mock.return_value.edit_features.return_value = {
-            'addResults': [],
-            'updateResults': [
-                {
-                    'objectId': 1,
-                    'success': True
-                },
-                {
-                    'objectId': 2,
-                    'success': True
-                },
-            ],
-            'deleteResults': [],
-        }
-        mocker.patch.object(arcgis.features.FeatureLayer, 'fromitem', new=fromitem_function_mock)
-        updater = palletjack.FeatureServiceInlineUpdater(gis_mock, new_dataframe, 'key')
-
-        updated_rows = updater.update_existing_features_in_hosted_feature_layer('1234', ['data', 'key'])
-
-        assert updated_rows == 2
-
-    def test_update_existing_features_in_hosted_feature_layer_all_matching_logs_properly(self, mocker, caplog):
-        pd_mock = mocker.Mock()
-        pd_mock.return_value = pd.DataFrame({
-            'OBJECTID': [1, 2],
-            'data': ['foo', 'bar'],
-            'key': ['a', 'b'],
-        })
-        mocker.patch.object(pd.DataFrame.spatial, 'from_layer', new=pd_mock)
-        new_dataframe = pd.DataFrame({
-            'data': ['FOO', 'BAR'],
-            'key': ['a', 'b'],
-        })
-        gis_mock = mocker.Mock()
-        fromitem_function_mock = mocker.Mock()
-        fromitem_function_mock.return_value.edit_features.return_value = {
-            'addResults': [],
-            'updateResults': [
-                {
-                    'objectId': 1,
-                    'success': True
-                },
-                {
-                    'objectId': 2,
-                    'success': True
-                },
-            ],
-            'deleteResults': [],
-        }
-        mocker.patch.object(arcgis.features.FeatureLayer, 'fromitem', new=fromitem_function_mock)
-        updater = palletjack.FeatureServiceInlineUpdater(gis_mock, new_dataframe, 'key')
-
-        with caplog.at_level(logging.DEBUG):
-            updated_rows = updater.update_existing_features_in_hosted_feature_layer('1234', ['data', 'key'])
-            assert '2 rows successfully updated:' in caplog.text
-            assert "Existing data: {'OBJECTID': 1, 'data': 'foo', 'key': 'a'}" in caplog.text
-            assert "New data: {'data': 'FOO', 'key': 'a'}" in caplog.text
-            assert "Existing data: {'OBJECTID': 2, 'data': 'bar', 'key': 'b'}" in caplog.text
-            assert "New data: {'data': 'BAR', 'key': 'b'}" in caplog.text
-
-    def test_update_existing_features_in_hosted_feature_layer_subsets_logging_fields_properly(self, mocker, caplog):
-        pd_mock = mocker.Mock()
-        pd_mock.return_value = pd.DataFrame({
-            'OBJECTID': [1, 2],
-            'data': ['foo', 'bar'],
-            'key': ['a', 'b'],
-            'extra_field': ['muck', 'slime']
-        })
-        mocker.patch.object(pd.DataFrame.spatial, 'from_layer', new=pd_mock)
-        new_dataframe = pd.DataFrame({
-            'data': ['FOO', 'BAR'],
-            'key': ['a', 'b'],
-        })
-        gis_mock = mocker.Mock()
-        fromitem_function_mock = mocker.Mock()
-        fromitem_function_mock.return_value.edit_features.return_value = {
-            'addResults': [],
-            'updateResults': [
-                {
-                    'objectId': 1,
-                    'success': True
-                },
-                {
-                    'objectId': 2,
-                    'success': True
-                },
-            ],
-            'deleteResults': [],
-        }
-        mocker.patch.object(arcgis.features.FeatureLayer, 'fromitem', new=fromitem_function_mock)
-        updater = palletjack.FeatureServiceInlineUpdater(gis_mock, new_dataframe, 'key')
-
-        with caplog.at_level(logging.DEBUG):
-            updated_rows = updater.update_existing_features_in_hosted_feature_layer('1234', ['data', 'key'])
-            assert '2 rows successfully updated:' in caplog.text
-            assert "Existing data: {'OBJECTID': 1, 'data': 'foo', 'key': 'a'}" in caplog.text
-            assert "New data: {'data': 'FOO', 'key': 'a'}" in caplog.text
-            assert "Existing data: {'OBJECTID': 2, 'data': 'bar', 'key': 'b'}" in caplog.text
-            assert "New data: {'data': 'BAR', 'key': 'b'}" in caplog.text
-
-
-class TestColorRampReclassifier:
-
-    def test_get_layer_id_returns_match_single_layer(self, mocker):
-        layers = {
-            'operationalLayers': [
-                {
-                    'title': 'foo'
-                },
-            ],
-        }
-        get_data_mock = mocker.Mock(return_value=layers)
-        webmap_item_mock = mocker.Mock()
-        webmap_item_mock.get_data = get_data_mock
-        reclassifier = palletjack.ColorRampReclassifier(webmap_item_mock, 'gis')
-
-        layer_id = reclassifier._get_layer_id('foo')
-        assert layer_id == 0
-
-    def test_get_layer_id_returns_match_many_layers(self, mocker):
-        layers = {
-            'operationalLayers': [
-                {
-                    'title': 'foo'
-                },
-                {
-                    'title': 'bar'
-                },
-                {
-                    'title': 'baz'
-                },
-            ],
-        }
-        get_data_mock = mocker.Mock(return_value=layers)
-        webmap_item_mock = mocker.Mock()
-        webmap_item_mock.get_data = get_data_mock
-        reclassifier = palletjack.ColorRampReclassifier(webmap_item_mock, 'gis')
-
-        layer_id = reclassifier._get_layer_id('bar')
-        assert layer_id == 1
-
-    def test_get_layer_id_returns_first_match(self, mocker):
-        layers = {
-            'operationalLayers': [
-                {
-                    'title': 'foo'
-                },
-                {
-                    'title': 'bar'
-                },
-                {
-                    'title': 'bar'
-                },
-            ],
-        }
-        get_data_mock = mocker.Mock(return_value=layers)
-        webmap_item_mock = mocker.Mock()
-        webmap_item_mock.get_data = get_data_mock
-        reclassifier = palletjack.ColorRampReclassifier(webmap_item_mock, 'gis')
-
-        layer_id = reclassifier._get_layer_id('bar')
-        assert layer_id == 1
-
-    def test_get_layer_id_raises_error_when_not_found(self, mocker):
-        layers = {
-            'operationalLayers': [
-                {
-                    'title': 'bar'
-                },
-            ],
-        }
-        get_data_mock = mocker.Mock(return_value=layers)
-        webmap_item_mock = mocker.Mock()
-        webmap_item_mock.title = 'test map'
-        webmap_item_mock.get_data = get_data_mock
-        reclassifier = palletjack.ColorRampReclassifier(webmap_item_mock, 'gis')
-
-        with pytest.raises(ValueError) as error_info:
-            layer_id = reclassifier._get_layer_id('foo')
-
-        assert 'Could not find "foo" in test map' in str(error_info.value)
-
-    def test_calculate_new_stops_with_manual_numbers(self):
-        dataframe = pd.DataFrame({'numbers': [100, 300, 500, 700, 900]})
-
-        stops = palletjack.ColorRampReclassifier._calculate_new_stops(dataframe, 'numbers', 5)
-
-        assert stops == [100, 279, 458, 637, 816]
-
-    def test_calculate_new_stops_mismatched_column_raises_error(self):
-        dataframe = pd.DataFrame({'numbers': [100, 300, 500, 700, 900]})
-
-        with pytest.raises(ValueError) as error_info:
-            stops = palletjack.ColorRampReclassifier._calculate_new_stops(dataframe, 'foo', 5)
-            assert 'Column `foo` not in dataframe`' in str(error_info)
-
-    def test_update_stops_values(self, mocker):
-        # renderer = data['operationalLayers'][layer_number]['layerDefinition']['drawingInfo']['renderer']
-        # stops = renderer['visualVariables'][0]['stops']
-
-        data = {
-            'operationalLayers': [{
-                'layerDefinition': {
-                    'drawingInfo': {
-                        'renderer': {
-                            'visualVariables': [{
-                                'stops': [{
-                                    'value': 0
-                                }, {
-                                    'value': 1
-                                }, {
-                                    'value': 2
-                                }, {
-                                    'value': 3
-                                }]
-                            }]
-                        }
-                    }
-                }
-            }],
-        }
-        get_data_mock = mocker.Mock(return_value=data)
-        webmap_item_mock = mocker.Mock()
-        webmap_item_mock.get_data = get_data_mock
-        update_mock = mocker.Mock()
-        webmap_item_mock.update = update_mock
-        reclassifier = palletjack.ColorRampReclassifier(webmap_item_mock, 'gis')
-
-        reclassifier._update_stop_values(0, [100, 200, 300, 400])
-
-        data['operationalLayers'][0]['layerDefinition']['drawingInfo']['renderer']['visualVariables'][0]['stops'] = [{
-            'value': 100
-        }, {
-            'value': 200
-        }, {
-            'value': 300
-        }, {
-            'value': 400
-        }]
-
-        assert update_mock.called_with(item_properties={'text': json.dumps(data)})
-
-
-class TestAttachments:
-
-    def test_create_attachment_action_df_adds_for_blank_existing_name(self, mocker):
-        input_df = pd.DataFrame({
-            'NAME': [np.nan],
-            'new_path': ['bee/foo.png'],
-        })
-
-        ops_df = palletjack.FeatureServiceAttachmentsUpdater._create_attachment_action_df(
-            mocker.Mock(), input_df, 'new_path'
-        )
-
-        test_df = pd.DataFrame({
-            'NAME': [np.nan],
-            'new_path': ['bee/foo.png'],
-            'new_filename': ['foo.png'],
-            'operation': ['add'],
-        })
-
-        tm.assert_frame_equal(ops_df, test_df)
-
-    def test_create_attachment_action_df_overwrites_for_different_existing_name(self, mocker):
-        input_df = pd.DataFrame({
-            'NAME': ['bar.png'],
-            'new_path': ['bee/foo.png'],
-        })
-
-        ops_df = palletjack.FeatureServiceAttachmentsUpdater._create_attachment_action_df(
-            mocker.Mock(), input_df, 'new_path'
-        )
-
-        test_df = pd.DataFrame({
-            'NAME': ['bar.png'],
-            'new_path': ['bee/foo.png'],
-            'new_filename': ['foo.png'],
-            'operation': ['overwrite'],
-        })
-
-        tm.assert_frame_equal(ops_df, test_df)
-
-    def test_create_attachment_action_df_does_nothing_for_same_name(self, mocker):
-        input_df = pd.DataFrame({
-            'NAME': ['foo.png'],
-            'new_path': ['bee/foo.png'],
-        })
-
-        ops_df = palletjack.FeatureServiceAttachmentsUpdater._create_attachment_action_df(
-            mocker.Mock(), input_df, 'new_path'
-        )
-
-        test_df = pd.DataFrame({
-            'NAME': ['foo.png'],
-            'new_path': ['bee/foo.png'],
-            'new_filename': ['foo.png'],
-            'operation': [np.nan],
-        })
-        test_df['operation'] = test_df['operation'].astype(object)
-
-        tm.assert_frame_equal(ops_df, test_df)
-
-    def test_create_attachment_action_df_does_all_three_ops(self, mocker):
-        input_df = pd.DataFrame({
-            'NAME': ['foo.png', 'bar.png', np.nan],
-            'new_path': ['bee/foo.png', 'bee/baz.png', 'bee/bin.png'],
-        })
-
-        ops_df = palletjack.FeatureServiceAttachmentsUpdater._create_attachment_action_df(
-            mocker.Mock(), input_df, 'new_path'
-        )
-
-        test_df = pd.DataFrame({
-            'NAME': ['foo.png', 'bar.png', np.nan],
-            'new_path': ['bee/foo.png', 'bee/baz.png', 'bee/bin.png'],
-            'new_filename': ['foo.png', 'baz.png', 'bin.png'],
-            'operation': [np.nan, 'overwrite', 'add'],
-        })
-
-        tm.assert_frame_equal(ops_df, test_df)
-
-    def test_create_attachment_action_df_do_nothing_after_others(self, mocker):
-        input_df = pd.DataFrame({
-            'NAME': ['bar.png', np.nan, 'foo.png'],
-            'new_path': ['bee/baz.png', 'bee/bin.png', 'bee/foo.png'],
-        })
-
-        ops_df = palletjack.FeatureServiceAttachmentsUpdater._create_attachment_action_df(
-            mocker.Mock(), input_df, 'new_path'
-        )
-
-        test_df = pd.DataFrame({
-            'NAME': ['bar.png', np.nan, 'foo.png'],
-            'new_path': ['bee/baz.png', 'bee/bin.png', 'bee/foo.png'],
-            'new_filename': ['baz.png', 'bin.png', 'foo.png'],
-            'operation': ['overwrite', 'add', np.nan],
-        })
-
-        tm.assert_frame_equal(ops_df, test_df)
-
-    def test_get_live_data_from_join_field_values_only_gets_matching_data(self, mocker):
-        live_features_df = pd.DataFrame({
-            'OBJECTID': [1, 2, 3],
-            'GlobalID': ['guid1', 'guid2', 'guid3'],
-            'attachment_key': [11, 12, 13],
-            'deleted': ['a', 'b', 'c'],
-        })
-
-        attachments_df = pd.DataFrame({
-            'attachment_key': [12, 13],
-            'attachments': ['foo', 'bar'],
-        })
-
-        live_data_subset = palletjack.FeatureServiceAttachmentsUpdater._get_live_oid_and_guid_from_join_field_values(
-            mocker.Mock(), live_features_df, 'attachment_key', attachments_df
-        )
-
-        test_df = pd.DataFrame({
-            'OBJECTID': [2, 3],
-            'GlobalID': ['guid2', 'guid3'],
-            'attachment_key': [12, 13],
-            'attachments': ['foo', 'bar'],
-        })
-
-        tm.assert_frame_equal(live_data_subset, test_df)
-
-    def test_get_current_attachment_info_by_oid_includes_nans_for_features_wo_attachments(self, mocker):
-
-        live_attachments = [
-            {
-                'PARENTOBJECTID': 1,
-                'PARENTGLOBALID': 'parentguid1',
-                'ID': 111,
-                'NAME': 'foo.png',
-                'CONTENTTYPE': 'image/png',
-                'SIZE': 42,
-                'KEYWORDS': '',
-                'IMAGE_PREVIEW': 'preview1',
-                'GLOBALID': 'guid1',
-                'DOWNLOAD_URL': 'url1'
-            },
-            {
-                'PARENTOBJECTID': 2,
-                'PARENTGLOBALID': 'parentguid2',
-                'ID': 222,
-                'NAME': 'bar.png',
-                'CONTENTTYPE': 'image/png',
-                'SIZE': 42,
-                'KEYWORDS': '',
-                'IMAGE_PREVIEW': 'preview2',
-                'GLOBALID': 'guid2',
-                'DOWNLOAD_URL': 'url2'
-            },
-        ]
-
-        updater_mock = mocker.Mock()
-        updater_mock.feature_layer.attachments.search.return_value = live_attachments
-
-        live_data_subset_df = pd.DataFrame({
-            'OBJECTID': [1, 2, 3],
-            'GlobalID': ['guid1', 'guid2', 'guid3'],
-            'attachment_key': [11, 12, 13],
-            'attachments': ['fee', 'ber', 'boo'],
-        })
-
-        current_attachments_df = palletjack.FeatureServiceAttachmentsUpdater._get_current_attachment_info_by_oid(
-            updater_mock, live_data_subset_df
-        )
-
-        test_df = pd.DataFrame({
-            'OBJECTID': [1, 2, 3],
-            'GlobalID': ['guid1', 'guid2', 'guid3'],
-            'attachment_key': [11, 12, 13],
-            'attachments': ['fee', 'ber', 'boo'],
-            'PARENTOBJECTID': [1., 2., np.nan],
-            'NAME': ['foo.png', 'bar.png', np.nan],
-            'ID': [111, 222, pd.NA],
-        })
-        test_df['ID'] = test_df['ID'].astype('Int64')
-
-        tm.assert_frame_equal(current_attachments_df, test_df)
-
-    def test_check_attachment_dataframe_for_invalid_column_names_doesnt_raise_with_valid_names(self, mocker):
-        dataframe = pd.DataFrame(columns=['foo', 'bar'])
-        invalid_names = ['baz', 'boo']
-        palletjack.FeatureServiceAttachmentsUpdater._check_attachment_dataframe_for_invalid_column_names(
-            dataframe, invalid_names
-        )
-
-    def test_check_attachment_dataframe_for_invalid_column_names_raises_with_one_invalid(self, mocker):
-        dataframe = pd.DataFrame(columns=['foo', 'bar'])
-        invalid_names = ['foo', 'boo']
-        with pytest.raises(RuntimeError) as exc_info:
-            palletjack.FeatureServiceAttachmentsUpdater._check_attachment_dataframe_for_invalid_column_names(
-                dataframe, invalid_names
-            )
-        assert exc_info.value.args[0] == 'Attachment dataframe contains the following invalid names: [\'foo\']'
-
-    def test_check_attachment_dataframe_for_invalid_column_names_raises_with_all_invalid(self, mocker):
-        dataframe = pd.DataFrame(columns=['foo', 'bar'])
-        invalid_names = ['foo', 'bar']
-        with pytest.raises(RuntimeError) as exc_info:
-            palletjack.FeatureServiceAttachmentsUpdater._check_attachment_dataframe_for_invalid_column_names(
-                dataframe, invalid_names
-            )
-        assert exc_info.value.args[0] == 'Attachment dataframe contains the following invalid names: [\'foo\', \'bar\']'
-
-    def test_add_attachments_by_oid_adds_and_doesnt_warn(self, mocker):
-        action_df = pd.DataFrame({
-            'OBJECTID': [1, 2],
-            'operation': ['add', 'add'],
-            'path': ['path1', 'path2'],
-        })
-
-        result_dict = [
-            {
-                'addAttachmentResult': {
-                    'success': True
-                }
-            },
-            {
-                'addAttachmentResult': {
-                    'success': True
-                }
-            },
-        ]
-
-        updater_mock = mocker.Mock()
-        updater_mock.feature_layer.attachments.add.side_effect = result_dict
-
-        with pytest.warns(None) as warning:
-            count = palletjack.FeatureServiceAttachmentsUpdater._add_attachments_by_oid(updater_mock, action_df, 'path')
-
-        assert count == 2
-        assert not warning
-
-    def test_add_attachments_by_oid_warns_on_failure_and_doesnt_count_that_one_and_continues(self, mocker):
-        action_df = pd.DataFrame({
-            'OBJECTID': [1, 2, 3],
-            'operation': ['add', 'add', 'add'],
-            'path': ['path1', 'path2', 'path3'],
-        })
-
-        result_dict = [
-            {
-                'addAttachmentResult': {
-                    'success': True
-                }
-            },
-            {
-                'addAttachmentResult': {
-                    'success': False
-                }
-            },
-            {
-                'addAttachmentResult': {
-                    'success': True
-                }
-            },
-        ]
-
-        feature_layer_mock = mocker.Mock()
-        feature_layer_mock.attachments.add.side_effect = result_dict
-
-        updater = palletjack.FeatureServiceAttachmentsUpdater(mocker.Mock())
-        updater.feature_layer = feature_layer_mock
-
-        with pytest.warns(UserWarning, match='Failed to attach path2 to OID 2'):
-            count = updater._add_attachments_by_oid(action_df, 'path')
-
-        assert count == 2
-        assert updater.failed_dict == {2: ('add', 'path2')}
-
-    def test_add_attachments_by_oid_handles_internal_agol_errors(self, mocker, caplog):
-        action_df = pd.DataFrame({
-            'OBJECTID': [1, 2],
-            'operation': ['add', 'add'],
-            'path': ['path1', 'path2'],
-        })
-
-        feature_layer_mock = mocker.Mock()
-        feature_layer_mock.attachments.add.side_effect = [
-            RuntimeError('foo'),
-            {
-                'addAttachmentResult': {
-                    'success': True
-                }
-            },
-        ]
-
-        updater = palletjack.FeatureServiceAttachmentsUpdater(mocker.Mock())
-        updater.feature_layer = feature_layer_mock
-
-        count = updater._add_attachments_by_oid(action_df, 'path')
-        assert count == 1
-        assert 'AGOL error while adding path1 to OID 1' in caplog.text
-        assert 'foo' in caplog.text
-        assert updater.failed_dict == {1: ('add', 'path1')}
-
-    def test_add_attachments_by_oid_skips_overwrite_and_nan(self, mocker):
-        action_df = pd.DataFrame({
-            'OBJECTID': [1, 2, 3],
-            'operation': ['add', 'overwrite', np.nan],
-            'path': ['path1', 'path2', 'path3'],
-        })
-
-        result_dict = [
-            {
-                'addAttachmentResult': {
-                    'success': True
-                }
-            },
-        ]
-
-        updater_mock = mocker.Mock()
-        updater_mock.feature_layer.attachments.add.side_effect = result_dict
-
-        count = palletjack.FeatureServiceAttachmentsUpdater._add_attachments_by_oid(updater_mock, action_df, 'path')
-
-        assert updater_mock.feature_layer.attachments.add.call_count == 1
-        assert count == 1
-
-    def test_overwrite_attachments_by_oid_overwrites_and_doesnt_warn(self, mocker):
-        action_df = pd.DataFrame({
-            'OBJECTID': [1, 2],
-            'operation': ['overwrite', 'overwrite'],
-            'path': ['path1', 'path2'],
-            'ID': ['existing1', 'existing2'],
-            'NAME': ['oldname1', 'oldname2'],
-        })
-
-        result_dict = [
-            {
-                'updateAttachmentResult': {
-                    'success': True
-                }
-            },
-            {
-                'updateAttachmentResult': {
-                    'success': True
-                }
-            },
-        ]
-
-        updater_mock = mocker.Mock()
-        updater_mock.feature_layer.attachments.update.side_effect = result_dict
-
-        with pytest.warns(None) as warning:
-            count = palletjack.FeatureServiceAttachmentsUpdater._overwrite_attachments_by_oid(
-                updater_mock, action_df, 'path'
-            )
-
-        assert count == 2
-        assert not warning
-
-    def test_overwrite_attachments_by_oid_warns_on_failure_and_doesnt_count_that_one_and_continues(self, mocker):
-        action_df = pd.DataFrame({
-            'OBJECTID': [1, 2, 3],
-            'operation': ['overwrite', 'overwrite', 'overwrite'],
-            'path': ['path1', 'path2', 'path3'],
-            'ID': [11, 22, 33],
-            'NAME': ['oldname1', 'oldname2', 'oldname3'],
-        })
-
-        result_dict = [
-            {
-                'updateAttachmentResult': {
-                    'success': True
-                }
-            },
-            {
-                'updateAttachmentResult': {
-                    'success': False
-                }
-            },
-            {
-                'updateAttachmentResult': {
-                    'success': True
-                }
-            },
-        ]
-
-        feature_layer_mock = mocker.Mock()
-        feature_layer_mock.attachments.update.side_effect = result_dict
-
-        updater = palletjack.FeatureServiceAttachmentsUpdater(mocker.Mock())
-        updater.feature_layer = feature_layer_mock
-
-        with pytest.warns(UserWarning, match='Failed to update oldname2, attachment ID 22, on OID 2 with path2'):
-            count = updater._overwrite_attachments_by_oid(action_df, 'path')
-
-        assert count == 2
-        assert updater.failed_dict == {2: ('update', 'path2')}
-
-    def test_overwrite_attachments_by_oid_handles_internal_agol_errors(self, mocker, caplog):
-        action_df = pd.DataFrame({
-            'OBJECTID': [1, 2],
-            'operation': ['overwrite', 'overwrite'],
-            'path': ['path1', 'path2'],
-            'ID': [11, 22],
-            'NAME': ['oldname1', 'oldname2'],
-        })
-
-        feature_layer_mock = mocker.Mock()
-        feature_layer_mock.attachments.update.side_effect = [
-            RuntimeError('foo'),
-            {
-                'updateAttachmentResult': {
-                    'success': True
-                }
-            },
-        ]
-
-        updater = palletjack.FeatureServiceAttachmentsUpdater(mocker.Mock())
-        updater.feature_layer = feature_layer_mock
-
-        count = updater._overwrite_attachments_by_oid(action_df, 'path')
-        assert count == 1
-        assert 'AGOL error while overwriting oldname1 (attachment ID 11) on OID 1 with path1' in caplog.text
-        assert 'foo' in caplog.text
-        assert updater.failed_dict == {1: ('update', 'path1')}
-
-    def test_overwrite_attachments_by_oid_skips_add_and_nan(self, mocker):
-        action_df = pd.DataFrame({
-            'OBJECTID': [1, 2, 3],
-            'operation': ['add', 'overwrite', np.nan],
-            'path': ['path1', 'path2', 'path3'],
-            'ID': [np.nan, 'existing2', 'existing3'],
-            'NAME': ['oldname1', 'oldname2', 'oldname3'],
-        })
-
-        result_dict = [
-            {
-                'updateAttachmentResult': {
-                    'success': True
-                }
-            },
-        ]
-
-        updater_mock = mocker.Mock()
-        updater_mock.feature_layer.attachments.update.side_effect = result_dict
-
-        count = palletjack.FeatureServiceAttachmentsUpdater._overwrite_attachments_by_oid(
-            updater_mock, action_df, 'path'
-        )
-
-        assert updater_mock.feature_layer.attachments.update.call_count == 1
-        assert count == 1
-
-    def test_create_attachments_dataframe_subsets_and_crafts_paths_properly(self, mocker):
-        input_df = pd.DataFrame({
-            'join': [1, 2, 3],
-            'pic': ['foo.png', 'bar.png', 'baz.png'],
-            'data': [11., 12., 13.],
-        })
-
-        attachment_df = palletjack.FeatureServiceAttachmentsUpdater.build_attachments_dataframe(
-            input_df, 'join', 'pic', '/foo/bar'
-        )
-
-        test_df = pd.DataFrame({
-            'join': [1, 2, 3],
-            'pic': ['foo.png', 'bar.png', 'baz.png'],
-            'full_file_path': [Path('/foo/bar/foo.png'),
-                               Path('/foo/bar/bar.png'),
-                               Path('/foo/bar/baz.png')]
-        })
-
-        #: Column of path objects won't test equal in assert_frame_equal, so we make lists of their str representations
-        #: and compare those separately from the rest of the dataframe
-        other_fields = ['join', 'pic']
-        tm.assert_frame_equal(attachment_df[other_fields], test_df[other_fields])
-        assert [str(path) for path in attachment_df['full_file_path']
-               ] == [str(path) for path in test_df['full_file_path']]
-
-    def test_create_attachments_dataframe_drops_missing_attachments(self, mocker):
-        input_df = pd.DataFrame({
-            'join': [1, 2, 3],
-            'pic': ['foo.png', None, ''],
-        })
-
-        attachment_df = palletjack.FeatureServiceAttachmentsUpdater.build_attachments_dataframe(
-            input_df, 'join', 'pic', '/foo/bar'
-        )
-
-        test_df = pd.DataFrame({
-            'join': [1],
-            'pic': ['foo.png'],
-            'full_file_path': [Path('/foo/bar/foo.png')],
-        })
-
-        #: Column of path objects won't test equal in assert_frame_equal, so we make lists of their str representations
-        #: and compare those separately from the rest of the dataframe
-        other_fields = ['join', 'pic']
-        tm.assert_frame_equal(attachment_df[other_fields], test_df[other_fields])
-        assert [str(path) for path in attachment_df['full_file_path']
-               ] == [str(path) for path in test_df['full_file_path']]
-
-
-class TestFeatureServiceOverwriter:
-
-    def test_truncate_existing_data_normal(self, mocker):
-        fl_mock = mocker.Mock()
-        fl_mock.manager.truncate.return_value = {
-            'submissionTime': 123,
-            'lastUpdatedTime': 124,
-            'status': 'Completed',
-        }
-        overwriter = palletjack.FeatureServiceOverwriter(mocker.Mock())
-
-        palletjack.FeatureServiceOverwriter._truncate_existing_data(overwriter, fl_mock, 0, 'abc')
-
-    def test_truncate_existing_raises_error_on_failure(self, mocker):
-        fl_mock = mocker.Mock()
-        fl_mock.manager.truncate.return_value = {
-            'submissionTime': 123,
-            'lastUpdatedTime': 124,
-            'status': 'Foo',
-        }
-        overwriter = palletjack.FeatureServiceOverwriter(mocker.Mock())
-
-        with pytest.raises(RuntimeError) as exc_info:
-            palletjack.FeatureServiceOverwriter._truncate_existing_data(overwriter, fl_mock, 0, 'abc')
-
-        assert exc_info.value.args[0] == 'Failed to truncate existing data from layer id 0 in itemid abc'
-
-    def test_truncate_existing_retries_on_HTTPError(self, mocker):
-        fl_mock = mocker.Mock()
-        fl_mock.manager.truncate.side_effect = [
-            urllib.error.HTTPError('url', 'code', 'msg', 'hdrs', 'fp'), {
-                'submissionTime': 123,
-                'lastUpdatedTime': 124,
-                'status': 'Completed',
-            }
-        ]
-        overwriter = palletjack.FeatureServiceOverwriter(mocker.Mock())
-
-        palletjack.FeatureServiceOverwriter._truncate_existing_data(overwriter, fl_mock, 0, 'abc')
-
-    def test_append_new_data_doesnt_raise_on_normal(self, mocker):
-        mock_df = mocker.Mock()
-        mock_fl = mocker.Mock()
-        mock_fl.append.return_value = (True, {'message': 'foo'})
-
-        overwriter = palletjack.FeatureServiceOverwriter(mocker.Mock())
-        overwriter._append_new_data(mock_fl, mock_df, 0, 'abc')
-
-    def test_append_new_data_retries_on_httperror(self, mocker):
-        mock_df = mocker.Mock()
-        mock_fl = mocker.Mock()
-        mock_fl.append.side_effect = [urllib.error.HTTPError('a', 'b', 'c', 'd', 'e'), (True, {'message': 'foo'})]
-
-        overwriter = palletjack.FeatureServiceOverwriter(mocker.Mock())
-        overwriter._append_new_data(mock_fl, mock_df, 0, 'abc')
-
-    def test_append_new_data_raises_on_False_result(self, mocker):
-        mock_df = mocker.Mock()
-        mock_fl = mocker.Mock()
-        mock_fl.append.return_value = (False, {'message': 'foo'})
-
-        overwriter = palletjack.FeatureServiceOverwriter(mocker.Mock())
-
-        with pytest.raises(RuntimeError) as exc_info:
-            overwriter._append_new_data(mock_fl, mock_df, 'abc', 0)
-
-        assert exc_info.value.args[
-            0] == 'Failed to append data to layer id 0 in itemid abc. Append should have been rolled back.'
-
-    def test_truncate_and_load_feature_service_normal(self, mocker):
-        mock_fl = mocker.MagicMock()
-        mock_fl.manager.truncate.return_value = {
-            'submissionTime': 123,
-            'lastUpdatedTime': 124,
-            'status': 'Completed',
-        }
-        mock_fl.properties.fields = [
-            {
-                'name': 'Foo'
-            },
-            {
-                'name': 'Bar'
-            },
-        ]
-        mock_fl.append.return_value = (True, {'recordCount': 42})
-
-        fl_class_mock = mocker.Mock()
-        fl_class_mock.fromitem.return_value = mock_fl
-        mocker.patch('arcgis.features.FeatureLayer', fl_class_mock)
-
-        new_dataframe = pd.DataFrame(columns=['Foo', 'Bar'])
-        mocker.patch.object(pd.DataFrame, 'spatial')
-
-        overwriter = palletjack.FeatureServiceOverwriter(mocker.Mock())
-
-        uploaded_features = overwriter.truncate_and_load_feature_service('abc', new_dataframe, 'foo/dir')
-
-        assert uploaded_features == 42
-
-    def test_truncate_and_load_append_fails_reload_works(self, mocker, caplog):
-        caplog.set_level(logging.DEBUG)
-        mock_fl = mocker.Mock()
-        mock_fl.manager.truncate.return_value = {
-            'submissionTime': 123,
-            'lastUpdatedTime': 124,
-            'status': 'Completed',
-        }
-        mock_fl.properties.fields = [
-            {
-                'name': 'Foo'
-            },
-            {
-                'name': 'Bar'
-            },
-        ]
-        mock_fl.append.side_effect = [(False, 'foo'), (True, {'recordCount': 42})]
-
-        fl_class_mock = mocker.Mock()
-        fl_class_mock.fromitem.return_value = mock_fl
-        mocker.patch('arcgis.features.FeatureLayer', fl_class_mock)
-
-        new_dataframe = pd.DataFrame(columns=['Foo', 'Bar'])
-        mocker.patch.object(pd.DataFrame, 'spatial')
-
-        overwriter = palletjack.FeatureServiceOverwriter(mocker.Mock())
-
-        with pytest.raises(
-            RuntimeError,
-            match='Failed to append data to layer id 0 in itemid abc. Append should have been rolled back.'
-        ):
-            uploaded_features = overwriter.truncate_and_load_feature_service('abc', new_dataframe, 'foo/dir')
-
-        assert 'Append failed; attempting to re-load truncated data...' in caplog.text
-        assert '42 features reloaded' in caplog.text
-
-    def test_truncate_and_load_field_check_fails_reload_works(self, mocker, caplog):
-        caplog.set_level(logging.DEBUG)
-        mock_fl = mocker.Mock()
-        mock_fl.manager.truncate.return_value = {
-            'submissionTime': 123,
-            'lastUpdatedTime': 124,
-            'status': 'Completed',
-        }
-        mock_fl.properties.fields = [
-            {
-                'name': 'Foo'
-            },
-            {
-                'name': 'Bar'
-            },
-        ]
-        mock_fl.append.return_value = (True, {'recordCount': 42})
-
-        fl_class_mock = mocker.Mock()
-        fl_class_mock.fromitem.return_value = mock_fl
-        mocker.patch('arcgis.features.FeatureLayer', fl_class_mock)
-
-        new_dataframe = pd.DataFrame(columns=['Foo', 'Bar', 'Baz'])
-        mocker.patch.object(pd.DataFrame, 'spatial')
-
-        overwriter = palletjack.FeatureServiceOverwriter(mocker.Mock())
-
-        with pytest.raises(
-            RuntimeError,
-            match='New dataset contains the following fields that are not present in the live dataset: {\'Baz\'}'
-        ):
-            uploaded_features = overwriter.truncate_and_load_feature_service('abc', new_dataframe, 'foo/dir')
-
-        assert 'Append failed; attempting to re-load truncated data...' in caplog.text
-        assert '42 features reloaded' in caplog.text
-
-    def test_truncate_and_load_append_fails_reload_fails(self, mocker, caplog):
-        caplog.set_level(logging.DEBUG)
-        mock_fl = mocker.Mock()
-        mock_fl.manager.truncate.return_value = {
-            'submissionTime': 123,
-            'lastUpdatedTime': 124,
-            'status': 'Completed',
-        }
-        mock_fl.properties = {
-            'fields': [
-                {
-                    'name': 'Foo'
-                },
-                {
-                    'name': 'Bar'
-                },
-            ]
-        }
-        mock_fl.append.side_effect = [(False, 'foo'), (False, 'bar')]
-
-        fl_class_mock = mocker.Mock()
-        fl_class_mock.fromitem.return_value = mock_fl
-        mocker.patch('arcgis.features.FeatureLayer', fl_class_mock)
-
-        new_dataframe = pd.DataFrame(columns=['Foo', 'Bar'])
-        mocker.patch.object(pd.DataFrame, 'spatial')
-
-        mocker.patch.object(palletjack.FeatureServiceOverwriter, '_save_truncated_data', return_value='/foo/bar.json')
-
-        overwriter = palletjack.FeatureServiceOverwriter(mocker.Mock())
-
-        with pytest.raises(RuntimeError) as exc_info:
-            uploaded_features = overwriter.truncate_and_load_feature_service('abc', new_dataframe, 'foo/dir')
-
-        assert exc_info.value.args[
-            0] == 'Failed to re-add truncated data after failed append; data saved to /foo/bar.json'
-        assert 'Append failed; attempting to re-load truncated data...' in caplog.text
-        assert 'features reloaded' not in caplog.text
-
-    def test_save_truncated_data_calls_write_with_json(self, mocker):
-        mock_df = pd.DataFrame({
-            'foo': [1],
-            'x': [11],
-            'y': [14],
-        })
-        #: Need to 'unload' the mocked arcpy used in tests above so that .from_xy() works. This could be pulled
-        #: out if these tests or the other tests are in their own files.
-        if 'arcpy' in sys.modules:
-            del sys.modules['arcpy']
-
-        mock_sdf = pd.DataFrame.spatial.from_xy(mock_df, 'x', 'y')
-
-        open_mock = mocker.MagicMock()
-        context_manager_mock = mocker.MagicMock()
-        context_manager_mock.return_value.__enter__.return_value = open_mock
-        mocker.patch('pathlib.Path.open', new=context_manager_mock)
-
-        palletjack.FeatureServiceOverwriter._save_truncated_data(mocker.Mock(), mock_sdf, 'foo')
-
-        test_json_string = '{"features": [{"geometry": {"spatialReference": {"wkid": 4326}, "x": 11, "y": 14}, "attributes": {"foo": 1, "x": 11, "y": 14, "OBJECTID": 1}}], "objectIdFieldName": "OBJECTID", "displayFieldName": "OBJECTID", "spatialReference": {"wkid": 4326}, "geometryType": "esriGeometryPoint", "fields": [{"name": "OBJECTID", "type": "esriFieldTypeOID", "alias": "OBJECTID"}, {"name": "foo", "type": "esriFieldTypeInteger", "alias": "foo"}, {"name": "x", "type": "esriFieldTypeInteger", "alias": "x"}, {"name": "y", "type": "esriFieldTypeInteger", "alias": "y"}]}'
-
-        assert open_mock.write.called_with(test_json_string)
-
-    def test_save_truncated_data_opens_file_with_right_name(self, mocker):
-        mock_df = pd.DataFrame({
-            'foo': [1],
-            'x': [11],
-            'y': [14],
-        })
-        #: Need to 'unload' the mocked arcpy used in tests above so that .from_xy() works. This could be pulled
-        #: out if these tests or the other tests are in their own files.
-        if 'arcpy' in sys.modules:
-            del sys.modules['arcpy']
-
-        mock_sdf = pd.DataFrame.spatial.from_xy(mock_df, 'x', 'y')
-
-        open_mock = mocker.MagicMock()
-        context_manager_mock = mocker.MagicMock()
-        context_manager_mock.return_value.__enter__.return_value = open_mock
-
-        datetime_mock = mocker.Mock()
-        datetime_mock.date.today.return_value = 'foo-date'
-        mocker.patch('palletjack.updaters.datetime', new=datetime_mock)
-
-        open_mock = mocker.MagicMock()
-        context_manager_mock = mocker.MagicMock()
-        context_manager_mock.return_value.__enter__.return_value = open_mock
-        mocker.patch('pathlib.Path.open', new=context_manager_mock)
-
-        out_path = palletjack.FeatureServiceOverwriter._save_truncated_data(mocker.Mock(), mock_sdf, 'foo')
-
-        assert out_path == Path('foo/old_data_foo-date.json')
-
-
-class TestFeatureServiceInlineUpdaterUpsert:
-
-    def test_upsert_new_data_doesnt_raise_on_normal(self, mocker):
-        mock_df = mocker.MagicMock()
-        mock_fl = mocker.MagicMock()
-        mock_fl.append.return_value = (True, {'message': 'foo'})
-        mocker.patch('palletjack.utils.rename_columns_for_agol')
-        updater = palletjack.FeatureServiceInlineUpdater(mocker.Mock(), mocker.Mock(), 'foo')
-
-        updater._upsert_new_data(mock_fl, mock_df, 'abc', 0)
-
-    def test_upsert_new_data_retries_on_httperror(self, mocker):
-        mock_df = mocker.MagicMock()
-        mock_fl = mocker.MagicMock()
-        mock_fl.append.side_effect = [Exception, (True, {'message': 'foo'})]
-        mocker.patch('palletjack.utils.rename_columns_for_agol')
-
-        updater = palletjack.FeatureServiceInlineUpdater(mocker.Mock(), mocker.Mock(), 'foo')
-        updater._upsert_new_data(mock_fl, mock_df, 'abc', 0)
-
-    def test_upsert_new_data_raises_on_False_result(self, mocker):
-        mock_df = mocker.MagicMock()
-        mock_fl = mocker.MagicMock()
-        mock_fl.append.return_value = (False, {'message': 'foo'})
-        mocker.patch('palletjack.utils.rename_columns_for_agol')
-
-        updater = palletjack.FeatureServiceInlineUpdater(mocker.Mock(), mocker.Mock(), 'foo')
-
-        with pytest.raises(RuntimeError) as exc_info:
-            updater._upsert_new_data(mock_fl, mock_df, 'abc', 0)
-
-        assert exc_info.value.args[
-            0] == 'Failed to append data to layer id 0 in itemid abc. Append should have been rolled back.'
-
-    def test_upsert_new_data_in_hosted_feature_layer_normal(self, mocker):
-        mock_fl = mocker.MagicMock()
-        # mock_fl.manager.truncate.return_value = {
-        #     'submissionTime': 123,
-        #     'lastUpdatedTime': 124,
-        #     'status': 'Completed',
-        # }
-        mock_fl.properties.fields = [
-            {
-                'name': 'Foo',
-                'type': 'esriFieldTypeString',
-                'nullable': True,
-            },
-            {
-                'name': 'Bar',
-                'type': 'esriFieldTypeString',
-                'nullable': True,
-            },
-        ]
-        mock_fl.properties.indexes = [
-            {
-                'fields': 'Foo',
-                'isUnique': True
-            },
-        ]
-
-        mock_fl.append.return_value = (True, {'recordCount': 42})
-
-        fl_class_mock = mocker.Mock()
-        fl_class_mock.fromitem.return_value = mock_fl
-        mocker.patch('arcgis.features.FeatureLayer', fl_class_mock)
-
-        new_dataframe = pd.DataFrame(columns=['Foo', 'Bar'])
-        mocker.patch.object(pd.DataFrame, 'spatial')
-
-        updater = palletjack.FeatureServiceInlineUpdater(mocker.Mock(), new_dataframe, 'Foo')
-
-        uploaded_features = updater.upsert_new_data_in_hosted_feature_layer('abc')
-
-        assert uploaded_features == 42
-
-    def test_upsert_new_data_in_hosted_feature_layer_handles_agol_field_renaming(self, mocker):
-        mock_fl = mocker.MagicMock()
-        # mock_fl.manager.truncate.return_value = {
-        #     'submissionTime': 123,
-        #     'lastUpdatedTime': 124,
-        #     'status': 'Completed',
-        # }
-        mock_fl.properties.fields = [
-            {
-                'name': 'Foo_field',
-                'type': 'esriFieldTypeString',
-                'nullable': True,
-            },
-            {
-                'name': 'Bar',
-                'type': 'esriFieldTypeString',
-                'nullable': True,
-            },
-        ]
-        mock_fl.properties.indexes = [
-            {
-                'fields': 'Bar',
-                'isUnique': True
-            },
-        ]
-        mock_fl.append.return_value = (True, {'recordCount': 42})
-
-        fl_class_mock = mocker.Mock()
-        fl_class_mock.fromitem.return_value = mock_fl
-        mocker.patch('arcgis.features.FeatureLayer', fl_class_mock)
-
-        new_dataframe = pd.DataFrame(columns=['Foo field', 'Bar'])
-        mocker.patch.object(pd.DataFrame, 'spatial')
-
-        updater = palletjack.FeatureServiceInlineUpdater(mocker.Mock(), new_dataframe, 'Bar')
-
-        uploaded_features = updater.upsert_new_data_in_hosted_feature_layer('abc')
-
-        assert uploaded_features == 42
-
-    def test_upsert_new_data_in_hosted_feature_layer_handles_manual_field_renaming(self, mocker):
-        mock_fl = mocker.MagicMock()
-
-        mock_fl.properties.fields = [
-            {
-                'name': 'Shape__Length',
-                'type': 'esriFieldTypeString',
-                'nullable': True,
-            },
-            {
-                'name': 'Bar',
-                'type': 'esriFieldTypeString',
-                'nullable': True,
-            },
-        ]
-        mock_fl.properties.indexes = [
-            {
-                'fields': 'Bar',
-                'isUnique': True
-            },
-        ]
-        mock_fl.append.return_value = (True, {'recordCount': 42})
-
-        fl_class_mock = mocker.Mock()
-        fl_class_mock.fromitem.return_value = mock_fl
-        mocker.patch('arcgis.features.FeatureLayer', fl_class_mock)
-
-        new_dataframe = pd.DataFrame(columns=['st_length(shape)', 'Bar'])
-        field_mapping = {'st_length(shape)': 'Shape__Length'}
-        mocker.patch.object(pd.DataFrame, 'spatial')
-
-        updater = palletjack.FeatureServiceInlineUpdater(
-            mocker.Mock(), new_dataframe, 'Bar', field_mapping=field_mapping
-        )
-
-        uploaded_features = updater.upsert_new_data_in_hosted_feature_layer('abc')
-
-        assert uploaded_features == 42
-
-
-class TestFeatureServiceInlineUpdaterInit:
-
-    def test_init_raises_on_missing_index_field(self, mocker):
-
-        new_dataframe = pd.DataFrame(columns=['Foo_field', 'Bar'])
-        # mocker.patch.object(pd.DataFrame, 'spatial')
-
-        with pytest.raises(KeyError) as exc_info:
-            updater = palletjack.FeatureServiceInlineUpdater(mocker.Mock(), new_dataframe, 'Baz')
-
-        assert exc_info.value.args[0] == 'Index column Baz not found in dataframe columns'
-
-    def test_init_renames_dataframe_columns(self, mocker):
-
-        new_dataframe = pd.DataFrame(columns=['Foo field', 'Bar'])
-        # mocker.patch.object(pd.DataFrame, 'spatial')
-
-        updater = palletjack.FeatureServiceInlineUpdater(mocker.Mock(), new_dataframe, 'Bar')
-
-        assert list(updater.new_dataframe.columns) == ['Foo_field', 'Bar']
-
-    def test_init_renames_index_column(self, mocker):
-
-        new_dataframe = pd.DataFrame(columns=['Foo field', 'Bar'])
-        # mocker.patch.object(pd.DataFrame, 'spatial')
-
-        updater = palletjack.FeatureServiceInlineUpdater(mocker.Mock(), new_dataframe, 'Foo field')
-
-        assert updater.index_column == 'Foo_field'
diff --git a/tests/test_utils.py b/tests/test_utils.py
index b99ec86..24da694 100644
--- a/tests/test_utils.py
+++ b/tests/test_utils.py
@@ -1,14 +1,23 @@
+import datetime
 import logging
+import re
 from collections import namedtuple
+from pathlib import Path
 
 import numpy as np
 import pandas as pd
 import pytest
+from arcgis import geometry
 from pandas import testing as tm
 
 import palletjack
 
 
+@pytest.fixture(scope='module')  #: only call this once per module
+def iris():
+    return pd.read_csv('https://raw.githubusercontent.com/mwaskom/seaborn-data/master/iris.csv')
+
+
 class TestRenameColumns:
 
     def test_rename_columns_for_agol_handles_special_and_space(self):
@@ -182,54 +191,6 @@ def test_retry_fails_after_four_failures(self, mocker):
         assert mock.function.call_count == 4
 
 
-class TestReplaceNanSeries:
-
-    def test_replace_nan_series_with_empty_strings_one_empty_one_non_empty_float(self, mocker):
-        df = pd.DataFrame({
-            'normal': [1., 2., 3.],
-            'empty': [np.nan, np.nan, np.nan],
-        })
-
-        fixed_df = palletjack.utils.replace_nan_series_with_empty_strings(df)
-
-        test_df = pd.DataFrame({
-            'normal': [1., 2., 3.],
-            'empty': ['', '', ''],
-        })
-
-        tm.assert_frame_equal(fixed_df, test_df)
-
-    def test_replace_nan_series_with_empty_strings_other_series_has_nan(self, mocker):
-        df = pd.DataFrame({
-            'normal': [1., 2., np.nan],
-            'empty': [np.nan, np.nan, np.nan],
-        })
-
-        fixed_df = palletjack.utils.replace_nan_series_with_empty_strings(df)
-
-        test_df = pd.DataFrame({
-            'normal': [1., 2., np.nan],
-            'empty': ['', '', ''],
-        })
-
-        tm.assert_frame_equal(fixed_df, test_df)
-
-    def test_replace_nan_series_with_empty_strings_other_series_is_empty_str(self, mocker):
-        df = pd.DataFrame({
-            'normal': ['', '', ''],
-            'empty': [np.nan, np.nan, np.nan],
-        })
-
-        fixed_df = palletjack.utils.replace_nan_series_with_empty_strings(df)
-
-        test_df = pd.DataFrame({
-            'normal': ['', '', ''],
-            'empty': ['', '', ''],
-        })
-
-        tm.assert_frame_equal(fixed_df, test_df)
-
-
 class TestCheckIndexColumnInFL:
 
     def test_check_index_column_in_feature_layer_doesnt_raise_on_normal(self, mocker):
@@ -326,7 +287,7 @@ def test_geocode_addr_builds_url_correctly(self, mocker):
 
         palletjack.utils.requests.get.return_value = response_mock
 
-        palletjack.utils.geocode_addr('foo', 'bar', 'foo_key', (0.015, 0.03))
+        palletjack.utils.Geocoding.geocode_addr('foo', 'bar', 'foo_key', (0.015, 0.03))
 
         palletjack.utils.requests.get.assert_called_with(
             'https://api.mapserv.utah.gov/api/v1/geocode/foo/bar', params={'apiKey': 'foo_key'}
@@ -352,7 +313,7 @@ def test_geocode_addr_handles_kwargs_for_geocoding_api(self, mocker):
 
         palletjack.utils.requests.get.return_value = response_mock
 
-        palletjack.utils.geocode_addr('foo', 'bar', 'foo_key', (0.015, 0.03), spatialReference=3857)
+        palletjack.utils.Geocoding.geocode_addr('foo', 'bar', 'foo_key', (0.015, 0.03), spatialReference=3857)
 
         palletjack.utils.requests.get.assert_called_with(
             'https://api.mapserv.utah.gov/api/v1/geocode/foo/bar',
@@ -382,7 +343,7 @@ def test_geocode_addr_returns_coords(self, mocker):
 
         palletjack.utils.requests.get.return_value = response_mock
 
-        result = palletjack.utils.geocode_addr('foo', 'bar', 'foo_key', (0.015, 0.03))
+        result = palletjack.utils.Geocoding.geocode_addr('foo', 'bar', 'foo_key', (0.015, 0.03))
 
         assert result == (123, 456, 100., 'bar')
 
@@ -395,7 +356,7 @@ def test_geocode_addr_returns_null_island_bad_status_code(self, mocker):
 
         palletjack.utils.requests.get.return_value = response_mock
 
-        result = palletjack.utils.geocode_addr('foo', 'bar', 'foo_key', (0.015, 0.03))
+        result = palletjack.utils.Geocoding.geocode_addr('foo', 'bar', 'foo_key', (0.015, 0.03))
 
         assert result == (0, 0, 0., 'No Match')
 
@@ -415,7 +376,7 @@ def bool_mock(self):
 
         palletjack.utils.requests.get.return_value = response_mock
 
-        result = palletjack.utils.geocode_addr('foo', 'bar', 'foo_key', (0.015, 0.03))
+        result = palletjack.utils.Geocoding.geocode_addr('foo', 'bar', 'foo_key', (0.015, 0.03))
 
         assert result == (0, 0, 0., 'No Match')
 
@@ -435,7 +396,7 @@ def bool_mock(self):
 
         palletjack.utils.requests.get.return_value = response_mock
 
-        result = palletjack.utils.geocode_addr('foo', 'bar', 'foo_key', (0.015, 0.03))
+        result = palletjack.utils.Geocoding.geocode_addr('foo', 'bar', 'foo_key', (0.015, 0.03))
 
         assert 'No response from GET; request timeout?' not in caplog.text
 
@@ -459,7 +420,7 @@ def test_geocode_addr_retries_on_exception(self, mocker):
 
         palletjack.utils.requests.get.side_effect = [Exception, response_mock]
 
-        result = palletjack.utils.geocode_addr('foo', 'bar', 'foo_key', (0.015, 0.03))
+        result = palletjack.utils.Geocoding.geocode_addr('foo', 'bar', 'foo_key', (0.015, 0.03))
 
         assert palletjack.utils.requests.get.call_count == 2
         assert result == (123, 456, 100., 'bar')
@@ -485,7 +446,7 @@ def test_geocode_addr_retries_on_missing_response(self, mocker, caplog):
 
         palletjack.utils.requests.get.side_effect = [None, response_mock]
 
-        result = palletjack.utils.geocode_addr('foo', 'bar', 'foo_key', (0.015, 0.03))
+        result = palletjack.utils.Geocoding.geocode_addr('foo', 'bar', 'foo_key', (0.015, 0.03))
 
         assert 'No response from GET; request timeout?' in caplog.text
         assert palletjack.utils.requests.get.call_count == 2
@@ -515,7 +476,7 @@ def test_geocode_addr_retries_on_bad_status_code(self, mocker, caplog):
 
         palletjack.utils.requests.get.side_effect = [first_response_mock, second_response_mock]
 
-        result = palletjack.utils.geocode_addr('foo', 'bar', 'foo_key', (0.015, 0.03))
+        result = palletjack.utils.Geocoding.geocode_addr('foo', 'bar', 'foo_key', (0.015, 0.03))
 
         assert 'Did not receive a valid geocoding response; status code: 500' in caplog.text
         assert palletjack.utils.requests.get.call_count == 2
@@ -530,7 +491,7 @@ def test_geocode_addr_handles_complete_geocode_failure(self, mocker, caplog):
         bad_response.status_code = 500
         palletjack.utils.requests.get.side_effect = [bad_response] * 4
 
-        result = palletjack.utils.geocode_addr('foo', 'bar', 'foo_key', (0.015, 0.03))
+        result = palletjack.utils.Geocoding.geocode_addr('foo', 'bar', 'foo_key', (0.015, 0.03))
 
         assert 'Did not receive a valid geocoding response; status code: 500' in caplog.messages
         assert palletjack.utils.requests.get.call_count == 4
@@ -556,7 +517,7 @@ def test_geocode_addr_sleeps_appropriately(self, mocker):
 
         palletjack.utils.requests.get.return_value = response_mock
 
-        palletjack.utils.geocode_addr('foo', 'bar', 'foo_key', (0.015, 0.03))
+        palletjack.utils.Geocoding.geocode_addr('foo', 'bar', 'foo_key', (0.015, 0.03))
 
         palletjack.utils.sleep.assert_called_once()
 
@@ -583,40 +544,46 @@ def test_calc_modulus_for_reporting_interval_handles_more_than_split_value(self)
 class TestValidateAPIKey:
 
     def test_validate_api_key_good_key(self, mocker):
-        mocker.patch('palletjack.utils.requests', autospec=True)
+        req_mock = mocker.patch('palletjack.utils.requests', autospec=True)
         response_mock = mocker.Mock()
         response_mock.json.return_value = {'status': 200, 'message': 'foo'}
-        palletjack.utils.requests.get.return_value = response_mock
-
-        check = palletjack.utils.validate_api_key('foo')
+        req_mock.get.return_value = response_mock
 
-        assert check == 'valid'
+        #: Should not raise
+        palletjack.utils.Geocoding.validate_api_key('foo')
 
     def test_validate_api_key_bad_key(self, mocker):
-        mocker.patch('palletjack.utils.requests', autospec=True)
+        req_mock = mocker.patch('palletjack.utils.requests', autospec=True)
         response_mock = mocker.Mock()
         response_mock.json.return_value = {'status': 400, 'message': 'Invalid API key'}
-        palletjack.utils.requests.get.return_value = response_mock
-
-        check = palletjack.utils.validate_api_key('foo')
+        req_mock.get.return_value = response_mock
 
-        assert check == 'Invalid API key'
+        with pytest.raises(ValueError, match=re.escape('API key validation failed: Invalid API key')):
+            palletjack.utils.Geocoding.validate_api_key('foo')
 
-    def test_validate_api_key_handles_exception(self, mocker, caplog):
-        mocker.patch('palletjack.utils.requests', autospec=True)
+    def test_validate_api_key_handles_network_exception(self, mocker, caplog):
+        req_mock = mocker.patch('palletjack.utils.requests', autospec=True)
         mocker.patch('palletjack.utils.sleep')
-        caplog.set_level(logging.DEBUG)
-        # response_mock = mocker.Mock()
-        # response_mock.json.return_value = {
-        #     'status': 400,
-        #     'message': 'foo'
-        # }
-        palletjack.utils.requests.get.side_effect = [Exception('Random Error')] * 4
-
-        check = palletjack.utils.validate_api_key('foo')
-        assert check == 'Could not determine key validity; check your API key and/or network connection'
-        assert palletjack.utils.requests.get.call_count == 4
-        assert 'Random Error' in caplog.messages
+        # caplog.set_level(logging.DEBUG)
+        req_mock.get.side_effect = [IOError('Random Error')] * 4
+
+        with pytest.raises(
+            RuntimeError,
+            match=re.escape('Could not determine key validity; check your API key and/or network connection')
+        ) as exc_info:
+            palletjack.utils.Geocoding.validate_api_key('foo')
+
+        assert req_mock.get.call_count == 4
+        assert 'Random Error' in str(exc_info.value.__cause__)
+
+    def test_validate_api_key_handles_other_response(self, mocker):
+        req_mock = mocker.patch('palletjack.utils.requests', autospec=True)
+        response_mock = mocker.Mock()
+        response_mock.json.return_value = {'status': 404, 'message': 'Weird Response'}
+        req_mock.get.return_value = response_mock
+
+        with pytest.warns(UserWarning, match=re.escape('Unhandled API key validation response 404: Weird Response')):
+            palletjack.utils.Geocoding.validate_api_key('foo')
 
 
 class TestFieldRenaming:
@@ -678,7 +645,7 @@ def test_authorize_pygsheets_auths_from_file(self, mocker):
         assert pygsheets_mock.authorize.called_once_with('file')
         assert client == 'authed'
 
-    def test_authorize_pygsheets_auths_from_custom_credentials(self, mocker, caplog):
+    def test_authorize_pygsheets_auths_from_custom_credentials_file_not_found(self, mocker, caplog):
         pygsheets_mock = mocker.patch.object(palletjack.utils, 'pygsheets')
         pygsheets_mock.authorize.side_effect = [FileNotFoundError, 'authed']
 
@@ -696,6 +663,24 @@ def test_authorize_pygsheets_auths_from_custom_credentials(self, mocker, caplog)
         ]
         assert client == 'authed'
 
+    def test_authorize_pygsheets_auths_from_custom_credentials_custom_object(self, mocker, caplog):
+        pygsheets_mock = mocker.patch.object(palletjack.utils, 'pygsheets')
+        pygsheets_mock.authorize.side_effect = [TypeError, 'authed']
+
+        caplog.set_level(logging.DEBUG, logger='palletjack.utils')
+        caplog.clear()
+
+        client = palletjack.utils.authorize_pygsheets('credentials')
+
+        assert 'Credentials file not found, trying as environment variable' in [rec.message for rec in caplog.records]
+
+        assert pygsheets_mock.authorize.call_count == 2
+        assert pygsheets_mock.authorize.call_args_list == [
+            mocker.call(service_file='credentials'),
+            mocker.call(custom_credentials='credentials')
+        ]
+        assert client == 'authed'
+
     def test_authorize_pygsheets_raises_after_failing_both(self, mocker, caplog):
         pygsheets_mock = mocker.patch.object(palletjack.utils, 'pygsheets')
         pygsheets_mock.authorize.side_effect = [FileNotFoundError, IOError]
@@ -806,3 +791,992 @@ def test_fix_numeric_empty_strings_handles_single_missing_shape_info_field(self)
 
         assert fixed_feature.attributes['a'] == ''
         assert fixed_feature.attributes['b'] is None
+
+
+class TestCheckFieldsMatch:
+
+    def test_check_live_and_new_field_types_match_normal(self, mocker):
+        new_df = pd.DataFrame({
+            'ints': [1, 2, 3],
+            'floats': [4., 5., 6.],
+            'strings': ['a', 'b', 'c'],
+            'OBJECTID': [11, 12, 13],
+            'GlobalID': [
+                'cc1cd617-1e55-4153-914d-8abb6ef22f24', '0f45d56f-249e-494a-863e-6b3999619bae',
+                'd3a64873-8a09-4351-9ea0-802e450329ea'
+            ],
+            'dates': ['2015-09-02T23:08:12+00:00', '2015-09-02T23:08:13+00:00', '2015-09-02T23:08:14+00:00']
+        })
+        new_df['datetimes'] = pd.to_datetime(new_df['dates']).dt.tz_localize(None)
+        properties_mock = mocker.Mock()
+        properties_mock.fields = [
+            {
+                'name': 'OBJECTID',
+                'type': 'esriFieldTypeOID'
+            },
+            {
+                'name': 'strings',
+                'type': 'esriFieldTypeString'
+            },
+            {
+                'name': 'ints',
+                'type': 'esriFieldTypeInteger'
+            },
+            {
+                'name': 'floats',
+                'type': 'esriFieldTypeDouble'
+            },
+            {
+                'name': 'GlobalID',
+                'type': 'esriFieldTypeGlobalID'
+            },
+            {
+                'name': 'datetimes',
+                'type': 'esriFieldTypeDate'
+            },
+        ]
+
+        #: If it raises an error, it failed.
+        checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+        checker.check_live_and_new_field_types_match(['ints', 'floats', 'strings', 'OBJECTID', 'GlobalID', 'datetimes'])
+
+    def test_check_live_and_new_field_types_match_converted(self, mocker):
+        new_df = pd.DataFrame({
+            'ints': [1, 2, 3],
+            'floats': [4.1, 5.1, 6.1],
+            'strings': ['a', 'b', 'c'],
+            'OBJECTID': [11, 12, 13],
+            'GlobalID': [
+                'cc1cd617-1e55-4153-914d-8abb6ef22f24', '0f45d56f-249e-494a-863e-6b3999619bae',
+                'd3a64873-8a09-4351-9ea0-802e450329ea'
+            ],
+            'dates': ['2015-09-02T23:08:12+00:00', '2015-09-02T23:08:13+00:00', '2015-09-02T23:08:14+00:00']
+        }).convert_dtypes()
+
+        new_df['datetimes'] = pd.to_datetime(new_df['dates']).dt.tz_localize(None)
+
+        properties_mock = mocker.Mock()
+        properties_mock.fields = [
+            {
+                'name': 'OBJECTID',
+                'type': 'esriFieldTypeOID'
+            },
+            {
+                'name': 'strings',
+                'type': 'esriFieldTypeString'
+            },
+            {
+                'name': 'ints',
+                'type': 'esriFieldTypeInteger'
+            },
+            {
+                'name': 'floats',
+                'type': 'esriFieldTypeDouble'
+            },
+            {
+                'name': 'GlobalID',
+                'type': 'esriFieldTypeGlobalID'
+            },
+            {
+                'name': 'datetimes',
+                'type': 'esriFieldTypeDate'
+            },
+        ]
+
+        #: If it raises an error, it failed.
+        checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+        checker.check_live_and_new_field_types_match(['ints', 'floats', 'strings', 'OBJECTID', 'GlobalID', 'datetimes'])
+
+    def test_check_live_and_new_field_types_match_raises_on_incompatible(self, mocker):
+        new_df = pd.DataFrame({
+            'ints': [1, 2, 3],
+        })
+
+        properties_mock = mocker.Mock()
+        properties_mock.fields = [{'name': 'ints', 'type': 'esriFieldTypeDouble'}]
+
+        with pytest.raises(
+            ValueError,
+            match=re.escape(
+                'Field type incompatibilities (field, live type, new type): [(\'ints\', \'esriFieldTypeDouble\', \'int64\')]'
+            )
+        ):
+            checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+            checker.check_live_and_new_field_types_match(['ints'])
+
+    def test_check_live_and_new_field_types_match_handles_nullable_int_with_nans(self, mocker):
+        new_df = pd.DataFrame({
+            'ints': [1, 2, None],
+        }).convert_dtypes()
+
+        properties_mock = mocker.Mock()
+        properties_mock.fields = [{'name': 'ints', 'type': 'esriFieldTypeInteger'}]
+
+        checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+
+        #: should not raise
+        checker.check_live_and_new_field_types_match(['ints'])
+
+    def test_check_live_and_new_field_types_match_raises_on_multiple_incompatible(self, mocker):
+        new_df = pd.DataFrame({
+            'ints': [1, 2, 3],
+            'floats': [1.1, 1.2, 1.3],
+        })
+        properties_mock = mocker.Mock()
+        properties_mock.fields = [
+            {
+                'name': 'ints',
+                'type': 'esriFieldTypeDouble'
+            },
+            {
+                'name': 'floats',
+                'type': 'esriFieldTypeString'
+            },
+        ]
+
+        with pytest.raises(
+            ValueError,
+            match=re.escape(
+                'Field type incompatibilities (field, live type, new type): [(\'ints\', \'esriFieldTypeDouble\', \'int64\'), (\'floats\', \'esriFieldTypeString\', \'float64\')]'
+            )
+        ):
+            checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+            checker.check_live_and_new_field_types_match(['ints', 'floats'])
+
+    def test_check_live_and_new_field_types_match_raises_on_incoming_int_as_float(self, mocker):
+        new_df = pd.DataFrame({
+            'a': [1, 2, 3],
+            'b': [1.1, 1.2, np.nan],
+        })
+        properties_mock = mocker.Mock()
+        properties_mock.fields = [
+            {
+                'name': 'a',
+                'type': 'esriFieldTypeDouble'
+            },
+            {
+                'name': 'b',
+                'type': 'esriFieldTypeInteger'
+            },
+        ]
+
+        with pytest.raises(
+            palletjack.IntFieldAsFloatError,
+            match=re.escape(
+                'Field type incompatibilities (field, live type, new type): [(\'a\', \'esriFieldTypeDouble\', \'int64\'), (\'b\', \'esriFieldTypeInteger\', \'float64\')]\nCheck the following int fields for null/np.nan values and convert to panda\'s nullable int dtype: b'
+            )
+        ):
+            checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+            checker.check_live_and_new_field_types_match(['a', 'b'])
+
+    def test_check_live_and_new_field_types_match_raises_on_timezone_aware_datetime(self, mocker):
+        new_df = pd.DataFrame({
+            'a': ['2020-04-29T01:09:29+00:00'],
+        })
+        new_df['a'] = pd.to_datetime(new_df['a'])
+        properties_mock = mocker.Mock()
+        properties_mock.fields = [
+            {
+                'name': 'a',
+                'type': 'esriFieldTypeDate'
+            },
+        ]
+
+        with pytest.raises(
+            palletjack.TimezoneAwareDatetimeError,
+            match=re.escape(
+                'Field type incompatibilities (field, live type, new type): [(\'a\', \'esriFieldTypeDate\', \'datetime64[ns, UTC]\')]\nCheck the following datetime fields for timezone aware dtypes values and convert to timezone-naive dtypes using pd.to_datetime(df[\'field\']).dt.tz_localize(None): a'
+            )
+        ):
+            checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+            checker.check_live_and_new_field_types_match(['a'])
+
+    def test_check_live_and_new_field_types_match_raises_on_notimplemented_esri_type(self, mocker):
+        new_df = pd.DataFrame({
+            'ints': [1, 2, 3],
+        })
+
+        properties_mock = mocker.Mock()
+        properties_mock.fields = [{'name': 'ints', 'type': 'esriFieldTypeXML'}]
+
+        with pytest.raises(NotImplementedError) as exc_info:
+            checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+            checker.check_live_and_new_field_types_match(['ints'])
+
+        assert 'Live field "ints" type "esriFieldTypeXML" not yet mapped to a pandas dtype' in str(exc_info.value)
+
+    def test_check_live_and_new_field_types_removes_SHAPE(self, mocker):
+        geocheck_mock = mocker.patch('palletjack.utils.FieldChecker._check_geometry_types')
+        new_df = pd.DataFrame({
+            'ints': [1, 2, 3],
+            'SHAPE': [geometry.Geometry([0, 0])] * 3,
+        }).convert_dtypes()
+
+        properties_mock = mocker.Mock()
+        properties_mock.fields = [{'name': 'ints', 'type': 'esriFieldTypeInteger'}]
+
+        #: If it raises an error, it failed.
+        checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+        checker.check_live_and_new_field_types_match(['ints', 'SHAPE'])
+        geocheck_mock.assert_called_once()
+
+    def test_check_geometry_types_normal(self, mocker):
+        new_df = pd.DataFrame.spatial.from_xy(
+            pd.DataFrame({
+                'OBJECTID': [11, 12, 13],
+                'x': [0, 0, 0],
+                'y': [0, 0, 0],
+            }), x_column='x', y_column='y'
+        )
+
+        properties_mock = mocker.Mock()
+        properties_mock.geometryType = 'esriGeometryPoint'
+        properties_mock.fields = {'a': ['b']}
+
+        #: If it raises an error, it failed.
+        checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+        checker._check_geometry_types()
+
+    def test_check_geometry_types_raises_on_multiple_types(self, mocker):
+        new_df = mocker.Mock()
+        new_df.columns = ['SHAPE']
+        new_df.spatial.geometry_type = [1, 2]
+
+        properties_mock = mocker.Mock()
+        properties_mock.geometryType = 'esriGeometryPoint'
+        properties_mock.fields = {'a': ['b']}
+
+        with pytest.raises(ValueError) as exc_info:
+            checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+            checker._check_geometry_types()
+
+        assert 'New dataframe has multiple geometry types' in str(exc_info.value)
+
+    def test_check_geometry_types_raises_on_incompatible_type(self, mocker):
+        new_df = mocker.Mock()
+        new_df.columns = ['SHAPE']
+        new_df.spatial.geometry_type = ['Polygon']
+
+        properties_mock = mocker.Mock()
+        properties_mock.geometryType = 'esriGeometryPoint'
+        properties_mock.fields = {'a': ['b']}
+
+        with pytest.raises(ValueError) as exc_info:
+            checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+            checker._check_geometry_types()
+
+        assert 'New dataframe geometry type "Polygon" incompatible with live geometry type "esriGeometryPoint"' in str(
+            exc_info.value
+        )
+
+    def test_check_geometry_types_raises_on_missing_SHAPE(self, mocker):
+        new_df = mocker.Mock()
+        new_df.columns = ['foo']
+
+        checker_mock = mocker.Mock()
+        checker_mock.new_dataframe = new_df
+
+        with pytest.raises(ValueError) as exc_info:
+            palletjack.utils.FieldChecker._check_geometry_types(checker_mock)
+
+        assert 'New dataframe does not have a SHAPE column' in str(exc_info.value)
+
+
+class TestNullableIntWarning:
+
+    def test_check_nullable_ints_shapely_warns_with_na(self, mocker):
+        new_df = pd.DataFrame({
+            'normal': [1, 2, 3],
+            'nulls': [4, 5, None],
+            'floats': [6.1, 7.1, 8.1],
+        }).convert_dtypes()
+
+        mocker.patch('palletjack.utils.importlib.util.find_spec', return_value=None)
+
+        checker_mock = mocker.Mock()
+        checker_mock.new_dataframe = new_df
+
+        with pytest.warns(
+            UserWarning,
+            match=re.escape(
+                'The following columns have null values that will be replaced by 0 due to shapely conventions: nulls'
+            )
+        ):
+            palletjack.utils.FieldChecker.check_nullable_ints_shapely(checker_mock)
+
+    def test_check_nullable_ints_shapely_doesnt_warn_on_float_nan(self, mocker):
+        new_df = pd.DataFrame({
+            'normal': [1, 2, 3],
+            'nulls': [4, 5, 6],
+            'floats': [None, 7.1, 8.1],
+        }).convert_dtypes()
+
+        mocker.patch('palletjack.utils.importlib.util.find_spec', return_value=None)
+
+        checker_mock = mocker.Mock()
+        checker_mock.new_dataframe = new_df
+        warning_mock = mocker.patch('palletjack.utils.warnings')
+
+        palletjack.utils.FieldChecker.check_nullable_ints_shapely(checker_mock)
+
+        assert warning_mock.warns.call_count == 0
+
+    def test_check_nullable_ints_shapely_doesnt_warn_when_using_arcpy(self, mocker):
+        new_df = pd.DataFrame({
+            'normal': [1, 2, 3],
+            'nulls': [4, 5, None],
+            'floats': [6.1, 7.1, 8.1],
+        }).convert_dtypes()
+
+        mocker.patch('palletjack.utils.importlib.util.find_spec', return_value='__spec__')
+
+        checker_mock = mocker.Mock()
+        checker_mock.new_dataframe = new_df
+        warning_mock = mocker.patch('palletjack.utils.warnings')
+
+        palletjack.utils.FieldChecker.check_nullable_ints_shapely(checker_mock)
+
+        assert warning_mock.warns.call_count == 0
+
+
+class TestFieldNullChecker:
+
+    def test_check_for_non_null_fields_raises_on_null_data_in_nonnullable_field(self, mocker):
+        properties_mock = mocker.Mock()
+        properties_mock.fields = [
+            {
+                'name': 'regular',
+                'nullable': True,
+                'defaultValue': None,
+            },
+            {
+                'name': 'non-nullable',
+                'nullable': False,
+                'defaultValue': None,
+            },
+        ]
+
+        new_df = pd.DataFrame({
+            'regular': ['a', 'b'],
+            'non-nullable': ['c', None],
+        })
+
+        checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+
+        with pytest.raises(ValueError) as exc_info:
+            checker.check_for_non_null_fields(['regular', 'non-nullable'])
+
+        assert 'The following fields cannot have null values in the live data but one or more nulls exist in the new data: non-nullable' in str(
+            exc_info.value
+        )
+
+    def test_check_for_non_null_fields_doesnt_raise_on_null_in_nullable_field(self, mocker):
+        properties_mock = mocker.Mock()
+        properties_mock.fields = [
+            {
+                'name': 'regular',
+                'nullable': True,
+                'defaultValue': None,
+            },
+        ]
+
+        new_df = pd.DataFrame({
+            'regular': ['a', None],
+        })
+
+        checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+
+        #: Should not raise an error
+        checker.check_for_non_null_fields(['regular'])
+
+    def test_check_for_non_null_fields_doesnt_raise_on_null_in_nonnullable_with_default(self, mocker):
+        properties_mock = mocker.Mock()
+        properties_mock.fields = [
+            {
+                'name': 'regular',
+                'nullable': True,
+                'defaultValue': 'foo',
+            },
+        ]
+
+        new_df = pd.DataFrame({
+            'regular': ['a', None],
+        })
+
+        checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+
+        #: Should not raise an error
+        checker.check_for_non_null_fields(['regular'])
+
+    def test_check_for_non_null_fields_skips_field(self, mocker):
+        properties_mock = mocker.Mock()
+        properties_mock.fields = [
+            {
+                'name': 'regular',
+                'nullable': True,
+                'defaultValue': None,
+            },
+            {
+                'name': 'non-nullable',
+                'nullable': False,
+                'defaultValue': None,
+            },
+        ]
+
+        new_df = pd.DataFrame({
+            'regular': ['a', 'b'],
+            'non-nullable': ['c', None],
+        })
+
+        checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+
+        #: Should not raise
+        checker.check_for_non_null_fields(['regular'])
+
+
+class TestFieldLength:
+
+    def test_check_field_length_normal_string(self, mocker):
+        properties_mock = mocker.Mock()
+        properties_mock.fields = [
+            {
+                'name': 'foo',
+                'type': 'esriFieldTypeString',
+                'length': 10,
+            },
+        ]
+
+        new_df = pd.DataFrame({
+            'foo': ['aaa', 'bbbb'],
+        })
+
+        checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+
+        #: Should not raise
+        checker.check_field_length(['foo'])
+
+    def test_check_field_length_raises_on_long_string(self, mocker):
+        properties_mock = mocker.Mock()
+        properties_mock.fields = [
+            {
+                'name': 'foo',
+                'type': 'esriFieldTypeString',
+                'length': 10,
+            },
+        ]
+
+        new_df = pd.DataFrame({
+            'foo': ['aaa', 'bbbb', 'this string is far too long'],
+        })
+
+        checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+
+        with pytest.raises(ValueError) as exc_info:
+            checker.check_field_length(['foo'])
+
+        assert 'Row 2, column foo in new data exceeds the live data max length of 10' in str(exc_info.value)
+
+    def test_check_field_length_uses_fields_arg(self, mocker):
+        properties_mock = mocker.Mock()
+        properties_mock.fields = [
+            {
+                'name': 'foo',
+                'type': 'esriFieldTypeString',
+                'length': 10,
+            },
+            {
+                'name': 'bar',
+                'type': 'esriFieldTypeString',
+                'length': 10,
+            },
+        ]
+        new_df = pd.DataFrame({
+            'foo': ['aaa', 'bbbb'],
+            'bar': ['a', 'way too long field'],
+        })
+
+        checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+
+        #: bar shouldn't trigger an exception
+        checker.check_field_length(['foo'])
+
+    def test_check_field_length_uses_ignores_new_field_not_in_live_data(self, mocker):
+        properties_mock = mocker.Mock()
+        properties_mock.fields = [
+            {
+                'name': 'foo',
+                'type': 'esriFieldTypeString',
+                'length': 10,
+            },
+        ]
+        new_df = pd.DataFrame({
+            'foo': ['aaa', 'bbbb'],
+            'bar': ['a', 'way too long field'],
+        })
+
+        checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+
+        #: bar shouldn't trigger an exception
+        checker.check_field_length(['foo'])
+
+    def test_check_field_length_works_with_int_field(self, mocker):
+        properties_mock = mocker.Mock()
+        properties_mock.fields = [
+            {
+                'name': 'foo',
+                'type': 'esriFieldTypeString',
+                'length': 10,
+            },
+            {
+                'name': 'int',
+                'type': 'esriFieldTypeInteger',
+            },
+        ]
+        new_df = pd.DataFrame({
+            'foo': ['aaa', 'bbbb'],
+            'int': [1, 2],
+        })
+
+        checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+
+        #: Should not raise
+        checker.check_field_length(['foo', 'int'])
+
+    def test_check_field_length_passes_with_only_int_fields(self, mocker, caplog):
+        caplog.set_level(logging.DEBUG)
+        properties_mock = mocker.Mock()
+        properties_mock.fields = [
+            {
+                'name': 'foo',
+                'type': 'esriFieldTypeInteger',
+            },
+            {
+                'name': 'int',
+                'type': 'esriFieldTypeInteger',
+            },
+        ]
+        new_df = pd.DataFrame({
+            'foo': ['aaa', 'bbbb'],
+            'int': [1, 2],
+        })
+
+        checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+
+        #: Should not raise
+        checker.check_field_length(['foo'])
+
+        assert 'No fields with length property' in caplog.text
+
+
+class TestFieldsPresent:
+
+    def test_check_fields_present_normal_in_both(self, mocker):
+        properties_mock = mocker.Mock()
+        properties_mock.fields = [{'name': 'foo'}, {'name': 'bar'}]
+
+        new_df = pd.DataFrame(columns=['foo', 'bar'])
+
+        checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+
+        #: Should not raise
+        checker.check_fields_present(['foo', 'bar'], False)
+
+    def test_check_fields_present_raises_missing_live(self, mocker):
+        properties_mock = mocker.Mock()
+        properties_mock.fields = [{'name': 'foo'}]
+
+        new_df = pd.DataFrame(columns=['foo', 'bar'])
+
+        checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+
+        with pytest.raises(RuntimeError) as exc_info:
+            checker.check_fields_present(['foo', 'bar'], False)
+        assert 'Fields missing in live data: bar' in str(exc_info.value)
+
+    def test_check_fields_present_raises_missing_new(self, mocker):
+        properties_mock = mocker.Mock()
+        properties_mock.fields = [{'name': 'foo'}, {'name': 'bar'}]
+
+        new_df = pd.DataFrame(columns=['foo'])
+
+        checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+
+        with pytest.raises(RuntimeError) as exc_info:
+            checker.check_fields_present(['foo', 'bar'], False)
+        assert 'Fields missing in new data: bar' in str(exc_info.value)
+
+    def test_check_fields_present_raises_missing_both(self, mocker):
+        properties_mock = mocker.Mock()
+        properties_mock.fields = [{'name': 'foo'}]
+
+        new_df = pd.DataFrame(columns=['bar'])
+
+        checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+
+        with pytest.raises(RuntimeError) as exc_info:
+            checker.check_fields_present(['foo', 'bar'], False)
+        assert 'Fields missing in live data: bar. Fields missing in new data: foo' in str(exc_info.value)
+
+    def test_check_fields_present_adds_oid_to_list_of_fields_to_check(self, mocker):
+        properties_mock = mocker.Mock()
+        properties_mock.fields = [{'name': 'foo'}, {'name': 'bar'}, {'name': 'OBJECTID'}]
+
+        new_df = pd.DataFrame(columns=['foo', 'bar', 'OBJECTID'])
+
+        checker = palletjack.utils.FieldChecker(properties_mock, new_df)
+
+        #: Should not raise
+        checker.check_fields_present(['foo', 'bar'], True)
+
+
+class TestSRSCheck:
+
+    def test_check_srs_wgs84_good_match(self, mocker):
+        checker_mock = mocker.Mock()
+        checker_mock.new_dataframe.spatial.sr.wkid = 4326
+
+        #: Should not raise
+        palletjack.utils.FieldChecker.check_srs_wgs84(checker_mock)
+
+    def test_check_srs_wgs84_raises_on_mismatch(self, mocker):
+        checker_mock = mocker.Mock()
+        checker_mock.new_dataframe.spatial.sr.wkid = 42
+
+        with pytest.raises(
+            ValueError,
+            match=re.escape('New dataframe SRS 42 is not wkid 4326. Reproject with appropriate transformation')
+        ):
+            palletjack.utils.FieldChecker.check_srs_wgs84(checker_mock)
+
+    def test_check_srs_wgs84_handles_string_and_int(self, mocker):
+        checker_mock = mocker.Mock()
+        checker_mock.new_dataframe.spatial.sr.wkid = '4326'
+
+        #: should not raise
+        palletjack.utils.FieldChecker.check_srs_wgs84(checker_mock)
+
+    def test_check_srs_wgs84_reports_uncastable_string(self, mocker):
+        checker_mock = mocker.Mock()
+        checker_mock.new_dataframe.spatial.sr.wkid = 'forty two'
+
+        with pytest.raises(ValueError, match=re.escape('Could not cast new SRS to int')) as exc_info:
+            palletjack.utils.FieldChecker.check_srs_wgs84(checker_mock)
+
+        assert hasattr(exc_info.value, '__cause__')
+        assert isinstance(exc_info.value.__cause__, ValueError)
+        assert 'invalid literal for int() with base 10:' in str(exc_info.value.__cause__)
+
+    def test_check_srs_wgs84_handles_srs_with_key_of_0(self, mocker):
+        checker_mock = mocker.Mock()
+        checker_mock.new_dataframe.spatial.sr = {0: 4326}
+
+        #: Should not raise
+        palletjack.utils.FieldChecker.check_srs_wgs84(checker_mock)
+
+
+class TestNullGeometryGenerators:
+
+    def test_get_null_geometries_point(self, mocker):
+        properties_mock = mocker.Mock()
+        properties_mock.geometryType = 'esriGeometryPoint'
+
+        nullo = palletjack.utils.get_null_geometries(properties_mock)
+
+        assert nullo == '{"x": 0, "y": 0, "spatialReference": {"wkid": 4326}}'
+
+    def test_get_null_geometries_polyline(self, mocker):
+        properties_mock = mocker.Mock()
+        properties_mock.geometryType = 'esriGeometryPolyline'
+
+        nullo = palletjack.utils.get_null_geometries(properties_mock)
+
+        assert nullo == '{"paths": [[[0, 0], [0.1, 0.1], [0.2, 0.2]]], "spatialReference": {"wkid": 4326}}'
+
+    def test_get_null_geometries_polygon(self, mocker):
+        properties_mock = mocker.Mock()
+        properties_mock.geometryType = 'esriGeometryPolygon'
+
+        nullo = palletjack.utils.get_null_geometries(properties_mock)
+
+        assert nullo == '{"rings": [[[0, 0.1], [0.1, 0.1], [0.1, 0], [0, 0]]], "spatialReference": {"wkid": 4326}}'
+
+    def test_get_null_geometries_raises_on_other(self, mocker):
+        properties_mock = mocker.Mock()
+        properties_mock.geometryType = 'other'
+
+        with pytest.raises(NotImplementedError) as exc_info:
+            nullo = palletjack.utils.get_null_geometries(properties_mock)
+
+        assert 'Null value generator for live geometry type other not yet implemented' in str(exc_info.value)
+
+
+class TestDeleteUtils:
+
+    def test_check_delete_oids_are_ints_casts_string_list_with_spaces(self):
+        oid_list = [' 1', ' 2', '3']
+
+        numeric_oids = palletjack.utils.DeleteUtils.check_delete_oids_are_ints(oid_list)
+
+        assert numeric_oids == [1, 2, 3]
+
+    def test_check_delete_oids_are_ints_casts_string_list_without_spaces(self):
+        oid_list = ['1', '2', '3']
+
+        numeric_oids = palletjack.utils.DeleteUtils.check_delete_oids_are_ints(oid_list)
+
+        assert numeric_oids == [1, 2, 3]
+
+    def test_check_delete_oids_are_ints_passes_on_int_list(self):
+        oid_list = [1, 2, 3]
+
+        numeric_oids = palletjack.utils.DeleteUtils.check_delete_oids_are_ints(oid_list)
+
+        assert numeric_oids == [1, 2, 3]
+
+    def test_check_delete_oids_are_ints_raises_first_non_int(self):
+        oid_list = ['one', 2, 3]
+
+        with pytest.raises(TypeError) as exc_info:
+            palletjack.utils.DeleteUtils.check_delete_oids_are_ints(oid_list)
+
+        assert 'Couldn\'t convert OBJECTID(s) `[\'one\']` to integer' in str(exc_info.value)
+
+    def test_check_delete_oids_are_ints_raises_later_non_ints(self):
+        oid_list = [1, 'two', 'three']
+
+        with pytest.raises(TypeError) as exc_info:
+            palletjack.utils.DeleteUtils.check_delete_oids_are_ints(oid_list)
+
+        assert 'Couldn\'t convert OBJECTID(s) `[\'two\', \'three\']` to integer' in str(exc_info.value)
+
+    def test_check_for_empty_oid_list_doesnt_raise_on_list(self):
+        oid_list = [1, 2, 3]
+        numeric_oids = [1, 2, 3]
+
+        palletjack.utils.DeleteUtils.check_for_empty_oid_list(oid_list, numeric_oids)
+
+    def test_check_for_empty_oid_list_raises_on_empty_list(self):
+        oid_list = []
+        numeric_oids = []
+
+        with pytest.raises(ValueError) as exc_info:
+            palletjack.utils.DeleteUtils.check_for_empty_oid_list(oid_list, numeric_oids)
+
+        assert 'No OBJECTIDs found in []' in str(exc_info.value)
+
+    def test_check_delete_oids_are_in_live_data_doesnt_warn_on_good_oids(self, mocker):
+        fl_mock = mocker.Mock()
+        fl_mock.query.return_value = {'objectIdFieldName': 'OBJECTID', 'objectIds': [1, 2, 3]}
+        oid_list = [1, 2, 3]
+        oid_numeric = [1, 2, 3]
+
+        palletjack.utils.DeleteUtils.check_delete_oids_are_in_live_data(oid_list, oid_numeric, fl_mock)
+
+    def test_check_delete_oids_are_in_live_data_warns_on_missing_oid(self, mocker):
+        fl_mock = mocker.Mock()
+        fl_mock.query.return_value = {'objectIdFieldName': 'OBJECTID', 'objectIds': [1, 2]}
+        oid_list = [1, 2, 3]
+        oid_numeric = [1, 2, 3]
+
+        with pytest.warns(UserWarning, match=re.escape('OBJECTIDs {3} were not found in the live data')):
+            number_of_missing_oids = palletjack.utils.DeleteUtils.check_delete_oids_are_in_live_data(
+                oid_list, oid_numeric, fl_mock
+            )
+
+        assert number_of_missing_oids == 1
+
+
+class TestSaveDataframeToJSON:
+
+    def test_save_feature_layer_to_json_calls_write_with_json_and_returns_right_path(self, mocker):
+
+        mock_df = pd.DataFrame({
+            'foo': [1],
+            'x': [11],
+            'y': [14],
+        })
+
+        mock_sdf = pd.DataFrame.spatial.from_xy(mock_df, 'x', 'y')
+        mock_fl = mocker.Mock()
+        mock_fl.query.return_value.sdf = mock_sdf
+        mock_fl.properties.name = 'flayer'
+
+        open_mock = mocker.MagicMock()
+        context_manager_mock = mocker.MagicMock()
+        context_manager_mock.return_value.__enter__.return_value = open_mock
+        mocker.patch('palletjack.utils.Path.open', new=context_manager_mock)
+
+        out_path = palletjack.utils.save_feature_layer_to_json(mock_fl, 'foo')
+
+        test_json_string = '{"features": [{"geometry": {"spatialReference": {"wkid": 4326}, "x": 11, "y": 14}, "attributes": {"foo": 1, "x": 11, "y": 14, "OBJECTID": 1}}], "objectIdFieldName": "OBJECTID", "displayFieldName": "OBJECTID", "spatialReference": {"wkid": 4326}, "geometryType": "esriGeometryPoint", "fields": [{"name": "foo", "type": "esriFieldTypeInteger", "alias": "foo"}, {"name": "x", "type": "esriFieldTypeInteger", "alias": "x"}, {"name": "y", "type": "esriFieldTypeInteger", "alias": "y"}]}'
+
+        open_mock.write.assert_called_with(test_json_string)
+        assert out_path == Path('foo', f'flayer_{datetime.date.today()}.json')
+
+    def test_save_feature_layer_to_json_doesnt_save_empty_data(self, mocker):
+
+        mock_df = pd.DataFrame()
+
+        mock_fl = mocker.Mock()
+        mock_fl.query.return_value.sdf = mock_df
+        mock_fl.properties.name = 'flayer'
+        spatial_mock = mocker.patch.object(mock_df, 'spatial')
+
+        # open_mock = mocker.MagicMock()
+        # context_manager_mock = mocker.MagicMock()
+        # context_manager_mock.return_value.__enter__.return_value = open_mock
+        write_mock = mocker.patch('palletjack.utils.Path.write_text')
+
+        out_path = palletjack.utils.save_feature_layer_to_json(mock_fl, 'foo')
+
+        write_mock.write.assert_not_called()
+        spatial_mock.to_featureset.assert_not_called()
+
+        assert out_path == 'No data to save in feature layer flayer'
+
+
+class TestDataFrameChunking:
+
+    def test_ceildiv_always_returns_at_least_one(self):
+        assert palletjack.utils.Chunking._ceildiv(1, 2) == 1
+
+    def test_ceildiv_returns_ceiling(self):
+        assert palletjack.utils.Chunking._ceildiv(3, 2) == 2
+
+    def test_chunk_dataframe_properly_chunks(self, iris):
+
+        chunks = 4
+        chunk_size = palletjack.utils.Chunking._ceildiv(len(iris), chunks)
+        dfs = palletjack.utils.Chunking._chunk_dataframe(iris, chunk_size)
+
+        assert len(dfs) == chunks
+        assert [len(df) for df in dfs] == [38, 38, 38, 36]
+
+    def test_chunk_dataframe_properly_chunks_even_sized_chunks(self, iris):
+
+        chunks = 5
+        chunk_size = palletjack.utils.Chunking._ceildiv(len(iris), chunks)
+        dfs = palletjack.utils.Chunking._chunk_dataframe(iris, chunk_size)
+
+        assert len(dfs) == chunks
+        assert [len(df) for df in dfs] == [30] * chunks
+
+    def test_recursive_dataframe_chunking_recurses_on_first_chunk_too_large(self, mocker):
+        #: 5 rows, 2 chunks, first chunk gets 4 instead of three.
+        #: two total calls: once for 5 elements, once for 4. first call chunks to 4 and 1, next chunks 4 into 3 and 1
+        df = pd.DataFrame(['a', 'b', 'c', 'd', 'e'], columns=['foo'])
+        mocker.patch('palletjack.utils.pd.DataFrame.spatial.to_featureset', return_value=mocker.Mock())
+        mocker.patch('palletjack.utils.Chunking._ceildiv')
+        sys_mock = mocker.patch('palletjack.utils.sys.getsizeof', side_effect=['foo', 4, 'foo', 2, 2, 1])
+        mocker.patch(
+            'palletjack.utils.Chunking._chunk_dataframe',
+            side_effect=[
+                [df.iloc[:4], df.iloc[4:]],  #: first chunking gives 4 and 1
+                [df.iloc[:3], df.iloc[3:4]],  #: second chunking gives 3 and 1 (3 being the max size)
+            ]
+        )
+        df_list = palletjack.utils.Chunking._recursive_dataframe_chunking(df, 3)
+
+        tm.assert_frame_equal(df_list[0], df.iloc[:3])
+        tm.assert_frame_equal(df_list[1], df.iloc[3:4])
+        tm.assert_frame_equal(df_list[2], df.iloc[4:])
+        assert sys_mock.call_count == 6
+
+    def test_recursive_dataframe_chunking_recurses_on_second_chunk_too_large(self, mocker):
+        #: 5 rows, 2 chunks, first chunk gets 1 instead of three, second gets 4
+        #: two total calls: once for 5 elements, once for 4. first call chunks to 1 and 4, next chunks 4 into 3 and 1
+        df = pd.DataFrame(['a', 'b', 'c', 'd', 'e'], columns=['foo'])
+        mocker.patch('palletjack.utils.pd.DataFrame.spatial.to_featureset', return_value=mocker.Mock())
+        mocker.patch('palletjack.utils.Chunking._ceildiv')
+        sys_mock = mocker.patch('palletjack.utils.sys.getsizeof', side_effect=['foo', 1, 4, 'foo', 2, 2])
+        mocker.patch(
+            'palletjack.utils.Chunking._chunk_dataframe',
+            side_effect=[
+                [df.iloc[:1], df.iloc[1:]],  #: first chunking gives 1 and 4
+                [df.iloc[1:4], df.iloc[4:]],  #: second chunking gives 3 and 1 (3 being the max size)
+            ]
+        )
+
+        df_list = palletjack.utils.Chunking._recursive_dataframe_chunking(df, 3)
+
+        tm.assert_frame_equal(df_list[0], df.iloc[:1])
+        tm.assert_frame_equal(df_list[1], df.iloc[1:4])
+        tm.assert_frame_equal(df_list[2], df.iloc[4:])
+        assert sys_mock.call_count == 6
+
+    def test_recursive_dataframe_chunking_recurses_on_middle_chunk_too_large(self, mocker):
+        #: 5 rows, 3 chunks, first chunk gets 1, second gets 3, third gets 1
+        #: two total calls: once for 5 elements, once for 3.
+        df = pd.DataFrame(['a', 'b', 'c', 'd', 'e'], columns=['foo'])
+        mocker.patch('palletjack.utils.pd.DataFrame.spatial.to_featureset', return_value=mocker.Mock())
+        mocker.patch('palletjack.utils.Chunking._ceildiv')
+        sys_mock = mocker.patch('palletjack.utils.sys.getsizeof', side_effect=['foo', 1, 3, 'foo', 2, 1, 1])
+        mocker.patch(
+            'palletjack.utils.Chunking._chunk_dataframe',
+            side_effect=[
+                [df.iloc[:1], df.iloc[1:4], df.iloc[4:]],  #: first chunking gives 1, 3, 1
+                [df.iloc[1:3], df.iloc[3:4]],  #: second chunking gives 2 and 1 (2 being the max size)
+            ]
+        )
+
+        df_list = palletjack.utils.Chunking._recursive_dataframe_chunking(df, 2)
+
+        tm.assert_frame_equal(df_list[0], df.iloc[:1])
+        tm.assert_frame_equal(df_list[1], df.iloc[1:3])
+        tm.assert_frame_equal(df_list[2], df.iloc[3:4])
+        tm.assert_frame_equal(df_list[3], df.iloc[4:])
+        assert sys_mock.call_count == 7
+
+    def test_recursive_dataframe_chunking_doesnt_recurse_when_not_needed(self, mocker):
+        #: 5 rows, 2 chunks, first chunk gets 3, second gets 2
+        #: only one call: first breaks into 3 and 2
+        df = pd.DataFrame(['a', 'b', 'c', 'd', 'e'], columns=['foo'])
+        mocker.patch('palletjack.utils.pd.DataFrame.spatial.to_featureset', return_value=mocker.Mock())
+        mocker.patch('palletjack.utils.Chunking._ceildiv')
+        sys_mock = mocker.patch('palletjack.utils.sys.getsizeof', side_effect=['foo', 3, 2])
+        mocker.patch(
+            'palletjack.utils.Chunking._chunk_dataframe',
+            side_effect=[
+                [df.iloc[:3], df.iloc[3:]],  #: first chunking gives 3, 2
+            ]
+        )
+
+        df_list = palletjack.utils.Chunking._recursive_dataframe_chunking(df, 3)
+
+        tm.assert_frame_equal(df_list[0], df.iloc[:3])
+        tm.assert_frame_equal(df_list[1], df.iloc[3:])
+        assert sys_mock.call_count == 3
+
+    def test_recursive_dataframe_chunking_doesnt_chunk_when_not_needed(self, mocker):
+        #: 5 rows, 2 chunks, first chunk gets 3, second gets 2
+        #: only one call: first breaks into 3 and 2
+        df = pd.DataFrame(['a', 'b', 'c', 'd', 'e'], columns=['foo'])
+        mocker.patch('palletjack.utils.pd.DataFrame.spatial.to_featureset', return_value=mocker.Mock())
+        mocker.patch('palletjack.utils.Chunking._ceildiv')
+        sys_mock = mocker.patch('palletjack.utils.sys.getsizeof', side_effect=['foo', 5])
+        chunking_mock = mocker.patch(
+            'palletjack.utils.Chunking._chunk_dataframe',
+            side_effect=[
+                [df],  #: no chunk needed
+            ]
+        )
+
+        df_list = palletjack.utils.Chunking._recursive_dataframe_chunking(df, 5)
+
+        tm.assert_frame_equal(df_list[0], df)
+        chunking_mock.assert_called_once()
+        assert sys_mock.call_count == 2
+
+    def test_build_upload_json_calls_null_string_fixer_appropriate_number_of_times(self, mocker):
+
+        mock_df = mocker.Mock()
+        mock_string_fixer = mocker.patch('palletjack.utils.fix_numeric_empty_strings')
+        mock_string_fixer.return_value.to_geojson = 'new_json'
+        mocker.patch('palletjack.utils.sys.getsizeof')
+        mocker.patch('palletjack.utils.Chunking._recursive_dataframe_chunking', return_value=[mocker.Mock()] * 5)
+
+        json_list = palletjack.utils.Chunking.build_upload_json(mock_df, 'foo_dict')
+
+        assert len(json_list) == 5
+        assert json_list == ['new_json'] * 5
+        assert mock_string_fixer.call_count == 5