Repository: mapillary/mapillary_tools (language: python)

Issue:
process_and_upload: import_path: support read-only file system
### Basic information
* Release version: `0.1.2`
* System: `Mac OS X`
* Capture Device: `Action Camera` | `Sony 200V`

### Steps to reproduce behavior

 1. `$ mapillary_tools process_and_upload --import_path /Volumes/Untitled/DCIM --user_name "xyz" --verbose --rerun --flag_duplicates --duplicate_distance 0.5 --duplicate_angle 1 --keep_original --advanced`
 
### Expected behavior

No error, just do the magic.

### Actual behavior

```
Traceback (most recent call last):
  File "/usr/local/bin/mapillary_tools", line 60, in <module>
    command.run(args)
  File "/usr/local/lib/python2.7/site-packages/mapillary_tools/commands/process_and_upload.py", line 91, in run
    process_user_properties(**({k: v for k, v in vars_args.iteritems()
  File "/usr/local/lib/python2.7/site-packages/mapillary_tools/process_user_properties.py", line 70, in process_user_properties
    user_properties)
  File "/usr/local/lib/python2.7/site-packages/mapillary_tools/processing.py", line 624, in create_and_log_process_in_list
    verbose)
  File "/usr/local/lib/python2.7/site-packages/mapillary_tools/processing.py", line 632, in create_and_log_process
    os.makedirs(log_root)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 150, in makedirs
    makedirs(head, mode)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 150, in makedirs
    makedirs(head, mode)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 30] Read-only file system: '/Volumes/Untitled/DCIM/100MSDCF/.mapillary'
```

### Additional information

Sony AS200V only mounts as read only file system.
Don't write anything in any location of the import_path.
Please, use a default temporary location for writing stuff.

### Mapillary upload also fails
```
$ mapillary_tools upload --import_path /Volumes/Untitled/DCIM --verbose --advanced            
No images to upload.
Please check if all images contain the required Mapillary metadata. If not, you can use "mapillary_tools process" to add them
```

PS#1 For `--verbose` there is very little output in the upload command
PS#2  After already having uploaded 253.9 K images  through the web interface, it would be nice that the command line tools finally start working.
Feature Request: Use only 1 file to log the image status
### Basic information
* Release version: `0.1.2`

Currently, the mapillary_tools log the image's status with one file for each step.
I have a folder with about 20 000 images and after the mapillary_tools processes, I have more than 500 000 files and 20 000 folders. It's a ressource waste, and moving/copying these images become very slow.

Could you use only one log file per image ? You could use a file with a dict inside, or use something like the ConfigParser module.
improvements to processing times
### Basic information
* Release version: master
* System: `Linux`

I'd like to see some improvements in the processing time if possible, at the moment `mapillary_tools process_and_upload` spends quite a while running through:

```
Running user_process for 2612 images, skipping 0 images.
Running import_meta_data_process for 2612 images, skipping 0 images.
Running geotag_process for 2612 images, skipping 0 images.
Running sequence_process for 2612 images, skipping 0 images.
Running upload_params_process for 2612 images, skipping 0 images.
Running mapillary_image_description for 2612 images, skipping 0 images.
```
Please run --interpolate_directions AFTER detecting duplicates
### Basic information
* Release version: `0.5.3`
* System: Android 10, running commands on Termux 
* Capture Device: Samsung A515g

### Steps to reproduce behavior

 1. Capture images in a sequence where you're stopped in place at some point, continuing to capture images. 
 2. Run a command meant to weed out duplicates, and interpolate directions. I use pretty harsh duplicate detection, since my phone's compass is terrible, and my GPS wanders anywhere in a 1m range while I'm stopped, given enough time. I use: 

mapillary_tools process --import_path storage/dcim/Trip1 --cutoff_time 30 --duplicate_distance 1.5 --duplicate_angle 360 --interpolate_directions --offset_angle 0 --overwrite_EXIF_direction_tag --user_name jesseakaraccoon --advanced

### Expected behavior

I expect a sequence with no duplicates, each image pointing at the next image in the sequence.

### Actual behavior

When I check the uploaded sequence, I notice that some images at intersections point in very odd directions. 

I suspect that this happens because image directions are interpolated first (causing images at stops to point in whatever directions the GPS drifted to while stopped), THEN duplicates are removed.

I suggest somehow interpolating directions AFTER weeding out duplicates.

### Corresponding data

Sorry, I only thought to submit this feature request AFTER editing sequences where this problem occurred. I don't have a screenshot / image key. I'll try to re-create this problem later. But I believe this is a known problem with --interpolate_directions within the community. 
0.5.0 probably cosmetic - logging 
When running mapillary_tools video_process (ie no upload)

"Video /media/user/MapDisk2/Recorda/20190327_171915_NF.mp4 has already been uploaded, contact support@mapillary for help with reuploading it if neccessary."
(multiple lines)

It would seem an alert about upload should not be in process only.,

In my case I did in fact upload a series of images (not movies) with that prefix, then deleted through the web GUI. I have not yet attempted to upload the recent process.





change python exif library
Using a [fork](https://github.com/mapillary/Piexif) as a dependency is an unfortunate choice since it prevent distros from packaging
Both the fork and the [original library](https://github.com/hMatoba/Piexif) seems to be unmaintained (2018-04-25 and 2019-12-27 as last commit)
Please consider switching to a more maintained library such as [pyexiv2](https://github.com/LeoHsiao1/pyexiv2)


Discussion / hints:
hi, sorry for the late reply.
We are planning to improve the logging procedure in the next releases. Unfortunately it is not so simple, as we need to make sure we keep exact logs of the upload state for each image, otherwise we would quickly end up with many duplicate uploads. Currently we are handling this by creating log files where ever the image is located, since otherwise, in case of using a default temporary location, we would have to be relying that the absolute path doesnt change at all, which in our experience does not work out well.
Nevertheless, we do have plans to improve the logging and the solution will work for read only file systems as well. 
We will keep you informed about the progress and also improve the output of `--verbose` in the upload command.

on linux `~/.config/mapillary_tools` or `~/.mapillary_tools/` would work. It's not so temporary so would survive machine restarts and avoids writing to the sdcard or other device which the images are stored on.
@andrewharvey I don't understand what you are trying to say with your comment.
Hi sorry for late reply, I m on vacation until 10th August with limited access to internet. We are aware of the issues and inconveniences with the current logging procedure and are planning to improve it in the next releases. We will keep you updated, thank you for the suggestions.
Any news on this?
I also end up with tons of empty/small files which slows backups.
@fgouget this is still in our priorities, we are planning a big improvement on logging but need time to figure out the best way forward
Waiting impatiently for logging improvements. Indeed, tons of 0-100 bytes log files slows everything down...
I actually find the scattered current logging system quite useful as is. (eg using find under Linux) I am however flexible!
Hello, 
Just to know if there is any update on this issue ? 
If there was just an option it could be useful too : "not create log file" or "delete log file post process". 
Waiting impatiently 
Please make sure that any new system is backward compatible too. I regularly have a few months between process and upload and don't want to repeat processing of hundreds and thousands of images. (again)
This is worse when running an upload directly from the SDCARD (30mins), but can still take a couple of minutes on an SSD.
hi, yes we will work on improving the processing time, it is not using threading as upload directly, so we likely start with that. Any suggestions are welcome
The `mapillary_image_description` stage takes maybe 10 times the time as the other stages, I'll try to run some benchmarking...
yes, we are noticing the same, it is due to writing  data into the image,  so we are considering ways to improve or omit that completely, you are welcome to contribute
I also just realized running --interpolate_directions before detecting duplicates causes another problem. 

I tried to change the --duplicate_angle parameter from 360 to 20, so I can try keeping multiple images from tight turns turns. Even though my compass can be pretty inaccurate (thus, why I use --interpolate_directions), it's accurate enough to pick up a turn.

However, because --interpolate_directions precedes duplicate detection, this means that first, the EXIF compass data for my images is overwritten. At places where my car stops, images are assigned random compass angles, as my GPS wanders around in random directions by tiny amounts. Then, since most of the random angles are greater than 20°, **almost no duplicates are removed.** 

You basically *must* use "--duplicate_angle 360" if you want to remove duplicates and --interpolate_directions in one command. 

Seen with this sequence at every stop, including the first few images: https://www.mapillary.com/app/?lat=40.598896836111116&lng=-124.15212985000001&z=17&tab=uploads&pKey=iTUabNjP2ZEfA4oysQ63rh
I dig this issue -- a very annoying one. 

I believe that in `mapillary_tools/process_sequence_properties.py:189` the processes of deduplication by distance should run **before** the `COMPUTE DIRECTIONS` part and the deduplication by direction - where it sits now. Looks like a simple fix to me.
Additional. Same log error also happens for a prefix I have never uploaded..

We need add more tests to make sure we don't break things, and then evaluate the library. Thanks for the information!

Fix patch:
diff --git a/CHANGELOG.MD b/CHANGELOG.MD
index 13af3ba5..a86f3e6e 100644
--- a/CHANGELOG.MD
+++ b/CHANGELOG.MD
@@ -220,3 +220,38 @@ Improvement:
 Fixes:
 - Fix organization description missing from the API #429
 - Skip BlackVue videos that have no GPS data found #430
+
+## 0.8.0 (2021-10-07)
+
+## Features
+- Immutable process and upload: do not modify images 0430c711f06678380ccd15f6b539c16d02eeb721 #276
+- The command `process` writes all metadata a single file (image description file) instead of multiple files 081e6f6daaa5827fc148d2f48062b56ab8ca732b #269
+- The command `upload` supports upload given a image folder and a image description file (for advanced usages)
+
+## Improvement
+- Simplify internal image states (now an image has only 3 states processed, not_processed, failed) a859732b69b76d3b9ba9c96bfc4287c9e738687c
+- Simplify command line UI (which introduces some breaking changes)
+- Improve the processing performance -- less image EXIF reading/writing #251
+- Improve sequence processing a859732b69b76d3b9ba9c96bfc4287c9e738687c #383 #395
+- More unit tests and integration tests 5263409dc6d12ef89e89f09e13cceaf7c52df244 f1d488b708a06e915a0cba48d110c7b36db28ec8
+- Use the logging module to log messages/events 5263409dc6d12ef89e89f09e13cceaf7c52df244 #342
+- Update/Improve README.md a728461bc4c6a1090633bc0ffcab5e9286dde793
+- Upgrade `piexif==1.1.3` and ` exifread==2.3.2` to latest a728461bc4c6a1090633bc0ffcab5e9286dde793 #398
+
+## Breaking changes
+### UI changes
+- Remove `--import_path`: Instead of `process --import_path path/to/images`, use `process path/to/images`
+- Remove `--advanced`: Not needed
+- Remove `--rerun`: always rerun (re-process) due to performance improvement and less states c3886884166df22a6a806ddb01b9f4ae9164412b
+
+### Other changes
+- Remove command `post_process`: No need because all information are written in the description file 60ee64749a97ec71b65c7f832f2f69ba967812da
+- Remove command `interpolate`
+- Remove command `process_csv`: Will add back as a geotag source
+- Remove all `extract_*` commands: No need because all information are written in the description file
+
+## Other fixes:
+- Fix missing GPS reference 60ee64749a97ec71b65c7f832f2f69ba967812da
+- https://github.com/mapillary/mapillary_tools/issues/433
+- https://github.com/mapillary/mapillary_tools/issues/437
+- https://github.com/mapillary/mapillary_tools/issues/338
diff --git a/MANIFEST.in b/MANIFEST.in
new file mode 100644
index 00000000..0f47326e
--- /dev/null
+++ b/MANIFEST.in
@@ -0,0 +1,1 @@
+include schema/*.json
diff --git a/README.md b/README.md
index c33ff0f9..a5615d4d 100644
--- a/README.md
+++ b/README.md
@@ -9,9 +9,14 @@ Mapillary Tools is a library for processing and uploading images to [Mapillary](
 * [Installation](#installation)
 * [Video Support](#video-support)
 * [Usage](#usage)
+    - [Process](#process)
+    - [Upload](#upload)
+    - [Video Process](#video-process)
+    - [Aliases](#aliases)
+    - [Authenticate](#authenticate)
 * [Advanced Usage](#advanced-usage)
-* [Command Specifications](#command-specifications)
-* [Camera Specific](#camera-specific)
+    - [Image Description](#image-description)
+    - [Zip Images](#zip-images)
 * [Troubleshooting](#troubleshooting)
 * [Development](#development)
 
@@ -19,14 +24,15 @@ Mapillary Tools is a library for processing and uploading images to [Mapillary](
 
 ## Quickstart
 
-Download the latest `mapillary_tools` binaries for your platform here: https://github.com/mapillary/mapillary_tools/releases/tag/v0.7.4
+Download the latest `mapillary_tools` binaries for your platform
+here: https://github.com/mapillary/mapillary_tools/releases/tag/v0.8.0
 
 See [more installation instructions](#installation) below.
 
 Upload imagery:
 
-```bash
-./mapillary_tools process_and_upload --import_path "path/to/images" --user_name "mapillary_username"
+```shell
+mapillary_tools process_and_upload "path/to/images/"
 ```
 
 ## Requirements
@@ -35,9 +41,7 @@ Upload imagery:
 
 To upload images to Mapillary, an account is required and can be created [here](https://www.mapillary.com/signup). When
 using the tools for the first time, user authentication is required. You will be prompted to enter your account
-credentials. Only Mapillary account credentials (email and password) are valid for authentication
-in `mapillary_tools`. Other Mapillary login options (Facebook, OpenStreetMap, Google+, or ArcGIS) are not supported
-with `mapillary_tools`.
+credentials.
 
 ### Metadata
 
@@ -50,17 +54,19 @@ information [here](https://help.mapillary.com/hc/en-us/articles/115001717829-Geo
 
 Python (3.6 and above) and git are required:
 
-```bash
+```shell
 python3 -m pip install --upgrade git+https://github.com/mapillary/mapillary_tools
 ```
 
-If you see "Permission Denied" error, try run the command above with `sudo`, or install it in your local [virtualenv](#development) (recommended).
+If you see "Permission Denied" error, try to run the command above with `sudo`, or install it in your
+local [virtualenv](#development) (recommended).
 
 ### Installing on Android Devices
 
-A command line program such as Termux is required. Installation can be done without root privileges.  The following commands will install Python 3, pip3, git, and all required libraries for mapillary_tools on Termux:
+A command line program such as Termux is required. Installation can be done without root privileges. The following
+commands will install Python 3, pip3, git, and all required libraries for mapillary_tools on Termux:
 
-```bash
+```shell
 pkg install python git build-essential libgeos openssl libjpeg-turbo
 python3 -m pip install --upgrade pip wheel
 python3 -m pip install --upgrade git+https://github.com/mapillary/mapillary_tools
@@ -68,689 +74,331 @@ python3 -m pip install --upgrade git+https://github.com/mapillary/mapillary_tool
 
 Termux must access the device's internal storage to process and upload images. To do this, use the following command:
 
-```bash
+```shell
 termux-setup-storage
 ```
 
-Finally, on devices running Android 11, using a command line program, mapillary_tools will process images very slowly if they are in shared internal storage during processing. It is advisable to first move images to the command line program’s native directory before running mapillary_tools.
-For an example using Termux, if imagery is stored in the folder “Internal storage/DCIM/mapillaryimages" the following command will move that folder from shared storage to Termux:
+Finally, on devices running Android 11, using a command line program, mapillary_tools will process images very slowly if
+they are in shared internal storage during processing. It is advisable to first move images to the command line
+program’s native directory before running mapillary_tools. For an example using Termux, if imagery is stored in the
+folder `Internal storage/DCIM/mapillaryimages` the following command will move that folder from shared storage to
+Termux:
 
-```bash
+```shell
 mv -v storage/dcim/mapillaryimages mapillaryimages
 ```
 
 ## Video Support
 
-To sample images from videos, you will also need to install `ffmpeg`.
+To [process videos](#video-process), you will also need to install `ffmpeg`.
 
-### Windows
+You can download `ffmpeg` from [here](https://ffmpeg.org/download.html). Make sure it is executable and put the
+downloaded binaries in your `$PATH`. You can also install `ffmpeg` with your favourite package manager in your system.
+For example:
 
-To install `ffmpeg` on Windows, follow
-these [instructions](http://adaptivesamples.com/how-to-install-ffmpeg-on-windows/).
+On macOS, use [`Homebrew`](https://brew.sh/):
 
-### macOS
-
-To install `ffmpeg` on macOS use [Homebrew](https://brew.sh). Once you have Homebrew installed, you can install `ffmpeg`
-by running:
-
-```bash
+```shell
 brew install ffmpeg
 ```
 
-### Ubuntu
+On Debian/Ubuntu:
 
-To install `ffmpeg` on Ubuntu:
-
-```bash
+```shell
 sudo apt install ffmpeg
 ```
 
 ## Usage
 
-All commands are executed with `mapillary_tools`.
-
-### Available commands
-
-To see the available commands, use the following in the command line (for Windows, adjust the command according the
-instructions for execution):
-
-```bash
-mapillary_tools -h
-```
-
-Executable `mapillary_tools` takes the following arguments:
-
-`-h, --help`: Show help and exit
-
-`--advanced`: Use the tools under an advanced level, with additional arguments and commands available
-
------
-
-`command`: Use one of the available commands:
-
-- `process`: Process the images including for instance, geotagging and sequence arrangement
-- `upload`: Upload images to Mapillary
-- `process_and_upload`: A bundled command for `process` and `upload`
-
------
-
-See the command specific help for required and optional arguments:
-
-- Show help for `process` command:
+### Process
 
-```bash
-mapillary_tools process -h
-```
-
------
-
-- Show advanced help for `process` command:
-
-```bash
-mapillary_tools process -h --advanced
-```
-
-### Examples
-
-For Windows, adjust the commands according the instructions for execution.
-
-#### Process Images
+The `process` command geotags images in the given directory. It extracts the required and optional metadata from image
+EXIF (or the other supported geotag sources), and writes all the metadata (or process errors) in
+an [image description](#image-description) file, which will be read during [upload](#upload).
 
-The command below processes all images in the directory and its sub-directories. It will update the images with
-Mapillary-specific metadata in the image EXIF for the user with user name `mapillary_user`. It requires that each image
-in the directory contains `capture time` and `GPS`. By default, only the Image Description EXIF tag is overwritten and
-duplicate images are flagged to be excluded from upload using default thresholds for duplicate distance 0.1 m and
-duplicate angle 5°.
-
-```bash
-mapillary_tools process --import_path "path/to/images" --user_name "mapillary_username"
-```
-
-#### Upload Images
-
-The command below uploads all images in a directory and its sub-directories. It requires Mapillary-specific metadata in
-the image EXIF. It works for images that are captured with Mapillary iOS or Android apps or processed with the `process`
-command.
-
- ```bash
-mapillary_tools upload --import_path "path/to/images"
-```
-
-#### Process and Upload Images
+#### Examples
 
-The command below runs `process` and `upload` consecutively for a directory.
+Process all images in the directory `path/to/images/` (and its sub-directories):
 
-```bash
-mapillary_tools process_and_upload --import_path "path/to/images" --user_name "mapillary_username"
+```shell
+mapillary_tools process "path/to/images/"
 ```
 
-## Advanced Usage
+Interpolate images in the directory `path/to/images/` on the GPX track read from `path/to/gpx_file.gpx`. The images are
+required to contain capture time in order to sort the images and interpolate them.
 
-Available commands for advanced usage:
-
-- Video Specific Commands:
-    - sample_video
-    - video_process
-    - video_process_and_upload
-- Process Unit Commands:
-    - extract_user_data
-    - extract_import_meta_data
-    - extract_geotag_data
-    - extract_sequence_data
-    - extract_upload_params
-    - exif_insert
-- Other Commands:
-    - process_csv
-    - interpolate
-    - authenticate
-    - post_process
-
-### Geotag and Upload
-
-- Run process and upload consecutively, while process is reading geotag data from a gpx track. It requires
-  that `capture time` information is embedded in the image EXIF. By default geotag data is stored only in the mapillary
-  image description, in the EXIF Image Description tag. If you would like the rest of the tags to be overwritten as
-  well, for example to be able to place images on the map for testing purposes, you should pass an additional
-  argument `--overwrite_all_EXIF_tags` to overwrite all EXIF tags, or in case you only want to overwrite a specific tag,
-  like for example the GPS tag, pass argument `--overwrite_EXIF_gps_tag`.
-
- ```bash
-mapillary_tools process --advanced --import_path "path/to/images" \
-    --user_name "mapillary_username" \
+```shell
+mapillary_tools process "path/to/images/" \
     --geotag_source "gpx" \
-    --geotag_source_path "path/to/gpx_file"
-
-mapillary_tools upload --import_path "path/to/images"
+    --geotag_source_path "path/to/gpx_file.gpx"
 ```
 
-or
+Process all images in the directory, specifying an angle offset of 90° for the camera direction and splitting images
+into sequences of images apart by less than 100 meters according to image `GPS` and less than 120 seconds according to
+image capture time.
 
- ```bash
-mapillary_tools process_and_upload --advanced --import_path "path/to/images" \
-    --user_name "mapillary_username" \
-    --geotag_source "gpx" \
-    --geotag_source_path "path/to/gpx_file"
+```shell
+mapillary_tools process "path/to/images/" \
+    --offset_angle 90 \
+    --cutoff_distance 100 \
+    --cutoff_time 120
 ```
 
-### Keep original images intact and Upload
-
-- To prevent data loss or control versions, the original images can be left intact by specifying the
-  flag `--keep_original`. This will result in the edited image being saved in a copy of the original image, instead of
-  the original image itself. Copies are saved in `{$import_path/$image_path/}.mapillary/process_images}` and are deleted
-  at the start of every processing run.
+### Upload
 
-```bash
-mapillary_tools process --advanced --import_path "path/to/images" \
-    --user_name "mapillary_username" \
-    --keep_original
+Images that have been successfully processed can be uploaded with the `upload` command.
 
-mapillary_tools upload --import_path "path/to/images"
-```
+#### Examples
 
-or
+Upload all processed images in the directory `path/to/images/` to user `mly_user` for organization `mly_organization_id`
+. It is optional to specify `--user_name` if you have only one user [authenticated](#authenticate).
 
- ```bash
-mapillary_tools process_and_upload --advanced --import_path "path/to/images" \
-    --user_name "mapillary_username" \
-    --keep_original
+```shell
+mapillary_tools upload "path/to/images/" --user_name "mly_user" --organization_id "mly_organization_id"
 ```
 
-### Derive image direction and Upload
+### Video Process
 
-- Derive image direction (image heading or camera angle) based on image latitude and longitude. If images are missing
-  direction, the direction is derived automatically, if direction is present, it will be derived and overwritten only if
-  the flag `--interpolate directions` is specified.
+Video process involves two commands:
 
- ```bash
-mapillary_tools process --advanced --import_path "path/to/images" \
-    --user_name "mapillary_username" \
-    --interpolate_directions
-
-mapillary_tools upload --import_path "path/to/images"
-```
-
-or
+1. `sample_video`: sample videos into images, and insert capture times to the image EXIF. Capture time is calculated
+   based on the video start time and sampling interval. This is where `ffmpeg` is being used.
+2. `process`: process (geotag) the sample images with the specified source
 
- ```bash
-mapillary_tools process_and_upload --advanced --import_path "path/to/images" \
-    --user_name "mapillary_username" \
-    --interpolate_directions
-```
+The two commands are usually combined into a single command `video_process`. See the examples below.
 
-### Video Sampling and Upload
+#### Examples
 
-- Sample the video(s) located in `path/to/videos` into the directory `path/to/images`, at a sample interval of 0.5
-  seconds and tag the sampled images with `capture time`. Note that the video frames will always be sampled into sub
-  directory `.mapillary/sampled_video_frames/"video_import_path"`, whether import path is specified or not. In
-  case `import_path` is specified the final path for the sampled video frames will
-  be `"import path"/.mapillary/sampled_video_frames/"video_import_path"` and in case `import_path` is not specified, the
-  final path for the sampled video frames will be `path/to/.mapillary/sampled_video_frames/"video_import_path"`.
+Sample the videos located in `path/to/videos/` at the default sampling rate 2 seconds, i.e. one video frame every two
+seconds. Video frames will be sampled into a sub-directory `path/to/videos/mapillary_sampled_video_frames`.
 
- ```bash
-mapillary_tools sample_video --import_path "path/to/images" \
-    --video_import_path "path/to/videos" \
-    --video_sample_interval 0.5 --advanced 
+```shell
+mapillary_tools sample_video "path/to/videos/"
 ```
 
-- Sample the video(s) located in `path/to/videos`, at a sample interval of 2 seconds (default value) and tag the
-  resulting images with `capture time`. And then process and upload the resulting images for
-  user `username_at_mapillary`, specifying a gpx track to be the source of geotag data. Additionally pass
-  the `--overwrite_all_EXIF_tags` so the extracted frames have all the tags set beside the Image Description tag.
-
-```bash
-mapillary_tools sample_video --video_import_path "path/to/videos" --advanced
+Sample the videos located in `path/to/videos/` to directory `path/to/sample_images/` at a sampling rate 0.5 seconds,
+i.e. two video frames every second.
 
-mapillary_tools process --advanced --import_path "path/to/.mapillary/sampled_video_frames/video_import_path" \
-    --user_name "mapillary_username" \
-    --geotag_source "gpx" \
-    --geotag_source_path "path/to/gpx_file" \
-    --overwrite_all_EXIF_tags
-
-mapillary_tools upload --import_path "path/to/.mapillary/sampled_video_frames/video_import_path"
+```shell
+mapillary_tools sample_video "path/to/videos/" "path/to/sample_images/" \
+    --video_sample_interval 0.5
 ```
 
-or
+Sample the videos located in `path/to/videos/` to the directory `path/to/sample_images/` at the default sampling rate 1
+second, i.e. one video frame every second, geotagging data from a gpx track stored in `path/to/gpx_file.gpx` video,
+assuming video start time can be extracted from the video file and deriving camera direction based on `GPS`.
 
-```bash
-mapillary_tools video_process_and_upload --video_import_path "path/to/videos" \
-    --user_name "mapillary_username" \
-    --advanced --geotag_source "gpx" \
-    --geotag_source_path "path/to/gpx_file" \
-    --overwrite_all_EXIF_tags
+```shell
+mapillary_tools video_process "path/to/videos/" "path/to/sample_images/" \
+    --geotag_source "gpx" \
+    --geotag_source_path "path/to/gpx_file.gpx" \
+    --video_sample_interval 1 \
+    --interpolate_directions
 ```
 
-### Process csv
-
-- Insert image capture time and gps data from a csv file, based on filename:
+Sample GoPro videos in directory `path/to/videos/` into import path `path/to/sample_images/` at a sampling rate 0.5
+seconds, i.e. two frames every second, reading geotag data from the GoPro videos in `path/to/videos/`.
 
-```bash
-mapillary_tools process_csv --import_path "path/to/images" \
-    --csv_path "path/to/csv_file" \
-    --filename_column 1 \
-    --timestamp_column 4 \
-    --latitude_column 2 \
-    --longitude_column 3 \
-    --advanced
+```shell
+mapillary_tools video_process "path/to/videos/" "path/to/sample_images/" \
+    --geotag_source "gopro_videos" \
+    --interpolate_directions \
+    --video_sample_interval 0.5
 ```
 
-- Insert image capture time and meta data from a csv file based on the order of image file names (in case filename
-  column is missing):
+Sample BlackVue videos in directory `path/to/videos/` at a sampling rate 0.2 seconds, i.e. 5 frames every second and
+process resulting video frames for user `mapillary_user`, reading geotag data from the BlackVue videos
+in `path/to/videos/` and specifying camera make and model.
 
-```bash
-mapillary_tools process_csv --import_path "path/to/images" \
-    --csv_path "path/to/csv_file" \
-    --timestamp_column 1 \
-    --meta_columns "6,7" \
-    --meta_names "random_name1,random_name2" \
-    --meta_types "double,string" \
-    --advanced
+```shell
+mapillary_tools video_process "path/to/videos/" \
+    --geotag_source "blackvue_videos" \
+    --device_make "Blackvue" \
+    --device_model "DR900S-2CH"
 ```
 
-## Command Specifications
-
-### `process`
-
-The `process` command will format the required and optional meta data into a Mapillary image description and insert it
-in the image EXIF Image Description tag. Images are required to contain image capture time, latitude, longitude and
-camera direction in the image EXIF. Under advanced usage, additional functionalities are available, for example latitude
-and longitude can be read from a gpx track file or a GoPro video, camera direction can be derived based on latitude and
-longitude, duplicates can be kept instead of excluded from the upload etc. See the command specific help for required
-and optional arguments, add `--advanced` to see additional advanced optional arguments.
+### Aliases
 
-#### Examples
-
-- process all images for user `mapillary_user`, in the directory `path/to/images` and its sub-directories:
-
-```bash
-mapillary_tools process --import_path "path/to/images" \
-    --user_name "mapillary_username"
-```
+#### `process_and_upload`
 
-- process all images for user `mapillary_user`, in the directory `path/to/images`, skipping the images in its
-  sub-directories, rerunning process for all images that were not already uploaded and printing out extra warnings or
-  errors.
+`process_and_upload` command will run `process` and `upload` commands consecutively with combined required and optional
+arguments. It is equivalent to:
 
-```bash
-mapillary_tools process --import_path "path/to/images" \
-    --user_name "mapillary_username" \
-    --verbose \
-    --rerun \
-    --skip_subfolders
+```shell
+mapillary_tools process "path/to/images/"
+mapillary_tools upload  "path/to/images/"
 ```
 
-#### Advanced Examples
+#### `video_process`
 
-- Process all images for user `mapillary_user`, in the directory `path/to/images` and its sub-directories, reading
-  geotag data from a gpx track stored in file `path/to/gpx_file`, specifying an offset of 2 seconds between the camera
-  and gps device, ie, camera is 2 seconds ahead of the gps device and specifying to keep duplicates to be uploaded
-  instead of flagging images as duplicates in case they are apart by equal or less then the default 0.1 m and differ by
-  the camera angle by equal or less than the default 5°. Additionally pass the `--overwrite_EXIF_gps_tag` to overwrite
-  values with the values obtained from the gpx track.
+`video_process` command will run `sample_video` and `process` commands consecutively with combined required and optional
+arguments. It is equivalent to:
 
-```bash
-mapillary_tools process --import_path "path/to/images" \
-    --user_name "mapillary_username" \
-    --advanced \
-    --geotag_source "gpx" \
-    --geotag_source_path "path/to/gpx_file" \
-    --offset_time 2 \
-    --keep_duplicates \
-    --overwrite_EXIF_gps_tag
+```shell
+mapillary_tools sample_video "path/to/videos/" "path/to/images/"
+mapillary_tools upload "path/to/images/"
 ```
 
-- Process all images for user `mapillary_user`, in the directory `path/to/images` and its sub-directories, specifying
-  the import to be private imagery belonging to an organization with organization username `mapillary_organization`. You
-  can find the organization username in your dashboard.
-
-```bash
-mapillary_tools process --import_path "path/to/images" \
-    --user_name "mapillary_username" \
-    --advanced \
-    --private \
-    --organization_username "mapillary_organization"
-```
+#### `video_process_and_upload`
 
-- Process all images for user `mapillary_user`, in the directory `path/to/images` and its sub-directories, specifying an
-  angle offset of 90° for the camera direction and splitting images into sequences of images apart by less than 100
-  meters according to image `GPS` and less than 120 seconds according to image `capture time`. Additionally pass
-  the `--overwrite_EXIF_direction_tag` to overwrite values with the additional specified offset.
+`video_process_and_upload` command will run `sample_video` and `process_and_upload` commands consecutively with combined
+required and optional arguments. It is equivalent to:
 
-```bash
-mapillary_tools process --import_path "path/to/images" \
-    --user_name "mapillary_username" \
-    --advanced \
-    --offset_angle 90 \
-    --cutoff_distance 100 \
-    --cutoff_time 120 \
-    --overwrite_EXIF_direction_tag
+```shell
+mapillary_tools sample_video "path/to/videos/" "path/to/videos/mapillary_sampled_video_frames/"
+mapillary_tools process_and_upload "path/to/videos/mapillary_sampled_video_frames/"
 ```
 
-### `upload`
-
-Images that have been successfully processed or were taken with the Mapillary app will contain the required Mapillary
-image description embedded in the image EXIF and can be uploaded with the `upload` command.
-
-The `upload` command will collect all the images in the import path, while checking for duplicate flags, processing and
-uploading logs. If image is flagged as duplicate, was logged with failed process or logged as successfully uploaded, it
-will not be added to the upload list.
+### Authenticate
 
-By default, 5 threads upload in parallel and the script retries 50 times upon encountering a failure. These can be
-customized by specifying additional arguments `--number_threads` and `--max_attempts` under `--advanced` usage or with
-environment variables in the command line:
-
-    NUMBER_THREADS=10
-    MAX_ATTEMPTS=100
+The command `authenticate` will update the user credentials stored in the config file.
 
 #### Examples
 
-- upload all images in the directory `path/to/images` and its sub directories:
+Authenticate new user:
 
-```bash
-mapillary_tools upload --import_path "path/to/images"
+```shell
+mapillary_tools authenticate
 ```
 
-- upload all images in the directory `path/to/images`, while skipping its sub directories and specifying to upload with
-  10 threads and 10 maximum attempts:
+Authenticate for user `mly_user`. If the user is already authenticated, it will update the credentials in the config:
 
-```bash
-mapillary_tools upload --import_path "path/to/images" \
-    --skip_subfolders \
-    --number_threads 10 \
-    --max_attempts 10 \
-    --advanced
+```shell
+mapillary_tools authenticate --user_name "mly_user"
 ```
 
-### `process_and_upload`
+## Advanced Usage
 
-`process_and_upload` command will run `process` and `upload` commands consecutively with combined required and optional
-arguments.
+### Image Description
+
+As the output, the `procss` command generates `mapillary_image_description.json` under the image directory by default.
+The file contains an array of objects, each of which records the metadata of one image in the image directory. The
+metadata is validated by [the image description schema](https://github.com/mapillary/mapillary_tools/tree/master/schema/image_description_schema.json).
+Here is a minimal example:
+
+```json
+[
+  {
+    "MAPLatitude": 58.5927694,
+    "MAPLongitude": 16.1840944,
+    "MAPCaptureTime": "2021_02_13_13_24_41_140",
+    "filename": "IMG_0291.jpg"
+  },
+  {
+    "error": {
+      "type": "MapillaryGeoTaggingError",
+      "message": "Unable to extract GPS Longitude or GPS Latitude from the image"
+    },
+    "filename": "IMG_0292.jpg"
+  }
+]
+```
+
+The `upload` command then takes the image description file as the input, [zip images](#zip-images) with the specified
+metadata, and then upload. The required `filename` property is used to associate images and metadata objects. Objects
+that contain `error` property will be ignored.
 
 #### Examples
 
-- process and upload all the images in directory `path/to/images` and its sub-directories for user `mapillary_user`.
+Write and read the image description file in another location. This is useful if the image directory is readonly.
 
-```bash
-mapillary_tools process_and_upload --import_path "path/to/images" \
-    --user_name "mapillary_username"
+```shell
+mapillary_tools process "path/to/images/" --desc_path "path/to/description.json"
+mapillary_tools upload  "path/to/images/" --desc_path "path/to/description.json"
 ```
 
-#### Advanced Examples
-
-- Process and upload all the images in directory `path/to/images` and its sub-directories for user `mapillary_user`,
-  while specifying duplicate distance and angle threshold so that duplicate images are consecutive images that are less
-  than 0.5 meter apart according to image `GPS` and have less than 1° camera angle difference according to image
-  direction.
+Edit the description with the script `./description_editor.py` before uploading.
 
-```bash
-mapillary_tools process_and_upload --import_path "path/to/images" \
-    --user_name "mapillary_username" \
-    --verbose \
-    --rerun \
-    --duplicate_distance 0.5 \
-    --duplicate_angle 1 \
-    --advanced
+```shell
+mapillary_tools process "path/to/images/" --desc_path - | ./description_editor.py > "path/to/description.json"
+mapillary_tools upload  "path/to/images/" --desc_path "path/to/description.json"
 ```
 
-### `sample_video`
-
-`sample_video` command will sample a video into images and insert `capture time` to the image EXIF. Capture time is
-calculated based on the `video start time` and sampling interval. Video start time can either be extracted from the
-video metadata or passed as an argument `--video_start_time` (milliseconds since UNIX epoch).
-
-#### Examples
-
-- Sample the video(s) located in `path/to/videos` at the default sampling rate 2 seconds, ie 1 video frame every 2
-  seconds. Video frames will be sampled into a sub directory `.mapillary/sampled_video_frames/video_import_path` at the
-  location of the video.
+Geotag from a special CSV format.
 
-```bash
-mapillary_tools sample_video --video_import_path "path/to/videos" --advanced
+```shell
+./special_csv_to_description.sh special.csv | mapillary_tools upload "path/to/images/" --desc_path -
 ```
 
-- Sample the video(s) located in `path/to/videos` to directory `path/to/images` at a sampling rate 0.5 seconds, ie two
-  video frames every second and specifying the video start time to be `156893940910` (milliseconds since UNIX epoch).
+### Zip Images
 
-```bash
-mapillary_tools sample_video --import_path "path/to/images" \
-    --video_import_path "path/to/videos" \
-    --video_sample_interval 0.5 \
-    --video_start_time 156893940910 \
-    --advanced
-```
-
-### `video_process`
+When uploading an image directory `mapillary_tools upload path/to/images/`, internally the `upload` command will zip
+sequences in the temporary directory (`TMPDIR`) and then upload these zip files.
 
-`video_process` command will run `video_sample` and `process` commands consecutively with combined required and optional
-arguments.
+Mapillary Tools provides `zip` command that allows users to specify where to store the zip files, usually somewhere with
+faster IO or more free space.
 
-#### Examples
+#### Examples:
 
-- In case video start capture time could not be extracted or specified, images should be tagged with `capture time` from
-  the external geotag source, by passing the argument `--use_gps_start_time`. To make sure the external source and
-  images are aligned ok, an offset in seconds can be specified.
+Zip processed images in `path/to/images/` and write zip files in `path/to/zipped_images/`:
 
-```bash
-mapillary_tools video_process --import_path "path/to/images" \
-    --video_import_path "path/to/videos" \
-    --user_name "mapillary_username" --advanced \
-    --geotag_source "gpx" \
-    --geotag_source_path "path/to/gpx" \
-    --use_gps_start_time \
-    --offset_time 2
+```shell
+mapillary_tools zip "path/to/images/" "path/to/zipped_images/"
 ```
 
-### `video_process_and_upload`
-
-`video_process_and_upload` command will run `video_sample`, `process` and `upload` commands consecutively with combined
-required and optional arguments.
-
-#### Examples
-
-- Sample the video(s) located in `path/to/videos` to directory `path/to/images` at the default sampling rate 1 second,
-  ie one video frame every second. Process and upload resulting video frames for user `mapillary_user`, reading geotag
-  data from a gpx track stored in `path/to/gpx_file` video, assuming video start time can be extracted from the video
-  file and deriving camera direction based on `GPS`.
+Choose the image description file to write when zipping images:
 
-```bash
-mapillary_tools video_process_and_upload --import_path "path/to/images" \
-    --video_import_path "path/to/videos" \
-    --user_name "mapillary_username" \
-    --advanced \
-    --geotag_source "gpx" \
-    --geotag_source_path "path/to/gpx_file" \
-    --video_sample_interval 1 \
-    --interpolate_directions
+```shell
+mapillary_tools zip --desc_path "path/to/image_description.json" "path/to/images/" "path/to/zipped_images/"
 ```
 
-### Process Unit Commands
-
-Process unit commands are commands executed by the `process` command. Usage of process unit commands requires the
-flag `--advanced` to be passed and might require some experience with mapillary_tools.
-
-#### `extract_user_data`
-
-`extract_user_data` will process user specific properties and initialize authentication in case of first import.
-Credentials are then stored in a global config file and read from there in further imports.
-
-#### `extract_import_meta_data`
-
-`extract_import_meta_data` will process import specific meta data which is not required, but can be very useful. Import
-meta data is read from EXIF and/or can be passed through additional arguments.
+Then upload the zip files separately:
 
-#### `extract_geotag_data`
-
-`extract_geotag_data` will process image capture date/time, latitude, longitude and camera angle. By default geotag data
-is read from image EXIF. Under advanced usage, a different source of latitude, longitude and camera direction can be
-specified. Geotagging can be adjusted for better quality, by specifying an offset angle for the camera direction or an
-offset time between the camera and gps device.
-
-#### `extract_sequence_data`
-
-`extract_sequence_data` will process the entire set of images located in the import path and create sequences, initially
-based on the file system structure, then based on image capture time and location and in the end splitting sequences
-longer than 500 images. By default, duplicates are flagged to be excluded from upload, using the default duplicate
-thresholds for distance 0.1 m and angle 5°. Optionally, duplicates can be kept and camera directions can be derived
-based on latitude and longitude.
-
-#### `extract_upload_params`
-
-`extract_upload_params` will process user specific upload parameters, required to safely upload images to Mapillary.
-
-#### `exif_insert`
-
-`exif_insert` will take all the meta data read and processed in the other processing unit commands and insert it in the
-image EXIF tag Image Description only, unless additional arguments are passed in order to overwrite the rest of EXIF
-tags as well.
-
-### Other Commands
-
-#### `authenticate`
-
-`authenticate` will update the user credentials stored in `~/.config/mapillary/config`. Mapillary acount `user_name`
-, `user_email` and `user_password` are required and can either be passed as arguments to the command or left unspecified
-and entered upon prompt.
-
-#### `interpolate`
-
-`interpolate` will interpolate identical timestamps in a csv file or stored in image EXIF or will interpolate missing
-gps data in a set of otherwise geotagged images.
-
-#### `process_csv`
-
-`process_csv` will parse the specified csv file and insert data in the image EXIF.
-
-#### `post_process`
-
-`post_process` provides functionalities to help summarize and organize the results of the `process` and/or `upload`
-commands.
-
-## Camera specific
-
-### BlackVue
-#### Local sampling
-
-- Sample one or more Blackvue videos in directory `path/to/videos` into import path `path/to/images` at a sampling rate
-  0.2 seconds, ie 5 frames every second and process resulting video frames for user `mapillary_user`, reading geotag
-  data from the Blackvue videos in `path/to/videos` and specifying camera make and model, specifying to derive camera
-  direction based on `GPS` and use the `GPS` start time. Note that video frames will be sampled
-  into `path/to/images/.mapillary/sampled_video_frames/"video_import_path"`. Video frames will be geotagged after all
-  the videos in the specified `video_import_path` have been sampled. In case video frames geotagging requires `rerun`,
-  there is no need to rerun the entire `video_process` command, in case video frame extraction was successful, rerunning
-  only the `process` command for the given `import_path` is sufficient. We encourage users to check and specify camera
-  make and model, since it helps with camera calibration and improves 3D reconstruction. If you want to check the video
-  frame placement on the map before uploading, specify `--overwrite_EXIF_gps_tag`.
-
-```bash
-mapillary_tools video_process --import_path "path/to/images" \
-    --video_import_path "path/to/videos" \
-    --user_name "mapillary_username" \
-    --advanced \
-    --geotag_source "blackvue_videos" \
-    --geotag_source_path "path/to/videos" \
-    --use_gps_start_time \
-    --interpolate_directions \
-    --video_sample_interval 0.2 \
-    --device_make "Blackvue" \
-    --device_model "DR900S-2CH" \
-    --overwrite_EXIF_gps_tag
-```
-
-### GoPro
-
-- Sample one or more GoPro videos in directory `path/to/videos` into import path `path/to/images` at a sampling rate 0.5
-  seconds, ie 2 frames every second and process resulting video frames for user `mapillary_user`, reading geotag data
-  from the GoPro videos in `path/to/videos` and specifying to derive camera direction based on `GPS`. Note that video
-  frames will be sampled into `path/to/images/.mapillary/sampled_video_frames/"video_import_path"`. Video frames will be
-  geotagged after all the videos in the specified `video_import_path` have been sampled. In case video frames geotagging
-  requires `rerun`, there is no need to rerun the entire `video_process` command, in case video frame extraction was
-  successful, rerunning only the `process` command for the given `import_path` is sufficient. If you want to check the
-  video frame placement on the map before uploading, specify `--overwrite_EXIF_gps_tag`.
-
-```bash
-mapillary_tools video_process --import_path "path/to/images" \
-    --video_import_path "path/to/videos" \
-    --user_name "mapillary_username" \
-    --advanced \
-    --geotag_source "gopro_videos" \
-    --geotag_source_path "path/to/videos" \
-    --interpolate_directions \
-    --video_sample_interval 0.5 \
-    --overwrite_EXIF_gps_tag
+```shell
+for zipfile in path/to/zipped_images/*.zip; do
+  mapillary_tools upload "$zipfile"
+  # optionally remove the zipfile after uploaded
+  rm "$zipfile"
+done
 ```
 
 ## Troubleshooting
 
 In case of any issues with the installation and usage of `mapillary_tools`, check this section in case it has already
-been addressed, otherwise, open an issue on Github.
+been addressed, otherwise, open an issue on GitHub.
 
 ### General
 
 - In case of any issues, it is always safe to try and rerun the failing command while specifying `--verbose` to see more
   information printed out. Uploaded images should not get uploaded more than once and should not be processed after
-  uploading. mapillary_tools should take care of that, if it occurs otherwise, please open an issue on Github.
+  uploading. mapillary_tools should take care of that, if it occurs otherwise, please open an issue on GitHub.
 - Make sure you run the latest version of `mapillary_tools`, which you can check with `mapillary_tools --version`. When
   installing the latest version, don't forget you need to specify `--upgrade`.
-- Advanced user are encouraged to explore the processed data and log files in
-  the `{image_path}/.mapillary/logs/{image_name}/` to get more insight in the failure.
+- Advanced user are encouraged to explore the processed data and the image description file in
+  the `path/to/images/mapillary_image_description.json` to get more insight in the failure.
 
 ### Run time issues
 
 - HTTP Errors can occur due to poor network connection or high load on the import pipeline. In most cases the images
   eventually get uploaded regardless. But in some cases HTTP Errors can occur due to authentication issues, which can be
   resolved by either removing the config file with the users credentials, located in `~/.config/mapillary/config` or
-  running the `authenticate` command available under advanced usage of `mapillary_tools`.
-
-- Windows users sometimes have issues with the prompt not functioning. This usually results in mapillary_tools just
-  hanging without printing anything or creating any logs in `{image_path}/.mapillary/logs/{image_name}`. In such cases
-  authentication should be run separately with the `authentication` command, while passing `user_name`, `user_email`
-  and `user_password` as command line arguments. This will avoid the prompt and will authenticate the user for all
-  further usage of the `process` command.
+  running `mapillary_tools authenticate`.
 
 - Missing required data is often the reason for failed uploads, especially if the processing included parsing external
   data like a gps trace. Images are aligned with a gps trace based on the image capture time and gps time, where the
-  default assumption is that both are in UTC. Check the begin and end date of your capture and the begin and end date of
-  the gps trace to make sure that the image capture time is in the scope of the gps trace. To correct any offset between
-  the two capture times, you can specify `--offset_time "offset time"`. Timezone differences can result in such issues,
-  if you know that the image capture time was stored in your current local timezone, while the gps trace is stored in
-  UTC, specify `--local_time`. If images do not contain capture time or the capture time is unreliable, while gps time
-  is accurate, specify `use_gps_start_time`.
-
-- In cases where the `import_path` is located on an external mount, images can potentially get overwritten, if breaking
-  the script with Ctrl+c. To keep the images intact, you can specify `--keep_original` and all the processed data will
-  be inserted in a copy of the original image. We are still in progress of improving this step of data import and will
-  make sure that no image gets overwritten at any point.
+  default assumption is that both are in UTC. Check the beginning and end date of your capture and the beginning and end
+  date of the gps trace to make sure that the image capture time is in the scope of the gps trace. To correct any offset
+  between the two capture times, you can specify `--offset_time "offset time"`.
 
 ### Upload quality issues
 
 - Some devices do not store the camera direction properly, often storing only 0. Camera direction will get derived based
-  on latitide and longitude only if the camera direction is not set or `--interpolate_directions` is specified. Before
+  on latitude and longitude only if the camera direction is not set or `--interpolate_directions` is specified. Before
   processing and uploading images, make sure that the camera direction is either correct or missing and in case it is
   present but incorrect, you specify `-interpolate_directions`.
-- Timestamp interpolation is required in case the latitude and longitude are stored in an external gps trace with a
-  higher capture frequency then the image capture frequency which results in identical image capture times.
-  Command `interpolate` can be used to interpolate image capture time:
-
-```bash
-mapillary_tools interpolate --data "identical_timestamps" --import_path "path/to/images" --advanced 
- ```
-
-- If `process` includes correction of existing EXIF tag values or extraction of missing EXIF tag values from external
-  sources and you want to test the placement on the map before uploading the images, make sure you
-  pass `--advanced --overwrite_all_EXIF_tags` so that the rest of tags beside Image Description tag will get updated
-  with the values obtained during `process`.
 
 ## Development
 
 Clone the repository:
 
-```bash
+```shell
 git clone git@github.com:mapillary/mapillary_tools.git
 cd mapillary_tools
 ```
 
 Set up the virtual environment. It is optional but recommended:
 
-```bash
+```shell
 python3 -m venv venv
 source venv/bin/activate # For Windows, run: .\venv\Scripts\activate
 # verify if the venv is activated
@@ -759,25 +407,24 @@ which python3
 
 Install dependencies:
 
-```
-python3 -m pip install -r requirements.txt
-python3 -m pip install -r requirements-dev.txt
+```shell
+python3 -m pip install -r requirements.txt -r requirements-dev.txt
 ```
 
 Run the code from the repository:
 
-```
+```shell
 python3 -m mapillary_tools --version
 ```
 
 Run tests:
 
-```
+```shell
 pytest tests
 ```
 
 Run linting:
 
-```
+```shell
 black mapillary_tools tests
 ```
diff --git a/mapillary_tools/__init__.py b/mapillary_tools/__init__.py
index 6852e092..44f4ad01 100644
--- a/mapillary_tools/__init__.py
+++ b/mapillary_tools/__init__.py
@@ -1,1 +1,1 @@
-VERSION = "0.7.4"
+VERSION = "0.8.0"
diff --git a/mapillary_tools/__main__.py b/mapillary_tools/__main__.py
index 9a07e0f9..5c44780f 100644
--- a/mapillary_tools/__main__.py
+++ b/mapillary_tools/__main__.py
@@ -1,96 +1,127 @@
-import sys
 import argparse
-from . import commands, VERSION
+import logging
+import sys
+
+from . import VERSION
+from .commands import authenticate
+from .commands import process
+from .commands import process_and_upload
+from .commands import sample_video
+from .commands import upload
+from .commands import video_process
+from .commands import video_process_and_upload
+from .commands import zip
+
+mapillary_tools_commands = [
+    process,
+    zip,
+    upload,
+    process_and_upload,
+    sample_video,
+    video_process,
+    video_process_and_upload,
+    authenticate,
+]
+
+
+LOG = logging.getLogger(__name__)
+
+
+def add_general_arguments(parser, command):
+    if command == "authenticate":
+        return
+
+    if command in ["sample_video", "video_process", "video_process_and_upload"]:
+        parser.add_argument(
+            "video_import_path",
+            help="Path to a video or directory with one or more video files.",
+        )
+        parser.add_argument(
+            "import_path",
+            help='Path to where the images from video sampling will be saved. If not specified, it will default to "mapillary_sampled_video_frames" under your video import path',
+            nargs="?",
+        )
+        parser.add_argument(
+            "--skip_subfolders",
+            help="Skip all subfolders and import only the images in the given video_import_path",
+            action="store_true",
+            default=False,
+            required=False,
+        )
+    elif command in ["upload"]:
+        parser.add_argument(
+            "import_path",
+            help="Path to your images",
+        )
+    elif command in ["zip"]:
+        parser.add_argument(
+            "import_path",
+            help="Path to your images",
+        )
+        parser.add_argument(
+            "zip_dir",
+            help="Path to store zipped images",
+        )
+    else:
+        parser.add_argument(
+            "import_path",
+            help="Path to your images",
+        )
+        parser.add_argument(
+            "--skip_subfolders",
+            help="Skip all subfolders and import only the images in the given import_path",
+            action="store_true",
+            default=False,
+            required=False,
+        )
+
+
+def configure_logger(logger: logging.Logger, level, stream=None) -> None:
+    """Configure the given logger."""
+    formatter = logging.Formatter("%(asctime)s - %(levelname)-6s - %(message)s")
+    handler = logging.StreamHandler(stream)
+    handler.setLevel(level)
+    handler.setFormatter(formatter)
+    logger.addHandler(handler)
 
 
 def main():
-    advanced = "--advanced" in sys.argv
-    version = "--version" in sys.argv
-    full_help = "--full_help" in sys.argv
-
-    if version:
-        print("")
-        print("Mapillary tools version : " + VERSION)
-        print("")
-        if len(sys.argv) < 3:
-            sys.exit()
-
-    # Create the top-level parser
     parser = argparse.ArgumentParser(
-        "Mapillary import tool",
-        usage="see -h for available tools and corresponding arguments, add --advanced to see additional advanced tools and/or arguments and --version to see version.",
-    )
-    parser.add_argument(
-        "--advanced",
-        help="Use the tools under an advanced level with additional arguments and tools available.",
-        action="store_true",
-        required=False,
-        default=False,
+        "mapillary_tool",
     )
     parser.add_argument(
         "--version",
-        help="Print mapillary tools version.",
-        action="store_true",
-        required=False,
-        default=False,
+        help="show the version of mapillary tools and exit",
+        action="version",
+        version=f"Mapillary tools version : {VERSION}",
     )
     parser.add_argument(
-        "--full_help",
-        help="Print full help for all the available commands and their arguments.",
+        "--verbose",
+        help="Show verbose",
         action="store_true",
-        required=False,
         default=False,
+        required=False,
     )
 
+    all_commands = [module.Command() for module in mapillary_tools_commands]
+
     subparsers = parser.add_subparsers(
-        help="Please choose one of the available tools", dest="tool", metavar="tool"
+        description="please choose one of the available subcommands",
     )
-
-    # Load the subcommands
-    advanced_commands = [
-        module.Command() for module in commands.mapillary_tools_advanced_commands
-    ]
-    basic_commands = [module.Command() for module in commands.mapillary_tools_commands]
-
-    # Create one subparser for each subcommand
-    all_commands = basic_commands
-    if advanced:
-        all_commands += advanced_commands
-
     for command in all_commands:
-        subparser = subparsers.add_parser(command.name, help=command.help)
-        commands.add_general_arguments(subparser, command.name)
-        command.add_basic_arguments(subparser)
-        if advanced:
-            command.add_advanced_arguments(subparser)
-
-    if full_help:
-        subparsers_actions = [
-            action
-            for action in parser._actions
-            if isinstance(action, argparse._SubParsersAction)
-        ]
-        for subparsers_action in subparsers_actions:
-            for choice, subparser in subparsers_action.choices.items():
-                print(f"Subcommand '{choice}'")
-                print(subparser.format_help())
+        cmd_parser = subparsers.add_parser(
+            command.name, help=command.help, conflict_handler="resolve"
+        )
+        add_general_arguments(cmd_parser, command.name)
+        command.add_basic_arguments(cmd_parser)
+        cmd_parser.set_defaults(func=command.run)
 
     args = parser.parse_args()
-
-    args_command = vars(args)["tool"]
-    del vars(args)["tool"]
-    if "advanced" in vars(args):
-        del vars(args)["advanced"]
-    if "version" in vars(args):
-        del vars(args)["version"]
-    if "full_help" in vars(args):
-        del vars(args)["full_help"]
-
-    # Run the selected subcommand if unit command, or in case of batch
-    # command, run several unit commands
-    for command in all_commands:
-        if args_command == command.name:
-            command.run(args)
+    log_level = logging.DEBUG if args.verbose else logging.INFO
+    configure_logger(LOG, log_level, sys.stderr)
+    LOG.setLevel(log_level)
+    LOG.debug(f"argparse vars: {vars(args)}")
+    args.func(vars(args))
 
 
 if __name__ == "__main__":
diff --git a/mapillary_tools/api_v4.py b/mapillary_tools/api_v4.py
index 457d1374..511962ca 100644
--- a/mapillary_tools/api_v4.py
+++ b/mapillary_tools/api_v4.py
@@ -32,4 +32,5 @@ def fetch_organization(
             "Authorization": f"OAuth {user_access_token}",
         },
     )
+    resp.raise_for_status()
     return resp
diff --git a/mapillary_tools/camera_support/prepare_blackvue_videos.py b/mapillary_tools/camera_support/prepare_blackvue_videos.py
index 2a1f4f52..b7eb2780 100644
--- a/mapillary_tools/camera_support/prepare_blackvue_videos.py
+++ b/mapillary_tools/camera_support/prepare_blackvue_videos.py
@@ -3,7 +3,7 @@
 
 
 def find_camera_model(videos_folder) -> bytes:
-    from mapillary_tools.uploader import get_video_file_list
+    from mapillary_tools.image_log import get_video_file_list
 
     file_list = get_video_file_list(videos_folder)
     if not file_list:
@@ -28,38 +28,3 @@ def apply_config_blackvue(vars_args):
     vars_args["geotag_source"] = "blackvue_videos"
     vars_args["duplicate_angle"] = 360
     return vars_args
-
-
-def get_blackvue_info(video_file):
-    response = {
-        "is_Blackvue_video": False,
-    }
-    with open(video_file, "rb") as f:
-        first_bytes = f.read(150)
-        video_details = first_bytes.split(b";")
-        # Check if file is Blackvue video
-        for idx, detail in enumerate(video_details):
-            if b"Pittasoft" in detail:
-                response["is_Blackvue_video"] = True
-                details_start = idx
-        if not response["is_Blackvue_video"]:
-            return response
-        response["header"] = video_details[details_start + 0]
-        response["model_info"] = video_details[details_start + 1]
-        response["firmware_version"] = video_details[details_start + 2]
-        response["language"] = video_details[details_start + 3]
-        # Firmwares before 1.004 don't have a separate field for front and back, just a long string.
-        # Assuming that first byte represents front and back
-        if int(video_details[details_start + 4][0]) == 1:
-            response["camera_direction"] = "Front"
-        elif int(video_details[details_start + 4][0]) == 2:
-            response["camera_direction"] = "Back"
-        try:
-            # Check that string is actually the SN, it could be missing since some firmwares don't output it
-            if video_details[details_start + 5][0:2] == response["model_info"][0:2]:
-                response["serial_number"] = video_details[details_start + 5]
-            else:
-                response["serial_number"] = None
-        except:
-            response["serial_number"] = None
-    return response
diff --git a/mapillary_tools/commands/__init__.py b/mapillary_tools/commands/__init__.py
index 7a3c1d59..3245c6b6 100644
--- a/mapillary_tools/commands/__init__.py
+++ b/mapillary_tools/commands/__init__.py
@@ -1,79 +1,7 @@
 from . import authenticate
-from . import exif_insert
-from . import extract_geotag_data
-from . import extract_import_meta_data
-from . import extract_sequence_data
-from . import extract_upload_params
-from . import extract_user_data
-from . import interpolate
-from . import post_process
 from . import process
 from . import process_and_upload
-from . import process_csv
 from . import sample_video
 from . import upload
 from . import video_process
 from . import video_process_and_upload
-
-mapillary_tools_advanced_commands = [
-    sample_video,
-    video_process,
-    video_process_and_upload,
-    extract_user_data,
-    extract_geotag_data,
-    extract_import_meta_data,
-    extract_sequence_data,
-    extract_upload_params,
-    exif_insert,
-    process_csv,
-    authenticate,
-    interpolate,
-    post_process,
-]
-
-mapillary_tools_commands = [process, upload, process_and_upload]
-
-
-def add_general_arguments(parser, command):
-    parser.add_argument(
-        "--advanced",
-        help="Use the tools under an advanced level with additional arguments and tools available.",
-        action="store_true",
-        required=False,
-        default=False,
-    )
-    parser.add_argument(
-        "--version",
-        help="Print mapillary tools version.",
-        action="store_true",
-        required=False,
-        default=False,
-    )
-
-    if command == "authenticate":
-        return
-
-    # print out warnings
-    parser.add_argument(
-        "--verbose",
-        help="print debug info",
-        action="store_true",
-        default=False,
-        required=False,
-    )
-
-    # import path
-    required = True
-    if command in [
-        "interpolate",
-        "video_process",
-        "video_process_and_upload",
-        "sample_video",
-    ]:
-        required = False
-
-    parser.add_argument(
-        "--import_path",
-        help="path to your photos, or in case of video, path where the photos from video sampling will be saved",
-        required=required,
-    )
diff --git a/mapillary_tools/commands/authenticate.py b/mapillary_tools/commands/authenticate.py
index 74fef897..36f3e123 100644
--- a/mapillary_tools/commands/authenticate.py
+++ b/mapillary_tools/commands/authenticate.py
@@ -1,11 +1,14 @@
+import argparse
+import inspect
+
 from ..edit_config import edit_config
 
 
 class Command:
     name = "authenticate"
-    help = "Helper tool : (Re)run authentication."
+    help = "authenticate Mapillary users"
 
-    def add_basic_arguments(self, parser):
+    def add_basic_arguments(self, parser: argparse.ArgumentParser):
         parser.add_argument(
             "--config_file",
             help="Full path to the config file to be edited. Default is ~/.config/mapillary/configs/MLY_CLIENT_ID",
@@ -44,8 +47,13 @@ def add_basic_arguments(self, parser):
             required=False,
         )
 
-    def add_advanced_arguments(self, parser):
-        pass
-
-    def run(self, args):
-        edit_config(**vars(args))
+    def run(self, vars_args: dict):
+        edit_config(
+            **(
+                {
+                    k: v
+                    for k, v in vars_args.items()
+                    if k in inspect.getfullargspec(edit_config).args
+                }
+            )
+        )
diff --git a/mapillary_tools/commands/exif_insert.py b/mapillary_tools/commands/exif_insert.py
deleted file mode 100644
index b39da659..00000000
--- a/mapillary_tools/commands/exif_insert.py
+++ /dev/null
@@ -1,80 +0,0 @@
-from ..insert_MAPJson import insert_MAPJson
-
-
-class Command:
-    name = "exif_insert"
-    help = "Process unit tool : Format and insert Mapillary image description into image EXIF ImageDescription."
-
-    def add_basic_arguments(self, parser):
-        parser.add_argument(
-            "--rerun", help="rerun the processing", action="store_true", required=False
-        )
-        parser.add_argument(
-            "--skip_subfolders",
-            help="Skip all subfolders and import only the images in the given directory path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-
-    def add_advanced_arguments(self, parser):
-        # master upload
-        parser.add_argument(
-            "--master_upload",
-            help="Process images with a master key, note: only used by Mapillary employees",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--skip_EXIF_insert",
-            help="Skip inserting the extracted data into image EXIF.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--keep_original",
-            help='Do not overwrite original images, instead save the processed images in a new directory called "processed_images" located in .mapillary in the import_path.',
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--overwrite_all_EXIF_tags",
-            help="Overwrite the rest of the EXIF tags, whose values are changed during the processing. Default is False, which will result in the processed values to be inserted only in the EXIF Image Description tag.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--overwrite_EXIF_time_tag",
-            help="Overwrite the capture time EXIF tag with the value obtained in process.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--overwrite_EXIF_gps_tag",
-            help="Overwrite the gps EXIF tag with the value obtained in process.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--overwrite_EXIF_direction_tag",
-            help="Overwrite the camera direction EXIF tag with the value obtained in process.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--overwrite_EXIF_orientation_tag",
-            help="Overwrite the orientation EXIF tag with the value obtained in process.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-
-    def run(self, args):
-        insert_MAPJson(**vars(args))
diff --git a/mapillary_tools/commands/extract_geotag_data.py b/mapillary_tools/commands/extract_geotag_data.py
deleted file mode 100644
index ae5dfc03..00000000
--- a/mapillary_tools/commands/extract_geotag_data.py
+++ /dev/null
@@ -1,89 +0,0 @@
-from ..process_geotag_properties import process_geotag_properties
-
-
-class Command:
-    name = "extract_geotag_data"
-    help = "Process unit tool : Extract and process time and location properties."
-
-    def add_basic_arguments(self, parser):
-        parser.add_argument(
-            "--rerun", help="rerun the processing", action="store_true", required=False
-        )
-        parser.add_argument(
-            "--skip_subfolders",
-            help="Skip all subfolders and import only the images in the given directory path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-
-    def add_advanced_arguments(self, parser):
-        parser.add_argument(
-            "--geotag_source",
-            help="Provide the source of date/time and gps information needed for geotagging.",
-            action="store",
-            choices=["exif", "gpx", "gopro_videos", "nmea", "blackvue_videos"],
-            default="exif",
-            required=False,
-        )
-        parser.add_argument(
-            "--geotag_source_path",
-            help="Provide the path to the file source of date/time and gps information needed for geotagging.",
-            action="store",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--local_time",
-            help="Assume image timestamps are in your local time",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--sub_second_interval",
-            help="Sub second time between shots. Used to set image times with sub-second precision",
-            type=float,
-            default=0.0,
-            required=False,
-        )
-        parser.add_argument(
-            "--offset_time",
-            default=0.0,
-            type=float,
-            help="time offset between the camera and the gps device, in seconds.",
-            required=False,
-        )
-        parser.add_argument(
-            "--offset_angle",
-            default=0.0,
-            type=float,
-            help="offset camera angle (90 for right facing, 180 for rear facing, -90 for left facing)",
-            required=False,
-        )
-        parser.add_argument(
-            "--use_gps_start_time",
-            help="Use GPS trace starting time in case of derivating timestamp from filename.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-
-    def run(self, args):
-        vars_args = vars(args)
-        if (
-            "geotag_source" in vars_args
-            and vars_args["geotag_source"] == "blackvue_videos"
-            and (
-                "device_make" not in vars_args
-                or ("device_make" in vars_args and not vars_args["device_make"])
-            )
-        ):
-            vars_args["device_make"] = "Blackvue"
-        if (
-            "device_make" in vars_args
-            and vars_args["device_make"]
-            and vars_args["device_make"].lower() == "blackvue"
-        ):
-            vars_args["duplicate_angle"] = 360
-        process_geotag_properties(**vars(args))
diff --git a/mapillary_tools/commands/extract_import_meta_data.py b/mapillary_tools/commands/extract_import_meta_data.py
deleted file mode 100644
index 9d76aa79..00000000
--- a/mapillary_tools/commands/extract_import_meta_data.py
+++ /dev/null
@@ -1,94 +0,0 @@
-from ..process_import_meta_properties import (
-    process_import_meta_properties,
-)
-
-
-class Command:
-    name = "extract_import_meta_data"
-    help = "Process unit tool: Extract and process import meta properties."
-
-    def add_basic_arguments(self, parser):
-        parser.add_argument(
-            "--rerun", help="rerun the processing", action="store_true", required=False
-        )
-        parser.add_argument(
-            "--skip_subfolders",
-            help="Skip all subfolders and import only the images in the given directory path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-
-    def add_advanced_arguments(self, parser):
-        # command specific args
-        parser.add_argument(
-            "--device_make",
-            help="Specify device manufacturer. Note this input has precedence over the input read from the import source file.",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--device_model",
-            help="Specify device model. Note this input has precedence over the input read from the import source file.",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--add_file_name",
-            help="Add original file name to EXIF. Note this input has precedence over the input read from the import source file.",
-            action="store_true",
-            required=False,
-        )
-        parser.add_argument(
-            "--windows_path",
-            help="If local file name is to be added with --add_file_name, added it as a windows path.",
-            action="store_true",
-            required=False,
-        )
-        parser.add_argument(
-            "--exclude_import_path",
-            help="If local file name is to be added exclude import_path from the name.",
-            action="store_true",
-            required=False,
-        )
-        parser.add_argument(
-            "--exclude_path",
-            help="If local file name is to be added, specify the path to be excluded.",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--add_import_date",
-            help="Add import date.",
-            action="store_true",
-            required=False,
-        )
-        parser.add_argument(
-            "--orientation",
-            help="Specify the image orientation in degrees. Note this might result in image rotation. Note this input has precedence over the input read from the import source file.",
-            choices=[0, 90, 180, 270],
-            type=int,
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--GPS_accuracy",
-            help="GPS accuracy in meters. Note this input has precedence over the input read from the import source file.",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--camera_uuid",
-            help="Custom string used to differentiate different captures taken with the same camera make and model.",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--custom_meta_data",
-            help='Add custom meta data to all images. Required format of input is a string, consisting of the meta data name, type and value, separated by a comma for each entry, where entries are separated by semicolon. Supported types are long, double, string, boolean, date. Example for two meta data entries "random_name1,double,12.34;random_name2,long,1234"',
-            default=None,
-            required=False,
-        )
-
-    def run(self, args):
-        process_import_meta_properties(**vars(args))
diff --git a/mapillary_tools/commands/extract_sequence_data.py b/mapillary_tools/commands/extract_sequence_data.py
deleted file mode 100644
index 98bf5f58..00000000
--- a/mapillary_tools/commands/extract_sequence_data.py
+++ /dev/null
@@ -1,71 +0,0 @@
-from ..process_sequence_properties import process_sequence_properties
-
-
-class Command:
-    name = "extract_sequence_data"
-    help = "Process unit tool : Extract and process sequence properties."
-
-    def add_basic_arguments(self, parser):
-        parser.add_argument(
-            "--rerun", help="rerun the processing", action="store_true", required=False
-        )
-        parser.add_argument(
-            "--skip_subfolders",
-            help="Skip all subfolders and import only the images in the given directory path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-
-    def add_advanced_arguments(self, parser):
-        parser.add_argument(
-            "--cutoff_distance",
-            default=600.0,
-            type=float,
-            help="maximum gps distance in meters within a sequence",
-            required=False,
-        )
-        parser.add_argument(
-            "--cutoff_time",
-            default=60.0,
-            type=float,
-            help="maximum time interval in seconds within a sequence",
-            required=False,
-        )
-        parser.add_argument(
-            "--interpolate_directions",
-            help="perform interpolation of directions",
-            action="store_true",
-            required=False,
-        )
-        parser.add_argument(
-            "--keep_duplicates",
-            help="keep duplicates, ie do not flag duplicates for upload exclusion, but keep them to be uploaded",
-            action="store_true",
-            required=False,
-            default=False,
-        )
-        parser.add_argument(
-            "--duplicate_distance",
-            default=0.1,
-            type=float,
-            help="max distance for two images to be considered duplicates in meters",
-            required=False,
-        )
-        parser.add_argument(
-            "--duplicate_angle",
-            default=5.0,
-            type=float,
-            help="max angle for two images to be considered duplicates in degrees",
-            required=False,
-        )
-        parser.add_argument(
-            "--offset_angle",
-            default=0.0,
-            type=float,
-            help="offset camera angle (90 for right facing, 180 for rear facing, -90 for left facing)",
-            required=False,
-        )
-
-    def run(self, args):
-        process_sequence_properties(**vars(args))
diff --git a/mapillary_tools/commands/extract_upload_params.py b/mapillary_tools/commands/extract_upload_params.py
deleted file mode 100644
index 087e7f1b..00000000
--- a/mapillary_tools/commands/extract_upload_params.py
+++ /dev/null
@@ -1,33 +0,0 @@
-from ..process_upload_params import process_upload_params
-
-
-class Command:
-    name = "extract_upload_params"
-    help = "Process unit tool : Extract and process upload parameters."
-
-    def add_basic_arguments(self, parser):
-        parser.add_argument(
-            "--rerun", help="rerun the processing", action="store_true", required=False
-        )
-        # user name for the import
-        parser.add_argument("--user_name", help="user name", required=True)
-        parser.add_argument(
-            "--skip_subfolders",
-            help="Skip all subfolders and import only the images in the given directory path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-
-    def add_advanced_arguments(self, parser):
-        # master upload
-        parser.add_argument(
-            "--master_upload",
-            help="Process images with a master key, note: only used by Mapillary employees",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-
-    def run(self, args):
-        process_upload_params(**vars(args))
diff --git a/mapillary_tools/commands/extract_user_data.py b/mapillary_tools/commands/extract_user_data.py
deleted file mode 100644
index d04257df..00000000
--- a/mapillary_tools/commands/extract_user_data.py
+++ /dev/null
@@ -1,53 +0,0 @@
-from ..process_user_properties import process_user_properties
-
-
-class Command:
-    name = "extract_user_data"
-    help = "Process unit tool : Extract and process user properties."
-
-    def add_basic_arguments(self, parser):
-        parser.add_argument(
-            "--rerun", help="rerun the processing", action="store_true", required=False
-        )
-        # user name for the import
-        parser.add_argument("--user_name", help="user name", required=True)
-        # organization level parameters
-        parser.add_argument(
-            "--organization_username",
-            help="Specify organization user name",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--organization_key",
-            help="Specify organization key",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--private",
-            help="Specify whether the import is private",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--skip_subfolders",
-            help="Skip all subfolders and import only the images in the given directory path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-
-    def add_advanced_arguments(self, parser):
-        # master upload
-        parser.add_argument(
-            "--master_upload",
-            help="Process images with a master key, note: only used by Mapillary employees",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-
-    def run(self, args):
-        process_user_properties(**vars(args))
diff --git a/mapillary_tools/commands/interpolate.py b/mapillary_tools/commands/interpolate.py
deleted file mode 100644
index 912e9c27..00000000
--- a/mapillary_tools/commands/interpolate.py
+++ /dev/null
@@ -1,79 +0,0 @@
-from ..interpolation import interpolation
-
-
-class Command:
-    name = "interpolate"
-    help = "Preprocess tool : Interpolate missing gps, identical timestamps, etc..."
-
-    def add_basic_arguments(self, parser):
-        parser.add_argument(
-            "--data",
-            action="store",
-            choices=["missing_gps", "identical_timestamps"],
-            help="Specify which data you want to interpolate.",
-            required=True,
-        )
-        parser.add_argument(
-            "--max_time_delta",
-            default=1.0,
-            type=float,
-            help="Maximum delta time in seconds, for an image with a timestamp out of scope to have missing gps extrapolated.",
-            required=False,
-        )
-        parser.add_argument(
-            "--file_in_path",
-            help="Input file path, in case the identical timestamps to be interpolated are in an external file.",
-            required=False,
-        )
-        parser.add_argument(
-            "--file_format",
-            help="Format of the input file, only csv supported for now.",
-            required=False,
-            default="csv",
-        )
-        parser.add_argument(
-            "--time_column",
-            help="Column with the timestamps, in case interpolating missing timestamps in a csv file.",
-            required=False,
-            type=int,
-            default=0,
-        )
-        parser.add_argument(
-            "--delimiter",
-            help="Delimiter between the columns, in case interpolating missing timestamps in a csv file.",
-            default=",",
-            required=False,
-        )
-        parser.add_argument(
-            "--time_utc",
-            help="Is the timestamp in utc",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--time_format",
-            help="Time format in a string",
-            default="%Y-%m-%dT%H:%M:%SZ",
-            required=False,
-        )
-        parser.add_argument(
-            "--header",
-            help="Specify whether the csv file includes a header, in case interpolating missing timestamps in a csv file..",
-            default=False,
-            required=False,
-            action="store_true",
-        )
-        parser.add_argument(
-            "--keep_original",
-            help='Do not overwrite original file, instead save the file with interpolated times in a new file by adding suffix "_processed" to the input file.',
-            action="store_true",
-            default=False,
-            required=False,
-        )
-
-    def add_advanced_arguments(self, parser):
-        pass
-
-    def run(self, args):
-        interpolation(**vars(args))
diff --git a/mapillary_tools/commands/post_process.py b/mapillary_tools/commands/post_process.py
deleted file mode 100644
index 5119b02b..00000000
--- a/mapillary_tools/commands/post_process.py
+++ /dev/null
@@ -1,90 +0,0 @@
-from ..post_process import post_process
-
-
-class Command:
-    name = "post_process"
-    help = "Post process tool : Post process for a given import path, including import summary and grouping/moving based on import status."
-
-    def add_basic_arguments(self, parser):
-        parser.add_argument(
-            "--skip_subfolders",
-            help="Skip all subfolders and import only the images in the given directory path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-
-    def add_advanced_arguments(self, parser):
-        parser.add_argument(
-            "--summarize",
-            help="Summarize import for given import path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--move_all_images",
-            help="Move all images in import_path according to import state.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--move_duplicates",
-            help="Move images in case they were flagged as duplicates.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--move_uploaded",
-            help="Move images according to upload state.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--move_sequences",
-            help="Move images into sequence folders.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--save_as_json",
-            help="Save summary or file status list in a json.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--list_file_status",
-            help="List file status for given import path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--push_images",
-            help="Push images uploaded in given import path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--split_import_path",
-            help="Provide the path where the images should be moved to based on the import status.",
-            action="store",
-            required=False,
-            default=None,
-        )
-        parser.add_argument(
-            "--save_local_mapping",
-            help="Save the mapillary photo uuid to local file mapping in a csv.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-
-    def run(self, args):
-        post_process(**vars(args))
diff --git a/mapillary_tools/commands/process.py b/mapillary_tools/commands/process.py
index 41d615ea..acf3fa73 100644
--- a/mapillary_tools/commands/process.py
+++ b/mapillary_tools/commands/process.py
@@ -1,108 +1,107 @@
 import inspect
+import argparse
 
 from ..insert_MAPJson import insert_MAPJson
-from ..post_process import post_process
 from ..process_geotag_properties import process_geotag_properties
 from ..process_import_meta_properties import (
     process_import_meta_properties,
 )
 from ..process_sequence_properties import process_sequence_properties
-from ..process_upload_params import process_upload_params
-from ..process_user_properties import process_user_properties
 
 
 class Command:
     name = "process"
-    help = "Main tool : Process image meta data and insert it in image EXIF."
+    help = "process images"
 
-    def add_basic_arguments(self, parser):
+    def add_basic_arguments(self, parser: argparse.ArgumentParser):
         parser.add_argument(
-            "--rerun", help="rerun the processing", action="store_true", required=False
+            "--skip_process_errors",
+            help="Skip process errors.",
+            action="store_true",
+            default=False,
+            required=False,
         )
-        # user properties
-        # user name for the import
-        parser.add_argument("--user_name", help="user name", required=True)
-        # organization level parameters
-        parser.add_argument(
-            "--organization_username",
-            help="Specify organization user name",
-            default=None,
+        group = parser.add_argument_group("process EXIF options")
+        group.add_argument(
+            "--overwrite_all_EXIF_tags",
+            help="Overwrite the rest of the EXIF tags, whose values are changed during the processing. Default is False, which will result in the processed values to be inserted only in the EXIF Image Description tag.",
+            action="store_true",
+            default=False,
             required=False,
         )
-        parser.add_argument(
-            "--organization_key",
-            help="Specify organization key",
-            default=None,
+        group.add_argument(
+            "--overwrite_EXIF_time_tag",
+            help="Overwrite the capture time EXIF tag with the value obtained in process.",
+            action="store_true",
+            default=False,
             required=False,
         )
-        parser.add_argument(
-            "--private",
-            help="Specify whether the import is private",
+        group.add_argument(
+            "--overwrite_EXIF_gps_tag",
+            help="Overwrite the GPS EXIF tag with the value obtained in process.",
             action="store_true",
             default=False,
             required=False,
         )
-        parser.add_argument(
-            "--skip_subfolders",
-            help="Skip all subfolders and import only the images in the given directory path.",
+        group.add_argument(
+            "--overwrite_EXIF_direction_tag",
+            help="Overwrite the camera direction EXIF tag with the value obtained in process.",
             action="store_true",
             default=False,
             required=False,
         )
-
-    def add_advanced_arguments(self, parser):
-        # master upload
-        parser.add_argument(
-            "--master_upload",
-            help="Process images with a master key, note: only used by Mapillary employees",
+        group.add_argument(
+            "--overwrite_EXIF_orientation_tag",
+            help="Overwrite the orientation EXIF tag with the value obtained in process.",
             action="store_true",
             default=False,
             required=False,
         )
-        # import meta
-        parser.add_argument(
+
+        group_metadata = parser.add_argument_group("process metadata options")
+        group_metadata.add_argument(
             "--device_make",
             help="Specify device manufacturer. Note this input has precedence over the input read from the import source file.",
             default=None,
             required=False,
         )
-        parser.add_argument(
+        group_metadata.add_argument(
             "--device_model",
             help="Specify device model. Note this input has precedence over the input read from the import source file.",
             default=None,
             required=False,
         )
-        parser.add_argument(
+        group_metadata.add_argument(
             "--add_file_name",
             help="Add original file name to EXIF. Note this input has precedence over the input read from the import source file.",
             action="store_true",
             required=False,
         )
-        parser.add_argument(
+        group_metadata.add_argument(
             "--exclude_import_path",
             help="If local file name is to be added exclude import_path from the name.",
             action="store_true",
             required=False,
         )
-        parser.add_argument(
+        group_metadata.add_argument(
             "--exclude_path",
             help="If local file name is to be added, specify the path to be excluded.",
             default=None,
             required=False,
         )
-        parser.add_argument(
+        group_metadata.add_argument(
             "--add_import_date",
             help="Add import date.",
             action="store_true",
             required=False,
         )
-        parser.add_argument(
+        group_metadata.add_argument(
             "--windows_path",
             help="If local file name is to be added with --add_file_name, added it as a windows path.",
             action="store_true",
             required=False,
         )
-        parser.add_argument(
+        group_metadata.add_argument(
             "--orientation",
             help="Specify the image orientation in degrees. Note this might result in image rotation. Note this input has precedence over the input read from the import source file.",
             choices=[0, 90, 180, 270],
@@ -110,107 +109,91 @@ def add_advanced_arguments(self, parser):
             default=None,
             required=False,
         )
-        parser.add_argument(
+        group_metadata.add_argument(
             "--GPS_accuracy",
             help="GPS accuracy in meters. Note this input has precedence over the input read from the import source file.",
             default=None,
             required=False,
         )
-        parser.add_argument(
+        group_metadata.add_argument(
             "--camera_uuid",
             help="Custom string used to differentiate different captures taken with the same camera make and model.",
             default=None,
             required=False,
         )
+        group_metadata.add_argument(
+            "--custom_meta_data",
+            help='Add custom meta data to all images. Required format of input is a string, consisting of the meta data name, type and value, separated by a comma for each entry, where entries are separated by semicolon. Supported types are long, double, string, boolean, date. Example for two meta data entries "random_name1,double,12.34;random_name2,long,1234"',
+            default=None,
+            required=False,
+        )
 
-        # geotagging
-        parser.add_argument(
+        group_geotagging = parser.add_argument_group("process geotagging options")
+        group_geotagging.add_argument(
+            "--desc_path",
+            help="Specify the path to store mapillary image descriptions as JSON. By default it is {IMPORT_PATH}/mapillary_image_description.json",
+            default=None,
+            required=False,
+        )
+        group_geotagging.add_argument(
             "--geotag_source",
-            help="Provide the source of date/time and gps information needed for geotagging.",
+            help="Provide the source of date/time and GPS information needed for geotagging",
             action="store",
             choices=["exif", "gpx", "gopro_videos", "nmea", "blackvue_videos"],
             default="exif",
             required=False,
         )
-        parser.add_argument(
+        group_geotagging.add_argument(
             "--geotag_source_path",
-            help="Provide the path to the file source of date/time and gps information needed for geotagging.",
+            help="Provide the path to the file source of date/time and GPS information needed for geotagging",
             action="store",
             default=None,
             required=False,
         )
-        parser.add_argument(
-            "--local_time",
-            help="Assume image timestamps are in your local time",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--sub_second_interval",
-            help="Sub second time between shots. Used to set image times with sub-second precision",
-            type=float,
-            default=0.0,
-            required=False,
-        )
-        parser.add_argument(
+        group_geotagging.add_argument(
             "--offset_time",
             default=0.0,
             type=float,
-            help="time offset between the camera and the gps device, in seconds.",
+            help="time offset, in seconds, that will be added to your image timestamps",
             required=False,
         )
-        parser.add_argument(
+        group_geotagging.add_argument(
             "--offset_angle",
             default=0.0,
             type=float,
-            help="offset camera angle (90 for right facing, 180 for rear facing, -90 for left facing)",
-            required=False,
-        )
-        parser.add_argument(
-            "--use_gps_start_time",
-            help="Use GPS trace starting time in case of derivating timestamp from filename.",
-            action="store_true",
-            default=False,
+            help="camera angle offset, in degrees, that will be added to your image camera angles",
             required=False,
         )
 
-        # sequence
-        parser.add_argument(
+        group_sequence = parser.add_argument_group("process sequence options")
+        group_sequence.add_argument(
             "--cutoff_distance",
             default=600.0,
             type=float,
-            help="maximum gps distance in meters within a sequence",
+            help="maximum GPS distance in meters within a sequence",
             required=False,
         )
-        parser.add_argument(
+        group_sequence.add_argument(
             "--cutoff_time",
             default=60.0,
             type=float,
             help="maximum time interval in seconds within a sequence",
             required=False,
         )
-        parser.add_argument(
+        group_sequence.add_argument(
             "--interpolate_directions",
             help="perform interploation of directions",
             action="store_true",
             required=False,
         )
-        parser.add_argument(
-            "--keep_duplicates",
-            help="keep duplicates, ie do not flag duplicates for upload exclusion, but keep them to be uploaded",
-            action="store_true",
-            required=False,
-            default=False,
-        )
-        parser.add_argument(
+        group_sequence.add_argument(
             "--duplicate_distance",
             help="max distance for two images to be considered duplicates in meters",
             type=float,
             default=0.1,
             required=False,
         )
-        parser.add_argument(
+        group_sequence.add_argument(
             "--duplicate_angle",
             help="max angle for two images to be considered duplicates in degrees",
             type=float,
@@ -218,139 +201,7 @@ def add_advanced_arguments(self, parser):
             required=False,
         )
 
-        # EXIF insert
-        parser.add_argument(
-            "--skip_EXIF_insert",
-            help="Skip inserting the extracted data into image EXIF.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--keep_original",
-            help='Do not overwrite original images, instead save the processed images in a new directory called "processed_images" located in .mapillary in the import_path.',
-            action="store_true",
-            default=False,
-            required=False,
-        )
-
-        # post process
-        parser.add_argument(
-            "--summarize",
-            help="Summarize import for given import path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--move_all_images",
-            help="Move all images in import_path according to import state.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--move_duplicates",
-            help="Move images in case they were flagged as duplicates.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--move_uploaded",
-            help="Move images according to upload state.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--move_sequences",
-            help="Move images into sequence folders.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--save_as_json",
-            help="Save summary or file status list in a json.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--list_file_status",
-            help="List file status for given import path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--push_images",
-            help="Push images uploaded in given import path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--split_import_path",
-            help="If splitting the import path into duplicates, sequences, success and failed uploads, provide a path for the splits.",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--save_local_mapping",
-            help="Save the mapillary photo uuid to local file mapping in a csv.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--overwrite_all_EXIF_tags",
-            help="Overwrite the rest of the EXIF tags, whose values are changed during the processing. Default is False, which will result in the processed values to be inserted only in the EXIF Image Description tag.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--overwrite_EXIF_time_tag",
-            help="Overwrite the capture time EXIF tag with the value obtained in process.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--overwrite_EXIF_gps_tag",
-            help="Overwrite the gps EXIF tag with the value obtained in process.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--overwrite_EXIF_direction_tag",
-            help="Overwrite the camera direction EXIF tag with the value obtained in process.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--overwrite_EXIF_orientation_tag",
-            help="Overwrite the orientation EXIF tag with the value obtained in process.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        # add custom meta data in a form of a string consisting of a triplet
-        # "name,type,value"
-        parser.add_argument(
-            "--custom_meta_data",
-            help='Add custom meta data to all images. Required format of input is a string, consisting of the meta data name, type and value, separated by a comma for each entry, where entries are separated by semicolon. Supported types are long, double, string, boolean, date. Example for two meta data entries "random_name1,double,12.34;random_name2,long,1234"',
-            default=None,
-            required=False,
-        )
-
-    def run(self, args):
-
-        vars_args = vars(args)
+    def run(self, vars_args: dict):
         if (
             "geotag_source" in vars_args
             and vars_args["geotag_source"] == "blackvue_videos"
@@ -366,22 +217,13 @@ def run(self, args):
             and vars_args["device_make"].lower() == "blackvue"
         ):
             vars_args["duplicate_angle"] = 360
-        process_user_properties(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(process_user_properties).args
-                }
-            )
-        )
 
         process_import_meta_properties(
             **(
                 {
                     k: v
                     for k, v in vars_args.items()
-                    if k in inspect.getargspec(process_import_meta_properties).args
+                    if k in inspect.getfullargspec(process_import_meta_properties).args
                 }
             )
         )
@@ -391,7 +233,7 @@ def run(self, args):
                 {
                     k: v
                     for k, v in vars_args.items()
-                    if k in inspect.getargspec(process_geotag_properties).args
+                    if k in inspect.getfullargspec(process_geotag_properties).args
                 }
             )
         )
@@ -401,17 +243,7 @@ def run(self, args):
                 {
                     k: v
                     for k, v in vars_args.items()
-                    if k in inspect.getargspec(process_sequence_properties).args
-                }
-            )
-        )
-
-        process_upload_params(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(process_upload_params).args
+                    if k in inspect.getfullargspec(process_sequence_properties).args
                 }
             )
         )
@@ -421,19 +253,7 @@ def run(self, args):
                 {
                     k: v
                     for k, v in vars_args.items()
-                    if k in inspect.getargspec(insert_MAPJson).args
-                }
-            )
-        )
-
-        print("Process done.")
-
-        post_process(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(post_process).args
+                    if k in inspect.getfullargspec(insert_MAPJson).args
                 }
             )
         )
diff --git a/mapillary_tools/commands/process_and_upload.py b/mapillary_tools/commands/process_and_upload.py
index 01ded81c..fcbdfeb9 100644
--- a/mapillary_tools/commands/process_and_upload.py
+++ b/mapillary_tools/commands/process_and_upload.py
@@ -1,470 +1,15 @@
-import inspect
-
-from ..insert_MAPJson import insert_MAPJson
-from ..post_process import post_process
-from ..process_geotag_properties import process_geotag_properties
-from ..process_import_meta_properties import (
-    process_import_meta_properties,
-)
-from ..process_sequence_properties import process_sequence_properties
-from ..process_upload_params import process_upload_params
-from ..process_user_properties import process_user_properties
-from ..upload import upload
+from .process import Command as ProcessCommand
+from .upload import Command as UploadCommand
 
 
 class Command:
     name = "process_and_upload"
-    help = "Batch tool : Process images and upload to Mapillary."
+    help = "process images and upload to Mapillary"
 
     def add_basic_arguments(self, parser):
-        parser.add_argument(
-            "--rerun", help="rerun the processing", action="store_true", required=False
-        )
-        # user properties
-        # user name for the import
-        parser.add_argument("--user_name", help="user name", required=True)
-        # organization level parameters
-        parser.add_argument(
-            "--organization_username",
-            help="Specify organization user name",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--organization_key",
-            help="Specify organization key",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--private",
-            help="Specify whether the import is private",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--dry_run",
-            help="Disable actual upload. Used for debugging only",
-            type=bool,
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--skip_subfolders",
-            help="Skip all subfolders and import only the images in the given directory path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-
-    def add_advanced_arguments(self, parser):
-        # master upload
-        parser.add_argument(
-            "--master_upload",
-            help="Process images with a master key, note: only used by Mapillary employees",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        # import meta
-        parser.add_argument(
-            "--device_make",
-            help="Specify device manufacturer. Note this input has precedence over the input read from the import source file.",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--device_model",
-            help="Specify device model. Note this input has precedence over the input read from the import source file.",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--add_file_name",
-            help="Add original file name to EXIF. Note this input has precedence over the input read from the import source file.",
-            action="store_true",
-            required=False,
-        )
-        parser.add_argument(
-            "--windows_path",
-            help="If local file name is to be added with --add_file_name, added it as a windows path.",
-            action="store_true",
-            required=False,
-        )
-        parser.add_argument(
-            "--exclude_import_path",
-            help="If local file name is to be added exclude import_path from the name.",
-            action="store_true",
-            required=False,
-        )
-        parser.add_argument(
-            "--exclude_path",
-            help="If local file name is to be added, specify the path to be excluded.",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--add_import_date",
-            help="Add import date.",
-            action="store_true",
-            required=False,
-        )
-        parser.add_argument(
-            "--orientation",
-            help="Specify the image orientation in degrees. Note this might result in image rotation. Note this input has precedence over the input read from the import source file.",
-            choices=[0, 90, 180, 270],
-            type=int,
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--GPS_accuracy",
-            help="GPS accuracy in meters. Note this input has precedence over the input read from the import source file.",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--camera_uuid",
-            help="Custom string used to differentiate different captures taken with the same camera make and model.",
-            default=None,
-            required=False,
-        )
-
-        # geotagging
-        parser.add_argument(
-            "--geotag_source",
-            help="Provide the source of date/time and gps information needed for geotagging.",
-            action="store",
-            choices=["exif", "gpx", "gopro_videos", "nmea", "blackvue_videos"],
-            default="exif",
-            required=False,
-        )
-        parser.add_argument(
-            "--geotag_source_path",
-            help="Provide the path to the file source of date/time and gps information needed for geotagging.",
-            action="store",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--local_time",
-            help="Assume image timestamps are in your local time",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--sub_second_interval",
-            help="Sub second time between shots. Used to set image times with sub-second precision",
-            type=float,
-            default=0.0,
-            required=False,
-        )
-        parser.add_argument(
-            "--offset_time",
-            default=0.0,
-            type=float,
-            help="time offset between the camera and the gps device, in seconds.",
-            required=False,
-        )
-        parser.add_argument(
-            "--offset_angle",
-            default=0.0,
-            type=float,
-            help="offset camera angle (90 for right facing, 180 for rear facing, -90 for left facing)",
-            required=False,
-        )
-        parser.add_argument(
-            "--use_gps_start_time",
-            help="Use GPS trace starting time in case of derivating timestamp from filename.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-
-        # sequence
-        parser.add_argument(
-            "--cutoff_distance",
-            default=600.0,
-            type=float,
-            help="maximum gps distance in meters within a sequence",
-            required=False,
-        )
-        parser.add_argument(
-            "--cutoff_time",
-            default=60.0,
-            type=float,
-            help="maximum time interval in seconds within a sequence",
-            required=False,
-        )
-        parser.add_argument(
-            "--interpolate_directions",
-            help="perform interploation of directions",
-            action="store_true",
-            required=False,
-        )
-        parser.add_argument(
-            "--keep_duplicates",
-            help="keep duplicates, ie do not flag duplicates for upload exclusion, but keep them to be uploaded",
-            action="store_true",
-            required=False,
-            default=False,
-        )
-        parser.add_argument(
-            "--duplicate_distance",
-            help="max distance for two images to be considered duplicates in meters",
-            type=float,
-            default=0.1,
-            required=False,
-        )
-        parser.add_argument(
-            "--duplicate_angle",
-            help="max angle for two images to be considered duplicates in degrees",
-            type=float,
-            default=5,
-            required=False,
-        )
-        # EXIF insert
-        parser.add_argument(
-            "--skip_EXIF_insert",
-            help="Skip inserting the extracted data into image EXIF.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--keep_original",
-            help='Do not overwrite original images, instead save the processed images in a new directory called "processed_images" located in .mapillary in the import_path.',
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--number_threads",
-            help="Specify the number of upload threads.",
-            type=int,
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--max_attempts",
-            help="Specify the maximum number of attempts to upload.",
-            type=int,
-            default=None,
-            required=False,
-        )
-
-        # post process
-        parser.add_argument(
-            "--summarize",
-            help="Summarize import for given import path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--move_all_images",
-            help="Move all images in import_path according to import state.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--move_duplicates",
-            help="Move images in case they were flagged as duplicates.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--move_uploaded",
-            help="Move images according to upload state.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--move_sequences",
-            help="Move images into sequence folders.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--save_as_json",
-            help="Save summary or file status list in a json.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--list_file_status",
-            help="List file status for given import path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--push_images",
-            help="Push images uploaded in given import path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--split_import_path",
-            help="If splitting the import path into duplicates, sequences, success and failed uploads, provide a path for the splits.",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--save_local_mapping",
-            help="Save the mapillary photo uuid to local file mapping in a csv.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--overwrite_all_EXIF_tags",
-            help="Overwrite the rest of the EXIF tags, whose values are changed during the processing. Default is False, which will result in the processed values to be inserted only in the EXIF Image Description tag.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--overwrite_EXIF_time_tag",
-            help="Overwrite the capture time EXIF tag with the value obtained in process.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--overwrite_EXIF_gps_tag",
-            help="Overwrite the gps EXIF tag with the value obtained in process.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--overwrite_EXIF_direction_tag",
-            help="Overwrite the camera direction EXIF tag with the value obtained in process.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--overwrite_EXIF_orientation_tag",
-            help="Overwrite the orientation EXIF tag with the value obtained in process.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        # add custom meta data in a form of a string consisting of a triplet
-        # "name,type,value"
-        parser.add_argument(
-            "--custom_meta_data",
-            help='Add custom meta data to all images. Required format of input is a string, consisting of the meta data name, type and value, separated by a comma for each entry, where entries are separated by semicolon. Supported types are long, double, string, boolean, date. Example for two meta data entries "random_name1,double,12.34;random_name2,long,1234"',
-            default=None,
-            required=False,
-        )
-
-    def run(self, args):
-
-        vars_args = vars(args)
-        if (
-            "geotag_source" in vars_args
-            and vars_args["geotag_source"] == "blackvue_videos"
-            and (
-                "device_make" not in vars_args
-                or ("device_make" in vars_args and not vars_args["device_make"])
-            )
-        ):
-            vars_args["device_make"] = "Blackvue"
-        if (
-            "device_make" in vars_args
-            and vars_args["device_make"]
-            and vars_args["device_make"].lower() == "blackvue"
-        ):
-            vars_args["duplicate_angle"] = 360
-
-        process_user_properties(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(process_user_properties).args
-                }
-            )
-        )
-
-        process_import_meta_properties(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(process_import_meta_properties).args
-                }
-            )
-        )
-
-        process_geotag_properties(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(process_geotag_properties).args
-                }
-            )
-        )
-
-        process_sequence_properties(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(process_sequence_properties).args
-                }
-            )
-        )
-
-        process_upload_params(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(process_upload_params).args
-                }
-            )
-        )
-
-        insert_MAPJson(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(insert_MAPJson).args
-                }
-            )
-        )
-        print("Process done.")
-
-        upload(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(upload).args
-                }
-            )
-        )
+        ProcessCommand().add_basic_arguments(parser)
+        UploadCommand().add_basic_arguments(parser)
 
-        post_process(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(post_process).args
-                }
-            )
-        )
+    def run(self, args: dict):
+        ProcessCommand().run(args)
+        UploadCommand().run(args)
diff --git a/mapillary_tools/commands/process_csv.py b/mapillary_tools/commands/process_csv.py
deleted file mode 100644
index 3e3b9d6a..00000000
--- a/mapillary_tools/commands/process_csv.py
+++ /dev/null
@@ -1,131 +0,0 @@
-from ..process_csv import process_csv
-
-
-class Command:
-    name = "process_csv"
-    help = "Preprocess tool : Parse csv and preprocess the images, to enable running process_and_upload."
-
-    def add_basic_arguments(self, parser):
-        parser.add_argument(
-            "--csv_path",
-            help="Provide the path to the csv file.",
-            action="store",
-            required=True,
-        )
-        parser.add_argument(
-            "--delimiter",
-            help="Delimiter between the columns.",
-            required=False,
-            action="store",
-            default=",",
-        )
-        parser.add_argument(
-            "--convert_gps_time",
-            help="Convert gps time in ticks to standard time.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--convert_utc_time",
-            help="Convert utc epoch time in seconds or milliseconds.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--filename_column",
-            help="Specify the column number of image filename, counting from 1 on.",
-            action="store",
-            required=False,
-            type=int,
-        )
-        parser.add_argument(
-            "--timestamp_column",
-            help="Specify the column number of image timestamp, counting from 1 on.",
-            action="store",
-            required=False,
-            type=int,
-        )
-        parser.add_argument(
-            "--latitude_column",
-            help="Specify the column number of image latitude, counting from 1 on.",
-            action="store",
-            required=False,
-            type=int,
-        )
-        parser.add_argument(
-            "--longitude_column",
-            help="Specify the column number of image longitude, counting from 1 on.",
-            action="store",
-            required=False,
-            type=int,
-        )
-        parser.add_argument(
-            "--heading_column",
-            help="Specify the column number of image heading, counting from 1 on.",
-            action="store",
-            required=False,
-            type=int,
-        )
-        parser.add_argument(
-            "--altitude_column",
-            help="Specify the column number of image altitude, counting from 1 on.",
-            action="store",
-            required=False,
-            type=int,
-        )
-        parser.add_argument(
-            "--gps_week_column",
-            help="Specify the column number of image timestamps gps week, counting from 1 on. Used only with --convert_gps_time.",
-            action="store",
-            required=False,
-            type=int,
-        )
-        parser.add_argument(
-            "--meta_columns",
-            help='Specify the column numbers containing meta data, separate numbers with commas, example "7,9,10".',
-            action="store",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--meta_names",
-            help='Specify the meta data names, separate names with commas, example "meta_data_1,meta_data2,meta_data3".',
-            action="store",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--meta_types",
-            help='Specify the meta data types, separate types with commas, example "string,string,long". Available types are [string, double, long, date, boolean]',
-            action="store",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--time_format",
-            help="Specify the format of the date/time.",
-            action="store",
-            default="%Y:%m:%d %H:%M:%S.%f",
-            required=False,
-        )
-        parser.add_argument(
-            "--header",
-            help="The csv file includes a header.",
-            default=False,
-            required=False,
-            action="store_true",
-        )
-
-    def add_advanced_arguments(self, parser):
-        parser.add_argument(
-            "--keep_original",
-            help='Do not overwrite original images, instead save the processed images in a new directory by adding suffix "_processed" to the import_path.',
-            action="store_true",
-            default=False,
-            required=False,
-        )
-
-    def run(self, args):
-        process_csv(**vars(args))
diff --git a/mapillary_tools/commands/sample_video.py b/mapillary_tools/commands/sample_video.py
index 376cef00..b9d55972 100644
--- a/mapillary_tools/commands/sample_video.py
+++ b/mapillary_tools/commands/sample_video.py
@@ -1,40 +1,38 @@
+import os
+import inspect
+import argparse
+
 from ..process_video import sample_video
 
 
 class Command:
     name = "sample_video"
-    help = "Main tool : Sample video into images."
+    help = "sample video into images"
 
-    def add_basic_arguments(self, parser):
-        # video specific args
-        parser.add_argument(
-            "--video_import_path",
-            help="Path to a video or directory with one or more video files.",
-            action="store",
-            required=True,
-        )
-        parser.add_argument(
+    def add_basic_arguments(self, parser: argparse.ArgumentParser):
+        group = parser.add_argument_group("video process options")
+        group.add_argument(
             "--video_sample_interval",
             help="Time interval for sampled video frames in seconds",
             default=2,
             type=float,
             required=False,
         )
-        parser.add_argument(
+        group.add_argument(
             "--video_duration_ratio",
             help="Real time video duration ratio of the under or oversampled video duration.",
             type=float,
             default=1.0,
             required=False,
         )
-        parser.add_argument(
+        group.add_argument(
             "--video_start_time",
             help="Video start time in epochs (milliseconds)",
             type=int,
             default=None,
             required=False,
         )
-        parser.add_argument(
+        group.add_argument(
             "--skip_subfolders",
             help="Skip all subfolders and import only the images in the given directory path.",
             action="store_true",
@@ -42,9 +40,26 @@ def add_basic_arguments(self, parser):
             required=False,
         )
 
-    def add_advanced_arguments(self, parser):
-        pass
+    def run(self, vars_args: dict):
+        video_import_path = vars_args["video_import_path"]
+        import_path = vars_args["import_path"]
+        if import_path is None:
+            if os.path.isdir(video_import_path):
+                import_path = os.path.join(
+                    video_import_path, "mapillary_sampled_video_frames"
+                )
+            else:
+                import_path = os.path.join(
+                    os.path.dirname(video_import_path), "mapillary_sampled_video_frames"
+                )
+            vars_args["import_path"] = import_path
 
-    def run(self, args):
-        # sample video
-        sample_video(**vars(args))
+        sample_video(
+            **(
+                {
+                    k: v
+                    for k, v in vars_args.items()
+                    if k in inspect.getfullargspec(sample_video).args
+                }
+            )
+        )
diff --git a/mapillary_tools/commands/upload.py b/mapillary_tools/commands/upload.py
index b0793960..bbc562d2 100644
--- a/mapillary_tools/commands/upload.py
+++ b/mapillary_tools/commands/upload.py
@@ -1,130 +1,50 @@
 import inspect
 
-from ..post_process import post_process
 from ..upload import upload
 
 
 class Command:
     name = "upload"
-    help = "Main tool : Upload images to Mapillary."
+    help = "upload images to Mapillary"
 
     def add_basic_arguments(self, parser):
-        # command specific args
-        parser.add_argument(
-            "--skip_subfolders",
-            help="Skip all subfolders and import only the images in the given directory path.",
-            action="store_true",
-            default=False,
+        group = parser.add_argument_group("upload options")
+        group.add_argument(
+            "--user_name", help="Upload to which Mapillary user account", required=False
+        )
+        group.add_argument(
+            "--organization_username",
+            help="Specify organization user name",
+            default=None,
             required=False,
         )
-
-    def add_advanced_arguments(self, parser):
-        parser.add_argument(
-            "--number_threads",
-            help="Specify the number of upload threads.",
-            type=int,
+        group.add_argument(
+            "--organization_key",
+            help="Specify organization key",
             default=None,
             required=False,
         )
-        parser.add_argument(
-            "--max_attempts",
-            help="Specify the maximum number of attempts to upload.",
-            type=int,
+        group.add_argument(
+            "--desc_path",
+            help="Specify the path to read image description",
             default=None,
             required=False,
         )
-        parser.add_argument(
+        group.add_argument(
             "--dry_run",
             help="Disable actual upload. Used for debugging only",
-            type=bool,
-            default=False,
-            required=False,
-        )
-
-        # post process
-        parser.add_argument(
-            "--summarize",
-            help="Summarize import for given import path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--move_all_images",
-            help="Move all images in import_path according to import state.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--move_duplicates",
-            help="Move images in case they were flagged as duplicates.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--move_uploaded",
-            help="Move images according to upload state.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--move_sequences",
-            help="Move images into sequence folders.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--save_as_json",
-            help="Save summary or file status list in a json.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--list_file_status",
-            help="List file status for given import path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--push_images",
-            help="Push images uploaded in given import path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--save_local_mapping",
-            help="Save the mapillary photo uuid to local file mapping in a csv.",
             action="store_true",
             default=False,
             required=False,
         )
 
-    def run(self, args):
-        vars_args = vars(args)
-
+    def run(self, vars_args: dict):
         upload(
             **(
                 {
                     k: v
                     for k, v in vars_args.items()
-                    if k in inspect.getargspec(upload).args
-                }
-            )
-        )
-
-        post_process(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(post_process).args
+                    if k in inspect.getfullargspec(upload).args
                 }
             )
         )
diff --git a/mapillary_tools/commands/video_process.py b/mapillary_tools/commands/video_process.py
index 6c018845..729519d7 100644
--- a/mapillary_tools/commands/video_process.py
+++ b/mapillary_tools/commands/video_process.py
@@ -1,466 +1,17 @@
-import inspect
-
+from .process import Command as ProcessCommand
+from .sample_video import Command as SampleCommand
 from ..apply_camera_specific_config import apply_camera_specific_config
-from ..insert_MAPJson import insert_MAPJson
-from ..post_process import post_process
-from ..process_geotag_properties import process_geotag_properties
-from ..process_import_meta_properties import (
-    process_import_meta_properties,
-)
-from ..process_sequence_properties import process_sequence_properties
-from ..process_upload_params import process_upload_params
-from ..process_user_properties import process_user_properties
-from ..process_video import sample_video
 
 
 class Command:
     name = "video_process"
-    help = "Batch tool : Sample video into images and process image meta data and insert it in image EXIF ImageDescription."
+    help = "sample video into images and process the images"
 
     def add_basic_arguments(self, parser):
-        parser.add_argument(
-            "--rerun", help="rerun the processing", action="store_true", required=False
-        )
-        # user properties
-        # user name for the import
-        parser.add_argument("--user_name", help="user name", required=True)
-        # organization level parameters
-        parser.add_argument(
-            "--organization_username",
-            help="Specify organization user name",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--organization_key",
-            help="Specify organization key",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--private",
-            help="Specify whether the import is private",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--skip_subfolders",
-            help="Skip all subfolders and import only the images in the given directory path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        # video specific args
-        parser.add_argument(
-            "--video_import_path",
-            help="Path to a video or a directory with one or more video files.",
-            action="store",
-            default=None,
-            required=True,
-        )
-        parser.add_argument(
-            "--video_sample_interval",
-            help="Time interval for sampled video frames in seconds",
-            default=2,
-            type=float,
-            required=False,
-        )
-        parser.add_argument(
-            "--video_duration_ratio",
-            help="Real time video duration ratio of the under or oversampled video duration.",
-            type=float,
-            default=1.0,
-            required=False,
-        )
-        parser.add_argument(
-            "--video_start_time",
-            help="Video start time in epochs (milliseconds)",
-            type=int,
-            default=None,
-            required=False,
-        )
-
-    def add_advanced_arguments(self, parser):
-        # master upload
-        parser.add_argument(
-            "--master_upload",
-            help="Process images with a master key, note: only used by Mapillary employees",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        # import meta
-        parser.add_argument(
-            "--device_make",
-            help="Specify device manufacturer. Note this input has precedence over the input read from the import source file.",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--device_model",
-            help="Specify device model. Note this input has precedence over the input read from the import source file.",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--add_file_name",
-            help="Add original file name to EXIF. Note this input has precedence over the input read from the import source file.",
-            action="store_true",
-            required=False,
-        )
-        parser.add_argument(
-            "--exclude_import_path",
-            help="If local file name is to be added exclude import_path from the name.",
-            action="store_true",
-            required=False,
-        )
-        parser.add_argument(
-            "--exclude_path",
-            help="If local file name is to be added, specify the path to be excluded.",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--windows_path",
-            help="If local file name is to be added with --add_file_name, added it as a windows path.",
-            action="store_true",
-            required=False,
-        )
-        parser.add_argument(
-            "--add_import_date",
-            help="Add import date.",
-            action="store_true",
-            required=False,
-        )
-        parser.add_argument(
-            "--orientation",
-            help="Specify the image orientation in degrees. Note this might result in image rotation. Note this input has precedence over the input read from the import source file.",
-            choices=[0, 90, 180, 270],
-            type=int,
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--GPS_accuracy",
-            help="GPS accuracy in meters. Note this input has precedence over the input read from the import source file.",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--camera_uuid",
-            help="Custom string used to differentiate different captures taken with the same camera make and model.",
-            default=None,
-            required=False,
-        )
-
-        # geotagging
-        parser.add_argument(
-            "--geotag_source",
-            help="Provide the source of date/time and gps information needed for geotagging.",
-            action="store",
-            choices=["exif", "gpx", "gopro_videos", "nmea", "blackvue_videos"],
-            default="exif",
-            required=False,
-        )
-        parser.add_argument(
-            "--geotag_source_path",
-            help="Provide the path to the file source of date/time and gps information needed for geotagging.",
-            action="store",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--local_time",
-            help="Assume image timestamps are in your local time",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--sub_second_interval",
-            help="Sub second time between shots. Used to set image times with sub-second precision",
-            type=float,
-            default=0.0,
-            required=False,
-        )
-        parser.add_argument(
-            "--offset_time",
-            default=0.0,
-            type=float,
-            help="time offset between the camera and the gps device, in seconds.",
-            required=False,
-        )
-        parser.add_argument(
-            "--offset_angle",
-            default=0.0,
-            type=float,
-            help="offset camera angle (90 for right facing, 180 for rear facing, -90 for left facing)",
-            required=False,
-        )
-        parser.add_argument(
-            "--use_gps_start_time",
-            help="Use GPS trace starting time in case of derivating timestamp from filename.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-
-        # sequence
-        parser.add_argument(
-            "--cutoff_distance",
-            default=600.0,
-            type=float,
-            help="maximum gps distance in meters within a sequence",
-            required=False,
-        )
-        parser.add_argument(
-            "--cutoff_time",
-            default=60.0,
-            type=float,
-            help="maximum time interval in seconds within a sequence",
-            required=False,
-        )
-        parser.add_argument(
-            "--interpolate_directions",
-            help="perform interpolation of directions",
-            action="store_true",
-            required=False,
-        )
-        parser.add_argument(
-            "--keep_duplicates",
-            help="keep duplicates, ie do not flag duplicates for upload exclusion, but keep them to be uploaded",
-            action="store_true",
-            required=False,
-            default=False,
-        )
-        parser.add_argument(
-            "--duplicate_distance",
-            help="max distance for two images to be considered duplicates in meters",
-            type=float,
-            default=0.1,
-            required=False,
-        )
-        parser.add_argument(
-            "--duplicate_angle",
-            help="max angle for two images to be considered duplicates in degrees",
-            type=float,
-            default=5,
-            required=False,
-        )
-        # EXIF insert
-        parser.add_argument(
-            "--skip_EXIF_insert",
-            help="Skip inserting the extracted data into image EXIF.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--keep_original",
-            help='Do not overwrite original images, instead save the processed images in a new directory called "processed_images" located in .mapillary in the import_path.',
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--overwrite_all_EXIF_tags",
-            help="Overwrite the rest of the EXIF tags, whose values are changed during the processing. Default is False, which will result in the processed values to be inserted only in the EXIF Image Description tag.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--overwrite_EXIF_time_tag",
-            help="Overwrite the capture time EXIF tag with the value obtained in process.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--overwrite_EXIF_gps_tag",
-            help="Overwrite the gps EXIF tag with the value obtained in process.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--overwrite_EXIF_direction_tag",
-            help="Overwrite the camera direction EXIF tag with the value obtained in process.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--overwrite_EXIF_orientation_tag",
-            help="Overwrite the orientation EXIF tag with the value obtained in process.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        # post process
-        parser.add_argument(
-            "--summarize",
-            help="Summarize import for given import path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--move_all_images",
-            help="Move all images in import_path according to import state.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--move_duplicates",
-            help="Move images in case they were flagged as duplicates.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--move_uploaded",
-            help="Move images according to upload state.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--move_sequences",
-            help="Move images into sequence folders.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--save_as_json",
-            help="Save summary or file status list in a json.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--list_file_status",
-            help="List file status for given import path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--push_images",
-            help="Push images uploaded in given import path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--split_import_path",
-            help="If splitting the import path into duplicates, sequences, success and failed uploads, provide a path for the splits.",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--save_local_mapping",
-            help="Save the mapillary photo uuid to local file mapping in a csv.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-
-        # add custom meta data in a form of a string consisting of a triplet
-        # "name,type,value"
-        parser.add_argument(
-            "--custom_meta_data",
-            help='Add custom meta data to all images. Required format of input is a string, consisting of the meta data name, type and value, separated by a comma for each entry, where entries are separated by semicolon. Supported types are long, double, string, boolean, date. Example for two meta data entries "random_name1,double,12.34;random_name2,long,1234"',
-            default=None,
-            required=False,
-        )
-
-    def run(self, args):
-        vars_args = vars(args)
-
-        vars_args = apply_camera_specific_config(vars_args)
-
-        sample_video(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(sample_video).args
-                }
-            )
-        )
-
-        process_user_properties(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(process_user_properties).args
-                }
-            )
-        )
-
-        process_import_meta_properties(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(process_import_meta_properties).args
-                }
-            )
-        )
-
-        process_geotag_properties(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(process_geotag_properties).args
-                }
-            )
-        )
-
-        process_sequence_properties(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(process_sequence_properties).args
-                }
-            )
-        )
-
-        process_upload_params(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(process_upload_params).args
-                }
-            )
-        )
-
-        insert_MAPJson(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(insert_MAPJson).args
-                }
-            )
-        )
-
-        print("Process done.")
+        SampleCommand().add_basic_arguments(parser)
+        ProcessCommand().add_basic_arguments(parser)
 
-        post_process(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(post_process).args
-                }
-            )
-        )
+    def run(self, args: dict):
+        vars_args = apply_camera_specific_config(args)
+        SampleCommand().run(vars_args)
+        ProcessCommand().run(vars_args)
diff --git a/mapillary_tools/commands/video_process_and_upload.py b/mapillary_tools/commands/video_process_and_upload.py
index d29a4bab..28f73bab 100644
--- a/mapillary_tools/commands/video_process_and_upload.py
+++ b/mapillary_tools/commands/video_process_and_upload.py
@@ -1,505 +1,18 @@
-import inspect
-
-from ..insert_MAPJson import insert_MAPJson
-from ..post_process import post_process
-from ..process_geotag_properties import process_geotag_properties
-from ..process_import_meta_properties import (
-    process_import_meta_properties,
-)
-from ..process_sequence_properties import process_sequence_properties
-from ..process_upload_params import process_upload_params
-from ..process_user_properties import process_user_properties
-from ..process_video import sample_video
-from ..upload import upload
+from .process import Command as ProcessCommand
+from .sample_video import Command as SampleCommand
+from .upload import Command as UploadCommand
 
 
 class Command:
     name = "video_process_and_upload"
-    help = (
-        "Batch tool : Sample video into images, process images and upload to Mapillary."
-    )
+    help = "sample video into images, process the images and upload to Mapillary"
 
     def add_basic_arguments(self, parser):
-        parser.add_argument(
-            "--rerun", help="rerun the processing", action="store_true", required=False
-        )
-        # user properties
-        # user name for the import
-        parser.add_argument("--user_name", help="user name", required=True)
-        # organization level parameters
-        parser.add_argument(
-            "--organization_username",
-            help="Specify organization user name",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--organization_key",
-            help="Specify organization key",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--private",
-            help="Specify whether the import is private",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        # video specific args
-        parser.add_argument(
-            "--video_import_path",
-            help="Path to a video or a directory with one or more video files.",
-            action="store",
-            default=None,
-            required=True,
-        )
-        parser.add_argument(
-            "--video_sample_interval",
-            help="Time interval for sampled video frames in seconds",
-            default=2,
-            type=float,
-            required=False,
-        )
-        parser.add_argument(
-            "--video_duration_ratio",
-            help="Real time video duration ratio of the under or oversampled video duration.",
-            type=float,
-            default=1.0,
-            required=False,
-        )
-        parser.add_argument(
-            "--video_start_time",
-            help="Video start time in epochs (milliseconds)",
-            type=int,
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--skip_subfolders",
-            help="Skip all subfolders and import only the images in the given directory path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-
-    def add_advanced_arguments(self, parser):
-        # master upload
-        parser.add_argument(
-            "--master_upload",
-            help="Process images with a master key, note: only used by Mapillary employees",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        # import meta
-        parser.add_argument(
-            "--device_make",
-            help="Specify device manufacturer. Note this input has precedence over the input read from the import source file.",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--device_model",
-            help="Specify device model. Note this input has precedence over the input read from the import source file.",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--add_file_name",
-            help="Add original file name to EXIF. Note this input has precedence over the input read from the import source file.",
-            action="store_true",
-            required=False,
-        )
-        parser.add_argument(
-            "--exclude_import_path",
-            help="If local file name is to be added exclude import_path from the name.",
-            action="store_true",
-            required=False,
-        )
-        parser.add_argument(
-            "--exclude_path",
-            help="If local file name is to be added, specify the path to be excluded.",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--windows_path",
-            help="If local file name is to be added with --add_file_name, added it as a windows path.",
-            action="store_true",
-            required=False,
-        )
-        parser.add_argument(
-            "--add_import_date",
-            help="Add import date.",
-            action="store_true",
-            required=False,
-        )
-        parser.add_argument(
-            "--orientation",
-            help="Specify the image orientation in degrees. Note this might result in image rotation. Note this input has precedence over the input read from the import source file.",
-            choices=[0, 90, 180, 270],
-            type=int,
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--GPS_accuracy",
-            help="GPS accuracy in meters. Note this input has precedence over the input read from the import source file.",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--camera_uuid",
-            help="Custom string used to differentiate different captures taken with the same camera make and model.",
-            default=None,
-            required=False,
-        )
-
-        # geotagging
-        parser.add_argument(
-            "--geotag_source",
-            help="Provide the source of date/time and gps information needed for geotagging.",
-            action="store",
-            choices=["exif", "gpx", "gopro_videos", "nmea", "blackvue_videos"],
-            default="exif",
-            required=False,
-        )
-        parser.add_argument(
-            "--geotag_source_path",
-            help="Provide the path to the file source of date/time and gps information needed for geotagging.",
-            action="store",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--local_time",
-            help="Assume image timestamps are in your local time",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--sub_second_interval",
-            help="Sub second time between shots. Used to set image times with sub-second precision",
-            type=float,
-            default=0.0,
-            required=False,
-        )
-        parser.add_argument(
-            "--offset_time",
-            default=0.0,
-            type=float,
-            help="time offset between the camera and the gps device, in seconds.",
-            required=False,
-        )
-        parser.add_argument(
-            "--offset_angle",
-            default=0.0,
-            type=float,
-            help="offset camera angle (90 for right facing, 180 for rear facing, -90 for left facing)",
-            required=False,
-        )
-        parser.add_argument(
-            "--use_gps_start_time",
-            help="Use GPS trace starting time in case of derivating timestamp from filename.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-
-        # sequence
-        parser.add_argument(
-            "--cutoff_distance",
-            default=600.0,
-            type=float,
-            help="maximum gps distance in meters within a sequence",
-            required=False,
-        )
-        parser.add_argument(
-            "--cutoff_time",
-            default=60.0,
-            type=float,
-            help="maximum time interval in seconds within a sequence",
-            required=False,
-        )
-        parser.add_argument(
-            "--interpolate_directions",
-            help="perform interploation of directions",
-            action="store_true",
-            required=False,
-        )
-        parser.add_argument(
-            "--keep_duplicates",
-            help="keep duplicates, ie do not flag duplicates for upload exclusion, but keep them to be uploaded",
-            action="store_true",
-            required=False,
-            default=False,
-        )
-        parser.add_argument(
-            "--duplicate_distance",
-            help="max distance for two images to be considered duplicates in meters",
-            type=float,
-            default=0.1,
-            required=False,
-        )
-        parser.add_argument(
-            "--duplicate_angle",
-            help="max angle for two images to be considered duplicates in degrees",
-            type=float,
-            default=5,
-            required=False,
-        )
-        # EXIF insert
-        parser.add_argument(
-            "--skip_EXIF_insert",
-            help="Skip inserting the extracted data into image EXIF.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--keep_original",
-            help='Do not overwrite original images, instead save the processed images in a new directory called "processed_images" located in .mapillary in the import_path.',
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--number_threads",
-            help="Specify the number of upload threads.",
-            type=int,
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--max_attempts",
-            help="Specify the maximum number of attempts to upload.",
-            type=int,
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--overwrite_all_EXIF_tags",
-            help="Overwrite the rest of the EXIF tags, whose values are changed during the processing. Default is False, which will result in the processed values to be inserted only in the EXIF Image Description tag.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--overwrite_EXIF_time_tag",
-            help="Overwrite the capture time EXIF tag with the value obtained in process.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--overwrite_EXIF_gps_tag",
-            help="Overwrite the gps EXIF tag with the value obtained in process.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--overwrite_EXIF_direction_tag",
-            help="Overwrite the camera direction EXIF tag with the value obtained in process.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--overwrite_EXIF_orientation_tag",
-            help="Overwrite the orientation EXIF tag with the value obtained in process.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        # post process
-        parser.add_argument(
-            "--summarize",
-            help="Summarize import for given import path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--move_all_images",
-            help="Move all images in import_path according to import state.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--move_duplicates",
-            help="Move images in case they were flagged as duplicates.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--move_uploaded",
-            help="Move images according to upload state.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--move_sequences",
-            help="Move images into sequence folders.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--save_as_json",
-            help="Save summary or file status list in a json.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--list_file_status",
-            help="List file status for given import path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--push_images",
-            help="Push images uploaded in given import path.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-        parser.add_argument(
-            "--split_import_path",
-            help="If splitting the import path into duplicates, sequences, success and failed uploads, provide a path for the splits.",
-            default=None,
-            required=False,
-        )
-        parser.add_argument(
-            "--save_local_mapping",
-            help="Save the mapillary photo uuid to local file mapping in a csv.",
-            action="store_true",
-            default=False,
-            required=False,
-        )
-
-        # add custom meta data in a form of a string consisting of a triplet
-        # "name,type,value"
-        parser.add_argument(
-            "--custom_meta_data",
-            help='Add custom meta data to all images. Required format of input is a string, consisting of the meta data name, type and value, separated by a comma for each entry, where entries are separated by semicolon. Supported types are long, double, string, boolean, date. Example for two meta data entries "random_name1,double,12.34;random_name2,long,1234"',
-            default=None,
-            required=False,
-        )
-
-    def run(self, args):
-        vars_args = vars(args)
-        if (
-            "geotag_source" in vars_args
-            and vars_args["geotag_source"] == "blackvue_videos"
-            and (
-                "device_make" not in vars_args
-                or ("device_make" in vars_args and not vars_args["device_make"])
-            )
-        ):
-            vars_args["device_make"] = "Blackvue"
-        if (
-            "device_make" in vars_args
-            and vars_args["device_make"]
-            and vars_args["device_make"].lower() == "blackvue"
-        ):
-            vars_args["duplicate_angle"] = 360
-
-        sample_video(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(sample_video).args
-                }
-            )
-        )
-
-        process_user_properties(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(process_user_properties).args
-                }
-            )
-        )
-
-        process_import_meta_properties(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(process_import_meta_properties).args
-                }
-            )
-        )
-
-        process_geotag_properties(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(process_geotag_properties).args
-                }
-            )
-        )
-
-        process_sequence_properties(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(process_sequence_properties).args
-                }
-            )
-        )
-
-        process_upload_params(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(process_upload_params).args
-                }
-            )
-        )
-
-        insert_MAPJson(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(insert_MAPJson).args
-                }
-            )
-        )
-
-        print("Process done.")
-
-        upload(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(upload).args
-                }
-            )
-        )
-
-        post_process(
-            **(
-                {
-                    k: v
-                    for k, v in vars_args.items()
-                    if k in inspect.getargspec(post_process).args
-                }
-            )
-        )
+        SampleCommand().add_basic_arguments(parser)
+        ProcessCommand().add_basic_arguments(parser)
+        UploadCommand().add_basic_arguments(parser)
+
+    def run(self, args: dict):
+        SampleCommand().run(args)
+        ProcessCommand().run(args)
+        UploadCommand().run(args)
diff --git a/mapillary_tools/commands/zip.py b/mapillary_tools/commands/zip.py
new file mode 100644
index 00000000..fadadb51
--- /dev/null
+++ b/mapillary_tools/commands/zip.py
@@ -0,0 +1,28 @@
+import inspect
+import argparse
+
+from ..upload import zip_images
+
+
+class Command:
+    name = "zip"
+    help = "Zip images into sequences"
+
+    def add_basic_arguments(self, parser: argparse.ArgumentParser):
+        parser.add_argument(
+            "--desc_path",
+            help="Specify the path to read image description",
+            default=None,
+            required=False,
+        )
+
+    def run(self, vars_args: dict):
+        zip_images(
+            **(
+                {
+                    k: v
+                    for k, v in vars_args.items()
+                    if k in inspect.getfullargspec(zip_images).args
+                }
+            )
+        )
diff --git a/mapillary_tools/config.py b/mapillary_tools/config.py
index 90030956..ed7d4888 100644
--- a/mapillary_tools/config.py
+++ b/mapillary_tools/config.py
@@ -1,7 +1,8 @@
 import configparser
 import os
+import typing as T
 
-from . import api_v4
+from . import api_v4, types
 
 
 _CLIENT_ID = api_v4.MAPILLARY_WEB_CLIENT_ID
@@ -34,9 +35,9 @@ def save_config(config: configparser.ConfigParser, config_path: str) -> None:
         config.write(cfg)
 
 
-def load_user(config: configparser.ConfigParser, user_name: str) -> dict:
+def load_user(config: configparser.ConfigParser, user_name: str) -> types.User:
     user_items = dict(config.items(user_name))
-    return user_items
+    return T.cast(types.User, user_items)
 
 
 def add_user(
@@ -50,14 +51,14 @@ def add_user(
 
 
 def set_user_items(
-    config: configparser.ConfigParser, user_name: str, user_items: dict
+    config: configparser.ConfigParser, user_name: str, user_items: types.User
 ) -> configparser.ConfigParser:
-    for key in user_items.keys():
-        config.set(user_name, key, user_items[key])
+    for key, val in user_items.items():
+        config.set(user_name, key, T.cast(str, val))
     return config
 
 
-def update_config(config_path: str, user_name: str, user_items) -> None:
+def update_config(config_path: str, user_name: str, user_items: types.User) -> None:
     config = load_config(config_path)
     if user_name not in config.sections():
         add_user(config, user_name, config_path)
diff --git a/mapillary_tools/error.py b/mapillary_tools/error.py
index 74e66cb0..3f59d070 100644
--- a/mapillary_tools/error.py
+++ b/mapillary_tools/error.py
@@ -1,6 +1,16 @@
-from . import ipc
+class MapillaryUserError(Exception):
+    pass
 
 
-def print_error(message):
-    print(message)
-    ipc.send_error(message)
+class MapillaryGeoTaggingError(MapillaryUserError):
+    pass
+
+
+class MapillaryStationaryBlackVueError(MapillaryUserError):
+    pass
+
+
+class MapillaryDuplicationError(MapillaryUserError):
+    def __init__(self, message, desc):
+        super().__init__(message)
+        self.desc = desc
diff --git a/mapillary_tools/exif_read.py b/mapillary_tools/exif_read.py
index 1757a375..13016d74 100644
--- a/mapillary_tools/exif_read.py
+++ b/mapillary_tools/exif_read.py
@@ -1,8 +1,7 @@
+import sys
 from typing import List, Optional, Tuple, Type, Union, Any
 import datetime
-import json
 import os
-import logging
 
 import exifread
 
@@ -10,13 +9,11 @@
 from exifread.utils import Ratio
 
 
-LOG = logging.getLogger()
-
-
 def eval_frac(value: Ratio) -> float:
-    if value.den == 0:
-        return -1.0
-    return float(value.num) / float(value.den)
+    try:
+        return float(value.num) / float(value.den)
+    except ZeroDivisionError:
+        return 0
 
 
 def format_time(time_string: str) -> Tuple[datetime.datetime, bool]:
@@ -43,9 +40,10 @@ def format_time(time_string: str) -> Tuple[datetime.datetime, bool]:
 
 def gps_to_decimal(values: List[Ratio], reference: str) -> float:
     sign = 1 if reference in "NE" else -1
-    degrees = eval_frac(values[0])
-    minutes = eval_frac(values[1])
-    seconds = eval_frac(values[2])
+    deg, min, sec = values
+    degrees = eval_frac(deg)
+    minutes = eval_frac(min)
+    seconds = eval_frac(sec)
     return sign * (degrees + minutes / 60 + seconds / 3600)
 
 
@@ -87,9 +85,9 @@ def __init__(self, filename: str, details: bool = False) -> None:
         self.filename = filename
         if isinstance(filename, str):
             with open(filename, "rb") as fp:
-                self.tags = exifread.process_file(fp, details=details)
+                self.tags = exifread.process_file(fp, details=details, debug=True)
         else:
-            self.tags = exifread.process_file(filename, details=details)
+            self.tags = exifread.process_file(filename, details=details, debug=True)
 
     def _extract_alternative_fields(
         self,
@@ -126,12 +124,19 @@ def extract_altitude(self) -> float:
         fields: List[str] = ["GPS GPSAltitude", "EXIF GPS GPSAltitude"]
         refs: List[str] = ["GPS GPSAltitudeRef", "EXIF GPS GPSAltitudeRef"]
         altitude, _ = self._extract_alternative_fields(fields, 0, float)
-        ref = (
-            0
-            if not any([True for x in refs if x in self.tags])
-            else [self.tags[x].values for x in refs if x in self.tags][0][0]
-        )
-        return altitude * altitude_ref[ref]
+        ref = 0
+        for x in refs:
+            t = self.tags.get(x)
+            if t is not None:
+                if t.values:
+                    if isinstance(t.values[0], Ratio):
+                        try:
+                            ref = int(eval_frac(t.values[0]))
+                        except ZeroDivisionError:
+                            pass
+                        else:
+                            break
+        return altitude * altitude_ref.get(ref, 1)
 
     def extract_capture_time(self) -> Optional[datetime.datetime]:
         """
@@ -167,7 +172,7 @@ def extract_capture_time(self) -> Optional[datetime.datetime]:
             capture_time_obj, subseconds = format_time(capture_time)
             sub_sec = "0"
             if not subseconds:
-                sub_sec = self.extract_subsec()
+                sub_sec = self._extract_subsec()
                 if isinstance(sub_sec, str):
                     sub_sec = sub_sec.strip()
             capture_time_obj = capture_time_obj + datetime.timedelta(
@@ -175,7 +180,7 @@ def extract_capture_time(self) -> Optional[datetime.datetime]:
             )
             return capture_time_obj
 
-    def extract_direction(self) -> float:
+    def extract_direction(self) -> Optional[float]:
         """
         Extract image direction (i.e. compass, heading, bearing)
         """
@@ -218,48 +223,34 @@ def extract_gps_time(self) -> Optional[datetime.datetime]:
         else:
             return None
 
-    def extract_image_size(self):
-        """
-        Extract image height and width
-        """
-        width, _ = self._extract_alternative_fields(
-            ["Image ImageWidth", "EXIF ExifImageWidth"], -1, int
-        )
-        height, _ = self._extract_alternative_fields(
-            ["Image ImageLength", "EXIF ExifImageLength"], -1, int
-        )
-        return width, height
-
-    def extract_image_description(self) -> Optional[str]:
-        description, _ = self._extract_alternative_fields(
-            ["Image ImageDescription"], None, str
-        )
-        return description
-
     def extract_lon_lat(self) -> Tuple[Optional[float], Optional[float]]:
-        if "GPS GPSLatitude" in self.tags and "GPS GPSLatitude" in self.tags:
-            lat: Optional[float] = gps_to_decimal(
-                self.tags["GPS GPSLatitude"].values,
-                self.tags["GPS GPSLatitudeRef"].values,
+        lat_tag = self.tags.get("GPS GPSLatitude")
+        lon_tag = self.tags.get("GPS GPSLongitude")
+        if lat_tag and lon_tag:
+            lat_ref_tag = self.tags.get("GPS GPSLatitudeRef")
+            lat = gps_to_decimal(
+                lat_tag.values, lat_ref_tag.values if lat_ref_tag else "N"
             )
-            lon: Optional[float] = gps_to_decimal(
-                self.tags["GPS GPSLongitude"].values,
-                self.tags["GPS GPSLongitudeRef"].values,
+            lon_ref_tag = self.tags.get("GPS GPSLongitudeRef")
+            lon = gps_to_decimal(
+                lon_tag.values, lon_ref_tag.values if lon_ref_tag else "E"
             )
-        elif (
-            "EXIF GPS GPSLatitude" in self.tags and "EXIF GPS GPSLatitude" in self.tags
-        ):
+            return lon, lat
+
+        lat_tag = self.tags.get("EXIF GPS GPSLatitude")
+        lon_tag = self.tags.get("EXIF GPS GPSLongitude")
+        if lat_tag and lon_tag:
+            lat_ref_tag = self.tags.get("EXIF GPS GPSLatitudeRef")
             lat = gps_to_decimal(
-                self.tags["EXIF GPS GPSLatitude"].values,
-                self.tags["EXIF GPS GPSLatitudeRef"].values,
+                lat_tag.values, lat_ref_tag.values if lat_ref_tag else "N"
             )
+            lon_ref_tag = self.tags.get("EXIF GPS GPSLongitudeRef")
             lon = gps_to_decimal(
-                self.tags["EXIF GPS GPSLongitude"].values,
-                self.tags["EXIF GPS GPSLongitudeRef"].values,
+                lon_tag.values, lon_ref_tag.values if lon_ref_tag else "E"
             )
-        else:
-            lon, lat = None, None
-        return lon, lat
+            return lon, lat
+
+        return None, None
 
     def extract_make(self) -> str:
         """
@@ -293,7 +284,7 @@ def extract_orientation(self) -> int:
             return 1
         return orientation
 
-    def extract_subsec(self) -> str:
+    def _extract_subsec(self) -> str:
         """
         Extract microseconds
         """
@@ -310,31 +301,21 @@ def extract_subsec(self) -> str:
         )
         return sub_sec
 
-    def mapillary_tag_exists(self):
-        """
-        Check existence of required Mapillary tags
-        """
-        description = self.extract_image_description()
-        if description is None:
-            return False
-
-        try:
-            description_values = json.loads(description)
-        except json.JSONDecodeError:
-            LOG.warning(f"Error JSON decoding ImageDescription: {description}")
-            return False
-
-        for requirement in [
-            "MAPCaptureTime",
-            "MAPLatitude",
-            "MAPLongitude",
-            "MAPSequenceUUID",
-            "MAPSettingsUserKey",
-        ]:
-            val = description_values.get(requirement)
-            if val is None:
-                return False
-            elif isinstance(val, str):
-                if not val.strip():
-                    return False
-        return True
+
+if __name__ == "__main__":
+    import pprint
+
+    for filename in sys.argv[1:]:
+        exif = ExifRead(filename, details=True)
+        pprint.pprint(
+            {
+                "capture_time": exif.extract_capture_time(),
+                "gps_time": exif.extract_gps_time(),
+                "direction": exif.extract_direction(),
+                "model": exif.extract_model(),
+                "make": exif.extract_make(),
+                "lon_lat": exif.extract_lon_lat(),
+                "altitude": exif.extract_altitude(),
+                "image_history": exif.extract_image_history(),
+            }
+        )
diff --git a/mapillary_tools/exif_write.py b/mapillary_tools/exif_write.py
index 1614d09a..c2bcf3d4 100644
--- a/mapillary_tools/exif_write.py
+++ b/mapillary_tools/exif_write.py
@@ -1,9 +1,11 @@
+import datetime
 import json
+import io
 
 import piexif
 
-from .error import print_error
 from .geo import decimal_to_dms
+from .types import FinalImageDescription
 
 
 class ExifEdit:
@@ -14,29 +16,22 @@ def __init__(self, filename: str):
         self._filename = filename
         self._ef = piexif.load(filename)
 
-    def add_image_description(self, data: dict) -> None:
+    def add_image_description(self, data: FinalImageDescription) -> None:
         """Add a dict to image description."""
         self._ef["0th"][piexif.ImageIFD.ImageDescription] = json.dumps(data)
 
     def add_orientation(self, orientation: int) -> None:
         """Add image orientation to image."""
         if orientation not in range(1, 9):
-            print_error(
-                "Error value for orientation, value must be in range(1,9), setting to default 1"
-            )
-            self._ef["0th"][piexif.ImageIFD.Orientation] = 1
-        else:
-            self._ef["0th"][piexif.ImageIFD.Orientation] = orientation
+            raise ValueError(f"orientation value {orientation} must be in range(1, 9)")
+        self._ef["0th"][piexif.ImageIFD.Orientation] = orientation
 
     def add_date_time_original(
-        self, date_time, time_format: str = "%Y:%m:%d %H:%M:%S.%f"
+        self, date_time: datetime.datetime, time_format: str = "%Y:%m:%d %H:%M:%S.%f"
     ):
         """Add date time original."""
-        try:
-            DateTimeOriginal = date_time.strftime(time_format)[:-3]
-            self._ef["Exif"][piexif.ExifIFD.DateTimeOriginal] = DateTimeOriginal
-        except Exception as e:
-            print_error("Error writing DateTimeOriginal, due to " + str(e))
+        DateTimeOriginal = date_time.strftime(time_format)[:-3]
+        self._ef["Exif"][piexif.ExifIFD.DateTimeOriginal] = DateTimeOriginal
 
     def add_lat_lon(self, lat: float, lon: float, precision: float = 1e7):
         """Add lat, lon to gps (lat, lon in float)."""
@@ -49,10 +44,6 @@ def add_lat_lon(self, lat: float, lon: float, precision: float = 1e7):
             abs(lat), int(precision)
         )
 
-    def add_image_history(self, data: dict) -> None:
-        """Add arbitrary string to ImageHistory tag."""
-        self._ef["0th"][piexif.ImageIFD.ImageHistory] = json.dumps(data)
-
     def add_altitude(self, altitude: float, precision: int = 100) -> None:
         """Add altitude (pre is the precision)."""
         ref = 0 if altitude > 0 else 1
@@ -72,6 +63,22 @@ def add_direction(self, direction, ref="T", precision=100):
         )
         self._ef["GPS"][piexif.GPSIFD.GPSImgDirectionRef] = ref
 
+    def dump_image_bytes(self) -> bytes:
+        try:
+            exif_bytes = piexif.dump(self._ef)
+        except piexif.InvalidImageDataError:
+            if self._ef.get("thumbnail") == b"":
+                # workaround https://github.com/hMatoba/Piexif/issues/30
+                del self._ef["thumbnail"]
+                if "1st" in self._ef:
+                    del self._ef["1st"]
+                exif_bytes = piexif.dump(self._ef)
+            else:
+                raise
+        output = io.BytesIO()
+        piexif.insert(exif_bytes, self._filename, output)
+        return output.read()
+
     def write(self, filename=None):
         """Save exif data to file."""
         if filename is None:
diff --git a/mapillary_tools/ffmpeg.py b/mapillary_tools/ffmpeg.py
index a0ea1e9d..425cf953 100644
--- a/mapillary_tools/ffmpeg.py
+++ b/mapillary_tools/ffmpeg.py
@@ -1,72 +1,65 @@
 import json
 import os
 import subprocess
+import logging
 
-
-# author https://github.com/stilldavid
+LOG = logging.getLogger(__name__)
 
 
 def get_ffprobe(path: str) -> dict:
-    """
-    Gets information about a media file
-    TODO: use the class in ffprobe.py - why doesn't it use json output?
-    """
     if not os.path.isfile(path):
         raise RuntimeError(f"No such file: {path}")
 
+    cmd = [
+        "ffprobe",
+        "-v",
+        "quiet",
+        "-print_format",
+        "json",
+        "-show_format",
+        "-show_streams",
+        path,
+    ]
+    LOG.info(f"Extracting video information: {' '.join(cmd)}")
     try:
-        j_str = subprocess.check_output(
-            [
-                "ffprobe",
-                "-v",
-                "quiet",
-                "-print_format",
-                "json",
-                "-show_format",
-                "-show_streams",
-                path,
-            ]
-        )
+        output = subprocess.check_output(cmd)
     except FileNotFoundError:
         raise RuntimeError(
             "ffprobe not found. Please make sure it is installed in your PATH. See https://github.com/mapillary/mapillary_tools#video-support for instructions"
         )
 
     try:
-        j_obj = json.loads(j_str)
+        j_obj = json.loads(output)
     except json.JSONDecodeError:
-        raise RuntimeError(f"Error JSON decoding {j_str.decode('utf-8')}")
+        raise RuntimeError(f"Error JSON decoding {output.decode('utf-8')}")
 
     return j_obj
 
 
-def extract_stream(source, dest, stream_id):
-    """
-    Get the data out of the file using ffmpeg
-    @param filename: mp4 filename
-    """
+def extract_stream(source: str, dest: str, stream_id: int) -> None:
     if not os.path.isfile(source):
         raise RuntimeError(f"No such file: {source}")
 
+    cmd = [
+        "ffmpeg",
+        "-i",
+        source,
+        "-y",  # overwrite - potentially dangerous
+        "-nostats",
+        "-loglevel",
+        "0",
+        "-codec",
+        "copy",
+        "-map",
+        "0:" + str(stream_id),
+        "-f",
+        "rawvideo",
+        dest,
+    ]
+
+    LOG.info(f"Extracting frames: {' '.join(cmd)}")
     try:
-        subprocess.check_output(
-            [
-                "ffmpeg",
-                "-i",
-                source,
-                "-y",  # overwrite - potentially dangerous
-                "-nostats",
-                "-loglevel",
-                "0",
-                "-codec",
-                "copy",
-                "-map",
-                "0:" + str(stream_id),
-                "-f",
-                "rawvideo",
-                dest,
-            ]
-        )
+        subprocess.check_output(cmd)
     except FileNotFoundError:
         raise RuntimeError(
             "ffmpeg not found. Please make sure it is installed in your PATH. See https://github.com/mapillary/mapillary_tools#video-support for instructions"
diff --git a/mapillary_tools/ffprobe.py b/mapillary_tools/ffprobe.py
index a41658ba..775c8e3d 100644
--- a/mapillary_tools/ffprobe.py
+++ b/mapillary_tools/ffprobe.py
@@ -1,18 +1,12 @@
-#!/usr/bin/python
-# Filename: ffprobe.py
-"""
-Based on Python wrapper for ffprobe command line tool. ffprobe must exist in the path.
-Author: Simon Hargreaves
-
-"""
-
-version = "0.5"
-
+import logging
 import typing as T
 import json
 import subprocess
 
 
+LOG = logging.getLogger(__name__)
+
+
 class FFProbe:
     video: T.List[dict]
     streams: T.List[dict]
@@ -28,6 +22,7 @@ def __init__(self, video_file: str):
             "json",
             self.video_file,
         ]
+        LOG.info(f"Extracting video information: {' '.join(cmd)}")
         try:
             output = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
         except FileNotFoundError:
diff --git a/mapillary_tools/geo.py b/mapillary_tools/geo.py
index 35bc8782..b3620f8c 100644
--- a/mapillary_tools/geo.py
+++ b/mapillary_tools/geo.py
@@ -1,15 +1,12 @@
-# -*- coding: utf-8 -*-
-
 import datetime
 import math
-import logging
+import itertools
+import bisect
 
-import pytz
-from typing import Any, List, Tuple
+from typing import List, Tuple, TypeVar, Iterable, Optional, NamedTuple
 
 WGS84_a = 6378137.0
 WGS84_b = 6356752.314245
-LOG = logging.getLogger()
 
 
 def ecef_from_lla(lat: float, lon: float, alt: float) -> Tuple[float, float, float]:
@@ -71,38 +68,6 @@ def get_max_distance_from_start(latlon_track):
     return max_distance
 
 
-def get_total_distance_traveled(latlon_track):
-    """
-    Returns the total distance traveled of a GPS track. Used to calculate whether or not the entire sequence was just stationary video
-    Takes a sequence of points as input
-    """
-    latlon_list = []
-    # Remove timestamps from list
-    for idx, point in enumerate(latlon_track):
-        lat = latlon_track[idx][1]
-        lon = latlon_track[idx][2]
-        alt = latlon_track[idx][3]
-        latlon_list.append([lat, lon, alt])
-
-    total_distance = 0
-    last_position = latlon_list[0]
-    for position in latlon_list:
-        total_distance += gps_distance(last_position, position)
-        last_position = position
-    return total_distance
-
-
-def gps_speed(distance: List[Any], delta_t: List[Any]) -> List[Any]:
-    # Most timestamps have 1 second resolution so change zeros in delta_t for
-    # 0.5 so that we don't divide by zero
-    delta_t_corrected = [0.5 if x == 0 else x for x in delta_t]
-    speed = [
-        distance / delta_t_corrected
-        for distance, delta_t_corrected in zip(distance, delta_t_corrected)
-    ]
-    return speed
-
-
 def decimal_to_dms(value, precision):
     """
     Convert decimal position to degrees, minutes, seconds in a fromat supported by EXIF
@@ -114,24 +79,6 @@ def decimal_to_dms(value, precision):
     return (deg, 1), (min, 1), (sec, precision)
 
 
-def gpgga_to_dms(gpgga):
-    """
-    Convert GPS coordinate in GPGGA format to degree/minute/second
-
-    Reference: http://us.cactii.net/~bb/gps.py
-    """
-    deg_min, dmin = gpgga.split(".")
-    degrees = int(deg_min[:-2])
-    minutes = float(f"{deg_min[-2:]}.{dmin}")
-    decimal = degrees + (minutes / 60)
-    return decimal
-
-
-def utc_to_localtime(utc_time):
-    utc_offset_timedelta = datetime.datetime.utcnow() - datetime.datetime.now()
-    return utc_time - utc_offset_timedelta
-
-
 def compute_bearing(start_lat, start_lon, end_lat, end_lon) -> float:
     """
     Get the compass bearing from start to end.
@@ -171,14 +118,6 @@ def diff_bearing(b1, b2):
     return d
 
 
-def offset_bearing(bearing, offset):
-    """
-    Add offset to bearing
-    """
-    bearing = (bearing + offset) % 360
-    return bearing
-
-
 def normalize_bearing(bearing: float, check_hex: bool = False) -> float:
     """
     Normalize bearing and convert from hex if
@@ -193,127 +132,58 @@ def normalize_bearing(bearing: float, check_hex: bool = False) -> float:
     return bearing
 
 
-class MapillaryInterpolationError(RuntimeError):
-    pass
+_IT = TypeVar("_IT")
 
 
-def interpolate_lat_lon(points: list, t: datetime.datetime, tolerant=10):
-    """
-    Return interpolated lat, lon and compass bearing for time t.
+# http://stackoverflow.com/a/5434936
+def pairwise(iterable: Iterable[_IT]) -> Iterable[Tuple[_IT, _IT]]:
+    """s -> (s0,s1), (s1,s2), (s2, s3), ..."""
+    a, b = itertools.tee(iterable)
+    next(b, None)
+    return zip(a, b)
 
-    Points is a list of tuples (time, lat, lon, elevation), t a datetime object.
-    """
-    if tolerant < 0:
-        raise ValueError(f"tolerant must be non-negative in seconds but got {tolerant}")
-
-    min_time: datetime.datetime = points[0][0]
-    max_time: datetime.datetime = points[-1][0]
-
-    if min_time > max_time:
-        raise ValueError(
-            f"Expect trace's start time {min_time} <= trace's end time {max_time}"
-        )
-
-    max_dt = datetime.timedelta(seconds=tolerant)
-
-    if min_time < t < max_time:
-        for i, point in enumerate(points):
-            if t < point[0]:
-                if i > 0:
-                    before = points[i - 1]
-                else:
-                    before = points[i]
-                after = points[i]
-                break
-    else:
-        if t < min_time - max_dt:
-            raise MapillaryInterpolationError(
-                f"Unable to interpolate the point captured at {t} because it is behind the trace start time {min_time} by {(min_time - t).total_seconds()} seconds",
-            )
-
-        if max_time + max_dt < t:
-            raise MapillaryInterpolationError(
-                f"Unable to interpolate the point captured at {t} because it is beyond the trace end time {max_time} by {(t - max_time).total_seconds()} seconds",
-            )
-
-        if t < min_time:
-            before = points[0]
-            after = points[1]
-        else:
-            before = points[-2]
-            after = points[-1]
-        bearing = compute_bearing(before[1], before[2], after[1], after[2])
 
-        if t == min_time:
-            x = points[0]
-            return x[1], x[2], bearing, x[3]
+class Point(NamedTuple):
+    time: datetime.datetime
+    lat: float
+    lon: float
+    alt: Optional[float]
 
-        if t == max_time:
-            x = points[-1]
-            return x[1], x[2], bearing, x[3]
 
-    # weight based on time
-    weight = (t - before[0]).total_seconds() / (after[0] - before[0]).total_seconds()
+def interpolate_lat_lon(points: List[Point], t: datetime.datetime):
+    if not points:
+        raise ValueError("Expect non-empty points")
+    # Make sure that points are sorted:
+    # for cur, nex in pairwise(points):
+    #     assert cur.time <= nex.time, "Points not sorted"
+    idx = bisect.bisect_left([x.time for x in points], t)
 
-    # simple linear interpolation in case points are not the same
-    if before[1] == after[1]:
-        lat = before[1]
-    else:
-        lat = before[1] - weight * before[1] + weight * after[1]
-
-    if before[2] == after[2]:
-        lon = before[2]
+    if 0 < idx < len(points):
+        before = points[idx - 1]
+        after = points[idx]
+    elif idx <= 0:
+        if 2 <= len(points):
+            before, after = points[0], points[1]
+        else:
+            before, after = points[0], points[0]
     else:
-        lon = before[2] - weight * before[2] + weight * after[2]
-
-    # camera angle
-    bearing = compute_bearing(before[1], before[2], after[1], after[2])
+        assert len(points) <= idx
+        if 2 <= len(points):
+            before, after = points[-2], points[-1]
+        else:
+            before, after = points[-1], points[-1]
 
-    # altitude
-    if before[3] is not None:
-        ele = before[3] - weight * before[3] + weight * after[3]
+    if before.time == after.time:
+        weight = 0.0
     else:
-        ele = None
-
-    return lat, lon, bearing, ele
-
-
-def write_gpx(filename, gps_trace):
-    time_format = "%Y-%m-%dT%H:%M:%S.%f"
-    gpx = "<gpx>" + "\n"
-    gpx += "<trk>" + "\n"
-    gpx += "<name>Mapillary GPX</name>" + "\n"
-    gpx += "<trkseg>" + "\n"
-    for point in gps_trace:
-        lat = point[1]
-        lon = point[2]
-        if lat == 0 or lon == 0:
-            continue
-        time = datetime.datetime.strftime(point[0], time_format)[:-3]
-        elevation = point[3] if len(point) > 3 else 0
-        gpx += '<trkpt lat="' + str(lat) + '" lon="' + str(lon) + '">' + "\n"
-        gpx += "<ele>" + str(elevation) + "</ele>" + "\n"
-        gpx += "<time>" + time + "</time>" + "\n"
-        gpx += "</trkpt>" + "\n"
-    gpx += "</trkseg>" + "\n"
-    gpx += "</trk>" + "\n"
-    gpx += "</gpx>" + "\n"
-    with open(filename, "w") as fout:
-        fout.write(gpx)
-
-
-def get_timezone_and_utc_offset(lat, lon):
-    # TODO Importing inside function because tzwhere is a temporary solution and dependency fails to install on windows
-    from tzwhere import tzwhere
-
-    tz = tzwhere.tzwhere(
-        forceTZ=True
-    )  # TODO: This library takes 2 seconds to initialize. Should be done only once if used for many videos
-    timezone_str = tz.tzNameAt(lat, lon)
-    if timezone_str is not None:
-        timezone = pytz.timezone(timezone_str)
-        dt = datetime.datetime.utcnow()
-        return [timezone_str, timezone.utcoffset(dt)]
+        weight = (t - before.time).total_seconds() / (
+            after.time - before.time
+        ).total_seconds()
+    lat = before.lat - weight * before.lat + weight * after.lat
+    lon = before.lon - weight * before.lon + weight * after.lon
+    bearing = compute_bearing(before.lat, before.lon, after.lat, after.lon)
+    if before.alt is not None and after.alt is not None:
+        alt: Optional[float] = before.alt - weight * before.alt + weight * after.alt
     else:
-        print("ERROR: Could not determine timezone")
-        return [None, None]
+        alt = None
+    return lat, lon, bearing, alt
diff --git a/mapillary_tools/geojson.py b/mapillary_tools/geojson.py
new file mode 100644
index 00000000..25200b1b
--- /dev/null
+++ b/mapillary_tools/geojson.py
@@ -0,0 +1,126 @@
+import typing as T
+
+import jsonschema
+
+from . import types
+
+
+def feature_collection_schema(features: dict) -> dict:
+    return {
+        "type": "object",
+        "properties": {
+            "type": {"type": "string", "enum": "FeatureCollection"},
+            "features": {
+                "type": "array",
+                "items": features,
+            },
+        },
+    }
+
+
+def feature_schema(geometry: dict, properties: T.Any) -> dict:
+    return {
+        "type": "object",
+        "properties": {
+            "type": {"type": "string", "enum": "Feature"},
+            "properties": properties,
+            "geometry_schema": geometry,
+        },
+    }
+
+
+def position_schema():
+    return {
+        "type": "array",
+        "items": {
+            "type": "number",
+        },
+    }
+
+
+def point_schema():
+    return {
+        "type": "object",
+        "properties": {
+            "type": {"type": "string"},
+            "coordinates": position_schema(),
+        },
+    }
+
+
+def single_feature_to_desc(
+    feature: dict, quiet=False
+) -> T.Optional[types.FinalImageDescriptionFromGeoJSON]:
+    input_schema = feature_schema(point_schema(), {})
+    if quiet:
+        try:
+            jsonschema.validate(instance=feature, schema=input_schema)
+        except jsonschema.exceptions.ValidationError:
+            return None
+    lng, lat = feature["geometry"]["coordinates"]
+    desc = {
+        "MAPLatitude": lng,
+        "MAPLongitude": lat,
+        **feature["properties"],
+    }
+    return T.cast(types.FinalImageDescriptionFromGeoJSON, desc)
+
+
+def feature_collection_to_desc(
+    feature_collection: dict,
+) -> T.List[types.FinalImageDescriptionFromGeoJSON]:
+    input_schema = feature_collection_schema(
+        feature_schema(
+            point_schema(),
+            True,
+        )
+    )
+    jsonschema.validate(instance=feature_collection, schema=input_schema)
+    desc = [
+        single_feature_to_desc(f, quiet=True) for f in feature_collection["features"]
+    ]
+    return [d for d in desc if d is not None]
+
+
+def single_desc_to_feature(
+    desc: types.FinalImageDescription, quiet=False
+) -> T.Optional[dict]:
+    input_schema = {
+        "type": "array",
+        "items": types.FinalImageDescriptionSchema,
+    }
+    if quiet:
+        try:
+            jsonschema.validate(instance=desc, schema=input_schema)
+        except jsonschema.exceptions.ValidationError:
+            return None
+    properties = {**desc}
+    del properties["MAPLongitude"]
+    del properties["MAPLatitude"]
+    return {
+        "type": "Feature",
+        "geometry": {
+            "type": "Point",
+            "coordinates": [desc["MAPLongitude"], desc["MAPLatitude"]],
+        },
+        "properties": properties,
+    }
+
+
+def desc_to_feature_collection(desc: T.List[types.FinalImageDescription]) -> dict:
+    features = [single_desc_to_feature(d) for d in desc]
+    return {
+        "type": "FeatureCollection",
+        "features": features,
+    }
+
+
+if __name__ == "__main__":
+    import json
+    import sys
+
+    with open(sys.argv[1]) as fp:
+        descs = json.load(fp)
+        descs = [d for d in descs if "error" not in d]
+        feature_collection = desc_to_feature_collection(descs)
+        print(json.dumps(feature_collection, indent=4))
diff --git a/mapillary_tools/gps_parser.py b/mapillary_tools/gps_parser.py
index e748c0a3..c2d99365 100644
--- a/mapillary_tools/gps_parser.py
+++ b/mapillary_tools/gps_parser.py
@@ -1,43 +1,43 @@
-#!/usr/bin/python
-
 import datetime
-
-from .geo import utc_to_localtime
-
 import typing as T
 import gpxpy
 import pynmea2
 
+from .types import GPXPoint
+
+
 """
 Methods for parsing gps data from various file format e.g. GPX, NMEA, SRT.
 """
 
 
-def get_lat_lon_time_from_gpx(
-    gpx_file, local_time=True
-) -> T.List[T.Tuple[datetime.datetime, float, float, float]]:
-    """
-    Read location and time stamps from a track in a GPX file.
-
-    Returns a list of tuples (time, lat, lon).
-
-    GPX stores time in UTC, by default we assume your camera used the local time
-    and convert accordingly.
-    """
+def get_lat_lon_time_from_gpx(gpx_file: str) -> T.List[GPXPoint]:
     with open(gpx_file, "r") as f:
         gpx = gpxpy.parse(f)
 
-    points = []
-    if len(gpx.tracks) > 0:
-        for track in gpx.tracks:
-            for segment in track.segments:
-                for point in segment.points:
-                    t = utc_to_localtime(point.time) if local_time else point.time
-                    points.append((t, point.latitude, point.longitude, point.elevation))
-    if len(gpx.waypoints) > 0:
-        for point in gpx.waypoints:
-            t = utc_to_localtime(point.time) if local_time else point.time
-            points.append((t, point.latitude, point.longitude, point.elevation))
+    points: T.List[GPXPoint] = []
+
+    for track in gpx.tracks:
+        for segment in track.segments:
+            for point in segment.points:
+                points.append(
+                    GPXPoint(
+                        point.time,
+                        lat=point.latitude,
+                        lon=point.longitude,
+                        alt=point.elevation,
+                    )
+                )
+
+    for point in gpx.waypoints:
+        points.append(
+            GPXPoint(
+                time=point.time,
+                lat=point.latitude,
+                lon=point.longitude,
+                alt=point.elevation,
+            )
+        )
 
     # sort by time just in case
     points.sort()
@@ -45,17 +45,7 @@ def get_lat_lon_time_from_gpx(
     return points
 
 
-def get_lat_lon_time_from_nmea(
-    nmea_file, local_time=True
-) -> T.List[T.Tuple[datetime.datetime, float, float, float]]:
-    """
-    Read location and time stamps from a track in a NMEA file.
-
-    Returns a list of tuples (time, lat, lon).
-
-    GPX stores time in UTC, by default we assume your camera used the local time
-    and convert accordingly.
-    """
+def get_lat_lon_time_from_nmea(nmea_file: str) -> T.List[GPXPoint]:
     with open(nmea_file, "r") as f:
         lines = f.readlines()
         lines = [l.rstrip("\n\r") for l in lines]
@@ -78,7 +68,7 @@ def get_lat_lon_time_from_nmea(
             data = pynmea2.parse(l)
             timestamp = datetime.datetime.combine(date, data.timestamp)
             lat, lon, alt = data.latitude, data.longitude, data.altitude
-            points.append((timestamp, lat, lon, alt))
+            points.append(GPXPoint(time=timestamp, lat=lat, lon=lon, alt=alt))
 
     points.sort()
     return points
diff --git a/mapillary_tools/gpx_from_blackvue.py b/mapillary_tools/gpx_from_blackvue.py
index 1643bd79..864f2839 100644
--- a/mapillary_tools/gpx_from_blackvue.py
+++ b/mapillary_tools/gpx_from_blackvue.py
@@ -1,23 +1,23 @@
-from typing import Tuple
+import logging
+import typing as T
 import datetime
 import io
-import os
 import re
-import sys
 
 import pynmea2
-from construct.core import RangeError, ConstError
 from pymp4.parser import Box
 
 from .geo import get_max_distance_from_start
-from .geo import write_gpx
+from .types import GPXPoint
 
 """
 Pulls geo data out of a BlackVue video files
 """
 
+LOG = logging.getLogger(__name__)
 
-def get_points_from_bv(path, use_nmea_stream_timestamp=False):
+
+def get_points_from_bv(path, use_nmea_stream_timestamp=False) -> T.List[GPXPoint]:
     points = []
     with open(path, "rb") as fd:
         fd.seek(0, io.SEEK_END)
@@ -25,20 +25,13 @@ def get_points_from_bv(path, use_nmea_stream_timestamp=False):
         fd.seek(0)
         date = None
 
-        first_gps_date = None
-        first_gps_time = None
+        first_gps_date: T.Optional[datetime.date] = None
+        first_gps_time: T.Optional[datetime.time] = None
         found_first_gps_date = False
         found_first_gps_time = False
 
         while fd.tell() < eof:
-            try:
-                box = Box.parse_stream(fd)
-            except RangeError:
-                print("error parsing blackvue GPS information, exiting")
-                sys.exit(1)
-            except ConstError:
-                print("error parsing blackvue GPS information, exiting")
-                sys.exit(1)
+            box = Box.parse_stream(fd)
 
             if box.type.decode("utf-8") == "free":
                 length = len(box.data)
@@ -92,7 +85,7 @@ def get_points_from_bv(path, use_nmea_stream_timestamp=False):
                                         # There are often Checksum errors in the GPS stream, better not to show errors to user
                                         pass
                                     except Exception:
-                                        print(
+                                        LOG.warning(
                                             "Warning: Error in parsing gps trace to extract date information, nmea parsing failed"
                                         )
                             if use_nmea_stream_timestamp:
@@ -114,8 +107,9 @@ def get_points_from_bv(path, use_nmea_stream_timestamp=False):
                                             points.append((timestamp, lat, lon, alt))
 
                                     except Exception as e:
-                                        print(
-                                            f"Error in parsing gps trace to extract time and gps information, nmea parsing failed due to {e}"
+                                        LOG.error(
+                                            f"Error in parsing GPS trace to extract time and GPS information, nmea parsing failed",
+                                            exc_info=e,
                                         )
 
                         # If there are no points after parsing just return empty vector
@@ -126,7 +120,8 @@ def get_points_from_bv(path, use_nmea_stream_timestamp=False):
                         if not use_nmea_stream_timestamp:
                             # If we use the camera timestamp, we need to get the timezone offset, since Mapillary backend expects UTC timestamps
                             first_gps_timestamp = datetime.datetime.combine(
-                                first_gps_date, first_gps_time
+                                T.cast(datetime.date, first_gps_date),
+                                T.cast(datetime.time, first_gps_time),
                             )
                             delta_t = points[0][0] - first_gps_timestamp
                             if delta_t.days > 0:
@@ -159,11 +154,10 @@ def get_points_from_bv(path, use_nmea_stream_timestamp=False):
                             # add date to points that don't have it yet, because GPRMC message came later
                             utc_points = []
                             for idx, point in enumerate(points):
-                                if type(points[idx][0]) != type(
-                                    datetime.datetime.today()
-                                ):
+                                if not isinstance(points[idx][0], datetime.datetime):
                                     timestamp = datetime.datetime.combine(
-                                        first_gps_date, points[idx][0]
+                                        T.cast(datetime.date, first_gps_date),
+                                        T.cast(datetime.time, points[idx][0]),
                                     )
                                 else:
                                     timestamp = points[idx][0]
@@ -179,7 +173,7 @@ def get_points_from_bv(path, use_nmea_stream_timestamp=False):
 
                 break
 
-        return points
+        return [GPXPoint(time=p[0], lat=p[1], lon=p[2], alt=p[3]) for p in points]
 
 
 def is_video_stationary(max_distance_from_start) -> bool:
@@ -191,12 +185,10 @@ def is_video_stationary(max_distance_from_start) -> bool:
     )
 
 
-def gpx_from_blackvue(bv_video, use_nmea_stream_timestamp=False) -> Tuple[str, bool]:
-    bv_data = get_points_from_bv(bv_video, use_nmea_stream_timestamp)
-    if not bv_data:
-        return "", True
-    basename, extension = os.path.splitext(bv_video)
-    gpx_path = basename + ".gpx"
-    bv_data.sort(key=lambda x: x[0])
-    write_gpx(gpx_path, bv_data)
-    return gpx_path, is_video_stationary(get_max_distance_from_start(bv_data))
+def gpx_from_blackvue(
+    bv_video, use_nmea_stream_timestamp=False
+) -> T.Tuple[T.List[GPXPoint], bool]:
+    points = get_points_from_bv(bv_video, use_nmea_stream_timestamp)
+    if not points:
+        return points, True
+    return points, is_video_stationary(get_max_distance_from_start(points))
diff --git a/mapillary_tools/gpx_from_exif.py b/mapillary_tools/gpx_from_exif.py
index 4009549d..af756fb2 100644
--- a/mapillary_tools/gpx_from_exif.py
+++ b/mapillary_tools/gpx_from_exif.py
@@ -1,51 +1,29 @@
-from . import exif_read
-from .geo import write_gpx
+from .types import GPXPoint, GPXPointAngle
+from .exif_read import ExifRead
+from .error import MapillaryGeoTaggingError
 
 
-def get_points_from_exif(file_list, verbose=False):
-    data = []
-    for file in file_list:
-        point = ()
-        try:
-            exif = exif_read.ExifRead(file)
-        except:
-            if verbose:
-                print(f"Warning, EXIF could not be read for image {file}.")
-            continue
-        try:
-            lon, lat = exif.extract_lon_lat()
-        except:
-            if verbose:
-                print(f"Warning {file} image latitude or longitude tag not in EXIF.")
-            continue
-        try:
-            timestamp = exif.extract_capture_time()
-        except:
-            if verbose:
-                print(f"Warning {file} image capture time tag not in EXIF.")
-            continue
-        if lon is not None and lat is not None and timestamp is not None:
-            point = point + (timestamp, lat, lon)
-        else:
-            continue
-        try:
-            altitude = exif.extract_altitude()
-            point = point + (altitude,)
-        except:
-            pass
-        try:
-            heading = exif.extract_direction()
-            point = point + (heading,)
-        except:
-            pass
-        if point:
-            data.append(point)
-    return data
+def gpx_from_exif(image: str) -> GPXPointAngle:
+    exif = ExifRead(image)
 
+    lon, lat = exif.extract_lon_lat()
+    if lat is None or lon is None:
+        raise MapillaryGeoTaggingError(
+            "Unable to extract GPS Longitude or GPS Latitude from the image"
+        )
 
-def gpx_from_exif(file_list, import_path, verbose=False):
-    data = get_points_from_exif(file_list, verbose)
-    data = sorted(data, key=lambda x: x[0])
-    gpx_path = import_path + ".gpx"
-    write_gpx(gpx_path, data)
-    return gpx_path
+    timestamp = exif.extract_capture_time()
+    if timestamp is None:
+        raise MapillaryGeoTaggingError("Unable to extract timestamp from the image")
+
+    angle = exif.extract_direction()
+
+    return GPXPointAngle(
+        point=GPXPoint(
+            time=timestamp,
+            lon=lon,
+            lat=lat,
+            alt=exif.extract_altitude(),
+        ),
+        angle=angle,
+    )
diff --git a/mapillary_tools/gpx_from_gopro.py b/mapillary_tools/gpx_from_gopro.py
index fe8dfb3c..b7575c1b 100644
--- a/mapillary_tools/gpx_from_gopro.py
+++ b/mapillary_tools/gpx_from_gopro.py
@@ -1,15 +1,10 @@
 import datetime
 import os
+import typing as T
 
 from .ffmpeg import extract_stream, get_ffprobe
-from .geo import write_gpx
 from .gpmf import parse_bin, interpolate_times
-
-# author https://github.com/stilldavid
-
-"""
-Pulls data out of a GoPro 5+ recording while GPS was enabled.
-"""
+from .types import GPXPoint
 
 
 def extract_bin(path: str) -> str:
@@ -30,7 +25,7 @@ def extract_bin(path: str) -> str:
     if stream_id is None:
         raise IOError("No GoPro metadata track found - was GPS turned on?")
 
-    basename, extension = os.path.splitext(path)
+    basename, _ = os.path.splitext(path)
     bin_path = basename + ".bin"
 
     extract_stream(path, bin_path, stream_id)
@@ -38,13 +33,13 @@ def extract_bin(path: str) -> str:
     return bin_path
 
 
-def get_points_from_gpmf(path: str) -> list:
+def get_points_from_gpmf(path: str) -> T.List[GPXPoint]:
     bin_path = extract_bin(path)
 
     gpmf_data = parse_bin(bin_path)
     rows = len(gpmf_data)
 
-    points = []
+    points: T.List[GPXPoint] = []
     for i, frame in enumerate(gpmf_data):
         t = frame["time"]
 
@@ -57,24 +52,17 @@ def get_points_from_gpmf(path: str) -> list:
 
         for point in frame["gps"]:
             points.append(
-                (
-                    point["time"],
-                    point["lat"],
-                    point["lon"],
-                    point["alt"],
-                    frame["gps_fix"],
+                GPXPoint(
+                    time=point["time"],
+                    lat=point["lat"],
+                    lon=point["lon"],
+                    alt=point["alt"],
+                    # frame["gps_fix"],
                 )
             )
 
     return points
 
 
-def gpx_from_gopro(gopro_video):
-    gopro_data = get_points_from_gpmf(gopro_video)
-
-    basename, extension = os.path.splitext(gopro_video)
-    gpx_path = basename + ".gpx"
-
-    write_gpx(gpx_path, sorted(gopro_data))
-
-    return gpx_path
+def gpx_from_gopro(gopro_video: str) -> T.List[GPXPoint]:
+    return get_points_from_gpmf(gopro_video)
diff --git a/mapillary_tools/image_log.py b/mapillary_tools/image_log.py
new file mode 100644
index 00000000..ffd49b09
--- /dev/null
+++ b/mapillary_tools/image_log.py
@@ -0,0 +1,71 @@
+import os
+import typing as T
+from typing import Generator, List, Optional
+
+from . import types
+
+
+def is_image_file(path: str) -> bool:
+    basename, ext = os.path.splitext(os.path.basename(path))
+    return ext.lower() in (".jpg", ".jpeg", ".tif", ".tiff", ".pgm", ".pnm")
+
+
+def is_video_file(path: str) -> bool:
+    basename, ext = os.path.splitext(os.path.basename(path))
+    return ext.lower() in (".mp4", ".avi", ".tavi", ".mov", ".mkv")
+
+
+def iterate_files(root: str, recursive=False) -> Generator[str, None, None]:
+    for dirpath, dirnames, files in os.walk(root, topdown=True):
+        if not recursive:
+            dirnames.clear()
+        else:
+            dirnames[:] = [name for name in dirnames if not name.startswith(".")]
+        for file in files:
+            yield os.path.join(dirpath, file)
+
+
+def get_video_file_list(video_file, skip_subfolders=False) -> T.List[str]:
+    files = iterate_files(video_file, not skip_subfolders)
+    return sorted(file for file in files if is_video_file(file))
+
+
+def get_total_file_list(import_path: str, skip_subfolders: bool = False) -> List[str]:
+    files = iterate_files(import_path, not skip_subfolders)
+    return sorted(file for file in files if is_image_file(file))
+
+
+_IMAGE_STATE: T.Dict[str, T.Dict[types.Process, T.Tuple[types.Status, T.Mapping]]] = {}
+
+
+def _create_and_log_process_in_memory(
+    image: str,
+    process: types.Process,
+    status: types.Status,
+    description: T.Mapping,
+) -> None:
+    _IMAGE_STATE.setdefault(image, {})[process] = (status, description)
+
+
+def log_failed_in_memory(image: str, process: types.Process, exc: Exception):
+    desc: T.Dict = {
+        "type": exc.__class__.__name__,
+        "message": str(exc),
+    }
+    exc_vars = vars(exc)
+    if exc_vars:
+        desc["vars"] = exc_vars
+    return _create_and_log_process_in_memory(image, process, "failed", desc)
+
+
+def log_in_memory(image: str, process: types.Process, desc: T.Mapping):
+    return _create_and_log_process_in_memory(image, process, "success", desc)
+
+
+def read_process_data_from_memory(
+    image: str, process: types.Process
+) -> Optional[T.Tuple[types.Status, T.Mapping]]:
+    state = _IMAGE_STATE.get(image)
+    if state is None:
+        return None
+    return state.get(process)
diff --git a/mapillary_tools/insert_MAPJson.py b/mapillary_tools/insert_MAPJson.py
index 90b2f918..c5cd5226 100644
--- a/mapillary_tools/insert_MAPJson.py
+++ b/mapillary_tools/insert_MAPJson.py
@@ -1,103 +1,138 @@
+import typing as T
+import json
+import logging
 import os
-import sys
+import uuid
 
 from tqdm import tqdm
 
-from . import processing
-from . import uploader
-from .error import print_error
+from . import image_log, types, processing, error
+
+LOG = logging.getLogger(__name__)
+
+
+def get_final_mapillary_image_description(
+    image: str,
+) -> T.Optional[T.Tuple[types.Status, T.Mapping]]:
+    ret = image_log.read_process_data_from_memory(image, "geotag_process")
+    if ret is None:
+        return None
+
+    status, geotag_desc = ret
+    if status != "success":
+        return status, geotag_desc
+
+    ret = image_log.read_process_data_from_memory(image, "sequence_process")
+    if ret is None:
+        return None
+
+    status, sequence_desc = ret
+    if status != "success":
+        return status, sequence_desc
+
+    description: dict = {}
+    description.update(T.cast(dict, geotag_desc))
+    # sequence desc overrides the image desc
+    description.update(T.cast(dict, sequence_desc))
+
+    ret = image_log.read_process_data_from_memory(image, "import_meta_data_process")
+    if ret is not None:
+        status, meta_desc = ret
+        if status == "success":
+            description.update(T.cast(dict, meta_desc))
+
+    description["MAPPhotoUUID"] = str(uuid.uuid4())
+
+    return status, T.cast(types.FinalImageDescription, description)
 
 
 def insert_MAPJson(
     import_path,
-    master_upload=False,
-    verbose=False,
-    rerun=False,
     skip_subfolders=False,
-    skip_EXIF_insert=False,
-    keep_original=False,
-    video_import_path=None,
+    skip_process_errors=False,
     overwrite_all_EXIF_tags=False,
     overwrite_EXIF_time_tag=False,
     overwrite_EXIF_gps_tag=False,
     overwrite_EXIF_direction_tag=False,
     overwrite_EXIF_orientation_tag=False,
+    desc_path: str = None,
 ):
-    # sanity check if video file is passed
-    if (
-        video_import_path
-        and not os.path.isdir(video_import_path)
-        and not os.path.isfile(video_import_path)
-    ):
-        print(f"Error, video path {video_import_path} does not exist, exiting...")
-        sys.exit(1)
-
-    # in case of video processing, adjust the import path
-    if video_import_path:
-        # set sampling path
-        video_sampling_path = "mapillary_sampled_video_frames"
-        video_dirname = (
-            video_import_path
-            if os.path.isdir(video_import_path)
-            else os.path.dirname(video_import_path)
-        )
-        import_path = (
-            os.path.join(os.path.abspath(import_path), video_sampling_path)
-            if import_path
-            else os.path.join(os.path.abspath(video_dirname), video_sampling_path)
-        )
-
     # basic check for all
     if not import_path or not os.path.isdir(import_path):
-        print_error(f"Error, import directory {import_path} does not exist, exiting...")
-        sys.exit(1)
-
-    # get list of file to process
-    process_file_list = processing.get_process_file_list(
-        import_path,
-        "mapillary_image_description",
-        rerun=rerun,
-        skip_subfolders=skip_subfolders,
-    )
-    if not len(process_file_list):
-        print("No images to run process finalization")
-        print(
-            "If the images have already been processed and not yet uploaded, they can be processed again, by passing the argument --rerun"
+        raise RuntimeError(
+            f"Error, import directory {import_path} does not exist, exiting..."
         )
 
-    for image in tqdm(
-        process_file_list, desc="Inserting mapillary image description in image EXIF"
-    ):
-        # check the processing logs
-        log_root = uploader.log_rootpath(image)
+    if desc_path is None:
+        desc_path = os.path.join(import_path, "mapillary_image_description.json")
 
-        duplicate_path = os.path.join(log_root, "duplicate")
+    images = image_log.get_total_file_list(import_path, skip_subfolders=skip_subfolders)
 
-        if os.path.isfile(duplicate_path):
+    descs: T.List[types.FinalImageDescriptionOrError] = []
+    for image in tqdm(images, unit="files", desc="Processing image description"):
+        ret = get_final_mapillary_image_description(image)
+        if ret is None:
             continue
 
-        final_mapillary_image_description = (
-            processing.get_final_mapillary_image_description(
-                log_root,
-                image,
-                master_upload,
-                verbose,
-                skip_EXIF_insert,
-                keep_original,
-                overwrite_all_EXIF_tags,
-                overwrite_EXIF_time_tag,
-                overwrite_EXIF_gps_tag,
-                overwrite_EXIF_direction_tag,
-                overwrite_EXIF_orientation_tag,
+        status, desc = ret
+
+        if status == "success":
+            try:
+                processing.overwrite_exif_tags(
+                    image,
+                    T.cast(types.FinalImageDescription, desc),
+                    overwrite_all_EXIF_tags,
+                    overwrite_EXIF_time_tag,
+                    overwrite_EXIF_gps_tag,
+                    overwrite_EXIF_direction_tag,
+                    overwrite_EXIF_orientation_tag,
+                )
+            except Exception:
+                LOG.warning(f"Failed to overwrite EXIF", exc_info=True)
+
+        relpath = os.path.relpath(image, import_path)
+        if status == "success":
+            descs.append(
+                T.cast(types.FinalImageDescription, {**desc, "filename": relpath})
+            )
+        else:
+            descs.append(
+                T.cast(
+                    types.FinalImageDescriptionError,
+                    {"error": desc, "filename": relpath},
+                )
             )
-        )
 
-        processing.create_and_log_process(
-            image,
-            "mapillary_image_description",
-            "success",
-            final_mapillary_image_description,
-            verbose=verbose,
-        )
+    if desc_path == "-":
+        print(json.dumps(descs, indent=4))
+    else:
+        with open(desc_path, "w") as fp:
+            json.dump(descs, fp, indent=4)
+
+    processed_images = [desc for desc in descs if "error" not in desc]
+    not_processed_images = T.cast(
+        T.List[types.FinalImageDescriptionError],
+        [desc for desc in descs if "error" in desc],
+    )
+    duplicated_images = [
+        desc
+        for desc in not_processed_images
+        if desc["error"].get("type") == error.MapillaryDuplicationError.__name__
+    ]
 
-    print("Sub process ended")
+    summary = {
+        "total_images": len(descs),
+        "processed_images": len(processed_images),
+        "failed_images": len(not_processed_images) - len(duplicated_images),
+        "duplicated_images": len(duplicated_images),
+    }
+
+    LOG.info(json.dumps(summary, indent=4))
+    if 0 < summary["failed_images"]:
+        if skip_process_errors:
+            LOG.warning("Skipping %s failed images", summary["failed_images"])
+        else:
+            raise RuntimeError(
+                f"Failed to process {summary['failed_images']} images. Check {desc_path} for details. Specify --skip_process_errors to skip these errors"
+            )
+    LOG.info(f"Check {desc_path} for details")
diff --git a/mapillary_tools/interpolation.py b/mapillary_tools/interpolation.py
deleted file mode 100644
index ff1bf0e6..00000000
--- a/mapillary_tools/interpolation.py
+++ /dev/null
@@ -1,238 +0,0 @@
-import csv
-import datetime
-import os
-import sys
-
-from tqdm import tqdm
-
-from . import process_csv
-from . import processing
-from . import uploader
-from .error import print_error
-from .exif_read import ExifRead
-from .exif_write import ExifEdit
-from .geo import interpolate_lat_lon
-from .process_import_meta_properties import add_meta_tag
-
-EPOCH = datetime.datetime.utcfromtimestamp(0)
-
-
-def format_datetime(
-    timestamps_interpolated, time_utc=False, time_format="%Y-%m-%dT%H:%M:%SZ"
-):
-    if time_utc:
-        try:
-            timestamps_formated = [
-                int((datetime_timestamp - EPOCH).total_seconds() * 1000.0)
-                for datetime_timestamp in timestamps_interpolated
-            ]
-        except:
-            print("Formating timestamps from datetime to UTC failed...")
-            sys.exit(1)
-    else:
-        try:
-            timestamps_formated = [
-                datetime_timestamp.strftime(time_format)
-                for datetime_timestamp in timestamps_interpolated
-            ]
-        except:
-            print(
-                f"Formating timestamps from datetime to time format {time_format} failed..."
-            )
-            sys.exit(1)
-    return timestamps_formated
-
-
-def interpolation(
-    data,
-    file_in_path=None,
-    file_format="csv",
-    time_column=0,
-    delimiter=",",
-    time_utc=False,
-    time_format="%Y-%m-%dT%H:%M:%SZ",
-    header=False,
-    keep_original=False,
-    import_path=None,
-    max_time_delta=1,
-    verbose=False,
-):
-    if not data:
-        print_error(
-            "Error, you must specify the data for interpolation."
-            + 'Choose between "missing_gps" or "identical_timestamps"'
-        )
-        sys.exit(1)
-
-    if not import_path and not file_in_path:
-        print_error(
-            "Error, you must specify a path to data, either path to directory with images or path to an external log file."
-        )
-        sys.exit(1)
-
-    if file_in_path:
-        if not os.path.isfile(file_in_path):
-            print_error("Error, specified input file does not exist, exiting...")
-            sys.exit(1)
-        if file_format != "csv":
-            print_error("Only csv file format is supported at the moment, exiting...")
-            sys.exit(1)
-
-        csv_data = process_csv.read_csv(
-            file_in_path, delimiter=delimiter, header=header
-        )
-
-        if data == "identical_timestamps":
-            timestamps = csv_data[time_column]
-            timestamps_datetime = [
-                process_csv.format_time(timestamp, time_utc, time_format)
-                for timestamp in timestamps
-            ]
-
-            timestamps_interpolated = processing.interpolate_timestamp(
-                timestamps_datetime
-            )
-
-            csv_data[time_column] = format_datetime(
-                timestamps_interpolated, time_utc, time_format
-            )
-
-            file_out = (
-                file_in_path
-                if not keep_original
-                else file_in_path[:-4] + "_processed." + file_format
-            )
-
-            with open(file_out, "w") as csvfile:
-                csvwriter = csv.writer(csvfile, delimiter=delimiter)
-                for row in zip(*csv_data):
-                    csvwriter.writerow(row)
-            sys.exit()
-        elif data == "missing_gps":
-            print_error(
-                "Error, missing gps interpolation in an external log file not supported yet, exiting..."
-            )
-            sys.exit(1)
-        else:
-            print_error("Error unsupported data for interpolation, exiting...")
-            sys.exit(1)
-
-    if import_path:
-        if not os.path.isdir(import_path):
-            print_error("Error, specified import path does not exist, exiting...")
-            sys.exit(1)
-
-        # get list of files to process
-        process_file_list = uploader.get_total_file_list(import_path)
-        if not len(process_file_list):
-            print("No images found in the import path " + import_path)
-            sys.exit(1)
-
-        if data == "missing_gps":
-            # get geotags from images and a list of tuples with images missing geotags
-            # and their timestamp
-            geotags, missing_geotags = processing.get_images_geotags(process_file_list)
-            if not len(missing_geotags):
-                print(
-                    f"No images in directory {import_path} missing geotags, exiting..."
-                )
-                sys.exit(1)
-            if not len(geotags):
-                print(f"No images in directory {import_path} with geotags.")
-                sys.exit(1)
-
-            sys.stdout.write(
-                f"Interpolating gps for {len(missing_geotags)} images missing geotags."
-            )
-
-            for image, timestamp in tqdm(
-                missing_geotags, desc="Interpolating missing gps"
-            ):
-                # interpolate
-                try:
-                    lat, lon, bearing, elevation = interpolate_lat_lon(
-                        geotags, timestamp, max_time_delta
-                    )
-                except Exception as e:
-                    print_error(
-                        f"Error, {e}, interpolation of latitude and longitude failed for image {image}"
-                    )
-                    continue
-                # insert into exif
-                exif_edit = ExifEdit(image)
-                if lat and lon:
-                    exif_edit.add_lat_lon(lat, lon)
-                else:
-                    print_error(
-                        f"Error, lat and lon not interpolated for image {image}."
-                    )
-                if bearing:
-                    exif_edit.add_direction(bearing)
-                else:
-                    if verbose:
-                        print(f"Warning, bearing not interpolated for image {image}.")
-                if elevation:
-                    exif_edit.add_altitude(elevation)
-                else:
-                    if verbose:
-                        print(f"Warning, altitude not interpolated for image {image}.")
-
-                meta = {}
-
-                add_meta_tag(meta, "booleans", "interpolated_gps", True)
-
-                exif_edit.add_image_history(meta["MAPMetaTags"])
-
-                file_out = image if not keep_original else image[:-4] + "_processed."
-                exif_edit.write(filename=file_out)
-
-        elif data == "identical_timestamps":
-
-            sys.stdout.write("Loading image timestamps.")
-
-            # read timestamps
-            timestamps = []
-            for image in tqdm(
-                process_file_list, desc="Interpolating identical timestamps"
-            ):
-
-                # load exif
-                exif = ExifRead(image)
-                timestamp = exif.extract_capture_time()
-                if timestamp:
-                    timestamps.append(timestamp)
-                else:
-                    print(f"Capture could not be extracted for image {image}.")
-
-            # interpolate
-            timestamps_interpolated = processing.interpolate_timestamp(timestamps)
-
-            print("")
-            sys.stdout.write("Interpolating identical timestamps.")
-            counter = 0
-
-            # write back
-            for image, timestamp in tqdm(
-                zip(process_file_list, timestamps_interpolated),
-                desc="Writing capture time in image EXIF",
-            ):
-
-                # print progress
-                counter += 1
-                sys.stdout.write(".")
-                if (counter % 100) == 0:
-                    print("")
-
-                # load exif
-                exif_edit = ExifEdit(image)
-                exif_edit.add_date_time_original(timestamp)
-
-                # write to exif
-                file_out = image if not keep_original else image[:-4] + "_processed."
-                exif_edit.write(filename=file_out)
-
-            sys.exit()
-        else:
-            print_error("Error unsupported data for interpolation, exiting...")
-            sys.exit(1)
-    print("")
diff --git a/mapillary_tools/ipc.py b/mapillary_tools/ipc.py
index 25fe16a2..098ac55e 100644
--- a/mapillary_tools/ipc.py
+++ b/mapillary_tools/ipc.py
@@ -1,9 +1,13 @@
 import json
 import os
 import struct
+import logging
 
+
+LOG = logging.getLogger(__name__)
 NODE_CHANNEL_FD = int(os.getenv("NODE_CHANNEL_FD", -1))
 
+
 if NODE_CHANNEL_FD == -1:
 
     def __write(obj):
@@ -31,10 +35,6 @@ def __write(obj):
         os.write(NODE_CHANNEL_FD, data.encode("utf-8"))
 
 
-def is_enabled():
-    return NODE_CHANNEL_FD != -1
-
-
 def send(type, payload):
     obj = {
         "type": type,
@@ -42,10 +42,5 @@ def send(type, payload):
     }
     try:
         __write(obj)
-    except Exception as e:
-        print(f"IPC error for: {obj}")
-        print(f"Error: {e}")
-
-
-def send_error(message):
-    send("error", {"message": message})
+    except Exception:
+        LOG.warning(f"IPC error sending: {obj}", exc_info=True)
diff --git a/mapillary_tools/login.py b/mapillary_tools/login.py
index f4160f16..68ca27fd 100644
--- a/mapillary_tools/login.py
+++ b/mapillary_tools/login.py
@@ -1,11 +1,14 @@
 import getpass
+import logging
 import os
+import typing as T
 
 import requests
 
-from . import api_v4, config
+from . import api_v4, config, types
 from .config import GLOBAL_CONFIG_FILEPATH
-from .error import print_error
+
+LOG = logging.getLogger(__name__)
 
 
 class HTTPError(Exception):
@@ -22,15 +25,13 @@ def wrap_http_exception(ex: requests.HTTPError):
     return HTTPError("\n".join(lines))
 
 
-def prompt_user_for_user_items(user_name: str) -> dict:
+def prompt_user_for_user_items(user_name: str) -> types.User:
     print(f"Sign in for user {user_name}")
     user_email = input("Enter your Mapillary user email: ")
     user_password = getpass.getpass("Enter Mapillary user password: ")
 
     try:
         data = api_v4.get_upload_token(user_email, user_password)
-        upload_token = data.get("access_token")
-        user_key = data.get("user_id")
     except requests.HTTPError as ex:
         if 400 <= ex.response.status_code < 500:
             resp = ex.response.json()
@@ -38,13 +39,23 @@ def prompt_user_for_user_items(user_name: str) -> dict:
             if subcode in [1348028, 1348092, 3404005]:
                 title = resp.get("error", {}).get("error_user_title")
                 message = resp.get("error", {}).get("error_user_msg")
-                print_error(f"{title}: {message}")
+                LOG.error(f"{title}: {message}")
                 return prompt_user_for_user_items(user_name)
             else:
                 raise wrap_http_exception(ex)
         else:
             raise wrap_http_exception(ex)
 
+    upload_token = T.cast(str, data.get("access_token"))
+    user_key = T.cast(str, data.get("user_id"))
+    if not isinstance(upload_token, str) or not isinstance(user_key, (str, int)):
+        raise RuntimeError(
+            f"Error extracting user_key or token from the login response: {data}"
+        )
+
+    if isinstance(user_key, int):
+        user_key = str(user_key)
+
     return {
         "MAPSettingsUsername": user_name,
         "MAPSettingsUserKey": user_key,
@@ -52,7 +63,18 @@ def prompt_user_for_user_items(user_name: str) -> dict:
     }
 
 
-def authenticate_user(user_name: str) -> dict:
+def list_all_users() -> T.List[types.User]:
+    if os.path.isfile(GLOBAL_CONFIG_FILEPATH):
+        global_config_object = config.load_config(GLOBAL_CONFIG_FILEPATH)
+        return [
+            config.load_user(global_config_object, user_name)
+            for user_name in global_config_object.sections()
+        ]
+    else:
+        return []
+
+
+def authenticate_user(user_name: str) -> types.User:
     if os.path.isfile(GLOBAL_CONFIG_FILEPATH):
         global_config_object = config.load_config(GLOBAL_CONFIG_FILEPATH)
         if user_name in global_config_object.sections():
diff --git a/mapillary_tools/post_process.py b/mapillary_tools/post_process.py
deleted file mode 100644
index dfd37e0f..00000000
--- a/mapillary_tools/post_process.py
+++ /dev/null
@@ -1,350 +0,0 @@
-import csv
-import json
-import os
-import sys
-
-from tqdm import tqdm
-
-from . import exif_read
-from . import ipc
-from . import processing
-from . import uploader
-
-
-def map_images_to_sequences(destination_mapping, total_files):
-    unique_sequence_uuids = []
-    sequence_counter = 0
-    for image in tqdm(
-        total_files, desc="Reading sequence information stored in log files"
-    ):
-        log_root = uploader.log_rootpath(image)
-        sequence_data_path = os.path.join(log_root, "sequence_process.json")
-
-        if os.path.isfile(sequence_data_path):
-            sequence_data = processing.load_json(sequence_data_path)
-        else:
-            sequence_data = None
-
-        if sequence_data and "MAPSequenceUUID" in sequence_data:
-            sequence_uuid = sequence_data["MAPSequenceUUID"]
-        else:
-            sequence_uuid = ""
-
-        if sequence_uuid:
-            if sequence_uuid not in unique_sequence_uuids:
-                sequence_counter += 1
-                unique_sequence_uuids.append(sequence_uuid)
-            if image in destination_mapping:
-                destination_mapping[image]["sequence"] = str(sequence_counter)
-            else:
-                destination_mapping[image] = {"sequence": str(sequence_counter)}
-        else:
-            print(f"MAPSequenceUUID could not be read for image {image}")
-
-    return destination_mapping
-
-
-def get_local_mapping(import_path):
-    total_files = uploader.get_total_file_list(import_path)
-
-    local_mapping = []
-    for file in tqdm(total_files, desc="Reading image uuids"):
-        image_file_uuid = None
-        relative_path = file.lstrip(os.path.abspath(import_path))
-        log_rootpath = uploader.log_rootpath(file)
-        image_description_json_path = os.path.join(
-            log_rootpath, "mapillary_image_description.json"
-        )
-        if os.path.isfile(image_description_json_path):
-            image_description_json = processing.load_json(image_description_json_path)
-            if "MAPPhotoUUID" in image_description_json:
-                image_file_uuid = image_description_json["MAPPhotoUUID"]
-            else:
-                print(
-                    "Error, photo uuid not in mapillary_image_description.json log file."
-                )
-        else:
-            image_exif = exif_read.ExifRead(file)
-            description = image_exif.extract_image_description()
-            if description is None:
-                description = ""
-            try:
-                image_description = json.loads(description)
-            except json.JSONDecodeError:
-                print(
-                    f"Warning, failed to JSON decoding image description {description} from {file}",
-                )
-                image_description = {}
-            if (
-                isinstance(image_description, dict)
-                and "MAPPhotoUUID" in image_description
-            ):
-                image_file_uuid = str(image_description["MAPPhotoUUID"])
-            else:
-                print(
-                    f"Warning, image {file} EXIF does not contain mapillary image description and mapillary_image_description.json log file does not exist. Try to process the image using mapillary_tools."
-                )
-        if image_file_uuid:
-            local_mapping.append((relative_path, image_file_uuid))
-    return local_mapping
-
-
-def post_process(
-    import_path,
-    split_import_path=None,
-    video_import_path=None,
-    summarize=False,
-    move_all_images=False,
-    move_duplicates=False,
-    move_uploaded=False,
-    move_sequences=False,
-    save_as_json=False,
-    list_file_status=False,
-    push_images=False,
-    skip_subfolders=False,
-    verbose=False,
-    save_local_mapping=False,
-):
-    # return if nothing specified
-    if not any(
-        [
-            summarize,
-            move_all_images,
-            list_file_status,
-            push_images,
-            move_duplicates,
-            move_uploaded,
-            save_local_mapping,
-            move_sequences,
-        ]
-    ):
-        print("No post processing action specified.")
-        return
-
-    # sanity check if video file is passed
-    if (
-        video_import_path
-        and not os.path.isdir(video_import_path)
-        and not os.path.isfile(video_import_path)
-    ):
-        print("Error, video path " + video_import_path + " does not exist, exiting...")
-        sys.exit(1)
-    if move_all_images:
-        move_sequences = True
-        move_duplicates = True
-        move_uploaded = True
-    # in case of video processing, adjust the import path
-    if video_import_path:
-        # set sampling path
-        video_sampling_path = "mapillary_sampled_video_frames"
-        video_dirname = (
-            video_import_path
-            if os.path.isdir(video_import_path)
-            else os.path.dirname(video_import_path)
-        )
-        import_path = (
-            os.path.join(os.path.abspath(import_path), video_sampling_path)
-            if import_path
-            else os.path.join(os.path.abspath(video_dirname), video_sampling_path)
-        )
-
-    # basic check for all
-    if not import_path or not os.path.isdir(import_path):
-        print("Error, import directory " + import_path + " does not exist, exiting...")
-        sys.exit(1)
-
-    if save_local_mapping:
-        local_mapping = get_local_mapping(import_path)
-        local_mapping_filepath = os.path.join(
-            os.path.dirname(import_path),
-            os.path.basename(import_path)
-            + "_mapillary_image_uuid_to_local_path_mapping.csv",
-        )
-        with open(local_mapping_filepath, "w") as csvfile:
-            csvwriter = csv.writer(csvfile, delimiter=",")
-            for row in local_mapping:
-                csvwriter.writerow(row)
-
-    if push_images:
-        to_be_pushed_files = uploader.get_success_only_manual_upload_file_list(
-            import_path, skip_subfolders
-        )
-        params = {}
-        for image in tqdm(to_be_pushed_files, desc="Pushing images"):
-            log_root = uploader.log_rootpath(image)
-            upload_params_path = os.path.join(log_root, "upload_params_process.json")
-            if os.path.isfile(upload_params_path):
-                with open(upload_params_path, "rb") as jf:
-                    params[image] = json.load(jf)
-
-        # flag finalization for each file
-        uploader.flag_finalization(to_be_pushed_files)
-
-    if summarize or list_file_status or move_uploaded:
-        # upload logs
-        uploaded_files = uploader.get_success_upload_file_list(
-            import_path, skip_subfolders
-        )
-        uploaded_files_count = len(uploaded_files)
-        failed_upload_files = uploader.get_failed_upload_file_list(
-            import_path, skip_subfolders
-        )
-        failed_upload_files_count = len(failed_upload_files)
-        to_be_finalized_files = uploader.get_finalize_file_list(import_path)
-        to_be_finalized_files_count = len(to_be_finalized_files)
-        to_be_uploaded_files = uploader.get_upload_file_list(
-            import_path, skip_subfolders
-        )
-        to_be_uploaded_files_count = len(to_be_uploaded_files)
-
-    if summarize or move_sequences:
-        total_files = uploader.get_total_file_list(import_path)
-        total_files_count = len(total_files)
-
-    if summarize or move_duplicates or list_file_status:
-        duplicates_file_list = processing.get_duplicate_file_list(
-            import_path, skip_subfolders
-        )
-        duplicates_file_list_count = len(duplicates_file_list)
-
-    if summarize:
-        summary_dict = {
-            "total images": total_files_count,
-            "upload summary": {
-                "successfully uploaded": uploaded_files_count,
-                "failed uploads": failed_upload_files_count,
-                "uploaded to be finalized": to_be_finalized_files_count,
-            },
-            "process summary": {},
-        }
-        # process logs
-        process_steps = [
-            "user_process",
-            "import_meta_process",
-            "geotag_process",
-            "sequence_process",
-            "upload_params_process",
-            "mapillary_image_description",
-        ]
-        for step in process_steps:
-            process_success = len(
-                processing.get_process_status_file_list(
-                    import_path, step, "success", skip_subfolders
-                )
-            )
-            process_failed = len(
-                processing.get_process_status_file_list(
-                    import_path, step, "failed", skip_subfolders
-                )
-            )
-            summary_dict["process summary"][step] = {
-                "failed": process_failed,
-                "success": process_success,
-            }
-        summary_dict["process summary"]["duplicates"] = duplicates_file_list_count
-        summary_dict["process summary"][
-            "processed_not_yet_uploaded"
-        ] = to_be_uploaded_files_count
-        print(f"Import summary for import path {import_path}:")
-        print(json.dumps(summary_dict, indent=4))
-
-        ipc.send("summary", summary_dict)
-
-        if save_as_json:
-            try:
-                processing.save_json(
-                    summary_dict,
-                    os.path.join(import_path, "mapillary_import_summary.json"),
-                )
-            except Exception as e:
-                print(
-                    f"Could not save summary into json at {os.path.join(import_path, 'mapillary_import_summary.json')}, due to {e}"
-                )
-
-    if list_file_status:
-        status_list_dict = {
-            "successfully uploaded": uploaded_files,
-            "failed uploads": failed_upload_files,
-            "uploaded to be finalized": to_be_finalized_files,
-            "duplicates": duplicates_file_list,
-            "processed_not_yet_uploaded": to_be_uploaded_files,
-        }
-        print("")
-        print(f"List of file status for import path {import_path} :")
-        print(json.dumps(status_list_dict, indent=4))
-        if save_as_json:
-            try:
-                processing.save_json(
-                    status_list_dict,
-                    os.path.join(
-                        import_path, "mapillary_import_image_status_list.json"
-                    ),
-                )
-            except Exception as e:
-                print(
-                    f"Could not save image status list into json at {os.path.join(import_path, 'mapillary_import_image_status_list.json')}, due to {e}"
-                )
-    split_import_path = split_import_path if split_import_path else import_path
-    if move_sequences or move_duplicates or move_uploaded:
-        if not os.path.isdir(split_import_path):
-            print(f"Split import path {split_import_path} does not exist.")
-            sys.exit(1)
-
-    destination_mapping = {}
-    if move_duplicates:
-        for image in duplicates_file_list:
-            destination_mapping[image] = {"basic": ["duplicates"]}
-    if move_uploaded:
-        for image in uploaded_files:
-            if image in destination_mapping:
-                destination_mapping[image]["basic"].append("uploaded")
-            else:
-                destination_mapping[image] = {"basic": ["uploaded"]}
-        for image in failed_upload_files:
-            if image in destination_mapping:
-                destination_mapping[image]["basic"].append("failed_upload")
-            else:
-                destination_mapping[image] = {"basic": ["failed_upload"]}
-        for image in to_be_finalized_files:
-            if image in destination_mapping:
-                destination_mapping[image]["basic"].append("uploaded_not_finalized")
-            else:
-                destination_mapping[image] = {"basic": ["uploaded_not_finalized"]}
-        for image in to_be_uploaded_files:
-            if image in destination_mapping:
-                destination_mapping[image]["basic"].append("to_be_uploaded")
-            else:
-                destination_mapping[image] = {"basic": ["to_be_uploaded"]}
-
-    if move_sequences:
-        destination_mapping = map_images_to_sequences(destination_mapping, total_files)
-
-    for image in destination_mapping:
-        basic_destination = (
-            destination_mapping[image]["basic"]
-            if "basic" in destination_mapping[image]
-            else []
-        )
-        sequence_destination = (
-            destination_mapping[image]["sequence"]
-            if "sequence" in destination_mapping[image]
-            else ""
-        )
-        image_destination_path = os.path.join(
-            *(
-                [split_import_path]
-                + basic_destination
-                + [os.path.dirname(image[len(os.path.abspath(import_path)) + 1 :])]
-                + [sequence_destination, os.path.basename(image)]
-            )
-        )
-        if not os.path.isdir(os.path.dirname(image_destination_path)):
-            os.makedirs(os.path.dirname(image_destination_path))
-        os.rename(image, image_destination_path)
-        image_logs_dir = uploader.log_rootpath(image)
-        destination_logs_dir = uploader.log_rootpath(image_destination_path)
-        if not os.path.isdir(image_logs_dir):
-            continue
-        if not os.path.isdir(os.path.dirname(destination_logs_dir)):
-            os.makedirs(os.path.dirname(destination_logs_dir))
-        os.rename(image_logs_dir, destination_logs_dir)
diff --git a/mapillary_tools/process_csv.py b/mapillary_tools/process_csv.py
deleted file mode 100644
index b379aa09..00000000
--- a/mapillary_tools/process_csv.py
+++ /dev/null
@@ -1,333 +0,0 @@
-import csv
-import datetime
-import os
-
-from tqdm import tqdm
-
-from . import process_import_meta_properties
-from . import processing
-from . import uploader
-from .exif_write import ExifEdit
-
-META_DATA_TYPES = ["string", "double", "long", "date", "boolean"]
-
-MILLISECONDS_PRECISION_CUT_OFF = 10000000000
-
-GPS_START = datetime.datetime(1980, 1, 6)
-
-SECS_IN_WEEK = 604800
-
-
-def format_time(timestamp, time_utc=False, time_format="%Y-%m-%dT%H:%M:%SZ"):
-    if time_utc:
-        division = 1.0 if float(timestamp) < MILLISECONDS_PRECISION_CUT_OFF else 1000.0
-        t_datetime = datetime.datetime.utcfromtimestamp(float(timestamp) / division)
-    else:
-        t_datetime = datetime.datetime.strptime(timestamp, time_format)
-    return t_datetime
-
-
-def validate_meta_data(meta_columns, meta_names, meta_types):
-    if any([x for x in [meta_columns, meta_names, meta_types]]):
-
-        # if any of the meta data arguments are passed all must be
-        if any([not x for x in [meta_columns, meta_names, meta_types]]):
-            raise RuntimeError(
-                "Error, if extracting meta data you need to specify meta_columns, meta_names and meta_types."
-            )
-
-        # get meta data column numbers
-        meta_columns = meta_columns.split(",")
-        try:
-            meta_columns = [int(field) - 1 for field in meta_columns]
-        except:
-            raise RuntimeError(
-                'Error, meta column numbers could not be extracted. Meta column numbers need to be separated with commas, example "7,9,10"'
-            )
-
-        # get meta data names and types
-        meta_names = meta_names.split(",")
-        meta_types = meta_types.split(",")
-
-        # exit if they are not all of same length
-        if len(meta_columns) != len(meta_names) or len(meta_types) != len(meta_names):
-            raise RuntimeError(
-                "Error, number of meta data column numbers, types and names must be the same."
-            )
-
-        # check if types are valid
-        for meta_type in meta_types:
-            if meta_type not in META_DATA_TYPES:
-                raise RuntimeError(
-                    "Error, invalid meta data type, valid types are "
-                    + str(META_DATA_TYPES)
-                )
-    return meta_columns, meta_names, meta_types
-
-
-def convert_from_gps_time(gps_time, gps_week=None):
-    """Convert gps time in ticks to standard time."""
-
-    gps_timestamp = float(gps_time)
-
-    if gps_week is not None:
-        # image date
-        converted_gps_time = GPS_START + datetime.timedelta(
-            seconds=int(gps_week) * SECS_IN_WEEK + gps_timestamp
-        )
-
-    else:
-        # TAI scale with 1970-01-01 00:00:10 (TAI) epoch
-        os.environ["TZ"] = "right/UTC"
-
-        # by definition
-        gps_time_as_gps = GPS_START + datetime.timedelta(seconds=gps_timestamp)
-
-        # constant offset
-        gps_time_as_tai = gps_time_as_gps + datetime.timedelta(seconds=19)
-        tai_epoch_as_tai = datetime.datetime(1970, 1, 1, 0, 0, 10)
-
-        # by definition
-        tai_timestamp = (gps_time_as_tai - tai_epoch_as_tai).total_seconds()
-
-        converted_gps_time = datetime.datetime.utcfromtimestamp(tai_timestamp)
-
-    # "right" timezone is in effect
-    return converted_gps_time
-
-
-def get_image_index(image, file_names):
-    image_index = None
-    try:
-        image_index = file_names.index(image)
-    except:
-        try:
-            file_names = [os.path.basename(entry) for entry in file_names]
-            image_index = file_names.index(os.path.basename(image))
-        except:
-            try:
-                image_index = [
-                    idx
-                    for idx, file_name in enumerate(file_names)
-                    if file_name in image
-                ][0]
-            except:
-                try:
-                    image_index = [
-                        idx
-                        for idx, file_name in enumerate(file_names)
-                        if ".".join(file_name.split(".")[:-1]) in image
-                    ][0]
-                except:
-                    pass
-    return image_index
-
-
-def parse_csv_geotag_data(
-    csv_data,
-    image_index,
-    column_indexes,
-    convert_gps_time=False,
-    convert_utc_time=False,
-    time_format="%Y:%m:%d %H:%M:%S.%f",
-):
-    timestamp = None
-    lat = None
-    lon = None
-    heading = None
-    altitude = None
-
-    timestamp_column = column_indexes[1]
-    latitude_column = column_indexes[2]
-    longitude_column = column_indexes[3]
-    heading_column = column_indexes[4]
-    altitude_column = column_indexes[5]
-    gps_week_column = column_indexes[6]
-
-    if timestamp_column is not None:
-        timestamp = csv_data[timestamp_column][image_index]
-        gps_week = None
-        if gps_week_column is not None:
-            gps_week = csv_data[gps_week_column][image_index]
-        timestamp = (
-            convert_from_gps_time(timestamp, gps_week)
-            if convert_gps_time
-            else format_time(timestamp, convert_utc_time, time_format)
-        )
-
-    if latitude_column is not None:
-        lat = float(csv_data[latitude_column][image_index])
-    if longitude_column is not None:
-        lon = float(csv_data[longitude_column][image_index])
-    if heading_column is not None:
-        heading = float(csv_data[heading_column][image_index])
-    if altitude_column is not None:
-        altitude = float(csv_data[altitude_column][image_index])
-
-    return timestamp, lat, lon, heading, altitude
-
-
-def parse_csv_meta_data(csv_data, image_index, meta_columns, meta_types, meta_names):
-    meta = {}
-    if meta_columns:
-        for field, meta_field in enumerate(meta_columns):
-            try:
-                tag_type = meta_types[field] + "s"
-                tag_value = csv_data[meta_field][image_index]
-                tag_key = meta_names[field]
-                process_import_meta_properties.add_meta_tag(
-                    meta, tag_type, tag_key, tag_value
-                )
-            except:
-                print(f"Error, meta data {tag_key} could not be extracted.")
-    return meta
-
-
-def read_csv(csv_path: str, delimiter=",", header=False) -> list:
-    with open(csv_path, "r") as csvfile:
-        csvreader = csv.reader(csvfile, delimiter=delimiter)
-        if header:
-            next(csvreader, None)
-
-        csv_data = list(zip(*csvreader))
-    return csv_data
-
-
-def process_csv(
-    import_path,
-    csv_path,
-    filename_column=None,
-    timestamp_column=None,
-    latitude_column=None,
-    longitude_column=None,
-    heading_column=None,
-    altitude_column=None,
-    gps_week_column=None,
-    time_format="%Y:%m:%d %H:%M:%S.%f",
-    convert_gps_time=False,
-    convert_utc_time=False,
-    delimiter=",",
-    header=False,
-    meta_columns=None,
-    meta_names=None,
-    meta_types=None,
-    verbose=False,
-    keep_original=False,
-):
-    # sanity checks
-    if not import_path or not os.path.isdir(import_path):
-        raise RuntimeError(
-            f"Error, import directory {import_path} doesnt not exist, exiting..."
-        )
-
-    if not csv_path or not os.path.isfile(csv_path):
-        raise RuntimeError(
-            "Error, csv file not provided or does not exist. Please specify a valid path to a csv file."
-        )
-
-    # get list of file to process
-    process_file_list = uploader.get_total_file_list(import_path)
-    if not len(process_file_list):
-        raise RuntimeError("No images found in the import path " + import_path)
-
-    if gps_week_column is not None and not convert_gps_time:
-        raise RuntimeError(
-            "Error, in order to parse timestamp provided as a combination of GPS week and GPS seconds, you must specify timestamp column and flag --convert_gps_time, exiting..."
-        )
-
-    if (not convert_gps_time or not convert_utc_time) and timestamp_column is None:
-        raise RuntimeError(
-            "Error, if specifying a flag to convert timestamp, timestamp column must be provided, exiting..."
-        )
-
-    column_indexes = [
-        filename_column,
-        timestamp_column,
-        latitude_column,
-        longitude_column,
-        heading_column,
-        altitude_column,
-        gps_week_column,
-    ]
-
-    if any(column == 0 for column in column_indexes):
-        raise RuntimeError(
-            "Error, csv column numbers start with 1, one of the columns specified is 0."
-        )
-
-    column_indexes = [col - 1 if col is not None else None for col in column_indexes]
-
-    # checks for meta arguments if any
-    meta_columns, meta_names, meta_types = validate_meta_data(
-        meta_columns, meta_names, meta_types
-    )
-
-    # open and process csv
-    csv_data = read_csv(csv_path, delimiter=delimiter, header=header)
-
-    # align by filename column if provided, otherwise align in order of image
-    # names
-    file_names = None
-    if filename_column:
-        file_names = csv_data[filename_column - 1]
-    else:
-        if verbose:
-            print(
-                "Warning, filename column not provided, images will be aligned with the csv data in order of the image filenames."
-            )
-
-    # process each image
-    for idx, image in tqdm(
-        enumerate(process_file_list), desc="Inserting csv data in image EXIF"
-    ):
-
-        # get image entry index
-        image_index = get_image_index(image, file_names) if file_names else idx
-        if image_index is None:
-            print(f"Warning, no entry found in csv file for image {image}")
-            continue
-
-        # get required data
-        timestamp, lat, lon, heading, altitude = parse_csv_geotag_data(
-            csv_data,
-            image_index,
-            column_indexes,
-            convert_gps_time,
-            convert_utc_time,
-            time_format,
-        )
-
-        # get meta data
-        meta = parse_csv_meta_data(
-            csv_data, image_index, meta_columns, meta_types, meta_names
-        )
-
-        # insert in image EXIF
-        exif_edit = ExifEdit(image)
-        if timestamp:
-            exif_edit.add_date_time_original(timestamp)
-        if lat and lon:
-            exif_edit.add_lat_lon(lat, lon)
-        if heading:
-            exif_edit.add_direction(heading)
-        if altitude:
-            exif_edit.add_altitude(altitude)
-        if meta:
-            exif_edit.add_image_history(meta["MAPMetaTags"])
-
-        filename = image
-        filename_keep_original = processing.processed_images_rootpath(image)
-
-        if os.path.isfile(filename_keep_original):
-            os.remove(filename_keep_original)
-
-        if keep_original:
-            if not os.path.isdir(os.path.dirname(filename_keep_original)):
-                os.makedirs(os.path.dirname(filename_keep_original))
-            filename = filename_keep_original
-
-        try:
-            exif_edit.write(filename=filename)
-        except:
-            print(f"Error, image EXIF could not be written back for image {image}")
-            return None
diff --git a/mapillary_tools/process_geotag_properties.py b/mapillary_tools/process_geotag_properties.py
index 5996db58..42a8035a 100644
--- a/mapillary_tools/process_geotag_properties.py
+++ b/mapillary_tools/process_geotag_properties.py
@@ -1,105 +1,77 @@
 import os
-import sys
+import typing as T
 
+from . import image_log
 from . import processing
-from .error import print_error
 
 
 def process_geotag_properties(
-    import_path,
+    import_path: T.Optional[str] = None,
+    video_import_path: T.Optional[str] = None,
     geotag_source="exif",
-    geotag_source_path=None,
+    geotag_source_path: T.Optional[str] = None,
     offset_time=0.0,
     offset_angle=0.0,
-    local_time=False,
-    sub_second_interval=0.0,
-    use_gps_start_time=False,
-    verbose=False,
-    rerun=False,
     skip_subfolders=False,
-    video_import_path=None,
-):
-    # sanity check if video file is passed
-    if (
-        video_import_path
-        and not os.path.isdir(video_import_path)
-        and not os.path.isfile(video_import_path)
-    ):
-        print("Error, video path " + video_import_path + " does not exist, exiting...")
-        sys.exit(1)
-
-    # in case of video processing, adjust the import path
-    if video_import_path:
-        # set sampling path
-        video_sampling_path = "mapillary_sampled_video_frames"
-        video_dirname = (
-            video_import_path
-            if os.path.isdir(video_import_path)
-            else os.path.dirname(video_import_path)
-        )
-        import_path = (
-            os.path.join(os.path.abspath(import_path), video_sampling_path)
-            if import_path
-            else os.path.join(os.path.abspath(video_dirname), video_sampling_path)
-        )
-
-    # basic check for all
+) -> None:
     if not import_path or not os.path.isdir(import_path):
         raise RuntimeError(
             f"Error, import directory {import_path} does not exist, exiting..."
         )
 
-    # get list of file to process
-    process_file_list = processing.get_process_file_list(
-        import_path, "geotag_process", rerun=rerun, skip_subfolders=skip_subfolders
+    process_file_list = image_log.get_total_file_list(
+        import_path,
+        skip_subfolders=skip_subfolders,
     )
+    if not process_file_list:
+        return
 
-    if not len(process_file_list):
-        print("No images to run geotag process")
-        print(
-            "If the images have already been processed and not yet uploaded, they can be processed again, by passing the argument --rerun"
-        )
-
-    # function calls
     if geotag_source == "exif":
-        processing.geotag_from_exif(
-            process_file_list, import_path, offset_time, offset_angle, verbose
-        )
+        return processing.geotag_from_exif(process_file_list, offset_time, offset_angle)
 
-    elif geotag_source == "gpx" or geotag_source == "nmea":
-        processing.geotag_from_gps_trace(
+    elif geotag_source == "gpx":
+        if geotag_source_path is None:
+            raise RuntimeError(
+                "GPX file is required to be specified with --geotag_source_path"
+            )
+        return processing.geotag_from_gpx_file(
+            process_file_list,
+            geotag_source_path,
+            offset_time=offset_time,
+            offset_angle=offset_angle,
+        )
+    elif geotag_source == "nmea":
+        if geotag_source_path is None:
+            raise RuntimeError(
+                "NMEA file is required to be specified with --geotag_source_path"
+            )
+        return processing.geotag_from_nmea_file(
             process_file_list,
-            geotag_source,
             geotag_source_path,
-            offset_time,
-            offset_angle,
-            local_time,
-            sub_second_interval,
-            use_gps_start_time,
-            verbose,
+            offset_time=offset_time,
+            offset_angle=offset_angle,
         )
     elif geotag_source == "gopro_videos":
-        processing.geotag_from_gopro_video(
+        if geotag_source_path is None:
+            geotag_source_path = video_import_path
+        if geotag_source_path is None:
+            raise RuntimeError("geotag_source_path is required")
+        return processing.geotag_from_gopro_video(
             process_file_list,
-            import_path,
             geotag_source_path,
-            offset_time,
-            offset_angle,
-            local_time,
-            sub_second_interval,
-            use_gps_start_time,
-            verbose,
+            offset_time=offset_time,
+            offset_angle=offset_angle,
         )
     elif geotag_source == "blackvue_videos":
-        processing.geotag_from_blackvue_video(
+        if geotag_source_path is None:
+            geotag_source_path = video_import_path
+        if geotag_source_path is None:
+            raise RuntimeError("geotag_source_path is required")
+        return processing.geotag_from_blackvue_video(
             process_file_list,
-            import_path,
             geotag_source_path,
-            offset_time,
-            offset_angle,
-            local_time,
-            sub_second_interval,
-            use_gps_start_time,
-            verbose,
+            offset_time=offset_time,
+            offset_angle=offset_angle,
         )
-    print("Sub process ended")
+    else:
+        raise RuntimeError(f"Invalid geotag source {geotag_source}")
diff --git a/mapillary_tools/process_import_meta_properties.py b/mapillary_tools/process_import_meta_properties.py
index e9e1da31..835bb2d0 100644
--- a/mapillary_tools/process_import_meta_properties.py
+++ b/mapillary_tools/process_import_meta_properties.py
@@ -1,96 +1,86 @@
 import os
-import sys
 import time
 
 from tqdm import tqdm
 
-from . import processing
-from .error import print_error
+from . import image_log
+from . import processing, VERSION
 from .exif_read import ExifRead
-from . import VERSION
+from .types import MetaProperties
+
 
 META_DATA_TYPES = {
     "strings": str,
     "doubles": float,
-    # "longs": long,
     "longs": int,
     "dates": int,
     "booleans": bool,
 }
 
 
-def validate_type(tag_type, tag_value):
-    if not isinstance(tag_value, META_DATA_TYPES[tag_type]):
-        try:
-            tag_value = META_DATA_TYPES[tag_type](tag_value)
-        except:
-            print_error(
-                f"Error, meta value {tag_type} can not be casted to the specified type {tag_value} and will therefore not be added."
-            )
-            return None
-    return tag_value
+def add_meta_tag(desc: MetaProperties, tag_type: str, key: str, value_before) -> None:
+    type_ = META_DATA_TYPES.get(tag_type)
 
+    if type_ is None:
+        raise RuntimeError(f"Invalid tag type: {tag_type}")
+
+    try:
+        value = type_(value_before)
+    except (ValueError, TypeError) as ex:
+        raise RuntimeError(
+            f'Unable to parse "{key}" in the custom metatags as {tag_type}'
+        ) from ex
 
-def add_meta_tag(mapillary_description, tag_type, key, value):
-    value = validate_type(tag_type, value)
-    if value is None:
-        return
     meta_tag = {"key": key, "value": value}
-    if "MAPMetaTags" in mapillary_description:
-        if tag_type in mapillary_description["MAPMetaTags"]:
-            mapillary_description["MAPMetaTags"][tag_type].append(meta_tag)
-        else:
-            mapillary_description["MAPMetaTags"][tag_type] = [meta_tag]
-    else:
-        mapillary_description["MAPMetaTags"] = {tag_type: [meta_tag]}
+    tags = desc.setdefault("MAPMetaTags", {})
+    tags.setdefault(tag_type, []).append(meta_tag)
 
 
-def parse_and_add_custom_meta_tags(mapillary_description, custom_meta_data):
+def parse_and_add_custom_meta_tags(desc: MetaProperties, custom_meta_data: str) -> None:
     # parse entry
     meta_data_entries = custom_meta_data.split(";")
     for entry in meta_data_entries:
         # parse name, type and value
         entry_fields = entry.split(",")
-
+        if len(entry_fields) != 3:
+            raise RuntimeError(
+                f'Unable to parse tag "{entry}" -- it must be "name,type,value"'
+            )
         # set name, type and value
         tag_name = entry_fields[0]
         tag_type = entry_fields[1] + "s"
         tag_value = entry_fields[2]
 
-        # insert name, type and value
-        add_meta_tag(mapillary_description, tag_type, tag_name, tag_value)
+        add_meta_tag(desc, tag_type, tag_name, tag_value)
 
 
 def finalize_import_properties_process(
-    image,
-    import_path,
+    image: str,
+    desc: MetaProperties,
+    import_path: str,
     orientation=None,
     device_make=None,
     device_model=None,
     GPS_accuracy=None,
     add_file_name=False,
     add_import_date=False,
-    verbose=False,
-    mapillary_description=None,
     custom_meta_data=None,
     camera_uuid=None,
     windows_path=False,
     exclude_import_path=False,
     exclude_path=None,
 ):
-    if mapillary_description is None:
-        mapillary_description = {}
     # always check if there are any command line arguments passed, they will
     if orientation is not None:
-        mapillary_description["MAPOrientation"] = orientation
+        desc["MAPOrientation"] = orientation
     if device_make is not None:
-        mapillary_description["MAPDeviceMake"] = device_make
+        desc["MAPDeviceMake"] = device_make
     if device_model is not None:
-        mapillary_description["MAPDeviceModel"] = device_model
+        desc["MAPDeviceModel"] = device_model
     if GPS_accuracy is not None:
-        mapillary_description["MAPGPSAccuracyMeters"] = float(GPS_accuracy)
+        desc["MAPGPSAccuracyMeters"] = float(GPS_accuracy)
     if camera_uuid is not None:
-        mapillary_description["MAPCameraUUID"] = camera_uuid
+        desc["MAPCameraUUID"] = camera_uuid
     if add_file_name:
         image_path = image
         if exclude_import_path:
@@ -100,57 +90,32 @@ def finalize_import_properties_process(
         if windows_path:
             image_path = image_path.replace("/", "\\")
 
-        mapillary_description["MAPFilename"] = image_path
+        desc["MAPFilename"] = image_path
 
     if add_import_date:
         add_meta_tag(
-            mapillary_description,
+            desc,
             "dates",
             "import_date",
             int(round(time.time() * 1000)),
         )
 
-    add_meta_tag(mapillary_description, "strings", "mapillary_tools_version", VERSION)
+    add_meta_tag(desc, "strings", "mapillary_tools_version", VERSION)
 
     if custom_meta_data:
-        parse_and_add_custom_meta_tags(mapillary_description, custom_meta_data)
+        parse_and_add_custom_meta_tags(desc, custom_meta_data)
 
-    processing.create_and_log_process(
-        image, "import_meta_data_process", "success", mapillary_description, verbose
-    )
+    return desc
 
 
-def get_import_meta_properties_exif(image, verbose=False):
-    import_meta_data_properties = {}
-    try:
-        exif = ExifRead(image)
-    except:
-        if verbose:
-            print(
-                "Warning, EXIF could not be read for image "
-                + image
-                + ", import properties not read."
-            )
-        return None
-    try:
-        import_meta_data_properties["MAPOrientation"] = exif.extract_orientation()
-    except:
-        if verbose:
-            print("Warning, image orientation tag not in EXIF.")
-    try:
-        import_meta_data_properties["MAPDeviceMake"] = exif.extract_make()
-    except:
-        if verbose:
-            print("Warning, camera make tag not in EXIF.")
-    try:
-        import_meta_data_properties["MAPDeviceModel"] = exif.extract_model()
-    except:
-        if verbose:
-            print("Warning, camera model tag not in EXIF.")
-    try:
-        import_meta_data_properties["MAPMetaTags"] = eval(exif.extract_image_history())
-    except:
-        pass
+def get_import_meta_properties_exif(image: str) -> MetaProperties:
+    import_meta_data_properties: MetaProperties = {}
+    # FIXME: might throw error here
+    exif = ExifRead(image)
+    import_meta_data_properties["MAPOrientation"] = exif.extract_orientation()
+    import_meta_data_properties["MAPDeviceMake"] = exif.extract_make()
+    import_meta_data_properties["MAPDeviceModel"] = exif.extract_model()
+    import_meta_data_properties["MAPMetaTags"] = eval(exif.extract_image_history())
 
     return import_meta_data_properties
 
@@ -163,67 +128,34 @@ def process_import_meta_properties(
     GPS_accuracy=None,
     add_file_name=False,
     add_import_date=False,
-    verbose=False,
-    rerun=False,
     skip_subfolders=False,
-    video_import_path=None,
     custom_meta_data=None,
     camera_uuid=None,
     windows_path=False,
     exclude_import_path=False,
     exclude_path=None,
-):
-    # sanity check if video file is passed
-    if (
-        video_import_path
-        and not os.path.isdir(video_import_path)
-        and not os.path.isfile(video_import_path)
-    ):
-        print("Error, video path " + video_import_path + " does not exist, exiting...")
-        sys.exit(1)
-
-    # in case of video processing, adjust the import path
-    if video_import_path:
-        # set sampling path
-        video_sampling_path = "mapillary_sampled_video_frames"
-        video_dirname = (
-            video_import_path
-            if os.path.isdir(video_import_path)
-            else os.path.dirname(video_import_path)
-        )
-        import_path = (
-            os.path.join(os.path.abspath(import_path), video_sampling_path)
-            if import_path
-            else os.path.join(os.path.abspath(video_dirname), video_sampling_path)
-        )
-
-    # basic check for all
+) -> None:
     if not import_path or not os.path.isdir(import_path):
-        print_error(
-            "Error, import directory " + import_path + " does not exist, exiting..."
-        )
-        sys.exit(1)
+        raise RuntimeError(f"Image directory {import_path} does not exist")
 
-    # get list of file to process
-    process_file_list = processing.get_process_file_list(
-        import_path, "import_meta_data_process", rerun, skip_subfolders=skip_subfolders
+    process_file_list = image_log.get_total_file_list(
+        import_path,
+        skip_subfolders=skip_subfolders,
     )
-    if not len(process_file_list):
-        print("No images to run import meta data process")
-        print(
-            "If the images have already been processed and not yet uploaded, they can be processed again, by passing the argument --rerun"
-        )
 
-    # map orientation from degrees to tags
+    if not process_file_list:
+        return
+
     if orientation is not None:
         orientation = processing.format_orientation(orientation)
 
-    # read import meta from image EXIF and finalize the import
-    # properties process
-    for image in tqdm(process_file_list, desc="Processing image import properties"):
-        import_meta_data_properties = get_import_meta_properties_exif(image, verbose)
-        finalize_import_properties_process(
+    for image in tqdm(
+        process_file_list, unit="files", desc="Processing import meta properties"
+    ):
+        import_meta_data_properties = get_import_meta_properties_exif(image)
+        desc = finalize_import_properties_process(
             image,
+            import_meta_data_properties,
             import_path,
             orientation,
             device_make,
@@ -231,12 +163,10 @@ def process_import_meta_properties(
             GPS_accuracy,
             add_file_name,
             add_import_date,
-            verbose,
-            import_meta_data_properties,
             custom_meta_data,
             camera_uuid,
             windows_path,
             exclude_import_path,
             exclude_path,
         )
-    print("Sub process ended")
+        image_log.log_in_memory(image, "import_meta_data_process", desc)
diff --git a/mapillary_tools/process_sequence_properties.py b/mapillary_tools/process_sequence_properties.py
index 0612b4c7..a5006592 100644
--- a/mapillary_tools/process_sequence_properties.py
+++ b/mapillary_tools/process_sequence_properties.py
@@ -1,43 +1,116 @@
 import typing as T
 import datetime
 import os
-import time
 import uuid
 
-from tqdm import tqdm
-
-from . import processing
-from . import uploader
-from .geo import compute_bearing, gps_distance, diff_bearing, gps_speed
+from . import image_log, types
+from .geo import compute_bearing, gps_distance, diff_bearing, pairwise
+from .error import MapillaryDuplicationError
 
 MAX_SEQUENCE_LENGTH = 500
-MAX_CAPTURE_SPEED = 45  # in m/s
-
-
-def finalize_sequence_processing(
-    sequence,
-    final_file_list,
-    final_directions,
-    final_capture_times,
-    verbose=False,
-):
-    for image, direction, capture_time in tqdm(
-        zip(final_file_list, final_directions, final_capture_times),
-        desc="Finalizing sequence process",
-    ):
-        mapillary_description = {
-            "MAPSequenceUUID": sequence,
-            "MAPCompassHeading": {
-                "TrueHeading": direction,
-                "MagneticHeading": direction,
-            },
-            "MAPCaptureTime": datetime.datetime.strftime(
-                capture_time, "%Y_%m_%d_%H_%M_%S_%f"
-            )[:-3],
-        }
-        processing.create_and_log_process(
-            image, "sequence_process", "success", mapillary_description, verbose=verbose
+
+
+class _GPXPoint:
+    desc: types.Image
+    filename: str
+
+    def __init__(self, desc: types.Image, filename: str):
+        self.desc = desc
+        self.filename = filename
+
+    @property
+    def lat(self) -> float:
+        return self.desc["MAPLatitude"]
+
+    @property
+    def lon(self) -> float:
+        return self.desc["MAPLongitude"]
+
+    @property
+    def time(self) -> datetime.datetime:
+        return types.map_capture_time_to_datetime(self.desc["MAPCaptureTime"])
+
+    @property
+    def angle(self) -> T.Optional[float]:
+        return self.desc.get("MAPCompassHeading", {}).get("TrueHeading")
+
+
+Sequence = T.List[_GPXPoint]
+
+
+def load_geotag_points(
+    images: T.List[str],
+) -> T.Generator[_GPXPoint, None, None]:
+    for image in images:
+        ret = image_log.read_process_data_from_memory(image, "geotag_process")
+        if ret is None:
+            continue
+        status, geotag_data = ret
+        if status != "success":
+            continue
+        yield _GPXPoint(T.cast(types.Image, geotag_data), image)
+
+
+def split_sequences(
+    sequence: Sequence,
+    cutoff_distance: float,
+    cutoff_time: float,
+) -> T.List[Sequence]:
+    sequences: T.List[T.List[_GPXPoint]] = []
+
+    if sequence:
+        sequences.append([sequence[0]])
+
+    for prev, cur in pairwise(sequence):
+        # invariant: prev is processed
+        distance = gps_distance(
+            (prev.lat, prev.lon),
+            (cur.lat, cur.lon),
         )
+        if cutoff_distance <= distance:
+            sequences.append([cur])
+            continue
+        time_diff = (cur.time - prev.time).total_seconds()
+        if cutoff_time <= time_diff:
+            sequences.append([cur])
+            continue
+        sequences[-1].append(cur)
+        # invariant: cur is processed
+
+    return sequences
+
+
+def find_duplicates(
+    sequence: Sequence,
+    duplicate_distance: float,
+    duplicate_angle: float,
+) -> T.List[int]:
+    if not sequence:
+        return []
+
+    duplicates = []
+    sequence_iter = iter(sequence)
+    prev = next(sequence_iter)
+    for idx, cur in enumerate(sequence_iter):
+        distance = gps_distance(
+            (prev.lat, prev.lon),
+            (cur.lat, cur.lon),
+        )
+        distance_duplicated = distance <= duplicate_distance
+
+        if prev.angle is not None and cur.angle is not None:
+            bearing_delta = diff_bearing(prev.angle, cur.angle)
+            angle_duplicated = bearing_delta <= duplicate_angle
+        else:
+            angle_duplicated = False
+
+        if distance_duplicated and angle_duplicated:
+            duplicates.append(idx + 1)
+            continue
+
+        prev = cur
+
+    return duplicates
 
 
 def process_sequence_properties(
@@ -45,196 +118,143 @@ def process_sequence_properties(
     cutoff_distance=600.0,
     cutoff_time=60.0,
     interpolate_directions=False,
-    keep_duplicates=False,
     duplicate_distance=0.1,
     duplicate_angle=5,
-    offset_angle=0.0,
-    verbose=False,
-    rerun=False,
     skip_subfolders=False,
-    video_import_path=None,
-):
-    # sanity check if video file is passed
-    if (
-        video_import_path
-        and not os.path.isdir(video_import_path)
-        and not os.path.isfile(video_import_path)
-    ):
-        raise RuntimeError(
-            f"Error, video path {video_import_path} does not exist, exiting..."
-        )
-
-    # in case of video processing, adjust the import path
-    if video_import_path:
-        # set sampling path
-        video_sampling_path = "mapillary_sampled_video_frames"
-        video_dirname = (
-            video_import_path
-            if os.path.isdir(video_import_path)
-            else os.path.dirname(video_import_path)
-        )
-        import_path = (
-            os.path.join(os.path.abspath(import_path), video_sampling_path)
-            if import_path
-            else os.path.join(os.path.abspath(video_dirname), video_sampling_path)
-        )
-
-    # basic check for all
+) -> None:
     if not import_path or not os.path.isdir(import_path):
         raise RuntimeError(f"Error, import directory {import_path} does not exist")
+    sequences = find_sequences(import_path, skip_subfolders)
+    for sequence in sequences:
+        process_sequence(
+            sequence,
+            cutoff_distance,
+            cutoff_time,
+            interpolate_directions,
+            duplicate_distance,
+            duplicate_angle,
+        )
+
 
-    sequences = find_sequences(
-        cutoff_distance, cutoff_time, import_path, rerun, skip_subfolders, verbose
+def process_sequence(
+    sequence: Sequence,
+    cutoff_distance: float,
+    cutoff_time: float,
+    interpolate_directions: bool,
+    duplicate_distance: float,
+    duplicate_angle: float,
+) -> None:
+    splitted_sequences: T.List[Sequence] = split_sequences(
+        sequence, cutoff_distance, cutoff_time
     )
 
-    # process for each sequence
-    for sequence in sequences:
-        file_list = sequence["file_list"]
-        directions = sequence["directions"]
-        latlons = sequence["latlons"]
-        capture_times = sequence["capture_times"]
-
-        # COMPUTE DIRECTIONS --------------------------------------
-        interpolated_directions = [
-            compute_bearing(ll1[0], ll1[1], ll2[0], ll2[1])
-            for ll1, ll2 in zip(latlons[:-1], latlons[1:])
-        ]
-        if len(interpolated_directions):
-            interpolated_directions.append(interpolated_directions[-1])
-        else:
-            interpolated_directions.append(directions[-1])
-        # use interpolated directions if direction not available or if flag for
-        # interpolate_directions
-        for i, d in enumerate(directions):
-            directions[i] = (
-                d
-                if (d is not None and not interpolate_directions)
-                else (interpolated_directions[i] + offset_angle) % 360.0
+    for sequence in splitted_sequences:
+        # duplication check
+        dup_indices = find_duplicates(
+            sequence,
+            duplicate_distance=duplicate_distance,
+            duplicate_angle=duplicate_angle,
+        )
+        dup_set = set(dup_indices)
+        dup_sequence = [image for idx, image in enumerate(sequence) if idx in dup_set]
+        for image in dup_sequence:
+            image_log.log_failed_in_memory(
+                image.filename,
+                "sequence_process",
+                MapillaryDuplicationError("duplicated", image.desc),
             )
-        # ---------------------------------------
 
-        # COMPUTE SPEED -------------------------------------------
-        computed_delta_ts = [
-            (t1 - t0).total_seconds()
-            for t0, t1 in zip(capture_times[:-1], capture_times[1:])
-        ]
-        computed_distances = [
-            gps_distance(l1, l0) for l0, l1 in zip(latlons[:-1], latlons[1:])
+        dedup_sequence = [
+            image for idx, image in enumerate(sequence) if idx not in dup_set
         ]
-        computed_speed = gps_speed(
-            computed_distances, computed_delta_ts
-        )  # in meters/second
-        if len([x for x in computed_speed if x > MAX_CAPTURE_SPEED]) > 0:
-            print(
-                f"Warning: The distance in sequence including images\n{file_list[0]}\nto\n{file_list[-1]}\nis too large for the time difference (very high apparent capture speed). Are you sure timestamps and locations are correct?"
-            )
 
-        # INTERPOLATE TIMESTAMPS, in case of identical timestamps
-        capture_times = processing.interpolate_timestamp(capture_times)
-
-        final_file_list = file_list[:]
-        final_directions = directions[:]
-        final_capture_times = capture_times[:]
-
-        # FLAG DUPLICATES --------------------------------------
-        if not keep_duplicates:
-            final_file_list = [file_list[0]]
-            final_directions = [directions[0]]
-            final_capture_times = [capture_times[0]]
-            prev_latlon = latlons[0]
-            prev_direction = directions[0]
-            for i, filename in enumerate(file_list[1:]):
-                log_root = uploader.log_rootpath(filename)
-                duplicate_flag_path = os.path.join(log_root, "duplicate")
-                sequence_process_success_path = os.path.join(
-                    log_root, "sequence_process_success"
+        # interpolate angles
+        interpolated_dedup_sequence: Sequence = []
+        for cur, nex in pairwise(dedup_sequence):
+            # should interpolate or not
+            if interpolate_directions or cur.angle is None:
+                new_bearing = compute_bearing(cur.lat, cur.lon, nex.lat, nex.lon)
+                new_desc: types.Image = T.cast(
+                    types.Image,
+                    {
+                        **cur.desc,
+                        "MAPCompassHeading": {
+                            "TrueHeading": new_bearing,
+                            "MagneticHeading": new_bearing,
+                        },
+                    },
                 )
-                k = i + 1
-                distance = gps_distance(latlons[k], prev_latlon)
-                if directions[k] is not None and prev_direction is not None:
-                    direction_diff = diff_bearing(directions[k], prev_direction)
-                else:
-                    # dont use bearing difference if no bearings are
-                    # available
-                    direction_diff = 360
-                if distance < duplicate_distance and direction_diff < duplicate_angle:
-                    open(duplicate_flag_path, "w").close()
-                    open(sequence_process_success_path, "w").close()
-                    open(
-                        sequence_process_success_path
-                        + "_"
-                        + str(time.strftime("%Y_%m_%d_%H_%M_%S", time.gmtime())),
-                        "w",
-                    ).close()
-                else:
-                    prev_latlon = latlons[k]
-                    prev_direction = directions[k]
-                    final_file_list.append(filename)
-                    final_directions.append(directions[k])
-                    final_capture_times.append(capture_times[k])
-        # ---------------------------------------
-
-        # FINALIZE ------------------------------------
-        for i in range(0, len(final_file_list), MAX_SEQUENCE_LENGTH):
-            finalize_sequence_processing(
-                str(uuid.uuid4()),
-                final_file_list[i : i + MAX_SEQUENCE_LENGTH],
-                final_directions[i : i + MAX_SEQUENCE_LENGTH],
-                final_capture_times[i : i + MAX_SEQUENCE_LENGTH],
-                verbose,
-            )
-    print("Sub process ended")
+                interpolated_dedup_sequence.append(_GPXPoint(new_desc, cur.filename))
+            else:
+                interpolated_dedup_sequence.append(cur)
 
+        if interpolated_dedup_sequence:
+            assert len(interpolated_dedup_sequence) == len(dedup_sequence) - 1
+        else:
+            assert len(dedup_sequence) <= 1
 
-def find_sequences(
-    cutoff_distance, cutoff_time, import_path, rerun, skip_subfolders, verbose
-) -> T.List[T.Dict]:
-    def _split(process_file_list: T.List[str]) -> T.List[T.Dict]:
-        if not process_file_list:
-            return []
-
-        (
-            file_list,
-            capture_times,
-            lats,
-            lons,
-            directions,
-        ) = processing.load_geotag_points(process_file_list, verbose)
-
-        if capture_times and lats and lons:
-            return processing.split_sequences(
-                capture_times,
-                lats,
-                lons,
-                file_list,
-                directions,
-                cutoff_time,
-                cutoff_distance,
-                verbose,
-            )
+        # interpolate the last image's angle
+        # can interpolate or not
+        if 2 <= len(dedup_sequence) and interpolated_dedup_sequence[-1] is not None:
+            # should interpolate or not
+            if interpolate_directions or dedup_sequence[-1] is None:
+                new_desc = T.cast(
+                    types.Image,
+                    {
+                        **dedup_sequence[-1].desc,
+                        "MAPCompassHeading": {
+                            "TrueHeading": interpolated_dedup_sequence[-1].angle,
+                            "MagneticHeading": interpolated_dedup_sequence[-1].angle,
+                        },
+                    },
+                )
+                interpolated_dedup_sequence.append(
+                    _GPXPoint(new_desc, dedup_sequence[-1].filename)
+                )
+            else:
+                interpolated_dedup_sequence.append(dedup_sequence[-1])
         else:
-            return []
+            interpolated_dedup_sequence.append(dedup_sequence[-1])
+
+        assert len(interpolated_dedup_sequence) == len(dedup_sequence)
+
+        # cut sequence per MAX_SEQUENCE_LENGTH images
+        for idx in range(0, len(interpolated_dedup_sequence), MAX_SEQUENCE_LENGTH):
+            sequence_uuid = str(uuid.uuid4())
+            for image in interpolated_dedup_sequence[idx : idx + MAX_SEQUENCE_LENGTH]:
+                desc: types.Sequence = {
+                    "MAPSequenceUUID": sequence_uuid,
+                }
+                heading = image.desc.get("MAPCompassHeading")
+                if heading is not None:
+                    desc["MAPCompassHeading"] = heading
+                image_log.log_in_memory(image.filename, "sequence_process", desc)
+
+
+def find_sequences(
+    import_path: str,
+    skip_subfolders: bool,
+) -> T.List[Sequence]:
+    sort_key = lambda image: image.time
 
     if skip_subfolders:
-        process_file_list = processing.get_process_file_list(
+        images = image_log.get_total_file_list(
             import_path,
-            "sequence_process",
-            rerun=rerun,
             skip_subfolders=True,
         )
-        return _split(process_file_list)
+        sequence = sorted(list(load_geotag_points(images)), key=sort_key)
+        return [sequence]
     else:
         sequences = []
 
         # sequence limited to the root of the files
         for root, dirs, files in os.walk(import_path, topdown=True):
             dirs[:] = [d for d in dirs if not d.startswith(".")]
-            process_file_list = processing.get_process_file_list(
+            images = image_log.get_total_file_list(
                 root,
-                "sequence_process",
-                rerun=rerun,
                 skip_subfolders=True,
             )
-            sequences.extend(_split(process_file_list))
+            sequence = sorted(list(load_geotag_points(images)), key=sort_key)
+            sequences.append(sequence)
 
         return sequences
diff --git a/mapillary_tools/process_upload_params.py b/mapillary_tools/process_upload_params.py
deleted file mode 100644
index 874a3c7d..00000000
--- a/mapillary_tools/process_upload_params.py
+++ /dev/null
@@ -1,102 +0,0 @@
-import os
-import sys
-
-from tqdm import tqdm
-
-from . import login
-from . import processing
-from . import uploader
-from .error import print_error
-
-
-def process_upload_params(
-    import_path,
-    user_name,
-    master_upload=False,
-    verbose=False,
-    rerun=False,
-    skip_subfolders=False,
-    video_import_path=None,
-):
-    # sanity check if video file is passed
-    if (
-        video_import_path
-        and not os.path.isdir(video_import_path)
-        and not os.path.isfile(video_import_path)
-    ):
-        print("Error, video path " + video_import_path + " does not exist, exiting...")
-        sys.exit(1)
-
-    # in case of video processing, adjust the import path
-    if video_import_path:
-        # set sampling path
-        video_sampling_path = "mapillary_sampled_video_frames"
-        video_dirname = (
-            video_import_path
-            if os.path.isdir(video_import_path)
-            else os.path.dirname(video_import_path)
-        )
-        import_path = (
-            os.path.join(os.path.abspath(import_path), video_sampling_path)
-            if import_path
-            else os.path.join(os.path.abspath(video_dirname), video_sampling_path)
-        )
-
-    # basic check for all
-    if not import_path or not os.path.isdir(import_path):
-        print_error(
-            "Error, import directory " + import_path + " does not exist, exiting..."
-        )
-        sys.exit(1)
-
-    # get list of file to process
-    process_file_list = processing.get_process_file_list(
-        import_path, "upload_params_process", rerun, skip_subfolders=skip_subfolders
-    )
-    if not len(process_file_list):
-        print("No images to run upload params process")
-        print(
-            "If the images have already been processed and not yet uploaded, they can be processed again, by passing the argument --rerun"
-        )
-
-    # sanity checks
-    if not user_name:
-        print_error("Error, must provide a valid user name, exiting...")
-        processing.create_and_log_process_in_list(
-            process_file_list, "upload_params_process", "failed", verbose
-        )
-        sys.exit(1)
-
-    if not master_upload:
-        credentials = login.authenticate_user(user_name)
-        user_upload_token = credentials["user_upload_token"]
-        user_key = credentials["MAPSettingsUserKey"]
-
-    for image in tqdm(process_file_list, desc="Processing image upload parameters"):
-        # check the status of the sequence processing
-        log_root = uploader.log_rootpath(image)
-        duplicate_flag_path = os.path.join(log_root, "duplicate")
-        upload_params_path = os.path.join(log_root, "upload_params_process.json")
-
-        if os.path.isfile(upload_params_path):
-            os.remove(upload_params_path)
-
-        if os.path.isfile(duplicate_flag_path) or master_upload:
-            continue
-
-        upload_params_properties = processing.get_upload_param_properties(
-            log_root, image, user_name, user_upload_token, user_key, verbose
-        )
-
-        processing.create_and_log_process(
-            image,
-            "upload_params_process",
-            "success",
-            upload_params_properties,
-            verbose=verbose,
-        )
-        # flag manual upload
-        log_manual_upload = os.path.join(log_root, "manual_upload")
-        open(log_manual_upload, "a").close()
-
-    print("Sub process ended")
diff --git a/mapillary_tools/process_user_properties.py b/mapillary_tools/process_user_properties.py
deleted file mode 100644
index 4be3ad29..00000000
--- a/mapillary_tools/process_user_properties.py
+++ /dev/null
@@ -1,114 +0,0 @@
-import os
-import typing as T
-
-import requests
-
-from . import processing, api_v4
-from . import login
-
-
-def get_user_properties(
-    user_name: str,
-    organization_key: T.Optional[str] = None,
-    private: bool = False,
-) -> T.Optional[T.Dict]:
-    # basic
-    user_items = login.authenticate_user(user_name)
-
-    if organization_key:
-        resp = api_v4.fetch_organization(
-            user_items["user_upload_token"], organization_key
-        )
-
-        try:
-            resp.raise_for_status()
-        except requests.RequestException:
-            raise RuntimeError(f"Invalid organization {resp.text}")
-
-        org = resp.json()
-
-        print(f"Organization ID: {org['id']}")
-        print(f"Organization name: {org['name']}")
-
-        user_items.update(
-            {"MAPOrganizationKey": organization_key, "MAPPrivate": private}
-        )
-
-    del user_items["user_upload_token"]
-
-    return user_items
-
-
-def process_user_properties(
-    import_path,
-    user_name,
-    organization_username=None,
-    organization_key=None,
-    private=False,
-    master_upload=False,
-    verbose=False,
-    rerun=False,
-    skip_subfolders=False,
-    video_import_path=None,
-):
-    # sanity check if video file is passed
-    if (
-        video_import_path
-        and not os.path.isdir(video_import_path)
-        and not os.path.isfile(video_import_path)
-    ):
-        raise RuntimeError(
-            f"Error, video path {video_import_path} does not exist, exiting..."
-        )
-
-    # in case of video processing, adjust the import path
-    if video_import_path:
-        # set sampling path
-        video_sampling_path = "mapillary_sampled_video_frames"
-        video_dirname = (
-            video_import_path
-            if os.path.isdir(video_import_path)
-            else os.path.dirname(video_import_path)
-        )
-        import_path = (
-            os.path.join(os.path.abspath(import_path), video_sampling_path)
-            if import_path
-            else os.path.join(os.path.abspath(video_dirname), video_sampling_path)
-        )
-
-    # basic check for all
-    if not import_path or not os.path.isdir(import_path):
-        raise RuntimeError(
-            f"Error, import directory {import_path} does not exist, exiting..."
-        )
-
-    # get list of file to process
-    process_file_list = processing.get_process_file_list(
-        import_path, "user_process", rerun=rerun, skip_subfolders=skip_subfolders
-    )
-    if not process_file_list:
-        print("No images to run user process")
-        print(
-            "If the images have already been processed and not yet uploaded, they can be processed again, by passing the argument --rerun"
-        )
-
-    # sanity checks
-    if not user_name:
-        raise RuntimeError("Error, must provide a valid user name, exiting...")
-
-    if private and not organization_username and not organization_key:
-        raise RuntimeError(
-            "Error, if the import belongs to a private repository, you need to provide a valid organization user name or key to which the private repository belongs to, exiting..."
-        )
-
-    user_properties = get_user_properties(
-        user_name,
-        organization_key,
-        private,
-    )
-
-    # write data and logs
-    processing.create_and_log_process_in_list(
-        process_file_list, "user_process", "success", verbose, user_properties
-    )
-    print("Sub process ended")
diff --git a/mapillary_tools/process_video.py b/mapillary_tools/process_video.py
index 9c5ed596..f3171541 100644
--- a/mapillary_tools/process_video.py
+++ b/mapillary_tools/process_video.py
@@ -1,25 +1,25 @@
 import datetime
-import io
 import os
-import struct
+import shutil
 import subprocess
+import typing as T
+import logging
 
-from pymp4.parser import Box
-from tqdm import tqdm
-
+from . import image_log
 from . import processing
-from . import uploader
 from .exif_write import ExifEdit
-from .ffprobe import FFProbe
 
 ZERO_PADDING = 6
-TIME_FORMAT = "%Y-%m-%d %H:%M:%S"
-TIME_FORMAT_2 = "%Y-%m-%dT%H:%M:%S.000000Z"
+LOG = logging.getLogger(__name__)
 
 
 def timestamp_from_filename(
-    video_filename, filename, start_time, interval=2.0, adjustment=1.0
-):
+    video_filename: str,
+    filename: str,
+    start_time: datetime.datetime,
+    interval=2.0,
+    adjustment=1.0,
+) -> datetime.datetime:
     seconds = (
         (int(filename.rstrip(".jpg").replace(f"{video_filename}_", "").lstrip("0")) - 1)
         * interval
@@ -30,10 +30,14 @@ def timestamp_from_filename(
 
 
 def timestamps_from_filename(
-    video_filename, full_image_list, start_time, interval=2.0, adjustment=1.0
-):
-    capture_times = []
-    for image in tqdm(full_image_list, desc="Deriving frame capture time"):
+    video_filename: str,
+    full_image_list: T.List[str],
+    start_time: datetime.datetime,
+    interval=2.0,
+    adjustment=1.0,
+) -> T.List[datetime.datetime]:
+    capture_times: T.List[datetime.datetime] = []
+    for image in full_image_list:
         capture_times.append(
             timestamp_from_filename(
                 video_filename,
@@ -47,92 +51,72 @@ def timestamps_from_filename(
 
 
 def sample_video(
-    video_import_path,
-    import_path,
+    video_import_path: str,
+    import_path: str,
     video_sample_interval=2.0,
     video_start_time=None,
     video_duration_ratio=1.0,
-    verbose=False,
     skip_subfolders=False,
 ):
-    if import_path is not None and not os.path.isdir(import_path):
-        raise RuntimeError(f"Error, import directory {import_path} does not exist")
-
-    # sanity check
-    if not os.path.isdir(video_import_path) and not os.path.isfile(video_import_path):
-        raise RuntimeError(f"Error, video path {video_import_path} does not exist")
-
-    # Adjust the import path
-    video_sampling_path = "mapillary_sampled_video_frames"
-    video_dirname = (
-        video_import_path
-        if os.path.isdir(video_import_path)
-        else os.path.dirname(video_import_path)
-    )
-    import_path = (
-        os.path.join(os.path.abspath(import_path), video_sampling_path)
-        if import_path is not None
-        else os.path.join(os.path.abspath(video_dirname), video_sampling_path)
-    )
+    if not os.path.exists(video_import_path):
+        raise RuntimeError(f'Error, video path "{video_import_path}" does not exist')
 
     video_list = (
-        uploader.get_video_file_list(video_import_path, skip_subfolders)
+        image_log.get_video_file_list(video_import_path, skip_subfolders)
         if os.path.isdir(video_import_path)
         else [video_import_path]
     )
 
-    for video in tqdm(video_list, desc="Extracting video frames"):
-        basename, _ = os.path.splitext(os.path.basename(video))
-        per_video_import_path = os.path.join(import_path, basename)
-        if not os.path.isdir(per_video_import_path):
-            os.makedirs(per_video_import_path)
-
-        print(f"Video sampling path set to {per_video_import_path}")
-        # check video logs
-        video_upload = processing.video_upload(
-            video_import_path, per_video_import_path, verbose
-        )
-        if video_upload:
-            print(
-                f"Video {video} has already been uploaded, contact support@mapillary for help with reuploading it if neccessary."
+    for video in video_list:
+        per_video_import_path = processing.video_sample_path(import_path, video)
+        if os.path.isdir(per_video_import_path):
+            images = image_log.get_total_file_list(per_video_import_path)
+            if images:
+                answer = input(
+                    f"The sample folder {per_video_import_path} already contains {len(images)} images.\nTo proceed, either DELETE the whole folder to restart the extraction (y), or skip the extraction (N)? [y/N] "
+                )
+                if answer in ["y", "Y"]:
+                    shutil.rmtree(per_video_import_path)
+        elif os.path.isfile(per_video_import_path):
+            answer = input(
+                f"The sample path {per_video_import_path} is found to be a file. To proceed, either DELETE it to restart the extraction (y), or skip the extraction (N)? [y/N] "
             )
+            if answer in ["y", "Y"]:
+                os.remove(per_video_import_path)
 
-        extract_frames(
-            video,
-            per_video_import_path,
-            video_sample_interval,
-            video_start_time,
-            video_duration_ratio,
-            verbose,
-        )
-
-    processing.create_and_log_video_process(video_import_path, import_path)
+    for video in video_list:
+        per_video_import_path = processing.video_sample_path(import_path, video)
+        if not os.path.exists(per_video_import_path):
+            os.makedirs(per_video_import_path)
+            extract_frames(
+                video,
+                per_video_import_path,
+                video_sample_interval,
+                video_duration_ratio,
+            )
 
 
 def extract_frames(
-    video_file,
-    import_path,
-    video_sample_interval=2.0,
-    video_start_time=None,
-    video_duration_ratio=1.0,
-    verbose=False,
-):
+    video_file: str,
+    import_path: str,
+    video_sample_interval: float = 2.0,
+    video_duration_ratio: float = 1.0,
+) -> None:
+    video_filename, ext = os.path.splitext(os.path.basename(video_file))
     command = [
         "ffmpeg",
         "-i",
         video_file,
-        "-loglevel",
-        "quiet",
         "-vf",
         f"fps=1/{video_sample_interval}",
-        "-qscale",
+        # video quality level
+        "-qscale:v",
         "1",
         "-nostdin",
+        f"{os.path.join(import_path, video_filename)}_%0{ZERO_PADDING}d.jpg",
     ]
 
-    video_filename, ext = os.path.splitext(os.path.basename(video_file))
-
-    command.append(f"{os.path.join(import_path, video_filename)}_%0{ZERO_PADDING}d.jpg")
+    LOG.info(f"Extracting frames: {' '.join(command)}")
     try:
         subprocess.call(command)
     except FileNotFoundError:
@@ -140,113 +124,35 @@ def extract_frames(
             "ffmpeg not found. Please make sure it is installed in your PATH. See https://github.com/mapillary/mapillary_tools#video-support for instructions"
         )
 
-    if video_start_time is not None:
-        video_start_time_obj = datetime.datetime.utcfromtimestamp(
-            video_start_time / 1000.0
-        )
-    else:
-        video_start_time_obj = get_video_start_time(video_file)
+    video_start_time = datetime.datetime.utcnow()
 
     insert_video_frame_timestamp(
         video_filename,
         import_path,
-        video_start_time_obj,
+        video_start_time,
         video_sample_interval,
         video_duration_ratio,
-        verbose,
     )
 
 
-def get_video_duration(video_file) -> float:
-    """Get video duration in seconds"""
-    probe = FFProbe(video_file)
-    duration = probe.video[0]["duration"]
-    try:
-        return float(duration)
-    except (TypeError, ValueError) as e:
-        raise RuntimeError(
-            f"could not parse duration {duration} from video {video_file} due to {e}"
-        )
-
-
 def insert_video_frame_timestamp(
-    video_filename,
-    video_sampling_path,
+    video_filename: str,
+    video_sampling_path: str,
     start_time: datetime.datetime,
-    sample_interval=2.0,
-    duration_ratio=1.0,
-    verbose=False,
-):
-    # get list of file to process
-    frame_list = uploader.get_total_file_list(video_sampling_path)
+    sample_interval: float = 2.0,
+    duration_ratio: float = 1.0,
+) -> None:
+    frame_list = image_log.get_total_file_list(video_sampling_path)
 
-    if not len(frame_list):
-        # WARNING LOG
-        print("No video frames were sampled.")
+    if not frame_list:
+        LOG.warning("No video frames were extracted.")
         return
 
     video_frame_timestamps = timestamps_from_filename(
         video_filename, frame_list, start_time, sample_interval, duration_ratio
     )
 
-    for image, timestamp in tqdm(
-        zip(frame_list, video_frame_timestamps), desc="Inserting frame capture time"
-    ):
+    for image, timestamp in zip(frame_list, video_frame_timestamps):
         exif_edit = ExifEdit(image)
         exif_edit.add_date_time_original(timestamp)
         exif_edit.write()
-
-
-def get_video_end_time(video_file) -> datetime.datetime:
-    """Get video end time in seconds"""
-    time_string = FFProbe(video_file).video[0]["tags"]["creation_time"]
-
-    try:
-        creation_time = datetime.datetime.strptime(time_string, TIME_FORMAT)
-    except ValueError:
-        try:
-            creation_time = datetime.datetime.strptime(time_string, TIME_FORMAT_2)
-        except ValueError:
-            raise RuntimeError(
-                f"Failed to parse {time_string} as {TIME_FORMAT} or {TIME_FORMAT_2}"
-            )
-
-    return creation_time
-
-
-def get_video_start_time(video_file) -> datetime.datetime:
-    """Get start time in seconds"""
-    video_end_time = get_video_end_time(video_file)
-    duration = get_video_duration(video_file)
-    return video_end_time - datetime.timedelta(seconds=duration)
-
-
-def get_video_start_time_blackvue(video_file):
-    with open(video_file, "rb") as fd:
-        fd.seek(0, io.SEEK_END)
-        eof = fd.tell()
-        fd.seek(0)
-
-        while fd.tell() < eof:
-            box = Box.parse_stream(fd)
-            if box.type.decode("utf-8") == "moov":
-                fd.seek(box.offset + 8, 0)
-
-                size = struct.unpack(">I", fd.read(4))[0]
-                typ = fd.read(4)
-
-                fd.seek(4, os.SEEK_CUR)
-
-                creation_time = struct.unpack(">I", fd.read(4))[0]
-                modification_time = struct.unpack(">I", fd.read(4))[0]
-                time_scale = struct.unpack(">I", fd.read(4))[0]
-                duration = struct.unpack(">I", fd.read(4))[0]
-
-                # from documentation
-                # in seconds since midnight, January 1, 1904
-                video_start_time_epoch = creation_time * 1000 - duration
-                epoch_start = datetime.datetime(year=1904, month=1, day=1)
-                video_start_time = epoch_start + datetime.timedelta(
-                    milliseconds=video_start_time_epoch
-                )
-                return video_start_time
diff --git a/mapillary_tools/processing.py b/mapillary_tools/processing.py
index 95e65eff..7d8a6013 100644
--- a/mapillary_tools/processing.py
+++ b/mapillary_tools/processing.py
@@ -1,685 +1,341 @@
-import base64
-from typing import Any, Dict, List, Optional, Tuple
-
+from typing import Dict, List, Tuple
+import typing as T
 import datetime
-import hashlib
-import json
 import os
-import time
-import uuid
-from collections import OrderedDict
 import logging
 
-from dateutil.tz import tzlocal
 from tqdm import tqdm
 
-from . import ipc
-from . import uploader
-from .error import print_error
+from . import image_log
+from . import types
+from .error import MapillaryGeoTaggingError, MapillaryStationaryBlackVueError
 from .exif_read import ExifRead
 from .exif_write import ExifEdit
-from .geo import (
-    normalize_bearing,
-    interpolate_lat_lon,
-    gps_distance,
-    MapillaryInterpolationError,
-)
+from .geo import normalize_bearing, interpolate_lat_lon, Point
 from .gps_parser import get_lat_lon_time_from_gpx, get_lat_lon_time_from_nmea
 from .gpx_from_blackvue import gpx_from_blackvue
 from .gpx_from_exif import gpx_from_exif
 from .gpx_from_gopro import gpx_from_gopro
-from .utils import force_decode
-
-"""
-auxillary processing functions
-"""
 
-
-LOG = logging.getLogger()
+LOG = logging.getLogger(__name__)
 
 
 def geotag_from_exif(
     process_file_list: List[str],
-    import_path: str,
     offset_time: float = 0.0,
     offset_angle: float = 0.0,
-    verbose: bool = False,
 ) -> None:
-    if offset_time == 0:
-        for image in tqdm(
-            process_file_list, desc="Extracting gps data from image EXIF"
-        ):
-            geotag_properties = get_geotag_properties_from_exif(
-                image, offset_angle, verbose
-            )
+    for image in tqdm(process_file_list, unit="files", desc="Extracting GPS from EXIF"):
 
-            create_and_log_process(
-                image, "geotag_process", "success", geotag_properties, verbose
-            )
-    else:
         try:
-            geotag_source_path = gpx_from_exif(process_file_list, import_path, verbose)
-            if not geotag_source_path or not os.path.isfile(geotag_source_path):
-                raise Exception
-        except Exception as e:
-            print_error(
-                f"Error, failed extracting data from exif due to {e}, exiting..."
-            )
-            raise e
+            point = gpx_from_exif(image)
+        except MapillaryGeoTaggingError as ex:
+            image_log.log_failed_in_memory(image, "geotag_process", ex)
+            continue
 
-        geotag_from_gps_trace(
-            process_file_list,
-            "gpx",
-            geotag_source_path,
-            offset_time,
-            offset_angle,
-            verbose=verbose,
+        corrected_time = point.point.time + datetime.timedelta(seconds=offset_time)
+
+        if point.angle is not None:
+            corrected_angle: T.Optional[float] = normalize_bearing(
+                point.angle + offset_angle
+            )
+        else:
+            corrected_angle = None
+
+        point = types.GPXPointAngle(
+            point=types.GPXPoint(
+                time=corrected_time,
+                lon=point.point.lon,
+                lat=point.point.lat,
+                alt=point.point.alt,
+            ),
+            angle=corrected_angle,
         )
 
+        image_log.log_in_memory(image, "geotag_process", point.as_desc())
 
-def get_geotag_properties_from_exif(
-    image: str, offset_angle: float = 0.0, verbose: bool = False
-) -> Optional[Dict]:
-    try:
-        exif = ExifRead(image)
-    except:
-        print_error(
-            "Error, EXIF could not be read for image "
-            + image
-            + ", geotagging process failed for this image since gps/time properties not read."
-        )
-        return None
-    # required tags
-    try:
-        lon, lat = exif.extract_lon_lat()
-    except:
-        print_error(
-            "Error, "
-            + image
-            + " image latitude or longitude tag not in EXIF. Geotagging process failed for this image, since this is required information."
-        )
-        return None
-    if lat is not None and lon is not None:
-        geotag_properties: Dict = {
-            "MAPLatitude": lat,
-            "MAPLongitude": lon,
-        }
-    else:
-        print_error(
-            "Error, "
-            + image
-            + " image latitude or longitude tag not in EXIF. Geotagging process failed for this image, since this is required information."
-        )
-        return None
-    try:
-        timestamp = exif.extract_capture_time()
-        if timestamp is None:
-            raise Exception
-    except:
-        print_error(
-            "Error, "
-            + image
-            + " image capture time tag not in EXIF. Geotagging process failed for this image, since this is required information."
-        )
-        return None
 
-    try:
-        geotag_properties["MAPCaptureTime"] = datetime.datetime.strftime(
-            timestamp, "%Y_%m_%d_%H_%M_%S_%f"
-        )[:-3]
-    except:
-        print_error(
-            f"Error, {image} image capture time tag incorrect format. Geotagging process failed for this image, since this is required information."
-        )
-        return None
+def video_sample_path(import_path: str, video_file_path: str) -> str:
+    video_filename = os.path.basename(video_file_path)
+    return os.path.join(import_path, video_filename)
+
 
-    # optional fields
-    try:
-        geotag_properties["MAPAltitude"] = exif.extract_altitude()
-    except:
-        if verbose:
-            print("Warning, image altitude tag not in EXIF.")
-    try:
-        heading = exif.extract_direction()
-        if heading is None:
-            heading = 0.0
-        heading = normalize_bearing(heading + offset_angle)
-        # bearing of the image
-        geotag_properties["MAPCompassHeading"] = {
-            "TrueHeading": heading,
-            "MagneticHeading": heading,
-        }
-    except:
-        if verbose:
-            print("Warning, image direction tag not in EXIF.")
+def is_sample_of_video(sample_path: str, video_filename: str) -> bool:
+    abs_sample_path = os.path.abspath(sample_path)
+    video_basename = os.path.basename(video_filename)
+    if video_basename == os.path.basename(os.path.dirname(abs_sample_path)):
+        sample_basename = os.path.basename(sample_path)
+        root, _ = os.path.splitext(video_basename)
+        return sample_basename.startswith(root + "_")
+    return False
 
-    return geotag_properties
+
+def _filter_video_samples(images: T.List[str], video_path: str) -> T.List[str]:
+    return [image for image in images if is_sample_of_video(image, video_path)]
 
 
 def geotag_from_gopro_video(
-    process_file_list,
-    import_path,
-    geotag_source_path,
-    offset_time,
-    offset_angle,
-    local_time,
-    sub_second_interval,
-    use_gps_start_time=False,
-    verbose=False,
-):
-    if geotag_source_path is None:
-        raise RuntimeError(
-            f"A path to your video directory is required to be specified in --geotag_source_path",
+    process_file_list: T.List[str],
+    geotag_source_path: str,
+    offset_time: float,
+    offset_angle: float,
+) -> None:
+    if os.path.isdir(geotag_source_path):
+        gopro_videos = image_log.get_video_file_list(geotag_source_path)
+        for gopro_video in gopro_videos:
+            trace = gpx_from_gopro(gopro_video)
+            _geotag_from_gpx(
+                _filter_video_samples(process_file_list, gopro_video),
+                trace,
+                offset_time,
+                offset_angle,
+            )
+    elif os.path.isfile(geotag_source_path):
+        trace = gpx_from_gopro(geotag_source_path)
+        _geotag_from_gpx(
+            _filter_video_samples(process_file_list, geotag_source_path),
+            trace,
+            offset_time,
+            offset_angle,
         )
-
-    if not os.path.isdir(geotag_source_path):
+    else:
         raise RuntimeError(
-            f"The path specified in geotag_source_path {geotag_source_path} is not a directory"
+            f"The geotag_source_path {geotag_source_path} does not exist"
         )
 
-    # for each video, create gpx trace and geotag the corresponding video
-    # frames
-    gopro_videos = uploader.get_video_file_list(geotag_source_path)
-    for gopro_video in gopro_videos:
-        gopro_video_filename, _ = os.path.splitext(os.path.basename(gopro_video))
-        gpx_path = gpx_from_gopro(gopro_video)
 
-        process_file_sublist = [
-            x
-            for x in process_file_list
-            if os.path.join(gopro_video_filename, gopro_video_filename + "_") in x
-        ]
+def _geotag_from_gpx(
+    images: List[str],
+    points: List[types.GPXPoint],
+    offset_time: float = 0.0,
+    offset_angle: float = 0.0,
+    read_image_time: T.Optional[T.Callable] = None,
+):
+    if read_image_time is None:
+        read_image_time = lambda img: ExifRead(img).extract_capture_time()
 
-        if not process_file_sublist:
-            print_error(
-                f"Error, no video frames extracted for video file {gopro_video} in import_path {import_path}"
-            )
-            create_and_log_process_in_list(
-                process_file_sublist, "geotag_process", "failed", verbose=verbose
+    if not points:
+        raise ValueError("Empty GPX list provided")
+
+    # need EXIF timestamps for sorting
+    pairs = []
+    for image in images:
+        capture_time = read_image_time(image)
+        if capture_time is None:
+            image_log.log_failed_in_memory(
+                image,
+                "geotag_process",
+                MapillaryGeoTaggingError("No capture time found in EXIF"),
             )
-            continue
+        else:
+            pairs.append((capture_time, image))
 
-        geotag_from_gps_trace(
-            process_file_sublist,
-            "gpx",
-            gpx_path,
-            offset_time,
-            offset_angle,
-            local_time,
-            sub_second_interval,
-            use_gps_start_time,
-            verbose,
-        )
+    sorted_points = sorted(points, key=lambda p: p.time)
+    sorted_pairs = sorted(pairs)
 
+    if sorted_pairs:
+        # assume: the ordered image timestamps are [2, 3, 4, 5]
+        # the ordered gpx timestamps are [5, 6, 7, 8]
+        # then the offset will be 5 - 2 = 3
+        time_delta = (sorted_points[0].time - sorted_pairs[0][0]).total_seconds()
+    else:
+        time_delta = 0.0
 
-def geotag_from_blackvue_video(
-    process_file_list,
-    import_path,
-    geotag_source_path,
-    offset_time,
-    offset_angle,
-    local_time,
-    sub_second_interval,
-    use_gps_start_time=False,
-    verbose=False,
-):
-    if geotag_source_path is None:
-        raise RuntimeError(
-            f"A path to your video directory is required to be specified in --geotag_source_path",
+    # same thing but different type
+    sorted_points_for_interpolation = [
+        Point(lat=p.lat, lon=p.lon, alt=p.alt, time=p.time) for p in sorted_points
+    ]
+
+    for exif_time, image in sorted_pairs:
+        exif_time = exif_time + datetime.timedelta(seconds=time_delta)
+        lat, lon, bearing, elevation = interpolate_lat_lon(
+            sorted_points_for_interpolation, exif_time
+        )
+
+        corrected_angle = normalize_bearing(bearing + offset_angle)
+        corrected_time = exif_time + datetime.timedelta(seconds=offset_time)
+        point = types.GPXPointAngle(
+            point=types.GPXPoint(
+                time=corrected_time,
+                lon=lon,
+                lat=lat,
+                alt=elevation,
+            ),
+            angle=corrected_angle,
         )
+        image_log.log_in_memory(image, "geotag_process", point.as_desc())
 
-    if not os.path.isdir(geotag_source_path):
+
+def geotag_from_blackvue_video(
+    process_file_list: T.List[str],
+    geotag_source_path: str,
+    offset_time: float = 0.0,
+    offset_angle: float = 0.0,
+) -> None:
+    if os.path.isdir(geotag_source_path):
+        blackvue_videos = image_log.get_video_file_list(geotag_source_path)
+    elif os.path.isfile(geotag_source_path):
+        blackvue_videos = [geotag_source_path]
+    else:
         raise RuntimeError(
-            f"The path specified in --geotag_source_path {geotag_source_path} is not a directory"
+            f"The geotag_source_path {geotag_source_path} does not exist"
         )
 
-    # for each video, create gpx trace and geotag the corresponding video
-    # frames
-    blackvue_videos = uploader.get_video_file_list(geotag_source_path)
     for blackvue_video in blackvue_videos:
-        blackvue_video_filename = (
-            os.path.basename(blackvue_video).replace(".mp4", "").replace(".MP4", "")
-        )
-        [gpx_path, is_stationary_video] = gpx_from_blackvue(
+        process_file_sublist = _filter_video_samples(process_file_list, blackvue_video)
+        if not process_file_sublist:
+            continue
+
+        [points, is_stationary_video] = gpx_from_blackvue(
             blackvue_video, use_nmea_stream_timestamp=False
         )
 
-        if not gpx_path or not os.path.isfile(gpx_path):
-            print_error(
-                f"Warning: Skipping blackvue video that has no GPS data found: {blackvue_video}"
-            )
+        if not points:
+            message = f"Skipping the BlackVue video {blackvue_video} -- no GPS found in the video"
+            for image in process_file_sublist:
+                image_log.log_failed_in_memory(
+                    image,
+                    "geotag_process",
+                    MapillaryGeoTaggingError(message),
+                )
             continue
 
         if is_stationary_video:
-            print_error(
-                f"Warning: Skipping stationary blackvue video: {blackvue_video}"
-            )
-            continue
-
-        process_file_sublist = [
-            x
-            for x in process_file_list
-            if os.path.join(blackvue_video_filename, blackvue_video_filename + "_") in x
-        ]
-
-        if not len(process_file_sublist):
-            print_error(
-                f"Error, no video frames extracted for video file {blackvue_video} in import_path {import_path}"
-            )
-            create_and_log_process_in_list(
-                process_file_sublist, "geotag_process", "failed", verbose=verbose
-            )
+            message = f"Skipping stationary BlackVue video {blackvue_video}"
+            for image in process_file_sublist:
+                image_log.log_failed_in_memory(
+                    image,
+                    "geotag_process",
+                    MapillaryStationaryBlackVueError(message),
+                )
             continue
 
-        geotag_from_gps_trace(
+        _geotag_from_gpx(
             process_file_sublist,
-            "gpx",
-            gpx_path,
+            points,
             offset_time,
             offset_angle,
-            local_time,
-            sub_second_interval,
-            use_gps_start_time,
-            verbose,
         )
 
 
-def geotag_from_gps_trace(
-    process_file_list,
-    geotag_source,
-    geotag_source_path,
+def geotag_from_nmea_file(
+    process_file_list: T.List[str],
+    geotag_source_path: str,
     offset_time=0.0,
     offset_angle=0.0,
-    local_time=False,
-    sub_second_interval=0.0,
-    use_gps_start_time=False,
-    verbose=False,
-):
-    if geotag_source == "gpx":
-        file_desc = "a GPX file"
-    elif geotag_source == "nmea":
-        file_desc = "an NMEA file"
-    else:
-        raise RuntimeError(f"Invalid geotag source {geotag_source}")
-
-    if geotag_source_path is None:
-        raise RuntimeError(
-            f"{file_desc} is required to be specified in --geotag_source_path",
-        )
-
+) -> None:
     if not os.path.isfile(geotag_source_path):
         raise RuntimeError(
-            f"The path specified in geotag_source_path {geotag_source_path} is not {file_desc}"
+            f"geotag_source_path {geotag_source_path} is not a NMEA file"
         )
 
-    # print time now to warn in case local_time
-    if local_time:
-        now = datetime.datetime.now(tzlocal())
-        print(
-            f"Your local timezone is {now.strftime('%Y-%m-%d %H:%M:%S %Z')}. If not, the geotags will be wrong."
-        )
-    else:
-        # if not local time to be used, warn UTC will be used
-        print(
-            "It is assumed that the image timestamps are in UTC. If not, try using the option --local_time."
-        )
-
-    # read gps file to get track locations
-    if geotag_source == "gpx":
-        gps_trace = get_lat_lon_time_from_gpx(geotag_source_path, local_time)
-    elif geotag_source == "nmea":
-        gps_trace = get_lat_lon_time_from_nmea(geotag_source_path, local_time)
-    else:
-        raise RuntimeError(f"Invalid geotag source {geotag_source}")
-
-    if not gps_trace:
-        print_error(
-            f"Error, gps trace file {geotag_source_path} was not read, images can not be geotagged."
-        )
-        create_and_log_process_in_list(
-            process_file_list, "geotag_process", "failed", verbose=verbose
-        )
+    if not process_file_list:
         return
 
-    pairs = [(ExifRead(f).extract_capture_time(), f) for f in process_file_list]
+    gps_trace = get_lat_lon_time_from_nmea(geotag_source_path)
 
-    if use_gps_start_time:
-        filtered_pairs: List[Tuple[datetime.datetime, str]] = [
-            p for p in pairs if p[0] is not None
-        ]
-        sorted_pairs = sorted(filtered_pairs)
-        if sorted_pairs:
-            # update offset time with the gps start time
-            offset_time += (sorted_pairs[0][0] - gps_trace[0][0]).total_seconds()
-            LOG.info(
-                f"Use GPS start time, which is same as using offset_time={offset_time}"
-            )
-
-    for capture_time, image in tqdm(
-        pairs,
-        desc="Inserting gps data into image EXIF",
-    ):
-        if capture_time is None:
-            print_error(f"Error, capture time could not be extracted for image {image}")
-            create_and_log_process(image, "geotag_process", "failed", verbose=verbose)
-        else:
-            try:
-                geotag_properties = get_geotag_properties_from_gps_trace(
-                    image, capture_time, gps_trace, offset_angle, offset_time
-                )
-            except MapillaryInterpolationError as ex:
-                raise RuntimeError(
-                    f"""Failed to interpolate image {image} with the geotag source file {geotag_source_path}. Try the following fixes:
-1. Specify --local_time to read the timestamps from the geotag source file as local time
-2. Use --use_gps_start_time to align the start time
-3. Manually shift the timestamps in the geotag source file with --offset_time OFFSET_IN_SECONDS
-"""
-                ) from ex
-
-            create_and_log_process(
-                image, "geotag_process", "success", geotag_properties, verbose
+    if not gps_trace:
+        message = f"No GPS extracted from the NMEA trace file {geotag_source_path}"
+        LOG.warning(message)
+        for image in process_file_list:
+            image_log.log_failed_in_memory(
+                image,
+                "geotag_process",
+                MapillaryGeoTaggingError(message),
             )
+        return
 
-
-def get_geotag_properties_from_gps_trace(
-    image,
-    capture_time: datetime.datetime,
-    gps_trace: list,
-    offset_angle=0.0,
-    offset_time=0.0,
-) -> dict:
-    capture_time = capture_time - datetime.timedelta(seconds=offset_time)
-
-    lat, lon, bearing, elevation = interpolate_lat_lon(gps_trace, capture_time)
-
-    geotag_properties = {
-        "MAPLatitude": lat,
-        "MAPLongitude": lon,
-    }
-
-    corrected_bearing = (bearing + offset_angle) % 360
-    geotag_properties["MAPCompassHeading"] = {
-        "TrueHeading": corrected_bearing,
-        "MagneticHeading": corrected_bearing,
-    }
-
-    geotag_properties["MAPCaptureTime"] = datetime.datetime.strftime(
-        capture_time, "%Y_%m_%d_%H_%M_%S_%f"
-    )[:-3]
-
-    if elevation is not None:
-        geotag_properties["MAPAltitude"] = elevation
-
-    return geotag_properties
-
-
-def get_upload_param_properties(
-    log_root: str,
-    image: str,
-    user_name: str,
-    user_upload_token: str,
-    user_key: str,
-    verbose: bool = False,
-) -> Optional[Dict]:
-    if not os.path.isdir(log_root):
-        print(
-            "Warning, sequence process has not been done for image "
-            + image
-            + ", therefore it will not be included in the upload params processing."
-        )
-        return None
-
-    # check if geotag process was a success
-    log_sequence_process_success = os.path.join(log_root, "sequence_process_success")
-    if not os.path.isfile(log_sequence_process_success):
-        print(
-            "Warning, sequence process failed for image "
-            + image
-            + ", therefore it will not be included in the upload params processing."
-        )
-        return None
-
-    upload_params_process_success_path = os.path.join(
-        log_root, "upload_params_process_success"
+    _geotag_from_gpx(
+        process_file_list,
+        gps_trace,
+        offset_time,
+        offset_angle,
     )
 
-    # load the sequence json
-    user_process_json_path = os.path.join(log_root, "user_process.json")
-    try:
-        user_data = load_json(user_process_json_path)
-    except:
-        print(
-            f"Warning, user data not read for image {image}, therefore it will not be included in the upload params processing."
-        )
-        return None
-
-    if "MAPSettingsUserKey" not in user_data:
-        print(
-            "Warning, user key not in user data for image {image}, therefore it will not be included in the upload params processing."
-        )
-        return None
-
-    user_key = user_data["MAPSettingsUserKey"]
-    organization_key = user_data.get("MAPOrganizationKey")
-    private = user_data.get("MAPPrivate", False)
 
-    # load the sequence json
-    sequence_process_json_path = os.path.join(log_root, "sequence_process.json")
-    try:
-        sequence_data = load_json(sequence_process_json_path)
-    except:
-        print(
-            "Warning, sequence data not read for image "
-            + image
-            + ", therefore it will not be included in the upload params processing."
+def geotag_from_gpx_file(
+    process_file_list: T.List[str],
+    geotag_source_path: str,
+    offset_time=0.0,
+    offset_angle=0.0,
+) -> None:
+    if not os.path.isfile(geotag_source_path):
+        raise RuntimeError(
+            f"The path specified in geotag_source_path {geotag_source_path} is not a GPX file"
         )
-        return None
 
-    if "MAPSequenceUUID" not in sequence_data:
-        print(
-            "Warning, sequence uuid not in sequence data for image "
-            + image
-            + ", therefore it will not be included in the upload params processing."
-        )
-        return None
+    if not process_file_list:
+        return
 
-    sequence_uuid = sequence_data["MAPSequenceUUID"]
+    gps_trace = get_lat_lon_time_from_gpx(geotag_source_path)
 
-    upload_params = {
-        "key": sequence_uuid,
-        "sequence_uuid": sequence_uuid,
-        "user_key": user_key,
-        "user_name": user_name,
-        "organization_key": organization_key,
-        "private": private,
-    }
+    if not gps_trace:
+        message = f"No GPS extracted from the GPX file {geotag_source_path}"
+        for image in process_file_list:
+            image_log.log_failed_in_memory(
+                image,
+                "geotag_process",
+                MapillaryGeoTaggingError(message),
+            )
+        return
 
-    x = base64.b64encode(image.encode("utf-8")).decode("utf-8")
-    s = f"{user_upload_token}{user_key}{x}"
-    settings_upload_hash = hashlib.sha256(s.encode("utf-8")).hexdigest()
-    save_json(
-        {"MAPSettingsUploadHash": settings_upload_hash},
-        os.path.join(log_root, "settings_upload_hash.json"),
+    _geotag_from_gpx(
+        process_file_list,
+        gps_trace,
+        offset_time,
+        offset_angle,
     )
-    return upload_params
 
 
-def get_final_mapillary_image_description(
-    log_root: str,
-    image: str,
-    master_upload: bool = False,
-    verbose: bool = False,
-    skip_EXIF_insert: bool = False,
-    keep_original: bool = False,
+def overwrite_exif_tags(
+    image_path: str,
+    desc: types.FinalImageDescription,
     overwrite_all_EXIF_tags: bool = False,
     overwrite_EXIF_time_tag: bool = False,
     overwrite_EXIF_gps_tag: bool = False,
     overwrite_EXIF_direction_tag: bool = False,
     overwrite_EXIF_orientation_tag: bool = False,
-) -> Optional[Dict]:
-    sub_commands = [
-        "user_process",
-        "geotag_process",
-        "sequence_process",
-        "upload_params_process",
-        "settings_upload_hash",
-        "import_meta_data_process",
-    ]
-    final_mapillary_image_description = {}
-
-    for sub_command in sub_commands:
-        sub_command_status = os.path.join(log_root, sub_command + "_failed")
-
-        if (
-            os.path.isfile(sub_command_status)
-            and sub_command != "import_meta_data_process"
-        ):
-            LOG.warning(
-                f"Warning, required {sub_command} failed for image {image}",
-                exc_info=True,
-            )
-            return None
-
-        sub_command_data_path = os.path.join(log_root, sub_command + ".json")
-        if (
-            not os.path.isfile(sub_command_data_path)
-            and sub_command != "import_meta_data_process"
-        ):
-            if (
-                sub_command == "settings_upload_hash"
-                or sub_command == "upload_params_process"
-            ) and master_upload:
-                continue
-            else:
-                LOG.warning(
-                    f"Warning, required {sub_command} did not result in a valid json file for image {image}",
-                    exc_info=True,
-                )
-                return None
-        if (
-            sub_command == "settings_upload_hash"
-            or sub_command == "upload_params_process"
-        ):
-            continue
-        try:
-            sub_command_data = load_json(sub_command_data_path)
-            if not sub_command_data:
-                if verbose:
-                    LOG.warning(
-                        f"Warning, no data read from json file {sub_command_data_path}",
-                        exc_info=True,
-                    )
-                return None
-
-            final_mapillary_image_description.update(sub_command_data)
-        except Exception:
-            if sub_command == "import_meta_data_process":
-                LOG.warning(
-                    "Warning, could not load json file " + sub_command_data_path,
-                    exc_info=True,
-                )
-                continue
-            else:
-                LOG.warning(
-                    "Warning, could not load json file " + sub_command_data_path,
-                    exc_info=True,
-                )
-                return None
-
-    # a unique photo ID to check for duplicates in the backend in case the
-    # image gets uploaded more than once
-    final_mapillary_image_description["MAPPhotoUUID"] = str(uuid.uuid4())
-
-    if skip_EXIF_insert:
-        return final_mapillary_image_description
-
-    image_exif = ExifEdit(image)
+) -> None:
+    modified = False
 
-    image_exif.add_image_description(final_mapillary_image_description)
+    image_exif = ExifEdit(image_path)
 
     # also try to set time and gps so image can be placed on the map for testing and
     # qc purposes
-    try:
-        if overwrite_all_EXIF_tags or overwrite_EXIF_time_tag:
-            image_exif.add_date_time_original(
-                datetime.datetime.strptime(
-                    final_mapillary_image_description["MAPCaptureTime"],
-                    "%Y_%m_%d_%H_%M_%S_%f",
-                )
-            )
-
-        if overwrite_all_EXIF_tags or overwrite_EXIF_gps_tag:
-            image_exif.add_lat_lon(
-                final_mapillary_image_description["MAPLatitude"],
-                final_mapillary_image_description["MAPLongitude"],
-            )
-
-        if overwrite_all_EXIF_tags or overwrite_EXIF_direction_tag:
-            image_exif.add_direction(
-                final_mapillary_image_description["MAPCompassHeading"]["TrueHeading"]
-            )
-
-        if overwrite_all_EXIF_tags or overwrite_EXIF_orientation_tag:
-            if "MAPOrientation" in final_mapillary_image_description:
-                image_exif.add_orientation(
-                    final_mapillary_image_description["MAPOrientation"]
-                )
-    except Exception:
-        LOG.warning("Error overwriting EXIF", exc_info=True)
-
-    filename_keep_original = processed_images_rootpath(image)
-    if os.path.isfile(filename_keep_original):
-        os.remove(filename_keep_original)
-
-    if keep_original:
-        if not os.path.isdir(os.path.dirname(filename_keep_original)):
-            os.makedirs(os.path.dirname(filename_keep_original))
-        target = filename_keep_original
-    else:
-        target = image
-
-    image_exif.write(filename=target)
-
-    return final_mapillary_image_description
+    if overwrite_all_EXIF_tags or overwrite_EXIF_time_tag:
+        dt = types.map_capture_time_to_datetime(desc["MAPCaptureTime"])
+        image_exif.add_date_time_original(dt)
+        modified = True
+
+    if overwrite_all_EXIF_tags or overwrite_EXIF_gps_tag:
+        image_exif.add_lat_lon(
+            desc["MAPLatitude"],
+            desc["MAPLongitude"],
+        )
+        modified = True
 
+    if overwrite_all_EXIF_tags or overwrite_EXIF_direction_tag:
+        heading = desc.get("MAPCompassHeading")
+        if heading is not None:
+            image_exif.add_direction(heading["TrueHeading"])
+            modified = True
 
-def get_geotag_data(log_root: str, image: str, verbose: bool = False) -> Optional[Dict]:
-    if not os.path.isdir(log_root):
-        if verbose:
-            print("Warning, no logs for image " + image)
-        return None
+    if overwrite_all_EXIF_tags or overwrite_EXIF_orientation_tag:
+        if "MAPOrientation" in desc:
+            image_exif.add_orientation(desc["MAPOrientation"])
+            modified = True
 
-    # check if geotag process was a success
-    log_geotag_process_success = os.path.join(log_root, "geotag_process_success")
-    if not os.path.isfile(log_geotag_process_success):
-        print(
-            "Warning, geotag process failed for image "
-            + image
-            + ", therefore it will not be included in the sequence processing."
-        )
-        return None
-    # load the geotag json
-    geotag_process_json_path = os.path.join(log_root, "geotag_process.json")
-    try:
-        geotag_data = load_json(geotag_process_json_path)
-        return geotag_data
-    except:
-        if verbose:
-            print(
-                "Warning, geotag data not read for image "
-                + image
-                + ", therefore it will not be included in the sequence processing."
-            )
-        return None
+    if modified:
+        image_exif.write()
 
 
-def format_orientation(orientation):
+def format_orientation(orientation: int) -> int:
     """
     Convert orientation from clockwise degrees to exif tag
 
     # see http://sylvana.net/jpegcrop/exif_orientation.html
     """
-    mapping = {
+    mapping: T.Mapping[int, int] = {
         0: 1,
         90: 8,
         180: 3,
@@ -689,398 +345,3 @@ def format_orientation(orientation):
         raise ValueError("Orientation value has to be 0, 90, 180, or 270")
 
     return mapping[orientation]
-
-
-def load_json(file_path: str):
-    try:
-        with open(file_path, "rb") as f:
-            return json.load(f)
-    except:
-        return {}
-
-
-def save_json(data: Dict[str, Any], file_path: str) -> None:
-    try:
-        buf = json.dumps(data, indent=4)
-    except Exception:
-        raise RuntimeError(f"Error JSON serializing {data}")
-    with open(file_path, "w") as f:
-        f.write(buf)
-
-
-def update_json(data, file_path, process):
-    original_data = load_json(file_path)
-    original_data[process] = data
-    save_json(original_data, file_path)
-
-
-def get_process_file_list(
-    import_path: str,
-    process: str,
-    rerun: bool = False,
-    skip_subfolders: bool = False,
-) -> List[str]:
-    files = uploader.iterate_files(import_path, not skip_subfolders)
-    sorted_files = sorted(
-        file
-        for file in files
-        if uploader.is_image_file(file) and preform_process(file, process, rerun)
-    )
-    return sorted_files
-
-
-def get_process_status_file_list(
-    import_path: str,
-    process: str,
-    status: str,
-    skip_subfolders: bool = False,
-) -> List[str]:
-    files = uploader.iterate_files(import_path, not skip_subfolders)
-    return sorted(
-        file
-        for file in files
-        if uploader.is_image_file(file) and process_status(file, process, status)
-    )
-
-
-def process_status(file_path: str, process: str, status: str) -> bool:
-    log_root = uploader.log_rootpath(file_path)
-    status_file = os.path.join(log_root, process + "_" + status)
-    return os.path.isfile(status_file)
-
-
-def get_duplicate_file_list(
-    import_path: str, skip_subfolders: bool = False
-) -> List[str]:
-    files = uploader.iterate_files(import_path, not skip_subfolders)
-    return sorted(
-        file for file in files if uploader.is_image_file(file) and is_duplicate(file)
-    )
-
-
-def is_duplicate(file_path: str) -> bool:
-    log_root = uploader.log_rootpath(file_path)
-    duplicate_flag_path = os.path.join(log_root, "duplicate")
-    return os.path.isfile(duplicate_flag_path)
-
-
-def preform_process(file_path: str, process: str, rerun: bool = False) -> bool:
-    log_root = uploader.log_rootpath(file_path)
-    process_succes = os.path.join(log_root, process + "_success")
-    upload_succes = os.path.join(log_root, "upload_success")
-    preform = not os.path.isfile(upload_succes) and (
-        not os.path.isfile(process_succes) or rerun
-    )
-    return preform
-
-
-def processed_images_rootpath(filepath: str) -> str:
-    return os.path.join(
-        os.path.dirname(filepath),
-        ".mapillary",
-        "processed_images",
-        os.path.basename(filepath),
-    )
-
-
-def video_upload(video_file, import_path, verbose=False):
-    import_paths = video_import_paths(video_file)
-    if not os.path.isdir(import_path):
-        os.makedirs(import_path)
-    if import_path not in import_paths:
-        import_paths.append(import_path)
-    else:
-        print(
-            f"Warning, {video_file} has already been sampled into {import_path}, please make sure all the previously sampled frames are deleted, otherwise the alignment might be incorrect"
-        )
-    for video_import_path in import_paths:
-        if os.path.isdir(video_import_path):
-            if len(uploader.get_success_upload_file_list(video_import_path)):
-                if verbose:
-                    print("no")
-                return 1
-    return 0
-
-
-def create_and_log_video_process(video_file, import_path):
-    log_root = uploader.log_rootpath(video_file)
-    if not os.path.isdir(log_root):
-        os.makedirs(log_root)
-    # set the log flags for process
-    log_process = os.path.join(log_root, "video_process.json")
-    import_paths = video_import_paths(video_file)
-    if import_path in import_paths:
-        return
-    import_paths.append(import_path)
-    video_process = load_json(log_process)
-    video_process.update({"sample_paths": import_paths})
-    save_json(video_process, log_process)
-
-
-def video_import_paths(video_file):
-    log_root = uploader.log_rootpath(video_file)
-    if not os.path.isdir(log_root):
-        return []
-    log_process = os.path.join(log_root, "video_process.json")
-    if not os.path.isfile(log_process):
-        return []
-    video_process = load_json(log_process)
-    if "sample_paths" in video_process:
-        return video_process["sample_paths"]
-    return []
-
-
-def create_and_log_process_in_list(
-    process_file_list: List[str],
-    process: str,
-    status: str,
-    verbose: bool = False,
-    mapillary_description: Optional[Dict[str, str]] = None,
-) -> None:
-    if mapillary_description is None:
-        mapillary_description = {}
-    for image in tqdm(process_file_list, desc="Logging"):
-        create_and_log_process(image, process, status, mapillary_description, verbose)
-
-
-def create_and_log_process(
-    image: str,
-    process: str,
-    status: str,
-    mapillary_description: Optional[Any] = None,
-    verbose: bool = False,
-) -> None:
-    if mapillary_description is None:
-        mapillary_description = {}
-
-    # set log path
-    log_root = uploader.log_rootpath(image)
-    # make all the dirs if not there
-    if not os.path.isdir(log_root):
-        os.makedirs(log_root)
-
-    # set the log flags for process
-    log_process = os.path.join(log_root, process)
-    log_process_succes = f"{log_process}_success"
-    log_process_failed = f"{log_process}_failed"
-    log_MAPJson = os.path.join(log_root, process + ".json")
-
-    if not mapillary_description:
-        status = "failed"
-
-    if status == "success":
-        suffix = str(time.strftime("%Y_%m_%d_%H_%M_%S", time.gmtime()))
-        save_json(mapillary_description, log_MAPJson)
-        open(log_process_succes, "w").close()
-        open(f"{log_process_succes}_{suffix}", "w").close()
-        # if there is a failed log from before, remove it
-        if os.path.isfile(log_process_failed):
-            os.remove(log_process_failed)
-    else:
-        open(log_process_failed, "w").close()
-        suffix = str(time.strftime("%Y_%m_%d_%H_%M_%S", time.gmtime()))
-        open(f"{log_process_failed}_{suffix}", "w").close()
-        # if there is a success log from before, remove it
-        if os.path.isfile(log_process_succes):
-            os.remove(log_process_succes)
-        # if there is meta data from before, remove it
-        if os.path.isfile(log_MAPJson):
-            if verbose:
-                print(
-                    f"Warning, {process} in this run has failed, previously generated properties will be removed."
-                )
-            os.remove(log_MAPJson)
-
-    decoded_image = force_decode(image)
-
-    ipc.send(
-        process,
-        {
-            "image": decoded_image,
-            "status": status,
-            "description": mapillary_description,
-        },
-    )
-
-
-def load_geotag_points(
-    process_file_list: List[str], verbose: bool = False
-) -> Tuple[List[str], List[datetime.datetime], List[float], List[float], List[float]]:
-    file_list = []
-    capture_times = []
-    lats = []
-    lons = []
-    directions = []
-
-    for image in tqdm(process_file_list, desc="Loading geotag points"):
-        log_root = uploader.log_rootpath(image)
-        geotag_data = get_geotag_data(log_root, image, verbose)
-        if not geotag_data:
-            create_and_log_process(image, "sequence_process", "failed", verbose=verbose)
-            continue
-
-        # assume all data needed available from this point on
-        file_list.append(image)
-        capture_times.append(
-            datetime.datetime.strptime(
-                geotag_data["MAPCaptureTime"], "%Y_%m_%d_%H_%M_%S_%f"
-            )
-        )
-        lats.append(geotag_data["MAPLatitude"])
-        lons.append(geotag_data["MAPLongitude"])
-        directions.append(
-            geotag_data["MAPCompassHeading"]["TrueHeading"]
-        ) if "MAPCompassHeading" in geotag_data else directions.append(0.0)
-
-        # remove previously created duplicate flags
-        duplicate_flag_path = os.path.join(log_root, "duplicate")
-        if os.path.isfile(duplicate_flag_path):
-            os.remove(duplicate_flag_path)
-
-    return file_list, capture_times, lats, lons, directions
-
-
-def split_sequences(
-    capture_times: List[datetime.datetime],
-    lats: List[float],
-    lons: List[float],
-    file_list: List[str],
-    directions: List[float],
-    cutoff_time: float,
-    cutoff_distance: float,
-    verbose: bool = False,
-) -> List[Dict]:
-    sequences: List[Dict] = []
-    # sort based on time
-    sort_by_time = list(zip(capture_times, file_list, lats, lons, directions))
-    sort_by_time.sort()
-    capture_times, file_list, lats, lons, directions = [
-        list(x) for x in zip(*sort_by_time)
-    ]
-    latlons = list(zip(lats, lons))
-
-    # initialize first sequence
-    sequence_index = 0
-    sequences.append(
-        {
-            "file_list": [file_list[0]],
-            "directions": [directions[0]],
-            "latlons": [latlons[0]],
-            "capture_times": [capture_times[0]],
-        }
-    )
-
-    if len(file_list) >= 1:
-        # diff in capture time
-        capture_deltas = [t2 - t1 for t1, t2 in zip(capture_times, capture_times[1:])]
-
-        # distance between consecutive images
-        distances = [gps_distance(ll1, ll2) for ll1, ll2 in zip(latlons, latlons[1:])]
-
-        # if cutoff time is given use that, else assume cutoff is
-        # 1.5x median time delta
-        if cutoff_time is None:
-            if verbose:
-                print(
-                    "Warning, sequence cut-off time is None and will therefore be derived based on the median time delta between the consecutive images."
-                )
-            median = sorted(capture_deltas)[len(capture_deltas) // 2]
-            if type(median) is not int:
-                median = median.total_seconds()
-            cutoff_time = 1.5 * median
-        else:
-            cutoff_time = float(cutoff_time)
-        cut = 0
-        for i, filepath in enumerate(file_list[1:]):
-            cut_time = capture_deltas[i].total_seconds() > cutoff_time
-            cut_distance = distances[i] > cutoff_distance
-            if cut_time or cut_distance:
-                cut += 1
-                # delta too big, start new sequence
-                sequence_index += 1
-                sequences.append(
-                    {
-                        "file_list": [filepath],
-                        "directions": [directions[1:][i]],
-                        "latlons": [latlons[1:][i]],
-                        "capture_times": [capture_times[1:][i]],
-                    }
-                )
-                if verbose:
-                    if cut_distance:
-                        print(
-                            f"Cut {cut}: Delta in distance {distances[i]} meters is bigger than cutoff_distance {cutoff_distance} meters at {file_list[i + 1]}"
-                        )
-                    elif cut_time:
-                        print(
-                            f"Cut {cut}: Delta in time {capture_deltas[i].total_seconds()} seconds is bigger then cutoff_time {cutoff_time} seconds at {file_list[i + 1]}"
-                        )
-            else:
-                # delta not too big, continue with current
-                # group
-                sequences[sequence_index]["file_list"].append(filepath)
-                sequences[sequence_index]["directions"].append(directions[1:][i])
-                sequences[sequence_index]["latlons"].append(latlons[1:][i])
-                sequences[sequence_index]["capture_times"].append(capture_times[1:][i])
-    return sequences
-
-
-def interpolate_timestamp(
-    capture_times: List[datetime.datetime],
-) -> List[datetime.datetime]:
-    """
-    Interpolate time stamps in case of identical timestamps
-    """
-
-    if len(capture_times) < 2:
-        return capture_times
-
-    # trace identical timestamps (always assume capture_times is sorted)
-    time_dict: OrderedDict[datetime.datetime, Dict] = OrderedDict()
-    for i, t in enumerate(capture_times):
-        if t not in time_dict:
-            time_dict[t] = {"count": 0, "pointer": 0}
-
-            if 0 < i:
-                interval = (t - capture_times[i - 1]).total_seconds()
-                time_dict[capture_times[i - 1]]["interval"] = interval
-
-        time_dict[t]["count"] += 1
-
-    keys = list(time_dict.keys())
-    if len(keys) >= 2:
-        # set time interval as the last available time interval
-        time_dict[keys[-1]]["interval"] = time_dict[keys[-2]]["interval"]
-    else:
-        # set time interval assuming capture interval is 1 second
-        time_dict[keys[0]]["interval"] = time_dict[keys[0]]["count"] * 1.0
-
-    timestamps = []
-
-    # interpolate timestamps
-    for t in capture_times:
-        d = time_dict[t]
-        s = datetime.timedelta(seconds=d["pointer"] * d["interval"] / float(d["count"]))
-        updated_time = t + s
-        time_dict[t]["pointer"] += 1
-        timestamps.append(updated_time)
-
-    return timestamps
-
-
-def get_images_geotags(process_file_list):
-    geotags = []
-    missing_geotags = []
-    for image in tqdm(sorted(process_file_list), desc="Reading gps data"):
-        exif = ExifRead(image)
-        timestamp = exif.extract_capture_time()
-        lon, lat = exif.extract_lon_lat()
-        altitude = exif.extract_altitude()
-        if timestamp and lon and lat:
-            geotags.append((timestamp, lat, lon, altitude))
-            continue
-        if timestamp and (not lon or not lat):
-            missing_geotags.append((image, timestamp))
-        else:
-            print_error(f"Error image {image} does not have captured time.")
-    return geotags, missing_geotags
diff --git a/mapillary_tools/types.py b/mapillary_tools/types.py
new file mode 100644
index 00000000..bc92a542
--- /dev/null
+++ b/mapillary_tools/types.py
@@ -0,0 +1,225 @@
+import datetime
+import sys
+import typing as T
+
+if sys.version_info >= (3, 8):
+    from typing import TypedDict, Literal  # pylint: disable=no-name-in-module
+else:
+    from typing_extensions import TypedDict, Literal
+
+
+class User(TypedDict, total=False):
+    MAPOrganizationKey: str
+    MAPSettingsUsername: str
+    MAPSettingsUserKey: str
+    user_upload_token: str
+
+
+class CompassHeading(TypedDict, total=True):
+    TrueHeading: float
+    MagneticHeading: float
+
+
+class ImageRequired(TypedDict, total=True):
+    MAPLatitude: float
+    MAPLongitude: float
+    MAPCaptureTime: str
+
+
+class Image(ImageRequired, total=False):
+    MAPAltitude: float
+    MAPPhotoUUID: str
+    MAPCompassHeading: CompassHeading
+
+
+class _SequenceOnly(TypedDict, total=True):
+    MAPSequenceUUID: str
+
+
+class Sequence(_SequenceOnly, total=False):
+    MAPCompassHeading: CompassHeading
+
+
+class MetaProperties(TypedDict, total=False):
+    MAPMetaTags: T.Dict
+    MAPDeviceMake: str
+    MAPDeviceModel: str
+    MAPGPSAccuracyMeters: float
+    MAPCameraUUID: str
+    MAPFilename: str
+    MAPOrientation: int
+
+
+class FinalImageDescription(_SequenceOnly, User, Image, MetaProperties):
+    pass
+
+
+class ImageDescriptionJSON(FinalImageDescription):
+    filename: str
+
+
+class FinalImageDescriptionError(TypedDict):
+    filename: str
+    error: T.Dict
+
+
+FinalImageDescriptionOrError = T.Union[
+    FinalImageDescriptionError, FinalImageDescription
+]
+
+
+class FinalImageDescriptionFromGeoJSON(FinalImageDescription):
+    pass
+
+
+UserItemSchema = {
+    "type": "object",
+    "properties": {
+        "MAPOrganizationKey": {"type": "string"},
+        "MAPSettingsUsername": {"type": "string"},
+        "MAPSettingsUserKey": {"type": "string"},
+        "user_upload_token": {"type": "string"},
+    },
+    "required": ["MAPSettingsUserKey", "user_upload_token"],
+    "additionalProperties": False,
+}
+
+FinalImageDescriptionSchema = {
+    "type": "object",
+    "properties": {
+        "MAPOrganizationKey": {
+            "type": "string",
+            "description": "Organization ID. Upload for which organization",
+        },
+        "MAPSettingsUsername": {"type": "string"},
+        "MAPSettingsUserKey": {
+            "type": "string",
+            "description": "User ID. Upload to which Mapillary user",
+        },
+        "MAPLatitude": {"type": "number", "description": "Latitude of the image"},
+        "MAPLongitude": {"type": "number", "description": "Longitude of the image"},
+        "MAPAltitude": {"type": "number", "description": "Altitude of the image"},
+        "MAPCaptureTime": {
+            "type": "string",
+            "description": "Capture time of the image",
+        },
+        "MAPPhotoUUID": {"type": "string"},
+        "MAPCompassHeading": {
+            "type": "object",
+            "properties": {
+                "TrueHeading": {"type": "number"},
+                "MagneticHeading": {"type": "number"},
+            },
+            "required": ["TrueHeading", "MagneticHeading"],
+            "additionalProperties": False,
+        },
+        "MAPSequenceUUID": {
+            "type": "string",
+            "description": "Arbitrary key used to group images",
+        },
+        "MAPMetaTags": {"type": "object"},
+        "MAPDeviceMake": {"type": "string"},
+        "MAPDeviceModel": {"type": "string"},
+        "MAPGPSAccuracyMeters": {"type": "number"},
+        "MAPCameraUUID": {"type": "string"},
+        "MAPFilename": {"type": "string"},
+        "MAPOrientation": {"type": "integer"},
+    },
+    "required": [
+        "MAPLatitude",
+        "MAPLongitude",
+        "MAPCaptureTime",
+    ],
+    "additionalProperties": False,
+}
+
+
+def merge_schema(*schemas: T.Dict):
+    for s in schemas:
+        assert s.get("type") == "object", "must be all object schemas"
+    properties = {}
+    all_required = []
+    additional_properties = True
+    for s in schemas:
+        properties.update(s.get("properties", {}))
+        all_required += s.get("required", [])
+        if "additionalProperties" in s:
+            additional_properties = s["additionalProperties"]
+    return {
+        "type": "object",
+        "properties": properties,
+        "required": list(set(all_required)),
+        "additionalProperties": additional_properties,
+    }
+
+
+ImageDescriptionJSONSchema = merge_schema(
+    FinalImageDescriptionSchema,
+    {
+        "type": "object",
+        "properties": {
+            "filename": {
+                "type": "string",
+                "description": "The image file's path relative to the image directory",
+            },
+        },
+        "required": [
+            "filename",
+        ],
+    },
+)
+
+Process = Literal[
+    "import_meta_data_process",
+    "geotag_process",
+    "sequence_process",
+    "mapillary_image_description",
+]
+
+Status = Literal["success", "failed"]
+
+
+def datetime_to_map_capture_time(time: datetime.datetime) -> str:
+    return datetime.datetime.strftime(time, "%Y_%m_%d_%H_%M_%S_%f")[:-3]
+
+
+def map_capture_time_to_datetime(time: str) -> datetime.datetime:
+    return datetime.datetime.strptime(time, "%Y_%m_%d_%H_%M_%S_%f")
+
+
+class GPXPoint(T.NamedTuple):
+    # Put it first for sorting
+    time: datetime.datetime
+    lat: float
+    lon: float
+    alt: T.Optional[float]
+
+    def as_desc(self) -> Image:
+        desc: Image = {
+            "MAPLatitude": self.lat,
+            "MAPLongitude": self.lon,
+            "MAPCaptureTime": datetime_to_map_capture_time(self.time),
+        }
+        if self.alt is not None:
+            desc["MAPAltitude"] = self.alt
+        return desc
+
+
+class GPXPointAngle(T.NamedTuple):
+    point: GPXPoint
+    angle: T.Optional[float]
+
+    def as_desc(self) -> Image:
+        desc = self.point.as_desc()
+        if self.angle is not None:
+            desc["MAPCompassHeading"] = {
+                "TrueHeading": self.angle,
+                "MagneticHeading": self.angle,
+            }
+        return desc
+
+
+if __name__ == "__main__":
+    import json
+
+    print(json.dumps(ImageDescriptionJSONSchema, indent=4))
diff --git a/mapillary_tools/upload.py b/mapillary_tools/upload.py
index f700c0c1..f9794efe 100755
--- a/mapillary_tools/upload.py
+++ b/mapillary_tools/upload.py
@@ -1,172 +1,111 @@
-import json
 import os
 import sys
+import typing as T
+import json
+import logging
 
-from . import uploader
-from . import processing
-from . import exif_read
+import requests
 
+from . import uploader, types, login, api_v4
 
-def verify_mapillary_tag(filepath):
-    filepath_keep_original = processing.processed_images_rootpath(filepath)
-    if os.path.isfile(filepath_keep_original):
-        filepath = filepath_keep_original
-    """
-    Check that image file has the required Mapillary tag
-    """
-    return exif_read.ExifRead(filepath).mapillary_tag_exists()
+LOG = logging.getLogger(__name__)
 
 
-def upload(
-    import_path,
-    skip_subfolders=False,
-    number_threads=None,
-    max_attempts=None,
-    video_import_path=None,
-    dry_run=False,
-):
-    """
-    Upload local images to Mapillary
-    Args:
-        import_path: Directory path to where the images are stored.
-        verbose: Print extra warnings and errors.
-        skip_subfolders: Skip images stored in subdirectories.
-
-    Returns:
-        Images are uploaded to Mapillary and flagged locally as uploaded.
-    """
-
-    # in case of video processing, adjust the import path
-    if video_import_path:
-        # sanity check if video file is passed
-        if not os.path.isdir(video_import_path) and not os.path.isfile(
-            video_import_path
-        ):
-            print(
-                "Error, video path " + video_import_path + " does not exist, exiting..."
-            )
-            sys.exit(1)
-
-        # set sampling path
-        video_sampling_path = "mapillary_sampled_video_frames"
-        video_dirname = (
-            video_import_path
-            if os.path.isdir(video_import_path)
-            else os.path.dirname(video_import_path)
-        )
-        import_path = (
-            os.path.join(os.path.abspath(import_path), video_sampling_path)
-            if import_path
-            else os.path.join(os.path.abspath(video_dirname), video_sampling_path)
+def read_image_descriptions(desc_path: str):
+    if not os.path.isfile(desc_path):
+        raise RuntimeError(
+            f"Image description file {desc_path} not found. Please process it first. Exiting..."
         )
 
+    descs: T.List[types.ImageDescriptionJSON] = []
+    if desc_path == "-":
+        try:
+            descs = json.load(sys.stdin)
+        except json.JSONDecodeError:
+            raise RuntimeError(f"Invalid JSON stream from stdin")
+    else:
+        with open(desc_path) as fp:
+            try:
+                descs = json.load(fp)
+            except json.JSONDecodeError:
+                raise RuntimeError(f" Invalid JSON file {desc_path}")
+    descs = [desc for desc in descs if "error" not in desc]
+    return descs
+
+
+def zip_images(
+    import_path: str,
+    zip_dir: str,
+    desc_path: T.Optional[str] = None,
+):
     # basic check for all
     if not import_path or not os.path.isdir(import_path):
-        print(f"Error, import directory {import_path} does not exist, exiting...")
-        sys.exit(1)
-
-    # get list of file to process
-    total_file_list = uploader.get_total_file_list(import_path, skip_subfolders)
-    upload_file_list = uploader.get_upload_file_list(import_path, skip_subfolders)
-    success_file_list = uploader.get_success_upload_file_list(
-        import_path, skip_subfolders
-    )
-    to_finalize_file_list = uploader.get_finalize_file_list(
-        import_path, skip_subfolders
-    )
-
-    if len(success_file_list) == len(total_file_list):
-        print("All images have already been uploaded")
-    else:
-        # verify the images in the upload list, they need to have the image
-        # description and certain MAP properties
-        upload_file_list = [f for f in upload_file_list if verify_mapillary_tag(f)]
-
-        if not len(upload_file_list) and not len(to_finalize_file_list):
-            print("No images to upload.")
-            print(
-                'Please check if all images contain the required Mapillary metadata. If not, you can use "mapillary_tools process" to add them'
+        raise RuntimeError(f"Error, import directory {import_path} does not exist")
+
+    if desc_path is None:
+        desc_path = os.path.join(import_path, "mapillary_image_description.json")
+
+    descs = read_image_descriptions(desc_path)
+
+    if not descs:
+        LOG.warning(f"No images found in {desc_path}. Exiting...")
+        return
+
+    uploader.zip_image_dir(import_path, descs, zip_dir)
+
+
+def fetch_user_items(
+    user_name: T.Optional[str] = None, organization_key: T.Optional[str] = None
+) -> types.User:
+    if user_name is None:
+        all_user_items = login.list_all_users()
+        if not all_user_items:
+            raise RuntimeError("No Mapillary account found. Add one with --user_name")
+        if len(all_user_items) == 1:
+            user_items = all_user_items[0]
+        else:
+            raise RuntimeError(
+                f"Found multiple Mapillary accounts. Please specify one with --user_name"
             )
-            sys.exit(1)
-
-        if upload_file_list:
-            # get upload params for the manual upload images, group them per sequence
-            # and separate direct upload images
-            params = {}
-            list_per_sequence_mapping = {}
-            direct_upload_file_list = []
-            for image in upload_file_list:
-                log_root = uploader.log_rootpath(image)
-
-                # read upload params
-                upload_params_path = os.path.join(
-                    log_root, "upload_params_process.json"
-                )
-                if os.path.isfile(upload_params_path):
-                    with open(upload_params_path, "r") as fp:
-                        params[image] = json.load(fp)
-                    sequence = params[image]["key"]
-                    list_per_sequence_mapping.setdefault(sequence, []).append(image)
-                else:
-                    direct_upload_file_list.append(image)
-
-                # read image descriptions
-                description_path = os.path.join(
-                    log_root, "mapillary_image_description.json"
-                )
-                if not os.path.isfile(description_path):
-                    raise RuntimeError(
-                        f"Please run process first because {description_path} is not generated"
-                    )
-                with open(description_path, "r") as fp:
-                    description = json.load(fp)
-                assert not set(description).intersection(
-                    params.get(image, {})
-                ), f"Parameter conflicting {description} and {params.get(image, {})}"
-                params.setdefault(image, {}).update(description)
-
-            # inform how many images are to be uploaded and how many are being skipped
-            # from upload
-
-            print(
-                f"Uploading {len(upload_file_list)} images with valid mapillary tags (Skipping {len(total_file_list) - len(upload_file_list)})"
+    else:
+        user_items = login.authenticate_user(user_name)
+
+    if organization_key is not None:
+        try:
+            resp = api_v4.fetch_organization(
+                user_items["user_upload_token"], organization_key
             )
+        except requests.HTTPError as ex:
+            raise login.wrap_http_exception(ex) from ex
+        org = resp.json()
+        LOG.info(f"Uploading to organization: {json.dumps(org)}")
+        user_items = T.cast(
+            types.User, {**user_items, "MAPOrganizationKey": organization_key}
+        )
+    return user_items
+
 
-            if direct_upload_file_list:
-                raise RuntimeError(
-                    f"Found {len(direct_upload_file_list)} files for direct upload which is not supported in v4"
-                )
-
-            total_sequences = len(list_per_sequence_mapping)
-            for idx, sequence_uuid in enumerate(list_per_sequence_mapping):
-                metadata = {
-                    "total_sequences": total_sequences,
-                    "sequence_idx": idx,
-                }
-                uploader.upload_sequence_v4(
-                    list_per_sequence_mapping[sequence_uuid],
-                    sequence_uuid,
-                    params,
-                    metadata=metadata,
-                    dry_run=dry_run,
-                )
-
-        if to_finalize_file_list:
-            params = {}
-            sequences = []
-            for image in to_finalize_file_list:
-                log_root = uploader.log_rootpath(image)
-                upload_params_path = os.path.join(
-                    log_root, "upload_params_process.json"
-                )
-                if os.path.isfile(upload_params_path):
-                    with open(upload_params_path, "rb") as jf:
-                        image_params = json.load(jf)
-                        sequence = image_params["key"]
-                        if sequence not in sequences:
-                            params[image] = image_params
-                            sequences.append(sequence)
-
-            uploader.flag_finalization(to_finalize_file_list)
-
-    uploader.print_summary(upload_file_list)
+def upload(
+    import_path: str,
+    desc_path: T.Optional[str] = None,
+    user_name: T.Optional[str] = None,
+    organization_key: T.Optional[str] = None,
+    dry_run=False,
+):
+    if os.path.isfile(import_path):
+        user_items = fetch_user_items(user_name, organization_key)
+        uploader.upload_zipfile(import_path, user_items, dry_run=dry_run)
+
+    elif os.path.isdir(import_path):
+        if desc_path is None:
+            desc_path = os.path.join(import_path, "mapillary_image_description.json")
+
+        descs = read_image_descriptions(desc_path)
+        if not descs:
+            LOG.warning(f"No images found in {desc_path}. Exiting...")
+            return
+
+        user_items = fetch_user_items(user_name, organization_key)
+        uploader.upload_image_dir(import_path, descs, user_items, dry_run=dry_run)
+    else:
+        raise RuntimeError(f"Expect {import_path} to be either file or directory")
diff --git a/mapillary_tools/upload_api_v4.py b/mapillary_tools/upload_api_v4.py
index b01d7254..17821f77 100644
--- a/mapillary_tools/upload_api_v4.py
+++ b/mapillary_tools/upload_api_v4.py
@@ -42,7 +42,7 @@ def upload(
         data: T.IO[bytes],
         offset: T.Optional[int] = None,
         chunk_size: int = DEFAULT_CHUNK_SIZE,
-    ) -> requests.Response:
+    ) -> str:
         if chunk_size <= 0:
             raise ValueError("Expect positive chunk size")
 
@@ -81,11 +81,17 @@ def upload(
             offset == self.entity_size
         ), f"offset ends at {offset} but the entity size is {self.entity_size}"
 
-        return resp
+        payload = resp.json()
+        try:
+            return payload["h"]
+        except KeyError:
+            raise RuntimeError(
+                f"Upload server error: File handle not found in the upload response {resp.text}"
+            )
 
     def finish(
         self, file_handle: str, organization_id: T.Optional[T.Union[str, int]] = None
-    ) -> requests.Response:
+    ) -> int:
         headers = {
             "Authorization": f"OAuth {self.user_access_token}",
         }
@@ -95,10 +101,61 @@ def finish(
         if organization_id is not None:
             data["organization_id"] = organization_id
 
-        return requests.post(
+        resp = requests.post(
             f"{MAPILLARY_GRAPH_API_ENDPOINT}/finish_upload", headers=headers, json=data
         )
 
+        resp.raise_for_status()
+
+        data = resp.json()
+
+        cluster_id = data.get("cluster_id")
+        if cluster_id is None:
+            raise RuntimeError(
+                f"Upload server error: failed to create the cluster {resp.text}"
+            )
+
+        return T.cast(int, cluster_id)
+
+
+# A mock class for testing only
+class FakeUploadService(UploadService):
+    upload_path = os.getenv("MAPILLARY_UPLOAD_PATH", "mapillary_public_uploads")
+
+    def upload(
+        self,
+        data: T.IO[bytes],
+        offset: T.Optional[int] = None,
+        chunk_size: int = DEFAULT_CHUNK_SIZE,
+    ) -> str:
+        if offset is None:
+            offset = self.fetch_offset()
+        os.makedirs(FakeUploadService.upload_path, exist_ok=True)
+        filename = os.path.join(FakeUploadService.upload_path, self.session_key)
+        with open(filename, "ab") as fp:
+            data.seek(offset, io.SEEK_CUR)
+            while True:
+                chunk = data.read(chunk_size)
+                if not chunk:
+                    break
+                fp.write(chunk)
+                for callback in self.callbacks:
+                    callback(chunk, None)
+        return self.session_key
+
+    def finish(
+        self, file_handle: str, organization_id: T.Optional[T.Union[str, int]] = None
+    ) -> int:
+        return 1
+
+    def fetch_offset(self) -> int:
+        try:
+            with open(self.session_key, "rb") as fp:
+                fp.seek(0, io.SEEK_END)
+                return fp.tell()
+        except FileNotFoundError:
+            return 0
+
 
 def _file_stats(fp: T.IO[bytes]):
     md5 = hashlib.md5()
@@ -140,7 +197,7 @@ def _file_stats(fp: T.IO[bytes]):
         ) as pbar:
             service.callbacks.append(lambda chunk, _: pbar.update(len(chunk)))
             try:
-                resp = service.upload(fp)
+                file_handle = service.upload(fp)
             except requests.HTTPError as ex:
                 raise wrap_http_exception(ex)
-    print(resp.json())
+    print(file_handle)
diff --git a/mapillary_tools/uploader.py b/mapillary_tools/uploader.py
index e19ee88a..41016ce9 100644
--- a/mapillary_tools/uploader.py
+++ b/mapillary_tools/uploader.py
@@ -1,7 +1,8 @@
 import io
-from typing import List, Optional, Iterable, Generator
+import uuid
+from typing import Optional, Iterable
+import typing as T
 import os
-import sys
 import tempfile
 import hashlib
 import logging
@@ -11,163 +12,18 @@
 
 import requests
 from tqdm import tqdm
+import jsonschema
 
-from . import upload_api_v4
-from . import ipc
-from .login import authenticate_user, wrap_http_exception
+from . import upload_api_v4, types, ipc, exif_write
+from .login import wrap_http_exception
 
 
 MIN_CHUNK_SIZE = 1024 * 1024  # 1MB
 MAX_CHUNK_SIZE = 1024 * 1024 * 32  # 32MB
-LOG = logging.getLogger()
+LOG = logging.getLogger(__name__)
 
 
-def is_image_file(path: str) -> bool:
-    basename, ext = os.path.splitext(os.path.basename(path))
-    return ext.lower() in (".jpg", ".jpeg", ".tif", ".tiff", ".pgm", ".pnm")
-
-
-def is_video_file(path: str) -> bool:
-    basename, ext = os.path.splitext(os.path.basename(path))
-    return ext.lower() in (".mp4", ".avi", ".tavi", ".mov", ".mkv")
-
-
-def flag_finalization(finalize_file_list):
-    for file in finalize_file_list:
-        finalize_flag = os.path.join(log_rootpath(file), "upload_finalized")
-        open(finalize_flag, "a").close()
-
-
-def iterate_files(root: str, recursive=False) -> Generator[str, None, None]:
-    for dirpath, dirnames, files in os.walk(root, topdown=True):
-        if not recursive:
-            dirnames.clear()
-        else:
-            dirnames[:] = [name for name in dirnames if not name.startswith(".")]
-        for file in files:
-            yield os.path.join(dirpath, file)
-
-
-def get_upload_file_list(import_path: str, skip_subfolders: bool = False) -> List[str]:
-    files = iterate_files(import_path, not skip_subfolders)
-    return sorted(
-        file for file in files if is_image_file(file) and preform_upload(file)
-    )
-
-
-# get a list of video files in a video_file
-def get_video_file_list(video_file, skip_subfolders=False):
-    files = iterate_files(video_file, not skip_subfolders)
-    return sorted(file for file in files if is_video_file(file))
-
-
-def get_total_file_list(import_path: str, skip_subfolders: bool = False) -> List[str]:
-    files = iterate_files(import_path, not skip_subfolders)
-    return sorted(file for file in files if is_image_file(file))
-
-
-def get_failed_upload_file_list(
-    import_path: str, skip_subfolders: bool = False
-) -> List[str]:
-    files = iterate_files(import_path, not skip_subfolders)
-    return sorted(file for file in files if is_image_file(file) and failed_upload(file))
-
-
-def get_success_upload_file_list(
-    import_path: str, skip_subfolders: bool = False
-) -> List[str]:
-    files = iterate_files(import_path, not skip_subfolders)
-    return sorted(
-        file for file in files if is_image_file(file) and success_upload(file)
-    )
-
-
-def success_upload(file_path: str) -> bool:
-    log_root = log_rootpath(file_path)
-    upload_success = os.path.join(log_root, "upload_success")
-    upload_finalization = os.path.join(log_root, "upload_finalized")
-    manual_upload = os.path.join(log_root, "manual_upload")
-    success = (
-        os.path.isfile(upload_success) and not os.path.isfile(manual_upload)
-    ) or (
-        os.path.isfile(upload_success)
-        and os.path.isfile(manual_upload)
-        and os.path.isfile(upload_finalization)
-    )
-    return success
-
-
-def get_success_only_manual_upload_file_list(import_path, skip_subfolders=False):
-    files = iterate_files(import_path, not skip_subfolders)
-    return sorted(
-        file
-        for file in files
-        if is_image_file(file) and success_only_manual_upload(file)
-    )
-
-
-def success_only_manual_upload(file_path: str):
-    log_root = log_rootpath(file_path)
-    upload_success = os.path.join(log_root, "upload_success")
-    manual_upload = os.path.join(log_root, "manual_upload")
-    success = os.path.isfile(upload_success) and os.path.isfile(manual_upload)
-    return success
-
-
-def preform_upload(file_path: str) -> bool:
-    log_root = log_rootpath(file_path)
-    process_success = os.path.join(log_root, "mapillary_image_description_success")
-    duplicate = os.path.join(log_root, "duplicate")
-    upload_succes = os.path.join(log_root, "upload_success")
-    upload = (
-        not os.path.isfile(upload_succes)
-        and os.path.isfile(process_success)
-        and not os.path.isfile(duplicate)
-    )
-    return upload
-
-
-def failed_upload(file_path: str) -> bool:
-    log_root = log_rootpath(file_path)
-    process_failed = os.path.join(log_root, "mapillary_image_description_failed")
-    duplicate = os.path.join(log_root, "duplicate")
-    upload_failed = os.path.join(log_root, "upload_failed")
-    failed = (
-        os.path.isfile(upload_failed)
-        and not os.path.isfile(process_failed)
-        and not os.path.isfile(duplicate)
-    )
-    return failed
-
-
-def get_finalize_file_list(
-    import_path: str, skip_subfolders: bool = False
-) -> List[str]:
-    files = iterate_files(import_path, not skip_subfolders)
-    return sorted(
-        file for file in files if is_image_file(file) and preform_finalize(file)
-    )
-
-
-def preform_finalize(file_path: str) -> bool:
-    log_root = log_rootpath(file_path)
-    upload_succes = os.path.join(log_root, "upload_success")
-    upload_finalized = os.path.join(log_root, "upload_finalized")
-    manual_upload = os.path.join(log_root, "manual_upload")
-    finalize = (
-        os.path.isfile(upload_succes)
-        and not os.path.isfile(upload_finalized)
-        and os.path.isfile(manual_upload)
-    )
-    return finalize
-
-
-def print_summary(file_list):
-    # inform upload has finished and print out the summary
-    print(f"Done uploading {len(file_list)} images.")  # improve upload summary
-
-
-def find_root_dir(file_list: Iterable[str]) -> Optional[str]:
+def _find_root_dir(file_list: Iterable[str]) -> Optional[str]:
     """
     find the common root path
     """
@@ -179,221 +35,298 @@ def find_root_dir(file_list: Iterable[str]) -> Optional[str]:
     elif len(dirs) == 1:
         return list(dirs)[0]
     else:
-        return find_root_dir(dirs)
+        return _find_root_dir(dirs)
+
+
+def _group_sequences_by_uuid(
+    image_descs: T.List[types.ImageDescriptionJSON],
+) -> T.Dict[str, T.Dict[str, types.FinalImageDescription]]:
+    sequences: T.Dict[str, T.Dict[str, types.FinalImageDescription]] = {}
+    missing_sequence_uuid = str(uuid.uuid4())
+    for desc in image_descs:
+        sequence_uuid = desc.get("MAPSequenceUUID", missing_sequence_uuid)
+        sequence = sequences.setdefault(sequence_uuid, {})
+        desc_without_filename = {**desc}
+        del desc_without_filename["filename"]
+        sequence[desc["filename"]] = T.cast(
+            types.FinalImageDescription, desc_without_filename
+        )
+    return sequences
 
 
-def upload_sequence_v4(
-    file_list: list,
-    sequence_uuid: str,
-    file_params: dict,
-    metadata: Optional[dict] = None,
-    dry_run=False,
-):
-    if metadata is None:
-        metadata = {}
+def _validate_descs(image_dir: str, image_descs: T.List[types.ImageDescriptionJSON]):
+    for desc in image_descs:
+        jsonschema.validate(instance=desc, schema=types.ImageDescriptionJSONSchema)
+    for desc in image_descs:
+        abspath = os.path.join(image_dir, desc["filename"])
+        if not os.path.isfile(abspath):
+            raise RuntimeError(f"Image path {abspath} not found")
 
-    first_image = list(file_params.values())[0]
-    user_name = first_image["user_name"]
 
-    def _read_captured_at(path):
-        return file_params.get(path, {}).get("MAPCaptureTime", "")
+def upload_image_dir(
+    image_dir: str,
+    image_descs: T.List[types.ImageDescriptionJSON],
+    user_items: types.User,
+    dry_run=False,
+):
+    jsonschema.validate(instance=user_items, schema=types.UserItemSchema)
+    _validate_descs(image_dir, image_descs)
+    sequences = _group_sequences_by_uuid(image_descs)
+    for sequence_idx, images in enumerate(sequences.values()):
+        cluster_id = _zip_and_upload_single_sequence(
+            image_dir,
+            images,
+            user_items,
+            sequence_idx,
+            len(sequences),
+            dry_run=dry_run,
+        )
 
-    # Sorting images by captured_at
-    file_list.sort(key=_read_captured_at)
 
-    root_dir = find_root_dir(file_list)
+def zip_image_dir(
+    image_dir: str, image_descs: T.List[types.ImageDescriptionJSON], zip_dir: str
+):
+    _validate_descs(image_dir, image_descs)
+    sequences = _group_sequences_by_uuid(image_descs)
+    os.makedirs(zip_dir, exist_ok=True)
+    for sequence_uuid, sequence in sequences.items():
+        # FIXME: do not use UUID as filename
+        zip_filename_wip = os.path.join(
+            zip_dir, f"mly_tools_{sequence_uuid}.{os.getpid()}.wip"
+        )
+        with open(zip_filename_wip, "wb") as fp:
+            sequence_md5 = _zip_sequence(image_dir, sequence, fp)
+        zip_filename = os.path.join(zip_dir, f"mly_tools_{sequence_md5}.zip")
+        os.rename(zip_filename_wip, zip_filename)
+
+
+class Notifier:
+    def __init__(self, sequnece_info: T.Dict):
+        self.uploaded_bytes = 0
+        self.sequence_info = sequnece_info
+
+    def notify_progress(self, chunk: bytes, _):
+        self.uploaded_bytes += len(chunk)
+        payload = {
+            "chunk_size": len(chunk),
+            "uploaded_bytes": self.uploaded_bytes,
+            **self.sequence_info,
+        }
+        ipc.send("upload", payload)
+
+
+def _zip_sequence(
+    image_dir: str,
+    sequences: T.Dict[str, types.FinalImageDescription],
+    fp: T.IO[bytes],
+    tqdm_desc: str = "Compressing",
+) -> str:
+    file_list = list(sequences.keys())
+    first_image = list(sequences.values())[0]
+
+    root_dir = _find_root_dir(file_list)
     if root_dir is None:
+        sequence_uuid = first_image.get("MAPSequenceUUID")
         raise RuntimeError(f"Unable to find the root dir of sequence {sequence_uuid}")
 
-    credentials = authenticate_user(user_name)
-    user_access_token = credentials["user_upload_token"]
+    sequence_md5 = hashlib.md5()
 
-    def _gen_notify_progress(uploaded_bytes: int):
-        def _notify_progress(chunk: bytes, _):
-            nonlocal uploaded_bytes
-            uploaded_bytes += len(chunk)
-            assert uploaded_bytes <= entity_size
-            payload = {
-                "chunk_size": len(chunk),
-                "sequence_path": root_dir,
-                "sequence_uuid": sequence_uuid,
-                "total_bytes": entity_size,
-                "uploaded_bytes": uploaded_bytes,
-            }
-            if metadata:
-                payload.update(metadata)
-            ipc.send("upload", payload)
+    file_list.sort(key=lambda path: sequences[path]["MAPCaptureTime"])
 
-        return _notify_progress
+    # compressing
+    with zipfile.ZipFile(fp, "w", zipfile.ZIP_DEFLATED) as ziph:
+        for file in tqdm(file_list, unit="files", desc=tqdm_desc):
+            relpath = os.path.relpath(file, root_dir)
+            abspath = os.path.join(image_dir, file)
+            edit = exif_write.ExifEdit(abspath)
+            edit.add_image_description(sequences[file])
+            image_bytes = edit.dump_image_bytes()
+            sequence_md5.update(image_bytes)
+            ziph.writestr(relpath, image_bytes)
 
-    def _build_desc(desc: str) -> str:
-        if metadata is not None:
-            total = metadata.get("total_sequences")
-            idx = metadata.get("sequence_idx")
-            if total is not None and idx is not None:
-                return f"{desc} {idx + 1}/{total}"
-            else:
-                return desc
-        else:
-            return desc
+    return sequence_md5.hexdigest()
 
-    with tempfile.NamedTemporaryFile() as fp:
-        # compressing
-        with zipfile.ZipFile(fp, "w", zipfile.ZIP_DEFLATED) as ziph:
-            with tqdm(
-                total=len(file_list), desc=_build_desc("Compressing"), unit="files"
-            ) as pbar:
-                for fullpath in file_list:
-                    relpath = os.path.relpath(fullpath, root_dir)
-                    ziph.write(fullpath, relpath)
-                    pbar.update(1)
+
+def upload_zipfile(zip_path: str, user_items: types.User, dry_run=False):
+    basename = os.path.basename(zip_path)
+    with open(zip_path, "rb") as fp:
         fp.seek(0, io.SEEK_END)
         entity_size = fp.tell()
 
         # chunk size
-        avg_image_size = int(entity_size / len(file_list))
+        # FIXME
+        avg_image_size = int(entity_size / 32)
         chunk_size = min(max(avg_image_size, MIN_CHUNK_SIZE), MAX_CHUNK_SIZE)
 
-        # md5sum
-        fp.seek(0, io.SEEK_SET)
-        md5 = hashlib.md5()
-        while True:
-            buf = fp.read(MAX_CHUNK_SIZE)
-            if not buf:
-                break
-            md5.update(buf)
-        md5sum = md5.hexdigest()
+        notifier = Notifier(
+            {
+                "sequence_path": zip_path,
+                "sequence_uuid": "",
+                "total_bytes": entity_size,
+                "sequence_idx": 0,
+                "total_sequences": 1,
+            }
+        )
+
+        return _upload_zipfile_fp(
+            user_items,
+            fp,
+            entity_size,
+            chunk_size,
+            session_key=basename,
+            tqdm_desc="Uploading",
+            notifier=notifier,
+            dry_run=dry_run,
+        )
+
+
+def _upload_zipfile_fp(
+    user_items: types.User,
+    fp: T.IO[bytes],
+    entity_size: int,
+    chunk_size: int,
+    session_key: str = None,
+    tqdm_desc: str = "Uploading",
+    notifier: Optional[Notifier] = None,
+    dry_run: bool = False,
+) -> int:
+    """
+    :param fp: the file handle to a zipped sequence file. Will always upload from the beginning
+    :param entity_size: the size of the whole zipped sequence file
+    :param session_key: the upload session key used to identify an upload
+    :return: cluster ID
+    """
+
+    if session_key is None:
+        session_key = str(uuid.uuid4())
+
+    user_access_token = user_items["user_upload_token"]
 
-        # uploading
-        service = upload_api_v4.UploadService(
+    # uploading
+    if dry_run:
+        upload_service: upload_api_v4.UploadService = upload_api_v4.FakeUploadService(
             user_access_token,
-            session_key=f"mly_tools_{md5sum}",
+            session_key=session_key,
             entity_size=entity_size,
         )
-
-        retryable_errors = (
-            requests.HTTPError,
-            requests.ConnectionError,
-            requests.Timeout,
+    else:
+        upload_service = upload_api_v4.UploadService(
+            user_access_token,
+            session_key=session_key,
+            entity_size=entity_size,
         )
 
-        retries = 0
+    retries = 0
 
-        # when it progresses, we reset retries
-        def _reset_retries(_, __):
-            nonlocal retries
-            retries = 0
+    retryable_errors = (
+        requests.HTTPError,
+        requests.ConnectionError,
+        requests.Timeout,
+    )
 
-        while True:
+    # when it progresses, we reset retries
+    def _reset_retries(_, __):
+        nonlocal retries
+        retries = 0
+
+    while True:
+        with tqdm(
+            total=upload_service.entity_size,
+            desc=tqdm_desc,
+            unit="B",
+            unit_scale=True,
+            unit_divisor=1024,
+        ) as pbar:
+            fp.seek(0, io.SEEK_SET)
             update_pbar = lambda chunk, _: pbar.update(len(chunk))
-            service.callbacks = [update_pbar, _reset_retries]
-            with tqdm(
-                total=entity_size,
-                desc=_build_desc("Uploading"),
-                unit="B",
-                unit_scale=True,
-                unit_divisor=1024,
-            ) as pbar:
-                fp.seek(0, io.SEEK_SET)
-                try:
-                    offset = service.fetch_offset()
-                    pbar.update(offset)
-                    service.callbacks.append(_gen_notify_progress(offset))
-                    upload_resp = service.upload(
-                        fp, chunk_size=chunk_size, offset=offset
+            try:
+                offset = upload_service.fetch_offset()
+                # set the initial progress
+                pbar.update(offset)
+                upload_service.callbacks = [
+                    update_pbar,
+                    _reset_retries,
+                ]
+                if notifier:
+                    notifier.uploaded_bytes = offset
+                    upload_service.callbacks.append(notifier.notify_progress)
+                file_handle = upload_service.upload(
+                    fp, chunk_size=chunk_size, offset=offset
+                )
+            except Exception as ex:
+                if retries < 200 and isinstance(ex, retryable_errors):
+                    retries += 1
+                    sleep_for = min(2 ** retries, 16)
+                    LOG.warning(
+                        f"Error uploading, resuming in {sleep_for} seconds",
+                        exc_info=True,
                     )
-                except Exception as ex:
-                    if retries < 200 and isinstance(ex, retryable_errors):
-                        retries += 1
-                        sleep_for = min(2 ** retries, 16)
-                        LOG.warning(
-                            f"Error uploading, resuming in {sleep_for} seconds",
-                            exc_info=True,
-                        )
-                        time.sleep(sleep_for)
-                    else:
-                        if not dry_run:
-                            for path in file_list:
-                                create_upload_log(path, "upload_failed")
-                        raise wrap_http_exception(ex) if isinstance(
-                            ex, requests.HTTPError
-                        ) else ex
+                    time.sleep(sleep_for)
                 else:
-                    break
-
-    upload_resp_json = upload_resp.json()
-    try:
-        file_handle = upload_resp_json["h"]
-    except KeyError:
-        raise RuntimeError(
-            f"File handle not found in the upload response {upload_resp.text}"
-        )
-
-    if dry_run:
-        return
-
-    organization_id = first_image.get("MAPOrganizationKey")
+                    if isinstance(ex, requests.HTTPError):
+                        raise wrap_http_exception(ex) from ex
+                    else:
+                        raise ex
+            else:
+                break
 
-    if organization_id is None:
-        print(f"Finishing upload {sequence_uuid}")
-    else:
-        print(f"Finishing upload {sequence_uuid} for organization {organization_id}")
+    organization_id = user_items.get("MAPOrganizationKey")
 
     # TODO: retry here
-    finish_resp = service.finish(file_handle, organization_id=organization_id)
     try:
-        finish_resp.raise_for_status()
+        return upload_service.finish(file_handle, organization_id=organization_id)
     except requests.HTTPError as ex:
-        for path in file_list:
-            create_upload_log(path, "upload_failed")
-        raise wrap_http_exception(ex)
-
-    # check cluster id
-    finish_data = finish_resp.json()
-    cluster_id = finish_data.get("cluster_id")
-    if cluster_id is None:
-        for path in file_list:
-            create_upload_log(path, "upload_failed")
-        raise RuntimeError(
-            f"Upload server error: failed to create the cluster {finish_resp.text}"
-        )
-    else:
-        print(f"Cluster {cluster_id} created")
+        raise wrap_http_exception(ex) from ex
 
-    for path in file_list:
-        create_upload_log(path, "upload_success")
 
-    flag_finalization(file_list)
+def _zip_and_upload_single_sequence(
+    image_dir: str,
+    sequences: T.Dict[str, types.FinalImageDescription],
+    user_items: types.User,
+    sequence_idx: int,
+    total_sequences: int,
+    dry_run=False,
+) -> int:
+    def _build_desc(desc: str) -> str:
+        return f"{desc} {sequence_idx + 1}/{total_sequences}"
 
+    file_list = list(sequences.keys())
+    first_image = list(sequences.values())[0]
+    sequence_uuid = first_image.get("MAPSequenceUUID")
 
-def log_rootpath(filepath: str) -> str:
-    return os.path.join(
-        os.path.dirname(filepath),
-        ".mapillary",
-        "logs",
-        os.path.splitext(os.path.basename(filepath))[0],
-    )
+    root_dir = _find_root_dir(file_list)
+    if root_dir is None:
+        raise RuntimeError(f"Unable to find the root dir of sequence {sequence_uuid}")
 
+    with tempfile.NamedTemporaryFile() as fp:
+        sequence_md5 = _zip_sequence(
+            image_dir, sequences, fp, tqdm_desc=_build_desc("Compressing")
+        )
 
-def create_upload_log(filepath: str, status: str) -> None:
-    assert status in ["upload_success", "upload_failed"], f"invalid status {status}"
-    upload_log_root = log_rootpath(filepath)
-    upload_log_filepath = os.path.join(upload_log_root, status)
-    opposite_status = {
-        "upload_success": "upload_failed",
-        "upload_failed": "upload_success",
-    }
-    upload_opposite_log_filepath = os.path.join(
-        upload_log_root, opposite_status[status]
-    )
-    suffix = str(time.strftime("%Y_%m_%d_%H_%M_%S", time.gmtime()))
-    if not os.path.isdir(upload_log_root):
-        os.makedirs(upload_log_root)
-        open(upload_log_filepath, "w").close()
-        open(f"{upload_log_filepath}_{suffix}", "w").close()
-    else:
-        if not os.path.isfile(upload_log_filepath):
-            open(upload_log_filepath, "w").close()
-            open(f"{upload_log_filepath}_{suffix}", "w").close()
-        if os.path.isfile(upload_opposite_log_filepath):
-            os.remove(upload_opposite_log_filepath)
+        fp.seek(0, io.SEEK_END)
+        entity_size = fp.tell()
+
+        # chunk size
+        avg_image_size = int(entity_size / len(sequences))
+        chunk_size = min(max(avg_image_size, MIN_CHUNK_SIZE), MAX_CHUNK_SIZE)
+
+        notifier = Notifier(
+            {
+                "sequence_path": root_dir,
+                "sequence_uuid": sequence_uuid,
+                "total_bytes": entity_size,
+                "sequence_idx": sequence_idx,
+                "total_sequences": total_sequences,
+            }
+        )
+
+        return _upload_zipfile_fp(
+            user_items,
+            fp,
+            entity_size,
+            chunk_size,
+            session_key=f"mly_tools_{sequence_md5}.zip",
+            tqdm_desc=_build_desc("Uploading"),
+            notifier=notifier,
+            dry_run=dry_run,
+        )
diff --git a/mapillary_tools/utils.py b/mapillary_tools/utils.py
deleted file mode 100644
index fe386f58..00000000
--- a/mapillary_tools/utils.py
+++ /dev/null
@@ -1,12 +0,0 @@
-def force_decode(string, codecs=None):
-    if codecs is None:
-        codecs = ["utf8", "cp1252"]
-    if isinstance(string, str):
-        return string
-    for i in codecs:
-        try:
-            return string.decode(i)
-        except UnicodeDecodeError:
-            pass
-    print(f"cannot decode string: {string}")
-    return string.decode("utf8", errors="replace")
diff --git a/mypy.ini b/mypy.ini
index bb4d32b3..8e39ec79 100644
--- a/mypy.ini
+++ b/mypy.ini
@@ -23,3 +23,6 @@ ignore_missing_imports = True
 
 [mypy-construct.*]
 ignore_missing_imports = True
+
+[mypy-jsonschema.*]
+ignore_missing_imports = True
diff --git a/requirements.txt b/requirements.txt
index 4a291e2a..79b57334 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,10 +1,9 @@
-exifread==2.1.2
-Piexif @ git+https://github.com/mapillary/Piexif
+exifread==2.3.2
+piexif==1.1.3
 gpxpy==0.9.8
 pymp4==1.1.0
 pynmea2==1.12.0
-python-dateutil==2.7.3
-pytz
 requests==2.20.0
 tqdm>=4.0,<5.0
-tzwhere
+typing_extensions
+jsonschema
diff --git a/schema/image_description_schema.json b/schema/image_description_schema.json
new file mode 100644
index 00000000..9f36b849
--- /dev/null
+++ b/schema/image_description_schema.json
@@ -0,0 +1,87 @@
+{
+    "type": "object",
+    "properties": {
+        "MAPOrganizationKey": {
+            "type": "string",
+            "description": "Organization ID. Upload for which organization"
+        },
+        "MAPSettingsUsername": {
+            "type": "string"
+        },
+        "MAPSettingsUserKey": {
+            "type": "string",
+            "description": "User ID. Upload to which Mapillary user"
+        },
+        "MAPLatitude": {
+            "type": "number",
+            "description": "Latitude of the image"
+        },
+        "MAPLongitude": {
+            "type": "number",
+            "description": "Longitude of the image"
+        },
+        "MAPAltitude": {
+            "type": "number",
+            "description": "Altitude of the image"
+        },
+        "MAPCaptureTime": {
+            "type": "string",
+            "description": "Capture time of the image"
+        },
+        "MAPPhotoUUID": {
+            "type": "string"
+        },
+        "MAPCompassHeading": {
+            "type": "object",
+            "properties": {
+                "TrueHeading": {
+                    "type": "number"
+                },
+                "MagneticHeading": {
+                    "type": "number"
+                }
+            },
+            "required": [
+                "TrueHeading",
+                "MagneticHeading"
+            ],
+            "additionalProperties": false
+        },
+        "MAPSequenceUUID": {
+            "type": "string",
+            "description": "Arbitrary key used to group images"
+        },
+        "MAPMetaTags": {
+            "type": "object"
+        },
+        "MAPDeviceMake": {
+            "type": "string"
+        },
+        "MAPDeviceModel": {
+            "type": "string"
+        },
+        "MAPGPSAccuracyMeters": {
+            "type": "number"
+        },
+        "MAPCameraUUID": {
+            "type": "string"
+        },
+        "MAPFilename": {
+            "type": "string"
+        },
+        "MAPOrientation": {
+            "type": "integer"
+        },
+        "filename": {
+            "type": "string",
+            "description": "The image file's path relative to the image directory"
+        }
+    },
+    "required": [
+        "MAPLatitude",
+        "MAPLongitude",
+        "MAPCaptureTime",
+        "filename"
+    ],
+    "additionalProperties": false
+}


Test patch:
diff --git a/tests/integration/test_process.py b/tests/integration/test_process.py
index 588d870b..9e3d614e 100644
--- a/tests/integration/test_process.py
+++ b/tests/integration/test_process.py
@@ -1,9 +1,11 @@
+import json
 import os
 import subprocess
+import zipfile
 
 import pytest
 import py.path
-
+import exifread
 
 EXECUTABLE = os.getenv("MAPILLARY_TOOLS_EXECUTABLE", "python3 -m mapillary_tools")
 IMPORT_PATH = "tests/integration/mapillary_tools_process_images_provider/data"
@@ -17,9 +19,8 @@
 
 
 @pytest.fixture
-def setup_config(tmpdir):
+def setup_config(tmpdir: py.path.local):
     config_path = tmpdir.mkdir("configs").join("CLIENT_ID")
-    print(f"config path: {config_path}")
     with open(config_path, "w") as fp:
         fp.write(CONFIG_CONTENT)
     yield config_path
@@ -28,187 +29,165 @@ def setup_config(tmpdir):
 
 
 @pytest.fixture
-def setup_data(tmpdir):
+def setup_data(tmpdir: py.path.local):
     data_path = tmpdir.mkdir("data")
-    print(f"config path: {data_path}")
     source = py.path.local(IMPORT_PATH)
     source.copy(data_path)
-    print(f"data dir content: {data_path.listdir()}")
     yield data_path
     if tmpdir.check():
         tmpdir.remove(ignore_errors=True)
 
 
 def test_basic():
-    for option in ["--version", "--help", "--full_help"]:
+    for option in ["--version", "--help"]:
         x = subprocess.run(f"{EXECUTABLE} {option}", shell=True)
-        assert x.returncode == 0
+        assert x.returncode == 0, x.stderr
 
 
-def test_process(setup_config, setup_data):
-    os.environ["GLOBAL_CONFIG_FILEPATH"] = str(setup_config)
+def test_process(setup_data: py.path.local):
     x = subprocess.run(
-        f"{EXECUTABLE} process --import_path {setup_data} --user_name {USERNAME}",
+        f"{EXECUTABLE} process {setup_data}",
         shell=True,
     )
-    assert x.returncode == 0
-
-
-def test_process_boolean_options(setup_config, setup_data):
-    os.environ["GLOBAL_CONFIG_FILEPATH"] = str(setup_config)
-    boolean_options = [
-        "--add_file_name",
-        "--add_import_date",
-        "--exclude_import_path",
-        "--interpolate_directions",
-        "--keep_duplicates",
-        "--keep_original",
-        "--list_file_status",
-        "--local_time",
-        "--move_all_images",
-        "--move_duplicates",
-        "--move_sequences",
-        "--move_uploaded",
-        "--overwrite_EXIF_direction_tag",
-        "--overwrite_EXIF_gps_tag",
-        "--overwrite_EXIF_orientation_tag",
-        "--overwrite_EXIF_time_tag",
-        "--overwrite_all_EXIF_tags",
-        # --private only works with organizations
-        # '--private',
-        # FIXME: AttributeError: module 'mapillary_tools.uploader' has no attribute 'process_upload_finalization'
-        # '--push_images',
-        "--rerun",
-        "--save_as_json",
-        # '--save_local_mapping',
-        "--skip_EXIF_insert",
-        "--skip_subfolders",
-        "--summarize",
-        "--use_gps_start_time",
-        "--verbose",
-        "--windows_path",
-    ]
-    for option in boolean_options:
-        x = subprocess.run(
-            f"{EXECUTABLE} --advanced process --import_path {setup_data} --user_name {USERNAME} {option}",
-            shell=True,
-        )
-        assert x.returncode == 0
-    all_options = " ".join(boolean_options)
+    assert x.returncode == 0, x.stderr
+    desc_path = os.path.join(setup_data, "mapillary_image_description.json")
+    with open(desc_path) as fp:
+        descs = json.load(fp)
+    for desc in descs:
+        assert "filename" in desc
+        assert os.path.isfile(os.path.join(setup_data, desc["filename"]))
+
+
+def validate_and_extract_zip(filename: str):
+    basename = os.path.basename(filename)
+    assert basename.startswith("mly_tools_"), filename
+    assert basename.endswith(".zip"), filename
+    ret = {}
+    import tempfile
+
+    with zipfile.ZipFile(filename) as zipf:
+        with tempfile.TemporaryDirectory() as tempdir:
+            zipf.extractall(path=tempdir)
+            for name in os.listdir(tempdir):
+                with open(os.path.join(tempdir, name), "rb") as fp:
+                    tags = exifread.process_file(fp)
+                    desc_tag = tags.get("Image ImageDescription")
+                    assert desc_tag is not None, tags
+                    desc = json.loads(str(desc_tag.values))
+                    assert isinstance(desc.get("MAPLatitude"), (float, int)), desc
+                    assert isinstance(desc.get("MAPLongitude"), (float, int)), desc
+                    assert isinstance(desc.get("MAPCaptureTime"), str), desc
+                    for key in desc.keys():
+                        assert key.startswith("MAP"), key
+                    ret[name] = desc
+    return ret
+
+
+def test_zip(tmpdir: py.path.local, setup_data: py.path.local):
+    zip_dir = tmpdir.mkdir("zip_dir")
     x = subprocess.run(
-        f"{EXECUTABLE} --advanced process --import_path {setup_data} --user_name {USERNAME} {all_options}",
+        f"{EXECUTABLE} process {setup_data}",
         shell=True,
     )
-    assert x.returncode == 0
-
-
-def test_extract_user_data(setup_config, setup_data):
-    os.environ["GLOBAL_CONFIG_FILEPATH"] = str(setup_config)
+    assert x.returncode == 0, x.stderr
     x = subprocess.run(
-        f"{EXECUTABLE} --advanced extract_user_data --import_path {setup_data} --user_name {USERNAME}",
+        f"{EXECUTABLE} zip {setup_data} {zip_dir}",
         shell=True,
     )
-    assert x.returncode == 0
+    assert x.returncode == 0, x.stderr
+    assert 0 < len(zip_dir.listdir())
+    for file in zip_dir.listdir():
+        validate_and_extract_zip(str(file))
 
 
-def test_extract_geotag_data(setup_config, setup_data):
+def test_upload_image_dir(
+    tmpdir: py.path.local, setup_config: py.path.local, setup_data: py.path.local
+):
     os.environ["GLOBAL_CONFIG_FILEPATH"] = str(setup_config)
+    upload_dir = tmpdir.mkdir("mapillary_public_uploads")
+    os.environ["MAPILLARY_UPLOAD_PATH"] = str(upload_dir)
     x = subprocess.run(
-        f"{EXECUTABLE} --advanced extract_geotag_data --import_path {setup_data}",
+        f"{EXECUTABLE} process {setup_data}",
         shell=True,
     )
-    assert x.returncode == 0
-
-
-def test_extract_import_meta_data(setup_config, setup_data):
-    os.environ["GLOBAL_CONFIG_FILEPATH"] = str(setup_config)
+    assert x.returncode == 0, x.stderr
     x = subprocess.run(
-        f"{EXECUTABLE} --advanced extract_import_meta_data --import_path {setup_data}",
+        f"{EXECUTABLE} upload {setup_data} --dry_run --user_name={USERNAME}",
         shell=True,
     )
-    assert x.returncode == 0
+    for file in upload_dir.listdir():
+        validate_and_extract_zip(str(file))
+    assert x.returncode == 0, x.stderr
 
 
-def test_extract_sequence_data(setup_config, setup_data):
+def test_upload_zip(
+    tmpdir: py.path.local, setup_data: py.path.local, setup_config: py.path.local
+):
     os.environ["GLOBAL_CONFIG_FILEPATH"] = str(setup_config)
+    upload_dir = tmpdir.mkdir("mapillary_public_uploads")
+    os.environ["MAPILLARY_UPLOAD_PATH"] = str(upload_dir)
+    zip_dir = tmpdir.mkdir("zip_dir")
     x = subprocess.run(
-        f"{EXECUTABLE} --advanced extract_sequence_data --import_path {setup_data}",
+        f"{EXECUTABLE} process {setup_data}",
         shell=True,
     )
-    assert x.returncode == 0
-
-
-def test_extract_upload_params(setup_config, setup_data):
-    os.environ["GLOBAL_CONFIG_FILEPATH"] = str(setup_config)
+    assert x.returncode == 0, x.stderr
     x = subprocess.run(
-        f"{EXECUTABLE} --advanced extract_upload_params --import_path {setup_data} --user_name {USERNAME}",
+        f"{EXECUTABLE} zip {setup_data} {zip_dir}",
         shell=True,
     )
-    assert x.returncode == 0
-
-
-def test_exif_insert(setup_config, setup_data):
-    os.environ["GLOBAL_CONFIG_FILEPATH"] = str(setup_config)
-    x = subprocess.run(
-        f"{EXECUTABLE} --advanced exif_insert --import_path {setup_data}", shell=True
-    )
-    assert x.returncode == 0
-
-
-def test_process_csv():
-    pass
-
-
-def test_authenticate():
-    pass
-
-
-@pytest.mark.skip(
-    "skip because no images in the data directory missing geotags, exiting..."
-)
-def test_interpolate_missing_gps(setup_config, setup_data):
-    os.environ["GLOBAL_CONFIG_FILEPATH"] = str(setup_config)
-    x = subprocess.run(
-        f"{EXECUTABLE} --advanced interpolate --import_path {setup_data} --data missing_gps",
-        shell=True,
-    )
-    assert x.returncode == 0
+    assert x.returncode == 0, x.stderr
+    for zfile in zip_dir.listdir():
+        x = subprocess.run(
+            f"{EXECUTABLE} upload {zfile} --dry_run --user_name={USERNAME}",
+            shell=True,
+        )
+        assert x.returncode == 0, x.stderr
+    for file in upload_dir.listdir():
+        validate_and_extract_zip(str(file))
 
 
-def test_interpolate_identical_timestamps(setup_config, setup_data):
+def test_process_and_upload(
+    tmpdir: py.path.local, setup_config: py.path.local, setup_data: py.path.local
+):
     os.environ["GLOBAL_CONFIG_FILEPATH"] = str(setup_config)
+    upload_dir = tmpdir.mkdir("mapillary_public_uploads")
+    os.environ["MAPILLARY_UPLOAD_PATH"] = str(upload_dir)
     x = subprocess.run(
-        f"{EXECUTABLE} --advanced interpolate --import_path {setup_data} --data identical_timestamps",
+        f"{EXECUTABLE} process_and_upload {setup_data} --dry_run --user_name={USERNAME}",
         shell=True,
     )
-    assert x.returncode == 0
+    assert x.returncode == 0, x.stderr
+    for file in upload_dir.listdir():
+        validate_and_extract_zip(str(file))
 
 
-def test_post_process_boolean_options(setup_config, setup_data):
+def test_process_boolean_options(
+    setup_config: py.path.local, setup_data: py.path.local
+):
     os.environ["GLOBAL_CONFIG_FILEPATH"] = str(setup_config)
     boolean_options = [
-        "--list_file_status",
-        "--move_all_images",
-        "--move_duplicates",
-        "--move_sequences",
-        "--move_uploaded",
-        # "--push_images",
-        "--save_as_json",
-        "--save_local_mapping",
+        "--add_file_name",
+        "--add_import_date",
+        "--exclude_import_path",
+        "--interpolate_directions",
+        "--overwrite_EXIF_direction_tag",
+        "--overwrite_EXIF_gps_tag",
+        "--overwrite_EXIF_orientation_tag",
+        "--overwrite_EXIF_time_tag",
+        "--overwrite_all_EXIF_tags",
         "--skip_subfolders",
-        "--summarize",
+        "--windows_path",
     ]
     for option in boolean_options:
         x = subprocess.run(
-            f"{EXECUTABLE} --advanced post_process --import_path {setup_data} {option}",
+            f"{EXECUTABLE} process {setup_data} {option}",
             shell=True,
         )
-        assert x.returncode == 0, x.returncode
-
-
-def test_download():
-    pass
-
-
-def test_send_videos_for_processing():
-    pass
+        assert x.returncode == 0
+    all_options = " ".join(boolean_options)
+    x = subprocess.run(
+        f"{EXECUTABLE} process {setup_data} {all_options}",
+        shell=True,
+    )
+    assert x.returncode == 0, x.stderr
diff --git a/tests/unit/test_exifread.py b/tests/unit/test_exifread.py
index 0f96a8b8..49d1f0f5 100644
--- a/tests/unit/test_exifread.py
+++ b/tests/unit/test_exifread.py
@@ -29,16 +29,6 @@ def load_exif_PIL(filename=TEST_EXIF_FILE):
     return exif_data
 
 
-def read_image_description_general(test_obj, filename):
-    exif_data_PIL = load_exif_PIL()
-    image_description_PIL = str(
-        exif_data_PIL[EXIF_PRIMARY_TAGS_DICT["ImageDescription"]]
-    )
-    exif_data_ExifRead = ExifRead(filename)
-    image_description_ExifRead = str(exif_data_ExifRead.extract_image_description())
-    test_obj.assertEqual(image_description_ExifRead, image_description_PIL)
-
-
 def read_image_history_general(test_obj, filename):
     exif_data_PIL = load_exif_PIL()
     image_history_PIL = str(exif_data_PIL[EXIF_PRIMARY_TAGS_DICT["ImageHistory"]])
@@ -137,9 +127,6 @@ def read_direction_general(test_obj, filename):
 class ExifReadTests(unittest.TestCase):
     """tests for main functions."""
 
-    def test_read_image_description(self):
-        read_image_description_general(self, TEST_EXIF_FILE)
-
     def test_read_orientation(self):
         read_orientation_general(self, TEST_EXIF_FILE)
 
diff --git a/tests/unit/test_geo.py b/tests/unit/test_geo.py
new file mode 100644
index 00000000..d1c89f78
--- /dev/null
+++ b/tests/unit/test_geo.py
@@ -0,0 +1,66 @@
+import datetime
+
+from mapillary_tools.geo import interpolate_lat_lon, Point
+
+
+def approximate(a, b):
+    for i, j in zip(a, b):
+        assert abs(i - j) <= 0.00001
+
+
+def test_interpolate():
+    points = [
+        Point(lon=1, lat=1, time=datetime.datetime.utcfromtimestamp(1), alt=1),
+        Point(lon=2, lat=2, time=datetime.datetime.utcfromtimestamp(2), alt=2),
+    ]
+    a = interpolate_lat_lon(points, datetime.datetime.utcfromtimestamp(-1))
+    approximate(a, (-1.0, -1.0, 44.978182941465036, -1.0))
+
+    a = interpolate_lat_lon(points, datetime.datetime.utcfromtimestamp(0.1))
+    approximate(
+        a,
+        (
+            0.09999999999999987,
+            0.09999999999999987,
+            44.978182941465036,
+            0.09999999999999987,
+        ),
+    )
+
+    a = interpolate_lat_lon(points, datetime.datetime.utcfromtimestamp(1))
+    approximate(a, (1.0, 1.0, 44.978182941465036, 1.0))
+
+    a = interpolate_lat_lon(points, datetime.datetime.utcfromtimestamp(1.2))
+    approximate(
+        a,
+        (
+            1.2000000000000002,
+            1.2000000000000002,
+            44.978182941465036,
+            1.2000000000000002,
+        ),
+    )
+
+    a = interpolate_lat_lon(points, datetime.datetime.utcfromtimestamp(2))
+    approximate(a, (2.0, 2.0, 44.978182941465036, 2.0))
+
+    a = interpolate_lat_lon(points, datetime.datetime.utcfromtimestamp(2.3))
+    approximate(a, (2.3, 2.3, 44.978182941465036, 2.3))
+
+
+def test_interpolate_single():
+    points = [
+        Point(lon=1, lat=1, time=datetime.datetime.utcfromtimestamp(1), alt=1),
+    ]
+    a = interpolate_lat_lon(points, datetime.datetime.utcfromtimestamp(-1))
+    approximate(a, (1.0, 1.0, 0.0, 1.0))
+    a = interpolate_lat_lon(points, datetime.datetime.utcfromtimestamp(0.1))
+    approximate(a, (1.0, 1.0, 0.0, 1.0))
+    a = interpolate_lat_lon(points, datetime.datetime.utcfromtimestamp(1))
+    approximate(a, (1.0, 1.0, 0.0, 1.0))
+    a = interpolate_lat_lon(points, datetime.datetime.utcfromtimestamp(1.2))
+    approximate(a, (1.0, 1.0, 0.0, 1.0))
+    a = interpolate_lat_lon(points, datetime.datetime.utcfromtimestamp(2))
+    approximate(a, (1.0, 1.0, 0.0, 1.0))
+    a = interpolate_lat_lon(points, datetime.datetime.utcfromtimestamp(2.3))
+    approximate(a, (1.0, 1.0, 0.0, 1.0))
diff --git a/tests/unit/test_sequence_processing.py b/tests/unit/test_sequence_processing.py
new file mode 100644
index 00000000..f94c80a0
--- /dev/null
+++ b/tests/unit/test_sequence_processing.py
@@ -0,0 +1,238 @@
+import uuid
+import datetime
+import itertools
+
+from mapillary_tools import process_sequence_properties as psp
+from mapillary_tools.types import (
+    Image,
+    datetime_to_map_capture_time,
+    map_capture_time_to_datetime,
+)
+from mapillary_tools import image_log, types
+
+
+def make_image(
+    lng: float, lat: float, time: float, angle: float = None, filename: str = None
+):
+    if filename is None:
+        filename = str(uuid.uuid4())
+
+    desc: Image = {
+        "MAPLatitude": lat,
+        "MAPLongitude": lng,
+        "MAPCaptureTime": datetime_to_map_capture_time(
+            datetime.datetime.fromtimestamp(time)
+        ),
+    }
+    if angle is not None:
+        desc["MAPCompassHeading"] = {
+            "TrueHeading": angle,
+            "MagneticHeading": angle,
+        }
+    return psp._GPXPoint(desc, filename=filename)
+
+
+def test_split_sequences():
+    sequence = [
+        # s1
+        make_image(1, 1, 1),
+        make_image(1.00001, 1.00001, 2),
+        make_image(1.00002, 1.00002, 2),
+        # s2
+        make_image(1.00090, 1.00090, 2),
+        make_image(1.00091, 1.00091, 3),
+        # s3
+        make_image(1.00092, 1.00092, 19),
+        make_image(1.00093, 1.00093, 28),
+    ]
+    psp.process_sequence(
+        sequence,
+        cutoff_distance=100,
+        cutoff_time=10,
+        interpolate_directions=False,
+        duplicate_distance=0,
+        duplicate_angle=0,
+    )
+    descs = []
+    for image in sequence:
+        status, desc = image_log.read_process_data_from_memory(
+            image.filename, "sequence_process"
+        )
+        assert status == "success", desc
+        descs.append({**desc, "_filename": image.filename})
+    actual_seqs = []
+    for key, seq in itertools.groupby(descs, key=lambda d: d["MAPSequenceUUID"]):
+        actual_seqs.append(list(seq))
+    assert len(actual_seqs) == 3
+    assert [img.filename for img in sequence[0:3]] == [
+        img["_filename"] for img in actual_seqs[0]
+    ]
+    assert [img.filename for img in sequence[3:5]] == [
+        img["_filename"] for img in actual_seqs[1]
+    ]
+    assert [img.filename for img in sequence[5:7]] == [
+        img["_filename"] for img in actual_seqs[2]
+    ]
+
+
+def test_duplication():
+    sequence = [
+        # s1
+        make_image(1, 1, 1, angle=0),
+        make_image(1.00001, 1.00001, 2, angle=1),
+        make_image(1.00002, 1.00002, 2, angle=-1),
+        make_image(1.00003, 1.00003, 2, angle=-2),
+        make_image(1.00009, 1.00009, 2, angle=5),
+        make_image(1.00090, 1.00090, 2, angle=5),
+        make_image(1.00091, 1.00091, 2, angle=-1),
+    ]
+    psp.process_sequence(
+        sequence,
+        cutoff_distance=100000,
+        cutoff_time=100,
+        interpolate_directions=False,
+        duplicate_distance=100,
+        duplicate_angle=5,
+    )
+    descs = []
+    for image in sequence:
+        status, desc = image_log.read_process_data_from_memory(
+            image.filename, "sequence_process"
+        )
+        descs.append((status, desc))
+
+    assert descs[0][0] == "success"
+    assert descs[1][0] == "failed"
+    assert descs[1][1]["type"] == "MapillaryDuplicationError"
+
+    assert descs[2][0] == "failed"
+    assert descs[2][1]["type"] == "MapillaryDuplicationError"
+
+    assert descs[3][0] == "failed"
+    assert descs[3][1]["type"] == "MapillaryDuplicationError"
+
+    assert descs[4][0] == "failed"
+    assert descs[4][1]["type"] == "MapillaryDuplicationError"
+
+    assert descs[5][0] == "success"
+    assert descs[6][0] == "success"
+
+
+def test_interpolation():
+    sequence = [
+        # s1
+        make_image(0, 0, 1, angle=2),
+        make_image(1, 0, 2, angle=123),
+        make_image(1, 1, 2, angle=344),
+        make_image(0, 1, 2, angle=22),
+        make_image(0, 0, 2, angle=-123),
+    ]
+    psp.process_sequence(
+        sequence,
+        cutoff_distance=1000000000,
+        cutoff_time=100,
+        interpolate_directions=True,
+        duplicate_distance=100,
+        duplicate_angle=5,
+    )
+    descs = []
+    for image in sequence:
+        status, desc = image_log.read_process_data_from_memory(
+            image.filename, "sequence_process"
+        )
+        assert status == "success", desc
+        descs.append(desc)
+
+    assert [90, 0, 270, 180, 180] == [
+        int(desc["MAPCompassHeading"]["TrueHeading"]) for desc in descs
+    ]
+
+
+def test_interpolation_single():
+    sequence = [
+        # s1
+        make_image(0, 0, 1, angle=123),
+    ]
+    psp.process_sequence(
+        sequence,
+        cutoff_distance=1000000000,
+        cutoff_time=100,
+        interpolate_directions=True,
+        duplicate_distance=100,
+        duplicate_angle=5,
+    )
+    descs = []
+    for image in sequence:
+        status, desc = image_log.read_process_data_from_memory(
+            image.filename, "sequence_process"
+        )
+        assert status == "success", desc
+        descs.append(desc)
+    assert [123] == [int(desc["MAPCompassHeading"]["TrueHeading"]) for desc in descs]
+
+
+def test_geotag_from_gpx_2():
+    from mapillary_tools.processing import _geotag_from_gpx
+
+    images = [
+        make_image(0, 0, 1),
+        make_image(0, 0, 4),
+        make_image(0, 0, 9),
+        make_image(0, 0, 17),
+        make_image(0, 0, 40),
+    ]
+    image_by_filename = {image.filename: image for image in images}
+
+    points = [
+        types.GPXPoint(
+            lon=1, lat=1.3, time=datetime.datetime.utcfromtimestamp(100), alt=1
+        ),
+        types.GPXPoint(
+            lon=2, lat=1, time=datetime.datetime.utcfromtimestamp(102), alt=2
+        ),
+        types.GPXPoint(
+            lon=2, lat=4, time=datetime.datetime.utcfromtimestamp(104), alt=3
+        ),
+        types.GPXPoint(
+            lon=9, lat=4, time=datetime.datetime.utcfromtimestamp(130), alt=3
+        ),
+    ]
+
+    def _read_image_time(image: str):
+        return map_capture_time_to_datetime(
+            image_by_filename[image].desc["MAPCaptureTime"]
+        )
+
+    _geotag_from_gpx(
+        [image.filename for image in images],
+        points,
+        offset_time=100,
+        offset_angle=369,
+        read_image_time=_read_image_time,
+    )
+    descs = []
+    for image in images:
+        ret = image_log.read_process_data_from_memory(image.filename, "geotag_process")
+        status, desc = ret
+        assert status == "success"
+        descs.append(desc)
+
+    assert descs[0]["MAPLatitude"] == 1.3
+    assert descs[0]["MAPLongitude"] == 1
+    assert descs[0]["MAPCaptureTime"] == "1970_01_01_00_03_20_000"
+    #
+    assert abs(descs[1]["MAPLatitude"] - 2.5) < 0.0001
+    assert abs(descs[1]["MAPLongitude"] - 2) < 0.001
+    assert descs[1]["MAPCaptureTime"] == "1970_01_01_00_03_23_000"
+
+    assert abs(descs[2]["MAPLatitude"] - 4) < 0.0001
+    assert abs(descs[2]["MAPLongitude"] - 3.0769230769230766) < 0.001
+    assert descs[2]["MAPCaptureTime"] == "1970_01_01_00_03_28_000"
+
+    assert abs(descs[3]["MAPLatitude"] - 4) < 0.0001
+    assert abs(descs[3]["MAPLongitude"] - 5.230769230769231) < 0.001
+    assert descs[3]["MAPCaptureTime"] == "1970_01_01_00_03_36_000"
+
+    assert abs(descs[4]["MAPLatitude"] - 4) < 0.0001
+    assert abs(descs[4]["MAPLongitude"] - 11.423076923076923) < 0.001
+    assert descs[4]["MAPCaptureTime"] == "1970_01_01_00_03_59_000"