Package: unifi-protect-backup
Unifi Protect BackupA Python based tool for backing up UniFi Protect event clips as they occur.The idea for this project came after realising that if something were to happen, e.g. a fire, or a burglary
that meant I could no longer access my UDM, all the footage recorded by all my nice expensive UniFi cameras
would have been rather pointless. With this tool, all motion and smart detection clips are immediately
backed up to off-site storage thanks torclone, and kept for the configured
retention period.GitHub:https://github.com/ep1cman/unifi-protect-backupPyPI:https://pypi.org/project/unifi-protect-backup/Free software: MITFeaturesListens to events in real-time via the Unifi Protect websocket APIEnsures any previous and/or missed events within the retention period are also backed upSupports uploading to awide range of storage systems usingrcloneAutomatic pruning of old clipsRequirementsPython 3.9+Unifi Protect version 1.20 or higher (as perpyunifiprotect)rcloneinstalled with at least one remote configured.SetupUnifi Protect Account SetupIn order to connect to your unifi protect instance, you will first need to setup a local admin account:Login to yourLocal Portalon your UniFiOS device, and click onUsersOpen theRolestab and clickAdd Rolein the top right.Give the role a name likeunifi protect backupand give itFull Managementpermissions for the unifi protect app.Now switch to theUsertab and clickAdd Userin the top right, and fill out the form. Specific Fields to pay attention to:Role: Must be the role created in the last stepAccount Type:Local Access OnlyClickAddin at the bottom Right.Select the newly created user in the list, and navigate to theAssignmentstab in the left-hand pane, and ensure all cameras are ticked.InstallationThe prefered way to run this tool is using a containerDocker ContainerYou can run this tool as a container if you prefer with the following command.
Remember to change the variable to make your setup.NoteAs of version 0.8.0, the event database needs to be persisted for the tool to function properly
please see the updated commands belowBacking up locally:By default, if no rclone config is provided clips will be backed up to/data.docker run \
  -e UFP_USERNAME='USERNAME' \
  -e UFP_PASSWORD='PASSWORD' \
  -e UFP_ADDRESS='UNIFI_PROTECT_IP' \
  -e UFP_SSL_VERIFY='false' \
  -v '/path/to/save/clips':'/data' \
  -v '/path/to/save/database':/config/database/ \
  ghcr.io/ep1cman/unifi-protect-backupBacking up to cloud storage:In order to backup to cloud storage you need to provide arclone.conffile.If you do not already have arclone.conffile you can create one as follows:$ docker run -it --rm -v $PWD:/root/.config/rclone --entrypoint rclone ghcr.io/ep1cman/unifi-protect-backup configFollow the interactive configuration process, this will create arclone.conffile in your current directory.Finally, start the container:docker run \
  -e UFP_USERNAME='USERNAME' \
  -e UFP_PASSWORD='PASSWORD' \
  -e UFP_ADDRESS='UNIFI_PROTECT_IP' \
  -e UFP_SSL_VERIFY='false' \
  -e RCLONE_DESTINATION='my_remote:/unifi_protect_backup' \
  -v '/path/to/save/clips':'/data' \
  -v '/path/to/rclone.conf':'/config/rclone/rclone.conf' \
  -v '/path/to/save/database':/config/database/ \
  ghcr.io/ep1cman/unifi-protect-backupInstalling on host:Installrclone. Instructions for your platform can be found here:https://rclone.org/install/#quickstartConfigure thercloneremote you want to backup to. Instructions can be found here:https://rclone.org/docs/#configurepip install unifi-protect-backupOptional: Installffprobeso thatunifi-protect-backupcan check the length of the clips it downloadsUsageUsage: unifi-protect-backup [OPTIONS]

  A Python based tool for backing up Unifi Protect event clips as they occur.

Options:
  --version                       Show the version and exit.
  --address TEXT                  Address of Unifi Protect instance  [required]
  --port INTEGER                  Port of Unifi Protect instance  [default: 443]
  --username TEXT                 Username to login to Unifi Protect instance  [required]
  --password TEXT                 Password for Unifi Protect user  [required]
  --verify-ssl / --no-verify-ssl  Set if you do not have a valid HTTPS Certificate for your
                                  instance  [default: verify-ssl]
  --rclone-destination TEXT       `rclone` destination path in the format {rclone remote}:{path on
                                  remote}. E.g. `gdrive:/backups/unifi_protect`  [required]
  --retention TEXT                How long should event clips be backed up for. Format as per the
                                  `--max-age` argument of `rclone`
                                  (https://rclone.org/filtering/#max-age-don-t-transfer-any-file-
                                  older-than-this)  [default: 7d]
  --rclone-args TEXT              Optional extra arguments to pass to `rclone rcat` directly.
                                  Common usage for this would be to set a bandwidth limit, for
                                  example.
  --rclone-purge-args TEXT        Optional extra arguments to pass to `rclone delete` directly.
                                  Common usage for this would be to execute a permanent delete
                                  instead of using the recycle bin on a destination.
                                  Google Drive example: `--drive-use-trash=false`
  --detection-types TEXT          A comma separated list of which types of detections to backup.
                                  Valid options are: `motion`, `person`, `vehicle`, `ring`
                                  [default: motion,person,vehicle,ring]
  --ignore-camera TEXT            IDs of cameras for which events should not be backed up. Use
                                  multiple times to ignore multiple IDs. If being set as an
                                  environment variable the IDs should be separated by whitespace.
  --file-structure-format TEXT    A Python format string used to generate the file structure/name
                                  on the rclone remote.For details of the fields available, see
                                  the projects `README.md` file.  [default: {camera_name}/{event.s
                                  tart:%Y-%m-%d}/{event.end:%Y-%m-%dT%H-%M-%S}
                                  {detection_type}.mp4]
  -v, --verbose                   How verbose the logging output should be.
                                  
                                      None: Only log info messages created by `unifi-protect-
                                      backup`, and all warnings
                                  
                                      -v: Only log info & debug messages created by `unifi-
                                      protect-backup`, and all warnings
                                  
                                      -vv: Log info & debug messages created by `unifi-protect-
                                      backup`, command output, and all warnings
                                  
                                      -vvv Log debug messages created by `unifi-protect-backup`,
                                      command output, all info messages, and all warnings
                                  
                                      -vvvv: Log debug messages created by `unifi-protect-backup`
                                      command output, all info messages, all warnings, and
                                      websocket data
                                  
                                      -vvvvv: Log websocket data, command output, all debug
                                      messages, all info messages and all warnings  [x>=0]
  --sqlite_path TEXT              Path to the SQLite database to use/create
  --color-logging / --plain-logging
                                  Set if you want to use color in logging output  [default: plain-
                                  logging]
  --download-buffer-size TEXT     How big the download buffer should be (you can use suffixes like
                                  "B", "KiB", "MiB", "GiB")  [default: 512MiB]
  --purge_interval TEXT           How frequently to check for file to purge.
                                  
                                  NOTE: Can create a lot of API calls, so be careful if your cloud
                                  provider charges you per api call  [default: 1d]
  --apprise-notifier TEXT         Apprise URL for sending notifications.
                                  E.g: ERROR,WARNING=tgram://[BOT KEY]/[CHAT ID]
                                  
                                  You can use this parameter multiple times to use more than one
                                  notification platform.
                                  
                                  The following notification tags are available (corresponding to
                                  the respective logging levels):
                                  
                                      ERROR, WARNING, INFO, DEBUG, EXTRA_DEBUG, WEBSOCKET_DATA
                                  
                                  If no tags are specified, it defaults to ERROR
                                  
                                  More details about supported platforms can be found here:
                                  https://github.com/caronc/apprise
  --skip-missing                  If set, events which are 'missing' at the start will be ignored. 
                                  Subsequent missing events will be downloaded (e.g. a missed event)  [default: False]
  --download-rate-limit FLOAT     Limit how events can be downloaded in one minute. Disabled by
                                  default
  --max-event-length INTEGER      Only download events shorter than this maximum length, in
                                  seconds  [default: 7200]
  --help                          Show this message and exit.The following environment variables can also be used instead of command line arguments (note, CLI arguments
always take priority over environment variables):UFP_USERNAMEUFP_PASSWORDUFP_ADDRESSUFP_PORTUFP_SSL_VERIFYRCLONE_RETENTIONRCLONE_DESTINATIONRCLONE_ARGSRCLONE_PURGE_ARGSIGNORE_CAMERASDETECTION_TYPESFILE_STRUCTURE_FORMATSQLITE_PATHDOWNLOAD_BUFFER_SIZECOLOR_LOGGINGPURGE_INTERVALAPPRISE_NOTIFIERSSKIP_MISSINGDOWNLOAD_RATELIMITMAX_EVENT_LENGTHFile path formattingBy default, the application will save clips in the following structure on the provided rclone remote:{camera_name}/{event.start:%Y-%m-%d}/{event.end:%Y-%m-%dT%H-%M-%S} {detection_type}.mp4If you wish for the clips to be structured differently you can do this using the--file-structure-formatoption. It uses standardpython format string syntax.The following fields are provided to the format string:event:TheEventobject as perhttps://github.com/briis/pyunifiprotect/blob/master/pyunifiprotect/data/nvr.pyduration_seconds:The duration of the event in secondsdetection_type:A nicely formatted list of the event detection type and the smart detection types (if any)camera_name:The name of the camera that generated this eventYou can optionally format theevent.start/event.endtimestamps as per thestrftimeformatby appending it after a:e.g to get just the date without the time:{event.start:%Y-%m-%d}Skipping initially missing eventsIf you prefer to avoid backing up the entire backlog of events, and would instead prefer to back up events that occur from
now on, you can use the--skip-missingflag. This does not enable the periodic check for missing event (e.g. one that was missed by a disconnection) but instead marks all missing events at start-up as backed up.If you use this feature it is advised that your run the tool once with this flag, then stop it once the database has been created and the events are ignored. Keeping this flag set permanently could cause events to be missed if the tool crashes and is restarted etc.A note aboutrclonebackends and disk wearThis tool attempts to not write the downloaded files to disk to minimise disk wear, and instead streams them directly to
rclone. Sadly, not all storage backends supported byrcloneallow "Stream Uploads". Please refer to theStreamUploadcolumn on this table to see which one do and don't:https://rclone.org/overview/#optional-featuresIf you are using a storage medium with poor write durability e.g. an SD card on a Raspberry Pi, it is advised to avoid
such backends.If you are running on a linux host you can setuprcloneto usetmpfs(which is in RAM) to store its temp files, but this will significantly increase memory usage of the tool.Running Docker Container (LINUX ONLY)Add the following arguments to your docker run command:-e RCLONE_ARGS='--temp-dir=/rclone_tmp'
--tmpfs /rclone_tmpRunning Directly (LINUX ONLY)sudo mkdir /mnt/tmpfs
sudo mount -o size=1G -t tmpfs none /mnt/tmpfs
$ unifi-protect-backup --rclone-args "--temp-dir=/mnt/tmpfs"To make this persist reboots add the following to/etc/fstab:tmpfs /mnt/tmpfs tmpfs nosuid,nodev,noatime 0 0Running Backup Tool as a Service (LINUX ONLY)You can create a service that will run the docker or local version of this backup tool. The service can be configured to launch on boot. This is likely the preferred way you want to execute the tool once you have it completely configured and tested so it is continiously running.First create a service configuration file. You can replaceprotectbackupin the filename below with the name you wish to use for your service, if you change it remember to change the other locations in the following scripts as well.sudo nano /lib/systemd/system/protectbackup.serviceNext edit the content and fill in the 4 placeholders indicated by {}, replace these placeholders (including the leading{and trailing}characters) with the values you are using.[Unit]
Description=Unifi Protect Backup

[Service]
User={your machine username}
Group={your machine user group, could be the same as the username}
Restart=on-abort
WorkingDirectory=/home/{your machine username}
ExecStart={put your complete docker or local command here}

[Install]
WantedBy=multi-user.targetNow enable the service and then start the service.sudo systemctl enable protectbackup.service
sudo systemctl start protectbackup.serviceTo check the status of the service use this command.sudo systemctl status protectbackup.service --no-pagerDebuggingIf you need to debug your rclone setup, you can invoke rclone directly like so:docker run \
    --rm \
    -v /path/to/rclone.conf:/config/rclone/rclone.conf \
    -e RCLONE_CONFIG='/config/rclone/rclone.conf' \
    --entrypoint rclone \
    ghcr.io/ep1cman/unifi-protect-backup \
    {rclone subcommand as per: https://rclone.org/docs/#subcommands}For example to check that your config file is being read properly and list the configured remotes:docker run \
    --rm \
    -v /path/to/rclone.conf:/config/rclone/rclone.conf \
    -e RCLONE_CONFIG='/config/rclone/rclone.conf' \
    --entrypoint rclone \
    ghcr.io/ep1cman/unifi-protect-backup \
    listremotesCreditsAll the contributors who have helped make this project:Heavily utilisespyunifiprotectby@briisAll the cloud functionality is provided byrcloneThis package was created withCookiecutterand thewaynerv/cookiecutter-pypackageproject template.Star History