Package: poof
% poof(1) Version 1.4.0 | Secure cloud storage backup documentationNamepoof- 2-way secure data sync/backup/restore against cloud storage.Synopsis|poofcommand|poof[--confdirTEXT]command|poof[--poofconfTEXT]command|poof[--rcloneconfTEXT]command|poof[--help]DescriptionBackup of local file system directories to cloud storage or other storage, in a
secure manner, and preserving the original files' attributes for later recovery.
The poof! tool offers additional security options:Secure delete the local file system files upon successful process completionEncrypt the remote copiesEliminates all traces of itself in the local file systempoof! leverages tried-and-true tools to perform its tasks:rclone- A command line program for managing files in
cloud storage.Experimental versions ofpoofleverage other operating system and third-party
tools, discussed in this documentation.  For details, see thepoofGitHub
repository.Options--help: Prints brief usage information--confdir: Specifies the configuration files directory--poofconf: Specifies the poof configuration file name--rcloneconf: Specifies the rclone configuration file nameCanonical file name mergesconfdir/poofconforconfdir/rcloneconf.Options are never used in production mode.  They were defined for initial
rollout, testing, etc. but the correct way of running poof! is by letting it
determine the optimal config file system locations on its own.Commandsbackup: Backup to remote without wiping out the local data.cconfig: Ensure that the rclone-poof.conf file exists; creates it if not present.config: Ensure that the poof.conf file exists; creates it if not present.cryptoggle: Toggle remote target encryption ON/OFF.download: Download the files from the cloud storage into their corresponding
directories.econfig: Generate the encrypted rclone configuration scaffold.lconfig: Set poof to run every 6 hours via launchd (macOS)lpurge: Purge poof from launchd and ~/Library/LaunchAgents (macOS)neuter: Neuter this poof installation by deleting its configuration and executables.paths: Output the platform-specific paths to the poof configuration files.upload: Upload to remote and wipe out the local data.verify: Verify the poof and rclone tool configurations.version: Print the software version and quit.Theconfigandcconfigcommands also output their associated
configuration file to the console, and make a copy to the clipboard.Supported StorageS3: AWS Simple Storage ServiceThe development team is evaluating other storage systems and plan to extend
support in poof! backup 2.0, expected to be released in mid 2024.Q2.Filespoof.conf: Cloud storage remote specification, paths to backup/upload, bucket and file
system information.rclone-poof.conf: rclone cloud storage specification for clear and encrypted backups
configuration and secrets.These files location is system-dependent.On macOS:  $HOME/Library/Application Support/poofOn Linux:  $HOME/.config/poofOne-time setupUsepiporpip3to install in the current Python scope (system or virtual
environment):IMPORTANT- the package name waspoof-backupuntil 01.Dec.2021.  Remove
that package and replace withpoofby itself.  Check if this is the case by
runningpoof version- 1.2.7 was the last one published under the old name.pip3installpoofInstallingrcloneInstallrclonein your system viahttps://rclone.org/install/rclonemakes an identical copy of directories and their contents to the
desired cloud storage, including the correct time stamps and permissions.  It's
a more reliable mechanism than building that logic inpoof.ConfigurationThepoofconfiguration forrcloneis specific to this tool.rcloneusers
may have other configurations that in no way conflict withpoofand vice
versa.For the first timepoofconfiguration, run:poofverifypoofvalidates that all component and configuration requirements are
satisfied:installed rclone? - PoofStatus.OK
exists poof.conf? - PoofStatus.MISSING_CONFIG_FILEGenerate a new, basicpoofconfiguration file and validate its contents:poofconfig&&poofverifyResults in:installed rclone? - PoofStatus.OK
exists poof.conf? - PoofStatus.OK
exists rclone-poof.conf? - PoofStatus.MISSING_CONFIG_FILEThercloneconfiguration file forpoofis still missing.  Generate a simple
cloud storagercloneconfiguration file:poofcconfig&&poofverifyThe verification fails becausepoofmust have at least one directory to backup
in addition to the backup of its own configuration, which defaults tounittestuntil overridden by the user.Verification fails becausepoofmust have a minimum of two directories to
backup or upload to the cloud storage:Apoofconfiguration backup directoryOne end-user backup directoryIn most cases,Documentsought to be the first end-user directory.List the configuration paths to make the necessary updates:poofpathsShows these paths on a Mac:poof.conf = /Users/joe-user/Library/Application Support/poof/poof.conf
rclone-poof.conf = /Users/joe-user/Library/Application Support/poof/rclone-poof.confIt shows these paths on Linux:poof.conf = /home/joe-user/.config/poof/poof.conf
rclone-poof.conf = /home/joe-user/.config/poof/rclone-poof.confEnter the full path(s) to each directory you wish to back up, no~or$HOME.
Inpoof.conf:{
  "bucket": "poofbackup-joe-user-206ce7879351",
  "confFile": "/Users/joe-user/Library/Application Support/poof/poof.conf",
  "paths": {
    "/Users/joe-user/Documents": "Documents",
    "/Users/joe-user/Downloads": "Downloads",
    "/Users/joe-user/Library/Application Support/poof": "poof-config"
  },
  "remote": "poof-backup"
}Last, configure the appropriate credentials inrclone.conffor the cloud
storage intended for backup.  This example uses an Amazon S3 configuration,
replace the bogus credentials with your own.Verify the configuation one last time:poofverifyWill show thatpoofis ready for normal operations:installed rclone? - PoofStatus.OK
exists poof.conf? - PoofStatus.OK
exists rclone-poof.conf? - PoofStatus.OK
configuration appears to be valid and has valid credentialsIMPORTANTpoofcreates or updates snapshots of the latest file system contents in the
local file system or the cloud storage.Restoring data from the cloud storage is a destructive operation in the target
file system.This is by design becausepoofclones and synchronizes the
source file system to the targets.  Backups are never incremental -- they are
alwayssnapshots.Regular backupspoofvalidates its own configuration before backing up/uploading or restoring
data.  It will fail if its own configuration or any of its required tools
configurations are invalid.Runpoof backupas often as needed or required to copy all the directories in
thepoofconfiguration to the cloud storage.  It may automated viacronorlaunchd.UploadRunpoof uploadwhen there is need to sync the local file system directories,
then removes all the local files and directories (local directories wipe).RestoreTo restore a backup from the cloud to the local file system:Validate the configurationRunpoof downloadThe file system synchronization process may take from a few minutes to several
hours, depending on the number of files involved, the lengt of the files, and
the connection speed.Encrypted backups/uploadspoofleveragesrcloneencrypted remotes, if they are defined and available,
beginning with version1.2.0.  Future releases will implementcryptconfiguration generators from withinpoof, for now this relies onrcloneuntil automation, key storage, and operational security issues are resolved.Encryption details:File content encryption usesNaCl SecretBoxFile and directory names are separated by '/', padded to a multiple of 16
bytes, then encrypted with EME and AES with a 256-bit key.Implications:File and directory names with the same exact name will encrypt the same wayFile and directory names which start the same won't have a common prefixAll names are encrypted to lower case alphanumeric stringsPadding characters (e.g. =) are strippedSupports case-insensitve remotes (e.g. Windows)ThercloneCrypt documentation provides a thorough discussion ofhow thecryptremote implementation works.Pre-requisitesWorkingpoofconfigurationWorkingrcloneconfiguration for poof with a working typecryptremoteSamplepoof.conf:{"bucket":"poofbackup-joe-user-206ce7879351","confFile":"/Users/joe-user/Library/Application Support/poof/poof.conf","paths":{"/Users/joe-user/CryptoWallet":"CryptoWallet","/Users/joe-user/Documents":"Documents","/Users/joe-user/Downloads":"Downloads","/Users/joe-user/Library/Application Support/poof":"poof-conf"},"remote":"poof-backup"}Sample validrclone-poof.conf.  The[poof-crypt[section was generated usingrcloneconfiguration for the password.  Notice that the remote definition uses
the target bucket inpoof.conf:[poof-backup]type=s3provider=AWSenv_auth=falseaccess_key_id=BOGUS-KEY-USE-YOURSsecret_access_key=BOGUS-SECRET-KEY-USE-YOURSregion=eu-west-1location_constraint=eu-west-1acl=privatestorage_class=STANDARD_IAchunk_size=8Mupload_concurrency=2server_side_encryption=AES256[poof-crypt]type=cryptremote=poof:poofbackup-joe-user-206ce7879351password=BOGUS-PASSWORDpassword2=BOGUS-PASSWORD2Enabling and disabling encryptionEnabling and disabling encryption is accomplished by editing theremoteattribute in thepoofconfiguration file, to point at thepoof-cryptremote
instead of thepoof-backupremote.{"bucket":"poofbackup-joe-user-206ce7879351","confFile":"/Users/joe-user/Library/Application Support/poof/poof.conf","paths":{"/Users/joe-user/CryptoWallet":"CryptoWallet","/Users/joe-user/Documents":"Documents","/Users/joe-user/Downloads":"Downloads","/Users/joe-user/Library/Application Support/poof":"poof-conf"},"remote":"poof-crypt"}Running the upload or backup commands copies the files and directories to the
cloud storage using encrypted directory and file names, and encrypting the files
to prevent unauthorized viewing by the cloud storage provider:poofbackupDisabling encryption only requires to point the remote back to the cloud storage
remote definition, instead of the encrypted remote.Effects on backup/upload and downloadFile and directory names are preserved, as in the cleartext backup, in the local
file system.File and directory names are encrypted in the cloud storage target.  File names
are transparent topoofandrclone- listing the encypted cloud file system
names with valid credentials shows them in cleartext on the client, but they are
obfuscated in the remote as described at the beginning of this section.**https://raw.githubusercontent.com/poof-backup/poof/master/assets/sample-S3-dir-list.png**Operational securitypoof! operates within a privacy continuum that ranges from simple data backup
and restore for safekeeping, to full target encryption and self-destruction in
case of risk of local system compromise.An expanded description of the operational considerations and workflows is
available viahttps://github.com/poof-backup/poof/blob/master/ops-docs/index.mdThe poof! model covers 4 data security threat levels:https://raw.githubusercontent.com/poof-backup/poof/master/assets/backup-continuum.pngLevel 1:  backup and restoreAs a user, I want to make periodic backups of one or more local file system
directories, and trust or otherwise have no privacy concerns regarding the cloud
storage provider.Level 2:  upload and restoreAs a user, I need to make a backup of my local directories but need to wipe them
out upon completion.  I trust or otherwise have no privacy concerns regarding
the cloud storage provider.Examples:Bought a new computer and must move the data directories from the old to the
new systemThe current system must be surrendered to a distrusted third-party like a
repair shopConfiguration info for threat levels 1 and 2Thepoof.confandrclone-poof.confconfiguration files are uploaded to the
cloud if the poof! configuration file is included as part of the configuration.
These files are stored in cleartext in the cloud storage, and can be viewed or
downloaded by anyone with access permissions.Level 3:  crypt backup, restoreAs a user, I want to make periodic backups of one or more local system
directories, and do not trust the cloud storage provider.In this situation, the user may distrust the storage provider and wants to
prevent their data from being mined or otherwise accessed without authorization.Level 4:  crypt upload, restoreAs a user, I need to make a backup of my local directories but need to wipe them
out upon completion.  I do not trust the cloud storage provider.Personal or business sensitive data is stored in one or more directories
managed by poof!Bought a new computer and must move the data directories from the old to the
new systemThe current system must be surrendered to a distrusted third-party like a
repair shopThe cloud storage provider is known or suspected to inspect or mine storage
contents or to grant access to third-parties to do soConfiguration info for threat levels 3 and 4The poof! configuration files are stored in cleartext in the local file system,
but are encrypted in the cloud storage if they are present in thepoof.confconfiguration file.In the case of a level 4 threat,poof uploadwill also wipe out its own
configuration, and remove itself and all its dependencies from the local file
system.rcloneis left alone because there may be other legitimate uses for
it other thanpoofintegration.Preserving the poof! configuration for threat levels 3 and 4Theconfigandcconfigcommands display the current configuration and copy
the configuration files to the clipboard.  The user may then store them in a
separate, secure, unrelated system for later restoring the files.  For example,
this command:poofconfigDisplays the configuration:{
  "bucket": "poofbackup-joe-user-206ce7879351",
  "confFile": "/Users/joe-user/Library/Application Support/poof/poof.conf",
  "paths": {
    "/Users/joe-user/CryptoWallet": "CryptoWallet",
    "/Users/joe-user/Documents": "Documents",
    "/Users/joe-user/Downloads": "Downloads",
    "/Users/joe-user/Library/Application Support/poof": "poof-conf"
  },
  "remote": "poof-crypt"
}The configuration file also available in the clipboard.  You may verify this by
pasting into any text editor, or using your GUI's tools for viewing the
clipboard.Generate these files, store them in a safe place, separate from the cloud
storage holding your backups or the system that you backed up, and use them to
restore your data to a secure, safe system, when the threat level drops.Runningpoofon a schedulepoofcan be configured to run via scheduler, whether manually or using its
built-in commands.  Setting up the scheduler usingpoofcommands guarantees
that all configuration files are in the pre-defined locations thatpoofknows,
and will wipe them out upon execution of theneutercommand, to eliminate
traces of itself.launchd (macOS only)lconfigdisplays or generates a newlaunchdconfiguration file, including
the corresponding.plistfile and enabling the configuration for the current
user vialaunchctlcommands.lpurgedisables thepoofschedule, and deletes the.plistfile and any
temporary log files associated with the scheduler output.crond (Linux, other UNIX-like OS)Not implemented as of version 1.2.10.BugsSee GitHub issues:https://github.com/poof-backup/poof/issuesFiles$HOME/Library/Application Support/poof/poof.conf$HOME/Library/Application Support/poof/rclone-poof.conf$HOME/Library/LaunchAgents/org.pypi.poof.plist/tmp/6CC9-4821-827B-8596B684ECA9/com.apple.ContentStoreAgent-$USER-err.dat/tmp/6CC9-4821-827B-8596B684ECA9/com.apple.ContentStoreAgent-$USER-out.datAuthorEugene "pr3d4t0r" Ciurana and the poof backup contributors <poof.project AT cime.net>See alsocrontab(1)rclone(1)launchctl(1)launchd(8)