Package: luh3417
LUH3417This is a tool to help you implement a WordPress development workflow. It has 3
main features:Snapshot— Take snapshots of a running WordPress instanceRestore— Restore those snapshots in-place or to a different
locationTransfer— Transfer one instance over another using automated
backup, validation and configuration rulesEverything can happen seamlessly inlocalorthrough SSH, allowing you to
work easily on remote servers from your local machine and to transfer instances
from one server to another.Thanks to this, putting your code to production is as simple as:python-mluh3417.transfer-gmy_project.pylocalproductionWhile thesnapshotandrestoreoperations can be used individually, it is
not recommended to use them as the main tools. Indeed,restorecan easily
override an instance without any previous backup. For this reason, it is better
to usetransferwhenever possible. It will ensure your safety within the
workflow that you defined.Installationpip install luh3417UsageLUH3417 is made to use with Python's-moption. This way, if you want to
invoke thesnapshotfeature, the base command will bepython -m luh3417.snapshot.If you prefer, there is also equivalent commands installed in thebindirectory, namelyluh3417_snapshot,luh3417_restoreandluh3417_transfer.All the locations can be in two formats:SSH—user@server:/location/on/serverLocal—/location/on/current/machineThis allows you to transfer data between remote servers and local machine quite
seamlessly.NOTE— You need to use an SSH agent in order for all the features
to work. No password prompt will show up. Usually it's as simple as to typessh-addin your terminal once during your session.snapshotCreates a snapshot of a running WordPress instance. A snapshot is an archive
containing:All PHP/theme/media/etc filesA DB dumpMeta information about how the snapshot was takenUsage syntax:python -m luh3417.snapshot [-h] [-n SNAPSHOT_BASE_NAME] [-t FILE_NAME_TEMPLATE] source backup_dirExample:python -m luh3417.snapshot root@prod-server.com:/var/www/html root@backup-server.com:/var/backups/wpAdditional options:-n/--snapshot-base-name— Base name for your snapshot file. See
the--file-name-templateoption to see how this name is used. The default
name is the database's name.-t/--file-name-template— This template will be used to generate
the snapshot file name. By default it is{base}_{time}.tar.gzbut you can
put whatever you want.{base}and{time}will be replaced respectively
by the base name (see--snapshot-base-name) and the ISO 8601 UTC date.
Independently of the name, the file will be placed in thebackup_dir.restoreRestores a snapshot either in-place to its original location using the embedded
meta-data or to another location using a patch on the meta-data.In addition to just restoring the files and database, the patch can trigger
changes inwp-settings.php, replace values in the database and much more.restorewill essentially override an instance with the content of a
backup, so make sure to use it wisely in order not to loose data. Also, seetransfer.Usage:python -m luh3417.restore [-p PATCH] [-a ALLOW_IN_PLACE] snapshotOptions:-p/--patch— Location to the patch file (see below)-a/--allow-in-place— Allows restoring the backup onto its original
location. This flag is required because otherwise it would be way too easy
to overrideRestore in-placeIf you want to restore a backup to its original location, you just need to
know the file's location and pass the-aflag.python -m luh3417.restore -a root@backup-server.com:/path/to/snapshot.tar.gzNOTE— If the snapshot was made locally, it will always be restored
locally because there is no way for LUH3417 to know the originating server so
it assumes that the snapshot file was not transferred to another machine.Restore to another locationIn order to restore to another location, you need to use a patch filepython -m luh3417.restore -p patch.json root@backup-server.com:/path/to/snapshot.tar.gzHere is an example of patch file:{"args":{"source":"root@new-server.com:/var/www/html"},"owner":"www-data:"}See below for detailed documentation of patch contentargs.sourceSet this one to define where to restore the archive.{"args":{"source":"root@new-server.com:/var/www/html"}}wp_configDatabase configuration from the WordPress{"wp_config":{"db_host":"localhost","db_name":"xxx","db_user":"xxx","db_password":"xxx"}}NOTE— You need to make sure you match those values inphp_defineunless you're usingtransferwhich sets them automaticallyownerThis changes the owner of the files to another one. This only works if:When restoring locally, you run asrootWhen restoring remotely, you login in asroot{"owner":"www-data:"}gitReplaces some directories with a Git repository at a given version{"git":[{"location":"wp-content/themes/jupiter-child","repo":"git@gitlab.com:your_company/jupiter_child.git","version":"master"}]}NOTE—.gitdirectories are excluded from snapshots, so unless
you specify this option there will be no git-enabled directories in the
restored files. On the other hand, git repositories will be created at
specified version, so it might not make sense to specify this option when
restoring a backup in-place.setup_queriesA list of SQL queries to be run after the DB was restored{"setup_queries":["delete from wp_options where option_name = 'gtm4wp-options';"]}php_defineValues to be changed or added inwp-config.php. Any JSON-serializable value
can be used.{"php_define":{"WP_CACHE":false,"WP_SENTRY_ENV":"new-env"}}replace_in_dumpA list of strings with their replacement to be changed in the dump before
restoring it. This is mainly used to change the domain name of the instance.
As WordPress serializes its settings, a simple replace is not possible. This
will use a holistic heuristic which will try to keep PHP-serialized values
correct even if quoted in a MySQL string.NOTE— PHP-serialized values are prefixed by their length, this is
why a simple replace cannot be effective: if the length changes then the
whole value gets corrupted.{"replace_in_dump":[{"search":"https://old-domain.com","replace":"https://new-domain.com"}]}mysql_rootIn order to create the database and set the user password, the script needs
a root access to MySQL. Today, the only supported method issocket, because
it is password-less. However it only works when the server is local and
properly configured (it's the default behavior in Debian-based distros).{"mysql_root":{"method":"socket","options":{"sudo_user":"root","mysql_user":"root"}}}About the options:sudo_user— don't set it if you don't need to sudo to use the socket,
set it torootor whichever user is right otherwise.mysql_user— name of the MySQL user to useouter_filesCreates files on the server's file system. If the file name is relative then
the file is created relatively to the WordPress's root, otherwise it is created
at the specified absolute location.{"outer_files":[{"name":"robots.txt","content":"User-agent: *\nDisallow: /\n"},{"name":"/etc/apache2/sites-available/my-host.conf","content":"<VirtualHost> ..."}]}NOTE— There is not (yet) any form of privilege escalation to
create those files, so the local/remote user must have the rights to create
those files.post_installThose are shell scripts which run on the host server after the install is
complete. Typically, you can enable your virtual host and reload Apache.{"post_install":["a2ensite my-website.com","systemctl reload apache2"]}dnsYou might want to use your DNS provider's API in order to configure the domain
that is going to target your website. LUH3417 integrates withlibcloudin order to
provide an abstraction over the most popular cloud providers.Here is an example entry:{"dns":{"providers":[{"domain":"my-corp.net","provider":"digitalocean","credentials":{"key":"xxxxxx",}}],"entries":[{"type":"alias","params":{"domain":"my-wp.my-corp.net","target":"load-balancer.my-corp.net"}},{"type":"ips","params":{"domain":"dns.my-corp.net","ips":["2606:4700:4700::1111","2606:4700:4700::1001","1.1.1.1","1.0.0.1"]}}]}}Let's break this downprovidersThat's a list of the providers, associated to a domain name. The different keys
are used like this:domain— root domain name managed by this providerprovider— domain name provider (you can get the listhere,
use the lower-case string value)credentials— kwargs to be passed to the constructor of the providerentriesEntries are either a single CNAME either a set of A/AAAA records for a same
domain name. LUH3417 will make sure that all records for this (sub-)domain
match your specification andwill delete other records for that sub-domain.Suppose the following situation:foo.my.orgresolves toA 1.2.3.4But you want it to be a CNAME ofbar.my.orgTheA 1.2.3.4entry will be deleted and aCNAME bar.my.orgwill be
createdNow, let's dig into the options"type" = "alias"That's when you want to create a CNAME.{"type":"alias","params":{"domain":"my-wp.my-corp.net","target":"load-balancer.my-corp.net"}}The two params are:domain— target (sub-)domaintarget— target of the CNAME (aka the value of the record)"type" = "ips"This will set your (sub-)domain to point on a set if IP addresses, preferably
v6 but legacy systems like v4 are still supported.{"type":"ips","params":{"domain":"dns.my-corp.net","ips":["2606:4700:4700::1111","2606:4700:4700::1001","1.1.1.1","1.0.0.1"]}}domain— is the target (sub-)domainips— is a list of IP address that will be set to AAAA and A recordstransferThe main goal of this package is to allow the setup of a custom workflow that
allows easy copy of WordPress instances from an environment from the other.The basic idea is the following:You can specify an origin and target environment namesThere is asettings generatorPython file which will generate all the
settings and patches appropriate for this transfer.It'syour responsibilityto write an settings generator, however there is
an a documented example attached in this repository.Usage:python -m luh3417.transfer [-h] -g SETTINGS_GENERATOR origin targetExample:python -m luh3417.transfer -g example/generator.py develop localTo see the content of the generator file, please refer to theexample/generator.pyfile and especially theallow_transfer()method's documentation which will explain the spirit of
the file.replaceSeeks and replaces serialized values. Values could be in quoted MySQL literals
or serialized PHP values, they will be replaced and the containing string will
be re-serialized.That's a complicated problem due to PHP serialized format which prefixes the
string's length to the string. If you want to replace a value you need to
replace its length.By example:echo serialize("foo");// s:3:"foo";echo serialize("long foo");// s:8:"long foo";//   ^-- see here how the prefix changed from 3 to 8This command will do the replacing in a file and output to another file. Be
careful, it will not warn you of overwrites.Usage:python -m luh3417.replace [-h] -i INPUT -o OUTPUT [-b BEFORE [BEFORE ...]] [-a AFTER [AFTER ...]] [-c CHARSET]Example:python -m luh3417.replace \
    -i dump.sql \
    -o dump_replaced.sql \
    -b old_domain.com \
    -a new_domain.comFAQWhy the nameLUH3417?It's a character from THX1138. The author is not particularly fan of this
movie, however it expresses quite well the feeling of working with WordPress
and especially setting up a professional workflow.Why using Python to code it?It felt to the author that this language was more appropriate for this task
than PHP.Do I need to write Python to use the transfer feature?Yes, fortunately it's pretty easy. The author started withDive Into Python.Why can't the transfer feature have a configuration file instead?A configuration file would mean imposing the skeleton of the author's workflow
onto all users. If such a workflow is suitable for your needs, example code
and tutorial are provided so just have to adapt the code for yourself.LicenseThis project is distributed under the terms of theWTFPL. It comes
void of warranties and if you break things it's on you.