Package: creep
Creep README fileOverviewCreep is an incremental deploy tool. It allows delta update from a local source
(directory, Git repository or archive) to a FTP or SSH remote server. Its
purpose is to deploy applications by incrementally copying local files to
remote servers (e.g. production) with optional pre-processing.Incremental deployment means Creep keeps track of deployed files on all remote
locations. Only modified files are transferred between two deployments. This
tracking mechanism depends on the type of directory used, for exemple Creep uses
revision hashes when deploying from a Git repository.InstallingYou will need an environment with Python 3 (>= 3.6) before installing Creep.Creep can be installed using either PIP or manual install from sources. If you
choose to use PIP, just type the following:$ pip3 install creep
$ creep -h # display help to ensure install worked properlyIf you prefer manual install checkout the Git repository anywhere you want. Then
create a symbolic link in your$PATHto filecreep/creep.py. Last step is
convenient but not mandatory, you can call Creep using full path tocreep.py:$ git clone https://github.com/r3c/creep.git
$ cd creep
$ sudo ln -s creep/creep.py /usr/bin/creep
$ creep -h # display helpAfter installation go to your project folder and continue reading to deploy your
first project.Quick startFirst go to the directory you want to deploy, e.g. thedistfolder of some
project. It can be inside a Git repository or just be a regular folder, Creep
will select a suitable default configuration either way. Create a new.creep.envfile inside this directory with following JSON content:$ cd path/to/your/dist/directory
$ cat > .creep.env << EOF
{
	"default": {
		"connection": "file:////tmp/creep-quickstart"
	}
}
EOFMind the quadruple slash infile:////tmp/creep-quickstartvalue, we'll see
later why this is required. Once file is saved create the directory and execute
Creep with no parameter:$ mkdir /tmp/creep-quickstart
$ creepCreep will notice you're deploying this project for the first time and ask you
to confirm. Answeryto continue. It will display the full list of files in
your project (by scanning file system or Git history) then ask you again to
confirm. Enteryand Creep will deploy your project to directory/tmp/creep-quickstart.Now if you try to execute Creep again you'll see a message saying no action is
required. This is because deployment location now contains an up-to-date
version of your project and Creep saved this information. Try to change some
files (andgit committhem if you were using Git) then execute the command
again. This time Creep will copy only the file you changed rather than the
full project.This basic example shows how to incrementally deploy a project to some local
directory. Next sections will show how to deploy to remote locations (FTP or
SSH) and register several deployment configurations.Environment fileAs we saw during quick start steps, Creep reads deployment location(s) from a
file called anenvironmentfile. It contains one or more named location(s)
pointing to destinations you want to deploy to. By default Creep will search
for an environment file named.creep.envin current directory, but we'll see
later how this can be customized.Environment configuration file uses JSON format and looks like this:{
	"default": {
		"connection": "file:///../webdev/my-project"
	},
	"integration": {
		"connection": "ftp://me:password@my-dev-server/www-data"
	},
	"production": {
		"connection": "ssh://login@my-prod-server.com/www-data"
	}
}Elements in the root object specify an available deployment location. Each one
must have at least aconnectionstring specifying protocol, address,
credentials and/or path. Read details below for more information about supported
protocols.Once environment configuration file is ready you can start using Creep. Just
typecreep <name>where<name>is name of a configured location to initiate
a deployment to this location. You can also specify multiple locations by
runningcreep <name1> <name2> ...or use special location*to deploy
everywhere (creep '*', don't forget to escape the*if you're running Creep
from within a shell). If you don't specify a name Creep will deploy to location
namedefault.When invoked Creep will fetch last deployed revision from remote location and
compute difference. When you deploy for the first time there is no last
deployed revision so Creep will perform a full deploy. After each successful
deployment it will save revision to remote location in a.creep.revfile.Storing revision information on remote location keeps related data altogether
and works well if you're not the only developer doing deployments. In case you
prefer storing them locally, just add a newlocaloption with valuetruefor all affected locations in your.creep.envfile:{
	...
	"integration": {
		"connection": "ftp://me:password@my-dev-server/www-data/",
		"local": true
	},
	...
}You can specify some options depending on the protocol you're using. To specify
options just add aoptionsJSON object property holding required options:{
	...
	"default": {
		"connection": "ssh://www-data@localhost/www.mywebsite.com/",
		"options": {
			"extra": "-o StrictHostKeyChecking=no"
		}
	},
	...
}Here is the list of supported protocols with expected connection string format
and available options:Local file system:Use connection formatfile:///pathwhere path is relative to source
directory.Note the use of triple slash///because file protocol has no hostname.FTP or FTPs:Use connection formatftp://user:pass@host:port/pathwhereuserandpassare optional credentials (anonymous login will be used if they're
missing),portis an optional FTP port which defaults to 21, andpathis
a path relative to FTP user home directory.Use schemeftps://instead offtp://to enable TLS support.Boolean optionpassiveenables (default) or disables passive mode.SSH:Use connection formatssh://user@host:port/pathwith same variables than
the ones used for FTP deployment. No password can be specified here so
you'll need to either enter password manually or setup SSH keys and start
SSH agent.String optionextracan be used to pass parameters to SSH command as shown
in example above.Path is relative by default in all protocols. Add an extra slash/before
your path to specify an absolute path, e.g.file:////opt/myprojectorssh://user@host//opt/myproject.Note that environment files describe information about external locations and
may contain passwords. For those reasons they should be excluded from your
versionning and kept only on machines performing deployments.It's often convenient to keep the environment file at the top-level of the
directory you want to deploy, so that runningcreepwithout any argument will
perform a deployment of this directory using settings from the.creep.envfile it found there. You can create multiple environment files in
sub-directories inside your projet in case they require different deployment
configurations, for example one insrcdirectory to deploy executable code
to your application server and another one inassetsdirectory to deploy
static files to your web server.Definition fileCreep supports another configuration file, calleddefinitionfile. It's used
to define how to detect changes in files and what preprocessing operations
should be applied on files upon transfer. Create a new.creep.deffile and put
it in the same directories your environment files are. As opposed to environment
files this one is bound to your project and should be shared along with other
project files.Definition configuration file uses JSON format and looks like this:{
	"tracker": "hash",
	"options": {
		"algorithm": "md5",
		"follow": false
	},
	"modifiers": [
		{
			"pattern": "\\.dist$",
			"filter": "false"
		}
	]
}Thetrackerproperty specifies how Creep should analyze differences between
work directory and target location. When this option is not specified Creep
will auto-detect it based on current environment. Theoptionsallows you to
tune behavior of the selectedtrackerby specifying custom parameters. Here
is the list of supported trackers and associated options:Git versionning:Specifygittracker so localgitexecutable is used to get diff between
two revisions. When using this mode Creep relies on Git history and only
needs to remember which revision has been deployed. It also allows you to
manually specify the revision you want to deploy through command line
argument.No options are available for this tracker type.File hash:Specifyhashtracker to have Creep computing a hash of each file to detect
differences. This mode has a higher overhead than Git since it has to save
a value for each file rather than one unique revision, but can work with any
regular folder.String optionalgorithmselects the hashing algorithm to be used among
sha1, sha256, sha512 or md5 (default).Boolean optionfollowspecifies whether symbolic links should be
followed or ignored (default).Themodifierspart defines actions to perform on files before they're sent to
remote locations (e.g. rename, compile, minify, obfuscate, etc.). Each modifier
must define a regular expressionpatternproperty to select files it affects,
and processing actions that will be applied on them.Here is an example ofmodifierssection in a definition configuration.
Remember backslashes must be escaped in JSON, hence the double\\used in
regular expression patterns:{
	...
	"modifiers": [
		{
			"pattern": "\\.dist$",
			"filter": "false"
		},
		{
			"pattern": "\\.min\\.js$"
		},
		{
			"pattern": "\\.js$",
			"modify": "uglifyjs --compress --mangle -- '{}'"
		},
		{
			"pattern": "(.*)\\.less$",
			"rename": "\\1.css",
			"modify": "lessc --clean-css '{}'",
			"link": "find . -name '*.less' | xargs grep -Fl \"$(basename '{}')\" || true"
		}
	]
	...
}Creep evaluates modifiers in sequence and one file can only match one modifier:
evaluation stops after the first matched one. For each file matching a modifier,
associated properties (if any) are applied to it. Available modifier properties
are listed below by execution order:renameproperty specifies a new name for file and supports\\nback
references on groups from the regular expression used inpattern(remember
backslashes must be escaped in JSON).In the example above, files ending with.lesswill have their extension
changed to.css: the back reference\\1captured original file name
without extension in associated pattern.linkproperty specifies a shell command expected to output path to all
files that must also be included in the deployment along with matched file.
Command can contain special{}token which will be replaced by absolute
path to matched file. Output must contain one path per line and each path
must be relative to source directory.In the example above, a command usingfindandgrepis used to list all
files referencing currently ones, so they're also sent to remote location
whenever the file they reference is changed.Note the regular expression could have been more specific, but the point is
to be sure to include all changed files when deploying ; a few false
positives will just cause a harmless extra cost due to additional files
being deployed while not changed.modifyproperty specifies a shell command expected to output replacement
contents for the file being sent to remote location. It supports{}token
similar to thelinkproperty above.In the example above, executableuglifyjsis called to minify JavaScript
files (ending with.js) ;uglifyjsprints result to standard output,
which is then used to overwrite contents of matched JavaScript files.Note presence of a rule which matches files with name ending with.min.js:
it doesn't specify any action but prevents the\.js$rule from being
triggered for files that are already minified.chmodproperty specifies the file permission to be applied in octal format.
Mode "0644" (read/write for owner, read-only for everyone else) is used if
this property is not set.filterproperty specifies a shell command used to decide whether file
should be excluded from deployment or not. It supports{}token similar to
thelinkproperty above. If command execution returns a non-zero exit code
then file won't be sent to remote location.In the example above, thefalsecommand is used to exclude files with name
ending with.distfrom deployment.Empty string value can also be used to always exclude files. It's equivalent
to thefalsecommand used in the example above but has better portability.Creep always appends two modifiers to filter to exclude environment and
definition files from deployments. You shouldn't need to change this behavior,
but you may do so by adding explicit modifiers matching them.You can also specify definition configuration as a JSON string instead of file
using-dcommand line option:creep -d '{"tracker": "hash"}'TroubleshootingThis project is still under develpement and may not behave as you would expect.
In case of issue the-v(verbose) switch may help you understanding how your
environment and definition files are used.If you can't figure out what's happening don't hesitate to open an issue on
GitHub or contact me!