Package: nomad
Nomad is a simple migration application, which specifically takes into account
properties of development with DVCS and is completely agnostic from ORM or
whatever you are using to access your database. It uses simple SQL scripts to
migrate and can run pre- and post-processing routines written in any language
(Python, Ruby or whatever do you use for your application).Tests status:,changelogLayoutNomad’s migration store is a directory withnomad.iniand directories with
migrations inside. Each such directory must containmigration.inito be
recognized as a migration and this directory name is an unique identifier of a
migration.Your directory tree thus will look like this:migrations/
  nomad.ini
  2011-11-11-first-migration/
    migration.ini
    up.sql
  2011-11-12-second-migration/
    migration.ini
    1-pre.py
    2-up.sql
    3-post.py
  2011-11-13-third-migration/
    migration.ini
    1-pre.py
    2-up.sql
    3-post.sql.j2Nomad uses table callednomadto track what was applied already. It’s just a
list of applied migrations and dates when they were applied.InterfaceTo start working, createnomad.ini, and initialize your database (I assume
it already exists):$ nomad initThen you can start creating your first migration:$ nomad create 0-initialPut your ALTERs and CREATEs in0-initial/up.sqland apply a migration:$ nomad apply -a # or nomad apply 0-initialNomad should report which migrations it applied successfully, but you can check
status of that withnomad ls-a(ornomad lsto see only unapplied
migrations).I guess it’s time to create new migration:$ nomad create 1-next -d 0-initial-d0-initialmeans you want your1-nextto depend on0-initial. This
means Nomad will never apply1-nextwithout applying0-initialfirst. You usually want to depend on migrations which created tables you’re
going to alter, or just to make it easier - on the latest available migration.UsageThere are three types of migration files thatnomadsupports:Plain SQL files with the extension.sql. Just put SQL commands you need
to execute in the migration folder and they will be executed.Executable files. All file extensions are supported as long as the file
is executable. These files must contain everything necessary to migrate
your data, including setting up a database connection.nomadwill pass
all of theConfigurationvariables as environmental variables, prefixed
with their section.Template files with the extension.j2. These templates will be
passed through the Jinja2 templating library. You must install thejinja2library for this functionality. The entireConfigurationis
available to the template files as a single dictionary. These could be
useful if you are distributing an application where the end user needs to
control some aspects of the migrations (ie. additional database users and
passwords, additonal database names, etc.).# nomad.ini
[db]
another_user = reader
another_pass = pass# migrations/0001-initial/up.sql.j2
CREATE ROLE {{ db.another_user }};
ALTER ROLE {{ db.another_user }} WITH NOSUPERUSER LOGIN PASSWORD '{{ db.another_pass }}' VALID UNTIL 'infinity';Files inside of each migration folder are executed in lexicographical order.ConfigurationNomad reads database connection information from the[nomad]section of thenomad.inifile.[nomad]
engine = sqla
url = pgsql://user:password@host:port/dbPossible configuration options:engine(required) - SQL engine to use, possible options:sqla- use SQLAlchemy as an adapter, supports everything SQLAlchemy supportsdbapi- use regular DB API, supportssqlite,mysqlandpgsqlurl(required) - URL to database, takes multiple options, see format belowpath- path to migrations (default: directory withnomad.ini)Each migration has its ownmigration.inifile, which, by default, has a
single configuration option,nomad.dependencies, defining which migration
(or migrations) this one depends.You may add your own configuration variables to either thenomad.iniormigration.inifiles and they will be available in your jinja2 templates
as a single dictionary and your executable files as environmental
variables.Note that ini-files are parsed with extended interpolation (use it like${var}or${section.var}).A few predefined variables are provided to every migration:confpath- path tonomad.iniconfdir- path to directory, containingnomad.inidir- path to directory of migrationExample configuration:configrationexecutabletemplate[nomad]
engine = sqla
url = someurl

[foo]
bar = zetaNOMAD_ENGINE = sqla
NOMAD_URL = someurl

FOO_BAR = zeta

NOMAD_CONFPATH = path
NOMAD_CONFDIR = dir1
NOMAD_DIR = dir2nomad.engine = sqla
nomad.url = someurl

foo.bar = zeta

nomad.confpath = path
nomad.confdir = dir1
nomad.dir = dir2URL formatNomad can read connection url to database in a few various ways.nomad.urlconfiguration option is a space separated list of descriptions of how Nomad can
obtain database connection url.The easiest one is simply an url (like in config example). The others are:file:<path-to-file>- a path to file containing connection urlenv:<var-name>- an environment variable (do not prefix with$)py:<python.mod>:<variable.name>- a Python path to a module,
containing a variable with connection urlcmd:<cmd-to-execute>- command to execute to get connection urljson:<path-to-file>:key.0.key- path to file with JSON and then path
to a connection url within JSON objectyaml:<path-to-file>:key.0.key- path to file with YAML and then path
to a connection url within YAML objectini:<path-to-file>:<section.key>- path to INI file (parsed by
configparser with extended interpolation) and then path to a connection url
within this fileAn example:[nomad]
url =
  ini:${confdir}/../settings.ini:db.url
  json:${confdir}/../settings.json:db.url
  sqlite:///${confdir}/../local.dbNotice that in all cases in the end you have to return URL to a database in
normal format, i.e.dbtype://user:pass@host:port/dbname?options.optionsare supported only by pgsql right now, whatever you put there, nomad
will doset ...before every migration. Note that if you do not supply
anything there, nomad setsstatement_timeoutto 1000 ms andlock_timeoutto 500 ms by default.Main ideasThere are no downgrades - nobody ever tests them, and they are rarely
necessary. Just write an upgrade if you need to cancel something.You can write migration in whatever language you want, Nomad only helps you
track applied migrations and dependencies..sqlis treated differently and executed against database, configured innomad.ini.Only.sql,.j2, and executable files (sorry, Windows! - though I am eager to
hear ideas how to support it) are executed. You can put READMEs, pieces of
documentation, whatever you want alongside your migrations.Name matters - everything is executed in order. Order is determined by using
human sort (so thatx-1.sqlis earlier thanx-10.sql, you can always
check sorting withls--sort=version).