Package: pov-fabric-helpers
Fabric HelpersThis is a collection of helpers we use inFabricscripts.  They’re primarily
intended to manage Ubuntu servers (12.04 LTS or 14.04 LTS).ContentsFabric HelpersHelpers (aka why would I want this?)Instance management APIUsageUsage as a git submoduleTesting Fabfiles with VagrantChangelog0.3 (2016-09-11)0.2 (2015-08-06)0.1 (2014-11-19)Helpers (aka why would I want this?)APT packages:ensure_apt_not_outdated()- runsapt-getupdateat most once a dayinstall_packages("vimscreenbuild-essential")install_missing_packages("vimscreenbuild-essential")if notpackage_installed('git'):...if notpackage_available('pov-admin-helpers'):...User accounts:ensure_user("myusername")SSH:ensure_known_host("example.comssh-rsaAAA....")Locales:ensure_locales("en","lt")Files and directories:ensure_directory("/srv/data",mode=0o700)upload_file('crontab','/etc/cron.d/mycrontab')generate_file('crontab.in','/etc/cron.d/mycrontab', context, use_jinja=True)download_file('/home/user/.ssh/authorized_keys','https://example.com/ssh.pubkey')GIT:git_clone("git@github.com:ProgrammersOfVilnius/project.git","/opt/project")git_update("/opt/project")PostgreSQL:ensure_postgresql_user("username")ensure_postgresql_db("dbname","owner")Apache:ensure_ssl_key(...)install_apache_website('apache.conf.in','example.com', context, use_jinja=True,modules='sslrewriteproxy_http')`Postfix:install_postfix_virtual_table('virtual','/etc/postfix/virtual.example.com')`make_postfix_public()Keeping a changelog in /root/Changelog (requires
/usr/sbin/new-changelog-entry frompov-admin-tools)changelog("#Installing stuff")changelog_append("#more stuff")changelog_banner("Installingstuff")run_and_changelog("apt-getinstall stuff")plus many other helpers havechangelogand/orchangelog_appendarguments to invoke these implicitly.Instance management APIAll of my fabfiles can manage severalinstancesof a particular service.
Externally this looks likefab instance1 task1 task2 instance2 task3which executes Fabric taskstask1andtask2on instanceinstance1and then executestask3oninstance2.An instance defines various parameters, such aswhat server hosts itwhere on the filesystem it liveswhat Unix user IDs are usedwhat database is used for this instanceetc.To facilitate thispov_fabricprovides three things:AnInstanceclass that should be subclassed to provide your own instancesfrompov_fabricimportInstanceasBaseInstanceclassInstance(BaseInstance):def__init__(self,name,host,home='/opt/sentry',user='sentry',dbname='sentry'):super(Instance,self).Instance.__init__(name,host)self.home=homeself.user=userself.dbname=dbnameand since that’s a bit repetitive there’s a helperfrompov_fabricimportInstanceasBaseInstanceInstance=BaseInstance.with_params(home='/opt/sentry',user='sentry',dbname='sentry',)which is equivalent to the original manual subclassing.(BTW you can also add parameters with no sensible default this way, e.g.BaseInstance.with_params(user=BaseInstance.REQUIRED).)AnInstance.define()class method that defines new instances and
creates tasks for selecting themInstance.define(name='testing',host='root@vagrantbox',)Instance.define(name='production',host='server1.pov.lt',)Instance.define(name='staging',host='server1.pov.lt',home='/opt/sentry-staging',user='sentry-staging',dbname='sentry-staging',)(BTW you can also define aliases withInstance.define_alias('prod','production').)Aget_instance()function that returns the currently selected instance
(or aborts with an error if the user didn’t select one)frompov_fabricimportget_instance@taskdeflook_around():instance=get_instance()withsettings(host_string=instance.host):run('hostname')Previously I used a slightly different command stylefab task1:instance1 task2:instance1 task3:instance2and this can still be supported if you write your tasks like this@taskdeflook_around(instance=None):instance=get_instance(instance)withsettings(host_string=instance.host):run('hostname')Be careful if you mix styles, e.g.fab instance1 task1 task2:instance2 task3will runtask1andtask3oninstance1and it will runtask2oninstance2.UsageGet the latest release from PyPI:pip install pov-fabric-helpersand then import the helpers you want in yourfabfile.pyfromfabric.apiimport...frompov_fabricimport...Usage as a git submoduleYou can add this repository as a git submodulecd~/src/projectgitsubmoduleaddhttps://github.com/ProgrammersOfVilnius/pov-fabric-helpersand in yourfabfile.pyaddsys.path.insert(0,os.path.join(os.path.dirname(__file__),'pov-fabric-helpers'))ifnotos.path.exists(os.path.join(sys.path[0],'pov_fabric.py')):sys.exit("Please run 'git submodule update --init'.")frompov_fabricimport...Testing Fabfiles with VagrantI don’t know about you, but I was never able to write a fabfile.py that worked
on the first try.Vagrantwas very useful for testing fabfiles without
destroying real servers in the process.  Here’s how:Create aVagrantfilesomewhere withVagrant.configure("2")do|config|config.vm.box="ubuntu/precise64"# Ubuntu 12.04config.vm.provider:virtualboxdo|vb|vb.customize["modifyvm",:id,"--memory","1024"]endendRunvagrant upRunvagrantssh-configand copy the snippet to your~/.ssh/config,
but change the name tovagrantbox, e.g.Host vagrantbox
  HostName 127.0.0.1
  User vagrant
  Port 2222
  UserKnownHostsFile /dev/null
  StrictHostKeyChecking no
  PasswordAuthentication no
  IdentityFile ~/.vagrant.d/insecure_private_key
  IdentitiesOnly yes
  LogLevel FATALTest thatssh vagrantboxworksIn yourfabfile.pycreate a testing instanceInstance.define(name='testing',host='vagrant@vagrantbox',...)Test withfab testing installetc.Changelog0.3 (2016-09-11)register_host_key()now takesfingerprintsso you can specify both
MD5 and SHA256 fingerprints.Use eitherregister_host_key(key, fingerprint=md5_fprint)orregister_host_key(key,fingerprints=[md5_fprint,sha256_fprint]).Low-level helperssh_key_fingerprint()now takesforce_md5so you
can insist on MD5 instead of whatever OpenSSH gives you by default (which is
SHA256 for modern OpenSSH).0.2 (2015-08-06)New helpers:git_update(),register_host_key(),ensure_locales(),changelog_banner(),run_and_changelog(),has_new_changelog_message(),install_missing_packages(),package_available(),upload_file(),generate_file(),ensure_directory(),download_file(),install_postfix_virtual_table(),install_apache_website(),ensure_ssl_key().New optional arguments for existing helpers:git_clone()now takesbranchandchangelog.ensure_user()now takesshell,home,create_home, andchangelog.install_packages()now takeschangelog.changelog()now takescontext.changelog_append()now takescontextandoptional.changelog_banner()now takescontextandoptional.Increased safety:all helpers check their arguments for unsafe shell metacharacters.changelog() and friends quote the arguments correctly.Improved instance API:allowstr.format(**instance)(by making Instance a subclass ofdict).allow instance aliases defined viaInstance.define_alias(alias, name)static method.Bugfixes:ensure_postgresql_db()now works correctly on Ubuntu 14.04.run_as_rootnow correctly handlesenv.host_stringwith no
username part.New low-level helpers you’re probably not interested in, unless you’re
writing your own helpers:aslist(),assert_shell_safe(),ssh_key_fingerprint(),render_jinja2(),render_sinterp(),parse_git_repo(),generate_ssl_config(),generate_ssl_key(),generate_ssl_csr(),get_postfix_setting(),parse_postfix_setting(),add_postfix_virtual_map(),add_postfix_setting(),run_as_root().0.1 (2014-11-19)First public release.Helpers:ensure_apt_not_outdated(),package_installed(),install_packages(),ensure_known_host(),ensure_user(),git_clone(),ensure_postgresql_user(),ensure_postgresql_db(),changelog(),changelog_append().Instance API:class Instance,Instance.with_params(),Instance.REQUIRED,Instance.define().instance._asdict().get_instance().Low-level helpers you’re probably not interested in, unless you’re
writing your own helpers:asbool(),postgresql_user_exists(),postgresql_db_exists().