Package: dotref
dotrefdotrefis a simple tool to manage dotfiles across multiple devices.
It supports creating directories, symlinks, templating and hierarchical configuration profiles to keep things DRY.Feature HighlightCreate symlinks to dotfiles and directoriesCreate directoriesConfiguration files templatingHierarchical configuration profiles to easily manage dotfiles across multiple devicesWhy another dotfile manager?While there are many dotfile managers available, most of them are either too complex (try to do everything, often in opinionated ways)
or way to limited.
Dotref is an attempt to make simple but useful tool, that focuses just on dotfile management ("do one thing and do it well").Some highlight of what makes dotref different from some other tools:Dotfiles are managed according to hierarchical profiles (that are just simple JSON files).
That was the main reason for the creation of dotref, as at least at the moment it was originally created, no other tool could do that.
This makes it very easy to manage configuration for multiple devices (e.g. servers, desktops, virtual machines) while following DRY principle.Dotref is a single Python script without any dependencies on third-party tools or libraries.
The whole script is just a few hundred lines long and can be easily reviewed and audited by anyone with basic Python knowledge.Dotref doesn't make any assumptions about where dotfiles will be stored.
For example, while many people keep them in a Git repository, they may as well be stored on Dropbox or Nextcloud.OverviewDotref works mostly by just creating symlinks to your dotfiles.
How does it know what to link where? This is described in dotref profile files.ProfilesA profile file is a simple JSON file that describes what you want dotref to do with your dotfiles.By default, all profiles are stored in adotrefdirectory, inside the directory containing your dotfiles.
It's often useful to keep dotfiles in a Git (or other VCS) repository and the dotref profiles should be kept there as well.Profile files can be created manually in your favorite text editor and they have the following structure:{"extends":"... List of parent profile names ...""vars":"... Object with variables for templating ...","create":"... List of directories to create ...","link":"... List of symlinks to create ...","template":"... List of templates to render ..."}Profile name is simply its filename without the.jsonextension.Let's take a look at profile structure:extendsis a list of parent profile names.
If a profileChildextends profileRootthen everything from theRootprofile will also be available in theChildprofile.
If there's an overlap (for example, a child defines a variable with the same name), theChildproperties will override its parent's properties.extendsfield can be omitted for a root profiles.varsis a JSON object with variables for template substitution. An example:"vars": { "email": "john.doe@example.com" }.createis a list of JSON objects describing what directories need to be created.
Each object must have anamefield with the directory name and can optionally havemodewith a string containing octal directory permissions.
An example:"create": [ { "name": "~/games", "mode": "750" } ].
Note thatmodeis always affected byumaskand if not specified the default value is777.linkis a list of JSON objects describing symlinks that need to be created.
Each object must havesrcanddstfields representing link source and destination.
An example:"link": [ { "src": "bashrc", "dst": "~/.bashrc" } ]will create a~/.bashrcsymlink pointing to thebashrcfile in the current directory.templateis a list of JSON objects describing what templates to render.
Each object must havesrcanddstfields representing template source file and its rendered file destination.
An example:"template": [ { "src": "bashrc.tpl", "dst": "~/.bashrc" } ]will create a~/.bashrcfile by rendering a template filebashrc.tpl.Profile hierarchyHaving profile inheritance allows to avoid copy-pasting the same configuration when managing multiple devices.
It can be best explained on example: one can imagine abaseprofile which symlinks configuration for some common tools, shell, text editor, etc.
We can create two other profiles that extendbase: aserverprofile with extra configurations that only relevant on a server,
and adesktopone for desktops.
As a parallel hierarchy, we could create two profiles:homeandwork, each defining variables relevant to corresponding environment.
Then the final profiles likehome_server,work_laptoporhome_laptopcould just extend corresponding profiles.
The whole hierarchy in this case can be visualized as follows:home_server        home_laptop    work_laptop
    |                  |              |
    |\                / \            /|
    | \              /   \          / |
    |  \            /     \        /  |
    |   \          /       \      /   |
 server  \- home -/         desktop   work
    \                         /
     \                       /
      \------- base --------/TemplatingTemplating in dotref is simply a variable substitution in any text file.
A template file is "rendered" when all of its variable placeholders (denoted with leading$sign) replaced with actual variable values from the desired profile.Templating is based on Python's built-intemplates.Here's an example of templating Git configuration file:[user]
    name = $git_name
    email = $email
    signingKey = $git_keyHeregit_name,emailandgit_keyare variables that must be defined in a profile.If a template uses undefined variable, it will throw an error when this template is used.NoteTemplating can also be used to just copy a file to a destination. If a file doesn't have any variables, it will be copied as-is.
This can be convenient for cases when symbolic links are undesirable.CommandsDotref must be invoked with a specific command, which is one of:version,profiles,init,status,sync,unlink.
The only exception is when called with-hflag, which prints help and doesn't require a command.VersionTheversioncommand just prints current dotref version.ProfilesTheprofilescommand prints a list of all profiles in a given repository.
By default, profile files are searched in thedotrefsubdirectory in a current directory, but this can be specified using thed DOTDIR, --dotdir DOTDIRargument.Theprofilescommand can also show detailed information about a single profile, when invoked with-p PROFILE, --profile PROFILEargument.
It will show ancestors tree of the profile and detailed info about which parts of the configuration were taken from which profile.InitTheinitcommand selects and remembers what profile should be used on a given system.
Thus it must always be invoked with the-p PROFILE, --profile PROFILEargument.The selected profile is stored in a file called dotref statefile, under thedotrefdirectory.
This file is named.dotref.jsonby default, but this can be overridden using the-s STATEFILE, --statefile STATEFILEargument.Theinitcommand can be used to change the current profile but this should be done with caution,
as applying one profile on top of another can lead to unexpected results.
It's best to first dodotref unlinkbefore runningdotref initagain.StatusThestatuscommand checks if the system configuration matches the current profile.
It's somewhat similar togit statusand for every entry of a profile (including all it's ancestors) it will show the status of the entry.More about statuses below.SyncThesynccommand tries to bring the system in the desired state described by the current profile.
This command is idempotent, so it can be safely executed multiple times and if the system is already in the desired state it will only print status without doing anything.A profile entry (directory, symlink or template) can be in one of the following states:OK: entry already in the desired state, no changes were madeMISSING: entry doesn't exist in its destinationCREATED: directory was created successfullyLINKED: symlink was created successfullyUNLINKED: symlink was removed successfullyRENDERED: template was rendered to its destination successfullyDIFFERS: rendered template version differs from the actual templateCONFLICT: a conflicting object (file/directory/symlink) already exists at the destinationIt's important to mention that all conflicts must be resolved manually.
For example, if dotref is told to link~/.bashrcbut it already exists and it's not a symlink to the desired file - this will cause a conflict,
and it's up to the user to decide what to do with the existing file.Thesyncoperation processes profile entries in the following order:First all directories are createdLinks are created after the directories, since a link destination could be inside a created directoryTemplates are rendered after the links and directories are created, since template destination could be in any of themUnlinkTheunlinkcommand is the opposite ofsync- it tries to safely remove everything that's described in the current profile and its ancestors.
Theunlinkoperation is very conservative and it won't remove created directories or rendered templates (unless they exactly match to the actual template).Quick StartLet's assume you have adotfilesdirectory with the following files in it:$ ls
bashrc  gitconfig.tpl  i3configThebashrcandi3configrepresent your config files for Bash and i3 respectively.
Thegitconfig.tplfile is a dotref template for the main Git config file:$ cat gitconfig.tpl
[user]
    name = $git_name
    email = $emailAs you can see, it uses two variables:git_nameandemail.Creating profilesThe first step is to create a dotref profiles directory and a root profile:mkdir dotrefLet's createdotref/base.jsonfile with the following content:{"vars":{"git_name":"jdoe","email":"john.doe@example.com"},"create":[{"name":"~/.config/git"}],"link":[{"src":"bashrc","dst":"~/.bashrc"}],"template":[{"src":"gitconfig.tpl","dst":"~/.config/git/config"}]}Here thevarsblock just defines values for our template variables.
Thecreateblock is to create a~/.config/gitdirectory, if it doesn't already exist.
Thelinktells dotref we want to create a symlink~/.bashrcthat will point at ourbashrcfile.
And finally thetemplateblock instructs dotref to render a template filegitconfig.tplat the~/.config/git/configdestination.Now let's create adotref/desktop.jsonfile with the following content:{"extends":["base"],"link":[{"src":"i3config","dst":"~/.config/i3"}]}The"extends": [ "base" ]line tells that this profileextendsbaseprofile,
so everything specified in thebaseprofile will be also available in thedesktopprofile.
And thelinkblock just tells that we want a~/.config/i3symlink to point to thei3configfile.We can see the merged view (view that combines profile entries from all of its ancestors)
of thedesktopprofile with thedotref profilescommand:$ dotref profiles -p desktop
desktop
+-- base

Variables:
    git_name: jdoe (base)
    email:    john.doe@example.com (base)

Create:
    ~/.config/git: default mode (base)

Link:
    i3config: ~/.config/i3
    bashrc:   ~/.bashrc (base)

Template:
    gitconfig.tpl: ~/.config/git/config (base)Selecting profileBefore applying a profile we need to select what profile we want to use on the system.
This is done withdotref initcommand:> dotref init -p desktop
Successfully initialized to use profile desktopDotref remembers the current profile by storing it in thedotref/.dotref.jsonfile, so we don't need to specify it again with every command.Applying profileBefore applying our profile let's check the current status of the system with thedotref statuscommand:$ dotref status
Profile: desktop

Create:
    [MISSING]  ~/.config/git

Link:
    [MISSING]  ./i3config  ->  ~/.config/i3
    [MISSING]  ./bashrc    ->  ~/.bashrc

Template:
    [MISSING]  ./gitconfig.tpl  ->  ~/.config/git/configThis output tells us that~/.config/gitdirectory is missing, as well as the config files for Bash and i3,
and the git config template is missing as well.To apply the profile rundotref sync:$ dotref sync
Profile: desktop

Create:
    [CREATED]  ~/.config/git

Link:
    [LINKED]   ./i3config  ->  ~/.config/i3
    [LINKED]   ./bashrc    ->  ~/.bashrc

Template:
    [RENDERED] ./gitconfig.tpl  ->  ~/.config/git/config

sync completed successfully and no conflicts were detectedHere the dotref tells us that it did everything we asked it to do.
Let's try and read git config:$ cat ~/.config/git/config
[user]
    name = jdoe
    email = john.doe@example.comWe can rundotref syncagain:$ dotref sync
Profile: desktop

Create:
    [OK]       ~/.config/git

Link:
    [OK]       ./i3config  ->  ~/.config/i3
    [OK]       ./bashrc    ->  ~/.bashrc

Template:
    [OK]       ./gitconfig.tpl  ->  ~/.config/git/config

sync completed successfully and no conflicts were detectedTheOKstates tell us that the dotref didn't do anything since the system is already in the desired state.
Let's now edit thegitconfig.tplfile to make it looks like this:[user]
    name = $git_name
    email = $email

[core]
    editor = nvimand rundotref syncagain:$ dotref sync
Profile: desktop

Create:
    [OK]       ~/.config/git

Link:
    [OK]       ./i3config  ->  ~/.config/i3
    [OK]       ./bashrc    ->  ~/.bashrc

Template:
    [RENDERED] ./gitconfig.tpl  ->  ~/.config/git/config

sync completed successfully and no conflicts were detectedDotref realized that the Git config template was changed and re-rendered it.Unlinking profileTo remove symlinks and rendered templates that dotref have created rundotref unlinkcommand:$ dotref unlink
Profile: desktop

Link:
    [UNLINKED] ./i3config  ->  ~/.config/i3
    [UNLINKED] ./bashrc    ->  ~/.bashrc

Template:
    [UNLINKED] ./gitconfig.tpl  ->  ~/.config/git/config

unlink completed successfully and no conflicts were detectedSimilarly tosync, theunlinkcommand is idempotent and can be safely executed multiple times.ExampleFor a real-world example seeovk/dotfilesrepository.CLI Usageusage: dotref [-h] [-p PROFILE] [-d DOTDIR] [-s STATEFILE] [-v] {init,sync,unlink,status,profiles,version}

Simple tool to manage dotfiles

positional arguments:
  {init,sync,unlink,status,profiles,version}
                        Command to execute

options:
  -h, --help            show this help message and exit
  -p PROFILE, --profile PROFILE
                        Name of the profile to use for init and profiles commands. Use profiles to see
                        all available profiles.
  -d DOTDIR, --dotdir DOTDIR
                        Directory containing dotref profiles and state file (default: dotref)
  -s STATEFILE, --statefile STATEFILE
                        Name of the state file in which dotref will keep current profile and settings (default:
                        .dotref.json in the DOTDIR directory)
  -v, --verbose         Produce more verbose outputTerminal ColorsBy default dotref will use ANSI colors if possible.
This can be disabled be setting eitherNO_COLORorDOTREF_NO_COLORenvironment variables
(as perno-colorconvention).InstallationDotref's only dependency is Python 3.6 or newer.Manual installationDotref is just a single Python file with no dependencies, and can be easily installed manually:curl https://raw.githubusercontent.com/ovk/dotref/master/dotref -o dotref
chmod +x dotref
sudo mv dotref /usr/local/binPyPiDotref is also available as PyPi packagedotref.It can be installed with Pip:sudo pip install dotrefArch Linux AUR PackageFor Arch Linux, dotref can be installed from the AURdotrefpackage.WindowsDotref works on Windows and has been tested in WSL2, MSYS2 and Cygwin (it may work in other environments as well).
It can be either installed manually or withpip.Shell CompletionArch package includes shell auto-completion for Bash and Fish shells, which will be installed automatically.
For other types of installations, the completionfilescan be downloaded and installed manually, if desired.