Package: ql
quillquill is the "driver" forspin.systems,
and packaged on PyPi asql.A more detailed description of the package namespace can be found below,
but for everyday usage the commands needed are:importquillasqlql.ssm.check_manifest()# Check all component repos' statusql.ssm.repos_df# print the summary dataframeql.fold.wire.standup()# build or rebuild any 'wire' module MMD documents as HTML pagesql.fold.cut.standup()# build or rebuild any 'cut' module Jinja templates as HTML pagesql.remote_push_manifest()# Add/commit/push any dirty repos, triggering CI buildThe last step will also rebuild any sites which have dirty repos (to avoid content partially falling behind,
when for instance the templates for a site are changed).Structureql⠶scan: Read.mmdfilesscan⠶lever: Parse.mmdfile formatql⠶manifest: Readspin.systemsconfigurationql⠶fold: Manage*.spin.systemssubdomainsfold⠶address: Manage subdomain address shorthandfold⠶wire: Emit HTML websites from.mmdfilesfold⠶cut: Emit HTML websites from.htmlJinja templates and.mdfilesHelper CLINote: for reasons unknown, thedefoptCLI only works on 3.10+Installation adds aqlcommand (for deployment) andcyl(for local preview).
These two commands handle repo-internal and -external output of generated sites respectively
(cylcurrently only for thefold.cutmodule, notfold.wire,
so not yet sufficient to replace in the CI pipeline).usage: ql [-h] [-d [DOMAINS_LIST ...]] [--incremental] [-n] [-r] [-w] [-v]
          [-g] [--internal | --no-internal] [--version]

Configure input filtering and output display.

options:
  -h, --help            show this help message and exit
  -d [DOMAINS_LIST ...], --domains-list [DOMAINS_LIST ...]
                        Names of the subdomains to build.
                        (default: None)
  --incremental         compute MD5 checksums for each of the generated files.
                        (default: False)
  -n, --no-render       Dry run mode.
                        (default: False)
  -r, --recheck         Only regenerate files whose input checksum has changed since the last
                        incremental run.
                        (default: False)
  -w, --watch           Rebuild the domains continuously (incompatible with incremental/recheck).
                        (default: False)
  -v, --verbose         Whether to announce what tasks are being carried out.
                        (default: False)
  -g, --gitlab-ci       Source the manifest repos (i.e. git pull them) and then stash
                        the changes after building, switch to the www branch, pop the
                        stash and push the changes (i.e. publish the website content).
                        (default: False)
  --internal, --no-internal
                        Whether to build sites internal (ql) or external (cyl,
                        not including 'fold.wire') to the repo.
                        (default: True)
  --version             show program's version number and exitSo for example:cyl-dporePopulates~/spin/cyl/porewith thecutmodule's portion ofpore.spin.systemsIncremental builds, rechecking, and watchingTowards the goal of implementing an incremental build system on CI, quill has 2 extra modes:anincrementalbuild mode: where in addition to building the entire site (or domain list),
MD5 checksums are computed for each of the generated files. In this mode, thebuild auditeris activated,
and stores a log as it generates the templates. Use the-iflag when you want to recreate the
inventory of template MD5 hashes.To run a build for the same domain as above, in incremental mode, run:cyl-dpore-iarecheckbuild mode: a subset of incremental builds in which only files whose input checksum
has changed since the last incremental run are regenerated. This makes the build substantially faster
as there is a reduced scope of templates to transform. Use the-rflag (alongside-i) when you
want to double check only the files in the template hash inventory and only regenerate what differs on disk.To run a recheck afterwards, and only build any files that changed in between builds, run:cyl-dpore-i-rawatchbuild mode: incompatible with incremental/recheck, this is for continuously rebuilding
(incremental mode is designed for repeated use, like on an incremental build system). Use the-wflag when you want to have the page rebuild automatically in the background.Watching is incompatible with incremental mode, but rebuilds following changes to a file.The corresponding command to the ones above would be:cyl-dpore-wUsage memoRequires directory of static site repositories at
location specified inspin.ini(by default
this is as a sibling directory../ss/), containing
a filemanifest.mmdbeginning with the apex domain and
immediately followed by a subdomain list.Below is a demo of accessing thespin.systemsmanifest file
(ssm) which implements theMMDclass, subclassingDoc,
which parses the file contents in a structured way (specifically,
as a list of colon-separated values).>>>frompprintimportpprint>>>fromquillimportssm>>>importpandasaspd>>>ssmParsedMMDfile(Documentof1block,containing1list)>>>ssm.listHeaderedlistwith16items>>>pprint(ssm.list.all_nodes)[-spin.systems:spin-systems:masterwww:,-:cal:qu-cal:masterwww,-,:log:spin-log:masterwww,-,:conf:qu-conf:masterwww,-,:pore:qu-pore:masterwww,-,:ocu:naiveoculus:masterwww,-,:arc:appendens:masterwww,-,:qrx:qu-arx:masterwww,-,:erg:spin-erg:masterwww,-,:opt:spin-opt:masterwww,-,:poll:qu-poll:masterwww,-,:arb:spin-arb:masterwww,-,:reed:qu-reed:masterwww,-,:noto:qu-noto:masterwww,-,:plot:qu-plot:masterwww,-,:doc:spin-doc:masterwww,-,:labs:qu-labs:masterwww]Similarlyssm.list.nodesgives just the subdomains, andssm.list.headergives the main site
domain.>>>ssm.list.header-spin.systems:spin-systems:>>>ssm.list.header.parts['spin.systems','spin-systems','master www']>>>ssm.list.nodes[0].parts['cal','qu-cal','master www']>>>ssm.list.nodes[1].parts['log','spin-log','master www']>>>ssm.list.nodes[2].parts['conf','qu-conf','master www']>>>pprint(ssm.all_parts)[['spin.systems','spin-systems','master www'],['cal','qu-cal','master www'],['log','spin-log','master www'],['conf','qu-conf','master www'],['pore','qu-pore','master www'],['ocu','naiveoculus','master www'],['arc','appendens','master www'],['qrx','qu-arx','master www'],['erg','spin-erg','master www'],['opt','spin-opt','master www'],['poll','qu-poll','master www'],['arb','spin-arb','master www'],['reed','qu-reed','master www'],['noto','qu-noto','master www'],['plot','qu-plot','master www'],['doc','spin-doc','master www'],['labs','qu-labs','master www']]>>>ssm.as_df()domainrepo_namebranches0spin.systemsspin-systemsmasterwww1calqu-calmasterwww2logspin-logmasterwww3confqu-confmasterwww4porequ-poremasterwww5ocunaiveoculusmasterwww6arcappendensmasterwww7qrxqu-arxmasterwww8ergspin-ergmasterwww9optspin-optmasterwww10pollqu-pollmasterwww11arbspin-arbmasterwww12reedqu-reedmasterwww13notoqu-notomasterwww14plotqu-plotmasterwww15docspin-docmasterwww16labsqu-labsmasterwwwThis is just for review purposes currently, and any further info can be added as long
as all the lines ("nodes") have the same number of colon-separated values.The manifest is parsed inmanifest⠶parsingbyparse_man_nodewhich is wrapped intossm.repos,
which will print out the repos' git addresses for each CNAME domain or subdomain:('spin.systems', 'git@gitlab.com:spin-systems/spin-systems.gitlab.io.git')
('cal', 'git@gitlab.com:qu-cal/qu-cal.gitlab.io.git')
('log', 'git@gitlab.com:spin-log/spin-log.gitlab.io.git')
('conf', 'git@gitlab.com:qu-conf/qu-conf.gitlab.io.git')
('pore', 'git@gitlab.com:qu-pore/qu-pore.gitlab.io.git')
('ocu', 'git@gitlab.com:naiveoculus/naiveoculus.gitlab.io.git')
('arc', 'git@gitlab.com:appendens/appendens.gitlab.io.git')
('qrx', 'git@gitlab.com:qu-arx/qu-arx.gitlab.io.git')
('erg', 'git@gitlab.com:spin-erg/spin-erg.gitlab.io.git')
('opt', 'git@gitlab.com:spin-opt/spin-opt.gitlab.io.git')
('poll', 'git@gitlab.com:qu-poll/qu-poll.gitlab.io.git')
('arb', 'git@gitlab.com:spin-arb/spin-arb.gitlab.io.git')
('reed', 'git@gitlab.com:qu-reed/qu-reed.gitlab.io.git')
('noto', 'git@gitlab.com:qu-noto/qu-noto.gitlab.io.git')
('plot', 'git@gitlab.com:qu-plot/qu-plot.gitlab.io.git')
('doc', 'git@gitlab.com:spin-doc/spin-doc.gitlab.io.git')
('labs', 'git@gitlab.com:qu-labs/qu-labs.gitlab.io.git')as well as a DataFrame which is modified by theql.ssm.check_manifest()to include 'live' views on
the repos (note that this method takes aadd_before_check=Trueargument, which controls whethergit add --allis run on each repo to check if it's 'dirty').>>>ssm.repos_dfdomainrepo_namebranchesgit_url0spin.systemsspin-systemsmasterwwwgit@gitlab.com:spin-systems/spin-systems.gitlab.io.git1calqu-calmasterwwwgit@gitlab.com:qu-cal/qu-cal.gitlab.io.git2logspin-logmasterwwwgit@gitlab.com:spin-log/spin-log.gitlab.io.git3confqu-confmasterwwwgit@gitlab.com:qu-conf/qu-conf.gitlab.io.git4porequ-poremasterwwwgit@gitlab.com:qu-pore/qu-pore.gitlab.io.git5ocunaiveoculusmasterwwwgit@gitlab.com:naiveoculus/naiveoculus.gitlab.io.git6arcappendensmasterwwwgit@gitlab.com:appendens/appendens.gitlab.io.git7qrxqu-arxmasterwwwgit@gitlab.com:qu-arx/qu-arx.gitlab.io.git8ergspin-ergmasterwwwgit@gitlab.com:spin-erg/spin-erg.gitlab.io.git9optspin-optmasterwwwgit@gitlab.com:spin-opt/spin-opt.gitlab.io.git10pollqu-pollmasterwwwgit@gitlab.com:qu-poll/qu-poll.gitlab.io.git11arbspin-arbmasterwwwgit@gitlab.com:spin-arb/spin-arb.gitlab.io.git12reedqu-reedmasterwwwgit@gitlab.com:qu-reed/qu-reed.gitlab.io.git13notoqu-notomasterwwwgit@gitlab.com:qu-noto/qu-noto.gitlab.io.git14plotqu-plotmasterwwwgit@gitlab.com:qu-plot/qu-plot.gitlab.io.git15docspin-docmasterwwwgit@gitlab.com:spin-doc/spin-doc.gitlab.io.git16labsqu-labsmasterwwwgit@gitlab.com:qu-labs/qu-labs.gitlab.io.git>>>ssm.check_manifest()domainrepo_namebranchesgit_urlbranchlocalclean0spin.systemsspin-systemsmasterwwwgit@gitlab.com:spin-systems/spin-systems.gitlab.io.gitwwwTrueTrue1calqu-calmasterwwwgit@gitlab.com:qu-cal/qu-cal.gitlab.io.gitwwwTrueTrue2logspin-logmasterwwwgit@gitlab.com:spin-log/spin-log.gitlab.io.gitwwwTrueTrue3confqu-confmasterwwwgit@gitlab.com:qu-conf/qu-conf.gitlab.io.gitwwwTrueTrue4porequ-poremasterwwwgit@gitlab.com:qu-pore/qu-pore.gitlab.io.gitwwwTrueTrue5ocunaiveoculusmasterwwwgit@gitlab.com:naiveoculus/naiveoculus.gitlab.io.gitwwwTrueTrue6arcappendensmasterwwwgit@gitlab.com:appendens/appendens.gitlab.io.gitwwwTrueTrue7qrxqu-arxmasterwwwgit@gitlab.com:qu-arx/qu-arx.gitlab.io.gitwwwTrueTrue8ergspin-ergmasterwwwgit@gitlab.com:spin-erg/spin-erg.gitlab.io.gitwwwTrueTrue9optspin-optmasterwwwgit@gitlab.com:spin-opt/spin-opt.gitlab.io.gitwwwTrueTrue10pollqu-pollmasterwwwgit@gitlab.com:qu-poll/qu-poll.gitlab.io.gitmasterTrueTrue11arbspin-arbmasterwwwgit@gitlab.com:spin-arb/spin-arb.gitlab.io.gitwwwTrueTrue12reedqu-reedmasterwwwgit@gitlab.com:qu-reed/qu-reed.gitlab.io.gitwwwTrueTrue13notoqu-notomasterwwwgit@gitlab.com:qu-noto/qu-noto.gitlab.io.gitwwwTrueTrue14plotqu-plotmasterwwwgit@gitlab.com:qu-plot/qu-plot.gitlab.io.gitwwwTrueTrue15docspin-docmasterwwwgit@gitlab.com:spin-doc/spin-doc.gitlab.io.gitwwwTrueTrue16labsqu-labsmasterwwwgit@gitlab.com:qu-labs/qu-labs.gitlab.io.gitwwwTrueTrueIn this example, thepollrepo is on the master branch, and the rest are on the www (web deploy) branch.Obviously this can then be used to clone the repositories locally
(or address them for any othergit-related task)One nice feature of GitLab (which at the time of writing GitHub doesn't
provide to my knowledge) is that these repos can all be private, and only
the static site will be hosted publicly (specified in 'Settings' > 'General')To clone a given repo (testing has all been with SSH URLs), there is theql.clone()function,
and subsequently the namespace can berefreshed to reflect the new addition (this is
done automatically within theclonefunction).>>>ql.ns{}>>>ql.clone(ql.ssm.repos_df.git_url[0],"spin.systems")Cloninginto'spin.systems'...remote:Enumeratingobjects:6,done.remote:Countingobjects:100%(6/6),done.remote:Compressingobjects:100%(6/6),done.remote:Total236(delta1),reused0(delta0),pack-reused230Receivingobjects:100%(236/236),34.16KiB|210.00KiB/s,done.Resolvingdeltas:100%(123/123),done.>>>ql.ns{'spin.systems':'https://gitlab.com/spin-systems/spin-systems.gitlab.io'}Lastly, the entire manifest of repos can be sourced fromssmandcloned into thens_pathdirectory. This is done on CI to build each site when a change takes place
in one of the source repos or the quill repo (the 'engine').ql.source_manifest()For now, if the directory named as thedomainentry of the row in thessm.repos_dftable exists, it will simply not touch it. If it doesn't exist, it will try to clone it.Et voila the namespace now contains all the repos (stored in the siblingssdirectory)>>>pprint(ql.ns){'arb':'https://gitlab.com/spin-arb/spin-arb.gitlab.io','arc':'https://gitlab.com/appendens/appendens.gitlab.io','cal':'https://gitlab.com/qu-cal/qu-cal.gitlab.io','conf':'https://gitlab.com/qu-conf/qu-conf.gitlab.io','doc':'https://gitlab.com/spin-doc/spin-doc.gitlab.io','erg':'https://gitlab.com/spin-erg/spin-erg.gitlab.io','labs':'https://gitlab.com/qu-labs/qu-labs.gitlab.io','log':'https://gitlab.com/spin-log/spin-log.gitlab.io','noto':'https://gitlab.com/qu-noto/qu-noto.gitlab.io','ocu':'https://gitlab.com/naiveoculus/naiveoculus.gitlab.io','opt':'https://gitlab.com/spin-opt/spin-opt.gitlab.io','plot':'https://gitlab.com/qu-plot/qu-plot.gitlab.io','poll':'https://gitlab.com/qu-poll/qu-poll.gitlab.io','pore':'https://gitlab.com/qu-pore/qu-pore.gitlab.io','qrx':'https://gitlab.com/qu-arx/qu-arx.gitlab.io','reed':'https://gitlab.com/qu-reed/qu-reed.gitlab.io','spin.systems':'https://gitlab.com/spin-systems/spin-systems.gitlab.io'}At the end ofsource_manifest, thessm.repos_dfDataFrame is updated with a columnlocalindicating whether each domain in the manifest is now in thensnamespace (i.e. whether a
local repo has been created), via thecheck_manifestmethod whichssm'sMMDclass inherits
from theDocclass.Thisupdate_manifestmethod will be expanded to supplement therepos_dfDataFrame with
other information worth knowing to do with thegitstatus of the repo in question, for those
which are locally available. This ensures no unnecessary computation is done before the extra
information is needed.The next thing we can do (having established that these are now cloned locally) is to read the CI YAML
as the 'layout' for each site, checking they're valid (according to thereferenceon YAML configs for GitLab Pages)>>>manifests=ql.yaml_manifests(as_dicts=False)>>>fork,minmanifests.items():print(k,end="\t");pprint(m)spin.systemsSiteCI:PagesJob:{Stage.Deploy,Script<subdirectory='site'>,Artifacts<paths=['public']>,Only<branch=['www']>}calSiteCI:PagesJob:{Stage.Deploy,Script<subdirectory='site'>,Artifacts<paths=['public']>,Only<branch=['www']>}logSiteCI:PagesJob:{Stage.Deploy,Script<subdirectory='site'>,Artifacts<paths=['public']>,Only<branch=['www']>}confSiteCI:PagesJob:{Stage.Deploy,Script<subdirectory='site'>,Artifacts<paths=['public']>,Only<branch=['www']>}poreSiteCI:PagesJob:{Stage.Deploy,Script<subdirectory='site'>,Artifacts<paths=['public']>,Only<branch=['www']>}ocuSiteCI:PagesJob:{Stage.Deploy,Script<subdirectory='site'>,Artifacts<paths=['public']>,Only<branch=['www']>}arcSiteCI:PagesJob:{Stage.Deploy,Script<subdirectory='site'>,Artifacts<paths=['public']>,Only<branch=['www']>}qrxSiteCI:PagesJob:{Stage.Deploy,Script<subdirectory='site'>,Artifacts<paths=['public']>,Only<branch=['www']>}ergSiteCI:PagesJob:{Stage.Deploy,Script<subdirectory='site'>,Artifacts<paths=['public']>,Only<branch=['www']>}optSiteCI:PagesJob:{Stage.Deploy,Script<subdirectory='site'>,Artifacts<paths=['public']>,Only<branch=['www']>}pollSiteCI:PagesJob:{Stage.Deploy,Script<subdirectory='site'>,Artifacts<paths=['public']>,Only<branch=['www']>}arbSiteCI:PagesJob:{Stage.Deploy,Script<subdirectory='site'>,Artifacts<paths=['public']>,Only<branch=['www']>}reedSiteCI:PagesJob:{Stage.Deploy,Script<subdirectory='site'>,Artifacts<paths=['public']>,Only<branch=['www']>}notoSiteCI:PagesJob:{Stage.Deploy,Script<subdirectory='site'>,Artifacts<paths=['public']>,Only<branch=['www']>}plotSiteCI:PagesJob:{Stage.Deploy,Script<subdirectory='site'>,Artifacts<paths=['public']>,Only<branch=['www']>}docSiteCI:PagesJob:{Stage.Deploy,Script<subdirectory='site'>,Artifacts<paths=['public']>,Only<branch=['www']>}labsSiteCI:PagesJob:{Stage.Deploy,Script<subdirectory='site'>,Artifacts<paths=['public']>,Only<branch=['www']>}Obviously these could all be made more elaborate for less trivial scripts, this is a first draft showing basic functionality and to enforce/check standardisation across all reposThis setup is deliberately brittle, so any changes will need to be validated in thefold.yaml_utilmodule,
and so the library itself incorporates testing (simpleassertstatements based on a clear expected implementation)The build directory can be set withchange_build_dir, and using this I set all of the repos to
build from "site" (but this can be changed at a later date):fordinql.alias_df.domain:ql.change_build_dir(d,"site")⇣Moved build path for log from 'site' --> /home/louis/spin/ss/log/site
Created build path for ocu at /home/louis/spin/ss/ocu/site
Moved build path for arc from 'site' --> /home/louis/spin/ss/arc/site
Created build path for erg at /home/louis/spin/ss/erg/site
Created build path for opt at /home/louis/spin/ss/opt/site
Created build path for arb at /home/louis/spin/ss/arb/site
Created build path for doc at /home/louis/spin/ss/doc/site
Created build path for cal at /home/louis/spin/ss/cal/site
Moved build path for conf from 'docs' --> /home/louis/spin/ss/conf/site
Created build path for pore at /home/louis/spin/ss/pore/site
Created build path for qrx at /home/louis/spin/ss/qrx/site
Created build path for poll at /home/louis/spin/ss/poll/site
Created build path for reed at /home/louis/spin/ss/reed/site
Created build path for noto at /home/louis/spin/ss/noto/site
Created build path for plot at /home/louis/spin/ss/plot/site
Created build path for labs at /home/louis/spin/ss/labs/siteTo commit these changes, I added some more functions to manage thegitrepos.ql.ssm.check_manifest()now has a column referring to whether the working tree
is clean or has changes to tracked files not staged for commit.The output will be something like this example (where the README inarcwas moved):ql.remote_push_manifest()⇣Skipping 'repo_dir=/home/louis/spin/ss/spin.systems' (working tree clean)
Skipping 'repo_dir=/home/louis/spin/ss/cal' (working tree clean)
Skipping 'repo_dir=/home/louis/spin/ss/log' (working tree clean)
Skipping 'repo_dir=/home/louis/spin/ss/conf' (working tree clean)
Skipping 'repo_dir=/home/louis/spin/ss/pore' (working tree clean)
Skipping 'repo_dir=/home/louis/spin/ss/ocu' (working tree clean)
Commit [repo_dir=/home/louis/spin/ss/arc] ⠶ Renamed site/README.md -> README.md
⇢ Pushing ⠶ origin
Skipping 'repo_dir=/home/louis/spin/ss/qrx' (working tree clean)
Skipping 'repo_dir=/home/louis/spin/ss/erg' (working tree clean)
Skipping 'repo_dir=/home/louis/spin/ss/opt' (working tree clean)
Skipping 'repo_dir=/home/louis/spin/ss/poll' (working tree clean)
Skipping 'repo_dir=/home/louis/spin/ss/arb' (working tree clean)
Skipping 'repo_dir=/home/louis/spin/ss/reed' (working tree clean)
Skipping 'repo_dir=/home/louis/spin/ss/noto' (working tree clean)
Skipping 'repo_dir=/home/louis/spin/ss/plot' (working tree clean)
Skipping 'repo_dir=/home/louis/spin/ss/doc' (working tree clean)
Skipping 'repo_dir=/home/louis/spin/ss/labs' (working tree clean)This function takes arefspecargument which indicates a particular path to add and push in each repo,
aspecific_domainsargument which can be a string of a single domain, or list of multiple,
and will be evaluated as the list of all domains if left asNone.Runningssm.check_manifest()again is required to updatessm.repos_df.Thisrepos_dfdataframe is also useful for comparing whatever
other properties you might want to check, e.g. which have a README>>>df=ql.ssm.repos_df>>>df["has_README"]=["README.md"in[x.nameforxin(ql.ns_path/d).iterdir()]fordindf.domain]>>>dfdomainrepo_namebranches...localcleanhas_README0spin.systemsspin-systemsmasterwww...TrueTrueTrue1calqu-calmasterwww...TrueTrueFalse2logspin-logmasterwww...TrueTrueTrue3confqu-confmasterwww...TrueTrueTrue4porequ-poremasterwww...TrueTrueFalse5ocunaiveoculusmasterwww...TrueTrueTrue6arcappendensmasterwww...TrueTrueTrue7qrxqu-arxmasterwww...TrueTrueFalse8ergspin-ergmasterwww...TrueTrueFalse9optspin-optmasterwww...TrueTrueFalse10pollqu-pollmasterwww...TrueTrueFalse11arbspin-arbmasterwww...TrueTrueFalse12reedqu-reedmasterwww...TrueTrueFalse13notoqu-notomasterwww...TrueTrueFalse14plotqu-plotmasterwww...TrueTrueTrue15docspin-docmasterwww...TrueTrueFalse16labsqu-labsmasterwww...TrueTrueFalseObviously this is the kind of thing to then follow up manually,
but it helps to have programmatic ways to view the set of directories.If after reviewingssm.repos_dfyou want to push, you can do so
either with an automated commit message or by passing it as thecommit_msgargument toremote_push_manifest(which will reuse the same commit message
if multiple repos are not clean).Note you can always manually go in and check thegit diffbeforehandstatic,src, andwireTo build all sites with asrcand/orstaticdirectory, runql.fold.cut.standup().
This builds templates from thesrcfolder withstaticjinja, and the static files
are just copied over (directly undersite/).To build all sites with a wire config, runql.fold.wire.standup().More detailsIn fact,standupreturns a dictionary of the domains, though for now
only one domain is in use with wires.emitters=ql.fold.wire.standup(verbose=False)emitters⇣{'poll': <quill.fold.wire.emitters.WireEmitter object at 0x7f304980eb20>}Then to push the sites 'live', runql.remote_push_manifest("Commit message goes here").AliasesThe "canonical names" displayed in the README are the aliases, which by and large
are the same as the domain names.These provide the titles of the index pages of each site.I might make more use of these in future, I originally only needed these
as it turns out the description on GitHub/GitLab isn't stored in the git repo
itself (silly as there's adescriptionbuilt in togit?)Using this let me generate a nice README for the spin.systems superrepo:spin.systemsspin.systems:sscal:q ⠶ callog:∫ ⠶ logconf:q ⠶ confpore:q ⠶ biorxocu:∫ ⠶ ocuarc:∫ ⠶ appqrx:q ⠶ arxerg:∫ ⠶ ergopt:∫ ⠶ optpoll:q ⠶ pollarb:∫ ⠶ arbreed:q ⠶ reednoto:q ⠶ ruinotoplot:q ⠶ plotspotdoc:∫ ⠶ doclabs:q ⠶ labsAddresses"Spin addresses" follow the above "{namespace}⠶{domain|alias}" format, and additionally:Some (initially only∫⠶log) subdomain repos are 'deploy' stage counterparts
to local 'dev' stage directories.Some (initially only∫⠶log) subdomain repos will be 'addressed' with a date
[and a zero-based counter for same-day entries]>>>example="∫⠶log⠶20⠶oct⠶25⠶0">>>eg_addr=ql.AddressPath(example)>>>eg_addr['∫','log','20','oct','25','0']The path has been parsed ("strictly" by default) into parts which are
'typed' strings.pprint(list(map(type,eg_addr)))⇣[<class 'ql.src.quill.scan.address.paths.NameSpaceString'>,
 <class 'ql.src.quill.scan.address.paths.DomainString'>,
 <class 'ql.src.quill.scan.address.paths.YyDigitString'>,
 <class 'ql.src.quill.scan.address.paths.mmmString'>,
 <class 'ql.src.quill.scan.address.paths.DdDigitString'>,
 <class 'ql.src.quill.scan.address.paths.FileIntString'>]A file path can be obtained from this usinginterpret_filepath,
which is bound to the class as thefilepathproperty:>>>eg_addr.filepathPosixPath('/home/louis/spin/l/20/10oct/25/0_digitalising_spin_addresses.mmd')>>>eg_addr.filepath.exists()True>>>ql.mmd(eg_addr.filepath)ParsedMMDfile(Documentof4blocks)This comes in handy when building components of the spin.systems site such astapwhich can then build parts for a particular domain
with>>>eg_addr=ql.AddressPath.from_parts(domain="poll",ymd=(2021,2,17))>>>eg_addr.filepathPosixPath('/home/louis/spin/ss/poll/transmission/21/02feb/17')This gives a simple date-based interface obeying the storage structure of quill,
though unlike files, paths to directories in this way may not exist
(instead they can be created as needed).TODOA next step could be a class representing the state of the websites [beyond CI], which can
then be cross-referenced against therepos_df(but the goal is not to entirely Python-ise
the site development, just the management of key aspects to do with the version control on disk)Make a pip installable binary wheel (bdist not currently working with SCM, just sdist)Make package capable of downloading missing data files in the event it is being distributed