Package: mkrefs
MkRefsTheMkRefspluginforMkDocsgenerates reference Markdown pages from a knowledge graph,
based on thekglabproject.No graph database is required; however, let us know if you'd like to
use one in particular.There are several planned use cases for theMkRefsplugin,
including:biblio– semantic bibliography entries, generated from RDFglossary– semantic glossary entries, generated from RDFapidocs– semanticapidocssupporting theDiátaxisgrammar for documentation, generated as RDF from Python source codedepend– semantic dependency graph for Python libraries, generated as RDF fromsetup.pyindex– semantic search index, generated as RDF from MkDocs contentOnly theapidocs,biblio, andglossarycomponents have been
added toMkRefsso far, although the other mentioned components
exist in separate projects and are being integrated.Contributing CodeWe welcome people getting involved as contributors to this open source
project!For detailed instructions please see:CONTRIBUTING.mdSemantic VersioningBeforeMkRefsreaches releasev1.0.0the
types and classes may undergo substantial changes and the project is
not guaranteed to have a consistent API.Even so, we'll try to minimize breaking changes.
We'll also be sure to provide careful notes.See:changelog.txtWhy does this matter?A key takeaway is that many software engineering aspects of open
source projects involve graphs, therefore a knowledge graph can
provide an integral part of an open source repository.
Moreover, by using semantic representation (RDF) projects that
integrate with each other can share (i.e., federate) common resources,
for example to share definitions, analyze mutual dependencies, etc.InstallationTo install the plugin usingpip:python3 -m pip install mkrefsThen add the plugin into themkdocs.ymlfile:plugins:-mkrefsIn addition, the following configuration parameter is expected:mkrefs_config- YAML configuration file forMkRefs; e.g.,mkrefs.ymlAPI DocsAapidocsparameter within the configuration file expects four
required sub-parameters:page– name of the generated Markdown page, e.g.,ref.mdtemplate– aJinja2 templateto generate Markdown, e.g.,ref.jinjapackage– name of the package being documentedgit– base URL for source modules in Git, e.g.,https://github.com/DerwenAI/mkrefs/blob/mainThere is an optionalincludesparameter, as a list of class
definitions to include.
If this is used, then all other classes get ignored.See the source code in this repo for examples of how to format
Markdown withindocstrings.
Specifically see the parameter documentation per method or function,
which differs slightly from pre-exisiting frameworks.Note that the name of the generated Markdown page for the
apidocs must appear in thenavsection of yourmkdocs.ymlconfiguration file.
See the structure used in this repo for an example.Best Practices: RDF representationYou can use this library outside of MkDocs, i.e., calling it
programmatically, to generate an RDF graph to represent your package
API reference:package_name = "mkrefs"
git_url = "https://github.com/DerwenAI/mkrefs/blob/main"
includes = [ "MkRefsPlugin", "PackageDoc" ]

pkg_doc = PackageDoc(package_name, git_url, includes)
pkg_doc.build()

kg = pkg_doc.get_rdf()ThePackageDoc.get_rdf()method returns an RDF graph as an instance
of ankglab.KnowledgeGraphobject.
For more details, seehttps://derwen.ai/docs/kgl/BibliographyAbiblioparameter within the configuration file expects four
required sub-parameters:graph– an RDF graph represented as a Turtle (TTL) file, e.g.,mkrefs.ttlpage– name of the generated Markdown page, e.g.,biblio.mdtemplate– aJinja2 templateto generate Markdown, e.g.,biblio.jinjaqueries–SPARQL queriesused to extract bibliography data from the knowledge graphSee themkrefs.ttlfile for an example bibliography represented in RDF.
This comes from the documentation for thepytextrankopen source project.In the example RDF, thebibovocabulary represents
bibliographic entries, and theFOAFvocabulary
represents authors.
This also uses two custom OWL relations from thederwenvocabulary:derw:citeKey– citekey used to identify a bibliography entry within the documentationderw:openAccess– open access URL for a bibliography entry (if any)Thequeriesparameter has three required SPARQL queries:entry– to select the identifiers for all of the bibliograpy entriesentry_author– a mapping to identify author links for each bibliography entryentry_publisher- the publisher link for each bibliography entry (if any)Note that the name of the generated Markdown page for the
bibliography must appear in thenavsection of yourmkdocs.ymlconfiguration file.
See the structure used in this repo for an example.You may use any valid RDF representation for a bibliography.
Just be sure to change the three SPARQL queries and the Jinja2
template accordingly.While this example uses an adaptation of theMLA Citation Style,
feel free to modify the Jinja2 template to generate whatever
bibliographic style you need.Best Practices: constructing bibliographiesAs much as possible, bibliography entries should use the conventions athttps://www.bibsonomy.org/for theircitation keys.Journal abbreviations should useISO 4standards,
for example fromhttps://academic-accelerator.com/Journal-Abbreviation/SystemLinks to online versions of cited works should useDOIforpersistent identifiers.When available,open accessURLs should be listed as well.GlossaryAglossaryparameter within the configuration file expects four
required sub-parameters:graph– an RDF graph represented as a Turtle (TTL) file, e.g.,mkrefs.ttlpage– name of the generated Markdown page, e.g.,glossary.mdtemplate– aJinja2 templateto generate Markdown, e.g.,glossary.jinjaqueries–SPARQL queriesused to extract glossary data from the knowledge graphSee themkrefs.ttlfile for an example glossary represented in RDF.
This example RDF comes from documentation for thepytextrankopen source project.In the example RDF, thecitovocabulary represents citations to locally represented bibliographic
entries.
Theskosvocabulary
provides support fortaxonomyfeatures, e.g., semantic relations among glossary entries.
This example RDF also uses a definition from thederwenvocabulary:derw:Topic– askos:Conceptused to represent glossary entriesThequeriesparameter has three required SPARQL queries:entry– to select the identifiers for all of the bibliograpy entriesentry_syn– a mapping of synonyms (if any)entry_ref– a mapping of external references (if any)entry_cite– citations to the local bibliography citekeys (if any)entry_hyp– a mapping ofhypernyms(if any)Note that the name of the generated Markdown page for the glossary
must appear in thenavsection of yourmkdocs.ymlconfiguration
file.
See the structure used in this repo for an example.You may use any valid RDF representation for a glossary.
Just be sure to change the three SPARQL queries and the Jinja2
template accordingly.UsageThe standard way to generate documentation with MkDocs is:mkdocs buildIf you'd prefer to generate reference pages programmatically using
Python scripts, see the code for usage of theMkRefsPluginclass,
plus some utility functions:load_kg()render_apidocs()render_biblio()render_glossary()There are also command lineentry pointsprovided, which can be
helpful during dev/test cycles on the semantic representation of your
content:mkrefs apidocs docs/mkrefs.yml
mkrefs biblio docs/mkrefs.yml
mkrefs glossary docs/mkrefs.ymlWhat is going on here?For example with the bibliography use case, when the plugin runs...It parses its configuration file to identify the target Markdown page to generate and the Jinja2 templateThe plugin also loads an RDF graph from the indicated TTL fileThree SPARQL queries are run to identify the unique entities to extract from the graphThe graph is serialized asJSON-LDTheauthor,publisher, and bibliographyentryentities are used todenormalizethe graph into a JSON data objectThe JSON is rendered using the Jinja2 template to generate the MarkdownThe Markdown page is parsed and rendered by MkDocs as HTML, etc.CaveatsWhile theMkDocsutility is astoundingly useful,
its documentation (and coding style) leave much room for improvement.
Thedocumentation for developing pluginsis not even close to what happens when its code executes.Consequently, theMkRefsproject is an attempt to reverse-engineer
the code from many other MkDocs plugins, while documenting its observed
event sequence, required parameters, limitations and workarounds, etc.Two issues persist, where you will see warnings even though theMkRefscode is handling configuration as recommended:WARNING -  Config value: 'mkrefs_config'. Warning: Unrecognised configuration name: mkrefs_configandINFO    -  The following pages exist in the docs directory, but are not included in the "nav" configuration:
  - biblio.md
  - glossary.md
  - ref.mdFor now, you can simply ignore both of these warnings.
Meanwhile, we'll work on eliminating them.Feature roadmapLet us know if you need features to parse and generateBibTeX.License and CopyrightSource code forMkRefsplus its logo, documentation, and examples
have anMIT licensewhich is
succinct and simplifies use in commercial applications.All materials herein are Copyright © 2021 Derwen, Inc.AcknowledgementsThis plugin code is based on the marvelous examples inhttps://github.com/byrnereese/mkdocs-plugin-templatewith kudos to@byrnereese,
and also many thanks to@louisguitton,@dmccreary,
and@LarrySwansonfor their inspiration and insights.