Package: obsidiantools
obsidiantools 🪨⚒️obsidiantoolsis a Python package for getting structured metadata about yourObsidian.md notesand analysing your vault.  Complement your Obsidian workflows by getting metrics and detail about all your notes in one place through the widely-used Python data stack.It's incredibly easy to explore structured data on your vault through this fluent interface.  This is all the code you need to generate avaultobject that stores all the data:importobsidiantools.apiasotoolsvault=otools.Vault(<VAULT_DIRECTORY>).connect().gather()These are the basics of the method calls:connect(): connect your notes together in a graph structure and get metadata on links (e.g. wikilinks, backlinks, etc.)  There ais the option to support the inclusion of 'attachment' files in the graph.gather(): gather the plaintext content from your notes in one place.  This includes the 'source text' that represent how your notes are written.  There are arguments to support what text you want to remove, e.g. remove code.See some of thekey featuresbelow - all accessible from thevaultobject either through a method or an attribute.The package is built to support the 'shortest path when possible' option for links.  This should cover the vast majority of vaults that people create.  See thewikifor more info on what sort of wikilink syntax is not well-supported and how the graph may be slightly different to what you see in the Obsidian app.💡 Key featuresThis is howobsidiantoolscan complement your workflows for note-taking:Access anetworkxgraph of your vault(vault.graph)NetworkX is the main Python library for network analysis, enabling sophisticated analyses of your vault.NetworkX also supports the ability to export your graph to other data formats.When instantiating avault, the analysis can also be filtered on specific subdirectories.Get summary stats about your notes & files, e.g. number of backlinks and wikilinks, in a Pandas dataframeGet the dataframe viavault.get_note_metadata()(notes / md files),vault.get_media_file_metadata()(media files that can be embedded in notes) andvault.get_canvas_file_metadata()(canvas files).Retrieve detail about your notes' links and metadata as built-in Python typesThe main indices of files aremd_file_index,media_file_indexandcanvas_file_index(canvas files).Check whether files included as links in the vault actually exist, viavaultattributes likenonexistent_notes,nonexistent_media_filesandnonexistent_canvas_files.Check whether actual files are isolated in the graph ('orphans'), viavaultattributes likeisolated_notes,isolated_media_filesandisolated_canvas_files.You can access all the note & file links in one place, or you can load them for an individual note:e.g.vault.backlinks_indexfor all backlinks in the vaulte.g.vault.get_backlinks(<NOTE>)for the backlinks of an individual notemd note info:The various types of links:Wikilinks (incl. header links, links with alt text)Embedded filesBacklinksMarkdown linksFront matter viavault.get_front_matter(<NOTE>)orvault.front_matter_indexTags viavault.get_tags(<NOTE>)orvault.tags_index.  Nested tags are supported.LaTeX math viavault.get_math(<NOTE>)orvault.math_indexAs long asgather()is called:Get source text of note (viavault.get_source_text(<NOTE>)).  This tries to represent how a note's text appears in Obsidian's 'source mode'.Get readable text of note (viavault.get_readable_text(<NOTE>)).  This tries to reduce note text to minimal markdown formatting, e.g. preserving paragraphs, headers and punctuation.  Only slight processing is needed for various forms of NLP analysis.canvas file info:The JSON content of each canvas file is stored as a Python dict invault.canvas_content_indexData to recreate the layout of content in a canvas file via thevault.canvas_graph_detail_indexdictCheck out the functionality in the demo repo.  Launch the '15 minutes' demo in a virtual machine via Binder:There are other API features that try to mirror the Obsidian.md app, for your convenience when working with Python, but they are no substitute for the interactivity of the app!The text from vault notes goes through this process: markdown → split out front matter from text → HTML → ASCII plaintext.⏲️ Installationpip install obsidiantoolsRequires Python 3.9 or higher.🖇️ DependenciesMain libraries:markdownpymdown-extensionshtml2textpandasnumpynetworkxLibraries for front matter and HTML:python-frontmatterbeautifulsoup4lxmlbleachAll of these libraries are needed so that the package can separate note text from front matter in a generalised approach.🏗️ TestsA small 'dummy vault' vault of lipsum notes is intests/vault-stub(generated with help of thelorem-markdownumtool).  Sense-checking on the API functionality was also done on a personal vault of over 800 notes.I am not sure how the parsing will work outside of Latin languages - if you have ideas on how that can be supported feel free to suggest a feature or pull request.⚖️ LicenceModified BSD (3-clause)