Package: archive-md-urls
archive-md-urls: Turn URLs in Markdown files into archive.org snapshotsarchive-md-urlsscans Markdown files for URLs and if possible turns them into links to snapshots fromarchive.org. If a publication date can be extracted from the file (more info), the snapshots closest to this date will be used. If no date can be found, the latest available snapshots are used instead.This is very useful when you use a static site generator for your personal homepage that supports Markdown for writing blogposts and pages, e.g.Pelican,JekyllorHugo. Older content published years ago is likely to containlink rot: links that are simply broken or now point to a different target compared to when you wrote the content. In an ideal scenario,archive-md-urlswill not only fix these URLs, but also link to a snapshot that shows how a website or social media profile/post looked like when you wrote the content.archive-md-urlstries to be smart and does not simply replace every URL it finds. Instead, it uses alist of URLs which are considered 'stable'and are therefore ignored: URLs that already point to archive.org snapshots, intra-site links (e.g. a link to another blogpost on the same homepage) and URLs that containpersistent identifiers.Example showcaseInput fileexample_blogpost.md:Tile: Example blog post
author: Stefan
date: 2013-11-06

This fake blog post from 2013 links to [example.com](http://www.example.com/), a homepage that has dramatically changed in the meantime.

But it also links to URLs which can be considered 'stable':-[here](https://web.archive.org/web/20000622042643/http://www.google.com/) we already link to an archive.org snapshot-[here](https://doi.org/10.1080/32498327493.2014.358732798) the link contains a persistent identifier-and [here]({filename}/blog/2012/2012-02-05-an-even-older-blogpost.md) we link to a different post on our own homepage (Pelican format, Jekyll and Hugo intra-site links are supported too)

In addition, google.com is mentioned but not explicitly linked.

And finally, [here](www.some-madeup-link-that-hasnt-been-archived.com) we link to a homepage that doesn't have any corresponding archive.org snapshots.Output fromarchive-md-urls example_blogpost.md:Tile: Example blog post
author: Stefan
date: 2013-11-06

This fake blog post from 2013 links to [example.com](http://web.archive.org/web/20131106211912/http://www.example.com/), a homepage that has dramatically changed in the meantime.

But it also links to URLs which can be considered 'stable':-[here](https://web.archive.org/web/20000622042643/http://www.google.com/) we already link to an archive.org snapshot-[here](https://doi.org/10.1080/32498327493.2014.358732798) the link contains a persistent identifier-and [here]({filename}/blog/2012/2012-02-05-an-even-older-blogpost.md) we link to a different post on our own homepage (Pelican format, Jekyll and Hugo intra-site links are supported too)

In addition, google.com is mentioned but not explicitly linked.

And finally, [here](www.some-madeup-link-that-hasnt-been-archived.com) we link to a homepage that doesn't have any corresponding archive.org snapshots.Note how only the first link to example.com has been altered and points to a snapshot close in time to the publication date of this fake blog post (6th November 2013). Also note that URLs which are mentioned but not explicitly linked are ignored.InstallYou can installarchive-md-urlsvia pip:python-mpipinstallarchive-md-urlsHowever, usingPipxis recommended:pipxinstallarchive-md-urlsUsageImportant:archive-md-urlsmodifies your files directly in-place. It is recommended that the files you want to change are under version control so you can review the changes.Once installed, you can pass any number of Markdown files or directories containing Markdown files toarchive-md-urls:# Update two filesarchive-md-urlsmy-file.mdanother-file.md
Updated13linksin2files.# Update files in a directoryarchive-md-urlsmyblog/content/blog/2014
Updated97linksin20files.# You can also combine files and directoriesarchive-md-urlsmyblog/content/blog/2014my-file.md
Updated103links21files.By default, directories are not searched recursively for Markdown files. For recursive search, use the-rflag (use this with caution!):# Update URLs in all Markdown files of myblogarchive-md-urls-rmyblog/content
Updated160linksin32files.Note that Markdown files are identified by the file ending.md, other file endings are ignored.A note about speedarchive-md-urlsusesasynciowithHTTPXto make asynchronous API calls. However, do not expect to get fast results, especially (but not only) when you try to change a larger amount of URLs. TheWayback Machine APIcan be slow or even unavailable. Ifarchive-md-urlshas to cancel the operation because of that, just re-run it on the same files again later. Links that have already been updated before will be skipped because archive.org links are considered stable.ContributingIf you would like to contribute to this project, please create apull request from a fork.To set up a local development environment, clone your fork and set up a virtual environment with your preferred tool. For example:# Here we just clone the main repository, change it to your fork's URLgitclonehttps://github.com/sbaack/archive-md-urls.gitcdarchive-md-urls
python-mvenv.venv&&source.venv/bin/activateInstall an editable version ofarchive-md-urls:makesetup# OR, if you can't use Gnu Make:python-mpipinstall-Upip-Ue.In addition, you'll needhatchto run tests. If you don't already have hatch installed via Pipx, Conda etc., install it in the project venv:python-mpipinstall-UhatchTests should pass before submitting a pull request:maketest# OR:hatchruntests:test