Package: pushl
PushlA simple tool that parses content feeds and sends out appropriate push notifications (WebSub, webmention, etc.) when they change.Seehttp://publ.beesbuzz.biz/blog/113-Some-thoughts-on-WebMentionfor the motivation.FeaturesSupports any feed supported byfeedparserandmf2py(RSS, Atom, HTML pages containingh-entry, etc.)Will send WebSub notifications for feeds which declare a WebSub hubWill send WebMention notifications for entries discovered on those feeds or specified directlyCan perform autodiscovery of additional feeds on entry pagesCan do a full backfill on Atom feeds configured withRFC 5005When configured to use a cache directory, can detect entry deletions and updates to implement the webmention update and delete protocols (as well as saving some time and bandwidth)Site setupIf you want to support WebSub, have your feed implementthe WebSub protocol. The short version is that you should have a<link rel="hub" href="http://path/to/hub" />in your feed's top-level element.There are a number of WebSub hubs available; I useSuperfeedr.ForWebMentions, configure your site templates with the various microformats; by default, Pushl will use the following tags as the top-level entry container, in descending order of priority:Anything with aclassofh-entryAn<article>tagAnything with aclassofentryFor more information on how to configure your site templates, see themicroformats h-entry specification.mf2 feed notesIf you're using an mf2 feed (i.e. an HTML-formatted page withh-entrydeclarations), only entries with au-urlproperty will be used for sending webmentions; further, Pushl will retrieve the page from that URL to ensure it has the full content. (This is to work around certain setups where theh-feedonly shows summary text.)Also, there is technically no requirement for an HTML page to declare anh-feed; all entities marked up withh-entrywill be consumed.InstallationYou can install it usingpipwith e.g.:pip3installpushlHowever, I recommend installing it in a virtual environment with e.g.:virtualenv3$HOME/pushl$HOME/pushl/bin/pip3installpushland then putting a symlink to$HOME/pushl/bin/pushlto a directory in your $PATH, e.g.ln-s$HOME/pushl/bin/pushl$HOME/bin/pushlUsageBasicpushl-c$HOME/var/pushl-cachehttp://example.com/feed.xmlWhile you can run it without the-cargument, its use is highly recommended so that subsequent runs are both less spammy and so that it can detect changes and deletions.Sending pings from individual entriesIf you just want to send webmentions from an entry page without processing an entire feed, the-e/--entryflag indicates that the following URLs are pages or entries, rather than feeds; e.g.pushl-ehttp://example.com/some/pagewill simply send the webmentions for that page.Additional feed discoveryThe-r/--recurseflag will discover any additional feeds that are declared on entries and process them as well. This is useful if you have per-category feeds that you would also like to send WebSub notifications on. For example,my sitehas per-category feeds which are discoverable from individual entries, sopushl -r http://beesbuzz.biz/feedwill send WebSub notifications for all of the categories which have recent changes.Note that-rand-ein conjunction will also cause the feed declared on the entry page to be processed further. While it is tempting to use this in a feed autodiscovery context e.g.pushl-rehttp://example.com/blog/this will also send webmentions from the blog page itself which is probablynotwhat you want to have happen.Backfilling old contentIf your feed implementsRFC 5005, the-aflag will scan past entries for WebMention as well. It is recommended to only use this flag when doing an initial backfill, as it can end up taking a long time on larger sites (and possibly make endpoint operators very grumpy at you). To send updates of much older entries it's better to just use-eto do it on a case-by-case basis.Dual-protocol/multi-domain websitesIf you have a website which has multiple URLs that can access it (for example, http+https, or multiple domain names), you generally only want WebMentions to be sent from the canonical URL. The best solution is to use<link rel="canonical">to declare which one is the real one, and Pushl will use that in sending the mentions; so, for example:pushl-rhttps://example.com/feedhttp://example.com/feedhttp://alt-domain.example.com/feedAs long as bothhttp://example.comandhttp://alt-domain.example.comdeclare thehttps://example.comversion as canonical, only the webmentions fromhttps://example.comwill be sent.If, for some reason, you can't userel="canonical"you can use the-s/--websub-onlyflag on Pushl to have it only send WebSub notifications for that feed; for example:pushl-rhttps://example.com/feed-shttps://other.example.com/feedwill send both Webmention and WebSub forhttps://example.combut only WebSub forhttps://other.example.com.Automated updatespushlcan be run from a cron job, although it's a good idea to useflock -nto prevent multiple instances from stomping on each other. An example cron job for updating a site might look like:*/5 * * * * flock -n $HOME/.pushl-lock pushl -rc $HOME/.pushl-cache http://example.com/feedMy setupIn my setup, I havepushlinstalled in my website's pipenv:cd$HOME/beesbuzz.biz
pipenvinstallpushland created this script as$HOME/beesbuzz.biz/pushl.sh:#!/bin/bashcd$(dirname"$0")LOG=logs/pushl-$(date+%Y%m%d.log)# redirect log outputif["$1"=="quiet"];thenexec>>$LOG2>&1elseexec2>&1|tee-a$LOGfi# add timestampdate# run pushlflock-n$HOME/var/pushl/run.lock$HOME/.local/bin/pipenvrunpushl-rvvkc$HOME/var/pushl\https://beesbuzz.biz/feed\?push=1\http://publ.beesbuzz.biz/feed\?push=1\https://tumblr.beesbuzz.biz/rss\https://novembeat.com/feed\?push=1\http://beesbuzz.biz/feed\?push=1\-shttp://beesbuzz.biz/feed-summaryhttps://beesbuzz.biz/feed-summary# while we're at it, clean out the log and pushl cache directoryfindlogs$HOME/var/pushl-typef-mtime+30-print-deleteThen I have a cron job:*/15 * * * * $HOME/beesbuzz.biz/pushl.sh quietwhich runs it every 15 minutes.I also have agit deployment hookfor my website, and its final step (after restartinggunicorn) is to runpushl.sh, in case a maximum latency of 15 minutes just isn't fast enough.