Package: rssfixer
rssfixerA tool to generate anRSSfeed from someWordPressblogs and other sources that for some reason don't generate their own feeds. This tool usesBeautifulSoupto parse the HTML andfeedgento generate the feed. I created this tool to be to follow news from companies that have forgotten the usefulness of RSS.InstallationCreate a virtual environment and simply runpython3 -m pip install rssfixer, full example below.python3-mvenvvenvsourcevenv/bin/activate
python3-mpipinstallrssfixerSupported page typesI've expanded the tools to blogs that I like to follow. If you have suggestions to add/change functionality please open anissueor start a newdiscussion.The basic formats of web pages supported are:--list- links are in simple ul-list--json- links, titles and sometimes description is accessible in a JSON structure--html- links and titles can be found by some unique HTML element--release- similar to--htmlexcept there are no links and you have to specify a target URLDuring testing it is useful to use--stdoutoption to see the generated feed. When I have time (and enough motivation) I might write a tool to try and find the right combination of options for a specified URL.Simple listAn example to generate a feed fornccgroupthat have the links in a simple ul-list by using the--listoption:$rssfixer--titlenccgroup--listhttps://research.nccgroup.com/
RSSfeedcreated:rss_feed.xmlYou can specify a filename and silence output:rssfixer--titlenccgroup--outputnccgroup.xml--quiethttps://research.nccgroup.com/The resulting file is availablehereas an example.Most times you would run the script from crontab to have an updated feed. Here is an example with a venv in/home/user/src/rssfixer.32****/home/user/src/rssfixer/bin/rssfixer--titlenccgroup--output/var/www/html/feeds/nccgroup.xml--quiet--listhttps://research.nccgroup.comJSONSome blogs liketruesec.comhave all blog links in a JSON object. You can use the--jsonoption to parse the JSON object and generate a feed. The same is true for Apple'ssecurity blogpage.An example forApple:rssfixer--title"Apple Security"--outputapple.xml--quiet--json--json-entriesblogs--json-urlslug--base-urlhttps://security.apple.com/blog/https://security.apple.com/blogIn this example--json-entries blogsspecifies that blog entries are located in a key calledblogsand that URLs are available in a key calledslug. Since the URL only includes the key (or slug) we specify the full URL to the blog with--base-url https://security.apple.com/blog/.An example fortruesec.com:rssfixer--titleTruesec--json--json-descriptionpreamble--quiet--outputtruesec.xmlhttps://www.truesec.com/hub/blogHere we must specify--json-description preambleto find the description or summary of the blog post.General HTMLPages with a more general HTML structure can be parsed with the--htmloption. You can specify the HTML tag for the entries, the URL and title of the blog entry.An example fortripwire.com:rssfixer--titleTripwire--outputtripwire.xml--quiet--html--base-urlhttps://www.tripwire.comhttp://www.tripwire.com/state-of-securityReleaseCheck for one entity on release pages likeSQLite(h3) and generate RSS feed with links to the download page (required argument--release-url). Easy way to get notified when a new version is released.rssfixer--release--outputsqlite.xml--release-entriesh3--release-urlhttps://sqlite.org/download.htmlhttps://sqlite.org/changes.htmlUsageCommand-line options (updated on commit bymarkdown-code-runner):usage: rssfixer [-h] (--html | --json | --list | --release) [--version]
                [--atom] [--base-url BASE_URL] [--release-url RELEASE_URL]
                [--release-entries RELEASE_ENTRIES]
                [--html-entries HTML_ENTRIES]
                [--html-entries-class HTML_ENTRIES_CLASS]
                [--html-url HTML_URL] [--html-title HTML_TITLE]
                [--html-title-class HTML_TITLE_CLASS]
                [--title-filter TITLE_FILTER]
                [--html-description HTML_DESCRIPTION]
                [--html-description-class HTML_DESCRIPTION_CLASS]
                [--json-entries JSON_ENTRIES] [--json-url JSON_URL]
                [--json-title JSON_TITLE]
                [--json-description JSON_DESCRIPTION] [--output OUTPUT]
                [--title TITLE] [--user-agent USER_AGENT]
                [--filter-type FILTER_TYPE] [--filter-name FILTER_NAME] [-q]
                [-d] [--stdout]
                url

Generate RSS feed for blog that don't publish a feed. Default is to find links
in a simple <ul>-list. Options are available to find links in other HTML
elements or JSON strings.

positional arguments:
  url                   URL for the blog

options:
  -h, --help            show this help message and exit
  --html                Find entries in HTML
  --json                Find entries in JSON
  --list                Find entries in HTML <ul>-list (default)
  --release             Find releases in HTML
  --version             show program's version number and exit
  --atom                Generate Atom feed
  --base-url BASE_URL   Base URL for the blog
  --release-url RELEASE_URL
                        Release URL for downloads
  --release-entries RELEASE_ENTRIES
                        Release selector for entries
  --html-entries HTML_ENTRIES
                        HTML selector for entries
  --html-entries-class HTML_ENTRIES_CLASS
                        Class name for entries
  --html-url HTML_URL   HTML selector for URL
  --html-title HTML_TITLE
                        HTML selector for title
  --html-title-class HTML_TITLE_CLASS
                        Flag to specify title class (regex)
  --title-filter TITLE_FILTER
                        Filter for title, ignore entries that don't match
  --html-description HTML_DESCRIPTION
                        HTML selector for description
  --html-description-class HTML_DESCRIPTION_CLASS
                        Flag to specify description class (regex)
  --json-entries JSON_ENTRIES
                        JSON key for entries (default: 'entries')
  --json-url JSON_URL   JSON key for URL (default: 'url')
  --json-title JSON_TITLE
                        JSON key for title
  --json-description JSON_DESCRIPTION
                        JSON key for description
  --output OUTPUT       Name of the output file
  --title TITLE         Title of the RSS feed (default: "My RSS Feed")
  --user-agent USER_AGENT
                        User agent to use for HTTP requests
  --filter-type FILTER_TYPE
                        Filter web page
  --filter-name FILTER_NAME
                        Filter web page
  -q, --quiet           Suppress output
  -d, --debug           Debug selection
  --stdout              Print to stdoutCommand-line examples for blogs# Apple Security Blog# Url: https://security.apple.com/blog/rssfixer--title"Apple Security"--outputapple.xml--quiet--json--json-entriesblogs--json-urlslug--base-urlhttps://security.apple.com/blog/https://security.apple.com/blog# nccgroup# Url: https://research.nccgroup.com/rssfixer--titlenccgroup--outputnccgroup.xml--quiet--listhttps://research.nccgroup.com# Tripwire# Url: https://www.tripwire.com/state-of-securityrssfixer--titleTripwire--outputtripwire.xml--quiet--html--base-urlhttps://www.tripwire.comhttp://www.tripwire.com/state-of-security# TrueSec# Url: https://www.truesec.com/hub/blogrssfixer--titleTruesec--outputtruesec.xml--quiet--json--json-descriptionpreamblehttps://www.truesec.com/hub/blog# SQLite# Url: https://sqlite.org/changes.htmlrssfixer--titleSQLite--release--release-entriesh3--release-urlhttps://sqlite.org/download.htmlhttps://sqlite.org/changes.html# Nucleus# https://nucleussec.com/category/cisa-kevrssfixer--title"Nucleus CISA KEV"--outputnucleus.xml--html--filter-typediv--filter-namerecent-post-widget--html-entriesdiv--html-titlediv--html-title-class"post-desc"--title-filterKEVhttps://nucleussec.com/category/cisa-kev# NCSC-SE# https://www.ncsc.se/publikationer/rssfixer--html--filter-typediv--filter-name'page-container'--html-entriesdiv--html-entries-class"news-text"--html-titleh2--html-title-class""--html-descriptionp--html-description-class""--html-urla--base-urlhttps://www.ncsc.se--stdout--atom--title"Feed for NCSC-SE"https://www.ncsc.se/publikationer/If you have other example use case please add them inshow usage examplesin discussions.