Dev Diary Week 3 - Parsing OPML Files
Tags: node, javascript, showdev
Published: 2020-04-30 15:48:04

---
title: Dev Diary Week 3 - Parsing OPML Files
published: true
description: This post is an entry in a weekly development diary on building a feed-aggregator-based blog on Glitch. This week, I added a way to parse multiple RSS feeds, via an OPML file.
tags: node, javascript, showdev
series: Dev Diary
cover_image: https://dev-to-uploads.s3.amazonaws.com/i/06l1787yedv7wynyna3m.jpg
---

> This post is an entry in a weekly development diary on building a feed-aggregator-based blog on [Glitch]. Only a small part of building an app is code; the bulk of your time is spent planning, experimenting, making mistakes, getting frustrated, and making progress in baby steps.

## The plan

[Last time], I worked on an API endpoint for parsing RSS feeds. It lets you pass in an RSS feed URL, and returns some JSON feed metadata and an array of feed entries. That's a good start! However, I want to combine entries from multiple feeds into one "content firehose" feed, so this week we'll look at how to do this.

## Parsing an OPML file

Now, I _could_ hard-code an array for feed URLs (maybe as a JSON file), but there's an open standard called [OPML] that I can use instead. This is an "outline" file, and essentially each entry for a feed URL looks something like this:

```xml
<outline
  text="What's displayed (usually title)"
  title="Title of feed"
  description=""
  type="rss"
  version="RSS"
  htmlUrl="https://link/to/site/"
  xmlUrl="https://link/to/feed/"
/>
```

The advantage to using an OPML file is that they're portable. You can export your susbcriptions to an OPML file to use with this app, and anyone can get your OPML file and import it into their feed reader of choice. Here's [mine]!

So, this week, I'm planning on doing the following:

1. Create a default OPML file for this app.
2. Parse the OPML file to get each individual feed URL.
3. Pass each feed to the feed parsing endpoint.
4. Collect all feed entries as JSON.

And, just as we used the [feedparser] npm package last week, we'll use the [opmlparser] package (by the same author) to handle the parsing work this week.

## How did it go?

I managed to complete all of this work in about two hours between meetings, thanks (again) to great example code in the [opmlparser GitHub repository]. This serves as a great reminder that writing great docs is just as important as writing solid code for your library/framework/app!

I worked off a [remix] of my [existing work] and added the opmlparser package to **package.json**, and then added the [opmlparser.js] file to build out the functionality I needed. I also added [a **subscriptions.opml** file] to the **public** directory that included two feeds: the Glitch team's posts on Dev.to, and my own posts on Dev.to. This will give us an interesting de-duplication problem to solve for next week, since both feeds contain my posts.

I then added a `/opml` [route] to **server.js** which does the following:

1. Gets the feeds in the **subscriptions.opml** file;
2. Sends them to the `/api/parse/:feed` endpoint;
3. Returns the feed content from each feed in the OPML file to the caller.

This will probably change over time — for now, I wanted a way to see that the OPML file was being parsed correctly.

### What went well?

Integrating the opmlparser library was really straightforward. Since it was built by the same author, the code in **opmlparser.js** is pretty much the same as in **feedparser.js**.

### What did I have trouble with?

I got a bit lost in async spaghetti while trying to send each feed in the OPML file to the `/api/parse/:feed` endpoint. 😂 There's lots of room for improvement here, but because I expect to rewrite this elsewhere, I didn't want to spend _too_ much time on it.

### What did I learn?

I learned a lot about OPML files! I'm used to working with JSON data, and looking at XML sometimes takes a while to figure out. At the end of the day, though, OPML is a variation on an outline document. Thinking about it this way helped me reason through things.

### What will I work on next week?

Next week, I'm going to start working on getting feeds into a database!

{% glitch rss-firehose-parseopml %}

## What's in _your_ OPML file?

OPML files were created to be shared! They're an open standard that most any feed reader will import or export. Here's what's in [mine] — share yours in the comments!

<!--Links-->

[Glitch]: https://glitch.com?utm_medium=weblink&utm_source=dev.to&utm_campaign=blog&utm_content=dev
[Last time]: https://dev.to/glitch/dev-diary-week-2-reading-a-feed-4mno
[OPML]: http://dev.opml.org/
[mine]: https://link/to/file.opml
[feedparser]: https://www.npmjs.com/package/feedparser/
[opmlparser]: https://www.npmjs.com/package/opmlparser/
[opmlparser GitHub repository]: https://github.com/danmactough/node-opmlparser
[remix]: https://glitch.com/~rss-firehose-parseopml?utm_medium=weblink&utm_source=dev.to&utm_campaign=blog&utm_content=dev
[existing work]: https://glitch.com/~rss-firehose?utm_medium=weblink&utm_source=dev.to&utm_campaign=blog&utm_content=dev
[opmlparser.js]: https://glitch.com/edit/#!/rss-firehose-parseopml?path=opmlparser.js%3A1%3A0
[a **subscriptions.opml** file]: https://glitch.com/edit/#!/rss-firehose-parseopml?path=public%2Fsubscriptions.opml%3A1%3A0
[route]: https://glitch.com/edit/#!/rss-firehose-parseopml?path=server.js%3A55%3A0
[mine]: https://gist.github.com/AngeloStavrow/804c557c5e9e7f39433dde145d953fda