Export Meetup.com Events to a Github Pages Jekyll site
Tags: graphql, groovy, json, java
Published: 2022-08-31 04:13:49

I help run the Omaha Java Users Group, and for a long time our communication has been through <a href="https://www.meetup.com/omahajava/">meetup.com/omahajava/</a>, however we do still have an <a href="http://ojug.org/">ojug.org</a> website too. At first, we would nicely copy the event to both Meetup and OJUG.org, however within a few months you can see that we simply stopped duplicating effort and the events on <a href="http://ojug.org/">ojug.org</a> quickly got out of date. Not a big deal at the time. But it would be nice to archive our past events to the ojug.org site (especially as we explore Meetup.com alternatives in the near future <a href="#footnote1">[1]</a>).

  (if you want the TL&amp;DR: see all the details at <a href="https://github.com/jeffsheets/ojug-meetup-export">https://github.com/jeffsheets/ojug-meetup-export</a>.)

<h3>Exporting Events from Meetup.com</h3>

The first step was to find a way to export <a href="https://www.meetup.com/omahajava/events/past/">our past events</a>. I had hoped for a CSV export through the admin UI, but didn't see anything. I then thought it might take some web scraping, or maybe some network dev tools api call watching. But noticed that there was a grayed out setting in the admin UI for the Meetup API 🤔. A quick google found the <a href="https://www.meetup.com/api/playground/#graphQl-playground">Meetup GraphQL API Playground</a> page. And to my surprise, sending a test query just worked! 🎉

After a little trial and error from their API docs, and increasing the result count to 100 to get all of our events without paging, I was able to export all of the past events to JSON with:

```graphql
  query($meetupId: String!) {
    groupByUrlname(urlname: $meetupId) {
      description
      pastEvents(input: {first: 100}) {
        count
        edges {
          node {
            title
            description
            dateTime
            going
          }
        }
      }
    }
  }
```

And some inputs of:

```json
  {"meetupId":"omahajava"}
```

Which gave <a href="https://github.com/jeffsheets/ojug-meetup-export/blob/main/src/meetup-events-export-2022-08-25.json">a nice JSON result</a>, like this:

```json
{
    "data": {
      "groupByUrlname": {
        "description": "Omaha's Java User Group [@omahajug](https://twitter.com/omahajug/). yadda yadda yadda",
        "pastEvents": {
          "count": 65,
          "edges": [
            {
              "node": {
                "title": "Angular JS for Java Developers",
                "description": "This month //etc etc etc",
                "dateTime": "2014-05-20T17:30-05:00",
                "going": 27
              }
            },
.....
```

<h3>ojug.org Tech</h3>

Before showing you how the event blog post pages were generated, a quick note on the tech for ojug.org (src at <a href="https://github.com/OJUG/ojug.github.io">https://github.com/OJUG/ojug.github.io</a>). It is running on a standard <a href="https://docs.github.com/en/pages/setting-up-a-github-pages-site-with-jekyll/about-github-pages-and-jekyll">Github Pages Jekyll workflow</a> stack. While we have thoughts to move that over to <a href="https://www.11ty.dev/">11ty</a>, for now <a href="https://jekyllrb.com/docs/posts/">Jekyll</a> is working fine. Create a new markdown file in the <a href="https://github.com/OJUG/ojug.github.io/tree/master/_posts">_posts folder</a>, merge to the main branch, and the workflow auto-kicks off and redeploys our ojug.org site like magic.

<h3>Generating Meetup event Jekyll posts</h3>

With this in mind, the blog needs .md files generated for each event in the events JSON. Using a little Groovy, this was done pretty quickly with a GSP template, some functions to prettify date formats, and a filename creation function. The groovy <a href="https://github.com/jeffsheets/ojug-meetup-export/blob/main/src/post.gsp">post.gsp template</a> looks like:

```
---
layout: post
title:  "&lt;%= longDate %> &lt;%= title %>"
---

&lt;%= description %>

(This past event was exported from Meetup.com)
(&lt;%= attended %> people had RSVP'd to this event in Meetup)
```

Then the code that generates the template for each JSON event is in <a href="https://github.com/jeffsheets/ojug-meetup-export/blob/main/src/PostGenerator.groovy">PostGenerator.groovy</a>:

```groovy
    void generatePosts() {
        def events = new JsonSlurper().parse(getClass().getResource(SRC_JSON)).data.groupByUrlname.pastEvents.edges
        events.each {
            def event = it.node
            def filename = makeFilename(event.title, event.dateTime)
            def outfile = new File("$DEST_FOLDER/$filename")
            def filecontents = new SimpleTemplateEngine()
                    .createTemplate(getClass().getResource('/post.gsp'))
                    .make([
                            title      : event.title.replaceAll('"', '\"'),
                            description: event.description,
                            longDate   : convertToLongDate(event.dateTime),
                            attended   : event.going
                    ])
                    .toString()
            outfile.write filecontents
        }
    }
```

When executed, nice markdown files are generated in the output directory, similar to <a href="https://github.com/OJUG/ojug.github.io/blob/master/_posts/2014-05-20-angular-js-for-java-developers.md">2014-05-20-angular-js-for-java-developers.md</a>:

```markdown
---
layout: post
title:  "May 20, 2014 Angular JS for Java Developers"
---

This month //etc etc etc

(This past event was exported from Meetup.com)
(27 people had RSVP'd to this event in Meetup)
```

The last step was to copy all of the new markdown files into the _posts directory, create a PR, merge it, and see the final results up at <a href="http://ojug.org/">ojug.org</a>!

![OJUG Homepage](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/893i5x9flrtm8iur34kw.png)

<h3>Wrap up</h3>

And that's it! Writing this blogpost probably took longer than the process to export Meetup to JSON, and generate new markdown files for the Jekyll blog! You can view all of the full source on my github at <a href="https://github.com/jeffsheets/ojug-meetup-export">https://github.com/jeffsheets/ojug-meetup-export</a>

<p id="footnote1">
[1] - A quick footnote, about Meetup.com... Over the years Meetup has been great for advertising our group, attracting new members, having a great user interface, and easily collecting RSVP's for events. We've always been lucky enough to have great sponsors to pay the ever increasing fee, which is now up to $197.98/year. However Meetup's decision to not allow us (or any group) to "freeze" the account, means that our sponsor has been paying that fee for 2.5 years with little benefit. Talking to other local tech meetup organizers, it became apparent that many of us are pondering ways to free ourselves from these fees. Our sponsors could throw some pretty great user group parties with the savings! There's a lot of functionality we'd have to replicate though, so I'll leave that full discussion for another time...
</p>