How to build a Gatsby plugin to display your DEV posts
Tags: gatsby, webdev, graphql, react
Published: 2020-01-28 13:48:00

_Note: Since [my last Gatsby post](https://dev.to/ascorbic/how-to-use-drupal-paragraphs-in-your-gatsby-site-104l) I got [a job at Gatsby](https://www.gatsbyjs.com/careers/) on the open source team! This isn't an official post though._

DEV has [a simple API](https://docs.dev.to/) that means you can use it as a basic CMS. It's also a good way to cross-post your DEV posts to your own site. There are already plugins that let you do this, but they don't support everything I need, and in any case it's a great excuse to learn how to write a Gatsby source plugin. Did you know that Gatsby is [the #1 hottest tech skill to learn in 2020](https://www.cnbc.com/2019/12/02/10-hottest-tech-skills-that-could-pay-off-most-in-2020-says-new-report.html), according to Udemy?

First up, you should have a working Gatsby site. It's super quick to get started if you already have Node etc set up, but it's worth looking at the [quick start](https://www.gatsbyjs.org/docs/quick-start/). In this post we'll be using the default blog starter, so use the command:

```bash
gatsby new my-gatsby-blog https://github.com/gatsbyjs/gatsby-starter-blog 
```

If you need more help getting set up, check out the [step-by-step tutorial](https://www.gatsbyjs.org/tutorial/). When you have a site that runs, come back here and you'll build a plugin.

You can do this directly in your site's `gatsby-node.js` file, but the tidiest way to add a data source in Gatsby is to create a custom plugin. You don't need to publish it to NPM, but you can if you want to share your creation with the world. You just need to create a folder called `plugins` in the root of your site. Then create a folder with a name that matches your plugin. The only required file is a `package.json`.

Call yours `gatsby-source-dev`, so create the file `/plugins/gatsby-source-dev/package.json`

```json
{
    "name": "gatsby-source-dev"
}
```
A Gatsby plugin can do all sorts of things, but you're building a source plugin, which adds a new data source. You can use this data like any other in your site. You'll be using it to create pages. 

Create the file `gatsby-node.js` in the same folder, which does the actual processing. This file will be loaded and run by Node when Gatsby builds your site. If you export a function called `sourceNodes` then it will be called at the stage when the data is being loaded and created.

This is the most basic `sourceNodes` function. It doesn't need to be async, but yours should be because you'll be loading data from an API.  The first argument passed to the function is an object with lots of useful properties for performing the creation of nodes. The most important is the `actions` property, which lets you do the actual node creation.

```javascript
// /plugins/gatsby-source-dev/gatsby-node.js

exports.sourceNodes = async ({ actions }) => {
// Do cool stuff
}

```

Now, you're not creating anything yet, but this is a valid (but useless) plugin. If you add it to your site config (we'll get to that later), then it will be called whenever you build the site. Now it's time to do something useful with it.

The first thing you're going to want to do is load the data from the DEV API. As this isn't running in a browser, you can't use `fetch` etc, but you can use `axios`, which is included with Gatsby. 

```javascript
// /plugins/gatsby-source-dev/gatsby-node.js
const axios = require(`axios`);

exports.sourceNodes = async ({ actions }) => {
    const result = await axios.get(`https://dev.to/api/articles/me/published`);

}

```

If you try loading this now, you will get a "401 unauthorized" error. To fix this you'll need to [get an API key](https://docs.dev.to/api/#section/Authentication) from DEV. 

You'll be passing this key as a header, but first you need to look at how you configure a plugin. You don't want to hard-code the key in your plugin, so you need to pass it in to the plugin from your config. 

To tell Gatsby about you plugin you need to include it in the plugins array in your `gatsby-config.js`, and it is also where you set the options.

```javascript
module.exports = {
// rest of your config here
    plugins: [
        {
            resolve: `gatsby-source-dev`,
            options: {
                apiKey: `your-key-here`
            }
        },
    ]

}
```

If you're using the blog starter you'll see a couple of entries for `gatsby-source-filesystem`. You can remove the one with `name: "blog"` because you'll be getting your posts from DEV instead. You'll also need to remove `gatsby-plugin-feed` for now, as it'll need extra configuration to understand the posts once you're done.

Now, this will pass the key through to the plugin but it's still hard-coded in the source, which you should not do as this key will let anyone post to DEV in your name. The answer is environment variables, which can be imported from a local file or set in your build environment such as Netlify or Gatsby Cloud. The local variables are loaded by the package `dotenv`, which is already installed with Gatsby.

First create the `.env` file in the root of your project:

```bash
GATSBY_DEV_API_KEY=your-key-here
```
...then update your `gatsby-config.js` to load and use it:

```javascript
// /gatsby-config.js
require("dotenv").config();

module.exports = {
// rest of your config here
    plugins: [
        {
            resolve: `gatsby-source-dev`,
            options: {
                apiKey: process.env.GATSBY_DEV_API_KEY
            }
        },
    ]

}
```

The plugin options are passed to all of the `gatsby-node` functions, so you need to update your function to use the API key:

```javascript
// /plugins/gatsby-source-dev/gatsby-node.js
const axios = require(`axios`);

exports.sourceNodes = async ({ actions }, { apiKey }) => {
    const result = await axios.get(`https://dev.to/api/articles/me/published`, {
        headers: { "api-key": apiKey }
    });
}

```

The second argument passed to the function is the options object, and you're getting the `apiKey` option. You're then passing that as a header to the DEV API.

Now it's time to make some nodes. Nodes are the building blocks of the Gatsby data layer, and you can query them to create pages and display data etc. 

You need to loop through the results and create a node for each. The nodes that you create will mostly be a 1:1 copy of the objects returned from the DEV API. However you do need to do a couple of small tweaks to make it work properly. The best way to grab a couple of properties and pass through the rest is to destructure the object:

```javascript
// ... etc 
    result.data.forEach(post => {
        const { id, body_markdown, ...data } = post;
        // do stuff
    });
};

```

Next you're going to need some more of the helper functions that Gatsby provides, so add those to the arguments:

```javascript
exports.sourceNodes = async (
    { actions, createNodeId, createContentDigest },
    { apiKey }
) => {
    const { createNode } = actions;

    const result = await axios.get(`https://dev.to/api/articles/me/published`, {
        headers: { "api-key": apiKey }
    });

    result.data.forEach(post => {
        const { id, body_markdown, ...data } = post;

        // do stuff
    });
};

```

Now to create the actual nodes. The two changes you need to make are to create an proper node id, and to tell Gatsby to convert the markdown in the post. The clever thing is that all you need to do is set the media type and content, and the built-in markdown plugin will do the rest. It will even let you use plugins to do custom things with the markdown. The default blog starter includes plugins to do syntax highlighting, copying linked files and more. Any images used in the markdown will be automatically downloaded and processed.

```javascript
exports.sourceNodes = async (
    { actions, createNodeId, createContentDigest },
    { apiKey }
) => {
    const { createNode } = actions;

    const result = await axios.get(`https://dev.to/api/articles/me/published`, {
        headers: { "api-key": apiKey }
    });

    result.data.forEach(post => {
        // Destructure two fields and assign the rest to `data`
        const { id, body_markdown, ...data } = post;

        // Create the node object
        const node = {
            // Create a node id
            id: createNodeId(id),
            internal: {
                // Tell Gatsby this is a new node type, so you can query it
                type: `DevArticle`,
                // Set the markdown content
                mediaType: `text/markdown`,
                content: body_markdown
            },
            // Spread in the rest of the data
            ...data
        };
        const contentDigest = createContentDigest(node);
        node.internal.contentDigest = contentDigest;

        createNode(node);
    });
};
```

That's the plugin done! Run `gatsby develop` (or restart it if it's already running) and [open GraphiQL](http://localhost:8000/___graphql) and you'll find `devArticles` and `allDevArticles` in the explorer.

You might get markdown errors when you run the build. The parser in Gatsby is less forgiving than the DEV one, so you may need to fix some errors. e.g. I had to remove a colon from the frontmatter on one of my posts.

You can then use these to create pages. If you've started with the Gatsby blog starter, as shown in the docs, you'll have a `gatsby-node.js` file in the root of your project with a `createPages` function already in it. We just need to change it a little to create pages for your DEV posts.

First change the `createPages` function to match the one below. It's almost the same, but has a few changes to the data structure:

```javascript
exports.createPages = async ({ graphql, actions }) => {
  const { createPage } = actions

  const result = await graphql(`
    query DevArticleQuery {
      allDevArticle(filter: { childMarkdownRemark: { id: { ne: null } } }) {
        nodes {
          slug
        }
      }
    }
  `)

  if (result.errors) {
    throw result.errors
  }

  // Create blog posts pages.
  const blogPost = path.resolve(`./src/templates/blog-post.js`)

  const posts = result.data.allDevArticle.nodes
  posts.forEach((node, index) => {
    const previous = index === posts.length - 1 ? null : posts[index + 1]
    const next = index === 0 ? null : posts[index - 1]
    
    createPage({
      path: `posts/${node.slug}`,
      component: blogPost,
      context: {
        slug: node.slug,
        previous,
        next,
      },
    })
  })
}

```

The default blog also has an `onCreateNode` function that adds a slug based on the source filename. You need to remove because it gets confused by our markdown which isn't created from files and has a slug defined already.

This is enough to create the pages, however if you run `gatsby develop` now you'll get a load of GraphQL errors, because the page queries are still expecting the old markdown node types. You'll need to change anywhere that queries the pages. 

First go to `/src/pages/index.js` and change the page query to:

```graphql
  query ArticleQuery {
    site {
      siteMetadata {
        title
      }
    }
    allDevArticle {
      nodes {
        title
        slug
        published_at(formatString: "MMMM DD, YYYY")
        description
      }
    }
  }
```

...Then update the component to:

```jsx
export const BlogIndex = ({ data, location }) => {
  const siteTitle = data.site.siteMetadata.title
  const posts = data.allDevArticle.nodes

  return (
    <Layout location={location} title={siteTitle}>
      <SEO title="All posts" />
      <Bio />
      {posts.map((node) => {
        const title = node.title || node.slug
        return (
          <article key={node.slug}>
            <header>
              <h3
                style={{
                  marginBottom: rhythm(1 / 4),
                }}
              >
                <Link style={{ boxShadow: `none` }} to={`posts/${node.slug}`}>
                  {title}
                </Link>
              </h3>
              <small>{node.published_at}</small>
            </header>
            <section>
              <p
                dangerouslySetInnerHTML={{
                  __html: node.description,
                }}
              />
            </section>
          </article>
        )
      })}
    </Layout>
  )
}

export default BlogIndex
```


If you run `gatsby develop` now you should be able to load the site and see a list of your posts on the front page. If you don't, carefully check the queries and error messages.

Now you need to update the post template. There's also not too much to change here.

Go to `src/templates/blog-post.js` and update the page query to match this:

```graphql

  query BlogPostBySlug($slug: String!) {
    site {
      siteMetadata {
        title
      }
    }
    devArticle(slug: { eq: $slug }) {
      id
      title
      description
      published_at(formatString: "MMMM DD, YYYY")
      childMarkdownRemark {
        html
      }
    }
  }
```

...and then edit the component to change the data structures:


```jsx
export const BlogPostTemplate = ({ data, pageContext, location }) => {
  const post = data.devArticle
  const siteTitle = data.site.siteMetadata.title
  const { previous, next } = pageContext

  return (
    <Layout location={location} title={siteTitle}>
      <SEO title={post.title} description={post.description} />
      <article>
        <header>
          <h1
            style={{
              marginTop: rhythm(1),
              marginBottom: 0,
            }}
          >
            {post.title}
          </h1>
          <p
            style={{
              ...scale(-1 / 5),
              display: `block`,
              marginBottom: rhythm(1),
            }}
          >
            {post.published_at}
          </p>
        </header>
        <section dangerouslySetInnerHTML={{ __html: post.childMarkdownRemark.html }} />
        <hr
          style={{
            marginBottom: rhythm(1),
          }}
        />
        <footer>
          <Bio />
        </footer>
      </article>

      <nav>
        <ul
          style={{
            display: `flex`,
            flexWrap: `wrap`,
            justifyContent: `space-between`,
            listStyle: `none`,
            padding: 0,
          }}
        >
          <li>
            {previous && (
              <Link to={`posts/${previous.slug}`} rel="prev">
                ← {previous.title}
              </Link>
            )}
          </li>
          <li>
            {next && (
              <Link to={`posts/${next.slug}`} rel="next">
                {next.title} →
              </Link>
            )}
          </li>
        </ul>
      </nav>
    </Layout>
  )
}

export default BlogPostTemplate
```

Now open the site, click through to the link and you should see the post!

You could leave it there, but right now if you're using the starter, [Kyle](https://dev.to/kylemathews) is getting all the credit for your posts. You can change that by using the data from your DEV profile.

Open the bio component in `src/components/bio.js` and edit it to get the data from one of the DEV posts:

```jsx
const Bio = () => {
  const {devArticle} = useStaticQuery(graphql`
    query {
      devArticle {
        user {
          name
          profile_image_90
          twitter_username
        }
      }
    }
  `)

  const user = devArticle.user;

  return (
    <div
      style={{
        display: `flex`,
        marginBottom: rhythm(2.5),
      }}
    >
      <img
        width={45}
        height={45}
        alt={user.name}
        src={user.profile_image_90}
      />
      <p>
        Written by <strong>{user.name}</strong>
        {` `}
        <a href={`https://twitter.com/${user.twitter_username}`}>
          Follow them on Twitter
        </a>
      </p>
    </div>
  )
}

export default Bio
```

Now you should be able to run it and see your own profile.

When you deploy this you should be aware that it won't rebuild automatically when you add new posts to DEV. You'll need to manually trigger a rebuild when you post, or set your site to rebuild automatically at regular intervals.

You can check out [the source code for this demo](https://github.com/ascorbic/gatsby-dev-demo) and [view the result](https://gatsby-dev-demo.netlify.com/), built for free on [Gatsby Cloud](https://gatsbyjs.com/cloud/) and hosted on Netlify.