Custom Gatsby.js Source Plugin
Tags: react, gatsby, graphql, javascript
Published: 2019-08-24 16:28:50

---
title: Custom Gatsby.js Source Plugin
published: true
description: We take a deep dive into how data and GraphQL works with Gatsby.js, and also create our own data nodes for Gatsby using an external API.
tags: React, Gatsby, GraphQL, javascript
canonical_url: https://codebushi.com/custom-gatsby-source-plugin/
---

One of the greatest superpowers of Gatsby.js is its ability to use data and content from a variety of sources, such as Wordpress, Markdown files, or APIs. During the build process, Gatsby will query for the data and turn it into content for a static site. There are a wide variety of plugins, called ["source plugins"](https://www.gatsbyjs.org/plugins/?=gatsby-source), that will help you pull data from different places. Today we'll be looking into how we can make our own source plugin and how they work under the hood.

<h4 class="mt-4 mb-4"><a href="https://gatsby-custom-source.surge.sh/">Gatsby Custom Source Plugin</a> <small>( <a href="https://github.com/codebushi/gatsby-custom-source-plugin">view source</a> )</small></h4>

The source we want is [https://rawg.io/apidocs](https://rawg.io/apidocs), which is an API for video games. Before we use the API, let's take a look at how data and sources work in Gatsby. We'll use the Gatsby default starter to kick things off.

```bash
# Download and install the starter using the Gatsby CLI
gatsby new games-source https://github.com/gatsbyjs/gatsby-starter-default
cd games-source
gatsby develop
```

In Gatsby, there is a GraphQL layer which helps you pull in data. You can build sites without the GraphQL layer, but the GraphQL helps to structure the data and makes it a lot easier to work with.

With the development server running, open up the GraphiQL playground at http://localhost:8000/___graphql. On the lefthand side, you should see a variety of dropdowns and checkboxes under the Explorer sidebar. This is the current data that's available to you via the GraphQL data layer, built into the Gatsby Default Starter. We can make a query for the site's metadata, which is coming from the `gatsby-config.js` file, using the following query:

```graphql
query {
  site {
    siteMetadata {
      author
      description
      title
    }
  }
}
```

If you paste that into the box and hit the "Play" button, you'll see the siteMetadata show up in the right box.

![Site Metadata](https://thepracticaldev.s3.amazonaws.com/i/x8nd0busndeocon94kmu.png)

Whenever we add a source plugin, new sources will appear in the Explorer sidebar and we can run queries on those sources. To see this in action, we can add our own custom test source into Gatsby.

Open up the file called `gatsby-node.js`. This file allows us to have access to Gatsby's APIs, the one we'll focus on today is the [sourceNodes](https://www.gatsbyjs.org/docs/node-apis/#sourceNodes) API. We can now make data "nodes" for the GraphQL data layer.

```javascript
// In gatsby-node.js
exports.sourceNodes = async ({ actions, createNodeId, createContentDigest }) => {
  // We'll make the newNode object here for clarity
  const newNode = {
    title: 'Test Node!',
    description: 'This is a test node with static data',
    id: createNodeId('TestNode-testid'), // required by Gatsby
    internal: {
      type: 'TestNode', // required by Gatsby
      contentDigest: createContentDigest('testnode') // required by Gatsby, must be unique
    }
  };
  // This is where we actually create the data node, by passing in the newNode object.
  actions.createNode(newNode);
};
```

We create the actual node using `actions.createNode()`, which takes in an object. There are certain properties that are required by Gatsby in order for a node to work. We use `createNodeId` and `createContentDigest`, which are helper functions provided by Gatsby, to help use fill in those required properties.

To see this new node, we have to restart our development server. In the terminal, terminate the server using `control + c` and restart it using `gatsby develop`.

Open up the GraphiQL playground again and paste in this query:

```graphql
query MyQuery {
  testNode {
    title
    description
    internal {
      contentDigest
      type
    }
  }
}
```

As you can see, we now have the test data node showing up!

![Test Data](https://thepracticaldev.s3.amazonaws.com/i/93s6gqbvrp1sbek6qtwd.png)

We now know how to create data nodes, so the next step is to use our video game API to get real data and create nodes. All the functionality in `gatsby-node.js` runs using Node.js, and because of that we'll need to install an additional npm package that allows us to fetch data using Node.js.

Shut down the development server and install the package `node-fetch`

```bash
yarn add node-fetch
```

You can use either `yarn add` or `npm install`, but since the Gatsby CLI uses yarn, it's probably the better choice right now.

Open up `gatsby-node.js` again with the following

```javascript
const fetch = require('node-fetch');

exports.sourceNodes = async ({ actions, createNodeId, createContentDigest }) => {
  try {
    // Fetch the data
    const res = await fetch(`https://api.rawg.io/api/games`);

    // Transform the data into json
    const data = await res.json();

    // Map over the results array, calling action.createNode on each item in the array
    data.results.forEach(game => {
      const node = {
        ...game, // We copy all of the properties from the game object
        id: createNodeId(`RAWG-game-${game.id}`), // Needs to be unique
        internal: {
          type: 'RAWGGame',
          contentDigest: createContentDigest(game) // We pass in the game object to make sure it's unique
        }
      };

      // Create the actual data node
      actions.createNode(node);
    });
  } catch (error) {
    console.log(error);
  }
};
```

Be sure to require the `node-fetch` package at the top of the file. We do a GET request on the endpoint `https://api.rawg.io/api/games`. I like this as an example because there are no other parameters needed to use this endpoint. Once we get the data, we loop over `data.results` and using each game object in the array, we make our data nodes.

Restart the development server and head over to [http://localhost:8000/\_\_\_graphql](http://localhost:8000/___graphql) once again. You can now run queries on the games like so:

```graphql
query MyQuery {
  allRawgGame {
    edges {
      node {
        id
        name
        background_image
      }
    }
  }
}
```

![Video Game GraphQL Data](https://thepracticaldev.s3.amazonaws.com/i/ojdtf70vfu9vk9id6ae7.png)

We can access all the data under `allRawgGame > edges > node`. Using the amazing GraphiQl explorer, we can toggle all the fields available to us with ease! So now we have this data, let's use it in our Gatsby site. Open up the main `src/pages/index.js` page.

```jsx
import { graphql } from 'gatsby';
import React from 'react';
import Layout from '../components/layout';
import SEO from '../components/seo';

// The GraphQL data is passed into the component via props.data
const IndexPage = props => {
  return (
    <Layout>
      <SEO title="Home" />
      {props.data.allRawgGame.edges.map(({ node }) => {
        return (
          <div key={node.id} style={{ marginBottom: 20 }}>
            <p>{node.name}</p>
            <p>
              <img src={node.background_image} alt={node.name} />
            </p>
          </div>
        );
      })}
    </Layout>
  );
};

// We use the GraphiQL query here
export const query = graphql`
  query MyQuery {
    allRawgGame {
      edges {
        node {
          id
          name
          background_image
        }
      }
    }
  }
`;

export default IndexPage;
```

Now if you go to the normal development site at [http://localhost:8000/](http://localhost:8000/) you'll see our games! During the `gatsby-build` process, this data will be injected into our static site and we can just serve it up, blazingly fast!

Now if you wanted to make this into a stand-alone source plugin, you can create a new directory called `plugins` in the root folder. You'll copy over the `gatsby-node.js` file into that folder and create a package.json that describes the source plugin and also the dependency `node-fetch`.

To use the plugin, you would add it into the `gatsby-config.js` file like the other plugins. If your source API needs an API key or other options, you can pass those options in when adding it into the `gatsby-config.js` file

```javascript
plugins: [
  {
    resolve: `gatsby-source-rawg`,
    options: {
      apiKey: 'API_KEY_HERE'
    }
  }
];
```

In the `gatsby-node.js` file, you can access these new options in the `sourceNodes` like so

```javascript
exports.sourceNodes = async ({ actions, createNodeId, createContentDigest }, options) => {
  console.log(options); // The options are passed as another argument
  try {
    const res = await fetch(`https://api.rawg.io/api/games`);
    const data = await res.json();
    data.results.forEach(game => {
      const node = {
        ...game,
        id: createNodeId(`RAWG-game-${game.id}`),
        internal: {
          type: 'RAWGGame',
          contentDigest: createContentDigest(game)
        }
      };
      actions.createNode(node);
    });
  } catch (error) {
    console.log(error);
  }
};
```

Also, it's worth noting that currently the images are NOT static, they will be loaded from whatever url we get back from the endpoint. We'll make these static using the Gatsby Image plugin in a future tutorial. Hope this was helpful!

<h4 class="mt-4 mb-4"><a href="https://gatsby-custom-source.surge.sh/">Gatsby Custom Source Plugin</a> <small>( <a href="https://github.com/codebushi/gatsby-custom-source-plugin">view source</a> )</small></h4>