Create dynamic sitemap in Next.js app
Tags: nextjs, seo, sitemap, react
Published: 2022-11-07 20:22:27


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/53kb9bjvnc0iicmxntvs.jpeg)


> SEO, or Search Engine Optimization, means setting up your website and content to show up through online search results. While many marketing tactics rely on you reaching out to your audience, SEO gives you the power to reach people when they are actively searching out information related to your products and services.


## Steps to create Dynamic Sitemap File

- Firstly we have to install [next-sitemap](https://www.npmjs.com/package/next-sitemap).
- Create next-sitemep.js file on the `next.config` file level.
- Define your base url by  adding this code in `next-sitemap.config.js` file to read website urls .

``` javascript
const siteUrl = "https://your-host"

module.exports = {
    siteUrl
}
```

> Now build the app and will find that `sitemap.xml` file will be automatically created with your static routes.


To auto generate  `robots.txt` file we just need to add `generateRobotsTxt: true ` to the exports object in 
`next-sitemap.config.js` file , so `next-sitemap.config.js`  file after updating will be like following :

``` javascript
const siteUrl = "https:your-host"

module.exports = {
    siteUrl,
    generateRobotsTxt: true
}
```

To make sure it will regenerated each time we build the app add the following two lines to `.gitignore` file

```
/public/sitemap.xml
/public/robots.xml
```

What if we need to exclude some pages from being added to sitemap ?
we can do it by updating `next-sitemap.config.js` file and adding exclude array to the exports object.

``` javascript
const siteUrl = "https:you-host"

module.exports = {
    siteUrl,
    generateRobotsTxt: true
    exclude: ["/<page-url>"]
} 
```

And to disallow those pages in robots.txt file, we can do it using `robotsTxtOptions` object .

So `next-sitemap.config.js` file will be like following after adding pages we need to disallow .

``` javascript
const siteUrl = "https:yuor-host";

module.exports = {
  siteUrl,
  generateRobotsTxt: true,
  robotsTxtOptions: {
    policies: [
      { userAgent: "*", disallow: "/<page-url>" },
      { userAgent: "*", allow: "/" },
    ],
  },
  exclude: ["/<page-url>"],
};
```

**In the above steps we had created a dynamic sitemap for the static pages we have ,[ex: example.com , example.com/about , … etc] .**

**This is great but what about the dynamic rendering pages ? [like : example.com/posts/[post_id] , … etc ] .**

**The problem is that we don’t have  params in those urls.**

**So what should we do to overcome this problem ?**

- Firstly we have to get all dynamic pages urls .
- Then we can create another sitemap for those pages .

let’s do it 🥷

create a folder called `server-sitemap.xml` in the **Pages** folder & create an `index.js` file inside.

In the `index.js` file we will use `getServerSideProps` to get urls for dynamic pages .

``` javascript
import Axios from "axios";

export async function getServerSideProps(context) {
  const response = await Axios.get(
    `base_url/<data>`,
  );
  const data = await response?.data;
  return {props:{data}};
}

export default function Site() {}

```

then we will create an array of objects `fields`  which contains `{loc : <page url> ,lastmod: <time now>} `.

``` javascript
import Axios from "axios";

export async function getServerSideProps(context) {
  const response = await Axios.get(
    `base_url/<data>`,
  );
  const data = await response?.data;
  const fields = data?.map((item) => ({
    loc: `https://your-host/${item?.url}`,
    lastmod: new Date().toISOString(),
  }));
  return {props:{data}};
}

export default function Site() {}
```

Then we will import `getServerSideSitemap`  function from `next-sitemap` and use it to generate sitemap links and return the result from `getServerSideProps`

``` javascript
import Axios from "axios";
import { getServerSideSitemap } from "next-sitemap";

export async function getServerSideProps(context) {
  const response = await Axios.get(
    `base_url/<data>`,
  );
  const data = await response?.data;
  const fields = data?.map((item) => ({
    loc: `https://your-host/${item?.url}`,
    lastmod: new Date().toISOString(),
  }));
  return getServerSideSitemap(context, fields);
}

export default function Site() {}
```

**Great , now we had created the server dynamic sitemap & if we go to the url `host/server-sitemap.xml` we will find that our sitemap includes fetched urls as expected .**

Last thing to do here is to inform google crawler that it’s not only one sitemap ,but two . 
And this will be done by adding `additionalSitemaps` key to `robotsTxtOptions` object in `next-sitemap.config.js` file and add our two sitemaps _[static one that we have created before for static routes & server side one that we created for URLs with params]._

``` javascript
const siteUrl = "https://your-host";

module.exports = {
  siteUrl,
  generateRobotsTxt: true,
  robotsTxtOptions: {
    policies: [
      { userAgent: "*", disallow: "/page-url" },
      { userAgent: "*", allow: "/" },
    ],
    additionalSitemaps: [
      `${siteUrl}/sitemap.xml`,
      `${siteUrl}/server-sitemap.xml`,
    ],
  },
  exclude: ["/page-url"],
};
```




**That’s all , now we had created a dynamic sitemap for static pages & dynamic sitemap for dynamic rendering pages .**