How personalisation works in Sitecore XM Cloud
Tags: sitecore, xmcloud, personalization, webdev
Published: 2024-07-11 11:07:43

In my previous article, I shared a comprehensive [troubleshooting guide for Sitecore XM Cloud tracking and personalisation](https://dev.to/byteminds_agency/troubleshooting-tracking-and-personalisation-in-sitecore-xm-cloud-2n6). The guide addresses common issues, explains investigation steps and provides solutions for these issues. 

However, understanding how the personalisation engine works in depth can further help in diagnosing persistent issues and developing personalised websites. This article visualises what happens behind the scenes when you enable personalisation and tracking in your Sitecore XM Cloud applications.

## Overview of the personalisation workflow

Before we start, let's familiarise ourselves with key elements of the diagram we are going to look at.

![Key elements of personalisation and tracking data flows](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mqidbrg3lvgkvmbbwumt.png)

1. On the left hand site we can see the **Browser**, it is responsible for sending requests to our application and displaying the result to end users.

2. The **JSS app** sits at the top centre of the diagram and it represents the Rendering Host role in Sitecore Headless topology. It is the application that processes incoming requests and handles the presentation layer. In this case it is a Next.js application based on the [XM Cloud foundation template](https://doc.sitecore.com/xmc/en/developers/xm-cloud/getting-started-with-xm-cloud.html).

3. **Edge / XM API** is shown on the right hand side, it is a GraphQL endpoint that returns layout definition and content for requested pages, including [personalised variants](https://doc.sitecore.com/xmc/en/users/xm-cloud/create-a-page-variant.html). It can be Experience Edge API for cloud-based setups or a local CM container API endpoint for development purposes.

4. Finally, at the bottom we can see **Tracking & Interactive API** powered by the embedded instance of Sitecore CDP & Personalize. This is where [audiences](https://doc.sitecore.com/xmc/en/users/xm-cloud/creating-an-audience.html) are stored, [conditions](https://doc.sitecore.com/xmc/en/users/xm-cloud/specifying-variables-for-conditions.html) are executed and [analytics](https://doc.sitecore.com/xmc/en/users/xm-cloud/analyze.html) is collected.

When we talk about Next.js applications, there are two main rendering methods: **Server-Side Rendering (SSR)** and **Static Site Generation (SSG)**. Sitecore XM Cloud personalisation engine works slightly differently with these two approaches so we will cover both of them to understand their specifics.

## Server-Side Rendering

This is what happens when a website visitor opens a page that has personalised variants.

**Step 1.** When a user loads the website or navigates to a new page, the browser sends an HTTP request to the Next.js application, including cookies and HTTP headers that can be used in personalisation conditions.

![Step 1. Browser request](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uw5fqmuxes3iuj20gbjg.gif)

**Step 2.** The Next.js application runs all registered middleware modules, with the [Personalize middleware](https://doc.sitecore.com/xmc/en/developers/jss/220/jss-xmc/personalization-in-jss-next-js-applications.html) being of particular interest. This middleware sends an API request to the GraphQL endpoint to fetch all personalised variants for the current page configured in the CMS.

If there are no personalised variants configured for this page or the page is not found, this middleware will exit and page generation will continue as usual.

![Step 2. Personalize middleware kicks in and sends a request to Edge / XM API](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lgaxkpr6kk7eqotdqfoo.gif)

**Step 3.** If a page has more than one variant, the middleware sends another API request to the Personalize API to detect if the current visitor matches any of the audiences configured for this page. This is where cookies and HTTP headers received from the browser will help as they will be passed to the Personalize API to identify the visitor.

![Step 3. Identifying the audience for the current visitor](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6nqvutp4r94y2ks8wrj1.gif)

**Step 4.** By combining responses from these two API requests, the middleware determines which personalised page variant is suitable for the current visitor (cat-themed page in the diagram below 😺).

If the visitor matches an audience configured for the page, the middleware will rewrite the page path to a special personalised variant path (for example, from `/_site_Test/Pets` to `/_variantId_0dd7b00680be49c6815ca4d0793a36da/_site_Test/Pets`) and this will instruct the Next.js application to use the specific page variant when rendering the page. So the personalised version of the page will be rendered on the server and returned to the browser. 

If the visitor does not match any audiences, then the default page variant will be rendered and returned to the user. 

![Step 4. Returning the personalised page](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fjvnouyg4pcc1z1ym17p.gif)

**Step 5.** Once the page is rendered in the browser, a special React component responsible for tracking will send an API request to the CDP Stream API to register the page view, including which personalised variant was shown. This data is later will be aggregated and shown in analytics reports.

![Step 5. Sending a page view event](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3aq7ibcnf6qnd00vbk9h.gif)

## Static Site Generation (SSG)

The SSG process flow is similar to SSR but has some specifics related to this rendering method. Now, let's see what are these differences.

**Step 1.** This step is exactly the same as for SSR - the browser sends an HTTP request to the Next.js application with cookies and HTTP headers.

![Step 1. Browser request](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4w4sym8f4mn9qhluzv5l.gif)

**Step 2.** This is where things get different from the SSR process. When the Personalize middleware kicks in, it checks if there are any pre-rendered page variants for this page (the default, cat-themed 😺 and dog-themed 🐶 variants in the diagram). If yes, it skips the API request to the Edge / XM API, otherwise it will fall back to the standard SSR process and fetch personalised variants for the current page.

![Step 2. Leveraging pre-rendered page variants](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v4d6njbrszjvn9lglug8.gif)

**Step 3.** This step is the same as in the SSR flow - if a page has personalised variants, the middleware sends an API request to the Personalize API to identify visitor's audience.

![Step 3. Identifying the audience for the current visitor](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jlhmjrprv2tk8uvcd28a.gif)

**Step 4.** If there is a match and personalised page variants are pre-generated, the middleware will rewrite the page path and then the appropriate personalised page variant will be chosen and returned to the browser (looks like it's the cat-themed variant again! 😺). 

If there are no pre-generated personalised variants, but they exist in the CMS and the visitor matches one of the audiences, then the middleware will rewrite the page path, the Next.js app will generate the page variant and save the static output for future requests. This is the default process, see notes at the end of the article to learn more about static generation of personalised page variants.

If the visitor does not match any audiences, then the default page variant will be returned to the user using the statically generated HTML if it exists.

![Step 4. Returning the personalised page](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2a1epfhsffuam4ekangy.gif)

**Step 5.** As with SSR, once the page is returned to the browser the `CdpPageView` React component will send an API request to track the page view event for reporting. 

![Step 5. Sending a page view event](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8rs3mcnlgi2b9031fl1r.gif)

As you can see, the flow is very similar for SSR and SSG. The SSG method with static HTML generation and skipping some API requests can give us a performance boost, especially for websites with high traffic and personalisation enabled on frequently visited pages.

## Notes
### Personalize middleware 

The middleware is provided by Sitecore as a part of the [JSS XM Cloud add-on for Next.js](https://doc.sitecore.com/xmc/en/developers/jss/220/jss-xmc/the-jss-xm-cloud-add-on-for-next-js.html). Please note that this add-on is compatible with JSS version 21.6 and later. For earlier versions the [Next.js Personalize add-on](https://doc.sitecore.com/xmc/en/developers/jss/215/jss-xmc/the-next-js-personalize-add-on.html) is used that is now obsolete. 

This add-on is only compatible with Sitecore XM Cloud due to specific naming conventions and pre-configured settings required for the embedded CDP and Personalize instance.

### Build-time static generation of personalised page variants

To expand on the step 4 of SSG process, let's see when exactly personalised page variants are generated. As you may know, hosting providers often limit the time available for SSG builds. By default, pre-generation of personalised page variants during build is disabled in to avoid long build times.

However, if sufficient build time is available (for example, your website does not have too many pages) or you have critical personalisation rules on key pages (for instance, you only have a small number of personalised variants on the homepage or an important campaign page), then SSG for personalised variants [can be explicitly enabled](https://doc.sitecore.com/xmc/en/developers/jss/220/jss-xmc/walkthrough--configuring-personalization-in-a-next-js-jss-app.html#enable-static-generation-for-personalized-variants). 

This can be done by modifying the file `src/lib/sitemap-fetcher/plugins/graphql-sitemap-service.ts` and setting the `includePersonalizedRoutes` parameter to `true` in the sitemap service constructor:

```
this._graphqlSitemapService = new MultisiteGraphQLSitemapService({
    clientFactory,
    sites: [...new Set(siteResolver.sites.map((site: SiteInfo) => site.name))],
    includePersonalizedRoutes: true,
});
```

Just make sure to watch your build time after enabling this setting to avoid build failing or incurring unnecessary hosting costs.

---

Sitecore XM Cloud provides robust support for personalisation out-of-the-box for both SSR and SSG rendering methods. The add-on with Personalize middleware and CDP tracking component streamlines the process of fetching, matching, and delivering personalised content to website visitors while tracking interactions for reporting. 

Hope this article helps to understand the entire process of personalisation and tracking in XM Cloud and allows you to build well-performing and personalised applications. Feel free to share your thoughts and questions in the comments!