Making a Dynamic Renderer with Golang from Scratch
Tags: go, webdev, react
Published: 2023-03-07 15:01:06

## Understanding Dynamic Rendering: Why It Matters for Your Website's SEO

When it comes to building a website, the three key components are HTML, CSS, and Javascript. In recent years, client-side rendering (CSR) has become increasingly popular. With this approach, the browser downloads an empty HTML shell and uses Javascript to generate the content on the client side.

![](https://web-dev.imgix.net/image/T4FyVKpzu4WKF1kBNvXepbi08t52/JFxoxQe847ntctVOdn5u.png?auto=format&w=845 align="center")

\[Image credits: [web.dev](%5Bhttps://web-dev.imgix.net/image/T4FyVKpzu4WKF1kBNvXepbi08t52/JFxoxQe847ntctVOdn5u.png?auto=format&w=845%5D(https://web-dev.imgix.net/image/T4FyVKpzu4WKF1kBNvXepbi08t52/JFxoxQe847ntctVOdn5u.png?auto=format&w=845)) \]

However, while CSR may work well for users, it creates a significant problem for search engines and other bots. Bots rely on the generated HTML content to gather data and index your website. But when a bot visits a client-side rendered page, there is no data available because the Javascript hasn't been executed yet. As a result, the bot only sees an empty website, which can severely impact your website's SEO.

That's where dynamic rendering comes in. Dynamic rendering is the process of rendering a fully formed HTML page on the server side and sending it to bots, while still using client-side rendering for users. This approach allows bots to access the fully rendered HTML content and gather the data they need to index your website, without affecting the user experience.

In this article, we'll explore dynamic rendering in more detail and show you how to build a dynamic renderer using Golang, a powerful and efficient programming language, from scratch. With this knowledge, you'll be able to create high-performance web applications that not only deliver a great user experience but also rank well in search engines.

## But do I even need Dynamic Rendering?

Well, that depends.

❌ **You don't need dynamic rendering when** -

* If you are working with meta frameworks like NextJS, NuxtJS, Remix, etc. Because these are server-side rendering applications.
    
* If your codebase is small and you can migrate it to server-side rendering applications
    

✅ **You need dynamic rendering when**

* When your codebase is huge and you don't have the bandwidth to migrate it to server-side rendering applications. Also, you want to improve your website as soon as possible.
    

But keep in mind,

> Dynamic rendering is a workaround and not a long-term fix for problems with javascript.

## Implementing Dynamic Rendering

Let's first discuss the high-level overview, before jumping into the code.

* Since we need to render the HTML before sending it to the bot, we need a server.
    
* We will define a middleware, it intercepts the request made to the web app.
    
* We will use the user agent to check if the request was made by a bot or a user.
    
* If the request was made by a user, we send the client-side rendered app.
    
* But if the request was made by a bot, we generate the HTML content (with something like **puppeteer**) and then send the rendered HTML to the bot
    

Now that we are done with the high-level overview, let's jump into the implementation

### Setting up server

For setting up the server, I will be using the Gin framework.

```go
func main() {
	r := gin.Default()
	r.Use(our_middleware)

	r.Static("/", "./frontend/dist")

	if err := r.Run(":3000"); err != nil {
		log.Fatal(err)
	}
}
```

This starts our server at port 3000. Our CSR frontend files are present in the folder `frontend/dist`. Now when someone sends a request for a page, we serve HTML, CSS, and JS from this folder.

Now we need middleware to intercept the traffic. So let's implement it.

### Adding a middleware

```go
func dynamicRenderer() gin.HandlerFunc {
	return func(c *gin.Context) {
		// Check if request is from a bot
		isBot := checkforBot()
		if isBot {
          // render page and send the rendered page
			return
		}

		// If not a bot, continue to serve as usual
		c.Next()
	}
}

func main() {
	r := gin.Default()
	r.Use(dynamicRenderer())

	r.Static("/", "./frontend/dist")

	if err := r.Run(":3000"); err != nil {
		log.Fatal(err)
	}
} 
```

Now that the middleware is set up, let's write the code for rendering the HTML file. But before that, we need to run our front end on some other port. Because the puppeteer scrapes data from the website. So let's write code to start the frontend

### Serving the actual front end from another port

```go
func main() {
	// code

	cmd := exec.Command("command","to","serve","your","frontend")
	cmd.Dir = "./frontend"
	cmd.Stderr = os.Stderr

    wait_for_files_to_be_served()

	if err := r.Run(":3000"); err != nil {
		log.Fatal(err)
	}
}
```

### Writing the code for the renderer

```go
func dynamicRenderer() gin.HandlerFunc {
	return func(c *gin.Context) {
		isBot := checkForBot()

		if isBot {
			// Connect to Puppeteer
			ctx, cancel := chromedp.NewContext(context.Background())
          // We cancel the connection once the response is sent
			defer cancel()

			// Navigate to the page and wait for it to load
			url := "http://localhost:" + reactPort + c.Request.URL.Path
			var html string
			
          err := chromedp.Run(ctx,
				chromedp.Navigate(url),
				chromedp.InnerHTML("html", &html, chromedp.NodeVisible, chromedp.ByQuery),
			)
			if err != nil {
				log.Println(err)
				c.AbortWithStatus(http.StatusInternalServerError)
				return
			}

			// Send back the rendered HTML
			c.Data(http.StatusOK, "text/html; charset=utf-8", []byte(html))
          // We are done serving
			c.AbortWithStatus(http.StatusOK)
		}

		// If not a bot, continue to serve the React app as usual
		c.Next()
	}
}
```

Here's a short explanation of what happening:-

* Once we know it's a bot, we connect to the puppeteer.
    
* We then navigate to the URL, using `chromedp.Navigate(url)` .
    
* We wait for all the children of the HTML tag to load by using `chromedp.NodeVisible`
    
* Once the javascript has generated all the HTML content we store it in `html` variable and send it to the bot.
    

> Well, that's the entire implementation !! (sort of). I have left some boring parts out, but if you want you can check this [**repository**](https://github.com/Avash027/dynamic-rendering-go).

## Now, let's look at the result

* **Our webpage looks like this**
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1678199585818/8bfc3ecb-cda8-42c5-aad9-81ee55f21c7a.png align="center")

* **When the user is requesting for a file, we get**
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1678199472768/0acce655-0561-48e3-a6e1-11bca335d9cd.png align="center")

* But when a bot requests a file, we get
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1678199517497/350e558a-888b-47e5-a936-a618b96266ed.png align="center")

Notice that the HTML is all rendered.

> **NOTE:** When we send a rendered page there is not JS. So it won't function. But for a bot that is not an issuse because it does not use JS in any way

Well that's all. Thanks for reading. If you have any doubt, you can post it it comment. I'll try my best to clear your doubts