How to use Protobuf with Go?
Tags: protobuf, contracts, go, messaging
Published: 2022-04-06 19:08:47

---
title: How to use Protobuf with Go?
published: true
description: How to use Protobuf with Golang
tags: protobuf,contracts, go, messaging
//cover_image: https://direct_url_to_image.jpg
---


# How to use Protobuf with Go?

> Have you ever been in a situation where you do not know the structure of data that services are either consuming or publishing?

Did you know that you have to write a schema only once which defines the structure data and the same schema can be used as validation before sending & receiving data using [Protocol Buffers](https://developers.google.com/protocol-buffers)?

In this tutorial, you will learn about using Protobuf. You will send & receive a message asynchronously using Protobuf with [RabbitMQ](https://www.rabbitmq.com/).

# Table Of Contents

- Prerequisites
- Protobuf
- Bootstrapping Project
- Creating Protobuf Messages
- Publishing a Message
- Consuming a Message
- Conclusion

# Prerequisites

This tutorial will be a hands-on demonstration. If you’d like to follow along, be sure you have the following:

- You have running [Docker Composer](https://docs.docker.com/compose/) in your machine.
- You have a basic understanding of [Docker](https://www.docker.com/), [GO](https://go.dev/), [Rabbitmq](https://www.rabbitmq.com/).

# Protobuf

Protobufs are the Messaging Contracts. They are schema that describes the structure of the message. These files are written in `.proto` extension which is language-neutral i.e write once & use anywhere. Proto Messages are the structured schema that is inside `.proto `files

You will write Proto Messages and with the compiler called [Proto Compiler](https://grpc.io/docs/protoc-installation/), you can generate the code which will have a way to serialize & deserialize data into that format.

There are many benefits of using Messaging Contracts like Protobuf :

1. With these Proto Messages, you will know what type of data you are sending & receiving
2. Serves as a documentation
3. Supports validation for serializing & deserializing
4. Proto messages are language-neutral and it supports various programming languages which means you can publish messages from Go & start consuming them from Php / Node etc

# Bootstrapping Project

For creating a new test project follow the steps below:

- Copy `Dockerfile` & `docker-compose.yaml` from [this repository](https://github.com/ujwaldhakal/golang-protobuf) which will dockerized Go source code, Proto compiler & RabbitMQ
- Create a file `publish.go` where you will write some code to publish events
- Create a file `consume.go` where you will write some code to consume events

# Creating Protobuf Messages

In an application, while you sending messages you will serialize data using the definition from Proto Messages. And when consuming messages you will deserialize them using the same Proto Messages. Let's create a Proto Message so your services can use them to send and receive messages.

1. Create a folder `messages` and inside that folder create a file called `user_registered.go`
2. Inside `user_registered.go` copy the following codes. Let's understand the code

- `syntax` defines which proto version you are going to use
- `go_package` defines the path where all your generated Protobuf code will be into
- ``import “validate/validate.proto”;` it is importing the validator so that you can validate message using various rules without writing it by yourself
- All message starts with a message keyword followed by name of the message. Inside the block, you can write key-value pairs where the key defines name & type & the value defines its order & rules if there are any. In our case, you are defining the first field `userId` which is an integer and should be greater than 0 and the next is an email which is a string and should not be empty.

{% gist https://gist.github.com/ujwaldhakal/e39291428a6cbd24026d62a59be1a8ba %}
\3. Run `make build-proto` on your project root. This command will generate two files `user_registered.pb.go and user_registered.pb.validate.go` that define type & includes all the validation logic that you specified on the text which you can later use on your code while serializing & deserializing.

# Publishing a Message

As you have successfully created a Proto Message. Let's use the same auto-generated code while sending messages to other services for this you need to write a way to publish messages. Follow the following steps:

1. Create a directory `pkg` and a file `rabbitmq.go` inside pkg dir where you will make a connection with RabbitMQ. The following code the function `getConnection` is importing library and making a connection with RabbitMQ server and upon error function `failOnError` is terminating the application with an error.

{% gist https://gist.github.com/ujwaldhakal/ad71f7df6aec58d7f3233a5739cc91e4 %}
\2. After successfully making a connection, write a function that will publish a message to RabbitMQ. The following code accepts `queueName` and payload as a `message` which will write a message to default exchange to JSON format.

{% gist https://gist.github.com/ujwaldhakal/5f1e9db47d1bb762d871189301e9eca5 %}

\3. Create `publish.go` which will import publish function from `rabbitmq.go` to send data. The following code is using a struct that was generated by the proto using the proto compiler and it is encoding them into JSON after that it is calling to publish the function that you created earlier to send data to `UserRegistered`queue.

Try running `make publish` to verify that your code is actually working if you see `Data has been published` the output that means you were successful in publishing a message.

{% gist https://gist.github.com/ujwaldhakal/1e214b7d9d4fc4d6afc3c014cdc7bcae %}


# Consuming a Message

Until now you defined a photo message then you created a `rabbitmq.go `to create a way to publish messages & you used the same function `publish.go` to publish messages to RabbitMq. Now let's consume the message that you published earlier.

1. Inside `rabbit. go` add a function that will help in consuming a message. The following code makes a connection to RabbitMq & returns a channel that you can leverage to listen to incoming messages.
{% gist https://gist.github.com/ujwaldhakal/088f9c5ad88c8cc8b2f658018c984d9f %}


\2. Inside `consume.go` listen to all the messages and then print them. Let's understand the following code :

- The main function is listening to message by creating a consuming client. While deserializing message it is importing `UserRegistered` struct from the auto-generated code to define the type of data. There might be cases where the producer might send invalid data due to various reasons so it is better to validate that data you are consuming are as per the contracts that you specified.
- `validateMessage` accepts the JSON data and tries to validate as per the `user_registered.proto` definition and if the data breaks any validation rules it will throw a validation error

{% gist https://gist.github.com/ujwaldhakal/ec7b96a5cfedcd5ecdf0272838519978 %}


# Conclusion

Throughout this tutorial, you have learned Protobuf & how to use them with GO & RabbitMQ.

Protobuf is one of the ways to implement Messaging Contracts there are more options like [Pact](https://docs.pact.io/), [AsyncApi](https://www.asyncapi.com/)

Now with this knowledge, you can use Protobuf on any programming language to send & receive the data. You have also learned that it was easier to predict the data that you are consuming or publishing.

Let me know if you have any comments /suggestions/feedback.

Source code: https://github.com/ujwaldhakal/golang-protobuf
Ref: https://stackshare.io/protobuf