12.Data Transmission Protocol: Protocol Buffers
Tags: zinx, go
Published: 2023-08-18 08:33:52

### [Zinx]

[<1.Building Basic Services with Zinx Framework>](https://dev.to/aceld/1building-basic-services-with-zinx-framework-296e)
[<2. Zinx-V0.2 Simple Connection Encapsulation and Binding with Business>](https://dev.to/aceld/2-zinx-v02-simple-connection-encapsulation-and-binding-with-business-4p9g)
[<3.Design and Implementation of the Zinx Framework's Routing Module>](https://dev.to/aceld/3design-and-implementation-of-the-zinx-frameworks-routing-module-583n)
[<4.Zinx Global Configuration>](https://dev.to/aceld/4zinx-global-configuration-3jam)
[<5.Zinx Message Encapsulation Module Design and Implementation>](https://dev.to/aceld/5zinx-message-encapsulation-module-design-and-implementation-3fmc)
[<6.Design and Implementation of Zinx Multi-Router Mode>](https://dev.to/aceld/6design-and-implementation-of-zinx-multi-router-mode-19nn)
[<7. Building Zinx's Read-Write Separation Model>](https://dev.to/aceld/7-building-zinxs-read-write-separation-model-ihj)
[<8.Zinx Message Queue and Task Worker Pool Design and Implementation>](https://dev.to/aceld/8zinx-message-queue-and-task-worker-pool-design-and-implementation-12gl)
[<9. Zinx Connection Management and Property Setting>](https://dev.to/aceld/9-zinx-connection-management-and-property-setting-28ha)

### [Zinx Application - MMO Game Case Study]

[<10. Application Case Study using the Zinx Framework>](https://dev.to/aceld/10-application-case-study-using-the-zinx-framework-3ich)
[<11. MMO Online Game AOI Algorithm>](https://dev.to/aceld/11-mmo-online-game-aoi-algorithm-l7d)
[<12.Data Transmission Protocol: Protocol Buffers>](https://dev.to/aceld/12data-transmission-protocol-protocol-buffers-1jek)
[<13. MMO Game Server Application Protocol>](https://dev.to/aceld/13-mmo-game-server-application-protocol-5adl)
[<14. Building the Project and User Login>](https://dev.to/aceld/14-building-the-project-and-user-login-b7p)
[<15. World Chat System Implementation>](https://dev.to/aceld/157-world-chat-system-implementation-18li)
[<16. Online Location Information Synchronization>](https://dev.to/aceld/16-online-location-information-synchronization-3cp1)
[<17. Moving position and non-crossing grid AOI broadcasting>](https://dev.to/aceld/17-moving-position-and-non-crossing-grid-aoi-broadcasting-4nb9)
[<18. Player Logout>](https://dev.to/aceld/18-player-logout-3eln)
[<19. Movement and AOI Broadcast Across Grids>](https://dev.to/aceld/19-movement-and-aoi-broadcast-across-grids-46fe)

---
### MMO-GAME Source Code

https://github.com/aceld/zinx/tree/master/zinx_app_demo/mmo_game

---



In the current section of the server-side project, the data protocol used for communication between the client and the server is represented using Protocol Buffers. This section will provide a separate introduction to the installation and basic usage of Protocol Buffers. If readers are already familiar with Protocol Buffers syntax and related protocols, they can skip this section.

## 12.1 Introduction to Protocol Buffers

Google Protocol Buffers, commonly referred to as Protobuf, is a lightweight and efficient structured data storage format developed by Google. It is platform-agnostic, language-agnostic, extensible, and can be used in various domains such as communication protocols and data storage. This makes it suitable for data storage and usage across different applications. It is also well-suited for data exchange between different languages. By implementing the same protocol format, typically defined in a ".proto" file, it can be compiled into different language versions and integrated into respective projects. This enables different languages to parse data serialized using Protobuf. Currently, the official website provides support for languages including C/C++, Python, Java, Go, and more. Google open-sourced Protocol Buffers on July 7, 2008.

## 12.2 Data Exchange Format

`ProtoBuf` is, in essence, a data exchange format, which serves as a layer of data protocol built upon application protocols. This is distinct from the previously discussed TLV protocol in this book. Data exchange formats possess superior readability and data assembly capabilities. However, the trade-off is that their parsing performance tends to be relatively poorer. `ProtoBuf` has been designed with optimizations that specifically target performance aspects.

`ProtoBuf`'s relative advantages include:

(1) Serialization results in smaller data volumes compared to Json and XML, making it suitable for network transmission.
(2) Support for cross-platform and multi-language usage.
(3) Good message format upgrade and compatibility.
(4) Fast serialization and deserialization speed, faster than Json processing speed.

`ProtoBuf`'s relative disadvantages include:

(1) Less widespread adoption compared to XML and JSON.
(2) Binary format leads to poorer readability.
(3) Lack of self-description.

## 12.3 Installing Protobuf Environment

This section primarily covers the installation of the Protobuf environment for the Golang programming language on the Linux (Ubuntu) platform. For other operating systems, you can refer to other resources or official documentation.

### 12.3.1 Installing Protobuf Compilation Tools
Step 1: Download the Protobuf source code:
```bash
Step 1: Download the Protobuf source code:
```

Alternatively, you can directly extract the compressed package:
```bash
unzip protobuf.zip
```
Step 2: Install the required library dependencies:
```bash
sudo apt-get install autoconf automake libtool curl make g++ unzip libffi-dev -y
```
Step 3: Generate the Configure configuration file:
```bash
cd protobuf/
./autogen.sh
```
Step 4: Configure the environment:
```bash
./configure
```
Step 5: Compile the source code (this process may take a while):
```bash
make
```
Step 6: Install the Protobuf-related executables:
```bash
sudo make install
```
Step 7: Refresh the dynamic libraries required by Protobuf. This step is crucial to avoid dynamic library path association failures that might prevent Protobuf-related command programs from starting:
```bash
sudo ldconfig
```
Step 8: Test whether Protobuf is installed successfully by executing the following command:
```bash
protoc -h
```
If it displays the expected information without any errors, the installation is successful.

### 12.3.2 Installing Protobuf Go Language Plugin

Since Protobuf doesn't directly support the Go programming language, developers need to manually install the relevant plugin.

Step 1: Obtain the proto Package (Go Language's proto API Interface)

```bash
go get -v -u github.com/golang/protobuf/proto
go get -v -u github.com/golang/protobuf/protoc-gen-go
```
Step 2: Compile the Code for Golang Support Plugin
```bash
cd $GOPATH/src/github.com/golang/protobuf/protoc-gen-go/
go build
```
Step 3: Place the Generated protoc-gen-go Executable

Place the generated protoc-gen-go executable in the /bin directory (or any other $PATH directory):

```bash
sudo cp protoc-gen-go /bin/
```


## 12.4 Protobuf Syntax
Protobuf commonly refers to user-defined struct types as "messages". The definition of a Protobuf message is typically written in a file with the ".proto" extension.

### 12.4.1 A Simple Example

Below is an example of a Protobuf file. You can name it "file.proto". The content of the file is as follows:

```proto
//file.proto

syntax = "proto3";            // Specify version information, an error will occur if not specified
package pb;                   // Package name for generating Go files later

// 'message' is a keyword, used to define a message type
message Person {
    string name = 1;           // Name
    int32 age = 2;              // Age
    repeated string emails = 3; // Email addresses (repeated indicates the field allows duplicates)
    repeated PhoneNumber phones = 4;   // Phone numbers
}

// 'enum' is a keyword, used to define an enumeration type
enum PhoneType {
    MOBILE = 0;
    HOME = 1;
    WORK = 2;
}

// 'message' is a keyword, used to define a message type that can be nested within another message type
message PhoneNumber {
    string number = 1;
    PhoneType type = 2;
}

```

A Proto file must have several essential elements:

At the top of the file, there needs to be a syntax version information. In the current project example, the Proto version used is "proto3".
A package declaration, which specifies the package name for generating Go files. This is typically relevant in scenarios using Golang applications.

### 12.4.2 Message Format Explanation

A message consists of fields, and the format of each field in a message is as follows:
(Field modifier +) Data type + Field name + Unique numeric tag

The unique numeric tags, like 1 and 2 in PhoneNumber:

```proto
message PhoneNumber {
    string number = 1;
    PhoneType type = 2;
}
```

represent a unique identifier for each field within the message. These numeric tags are used to identify fields in the binary format of the message and cannot be repeated within the same message. Once a message is defined, these tags cannot be changed.

### 12.4.3 Data Types

The data types supported by the Protobuf protocol and their corresponding types in Golang are explained in Table 21-1 below:

Table 12-1: Protobuf Data Types

| .proto Type | Go Type  | Notes                                                           |
|-------------|----------|-----------------------------------------------------------------|
| double      | float64  | 64-bit floating-point number                                   |
| float       | float32  | 32-bit floating-point number                                   |
| int32       | int32    | Variable-length encoding; inefficient for negative values; use sint64 for negative values if possible |
| uint32      | uint32   | Variable-length encoding                                       |
| uint64      | uint64   | Variable-length encoding                                       |
| sint32      | int32    | Variable-length encoding; much more efficient than int32 for negative values |
| sint64      | int64    | Variable-length encoding; signed integer value; more efficient encoding than standard int64 |
| fixed32     | uint32   | 4 bytes; more efficient than uint32 if the value is always greater than 228 |
| fixed64     | uint64   | 8 bytes; more efficient than uint64 if the value is always greater than 256 |
| sfixed32    | int32    | 4 bytes                                                         |
| sfixed64    | int64    | 8 bytes                                                         |
| bool        | bool     | 1 byte                                                          |
| string      | string   | The string must be UTF-8 or 7-bit ASCII encoded text          |
| bytes       | []byte   | May contain arbitrary sequences of bytes                     |


In the above table, if you need to define a data type in Golang, you can refer to the corresponding ".proto Type" defined in the Proto protocol file.

### 12.4.4 Default Values

When a message is being parsed and the encoded information does not include a specific element, the corresponding field in the parsed object is set to a default value. Different data types have default values as specified in Table 12-2:

Table 12-2: Protobuf Default Values

| Data Type Range | Default Value                        |
|-----------------|--------------------------------------|
| String types    | Default is an empty string           |
| Binary types    | Default is an empty byte array       |
| Boolean types   | Default is false                     |
| Numeric types   | Default is 0                         |

## 12.5 Compiling Protobuf
You can compile ".proto" files into code using the protocol compiler as follows:

```bash
$ protoc --proto_path=IMPORT_PATH --go_out=DST_DIR path/to/file.proto
```

In the above command, the "protoc" instruction is the compiler of the previously installed Protobuf protocol. The two parameters are defined as follows:

(1) `--proto_path` specifies the path for importing packages from ".proto" files. Multiple paths can be specified. If ignored, the current directory is assumed by default.
(2) `--go_out` specifies the folder where the generated Go language code files will be placed.

Developers can also compile multiple .proto files simultaneously using the following approach:

```bash
protoc --go_out=./ *.proto
```
During compilation, the Protobuf compiler will generate ".pd.go" files from ".proto" files. These files should not be modified by developers. If changes are needed, the ".proto" file should be edited and then recompiled. The ".pd.go" files can be directly imported and referenced in program code. They include predefined data structures and related methods.

## 12.6 Golang Programming with Protobuf Protocol

In the previous section, the Protobuf file defined the package as "pb," and the generated pb files were placed in the "protocolbuffer_excise" directory under the current path. Now, you can write Golang code to utilize the defined data protocol. The code is as shown below:

```go
package main

import (
    "fmt"
    "github.com/golang/protobuf/proto"
    "protocolbuffer_excise/pb"
)

func main() {
    person := &pb.Person{
        Name:   "Aceld",
        Age:    16,
        Emails: []string{"https://github.com/aceld", "https://yuque.com/aceld"},
        Phones: []*pb.PhoneNumber{
            &pb.PhoneNumber{
                Number: "13113111311",
                Type:   pb.PhoneType_MOBILE,
            },
            &pb.PhoneNumber{
                Number: "14141444144",
                Type:   pb.PhoneType_HOME,
            },
            &pb.PhoneNumber{
                Number: "19191919191",
                Type:   pb.PhoneType_WORK,
            },
        },
    }

    data, err := proto.Marshal(person)
    if err != nil {
        fmt.Println("marshal err:", err)
    }

    newdata := &pb.Person{}
    err = proto.Unmarshal(data, newdata)
    if err != nil {
        fmt.Println("unmarshal err:", err)
    }

    fmt.Println(newdata)
}
```
The above code can now use the defined "Person" data protocol. The `Marshal()` method encodes the in-memory "Person" struct into serialized Proto data. The `Unmarshal()` method parses the serialized binary data into a "Person" struct for use in business code.

---
### MMO-GAME Source Code
https://github.com/aceld/zinx/tree/master/zinx_app_demo/mmo_game

---
### [Zinx]

[<1.Building Basic Services with Zinx Framework>](https://dev.to/aceld/1building-basic-services-with-zinx-framework-296e)
[<2. Zinx-V0.2 Simple Connection Encapsulation and Binding with Business>](https://dev.to/aceld/2-zinx-v02-simple-connection-encapsulation-and-binding-with-business-4p9g)
[<3.Design and Implementation of the Zinx Framework's Routing Module>](https://dev.to/aceld/3design-and-implementation-of-the-zinx-frameworks-routing-module-583n)
[<4.Zinx Global Configuration>](https://dev.to/aceld/4zinx-global-configuration-3jam)
[<5.Zinx Message Encapsulation Module Design and Implementation>](https://dev.to/aceld/5zinx-message-encapsulation-module-design-and-implementation-3fmc)
[<6.Design and Implementation of Zinx Multi-Router Mode>](https://dev.to/aceld/6design-and-implementation-of-zinx-multi-router-mode-19nn)
[<7. Building Zinx's Read-Write Separation Model>](https://dev.to/aceld/7-building-zinxs-read-write-separation-model-ihj)
[<8.Zinx Message Queue and Task Worker Pool Design and Implementation>](https://dev.to/aceld/8zinx-message-queue-and-task-worker-pool-design-and-implementation-12gl)
[<9. Zinx Connection Management and Property Setting>](https://dev.to/aceld/9-zinx-connection-management-and-property-setting-28ha)

### [Zinx Application - MMO Game Case Study]

[<10. Application Case Study using the Zinx Framework>](https://dev.to/aceld/10-application-case-study-using-the-zinx-framework-3ich)
[<11. MMO Online Game AOI Algorithm>](https://dev.to/aceld/11-mmo-online-game-aoi-algorithm-l7d)
[<12.Data Transmission Protocol: Protocol Buffers>](https://dev.to/aceld/12data-transmission-protocol-protocol-buffers-1jek)
[<13. MMO Game Server Application Protocol>](https://dev.to/aceld/13-mmo-game-server-application-protocol-5adl)
[<14. Building the Project and User Login>](https://dev.to/aceld/14-building-the-project-and-user-login-b7p)
[<15. World Chat System Implementation>](https://dev.to/aceld/157-world-chat-system-implementation-18li)
[<16. Online Location Information Synchronization>](https://dev.to/aceld/16-online-location-information-synchronization-3cp1)
[<17. Moving position and non-crossing grid AOI broadcasting>](https://dev.to/aceld/17-moving-position-and-non-crossing-grid-aoi-broadcasting-4nb9)
[<18. Player Logout>](https://dev.to/aceld/18-player-logout-3eln)
[<19. Movement and AOI Broadcast Across Grids>](https://dev.to/aceld/19-movement-and-aoi-broadcast-across-grids-46fe)

---
Author:
discord: https://discord.gg/xQ8Xxfyfcz
zinx:   https://github.com/aceld/zinx
github: https://github.com/aceld
aceld's home: https://yuque.com/aceld