High Performance Web Framework Tasting-Database Operations
Tags: webdev, beginners, tutorial, go
Published: 2022-11-13 14:25:20

## Introduction

In the [previous post](https://dev.to/justlorain/high-performance-go-http-framework-tasting-25li), we gave a brief introduction to the high-performance Go HTTP framework **Hertz** and completed [a simple demo](https://github.com/justlorain/userdemo) using Hertz to get you started.

In this post, you'll learn more about using the Hertz framework with [an official demo](https://github.com/cloudwego/hertz-examples/tree/main/bizdemo/hertz_gorm).

And we'll highlight the following features:

- Use `thrift` IDL to define `HTTP` interface

- Use `hz` to generate code

- Use `Hertz` binding and validate

- Use `GORM` and `MySQL`

## Installation

Run the following command to get the official demo:

```Shell
git clone https://github.com/cloudwego/hertz-examples.git
cd bizdemo/hertz_gorm
```

## Project Structure

```Shell
hertz_gorm
├── biz
|   ├── dal             // Logic code that interacts with the database
│   ├── handler         // Main logical code that handles HTTP requests                  
│   ├── hertz_gen       // Scaffolding generated by hertz from idl files
|   ├── model           // Go struct corresponding to the database table
|   ├── pack            // Transformation between database model and response model
|   ├── router          // Middleware and mapping of routes to handlers
├── go.mod              // go.mod                 
├── idl                 // thift idl                  
├── main.go             // Initialize and start the server              
├── router.go           // Sample route registration
├── router_gen.go       // Route registration    
├── docker-compose.yml  // docker-compose.yml
├── Makefile            // Makefile
```

This is the basic architecture for the project. It's pretty clean and simple, and `hz` generated a lot of scaffolding code for us as well.

## Define IDL

`hz` is a tool provided by the Hertz framework for generating code. Currently, hz can generate scaffolding for Hertz projects based on thrift and protobuf IDL.

The definition of an excellent IDL file plays an important role in developing with Hertz. We will use the thrift IDL for this project as an example.

We can use api annotations to let `hz` help us with parameter binding and validation, route registration code generation, etc.

`hz` will generate the go tag based on the following api annotations so that Hertz can retrieve these values using reflection and parse them.

### Field Annotation

The [go-tagexpr](https://github.com/bytedance/go-tagexpr) open source library is used for parameter binding and validation of the Field annotation, as shown in the following example for `CreateUserRequest`:

```Thrift
// api.thrift
struct CreateUserRequest{
    1: string name      (api.body="name", api.form="name",api.vd="(len($) > 0 && len($) < 100)")
    2: Gender gender    (api.body="gender", api.form="gender",api.vd="($ == 1||$ == 2)")
    3: i64    age       (api.body="age", api.form="age",api.vd="$>0")
    4: string introduce (api.body="introduce", api.form="introduce",api.vd="(len($) > 0 && len($) < 1000)")
}
```

The `form` annotation allows `hz` to automatically bind the parameters in the form of an HTTP request body for us, saving us the trouble of manually binding them using methods such as `PostForm`. 

The `vd` annotation allows for parameter validation. For example, `CreateUserRequest` uses the `vd` annotation to ensure that the `gender` field is only 1 or 2.

You may refer to [here](https://github.com/bytedance/go-tagexpr/tree/master/validator) for more information about parameter validation syntax.

### Method Annotation

The Method annotation can be used to generate route registration code.

Consider the following UserService:

```Thrift
// api.thrift
service UserService {
   UpdateUserResponse UpdateUser(1:UpdateUserRequest req)(api.post="/v1/user/update/:user_id")
   DeleteUserResponse DeleteUser(1:DeleteUserRequest req)(api.post="/v1/user/delete/:user_id")
   QueryUserResponse  QueryUser(1: QueryUserRequest req)(api.post="/v1/user/query/")
   CreateUserResponse CreateUser(1:CreateUserRequest req)(api.post="/v1/user/create/")
}
```

We defined POST methods and routes using `post` annotations, and `hz` will generate handler methods for each route as well as route grouping, middleware embedding scaffolding, etc. As shown in `biz/router/user_gorm/api.go` and `biz/handler/user_gorm/user_service.go`.

And we can also define the business error code in the idl file:

```Thrift
// api.thrift
enum Code {
     Success         = 1
     ParamInvalid    = 2
     DBErr           = 3
}
```

`hz` will generate constants and related methods for us based on these.

```Go
// biz/hertz_gen/user_gorm/api.go
type Code int64

const (
    Code_Success      Code = 1
    Code_ParamInvalid Code = 2
    Code_DBErr        Code = 3
)
```

### Generate Code with hz

After we finish writing IDL, we can generate the scaffolding code for us by using `hz`.

Execute the following command to generate code:

```Shell
hz new --model_dir biz/hertz_gen -mod github.com/cloudwego/hertz-examples/bizdemo/hertz_gorm -idl idl/api.thrift
```

Execute the following command to update the code if you edit the IDL after the first generated:

```Shell
hz update --model_dir biz/hertz_gen -idl idl/api.thrift
```

Of course, the project has already generated the code for you, so you don't need to execute it. When you actually use Hertz for web development yourself, I'm sure you'll find it a very efficient and fun tool.

## Use Middleware

In this project, we configured the root route group to use the gzip middleware for all routes to improve performance.

```Go
// biz/router/user_gorm/middleware.go
func rootMw() []app.HandlerFunc {
    // your code...
    // use gzip middleware
    return []app.HandlerFunc{gzip.Gzip(gzip.DefaultCompression)}
}
```

Just add one line of code to the generated scaffolding code, very easy. You can also refer to the [hertz-contrib/gzip](https://github.com/hertz-contrib/gzip) for more custom configuration.

## Manipulating database with GORM

### Configure GORM

To use `GORM` with a database, you first need to connect to the database using a driver and configure `GORM`, as shown in `biz/dal/mysql/init.go`.

```Go
// biz/dal/mysql/user.go
package mysql

import (
   "gorm.io/driver/mysql"
   "gorm.io/gorm"
   "gorm.io/gorm/logger"
)

var dsn = "gorm:gorm@tcp(localhost:9910)/gorm?charset=utf8&parseTime=True&loc=Local"

var DB *gorm.DB

func Init() {
   var err error
   DB, err = gorm.Open(mysql.Open(dsn), &gorm.Config{
      SkipDefaultTransaction: true,
      PrepareStmt:            true,
      Logger:                 logger.Default.LogMode(logger.Info),
   })
   if err != nil {
      panic(err)
   }
}
```

Here we connect with MySQL database by means of DSN and maintain a global database operation object `DB`.

In terms of GORM configuration, since this project does not involve the operation of multiple tables at the same time, we can configure `SkipDefaultTransaction` to `true` to skip the default transaction, and enable caching through `PrepareStmt` to improve efficiency.

We also use the default logger so that we can clearly see the `SQL` generated for us by GORM.

### Manipulating MySQL

`GORM` concatenates `SQL` statements to perform CRUD, so the code is very concise and easy to read, where all the database operations are in `biz/dal/mysql/user.go`.

We also declare a model corresponding to the database table, the `gorm.Model` contains some common fields, which `GORM` can automatically fill in for us, and support operations such as soft deletion.

```Go
// biz/model/user.go
type User struct {
   gorm.Model
   Name      string `json:"name" column:"name"`
   Gender    int64  `json:"gender" column:"gender"`
   Age       int64  `json:"age" column:"age"`
   Introduce string `json:"introduce" column:"introduce"`
}
```

## Handle HTTP Request

In this section, we'll explore the handler (`biz/handler/user_gorm/user_service.go`), which is the main business logic code.

### CreateUser & DeleteUser & UpdateUser

#### CreateUser

Since we are using api annotations in the thift IDL, `BindAndValidate` will do the parameter binding and validation for us . Very conveniently, all valid parameters will be injected into `CreateUserRequest`.

If there is an error, we can use the `JSON` method to return the data in JSON format . Whether it is `CreateUserResponse` or the business code, we can directly use the code generated by `hz`.

After that, we can insert a new user into MySQL by calling the `CreateUser` in the `dal` layer, passing in the encapsulated arguments.

If there is an error, we return JSON with the error code and information, just like we did in the beginning. Otherwise, the correct service code is returned to represent the successful creation of the user.

```Go
// biz/handler/user_gorm/user_service.go
// CreateUser .
// @router /v1/user/create/ [POST]
func CreateUser(ctx context.Context, c *app.RequestContext) {
   var err error
   var req user_gorm.CreateUserRequest
   err = c.BindAndValidate(&req)
   if err != nil {
      c.JSON(200, &user_gorm.CreateUserResponse{Code: user_gorm.Code_ParamInvalid, Msg: err.Error()})
      return
   }
   if err = mysql.CreateUser([]*model.User{
      {
         Name:      req.Name,
         Gender:    int64(req.Gender),
         Age:       req.Age,
         Introduce: req.Introduce,
      },
   }); err != nil {
      c.JSON(200, &user_gorm.CreateUserResponse{Code: user_gorm.Code_DBErr, Msg: err.Error()})
      return
   }

   resp := new(user_gorm.CreateUserResponse)
   resp.Code = user_gorm.Code_Success
   c.JSON(200, resp)
}
```

#### DeleteUser

The logic for `DeleteUser` and `CreateUser` is almost identical: Bind and validate the arguments, use `mysql.DeleteUser` to delete the user, and return if there is an error, otherwise, return success.

```Go
// biz/handler/user_gorm/user_service.go
// DeleteUser .
// @router /v1/user/delete/:user_id [POST]
func DeleteUser(ctx context.Context, c *app.RequestContext) {
   var err error
   var req user_gorm.DeleteUserRequest
   err = c.BindAndValidate(&req)
   if err != nil {
      c.JSON(200, &user_gorm.DeleteUserResponse{Code: user_gorm.Code_ParamInvalid, Msg: err.Error()})
      return
   }
   if err = mysql.DeleteUser(req.UserID); err != nil {
      c.JSON(200, &user_gorm.DeleteUserResponse{Code: user_gorm.Code_DBErr, Msg: err.Error()})
      return
   }

   c.JSON(200, &user_gorm.DeleteUserResponse{Code: user_gorm.Code_Success})
}
```

#### UpdateUser

`UpdateUser` is much the same, with the notable model transformation from an object that receives HTTP request parameters to a data access object that corresponds to a database table.

```Go
// biz/handler/user_gorm/user_service.go
// UpdateUser .
// @router /v1/user/update/:user_id [POST]
func UpdateUser(ctx context.Context, c *app.RequestContext) {
    var err error
    var req user_gorm.UpdateUserRequest
    err = c.BindAndValidate(&req)
    if err != nil {
            c.JSON(200, &user_gorm.UpdateUserResponse{Code: user_gorm.Code_ParamInvalid, Msg: err.Error()})
            return
    }

    u := &model.User{}
    u.ID = uint(req.UserID)
    u.Name = req.Name
    u.Gender = int64(req.Gender)
    u.Age = req.Age
    u.Introduce = req.Introduce

    if err = mysql.UpdateUser(u); err != nil {
            c.JSON(200, &user_gorm.UpdateUserResponse{Code: user_gorm.Code_DBErr, Msg: err.Error()})
            return
    }

    c.JSON(200, &user_gorm.UpdateUserResponse{Code: user_gorm.Code_Success})
}
```

### QueryUser

What's worth noting in `QueryUser` is that we're doing paging and a transformation from `model.User` to `user_gorm.User`, which is the reverse of the operation we just mentioned in UpdateUser.

With a simple paging formula `startIndex = (currentPage - 1) * pageSize`, we're paging the data as we're querying it.

And this time we've wrapped our transformation model in `biz/pack/user.go`.

```Go
// biz/pack/user.go
// Users Convert model.User list to user_gorm.User list
func Users(models []*model.User) []*user_gorm.User {
    users := make([]*user_gorm.User, 0, len(models))
    for _, m := range models {
            if u := User(m); u != nil {
                    users = append(users, u)
            }
    }
    return users
}

// User Convert model.User to user_gorm.User
func User(model *model.User) *user_gorm.User {
    if model == nil {
            return nil
    }
    return &user_gorm.User{
            UserID:    int64(model.ID),
            Name:      model.Name,
            Gender:    user_gorm.Gender(model.Gender),
            Age:       model.Age,
            Introduce: model.Introduce,
    }
}
// biz/handler/user_gorm/user_service.go
// QueryUser .
// @router /v1/user/query/ [POST]
func QueryUser(ctx context.Context, c *app.RequestContext) {
    var err error
    var req user_gorm.QueryUserRequest
    err = c.BindAndValidate(&req)
    if err != nil {
            c.JSON(200, &user_gorm.QueryUserResponse{Code: user_gorm.Code_ParamInvalid, Msg: err.Error()})
            return
    }

    users, total, err := mysql.QueryUser(req.Keyword, req.Page, req.PageSize)
    if err != nil {
            c.JSON(200, &user_gorm.QueryUserResponse{Code: user_gorm.Code_DBErr, Msg: err.Error()})
            return
    }
    c.JSON(200, &user_gorm.QueryUserResponse{Code: user_gorm.Code_Success, Users: pack.Users(users), Totoal: total})
}
```

The rest of the business logic is the same as before, and we're done with all the handler functions.

## Run Demo

- Run mysql docker

```Shell
cd bizdemo/hertz_gorm && docker-compose up
```

- Generate MySQL table

Connect MySQL and execute user.sql

- Run demo

```Shell
cd bizdemo/hertz_gorm
go build -o hertz_gorm && ./hertz_gorm
```

## Summary

That's it for this post. Hopefully it will give you a quick overview of how to develop with `Hertz` and `GORM`. Both of them are well documented . Feel free to check out the official documentation for more information.

## Reference List

- https://www.cloudwego.io/docs/hertz/

- https://gorm.io/

- https://github.com/bytedance/go-tagexpr

- https://github.com/hertz-contrib/gzip