How to securely store passwords?
Tags: go, tutorial, webdev, security
Published: 2021-01-16 22:11:59

Hello everyone, welcome back to the backend master class!

In this lecture, we’re gonna learn how to securely store users’ password in the database.

{% youtube B3xnJI2lHmc %}

Here's:

- Link to the [full series playlist on Youtube](https://bit.ly/backendmaster)
- And its [Github repository](https://github.com/techschool/simplebank)

## How to store password

As you already know, we should never ever store naked passwords! So the idea is to hash it first, and only store that hash value.

Basically, the password will be hashed using `brypt` hashing function to produce a hash value.

Besides the input password, `bcrypt` requires a `cost` parameter, which will decide the number of key expansion rounds or iterations of the algorithm.

`Bcrypt` also generates a random `salt` to be used in those iterations, which will help protect against the [rainbow table attack](https://en.wikipedia.org/wiki/Rainbow_table). Because of this random `salt`, the algorithm will give you a completely different output hash value even if the same input password is provided.

The `cost` and `salt` will also be added to the hash to produce the final hash string, which looks something like this:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/bv47cvavn5egi1fmjp72.png)

In this hash string, there are 4 components:

- The first part is the `hash algorithm identifier`. `2A` is the identifier of the `bcrypt` algorithm.
- The second part is the `cost`. In this case, the cost is `10`, which means there will be `2^10 = 1024` rounds of key expansion.
- The third part is the `salt` of length `16 bytes`, or `128 bits`. It is encoded using `base64` format, which will generate a string of `22` characters.
- Finally, the last part is the `24 bytes` hash value, encoded as `31` characters.

All of these 4 parts are concatenated together into a single hash string, and it is the string that we will store in the database.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/0q79j58zhd2totxlgabn.png)

So that’s the process of hashing users’ password!

But when users login, how can we verify that the password that they entered is correct or not?

Well, first we have to find the `hashed_password` stored in the DB by `username`.

Then we use the `cost` and `salt` of that `hashed_password` as the arguments to hash the `naked_password` users just entered with `bcrypt`. The output of this will be another hash value.

Then all we have to do is to compare the 2 hash values. If they’re the same, then the password is correct.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/x9wk0fa5xai45ucuputp.png)

Alright, now let’s see how to implement these logics in `Golang`.

## Implement functions to hash and compare passwords

In the [previous lecture](https://dev.to/techschoolguru/how-to-handle-db-errors-in-golang-correctly-11ek), we have generated the code to create a new user in the database. And `hashed_password` is one of the input parameters of the `CreateUser()` function.

```go
type CreateUserParams struct {
    Username       string `json:"username"`
    HashedPassword string `json:"hashed_password"`
    FullName       string `json:"full_name"`
    Email          string `json:"email"`
}

func (q *Queries) CreateUser(ctx context.Context, arg CreateUserParams) (User, error) {
    row := q.db.QueryRowContext(ctx, createUser,
        arg.Username,
        arg.HashedPassword,
        arg.FullName,
        arg.Email,
    )
    var i User
    err := row.Scan(
        &i.Username,
        &i.HashedPassword,
        &i.FullName,
        &i.Email,
        &i.PasswordChangedAt,
        &i.CreatedAt,
    )
    return i, err
}
```

Also, in this `createRandomUser()` function of the unit test in `db/sqlc/user_test.go`, we’re using a simple `"secret"` string for the `hash_password` field, which doesn’t reflect the real correct values this field should hold.

```go
func createRandomUser(t *testing.T) User {
    arg := CreateUserParams{
        Username:       util.RandomOwner(),
        HashedPassword: "secret",
        FullName:       util.RandomOwner(),
        Email:          util.RandomEmail(),
    }

    ...
}
```

So today we’re gonna update it to use a real hash string.

### Hash password function

First, let’s create a new file `password.go` inside the `util` package. In this file, I’m gonna define a new function: `HashPassword()`.

It will take a `password` string as input, and will return a `string` or an `error`. This function will compute the `bcrypt` hash string of the input `password`.

```go
// HashPassword returns the bcrypt hash of the password
func HashPassword(password string) (string, error) {
    hashedPassword, err := bcrypt.GenerateFromPassword([]byte(password), bcrypt.DefaultCost)
    if err != nil {
        return "", fmt.Errorf("failed to hash password: %w", err)
    }
    return string(hashedPassword), nil
}
```

In this function, we call `bcrypt.GenerateFromPassword()`. It requires 2 input parameters: the `password` of type `[]byte` slice, and a `cost` of type `int`.

So we have to convert the input `password` from `string` to `[]byte` slice. 

For `cost`, I use the `bcrypt.DefaultCost` value, which is `10`.

The output of this function will be the `hashedPassword` and an `error`. If the `error` is `not nil`,  then we just return an empty hashed string, and wrap the `error` with a message saying: `"failed to hash password"`.

Otherwise, we convert the `hashedPassword` from `[]byte` slice to `string`, and return it with a `nil` error.

### Compare passwords function

Next, we will write another function to check if a password is correct or not: `CheckPassword()`.

This function will take 2 input arguments: a `password` to check, and the `hashedPassword` to compare. It will return an `error` as output.

Basically, this function will check if the input `password` is correct when comparing to the provided `hashedPassword` or not.

As the standard `bcrypt` package has already implemented this feature, all we have to do is to call `bcrypt.CompareHashAndPassword()` function, and pass in the `hashedPassword` and naked `password`, after converting them from `string` to `[]byte` slices.

```go
// CheckPassword checks if the provided password is correct or not
func CheckPassword(password string, hashedPassword string) error {
    return bcrypt.CompareHashAndPassword([]byte(hashedPassword), []byte(password))
}
```

And that’s it. We’re done!

## Write unit test for HashPassword and CheckPassword functions

Now let’s write some unit tests to make sure these 2 functions work as expected.

I’m gonna create a new file `password_test.go` inside the `util` package. Then let’s define function `TestPassword()` with a `testing.T` object as input.

First I will generate a new `password` as a random string of `6` characters. Then we get the `hashedPassword` by calling `HashPassword()` function with the generated password.

We require no errors to be returned, and the `hashedPassword` string should be not empty.

```go
func TestPassword(t *testing.T) {
    password := RandomString(6)

    hashedPassword, err := HashPassword(password)
    require.NoError(t, err)
    require.NotEmpty(t, hashedPassword)

    err = CheckPassword(password, hashedPassword1)
    require.NoError(t, err)
}
```

Next we call `CheckPassword()` function with the `password` and `hashedPassword` parameters.

As this is the same `password` we used to create the `hashedPassword`, this function should return no errors, which means correct password.

Let’s also test the case where an `incorrect password` is provided!

I will generate a new random `wrongPassword` string, and call `CheckPassword()` again with this `wrongPassword` argument. This time, we expect an `error` to be returned, since the provided password is incorrect.

```go
func TestPassword(t *testing.T) {
    password := RandomString(6)

    hashedPassword, err := HashPassword(password)
    require.NoError(t, err)
    require.NotEmpty(t, hashedPassword)

    wrongPassword := RandomString(6)
    err = CheckPassword(wrongPassword, hashedPassword)
    require.EqualError(t, err, bcrypt.ErrMismatchedHashAndPassword.Error())
}
```

To be exact, we use `require.EqualError()` to compare the output error. It must be equal to the `bcrypt.ErrMismatchedHashAndPassword` error.

OK, the test is now completed. Let’s run it!

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/90uzylad79bl73vlglxb.jpg)

It passed! Awesome!

## Update the existing code to use HashPassword function

So the `HashPassword()` function is working properly. Let’s go back to the `user_test.go` file and use it in the `createRandomUser()` function.

Here I’m gonna create a new `hashedPassword` value by calling `util.HashPassword()` function with a random string of `6` characters.

We require no errors, then change the `"secret"` constant to `hashedPassword` instead:

```go
func createRandomUser(t *testing.T) User {
    hashedPassword, err := util.HashPassword(util.RandomString(6))
    require.NoError(t, err)

    arg := CreateUserParams{
        Username:       util.RandomOwner(),
        HashedPassword: hashedPassword,
        FullName:       util.RandomOwner(),
        Email:          util.RandomEmail(),
    }

    ...
}
```

Alright, let’s run the whole `db` package test!

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/n7dqkprr39m8iz68yr14.jpg)

All passed!

Now if we open the database in Table Plus and check the users table, we can see that the `hashed_password` column is now containing the correct `bcrypt` hashed string.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/ogkirl4c5og31kp2j3rg.jpg)

It looks just like the example that I shown you in the beginning of this video.

## Make sure all hashed passwords are different

One thing we want to make sure of is: if the `same password` is hashed twice, `2 different hash values` should be produced.

So let’s go back to the `TestPassword()` function. I’m gonna change the `hashPassword` variable’s name to `hashedPassword1`.

Then let’s duplicate the hash password code block, and change the variable’s name to `hashedPassword2`.

```go
func TestPassword(t *testing.T) {
    password := RandomString(6)

    hashedPassword1, err := HashPassword(password)
    require.NoError(t, err)
    require.NotEmpty(t, hashedPassword1)

    err = CheckPassword(password, hashedPassword1)
    require.NoError(t, err)

    wrongPassword := RandomString(6)
    err = CheckPassword(wrongPassword, hashedPassword1)
    require.EqualError(t, err, bcrypt.ErrMismatchedHashAndPassword.Error())

    hashedPassword2, err := HashPassword(password)
    require.NoError(t, err)
    require.NotEmpty(t, hashedPassword2)
    require.NotEqual(t, hashedPassword1, hashedPassword2)
}
```

What we expect to see is: the value of `hashedPassword2` should be different from the value of `hashedPassword1`. So here I use `require.NotEqual()` to check this condition.

OK, let’s rerun the test.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/evnlhng02niihnityz7y.jpg)

It passed! Excellent!

To really understand why it passed, we have to open the implementation of the `bcrypt.GenerateFromPassword()` function.

```go
func GenerateFromPassword(password []byte, cost int) ([]byte, error) {
    p, err := newFromPassword(password, cost)
    if err != nil {
        return nil, err
    }
    return p.Hash(), nil
}

func newFromPassword(password []byte, cost int) (*hashed, error) {
    if cost < MinCost {
        cost = DefaultCost
    }
    p := new(hashed)
    p.major = majorVersion
    p.minor = minorVersion

    err := checkCost(cost)
    if err != nil {
        return nil, err
    }
    p.cost = cost

    unencodedSalt := make([]byte, maxSaltSize)
    _, err = io.ReadFull(rand.Reader, unencodedSalt)
    if err != nil {
        return nil, err
    }

    p.salt = base64Encode(unencodedSalt)
    hash, err := bcrypt(password, p.cost, p.salt)
    if err != nil {
        return nil, err
    }
    p.hash = hash
    return p, err
}
```

As you can see here, in the `newFromPassword()` function, a `random salt` value is generated, and it is used in the `bcrypt()` function to generate the hash.

So now you know, because of this `random salt`, the generated hash value will be different everytime.

## Implement the create user API

Next step, I’m gonna use the `HashPassword()` function that we’ve written to implement the `create user API` for our simple bank.

Let’s create a new file `user.go` inside the `api` package.

This API will be very much alike the `create account API` that we’ve [implemented before](https://dev.to/techschoolguru/implement-restful-http-api-in-go-using-gin-4ap1), so I’m just gonna copy it from the `api/account.go` file.

Then let’s change this `struct` to `createUserRequest`.

The first parameter is `username`. It is a `required` field.

And let’s say we don’t allow it to contain any kind of special characters, so here I’m gonna use the `alphanum` tag, which is already provided by the [validator package](https://github.com/go-playground/validator). It basically means that this field should contain ASCII alphanumeric characters only.

The second field is `password`. It is also `required`. And normally we don’t want the password to be too short because it would be very easy to hack. So here let’s use the `min` tag to say that the length of the password should be at least `6` characters.

```go
type createUserRequest struct {
    Username string `json:"username" binding:"required,alphanum"`
    Password string `json:"password" binding:"required,min=6"`
    FullName string `json:"full_name" binding:"required"`
    Email    string `json:"email" binding:"required,email"`
}
```

The third field is `full_name` of the user. There’s no specific requirements for this field, except that it is `required`.

Then, the last field is `email`, which is very important because it would be the main communication channel between the users and our system. We can use the `email` tag provided by validator package to make sure that the value of this field is a correct email address.

There are many other useful built-in tags that were already implemented by the validator package, you can check them out in its [documentation](https://pkg.go.dev/github.com/go-playground/validator/v10) or [github page](https://github.com/go-playground/validator).

Now let’s go back to the code to complete this `createUser()` function.

```go
func (server *Server) createUser(ctx *gin.Context) {
    var req createUserRequest
    if err := ctx.ShouldBindJSON(&req); err != nil {
        ctx.JSON(http.StatusBadRequest, errorResponse(err))
        return
    }

    hashedPassword, err := util.HashPassword(req.Password)
    if err != nil {
        ctx.JSON(http.StatusInternalServerError, errorResponse(err))
        return
    }

    arg := db.CreateUserParams{
        Username:       req.Username,
        HashedPassword: hashedPassword,
        FullName:       req.FullName,
        Email:          req.Email,
    }

    ...   
}
```

Here we use the `ctx.ShouldBindJSON()` function to bind the input parameters from the `context` into the `createUserRequest` object.

If any of the parameters are invalid, we just return `400 Bad Request` status to the client. Otherwise, we will use them build the `db.CreateUserParams` object.

There are 4 fields that need to be set: `Username`, `HashedPassword`, `Fullname`, and `Email`.

So first, we compute the `hashedPassword` by calling `util.HashPassword()` function and pass in the input `request.Password` value.

If this function returns a `not nil` error, then we just return a status `500 Internal Server Error` to the client.

Else, we will build the `CreateUserParams` object, where `Username` is `request.Username`, `HashedPassword` is the computed `hashedPassword`, `FullName` is `request.FullName`, and `Email` is `request.Email`.

```go
func (server *Server) createUser(ctx *gin.Context) {
    ...

    user, err := server.store.CreateUser(ctx, arg)
    if err != nil {
        if pqErr, ok := err.(*pq.Error); ok {
            switch pqErr.Code.Name() {
            case "unique_violation":
                ctx.JSON(http.StatusForbidden, errorResponse(err))
                return
            }
        }
        ctx.JSON(http.StatusInternalServerError, errorResponse(err))
        return
    }

    ctx.JSON(http.StatusOK, user)
}
```

Then we call `server.store.CreateUser()` with this input argument. It will return the created `user` object or an `error`.

Just like in the `create account API`, if error is not nil, then there are some possible scenarios. Keep in mind that, in the users table, we have 2 `unique constraints`:

- One is for the primary key `username`,
- And the other is for the `email` column.

We don’t have a `foreign key` in this table, so here we only need to keep the `unique_violation` code name to return status `403 Forbidden` in case an user with the same `username` or `email` already exists.

Finally, if no errors occur, we just return status `200 OK` with the created `user` to the client.

OK, so now the `createUser` API handler is completed. The last step we must do is to register a route for it in the `api/server.go` file.

Here, in this `NewServer()` function, I’m gonna add a new route with method `POST`. Its path should be `/users`, and its handler function is `server.createUser`

```go
// NewServer creates a new HTTP server and set up routing.
func NewServer(store db.Store) *Server {
    server := &Server{store: store}
    router := gin.Default()

    if v, ok := binding.Validator.Engine().(*validator.Validate); ok {
        v.RegisterValidation("currency", validCurrency)
    }

    router.POST("/users", server.createUser)

    router.POST("/accounts", server.createAccount)
    router.GET("/accounts/:id", server.getAccount)
    router.GET("/accounts", server.listAccounts)

    router.POST("/transfers", server.createTransfer)

    server.router = router
    return server
}
```

And that’s it! We’re done!

## Test the create user API

Let’s open the terminal and run `make server` to start the server.

I’m gonna use [Postman](https://www.postman.com/) to test the new API.

Let’s select method `POST` and fill in the URL: <http://localhost:8080/users>

For the request body, let’s choose `raw`, and select `JSON` format. I'm gonna use this JSON data:

```json
{
    "username": "quang1",
    "full_name": "Quang Pham",
    "email": "quang@email.com",
    "password": "secret"
}
```

OK, let’s send this request!

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/rccwgqu8l97no4h434ce.jpg)

It’s successful! We’ve got the created `user` object here with all correct field values.

Let’s open the database to find this user.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/la3mih4va95g4yhuh22d.jpg)

Here it is! So our API is working well in the happy case.

Now let’s see what happens if I send this same request the second time.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/r6fj79ni1m81b7byp57r.jpg)

We’ve got a `403 Forbidden` status code. And the reason is that the `unique constraint` for `username` is violated.

We’re trying to create another user with the same `username`, So clearly it should not be allowed!

Now let’s try changing the `username` to `quang2`, but keep the `email` value the same, and send the request again.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/py1g4yoemdjjp5yhv8hd.jpg)

We still got `403 Forbidden`. But this time, the error is because the `email` unique constraint is violated. Exactly what we expected!

If I change the `email` to `quang2@email.com`, then the request will be successful, since this email doesn’t belong to any other users.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/gfa81bkj7n61nvg0apkt.jpg)

OK, now let’s try an invalid `username`, such as `quang#2`:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/4vbdkpyqzgv5gehy7opb.jpg)

This time, the status code is `400 Bad Request`. And the reason is: the field validation for `username` failed on the `alphanum` tag. There’s a special character `#` in the `username`, which is not alphanumeric.

Next, let’s try an invalid email. I’m gonna change the `username` to `quang3`, and `email` to `quang3email.com`, without the `@` character.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/143icuv62eb39ofmpj8n.jpg)

We’ve got `400 Bad Request` status again. And the error is: field validation for `email` failed on the `email` tag, which is exactly what we want.

OK now let’s fix the `email` address, and change the `password` to a very short value, such as `"123"`. Then send the request one more time.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/ktclklly7jy3tnml757c.jpg)

This time, we’ve got an error because the `password` field validation failed on the `min` tag. It doesn’t satisfy the minimum length constraint of `6` characters.

## API should not expose hashed password

Before we finish, there’s one more thing I want to tell you. Let’s fix the `password` value and send the request again.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/bebtbkl23o4fk7c88lel.jpg)

Now it’s successful. But you can notice that the `hashed_password` value is also returned, which doesn’t seem right, because the client will never need to use this value for anything.

And it might raise some security concerns, as this piece of sensitive information is being transmitted in the public.

It would be better to remove this field from the response body.

To do that, I’m gonna declare a new `createUserResponse struct` in the `api/user.go` file. It will contain almost all fields of the `db.User` struct, except for the `HashedPassword` field that should be removed.

```go
type createUserResponse struct {
    Username          string    `json:"username"`
    FullName          string    `json:"full_name"`
    Email             string    `json:"email"`
    PasswordChangedAt time.Time `json:"password_changed_at"`
    CreatedAt         time.Time `json:"created_at"`
}
```

Then here, at the end of the `createUser()` handler function, we build a new `createUserResponse` object, where `Username` is `user.Username`, `FullName` is `user.FullName`, `Email` is `user.Email`, `PasswordChangedAt` is `user.PasswordChangedAt`, and `CreatedAt` is `user.CreatedAt`.

```go
func (server *Server) createUser(ctx *gin.Context) {
    ...

    user, err := server.store.CreateUser(ctx, arg)
    if err != nil {
        if pqErr, ok := err.(*pq.Error); ok {
            switch pqErr.Code.Name() {
            case "unique_violation":
                ctx.JSON(http.StatusForbidden, errorResponse(err))
                return
            }
        }
        ctx.JSON(http.StatusInternalServerError, errorResponse(err))
        return
    }

    rsp := createUserResponse{
        Username:          user.Username,
        FullName:          user.FullName,
        Email:             user.Email,
        PasswordChangedAt: user.PasswordChangedAt,
        CreatedAt:         user.CreatedAt,
    }
    ctx.JSON(http.StatusOK, rsp)
}
```

Finally, we return the `response` object instead of `user`. And we’re done!

Let’s restart the server. Then go back to Postman, update the `username` and `email` to new values, and send the request.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/xdmenmht3ipl9x0f19g2.jpg)

It’s successful. And now there’s no `hashed_password` field in the response body anymore. Perfect!

So that brings us to the end of this lecture. I hope you have learned something useful.

Thank you for reading, and see you in the next one!

---

If you like the article, please subscribe to our [Youtube channel](http://bit.ly/techschoolguru) and follow us on [Twitter](https://twitter.com/techschoolguru) or [Facebook](https://www.facebook.com/techschool.guru) for more tutorials in the future.

---
If you want to join me on my current amazing team at Voodoo, check out [our job openings here](https://bit.ly/voodoojobs). Remote or onsite in Paris/Amsterdam/London/Berlin/Barcelona with visa sponsorship.