Using Golang Concurrency in Production
Tags: go, concurrency, docker, vue
Published: 2020-01-19 23:43:04

---
title: Using Golang Concurrency in Production
published: true
description: How to actually use Golang goroutines and channels in a production application
tags: #go #concurrency #docker #vuejs
cover_image: https://thepracticaldev.s3.amazonaws.com/i/xomfm3cx84cftpps3iq6.jpg
---

In this article, we will see how Asynchronous Programming is achieved in Golang using a real-world application.

**Asynchronous**
Our daily chores are filled with concurrent(asynchronous) activities.
For instance, When making a coffee, you boil water, put the coffee in your mug, add other ingredients you need, then finally add the boiled water into the mug. Your coffee is ready. 

**Synchronous**
From the instance above, doing this synchronously will mean you waiting for a task to be done first before undertaking another. That is, Place the water on the heater, do nothing else until that water boils. Of course, we consider this approach a total waste of our time and somewhat inefficient. 
Hence, it is inefficient to implement a feature that is asynchronous in nature synchronously. 

I wrote a real-world program: **Username Lookup Application**, which demonstrates how asynchronous programming is utilized in making HTTP calls to different endpoints at the same time and retrieving data. This is how it works, you provide a username to look up, then that username is checked across the accounts you specify(such as Twitter, Instagram, Github, and so on). Visit the application [here](https://username-across.herokuapp.com/)
You can also get the code on [Github](https://github.com/victorsteven/Username-Across-Platforms)


#The Application Overview.

**Golang** is used on the backend
**VueJS** is used on the frontend
**Docker** is used to deploying on **Heroku**
**Travis** is used for Continous Integration


#Steps Taken

##Step 1: Basic Setup
Create the root folder for the project titled: **username_across_platforms**
```mkdir username_across_platforms```

Initialize **go modules** with the name of the folder created above:
```go mod init username_across_platforms```

#Step 2: Client Implementation(for HTTP calls)

Create the **server** package: 

```mkdir server```

Since we will be making HTTP requests to the different urls, we will need a client. Note, the client here is not the frontend. It is used for HTTP calls on the server-side. 
Inside the server package(folder), create the client package(directory):
```cd server && mkdir client```

Then create **client.go** inside the **client** package:
```cd client && touch client.go```
{%gist https://gist.github.com/victorsteven/4c424c2f4f9c4756c4530287aa5f6a5d %}

From the above file, you might be wondering why we used an interface and stuff, this will make sense to you when you see the test files. We will need to mock the **GetValue** method. We can't do this except the method is defined in an interface.
Something else I want you to observe is how we implemented the interface.
We defined a **clientCall** struct which the **GetValue** method 'belongs to'. Then the struct now implements the interface in this line:
```	ClientCall HTTPClient = &clientCall{}```
The struct also has the **http.Client. This will help us replace the actual **http.Client** with a fake one, so we don't make a real http call while writing test cases.
Still, in the same package, create the test file: 
```touch client_test.go```
{%gist https://gist.github.com/victorsteven/a1ba68bbd8fd534c1ab737d4b5f9a628 %}
From the above file, we fake the real http call with the help of the **RoundTripFunc**, another option you might consider is using **httptest.Server**. 
You could see how the **Transport** of the **http.Client** is swapped with our RoundTripFunc in the **NewFakeClient** function.


##Step 3: Provider Implementation
Since our client is in place with enough unit tests to back it, let us create the provider that calls the **GetValue** method of the client and pass the response it obtained to a channel.
From the **server** package(directory), create the **provider** package then the provider.go file:
```mkdir provider```
```cd provider && touch provider.go```
{%gist https://gist.github.com/victorsteven/8843d9a764aee721845d1488d1466660 %}

As seen in the file, the **CheckUrl** method is defined in an interface(because we need to mock it in the future while writing unit tests). In the method implementation, we passed the url to look up and the channel to send the response or the error if not available. The main reason we used the channel here is, the **checkUrl** method will be called in a different **goroutines** when will implement the **service**.
In a nutshell, the **checkUrl** method checks a url, for example https://twitter.com/stevensunflash, if the url does not exist, **cant_access_resource** is sent to the channel. If the url exists but the username **stevensunflash** is not found, **no_match** is sent to the channel, if the desired username is found, we send the **url** to the channel.

Now let's test the implementation.
Create the **provider_test.go** file:
```touch provider_test.go```
{%gist https://gist.github.com/victorsteven/d13220ea98ace2cd82bae85884a78884 %}
Observe closely that we mocked the client's **GetValue** method here, this is one of the uses of defining that method in an interface from the client package. You could see how we returned the response we want from the client without hitting a real endpoint. This has also helped us achieve **unit testing** on the provider without calling the real **GetValue** method from the client package. How Sweet!😴


##Step 4: Service Implementation(Launching Some Goroutines🚀)
Now let's fire up some goroutines to get multiple urls responses at the same time.
From the **server** package(directory), create the **service** package(directory), then the **service.go** file:
```mkdir service```

```cd service && touch service.go```

{%gist https://gist.github.com/victorsteven/c08e2bff558cfcae18305b171613c424 %}

The **UsernameCheck** method receive a slice of urls to process, we already have the **checkUrl** method we can use to check a url, defined in the provider's package. Now, we loop through the given urls and launch a **goroutine** for each url. Remember, any response or error obtained is sent to the channel. We then get the value for each url from the channel and put it inside the **links** slice.
The result set can have three cases: 
- cant_access_resource
- no_match
- valid result(url)
We further filtered the links slice to get only valid urls.

Now, let's write some tests to prove that our code is working.
Create the **service_test.go** file:
```touch service_test.go```
{%gist https://gist.github.com/victorsteven/27966fdb0c40a78934e873dad391e22d %}

Observe from the test that we also mocked the **client** so that we don't hit the actual endpoint.


##Step 5: Controller Implementation(Returning the response to the caller)
Now, let's send back an HTTP response to the caller.
From the **server** package(directory), create the **controller** package(directory) and then the **controller.go** file
```mkdir controller```
```cd controller && controller.go```
{%gist https://gist.github.com/victorsteven/3bd8391c38cf83916e1d4c315f7b6131 %}

Nothing fancy, the controller receives the request from the caller pass it down to the service(which concurrently uses the provider's **checkUrls** method), the service passes back to the controller the urls it could process, then the controller sends the urls to the caller.

Let's also test the controller, create the **controller_test.go** file
```touch controller_test.go```
As seen above to achieve unit test, we must mock the service's **UsernameCheck** method, and return anything we like. We could mock the service easily with the help of the **usernameService** interface.

Another thing to observe from the tests is, the **json** passed from the caller is of this format: 
```
`["url1","url2","url3"]`
```
Anything outside a format like this won't work. We have the tests above to prove that.

##Step 6: Wiring up the application
Though we have unit tests to prove that our application is working, lets still test it on the browser.
From the **server** package(directory), create the **app** package(directory), 

```mkdir app```

then create two files: 
-app.go
-route.go


a. app.go

``` cd app && touch app.go```
{%gist https://gist.github.com/victorsteven/5295104e662acd902ebeb43b82aaf43c %}
Since we will later deploy this to [heroku](https://www.heroku.com), we checked for the **Heroku** port.


b. route.go
```touch route.go```
{%gist https://gist.github.com/victorsteven/3eeec5d4487bb4504d12c57df8f7ad31 %}

Observe from the route that we called a middleware we have not defined yet. This middleware will enable us to make API calls between the **server** and the **client**(frontend), which we will define shortly. 


##The Middleware
From the **server** package, create the middleware package(directory), then the **cors.go** file:
```mkdir middleware && touch cors.go```

{%gist https://gist.github.com/victorsteven/dbc0b52bd6924bf9662195508c9bf854 %}

##Running the app
We now need to create the **main.go** file in the **server** directory:
```touch main.go```
{%gist https://gist.github.com/victorsteven/87adb470022930cb880a5347c2264858 %} 

We called the **StartApp** function we defined in the **app** package.

Run the application, from the path: **username_across_platforms/server**
```
go run main.go
```
![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/hrjqsdovqdt2cydq12mr.png)


So, run the application and use **Postman** to test, or simply skip to the next step, where will use a **vuejs** as our frontend.
Remember, if you want to use **Postman** or your favorite testing tool, 
pass the JSON like this: 
![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/d0waiahi0hqc6v8g3u4a.png)


Step 7: The Client(Frontend)
All we have done thus far is **server** stuff. Let's now see a beautiful representation of our hard work💪.
We will be using **vuejs**, you can also use **reactjs** if you wish. After all, this is just a SPA(Single Page App)

The first thing to do is install the [https://cli.vuejs.org/](vue-cli), if you have installed that before.

From the root directory of the project(path: "username_across_platforms/"), create a new **Vue** project called **client**. 
```
vue create client
```
It will prompt you for some stuff, choose all default.
When completed, enter the client(frontend) directory:
```cd client```

💥**Very Important**💥
The application just installed already have **git** initialized, remove the **.git** file. From the terminal, in the path: **username_across_platforms/client**, execute:
```rm -rf .git```

Next step, install **vuetify** we will use for the UI
```vue add vuetify```

Since we will be making an api call, lets install **axios**
```yarn add axios --save```

Great!

Next, locate the **App.vue** file inside the **src** directory and replace the content with: 
{%gist https://gist.github.com/victorsteven/56f4eeebaa916e996774e4b783207220 %}

Observe above that we imported a file we have not defined(env.js). To enable us to test both on local and production, we need to inform our application at any point in time the url to use.
In the same directory path as **App.vue**, create the **env.js** file:
{%gist https://gist.github.com/victorsteven/1c075c62555066c2287b839d3735e595 %}


Now, let's start the Frontend app:
From the path: **username_across_platforms/client**
Run:
```npm run serve```

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/e1qfkqgaag9od4py5g1u.png)

Now fire up your browser and visit: **http://localhost:8080**

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/ysuuz8zzzeq9nda5sgf6.png)

Aww😍. You're welcome!


##Step 8: Hosting
We will deploy this awesome app to [heroku](https://www.heroku.com) for free. We can achieve easily using **Docker**.
From the project root (path: username_across_platforms/), create the **Dockerfile**
{%gist https://gist.github.com/victorsteven/76bc5312b95dbda15006ab220da74393 %}

Since **heroku** is used for deployment, create the **heroku.yml** file, this file tells Heroku that we are dockerizing the app:
From the root directory:
```touch heroku.yml```
{%gist https://gist.github.com/victorsteven/7ccc2a30fd1794087b1bf4ec2be58480 %}



If you have following along, push your code to github, remember to initialize git from the root directory(path: username_across_platforms/).

###Pushing to Heroku.
From the root directory
- Install the **heroku-cli**
- Login to heroku: 
```heroku login```
- Create the heroku application:
```heroku create```
- Tell heroku that we are deploying a container to this stack:
```heroku stack:set container```
- Push to heroku:

```git add .```
```git commit -m "Heroku deployment"```
```git push heroku master```

Now visit the application when deployment is done using:
```heroku open```

Behold the application🔥 

##Bonus 
- I added integration tests for the server implementation
- I also used Travis CI for Continuous Integration

Get all these from the repository: 
https://github.com/victorsteven/Username-Across-Platforms


#Conclusion
So there you have it! A full-fledged application that uses Golang awesome concurrency feature.
You can also visit other articles here or in my [medium account](https://medium.com/@victorsteven). 
Don't forget to follow for future articles.

Get the complete code on [Github](https://github.com/victorsteven/Username-Across-Platforms)


Happy Coding.