BYOCID Part[0]
By: Alireza Goudarzi
Tags: Software Development, Ci Cd Pipeline, Gcp, Cloud, Gcloud

Build Your Own Continuous Integration/Delivery will be a short series sharing my recent attempts of finding a descent solution for myself and to my surprise I didn’t even by the everyone who claim they have one. The goal is to build a lightweight development/integration/deployment pipeline that is fully customizable to your needs. (and yes I know that’s not how it is commonly referred to but CYOCICD is objectively no better). So let’s get down to it.

A [Cautionary] Tale of a Modern Application Infrastructure

Blessed the bits that make all of this possible — It all started when the first characters where transmitted (a word that I like and has been depreciated due to other connotations outside of computer networking [today you would call this IT, so you see it is all good old meaningful words for me, no useless acronyms that doesn’t mean any thing to anyone]) across ARPANET. A lot of people instantly realized a lot can be done with this. The DoD thought there are people that could be murdered, NSA and CIA realized there are democracies that could be unrooted, and of course students at Stanford and MIT realized they can exchange money for some pod (true story)! And that was all in the 70s. Since then we have seen decades of system development divided in application layer, middleware, operating systems, the network layer, the communication layer and for a while it worked. But to make more complex systems operating at large scales and in a robust and maintainable way we needed something more.

A revolution started when Amazon offered the AWS and the subsequent wave of everything-as-service and anything-as-code means the same methodology and tools that made software so effective in the last decade can be used to design, develop, and deploy, and maintain the entire infrastructure stack. Not many people have fully embraced the wave, but the ones who do will be in for a treat going forward. And to review it is the ability to iteratively, create, revise, and refine in a distributed and collaborative way, and to do so continuously is what turned the software industry around. So let’s see how can this be done.

P.S. Below I’m going to do a little preliminary intro for those of you who would like to follow but don’t have the right background. So if you are just looking for the CI/CD stuff you can skip ahead to the next articles.

Hello World 2020

For better or worse (but most probably for better), most of the application development has been shifting to web-based front-end technologies. The cool thing about this is the unification of both technology stack and user experience across different platforms and environments, wide understanding of said technologies by many developers and ease of porting and deploying new apps. Now in the old days if you were going to make a desktop app or a web app and share it with any one you had to jump through hoops. Today, with the ubiquity of cloud technologies and providers this is very easy. For the rest of these series I will use Google Cloud Platform and a combination of Vue.js for frontend of the app and Python for and backend. If any of you would like to follow my steps you need set yourselves up with a google account, signup for GCP and start with your $300 of GCP credit. Remember although you have $300 worth of free credit to use GCP you still need to enter a billing account and enable API access. Also setup the gcloud SDK on your computers. Most things i’ll be doing here are available through the GCP web console, but as you will come to enjoy too, command line is still king.

Before you can start you need to setup your gcloud . Simply issue this command:

gcloud init

If this doesn’t take you to the Google login page you can explicitly login with gcloud using:

gcloud auth login

From the command line, create a directory for your new app and change directory into it:

mkdir myapp; cd myapp

Then create a file called main.py and enter these lines inside it:

def Main(Request): return "Hello World 2020"

Note that the name main.py is mandatory here. The lines above define a function called main which will be the entry point of the app which means every time the app will receive a request this functions is called. Also our app will be hosted on GCP so it is really GCP that is going to call your Main and provide the Request argument and fill it with the all the information about the request that the app is receiving. The function itself is only returning a string which will be sent back to the user. So let’s deploy this to the cloud. from the app directory you can issue this command:

gcloud functions deploy Main --runtime python37 --trigger-http\

--allow-unauthenticated

This will take a couple of minutes to run because GCP needs to setup the necessary infrastructure to host your app and service it to your potentially millions of users. The — trigger-http flag will tell GCP that your app will be accessed through the web and — allow-unauthenticated will tell GCP to allow the app to be accessed publicly.

If this commands is successful it will return a message with a line that will look like this:

url: https://us-central1-myproj.cloudfunctions.net/Main

This is the publicly accessible url of your app that you can share with your millions of users. Try visiting the url and it will show you those globally coveted words of:

Hello World 2020

More Details

So what I call app here is really implemented as something called Cloud Function. This is a fully managed web endpoint that you can send requests to. It is intention is to serve functions that receive a request, processes it either in a standalone way or by in turn connecting to other downstream services and produce an answer. Typically this is called by other software that need receive structured output to be processed algorithmically. However, this now gives us the easiest way that I know today to share an app with my audience at scale without worrying about resource provisioning. So I’m really stretching that intentions here. But as you will see this can go very far and in a quite systematic way give us basic building blocks to create even the most sophisticated apps without worrying about infrastructure networking and etc.

What’s Next

In the future articles of this series, we will see how to have a real app with interface and functionality with control over the resources needed for the app and the distribution of the app. We will then show that all of this can be integrated into a pipeline for automatic integration and deployment of the revisions of the app every time you make an update to this app.