Jenkins and Buddy
By: Ted Hagos
Tags: Continuous Integration, Ci Cd Pipeline, Continuous Delivery, Deployment Automation, Jenkins

Photo by Markus Spiske on Unsplash

A CI/CD (Continuous Integration/Continuous Deployment) practice is common-place nowadays. More and more dev teams are convinced and already sold on CI benefits. In the age of multi-regional teams spread across timezones, CI practices definitely speed up the development process.

CI has been abuzz since 2007. The exact and discrete origins of the practice are hard to pin down. Many believe it’s earlier than 2007. Some say it’s 2004 when Grady Booch mentioned “micro-processes” and continuous testing in his book “Object-Oriented Analysis and Design with Applications”; during those days, there weren’t any CI software. Development teams had to write their own tools to automate the build.

More and more teams adopted the CI practice, and as a result, some people wrote the first CI applications. One of the earlier apps that have seen massive adoption was CruiseControl and Hudson.

Hudson eventually took over CruiseControl in terms of popularity. Hudson’s author, Kohsuke Kawaguchi, was working for Sun Microsystems at the time. When Oracle took over Sun Micro sometime in 2010, it announced to trademark Hudson; the Hudson community fled and established Hudson under a new name, Jenkins.

Jenkins is the go-to solution for CI/CD for quite some time now; that’s the first-mover advantage for you; it’s one of the older and more established CI/CD tools. It’s basically an automation server written in Java. The basic idea is for you continuously test and build your software. The usual tasks of pulling code from source control, running test scripts, and creating builds are handled via plugins.

Jenkins is still popular now; many teams still rely on it for their CI practice, but we’re a long way away from 2007. Technology has moved. It’s already the age of containerization and cloud software.

In this article, we will revisit Jenkins and Buddy; the latter is a newcomer on the CI/CD scene, but it’s got some neat tricks that I’d like to show you. We will create a small project in Angular, and then we’ll deploy the project on Jenkins. After that, we’ll do the same task on Buddy.

Building a sample project

To do test runs on CI/CD platforms, it helps to start with a simple project. So, we’ll create one in Angular.

Assuming you already have Node and NPM installed, you can create an Angular project using the Angular CLI, which we will get right now. On a terminal or cmd window, run the command:

npm install -g @angular-cli

That should take care of the Angular CLI tool. To create (and run) an Angular project, use the following command:

ng new helloangular

The tool will ask if you’d like to include Angular Routing (press N for No), then it will ask what styling library would you like to use (choose CSS).

Angular CLI will bootstrap the new project. After it’s done pulling some files from the repos, run the following commands:

cd helloangular ng serve

Angular CLI comes with its own webserver. The ng serve command builds the project and serves it at http://localhost:4200. The bootstrapped project has default content, and it even comes with some starter tests too. If you’d like to run the starter tests, run the following command:

ng test

Now that our sample project is built, we can push it to GitHub.

Log in to your GitHub account and create a new repo; name it whatever you like, but I’ll name mine “helloangular”. After that, go back to the terminal and run the following commands:

git remote add origin git@github.com:username/helloangular.git git add . git commit -m “first commit” git push -u origin master”

The command above associates our local project (helloangular) to the remote repo we created in GitHub. Please be sure to replace username with your GitHub username.

This takes care of the test project. We can now move on to Jenkins.

Jenkins

To use Jenkins, here are the general steps.

You have to get the software and install it somewhere; a server in your office, data center or Amazon EC2, etc. You need to download plugins. Part of the Jenkins setup is downloading the common plugins. If your build workflow is quite simple, the common plugins will probably suffice. For most Java projects, the common plugins will do. Create a pipeline and define your tasks.

Getting Jenkins

Jenkins is available for macOS, Linux, and Windows. The process of installing Jenkins vary quite a bit, depending on which OS you’re using. On Windows, the installer comes as an MSI package, and then you run it as a service. On macOS, the suggested way to install is via the Homebrew package manager; subsequently, you will also start, stop and restart Jenkins via the brew command — according to the official Jenkins webpage, the native installer for macOS has been deprecated, as of time of writing. On Linux, you can get Jenkins by pulling it from the repositories. The official Jenkins webpage (https://jenkins.io/download) keeps updated information on how to install Jenkins on various platforms.

Native installers and package managers aren’t the only way to get Jenkins. In the age of containers, we can also run Jenkins via Docker; this is the route I recommend because using Docker lets us run Jenkins in an isolated way. There’s no need to pollute your workstation, especially if you’re simply trying out how a software works.

Assuming you already have Docker and already know your way around it, we can get Jenkins by pulling its official image from DockerHub (https://hub.docker.com). On a terminal or cmd window, type the following:

docker pull jenkins/jenkins:lts

When Docker is done pulling the image, we can run it with the following command:

docker start -p 8080:8080 jenkins/jenkins:lts

This will start the Jenkins server, and you can begin to use it from your browser; but if you run Jenkins this way, whatever configuration or changes you make in this session won’t survive a container reboot or shutdown. When you start Jenkins for the second time, you will lose everything you’ve done in the previous session. Jenkins cannot persist data when ran this way.

We need a way to persist files for Jenkins while it’s running in Docker. We can do that by using Docker volumes.

If you’re on Windows, the way to run Jenkins with Docker volumes is as follows:

docker container run -v d:\jenkins_docker:/var/jenkins_home -p 8080:8080 — name jenk jenkins/jenkins:lts

The -v flag (volume flag) tells Docker to map the folder d:\jenkins_docker (on the Windows host machine) to the folder /var/jenkins_home (in the Docker container). The folder /var/jenkins_home is where Jenkins keeps all its files. When we install a plugin, create a project, or any other configuration change, Jenkins will keep it in this folder.

If you’re on Linux or macOS, we need to do it in a slightly different way. See the commands below:

docker volume create jenkins_docker

docker container run -v jenkins_docker:/var/jenkins_home -p 8080:8080 — name jenk jenkins/jenkins:lts

A Docker volume in macOS isn’t automatically a folder in the host OS; that’s why we needed to create a volume first.

When Jenkins is running, go to http://localhost:8080 to start using it. The Jenkins page is shown below:

During setup, an initial password was generated and stored at /var/jenkins_home/secrets/initialAdminPassword; but you need to remember that this folder is inside Docker, so you can’t simply use cat to get it. To view the initialAdminPassword file, you need to use the following command:

docker exec <container id> cat /var/jenkins_home/secrets/initialAdminPassword

Where <container id> is the Docker container id; which you can get by issuing the command:

docker ps -a

You should see something like the screenshot shown below:

In this case, my Jenkins instance is running in container 32312fc81bba. So, the command to get the initialAdminPassword is as follows:

docker exec 32312fc81bba cat /var/jenkins_home/secrets/initialAdminPassword

You should see something like the picture below:

Copy the result of the cat command and paste it into the Administrator password field.

On the page that follows, you’ll see two options to get the plugins. You can select your plugins or install the suggested ones — I recommend choosing the latter if you’re a first time user of Jenkins.

The following page shows Jenkins’ installation progress. This can take some time depending on your Internet connection. This is also a source of frustration and confusion for many first time users. This process is prone to failure. Try Googling “Jenkins won’t install suggested plugins,” and results will flood you. I was one of the statistics, but I got through the hurdle by merely changing my DNS settings (TIP: use Google’s 8.8.8.8 in your machine’s DNS setting).

If one of the plugins fail to install, Jenkins will give you an option to retry. If all of the plugins are installed without issues, you’ll get to the next page.

Fill up the username, password, full name, and email address, then click “Save and Continue”. On the page that follows, confirm the URL for Jenkins; simply accept the default (localhost:8080), then click “Save and Finish.”

Using Jenkins

After you log in, Jenkins shows the welcome page; from here, you can start using it by “Creating a Job” (which incidentally is also the name of the link). Click the link to get started.

On the page that follows, enter the name for the job. Give it something descriptive.

Choose “Freestyle project” (click to choose it). Then click “OK”.

On the following page, go to “Source Code Management,” choose “Git,” and then provide the “Repository URL” of your GitHub project, as shown below.

There’s no need to provide Credentials since I created the helloangular project as a public repo.

The next piece of configuration we need to do is to identify the trigger for the build. Go to the “Build triggers” section and find the trigger appropriate for your workflow. In this example, I simply chose “Poll SCM”, then on the schedule, I provided the “* * * * *” (five asterisks) — it’s a CRON expression which means we poll every minute.

Dealing with Plugins

Jenkins is a lot friendlier to Java-based projects; after all, it cut its teeth on Java projects (and is itself, written in Java). When your project isn’t Java-based (like our example), you may have to deal with plugins.

For our sample project, I need to, at least, execute the following commands:

npm install -g @angular/cli -y npm install npm update ng test

We’ll need a NodeJS environment for these commands. So, let’s get some plugins. Go back to Jenkins home page, then go to Jenkins > Manage Jenkins > Manage Plugins.

Type “node” into the search field. Look for the NodeJS plugin and install it. When it’s installed, go to the Jenkins tab again (top left corner of the home page) then Jenkins > Manage Jenkins > Global Tool Configuration.

Give the NodeJS configuration a name (I gave it the name “node,” as you can see), then choose the NodeJS version. I chose 14.14.0; this is the latest version at the time of writing.

To test the NodeJS installation, go back to the Jenkins home page, then go to Jenkins > New Item. Give it a name, then choose Pipeline. On the Pipeline definition, type the following code:

pipeline { agent any tools {nodejs “node”} stages { stage(‘NPMs’) { steps { sh ‘ls’ sh ‘npm config ls’ sh ‘npm install -g @angular/cli -y’ } } } }

The preceding code is written in Groovy. You need to learn it if you will write Pipeline definitions for Jenkins. What the previous code does is to build a Node task and run some NPM commands in it. Click “Apply” and then “Save” to save the definition. To run this task, go Jenkins > all > (the name you gave the Pipeline), then choose “Build Now”. If it goes through without error, then we know the Plugin was installed correctly.

Going back to the Build Script

Let’s go back to the Freestyle job we created earlier, go to Configure, then go the “Build Environment”; we need to set up the NodeJS in that environment.

Click the “Provide Node and npm bin/ folder to PATH”, then choose the Node environment. I usually leave the “Cache location” with the default value.

Then on the Build section, add a build step; an “Execute shell” build step should suffice. We can use the following code:

npm install npm install -g @angular/cli -y ng build You can now run the build.

The basic idea is for you to keep working on your project and keep pushing changes to the repo. This build will run continuously. If something breaks, you’ll see it in the status of the builds.

CI/CD hosted/managed solution

If this is your first time around with CI software, seeing the Jenkins solution in the previous section can be a bit overwhelming. There can be some learning curve when you use Jenkins. There will be some experimentation, some research, lots of visits to Stackoverflow, and some hair-pulling at times. It’s not for everybody. You can’t approach Jenkins, thinking that the plugins will always save you. You need to put some time and effort into learning how to use the tool effectively.

Not everybody has the time to spare in learning how to use a CI tool. This is the reason why there is a business case for managed CI/CD solutions. DevOps has now joined the “as a Service” bandwagon. You can avoid the learning curve and time investment for wrangling your own CI/CD solution by simply paying the experts to do it for you. There is no shortage of DevOps as as a Service companies; at the time of writing, somebody compiled a list of the “30 best hosted CI/CD integration services”. You can find them here.

You can find the usual suspects in the article — Circle CI, Shippable, GitLab CI, Travis, CodeShip, and many more.

We’re about to experiment with another CI/CD solution; it’s not on Slant’s list, but I think it ought to be. Let’s head over to https://buddy.works.

Buddy

Buddy is a newcomer. It’s the new kid on the block. So, it’s got the late-mover advantage. The people behind Buddy knows the frustrations and baggage of DevOps practice; they built the platform with careful attention to user experience. You don’t need to read a manual to figure out how to do things.

Sign up to the website if you don’t have an account yet. You can use it for free; there are limitations, of course, but the free tier is enough so you can get the feel of it.’

Once signed in, try to create a new project. Click the button that says, “Create a new project”.

Choose the Git hosting provider. I chose GitHub. Buddy will retrieve the list of your repos from GitHub. You can scroll through the list to find your project.

You can also type on the “repository” field to filter the repos so that you can find it faster.

The name of the project appears in view. I found the repo I want to use for this project (helloangular); click to select it.

Buddy prompts to “Add a new pipeline”, click the button to do so. On the page that follows, give the pipeline a name.

Choose the branch of the repo. I chose to work on the master branch.

Next, choose the Trigger mode. I decided “On Push” so that every time I push some changes to the repo, this pipeline runs.

On the page that follows, we can choose the “Actions” that will run we trigger the pipeline. There’s a whole slew of actions to choose from; unlike in Jenkins, you don’t have to install Plugins anymore; and in case you don’t find the action you need, there’s an option to get in touch with the Buddy support team so that they can do it for you.

For our test run, let’s choose NodeJS; click to select it.

The script is already populated with some commands; you can edit it to suit your needs. Click “Add this action” to proceed.

At this point, you might want to see if this action works as expected before you add others.

To test the action, click on “Pipelines” on the left-hand side of the page. You should see the “Run” button; click it to proceed.

Put in some comment, then click the “Run now” button.

The pipeline gets into gear and plows away. Some screen animations indicate that Buddy is hard at work, but if you want to peek at what’s happening, click the “Progress details” button.

A screen with more details appears on the page that follows. You can click on the “Logs” button to see the actual CLI logs.

If the task succeeds, you’ll see a green “PASSED” button on the left lower portion of the Pipeline.

When you keep working on the Actions — editing and adding commands, then rerunning the pipeline; you might stumble into an error (shown in the screenshot below).

This is one of the limitations of the Free Plan. You’re limited to 512MB of cache space. Fortunately, this is easy to solve. Select the pipeline by clicking it. Then click “Settings”.

Make sure to resize your browser wide enough to see the menu items on the right-hand side of the page. Choose “Clear pipeline cache”.

After that, rerun the pipeline, as usual — It’s a good idea to always clear the pipeline cache in between runs, at least until you upgrade to a paid plan.

Let’s add another action. You can do that by clicking on the plus symbol (+) at the bottom portion of the Action (shown below).

Clicking the plus symbol at the bottom of the Action will chain another Action right after the current one. Clicking the plus symbol at the top of the Action will add another Action before the current one — see what I mean? The Buddy team paid attention to UX.

For the next Action, I chose “Angular CLI”. Then, on the script editor, I added just one line (shown below)

ng build

Once done, I added another Action; daisy-chaining on my recently added Angular action. I chose “Build Docker Image”.

Setup the Dockerfile, then tell Buddy where to find it.

Once you’re happy with the selection, click “Use this Dockerfile” — I actually added this Dockerfile to my project earlier; I didn’t walk you through it anymore. I assumed you’re already familiar with this process.

Now I’ve got three Actions on my pipeline (shown in the picture below). This should be enough for my test run.

By the way, you can re-arrange the sequence of the Actions by simply dragging and dropping them.

The next bit I want to show you is the notifications feature of Buddy. At the lower portion of the Pipeline screen, you’ll see two buttons;

“Add the first on failed action” — Clicking this button will let you choose an Action that will be carried out when the Pipeline fails “Add the first on back to normal action” — Clicking this button will let you choose an Action that will run when the Pipeline is back to normal

Click the “Add the first on failed action”.

When the Pipeline fails, I simply would like to send an email. So, add an Email action.

Fill up the email body, choose the type, and fill up the recipients of the email.

Click “Save this action”. You can test it by clicking the “Test action” link.

Now, do the same thing to “Add the first on back to normal action”.

My Pipeline is complete. I’ve got a set of actions, and I can be notified when the pipeline succeeds (or fails).

Conclusion

CI/CD platforms have certainly gone a long way since CruiseControl and Jenkins. If your team doesn’t have a lot of expertise on Jenkins, the initial setup and learning curve could take away valuable time — which I’m sure can be focused elsewhere (the actual project, perhaps). It’s a build or buy decision.

If you choose to use a hosted/managed CI/CD solution, there are plenty to choose from; but you can’t go wrong with Buddy. The experience is friction-less. Try it out so you can experience the ease of setup yourself.

There are a lot of things that Buddy got right, but what struck me most was the fact that I don’t have to learn another scripting language (Groovy, YAML, etc) to write a Pipeline. It’s point-and-click and drag-and-drop.