Building a Command-Line Interface with Ruby
Tags: ruby, programming, showdev, cnc2018
Published: 2018-02-20 00:54:13

---
title: Building a Command-Line Interface with Ruby
published: true
description: 
tags: #ruby #programming #showdev #cnc2018
cover_image: https://thepracticaldev.s3.amazonaws.com/i/3koex4fiqonbab9p6dtz.png
---

In this post I’ll talk about my experience building a command-line interface application with Ruby; generating a gem directory, scraping the data with Nokogiri, and putting together the user interface.

## Draw Your Own Map
After several weeks of gradually levelling up my Ruby-wielding skills through [Flatiron School](https://flatironschool.com/), this was my first solo project --a command-line interface application. In the exercises leading up to it, there had been tests to pass or fail, to tell me when I was indeed headed in the right direction. But this time my only guides were my vision of the end result and these project requirements:

- Provide a CLI.
- CLI must provide access to data from a webpage.
- The data provided must go at least one level deep, generally by showing the user a list of available data and then being able to drill down into a specific item.
- Use good object-oriented design patterns.


## Finding North
Before I could write a line of code, I needed to pick a website to scrape. This would dictate the kind of data my application would provide access to i.e., its purpose. And, I needed to choose something I could scrape from without too much difficulty.

Thankfully, I had a website in my back pocket that I suspected would work just fine. The [Hack Design](https://hackdesign.org/) website provides lessons about design in various categories. Its pages lent themselves to the one layer deep model that was required. And they were rendered via static HTML putting them just in the range of difficulty I was hoping for. Being able to access these lessons from the minimal environment of the command-line struck me as a cool idea. So after a cursory assessment of whether I’d be able to scrape the content I needed, I decided to go for it.


## Set Up
I knew at the beginning that I wanted to be able to wrap my application in the self-contained, distributable format of a gem. [Bundler](http://bundler.io/v1.16/guides/creating_gem.html#getting-started) made it easy to get started with a scaffold directory.

![Getting Started with Gems using Bundler](https://raw.githubusercontent.com/Adjoa/words-with-pictures/master/pictures/bundler.PNG "Getting Started with Gems using Bundler")

The first time you run the `bundle gem` command, you have the option of including a `CODE_OF_CONDUCT.md` and `LICENSE.txt`. I chose the license. I updated my `.gemspec` file with details about my gem: a short summary of what my gem would do and a slightly more detailed description of the same. As I nudged its functionality forward, I added a short list of other gems it would depend on to function.


## A View of the Destination 
I started by writing code for a user interface that would mimic the intended behaviour of my application. The user would be greeted by a welcome message and options to: view categories, view lessons, view a random lesson, or quit. I didn’t have any categories or lessons yet so, I threw in some fillers to start. This code lives in `lib/hack_design/cli.rb`. I would call this code in an executable file called `bin/hack-design` and run my program.

I didn’t have any tests but, I knew what I expected my code to do. Debugging was a matter of trial-and-error-ing my way to success. This technique would take me through the development of my app.

## Fits and Starts
I intended to make the data two layers deep --providing content (including exercises) from within a lesson, and lessons from within a category. Taking my cue from the UI I had built, I created classes to model a `Category`, a `Lesson`, an `Exercise`, and a `Scraper`. I used `./bin/console` to test drive these classes by looking for expected behaviour. However, as I set out to teach my `Scraper` how to find and gather categories, life threw me a few curve balls which effectively stalled my progress for about 2 weeks.

When I returned, I was anxious about the time I had lost. Closer investigation of the data from my `Scraper` soon revealed that scraping each category would prove to be more difficult that I first imagined. So, I thought about it. First and foremost, I wanted to provide users with lesson content. Was listing categories really that essential to the purpose of the application? I decided not, and began to simplify.

I threw out the `Category` and `Exercise` classes. And I was left with what I needed: a `Lesson` class whose objects would organize data about each lesson, a `Scraper` to gather this data from the website, and a `CLI` to manage the user interface. I refactored the UI to reflect the changes. It would list lessons to choose from, not categories. And, the data was now just one layer deep.

## Keep Your Data Sources Close
With my UI was up and running, it was time to supply it with real data. I knew it would take a few tries to get the search queries right. From my previous attempt at scraping, I also knew that too many queries would very likely get me blocked. I decided to solve this problem by copying the pages I’d be scraping to a fixtures folder. This would allow me to keep my requests to the live site to a minimum.

The way Hack Design is set up, all 51 lessons can be found listed on a lesson [homepage](https://hackdesign.org/lessons). From that page, each lesson is links to its own page. Copying down the source code of the homepage was simply accomplished by using [cURL](https://curl.haxx.se/docs/manpage.html) to get the HTML and shovelling it into a new file.

```bash
curl http://hackdesign.org/lessons/ >> fixtures/site/lessons.html
```

The other 51 pages however, would pose a greater challenge. No way was I going to navigate to each page and copy down hundreds of lines of HTML for 51 individual files. I wrote a little Bash script to do it instead.

```bash
#!/bin/bash
#get-lesson.sh

typeset -i - END
let END=50 i=0
while ((i<=END)); do
  echo “Script starting now…”
  curl https://hackdesign.org/lessons/$i -O
  let i++
done
echo “Done”
```

## Scraping Forward with Nokogiri
Before long, I had all the files I needed to get to scraping. For this task, I used an HTML Parser called Nokogiri. It has the ability to search documents using CSS. I used CSS selectors to zero in on the HTML elements containing the data I wanted. In keeping with the setup of the Hack Design website, the `Scraper` has two methods.

`::scrape_lessons_page` scrapes the lesson homepage. creates a hash for each lesson, adds the lesson title and url to that hash, then adds that hash to an array of all the lessons.

`::scrape_lesson`scrapes a lesson's content page. It adds the instructor name and other content for a particular lesson to a hash and returns the result.

## Homecoming: Return to the CLI
The `CLI` is where the `Scraper` and `Lesson` classes join forces to produce `Lesson` objects that model individual lessons filled with content. Here, hashes act as the glue between the `Scraper` and `Lesson` classes, carrying data from the `Scraper` to the `Lesson`, allowing them to work together while remaining independent and focused in purpose.

A title and a URL are all I need to create a `Lesson` object.  I created 51 `Lesson` objects by iterating over the array of hashes from `::scrape_lessons_page`. Then I passed the URL of each `Lesson` object to `::scrape_lesson` which returned a hash of data I could add to my ready-made `Lesson` objects. This process went smoothly until… it didn’t.

One of the pages was breaking my `Scraper`. Using Pry and some temporary tweaks to my code, I was able to track down the page that was causing trouble. Turns out [lesson 41](https://hackdesign.org/lessons/41) has a list within one of its exercises. However, my `Scraper` was identifying exercises as list items. On discovering this list within a list, it would class the list item as another exercise and then crash when it didn't find the typical contents of an exercise inside. It needed a way to differentiate between an exercise and its content. To do this, I made the CSS selectors a little more specific. I had fun debugging this one. 

51 completed `Lesson` objects made filling out the user interface wonderfully straightforward. I could just loop through all the `Lesson` objects and display the information they contained accordingly. I added a few methods to enable navigation from one lesson’s content to another without returning to the list, and added a bit of color with the [colorize](https://github.com/fazibear/colorize) gem. And then, I was done. I had built a Ruby command-line-application! 

To wrap up, I packaged, installed, and tested my gem. Bundler makes [packaging and releasing your gem](http://bundler.io/v1.16/guides/creating_gem.html#releasing-the-gem) as easy as it does getting started. I chose not to release my gem but, I have included instructions on how to package and install it locally. Check out my code, [here](http://github.com/Adjoa/hack_design-cli-app).

## Fond Farewell
Completing this project was an exercise in confronting fear of failure. Every step was absolutely worth it. I am proud of this simpler, final product and really excited about what’s next.

Thanks for taking the time to read this post. See you in the next one!



Cover Art: [Mayumi Matsumoto](https://matsumayu.deviantart.com/)