The Story of How I wrote A CLI for Your Bookmarks using Larder.io
Tags: ruby, cli
Published: 2018-10-28 18:06:48

---
title: The Story of How I wrote A CLI for Your Bookmarks using Larder.io
published: true
description: How Lard came to be, what I learned, and what's next for the project
tags: ruby,cli
cover_image: https://github.com/hawkins/lard/raw/master/screenshots/folder.png
---

The story starts when I began using [Larder](https://larder.io) recently, and while I loved the mobile app and browser integrations, I really just wanted to use it from the command line too. I checked the roadmap and found that a CLI was suggested back in 2016, but there had been no other motion on that effort. A quick glance revealed they had a [nice and simple API](http://developer.larder.io), so I popped open a terminal and got to work.

I've been writing more and more Ruby lately, and I've been loving it! I thought this would be a fun time to become a publisher of ruby gems and not just a consumer, so I set out to write the project in Ruby. (Also, I heard great things about a few key Ruby libraries that would make this effortless, but we'll get to that.) It wasn't long before I had [a first commit](https://github.com/hawkins/lard/commit/621385ea2eb602fcdc57da9c574dc9799c0d66a8) that printed out info on a user based on a provided token from a `yaml` file. Not half an hour later, I had added [`lard folders`](https://github.com/hawkins/lard/commit/83598fe557932de591e80a3ffb96f8b1e4d1c681) and [pretty-printed](https://github.com/hawkins/lard/commit/fa0b694744b4be3299403e39b19fafd00f98c6ea) them too. I'll post pictures of the output later, but for now, you can see this on the project's readme: {% github https://github.com/hawkins/lard %}

It started simply:

```ruby
# lib/lard.rb
require 'httparty'
class Lard
  include HTTParty
  base_uri 'https://larder.io'
  def initialize(token)
    @options = {
      headers: {
        'Authorization' => "Token #{token}"
      }
    }
  end
  def user
    res = self.class.get '/api/1/@me/user', @options
    JSON.parse res.body, symbolize_names: true
  end
end

# ...
# bin/lard
#!/usr/bin/ruby
require 'yaml'
require './lib/lard'

config = YAML.load_file 'lard.yml'
lard = Lard.new config['token']
puts lard.user.inspect

```

But hold up a second - am I really calling it "Lard?" What, am I 5 years old? Well... sort of. I actually have some decent reasons though, so bear with me:

- I'm not associated with Larder, so this is third party and can't use their name as my own
- It's a command line app, so the less typing, the better - short names rule!
- It's a Larder client first and foremost, though, so the name should be memorable
- And, again, I'm five years old, so I think it's just silly enough that its other merits stand strong to make it a good name

I could tell this was going to be fun, and not too stressful, so I was sold on keeping the project and making it feature-complete with regards to what their API provided as quickly as possible. So, to make sure I wouldn't step on anyone behind Larder's toes, I shot them a tweet asking if it would be alright to open-source my work (my repo was private, at the time). {% twitter 1054206203839070210 %}
They very kindly agreed... {% twitter 1054207395268685826 %} so I was off to the races!

I wrote a little bit more code to fetch bookmarks from a folder, broke my ruby install by mucking about with rvm and rbenv and finding they had collided on my mac, and finally went to bed.

---

The next morning I woke up early to work on the project and fix my ruby install (exciting, right!?) and finish fetching bookmarks. Eventually I ran off to work, but when I came home, I spent all evening fixing the ruby install to compile with openssl as needed, and got work late that night since I was too hyped to sleep. I only coded an hour that night, but I got a [lot done](https://github.com/hawkins/lard/blob/129cafe5a3fe8801905433f1b89180123d1a0762/lib/lard.rb#L35toL47) as I was able to pretty-print out an entire folder of bookmarks, paginating with the Larder API and all.


---

The next day, I decided it was time to quit writing my own CLI kit and adopt such a wildly popular one, [Thor](http://whatisthor.com/). It was *super* nice, and this was where I really started learning in this project. I had already learned how to make basic HTTP requests with `net/http` and `HTTParty`, but now I was learning a speedy and nice way to write command line interfaces in Ruby without breaking a sweat.

I also [almost admitted I was doing all this on a mac](6c88e7e5a33525de182df74ade778a9d5101d1d3) by committing the `.DS_Store` file. 🤫

Then, I [published Lard 0.0.1](https://github.com/hawkins/lard/commit/2452d940b0a12212919fa3eef5eba77d1520f66a), the first version of the gem! Woohoo!! SPOILER ALERT: I didn't know it at the time, but `gem install lard` didn't bring along Lard's dependencies, so don't bother installing this particular version...

Finally I made a hackjob at [posting a bookmark](https://github.com/hawkins/lard/commit/b9a5549a8416529d467faeb834b1afb8c16b4ef7) (which turns out you can post *and* edit at the same endpoint, it matches based on the URL you provide). But it didn't work yet - I was seeing an HTTP error `301: Redirection`, and my `POST` request was getting an error back:

```
{"detail":"Method \"GET\" not allowed."}
```

But I was sending a `POST` request! I spent a long time debugging this. After a while I cracked open wireshark and saw that I was encountering a `301: Redirection` and sending a `GET` request to the redirected URL. Now that was odd. I didn't know why that would happen. Normally HTTP requests follow through `300`'s like they weren't even there, right?

Well, not this request. It was late, I was tired, and I just wanted this to work so I could start using the CLI myself and not just developing it. I decided to get some air, went on a walk, picked up a milkshake to ruin whatever few calories I'd managed to burn on the walk. I chatted with my friend and compiler partner Cameron about the issues, and found the solution on my walk home. I was so burned out I didn't want to write the code for it though, so I opened an issue for it and went to bed. {% github https://github.com/hawkins/lard/issues/4 %}

---


The next morning, I was cooled off and wrote the [patch to fix httparty](https://github.com/hawkins/lard/commit/5fb28de6e0383c832b2c286ac46715bd4dbbd359#diff-d4e9001d759eec3f96532cd6db835a4eR103) (yes, it was one stupid function call to change an option otherwise not exposed and one that seems contrary to HTTP 300 semantics). Now I don't want to trash talk this library - other than this gripe, it was *super* nice. But this hobby project should be fun, and I don't want headaches - I get enough of those living with chronic migraines! So just know that if you use HTTParty, that's perfectly fine, but this very minor gripe was enough to give it a bad taste for me. I have the luxury of making very simple requests in this project, so going back to `net/http` where I know exactly what I'm getting isn't a big deal at all.

I kept using HTTParty for now, but I knew it wouldn't be here to stay much longer. I wanted to do fun things, features, not refactoring. So later that night, wrote [`lard tags`](https://github.com/hawkins/lard/commit/bb92194d6bc9d3ac176d464cfe0135a9a6f03947), [`lard search`](https://github.com/hawkins/lard/commit/be20eeded762242fe0a477751c4b5d65c98336e3), and [`lard login`](https://github.com/hawkins/lard/commit/0ef35314e22021cbe5f483ff4b12d400bf647712).

I tweeted back to Larder now that I had something to show, and they were excited! {% twitter 1055299299922362370 %}

{% twitter 1055303873638744064 %}

That got me way too hyped, and I really needed to go to bed, so I kept tagging and publishing new versions between features hoping that would make me stop and rest, but [alas](https://github.com/hawkins/lard/commit/f99fa936dec3ac89472064546ebef686ad585ef4):

```
Bump 0.0.4  … committed 4 days ago

This should've been 2 at this point but I kept thinking I'd stop and go
to bed, but here we are. A few bugs and features later, and this is it.
```

then only [8 minutes later](https://github.com/hawkins/lard/commit/02f4a5084d8c4f000865aa1c8a5914e641839aaf):

```
Correct nil check  … committed 4 days ago 

I said I was done for the night, but I have no self control when it
comes to side projects
```

Finally after this, I published version `0.0.5` (still not installing gem dependencies with it, mind you) and went to sleep.

---

The gang at Larder told me the API limits folder requests to 200 items per request, so I applied that in my code as soon as I got a chance to the [next evening](https://github.com/hawkins/lard/commit/098c219b57927671cc9020acfe83dd5bf933431d). Then I did lots of refactoring type things to make working in the project feel less-icky, starting with [error messages](https://github.com/hawkins/lard/commit/f00460d60d79c9c1291741dad8c15a1c67f324f3). Then I learned about the magic that is [rubocop](https://github.com/hawkins/lard/commit/013b619dfdc1a60204be7fd7f18d26ac68c042de),and I desperately questioned how I ever wrote large scale ruby projects without it. Wow! Nicest linter I've seen in a long time. My pet language, Druid, has a lot to learn from this for it's "friendly compiler" effort...

Most excitingly, I [set up Travis CI](https://github.com/hawkins/lard/commit/34901845e2f921afdd86c0e857c28f29b7f29df6)! If you're not familiar with Travis, Travis CI is a super awesome and free continuous integration platform that integrates insanely well with GitHub and practically every popular language and software packaging platform like RubyGems. Perfect for running tests, linting, and deploying updates to your package automatically when you open a PR or tag a release! Check out Lard's [status page on Travis](https://travis-ci.org/hawkins/lard) where you can see the latest builds it has ran, what triggered them, and whether tests are passing/failing!

---

The next day, October 26th (6 days straight working on Lard!), Josh Sharp from Larder opened an issue which caught me by surprise: {% github https://github.com/hawkins/lard/issues/7 %}

Woah, one of the two devs from Larder tried my project? Awesome!! And it failed catastrophically, OOOOOPS!! I was so embarrassed, but I was excited because he'd clearly found an issue that I could learn from! That's why I love open sourcing everything I do, because someone is bound to have a different experience, and I LOVE learning from these opportunities!

Right away, I had a theory: {% github https://github.com/hawkins/lard/issues/7#issuecomment-433578790 %}

... and I learned two things at once:

1. I should do some *very basic googling* before responding to issues. Even if I really just want to fix the user's issue as quick as possible (with the `gem install *` commands I gave him), I might be able to give a more insightful and easier solution after a quick search
2. Sure enough, I was right about how Gem dependencies don't come from `Gemfile`: {% github https://github.com/hawkins/lard/issues/7#issuecomment-433579472 %}

So I [patched it](https://github.com/hawkins/lard/commit/7562bb667de23145576439df0afea84ecd0c438f) and deployed it as `0.0.6` so Josh could try again later.

Then, more refactoring. I decided to [split the CLI from the lib](https://github.com/hawkins/lard/commit/8b124cb3cb36227931a1c6e8c70094885b91d609) so that the CLI is easier to work with and other packages can leverage `lard` programatically in the future. Pretty straight forward stuff, just turning `LardHTTP` from a module to a class, maintaining some state, and leveraging that library from the CLI.

What I didn't know about this was how it could work simultaneously in production and development, though. Since this is my first foray into publishing a Ruby Gem, I've never had to link libraries from the file system, I could always just say "Hey Gem, find that thing for me" and be on my way. Not this time!

I first [kept `require 'lard'` in the code](https://github.com/hawkins/lard/commit/8b124cb3cb36227931a1c6e8c70094885b91d609#diff-d4e9001d759eec3f96532cd6db835a4eR6) that would be deployed to production. This works, because when someone runs `gem install lard`, Gem will know where `lard` is, so `require 'lard'` will resolve successfully. In development, though, `lard` is not installed, we just have its source files. So at the time, I would manually change this line to `require '../lib/lard'` (or use `require_relative`) and change it back to `require 'lard'` before committing, so production code would not be looking relatively, but development code would. Gross.

After some digging, I eventually found that leaving `require 'lard'` in the code is the right move, but that I can run the development binary `bin/lard` with a command line argument to link the `lard` library from the source code by using [`ruby bin/lard -Ilib`](https://github.com/hawkins/lard/commit/20274836eea0aa41e4f7f2281feeeddc4d9dff7c#diff-04c6e90faac2675aa89e2176d2eec7d8R25). This looks familiar to you if you know GCC, but here's how it works:

- We run the `ruby` binary
- We pass it `bin/lard`, a ruby file from my source code that contains the CLI code (and the infamous `require 'lard'` call at the top of its file)
- We specify the `-Ilib` argument to map the `lib` folder (where `lib/lard.rb` exists) as an include folder for this runtime
- Ruby processes `require 'lard'`
- Gem is unable to resolve `'lard'` from its normal folder, and sees that it is not relative
- Gem finds that the `lib` folder was linked, though, and that it contains a `lard.rb` file - so this `require` is resolved to this file! 

This is great. Now, I can just run that command during development and be perfectly comfortable to not commit broken `require` statements. Woohoo!

---

Now technically day 7, even tho it's [12 AM on a Friday night/ Saturday morning](https://github.com/hawkins/lard/commit/0bc94fd258dc1a86a32733047a7fe7040147cce6), I cleaned up the `lard user` command to pretty-print user details finally.

At this point, the library was technically complete in everything I wanted it to do at a minimum to be a viable command line interface for Larder!

So I celebrated by going back to sleep finally and didn't get back at it until the next evening (same day, next night). I'm a little bit nocturnal on the weekends, if you haven't noticed...

This is where I really made Travis shine - by configuring it to [automatically deploy new gem versions](https://github.com/hawkins/lard/commit/10cbeecdf23f25b688fca5deeffdbf2e79682c11) when I tag a release in git. Yaha! (I also took some time to go back and do this to Prettier Webpack Plugin, like I should have done years ago) Hey, why not, shameless plug: {% github https://github.com/hawkins/prettier-webpack-plugin %}

Then, after some linter warnings, I finally got around to [removing `httparty`](https://github.com/hawkins/lard/commit/bd2e2f695c21ee6979040eefd29e1807241e58be). Whew! It felt great to have a little bit more control over what's happening, and to know I don't have to read the library author's mind to know how to use it anymore. Alright alright, sorry, let's move on.


---

Continuing the trend of "I'll sleep when it's done!", I kept on with Day #8 at midnight. The next issue I tackled was another interesting learning point for publishing Ruby Gems. Check it out: {% github https://github.com/hawkins/lard/issues/11 %}

I wanted to add a User-Agent to all network traffic coming from Lard to indicate it was coming from Lard. Trouble is, like the whole `require 'lard'` headache up above, I didn't want to have to make code changes here to be in sync with the gem version itself manually. That would be silly! Computers are smarter than me, surely they can do this themselves. My first thought was a `rake` task that would complain if I didn't change it manually. So when I tag a release, Travis would find that rake screams about the release not updating the User-Agent, so I at least get forced to change it without publishing an accidentally-wrong User-Agent.

Okay, but that doesn't fix the root issue. I thought back to how `gemspec`'s are actually written in Ruby and leverage the `Gem` library. So I thought "maybe Gem can tell me what version of `'lard'` I'm using?"

Sure enough, it can, with `Gem.loaded_specs['lard'].version`. Bingo! But... wait. No, no dice. That only works in production, when gems are loaded from the local gem repository. Not in development, when they're linked virtually. Oof... well, okay. I thought I could add a constant to the library and pull that into `lard.gemspec` then, since `gemspec`'s are just Ruby code after all. [So I tried it...](https://github.com/hawkins/lard/pull/12/commits/5d684becf759b56ec1d09b4333738b9317fe5178) and after realizing I [neglected to mark `lib/lard.rb` as requirable](https://github.com/hawkins/lard/pull/12/commits/2eead1c3be74c7a76fd8e5151ad20968fec7e953#diff-6e7ac0d946e187ec51a3d99ab3a146fcR10) and wasting `0.0.7` as a test for this, I found that this version style would otherwise work!

In the process, I learned that RubyGems offers a pattern of [pre-release versions](https://guides.rubygems.org/patterns/#prerelease-gems), which are meant for shipping possibly-broken code like what I just wrote. Awesome! I just fixed two problems with one PR!

Finally, I decided it was time for proper tests. Now I love testing, especially when automated via Travis, but I've been struggling with how to test a program like this properly for a few reasons:

- How do we test effectively when our library relies so heavily on external network connections?
- What about our CLI, what do we test? The final binary, or the modules of it?
- How do we avoid testing the CLI kit's underlying capabilities (Thor)?

After brainstorming that at 1am (always a good idea!), I opened an issue with some thoughts and got to work:

{% github https://github.com/hawkins/lard/issues/15 %}

So to end out the night, I installed [RSpec](https://rspec.info) and wrote some [basic tests](https://github.com/hawkins/lard/commit/141ea01cb51ce6b0c5c348b7610d5569f9463076#diff-9732b19df1dbcad692c8f2099ce2bcd1R3) to get the ball rolling.

```ruby

# spec/lard_spec.rb
require 'lard'

RSpec.describe Lard, '#authorized' do
  it 'returns false to indicate user is unauthorized' do
    l = Lard.new
    expect(l.authorized).to be false
  end

  it 'returns true to indicate user has a token' do
    # Though we can't actually guarantee the token is authorized
    # without an http request
    l = Lard.new 'token'
    expect(l.authorized).to be true
  end
end

RSpec.describe Lard, '#api_url_prefix' do
  it 'returns the Larder API\'s Base URL' do
    l = Lard.new
    url = l.api_url_prefix
    expect(url).to be_an_instance_of String
    expect(url).to be_an_start_with 'https://'
  end
end

RSpec.describe Lard, '#get' do
  it 'will fail if not authorized' do
    l = Lard.new
    expect(l.authorized).to be false
    expect { l.get 'user' }.to raise_error(RuntimeError)
  end

  it 'can fetch user' do
    l = Lard.new 'token'
    expect(l.get('user')).to be_an_instance_of Hash
    expect(l.get('user')[:links]).to be_an_instance_of Integer
  end
end

# spec/spec_helper.rb
require 'webmock/rspec'
WebMock.disable_net_connect!(allow_localhost: true)

RSpec.configure do |config|
  # Handle web request mocking
  config.before(:each) do
    user_body = { 'id' => 1,
                  'username' => 'username',
                  'first_name' => 'greatest',
                  'last_name' => 'ever',
                  'timezone' => 'America/New_York',
                  'avatar' => nil,
                  'date_joined' => '2016-01-21T03:03:33Z',
                  'links' => 500,
                  'is_trial' => true }
    stub_request(:get, 'https://larder.io/api/1/@me/user/')
      .with(
        headers: {
          'Accept' => '*/*',
          'Authorization' => 'Token token',
          'Host' => 'larder.io',
          'User-Agent' => ['Lard/0.0.8', 'Ruby']
        }
      )
      .to_return(status: 200, body: user_body.to_json)
  end
# ... the rest of `rspec --init` output
end
```


# Recap: What I learned

I learned a *ton* in just 8 short days of writing an MVP command line interface for bookmarks management while working full time on other projects, so it's hard to sum it all up, but I've sat here for an hour now writing a silly story no one will ever read, so I might as well sum it up so I can regale my grand-children with the escapades of my youth:

- Indie devs rock! The two behind [Larder](https://larder.io), [Josh Sharp](http://joshsharp.com.au/) and [Belle Cooper](http://bellebcooper.com/) made an awesome product and were super supportive through the whole process. I can't wait to make Lard as awesome as possible, even if I'm the only user, just as a tribute to Larder itself.
- Language version management is hard. My mac came with Ruby installed, but it's sort of known that you shouldn't use it - so I installed rbenv some time ago. Well, I didn't remember that I had Ruby installed on my mac when I started this, so I grabbed rvm this time. Whoops! That put me in quite a pickle, but I learned how rvm links Ruby and its dependent libraries (having a non-system dependent OpenSSL being an issue for me) and managed to get that solved.
- `Gemfile != gemspec`. Almost every Ruby project I've used had a `Gemfile` that installed everything you need. This was my first Ruby gem I wrote myself, so I had to learn how to specify runtime dependencies on the Gem itself, as well as development dependencies that can belong in the `Gemfile`. Still a little unclear as to how to reduce duplication between them, but that's a story for another day.
- Rubocop rules. I've always loved linters, especially auto-formatters like [Prettier](https://prettier.io), but Rubocop is the best of both worlds in my opinion.
- Don't reach for the libraries until you know you need them. I generally don't, but in hobby projects like this, sometimes I reach for the big guns just to move quickly so I can have fun and not do drudge-work. This time, that gave me a big headache, so I've got more ammo to support rolling my own infrastructure on the next project. (Like we're doing over at Druid!)
- Ruby Gems are super smart. From linking development dependencies, making your gemfile just plain Ruby so you can avoid repeating yourself by importing some constants from your code to your spec, I was super pleased with working with Ruby Gems all around. Writing them and publishing them is a breeze! Especially with Travis.
- Oh, Travis CI freaking rules. I already knew this, but setting up auto deployment was the real hot stuff on it this time.
- Don't over-think your testing struggles. I could have set up testing way sooner, mocking HTTP requests is very much a solved problem when it comes to ruby testing, as my brief time with RSpec and Webmock showed last night. I wish I had set up testing sooner, but I had a lot to learn and iterate on so I put it off too long.

## What's next for Lard?

Well, first of all, *TESTS*! I'm absolutely getting proper testing in place as soon as I sit down to code again on Lard. But then, exciting new features:

- Improving `lard bookmark` to make it more dynamic and flexible to how you want to describe or change your bookmarks
- Aliasing commands with Thor (so `lard s` means `lard search`, etc)
- Working with your bookmarks offline, so you can make changes and upload them to Larder servers next chance you're online. Or, work entirely offline, if that's your workflow. (But Larder rules, so Lard will always be a Larder client first and foremost)

## Conclusion

Writing Lard was a blast, and I'm stoked on how it will help my workflow and how much I learned about Ruby and its ecosystem. I'm excited to see how far I can take the project, and I hope I convinced at least one reader out there somewhere to either Try out Lard and break my code or to make their own project writeup someday!

If you want to try Lard, feel free to run `gem install lard` and `lard help` to get started. Or pop over to my GitHub to check it out and say hello! {% github https://github.com/hawkins/lard %}

Alright, well I've spent entirely too much time in front of screens this week, so it's time I get out and enjoy the last part of this weekend. Happy weekend, everybody, and thanks for reading!