Small, sharp tools can cut you
Tags: commandline, unix
Published: 2020-05-06 16:47:05

---
title: Small, sharp tools can cut you
published: true
date: 2020-05-06 15:33:37 UTC
tags: command-line, unix
canonical_url: https://avdi.codes/the-limits-of-small-sharp-tools/
cover_image: https://i1.wp.com/avdi.codes/wp-content/uploads/2020/05/IMG_20200506_102552-scaled.jpg?w=2560&ssl=1
---

The household where I’ve been spending stay-at-home has one of the most pragmatic kitchen organization schemes I’ve ever run across. For instance, there’s “the drawer of things that scoop”, and the “the drawer of things that cut or stab”. As a result I rarely have to ask someone where to find a tool.

I’m told that the “drawer of things that cut or stab” was once “the drawer of things that cut or stab **or burn** ”. This posed problems during power outages, however. When you’re rummaging around in the dark for a candle and a lighter, you’d rather it not be in a drawer full of blades and sharp points.

The UNIX command-line philosophy is famously to provide a drawer full of [small, sharp tools](https://www.brandur.org/small-sharp-tools). Years ago, in one of my many iterations of building an ebook-construction toolchain, I decided to pose myself a challenge: lean fully into the UNIX command-line model. Build the toolchain using only single-purpose command-line tools, using each one for what it does best.

For text replacements and munging: [sed](https://en.wikipedia.org/wiki/Sed). For linewise filtering and splicing: [awk](https://en.wikipedia.org/wiki/AWK). For macro expansion, [m4](https://en.wikipedia.org/wiki/M4_(computer_language)). For XML extraction and transformation, [xmlstarlet](http://xmlstar.sourceforge.net/). For HTTP downloads and uploads, [curl](https://curl.haxx.se/). For document conversions, [pandoc](https://pandoc.org/) and [Calibre](https://calibre-ebook.com/). Tying it all together: [make](https://en.wikipedia.org/wiki/Make_(software)) and [bash](https://en.wikipedia.org/wiki/Bash_%28Unix_shell%29).

It was a nightmare. You don’t realize how much you take string-interpolation and concatenation for granted in your programming language of choice until you have to deal with a half a dozen text processing tools each of which has just _slightly_ different syntactical rules for expanding variables.

And then there’s all the character escaping. Working with a conglomeration of UNIX text tools inevitably means that at some point you will need to tunnel some code or annotated data _through_ one tool into another, so that it will be expanded in the right context. Which means escaping, and double-escaping, and oh… does double-escaping cancel out the escaping in this dialect?

This mess is exactly why Perl was created. By the time _I_ came to Perl it was a general-purpose programming language that came highly recommended for chewing through large text-munging jobs. But for its original audience, Perl was a way to have `sed` and `awk` and `m4` and `grep` and `cat` and `cut` all with one _consistent_ programming model, with consistent data types and consistent variable scoping and consistent expansion and interpolation rules.

And then there was Ruby, which was just Perl but undergirded with Smalltalk’s object model instead of being constructed on a foundation of [creeping chaos](https://lovecraft.fandom.com/wiki/Azathoth). And Ruby begat [Rake](https://github.com/ruby/rake), and when it came time to build yet _another_ ebook construction toolchain I built it in Rake, and all was well.

Getting back to the original point… drawers full of small, sharp, self-contained tools are fine in well-lit kitchens when all you need is to pick out _one_ and use it by itself. When a project requires a half a dozen of those capabilities together, with rapid context-switching between them… well, keep a box of band-aids handy.

At that point you don’t want to treat those capabilities as separate tasks anymore. You want them to be verbs and nouns in a larger, cohesive language of construction. A monolith. A food-processor.