Moving to deps.edn
Tags: clojure
Published: 2021-12-09 01:05:34

I've been using Clojure for ... some time now; I think I started experimenting with it in 2009, possibly earlier. At both Aviso and Walmart I have used, and often fought with, [Leiningen](https://leiningen.org/), the standard build tool. 

Finally, I'm changing over to using `clj` and [deps.edn](https://clojure.org/guides/deps_and_cli). As is often the case, the inertia of using the tool you know can get in the way of learning a tool that may just fit your needs better.

_deps_ is another case of Clojure's emphasis on data and simplicity, and on _explicitness_ as a better goal than _conciseness_.

At its core, _deps_ does two things:
* Build the classpath (resolving dependencies and downloading as needed)
* Start a Clojure instance and execute a function using the classpath

There's a few bells and whistles built on top of these two essentials, such as _aliases_ to adjust the classpath and the executed function and function arguments, such as to launch a test runner.

Most importantly, though, deps's more narrow focus makes it easier to understand what's going on as it executes, and to build new and customized functionality that every project seems to need.

For Lacinia (and my other open source projects), I had a goal:
* Store the version number of the library in *one* place
* Include that version number in [generated documentation](https://github.com/weavejester/codox)

In Leiningen, your basic meta-data (`defproject`) includes the version number right there, accessible to any plugins.  With deps, a version number doesn't really make sense ... remember, we're just putting together the classpath and running some Clojure code.

So, if it comes to building and publishing a JAR, you need to implement some project-level build tools.

You do this by defining a `:build` alias:

```clojure
  ;; clj -T:build <command>

  :build {:deps {io.github.clojure/tools.build {:git/tag "v0.7.4" :git/sha "ac442da"}
                 slipset/deps-deploy {:mvn/version "0.2.0"}}
          :ns-default build}}
```

The -T option alias also includes the current directory as a source, so we can create a `build.clj` file there:


```clojure
(ns build
  (:require [clojure.tools.build.api :as b]
            [deps-deploy.deps-deploy :as d]))

(def lib 'com.walmartlabs/lacinia)
(def version "1.1-alpha-7")
(def class-dir "target/classes")
(def jar-file (format "target/%s-%s.jar" (name lib) version))
(def copy-srcs ["src" "resources"])

(defn clean
  [_params]
  (b/delete {:path "target"}))

(defn jar
  [_params]
  (let [basis (b/create-basis)]
    (b/write-pom {:class-dir class-dir
                  :lib lib
                  :version version
                  :basis basis
                  :src-dirs ["src"]
                  :resource-dirs ["resources"]})
    (b/copy-dir {:src-dirs copy-srcs
                 :target-dir class-dir})
    (b/jar {:class-dir class-dir
            :jar-file jar-file}))
  (println "Created:" jar-file))

(defn deploy
  [_params]
  (clean nil)
  (jar nil)
  (d/deploy {:installer :remote
             :artifact jar-file
             :pom-file (b/pom-path {:lib lib :class-dir class-dir})
             :sign-releases? true
             :sign-key-id (or (System/getenv "CLOJARS_GPG_ID")
                              (throw (RuntimeException. "CLOJARS_GPG_ID environment variable not set")))}))
```

Each function here becomes a runnable command, such as `clj -T:build clean`. Each of these functions accepts a `params` map and returns it (these are values provided as command arguments, read as Clojure forms, and assembled into a map of keys and values).

Core to all of this is `clojure.tools.build.api/create-basis`; a basis is a description of the features of the project, including the classpath with source directories and third party dependencies resolved and expanded.  Once you have the basis, other `build.api` commands are used to generate Jar files and deploy to Clojars.

The above `build.clj` code is good so far ... a bit of boilerplate, but it is reasonably expressive and not too verbose.  It's very easy to see how to customize this for different projects.

But back to my challenge - I want to generate documentation with the right version number. Normally, Codox is invoked via an alias defined in `deps.edn`:

```clojure
 ;; clj -Xdev:codox

  :codox
  {:extra-deps {codox/codox {:mvn/version "0.10.7"}}
   :exec-fn codox.main/generate-docs
   :exec-args {:metadata {:doc/format :markdown}
               :name "com.walmartlabs/lacinia"
               :version "1.1-alpha-7"
               :description "Clojure-native implementation of GraphQL"}}}
```
 
This adds Codox to the classpath along with the project dependencies, and the dependencies of the `:dev` alias, and invokes `codox.main/generate-docs` to do the actual work.

Alas, this approach still requires keeping a second copy of the project version synchronized.  

Ideally, there should be a way to _inject_ the true version number, from `build.clj`, into the `:exec-args` above, but that goes against he grain of deps; the contents of `deps.edn` are just data; there's no functions to be called to read the version number from somewhere else.

The next best thing is to have a have a `codox` build command, defined in `build.clj`, that calls `codox.main/generate-docs`.

I did a few iterations on that approach but there are problems:  the  classpath for `-T` commands _does not include_ the project's classpath, and Codox needs to be able to load namespaces as part of how it organizes and generates the documentation.

Eventually, I came across the following solution:

- Build a classpath that includes the main dependencies, and Codox
- Build and execute a Java command to run Codox

It ends up looking like this:

```clojure
(defn codox
  [_params]
  (let [basis (b/create-basis {:extra '{:deps {codox/codox {:mvn/version "0.10.8"}}}
                               ;; This is needed because some of the namespaces
                               ;; rely on optional dependencies provided by :dev
                               :aliases [:dev]})
        expression `(do
                      ((requiring-resolve 'codox.main/generate-docs)
                       {:metadata {:doc/format :markdown}
                        :name ~(str lib)
                        :version ~version
                        :description "Clojure-native implementation of GraphQL"})
                      nil)
        process-params (b/java-command
                         {:basis basis
                          :main "clojure.main"
                          :main-args ["--eval" (pr-str expression)]})]
    (b/process process-params)))
```

At the top, we build a new basis that includes the project's base dependencies, the `:dev` alias dependencies, and Codox (and its dependencies). We're going to start a process to run a Java command to execute a Clojure expression - this expression ultimately is what invokes `generate-docs` - and `build.api` provides all the heavy lifting related to setting up the classpath and starting processes.

Even though the above code isn't a macro definition, we can still use the Clojure _syntax quote_ (the backtick) to create a snippet of Clojure code that gets included on the command line to the Java process.

So, running `clj -T:build codox` will invoke the `codox` function, which in-turn starts a second process running Java with an entirely different classpath.

At the end of the day, we have a short command to generate documentation, and our library's version number appears in just one place, `build.clj`.

I'm just scratching the surface of what can be accomplished using these techniques, but I'm excited at the possibilities of using real Clojure code to tackle some of our interesting build- and deploy-time requirements.