Adding a custom watcher to Phoenix
Tags: elixir, phoenix, node, chokidar
Published: 2020-04-29 17:47:48

---
title: Adding a custom watcher to Phoenix
published: true
description: Add a node-js watcher to the Phoenix dev config
tags: elixir, phoenix, node, chokidar 
---

At [Contact-Stack](http://www.contact-stack.com), we've recently added [Elm](https://elm-lang.org/) to one of our [Phoenix](http://www.phoenixframework.org/) projects. Elm is a fantastic language with a clear syntax, functional design, immutable data and a helpful compiler. 

For reasons of mild personal preference, we opted to not use [elm-webpack-loader](https://github.com/elm-community/elm-webpack-loader) when integrating the Elm code with the current Javascript setup that we already have from Phoenix. Though ideally, we would still like the experience we have when we edit the Javascript, ie. it is rebuilt and Phoenix reloads the current browser with the newly built assets.

Unfortunately for us the Elm compiler doesn't have a 'watch' mode so we can't rely on that. We need a separate process to run the Elm compiler whenever there is a change. I cannot find it now but I've seen comments from [Richard Feldman](https://twitter.com/rtfeldman), a prominent member of the Elm community, suggesting that the [Chokidar](https://github.com/paulmillr/chokidar) project can be used to set up a simple watcher which runs the Elm compiler. Chokidar is a node project that does a great job of wrapping some of the node standard library functionality to provide a robust watcher. It is used by a number of high profile node projects, including [Webpack](https://webpack.js.org/), to provide file watching functionality.

For reference, the exact build command that I would like run is: 
```
elm make src/Main.elm --output=../priv/static/js/elm.js
```
From within the `assets` directory in the standard Phoenix project layout.

Now to start, we might consider adding the [`chokidar-cli`](https://github.com/kimmobrunfeldt/chokidar-cli) which allows you to set up watchers with a simple command. We can add chokidar-cli with `yarn add -D chokidar` and then run:

```
chokidar "**/*.elm" -i node_modules -c "elm make src/Main.elm --output=../priv/static/js/elm.js"
```
If we run that in the `assets` directory it works great so maybe that's a good start. We add `-i node_modules` as for some reason chokidar starts tracking some files in `node_modules` too and we don't need it to. 

How do we go about adding it to Phoenix? If we look in the `config/dev.exs` we see a block that looks like this:

```
config :contact_stack, ContactStackWeb.Endpoint,                                                                                                           
  http: [port: 4000],                                                                                                                                      
  debug_errors: true,                                                                                                                                      
  code_reloader: true,                                                                                                                                     
  check_origin: false,                                                                                                                                     
  watchers: [                                                                                                                                              
    node: [                                                                                                                                                
      "node_modules/webpack/bin/webpack.js",                                                                                                               
      "--mode",                                                                                                                                            
      "development",                                                                                                                                       
      "--watch-stdin",                                                                                                                                     
      cd: Path.expand("../assets", __DIR__)                                                                                                                
    ]                                                                                                                                                      
  ]                                                                                                                                                        
```
The relevant entry, as you might guess, is the `watchers` list. This is a list of key-value pairs that each provide a program and a set of arguments for Phoenix to run as part of its watcher functionality. So in this case, it is going to run `node` with that list of arguments which will result in it running `webpack` in development mode. The last part is to ask Phoenix to run it in the `assets` directory.

So we could try to extend this way:

```diff
   watchers: [
     node: [
       "node_modules/webpack/bin/webpack.js",
       "--mode",
       "development",
       "--watch-stdin",
       cd: Path.expand("../assets", __DIR__)
+    ],
+    node: [
+      "node_modules/.bin/chokidar",
+      "**/*.elm",
+      "-i",
+      "node_modules",
+      "-c",
+      "elm make src/Main.elm --output=../priv/static/js/elm.js",
+      cd: Path.expand("../assets", __DIR__)
     ]
   ]
```

And actually, this seems to work great. We run this and, every time we save an Elm file, Phoenix runs the Elm compiler with the command we've provided. 

Unfortunately, if we kill the Phoenix server and check the running processes on our machine:
```
^C
BREAK: (a)bort (A)bort with dump (c)ontinue (p)roc info (i)nfo
       (l)oaded (v)ersion (k)ill (D)b-tables (d)istribution
$ ps -ef | grep chokidar
michael  17499     1  0 16:16 ?        00:00:00 /home/michael/.nave/installed/12.14.0/bin/node node_modules/.bin/chokidar "**/*.elm" -c "elm make src/Main.elm --output=../priv/static/js/elm.js"
```
We can see that the Chokidar process is still running. This is not great. We want to be able to freely restart our Phoenix dev server as often as we like without building up a back log of Chokidar processes which are all watching our Elm files and each running the Elm compiler on every change.

So why is this happening? Well, I'm guessing that Phoenix uses [Elixir's Port functionality](https://hexdocs.pm/elixir/Port.html) to run the watcher subprocesses and Elixir's Ports functionality as a [big warning about zombie processes](https://hexdocs.pm/elixir/Port.html#module-zombie-operating-system-processes) that are left over when the main Elixir system process has stopped. It seems that Elixir doesn't, or perhaps can't, proactivity kill subprocesses that it has started when closing down. Rather it relies on those subprocesses noticing that their standard input has been closed and exiting themselves. I haven't come across this mechanism outside of Elixir but it might well be common. And if we glance up at the `webpack` watcher config again we'll see that they're using a [`--watch-stdin`](https://webpack.js.org/api/cli/#watch-options) flag. Coincidence? Probably not.

Unfortunately, `chokidar-cli` doesn't have a `--watch-stdin` flag nor any search results for `stdin` [in the code](https://github.com/kimmobrunfeldt/chokidar-cli/search?q=stdin&type=Code) so it seems like we can't rely on that. 

But `webpack` is written in Javascript running on node so it must be possible and the main [`chokidar` package](https://www.npmjs.com/package/chokidar) is a library that allows you to easily access the file-watching functionality.

If we go splunking through the webpack code looking for references to standard input then we come across [these lines](https://github.com/webpack/webpack-cli/blob/145c552d1ace3303607fe4d204106fe9437e24a0/packages/webpack-cli/lib/utils/Compiler.js#L176-L181) in the webpack-cli project. Now I don't perfectly understand what is going on here but it seems that it is listening for a `'end'` event on the standard input stream and using `process.exit()` to close the whole program if it happens. That seems to fit with what Elixir's Ports expect.

If we combine that with some relatively basic `chokidar` library usage as they outline in the README then we get something like this:

```javascript
const chokidar = require("chokidar");
const { execSync } = require("child_process");

// Exit the process when standard input closes due to:
//   https://hexdocs.pm/elixir/1.10.2/Port.html#module-zombie-operating-system-processes
//
process.stdin.on("end", function() {
    console.log("standard input end");
    process.exit();
});

process.stdin.resume();

// Set up chokidar to watch all elm files and rebuild the elm app ignoring process errors
chokidar.watch("**/*.elm", { ignored: "node_modules" }).on("all", (event, path) => {
    console.log(event, path);
    try {
        execSync("./node_modules/.bin/elm make src/Main.elm --output=../priv/static/js/elm.js");
    } catch (error) {}
});
```
And if we save it in a file called `assets/watch-elm.js`. And then we change our `config/dev.exs` config to read:
```
    node: [
      "./watch-elm.js",
       cd: Path.expand("../assets", __DIR__)
    ]
```
Then we can run `mix phx.server` and see that not only does the Elm compiler get run correctly on changes but when we kill our dev server the watcher process dies too. Success!

# Conclusion

Adding new watcher processes to Phoenix is relatively easy in some ways but this matter of watching standard-input is a bit confusing and is probably handled quite differently in different languages.
 
# Notes

The Phoenix documentation does provide a helper bash script that you can use to wrap an executable and which does the 'listening for standard input to close' for you. I have used that successfully when having to run a subprocess during Elixir tests but I wasn't unable to get it working on first try in this situation. Possibly the complexities of the extra escaping of the command line arguments got the better of me. I'm not sure.