Three jq Tips You May Need to Know
Tags: jq, cli, json, productivity
Published: 2020-11-04 20:02:33

---
title: Three jq Tips You May Need to Know
published: true
date: 2020-11-03 13:50:55 UTC
tags: jq, CLI, JSON, productivity
canonical_url: 
---

![jq](https://miro.medium.com/max/1000/1*Zb1Sf4nO9QqCcREfy4vfdA.png)

[jq](https://stedolan.github.io/jq/manual/) is famously known as a powerful command-line JSON parser. Besides performing as a parser, jq has many other handy usages as well. Here, I'd like to share 3 jq tips you may need to know.

### JSON Prettier

When we `curl` API to fetch JSON response with raw data. It certainly looks messy:

```bash
$ curl https://api.spacexdata.com/v4/launches/latest
{"fairings":{"reused":false,"recovery_attempt":true,"recovered":null,"ships":["5ea6ed2e080df4000697c907","5ea6ed2e080df4000697c908"]},..."id":"5ef6a2e70059c33cee4a8293"}
```

Now let's toss raw data into `jq` and make it look prettier:

```bash
$ curl https://api.spacexdata.com/v4/launches/latest | jq
{
  "fairings": {
    "reused": false,
    "recovery_attempt": true,
    "recovered": null,
    "ships": [
      "5ea6ed2e080df4000697c907",
      "5ea6ed2e080df4000697c908"
    ]
  },
  ...
  "id": "5ef6a2e70059c33cee4a8293"
}
```

In the opposite way, when we got some pretty-printed JSON data and want to concatenate them to a one-line string, using `jq -c`:

```bash
$ curl https://api.spacexdata.com/v4/launches/latest | jq > example.json

$ cat example.json
{
  "fairings": {
    "reused": false,
    "recovery_attempt": true,
    "recovered": null,
    "ships": [
      "5ea6ed2e080df4000697c907",
      "5ea6ed2e080df4000697c908"
    ]
  },
  ...
  "id": "5ef6a2e70059c33cee4a8293"
}

$ cat example.json | jq -c
{"fairings":{"reused":false,"recovery_attempt":true,"recovered":null,"ships":["5ea6ed2e080df4000697c907","5ea6ed2e080df4000697c908"]},..."id":"5ef6a2e70059c33cee4a8293"}
```

### JSON Validator

jq can be served as a simple JSON validator, for instance:

```bash
$ echo '{"fairings":{"reused":false,, "recovery_attempt":true}}' | jq
parse error: Expected value before ',' at line 1, column 29
```

Because its output doesn't give much information to pinpoint directly the error in an obvious way, it's better to use a proper validator for some complex cases. But for some quick checks to validate JSON data, jq can be considered as a suitable choice.

### JSON Pathfinder

Analyzing JSON files and finding the correct path syntax to a certain value sometimes can be troublesome. Many times, I was punched hard by the frustration of trial and error. Until one day, this magic jq command line brings me the light: `jq -r 'paths(scalars) as $p | "." + ([([$p[] | tostring] | join(".")), (getpath($p) | tojson)] | join(": "))'`.

Let's use the same `curl` data from the previous example, in this case, I want to get the path syntax of `ships` ids:

```bash
$ curl https://api.spacexdata.com/v4/launches/latest \
| jq -r 'paths(scalars) as $p | "." + ([([$p[] | tostring] | join(".")), (getpath($p) | tojson)] | join(": "))' \
| grep '.ships'
.fairings.ships.0: "5ea6ed2e080df4000697c907"
.fairings.ships.1: "5ea6ed2e080df4000697c908"
.ships.0: "5ea6ed2f080df4000697c910"
.ships.1: "5ea6ed2f080df4000697c90b"
.ships.2: "5ea6ed2e080df4000697c907"
.ships.3: "5ea6ed2e080df4000697c908"
```

From the output, now I know the right path syntax is `.ships[]`.

As you can see from this example, this magic jq command line will print each path and value pair from the input JSON. Combing with `grep` command, it's sufficient to be used as a handy JSON pathfinder.

---

If you do know other hot jq tips, do not hesitate to share them with me.