Project Idea 02 – journalctl Parser
Tags: ideas, sideprojects, journalctl, linux
Published: 2020-05-09 09:00:59

*This post published on my blog [before](https://aligoren.com/project-idea-02-journalctl-parser/)*

Hi everyone. Today I'm going to tell you about my project idea. Before this post, I published another one.

## What's this journalctl?

Let's dig into journalctl man page using the below command;

`man journalctl`

We will see an output like that

```text
journalctl may be used to query the contents of the systemd(1) 
journal as written by systemd-journald.service(8)
```

So, it's a command to get systemd logs and it uses `systemd-journald.service`

## What is systemd-journal service

systemd-journald is a system service that collects and stores logging data. It creates and maintains structured, indexed journals based on logging information that is received from a variety of sources. These are can be message types;

- Kernel Messages
- Simple Log Messages
- Audit records

There are a lot of messages you can find.

## Let's See Some journalctl Commands

If you're using `journalctl` without any parameters it will show full output;

```bash
journalctl
```

**The output**

```output
-- Logs begin at Sat 2020-01-18 21:00:40 +03, end at Sat 2020-05-09 10:47:50 +03
Jan 18 21:00:40 opcode kernel: microcode: microcode updated early to revision 0x
Jan 18 21:00:40 opcode kernel: Linux version 5.3.0-26-generic (buildd@lgw01-amd6
Jan 18 21:00:40 opcode kernel: Command line: BOOT_IMAGE=/boot/vmlinuz-5.3.0-26-g
Jan 18 21:00:40 opcode kernel: KERNEL supported cpus:
Jan 18 21:00:40 opcode kernel:   Intel GenuineIntel
```

You can get json output in pretty format

```bash
journalctl -o json-pretty
```

**The output**

```json
{
        "__CURSOR" : "s=a62023d453d2404c949ce66c81b4b97a;i=1;b=5bf547fda1f147129
        "__REALTIME_TIMESTAMP" : "1579370440473152",
        "__MONOTONIC_TIMESTAMP" : "5696941",
        "_BOOT_ID" : "5bf547fda1f147129ac28544e4d1b35f",
        "_SOURCE_MONOTONIC_TIMESTAMP" : "0",
        "_TRANSPORT" : "kernel",
        "PRIORITY" : "6",
        "SYSLOG_FACILITY" : "0",
        "SYSLOG_IDENTIFIER" : "kernel",
        "MESSAGE" : "microcode: microcode updated early to revision 0x27, date =
        "_MACHINE_ID" : "35bb650aeefb48379f3b1920848e2a5a",
        "_HOSTNAME" : "opcode"
}
// more pages here
```

You can also get specific outputs for instance chrome's logs;

```bash
journalctl _COMM=chrome
```

**The output**

```output
-- Logs begin at Sat 2020-01-18 21:00:40 +03, end at Sat 2020-05-09 10:47:50 +03
Jan 20 20:49:09 opcode chrome[8566]: Failed to load module "canberra-gtk-module"
Jan 20 20:49:09 opcode chrome[8566]: Failed to load module "canberra-gtk-module"
Jan 20 20:49:10 opcode audit[8804]: AVC apparmor="DENIED" operation="sendmsg" pr
Jan 20 20:50:24 opcode chromium_chromium.desktop[8566]: [9131:1:0120/205024.3276
Jan 20 20:50:24 opcode chromium_chromium.desktop[8566]: [9131:1:0120/205024.3
```

As you see, these are the oldest messages. What about the current boot's log or specific boot's messages?

To get a list of boots, use this command;

```bash
journalctl --list-boots
```

**The output**

```output
-92 5bf547fda1f147129ac28544e4d1b35f Sat 2020-01-18 21:00:40 +03—Sat 2020-01-18 
-91 f6a4dc011a8847bb94572a02de1c8401 Sat 2020-01-18 21:25:32 +03—Sun 2020-01-19
// more than this
```

To see boot 91's message, use this command;

```bash
journalctl -b 91
```

There are many commands you should know.

## What Will We Do?

As we see journalctl useful command to understand system or application logs. But it's also hard to understand. You have to use a terminal, you have to know all commands. (In this idea you have to) but the end-user may don't want to know all commands.

We can write a parser in our best programming language. It can be a web project or another terminal project or GUI application.

### Users can filter logs between two dates

To do this idea, use this command;

```bash
journalctl -S "2020-01-01 00:00:00" -U "2020-01-02 00:00:00"
```

- **-S**: since
- **-U**: until

And search about this command

### Users can filter logs by specific services

For example, you want to see logs for apache2 use this command;

```bash
journalctl -u apache2.service
```
- **-u**: unit

### Users can filter logs by specific binary

For example, you want to see logs for chrome use this command;

```bash
journalctl _COMM=chrome
```
- **_COMM**: match for the script name is added to the query

### Users can see all boots

I'm an end-user who wants to see all boots. But it's really hard to see for me. Use this command;

```bash
journalctl --list-boots
```

### Users can see logs from different boots

For instance, we want to see the logs for boot 35, we should use this command;

```bash
journalctl -b 35
```
### Users can filter logs by priority

To filter logs by priority use this command;

```bash
journalctl -p 0
```

You can specify the number or level key.

```bash
journalctl -p crit
```

- **-p**: priority

These are log levels;

- **0**: emerg
- **1**: alert 
- **2**: crit
- **3**: err
- **4**: warning
- **5**: notice
- **6**: info"
- **7**: debug"

## Technologies

You can use various technologies to achieve this idea. For example, golang really good programming language. I believe you can do that in Python easily. I'll choose NodeJS to do that.

## EOL

Actually these are my thoughts. You can extend them. Your project will have better features than my project's features.

Sorry for the grammar mistakes.

Thanks for reading ^_^ and if there is something wrong, tell me.

## Resources

These resources helped a lot while thinking of this idea. I learned many new things. Remember that you can learn new things while thinking about something.

- https://www.howtogeek.com/499623/how-to-use-journalctl-to-read-linux-system-logs/
- https://www.linode.com/docs/quick-answers/linux/how-to-use-journalctl/
- https://www.digitalocean.com/community/tutorials/how-to-use-journalctl-to-view-and-manipulate-systemd-logs
- https://stackabuse.com/executing-shell-commands-with-node-js/
- https://unix.stackexchange.com/questions/225401/how-to-see-full-log-from-systemctl-status-service
- https://www.thegeekdiary.com/beginners-guide-to-journalctl-how-to-use-journalctl-to-view-and-manipulate-systemd-logs/
- https://coreos.com/os/docs/latest/reading-the-system-log.html
- https://net2.com/how-to-analyze-linux-systemd-logs-using-journalctl-advanced-filtering-options/
- https://www.maketecheasier.com/use-journalctl-read-linux-system-logs/
- https://www.shellhacks.com/journalctl-tail-service-logs-systemd-journal/
- https://www.freedesktop.org/software/systemd/man/journalctl.html
- https://www.golinuxcloud.com/view-logs-using-journalctl-filter-journald/
- https://www.tecmint.com/manage-systemd-logs-using-journalctl/