journalを別logとjsonに出力する

## この記事について

journalをログに保存するshellscriptの自分用テンプレです。

journalってそのままログ…なんですが別のログにしたいことがあるのでスクリプトを作りました。

## 別logファイル出力

手元に残るscriptをテンプレ代わりに置きます。ここにgrepでフィルターかけて使うことが多いです。

こんなことしなくても一行で書けるだろう…おっしゃる通りです。ちょっと変形して使ったりできるので一応残してあります。

```sh:journal_csv.sh
#!/bin/bash

datetime=$(date '+%Y%m%d-%H%M%S')
# CSVファイルのパス
CSV_FILE="Journal_logs_${datetime}.log"


# journalctl コマンドで最新のログを取得する関数
get_latest_journal() {
    journalctl -n 30 --no-pager
}

# CSVファイルにヘッダーを書き込む（必要なら）
echo "journal" > $CSV_FILE

# 最初のログスナップショットを取得
latest_journal=$(get_latest_journal)

# 無限ループでログの変更を監視
while true; do
    current_journal=$(get_latest_journal)
    if [ "$latest_journal" != "$current_journal" ]; then
        # 一時ファイルに現在のログを一行ずつ追加
        echo "$current_journal" | while IFS= read -r line; do
            # 空行はスキップ
            [ -z "$line" ] && continue
            # 重複行があれば除外
            grep -vFx "$line" $CSV_FILE > temp.csv
            echo "$line" >> temp.csv
            mv temp.csv $CSV_FILE
        done

        latest_journal=$current_journal
    fi
    sleep 0.2
done
```

普通にこんな感じで出力されます。

<img width= 700 src =https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/565396/37af8cc5-6987-6c79-5a3b-b8ba87c332ce.png>

## json出力

json出力です。
色々と考えることがありjson版を最初に作りました。csv出力はおまけです。

```sh:journal_json.sh
#!/bin/bash

THING_NAME=$(uname -n)
# MQTTのパラメータ
MQTT_HOST="127.0.0.1"
MQTT_TOPIC=${THING_NAME}_mos/${THING_NAME}_logs/Journal
echo $MQTT_TOPIC

# journalctl コマンドで最新のログを取得する関数
get_latest_journal() {
    journalctl -n 30 --no-pager
}

# 最初のログスナップショットを取得
latest_journal=$(get_latest_journal)

# 無限ループでログの変更を監視
while true; do
    current_journal=$(get_latest_journal)
    if [ "$latest_journal" != "$current_journal" ]; then
        # 一時ファイルに現在のログを一行ずつ追加
        echo "$current_journal" | while IFS= read -r line; do
            # 空行はスキップ
            [ -z "$line" ] && continue
            # 重複行があれば除外
            if ! grep -Fxq "$line" <<< "$latest_journal"; then
                timestamp=$(date '+%Y-%m-%dT%H:%M:%S')
                json_payload=$(jq -n --arg timestamp "$timestamp" --arg line "$line" '{($timestamp): $line}')
                echo $json_payload
                mosquitto_pub -h "$MQTT_HOST" -t "$MQTT_TOPIC" -m "$json_payload"
            fi
        done

        latest_journal=$current_journal
    fi
    sleep 0.2
done
```

DEMOとしてlocalhostでpubsubです。

```sh:pub
~ $ ./journal_json.sh
{ "2024-07-09T21:44:26": "Jul 09 21:44:25 VNCcamServer1 systemd[1]: Stopped suricata.service - Suricata IDS/IDP daemon." }
{ "2024-07-09T21:44:26": "Jul 09 21:44:25 VNCcamServer1 systemd[1]: suricata.service: Consumed 7.155s CPU time." }
{ "2024-07-09T21:44:26": "Jul 09 21:44:25 VNCcamServer1 systemd[1]: Starting suricata.service - Suricata IDS/IDP daemon..." }
{ "2024-07-09T21:44:26": "Jul 09 21:44:25 VNCcamServer1 suricata[24596]: i: suricata: This is Suricata version 7.0.6 RELEASE running in SYSTEM mode" }
{ "2024-07-09T21:44:26": "Jul 09 21:44:25 VNCcamServer1 suricata[24596]: W: ioctl: Failure when trying to get MTU via ioctl for 'eth1': No such device (19)" }
{ "2024-07-09T21:44:26": "Jul 09 21:44:25 VNCcamServer1 systemd[1]: Started suricata.service - Suricata IDS/IDP daemon." }
{ "2024-07-09T21:44:27": "Jul 09 21:44:26 VNCcamServer1 kernel: brcmfmac mmc1:0001:1 wlan0: entered promiscuous mode" }
{ "2024-07-09T21:44:35": "Jul 09 21:44:35 VNCcamServer1 kernel: iptables denied: IN=wlan0 OUT= MAC=ff:ff:ff:ff:ff:ff:40:ae:30:62:62:8d:08:00 SRC=192.168.11.1 DST=192.168.11.255 LEN=397 TOS=0x00 PREC=0x00 TTL=64 ID=54160 DF PROTO=UDP SPT=60161 DPT=20002 LEN=377 " }
{ "2024-07-09T21:44:40": "Jul 09 21:44:39 VNCcamServer1 kernel: iptables denied: IN=wlan0 OUT= MAC=ff:ff:ff:ff:ff:ff:40:ae:30:62:62:8d:08:00 SRC=192.168.11.1 DST=192.168.11.255 LEN=397 TOS=0x00 PREC=0x00 TTL=64 ID=54974 DF PROTO=UDP SPT=34506 DPT=20002 LEN=377 " }
{ "2024-07-09T21:45:05": "Jul 09 21:45:05 VNCcamServer1 kernel: iptables denied: IN=wlan0 OUT= MAC=ff:ff:ff:ff:ff:ff:40:ae:30:62:62:8d:08:00 SRC=192.168.11.1 DST=192.168.11.255 LEN=397 TOS=0x00 PREC=0x00 TTL=64 ID=57684 DF PROTO=UDP SPT=51973 DPT=20002 LEN=377 " }
{ "2024-07-09T21:45:10": "Jul 09 21:45:09 VNCcamServer1 kernel: iptables denied: IN=wlan0 OUT= MAC=ff:ff:ff:ff:ff:ff:40:ae:30:62:62:8d:08:00 SRC=192.168.11.1 DST=192.168.11.255 LEN=397 TOS=0x00 PREC=0x00 TTL=64 ID=58091 DF PROTO=UDP SPT=49818 DPT=20002 LEN=377 " }
```

```sh:sub
~ $ mosquitto_sub -h 127.0.0.1 -t "VNCcamServer1_mos/VNCcamServer1_logs/Journal" -v
 "2024-07-09T21:44:26": "Jul 09 21:44:25 VNCcamServer1 systemd[1]: Stopped suricata.service - Suricata IDS/IDP daemon."
}
VNCcamServer1_mos/VNCcamServer1_logs/Journal {
  "2024-07-09T21:44:26": "Jul 09 21:44:25 VNCcamServer1 systemd[1]: suricata.service: Consumed 7.155s CPU time."
}
VNCcamServer1_mos/VNCcamServer1_logs/Journal {
  "2024-07-09T21:44:26": "Jul 09 21:44:25 VNCcamServer1 systemd[1]: Starting suricata.service - Suricata IDS/IDP daemon..."
}
VNCcamServer1_mos/VNCcamServer1_logs/Journal {
  "2024-07-09T21:44:26": "Jul 09 21:44:25 VNCcamServer1 suricata[24596]: i: suricata: This is Suricata version 7.0.6 RELEASE running in SYSTEM mode"
}
VNCcamServer1_mos/VNCcamServer1_logs/Journal {
  "2024-07-09T21:44:26": "Jul 09 21:44:25 VNCcamServer1 suricata[24596]: W: ioctl: Failure when trying to get MTU via ioctl for 'eth1': No such device (19)"
}
VNCcamServer1_mos/VNCcamServer1_logs/Journal {
  "2024-07-09T21:44:26": "Jul 09 21:44:25 VNCcamServer1 systemd[1]: Started suricata.service - Suricata IDS/IDP daemon."
}
VNCcamServer1_mos/VNCcamServer1_logs/Journal {
  "2024-07-09T21:44:27": "Jul 09 21:44:26 VNCcamServer1 kernel: brcmfmac mmc1:0001:1 wlan0: entered promiscuous mode"
}
VNCcamServer1_mos/VNCcamServer1_logs/Journal {
  "2024-07-09T21:44:35": "Jul 09 21:44:35 VNCcamServer1 kernel: iptables denied: IN=wlan0 OUT= MAC=ff:ff:ff:ff:ff:ff:40:ae:30:62:62:8d:08:00 SRC=192.168.11.1 DST=192.168.11.255 LEN=397 TOS=0x00 PREC=0x00 TTL=64 ID=54160 DF PROTO=UDP SPT=60161 DPT=20002 LEN=377 "
}
VNCcamServer1_mos/VNCcamServer1_logs/Journal {
  "2024-07-09T21:44:40": "Jul 09 21:44:39 VNCcamServer1 kernel: iptables denied: IN=wlan0 OUT= MAC=ff:ff:ff:ff:ff:ff:40:ae:30:62:62:8d:08:00 SRC=192.168.11.1 DST=192.168.11.255 LEN=397 TOS=0x00 PREC=0x00 TTL=64 ID=54974 DF PROTO=UDP SPT=34506 DPT=20002 LEN=377 "
}
VNCcamServer1_mos/VNCcamServer1_logs/Journal {
  "2024-07-09T21:45:05": "Jul 09 21:45:05 VNCcamServer1 kernel: iptables denied: IN=wlan0 OUT= MAC=ff:ff:ff:ff:ff:ff:40:ae:30:62:62:8d:08:00 SRC=192.168.11.1 DST=192.168.11.255 LEN=397 TOS=0x00 PREC=0x00 TTL=64 ID=57684 DF PROTO=UDP SPT=51973 DPT=20002 LEN=377 "
}
VNCcamServer1_mos/VNCcamServer1_logs/Journal {
  "2024-07-09T21:45:10": "Jul 09 21:45:09 VNCcamServer1 kernel: iptables denied: IN=wlan0 OUT= MAC=ff:ff:ff:ff:ff:ff:40:ae:30:62:62:8d:08:00 SRC=192.168.11.1 DST=192.168.11.255 LEN=397 TOS=0x00 PREC=0x00 TTL=64 ID=58091 DF PROTO=UDP SPT=49818 DPT=20002 LEN=377 "
}
```