Repository: caddyserver/caddy (language: go)

Issue:
Basic Auth causes Caddy to use an extraordinary amount of CPU while viewing the NetData Dashboard
## 1. Environment

### 1a. Operating system and version

```
Ubuntu 22.04, but both caddy and netdata are running using docker compose.
```

### 1b. Caddy version (run `caddy version` or paste commit SHA)

```
v2.6.2 h1:wKoFIxpmOJLGl3QXoo6PNbYvGW4xLEgo32GPBEjWL8o=
```

## 2. Description

This is same issue as an old one https://github.com/caddyserver/caddy/issues/4267

### 2a. What happens (briefly explain what is wrong)

Here is what happens on caddy container restart with basic auth enabled:
![изображение](https://user-images.githubusercontent.com/1354301/215209957-9de10ca5-95be-48f5-9dda-c5fe65efbb89.png)
And here is same 1-minute window with caddy restart without basic auth:
![изображение](https://user-images.githubusercontent.com/1354301/215213865-565bcff7-fede-47a7-94aa-2094ecdb50c4.png)

### 2b. Why it's a bug (if it's not obvious)

Too high CPU usage.

### 2c. Log output

Looks like this 15-sec long high CPU load happens because of just 18 requests sent by netdata (IP/DOMAIN redacted):
```
{"level":"info","ts":1674856181.5576816,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
{"level":"warn","ts":1674856181.564568,"msg":"Caddyfile input is not formatted; run the 'caddy fmt' command to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
2023/01/27 21:49:41.590	INFO	admin	admin endpoint started	{"address": "localhost:2019", "enforce_origin": false, "origins": ["//localhost:2019", "//[::1]:2019", "//127.0.0.1:2019"]}
2023/01/27 21:49:41.593	INFO	http	server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS	{"server_name": "srv0", "https_port": 443}
2023/01/27 21:49:41.594	INFO	http	enabling automatic HTTP->HTTPS redirects	{"server_name": "srv0"}
2023/01/27 21:49:41.605	INFO	tls.cache.maintenance	started background certificate maintenance	{"cache": "0xc000439340"}
2023/01/27 21:49:41.610	INFO	http	enabling HTTP/3 listener	{"addr": ":443"}
{"level":"info","ts":1674856181.6112509,"msg":"failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size for details."}
2023/01/27 21:49:41.613	INFO	http.log	server running	{"name": "srv0", "protocols": ["h1", "h2", "h3"]}
2023/01/27 21:49:41.613	INFO	http.log	server running	{"name": "remaining_auto_https_redirects", "protocols": ["h1", "h2", "h3"]}
2023/01/27 21:49:41.613	INFO	http	enabling automatic TLS certificate management	{"domains": ["etesync.DOMAIN.TLD", "netdata.primary.DOMAIN.TLD"]}
2023/01/27 21:49:41.620	INFO	autosaved config (load with --resume flag)	{"file": "/config/caddy/autosave.json"}
2023/01/27 21:49:41.620	INFO	serving initial configuration
2023/01/27 21:49:41.620	INFO	tls	cleaning storage unit	{"description": "FileStorage:/data/caddy"}
2023/01/27 21:49:41.629	INFO	tls	finished cleaning storage units
2023/01/27 21:50:25.616	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=system.cpu&_=1674856210484&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "powerman", "duration": 14.857551031, "size": 0, "status": 0}
2023/01/27 21:50:25.633	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=system.cpu&_=1674856210427&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "powerman", "duration": 14.992071428, "size": 0, "status": 0}
2023/01/27 21:50:25.633	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=system.ram&_=1674856210451&format=array&points=300&group=average&gtime=0&options=absolute%7Cpercentage%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used%7Cbuffers%7Cactive%7Cwired"}, "user_id": "powerman", "duration": 14.983164857, "size": 0, "status": 0}
2023/01/27 21:50:25.635	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=system.net&_=1674856210477&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "powerman", "duration": 14.917745559, "size": 0, "status": 0}
2023/01/27 21:50:25.661	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=system.io&_=1674856210461&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=in"}, "user_id": "powerman", "duration": 14.996486785, "size": 0, "status": 0}
2023/01/27 21:50:25.662	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=system.net&_=1674856210467&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "powerman", "duration": 14.96954706, "size": 0, "status": 0}
2023/01/27 21:50:25.663	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=system.cpu&_=1674856210494&format=json&points=111&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300&dimensions=iowait"}, "user_id": "powerman", "duration": 14.725561648, "size": 0, "status": 0}
2023/01/27 21:50:25.669	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=system.io&_=1674856210505&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=out"}, "user_id": "powerman", "duration": 14.711465941, "size": 0, "status": 0}
2023/01/27 21:50:25.669	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=system.cpu&_=1674856210506&format=json&points=111&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300&dimensions=softirq"}, "user_id": "powerman", "duration": 14.862092634, "size": 0, "status": 0}
2023/01/27 21:50:25.676	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=system.net&_=1674856225496&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "powerman", "duration": 0.042953096, "size": 1887, "status": 200}
2023/01/27 21:50:25.678	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=system.net&_=1674856225495&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "powerman", "duration": 0.04421093, "size": 1887, "status": 200}
2023/01/27 21:50:25.681	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=system.cpu&_=1674856225440&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "powerman", "duration": 0.047251259, "size": 3653, "status": 200}
2023/01/27 21:50:25.681	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=system.ram&_=1674856225459&format=array&points=300&group=average&gtime=0&options=absolute%7Cpercentage%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used%7Cbuffers%7Cactive%7Cwired"}, "user_id": "powerman", "duration": 0.047498366, "size": 1473, "status": 200}
2023/01/27 21:50:25.681	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=system.io&_=1674856225535&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=out"}, "user_id": "powerman", "duration": 0.015849202, "size": 1522, "status": 200}
2023/01/27 21:50:25.681	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=system.cpu&_=1674856225539&format=json&points=111&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300&dimensions=softirq"}, "user_id": "powerman", "duration": 0.018101394, "size": 927, "status": 200}
2023/01/27 21:50:25.682	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=system.io&_=1674856225473&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=in"}, "user_id": "powerman", "duration": 0.048085624, "size": 1295, "status": 200}
2023/01/27 21:50:25.682	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=system.cpu&_=1674856225517&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "powerman", "duration": 0.016747384, "size": 1286, "status": 200}
2023/01/27 21:50:25.682	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=system.cpu&_=1674856225518&format=json&points=111&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300&dimensions=iowait"}, "user_id": "powerman", "duration": 0.016788965, "size": 929, "status": 200}
2023/01/27 21:50:31.924	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=system.net&_=1674856231812&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "powerman", "duration": 0.005819312, "size": 1887, "status": 200}
2023/01/27 21:50:31.928	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=system.net&_=1674856231811&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "powerman", "duration": 0.009109117, "size": 1884, "status": 200}
```


### 2d. Workaround(s)

Disabling basic auth is not a viable option, so…

I think it should be fixed by caddy avoiding hashing same password in more than one goroutine at same time.

## 3. Tutorial (minimal steps to reproduce the bug)

Caddyfile used in docker image `caddy:2.6.2`:
```
{
    email powerman@DOMAIN.TLD
    log default {
        format filter {
            wrap console
            fields {
                request>headers delete
                request>remote_port delete
                request>tls delete
                resp_headers delete
            }
        }
    }
}

netdata.primary.DOMAIN.TLD {
    basicauth {
        powerman HASH
    }

    reverse_proxy netdata:19999

    log
}
```



Discussion / hints:
And here are logs from second run (no basic auth, no high CPU usage):
```
{"level":"info","ts":1674857193.7385495,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
{"level":"warn","ts":1674857193.7581072,"msg":"Caddyfile input is not formatted; run the 'caddy fmt' command to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
2023/01/27 22:06:33.764	INFO	admin	admin endpoint started	{"address": "localhost:2019", "enforce_origin": false, "origins": ["//localhost:2019", "//[::1]:2019", "//127.0.0.1:2019"]}
2023/01/27 22:06:33.765	INFO	http	server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS	{"server_name": "srv0", "https_port": 443}
2023/01/27 22:06:33.767	INFO	http	enabling automatic HTTP->HTTPS redirects	{"server_name": "srv0"}
2023/01/27 22:06:33.770	INFO	http	enabling HTTP/3 listener	{"addr": ":443"}
{"level":"info","ts":1674857193.7712579,"msg":"failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size for details."}
2023/01/27 22:06:33.771	INFO	http.log	server running	{"name": "srv0", "protocols": ["h1", "h2", "h3"]}
2023/01/27 22:06:33.772	INFO	http.log	server running	{"name": "remaining_auto_https_redirects", "protocols": ["h1", "h2", "h3"]}
2023/01/27 22:06:33.772	INFO	http	enabling automatic TLS certificate management	{"domains": ["etesync.DOMAIN.TLD", "netdata.primary.DOMAIN.TLD"]}
2023/01/27 22:06:33.776	INFO	autosaved config (load with --resume flag)	{"file": "/config/caddy/autosave.json"}
2023/01/27 22:06:33.776	INFO	serving initial configuration
2023/01/27 22:06:33.778	INFO	tls.cache.maintenance	started background certificate maintenance	{"cache": "0xc0003ef1f0"}
2023/01/27 22:06:33.779	INFO	tls	cleaning storage unit	{"description": "FileStorage:/data/caddy"}
2023/01/27 22:06:33.783	INFO	tls	finished cleaning storage units
2023/01/27 22:06:34.543	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857190528&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.016873802, "size": 54, "status": 404}
2023/01/27 22:06:34.545	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=anomaly_detection.anomaly_detection_on_4bfab546-9cd2-11ed-9b29-0242ac150003&_=1674857190521&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.019989494, "size": 1274, "status": 200}
2023/01/27 22:06:34.546	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857190524&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.020723033, "size": 587, "status": 200}
2023/01/27 22:06:34.548	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857190529&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.021056683, "size": 54, "status": 404}
2023/01/27 22:06:34.550	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.mem_usage_limit&_=1674857190526&format=array&points=300&group=average&gtime=0&options=absolute%7Cpercentage%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.023808213, "size": 510, "status": 200}
2023/01/27 22:06:34.551	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857190534&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.017727663, "size": 54, "status": 404}
2023/01/27 22:06:34.553	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857190535&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.021783383, "size": 1487, "status": 200}
2023/01/27 22:06:34.555	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857190531&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.021717425, "size": 54, "status": 404}
2023/01/27 22:06:34.655	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857194597&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.002893958, "size": 54, "status": 404}
2023/01/27 22:06:34.705	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=anomaly_detection.anomaly_detection_on_4bfab546-9cd2-11ed-9b29-0242ac150003&_=1674857194646&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.00320833, "size": 1274, "status": 200}
2023/01/27 22:06:34.712	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857194658&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.003326472, "size": 587, "status": 200}
2023/01/27 22:06:34.712	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857194661&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.004452293, "size": 54, "status": 404}
2023/01/27 22:06:34.745	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.mem_usage_limit&_=1674857194689&format=array&points=300&group=average&gtime=0&options=absolute%7Cpercentage%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.003251551, "size": 510, "status": 200}
2023/01/27 22:06:34.752	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857194701&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.002575448, "size": 54, "status": 404}
2023/01/27 22:06:34.786	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857194726&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.003200075, "size": 1487, "status": 200}
2023/01/27 22:06:34.792	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857194736&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.002678299, "size": 54, "status": 404}
2023/01/27 22:06:35.406	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=anomaly_detection.anomaly_detection_on_4bfab546-9cd2-11ed-9b29-0242ac150003&_=1674857195339&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.003603535, "size": 1272, "status": 200}
2023/01/27 22:06:35.410	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857195350&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.008347508, "size": 587, "status": 200}
2023/01/27 22:06:35.412	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.mem_usage_limit&_=1674857195362&format=array&points=300&group=average&gtime=0&options=absolute%7Cpercentage%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.003621404, "size": 515, "status": 200}
2023/01/27 22:06:35.435	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857195376&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.00239467, "size": 54, "status": 404}
2023/01/27 22:06:35.443	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857195387&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.003077917, "size": 54, "status": 404}
2023/01/27 22:06:35.449	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857195397&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.002558217, "size": 54, "status": 404}
2023/01/27 22:06:35.481	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857195408&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.008403623, "size": 54, "status": 404}
2023/01/27 22:06:35.486	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857195417&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.014153445, "size": 1487, "status": 200}
2023/01/27 22:06:36.406	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=anomaly_detection.anomaly_detection_on_4bfab546-9cd2-11ed-9b29-0242ac150003&_=1674857196343&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.004395629, "size": 1266, "status": 200}
2023/01/27 22:06:36.415	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857196366&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.003531378, "size": 593, "status": 200}
2023/01/27 22:06:36.445	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.mem_usage_limit&_=1674857196382&format=array&points=300&group=average&gtime=0&options=absolute%7Cpercentage%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.002583208, "size": 524, "status": 200}
2023/01/27 22:06:36.447	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857196396&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.004883348, "size": 54, "status": 404}
2023/01/27 22:06:36.473	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857196422&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.003468039, "size": 54, "status": 404}
2023/01/27 22:06:36.475	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857196408&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.004067846, "size": 54, "status": 404}
2023/01/27 22:06:36.505	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857196431&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.010928318, "size": 54, "status": 404}
2023/01/27 22:06:36.508	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857196440&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.0105481, "size": 1503, "status": 200}
2023/01/27 22:06:37.395	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=anomaly_detection.anomaly_detection_on_4bfab546-9cd2-11ed-9b29-0242ac150003&_=1674857197339&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.002796509, "size": 1269, "status": 200}
2023/01/27 22:06:37.426	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857197365&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.004307892, "size": 600, "status": 200}
2023/01/27 22:06:37.428	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.mem_usage_limit&_=1674857197374&format=array&points=300&group=average&gtime=0&options=absolute%7Cpercentage%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.006848158, "size": 529, "status": 200}
2023/01/27 22:06:37.455	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857197395&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.002424421, "size": 54, "status": 404}
2023/01/27 22:06:37.456	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857197403&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.004466555, "size": 54, "status": 404}
2023/01/27 22:06:37.475	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857197421&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.003091015, "size": 54, "status": 404}
2023/01/27 22:06:37.482	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857197429&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.002963266, "size": 54, "status": 404}
2023/01/27 22:06:37.519	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857197459&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.005963655, "size": 1511, "status": 200}
2023/01/27 22:06:38.411	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=anomaly_detection.anomaly_detection_on_4bfab546-9cd2-11ed-9b29-0242ac150003&_=1674857198339&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.0029393, "size": 1271, "status": 200}
2023/01/27 22:06:38.423	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.mem_usage_limit&_=1674857198375&format=array&points=300&group=average&gtime=0&options=absolute%7Cpercentage%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.002938519, "size": 534, "status": 200}
2023/01/27 22:06:38.457	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857198396&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.004131537, "size": 54, "status": 404}
2023/01/27 22:06:38.459	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857198403&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.006631539, "size": 54, "status": 404}
2023/01/27 22:06:38.488	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857198422&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.005221857, "size": 54, "status": 404}
2023/01/27 22:06:38.491	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857198430&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.00907257, "size": 54, "status": 404}
2023/01/27 22:06:38.546	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857198483&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.002972665, "size": 1513, "status": 200}
2023/01/27 22:06:38.772	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857198366&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.002998192, "size": 609, "status": 200}
2023/01/27 22:06:39.396	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=anomaly_detection.anomaly_detection_on_4bfab546-9cd2-11ed-9b29-0242ac150003&_=1674857199339&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.003698237, "size": 1262, "status": 200}
2023/01/27 22:06:39.505	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857199365&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.00254393, "size": 617, "status": 200}
2023/01/27 22:06:39.507	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.mem_usage_limit&_=1674857199399&format=array&points=300&group=average&gtime=0&options=absolute%7Cpercentage%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.004838995, "size": 540, "status": 200}
2023/01/27 22:06:40.729	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=anomaly_detection.anomaly_detection_on_4bfab546-9cd2-11ed-9b29-0242ac150003&_=1674857200644&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.026703288, "size": 1274, "status": 200}
2023/01/27 22:06:40.730	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857200645&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.026595035, "size": 623, "status": 200}
2023/01/27 22:06:40.735	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.mem_usage_limit&_=1674857200647&format=array&points=300&group=average&gtime=0&options=absolute%7Cpercentage%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.032090689, "size": 545, "status": 200}
2023/01/27 22:06:40.735	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857200648&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.033319291, "size": 54, "status": 404}
2023/01/27 22:06:40.739	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857200652&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.018869709, "size": 54, "status": 404}
2023/01/27 22:06:40.739	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857200649&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.019248098, "size": 54, "status": 404}
2023/01/27 22:06:40.739	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857200650&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.019194463, "size": 54, "status": 404}
2023/01/27 22:06:40.739	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857200653&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.020260692, "size": 1531, "status": 200}
2023/01/27 22:06:41.706	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=anomaly_detection.anomaly_detection_on_4bfab546-9cd2-11ed-9b29-0242ac150003&_=1674857201643&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.003190303, "size": 1271, "status": 200}
2023/01/27 22:06:41.708	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857201651&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.005317572, "size": 622, "status": 200}
2023/01/27 22:06:41.713	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.mem_usage_limit&_=1674857201659&format=array&points=300&group=average&gtime=0&options=absolute%7Cpercentage%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.002966939, "size": 544, "status": 200}
2023/01/27 22:06:41.714	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857201664&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.004505234, "size": 54, "status": 404}
2023/01/27 22:06:41.717	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857201670&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.002479455, "size": 54, "status": 404}
2023/01/27 22:06:41.726	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857201677&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.002994047, "size": 54, "status": 404}
2023/01/27 22:06:41.731	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857201682&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.002154518, "size": 54, "status": 404}
2023/01/27 22:06:41.740	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857201689&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.003415677, "size": 1534, "status": 200}
2023/01/27 22:06:42.706	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=anomaly_detection.anomaly_detection_on_4bfab546-9cd2-11ed-9b29-0242ac150003&_=1674857202644&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.003814392, "size": 1272, "status": 200}
2023/01/27 22:06:42.709	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857202652&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.006616264, "size": 627, "status": 200}
2023/01/27 22:06:42.713	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.mem_usage_limit&_=1674857202661&format=array&points=300&group=average&gtime=0&options=absolute%7Cpercentage%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.003771147, "size": 548, "status": 200}
2023/01/27 22:06:42.720	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857202669&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.003749291, "size": 54, "status": 404}
2023/01/27 22:06:42.727	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857202674&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.00343937, "size": 54, "status": 404}
2023/01/27 22:06:42.733	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857202681&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.003234031, "size": 54, "status": 404}
2023/01/27 22:06:42.742	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857202687&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.003657111, "size": 54, "status": 404}
2023/01/27 22:06:42.744	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857202693&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.005698081, "size": 1540, "status": 200}
2023/01/27 22:06:43.706	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857203656&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.003876574, "size": 628, "status": 200}
2023/01/27 22:06:43.708	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=anomaly_detection.anomaly_detection_on_4bfab546-9cd2-11ed-9b29-0242ac150003&_=1674857203643&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.005120327, "size": 1273, "status": 200}
2023/01/27 22:06:43.731	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.mem_usage_limit&_=1674857203668&format=array&points=300&group=average&gtime=0&options=absolute%7Cpercentage%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.004017128, "size": 554, "status": 200}
2023/01/27 22:06:43.736	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857203691&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.004432244, "size": 54, "status": 404}
2023/01/27 22:06:43.736	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857203684&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.004271618, "size": 54, "status": 404}
2023/01/27 22:06:43.737	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857203673&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.0120167, "size": 54, "status": 404}
2023/01/27 22:06:43.768	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857203709&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.004613788, "size": 54, "status": 404}
2023/01/27 22:06:43.769	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857203716&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.006404427, "size": 1540, "status": 200}
2023/01/27 22:06:44.708	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=anomaly_detection.anomaly_detection_on_4bfab546-9cd2-11ed-9b29-0242ac150003&_=1674857204644&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.004865753, "size": 1274, "status": 200}
2023/01/27 22:06:44.710	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857204656&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.007906448, "size": 631, "status": 200}
2023/01/27 22:06:44.727	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.mem_usage_limit&_=1674857204668&format=array&points=300&group=average&gtime=0&options=absolute%7Cpercentage%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.004123232, "size": 553, "status": 200}
2023/01/27 22:06:44.727	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857204674&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.005293231, "size": 54, "status": 404}
2023/01/27 22:06:44.731	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857204685&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.002464728, "size": 54, "status": 404}
2023/01/27 22:06:44.738	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857204690&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.00298238, "size": 54, "status": 404}
2023/01/27 22:06:44.772	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857204709&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.006741091, "size": 54, "status": 404}
2023/01/27 22:06:44.773	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857204715&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.010064841, "size": 1548, "status": 200}
2023/01/27 22:06:45.708	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=anomaly_detection.anomaly_detection_on_4bfab546-9cd2-11ed-9b29-0242ac150003&_=1674857205644&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.004117897, "size": 1275, "status": 200}
2023/01/27 22:06:45.709	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857205656&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.006277861, "size": 636, "status": 200}
2023/01/27 22:06:45.714	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.mem_usage_limit&_=1674857205668&format=array&points=300&group=average&gtime=0&options=absolute%7Cpercentage%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.002483206, "size": 553, "status": 200}
2023/01/27 22:06:45.721	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857205674&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.00235675, "size": 54, "status": 404}
2023/01/27 22:06:45.746	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857205684&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.003455993, "size": 54, "status": 404}
2023/01/27 22:06:45.746	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857205690&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.004642546, "size": 54, "status": 404}
2023/01/27 22:06:45.767	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857205709&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.003966699, "size": 54, "status": 404}
2023/01/27 22:06:45.767	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857205715&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.004994952, "size": 1549, "status": 200}
2023/01/27 22:06:46.702	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=anomaly_detection.anomaly_detection_on_4bfab546-9cd2-11ed-9b29-0242ac150003&_=1674857206645&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.003536249, "size": 1275, "status": 200}
2023/01/27 22:06:46.708	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857206658&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.002089335, "size": 637, "status": 200}
2023/01/27 22:06:46.717	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.mem_usage_limit&_=1674857206669&format=array&points=300&group=average&gtime=0&options=absolute%7Cpercentage%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.004395815, "size": 557, "status": 200}
2023/01/27 22:06:46.737	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857206685&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.004361243, "size": 54, "status": 404}
2023/01/27 22:06:46.741	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857206677&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.007926892, "size": 54, "status": 404}
2023/01/27 22:06:46.743	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857206690&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.003517074, "size": 54, "status": 404}
2023/01/27 22:06:46.778	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857206714&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.004732342, "size": 54, "status": 404}
2023/01/27 22:06:46.779	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857206721&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.006095915, "size": 1549, "status": 200}
2023/01/27 22:06:47.706	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=anomaly_detection.anomaly_detection_on_4bfab546-9cd2-11ed-9b29-0242ac150003&_=1674857207645&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.003787906, "size": 1276, "status": 200}
2023/01/27 22:06:47.708	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857207656&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.006549881, "size": 643, "status": 200}
2023/01/27 22:06:47.716	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.mem_usage_limit&_=1674857207668&format=array&points=300&group=average&gtime=0&options=absolute%7Cpercentage%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.002991965, "size": 563, "status": 200}
2023/01/27 22:06:47.731	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857207678&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.004254929, "size": 54, "status": 404}
2023/01/27 22:06:47.739	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857207685&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.004799541, "size": 54, "status": 404}
2023/01/27 22:06:47.745	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857207691&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.003943605, "size": 54, "status": 404}
2023/01/27 22:06:47.776	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857207720&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.004162079, "size": 1554, "status": 200}
2023/01/27 22:06:47.778	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857207712&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.004739324, "size": 54, "status": 404}
2023/01/27 22:06:48.705	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=anomaly_detection.anomaly_detection_on_4bfab546-9cd2-11ed-9b29-0242ac150003&_=1674857208645&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.003364284, "size": 1275, "status": 200}
2023/01/27 22:06:48.711	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857208658&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.0020949, "size": 646, "status": 200}
2023/01/27 22:06:48.718	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.mem_usage_limit&_=1674857208669&format=array&points=300&group=average&gtime=0&options=absolute%7Cpercentage%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.002854553, "size": 568, "status": 200}
2023/01/27 22:06:48.730	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857208679&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.002809946, "size": 54, "status": 404}
2023/01/27 22:06:48.736	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857208686&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.002395452, "size": 54, "status": 404}
2023/01/27 22:06:48.743	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857208692&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.002628188, "size": 54, "status": 404}
2023/01/27 22:06:48.777	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857208713&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.003489238, "size": 54, "status": 404}
2023/01/27 22:06:48.777	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857208720&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.004943911, "size": 1560, "status": 200}
2023/01/27 22:06:49.706	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=anomaly_detection.anomaly_detection_on_4bfab546-9cd2-11ed-9b29-0242ac150003&_=1674857209646&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.003791295, "size": 1272, "status": 200}
2023/01/27 22:06:49.707	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857209656&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.004400378, "size": 647, "status": 200}
2023/01/27 22:06:49.715	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.mem_usage_limit&_=1674857209668&format=array&points=300&group=average&gtime=0&options=absolute%7Cpercentage%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.002660121, "size": 569, "status": 200}
2023/01/27 22:06:49.736	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857209688&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.003868641, "size": 54, "status": 404}
2023/01/27 22:06:49.736	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857209678&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.003501623, "size": 54, "status": 404}
2023/01/27 22:06:49.741	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857209694&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.002246172, "size": 54, "status": 404}
2023/01/27 22:06:49.776	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857209712&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.002352435, "size": 54, "status": 404}
2023/01/27 22:06:49.778	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857209721&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.005057668, "size": 1558, "status": 200}
2023/01/27 22:06:50.706	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=anomaly_detection.anomaly_detection_on_4bfab546-9cd2-11ed-9b29-0242ac150003&_=1674857210645&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.00309523, "size": 1274, "status": 200}
2023/01/27 22:06:50.706	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857210656&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.0039917, "size": 648, "status": 200}
2023/01/27 22:06:50.729	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.mem_usage_limit&_=1674857210668&format=array&points=300&group=average&gtime=0&options=absolute%7Cpercentage%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.006102861, "size": 577, "status": 200}
2023/01/27 22:06:50.736	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857210686&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.003148456, "size": 54, "status": 404}
2023/01/27 22:06:50.746	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857210678&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.012710194, "size": 54, "status": 404}
2023/01/27 22:06:50.749	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857210693&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.007187387, "size": 54, "status": 404}
2023/01/27 22:06:50.776	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857210713&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.00379729, "size": 54, "status": 404}
2023/01/27 22:06:50.777	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857210720&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.004774085, "size": 1559, "status": 200}
2023/01/27 22:06:51.706	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=anomaly_detection.anomaly_detection_on_4bfab546-9cd2-11ed-9b29-0242ac150003&_=1674857211646&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.003561227, "size": 1264, "status": 200}
2023/01/27 22:06:51.713	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857211662&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.004033434, "size": 648, "status": 200}
2023/01/27 22:06:51.723	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.mem_usage_limit&_=1674857211672&format=array&points=300&group=average&gtime=0&options=absolute%7Cpercentage%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.00352509, "size": 581, "status": 200}
2023/01/27 22:06:51.728	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857211678&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.00224675, "size": 54, "status": 404}
2023/01/27 22:06:51.736	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857211686&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.002583182, "size": 54, "status": 404}
2023/01/27 22:06:51.742	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857211694&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.002398049, "size": 54, "status": 404}
2023/01/27 22:06:51.776	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857211721&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.003651818, "size": 1558, "status": 200}
2023/01/27 22:06:51.778	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857211713&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.004306317, "size": 54, "status": 404}
2023/01/27 22:06:52.706	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=anomaly_detection.anomaly_detection_on_4bfab546-9cd2-11ed-9b29-0242ac150003&_=1674857212646&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.003354422, "size": 1264, "status": 200}
2023/01/27 22:06:52.713	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857212662&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.003282463, "size": 651, "status": 200}
2023/01/27 22:06:52.722	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.mem_usage_limit&_=1674857212672&format=array&points=300&group=average&gtime=0&options=absolute%7Cpercentage%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.00342175, "size": 585, "status": 200}
2023/01/27 22:06:52.729	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857212678&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.003385684, "size": 54, "status": 404}
2023/01/27 22:06:52.739	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857212686&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.006028412, "size": 54, "status": 404}
2023/01/27 22:06:52.745	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857212694&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.004413676, "size": 54, "status": 404}
2023/01/27 22:06:52.778	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857212713&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.004123185, "size": 54, "status": 404}
2023/01/27 22:06:52.778	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857212721&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.00582408, "size": 1564, "status": 200}
2023/01/27 22:06:53.705	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=anomaly_detection.anomaly_detection_on_4bfab546-9cd2-11ed-9b29-0242ac150003&_=1674857213646&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.002759679, "size": 1269, "status": 200}
2023/01/27 22:06:53.712	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857213661&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.002753794, "size": 655, "status": 200}
2023/01/27 22:06:53.721	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.mem_usage_limit&_=1674857213672&format=array&points=300&group=average&gtime=0&options=absolute%7Cpercentage%7Cjsonwrap%7Cnonzero&after=-300&dimensions=used"}, "user_id": "", "duration": 0.002951462, "size": 589, "status": 200}
2023/01/27 22:06:53.727	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857213679&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.002608213, "size": 54, "status": 404}
2023/01/27 22:06:53.747	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857213694&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=received"}, "user_id": "", "duration": 0.004563211, "size": 54, "status": 404}
2023/01/27 22:06:53.747	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth0&_=1674857213686&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.004204536, "size": 54, "status": 404}
2023/01/27 22:06:53.775	ERROR	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.net_eth1&_=1674857213713&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=sent"}, "user_id": "", "duration": 0.002304011, "size": 54, "status": 404}
2023/01/27 22:06:53.777	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=cgroup_caddy.cpu_limit&_=1674857213721&format=json&points=262&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-300"}, "user_id": "", "duration": 0.004966178, "size": 1568, "status": 200}
```

> `{"level":"info","ts":1674857193.7712579,"msg":"failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size for details."}`

Did you try to do this? It should help with performance over HTTP/3.

> `2023/01/27 21:50:25.661	INFO	http.log.access	handled request	{"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/3.0", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/api/v1/data?chart=system.io&_=1674856210461&format=array&points=300&group=average&gtime=0&options=absolute%7Cjsonwrap%7Cnonzero&after=-300&dimensions=in"}, "user_id": "powerman", "duration": 14.996486785, "size": 0, "status": 0}`

Requests are taking 15 seconds :thinking: that seems way too long.

Can you post your logs _without_ removing the HTTP headers? That's relevant to understanding how the requests look and give us more clues.
The problem isn't in headers, it's a slow hashing algo.

Here is I'm making request using `httpie` (wget/curl-like) tool after caddy restart to non-existing url. Netdata isn't open in browser, so it doesn't send it's own requests, so my request comes first. I sent 3 requests, 1st without auth, 2nd and 3rd with auth. It's clear 2nd request takes **extra 2.2s** to process compared to 3rd - I believe this is password hashing time:

```
2023/01/28 09:37:56.227 ERROR   http.log.access handled request {"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/1.1", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/nosuch"}, "user_id": "", "duration": 0.004400888, "size": 0, "status": 401}
2023/01/28 09:38:03.031 ERROR   http.log.access handled request {"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/1.1", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/nosuch"}, "user_id": "powerman", "duration": 2.205908112, "size": 93, "status": 404}
2023/01/28 09:38:07.890 ERROR   http.log.access handled request {"request": {"remote_ip": "1.2.3.4", "proto": "HTTP/1.1", "method": "GET", "host": "netdata.primary.DOMAIN.TLD", "uri": "/nosuch"}, "user_id": "powerman", "duration": 0.003090165, "size": 93, "status": 404}
```

Here are headers (from httpie output) for 2nd request (password/auth base64/domain redacted):
```
$ http -v --auth powerman:passwordpassword https://netdata.primary.DOMAIN.TLD/nosuch
GET /nosuch HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Authorization: Basic BASE64BASE64BASE64BASE64BASE64BASE64
Connection: keep-alive
Host: netdata.primary.DOMAIN.TLD
User-Agent: HTTPie/2.6.0



HTTP/1.1 404 Not Found
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Alt-Svc: h3=":443"; ma=2592000
Cache-Control: no-cache, no-store, must-revalidate
Content-Encoding: gzip
Content-Type: text/html; charset=utf-8
Date: Sat, 28 Jan 2023 09:38:03 UTC
Expires: Sat, 28 Jan 2023 09:38:04 UTC
Pragma: no-cache
Server: Caddy, Netdata Embedded HTTP Server v1.37.1
Transfer-Encoding: chunked

File does not exist, or is not accessible: &#x2F;usr&#x2F;share&#x2F;netdata&#x2F;web&#x2F;nosuch
```

Netdata sends 18 such requests at once, so if processing 1 takes 2.2s then no surprise parallel handling of 18 takes about 30s.
Actually, it's easy to prove the time is spend for hashing and has nothing with HTTP or headers:
```
/srv # time caddy hash-password --plaintext passwordpassword
$2a$14$dWI/7EzbFr7kWYCXptRbAefkWfYOihRFgAbk.bzxlvoiaJ5qahJba
real    0m 2.18s
user    0m 1.41s
sys     0m 0.05s
```
It may worth to note this is running on small DigitalOcean instance ($6/mo, 1 shared vCPU), so hashing speed about 2s per 1 hash may be normal. The issue is in handling parallel requests which cames with same password - processing them all should took about 2s, not 2s multiple amount of parallel requests.
Why I want to see the headers is that 15 seconds is way higher than the expected ~300ms it should take for password hashing. Is the HTTP request causing a single request to attempt hashing multiple times for some reason? That's what I want to figure out. To replicate the exact scenario from your logs, we need the HTTP headers to compare and replicate it using curl.

Plus the hash results should be cached, so only the first request with the same password should take a long time; see https://github.com/caddyserver/caddy/pull/3465 is something in the headers causing the hash cache to not work correctly?

Your hashed password is also likely using too high a work factor for your hardware. Try rehashing the password with a bcrypt number like 12. The default is set to 14 (which I don't agree with FWIW, I do want to lower it).
Oh, I think I see what's going on now.

The possible explanation is that multiple requests were fired all at once, and since the password was not cached yet -- the first request was still in process of hashing, and it only gets added to the cache once it's done -- they all tried to hash the same password in parallel.

If only a single request was made up-front with the password first, then it should only try once to hash, and subsequent requests should hit the cache.

@mholt would it makes sense to like, "lock" on a hashing job, so a subsequent request can check if there's another in-flight request hashing the same string? Then once the first is done they can just use the result from cache. Would avoid the increased load from doing it in parallel on an underpowered VM.

Fix patch:
diff --git a/go.mod b/go.mod
index b762aee6e72..63cf4f2def0 100644
--- a/go.mod
+++ b/go.mod
@@ -32,6 +32,7 @@ require (
 	go.uber.org/zap v1.23.0
 	golang.org/x/crypto v0.4.0
 	golang.org/x/net v0.5.0
+	golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4
 	golang.org/x/term v0.4.0
 	google.golang.org/genproto v0.0.0-20221027153422-115e99e71e1c
 	gopkg.in/natefinch/lumberjack.v2 v2.0.0
diff --git a/modules/caddyhttp/caddyauth/basicauth.go b/modules/caddyhttp/caddyauth/basicauth.go
index eb6fd5928d2..f515a72c65a 100644
--- a/modules/caddyhttp/caddyauth/basicauth.go
+++ b/modules/caddyhttp/caddyauth/basicauth.go
@@ -26,6 +26,7 @@ import (
 	"time"
 
 	"github.com/caddyserver/caddy/v2"
+	"golang.org/x/sync/singleflight"
 )
 
 func init() {
@@ -142,6 +143,7 @@ func (hba *HTTPBasicAuth) Provision(ctx caddy.Context) error {
 	if hba.HashCache != nil {
 		hba.HashCache.cache = make(map[string]bool)
 		hba.HashCache.mu = new(sync.RWMutex)
+		hba.HashCache.g = new(singleflight.Group)
 	}
 
 	return nil
@@ -190,12 +192,18 @@ func (hba HTTPBasicAuth) correctPassword(account Account, plaintextPassword []by
 	if ok {
 		return same, nil
 	}
-
 	// slow track: do the expensive op, then add it to the cache
-	same, err := compare()
+	// but perform it in a singleflight group so that multiple
+	// parallel requests using the same password don't cause a
+	// thundering herd problem by all performing the same hashing
+	// operation before the first one finishes and caches it.
+	v, err, _ := hba.HashCache.g.Do(cacheKey, func() (any, error) {
+		return compare()
+	})
 	if err != nil {
 		return false, err
 	}
+	same = v.(bool)
 	hba.HashCache.mu.Lock()
 	if len(hba.HashCache.cache) >= 1000 {
 		hba.HashCache.makeRoom() // keep cache size under control
@@ -223,6 +231,7 @@ func (hba HTTPBasicAuth) promptForCredentials(w http.ResponseWriter, err error)
 // compute on every HTTP request.
 type Cache struct {
 	mu *sync.RWMutex
+	g  *singleflight.Group
 
 	// map of concatenated hashed password + plaintext password + salt, to result
 	cache map[string]bool


Test patch: