Project: Grafana Loki (Observability and Analysis / Observability)
File: drop.md

title: drop menuTitle: description: The 'drop' Promtail pipeline stage. aliases: ../../../clients/promtail/stages/drop/ weight: The `drop` stage is a filtering stage that lets you drop logs based on several options. It's important to note that if you provide multiple options they will be treated like an AND clause, where each option has to be true to drop the log. If you wish to drop with an OR clause, then specify multiple drop stages. There are examples below to help explain. ```yaml drop: [source: [<string>] | <string>] [separator: <string> | default = ";"] [expression: <string>] [value: <string>] [older_than: <duration>] [longer_than: <string>|<int>] [dropcounterreason: <string> | default = "drop_stage"] ``` The following are examples showing the use of the `drop` stage. Simple `drop` stage configurations only specify one of the options, or two options when using the `source` option. Given the pipeline: ```yaml drop: source: ["level","msg"] ``` Drops any log line that has an extracted data field of at least `level` or `msg`. This example pipeline drops any log line with the substring "debug" in it: ```yaml drop: expression: ".debug." ``` Given the pipeline: ```yaml json: expressions: level: msg: drop: source: ["level","msg"] separator: "#" expression: "(error|ERROR)#.\/loki\/api\/push." ``` Drops both of these log lines: ``` {"time":"2019-01-01T01:00:00.000000001Z", "level": "error", "msg":"11.11.11.11 - "POST /loki/api/push/ HTTP/1.1" 200 932 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 GTB6"} {"time":"2019-01-01T01:00:00.000000001Z", "level": "ERROR", "msg":"11.11.11.11 - "POST /loki/api/push/ HTTP/1.1" 200 932 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 GTB6"} ``` Given the pipeline: ```yaml json: expressions: level: msg: drop: source: "level" value: "error" ``` Would drop this log line: ``` {"time":"2019-01-01T01:00:00.000000001Z", "level": "error", "msg":"11.11.11.11 - "POST /loki/api/push/ HTTP/1.1" 200 932 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 GTB6"} ``` {{% admonition type="note" %}} For `olderthan` to work, you must be using the stage to set the timestamp from the ingested log line before_ applying the `drop` stage. {{% /admonition %}} Given the pipeline: ```yaml json: expressions: time: msg: timestamp: source: time format: RFC3339 drop: older_than: 24h dropcounterreason: "linetooold" ``` With a current ingestion time of 2020-08-12T12:00:00Z would drop this log line when read from a file: ``` {"time":"2020-08-11T11:00:00Z", "level": "error", "msg":"11.11.11.11 - "POST /loki/api/push/ HTTP/1.1" 200 932 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 GTB6"} ``` However it would not drop this log line: ``` {"time":"2020-08-11T13:00:00Z", "level": "error", "msg":"11.11.11.11 - "POST /loki/api/push/ HTTP/1.1" 200 932 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 GTB6"} ``` In this example the current time is 2020-08-12T12:00:00Z and `older_than` is 24h. All log lines which have a timestamp older than 2020-08-11T12:00:00Z will be dropped. All lines dropped by this drop stage would also increment the `logentrydroppedlinestotal` metric with a label `reason="linetoo_old"` Given the pipeline: ```yaml drop: longer_than: 8kb dropcounterreason: "linetoolong" ``` Would drop any log line longer than 8kb bytes, this is useful when Loki would reject a line for being too long. All lines dropped by this drop stage would also increment the `logentrydroppedlinestotal` metric with a label `reason="linetoo_long"` Complex `drop` stage configurations specify multiple options in one stage or specify multiple drop stages Given the pipeline: ```yaml drop: expression: ".debug." longer_than: 1kb ``` Would drop all logs that contain the word debug AND are longer than 1kb bytes Given the pipeline: ```yaml json: expressions: time: msg: timestamp: source: time format: RFC3339 drop: older_than: 24h drop: longer_than: 8kb drop: source: "msg" expression: ".trace." ``` Would drop all logs older than 24h OR longer than 8kb bytes OR have a json `msg` field containing the word trace