GitHub repository: JamesJJ/fluent-bit-output-deduplicated-post
Primary language: Go
Created: 2020-04-19T13:37:11Z
Description: A FluentBit output plugin that can deduplicate, decorate, batch and compress incoming events, and then POST them to an upstream HTTP server. The original purpose was to process a high volume log source as input e.g. web service logs, match a list of specific clients IP, and POST upstream once daily a batch of the deduplicated matching callers (a very small amount of data compared to the original incoming event flow)