Falco Runtime Security: 20210501 plugin system

# Plugin System

## Summary

This is a proposal to create an infrastructure to extend the functionality of the Falco libraries via plugins.

Plugins will allow users to easily extend the functionality of the libraries and, as a consequence, of Falco and any other tool based on the libraries.

This proposal, in particular, focuses on two types of plugins: source plugins and extractor plugins.

## Motivation

[libscap](https://github.com/falcosecurity/libs/tree/master/userspace/libscap) and [libsinsp](https://github.com/falcosecurity/libs/tree/master/userspace/libsinsp) provide a powerful data capture framework, with a rich set of features that includes:

- data capture
- trace files management
- enrichment
- filtering
- formatting and screen rendering
- Lua scripting (chisels)

These features have been designed with one specific input in mind: system calls. However, they are generically adaptable to a broad set of inputs, such as cloud logs.

With this proposal, we want to dramatically extend the scope of what the libraries, Falco and other tools can be applied to. We want to do it in a way that is easy, efficient and empowers anyone in the community to write a plugin.

## Goals

- To design and implement a plugin framework that makes the libraries more modular and extensible
- To have a framework that is easy to use
- To support dynamic loading of plugins, so that the libraries can be extended without having to be recompiled and relinked
- To enable users to write plugins in any language, with a particular focus on Go, C and C++
- To have an efficient plugin framework so that, performance-wise, writing a plugin is as close as possible as extending the libraries internal source code
- To make it possible to write plugins for Linux, MacOS and Windows

## Non-Goals

- To implement plugins other than source and extractor: to be approached as separate task
- To document the plugin framework and interface: to be approached as separate task

## Proposal

### Plugin Common Information

Both source and extractor plugins have the following:

- A required api version, to ensure compatibility with the plugin framework.
- A name
- A description
- A version
- A contact field for the plugin authors (website, github repo, twitter, etc).
- Functions to initialize and destroy the plugin internal state.

### Plugin types

Initially, we will implement support for two types of plugins: source plugins and extractor plugins.

#### Source Plugin

A source plugin implements a new sinsp/scap event source. It has the ability to "open" and "close" a session that provides events. It also has the ability to return an event to the plugin framework via a next() method. Events returned by source plugins have an "event source", which describes the information in the event. This is distinct from the plugin name to allow for multiple kinds of plugins to generate the same kind of events. For example, there might be plugins gke-audit-bridge, eks-audit-bridge, ibmcloud-audit-bridge, etc. that all fetch [K8s Audit](https://kubernetes.io/docs/tasks/debug-application-cluster/audit/) information. The plugins would have different names but would have the same event source "k8s_audit".

Source plugins also have the ability to extract information from events based on fields. For example, a field proc.name extracts a process name from a syscall event. The plugin returns a set of supported fields, and there are functions to extract a value given an event and field. The plugin framework can then build filtering expressions/Falco rule conditions based on these fields combined with relational and/or logical operators. For example, given an expression "ct.name=root and ct.region=us-east-1", the plugin framework handles parsing the expression, calling the plugin to extract values for a given event, and determining the result of the expression. In a Falco output string like "An EC2 Node was created (name=%ec2.name region=%ct.region)", the plugin framework handles parsing the output string, calling the plugin to extract values for a given event, and building the resolved string.

Source plugins also provide an "id", which is globally unique and is used in capture files (see below).

#### Extractor Plugin

An extractor plugin focuses only on field extraction from events generated by other plugins, or by the core libraries. It does *not* provide an event source, but can extract fields from other event sources. An example is json field extraction, where a plugin might be able to extract fields from arbitrary json payloads.

An extractor plugin provides an optional set of event sources. When the framework receives an event with an event source in the plugin's set of event sources, fields in expressions/Falco outputs will be extracted from events using the plugin. An extractor plugin can also *not* name a set of event sources. In this case, fields will be extracted from *all* events, regardless of source. In this case, the extractor plugin must detect the format of arbitrary payloads and be able to return NULL/no value when the payload is not supported.

### Support for Plugin Events in Capture Files.

libscap will define a new event type called "pluginevent" that contains two fields:

* "plugin ID": This uniquely identifies the plugin that generated this event.
* "event_data": This is a variable-length data buffer containing the event data, as returned by the plugin.

Defining an event for plugins allows creating capture files from plugins. These capture files can be saved, read, filtered, etc, like any other capture file, allowing for later analysis/display/etc.

### Plugins format

Plugins are dynamic libraries (.so files in Unix, .dll files in windows) that export a minimum set of functions that the libraries will recognize.

Plugins are versioned using semantic versioning to minimize regressions and compatibility issues.

Plugins can be written in any language, as long as they export the required functions. Go, however, is the preferred language to write plugins, followed by C/C++.

### Protecting from plugin issues

The libraries will do everything possible to validate the data coming from the plugins and protect Falco and the other consumers from corrupted data. However, for performance reasons, plugins will be "trusted": they will run in the same thread and address space as Falco and they could crash the program. We assume that the user will be in control of plugin loading and will make sure only trusted plugins are loaded/packaged with Falco.

### Plugin/Event Source registries

Every source plugin requires its own, unique plugin ID to interoperate with Falco and the other plugins. The plugin ID will be used by the libs to properly process incoming events (for example, when saving events to file and loading them back), and by plugins to unambiguously recognize their dependencies.

To facilitate the allocation and distribution of plugin IDs, we will require that plugin developers request IDs for their plugins to the Fa