Project: Vector (Observability and Analysis / Observability)
File: target.md

This document specifies requirements for installation targets for the integration of Vector. The key words MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, SHOULD NOT, RECOMMENDED, MAY, and OPTIONAL in this document are to be interpreted as described in [RFC 2119]. Other words, such as "agent", "aggregator", "node", and "service" are to be interpreted as described in the . - - In its simplest form, installing Vector consists of downloading the binary and making it executable, but leaves much to be desired for users looking to integrate Vector in real-world production environments. To adhere with Vector's , Vector must also be opinionated about how it's deployed, providing easy facilities for adopting Vector's , , and Vector. Vector supports a number of installation targets that can be categorized into: Virtual/Physical Machine Orchestration Platform The primary differentiator between the two being that Virtual/Physical Machines provide a single node as the deployment target, whereas Orchestration Platforms allow for a scheduler to deploy Vector across a number of nodes. These categories have their own requirements for each . Examples of Virtual/Physical Machine targets include, but are not limited to: Debian Docker RHEL Windows Examples of Orchestration Platform targets include, but are not limited to: Kubernetes When supporting a target, Vector must support them through the paradigm of architectures: Targets MUST support the by providing a single command that deploys Vector and achieves the . Targets SHOULD support the by providing a single command that deploys Vector and achieves the . Targets MAY support the by providing a single command that deploys Vector and achieves the . The deploys Vector on each individual node for distributed data collection and processing. Along with general requirements, the following requirements define support for this architecture: Architecture MUST deploy as a daemon on existing nodes, one Vector process per node. MUST deploy with Vector's . Sizing MUST deploy as a good infrastructure citizen, giving resource priority to other services on the same node. SHOULD be limited to 1 vCPUs by default, MUST be overridable by the user. SHOULD be limited to 2 GiB of memory per vCPU by default, MUST be overridable by the user. SHOULD be limited to 1 GiB of disk space, MUST be overridable by the user. The deploys Vector onto dedicated nodes for data aggregation. Along with general requirements, the following requirements define support for this architecture: Architecture MUST deploy as a service with reserved/dedicated resources. SHOULD deploy with a persistent disk that is available between deployments by default, MUST be overridable by the user if they do not want a persistent disk. MUST deploy with Vector's . Configured Vector ports, including non-default user configured ports, SHOULD be automatically accessible within the Cluster or

Configured Vector sources, including non-default user configured sources, SHOULD be automatically discoverable via target service discovery mechanisms. Sizing MUST have dedicated/reserved resources that cannot be stolen by other services, preventing the "noisy neighbor" problem to the degree possible. The Vector service SHOULD NOT be artificially limited with resource limiters such as cgroups. SHOULD require 8 vCPUs by default, MUST be overridable by the user. SHOULD require 2 GiB of memory per vCPU (16 GiB in this case) by default, MUST be overridable by the user. SHOULD request 36 GiB of disk space per vCPU by default (288 GiB in this case), MUST be overridable by the user. The following are additional requirements for Orchestration Platform installation targets: High Availability SHOULD deploy across multiple nodes by default, MUST be overridable by the user. SHOULD deploy across multiple availability zones by default, MUST be overridable by the user. Scaling SHOULD provide facilities for provisioning a load balancer to enable horizontal scaling out of the box. MUST be overridable by the user. Cloud-managed load balancers (i.e., AWS NLB) SHOULD be supported in addition to self-managed load balancers (i.e., HAProxy). Cloud-managed load balancers SHOULD be prioritized by default over self-managed load balancers. Network load balancers (layer-4) SHOULD be prioritized over HTTP load balancers (layer-7) Autoscaling SHOULD be enabled by default, driven by an average of 85% CPU utilization and a stabilization period of 5 minutes. The deploys Vector on each individual node as an agent and as a separate service as an aggregator. The requirements for both the and the apply to this architecture. This architecture SHOULD NOT be installed on Virtual/Physical Machine targets as there is little added benefit. Setup An unprivileged Vector service account SHOULD be created upon installation for running the Vector process. Data hardening Swap SHOULD be disabled to prevent in-flight data from leaking to disk. Swap would also make Vector prohibitively slow. Vector's data directory SHOULD be read and write restricted to Vector's dedicated service account. Core dumps SHOULD be prevented for the Vector process to prevent in flight data from leaking to disk. Process hardening Vector's artifacts All communication during the setup process, such as downloading Vector artifacts, MUST use encrypted channels. Downloaded Vector artifacts MUST be verified against the provided checksum. The latest Vector version SHOULD be downloaded unless otherwise specified by the user. Vector's configuration Vector's configuration directory SHOULD be read restricted to Vector's service account. Vector's runtime Vector SHOULD be run under an unprivileged, dedicated service account. Vector's service account SHOULD NOT have the ability to overwrite Vector's binary or configuration files. The only directory the Vector service account should write to is Vectors data directory. Network hardening Configured sources and sinks SHOULD use encrypted channels by default.