Project: Vector (Observability and Analysis / Observability)
File: 2023-10-30-secrets-in-disk-buffers.md

date: "2023-10-30" title: "Secrets in Disk Buffers" description: "" authors: ["tobz"] pr_numbers: [18816] release: "0.34.0" hideonrelease_notes: false badges: type: "announcement" Starting with Vector's `0.34.0` release, secrets in events will now be stored in disk buffers. These secrets are stored unencrypted. For some source components in Vector, such as the Datadog Agent or Splunk HEC sources, these components have the ability to store the API key received in requests by Vector in order to re-use the same API key when sending those events back out to a compatible service. This allows users, for example, to set up Vector as an aggregator for all of their Datadog Agent processes, reusing the original API key, or keys, as the events are then forwarded to the Datadog API. Prior to , these event secrets (and other event metadata) were not stored when using disk buffers. This represented a loss of functionality when users switched from the default in-memory buffers to disk buffers. In order to bring this functionality up to par, we added support for storing event secrets/metadata when writing events to disk buffers. Naturally, event secrets represent sensitive data such as API keys and more. However, Vector currently stores these event secrets unencrypted in disk buffers. Firstly, if you're not using disk buffers, then there is no change to Vector's behavior and you can stop reading here. There are two main scenarios where a configuration might now start storing secrets in disk buffers: When you are using a source component which has the ability to store secrets When you are using `remap` and adding secrets directly to events Some source components store secrets (specifically, API keys) on an event in order to facilitate Vector acting similarly to a proxy, using as much of the original request/event data as possible. Only two sources currently provide such behavior: `datadog_agent` source (stores the `DD-API-KEY` header value; enabled by default) `splunk_hec` source (stores the `Authorization` header value; disabled by default) However, for both of these sinks, this behavior can be disabled by setting `storeapikey` to `false` for the `datadogagent` source, or setting `storehectoken` to `false` for the `splunkhec` source. When using the `remap` transform, VRL exposes helper functions to set secrets on events. If your `remap` usage includes setting secrets, then those secrets would also now be in scope for getting stored in disk buffers. As mentioned above, secrets will now be stored in disk buffer data files, and will be unencrypted. The data directory that Vector is configured to use should be locked down as tightly as possible so that only the user/group that runs the Vector process has read/write access. By default on Unix-based platforms, Vector will attempt to set file permissions for the disk buffer directory/files to only be readable/writeable by the process user, and only readable by the process group. This does not occur on Windows. This is not the end of the story for storing secrets in disk buffers. We do have tentative plans to eventually support encrypting secrets in disk buffers, and potentially support encrypting all event data itself. This work depends on capabilities Vector does not currently have, such as being able to securely pass a decryption key into the process, and where a long-lived decryption key would live. These issues need to be tackled first before we can provide a robust encryption solution for disk buffers.