Project: Puppet (Provisioning / Automation & Configuration)
File: indirector.md

This document describes Puppet's indirector subsystem, but it has a number of limitations described below. As a result, don't introduce any new indirections or termini. Puppet's indirector supports pluggable backends (termini) for a variety of key-value stores (indirections). Each indirection type corresponds to a particular Ruby class (the "Indirected Class" below) and values are instances of that class. Each instance's key is available from its name method. The termini can be local (e.g., on-disk files) or remote (e.g., using a REST interface to talk to a puppet master). An indirector has five methods, which are mapped into HTTP verbs for the REST interface: `find(key)` - get a single value (mapped to GET or POST with a singular endpoint) `search(key)` - get a list of matching values (mapped to GET with a plural endpoint) `head(key)` - return true if the key exists (mapped to HEAD) `destroy(key)` - remove the key and value (mapped to DELETE) `save(instance)` - write the instance to the store, using the instance's name as the key (mapped to PUT) These methods are available via the indirection class method on the indirected classes. For example, the following: ```ruby catalog = Puppet::Resource::Catalog.indirection.find('foo.example.com') ``` Will retrieve the catalog for the node `foo.example.com` based on the currently configured catalog terminus. If the terminus is the compiler, a new catalog will be compiled. If the terminus is `json`, it will be loaded from disk. At startup, each indirection is configured with a terminus. In most cases, this is the default terminus defined by the indirected class, but it can be overridden by the application or face, or overridden with the route_file configuration. The available termini differ for each indirection, and are listed below. Indirections can also have a cache, represented by a second terminus. This is a write-through cache: modifications are written both to the cache and to the primary terminus. Values fetched from the terminus are written to the cache. REST endpoints have the form /{prefix}/{version}/{indirection}/{key}?environment={environment}, where the indirection can be singular or plural, following normal English spelling rules. However, like most things in the English language, there are . On the server side, REST responses are generated from the locally-configured endpoints. Below is the list of all indirections, their associated terminus classes, and how you select between them. In general, the appropriate terminus class is selected by the application for you (e.g., puppet agent would always use the rest terminus for most of its indirected classes), but some classes are tunable via normal settings. These will have terminus setting documentation listed with them. Indirected Class: `Puppet::Resource::Catalog` Terminus Setting: `catalog_terminus` `compiler` terminus Compiles catalogs on demand using Puppet's compiler. `json` terminus Store catalogs as flat files, serialized using JSON. `msgpack` terminus Store catalogs as flat files, serialized using MessagePack. `rest` terminus Find resource catalogs over HTTP via REST. `store_configs` terminus Part of the "storeconfigs"

Should not be directly set by end users. `yaml` terminus Store catalogs as flat files, serialized using YAML. Where to find external data bindings. Indirected Class: `Puppet::DataBinding` Terminus Setting: `databindingterminus` `hiera` terminus Retrieve data using Hiera. `none` terminus A Dummy terminus that always throws :nosuchkey for data lookups. Indirected Class: `Puppet::Node::Facts` Terminus Setting: `facts_terminus` `facter` terminus Retrieve facts from Facter. This provides a somewhat abstract interface between Puppet and Facter. It's only somewhat abstract because it always returns the local host's facts, regardless of what you attempt to find. `memory` terminus Keep track of facts in memory but nowhere else. This is used for one-time compiles, such as what the stand-alone puppet does. To use this terminus, you must load it with the data you want it to contain. `network_device` terminus Retrieve facts from a network device. `store_configs` terminus Part of the "storeconfigs" feature. Should not be directly set by end users. `yaml` terminus Store client facts as flat files, serialized using YAML, or return deserialized facts from disk. Indirected Class: `Puppet::FileBucket::File` `file` terminus Store files in a directory set based on their checksums. `rest` terminus This is a REST based mechanism to send/retrieve file to/from the filebucket `selector` terminus Select the terminus based on the request Indirected Class: `Puppet::FileServing::Content` `file` terminus Retrieve file contents from disk. `file_server` terminus Retrieve file contents using Puppet's fileserver. `http` terminus Retrieve file contents from a remote HTTP server. `rest` terminus Retrieve file contents via a REST HTTP interface. `selector` terminus Select the terminus based on the request Indirected Class: `Puppet::FileServing::Metadata` `file` terminus Retrieve file metadata directly from the local filesystem. `file_server` terminus Retrieve file metadata using Puppet's fileserver. `http` terminus Retrieve file metadata from a remote HTTP server. `rest` terminus Retrieve file metadata via a REST HTTP interface. `selector` terminus Select the terminus based on the request Where to find node information. A node is composed of its name, its facts, and its environment. Indirected Class: `Puppet::Node` Terminus Setting: `node_terminus` `exec` terminus Call an external program to get node information. See the External Nodes page for more information. `ldap` terminus Search in LDAP for node configuration information. See the LDAP Nodes page for more information. This will first search for whatever the certificate name is, then (if that name contains a .) for the short name, then default. `memory` terminus Keep track of nodes in memory but nowhere else. This is used for one-time compiles, such as what the stand-alone puppet does. To use this terminus, you must load it with the data you want it to contain; it is only useful for developers and should generally not be chosen by a normal user. `msgpack` terminus Store node information as flat files, serialized using MessagePack, or deserialize stored MessagePack nodes. `plain` terminus Always return an empty node object. Assumes you keep track of nodes in flat file

You should use it when you don't have some other, functional source you want to use, as the compiler will not work without a valid node terminus. Note that class is responsible for merging the node's facts into the node instance before it is returned. `rest` terminus Get a node via REST. Puppet agent uses this to allow the puppet master to override its environment. `store_configs` terminus Part of the "storeconfigs" feature. Should not be directly set by end users. `yaml` terminus Store node information as flat files, serialized using YAML, or deserialize stored YAML nodes. Indirected Class: `Puppet::Transaction::Report` `msgpack` terminus Store last report as a flat file, serialized using MessagePack. `processor` terminus Puppet's report processor. Processes the report with each of the report types listed in the reports' setting. `rest` terminus Get server report over HTTP via REST. `yaml` terminus Store last report as a flat file, serialized using YAML. Indirected Class: `Puppet::Resource` `ral` terminus Manipulate resources with the resource abstraction layer. Only used internally. `store_configs` terminus Part of the "storeconfigs" feature. Should not be directly set by end users. `rest` terminus Get puppet master's status via REST. Useful because it tests the health of both the web server and the indirector. Here are specific issues with the indirector: The indirector relies on mutable global state, such as the list of indirections and which termini and caches are currently in use. Termini can be configured in settings, code or `routes.yaml`, but not all termini can be configured the same way. For example, there is a `nodeterminus` setting, but no `reportterminus`. There's a `catalogcacheterminus`, but no `factcacheterminus`. There can only be one termini for an indirection at any one time. However, some applications like `puppet catalog find` need to make a REST request and then write the catalog to disk, which . The indirector maintains an optional terminus cache, which is often used to overcome the previous limitation. For example, `puppet agent` downloads the catalog using the `rest` terminus, but saves the cached catalog as a side effect. The caller is aware of the cache, and or . It's not clear when objects are cleared from the cache. If an exception occurs when calling the terminus, the caller can't tell whether it occurred in the terminus or the cache terminus. Exceptions from the remote side of the `rest` terminus, don't propagate back. So if `find` returns nil, the caller doesn't know if it the resource doesn't exist, or an exception occurred for a different reason, such as the environment doesn't exist. The `failon404` option was created to handle that case, but is meaningless for non-REST termini. Can't process the HTTP response header to do something intelligent, e.g. to switch facts to JSON instead of PSON Doesn't support streaming as objects must be fully loaded in memory. Concrete termini, e.g. `Puppet::Node::Facts::Rest`, must inherit from an abstract terminus like `Puppet::Indirector::REST`, `Puppet::Indirector::Code`, etc. Also concrete termini can't inherit from other concrete termini.