Project: Logging Operator (Kube Logging) (Observability and Analysis / Observability)
File: opensearch.md

title: OpenSearch weight: 200 generated_file: true For details, see . For an example deployment, see . ```yaml spec: opensearch: host: opensearch-cluster.default.svc.cluster.local port: 9200 scheme: https ssl_verify: false sslversion: TLSv12 buffer: timekey: 1m timekey_wait: 30s timekeyuseutc: true ``` Send your logs to OpenSearch Specify the application name for the rollover index to be created. Default: default Configure bulkmessage request splitting threshold size. Default value is 20MB. (20 * 1024 * 1024) If you specify this size as negative number, bulkmessage request splitting feature will be disabled. Default: 20MB catchtransportexceptiononretry (default: true) Default: true compression_level This parameter adds additional headers to request. Example: `{"token":"secret"}` Default: {} Specify the string and its value to be replaced in form of hash. Can contain multiple key value pair that would be replaced in the specified templatefile. This setting only creates template and to add rollover index please check the rolloverindex configuration. Use @type opensearchdatastream You can specify Opensearch data stream name by this parameter. This parameter is mandatory for opensearchdatastream. Specify an existing index template for the data stream. If not present, a new template is created and named after the data stream. Default: datastreamname maxretrygetosversion Default: 1 emiterrorformissingid Default: false emiterrorlabel_event (default: true) Default: true AWS Endpoint Credentials Indicates whether to backup chunk when ignore exception occurs. Default: true failondetectingosversionretryexceed (default: true) Default: true Indicates whether to fail when maxretryputting_template is exceeded. If you have multiple output plugin, you could use this property to do not fail on Fluentd statup.(default: true) Default: true Flatten separator You can specify OpenSearch host by this parameter. Default: localhost You can specify multiple OpenSearch hosts with separator ",". If you specify hosts option, host and port options are ignored. With http_backend typhoeus, the opensearch plugin uses typhoeus faraday http backend. Typhoeus can handle HTTP keepalive. Default: excon httpbackendexcon_nonblock Default: true Field on your data to identify the data uniquely A list of exception that will be ignored - when the exception occurs the chunk will be discarded and the buffer retry mechanism won't be called. It is possible also to specify classes at higher level in the hierarchy. With this option set to true, Fluentd manifests the index name in the request URL (rather than in the request body). You can use this option to enforce an URL-based access control. This will add the Fluentd tag in the JSON record. Default: false Adds a @timestamp field to the log, following all settings logstashformat does, except without the restrictions on indexname. This allows one to log to an alias in OpenSearch and utilize the rollover API. Default: false Specify this to override the index date pattern for creating a rollover index. Default: now/d The index name to write events to Default: fluentd index_separator Default: - logos400_reason Default: false Set the Logstash date format. Default: %Y.%m.%d Enable Logstash log format. Default: false Set the Logstash prefix. Default: logstash Set the Logstash prefix separator. Default: - maxretrygetosversion Default: 15 You can specify times of retry putting template. Default: 10 parent_key Password for HTTP Basic authentication. Path for HTTP Basic

This param is to set a pipeline ID of your OpenSearch to be added into the request, you can configure ingest node. You can specify OpenSearch port by this parameter. Default: 9200 With default behavior, OpenSearch client uses Yajl as JSON encoder/decoder. Oj is the alternative high performance JSON encoder/decoder. When this parameter sets as true, OpenSearch client uses Oj as JSON encoder/decoder. Default: false Indicates that the plugin should reset connection on any error (reconnect on next send). By default it will reconnect only on "host unreachable exceptions". We recommended to set this true in the presence of OpenSearch shield. Default: false When reload_connections true, this is the integer number of operations after which the plugin will reload the connections. The default value is 10000. You can tune how the OpenSearch-transport host reloading feature works.(default: true) Default: true Indicates that the OpenSearch-transport will try to reload the nodes addresses if there is a failure while making the request, this can be useful to quickly remove a dead node from the list of addresses. Default: false Remove keys on update will not update the configured keys in OpenSearch when a record is being updated. This setting only has any effect if the write operation is update or upsert. This setting allows removekeysonupdate to be configured with a key in each record, in much the same way as targetindex_key works. You can specify HTTP request timeout. Default: 5s You can set in the OpenSearch-transport how often dead connections from the OpenSearch-transport's pool will be resurrected. Default: 60s This setting allows custom routing of messages in response to bulk request failures. The default behavior is to emit failed records using the same tag that was provided. routing_key CA certificate Client certificate Client certificate key Client key password Connection scheme Default: http selectorclassname The threshold for chunk flush performance check. Parameter type is float, not time, default: 20.0 (seconds) If chunk flush takes longer time than this threshold, Fluentd logs a warning message and increases the `fluentdoutputstatusslowflush_count` metric. The default Sniffer used by the OpenSearch::Transport class works well when Fluentd has a direct connection to all of the OpenSearch servers and can make effective use of the nodes API. This doesn't work well when Fluentd must connect through a load balancer or proxy. The `snifferclass_name` parameter gives you the ability to provide your own Sniffer class to implement whatever connection reload logic you require. In addition, there is a new Fluent::Plugin::OpenSearchSimpleSniffer class which reuses the hosts given in the configuration, which is typically the hostname of the load balancer or proxy. For example, a configuration like this would cause connections to logging-os to reload every 100 operations: . Skip ssl verification (default: true) Default: true If you want to configure SSL/TLS version, you can specify sslversion parameter. [SSLv23, TLSv1, TLSv11, TLSv1_2] By default, record body is wrapped by 'doc'. This behavior can not handle update script requests. You can set this to suppress doc wrapping and allow record body to be

Default: false Suppress type name to avoid warnings in OpenSearch This will add the Fluentd tag in the JSON record. Default: tag targetindexaffinity Default: false Tell this plugin to find the index name to write to in the record under this key in preference to other mechanisms. Key can be specified as path to nested record using dot ('.') as a separator. The path to the file containing the template to install. The name of the template to define. If a template by the name given is already present, it will be left unchanged, unless template_overwrite is set, in which case the template will be updated. Always update the template, even if it already exists. Default: false Specify index templates in form of hash. Can contain multiple templates. By default, when inserting records in Logstash format, @timestamp is dynamically created with the time at log ingestion. If you'd like to use a custom time, include an @timestamp with your record. timekeyexclude_timestamp Default: false The format of the time stamp field (@timestamp or what you specify with `timekey`). This parameter only has an effect when logstashformat is true as it only affects the name of the index we write to. With logstashformat true, OpenSearch plugin parses timestamp field for generating index name. If the record has invalid timestamp value, this plugin emits an error event to @ERROR label with `timeparseerrortag` configured tag. Should the record not include a time_key, define the degree of sub-second time precision to preserve from the time portion of the routed event. truncatecachesinterval Default unrecoverableerrortypes parameter is set up strictly. Because rejectedexecutionexception is caused by exceeding OpenSearch's thread pool capacity. Advanced users can increase its capacity, but normal users should follow default behavior. unrecoverablerecordtypes Specify wether to use legacy template or not. Default: true User for HTTP Basic authentication. This plugin will escape required URL encoded characters within %{} placeholders. e.g. `%{demo+}` By default, the records inserted into index logstash-YYMMDD with UTC (Coordinated Universal Time). This option allows to use local time if you describe `utc_index` to false. Default: true When you use mismatched OpenSearch server and client libraries, fluent-plugin-opensearch cannot send data into OpenSearch. Default: false verifyosversionatstartup (default: true) Default: true This is debugging purpose option to enable to obtain transporter layer log. Default: false The write_operation can be any of: (index,create,update,upsert) Default: index AWS access key id. This parameter is required when your agent is not running on EC2 instance with an IAM Role. Typically, you can use AssumeRole for cross-account access or federation. Set with AWSCONTAINERCREDENTIALSRELATIVEURI environment variable value AWS region. It should be in form like us-east-1, us-west-2. Default nil, which means try to find from environment variable AWS_REGION. AWS secret key. This parameter is required when your agent is not running on EC2 instance with an IAM Role. By default, the AWS Security Token Service (AWS STS) is available as a global service, and all AWS STS requests go to a single endpoint at https://sts.amazonaws.com. AWS recommends using Regional AWS STS endpoints instead of the global endpoint to reduce latency, build in redundancy, and increase session token validity. https://docs.aws.amazon.com/IAM/latest/UserGuide/idcredentialstemp_enable-regions.html AWS connection url.