cwielenb/CNCF-Dataset huggingface dataset:
Project: Netdata (Observability and Analysis / Observability)
File: postgres_db_transactions_rollback_ratio.md

This alert calculates the `PostgreSQL database transactions rollback ratio` for the last five minutes. If you receive this alert, it means that the percentage of `aborted transactions` in the specified PostgreSQL database is higher than the defined threshold. In a PostgreSQL database, the transactions rollback ratio represents the proportion of aborted transactions (those that roll back) in relation to the total number of transactions processed. A high rollback ratio may indicate issues with the application logic, database performance or excessive `deadlocks` causing transactions to be aborted frequently. Check the PostgreSQL logs for any error messages or unusual activities related to transactions that might help identify the cause of the high rollback ratio. ``` vi /var/log/postgresql/postgresql.log ``` Replace `/var/log/postgresql/postgresql.log` with the appropriate path to your PostgreSQL log file. Investigate recent database changes or application code modifications that might have led to the increased rollback ratio. Examine the PostgreSQL database table and index statistics to identify potential performance bottlenecks. ``` SELECT relname, seqscan, idxscan, ntupins, ntupupd, ntupdel, ntuphotupd, nlivetup, ndeadtup, lastvacuum, last_analyze FROM pgstatall_tables WHERE schemaname = 'yourschemaname'; ``` Replace `yourschemaname` with the appropriate schema name. Identify the most frequent queries that cause transaction rollbacks using pgstatstatements view: ``` SELECT substring(query, 1, 50) as shortquery, calls, totaltime, rows, 100.0 * sharedblkshit/nullif(sharedblkshit + sharedblksread, 0) AS hit_percent FROM pgstatstatements WHERE calls > 50 ORDER BY (total_time / calls) DESC; ``` Investigate database locks and deadlocks using pg_locks: ``` SELECT database, relation::regclass, mode, transactionid AS tid, virtualtransaction AS vtid, pid, granted FROM pgcatalog.pglocks; ``` Make necessary changes in the application logic or database configuration to resolve the issues causing a high rollback ratio. Consult a PostgreSQL expert, if needed.
Project: Grafana (Observability and Analysis / Observability)
File: release-notes-8-3-4.md

_build: list: false labels: products: cloud enterprise oss title: Release notes for Grafana 8.3.4 <!-- Auto generated by update changelog github action --> Alerting: Allow configuration of non-ready alertmanagers. , Alerting: Allow customization of Google chat message. , Alerting: Allow customization of Google chat message (#43568). , AppPlugins: Support app plugins with only default nav. , InfluxDB: InfluxQL: query editor: skip fields in metadata queries. , Postgres/MySQL/MSSQL: Cancel in-flight SQL query if user cancels query in grafana. , Prometheus: Forward oauth tokens after prometheus datasource migration. , Azure Monitor: Bug fix for variable interpolations in metrics dropdowns. , Azure Monitor: Improved error messages for variable queries. , CloudMonitoring: Fixes broken variable queries that use group bys. , Configuration: You can now see your expired API keys if you have no active ones. , Elasticsearch: Fix handling multiple datalinks for a single field. , Export: Fix error being thrown when exporting dashboards using query variables that reference the default datasource. , ImportDashboard: Fixes issue with importing dashboard and name ending up in uid. , Login: Page no longer overflows on mobile. , Plugins: Set backend metadata property for core plugins. , Prometheus: Fill missing steps with null values. , Prometheus: Fix interpolation of $\\rate_interval variable. , Prometheus: Interpolate variables with curly brackets syntax. , Prometheus: Respect the http-method data source setting. , Table: Fixes issue with field config applied to wrong fields when hiding columns. , Toolkit: Fix bug with rootUrls not being properly parsed when signing a private plugin. , Variables: Fix so data source variables are added to adhoc configuration. , Toolkit: Revert build config so tslib is bundled with plugins to prevent plugins from crashing. ,
Project: Vector (Observability and Analysis / Observability)
File: splunk_hec_metrics.md

title: Splunk HEC metrics description: Deliver metric data to Splunk's component_kind: sink layout: component tags: ["splunk", "hec", "http event collector", "component", "sink", "metrics"] {{/* This doc is generated using: The template in layouts/docs/component.html The relevant CUE data in cue/reference/components/... */}}
Project: Vector (Observability and Analysis / Observability)
File: 2021-07-14-vector-graph.md

date: "2021-07-14" title: "Visualizing your Vector topology" description: "Introducing the new `graph` CLI subcommand for outputting your Vector topology as a graph" authors: ["jszwedko"] pr_numbers: [8116] release: "0.15.0" hideonrelease_notes: false badges: type: new feature domains: ["cli"] This release adds a new subcommand `vector graph` to output the topology specified by you Vector configuration as a graph in format. This output can then be visualized using [Graphviz] to produce an image. For example, if you had a config, `vector.toml`, like: ```toml [sources.internal_metrics] type = "internal_metrics" [sources.dd_logs] type = "datadog_logs" # required acknowledgements = false # optional, default address = "0.0.0.0:8282" # required [sources.file_gen] type = "file" include = ["/var/log/file_gen//*.log"] read_from = "beginning" [transforms.remap] type = "remap" inputs = ["file_gen"] source = ''' .agent_name = "vector" parsed, err = parse_json(.message) if err == null { .message = parsed .format = "json" } else { .format = "ascii" } matches = parse_regex!(.file, r'.*/(?P<num>\d+)-(?P<name>\w+).log') .origin, err = .host + "/" + matches.name + "/" + matches.num if err != null { log("Failed to parse origin from file name", level: "error") } ''' [sinks.prometheus] type = "prometheus_exporter" inputs = ["internal_metrics"] address = "0.0.0.0:9598" [sinks.ddlogsegress] type = "datadog_logs" inputs = ["dd_logs"] defaultapikey = "" encoding.codec = "json" request.concurrency = "adaptive" batch.max_bytes = 5242880 request.ratelimitnum = 1000 [sinks.blackhole] type = "blackhole" inputs = ["remap"] ``` And you ran the new `vector graph --config vector.toml` you would see: ```dot digraph { "internal_metrics" [shape=trapezium] "dd_logs" [shape=trapezium] "file_gen" [shape=trapezium] "remap" [shape=diamond] "file_gen" -> "remap" "prometheus" [shape=invtrapezium] "internal_metrics" -> "prometheus" "ddlogsegress" [shape=invtrapezium] "ddlogs" -> "ddlogs_egress" "blackhole" [shape=invtrapezium] "remap" -> "blackhole" } ``` To render this, if you have [Graphviz] installed, you could do: ```shell vector graph --config vector.toml | dot -Tpng > graph.png ``` To get an image that looks like: If you don't have .
Project: Logging Operator (Kube Logging) (Observability and Analysis / Observability)
File: azurestore.md

title: Azure Storage weight: 200 generated_file: true Azure Storage output plugin buffers logs in local file and upload them to Azure Storage periodically. More info at https://github.com/microsoft/fluent-plugin-azure-storage-append-blob Automatically create container if not exists Default: true Available in Logging operator version 4.5 and later. Azure Cloud to use, for example, AzurePublicCloud, AzureChinaCloud, AzureGermanCloud, AzureUSGovernmentCloud, AZURESTACKCLOUD (in uppercase). This field is supported only if the fluentd plugin honors it, for example, https://github.com/elsesiy/fluent-plugin-azure-storage-append-blob-lts Your azure storage container Azure Instance Metadata Service API Version Object key format Default: %{path}%{timeslice}%{index}.%{file_extension} Your azure storage access key Your azure storage account Your azure storage sas token Compat format type: outfile, json, ltsv (default: outfile) Default: json Path prefix of the files on Azure The threshold for chunk flush performance check. Parameter type is float, not time, default: 20.0 (seconds) If chunk flush takes longer time than this threshold, fluentd logs warning message and increases metric fluentdoutputstatusslowflush_count.
Project: Inspektor Gadget (Observability and Analysis / Observability)
File: sni.md

title: 'Using trace sni' weight: 20 description: > Trace Server Name Indication (SNI) from TLS requests. The trace sni gadget is used to trace the requests sent as part of TLS handshakes. The SNI tracer will show which pods are making which SNI requests. To start it, we can run: ```bash $ kubectl gadget trace sni K8S.NODE K8S.NAMESPACE K8S.POD PID TID COMM NAME ``` To generate some output for this example, let's create a demo pod in another terminal: ```bash $ kubectl run -it ubuntu --image ubuntu:latest -- /bin/bash root@ubuntu:/# apt update && apt install -y wget && wget wikimedia.org (...) HTTP request sent, awaiting response... 301 Moved Permanently Location: https://www.wikimedia.org/ [following] (...) root@ubuntu:/# wget www.github.com (...) HTTP request sent, awaiting response... 301 Moved Permanently Location: https://github.com/ [following] (...) ``` Go back to the first terminal and see: ``` K8S.NODE K8S.NAMESPACE K8S.POD PID TID COMM NAME minikube default ubuntu 3917791 3917791 wget www.github.com minikube default ubuntu 3917791 3917791 wget github.com minikube default ubuntu 3917812 3917812 wget wikimedia.org minikube default ubuntu 3917812 3917812 wget www.wikimedia.org ``` We can see that each time our `wget` client connected to a different server, our tracer caught the Server Name Indication requested. Congratulations! You reached the end of this guide! You can now delete the pod you created: ```bash $ kubectl delete pod ubuntu pod "ubuntu" deleted ``` Run the gadget in a terminal ```bash $ sudo ig trace sni -r docker -c test-trace-sni RUNTIME.CONTAINERNAME PID TID COMM NAME ``` Run a containers that establishs a TLS connection with a remote endpoint: ```bash $ docker run -it --rm --name test-trace-sni busybox /bin/sh -c "wget https://example.com" Connecting to example.com (93.184.216.34:443) wget: note: TLS certificate validation not implemented saving to 'index.html' index.html 100% |*| 1256 0:00:00 ETA 'index.html' saved ``` The gadget will show that Server Name Indication used by the request. ```bash $ sudo ig trace sni -r docker -c test-trace-sni RUNTIME.CONTAINERNAME PID TID COMM NAME test-trace-sni 3944366 3944366 wget example.com ```
Project: Checkmk (Observability and Analysis / Observability)
File: 15198.md

key | value | date | 2024-04-09T12:24:12+00:00 version | 2.4.0b1 class | security edition | cre component | wato level | 1 compatible | yes Prior to this Werk, the mechanism to lock user accounts after too many failed login attempts was only effective for the web form login method. Login attempts via the REST API and basic authentication did not count towards the lockout mechanism. As a result, an attacker could try to brute-force user passwords without triggering the lockout mechanism. This Werk adds the same locking mechanism to login via the REST API and basic authentication for human user accounts. Note that automation accounts are remain unaffected by the lockout mechanism to avoid having them locked by malicious intent. It is therefore important to use long, random automation secrets. This issue was found during internal review. Affected Versions: 2.3.0 (beta) 2.2.0 2.1.0 2.0.0 (EOL) Mitigations: If updating is not possible, the brute-force attempts can be hindered by using a strong password policy. Vulnerability Management: We have rated the issue with a CVSS Score of 5.9 (Medium) with the following CVSS vector: `CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:N/A:N` and assigned CVE `CVE-2024-28825`.
Project: Checkmk (Observability and Analysis / Observability)
File: 15318.md

key | value | compatible | yes version | 2.3.0b1 date | 2024-01-24T07:27:37+00:00 level | 1 class | fix component | ec edition | cre Prior to this change, you may have seen the following log output: ``` undefined action "{aname}, must be one of {", ".join(table.keys()}" ``` The variables are now interpolated correctly.
Project: Headlamp (Observability and Analysis / Observability)
File: lib_k8s_node.KubeNode.md

title: "Interface: KubeNode" linkTitle: "KubeNode" slug: "libk8snode.KubeNode" .KubeNode `KubeNode` `Optional` apiVersion: `string` . _ kind: `string` Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. In CamelCase. Cannot be updated. `see` . _ metadata: . _ spec: `Object` | Name | Type | | : | : | | `podCIDR` | `string` | | `taints` | { `effect`: `string` ; `key`: `string` }[] | _ status: `Object` | Name | Type | | : | : | | `addresses` | { `address`: `string` ; `type`: `string` }[] | | `allocatable` | { `cpu`: `any` ; `ephemeralStorage`: `any` ; `hugepages1Gi`: `any` ; `hugepages2Mi`: `any` ; `memory`: `any` ; `pods`: `any` } | | `allocatable.cpu` | `any` | | `allocatable.ephemeralStorage` | `any` | | `allocatable.hugepages_1Gi` | `any` | | `allocatable.hugepages_2Mi` | `any` | | `allocatable.memory` | `any` | | `allocatable.pods` | `any` | | `capacity` | { `cpu`: `any` ; `ephemeralStorage`: `any` ; `hugepages1Gi`: `any` ; `hugepages2Mi`: `any` ; `memory`: `any` ; `pods`: `any` } | | `capacity.cpu` | `any` | | `capacity.ephemeralStorage` | `any` | | `capacity.hugepages_1Gi` | `any` | | `capacity.hugepages_2Mi` | `any` | | `capacity.memory` | `any` | | `capacity.pods` | `any` | | `conditions` | `Omit`<, ``"lastProbeTime"`` \| ``"lastUpdateTime"``\> & { `lastHeartbeatTime`: `string` }[] | | `nodeInfo` | { `architecture`: `string` ; `bootID`: `string` ; `containerRuntimeVersion`: `string` ; `kernelVersion`: `string` ; `kubeProxyVersion`: `string` ; `kubeletVersion`: `string` ; `machineID`: `string` ; `operatingSystem`: `string` ; `osImage`: `string` ; `systemUUID`: `string` } | | `nodeInfo.architecture` | `string` | | `nodeInfo.bootID` | `string` | | `nodeInfo.containerRuntimeVersion` | `string` | | `nodeInfo.kernelVersion` | `string` | | `nodeInfo.kubeProxyVersion` | `string` | | `nodeInfo.kubeletVersion` | `string` | | `nodeInfo.machineID` | `string` | | `nodeInfo.operatingSystem` | `string` | | `nodeInfo.osImage` | `string` | | `nodeInfo.systemUUID` | `string` |
Project: Logging Operator (Kube Logging) (Observability and Analysis / Observability)
File: gcs.md

title: Google Cloud Storage weight: 200 generated_file: true Store logs in Google Cloud Storage. For details, see . ```yaml spec: gcs: project: logging-example bucket: banzai-log-test path: logs/${tag}/%Y/%m/%d/ ``` Permission for the object in GCS: `authread` `ownerfull` `ownerread` `private` `projectprivate` `public_read` Create GCS bucket if it does not exists Default: true Name of a GCS bucket Number of times to retry requests on server error Default timeout to use in requests GCS service account credentials in JSON format Customer-supplied, AES-256 encryption key Max length of `%{hex_random}` placeholder(4-16) Default: 4 Path of GCS service account credentials JSON file Format of GCS object keys Default: `%{path}%{timeslice}%{index}.%{file_extension}` User provided web-safe keys and arbitrary string values that will returned with requests for the file as "x-goog-meta-" response headers. Overwrite already existing path Default: false Path prefix of the files on GCS Project identifier for GCS The threshold for chunk flush performance check. Parameter type is float, not time, default: 20.0 (seconds) If chunk flush takes longer time than this threshold, fluentd logs warning message and increases metric fluentdoutputstatusslowflush_count. Storage class of the file: `dra` `nearline` `coldline` `multi_regional` `regional` `standard` Archive format on GCS: gzip json text Default: gzip Enable the decompressive form of transcoding Key Value
Project: Headlamp (Observability and Analysis / Observability)
File: lib_k8s_pod.ExecOptions.md

title: "Interface: ExecOptions" linkTitle: "ExecOptions" slug: "libk8spod.ExecOptions" .ExecOptions `ExecOptions` `Optional` additionalProtocols: `string`[] Additional WebSocket protocols to use when connecting. . _ `Optional` cluster: `string` . _ `Optional` command: `string`[] _ `Optional` isJson: `boolean` Whether the stream is expected to receive JSON data. . _ `Optional` reconnectOnFailure: `boolean` Whether to attempt to reconnect the WebSocket connection if it fails. . _ `Optional` stderr: `boolean` . _ `Optional` stdin: `boolean` . _ `Optional` stdout: `boolean` . _ `Optional` tty: `boolean` . `Optional` connectCb(): `void` A callback function to execute when the WebSocket connection is established. `void` . _ `Optional` failCb(): `void` A callback function to execute when the WebSocket connection fails. `void` .
Project: Vector (Observability and Analysis / Observability)
File: dnstap.md

title: dnstap description: Collect DNS logs from a -compatible server component_kind: source layout: component tags: ["dnstap", "dns", "component", "source", "logs"] {{/* This doc is generated using: The template in layouts/docs/component.html The relevant CUE data in cue/reference/components/... */}}
Project: Checkmk (Observability and Analysis / Observability)
File: 16171.md

key | value | compatible | yes version | 2.4.0b1 date | 2024-02-21T13:16:55+00:00 level | 1 class | fix component | checks edition | cre Cloudwatch alarms weren't fetched properly for environments with a lot of alarms. It resulted in missing alarms in the 'AWS/CloudWatch Alarms' service.
Project: Grafana Loki (Observability and Analysis / Observability)
File: EXTERNAL_PLUGINS.md

This is a list of plugins that can be compiled outside of Telegraf and used via the `execd` , , or . Check out the for more information on writing and contributing a plugin. Pull requests welcome. - Generate random numbers - Gather account information from Twitter accounts - Gather account information from YouTube channels - Simple plugin to gather/monitor alarms generated in AWS. - Gather 3d print information from the octoprint API. - Gather systemd boot and unit timestamp metrics. - Aggregation and compression of metrics to send Amazon Kinesis.
Project: PowerfulSeal (Observability and Analysis / Chaos Engineering)
File: pods-rescheduled.md

layout: default title: Pods reschedule SLO description: "" permalink: /pods-rescheduled parent: Writing policies nav_order: 2 {: .no_toc } {: .no_toc .text-delta } TOC {:toc} Let's say we want to verify that if pod is deleted, it will be rescheduled and running with a specified time. You can do that easily with the Seal. ```yaml scenarios: name: Check pod rescheduling SLO description: > Verifies that after a pod is killed, it's succesfully rescheduled after 30 seconds. steps: podAction: matches: namespace: some-namespace filters: randomSample: size: 1 actions: kill: force: true wait: seconds: 30 podAction: matches: namespace: some-namespace actions: checkPodState: state: Running checkPodCount: count: 2 ```
Project: VictoriaMetrics (Observability and Analysis / Observability)
File: Logstash.md

weight: 3 title: Logstash setup disableToc: true menu: docs: parent: "victorialogs-data-ingestion" weight: 3 aliases: /VictoriaLogs/data-ingestion/Logstash.html Specify section in the `logstash.conf` file for sending the collected logs to : ```conf output { elasticsearch { hosts => ["http://localhost:9428/insert/elasticsearch/"] parameters => { "msgfield" => "message" "timefield" => "@timestamp" "streamfields" => "host.name,process.name" } } } ``` Substitute `localhost:9428` address inside `hosts` with the real TCP address of VictoriaLogs. See for details on the `parameters` section. It is recommended verifying whether the initial setup generates the needed and uses the correct . This can be done by specifying `debug` and inspecting VictoriaLogs logs then: ```conf output { elasticsearch { hosts => ["http://localhost:9428/insert/elasticsearch/"] parameters => { "msgfield" => "message" "timefield" => "@timestamp" "streamfields" => "host.name,process.name" "debug" => "1" } } } ``` If some must be skipped during data ingestion, then they can be put into `ignore_fields` . For example, the following config instructs VictoriaLogs to ignore `log.offset` and `event.original` fields in the ingested logs: ```conf output { elasticsearch { hosts => ["http://localhost:9428/insert/elasticsearch/"] parameters => { "msgfield" => "message" "timefield" => "@timestamp" "streamfields" => "host.hostname,process.name" "ignore_fields" => "log.offset,event.original" } } } ``` If the Logstash sends logs to VictoriaLogs in another datacenter, then it may be useful enabling data compression via `http_compression: true` option. This usually allows saving network bandwidth and costs by up to 5 times: ```conf output { elasticsearch { hosts => ["http://localhost:9428/insert/elasticsearch/"] parameters => { "msgfield" => "message" "timefield" => "@timestamp" "streamfields" => "host.hostname,process.name" } http_compression => true } } ``` By default, the ingested logs are stored in the `(AccountID=0, ProjectID=0)` . If you need storing logs in other tenant, then specify the needed tenant via `custom_headers` at `output.elasticsearch` section. For example, the following `logstash.conf` config instructs Logstash to store the data to `(AccountID=12, ProjectID=34)` tenant: ```conf output { elasticsearch { hosts => ["http://localhost:9428/insert/elasticsearch/"] custom_headers => { "AccountID" => "1" "ProjectID" => "2" } parameters => { "msgfield" => "message" "timefield" => "@timestamp" "streamfields" => "host.hostname,process.name" } } } ``` See also: . . . .
Project: Checkmk (Observability and Analysis / Observability)
File: 16336.md

key | value | compatible | yes version | 2.4.0b1 date | 2024-02-08T17:22:52+00:00 level | 1 class | fix component | checks edition | cre The output of the 'lspci' command was not consistent. This caused the parsing of the lnx_video section to fail. As a consequnce, the graphics cards were not shown in the HW/SW inventory. This has now been fixed and the graphics cards will be shown.
Project: Logging Operator (Kube Logging) (Observability and Analysis / Observability)
File: sqs.md

title: SQS weight: 200 generated_file: true Fluentd output plugin for SQS. AWS access key id AWS secret key Create SQS queue Default: true Delivery delay seconds Default: 0 Include tag Default: true Message group id for FIFO queue SQS queue name - required if sqs_url is not set AWS region Default: ap-northeast-1 SQS queue url e.g. `https://sqs.us-west-2.amazonaws.com/123456789012/myqueue` The threshold for chunk flush performance check. Parameter type is float, not time, default: 20.0 (seconds) If chunk flush takes longer time than this threshold, fluentd logs warning message and increases metric fluentdoutputstatusslowflush_count. Tags property name in json Default: 'tag' ```yaml apiVersion: logging.banzaicloud.io/v1beta1 kind: Output metadata: name: sqs-output-sample spec: sqs: queue_name: some-aws-sqs-queue create_queue: false region: us-east-1 ``` Fluentd config result: ```xml <match > @type sqs @id test_sqs queue_name some-aws-sqs-queue create_queue false region us-east-1 </match> ```
Project: Vector (Observability and Analysis / Observability)
File: apt.md

title: Install Vector using APT short: APT weight: 1 , or APT, is a free package manager that handles the installation and removal of software on [Debian], [Ubuntu], and other [Linux] distributions. Our APT repositories are provided by [Datadog]. This page covers installing and managing Vector through the . x86_64 ARM64 ARMv7 First, add the Vector repo: ```shell bash -c "$(curl -L https://setup.vector.dev)" ``` Then you can install the `vector` package: ```shell sudo apt-get install vector ``` {{< tabs default="Upgrade Vector" >}} {{< tab title="Upgrade Vector" >}} ```bash sudo apt-get upgrade vector ``` {{< /tab >}} {{< tab title="Uninstall Vector" >}} ```bash sudo apt remove vector ``` {{< /tab >}} {{< /tabs >}} {{< jump "/docs/administration/management" "apt-dpkg-rpm-yum-pacman" >}}