cwielenb/CNCF-Dataset huggingface dataset:
Project: Vector (Observability and Analysis / Observability)
File: 2020-07-10-add-reduce-transform.md

date: "2020-07-17" title: "New Reduce transform" description: "Canonical Log Lines in Vector" authors: ["hoverbear"] hideonrelease_notes: false pr_numbers: [2870] release: "0.10.0" badges: type: "new feature" domains: ["transforms"] Fan of ? We are too. You can now find a new ! This allows you to turn a stream of many small events into a stream of less small events! The fantastic article by Brandur Leach describes them best: Theyre a simple idea: in addition to their normal log traces, requests (or some other unit of work thats executing) also emit one long log line at the end that pulls all its key telemetry into one place. Here's a canonical example: ```log [2019-03-18 22:48:32.999] canonical-log-line alloccount=9123 authtype=apikey databasequeries=34 duration=0.009 httpmethod=POST httppath=/v1/charges httpstatus=200 keyid=mk123 permissionsused=accountwrite rateallowed=true ratequota=100 rateremaining=99 requestid=req123 team=acquiring userid=usr123 ``` Let's build similar in Vector! We'll take a series of events: ```log title=input.log {"timestamp": "...", "message": "Received GET /path", "requestid": "abcd1234", "requestpath": "/path", "request_params": "..."} {"timestamp": "...", "message": "Executed query in 5.2ms", "requestid": "abcd1234", "query": "SELECT * FROM table", "queryduration_ms": 5.2} {"timestamp": "...", "message": "Rendered partial partial.erb in 2.3ms", "requestid": "abcd1234", "template": "partial.erb", "renderduration_ms": 2.3} {"timestamp": "...", "message": "Executed query in 7.8ms", "requestid": "abcd1234", "query": "SELECT * FROM table", "queryduration_ms": 7.8} {"timestamp": "...", "message": "Sent 200 in 15.2ms", "requestid": "abcd1234", "responsestatus": 200, "responsedurationms": 5.2} ``` Then output this (but not formatted so nicely!): ```json title=output.log { "timestamp_start": "...", "timestamp_end": "...", "request_id": "abcd1234", "request_path": "/path", "request_params": "...", "querydurationms": 13.0, "renderdurationms": 2.3, "status": 200, "responsedurationms": 5.2 } ``` We'll run this config: ```toml title=vector.toml data_dir = "tmp" [sources.source0] include = ["input.log"] startatbeginning = true type = "file" fingerprinting.strategy = "deviceandinode" [transforms.transform0] inputs = ["source0"] type = "json_parser" field = "message" [transforms.transform1] inputs = ["transform0"] type = "reduce" identifierfields = ["requestid"] endswhen.type = "checkfields" endswhen."responsestatus.exists" = true merge_strategies.message = "discard" merge_strategies.query = "discard" merge_strategies.template = "discard" mergestrategies.queryduration_ms = "sum" mergestrategies.renderduration_ms = "sum" mergestrategies.responseduration_ms = "sum" [sinks.sink0] healthcheck = true inputs = ["transform1"] type = "file" path = "output.log" encoding = "ndjson" buffer.type = "memory" buffer.max_events = 500 buffer.when_full = "block"s ``` We hope you find this useful!
Project: Vector (Observability and Analysis / Observability)
File: 2019-12-16-ec2-metadata.md

date: "2020-03-31" title: "New AWS EC2 Metadata Transform" description: "Enrich your events with EC2 metadata" authors: ["binarylogic"] pr_numbers: [1325] release: "0.6.0" hideonrelease_notes: false badges: type: "new feature" domains: ["transforms"] transforms: ["awsec2metadata"] aliases: ["/blog/ec2-metadata"] Are your events the laughing-stock of the data warehouse? Then enrich them with our brand spanking new . <!--more--> Configuration isn't complicated, just add and hook up the transform. If you don't want all enrichments added then white-list them with the `fields` option: ```toml [transforms.fillmeup] type = "awsec2metadata" inputs = ["my-source-id"] fields = [ "instance-id", "local-hostname", "public-hostname", "public-ipv4", "ami-id", "availability-zone", "region", ] ``` For more guidance get on the . Data is better when it's thicc
Project: Icinga (Observability and Analysis / Observability)
File: 01-about.md

is a monitoring system which checks the availability of your network resources, notifies users of outages, and generates performance data for reporting. Scalable and extensible, Icinga can monitor large, complex environments across multiple locations. This includes your data center as well as your private, public, or hybrid clouds. Icinga 2 is the monitoring server and requires on top in your Icinga Stack. The can be easily managed with either the , config management tools or plain text within the . Once Icinga Server and Web are running in your distributed environment, make sure to check out the many for even better monitoring. You can follow the development and release milestones on . Please follow our release announcements on too. Check the project website at for status updates. Join the for questions or get in touch for . There are many ways to contribute to Icinga -- whether it be sending patches, testing, reporting bugs or reviewing and updating the documentation. Every contribution is appreciated! Please continue reading in the . For reporting security issues please visit . The Git repository is located on . Icinga 2 is written in C++ and can be built on Linux/Unix and Windows. Read more about development builds in the . Icinga 2 and the Icinga 2 documentation are licensed under the terms of the GNU General Public License Version 2. You will find a copy of this license in the LICENSE file included in the source package.
Project: Grafana (Observability and Analysis / Observability)
File: release-notes-8-4-1.md

hide_menu: true labels: products: cloud enterprise oss title: Release notes for Grafana 8.4.1 <!-- Auto generated by update changelog github action --> Cloudwatch: Add support for AWS/PrivateLink\* metrics and dimensions. , Configuration: Add ability to customize okta login button name and icon. , Tempo: Switch out Select with AsyncSelect component to get loading state in Tempo Search. , Alerting: Fix migrations by making sendalertsto field nullable. ,
Project: Checkmk (Observability and Analysis / Observability)
File: 16407.md

key | value | date | 2024-03-07T13:27:55+00:00 version | 2.4.0b1 class | fix edition | cee component | omd level | 1 compatible | yes The `omd update` command has the capability to undo the changes it has done up until `Verifying Checkmk configuration...`. However, if any change after `Verifying Checkmk configuration...` is persisted. Due to a regression caused by Werk #15725, the file `config.pb` is deleted during this verification. If the update aborts during the verification, then users will see the following error: ``` Starting cmc...Failed (Config /omd/sites/proddmz/var/checkmk/core/config.pb missing, run "cmk -U" and try again) ``` With this Werk, `config.pb` will be deleting only while `Updating Checkmk configuration...`.
Project: Flagsmith (Observability and Analysis / Feature Flagging)
File: task-processor.md

Flagsmith has the ability to consume asynchronous tasks using a separate task processor service. If flagsmith is run without the asynchronous processor, the flagsmith API will run any asynchronous tasks in a separate, unmanaged thread. The task processor can be run using the flagsmith/flagsmith-api image with a slightly different entrypoint. It should be pointed to the same database that the API container is using. To enable the API sending tasks to the processor, you must set the `TASKRUNMETHOD` to `TASK_PROCESSOR` in the flagsmith container running the flagsmith application. A basic docker-compose setup might look like: ```yaml postgres: image: postgres:15.5-alpine environment: POSTGRES_PASSWORD: password POSTGRES_DB: flagsmith containername: flagsmithpostgres flagsmith: image: flagsmith/flagsmith-api environment: DJANGOALLOWEDHOSTS: '*' DATABASE_URL: postgresql://postgres:password@postgres:5432/flagsmith ENV: prod TASKRUNMETHOD: TASK_PROCESSOR ports: '8000:8000' depends_on: postgres links: postgres flagsmith_processor: build: dockerfile: api/Dockerfile context: . environment: DATABASE_URL: postgresql://postgres:password@postgres:5432/flagsmith command: run-task-processor depends_on: flagsmith postgres ``` The processor exposes a number of configuration options to tune the processor to your needs / setup. These configuration options are via command line arguments when starting the processor. | Argument | Description | Default | | - | - | - | | `--sleepintervalms` | The amount of ms each worker should sleep between checking for a new task | 2000 | | `--numthreads` | The number of worker threads to run per task processor instance | 5 | | `--graceperiodms` | The amount of ms before a worker thread is considered 'stuck'. | 20000 | There are a number of options for monitoring the task processor's health. The task processor includes a management command which checks the health of the worker threads which are running tasks. ``` python manage.py checktaskprocessorthreadhealth ``` The command will exit with either a success exit code (0) or a failure exit code (1). To monitor that the API can send tasks to the processor and that they are successfully run, there is a custom health check which can be enabled on the general health endpoint (`GET /health?format=json`). This health check needs to be enabled manually, which can be done by setting the `ENABLETASKPROCESSORHEALTHCHECK` environment variable to `True` (in the flagsmith application container, not the task processor). Note that this health check is not considered "critical" and hence, the endpoint will return a 200 OK regardless of whether the task processor is sucessfully processing tasks or not. Within the API, there is an endpoint which returns, in JSON format, statistics about the tasks consumed / to be consumed by the processor. This endpoint is available at `GET /processor/monitoring`. This will respond with the following data: ```json { "waiting": 1 } ```
Project: Micrometer (Observability and Analysis / Observability)
File: README.md

](https://circleci.com/gh/micrometer-metrics/micrometer) ](https://www.apache.org/licenses/LICENSE-2.0) ](https://search.maven.org/artifact/io.micrometer/micrometer-core) ](https://javadocs.dev/io.micrometer) ](https://ge.micrometer.io/) An application metrics facade for the most popular monitoring tools. Instrument your code with dimensional metrics with a vendor neutral interface and decide on the monitoring backend at the last minute. More info on . Micrometer artifacts work with Java 8 or later with a few exceptions such as the micrometer-java11 module and micrometer-jetty11. See . Join the to share your questions, concerns, and feature requests. Snapshots are published to `repo.spring.io` for every successful build on the `main` branch and maintenance branches. To use: ```groovy repositories { maven { url 'https://repo.spring.io/snapshot' } } dependencies { implementation 'io.micrometer:micrometer-core:latest.integration' } ``` Milestone releases are published to https://repo.spring.io/milestone. Include that as a maven repository in your build configuration to use milestone releases. Note that milestone releases are for testing purposes and are not intended for production use. The reference documentation is managed in the and published to https://micrometer.io/. See our for information about contributing to Micrometer. Licensed under Sponsored by
Project: Headlamp (Observability and Analysis / Observability)
File: lib_k8s_token.KubeToken.md

title: "Interface: KubeToken" linkTitle: "KubeToken" slug: "libk8stoken.KubeToken" .KubeToken `KubeToken` `Optional` apiVersion: `string` . _ kind: `string` Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. In CamelCase. Cannot be updated. `see` . _ metadata: . _ spec: `Object` | Name | Type | | : | : | | `audiences` | `string`[] | | `expirationSeconds` | `number` | _ status: `Object` | Name | Type | | : | : | | `expirationTimestamp` | `string` | | `token` | `string` |
Project: Grafana Loki (Observability and Analysis / Observability)
File: _index.md

title : "Loki-Operator" description: "A Kubernetes Operator for Loki provided by the Grafana Loki SIG operator. " lead: "A Kubernetes Operator for Loki provided by the Grafana Loki SIG operator. " date: 2021-04-30T09:45:00+00:00 lastmod: 2021-04-30T09:45:00+00:00 draft: false images: []
Project: Infracost (Observability and Analysis / Continuous Optimization)
File: SECURITY.md

If you believe you have found a security vulnerability within Infracost, please let us know right away. We'll try and fix the problem as soon as possible. Do not report vulnerabilities using public GitHub issues. Instead, email with a detailed account of the issue. Please submit one issue per email, this helps us triage vulnerabilities. Once we've received your email we'll keep you updated as we fix the vulnerability. We release patches for security vulnerabilities as soon as they are found and fixed. Please refer to the below table to understand which CLI versions are eligible for security patches. | Version | Supported | ||| | 0.10.x | | | 0.9.x | | | < 0.9.0 | |
Project: Netdata (Observability and Analysis / Observability)
File: vm.stats.vm.v_swappgs.md

<!--startmeta customediturl: "https://github.com/netdata/netdata/edit/master/src/collectors/freebsd.plugin/integrations/vm.stats.vm.v_swappgs.md" meta_yaml: "https://github.com/netdata/netdata/edit/master/src/collectors/freebsd.plugin/metadata.yaml" sidebarlabel: "vm.stats.vm.vswappgs" learn_status: "Published" learnrelpath: "Collecting Metrics/FreeBSD" most_popular: False message: "DO NOT EDIT THIS FILE DIRECTLY, IT IS GENERATED BY THE COLLECTOR'S metadata.yaml FILE" endmeta--> <img src="https://netdata.cloud/img/memory.svg" width="150"/> Plugin: freebsd.plugin Module: vm.stats.vm.v_swappgs <img src="https://img.shields.io/badge/maintained%20by-Netdata-%2300ab44" /> The metric swap amount of data read from and written to SWAP. This collector is supported on all platforms. This collector supports collecting metrics from multiple instances of this integration, including remote instances. This integration doesn't support auto-detection. The default configuration for this integration does not impose any limits on data collection. The default configuration for this integration is not expected to impose a significant performance impact on the system. Metrics grouped by scope. The scope defines the instance that the metric belongs to. An instance is uniquely identified by a set of labels. This metric shows events happening on SWAP. This scope has no labels. Metrics: | Metric | Dimensions | Unit | |:|:-|:-| | mem.swapio | io, out | KiB/s | The following alerts are available: | Alert name | On metric | Description | |:|:-|:| | | mem.swapio | percentage of the system RAM swapped in the last 30 minutes | No action required. The configuration file name for this integration is `netdata.conf`. Configuration for this specific integration is located in the `[plugin:freebsd]` section within that file. The file format is a modified INI syntax. The general structure is: ```ini [section1] option1 = some value option2 = some other value [section2] option3 = some third value ``` You can edit the configuration file using the `edit-config` script from the Netdata . ```bash cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata sudo ./edit-config netdata.conf ``` <details><summary>Config options</summary> | Name | Description | Default | Required | |:-|:--|:-|:--:| | vm.stats.vm.v_swappgs | Enable or disable infoormation about SWAP I/O metric. | yes | no | </details> There are no configuration examples.
Project: Checkmk (Observability and Analysis / Observability)
File: 16456.md

key | value | date | 2024-03-11T09:10:30+00:00 version | 2.4.0b1 class | fix edition | cme component | omd level | 1 compatible | yes This situation can be fixed by stopping and starting remote sites via 'omd stop' and 'omd start',
Project: Netdata (Observability and Analysis / Observability)
File: vm.vmtotal.md

<!--startmeta customediturl: "https://github.com/netdata/netdata/edit/master/src/collectors/freebsd.plugin/integrations/vm.vmtotal.md" meta_yaml: "https://github.com/netdata/netdata/edit/master/src/collectors/freebsd.plugin/metadata.yaml" sidebar_label: "vm.vmtotal" learn_status: "Published" learnrelpath: "Collecting Metrics/FreeBSD" most_popular: False message: "DO NOT EDIT THIS FILE DIRECTLY, IT IS GENERATED BY THE COLLECTOR'S metadata.yaml FILE" endmeta--> <img src="https://netdata.cloud/img/memory.svg" width="150"/> Plugin: freebsd.plugin Module: vm.vmtotal <img src="https://img.shields.io/badge/maintained%20by-Netdata-%2300ab44" /> Collect Virtual Memory information from host. The plugin calls function `sysctl` to collect data. This collector is supported on all platforms. This collector supports collecting metrics from multiple instances of this integration, including remote instances. This integration doesn't support auto-detection. The default configuration for this integration does not impose any limits on data collection. The default configuration for this integration is not expected to impose a significant performance impact on the system. Metrics grouped by scope. The scope defines the instance that the metric belongs to. An instance is uniquely identified by a set of labels. These metrics show an overall vision about processes running. This scope has no labels. Metrics: | Metric | Dimensions | Unit | |:|:-|:-| | system.active_processes | active | processes | | system.processes | running, blocked | processes | | mem.real | used | MiB | The following alerts are available: | Alert name | On metric | Description | |:|:-|:| | | system.active_processes | system process IDs (PID) space utilization | No action required. The configuration file name for this integration is `netdata.conf`. Configuration for this specific integration is located in the `[plugin:freebsd:vm.vmtotal]` section within that file. The file format is a modified INI syntax. The general structure is: ```ini [section1] option1 = some value option2 = some other value [section2] option3 = some third value ``` You can edit the configuration file using the `edit-config` script from the Netdata . ```bash cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata sudo ./edit-config netdata.conf ``` <details><summary>Config Options</summary> | Name | Description | Default | Required | |:-|:--|:-|:--:| | enable total processes | Number of active processes. | yes | no | | processes running | Show number of processes running or blocked. | yes | no | | real memory | Memeory used on host. | yes | no | </details> There are no configuration examples.
Project: Grafana Pyroscope (Observability and Analysis / Observability)
File: python-span-profiles.md

title: Span profiles with Traces to profiles for Python menuTitle: Span profiles with Traces to profiles (Python) description: Learn about and configure Span profiles with Traces to profiles in Grafana for Python applications. weight: 104 Span Profiles represents a major shift in profiling methodology, enabling deeper analysis of both tracing and profiling data. Traditional continuous profiling provides an application-wide view over fixed intervals. In contrast, Span Profiles delivers focused, dynamic analysis on specific execution scopes within applications, such as individual requests or specific trace spans. This shift enables a more granular view of performance, enhancing the utility of profiles by linking them directly with traces for a comprehensive understanding of application behavior. As a result, engineering teams can more efficiently identify and address performance bottlenecks. To learn more about Span Profiles, refer to . Pyroscope integrates with distributed tracing systems supporting the standard. This integration lets you link traces with the profiling data and find resource usage for specific lines of code for your trace spans. {{< admonition type="note" >}} Only CPU profiling is supported at the moment. Because of how sampling profilers work, spans shorter than the sample interval may not be captured. {{< /admonition >}} To use Span Profiles, you need to: Configure a client-side package to link traces and profiles: Your applications must be instrumented for profiling and tracing before you can use span profiles. Profiling: Your application must be instrumented with Pyroscope's Python instrumentation library. Refer to the guide for instructions. Tracing: Your application must be instrumented with OpenTelemetry traces. Refer to the guide for isntructions. To start collecting Span Profiles for your Python application, you need to include in your code. This package provides a implementation, which connects the two telemetry signals (traces and profiles) together. ```shell pip install pyroscope-otel ``` Next, create and register the `PyroscopeSpanProcessor`: ```python from pyroscope-otel import PyroscopeSpanProcessor from opentelemetry import trace from opentelemetry.sdk.trace import TracerProvider provider = TracerProvider() provider.addspanprocessor(PyroscopeSpanProcessor()) trace.settracerprovider(provider) ``` With the span processor registered, spans created automatically (for example, HTTP handlers) and manually will have profiling data associated with them. To view the span profiles in Grafana Tempo, you need to have a Grafana instance running and a data source configured to link traces and profiles. Refer to the to see how to configure the visualization to link traces with profiles. Check out the directory for a complete demo application of span profiles in multiple languages.
Project: Checkmk (Observability and Analysis / Observability)
File: 16687.md

key | value | date | 2024-04-15T07:19:19+00:00 version | 2.4.0b1 class | fix edition | cre component | checks level | 1 compatible | yes This only affects plugin developers using the new API in `cmk.serversidecalls`. The config generation for custom plugins crashed when passwords where in cluded in the command. The mechanism how to pass `Secret`s has changed since 2.3.0b3. To learn how to adjust your special agent or active check to the new API, please refer to the Sphinx documentation of the `cmk.serversidecalls.v1.Secret` object.