cwielenb/CNCF-Dataset huggingface dataset:
Project: Opstrace (Observability and Analysis / Observability)
File: sending-logs-with-fluentd.md

In this guide we show how to configure a Fluentd instance to securely send logs to your Opstrace instance. We appreciate Fluentd for its robustness, especially with respect to log file rotation handling, node-local log data , and its mature error handling architecture. It is worth mentioning that Fluentd has decent debuggability, with insightful logging around retries and errors in particular. An Opstrace instance. A decision: for which Opstrace tenant would you like to send data? An Opstrace tenant authentication token file (for the tenant of your choice). Also see . A Fluentd instance can be instructed to send logs to an Opstrace instance by using the `@type loki` output plugin (, ). What follows is an example for a block matching all log entries, and for sending them to your Opstrace instance: ```xml <match > @type loki url <string> insecure_tls <boolean> bearertokenfile <filepath> </match> ``` This is the tenant-specific data API endpoint of an Opstrace instance for ingesting metrics. The endpoint URL is constructed using the tenant name and the instance name: ```xml url https://loki.${TENANTNAME}.${INSTANCENAME}.opstrace.io/loki/api/v1/push ``` Example: ```xml url https://loki.default.test.opstrace.io/loki/api/v1/push ``` Your Opstrace instance exposes the logs ingestion API securely. For authenticating the client towards the Opstrace instance., the client has to present a tenant-specific data API authentication token with every request. To achieve that, we use the `bearertokenfile` parameter. Example: ```xml bearertokenfile /var/run/default-tenant/tenant-api-token-default ``` TLS is used for establishing a secure, private transport between the Fluentd instance and the Opstrace instance.. By default, Fluentd attempts to verify the certificate exposed by the Opstrace instance. against its trust store, for establishing authenticity of the Opstrace instance towards itself. Depending on the `cert_issuer` instance. configuration option (see ) you may want to disable server certificate verification for testing purposes, which can be achieved via ```xml insecure_tls true ``` This is required for example when using `letsencrypt-staging` as `cert_issuer`. We recommend reading into the documentation. For example, when choosing a node-local Fluentd buffer of one can maximize the likelihood to recover from failures without losing valuable log data (the node-local persistent buffer can be flushed eventually -- Fluentd's default retrying timeout is 3 days). What follows is a configuration structure that serves as a starting point for more advanced flushing and buffering configuration: ```xml <match > @type loki url <string> insecure_tls <boolean> bearertokenfile <filepath> <buffer> @type file path <filepath> chunklimitsize 512kb flush_interval 3s </buffer> </match> ``` Note that Fluentd can also back up log entries that it failed to flush. (buffering parameters, flushing parameters, retrying parameters)
Project: Krkn (Observability and Analysis / Chaos Engineering)
File: pod_network_scenarios.md

Scenario to block the traffic ( Ingress/Egress ) of a pod matching the labels for the specified duration of time to understand the behavior of the service/other services which depend on it during downtime. This helps with planning the requirements accordingly, be it improving the timeouts or tweaking the alerts etc. With the current network policies, it is not possible to explicitly block ports which are enabled by allowed network policy rule. This chaos scenario addresses this issue by using OVS flow rules to block ports related to the pod. It supports OpenShiftSDN and OVNKubernetes based networks. ``` id: podnetworkoutage config: namespace: openshift-console # Required - Namespace of the pod to which filter need to be applied direction: # Optioinal - List of directions to apply filters ingress # Blocks ingress traffic, Default both egress and ingress ingress_ports: # Optional - List of ports to block traffic on 8443 # Blocks 8443, Default [], i.e. all ports. label_selector: 'component=ui' # Blocks access to openshift console ``` Scenario to introduce network latency, packet loss, and bandwidth restriction in the Pod's network interface. The purpose of this scenario is to observe faults caused by random variations in the network. ``` id: podegressshaping config: namespace: openshift-console # Required - Namespace of the pod to which filter need to be applied. label_selector: 'component=ui' # Applies traffic shaping to access openshift console. network_params: latency: 500ms # Add 500ms latency to egress traffic from the pod. ``` ``` id: podingressshaping config: namespace: openshift-console # Required - Namespace of the pod to which filter need to be applied. label_selector: 'component=ui' # Applies traffic shaping to access openshift console. network_params: latency: 500ms # Add 500ms latency to egress traffic from the pod. ``` Pick the pods to introduce the network anomaly either from labelselector or podname. Identify the pod interface name on the node. Set traffic shaping config on pod's interface using tc and netem. Wait for the duration time. Remove traffic shaping config on pod's interface. Remove the job that spawned the pod.
Project: SkyWalking (Observability and Analysis / Observability)
File: agent-compatibility.md

SkyWalking 8.0+ uses v3 protocols. Agents don't have to keep the identical versions as the OAP backend. | OAP Server Version | Java | Python | NodeJS | LUA | Kong | Browser Agent | Rust | PHP | Go | Rover | Satellite | |--|-|--|--|--||||--|||| | 8.0.1 - 8.1.0 | 8.0.0 - 8.3.0 | < = 0.6.0 | < = 0.3.0 | All | All | No | All | No | No | No | No | | 8.2.0 - 8.3.0 | 8.0.0 - 8.3.0 | < = 0.6.0 | < = 0.3.0 | All | All | All | All | No | No | No | No | | 8.4.0 - 8.8.1 | \> = 8.0.0 | All | All | All | All | All | All | All | No | No | No | | 8.9.0+ | \> = 8.0.0 | All | All | All | All | All | All | All | No | No | \> = 0.4.0 | | 9.0.0 | \> = 8.0.0 | All | All | All | All | All | All | All | No | No | \> = 0.4.0 | | 9.1.0+ | \> = 8.0.0 | All | All | All | All | All | All | All | No | \> = 0.1.0 | \> = 1.0.0 | | 9.5.0+ | \> = 8.0.0 & \> = 9.0.0 | All | All | All | All | All | All | All | \> = 0.1.0 | \> = 0.5.0 | \> = 1.2.0 | All following agent implementations are a part of the SkyWalking ecosystem. All the source codes and their distributions don't belong to the Apache Software Foundation. | OAP Server Version | DotNet | cpp2sky | |--||--| | 8.0.1 - 8.3.0 | 1.0.0 - 1.3.0 | < = 0.2.0 | | 8.4.0+ | \> = 1.0.0 | All | | 9.0.0+ | \> = 1.0.0 | All | All these projects are maintained by their own communities, and please reach them if you face any compatibility issues. _ All above compatibility are only references, and if you face an `unimplemented` error, it means you need to upgrade the OAP backend to support newer features in the agents.
Project: SkyWalking (Observability and Analysis / Observability)
File: Component-library-settings.md

Component library settings are about your own or third-party libraries used in the monitored application. In agent or SDK, regardless of whether the library name is collected as ID or String (literally, e.g. SpringMVC), the collector formats data in ID for better performance and less storage requirements. Also, the collector conjectures the remote service based on the component library. For example: if the component library is MySQL Driver library, then the remote service should be MySQL Server. For these two reasons, the collector requires two parts of settings in this file: Component library ID, names and languages. Remote server mapping based on the local library. All component names and IDs must be defined in this file. Define all names and IDs from component libraries which are used in the monitored application. This uses a two-way mapping strategy. The agent or SDK could use the value (ID) to represent the component name in uplink data. Name: the component name used in agent and UI ID: Unique ID. All IDs are reserved once they are released. Languages: Program languages may use this component. Multi languages should be separated by `,`. Java and multi languages shared: (0, 3000) .NET Platform reserved: [3000, 4000) Node.js Platform reserved: [4000, 5000) Go reserved: [5000, 6000) Lua reserved: [6000, 7000) Python reserved: [7000, 8000) PHP reserved: [8000, 9000) C++ reserved: [9000, 10000) Javascript reserved: [10000, 11000) Rust reserved: [11000, 12000) Example: ```yaml Tomcat: id: 1 languages: Java HttpClient: id: 2 languages: Java,C#,Node.js Dubbo: id: 3 languages: Java H2: id: 4 languages: Java ``` The remote server will be conjectured by the local component. The mappings are based on names in the component library. Key: client component library name Value: server component name ```yaml Component-Server-Mappings: Jedis: Redis StackExchange.Redis: Redis Redisson: Redis Lettuce: Redis Zookeeper: Zookeeper SqlClient: SqlServer Npgsql: PostgreSQL MySqlConnector: Mysql EntityFrameworkCore.InMemory: InMemoryDatabase ```
Project: Zipkin (Observability and Analysis / Observability)
File: bug.md

name: Bug about: If youve found a bug, spend the time to write a failing test. Bugs with tests get fixed and stay fixed. If you have a solution in mind, skip raising an issue and open a pull request instead. labels: bug A clear and concise description of what the bug is. If you have a solution in mind, skip raising an issue and open a pull request instead. If this is a UI issue... Attach a screen shot or animated gif showing what you think is wrong Include JSON of a trace that produces it, being careful to not include private data You can literally include the JSON in Otherwise, you can use a or pastebin such as Regardless, the best is to spend some time to write a failing test. Bugs with tests get fixed and stay fixed. Steps to reproduce the behavior: Suggest what you think correct behaviour should be. Note, it may be solved differently depending on the problem.
Project: Netdata (Observability and Analysis / Observability)
File: 1m_received_traffic_overflow.md

Network interfaces are categorized primarily on the bandwidth they can operate (1 Gbps, 10 Gbps, etc). High network utilization occurs when the volume of data on a network link approaches the capacity of the link. Netdata agent calculates the average outbound utilization for a specific network interface over the last minute. High outbound utilization increases latency and packet loss because packet bursts are buffered This alarm may indicate either network congestion or malicious activity. Prioritize important traffic Quality of service (QoS) is the use of routing prioritization to control traffic and ensure the performance of critical applications. QoS works best when low-priority traffic exists that can be dropped when congestion occurs. The higher-priority traffic must fit within the bandwidth limitations of the link or path. Add more bandwidth For Cloud infrastructures, adding bandwidth might be easy. It depends on your cloud infrastracture and your cloud provider. Some of them either offer you the service to upgrade machines to a higher bandwidth rate or upgrade you machine to a more powerful one with higher bandwidth rate. For Bare-metal machines, you will need either a hardware upgrade or the addition of a network card using link aggregation to combine multiple network connections in parallel (e.g LACP). is a traffic shaping helper. It has a very simple shell scripting language to express traffic shaping. is a command wrapper that makes it easy to set up traffic control of network bandwidth/latency/packet-loss/packet-corruption/etc.
Project: Checkmk (Observability and Analysis / Observability)
File: 16628.md

key | value | date | 2024-04-04T14:47:34+00:00 version | 2.4.0b1 class | fix edition | cre component | wato level | 1 compatible | yes Before this werk, the site failed to redirect users to the "Change Password" page following a successful login when two-factor authentication (2FA) was enabled. This werk resolves the issue by ensuring that, after completing 2FA, users are now redirected correctly.
Project: Flagsmith (Observability and Analysis / Feature Flagging)
File: edge-proxy.md

title: Edge Proxy The Flagsmith Edge Proxy is a service that you host yourself, that allows you to run an instance of the Flagsmith Engine close to your servers. If you are running Flagsmith within a server-side environment and you want to have very low latency flags, you have two options: Run the Edge Proxy within in your own infrastructure and connect to it from your server-side SDKs Run your server-side SDKs in . The main benefit to running the Edge Proxy is that you reduce your polling requests against the Flagsmith API itself. The main benefit to running server side SDKs in is that you get the lowest possible latency. :::info The Edge Proxy has the same . ::: You can think of the Edge Proxy as a copy of our Python Server Side SDK, running in , with an API interface that is compatible with the Flagsmith SDK API. The Edge Proxy runs as a lightweight Docker container. It connects to the Flagsmith API (either powered by us at api.flagsmith.com or self hosted by you) to get Environment Flags and Segment rules. You can then point the Flagsmith SDKs to your Edge Proxy; it implements all the current SDK endpoints. This means you can serve a very large number of requests close to your infrastructure and users, at very low latency. Check out the . The Proxy also acts as a local cache, allowing you to make requests to the Proxy without hitting the core API. The Edge Proxy can currently serve ~2,000 requests per second (RPS) at a mean latency of ~7ms on an M1 MacBook Pro with a simple set of flags. Working with more complex Environments with many Segment rules will bring this RPS number down. It is stateless and hence close to perfectly scalable being deployed behind a load balancer. There is one caveat with the Edge Proxy. Because it is entirely stateless, it is not able to persist Trait data into any sort of datastore. This means that you have to provide the full complement of Traits when requesting the Flags for a particular Identity. Our SDKs all provide relevant methods to achieve this. An example using `curl` would read as follows: ```bash curl -X "POST" "http://localhost:8000/api/v1/identities/?identifier=doitallinonegoidentity" \ -H 'X-Environment-Key: n9fbf9h3v4fFgH3U3ngWhb' \ -H 'Content-Type: application/json; charset=utf-8' \ -d $'{ "traits": [ { "trait_value": 123.5, "traitkey": "mytrait_key" }, { "trait_value": true, "traitkey": "myother_key" } ], "identifier": "doitallinonegoidentity" }' ``` Note that the Edge Proxy will currently not send the Trait data back to the Core API. Please see the . The standard Flagsmith architecture: With the proxy added to the mix:
Project: Headlamp (Observability and Analysis / Observability)
File: lib_k8s_job.KubeJob.md

title: "Interface: KubeJob" linkTitle: "KubeJob" slug: "libk8sjob.KubeJob" .KubeJob `KubeJob` `Optional` apiVersion: `string` . _ kind: `string` Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. In CamelCase. Cannot be updated. `see` . _ metadata: . _ spec: `Object` | Name | Type | | : | : | | `selector` | | | `template` | { `metadata?`: ; `spec`: } | | `template.metadata?` | | | `template.spec` | | _ status: `Object`
Project: Grafana Mimir (Observability and Analysis / Observability)
File: before-you-begin.md

aliases: ../../operators-guide/using-exemplars/before-you-begin/ ../../operators-guide/use-exemplars/before-you-begin/ description: Refer to this checklist before you begin using exemplars in Grafana Mimir. menuTitle: Before you begin title: Before you begin using exemplars with Grafana Mimir weight: 20 Follow the checklist to ensure that your application is generating metrics, traces, and exemplars. Verify that your application is using the official Prometheus client libraries. Ensure that the client library you choose is emitting metrics in format by referencing its documentation. For the Prometheus Go client library, for example, this requires you to set `EnableOpenMetrics` to `true`. For the Java library, follow on setting the proper header format. Obtain the trace ID for the current request and include the trace ID in calls to emit metrics. For histograms, use the `ObserveWithExemplar` method to emit the trace ID along with a value for the histogram. These functions are from the Go library but you can find similar functions in the other libraries. For counters, use the `AddWithExemplar` method to emit the trace ID along with a counter increment. Verify that metrics are being generated with exemplars by running the following command in a shell: `curl -H "Accept: application/openmetrics-text" http://<your application>/metrics | grep -i "traceid"`. Configure your Prometheus server or Grafana Agent to store and send exemplars. To configure Grafana Agent to send exemplars: Confirm that the Agent is scraping exemplars by verifying that the `prometheusremotestorageexemplarstotal` metric is a non-zero value. Add the option `sendexemplars: true` under the `remotewrite` configuration block in the Grafana Agent configuration file. To configure a Prometheus server to send exemplars: Run Prometheus with the `--enable-feature=exemplar-storage` flag. Confirm that Prometheus is scraping exemplars by verifying that the `prometheusremotestorageexemplarstotal` metric is a non-zero value. Add the option `sendexemplars: true` under the `remotewrite` configuration block in the Prometheus configuration file. See also:
Project: Checkmk (Observability and Analysis / Observability)
File: 15723.md

key | value | date | 2024-01-23T11:58:49+00:00 version | 2.3.0b1 class | fix edition | cre component | core level | 1 compatible | yes The `cmk-update-config` command, which is usually executed as part of `omd update` expects the site to be stopped during execution. This is always ensured during `omd update`. However, the command can also be executed manually later on. In this situation the site can be running. To prevent unexpected results, the command now protects against an accidentally running site on invocation.
Project: Headlamp (Observability and Analysis / Observability)
File: oidc.md

title: Accessing using OpenID Connect linktitle: OIDC weight: 10 Headlamp supports OIDC for cluster users to effortlessly log in using a "Sign in" button. For OIDC to be used, Headlamp needs to know how to configure it, so you have to provide the different OIDC-related arguments to Headlamp from your OIDC provider. Those are: the client ID: `-oidc-client-id` or env var `HEADLAMPCONFIGOIDCCLIENTID` the client secret: `-oidc-client-secret` or env var `HEADLAMPCONFIGOIDCCLIENTSECRET` the issuer URL: `-oidc-idp-issuer-url` or env var `HEADLAMPCONFIGOIDCIDPISSUER_URL` (optionally) the OpenId scopes: `-oidc-scopes` or env var `HEADLAMPCONFIGOIDC_SCOPES` and you have to tell the OIDC provider about the callback URL, which in Headlamp it is your URL + the `/oidc-callback` path, e.g.: `https://YOUR_URL/oidc-callback`. Besides the mandatory openid scope, Headlamp also requests the optional profile and email . Scopes can be overridden by using the `-oidc-scopes` option. Remember to include the default ones if you need them when using that option. For example, if you need to keep the default scopes and add Github's `repo`, then add them all to the option: `-oidc-scopes=profile,email,repo` Note: Before Headlamp 0.3.0, a scope groups was also included, as it's used by Dex and other services, but since it's not part of the default spec, it was removed in the mentioned version. If you are interested in a comprehensive example of using OIDC and Headlamp, you can check the . If you are using Dex and want to configure Headlamp to use it for OIDC, then you have to: Add the callback URL (e.g. `https://YOUR_URL/oidc-callback`) to Dex's `staticClient.redirectURIs` Set `-oidc-client-id` as Dex's `staticClient.id` Set `-oidc-client-secret` as Dex's `staticClient.secret` Set `-oidc-idp-issuer-url` as Dex's URL (same as in `--oidc-issuer-url` in the Kubernetes APIServer) Set `-oidc-scopes` if needed, e.g. `-oidc-scopes=profile,email,groups` Note If you already have another static client configured for Kubernetes to be used by the (OpenID Connect) configuration, it is important to ensure that a single static client ID i.e `-oidc-client-id` is used for both Dex and Headlamp. Additionally, the redirectURIs need to be specified for each client.
Project: Flagsmith (Observability and Analysis / Feature Flagging)
File: testing-push-notifications.md

title: Testing Push Notifications Sending out Push Notifications (and Email messages!) accurately and without error is notoriously difficult. Combine the fact that they can be extremely sensitive/error prone across dev/staging/production environments, along with the inability to recall them if you've sent them out in error, and you have a subject area that can strike fear into even the most hardened engineer. Deploying code that sends out messages to potentially millions of users is not a relaxing experience. When you send a push notification via , you can test things out by sending messages to a specific device via a device token. The problem is that its quite technical to get set up, and these tokens are not readily surfaced within an app. In addition, if you want to test out a campaign its likely you will want to try sending push notifications to more than 1 device so you can test between QA, product owners, developers and marketing when validating behavior and fixing bugs. We want to be able to test out the end-to-end process of sending marketing push notifications, before rolling them out in production. This often means working across marketing, product and engineering teams in what can become a complex, error prone process. Testing this process before pushing things live is one of the best ways of making sure that your procedures are working. You can utilise Flagsmith in combination with Firebase FCM topics to make this easier. When sending a push notification to a topic, Firebase will send that notification to all the users that are subscribed to that named topic. We will eventually use these topics to target marketing messages to our user-base, but we can also use them to test out the messaging with our team before sending them out to all our users. We're going to create a flag in Flagsmith called `fcmmarketingbeta`. We want anyone with this flag enabled in Flagsmith to be subscribed to the `marketing` FCM topic. We will then send messages out via this topic. We use the following code in our React Native application to do this: ```javascript const isInMarketingBeta = flagsmith.hasFeature('fcmmarketingbeta'); if (isInMarketingBeta) { messaging().subscribeToTopic('marketing'); } ``` Once our code is deployed, we can start enabling this flag for individual users of our application. Using the aspect of Flagsmith, we can enable this flag for our user in the application, as well as enabling it for some of our close team mates. Refreshing the application will cause the `messaging().subscribeToTopic('marketing');` code to execute, adding us to the FCM topic. Our device has now appeared in this topic, and we can use the topic to start sending test marketing messages. We can now make use of to add all our team to this beta FCM group. Create a Segment, add a rule that will include our team (in our case we match against the domain name of an email Trait), and override the `fcmmarketingbeta` Flag with this Segment. We're now able to test these messages with our entire team, no further code or deployments required!