Project: SkyWalking (Observability and Analysis / Observability)
File: exporter.md

SkyWalking provides the essential functions of observability, including metrics aggregation, trace, log, alerting, and profiling. In many real-world scenarios, users may want to forward their data to a 3rd party system for further in-depth analysis. Exporter has made that possible. The exporter is an independent module that has to be manually activated. Right now, we provide the following exporting channels: gRPC Exporter Kafka Exporter Metrics gRPC exporter uses SkyWalking's native export service definition. Here is the proto definition: . ```proto service MetricExportService { rpc export (stream ExportMetricValue) returns (ExportResponse) { } rpc subscription (SubscriptionReq) returns (SubscriptionsResp) { } } ``` To activate the exporter, you should set `${SWEXPORTER:default}` and `${SWEXPORTERENABLEGRPC_METRICS:true}`, configure the target gRPC server address. ```yaml exporter: selector:${SW_EXPORTER:default} default: enableGRPCMetrics: ${SWEXPORTERENABLEGRPCMETRICS:true} gRPCTargetHost: ${SWEXPORTERGRPC_HOST:127.0.0.1} gRPCTargetPort: ${SWEXPORTERGRPC_PORT:9870} ... ``` `gRPCTargetHost`:`gRPCTargetPort` is the expected target service address. You could set any gRPC server to receive the data. Target gRPC service needs to go on standby; otherwise, the OAP startup may fail. Subscription implementation. Return the expected metrics name list with event type (incremental or total). All names must match the OAL/MAL script definition. Return empty list, if you want to export all metrics in the incremental event type. Export implementation. Stream service. All subscribed metrics will be sent here based on the OAP core schedule. Also, if the OAP is deployed as a cluster, this method will be called concurrently. Trace kafka exporter pushes messages to the Kafka Broker and Topic `skywalking-export-trace` to export the trace. Here is the message: ``` ProducerRecord<String, Bytes> Key: TraceSegmentId Value: Bytes of SegmentObject ``` The `SegmentObject` definition follows the protocol: . ```proto // The segment is a collection of spans. It includes all collected spans in a simple one request context, such as a HTTP request process. message SegmentObject { string traceId = 1; string traceSegmentId = 2; repeated SpanObject spans = 3; string service = 4; string serviceInstance = 5; bool isSizeLimited = 6; } ``` To activate the exporter, you should set `${SWEXPORTER:default}` and `${SWEXPORTERENABLEKAFKA_TRACE:true}`, configure the Kafka server addresses. ```yaml exporter: selector:${SW_EXPORTER:default} default: enableKafkaTrace: ${SWEXPORTERENABLEKAFKATRACE:true} kafkaBootstrapServers: ${SWEXPORTERKAFKA_SERVERS:localhost:9092} kafkaProducerConfig: ${SWEXPORTERKAFKAPRODUCERCONFIG:""} kafkaTopicTrace: ${SWEXPORTERKAFKATOPICTRACE:skywalking-export-trace} exportErrorStatusTraceOnly: ${SWEXPORTERKAFKATRACEFILTER_ERROR:false} ... ``` `exportErrorStatusTraceOnly=true` represents that only export the error status trace segments through the Kafka channel. Log kafka exporter pushes messages to the Kafka Broker and Topic `skywalking-export-log` to export the log. Here is the message: ``` ProducerRecord<String, Bytes> Key: LogRecordId Value: Bytes of LogData ``` The `LogData` definition follows the protocol: . ```proto message LogData { int64 timestamp = 1; string service = 2; string serviceInstance = 3; string endpoint = 4; LogDataBody body = 5; TraceContext traceContext = 6; LogTags tags = 7; string layer = 8; } ``` To activate the exporter, you should set `${SWEXPORTER:default}` and `${SWEXPORTERENABLEKAFKA_LOG:true}`, configure the Kafka server addresses. ```yaml exporter: selector:${SW_EXPORTER:default} default: enableKafkaLog: ${SWEXPORTERENABLEKAFKALOG:true} kafkaBootstrapServers: ${SWEXPORTERKAFKA_SERVERS:localhost:9092} kafkaProducerConfig: ${SWEXPORTERKAFKAPRODUCERCONFIG:""} kafkaTopicLog: ${SWEXPORTERKAFKATOPICLOG:skywalking-export-log} ... ```