Project: Netdata (Observability and Analysis / Observability)
File: how-to-write-a-module.md

<!-- title: "How to write a Netdata collector in Go" description: "This guide will walk you through the technical implementation of writing a new Netdata collector in Golang, with tips on interfaces, structure, configuration files, and more." customediturl: "https://github.com/netdata/netdata/blob/master/src/go/collectors/go.d.plugin/docs/how-to-write-a-module.md" sidebar_label: "How to write a Netdata collector in Go" learn_status: "Published" learntopictype: "Tasks" learnrelpath: "Developers/External plugins/go.d.plugin" sidebar_position: 20 --> Take a look at our . this repository to your personal GitHub account. locally the forked repository (e.g `git clone https://github.com/odyslam/go.d.plugin`). Using a terminal, `cd` into the directory (e.g `cd go.d.plugin`) :exclamation: You can skip most of these steps if you first experiment directy with the existing , which will give you an idea of how things work. Let's assume you want to write a collector named `example2`. The steps are: Add the source code to . . . . Add the configuration to . Add the module to . Import the module in . Update the . To build it, run `make` from the plugin root dir. This will create a new `go.d.plugin` binary that includes your newly developed collector. It will be placed into the `bin` directory (e.g `go.d.plugin/bin`) Run it in the debug mode `bin/godplugin -d -m <MODULE_NAME>`. This will output the `STDOUT` of the collector, the same output that is sent to the Netdata Agent and is transformed into charts. You can read more about this collector API in our . If you want to test the collector with the actual Netdata Agent, you need to replace the `go.d.plugin` binary that exists in the Netdata Agent installation directory with the one you just compiled. Once you the Netdata Agent, it will detect and run it, creating all the charts. It is advised not to remove the default `go.d.plugin` binary, but simply rename it to `go.d.plugin.old` so that the Agent doesn't run it, but you can easily rename it back once you are done. Run `make clean` when you are done with testing. Every module should implement the following interface: ``` type Module interface { Init() bool Check() bool Charts() *Charts Collect() map[string]int64 Cleanup() } ``` `Init` does module initialization. If it returns `false`, the job will be disabled. We propose to use the following template: ``` // example.go func (e *Example) Init() bool { err := e.validateConfig() if err != nil { e.Errorf("config validation: %v", err) return false } someValue, err := e.initSomeValue() if err != nil { e.Errorf("someValue init: %v", err) return false } e.someValue = someValue // ... return true } ``` Move specific initialization methods into the `init.go` file. See . `Check` returns whether the job is able to collect metrics. Called after `Init` and only if `Init` returned `true`. If it returns `false`, the job will be disabled. The simplest way to implement `Check` is to see if we are getting any metrics from `Collect`. A lot of modules use such approach. ``` // example.go func (e *Example) Check() bool { return len(e.Collect()) > 0 } ``` :exclamation: Netdata module produces , not raw metrics. Use package to create them, it contains charts and dimensions structs. `Charts` returns the (`*module.Charts`). Called after `Check` and only if `Check` returned

If it returns `nil`, the job will be disabled :warning: Make sure not to share returned value between module instances (jobs). Usually charts initialized in `Init` and `Chart` method just returns the charts instance: ``` // example.go func (e Example) Charts() Charts { return e.charts } ``` `Collect` collects metrics. Called only if `Check` returned `true`. Called every `update_every` seconds. `map[string]int64` keys are charts dimensions ids'. We propose to use the following template: ``` // example.go func (e *Example) Collect() map[string]int64 { ms, err := e.collect() if err != nil { e.Error(err) } if len(ms) == 0 { return nil } return ms } ``` Move metrics collection logic into the `collect.go` file. See . `Cleanup` performs the job cleanup/teardown. Called if `Init` or `Check` fails, or we want to stop the job after `Collect`. If you have nothing to clean up: ``` // example.go func (Example) Cleanup() {} ``` The general idea is to not put everything in a single file. We recommend using one file per logical area. This approach makes it easier to maintain the module. Suggested minimal layout: | Filename | Contains | ||--| | | Module configuration, implementation and registration. | | | Charts, charts templates and constructor functions. | | | Initialization methods. | | | Metrics collection implementation. | | | Public methods/functions tests. | | | Files containing sample data. | :exclamation: See the example . Don't overload this file with the implementation details. Usually it contains only: module registration. module configuration. . :exclamation: See the example: . Put charts, charts templates and charts constructor functions in this file. :exclamation: See the example: . All the module initialization details should go in this file. make a function for each value that needs to be initialized. a function should return a value(s), not implicitly set/change any values in the main struct. ``` // init.go // Prefer this approach. func (e Example) initSomeValue() (someValue, error) { // ... return someValue, nil } // This approach is ok too, but we recommend to not use it. func (e *Example) initSomeValue() error { // ... m.someValue = someValue return nil } ``` :exclamation: See the example: . This file is the entry point for the metrics collection. Feel free to split it into several files if you think it makes the code more readable. Use `collect` prefix for the filenames: `collectthis.go`, `collect_that.go`, etc. ``` // collect.go func (e *Example) collect() (map[string]int64, error) { collected := make(map[string])int64 // ... // ... // ... return collected, nil } ``` :exclamation: See the example: . if you have no experience in testing we recommend starting with . we use `assert` and `require` packages from library, check . Testing is mandatory. test only public functions and methods (`New`, `Init`, `Check`, `Charts`, `Cleanup`, `Collect`). do not create a test function per a case, use . Prefer `map[string]struct{ ... }` over `[]struct{ ... }`. use helper functions to prepare test cases to keep them clean and readable. Put files with sample data in this directory if you need any. Its name should be . Directory and file names that begin with "." or "_" are ignored by the go tool, as are directories named "testdata". There are for writing a module.