Repository: akvorado/akvorado

Issue:
Kafka consumers wont ever exceed 16
This can be due to the fact clickhouse wont spawn more threads than there is logical cores, but there's also a thread limit on clickhouse that's enforced with other various settings which are defaulted to 16 everywhere in clickhouse.

It seems the most common way to bypass the limitation is to set kafka_disable_num_consumers_limit to 1.

Would worth documenting this and providing a way to turn this on, as it can be quite limiting.

Hints from issue comments:
@vincentbernat: Is it a settings to set in engine or in ClickHouse configuration? In both case, we can put it in the documentation.@kair1337: Hi @vincentbernat 

I think the way to do this properly is like so, in the docker compose
`- ../config/clickhouse/kafka_tweaks.xml:/etc/clickhouse-server/config.d/kafka_tweaks.xml:ro`

kafka_tweaks.xml:
```
<clickhouse>
    <profiles>
        <default>
            <kafka_disable_num_consumers_limit>1</kafka_disable_num_consumers_limit>
        </default>
    </profiles>

    <background_message_broker_schedule_pool_size>32</background_message_broker_schedule_pool_size>
    <background_distributed_schedule_pool_size>32</background_distributed_schedule_pool_size>
</clickhouse>
```


As per the title of the issue, by default clickhouse will limit you to 16 consumers for Kafka which isn't enough in heavy deployments. Also if using a VM it will /2 the vCPUs as your limit, kafka_disable_num_consumers_limit turns that off and allows it to go past 16 in general so long as the other two settings which are defaulted to 16 are set higher than 16.

I haven't had the chance to fully test this myself on a heavy running Akvorado instance, but I've noticed the main bottleneck has been not being able to scale out the consumers, which this should resolve.@vincentbernat: I plan to remove the use of Kafka engine in the next version (for other reasons). It would be replaced by another component doing batched insert in ClickHouse.@IgorSh16: Hello!

My server has 40 CPU cores. Usually clickhouse-serv process consumes 1200% CPU or less. But when autonomous system is under attack, CPU usage of clickhouse-serv process
reaches up to 1600% and doesn't grow more. 
Kafka can't write all flows into ClickHouse right in the moment, probably, store it in the queues and will push them later, when CPU usage become less 1600%
But I would like to have information in the database at the right time.

I've created file kafka_tweaks.xml and added "- ../config/clickhouse/kafka_tweaks.xml:/etc/clickhouse-server/config.d/kafka_tweaks.xml:ro" into docker-compose.yml

Tried to restart in different way
```
docker compose restart
docker compose down
docker compose up -d
docker exec docker-clickhouse-1 service clickhouse-server restart
```

Logs show that config was merged...

```
docker logs docker-clickhouse-1 2>&1

Merging configuration file '/etc/clickhouse-server/config.d/akvorado.xml'.
Merging configuration file '/etc/clickhouse-server/config.d/docker_related_config.xml'.
Merging configuration file '/etc/clickhouse-server/config.d/kafka_tweaks.xml'.
```


...and new system settings are in /var/lib/clickhouse/preprocessed_configs/config.xml file

```
docker exec docker-clickhouse-1 cat /var/lib/clickhouse/preprocessed_configs/config.xml | grep consumer
            <kafka_disable_num_consumers_limit>1</kafka_disable_num_consumers_limit>

docker exec docker-clickhouse-1 cat /var/lib/clickhouse/preprocessed_configs/config.xml | grep _pool_size
    <background_message_broker_schedule_pool_size>16</background_message_broker_schedule_pool_size>
    <background_distributed_schedule_pool_size>16</background_distributed_schedule_pool_size>
    <background_message_broker_schedule_pool_size>32</background_message_broker_schedule_pool_size>
    <background_distributed_schedule_pool_size>32</background_distributed_schedule_pool_size>
```



Both files clickhouse-server.log and clickhouse-server.err.log don't contain errors related to these 3 settings
But clickhouse-client shows old unupdated settings and CPU usage of clickhouse-server reach 1600% and not more

```
SELECT name,value FROM system.settings WHERE name LIKE '%consumer%' FORMAT TabSeparated

kafka_disable_num_consumers_limit       0

SELECT * FROM system.settings WHERE name LIKE '%_pool_size%' FORMAT TabSeparated

background_message_broker_schedule_pool_size    16      
background_distributed_schedule_pool_size       16

SELECT version()  FORMAT TabSeparated

24.3.10.33
```


Maybe you know what is my mistake?@randomizedcoder: Cool project.  It's interesting you comment about moving away from Kafka.  It would be great to understand more about why.

I'm doing a similar kind of project, and am definitely concerned about consumer lag.  In our production environment we DO see production lag, so I've been working to move to ProtobufList 
https://www.mux.com/blog/latency-and-throughput-tradeoffs-of-clickhouse-kafka-table-engine@vincentbernat: Reasons include:
- Kafka will be inserted between inlet and outlet. Putting it between outlet and ClickHouse would require to serialize/unserialize again.
- Instead, batch write directly using protobuf format used internally by ClickHouse protocol.

Interesting article. Thanks!

Solution patch:
diff --git a/.env b/.env
index 75f7efd4..41b0d4b9 100644
--- a/.env
+++ b/.env
@@ -10,8 +10,8 @@ COMPOSE_FILE=${COMPOSE_FILE}:docker/docker-compose-ipinfo.yml
 # GEOIPUPDATE_LICENSE_KEY=...
 # And you need to update the paths in config/akvorado.yaml.
 
-## Monitoring
-# COMPOSE_FILE=${COMPOSE_FILE}:docker/docker-compose-monitoring.yml
+## Prometheus
+# COMPOSE_FILE=${COMPOSE_FILE}:docker/docker-compose-prometheus.yml
 
 ## Grafana
 # COMPOSE_FILE=${COMPOSE_FILE}:docker/docker-compose-grafana.yml
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 653d764d..df96b063 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -16,7 +16,7 @@ jobs:
     permissions: {}
     if: |
       !startsWith(github.event.head_commit.message, 'build: update Nix dependency hashes') ||
-       startsWith(github.ref, 'refs/tags/')
+       github.ref_type == 'tag'
     steps:
       - name: Fetch dependabot metadata
         if: ${{ github.actor == 'dependabot[bot]' && github.event_name == 'pull_request' }}
@@ -47,8 +47,6 @@ jobs:
         run: docker compose -f docker/docker-compose-dev.yml up --wait --wait-timeout 60 --quiet-pull
       - name: Setup
         uses: ./.github/actions/setup
-
-      # Install dependencies
       - name: Install dependencies
         run: sudo apt-get install -qqy shared-mime-info curl
 
@@ -211,7 +209,7 @@ jobs:
       - build-linux
       - licenses
     runs-on: ubuntu-22.04
-    if: github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/')
+    if: github.ref == 'refs/heads/main' || github.ref_type == 'tag'
     permissions:
       contents: read
       packages: write
@@ -243,7 +241,7 @@ jobs:
         with:
           context: .
           file: docker/Dockerfile
-          platforms: ${{ startsWith(github.ref, 'refs/tags/') && 'linux/amd64,linux/arm64' || 'linux/amd64' }}
+          platforms: ${{ github.ref_type == 'tag' && 'linux/amd64,linux/arm64' || 'linux/amd64' }}
           push: true
           tags: ${{ steps.meta.outputs.tags }}
           labels: ${{ steps.meta.outputs.labels }}
@@ -259,7 +257,7 @@ jobs:
       - build-js
       - docker
     runs-on: ubuntu-latest
-    if: startsWith(github.ref, 'refs/tags/')
+    if: github.ref_type == 'tag'
     permissions:
       contents: write
       actions: read
@@ -299,6 +297,7 @@ jobs:
         with:
           body_path: changelog.md
           draft: true
+          prerelease: ${{ contains(github.ref, 'alpha') || contains(github.ref, 'beta') || contains(github.ref, 'rc') }}
           fail_on_unmatched_files: true
           files: |
             akvorado
diff --git a/.gitignore b/.gitignore
index deec8781..d1c35200 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,4 +1,4 @@
-/bin/
+/bin/akvorado
 /test/
 /orchestrator/clickhouse/data/asns.csv
 /orchestrator/clickhouse/data/tcp.csv
@@ -7,6 +7,7 @@
 /common/schema/definition_gen.go
 mock_*.go
 *_enumer.go
+*.pb.go
 
 /console/data/frontend/
 
diff --git a/Makefile b/Makefile
index 45db462b..8a8260b9 100644
--- a/Makefile
+++ b/Makefile
@@ -6,7 +6,6 @@ DATE    ?= $(shell date +%FT%T%z)
 VERSION ?= $(shell git describe --tags --always --dirty --match=v* 2> /dev/null || \
 			cat .version 2> /dev/null || echo v0)
 PKGS     = $(or $(PKG),$(shell env GO111MODULE=on $(GO) list ./...))
-BIN      = bin
 
 GO      = go
 NPM     = npm
@@ -19,18 +18,19 @@ M = $(shell if [ "$$(tput colors 2> /dev/null || echo 0)" -ge 8 ]; then printf "
 GENERATED_JS = \
 	console/frontend/node_modules
 GENERATED_GO = \
+	common/pb/rawflow.pb.go \
+	common/pb/rawflow_vtproto.pb.go \
 	common/schema/definition_gen.go \
 	orchestrator/clickhouse/data/asns.csv \
 	orchestrator/clickhouse/data/protocols.csv \
 	orchestrator/clickhouse/data/tcp.csv \
 	orchestrator/clickhouse/data/udp.csv \
 	console/filter/parser.go \
-	inlet/core/asnprovider_enumer.go \
-	inlet/core/netprovider_enumer.go \
-	inlet/flow/decoder/timestampsource_enumer.go \
-	inlet/metadata/provider/snmp/authprotocol_enumer.go \
-	inlet/metadata/provider/snmp/privprotocol_enumer.go \
-	inlet/metadata/provider/gnmi/ifspeedpathunit_enumer.go \
+	outlet/core/asnprovider_enumer.go \
+	outlet/core/netprovider_enumer.go \
+	outlet/metadata/provider/snmp/authprotocol_enumer.go \
+	outlet/metadata/provider/snmp/privprotocol_enumer.go \
+	outlet/metadata/provider/gnmi/ifspeedpathunit_enumer.go \
 	console/homepagetopwidget_enumer.go \
 	common/kafka/saslmechanism_enumer.go
 GENERATED_TEST_GO = \
@@ -42,20 +42,17 @@ GENERATED = \
 	console/data/frontend
 
 .PHONY: all
-all: fmt lint $(GENERATED) | $(BIN) ; $(info $(M) building executable…) @ ## Build program binary
+all: fmt lint $(GENERATED) ; $(info $(M) building executable…) @ ## Build program binary
 	$Q $(GO) build \
 		-tags release \
 		-ldflags '-X $(MODULE)/common/helpers.AkvoradoVersion=$(VERSION)' \
-		-o $(BIN)/$(basename $(MODULE)) main.go
+		-o bin/$(basename $(MODULE)) main.go
 
 .PHONY: all_js
 all_js: .fmt-js~ .lint-js~ $(GENERATED_JS) console/data/frontend
 
 # Tools
 
-$(BIN):
-	@mkdir -p $@
-
 ENUMER = go tool enumer
 GOCOV = go tool gocov
 GOCOVXML = go tool gocov-xml
@@ -65,11 +62,16 @@ MOCKGEN = go tool mockgen
 PIGEON = go tool pigeon
 REVIVE = go tool revive
 WWHRD = go tool wwhrd
+BUF = go run github.com/bufbuild/buf/cmd/buf@v1.55.1
 
 # Generated files
 
 .DELETE_ON_ERROR:
 
+common/pb/rawflow_vtproto.pb.go: common/pb/rawflow.pb.go
+%.pb.go: buf.gen.yaml %.proto ; $(info $(M) compiling protocol buffers $@…)
+	$Q $(BUF) generate --path $(@:.pb.go=.proto)
+
 common/clickhousedb/mocks/mock_driver.go: go.mod ; $(info $(M) generate mocks for ClickHouse driver…)
 	$Q $(MOCKGEN) -package mocks -build_constraint "!release" -destination $@ \
 		github.com/ClickHouse/clickhouse-go/v2/lib/driver Conn,Row,Rows,ColumnType
@@ -81,18 +83,16 @@ conntrackfixer/mocks/mock_conntrackfixer.go: go.mod ; $(info $(M) generate mocks
 		touch $@ ; \
 	fi
 
-inlet/core/asnprovider_enumer.go: go.mod inlet/core/config.go ; $(info $(M) generate enums for ASNProvider…)
-	$Q $(ENUMER) -type=ASNProvider -text -transform=kebab -trimprefix=ASNProvider inlet/core/config.go
-inlet/core/netprovider_enumer.go: go.mod inlet/core/config.go ; $(info $(M) generate enums for NetProvider…)
-	$Q $(ENUMER) -type=NetProvider -text -transform=kebab -trimprefix=NetProvider inlet/core/config.go
-inlet/flow/decoder/timestampsource_enumer.go: go.mod inlet/flow/decoder/config.go ; $(info $(M) generate enums for TimestampSource…)
-	$Q $(ENUMER) -type=TimestampSource -text -transform=kebab -trimprefix=TimestampSource inlet/flow/decoder/config.go
-inlet/metadata/provider/snmp/authprotocol_enumer.go: go.mod inlet/metadata/provider/snmp/config.go ; $(info $(M) generate enums for AuthProtocol…)
-	$Q $(ENUMER) -type=AuthProtocol -text -transform=kebab -trimprefix=AuthProtocol inlet/metadata/provider/snmp/config.go
-inlet/metadata/provider/snmp/privprotocol_enumer.go: go.mod inlet/metadata/provider/snmp/config.go ; $(info $(M) generate enums for PrivProtocol…)
-	$Q $(ENUMER) -type=PrivProtocol -text -transform=kebab -trimprefix=PrivProtocol inlet/metadata/provider/snmp/config.go
-inlet/metadata/provider/gnmi/ifspeedpathunit_enumer.go: go.mod inlet/metadata/provider/gnmi/config.go ; $(info $(M) generate enums for IfSpeedPathUnit…)
-	$Q $(ENUMER) -type=IfSpeedPathUnit -text -transform=kebab -trimprefix=Speed inlet/metadata/provider/gnmi/config.go
+outlet/core/asnprovider_enumer.go: go.mod outlet/core/config.go ; $(info $(M) generate enums for ASNProvider…)
+	$Q $(ENUMER) -type=ASNProvider -text -transform=kebab -trimprefix=ASNProvider outlet/core/config.go
+outlet/core/netprovider_enumer.go: go.mod outlet/core/config.go ; $(info $(M) generate enums for NetProvider…)
+	$Q $(ENUMER) -type=NetProvider -text -transform=kebab -trimprefix=NetProvider outlet/core/config.go
+outlet/metadata/provider/snmp/authprotocol_enumer.go: go.mod outlet/metadata/provider/snmp/config.go ; $(info $(M) generate enums for AuthProtocol…)
+	$Q $(ENUMER) -type=AuthProtocol -text -transform=kebab -trimprefix=AuthProtocol outlet/metadata/provider/snmp/config.go
+outlet/metadata/provider/snmp/privprotocol_enumer.go: go.mod outlet/metadata/provider/snmp/config.go ; $(info $(M) generate enums for PrivProtocol…)
+	$Q $(ENUMER) -type=PrivProtocol -text -transform=kebab -trimprefix=PrivProtocol outlet/metadata/provider/snmp/config.go
+outlet/metadata/provider/gnmi/ifspeedpathunit_enumer.go: go.mod outlet/metadata/provider/gnmi/config.go ; $(info $(M) generate enums for IfSpeedPathUnit…)
+	$Q $(ENUMER) -type=IfSpeedPathUnit -text -transform=kebab -trimprefix=Speed outlet/metadata/provider/gnmi/config.go
 console/homepagetopwidget_enumer.go: go.mod console/config.go ; $(info $(M) generate enums for HomepageTopWidget…)
 	$Q $(ENUMER) -type=HomepageTopWidget -text -json -transform=kebab -trimprefix=HomepageTopWidget console/config.go
 common/kafka/saslmechanism_enumer.go: go.mod common/kafka/config.go ; $(info $(M) generate enums for SASLMechanism…)
@@ -141,6 +141,17 @@ changelog.md: docs/99-changelog.md # To be used by GitHub actions only.
 	$Q >> $@ echo "**Docker image**: \`docker pull ghcr.io/$${GITHUB_REPOSITORY}:$${GITHUB_REF##*/v}\`"
 	$Q >> $@ echo "**Full changelog**: https://github.com/$${GITHUB_REPOSITORY}/compare/v$$(< docs/99-changelog.md sed -n '/^## '$${GITHUB_REF##*/v}' -/,/^## /{s/^## \([0-9.]*\) -.*/\1/p}' | tail -1)...v$${GITHUB_REF##*/v}"
 
+# Update default.pgo with the locally running "docker compose" instance.
+# Use: "make -j default.pgo".
+default.pgo: default-inlet.pgo default-outlet.pgo default-console.pgo
+	$Q go tool pprof -proto $^ > $@
+	$Q rm $^
+default-%.pgo:
+	$Q container=akvorado-akvorado-$*-1 ; \
+	   ip=$$(docker container inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' $$container) ; \
+	   [ -n $$ip ] ; \
+	   curl -so $@ "http://$$ip:8080/debug/pprof/profile?seconds=30"
+
 # Tests
 
 .PHONY: check test tests test-race test-short test-bench test-coverage
@@ -226,7 +237,7 @@ licensecheck: console/frontend/node_modules ; $(info $(M) check dependency licen
 
 .PHONY: clean
 clean: ; $(info $(M) cleaning…)	@ ## Cleanup everything
-	@rm -rf test $(GENERATED) inlet/flow/decoder/flow-*.pb.go *~ bin
+	@rm -rf test $(GENERATED) *~ bin/akvorado
 
 .PHONY: help
 help:
diff --git a/buf.gen.yaml b/buf.gen.yaml
new file mode 100644
index 00000000..76ba197d
--- /dev/null
+++ b/buf.gen.yaml
@@ -0,0 +1,13 @@
+---
+version: v2
+plugins:
+  - local: [ "go", "tool", "protoc-gen-go" ]
+    out: .
+    opt:
+      - paths=source_relative
+  - local: [ "go", "tool", "protoc-gen-go-vtproto" ]
+    out: .
+    opt:
+      - paths=source_relative
+      - features=size+marshal+unmarshal+pool
+      - pool=akvorado/common/pb.RawFlow
diff --git a/cmd/inlet.go b/cmd/inlet.go
index 0feba71a..9a25588c 100644
--- a/cmd/inlet.go
+++ b/cmd/inlet.go
@@ -5,24 +5,14 @@ package cmd
 
 import (
 	"fmt"
-	"reflect"
 
-	"github.com/gin-gonic/gin"
-	"github.com/go-viper/mapstructure/v2"
 	"github.com/spf13/cobra"
 
 	"akvorado/common/daemon"
-	"akvorado/common/helpers"
 	"akvorado/common/httpserver"
 	"akvorado/common/reporter"
-	"akvorado/common/schema"
-	"akvorado/inlet/core"
 	"akvorado/inlet/flow"
 	"akvorado/inlet/kafka"
-	"akvorado/inlet/metadata"
-	"akvorado/inlet/metadata/provider/snmp"
-	"akvorado/inlet/routing"
-	"akvorado/inlet/routing/provider/bmp"
 )
 
 // InletConfiguration represents the configuration file for the inlet command.
@@ -30,11 +20,7 @@ type InletConfiguration struct {
 	Reporting reporter.Configuration
 	HTTP      httpserver.Configuration
 	Flow      flow.Configuration
-	Metadata  metadata.Configuration
-	Routing   routing.Configuration
 	Kafka     kafka.Configuration
-	Core      core.Configuration
-	Schema    schema.Configuration
 }
 
 // Reset resets the configuration for the inlet command to its default value.
@@ -43,14 +29,8 @@ func (c *InletConfiguration) Reset() {
 		HTTP:      httpserver.DefaultConfiguration(),
 		Reporting: reporter.DefaultConfiguration(),
 		Flow:      flow.DefaultConfiguration(),
-		Metadata:  metadata.DefaultConfiguration(),
-		Routing:   routing.DefaultConfiguration(),
 		Kafka:     kafka.DefaultConfiguration(),
-		Core:      core.DefaultConfiguration(),
-		Schema:    schema.DefaultConfiguration(),
 	}
-	c.Metadata.Providers = []metadata.ProviderConfiguration{{Config: snmp.DefaultConfiguration()}}
-	c.Routing.Provider.Config = bmp.DefaultConfiguration()
 }
 
 type inletOptions struct {
@@ -66,7 +46,7 @@ var inletCmd = &cobra.Command{
 	Use:   "inlet",
 	Short: "Start Akvorado's inlet service",
 	Long: `Akvorado is a Netflow/IPFIX collector. The inlet service handles flow ingestion,
-enrichment and export to Kafka.`,
+and export to Kafka.`,
 	Args: cobra.ExactArgs(1),
 	RunE: func(cmd *cobra.Command, args []string) error {
 		config := InletConfiguration{}
@@ -103,48 +83,19 @@ func inletStart(r *reporter.Reporter, config InletConfiguration, checkOnly bool)
 	if err != nil {
 		return fmt.Errorf("unable to initialize http component: %w", err)
 	}
-	schemaComponent, err := schema.New(config.Schema)
-	if err != nil {
-		return fmt.Errorf("unable to initialize schema component: %w", err)
-	}
-	flowComponent, err := flow.New(r, config.Flow, flow.Dependencies{
-		Daemon: daemonComponent,
-		HTTP:   httpComponent,
-		Schema: schemaComponent,
-	})
-	if err != nil {
-		return fmt.Errorf("unable to initialize flow component: %w", err)
-	}
-	metadataComponent, err := metadata.New(r, config.Metadata, metadata.Dependencies{
-		Daemon: daemonComponent,
-	})
-	if err != nil {
-		return fmt.Errorf("unable to initialize metadata component: %w", err)
-	}
-	routingComponent, err := routing.New(r, config.Routing, routing.Dependencies{
-		Daemon: daemonComponent,
-	})
-	if err != nil {
-		return fmt.Errorf("unable to initialize routing component: %w", err)
-	}
 	kafkaComponent, err := kafka.New(r, config.Kafka, kafka.Dependencies{
 		Daemon: daemonComponent,
-		Schema: schemaComponent,
 	})
 	if err != nil {
 		return fmt.Errorf("unable to initialize Kafka component: %w", err)
 	}
-	coreComponent, err := core.New(r, config.Core, core.Dependencies{
-		Daemon:   daemonComponent,
-		Flow:     flowComponent,
-		Metadata: metadataComponent,
-		Routing:  routingComponent,
-		Kafka:    kafkaComponent,
-		HTTP:     httpComponent,
-		Schema:   schemaComponent,
+	flowComponent, err := flow.New(r, config.Flow, flow.Dependencies{
+		Daemon: daemonComponent,
+		HTTP:   httpComponent,
+		Kafka:  kafkaComponent,
 	})
 	if err != nil {
-		return fmt.Errorf("unable to initialize core component: %w", err)
+		return fmt.Errorf("unable to initialize flow component: %w", err)
 	}
 
 	// Expose some information and metrics
@@ -159,129 +110,8 @@ func inletStart(r *reporter.Reporter, config InletConfiguration, checkOnly bool)
 	// Start all the components.
 	components := []interface{}{
 		httpComponent,
-		metadataComponent,
-		routingComponent,
 		kafkaComponent,
-		coreComponent,
 		flowComponent,
 	}
 	return StartStopComponents(r, daemonComponent, components)
 }
-
-// InletConfigurationUnmarshallerHook renames SNMP configuration to metadata and
-// BMP configuration to routing.
-func InletConfigurationUnmarshallerHook() mapstructure.DecodeHookFunc {
-	return func(from, to reflect.Value) (interface{}, error) {
-		if from.Kind() != reflect.Map || from.IsNil() || to.Type() != reflect.TypeOf(InletConfiguration{}) {
-			return from.Interface(), nil
-		}
-
-		// snmp → metadata
-		{
-			var snmpKey, metadataKey *reflect.Value
-			fromKeys := from.MapKeys()
-			for i, k := range fromKeys {
-				k = helpers.ElemOrIdentity(k)
-				if k.Kind() != reflect.String {
-					return from.Interface(), nil
-				}
-				if helpers.MapStructureMatchName(k.String(), "Snmp") {
-					snmpKey = &fromKeys[i]
-				} else if helpers.MapStructureMatchName(k.String(), "Metadata") {
-					metadataKey = &fromKeys[i]
-				}
-			}
-			if snmpKey != nil {
-				if metadataKey != nil {
-					return nil, fmt.Errorf("cannot have both %q and %q", snmpKey.String(), metadataKey.String())
-				}
-
-				// Build the metadata configuration
-				providerValue := gin.H{}
-				metadataValue := gin.H{}
-				// Dispatch values from snmp key into metadata
-				snmpMap := helpers.ElemOrIdentity(from.MapIndex(*snmpKey))
-				snmpKeys := snmpMap.MapKeys()
-			outerSNMP:
-				for i, k := range snmpKeys {
-					k = helpers.ElemOrIdentity(k)
-					if k.Kind() != reflect.String {
-						continue
-					}
-					if helpers.MapStructureMatchName(k.String(), "PollerCoalesce") {
-						metadataValue["MaxBatchRequests"] = snmpMap.MapIndex(snmpKeys[i]).Interface()
-						continue
-					}
-					metadataConfig := reflect.TypeOf(metadata.Configuration{})
-					for j := range metadataConfig.NumField() {
-						if helpers.MapStructureMatchName(k.String(), metadataConfig.Field(j).Name) {
-							metadataValue[k.String()] = snmpMap.MapIndex(snmpKeys[i]).Interface()
-							continue outerSNMP
-						}
-					}
-					providerValue[k.String()] = snmpMap.MapIndex(snmpKeys[i]).Interface()
-				}
-
-				providerValue["type"] = "snmp"
-				metadataValue["provider"] = providerValue
-				from.SetMapIndex(reflect.ValueOf("metadata"), reflect.ValueOf(metadataValue))
-				from.SetMapIndex(*snmpKey, reflect.Value{})
-			}
-		}
-
-		// bmp → routing
-		{
-			var bmpKey, routingKey *reflect.Value
-			fromKeys := from.MapKeys()
-			for i, k := range fromKeys {
-				k = helpers.ElemOrIdentity(k)
-				if k.Kind() != reflect.String {
-					return from.Interface(), nil
-				}
-				if helpers.MapStructureMatchName(k.String(), "Bmp") {
-					bmpKey = &fromKeys[i]
-				} else if helpers.MapStructureMatchName(k.String(), "Routing") {
-					routingKey = &fromKeys[i]
-				}
-			}
-			if bmpKey != nil {
-				if routingKey != nil {
-					return nil, fmt.Errorf("cannot have both %q and %q", bmpKey.String(), routingKey.String())
-				}
-
-				// Build the routing configuration
-				providerValue := gin.H{}
-				routingValue := gin.H{}
-				// Dispatch values from bmp key into routing
-				bmpMap := helpers.ElemOrIdentity(from.MapIndex(*bmpKey))
-				bmpKeys := bmpMap.MapKeys()
-			outerBMP:
-				for i, k := range bmpKeys {
-					k = helpers.ElemOrIdentity(k)
-					if k.Kind() != reflect.String {
-						continue
-					}
-					routingConfig := reflect.TypeOf(routing.Configuration{})
-					for j := range routingConfig.NumField() {
-						if helpers.MapStructureMatchName(k.String(), routingConfig.Field(j).Name) {
-							routingValue[k.String()] = bmpMap.MapIndex(bmpKeys[i]).Interface()
-							continue outerBMP
-						}
-					}
-					providerValue[k.String()] = bmpMap.MapIndex(bmpKeys[i]).Interface()
-				}
-
-				providerValue["type"] = "bmp"
-				routingValue["provider"] = providerValue
-				from.SetMapIndex(reflect.ValueOf("routing"), reflect.ValueOf(routingValue))
-				from.SetMapIndex(*bmpKey, reflect.Value{})
-			}
-		}
-
-		return from.Interface(), nil
-	}
-}
-
-func init() {
-	helpers.RegisterMapstructureUnmarshallerHook(InletConfigurationUnmarshallerHook())
-}
diff --git a/cmd/orchestrator.go b/cmd/orchestrator.go
index 218972f0..120af630 100644
--- a/cmd/orchestrator.go
+++ b/cmd/orchestrator.go
@@ -37,6 +37,7 @@ type OrchestratorConfiguration struct {
 	Schema       schema.Configuration
 	// Other service configurations
 	Inlet        []InletConfiguration        `validate:"dive"`
+	Outlet       []OutletConfiguration       `validate:"dive"`
 	Console      []ConsoleConfiguration      `validate:"dive"`
 	DemoExporter []DemoExporterConfiguration `validate:"dive"`
 }
@@ -45,6 +46,8 @@ type OrchestratorConfiguration struct {
 func (c *OrchestratorConfiguration) Reset() {
 	inletConfiguration := InletConfiguration{}
 	inletConfiguration.Reset()
+	outletConfiguration := OutletConfiguration{}
+	outletConfiguration.Reset()
 	consoleConfiguration := ConsoleConfiguration{}
 	consoleConfiguration.Reset()
 	*c = OrchestratorConfiguration{
@@ -58,6 +61,7 @@ func (c *OrchestratorConfiguration) Reset() {
 		Schema:       schema.DefaultConfiguration(),
 		// Other service configurations
 		Inlet:        []InletConfiguration{inletConfiguration},
+		Outlet:       []OutletConfiguration{outletConfiguration},
 		Console:      []ConsoleConfiguration{consoleConfiguration},
 		DemoExporter: []DemoExporterConfiguration{},
 	}
@@ -83,14 +87,19 @@ components and centralizes configuration of the various other components.`,
 		OrchestratorOptions.Path = args[0]
 		OrchestratorOptions.BeforeDump = func(metadata mapstructure.Metadata) {
 			// Override some parts of the configuration
-			if !slices.Contains(metadata.Keys, "ClickHouse.Kafka.Brokers[0]") {
-				config.ClickHouse.Kafka.Configuration = config.Kafka.Configuration
-			}
 			for idx := range config.Inlet {
 				if !slices.Contains(metadata.Keys, fmt.Sprintf("Inlet[%d].Kafka.Brokers[0]", idx)) {
 					config.Inlet[idx].Kafka.Configuration = config.Kafka.Configuration
 				}
-				config.Inlet[idx].Schema = config.Schema
+			}
+			for idx := range config.Outlet {
+				if !slices.Contains(metadata.Keys, fmt.Sprintf("Outlet[%d].ClickHouse.Servers[0]", idx)) {
+					config.Outlet[idx].ClickHouseDB = config.ClickHouseDB
+				}
+				if !slices.Contains(metadata.Keys, fmt.Sprintf("Outlet[%d].Kafka.Brokers[0]", idx)) {
+					config.Outlet[idx].Kafka.Configuration = config.Kafka.Configuration
+				}
+				config.Outlet[idx].Schema = config.Schema
 			}
 			for idx := range config.Console {
 				if !slices.Contains(metadata.Keys, fmt.Sprintf("Console[%d].ClickHouse.Servers[0]", idx)) {
@@ -144,14 +153,12 @@ func orchestratorStart(r *reporter.Reporter, config OrchestratorConfiguration, c
 	if err != nil {
 		return fmt.Errorf("unable to initialize ClickHouse component: %w", err)
 	}
-
 	geoipComponent, err := geoip.New(r, config.GeoIP, geoip.Dependencies{
 		Daemon: daemonComponent,
 	})
 	if err != nil {
 		return fmt.Errorf("unable to initialize GeoIP component: %w", err)
 	}
-
 	clickhouseComponent, err := clickhouse.New(r, config.ClickHouse, clickhouse.Dependencies{
 		Daemon:     daemonComponent,
 		HTTP:       httpComponent,
@@ -171,6 +178,9 @@ func orchestratorStart(r *reporter.Reporter, config OrchestratorConfiguration, c
 	for idx := range config.Inlet {
 		orchestratorComponent.RegisterConfiguration(orchestrator.InletService, config.Inlet[idx])
 	}
+	for idx := range config.Outlet {
+		orchestratorComponent.RegisterConfiguration(orchestrator.OutletService, config.Outlet[idx])
+	}
 	for idx := range config.Console {
 		orchestratorComponent.RegisterConfiguration(orchestrator.ConsoleService, config.Console[idx])
 	}
@@ -188,7 +198,7 @@ func orchestratorStart(r *reporter.Reporter, config OrchestratorConfiguration, c
 	}
 
 	// Start all the components.
-	components := []interface{}{
+	components := []any{
 		geoipComponent,
 		httpComponent,
 		clickhouseDBComponent,
@@ -198,136 +208,224 @@ func orchestratorStart(r *reporter.Reporter, config OrchestratorConfiguration, c
 	return StartStopComponents(r, daemonComponent, components)
 }
 
-// OrchestratorConfigurationUnmarshallerHook migrates GeoIP configuration from inlet
-// component to clickhouse component and ClickHouse database configuration from
-// clickhouse component to clickhousedb component.
-func OrchestratorConfigurationUnmarshallerHook() mapstructure.DecodeHookFunc {
-	return func(from, to reflect.Value) (interface{}, error) {
+// orchestratorGeoIPMigrationHook migrates GeoIP configuration from inlet
+// component to clickhouse component
+func orchestratorGeoIPMigrationHook() mapstructure.DecodeHookFunc {
+	return func(from, to reflect.Value) (any, error) {
 		if from.Kind() != reflect.Map || from.IsNil() || to.Type() != reflect.TypeOf(OrchestratorConfiguration{}) {
 			return from.Interface(), nil
 		}
 
-	inletgeoip:
-		// inlet/geoip → geoip
-		for {
-			var (
-				inletKey, geoIPKey, inletGeoIPValue *reflect.Value
-			)
+		var (
+			inletKey, geoIPKey, inletGeoIPValue *reflect.Value
+		)
+
+		fromKeys := from.MapKeys()
+		for i, k := range fromKeys {
+			k = helpers.ElemOrIdentity(k)
+			if k.Kind() != reflect.String {
+				return from.Interface(), nil
+			}
+			if helpers.MapStructureMatchName(k.String(), "Inlet") {
+				inletKey = &fromKeys[i]
+			} else if helpers.MapStructureMatchName(k.String(), "GeoIP") {
+				geoIPKey = &fromKeys[i]
+			}
+		}
+		if inletKey == nil {
+			return from.Interface(), nil
+		}
 
-			fromKeys := from.MapKeys()
-			for i, k := range fromKeys {
+		// Take the first geoip configuration and delete the others
+		inletConfigs := helpers.ElemOrIdentity(from.MapIndex(*inletKey))
+		if inletConfigs.Kind() != reflect.Slice {
+			inletConfigs = reflect.ValueOf([]any{inletConfigs.Interface()})
+		}
+		for i := range inletConfigs.Len() {
+			fromInlet := helpers.ElemOrIdentity(inletConfigs.Index(i))
+			if fromInlet.Kind() != reflect.Map {
+				return from.Interface(), nil
+			}
+			fromInletKeys := fromInlet.MapKeys()
+			for _, k := range fromInletKeys {
 				k = helpers.ElemOrIdentity(k)
 				if k.Kind() != reflect.String {
-					break inletgeoip
+					return from.Interface(), nil
+				}
+				if helpers.MapStructureMatchName(k.String(), "GeoIP") {
+					if inletGeoIPValue == nil {
+						v := fromInlet.MapIndex(k)
+						inletGeoIPValue = &v
+					}
 				}
-				if helpers.MapStructureMatchName(k.String(), "Inlet") {
-					inletKey = &fromKeys[i]
-				} else if helpers.MapStructureMatchName(k.String(), "GeoIP") {
-					geoIPKey = &fromKeys[i]
+			}
+		}
+		if inletGeoIPValue == nil {
+			return from.Interface(), nil
+		}
+		if geoIPKey != nil {
+			return nil, errors.New("cannot have both \"GeoIP\" in inlet and clickhouse configuration")
+		}
+
+		from.SetMapIndex(reflect.ValueOf("geoip"), *inletGeoIPValue)
+		for i := range inletConfigs.Len() {
+			fromInlet := helpers.ElemOrIdentity(inletConfigs.Index(i))
+			fromInletKeys := fromInlet.MapKeys()
+			for _, k := range fromInletKeys {
+				k = helpers.ElemOrIdentity(k)
+				if helpers.MapStructureMatchName(k.String(), "GeoIP") {
+					fromInlet.SetMapIndex(k, reflect.Value{})
 				}
 			}
-			if inletKey == nil {
-				break inletgeoip
+		}
+
+		return from.Interface(), nil
+	}
+}
+
+// orchestratorClickHouseMigrationHook migrates ClickHouse database
+// configuration from clickhouse component to clickhousedb component
+func orchestratorClickHouseMigrationHook() mapstructure.DecodeHookFunc {
+	return func(from, to reflect.Value) (any, error) {
+		if from.Kind() != reflect.Map || from.IsNil() || to.Type() != reflect.TypeOf(OrchestratorConfiguration{}) {
+			return from.Interface(), nil
+		}
+
+		var clickhouseKey, clickhouseDBKey *reflect.Value
+		fromKeys := from.MapKeys()
+		for i, k := range fromKeys {
+			k = helpers.ElemOrIdentity(k)
+			if k.Kind() != reflect.String {
+				continue
+			}
+			if helpers.MapStructureMatchName(k.String(), "ClickHouse") {
+				clickhouseKey = &fromKeys[i]
+			} else if helpers.MapStructureMatchName(k.String(), "ClickHouseDB") {
+				clickhouseDBKey = &fromKeys[i]
 			}
+		}
 
-			// Take the first geoip configuration and delete the others
-			inletConfigs := helpers.ElemOrIdentity(from.MapIndex(*inletKey))
-			if inletConfigs.Kind() != reflect.Slice {
-				inletConfigs = reflect.ValueOf([]interface{}{inletConfigs.Interface()})
+		if clickhouseKey != nil {
+			var clickhouseDB reflect.Value
+			if clickhouseDBKey != nil {
+				clickhouseDB = helpers.ElemOrIdentity(from.MapIndex(*clickhouseDBKey))
+			} else {
+				clickhouseDB = reflect.ValueOf(gin.H{})
 			}
-			for i := range inletConfigs.Len() {
-				fromInlet := helpers.ElemOrIdentity(inletConfigs.Index(i))
-				if fromInlet.Kind() != reflect.Map {
-					break inletgeoip
+
+			clickhouse := helpers.ElemOrIdentity(from.MapIndex(*clickhouseKey))
+			if clickhouse.Kind() == reflect.Map {
+				clickhouseKeys := clickhouse.MapKeys()
+				// Fields to migrate from clickhouse to clickhousedb
+				fieldsToMigrate := []string{
+					"Servers", "Cluster", "Database", "Username", "Password",
+					"MaxOpenConns", "DialTimeout", "TLS",
 				}
-				fromInletKeys := fromInlet.MapKeys()
-				for _, k := range fromInletKeys {
+				found := false
+				for _, k := range clickhouseKeys {
 					k = helpers.ElemOrIdentity(k)
 					if k.Kind() != reflect.String {
-						break inletgeoip
+						continue
 					}
-					if helpers.MapStructureMatchName(k.String(), "GeoIP") {
-						if inletGeoIPValue == nil {
-							v := fromInlet.MapIndex(k)
-							inletGeoIPValue = &v
+					for _, field := range fieldsToMigrate {
+						if helpers.MapStructureMatchName(k.String(), field) {
+							if clickhouseDBKey != nil {
+								return nil, errors.New("cannot have both \"ClickHouseDB\" and ClickHouse database settings in \"ClickHouse\"")
+							}
+							clickhouseDB.SetMapIndex(k, helpers.ElemOrIdentity(clickhouse.MapIndex(k)))
+							clickhouse.SetMapIndex(k, reflect.Value{})
+							found = true
+							break
 						}
 					}
 				}
+				if clickhouseDBKey == nil && found {
+					from.SetMapIndex(reflect.ValueOf("clickhousedb"), clickhouseDB)
+				}
 			}
-			if inletGeoIPValue == nil {
-				break inletgeoip
+		}
+
+		return from.Interface(), nil
+	}
+}
+
+// orchestratorInletToOutletMigrationHook migrates inlet configuration to outlet
+// configuration. This only works if there is no outlet configuration and if
+// there is only one inlet configuration.
+func orchestratorInletToOutletMigrationHook() mapstructure.DecodeHookFunc {
+	return func(from, to reflect.Value) (any, error) {
+		if from.Kind() != reflect.Map || from.IsNil() || to.Type() != reflect.TypeOf(OrchestratorConfiguration{}) {
+			return from.Interface(), nil
+		}
+
+		// inlet fields (Metadata, Routing, Core, Schema) → outlet
+		var inletKey, outletKey *reflect.Value
+		fromKeys := from.MapKeys()
+		for i, k := range fromKeys {
+			k = helpers.ElemOrIdentity(k)
+			if k.Kind() != reflect.String {
+				continue
 			}
-			if geoIPKey != nil {
-				return nil, errors.New("cannot have both \"GeoIP\" in inlet and clickhouse configuration")
+			if helpers.MapStructureMatchName(k.String(), "Inlet") {
+				inletKey = &fromKeys[i]
+			} else if helpers.MapStructureMatchName(k.String(), "Outlet") {
+				outletKey = &fromKeys[i]
 			}
+		}
 
-			from.SetMapIndex(reflect.ValueOf("geoip"), *inletGeoIPValue)
-			for i := range inletConfigs.Len() {
-				fromInlet := helpers.ElemOrIdentity(inletConfigs.Index(i))
-				fromInletKeys := fromInlet.MapKeys()
-				for _, k := range fromInletKeys {
-					k = helpers.ElemOrIdentity(k)
-					if helpers.MapStructureMatchName(k.String(), "GeoIP") {
-						fromInlet.SetMapIndex(k, reflect.Value{})
-					}
-				}
+		if inletKey != nil {
+			inletConfigs := helpers.ElemOrIdentity(from.MapIndex(*inletKey))
+			if inletConfigs.Kind() != reflect.Slice {
+				inletConfigs = reflect.ValueOf([]any{inletConfigs.Interface()})
 			}
-			break
-		}
 
-		{
-			// clickhouse database fields → clickhousedb
-			var clickhouseKey, clickhouseDBKey *reflect.Value
-			fromKeys := from.MapKeys()
-			for i, k := range fromKeys {
-				k = helpers.ElemOrIdentity(k)
-				if k.Kind() != reflect.String {
-					continue
-				}
-				if helpers.MapStructureMatchName(k.String(), "ClickHouse") {
-					clickhouseKey = &fromKeys[i]
-				} else if helpers.MapStructureMatchName(k.String(), "ClickHouseDB") {
-					clickhouseDBKey = &fromKeys[i]
-				}
+			// Fields to migrate from inlet to outlet
+			fieldsToMigrate := []string{
+				// Current keys
+				"Metadata", "Routing", "Core", "Schema",
+				// Older keys (which will be migrated)
+				"BMP", "SNMP",
 			}
 
-			if clickhouseKey != nil {
-				var clickhouseDB reflect.Value
-				if clickhouseDBKey != nil {
-					clickhouseDB = helpers.ElemOrIdentity(from.MapIndex(*clickhouseDBKey))
-				} else {
-					clickhouseDB = reflect.ValueOf(gin.H{})
+			// Process each inlet configuration
+			for i := range inletConfigs.Len() {
+				fromInlet := helpers.ElemOrIdentity(inletConfigs.Index(i))
+				if fromInlet.Kind() != reflect.Map {
+					continue
 				}
+				modified := false
+				toOutlet := reflect.ValueOf(gin.H{})
 
-				clickhouse := helpers.ElemOrIdentity(from.MapIndex(*clickhouseKey))
-				if clickhouse.Kind() == reflect.Map {
-					clickhouseKeys := clickhouse.MapKeys()
-					// Fields to migrate from clickhouse to clickhousedb
-					fieldsToMigrate := []string{
-						"Servers", "Cluster", "Database", "Username", "Password",
-						"MaxOpenConns", "DialTimeout", "TLS",
+				// Migrate fields from inlet to outlet
+				fromInletKeys := fromInlet.MapKeys()
+				for _, k := range fromInletKeys {
+					k = helpers.ElemOrIdentity(k)
+					if k.Kind() != reflect.String {
+						continue
 					}
-					found := false
-					for _, k := range clickhouseKeys {
-						k = helpers.ElemOrIdentity(k)
-						if k.Kind() != reflect.String {
+					for _, field := range fieldsToMigrate {
+						if !helpers.MapStructureMatchName(k.String(), field) {
 							continue
 						}
-						for _, field := range fieldsToMigrate {
-							if helpers.MapStructureMatchName(k.String(), field) {
-								if clickhouseDBKey != nil {
-									return nil, errors.New("cannot have both \"ClickHouseDB\" and ClickHouse database settings in \"ClickHouse\"")
-								}
-								clickhouseDB.SetMapIndex(k, helpers.ElemOrIdentity(clickhouse.MapIndex(k)))
-								clickhouse.SetMapIndex(k, reflect.Value{})
-								found = true
-								break
-							}
+						// We can only do a migration if we have no existing
+						// outlet configuration AND only one inlet configuration.
+						if outletKey != nil {
+							return nil, fmt.Errorf("cannot have both \"inlet\" configuration with %q field and \"outlet\" configuration", field)
 						}
+						if inletConfigs.Len() > 1 {
+							return nil, fmt.Errorf("cannot migrate %q from %q to %q as there are several inlet configurations", field, "inlet", "outlet")
+						}
+						toOutlet.SetMapIndex(k, helpers.ElemOrIdentity(fromInlet.MapIndex(k)))
+						fromInlet.SetMapIndex(k, reflect.Value{})
+						modified = true
+						break
 					}
-					if clickhouseDBKey == nil && found {
-						from.SetMapIndex(reflect.ValueOf("clickhousedb"), clickhouseDB)
-					}
+				}
+
+				if modified {
+					// We know there is no existing outlet configuration.
+					outletConfigs := reflect.ValueOf([]any{toOutlet})
+					from.SetMapIndex(reflect.ValueOf("outlet"), outletConfigs)
 				}
 			}
 		}
@@ -337,5 +435,7 @@ func OrchestratorConfigurationUnmarshallerHook() mapstructure.DecodeHookFunc {
 }
 
 func init() {
-	helpers.RegisterMapstructureUnmarshallerHook(OrchestratorConfigurationUnmarshallerHook())
+	helpers.RegisterMapstructureUnmarshallerHook(orchestratorGeoIPMigrationHook())
+	helpers.RegisterMapstructureUnmarshallerHook(orchestratorClickHouseMigrationHook())
+	helpers.RegisterMapstructureUnmarshallerHook(orchestratorInletToOutletMigrationHook())
 }
diff --git a/cmd/outlet.go b/cmd/outlet.go
new file mode 100644
index 00000000..97780883
--- /dev/null
+++ b/cmd/outlet.go
@@ -0,0 +1,306 @@
+// SPDX-FileCopyrightText: 2022 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+package cmd
+
+import (
+	"fmt"
+	"reflect"
+
+	"github.com/gin-gonic/gin"
+	"github.com/go-viper/mapstructure/v2"
+	"github.com/spf13/cobra"
+
+	"akvorado/common/clickhousedb"
+	"akvorado/common/daemon"
+	"akvorado/common/helpers"
+	"akvorado/common/httpserver"
+	"akvorado/common/reporter"
+	"akvorado/common/schema"
+	"akvorado/outlet/clickhouse"
+	"akvorado/outlet/core"
+	"akvorado/outlet/flow"
+	"akvorado/outlet/kafka"
+	"akvorado/outlet/metadata"
+	"akvorado/outlet/metadata/provider/snmp"
+	"akvorado/outlet/routing"
+	"akvorado/outlet/routing/provider/bmp"
+)
+
+// OutletConfiguration represents the configuration file for the outlet command.
+type OutletConfiguration struct {
+	Reporting    reporter.Configuration
+	HTTP         httpserver.Configuration
+	Metadata     metadata.Configuration
+	Routing      routing.Configuration
+	Kafka        kafka.Configuration
+	ClickHouseDB clickhousedb.Configuration
+	ClickHouse   clickhouse.Configuration
+	Core         core.Configuration
+	Schema       schema.Configuration
+}
+
+// Reset resets the configuration for the outlet command to its default value.
+func (c *OutletConfiguration) Reset() {
+	*c = OutletConfiguration{
+		HTTP:         httpserver.DefaultConfiguration(),
+		Reporting:    reporter.DefaultConfiguration(),
+		Metadata:     metadata.DefaultConfiguration(),
+		Routing:      routing.DefaultConfiguration(),
+		Kafka:        kafka.DefaultConfiguration(),
+		ClickHouseDB: clickhousedb.DefaultConfiguration(),
+		ClickHouse:   clickhouse.DefaultConfiguration(),
+		Core:         core.DefaultConfiguration(),
+		Schema:       schema.DefaultConfiguration(),
+	}
+	c.Metadata.Providers = []metadata.ProviderConfiguration{{Config: snmp.DefaultConfiguration()}}
+	c.Routing.Provider.Config = bmp.DefaultConfiguration()
+}
+
+type outletOptions struct {
+	ConfigRelatedOptions
+	CheckMode bool
+}
+
+// OutletOptions stores the command-line option values for the outlet
+// command.
+var OutletOptions outletOptions
+
+var outletCmd = &cobra.Command{
+	Use:   "outlet",
+	Short: "Start Akvorado's outlet service",
+	Long: `Akvorado is a Netflow/IPFIX collector. The outlet service handles flow ingestion,
+enrichment and export to Kafka.`,
+	Args: cobra.ExactArgs(1),
+	RunE: func(cmd *cobra.Command, args []string) error {
+		config := OutletConfiguration{}
+		OutletOptions.Path = args[0]
+		if err := OutletOptions.Parse(cmd.OutOrStdout(), "outlet", &config); err != nil {
+			return err
+		}
+
+		r, err := reporter.New(config.Reporting)
+		if err != nil {
+			return fmt.Errorf("unable to initialize reporter: %w", err)
+		}
+		return outletStart(r, config, OutletOptions.CheckMode)
+	},
+}
+
+func init() {
+	RootCmd.AddCommand(outletCmd)
+	outletCmd.Flags().BoolVarP(&OutletOptions.ConfigRelatedOptions.Dump, "dump", "D", false,
+		"Dump configuration before starting")
+	outletCmd.Flags().BoolVarP(&OutletOptions.CheckMode, "check", "C", false,
+		"Check configuration, but does not start")
+}
+
+func outletStart(r *reporter.Reporter, config OutletConfiguration, checkOnly bool) error {
+	// Initialize the various components
+	daemonComponent, err := daemon.New(r)
+	if err != nil {
+		return fmt.Errorf("unable to initialize daemon component: %w", err)
+	}
+	httpComponent, err := httpserver.New(r, config.HTTP, httpserver.Dependencies{
+		Daemon: daemonComponent,
+	})
+	if err != nil {
+		return fmt.Errorf("unable to initialize http component: %w", err)
+	}
+	schemaComponent, err := schema.New(config.Schema)
+	if err != nil {
+		return fmt.Errorf("unable to initialize schema component: %w", err)
+	}
+	flowComponent, err := flow.New(r, flow.Dependencies{
+		Schema: schemaComponent,
+	})
+	if err != nil {
+		return fmt.Errorf("unable to initialize flow component: %w", err)
+	}
+	metadataComponent, err := metadata.New(r, config.Metadata, metadata.Dependencies{
+		Daemon: daemonComponent,
+	})
+	if err != nil {
+		return fmt.Errorf("unable to initialize metadata component: %w", err)
+	}
+	routingComponent, err := routing.New(r, config.Routing, routing.Dependencies{
+		Daemon: daemonComponent,
+	})
+	if err != nil {
+		return fmt.Errorf("unable to initialize routing component: %w", err)
+	}
+	kafkaComponent, err := kafka.New(r, config.Kafka, kafka.Dependencies{
+		Daemon: daemonComponent,
+	})
+	if err != nil {
+		return fmt.Errorf("unable to initialize Kafka component: %w", err)
+	}
+	clickhouseDBComponent, err := clickhousedb.New(r, config.ClickHouseDB, clickhousedb.Dependencies{
+		Daemon: daemonComponent,
+	})
+	if err != nil {
+		return fmt.Errorf("unable to initialize ClickHouse component: %w", err)
+	}
+	clickhouseComponent, err := clickhouse.New(r, config.ClickHouse, clickhouse.Dependencies{
+		ClickHouse: clickhouseDBComponent,
+		Schema:     schemaComponent,
+	})
+	if err != nil {
+		return fmt.Errorf("unable to initialize outlet ClickHouse component: %w", err)
+	}
+	coreComponent, err := core.New(r, config.Core, core.Dependencies{
+		Daemon:     daemonComponent,
+		Flow:       flowComponent,
+		Metadata:   metadataComponent,
+		Routing:    routingComponent,
+		Kafka:      kafkaComponent,
+		ClickHouse: clickhouseComponent,
+		HTTP:       httpComponent,
+		Schema:     schemaComponent,
+	})
+	if err != nil {
+		return fmt.Errorf("unable to initialize core component: %w", err)
+	}
+
+	// Expose some information and metrics
+	addCommonHTTPHandlers(r, "outlet", httpComponent)
+	versionMetrics(r)
+
+	// If we only asked for a check, stop here.
+	if checkOnly {
+		return nil
+	}
+
+	// Start all the components.
+	components := []any{
+		httpComponent,
+		clickhouseDBComponent,
+		clickhouseComponent,
+		flowComponent,
+		metadataComponent,
+		routingComponent,
+		kafkaComponent,
+		coreComponent,
+	}
+	return StartStopComponents(r, daemonComponent, components)
+}
+
+// OutletConfigurationUnmarshallerHook renames SNMP configuration to metadata and
+// BMP configuration to routing.
+func OutletConfigurationUnmarshallerHook() mapstructure.DecodeHookFunc {
+	return func(from, to reflect.Value) (interface{}, error) {
+		if from.Kind() != reflect.Map || from.IsNil() || to.Type() != reflect.TypeOf(OutletConfiguration{}) {
+			return from.Interface(), nil
+		}
+
+		// snmp → metadata
+		{
+			var snmpKey, metadataKey *reflect.Value
+			fromKeys := from.MapKeys()
+			for i, k := range fromKeys {
+				k = helpers.ElemOrIdentity(k)
+				if k.Kind() != reflect.String {
+					return from.Interface(), nil
+				}
+				if helpers.MapStructureMatchName(k.String(), "Snmp") {
+					snmpKey = &fromKeys[i]
+				} else if helpers.MapStructureMatchName(k.String(), "Metadata") {
+					metadataKey = &fromKeys[i]
+				}
+			}
+			if snmpKey != nil {
+				if metadataKey != nil {
+					return nil, fmt.Errorf("cannot have both %q and %q", snmpKey.String(), metadataKey.String())
+				}
+
+				// Build the metadata configuration
+				providerValue := gin.H{}
+				metadataValue := gin.H{}
+				// Dispatch values from snmp key into metadata
+				snmpMap := helpers.ElemOrIdentity(from.MapIndex(*snmpKey))
+				snmpKeys := snmpMap.MapKeys()
+			outerSNMP:
+				for i, k := range snmpKeys {
+					k = helpers.ElemOrIdentity(k)
+					if k.Kind() != reflect.String {
+						continue
+					}
+					if helpers.MapStructureMatchName(k.String(), "PollerCoalesce") {
+						continue
+					}
+					if helpers.MapStructureMatchName(k.String(), "Workers") {
+						continue
+					}
+					metadataConfig := reflect.TypeOf(metadata.Configuration{})
+					for j := range metadataConfig.NumField() {
+						if helpers.MapStructureMatchName(k.String(), metadataConfig.Field(j).Name) {
+							metadataValue[k.String()] = snmpMap.MapIndex(snmpKeys[i]).Interface()
+							continue outerSNMP
+						}
+					}
+					providerValue[k.String()] = snmpMap.MapIndex(snmpKeys[i]).Interface()
+				}
+
+				providerValue["type"] = "snmp"
+				metadataValue["provider"] = providerValue
+				from.SetMapIndex(reflect.ValueOf("metadata"), reflect.ValueOf(metadataValue))
+				from.SetMapIndex(*snmpKey, reflect.Value{})
+			}
+		}
+
+		// bmp → routing
+		{
+			var bmpKey, routingKey *reflect.Value
+			fromKeys := from.MapKeys()
+			for i, k := range fromKeys {
+				k = helpers.ElemOrIdentity(k)
+				if k.Kind() != reflect.String {
+					return from.Interface(), nil
+				}
+				if helpers.MapStructureMatchName(k.String(), "Bmp") {
+					bmpKey = &fromKeys[i]
+				} else if helpers.MapStructureMatchName(k.String(), "Routing") {
+					routingKey = &fromKeys[i]
+				}
+			}
+			if bmpKey != nil {
+				if routingKey != nil {
+					return nil, fmt.Errorf("cannot have both %q and %q", bmpKey.String(), routingKey.String())
+				}
+
+				// Build the routing configuration
+				providerValue := gin.H{}
+				routingValue := gin.H{}
+				// Dispatch values from bmp key into routing
+				bmpMap := helpers.ElemOrIdentity(from.MapIndex(*bmpKey))
+				bmpKeys := bmpMap.MapKeys()
+			outerBMP:
+				for i, k := range bmpKeys {
+					k = helpers.ElemOrIdentity(k)
+					if k.Kind() != reflect.String {
+						continue
+					}
+					routingConfig := reflect.TypeOf(routing.Configuration{})
+					for j := range routingConfig.NumField() {
+						if helpers.MapStructureMatchName(k.String(), routingConfig.Field(j).Name) {
+							routingValue[k.String()] = bmpMap.MapIndex(bmpKeys[i]).Interface()
+							continue outerBMP
+						}
+					}
+					providerValue[k.String()] = bmpMap.MapIndex(bmpKeys[i]).Interface()
+				}
+
+				providerValue["type"] = "bmp"
+				routingValue["provider"] = providerValue
+				from.SetMapIndex(reflect.ValueOf("routing"), reflect.ValueOf(routingValue))
+				from.SetMapIndex(*bmpKey, reflect.Value{})
+			}
+		}
+
+		return from.Interface(), nil
+	}
+}
+
+func init() {
+	helpers.RegisterMapstructureUnmarshallerHook(OutletConfigurationUnmarshallerHook())
+}
diff --git a/common/clickhousedb/config.go b/common/clickhousedb/config.go
index 34922dab..04fa78fa 100644
--- a/common/clickhousedb/config.go
+++ b/common/clickhousedb/config.go
@@ -6,6 +6,8 @@ package clickhousedb
 import (
 	"time"
 
+	"github.com/ClickHouse/ch-go"
+
 	"akvorado/common/helpers"
 )
 
@@ -55,3 +57,19 @@ func (c *Component) ClusterName() string {
 func (c *Component) DatabaseName() string {
 	return c.config.Database
 }
+
+// ChGoOptions returns options suitable to use with ch-go and the list of
+// available servers.
+func (c *Component) ChGoOptions() (ch.Options, []string) {
+	tlsConfig, _ := c.config.TLS.MakeTLSConfig()
+	return ch.Options{
+		Address:     c.config.Servers[0],
+		Database:    c.config.Database,
+		User:        c.config.Username,
+		Password:    c.config.Password,
+		Compression: ch.CompressionLZ4,
+		ClientName:  "akvorado",
+		DialTimeout: c.config.DialTimeout,
+		TLS:         tlsConfig,
+	}, c.config.Servers
+}
diff --git a/common/helpers/ether.go b/common/helpers/ether.go
index 11c22e19..3fa19453 100644
--- a/common/helpers/ether.go
+++ b/common/helpers/ether.go
@@ -3,9 +3,24 @@
 
 package helpers
 
+import "net"
+
 const (
 	// ETypeIPv4 is the ether type for IPv4
 	ETypeIPv4 = 0x800
 	// ETypeIPv6 is the ether type for IPv6
 	ETypeIPv6 = 0x86dd
 )
+
+// MACToUint64 converts a MAC address to an uint64
+func MACToUint64(mac net.HardwareAddr) uint64 {
+	if len(mac) != 6 {
+		return 0
+	}
+	return uint64(mac[0])<<40 |
+		uint64(mac[1])<<32 |
+		uint64(mac[2])<<24 |
+		uint64(mac[3])<<16 |
+		uint64(mac[4])<<8 |
+		uint64(mac[5])
+}
diff --git a/common/helpers/mapstructure.go b/common/helpers/mapstructure.go
index 23dba6d1..07fadde4 100644
--- a/common/helpers/mapstructure.go
+++ b/common/helpers/mapstructure.go
@@ -21,6 +21,57 @@ func RegisterMapstructureUnmarshallerHook(hook mapstructure.DecodeHookFunc) {
 	mapstructureUnmarshallerHookFuncs = append(mapstructureUnmarshallerHookFuncs, hook)
 }
 
+// RegisterMapstructureDeprecatedFields registers a decoder hook removing
+// deprecated fields. This should only be done during init.
+func RegisterMapstructureDeprecatedFields[V any](fieldNames ...string) {
+	RegisterMapstructureUnmarshallerHook(func(from, to reflect.Value) (any, error) {
+		var zeroV V
+		from = ElemOrIdentity(from)
+		to = ElemOrIdentity(to)
+		if !SameTypeOrSuperset(to.Type(), reflect.TypeOf(zeroV)) {
+			return from.Interface(), nil
+		}
+		if from.Kind() != reflect.Map {
+			return from.Interface(), nil
+		}
+
+		mapKeys := from.MapKeys()
+		for _, key := range mapKeys {
+			var keyStr string
+			if ElemOrIdentity(key).Kind() == reflect.String {
+				keyStr = ElemOrIdentity(key).String()
+			} else {
+				continue
+			}
+			for _, fieldName := range fieldNames {
+				if MapStructureMatchName(keyStr, fieldName) {
+					from.SetMapIndex(key, reflect.Value{})
+				}
+			}
+		}
+
+		return from.Interface(), nil
+	})
+}
+
+// SameTypeOrSuperset returns true if "input" and "ref" type are the same or
+// when "input" has "ref" as a squashed field.
+func SameTypeOrSuperset(input, ref reflect.Type) bool {
+	if input == ref {
+		return true
+	}
+	if input.Kind() != reflect.Struct {
+		return false
+	}
+	for i := 0; i < input.NumField(); i++ {
+		field := input.Field(i)
+		if tag := field.Tag.Get("mapstructure"); tag == ",squash" && field.Type == ref {
+			return true
+		}
+	}
+	return false
+}
+
 // GetMapStructureDecoderConfig returns a decoder config for
 // mapstructure with all registered hooks as well as appropriate
 // default configuration.
diff --git a/common/helpers/release.go b/common/helpers/release.go
new file mode 100644
index 00000000..fc27aecb
--- /dev/null
+++ b/common/helpers/release.go
@@ -0,0 +1,13 @@
+// SPDX-FileCopyrightText: 2022 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+//go:build release
+
+package helpers
+
+// Testing reports whether the current code is being run in a test. It always
+// return false in release mode and therefore its test has no performance
+// impact.
+func Testing() bool {
+	return false
+}
diff --git a/common/kafka/config.go b/common/kafka/config.go
index 5ed0d883..a6c305b9 100644
--- a/common/kafka/config.go
+++ b/common/kafka/config.go
@@ -6,17 +6,19 @@
 package kafka
 
 import (
-	"context"
-	"crypto/sha256"
-	"crypto/sha512"
 	"fmt"
 	"reflect"
 
 	"akvorado/common/helpers"
+	"akvorado/common/reporter"
 
-	"github.com/IBM/sarama"
 	"github.com/gin-gonic/gin"
 	"github.com/go-viper/mapstructure/v2"
+	"github.com/twmb/franz-go/pkg/kgo"
+	"github.com/twmb/franz-go/pkg/sasl"
+	"github.com/twmb/franz-go/pkg/sasl/oauth"
+	"github.com/twmb/franz-go/pkg/sasl/plain"
+	"github.com/twmb/franz-go/pkg/sasl/scram"
 	"golang.org/x/oauth2/clientcredentials"
 )
 
@@ -26,8 +28,6 @@ type Configuration struct {
 	Topic string `validate:"required"`
 	// Brokers is the list of brokers to connect to.
 	Brokers []string `min=1,dive,validate:"listen"`
-	// Version is the version of Kafka we assume to work
-	Version Version
 	// TLS defines TLS configuration
 	TLS helpers.TLSConfiguration
 	// SASL defines SASL configuration
@@ -53,7 +53,6 @@ func DefaultConfiguration() Configuration {
 	return Configuration{
 		Topic:   "flows",
 		Brokers: []string{"127.0.0.1:9092"},
-		Version: Version(sarama.V2_8_1_0),
 		TLS: helpers.TLSConfiguration{
 			Enable: false,
 			Verify: true,
@@ -61,29 +60,6 @@ func DefaultConfiguration() Configuration {
 	}
 }
 
-// Version represents a supported version of Kafka
-type Version sarama.KafkaVersion
-
-// UnmarshalText parses a version of Kafka
-func (v *Version) UnmarshalText(text []byte) error {
-	version, err := sarama.ParseKafkaVersion(string(text))
-	if err != nil {
-		return err
-	}
-	*v = Version(version)
-	return nil
-}
-
-// String turns a Kafka version into a string
-func (v Version) String() string {
-	return sarama.KafkaVersion(v).String()
-}
-
-// MarshalText turns a Kafka version into a string
-func (v Version) MarshalText() ([]byte, error) {
-	return []byte(v.String()), nil
-}
-
 // SASLMechanism defines an SASL algorithm
 type SASLMechanism int
 
@@ -100,62 +76,67 @@ const (
 	SASLOauth
 )
 
-// NewConfig returns a Sarama Kafka configuration ready to use.
-func NewConfig(config Configuration) (*sarama.Config, error) {
-	kafkaConfig := sarama.NewConfig()
-	kafkaConfig.Version = sarama.KafkaVersion(config.Version)
-	kafkaConfig.ClientID = fmt.Sprintf("akvorado-%s", helpers.AkvoradoVersion)
+// NewConfig returns a slice of kgo.Opt configurations ready to use.
+func NewConfig(r *reporter.Reporter, config Configuration) ([]kgo.Opt, error) {
+	opts := []kgo.Opt{
+		kgo.SeedBrokers(config.Brokers...),
+		kgo.ClientID(fmt.Sprintf("akvorado-%s", helpers.AkvoradoVersion)),
+		kgo.WithLogger(NewLogger(r)),
+	}
+
+	// TLS configuration
 	tlsConfig, err := config.TLS.MakeTLSConfig()
 	if err != nil {
 		return nil, err
 	}
 	if tlsConfig != nil {
-		kafkaConfig.Net.TLS.Enable = true
-		kafkaConfig.Net.TLS.Config = tlsConfig
+		opts = append(opts, kgo.DialTLSConfig(tlsConfig))
 	}
-	// SASL
+
+	// SASL configuration
 	if config.SASL.Mechanism != SASLNone {
-		kafkaConfig.Net.SASL.Enable = true
-		kafkaConfig.Net.SASL.User = config.SASL.Username
-		kafkaConfig.Net.SASL.Password = config.SASL.Password
+		var mechanism sasl.Mechanism
 		switch config.SASL.Mechanism {
 		case SASLPlain:
-			kafkaConfig.Net.SASL.Mechanism = sarama.SASLTypePlaintext
+			mechanism = plain.Auth{
+				User: config.SASL.Username,
+				Pass: config.SASL.Password,
+			}.AsMechanism()
 		case SASLScramSHA256:
-			kafkaConfig.Net.SASL.Handshake = true
-			kafkaConfig.Net.SASL.Mechanism = sarama.SASLTypeSCRAMSHA256
-			kafkaConfig.Net.SASL.SCRAMClientGeneratorFunc = func() sarama.SCRAMClient {
-				return &xdgSCRAMClient{HashGeneratorFcn: sha256.New}
-			}
+			mechanism = scram.Auth{
+				User: config.SASL.Username,
+				Pass: config.SASL.Password,
+			}.AsSha256Mechanism()
 		case SASLScramSHA512:
-			kafkaConfig.Net.SASL.Handshake = true
-			kafkaConfig.Net.SASL.Mechanism = sarama.SASLTypeSCRAMSHA512
-			kafkaConfig.Net.SASL.SCRAMClientGeneratorFunc = func() sarama.SCRAMClient {
-				return &xdgSCRAMClient{HashGeneratorFcn: sha512.New}
-			}
+			mechanism = scram.Auth{
+				User: config.SASL.Username,
+				Pass: config.SASL.Password,
+			}.AsSha512Mechanism()
 		case SASLOauth:
-			kafkaConfig.Net.SASL.Mechanism = sarama.SASLTypeOAuth
-			kafkaConfig.Net.SASL.TokenProvider = newOAuthTokenProvider(
-				context.Background(), // TODO should be bound to the component lifecycle, but no component here
-				tlsConfig,
-				clientcredentials.Config{
-					ClientID:     config.SASL.Username,
-					ClientSecret: config.SASL.Password,
-					TokenURL:     config.SASL.OAuthTokenURL,
-					Scopes:       config.SASL.OAuthScopes,
-				})
+			mechanism = oauth.Oauth(
+				newOAuthTokenProvider(
+					tlsConfig,
+					clientcredentials.Config{
+						ClientID:     config.SASL.Username,
+						ClientSecret: config.SASL.Password,
+						TokenURL:     config.SASL.OAuthTokenURL,
+						Scopes:       config.SASL.OAuthScopes,
+					}),
+			)
 		default:
 			return nil, fmt.Errorf("unknown SASL mechanism: %s", config.SASL.Mechanism)
 		}
+		opts = append(opts, kgo.SASL(mechanism))
 	}
-	return kafkaConfig, nil
+
+	return opts, nil
 }
 
 // ConfigurationUnmarshallerHook normalize Kafka configuration:
 //   - move SASL related parameters from TLS section to SASL section
 func ConfigurationUnmarshallerHook() mapstructure.DecodeHookFunc {
 	return func(from, to reflect.Value) (interface{}, error) {
-		if from.Kind() != reflect.Map || from.IsNil() || to.Type() != reflect.TypeOf(Configuration{}) {
+		if from.Kind() != reflect.Map || from.IsNil() || !helpers.SameTypeOrSuperset(to.Type(), reflect.TypeOf(Configuration{})) {
 			return from.Interface(), nil
 		}
 
@@ -204,6 +185,6 @@ func ConfigurationUnmarshallerHook() mapstructure.DecodeHookFunc {
 }
 
 func init() {
+	helpers.RegisterMapstructureDeprecatedFields[Configuration]("Version")
 	helpers.RegisterMapstructureUnmarshallerHook(ConfigurationUnmarshallerHook())
-
 }
diff --git a/common/kafka/logs.go b/common/kafka/logs.go
index 16afe44d..0c182ccb 100644
--- a/common/kafka/logs.go
+++ b/common/kafka/logs.go
@@ -4,60 +4,40 @@
 package kafka
 
 import (
-	"fmt"
-	"sync/atomic"
-
-	"github.com/IBM/sarama"
+	"github.com/twmb/franz-go/pkg/kgo"
 
+	"akvorado/common/helpers"
 	"akvorado/common/reporter"
 )
 
-func init() {
-	// The logger in Sarama is global. Do the same.
-	sarama.Logger = &GlobalKafkaLogger
-}
-
-// GlobalKafkaLogger is the logger instance registered to sarama.
-var GlobalKafkaLogger kafkaLogger
-
+// kafkaLogger implements kgo.Logger interface.
 type kafkaLogger struct {
-	r atomic.Pointer[reporter.Reporter]
+	r *reporter.Reporter
 }
 
-// Register register the provided reporter to be used for logging with sarama.
-func (l *kafkaLogger) Register(r *reporter.Reporter) {
-	l.r.Store(r)
-}
-
-// Unregister removes the currently registered reporter.
-func (l *kafkaLogger) Unregister() {
-	var noreporter *reporter.Reporter
-	l.r.Store(noreporter)
-}
-
-func (l *kafkaLogger) Print(v ...interface{}) {
-	r := l.r.Load()
-	if r != nil {
-		if e := r.Debug(); e.Enabled() {
-			e.Msg(fmt.Sprint(v...))
-		}
-	}
+// NewLogger creates a new kafka logger using the provided reporter.
+func NewLogger(r *reporter.Reporter) kgo.Logger {
+	return &kafkaLogger{r: r}
 }
 
-func (l *kafkaLogger) Println(v ...interface{}) {
-	r := l.r.Load()
-	if r != nil {
-		if e := r.Debug(); e.Enabled() {
-			e.Msg(fmt.Sprint(v...))
-		}
+// Level returns the current log level.
+func (l *kafkaLogger) Level() kgo.LogLevel {
+	if !helpers.Testing() {
+		return kgo.LogLevelInfo
 	}
+	return kgo.LogLevelDebug
 }
 
-func (l *kafkaLogger) Printf(format string, v ...interface{}) {
-	r := l.r.Load()
-	if r != nil {
-		if e := r.Debug(); e.Enabled() {
-			e.Msg(fmt.Sprintf(format, v...))
-		}
+// Log logs a message at the specified level.
+func (l *kafkaLogger) Log(level kgo.LogLevel, msg string, keyvals ...any) {
+	switch level {
+	case kgo.LogLevelError:
+		l.r.Error().Fields(keyvals).Msg(msg)
+	case kgo.LogLevelWarn:
+		l.r.Warn().Fields(keyvals).Msg(msg)
+	case kgo.LogLevelInfo:
+		l.r.Info().Fields(keyvals).Msg(msg)
+	case kgo.LogLevelDebug:
+		l.r.Debug().Fields(keyvals).Msg(msg)
 	}
 }
diff --git a/common/kafka/oauth.go b/common/kafka/oauth.go
index f088a81a..d363cd24 100644
--- a/common/kafka/oauth.go
+++ b/common/kafka/oauth.go
@@ -8,34 +8,29 @@ import (
 	"crypto/tls"
 	"net/http"
 
-	"github.com/IBM/sarama"
+	"github.com/twmb/franz-go/pkg/sasl/oauth"
 	"golang.org/x/oauth2"
 	"golang.org/x/oauth2/clientcredentials"
 )
 
-// tokenProvider implements sarama.AccessTokenProvider.
+// tokenProvider implements OAuth token provider for franz-go.
 type tokenProvider struct {
 	tokenSource oauth2.TokenSource
 }
 
-// newOAuthTokenProvider returns a sarama.AccessTokenProvider using OAuth credentials.
-func newOAuthTokenProvider(ctx context.Context, tlsConfig *tls.Config, oauthConfig clientcredentials.Config) sarama.AccessTokenProvider {
-	httpClient := &http.Client{Transport: &http.Transport{
-		Proxy:           http.ProxyFromEnvironment,
-		TLSClientConfig: tlsConfig,
-	}}
-	ctx = context.WithValue(ctx, oauth2.HTTPClient, httpClient)
-
-	return &tokenProvider{
-		tokenSource: oauthConfig.TokenSource(context.Background()),
-	}
-}
-
-// Token returns a new *sarama.AccessToken or an error as appropriate.
-func (t *tokenProvider) Token() (*sarama.AccessToken, error) {
-	token, err := t.tokenSource.Token()
-	if err != nil {
-		return nil, err
+// newOAuthTokenProvider returns a token provider function using OAuth credentials.
+func newOAuthTokenProvider(tlsConfig *tls.Config, oauthConfig clientcredentials.Config) func(context.Context) (oauth.Auth, error) {
+	return func(ctx context.Context) (oauth.Auth, error) {
+		httpClient := &http.Client{Transport: &http.Transport{
+			Proxy:           http.ProxyFromEnvironment,
+			TLSClientConfig: tlsConfig,
+		}}
+		ctx = context.WithValue(ctx, oauth2.HTTPClient, httpClient)
+		tokenSource := oauthConfig.TokenSource(ctx)
+		token, err := tokenSource.Token()
+		if err != nil {
+			return oauth.Auth{}, err
+		}
+		return oauth.Auth{Token: token.AccessToken}, nil
 	}
-	return &sarama.AccessToken{Token: token.AccessToken}, nil
 }
diff --git a/common/kafka/scram.go b/common/kafka/scram.go
deleted file mode 100644
index 73536e56..00000000
--- a/common/kafka/scram.go
+++ /dev/null
@@ -1,32 +0,0 @@
-// SPDX-FileCopyrightText: 2013 Shopify
-// SPDX-License-Identifier: MIT
-
-// From https://github.com/IBM/sarama/blob/main/examples/sasl_scram_client/scram_client.go
-
-package kafka
-
-import "github.com/xdg-go/scram"
-
-type xdgSCRAMClient struct {
-	*scram.Client
-	*scram.ClientConversation
-	scram.HashGeneratorFcn
-}
-
-func (x *xdgSCRAMClient) Begin(userName, password, authzID string) (err error) {
-	x.Client, err = x.HashGeneratorFcn.NewClient(userName, password, authzID)
-	if err != nil {
-		return err
-	}
-	x.ClientConversation = x.Client.NewConversation()
-	return nil
-}
-
-func (x *xdgSCRAMClient) Step(challenge string) (response string, err error) {
-	response, err = x.ClientConversation.Step(challenge)
-	return
-}
-
-func (x *xdgSCRAMClient) Done() bool {
-	return x.ClientConversation.Done()
-}
diff --git a/common/pb/rawflow.go b/common/pb/rawflow.go
new file mode 100644
index 00000000..b322598a
--- /dev/null
+++ b/common/pb/rawflow.go
@@ -0,0 +1,78 @@
+// SPDX-FileCopyrightText: 2024 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+// Package pb contains the definition of RawFlow, the protobuf-based
+// structure to exchange flows between the inlet and the outlet.
+package pb
+
+import (
+	"errors"
+	"fmt"
+
+	"akvorado/common/helpers/bimap"
+)
+
+// Version is the version of the schema. On incompatible changes, this should be
+// bumped.
+var Version = 5
+
+var decoderMap = bimap.New(map[RawFlow_Decoder]string{
+	RawFlow_DECODER_NETFLOW: "netflow",
+	RawFlow_DECODER_SFLOW:   "sflow",
+})
+
+// MarshalText turns a decoder to text
+func (d RawFlow_Decoder) MarshalText() ([]byte, error) {
+	got, ok := decoderMap.LoadValue(d)
+	if ok {
+		return []byte(got), nil
+	}
+	return nil, errors.New("unknown decoder")
+}
+
+// UnmarshalText provides a decoder from text
+func (d *RawFlow_Decoder) UnmarshalText(input []byte) error {
+	if len(input) == 0 {
+		*d = RawFlow_DECODER_UNSPECIFIED
+		return nil
+	}
+	got, ok := decoderMap.LoadKey(string(input))
+	if ok {
+		*d = got
+		return nil
+	}
+	return errors.New("unknown decoder")
+}
+
+var tsMap = bimap.New(map[RawFlow_TimestampSource]string{
+	RawFlow_TS_INPUT:                  "input", // this is the default value
+	RawFlow_TS_NETFLOW_FIRST_SWITCHED: "netflow-first-switched",
+	RawFlow_TS_NETFLOW_PACKET:         "netflow-packet",
+})
+
+// MarshalText turns a timestamp source to text
+func (ts RawFlow_TimestampSource) MarshalText() ([]byte, error) {
+	got, ok := tsMap.LoadValue(ts)
+	if ok {
+		return []byte(got), nil
+	}
+	return nil, errors.New("unknown timestamp source")
+}
+
+// UnmarshalText provides a timestamp source from text
+func (ts *RawFlow_TimestampSource) UnmarshalText(input []byte) error {
+	if len(input) == 0 {
+		*ts = RawFlow_TS_INPUT
+		return nil
+	}
+	if string(input) == "udp" {
+		*ts = RawFlow_TS_INPUT
+		return nil
+	}
+	got, ok := tsMap.LoadKey(string(input))
+	if ok {
+		*ts = got
+		return nil
+	}
+	return fmt.Errorf("unknown timestamp source %q", string(input))
+}
diff --git a/common/pb/rawflow.proto b/common/pb/rawflow.proto
new file mode 100644
index 00000000..6737482e
--- /dev/null
+++ b/common/pb/rawflow.proto
@@ -0,0 +1,26 @@
+syntax = "proto3";
+package input;
+option go_package = "akvorado/common/pb";
+
+// RawFlow is an undecoded flow with some options.
+message RawFlow {
+    uint64 time_received = 1;    // when the flow was received
+    bytes payload = 2;           // payload of the flow
+    bytes source_address = 3;    // source IPv6 address
+    bool use_source_address = 4; // use source address as exporter address
+
+    // Decoding options
+    enum Decoder {
+        DECODER_UNSPECIFIED = 0;
+        DECODER_NETFLOW = 1;
+        DECODER_SFLOW = 2;
+        DECODER_GOB = 3;
+    }
+    enum TimestampSource {
+        TS_INPUT = 0;
+        TS_NETFLOW_PACKET = 1;
+        TS_NETFLOW_FIRST_SWITCHED = 2;
+    }
+    Decoder decoder = 5;
+    TimestampSource timestamp_source = 6;
+}
diff --git a/common/remotedatasourcefetcher/root.go b/common/remotedatasourcefetcher/root.go
index 22f0aa0c..ee7e0990 100644
--- a/common/remotedatasourcefetcher/root.go
+++ b/common/remotedatasourcefetcher/root.go
@@ -8,7 +8,6 @@ import (
 	"context"
 	"encoding/json"
 	"errors"
-	"fmt"
 	"net/http"
 	"sync"
 	"time"
@@ -21,7 +20,9 @@ import (
 	"akvorado/common/reporter"
 )
 
-// ProviderFunc is the callback function to call when a datasource is refreshed  implementZ.
+// ProviderFunc is the callback function to call when a datasource is refreshed.
+// The error returned is used for metrics. One should avoid having too many
+// different errors.
 type ProviderFunc func(ctx context.Context, name string, source RemoteDataSource) (int, error)
 
 // Component represents a remote data source fetcher.
@@ -49,9 +50,27 @@ func New[T interface{}](r *reporter.Reporter, provider ProviderFunc, dataType st
 	return &c, nil
 }
 
-// Fetch retrieves data from a configured RemoteDataSource, and returns a list of results
-// decoded from JSON to generic type.
-// Fetch should be used in UpdateRemoteDataSource implementations to update internal data from results.
+var (
+	// ErrBuildRequest is triggered when we cannot build an HTTP request
+	ErrBuildRequest = errors.New("cannot build HTTP request")
+	// ErrFetchDataSource is triggered when we cannot fetch the data source
+	ErrFetchDataSource = errors.New("cannot fetch data source")
+	// ErrStatusCode is triggered if status code is not 200
+	ErrStatusCode = errors.New("unexpected HTTP status code")
+	// ErrJSONDecode is triggered for any decoding issue
+	ErrJSONDecode = errors.New("cannot decode JSON")
+	// ErrMapResult is triggered when we cannot map the JSON result to the expected structure
+	ErrMapResult = errors.New("cannot map JSON")
+	// ErrJQExecute is triggered when we cannot execute the jq filter
+	ErrJQExecute = errors.New("cannot execute jq filter")
+	// ErrEmpty is triggered if the results are empty
+	ErrEmpty = errors.New("empty result")
+)
+
+// Fetch retrieves data from a configured RemoteDataSource, and returns a list
+// of results decoded from JSON to generic type. Fetch should be used in
+// UpdateRemoteDataSource implementations to update internal data from results.
+// It outputs errors without details because they are used for metrics.
 func (c *Component[T]) Fetch(ctx context.Context, name string, source RemoteDataSource) ([]T, error) {
 	var results []T
 	l := c.r.With().Str("name", name).Str("url", source.URL).Logger()
@@ -61,31 +80,30 @@ func (c *Component[T]) Fetch(ctx context.Context, name string, source RemoteData
 		Proxy: http.ProxyFromEnvironment,
 	}}
 	req, err := http.NewRequestWithContext(ctx, source.Method, source.URL, nil)
+	if err != nil {
+		l.Err(err).Msg("unable to build new request")
+		return results, ErrBuildRequest
+	}
 	for headerName, headerValue := range source.Headers {
 		req.Header.Set(headerName, headerValue)
 	}
 	req.Header.Set("accept", "application/json")
-	if err != nil {
-		l.Err(err).Msg("unable to build new request")
-		return results, fmt.Errorf("unable to build new request: %w", err)
-	}
 	resp, err := client.Do(req)
 	if err != nil {
 		l.Err(err).Msg("unable to fetch data source")
-		return results, fmt.Errorf("unable to fetch data source: %w", err)
+		return results, ErrFetchDataSource
 	}
 	defer resp.Body.Close()
 	if resp.StatusCode != 200 {
-		err := fmt.Errorf("unexpected status code %d: %s", resp.StatusCode, resp.Status)
-		l.Error().Msg(err.Error())
-		return results, err
+		l.Error().Int("status", resp.StatusCode).Msg("unexpected status code")
+		return results, ErrStatusCode
 	}
 	reader := bufio.NewReader(resp.Body)
 	decoder := json.NewDecoder(reader)
 	var got interface{}
 	if err := decoder.Decode(&got); err != nil {
 		l.Err(err).Msg("cannot decode JSON output")
-		return results, fmt.Errorf("cannot decode JSON output: %w", err)
+		return results, ErrJSONDecode
 	}
 
 	iter := source.Transform.Query.RunWithContext(ctx, got)
@@ -96,7 +114,7 @@ func (c *Component[T]) Fetch(ctx context.Context, name string, source RemoteData
 		}
 		if err, ok := v.(error); ok {
 			l.Err(err).Msg("cannot execute jq filter")
-			return results, fmt.Errorf("cannot execute jq filter: %w", err)
+			return results, ErrJQExecute
 		}
 		var result T
 		config := &mapstructure.DecoderConfig{
@@ -110,14 +128,13 @@ func (c *Component[T]) Fetch(ctx context.Context, name string, source RemoteData
 		}
 		if err := decoder.Decode(v); err != nil {
 			l.Err(err).Msgf("cannot map returned value for %#v", v)
-			return results, fmt.Errorf("cannot map returned value: %w", err)
+			return results, ErrMapResult
 		}
 		results = append(results, result)
 	}
 	if len(results) == 0 {
-		err := errors.New("empty results")
-		l.Error().Msg(err.Error())
-		return results, err
+		l.Error().Msg("empty result")
+		return results, ErrEmpty
 	}
 	return results, nil
 }
@@ -144,10 +161,7 @@ func (c *Component[T]) Start() error {
 				customBackoff := backoff.NewExponentialBackOff()
 				customBackoff.MaxElapsedTime = 0
 				customBackoff.MaxInterval = source.Interval
-				customBackoff.InitialInterval = source.Interval / 10
-				if customBackoff.InitialInterval > time.Second {
-					customBackoff.InitialInterval = time.Second
-				}
+				customBackoff.InitialInterval = min(time.Second, source.Interval/10)
 				return backoff.NewTicker(customBackoff)
 			}
 			newRegularTicker := func() *time.Ticker {
diff --git a/common/reporter/metrics.go b/common/reporter/metrics.go
index 25321e8f..c3b2a62c 100644
--- a/common/reporter/metrics.go
+++ b/common/reporter/metrics.go
@@ -125,3 +125,8 @@ func (r *Reporter) MetricCollectorForCurrentModule(c prometheus.Collector) {
 func (r *Reporter) MetricDesc(name, help string, variableLabels []string) *MetricDesc {
 	return r.metrics.Desc(1, name, help, variableLabels)
 }
+
+// MetricDesc2 defines a new metric description. It skips one callstack.
+func (r *Reporter) MetricDesc2(name, help string, variableLabels []string) *MetricDesc {
+	return r.metrics.Desc(2, name, help, variableLabels)
+}
diff --git a/common/schema/clickhouse.go b/common/schema/clickhouse.go
index 2844cc98..1dcd428d 100644
--- a/common/schema/clickhouse.go
+++ b/common/schema/clickhouse.go
@@ -4,9 +4,15 @@
 package schema
 
 import (
+	"encoding/base32"
 	"fmt"
+	"hash/fnv"
+	"net/netip"
 	"slices"
 	"strings"
+	"time"
+
+	"github.com/ClickHouse/ch-go/proto"
 )
 
 // ClickHouseDefinition turns a column into a declaration for ClickHouse
@@ -21,6 +27,33 @@ func (column Column) ClickHouseDefinition() string {
 	return strings.Join(result, " ")
 }
 
+// newProtoColumn turns a column into its proto.Column definition
+func (column Column) newProtoColumn() proto.Column {
+	if strings.HasPrefix(column.ClickHouseType, "Enum8(") {
+		// Enum8 is a special case. We do not want to use ColAuto as it comes
+		// with a performance penalty due to conversion between key values.
+		return new(proto.ColEnum8)
+	}
+
+	col := &proto.ColAuto{}
+	err := col.Infer(proto.ColumnType(column.ClickHouseType))
+	if err != nil {
+		panic(fmt.Sprintf("unhandled ClickHouse type %q", column.ClickHouseType))
+	}
+	return col.Data
+}
+
+// wrapProtoColumn optionally wraps the proto.Column for use in proto.Input
+func (column Column) wrapProtoColumn(in proto.Column) proto.Column {
+	if strings.HasPrefix(column.ClickHouseType, "Enum8(") {
+		// Enum8 is a special case. See above.
+		ddl := column.ClickHouseType[6 : len(column.ClickHouseType)-1]
+		return proto.Wrap(in, ddl)
+	}
+
+	return in
+}
+
 // ClickHouseTableOption is an option to alter the values returned by ClickHouseCreateTable() and ClickHouseSelectColumns().
 type ClickHouseTableOption int
 
@@ -29,18 +62,12 @@ const (
 	ClickHouseSkipMainOnlyColumns ClickHouseTableOption = iota
 	// ClickHouseSkipGeneratedColumns skips the columns with a GenerateFrom value
 	ClickHouseSkipGeneratedColumns
-	// ClickHouseSkipTransformColumns skips the columns with a TransformFrom value
-	ClickHouseSkipTransformColumns
 	// ClickHouseSkipAliasedColumns skips the columns with a Alias value
 	ClickHouseSkipAliasedColumns
 	// ClickHouseSkipTimeReceived skips the time received column
 	ClickHouseSkipTimeReceived
-	// ClickHouseUseTransformFromType uses the type from TransformFrom if any
-	ClickHouseUseTransformFromType
 	// ClickHouseSubstituteGenerates changes the column name to use the default generated value
 	ClickHouseSubstituteGenerates
-	// ClickHouseSubstituteTransforms changes the column name to use the transformed value
-	ClickHouseSubstituteTransforms
 )
 
 // ClickHouseCreateTable returns the columns for the CREATE TABLE clause in ClickHouse.
@@ -72,27 +99,12 @@ func (schema Schema) clickhouseIterate(fn func(Column), options ...ClickHouseTab
 		if slices.Contains(options, ClickHouseSkipGeneratedColumns) && column.ClickHouseGenerateFrom != "" && !column.ClickHouseSelfGenerated {
 			continue
 		}
-		if slices.Contains(options, ClickHouseSkipTransformColumns) && column.ClickHouseTransformFrom != nil {
-			continue
-		}
 		if slices.Contains(options, ClickHouseSkipAliasedColumns) && column.ClickHouseAlias != "" {
 			continue
 		}
-		if slices.Contains(options, ClickHouseUseTransformFromType) && column.ClickHouseTransformFrom != nil {
-			for _, ocol := range column.ClickHouseTransformFrom {
-				// We assume we only need to use name/type
-				column.Name = ocol.Name
-				column.ClickHouseType = ocol.ClickHouseType
-				fn(column)
-			}
-			continue
-		}
 		if slices.Contains(options, ClickHouseSubstituteGenerates) && column.ClickHouseGenerateFrom != "" {
 			column.Name = fmt.Sprintf("%s AS %s", column.ClickHouseGenerateFrom, column.Name)
 		}
-		if slices.Contains(options, ClickHouseSubstituteTransforms) && column.ClickHouseTransformFrom != nil {
-			column.Name = fmt.Sprintf("%s AS %s", column.ClickHouseTransformTo, column.Name)
-		}
 		fn(column)
 	}
 }
@@ -119,3 +131,256 @@ func (schema Schema) ClickHousePrimaryKeys() []string {
 	}
 	return cols
 }
+
+// ClickHouseHash returns an hash of the inpt table in ClickHouse
+func (schema Schema) ClickHouseHash() string {
+	hash := fnv.New128()
+	create := schema.ClickHouseCreateTable(ClickHouseSkipGeneratedColumns, ClickHouseSkipAliasedColumns)
+	hash.Write([]byte(create))
+	hashString := base32.StdEncoding.WithPadding(base32.NoPadding).EncodeToString(hash.Sum(nil))
+	return fmt.Sprintf("%sv5", hashString)
+}
+
+// AppendDateTime adds a DateTime value to the provided column
+func (bf *FlowMessage) AppendDateTime(columnKey ColumnKey, value uint32) {
+	col := bf.batch.columns[columnKey]
+	if value == 0 || col == nil || bf.batch.columnSet.Test(uint(columnKey)) {
+		return
+	}
+	bf.batch.columnSet.Set(uint(columnKey))
+	col.(*proto.ColDateTime).AppendRaw(proto.DateTime(value))
+	bf.appendDebug(columnKey, value)
+}
+
+// AppendUint adds an UInt64/32/16/8 or Enum8 value to the provided column
+func (bf *FlowMessage) AppendUint(columnKey ColumnKey, value uint64) {
+	col := bf.batch.columns[columnKey]
+	if value == 0 || col == nil || bf.batch.columnSet.Test(uint(columnKey)) {
+		return
+	}
+	switch col := col.(type) {
+	case *proto.ColUInt64:
+		col.Append(value)
+	case *proto.ColUInt32:
+		col.Append(uint32(value))
+	case *proto.ColUInt16:
+		col.Append(uint16(value))
+	case *proto.ColUInt8:
+		col.Append(uint8(value))
+	case *proto.ColEnum8:
+		col.Append(proto.Enum8(value))
+	default:
+		panic(fmt.Sprintf("unhandled uint type %q", col.Type()))
+	}
+	bf.batch.columnSet.Set(uint(columnKey))
+	bf.appendDebug(columnKey, value)
+}
+
+// AppendString adds a String value to the provided column
+func (bf *FlowMessage) AppendString(columnKey ColumnKey, value string) {
+	col := bf.batch.columns[columnKey]
+	if value == "" || col == nil || bf.batch.columnSet.Test(uint(columnKey)) {
+		return
+	}
+	switch col := col.(type) {
+	case *proto.ColLowCardinality[string]:
+		col.Append(value)
+	default:
+		panic(fmt.Sprintf("unhandled string type %q", col.Type()))
+	}
+	bf.batch.columnSet.Set(uint(columnKey))
+	bf.appendDebug(columnKey, value)
+}
+
+// AppendIPv6 adds an IPv6 value to the provided column
+func (bf *FlowMessage) AppendIPv6(columnKey ColumnKey, value netip.Addr) {
+	col := bf.batch.columns[columnKey]
+	if !value.IsValid() || col == nil || bf.batch.columnSet.Test(uint(columnKey)) {
+		return
+	}
+	switch col := col.(type) {
+	case *proto.ColIPv6:
+		col.Append(value.As16())
+	case *proto.ColLowCardinality[proto.IPv6]:
+		col.Append(value.As16())
+	default:
+		panic(fmt.Sprintf("unhandled string type %q", col.Type()))
+	}
+	bf.batch.columnSet.Set(uint(columnKey))
+	bf.appendDebug(columnKey, value)
+}
+
+// AppendArrayUInt32 adds an Array(UInt32) value to the provided column
+func (bf *FlowMessage) AppendArrayUInt32(columnKey ColumnKey, value []uint32) {
+	col := bf.batch.columns[columnKey]
+	if len(value) == 0 || col == nil || bf.batch.columnSet.Test(uint(columnKey)) {
+		return
+	}
+	bf.batch.columnSet.Set(uint(columnKey))
+	col.(*proto.ColArr[uint32]).Append(value)
+	bf.appendDebug(columnKey, value)
+}
+
+// AppendArrayUInt128 adds an Array(UInt128) value to the provided column
+func (bf *FlowMessage) AppendArrayUInt128(columnKey ColumnKey, value []UInt128) {
+	col := bf.batch.columns[columnKey]
+	if len(value) == 0 || col == nil || bf.batch.columnSet.Test(uint(columnKey)) {
+		return
+	}
+	bf.batch.columnSet.Set(uint(columnKey))
+	col.(*proto.ColArr[proto.UInt128]).Append(value)
+	bf.appendDebug(columnKey, value)
+}
+
+func (bf *FlowMessage) appendDebug(columnKey ColumnKey, value any) {
+	if !debug {
+		return
+	}
+	if bf.OtherColumns == nil {
+		bf.OtherColumns = make(map[ColumnKey]any)
+	}
+	bf.OtherColumns[columnKey] = value
+}
+
+// check executes some sanity checks when in debug mode. It should be called
+// only after finalization.
+func (bf *FlowMessage) check() {
+	if !debug {
+		return
+	}
+	if debug {
+		// Check that all columns have the right amount of rows
+		for idx, col := range bf.batch.columns {
+			if col == nil {
+				continue
+			}
+			if col.Rows() != bf.batch.rowCount {
+				panic(fmt.Sprintf("row %s has a count of %d instead of %d", ColumnKey(idx), col.Rows(), bf.batch.rowCount))
+			}
+		}
+	}
+}
+
+// appendDefaultValue appends a default/zero value to the given column.
+func (bf *FlowMessage) appendDefaultValues() {
+	for idx, col := range bf.batch.columns {
+		// Skip unpopulated columns
+		if col == nil {
+			continue
+		}
+		// Or columns already set
+		if bf.batch.columnSet.Test(uint(idx)) {
+			continue
+		}
+		// Put the default value depending on the real type
+		switch col := col.(type) {
+		case *proto.ColUInt64:
+			col.Append(0)
+		case *proto.ColUInt32:
+			col.Append(0)
+		case *proto.ColUInt16:
+			col.Append(0)
+		case *proto.ColUInt8:
+			col.Append(0)
+		case *proto.ColIPv6:
+			col.Append([16]byte{})
+		case *proto.ColDateTime:
+			col.Append(time.Unix(0, 0))
+		case *proto.ColEnum8:
+			col.Append(0)
+		case *proto.ColLowCardinality[string]:
+			col.Append("")
+		case *proto.ColLowCardinality[proto.IPv6]:
+			col.Append(proto.IPv6{})
+		case *proto.ColArr[uint32]:
+			col.Append([]uint32{})
+		case *proto.ColArr[proto.UInt128]:
+			col.Append([]proto.UInt128{})
+		default:
+			panic(fmt.Sprintf("unhandled ClickHouse type %q", col.Type()))
+		}
+	}
+}
+
+// Undo reverts the current changes. This should revert the various Append() functions.
+func (bf *FlowMessage) Undo() {
+	for idx, col := range bf.batch.columns {
+		if col == nil {
+			continue
+		}
+		if !bf.batch.columnSet.Test(uint(idx)) {
+			continue
+		}
+		switch col := col.(type) {
+		case *proto.ColUInt64:
+			*col = (*col)[:len(*col)-1]
+		case *proto.ColUInt32:
+			*col = (*col)[:len(*col)-1]
+		case *proto.ColUInt16:
+			*col = (*col)[:len(*col)-1]
+		case *proto.ColUInt8:
+			*col = (*col)[:len(*col)-1]
+		case *proto.ColIPv6:
+			*col = (*col)[:len(*col)-1]
+		case *proto.ColDateTime:
+			col.Data = col.Data[:len(col.Data)-1]
+		case *proto.ColEnum8:
+			*col = (*col)[:len(*col)-1]
+		case *proto.ColLowCardinality[string]:
+			col.Values = col.Values[:len(col.Values)-1]
+		case *proto.ColLowCardinality[proto.IPv6]:
+			col.Values = col.Values[:len(col.Values)-1]
+		case *proto.ColArr[uint32]:
+			l := len(col.Offsets)
+			if l > 0 {
+				start := uint64(0)
+				if l > 1 {
+					start = col.Offsets[l-2]
+				}
+				data := col.Data.(*proto.ColUInt32)
+				*data = (*data)[:start]
+				col.Data = data
+				col.Offsets = col.Offsets[:l-1]
+			}
+		case *proto.ColArr[proto.UInt128]:
+			l := len(col.Offsets)
+			if l > 0 {
+				start := uint64(0)
+				if l > 1 {
+					start = col.Offsets[l-2]
+				}
+				data := col.Data.(*proto.ColUInt128)
+				*data = (*data)[:start]
+				col.Data = data
+				col.Offsets = col.Offsets[:l-1]
+			}
+		default:
+			panic(fmt.Sprintf("unhandled ClickHouse type %q", col.Type()))
+		}
+	}
+	bf.reset()
+}
+
+// Finalize finalizes the current FlowMessage. It can then be reused for the
+// next one. It is crucial to always call Finalize, otherwise the batch could be
+// faulty.
+func (bf *FlowMessage) Finalize() {
+	bf.AppendDateTime(ColumnTimeReceived, bf.TimeReceived)
+	bf.AppendUint(ColumnSamplingRate, bf.SamplingRate)
+	bf.AppendIPv6(ColumnExporterAddress, bf.ExporterAddress)
+	bf.AppendUint(ColumnSrcAS, uint64(bf.SrcAS))
+	bf.AppendUint(ColumnDstAS, uint64(bf.DstAS))
+	bf.AppendUint(ColumnSrcNetMask, uint64(bf.SrcNetMask))
+	bf.AppendUint(ColumnDstNetMask, uint64(bf.DstNetMask))
+	bf.AppendIPv6(ColumnSrcAddr, bf.SrcAddr)
+	bf.AppendIPv6(ColumnDstAddr, bf.DstAddr)
+	bf.AppendIPv6(ColumnNextHop, bf.NextHop)
+	if !bf.schema.IsDisabled(ColumnGroupL2) {
+		bf.AppendUint(ColumnSrcVlan, uint64(bf.SrcVlan))
+		bf.AppendUint(ColumnDstVlan, uint64(bf.DstVlan))
+	}
+	bf.batch.rowCount++
+	bf.appendDefaultValues()
+	bf.reset()
+	bf.check()
+}
diff --git a/common/schema/definition.go b/common/schema/definition.go
index 7eed59af..afc419e6 100644
--- a/common/schema/definition.go
+++ b/common/schema/definition.go
@@ -12,8 +12,6 @@ import (
 	"akvorado/common/helpers/bimap"
 
 	"github.com/bits-and-blooms/bitset"
-	"google.golang.org/protobuf/encoding/protowire"
-	"google.golang.org/protobuf/reflect/protoreflect"
 )
 
 // InterfaceBoundary identifies wether the interface is facing inside or outside the network.
@@ -133,9 +131,6 @@ const (
 	ColumnDst3rdAS
 	ColumnDstCommunities
 	ColumnDstLargeCommunities
-	ColumnDstLargeCommunitiesASN
-	ColumnDstLargeCommunitiesLocalData1
-	ColumnDstLargeCommunitiesLocalData2
 	ColumnInIfName
 	ColumnOutIfName
 	ColumnInIfDescription
@@ -212,7 +207,6 @@ func flows() Schema {
 				ClickHouseType:      "DateTime",
 				ClickHouseCodec:     "DoubleDelta, LZ4",
 				ConsoleNotDimension: true,
-				ProtobufType:        protoreflect.Uint64Kind,
 			},
 			{Key: ColumnSamplingRate, NoDisable: true, ClickHouseType: "UInt64", ConsoleNotDimension: true},
 			{Key: ColumnExporterAddress, ParserType: "ip", ClickHouseType: "LowCardinality(IPv6)"},
@@ -385,25 +379,10 @@ END`,
 				ClickHouseType:     "Array(UInt32)",
 			},
 			{
-				Key:                ColumnDstLargeCommunities,
-				ClickHouseMainOnly: true,
-				ClickHouseType:     "Array(UInt128)",
-				ClickHouseTransformFrom: []Column{
-					{
-						Key:            ColumnDstLargeCommunitiesASN,
-						ClickHouseType: "Array(UInt32)",
-					},
-					{
-						Key:            ColumnDstLargeCommunitiesLocalData1,
-						ClickHouseType: "Array(UInt32)",
-					},
-					{
-						Key:            ColumnDstLargeCommunitiesLocalData2,
-						ClickHouseType: "Array(UInt32)",
-					},
-				},
-				ClickHouseTransformTo: "arrayMap((asn, l1, l2) -> ((bitShiftLeft(CAST(asn, 'UInt128'), 64) + bitShiftLeft(CAST(l1, 'UInt128'), 32)) + CAST(l2, 'UInt128')), DstLargeCommunitiesASN, DstLargeCommunitiesLocalData1, DstLargeCommunitiesLocalData2)",
-				ConsoleNotDimension:   true,
+				Key:                 ColumnDstLargeCommunities,
+				ClickHouseMainOnly:  true,
+				ClickHouseType:      "Array(UInt128)",
+				ConsoleNotDimension: true,
 			},
 			{Key: ColumnInIfName, ParserType: "string", ClickHouseType: "LowCardinality(String)"},
 			{Key: ColumnInIfDescription, ParserType: "string", ClickHouseType: "LowCardinality(String)", ClickHouseNotSortingKey: true},
@@ -414,13 +393,6 @@ END`,
 				Key:                     ColumnInIfBoundary,
 				ClickHouseType:          fmt.Sprintf("Enum8('undefined' = %d, 'external' = %d, 'internal' = %d)", InterfaceBoundaryUndefined, InterfaceBoundaryExternal, InterfaceBoundaryInternal),
 				ClickHouseNotSortingKey: true,
-				ProtobufType:            protoreflect.EnumKind,
-				ProtobufEnumName:        "Boundary",
-				ProtobufEnum: map[int]string{
-					int(InterfaceBoundaryUndefined): "UNDEFINED",
-					int(InterfaceBoundaryExternal):  "EXTERNAL",
-					int(InterfaceBoundaryInternal):  "INTERNAL",
-				},
 			},
 			{Key: ColumnEType, ClickHouseType: "UInt32"}, // TODO: UInt16 but hard to change, primary key
 			{Key: ColumnProto, ClickHouseType: "UInt32"}, // TODO: UInt8 but hard to change, primary key
@@ -574,9 +546,9 @@ END`,
 	}.finalize()
 }
 
-func (column *Column) shouldBeProto() bool {
-	return column.ClickHouseTransformFrom == nil &&
-		(column.ClickHouseGenerateFrom == "" || column.ClickHouseSelfGenerated) &&
+// shouldProvideValue tells if we should send a value for this column to ClickHouse.
+func (column *Column) shouldProvideValue() bool {
+	return (column.ClickHouseGenerateFrom == "" || column.ClickHouseSelfGenerated) &&
 		column.ClickHouseAlias == ""
 }
 
@@ -592,30 +564,12 @@ func (schema Schema) finalize() Schema {
 			column.Name = name
 		}
 
-		// Also true name for columns in ClickHouseTransformFrom
-		for idx, ecolumn := range column.ClickHouseTransformFrom {
-			if ecolumn.Name == "" {
-				name, ok := columnNameMap.LoadValue(ecolumn.Key)
-				if !ok {
-					panic(fmt.Sprintf("missing name mapping for %d", ecolumn.Key))
-				}
-				column.ClickHouseTransformFrom[idx].Name = name
-			}
-		}
-
 		// Non-main columns with an alias are NotSortingKey
 		if !column.ClickHouseMainOnly && column.ClickHouseAlias != "" {
 			column.ClickHouseNotSortingKey = true
 		}
 
-		// Transform implicit dependencies
-		for idx := range column.ClickHouseTransformFrom {
-			deps := column.ClickHouseTransformFrom[idx].Depends
-			deps = append(deps, column.Key)
-			slices.Sort(deps)
-			column.ClickHouseTransformFrom[idx].Depends = slices.Compact(deps)
-			column.Depends = append(column.Depends, column.ClickHouseTransformFrom[idx].Key)
-		}
+		// Deduplicate dependencies
 		slices.Sort(column.Depends)
 		column.Depends = slices.Compact(column.Depends)
 
@@ -639,7 +593,6 @@ func (schema Schema) finalize() Schema {
 					panic(fmt.Sprintf("missing name mapping for %q", column.Name))
 				}
 				column.ClickHouseAlias = strings.ReplaceAll(column.ClickHouseAlias, "Src", "Dst")
-				column.ClickHouseTransformFrom = slices.Clone(column.ClickHouseTransformFrom)
 				ncolumns = append(ncolumns, column)
 			}
 		} else if strings.HasPrefix(column.Name, "InIf") {
@@ -650,53 +603,12 @@ func (schema Schema) finalize() Schema {
 					panic(fmt.Sprintf("missing name mapping for %q", column.Name))
 				}
 				column.ClickHouseAlias = strings.ReplaceAll(column.ClickHouseAlias, "InIf", "OutIf")
-				column.ClickHouseTransformFrom = slices.Clone(column.ClickHouseTransformFrom)
 				ncolumns = append(ncolumns, column)
 			}
 		}
 	}
 	schema.columns = ncolumns
 
-	// Set Protobuf index and type
-	protobufIndex := 1
-	ncolumns = []Column{}
-	for _, column := range schema.columns {
-		pcolumns := []*Column{&column}
-		for idx := range column.ClickHouseTransformFrom {
-			pcolumns = append(pcolumns, &column.ClickHouseTransformFrom[idx])
-		}
-		for _, column := range pcolumns {
-			if column.ProtobufIndex == 0 {
-				if !column.shouldBeProto() {
-					column.ProtobufIndex = -1
-					continue
-				}
-
-				column.ProtobufIndex = protowire.Number(protobufIndex)
-				protobufIndex++
-			}
-
-			if column.ProtobufType == 0 &&
-				column.shouldBeProto() {
-				switch column.ClickHouseType {
-				case "String", "LowCardinality(String)", "FixedString(2)":
-					column.ProtobufType = protoreflect.StringKind
-				case "UInt64":
-					column.ProtobufType = protoreflect.Uint64Kind
-				case "UInt32", "UInt16", "UInt8":
-					column.ProtobufType = protoreflect.Uint32Kind
-				case "IPv6", "LowCardinality(IPv6)":
-					column.ProtobufType = protoreflect.BytesKind
-				case "Array(UInt32)":
-					column.ProtobufType = protoreflect.Uint32Kind
-					column.ProtobufRepeated = true
-				}
-			}
-		}
-		ncolumns = append(ncolumns, column)
-	}
-	schema.columns = ncolumns
-
 	// Build column index
 	maxKey := ColumnTimeReceived
 	for _, column := range schema.columns {
@@ -707,9 +619,6 @@ func (schema Schema) finalize() Schema {
 	schema.columnIndex = make([]*Column, maxKey+1)
 	for i, column := range schema.columns {
 		schema.columnIndex[column.Key] = &schema.columns[i]
-		for j, column := range column.ClickHouseTransformFrom {
-			schema.columnIndex[column.Key] = &schema.columns[i].ClickHouseTransformFrom[j]
-		}
 	}
 
 	// Update disabledGroups
diff --git a/common/schema/message.go b/common/schema/message.go
new file mode 100644
index 00000000..2a9fb814
--- /dev/null
+++ b/common/schema/message.go
@@ -0,0 +1,110 @@
+// SPDX-FileCopyrightText: 2025 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+package schema
+
+import (
+	"net/netip"
+
+	"github.com/ClickHouse/ch-go/proto"
+	"github.com/bits-and-blooms/bitset"
+)
+
+// FlowMessage is the abstract representation of a flow through various subsystems.
+type FlowMessage struct {
+	TimeReceived uint32
+	SamplingRate uint64
+
+	// For exporter classifier
+	ExporterAddress netip.Addr
+
+	// For interface classifier
+	InIf    uint32
+	OutIf   uint32
+	SrcVlan uint16
+	DstVlan uint16
+
+	// For routing component
+	SrcAddr netip.Addr
+	DstAddr netip.Addr
+	NextHop netip.Addr
+
+	// Core component may override them
+	SrcAS      uint32
+	DstAS      uint32
+	SrcNetMask uint8
+	DstNetMask uint8
+
+	// Only for tests
+	OtherColumns map[ColumnKey]any
+
+	batch  clickhouseBatch
+	schema *Schema
+}
+
+// clickhouseBatch stores columns for efficient streaming. It is embedded
+// inside a FlowMessage.
+type clickhouseBatch struct {
+	columns   []proto.Column // Indexed by ColumnKey
+	columnSet bitset.BitSet  // Track which columns have been set
+	rowCount  int            // Number of rows accumulated
+	input     proto.Input    // Input including all columns to stream to ClickHouse
+}
+
+// reset resets a flow message. All public fields are set to 0,
+// but the current ClickHouse batch is left untouched.
+func (bf *FlowMessage) reset() {
+	*bf = FlowMessage{
+		batch:  bf.batch,
+		schema: bf.schema,
+	}
+	bf.batch.columnSet.ClearAll()
+}
+
+// Clear clears all column data.
+func (bf *FlowMessage) Clear() {
+	bf.reset()
+	bf.batch.input.Reset()
+	bf.batch.rowCount = 0
+}
+
+// ClickHouseProtoInput returns the proto.Input that can be used to stream results
+// to ClickHouse.
+func (bf *FlowMessage) ClickHouseProtoInput() proto.Input {
+	return bf.batch.input
+}
+
+// NewFlowMessage creates a new FlowMessage for the given schema with ClickHouse batch initialized.
+func (schema *Schema) NewFlowMessage() *FlowMessage {
+	bf := &FlowMessage{
+		schema: schema,
+	}
+
+	maxKey := ColumnKey(0)
+	for _, column := range bf.schema.columns {
+		if column.Key > maxKey {
+			maxKey = column.Key
+		}
+	}
+
+	bf.batch.columns = make([]proto.Column, maxKey+1)
+	bf.batch.columnSet = *bitset.New(uint(maxKey + 1))
+	bf.batch.rowCount = 0
+
+	for _, column := range bf.schema.columns {
+		if !column.Disabled && column.shouldProvideValue() {
+			bf.batch.columns[column.Key] = column.newProtoColumn()
+			bf.batch.input = append(bf.batch.input, proto.InputColumn{
+				Name: column.Name,
+				Data: column.wrapProtoColumn(bf.batch.columns[column.Key]),
+			})
+		}
+	}
+
+	return bf
+}
+
+// FlowCount return the number of flows batched
+func (bf *FlowMessage) FlowCount() int {
+	return bf.batch.rowCount
+}
diff --git a/common/schema/protobuf.go b/common/schema/protobuf.go
deleted file mode 100644
index 9e2e9613..00000000
--- a/common/schema/protobuf.go
+++ /dev/null
@@ -1,262 +0,0 @@
-// SPDX-FileCopyrightText: 2022 Free Mobile
-// SPDX-License-Identifier: AGPL-3.0-only
-
-package schema
-
-import (
-	"encoding/base32"
-	"fmt"
-	"hash/fnv"
-	"net/netip"
-	"slices"
-	"strings"
-
-	"github.com/bits-and-blooms/bitset"
-	"google.golang.org/protobuf/encoding/protowire"
-	"google.golang.org/protobuf/reflect/protoreflect"
-)
-
-// ProtobufMessageHash returns the name of the protobuf definition.
-func (schema Schema) ProtobufMessageHash() string {
-	name, _ := schema.protobufMessageHashAndDefinition()
-	return name
-}
-
-// ProtobufDefinition returns the protobuf definition.
-func (schema Schema) ProtobufDefinition() string {
-	_, definition := schema.protobufMessageHashAndDefinition()
-	return definition
-}
-
-// protobufMessageHashAndDefinition returns the name of the protobuf definition
-// along with the protobuf definition itself (.proto file).
-func (schema Schema) protobufMessageHashAndDefinition() (string, string) {
-	lines := []string{}
-	enums := map[string]string{}
-
-	hash := fnv.New128()
-	for _, column := range schema.Columns() {
-		for _, column := range append([]Column{column}, column.ClickHouseTransformFrom...) {
-			if column.ProtobufIndex < 0 {
-				continue
-			}
-
-			t := column.ProtobufType.String()
-
-			// Enum definition
-			if column.ProtobufType == protoreflect.EnumKind {
-				if _, ok := enums[column.ProtobufEnumName]; !ok {
-					definition := []string{}
-					keys := []int{}
-					for key := range column.ProtobufEnum {
-						keys = append(keys, key)
-					}
-					slices.Sort(keys)
-					for _, key := range keys {
-						definition = append(definition, fmt.Sprintf("%s = %d;", column.ProtobufEnum[key], key))
-					}
-					enums[column.ProtobufEnumName] = fmt.Sprintf("enum %s { %s }",
-						column.ProtobufEnumName,
-						strings.Join(definition, " "))
-				}
-				t = column.ProtobufEnumName
-			}
-
-			// Column definition
-			if column.ProtobufRepeated {
-				t = fmt.Sprintf("repeated %s", t)
-			}
-			line := fmt.Sprintf("%s %s = %d;",
-				t,
-				column.Name,
-				column.ProtobufIndex,
-			)
-			lines = append(lines, line)
-			hash.Write([]byte(line))
-		}
-	}
-
-	enumDefinitions := []string{}
-	for _, v := range enums {
-		enumDefinitions = append(enumDefinitions, v)
-		hash.Write([]byte(v))
-	}
-	hashString := base32.StdEncoding.WithPadding(base32.NoPadding).EncodeToString(hash.Sum(nil))
-
-	return hashString, fmt.Sprintf(`
-syntax = "proto3";
-
-message FlowMessagev%s {
- %s
-
- %s
-}
-`, hashString, strings.Join(enumDefinitions, "\n "), strings.Join(lines, "\n "))
-}
-
-// ProtobufMarshal transforms a basic flow into protobuf bytes. The provided flow should
-// not be reused afterwards.
-func (schema *Schema) ProtobufMarshal(bf *FlowMessage) []byte {
-	schema.ProtobufAppendVarint(bf, ColumnTimeReceived, bf.TimeReceived)
-	schema.ProtobufAppendVarint(bf, ColumnSamplingRate, uint64(bf.SamplingRate))
-	schema.ProtobufAppendIP(bf, ColumnExporterAddress, bf.ExporterAddress)
-	schema.ProtobufAppendVarint(bf, ColumnSrcAS, uint64(bf.SrcAS))
-	schema.ProtobufAppendVarint(bf, ColumnDstAS, uint64(bf.DstAS))
-	schema.ProtobufAppendVarint(bf, ColumnSrcNetMask, uint64(bf.SrcNetMask))
-	schema.ProtobufAppendVarint(bf, ColumnDstNetMask, uint64(bf.DstNetMask))
-	schema.ProtobufAppendIP(bf, ColumnSrcAddr, bf.SrcAddr)
-	schema.ProtobufAppendIP(bf, ColumnDstAddr, bf.DstAddr)
-	schema.ProtobufAppendIP(bf, ColumnNextHop, bf.NextHop)
-	if !schema.IsDisabled(ColumnGroupL2) {
-		schema.ProtobufAppendVarint(bf, ColumnSrcVlan, uint64(bf.SrcVlan))
-		schema.ProtobufAppendVarint(bf, ColumnDstVlan, uint64(bf.DstVlan))
-	}
-
-	// Add length and move it as a prefix
-	end := len(bf.protobuf)
-	payloadLen := end - maxSizeVarint
-	bf.protobuf = protowire.AppendVarint(bf.protobuf, uint64(payloadLen))
-	sizeLen := len(bf.protobuf) - end
-	result := bf.protobuf[maxSizeVarint-sizeLen : end]
-	copy(result, bf.protobuf[end:end+sizeLen])
-	bf.protobuf = result
-
-	return result
-}
-
-// ProtobufAppendVarint append a varint to the protobuf representation of a flow.
-func (schema *Schema) ProtobufAppendVarint(bf *FlowMessage, columnKey ColumnKey, value uint64) {
-	// Check if value is 0 to avoid a lookup.
-	if value > 0 {
-		schema.ProtobufAppendVarintForce(bf, columnKey, value)
-	}
-}
-
-// ProtobufAppendVarintForce append a varint to the protobuf representation of a flow, even if it is a 0-value.
-func (schema *Schema) ProtobufAppendVarintForce(bf *FlowMessage, columnKey ColumnKey, value uint64) {
-	column, _ := schema.LookupColumnByKey(columnKey)
-	column.ProtobufAppendVarintForce(bf, value)
-}
-
-// ProtobufAppendVarint append a varint to the protobuf representation of a flow.
-func (column *Column) ProtobufAppendVarint(bf *FlowMessage, value uint64) {
-	if value > 0 {
-		column.ProtobufAppendVarintForce(bf, value)
-	}
-}
-
-// ProtobufAppendVarintForce append a varint to the protobuf representation of a flow, even when 0.
-func (column *Column) ProtobufAppendVarintForce(bf *FlowMessage, value uint64) {
-	bf.init()
-	if column.protobufCanAppend(bf) {
-		bf.protobuf = protowire.AppendTag(bf.protobuf, column.ProtobufIndex, protowire.VarintType)
-		bf.protobuf = protowire.AppendVarint(bf.protobuf, value)
-		bf.protobufSet.Set(uint(column.ProtobufIndex))
-		if debug {
-			column.appendDebug(bf, value)
-		}
-	}
-}
-
-func (column Column) protobufCanAppend(bf *FlowMessage) bool {
-	return column.ProtobufIndex > 0 &&
-		!column.Disabled &&
-		(column.ProtobufRepeated || !bf.protobufSet.Test(uint(column.ProtobufIndex)))
-}
-
-// ProtobufAppendBytes append a slice of bytes to the protobuf representation
-// of a flow.
-func (schema *Schema) ProtobufAppendBytes(bf *FlowMessage, columnKey ColumnKey, value []byte) {
-	if len(value) > 0 {
-		schema.ProtobufAppendBytesForce(bf, columnKey, value)
-	}
-}
-
-// ProtobufAppendBytesForce append a slice of bytes to the protobuf representation
-// of a flow, even when empty
-func (schema *Schema) ProtobufAppendBytesForce(bf *FlowMessage, columnKey ColumnKey, value []byte) {
-	column, _ := schema.LookupColumnByKey(columnKey)
-	column.ProtobufAppendBytesForce(bf, value)
-}
-
-// ProtobufAppendBytes append a slice of bytes to the protobuf representation
-// of a flow.
-func (column *Column) ProtobufAppendBytes(bf *FlowMessage, value []byte) {
-	if len(value) > 0 {
-		column.ProtobufAppendBytesForce(bf, value)
-	}
-}
-
-// ProtobufAppendBytesForce append a slice of bytes to the protobuf representation
-// of a flow, even when empty
-func (column *Column) ProtobufAppendBytesForce(bf *FlowMessage, value []byte) {
-	bf.init()
-	if column.protobufCanAppend(bf) {
-		bf.protobuf = protowire.AppendTag(bf.protobuf, column.ProtobufIndex, protowire.BytesType)
-		bf.protobuf = protowire.AppendBytes(bf.protobuf, value)
-		bf.protobufSet.Set(uint(column.ProtobufIndex))
-		if debug {
-			column.appendDebug(bf, value)
-		}
-	}
-}
-
-// ProtobufAppendIP append an IP to the protobuf representation
-// of a flow.
-func (schema *Schema) ProtobufAppendIP(bf *FlowMessage, columnKey ColumnKey, value netip.Addr) {
-	if value.IsValid() {
-		column, _ := schema.LookupColumnByKey(columnKey)
-		column.ProtobufAppendIPForce(bf, value)
-	}
-}
-
-// ProtobufAppendIP append an IP to the protobuf representation
-// of a flow.
-func (column *Column) ProtobufAppendIP(bf *FlowMessage, value netip.Addr) {
-	if value.IsValid() {
-		column.ProtobufAppendIPForce(bf, value)
-	}
-}
-
-// ProtobufAppendIPForce append an IP to the protobuf representation
-// of a flow, even when not valid
-func (column *Column) ProtobufAppendIPForce(bf *FlowMessage, value netip.Addr) {
-	bf.init()
-	if column.protobufCanAppend(bf) {
-		v := value.As16()
-		bf.protobuf = protowire.AppendTag(bf.protobuf, column.ProtobufIndex, protowire.BytesType)
-		bf.protobuf = protowire.AppendBytes(bf.protobuf, v[:])
-		bf.protobufSet.Set(uint(column.ProtobufIndex))
-		if debug {
-			column.appendDebug(bf, value)
-		}
-	}
-}
-
-func (column *Column) appendDebug(bf *FlowMessage, value interface{}) {
-	if bf.ProtobufDebug == nil {
-		bf.ProtobufDebug = make(map[ColumnKey]interface{})
-	}
-	if column.ProtobufRepeated {
-		if current, ok := bf.ProtobufDebug[column.Key]; ok {
-			bf.ProtobufDebug[column.Key] = append(current.([]interface{}), value)
-		} else {
-			bf.ProtobufDebug[column.Key] = []interface{}{value}
-		}
-	} else {
-		bf.ProtobufDebug[column.Key] = value
-	}
-}
-
-// Bytes returns protobuf bytes. The flow should have been processed by
-// `ProtobufMarshal` first.
-func (bf *FlowMessage) Bytes() []byte {
-	return bf.protobuf
-}
-
-func (bf *FlowMessage) init() {
-	if bf.protobuf == nil {
-		bf.protobuf = make([]byte, maxSizeVarint, 500)
-		bf.protobufSet = *bitset.New(uint(ColumnLast))
-	}
-}
diff --git a/common/schema/types.go b/common/schema/types.go
index 9dba1416..33633d4b 100644
--- a/common/schema/types.go
+++ b/common/schema/types.go
@@ -4,11 +4,8 @@
 package schema
 
 import (
-	"net/netip"
-
+	"github.com/ClickHouse/ch-go/proto"
 	"github.com/bits-and-blooms/bitset"
-	"google.golang.org/protobuf/encoding/protowire"
-	"google.golang.org/protobuf/reflect/protoreflect"
 )
 
 // Schema is the data schema.
@@ -42,18 +39,18 @@ type Column struct {
 	// instead of being retrieved from the protobuf. `TransformFrom' and
 	// `TransformTo' work in pairs. The first one is the set of column in the
 	// raw table while the second one is how to transform it for the main table.
-	ClickHouseType             string
-	ClickHouseMaterializedType string
-	ClickHouseCodec            string
-	ClickHouseAlias            string
-	ClickHouseNotSortingKey    bool
-	ClickHouseGenerateFrom     string
-	ClickHouseTransformFrom    []Column
-	ClickHouseTransformTo      string
-	ClickHouseMainOnly         bool
-	// ClickHouseSelfGenerated identifies a column as being formatted using itself as source
-	ClickHouseSelfGenerated bool
-
+	ClickHouseType             string // ClickHouse type for the column
+	ClickHouseMaterializedType string // ClickHouse type when we request materialization
+	ClickHouseCodec            string // Compression codec
+	ClickHouseAlias            string // Alias expression
+	// ClickHouseNotSortingKey is to be used for columns whose content is
+	// derived from another column. Like Exporter* all derive from
+	// ExporterAddress.
+	ClickHouseNotSortingKey bool
+	// ClickHouseGenerateFrom computes the content of the column using another column
+	ClickHouseGenerateFrom  string
+	ClickHouseMainOnly      bool // Only include this column in the main table
+	ClickHouseSelfGenerated bool // Generated (partly) from its own value
 	// ClickHouseMaterialized indicates that the column was materialized (and is not by default)
 	ClickHouseMaterialized bool
 
@@ -61,55 +58,13 @@ type Column struct {
 	// truncatable when used as a dimension.
 	ConsoleNotDimension bool
 	ConsoleTruncateIP   bool
-
-	// For protobuf. The index is automatically derived from the position,
-	// unless specified. Use -1 to not include the column into the protobuf
-	// schema.
-	ProtobufIndex    protowire.Number
-	ProtobufType     protoreflect.Kind // Uint64Kind, Uint32Kind, BytesKind, StringKind, EnumKind
-	ProtobufEnum     map[int]string
-	ProtobufEnumName string
-	ProtobufRepeated bool
 }
 
 // ColumnKey is the name of a column
-type ColumnKey int
+type ColumnKey uint
 
 // ColumnGroup represents a group of columns
 type ColumnGroup uint
 
-// FlowMessage is the abstract representation of a flow through various subsystems.
-type FlowMessage struct {
-	TimeReceived uint64
-	SamplingRate uint32
-
-	// For exporter classifier
-	ExporterAddress netip.Addr
-
-	// For interface classifier
-	InIf    uint32
-	OutIf   uint32
-	SrcVlan uint16
-	DstVlan uint16
-
-	// For geolocation or BMP
-	SrcAddr netip.Addr
-	DstAddr netip.Addr
-	NextHop netip.Addr
-
-	// Core component may override them
-	SrcAS          uint32
-	DstAS          uint32
-	GotASPath      bool
-	GotCommunities bool
-
-	SrcNetMask uint8
-	DstNetMask uint8
-
-	// protobuf is the protobuf representation for the information not contained above.
-	protobuf      []byte
-	protobufSet   bitset.BitSet
-	ProtobufDebug map[ColumnKey]interface{} `json:"-"` // for testing purpose
-}
-
-const maxSizeVarint = 10 // protowire.SizeVarint(^uint64(0))
+// UInt128 is an unsigned 128-bit number
+type UInt128 = proto.UInt128
diff --git a/config/akvorado.yaml b/config/akvorado.yaml
index 46d98c8c..62249bee 100644
--- a/config/akvorado.yaml
+++ b/config/akvorado.yaml
@@ -6,7 +6,6 @@
 
 kafka:
   topic: flows
-  version: 3.3.1
   brokers:
     - kafka:9092
   topic-configuration:
@@ -76,6 +75,7 @@ clickhouse:
     #     { prefix: (.ipv4Prefix // .ipv6Prefix), tenant: "google-cloud", region: .scope }
 
 inlet: !include "inlet.yaml"
+outlet: !include "outlet.yaml"
 console: !include "console.yaml"
 
 # Remove the following line if you don't want to get demo data
diff --git a/config/demo.yaml b/config/demo.yaml
index 16d96c26..6ac428a2 100644
--- a/config/demo.yaml
+++ b/config/demo.yaml
@@ -96,7 +96,7 @@
         21: "core"
       listen: :161
     bmp: &bmp
-      target: akvorado-inlet:10179
+      target: akvorado-outlet:10179
       routes:
         - prefixes: 192.0.2.0/24,2a01:db8:cafe:1::/64
           aspath: 64501
diff --git a/config/inlet.yaml b/config/inlet.yaml
index 41790c5f..41525491 100644
--- a/config/inlet.yaml
+++ b/config/inlet.yaml
@@ -1,13 +1,4 @@
 ---
-kafka:
-  compression-codec: zstd
-metadata:
-  workers: 10
-  provider:
-    type: snmp
-    credentials:
-      ::/0:
-        communities: public
 flow:
   inputs:
     - type: udp
@@ -20,22 +11,3 @@ flow:
       listen: :6343
       workers: 6
       receive-buffer: 10485760
-core:
-  workers: 6
-  exporter-classifiers:
-    # This is an example. This should be customized depending on how
-    # your exporters are named.
-    - ClassifySiteRegex(Exporter.Name, "^([^-]+)-", "$1")
-    - ClassifyRegion("europe")
-    - ClassifyTenant("acme")
-    - ClassifyRole("edge")
-  interface-classifiers:
-    # This is an example. This must be customized depending on the
-    # descriptions of your interfaces. In the following, we assume
-    # external interfaces are named "Transit: Cogent" Or "IX:
-    # FranceIX".
-    - |
-      ClassifyConnectivityRegex(Interface.Description, "^(?i)(transit|pni|ppni|ix):? ", "$1") &&
-      ClassifyProviderRegex(Interface.Description, "^\\S+?\\s(\\S+)", "$1") &&
-      ClassifyExternal()
-    - ClassifyInternal()
diff --git a/config/outlet.yaml b/config/outlet.yaml
new file mode 100644
index 00000000..a3d9aa5a
--- /dev/null
+++ b/config/outlet.yaml
@@ -0,0 +1,27 @@
+---
+metadata:
+  provider:
+    type: snmp
+    credentials:
+      ::/0:
+        communities: public
+kafka:
+  workers: 6
+core:
+  exporter-classifiers:
+    # This is an example. This should be customized depending on how
+    # your exporters are named.
+    - ClassifySiteRegex(Exporter.Name, "^([^-]+)-", "$1")
+    - ClassifyRegion("europe")
+    - ClassifyTenant("acme")
+    - ClassifyRole("edge")
+  interface-classifiers:
+    # This is an example. This must be customized depending on the
+    # descriptions of your interfaces. In the following, we assume
+    # external interfaces are named "Transit: Cogent" Or "IX:
+    # FranceIX".
+    - |
+      ClassifyConnectivityRegex(Interface.Description, "^(?i)(transit|pni|ppni|ix):? ", "$1") &&
+      ClassifyProviderRegex(Interface.Description, "^\\S+?\\s(\\S+)", "$1") &&
+      ClassifyExternal()
+    - ClassifyInternal()
diff --git a/console/admonitions.go b/console/admonitions.go
new file mode 100644
index 00000000..38f40292
--- /dev/null
+++ b/console/admonitions.go
@@ -0,0 +1,190 @@
+// SPDX-FileCopyrightText: 2025 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+package console
+
+import (
+	"bytes"
+	"regexp"
+	"strings"
+
+	"github.com/yuin/goldmark"
+	"github.com/yuin/goldmark/ast"
+	"github.com/yuin/goldmark/parser"
+	"github.com/yuin/goldmark/renderer"
+	"github.com/yuin/goldmark/text"
+	"github.com/yuin/goldmark/util"
+)
+
+var (
+	admonitionRegexp = regexp.MustCompile(`^\[!(IMPORTANT|NOTE|TIP|WARNING|CAUTION)\]$`)
+	admonitionIcons  = map[string]string{
+		"IMPORTANT": `<path stroke-linecap="round" stroke-linejoin="round" d="M12 9v3.75m9-.75a9 9 0 1 1-18 0 9 9 0 0 1 18 0Zm-9 3.75h.008v.008H12v-.008Z" />`,
+		"NOTE":      `<path stroke-linecap="round" stroke-linejoin="round" d="m11.25 11.25.041-.02a.75.75 0 0 1 1.063.852l-.708 2.836a.75.75 0 0 0 1.063.853l.041-.021M21 12a9 9 0 1 1-18 0 9 9 0 0 1 18 0Zm-9-3.75h.008v.008H12V8.25Z" />`,
+		"TIP":       `<path stroke-linecap="round" stroke-linejoin="round" d="M12 18v-5.25m0 0a6.01 6.01 0 0 0 1.5-.189m-1.5.189a6.01 6.01 0 0 1-1.5-.189m3.75 7.478a12.06 12.06 0 0 1-4.5 0m3.75 2.383a14.406 14.406 0 0 1-3 0M14.25 18v-.192c0-.983.658-1.823 1.508-2.316a7.5 7.5 0 1 0-7.517 0c.85.493 1.509 1.333 1.509 2.316V18" />`,
+		"WARNING":   `<path stroke-linecap="round" stroke-linejoin="round" d="M12 9v3.75m-9.303 3.376c-.866 1.5.217 3.374 1.948 3.374h14.71c1.73 0 2.813-1.874 1.948-3.374L13.949 3.378c-.866-1.5-3.032-1.5-3.898 0L2.697 16.126ZM12 15.75h.007v.008H12v-.008Z" />`,
+		"CAUTION":   `<path stroke-linecap="round" stroke-linejoin="round" d="M12 9v3.75m0-10.036A11.959 11.959 0 0 1 3.598 6 11.99 11.99 0 0 0 3 9.75c0 5.592 3.824 10.29 9 11.622 5.176-1.332 9-6.03 9-11.622 0-1.31-.21-2.57-.598-3.75h-.152c-3.196 0-6.1-1.25-8.25-3.286Zm0 13.036h.008v.008H12v-.008Z" />`,
+	}
+	kindAdmonition = ast.NewNodeKind("Admonition")
+)
+
+// admonitionNode represents an admonition (kind of like a blockquote with an icon)
+type admonitionNode struct {
+	ast.BaseBlock
+	AdmonitionType string
+}
+
+// newAdmonitionNode creates a new admonition node
+func newAdmonitionNode(admonitionType string) *admonitionNode {
+	return &admonitionNode{
+		AdmonitionType: admonitionType,
+	}
+}
+
+// Dump implements ast.Node
+func (n *admonitionNode) Dump(source []byte, level int) {
+	ast.DumpHelper(n, source, level, nil, nil)
+}
+
+// Kind implements ast.Node
+func (n *admonitionNode) Kind() ast.NodeKind {
+	return kindAdmonition
+}
+
+// admonitionTransformer converts blockquotes to admonitions.
+type admonitionTransformer struct{}
+
+// Transform transforms the AST to turns blockquotes into admonitions.
+func (t *admonitionTransformer) Transform(node *ast.Document, reader text.Reader, _ parser.Context) {
+	walker := func(n ast.Node) ast.WalkStatus {
+		blockquote, ok := n.(*ast.Blockquote)
+		if !ok {
+			return ast.WalkContinue
+		}
+
+		// Check if the first child is a paragraph with admonition syntax
+		firstChild := blockquote.FirstChild()
+		if firstChild == nil {
+			return ast.WalkSkipChildren
+		}
+		paragraph, ok := firstChild.(*ast.Paragraph)
+		if !ok {
+			return ast.WalkSkipChildren
+		}
+		lines := paragraph.Lines()
+		if lines.Len() == 0 {
+			return ast.WalkSkipChildren
+		}
+		firstLine := lines.At(0)
+		lineContent := bytes.TrimSpace(firstLine.Value(reader.Source()))
+		matches := admonitionRegexp.FindSubmatch(lineContent)
+		if matches == nil {
+			return ast.WalkSkipChildren
+		}
+
+		// Recreate the paragraph with the admonition marker removed. We assume it was alone on its line.
+		newSegments := text.NewSegments()
+		for i := 1; i < lines.Len(); i++ {
+			line := lines.At(i)
+			newSegments.Append(line)
+		}
+		paragraph.SetLines(newSegments)
+		for child := paragraph.FirstChild(); child != nil; {
+			next := child.NextSibling()
+			if s, ok := child.(*ast.Text); ok && s.Segment.Stop <= firstLine.Stop {
+				paragraph.RemoveChild(paragraph, child)
+			}
+			child = next
+		}
+
+		// Move all children from blockquote to admonition and replace the
+		// blockquote with admonition.
+		admonitionType := string(matches[1])
+		admonitionNode := newAdmonitionNode(admonitionType)
+		admonitionNode.AppendChild(admonitionNode, paragraph)
+		for child := blockquote.FirstChild(); child != nil; {
+			next := child.NextSibling()
+			blockquote.RemoveChild(blockquote, child)
+			admonitionNode.AppendChild(admonitionNode, child)
+			child = next
+		}
+		parent := blockquote.Parent()
+		parent.ReplaceChild(parent, blockquote, admonitionNode)
+
+		return ast.WalkSkipChildren
+	}
+
+	var walk func(ast.Node) ast.WalkStatus
+	// This is almost like ast.Walk(), except it keeps a reference to the next
+	// sibling in case the current element gets removed. We can switch back to
+	// ast.Walk() once https://github.com/yuin/goldmark/pull/523 is merged.
+	walk = func(n ast.Node) ast.WalkStatus {
+		status := walker(n)
+		if status == ast.WalkStop {
+			return status
+		}
+		if status != ast.WalkSkipChildren {
+			for c := n.FirstChild(); c != nil; {
+				next := c.NextSibling()
+				if st := walk(c); st == ast.WalkStop {
+					return st
+				}
+				c = next
+			}
+		}
+		return ast.WalkContinue
+	}
+
+	walk(node)
+}
+
+// admonitionRenderer renders admonitions to HTML
+type admonitionRenderer struct{}
+
+// RegisterFuncs implements renderer.NodeRenderer
+func (r *admonitionRenderer) RegisterFuncs(reg renderer.NodeRendererFuncRegisterer) {
+	reg.Register(kindAdmonition, r.renderAdmonition)
+}
+
+func (r *admonitionRenderer) renderAdmonition(w util.BufWriter, _ []byte, node ast.Node, entering bool) (ast.WalkStatus, error) {
+	n := node.(*admonitionNode)
+
+	if entering {
+		admonitionClass := "admonition admonition-" + strings.ToLower(n.AdmonitionType)
+		admonitionIcon := admonitionIcons[n.AdmonitionType]
+
+		_, _ = w.WriteString(`<div class="` + admonitionClass + `" dir="auto">`)
+		_, _ = w.WriteString(`<p class="admonition-title" dir="auto">`)
+		_, _ = w.WriteString(`<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" aria-hidden="true">`)
+		_, _ = w.WriteString(admonitionIcon)
+		_, _ = w.WriteString(`</svg>`)
+
+		title := strings.ToLower(n.AdmonitionType)
+		if len(title) > 0 {
+			title = strings.ToUpper(title[:1]) + title[1:]
+		}
+		_, _ = w.WriteString(title)
+		_, _ = w.WriteString(`</p>`)
+	} else {
+		_, _ = w.WriteString(`</div>`)
+	}
+
+	return ast.WalkContinue, nil
+}
+
+// admonitionExtension extends goldmark with admonitions
+type admonitionExtension struct{}
+
+// Extend implements goldmark.Extender
+func (e *admonitionExtension) Extend(m goldmark.Markdown) {
+	m.Parser().AddOptions(
+		parser.WithASTTransformers(
+			util.Prioritized(&admonitionTransformer{}, 500),
+		),
+	)
+	m.Renderer().AddOptions(
+		renderer.WithNodeRenderers(
+			util.Prioritized(&admonitionRenderer{}, 500),
+		),
+	)
+}
diff --git a/console/data/docs/00-intro.md b/console/data/docs/00-intro.md
index 0ead3e1f..595a4a9d 100644
--- a/console/data/docs/00-intro.md
+++ b/console/data/docs/00-intro.md
@@ -1,9 +1,9 @@
 # Introduction
 
-*Akvorado*[^name] receives flows (currently Netflow/IPFIX and sFlow), enriches
-them with interface names (using SNMP), geo information (using
-[IPinfo](https://ipinfo.io/) or MaxMind), and exports them to Kafka, then
-ClickHouse. It also exposes a web interface to browse the result.
+*Akvorado*[^name] receives network flows (currently Netflow/IPFIX and sFlow), enriches
+them with interface names (using SNMP), geographic information (using
+[IPinfo](https://ipinfo.io/) or MaxMind), and exports them to ClickHouse via
+Kafka. It also provides a web interface to browse the results.
 
 [^name]: [Akvorado][] means "water wheel" in Esperanto.
 
@@ -21,77 +21,81 @@ install the `docker-compose-v2` package. On macOS, you can use the
 # mkdir akvorado
 # cd akvorado
 # curl -sL https://github.com/akvorado/akvorado/releases/latest/download/docker-compose-quickstart.tar.gz | tar zxvf -
-# docker compose up -d
+# docker compose up
 ```
 
 Monitor the output of `docker compose ps`. Once `akvorado-console` service is
-"healthy", *Akvorado* web interface should be running on port 8081.
-
-A few synthetic flows are generated in the background. To disable them:
-
-1. Remove the reference for `docker-compose-demo.yml` from `.env`,
-2. Comment the last line of `akvorado.yaml`, and
-3. Run `docker compose up -d --remove-orphans`.
-
-If you want to send you own flows, the inlet is accepting both NetFlow
-(port 2055) and sFlow (port 6343). You should also customize some
-settings in `akvorado.yaml`. They are described in details in the
-[“configuration” section](02-configuration.md) section of the
-documentation.
-
-- `clickhouse` → `asns` to give names to your internal AS numbers (optional)
-- `clickhouse` → `networks` to attach attributes to your networks (optional)
-- `inlet` → `metadata` → `provider` → `communities` to set the communities to
-  use for SNMP queries (mandatory)
-- `inlet` → `core` → `exporter-classifiers` to define rules to attach
-  attributes to your exporters (optional)
-- `inlet` → `core` → `interface-classifiers` to define rules to attach
-  attributes to your interfaces, including the "boundary" attribute
-  which is used by default by the web interface (mandatory)
-
-The last point is very important: without classification, the homepage widgets
-won't work and nothing will be displayed by default in the “Visualize” tab. The
-provided configuration expects your interface to be named `Transit: Cogent`,
-`PNI: Akamai`, `PPNI: Fastly`, or `IX: DECIX`.
+present "healthy", *Akvorado* web interface should be running on port 8081. It
+can take a few minutes. The demo includes synthetic flow data to help you
+explore the features.
+
+### Next steps
+
+To connect your own network devices:
+
+1. Disable demo data:
+   - Remove the reference to `docker-compose-demo.yml` from `.env`
+   - Comment out the last line in `akvorado.yaml`
+
+1. Customize the configuration in `akvorado.yaml`:
+   - Set SNMP communities for your devices in `outlet` → `metadata` → `provider` → `communities`
+   - Configure interface classification rules in `outlet` → `core` → `interface-classifiers`
+
+1. Configure your routers/switches to send flows to *Akvorado*:
+   - NetFlow/IPFIX: port 2055
+   - sFlow: port 6343
+   
+1. Restart all containers:
+   - `docker compose down --volumes`
+   - `docker compose up -d`
+
+> [!TIP]
+> Interface classification is essential for the web interface to work properly.
+> Without it, you won't see data in the dashboard widgets or visualization tab.
+> See the [configuration guide](02-configuration.md#classification) for details.
+
+### Need help?
+
+- Check the [installation guide](01-install.md) for other deployment options
+- Read the [configuration guide](02-configuration.md) for detailed setup instructions
+- Review the [operations guide](04-operations.md) for router configuration examples
+- Check the [troubleshooting guide](05-troubleshooting.md) if you run into an issue
 
 You can get all the expanded configuration (with default values) with
 `docker compose exec akvorado-orchestrator akvorado orchestrator
 --check --dump /etc/akvorado/akvorado.yaml`.
 
-Once you are ready, you can run everything in the background with
-`docker compose up -d`.
+> [!IMPORTANT]
+> Please, do not open an issue or start a discussion unless you have read the
+> various chapters of the documentation, notably the [troubleshooting
+> guide](05-troubleshooting.md).
 
 ## Big picture
 
 ![General design](design.svg)
 
-*Akvorado* is split into three components:
+*Akvorado* is split into four components:
+
+- The **inlet service** receives flows from exporters and forwards them unparsed
+  to Kafka.
 
-- The **inlet service** receives flows from exporters. It poll each
-  exporter using SNMP to get the *system name*, the *interface names*,
-  *descriptions* and *speeds*. It applies rules to add attributes to
-  exporters. Interface rules attach to each interface a *boundary*
-  (external or internal), a *network provider* and a *connectivity
-  type* (PNI, IX, transit). Optionally, it may also receive BGP routes
-  through the BMP protocol to get the *AS number*, the *AS path*, and
-  the communities. The flow is exported to *Kafka*, serialized using
-  *Protobuf*.
+- The **outlet service** consumes flows from Kafka, parses them, and enriches
+  them with metadata. It polls each exporter using SNMP to get the *system
+  name*, the *interface names*, *descriptions* and *speeds*. It applies rules to
+  add attributes to exporters. Interface rules attach to each interface a
+  *boundary* (external or internal), a *network provider* and a *connectivity
+  type* (PNI, IX, transit). Optionally, it may also receive BGP routes through
+  the BMP protocol to get the *AS number*, the *AS path*, and the communities.
+  The enriched flows are then exported to ClickHouse.
 
 - The **orchestrator service** configures the internal and external components.
   It creates the *Kafka topic* and configures *ClickHouse* to receive the flows
-  from Kafka. It exposes configuration settings for the other services to use.
-  It provides to ClickHouse additional data, notably *GeoIP* data.
-
-- The **console service** exposes a web interface to look and
-  manipulate the flows stored inside the ClickHouse database.
-
-## Serialized flow schemas
+  from the outlet service. It exposes configuration settings for the other
+  services to use. It provides to ClickHouse additional data, notably *GeoIP*
+  data.
 
-Flows sent to Kafka are encoded with a versioned schema. When the schema
-changes, a different Kafka topic is used. For example, the
-`flows-ZUYGDTE3EBIXX352XPM3YEEFV4` topic receive serialized flows using a
-specific version of the schema. The inlet service exports the schema with its
-HTTP service, via the `/api/v0/inlet/flow.proto` endpoint.
+- The **console service** provides a web interface to view and manipulate the
+  flows stored in the ClickHouse database.
 
 ## ClickHouse database schemas
 
diff --git a/console/data/docs/01-install.md b/console/data/docs/01-install.md
index 6d650eac..1ba881a4 100644
--- a/console/data/docs/01-install.md
+++ b/console/data/docs/01-install.md
@@ -1,11 +1,10 @@
 # Installation
 
-*Akvorado* is written in Go. It provides its 3 components into a
-single binary or Docker image. It also requires an installation of
+*Akvorado* is written in Go. It provides its 4 components in a single binary or
+Docker image. It also requires an installation of
 [Kafka](https://kafka.apache.org/quickstart) and
-[ClickHouse](https://clickhouse.com/docs/en/getting-started/install/).
-They have to be installed separately. For ClickHouse, the minimal
-version is 22.4 (due to the use of the `INTERPOLATE` directive).
+[ClickHouse](https://clickhouse.com/docs/en/getting-started/install/). These
+must be installed separately. For ClickHouse, the minimum version is 22.4.
 
 ## Docker image
 
@@ -22,7 +21,7 @@ Check the `docker/docker-compose.yml` file for an example on how to deploy
 the [quick start procedure](00-intro.md#quick-start). This documentation assumes
 you are running the `docker compose` setup.
 
-If you want to compile the Docker image yourself, you can use `make docker`.
+If you want to compile the Docker image yourself, use `make docker`.
 
 ## Pre-built binary
 
@@ -94,7 +93,7 @@ The following `make` targets are available:
 
 Be sure to read the [changelog](99-changelog.md) before attempting an upgrade.
 Upgrade the orchestrator first. This will update the ClickHouse database if
-needed. Then, upgrade all inlets. Then the console.
+needed. Then, upgrade all inlets and outlets. Then the console.
 
 When using `docker compose`, use the following commands to fetch an updated
 `docker-compose.yml` file and update your installation.
@@ -109,6 +108,5 @@ When using `docker compose`, use the following commands to fetch an updated
 
 The `docker-compose-upgrade.tar.gz` tarball ships `.env.dist` instead of `.env`.
 You may want to check the differences with your setup (most of the time, there
-shouldn't have any). Note that if Zookeeper or Kakfa gets upgraded in the
-process, this can be disruptive. You may want to review the changes in
-`docker/versions.yml`.
+shouldn't be any). Note that if Kafka gets upgraded in the process, this can
+be disruptive. You may want to review the changes in `docker/versions.yml`.
diff --git a/console/data/docs/02-configuration.md b/console/data/docs/02-configuration.md
index 4fc9d9ca..17063e54 100644
--- a/console/data/docs/02-configuration.md
+++ b/console/data/docs/02-configuration.md
@@ -8,13 +8,13 @@ retrieve their configuration.
 The default configuration can be obtained with `docker compose exec
 akvorado-orchestrator akvorado orchestrator --dump --check /dev/null`. Note that
 some sections are generated from the configuration of another section. Notably,
-all Kafka configuration comes from upper-level `kafka` key. Durations must be
+all Kafka configuration comes from the upper-level `kafka` key. Durations must be
 written using strings like `10h20m` or `5s`. Valid time units are `ms`, `s`,
 `m`, and `h`.
 
 It is also possible to override configuration settings using
 environment variables. You need to remove any `-` from key names and
-use `_` to handle nesting. Then, put `AKVORADO_CFG_ORCHESTRATOR_` as a
+use `_` to handle nesting. Then, add `AKVORADO_CFG_ORCHESTRATOR_` as a
 prefix. For example, let's consider the following configuration file:
 
 ```yaml
@@ -37,8 +37,8 @@ AKVORADO_CFG_ORCHESTRATOR_KAFKA_BROKERS=192.0.2.1:9092,192.0.2.2:9092
 
 The orchestrator service has its own configuration, as well as the
 configuration for the other services under the key matching the
-service name (`inlet` and `console`). For each service, it is possible
-to provide a list of configuration. A service can query the
+service name (`inlet`, `outlet`, and `console`). For each service, it is possible
+to provide a list of configurations. A service can query the
 configuration it wants by appending an index to the configuration URL.
 If the index does not match a provided configuration, the first
 configuration is provided.
@@ -48,17 +48,15 @@ gets a section of the configuration file matching its name.
 
 ## Inlet service
 
-This service is configured under the `inlet` key. The main components
-of the inlet services are `flow`, `kafka`, and `core`.
+This service is configured under the `inlet` key. The inlet service receives
+Netflow/IPFIX/sFlow packets and forwards them to Kafka. The main components are
+`flow` and `kafka`.
 
 ### Flow
 
 The flow component handles incoming flows. It accepts the `inputs` key
-to define the list of inputs to receive incoming flows and the
-`rate-limit` key to have an hard-limit on the number of flows/second
-accepted per exporter. When set, the provided rate limit will be
-enforced for each exporter and the sampling rate of the surviving
-flows will be adapted.
+to define the list of inputs to receive incoming flows. The flows are
+encapsulated into protobuf messages and sent to Kafka without parsing.
 
 Each input has a `type` and a `decoder`. For `decoder`, both
 `netflow` or `sflow` are supported. As for the `type`, both `udp`
@@ -90,7 +88,6 @@ flow:
       decoder: sflow
       listen: :6343
       workers: 3
-  workers: 2
 ```
 
 The `file` input should only be used for testing. It supports a
@@ -110,13 +107,54 @@ flow:
       paths:
        - /tmp/flow1.raw
        - /tmp/flow2.raw
-  workers: 2
 ```
 
 Without configuration, *Akvorado* will listen for incoming
 Netflow/IPFIX and sFlow flows on a random port (check the logs to know
 which one).
 
+### Kafka
+
+The inlet service exports received flows to a Kafka topic using the [protocol
+buffers format][].
+
+[protocol buffers format]: https://developers.google.com/protocol-buffers
+
+The following keys are accepted:
+
+- `topic`, `brokers`, `tls`, and `version` keys are described in the
+  configuration for the [orchestrator service](#kafka-2) (the values of these
+  keys are copied from the orchestrator configuration, unless `brokers` is
+  explicitely set)
+- `compression-codec` defines the compression codec to use to compress
+  messages: `none`, `gzip`, `snappy`, `lz4` (the default)*, or `zstd`.
+- `queue-size` defines the maximum number of messages to buffer to Kafka.
+
+The topic name is automatically suffixed by a version number, in case the
+protobuf schema changes in a backward-incompatible way.
+
+## Outlet service
+
+This service is configured under the `outlet` key. The outlet service
+consumes flows from Kafka, parses them, enriches them with metadata
+and routing information, and exports them to ClickHouse. The main
+components are `kafka`, `metadata`, `routing`, and `core`.
+
+### Kafka
+
+The outlet's Kafka component consumes flows from the Kafka topic. The following
+keys are accepted:
+
+- `topic`, `brokers`, `tls`, and `version` keys are described in the
+  configuration for the [orchestrator service](#kafka-2) (the values of these
+  keys are copied from the orchestrator configuration, unless `brokers` is
+  explicitly set)
+- `workers` defines the number of Kafka workers to use
+- `consumer-group` defines the consumer group ID for Kafka consumption
+- `fetch-min-bytes` defines the minimum number of bytes to fetch from Kafka
+- `fetch-max-wait-time` defines how much maximum time to wait for the minimum
+  number of bytes to become available
+
 ### Routing
 
 The routing component optionally fetches source and destination AS numbers, as
@@ -201,162 +239,12 @@ RIS instances known holding the RIB.
 BioRIS currently supports setting prefix, AS, AS Path and communities for the
 given flow.
 
-### Kafka
-
-Received flows are exported to a Kafka topic using the [protocol buffers
-format][]. Each flow is written in the [length-delimited format][].
-
-[protocol buffers format]: https://developers.google.com/protocol-buffers
-[length-delimited format]: https://cwiki.apache.org/confluence/display/GEODE/Delimiting+Protobuf+Messages
-
-The following keys are accepted:
-
-- `topic`, `brokers`, `tls`, and `version` keys are described in the
-  configuration for the [orchestrator service](#kafka-1) (the values of these
-  keys are copied from the orchestrator configuration, unless `brokers` is
-  explicitely set)
-- `flush-interval` defines the maximum flush interval to send received
-  flows to Kafka
-- `flush-bytes` defines the maximum number of bytes to store before
-  flushing flows to Kafka
-- `max-message-bytes` defines the maximum size of a message (it should
-  be equal or smaller to the same setting in the broker configuration)
-- `compression-codec` defines the compression codec to use to compress
-  messages (`none`, `gzip`, `snappy`, `lz4` and `zstd`)
-- `queue-size` defines the size of the internal queues to send
-  messages to Kafka. Increasing this value will improve performance,
-  at the cost of losing messages in case of problems.
-
-The topic name is suffixed by a hash of the schema.
-
-### Core
-
-The core component queries the `metadata` component to
-enrich the flows with additional information. It also classifies
-exporters and interfaces into groups with a set of classification
-rules.
-
-The following configuration keys are accepted:
-
-- `workers` key define how many workers should be spawned to process
-  incoming flows
-- `exporter-classifiers` is a list of classifier rules to define a group
-  for exporters
-- `interface-classifiers` is a list of classifier rules to define
-  connectivity type, network boundary and provider for an interface
-- `classifier-cache-duration` defines how long to keep the result of a previous
-  classification in memory to reduce CPU usage.
-- `default-sampling-rate` defines the default sampling rate to use
-  when the information is missing. If not defined, flows without a
-  sampling rate will be rejected. Use this option only if your
-  hardware is unable to advertise a sampling rate. This can either be
-  a single value or a map from subnets to sampling rates.
-- `override-sampling-rate` defines the sampling rate instead of the
-  one received in the flows. This is useful if a device lie about its
-  sampling rate. This is a map from subnets to sampling rates (but it
-  would also accept a single value).
-- `asn-providers` defines the source list for AS numbers. The available sources
-  are `flow`, `flow-except-private` (use information from flow except if the ASN
-  is private), `routing`, `routing-except-private`, and `geo-ip`. The default
-  value is `flow`, `routing`, `geo-ip`. `geo-ip` should only be used at the end as
-  there is no fallback possible.
-- `net-providers` defines the sources for prefix lengths and nexthop. `flow` uses the value
-  provided by the flow message (if any), while `routing` looks it up using the BMP
-  component. If multiple sources are provided, the value of the first source
-  providing a non-default route is taken. The default value is `flow` and `routing`.
-
-Classifier rules are written using [Expr][].
-
-Exporter classifiers gets the classifier IP address and its hostname.
-If they can make a decision, they should invoke one of the
-`Classify()` functions with the target element as an argument. Once
-classification is done for an element, it cannot be changed by a
-subsequent rule. All strings are normalized (lower case, special chars
-removed).
-
-- `Exporter.IP` for the exporter IP address
-- `Exporter.Name` for the exporter name
-- `ClassifyGroup()` to classify the exporter to a group
-- `ClassifyRole()` to classify the exporter for a role (`edge`, `core`)
-- `ClassifySite()` to classify the exporter to a site (`paris`, `berlin`, `newyork`)
-- `ClassifyRegion()` to classify the exporter to a region (`france`, `italy`, `caraibes`)
-- `ClassifyTenant()` to classify the exporter to a tenant (`team-a`, `team-b`)
-- `Reject()` to reject the flow
-- `Format()` to format a string: `Format("name: %s", Exporter.Name)`
-
-As a compatibility `Classify()` is an alias for `ClassifyGroup()`.
-Here is an example, assuming routers are named
-`th2-ncs55a1-1.example.fr` or `milan-ncs5k8-2.example.it`:
-
-```yaml
-exporter-classifiers:
-  - ClassifySiteRegex(Exporter.Name, "^([^-]+)-", "$1")
-  - Exporter.Name endsWith ".it" && ClassifyRegion("italy")
-  - Exporter.Name matches "^(washington|newyork).*" && ClassifyRegion("usa")
-  - Exporter.Name endsWith ".fr" && ClassifyRegion("france")
-```
-
-Interface classifiers gets the following information and, like exporter
-classifiers, should invoke one of the `Classify()` functions to make a
-decision:
-
-- `Exporter.IP` for the exporter IP address
-- `Exporter.Name` for the exporter name
-- `Interface.Index` for the interface index
-- `Interface.Name` for the interface name
-- `Interface.Description` for the interface description
-- `Interface.Speed` for the interface speed
-- `Interface.VLAN` for VLAN number (you need to enable `SrcVlan` and `DstVlan` in schema)
-- `ClassifyConnectivity()` to classify for a connectivity type (transit, PNI, PPNI, IX, customer, core, ...)
-- `ClassifyProvider()` to classify for a provider (Cogent, Telia, ...)
-- `ClassifyExternal()` to classify the interface as external
-- `ClassifyInternal()` to classify the interface as internal
-- `SetName()` to change the interface name
-- `SetDescription()` to change the interface description
-- `Reject()` to reject the flow
-- `Format()` to format a string: `Format("name: %s", Interface.Name)`
-
-Once an interface is classified for a given criteria, it cannot be
-changed by later rule. Once an interface is classified for all
-criteria, remaining rules are skipped. Connectivity and provider are
-normalized (lower case, special chars removed).
-
-Each `Classify()` function, with the exception of `ClassifyExternal()`
-and `ClassifyInternal()` have a variant ending with `Regex` which
-takes a string and a regex before the original string and do a regex
-match. The original string is expanded using the matching parts of the
-regex. The syntax is the one [from Go][]. If you want to use Perl
-character classes, such as `\d` or `\w`, you need to escape the
-backslash character: `\\d` and `\\w`. To test your regex, you can use
-a site like [regular expressions 101][]. Be sure to use the "Golang"
-flavor. You can use the substition function. In this case, append `.*`
-to your regex to get the [expected result][] (you can keep it in the
-final regex if you prefer).
-
-[regular expressions 101]: https://regex101.com/
-[expected result]: https://regex101.com/r/eg6drf/1
-
-Here is an example, assuming interface descriptions for external
-facing interfaces look like `Transit: Cogent 1-3834938493` or `PNI:
-Netflix (WL6-1190)`.
-
-```yaml
-interface-classifiers:
-  - |
-    ClassifyConnectivityRegex(Interface.Description, "^(?i)(transit|pni|ppni|ix):? ", "$1") &&
-    ClassifyProviderRegex(Interface.Description, "^[^ ]+? ([^ ]+)", "$1") &&
-    ClassifyExternal()
-  - ClassifyInternal()
-```
-
-[expr]: https://expr-lang.org/docs/language-definition
-[from Go]: https://github.com/google/re2/wiki/Syntax
-
 ### Metadata
 
 Flows only include interface indexes. To associate them with an interface name
-and description, metadata are polled. A cache is maintained. There are several
-providers available to poll metadata. The following keys are accepted:
+and description, metadata are retrieved from the exporting routers. A cache is
+maintained. There are several providers available to poll metadata. The
+following keys are accepted:
 
 - `cache-duration` tells how much time to keep data in the cache
 - `cache-refresh` tells how much time to wait before updating an entry
@@ -365,13 +253,13 @@ providers available to poll metadata. The following keys are accepted:
   about to expire or need an update
 - `cache-persist-file` tells where to store cached data on shutdown and
   read them back on startup
-- `workers` tell how many workers to spawn to fetch metadata.
-- `max-batch-requests` define how many requests can be batched together
+- `query-timeout` tells how long to wait for a provider to answer a query
+- `initial-delay` tells how long to wait after starting before applying the
+  standard query timeout
 - `providers` defines the provider configurations
 
-As flows missing interface information are discarded, persisting the
-cache is useful to quickly be able to handle incoming flows. By
-default, no persistent cache is configured.
+As flows missing any interface information are discarded, persisting the cache
+is useful to quickly be able to handle incoming flows.
 
 The `providers` key contains the configuration of the providers. For each, the
 provider type is defined by the `type` key. When using several providers, they
@@ -555,48 +443,138 @@ metadata:
         transform: .exporters[]
 ```
 
-### HTTP
+### Core
 
-The builtin HTTP server serves various pages. Its configuration
-supports the following keys:
+The core component processes flows from Kafka, queries the `metadata` component to
+enrich the flows with additional information, and classifies
+exporters and interfaces into groups with a set of classification
+rules. It also handles flow rate limiting.
 
-- `listen` defines the address and port to listen to.
-- `profiler` enables [Go profiler HTTP
-  interface](https://pkg.go.dev/net/http/pprof). Check the [troubleshooting
-  section](05-troubleshooting.html#profiling) for details. It is enabled by
-  default.
-- `cache` defines the cache backend to use for some HTTP requests. It accepts a
-  `type` key which can be either `memory` (the default value) or `redis`. When
-  using the Redis backend, the following additional keys are also accepted:
-  `protocol` (`tcp` or `unix`), `server` (host and port), `username`,
-  `password`, and `db` (an integer to specify which database to use).
+The following configuration keys are accepted:
+
+- `exporter-classifiers` is a list of classifier rules to define a group
+  for exporters
+- `interface-classifiers` is a list of classifier rules to define
+  connectivity type, network boundary and provider for an interface
+- `classifier-cache-duration` defines how long to keep the result of a previous
+  classification in memory to reduce CPU usage.
+- `default-sampling-rate` defines the default sampling rate to use
+  when the information is missing. If not defined, flows without a
+  sampling rate will be rejected. Use this option only if your
+  hardware is unable to advertise a sampling rate. This can either be
+  a single value or a map from subnets to sampling rates.
+- `override-sampling-rate` defines the sampling rate instead of the
+  one received in the flows. This is useful if a device lie about its
+  sampling rate. This is a map from subnets to sampling rates (but it
+  would also accept a single value).
+- `asn-providers` defines the source list for AS numbers. The available sources
+  are `flow`, `flow-except-private` (use information from flow except if the ASN
+  is private), `routing`, `routing-except-private`, and `geo-ip`. The default
+  value is `flow`, `routing`, `geo-ip`. `geo-ip` should only be used at the end as
+  there is no fallback possible.
+- `net-providers` defines the sources for prefix lengths and nexthop. `flow` uses the value
+  provided by the flow message (if any), while `routing` looks it up using the BMP
+  component. If multiple sources are provided, the value of the first source
+  providing a non-default route is taken. The default value is `flow` and `routing`.
+
+#### Classification
+
+Classifier rules are written in a language called [Expr][].
+
+Interface classifiers gets exporter and interface-related information as input.
+If they can make a decision, they should invoke one of the `Classify()`
+functions with the target element as an argument. Once classification is done
+for an element, it cannot be changed by a subsequent rule. All strings are
+normalized (lower case, special chars removed).
+
+- `Exporter.IP` for the exporter IP address
+- `Exporter.Name` for the exporter name
+- `Interface.Index` for the interface index
+- `Interface.Name` for the interface name
+- `Interface.Description` for the interface description
+- `Interface.Speed` for the interface speed
+- `Interface.VLAN` for VLAN number (you need to enable `SrcVlan` and `DstVlan` in schema)
+- `ClassifyConnectivity()` to classify for a connectivity type (transit, PNI, PPNI, IX, customer, core, ...)
+- `ClassifyProvider()` to classify for a provider (Cogent, Telia, ...)
+- `ClassifyExternal()` to classify the interface as external
+- `ClassifyInternal()` to classify the interface as internal
+- `SetName()` to change the interface name
+- `SetDescription()` to change the interface description
+- `Reject()` to reject the flow
+- `Format()` to format a string: `Format("name: %s", Interface.Name)`
+
+Once an interface is classified for a given criteria, it cannot be
+changed by later rule. Once an interface is classified for all
+criteria, remaining rules are skipped. Connectivity and provider are
+normalized (lower case, special chars removed).
+
+Each `Classify()` function, with the exception of `ClassifyExternal()`
+and `ClassifyInternal()` have a variant ending with `Regex` which
+takes a string and a regex before the original string and do a regex
+match. The original string is expanded using the matching parts of the
+regex. The syntax is the one [from Go][]. If you want to use Perl
+character classes, such as `\d` or `\w`, you need to escape the
+backslash character: `\\d` and `\\w`. To test your regex, you can use
+a site like [regular expressions 101][]. Be sure to use the "Golang"
+flavor. You can use the substition function. In this case, append `.*`
+to your regex to get the [expected result][] (you can keep it in the
+final regex if you prefer).
+
+[regular expressions 101]: https://regex101.com/
+[expected result]: https://regex101.com/r/eg6drf/1
+
+Here is an example, assuming interface descriptions for external
+facing interfaces look like `Transit: Cogent 1-3834938493` or `PNI:
+Netflix (WL6-1190)`.
 
 ```yaml
-http:
-  listen: :8000
-  cache:
-    type: redis
-    username: akvorado
-    password: akvorado
+interface-classifiers:
+  - |
+    ClassifyConnectivityRegex(Interface.Description, "^(?i)(transit|pni|ppni|ix):? ", "$1") &&
+    ClassifyProviderRegex(Interface.Description, "^[^ ]+? ([^ ]+)", "$1") &&
+    ClassifyExternal()
+  - ClassifyInternal()
 ```
 
-Note that the cache backend is currently only useful with the console. You need
-to define the cache in the `http` key of the `console` section for it to be
-useful (not in the `inlet` section).
+The first rule says “extract the connectivity (transit, pni, ppni or ix) from
+the interface description, and if successful, use the second part of the
+description as the provider, and if successful, considers the interface as an
+external one”. The second rule says “if an interface was not classified as
+external or internal, consider it as an internal one.”
 
-### Reporting
+Exporter classifiers gets the classifier IP address and its hostname. Like the
+interface classifiers, they should invoke one of the `Classify()` functions to
+make a decision:
 
-Reporting encompasses logging and metrics. Currently, as *Akvorado* is
-expected to be run inside Docker, logging is done on the standard
-output and is not configurable. As for metrics, they are reported by
-the HTTP component on the `/api/v0/inlet/metrics` endpoint and there is
-nothing to configure either.
+- `Exporter.IP` for the exporter IP address
+- `Exporter.Name` for the exporter name
+- `ClassifyGroup()` to classify the exporter to a group
+- `ClassifyRole()` to classify the exporter for a role (`edge`, `core`)
+- `ClassifySite()` to classify the exporter to a site (`paris`, `berlin`, `newyork`)
+- `ClassifyRegion()` to classify the exporter to a region (`france`, `italy`, `caraibes`)
+- `ClassifyTenant()` to classify the exporter to a tenant (`team-a`, `team-b`)
+- `Reject()` to reject the flow
+- `Format()` to format a string: `Format("name: %s", Exporter.Name)`
+
+As a compatibility `Classify()` is an alias for `ClassifyGroup()`.
+Here is an example, assuming routers are named
+`th2-ncs55a1-1.example.fr` or `milan-ncs5k8-2.example.it`:
+
+```yaml
+exporter-classifiers:
+  - ClassifySiteRegex(Exporter.Name, "^([^-]+)-", "$1")
+  - Exporter.Name endsWith ".it" && ClassifyRegion("italy")
+  - Exporter.Name matches "^(washington|newyork).*" && ClassifyRegion("usa")
+  - Exporter.Name endsWith ".fr" && ClassifyRegion("france")
+```
+
+[expr]: https://expr-lang.org/docs/language-definition
+[from Go]: https://github.com/google/re2/wiki/Syntax
 
 ## Orchestrator service
 
-The two main components of the orchestrator service are `clickhouse` and
-`kafka`. It also uses the [HTTP](#http), and [reporting](#reporting) from the
-inlet service and accepts the same configuration settings.
+The three main components of the orchestrator service are `schema`,
+`clickhouse`, and `kafka`.
 
 ### Schema
 
@@ -805,26 +783,9 @@ configure a ClickHouse database. It also provisions and keep
 up-to-date a ClickHouse database. The following keys can be
 provided inside `clickhouse`:
 
-- `kafka` defines the configuration for the Kafka consumer. The accepted keys are:
-  - `consumers` defines the number of consumers to use to consume messages from
-    the Kafka topic. It is silently bound by the maximum number of threads
-    ClickHouse will use (by default, the number of CPUs). It should also be less
-    than the number of partitions: the additional consumers will stay idle.
-  - `group-name` defines the group name consumers will use to consume messages from the
-    Kafka topic.
-    The default value is "clickhouse".
-  - `engine-settings` defines a list of additional settings for the Kafka engine
-    in ClickHouse. Check [ClickHouse documentation][] for possible values. You
-    can notably tune `kafka_max_block_size`, `kafka_poll_timeout_ms`,
-    `kafka_poll_max_batch_size`, and `kafka_flush_interval_ms`.
 - `resolutions` defines the various resolutions to keep data
 - `max-partitions` defines the number of partitions to use when
   creating consolidated tables
-- `system-log-ttl` defines the TTL for system log tables. Set to 0 to disable.
-  As these tables are partitioned by month, it's useless to use a too low value.
-  The default value is 30 days. This requires a restart of ClickHouse.
-- `prometheus-endpoint` defines the endpoint to configure to expose ClickHouse
-  metrics to Prometheus. When not defined, this is left unconfigured.
 - `networks` maps subnets to attributes. Attributes are `name`, `role`, `site`,
   `region`, and `tenant`. They are exposed as `SrcNetName`, `DstNetName`,
   `SrcNetRole`, `DstNetRole`, etc. It is also possible to override GeoIP
@@ -918,9 +879,8 @@ refreshed. For a given database, the latest paths override the earlier ones.
 
 ## Console service
 
-The main components of the console service are `http`, `console`,
-`authentication` and `database`. `http` accepts the [same configuration](#http)
-as the inlet service.
+The main components of the console service are `console`, `authentication` and
+`database`.
 
 The console itself accepts the following keys:
 
@@ -942,8 +902,8 @@ The console itself accepts the following keys:
    homepage. It defaults to 24 hours.
 
 It also takes a `clickhouse` key, accepting the [same
-configuration](#clickhouse) as the orchestrator service. These keys are copied
-from the orchestrator, unless `servers` is set explicitely.
+configuration](#clickhouse-database) as the orchestrator service. These keys are
+copied from the orchestrator, unless `servers` is set explicitely.
 
 Here is an example:
 
@@ -1090,3 +1050,44 @@ repeating a lot of stuff.
 
 [YAML anchors]: https://www.linode.com/docs/guides/yaml-anchors-aliases-overrides-extensions/
 [clickhouse documentation]: https://clickhouse.com/docs/en/engines/table-engines/integrations/kafka/#table_engine-kafka-creating-a-table
+
+## Common configuration settings
+
+All services also embeds an HTTP and a reporting component.
+
+### HTTP
+
+The builtin HTTP server serves various pages. Its configuration
+supports the following keys:
+
+- `listen` defines the address and port to listen to.
+- `profiler` enables [Go profiler HTTP
+  interface](https://pkg.go.dev/net/http/pprof). Check the [troubleshooting
+  section](05-troubleshooting.html#profiling) for details. It is enabled by
+  default.
+- `cache` defines the cache backend to use for some HTTP requests. It accepts a
+  `type` key which can be either `memory` (the default value) or `redis`. When
+  using the Redis backend, the following additional keys are also accepted:
+  `protocol` (`tcp` or `unix`), `server` (host and port), `username`,
+  `password`, and `db` (an integer to specify which database to use).
+
+```yaml
+http:
+  listen: :8000
+  cache:
+    type: redis
+    username: akvorado
+    password: akvorado
+```
+
+Note that the cache backend is currently only useful with the console. You need
+to define the cache in the `http` key of the `console` section for it to be
+useful.
+
+### Reporting
+
+Reporting encompasses logging and metrics. Currently, as *Akvorado* is expected
+to be run inside Docker, logging is done on the standard output and is not
+configurable. As for metrics, they are reported by the HTTP component on the
+`/api/v0/XXX/metrics` endpoint (where `XXX` is the service name) and there is
+nothing to configure either.
diff --git a/console/data/docs/03-usage.md b/console/data/docs/03-usage.md
index 676950df..f2ad5d98 100644
--- a/console/data/docs/03-usage.md
+++ b/console/data/docs/03-usage.md
@@ -2,7 +2,7 @@
 
 *Akvorado* uses a subcommand system. Each subcommand comes with its
 own set of options. It is possible to get help using `akvorado
---help`. Each service is started using the matchin subcommand. When
+--help`. Each service is started using the matching subcommand. When
 started from a TTY, a service displays logs in a fancy way. Without a
 TTY, logs are output formatted as JSON.
 
@@ -11,7 +11,7 @@ TTY, logs are output formatted as JSON.
 Each service accepts a set of common options as flags.
 
 The `--check` option will check if the provided configuration is
-correct and stops here. The `--dump` option will dump the parsed
+correct and stops there. The `--dump` option will dump the parsed
 configuration, along with the default values. It should be combined
 with `--check` if you don't want the service to start.
 
@@ -26,6 +26,7 @@ file and the other services should point to it.
 ```console
 $ akvorado orchestrator /etc/akvorado/config.yaml
 $ akvorado inlet http://orchestrator:8080
+$ akvorado outlet http://orchestrator:8080
 $ akvorado console http://orchestrator:8080
 $ akvorado console http://orchestrator:8080#2
 ```
@@ -41,16 +42,24 @@ service-specific endpoints:
 Each endpoint is also exposed under the service namespace. The idea is
 to be able to expose an unified API for all services under a single
 endpoint using an HTTP proxy. For example, the `inlet` service also
-exposes its metrics under `/api/v0/inlet/metrics`.
+exposes its metrics under `/api/v0/inlet/metrics` and the `outlet`
+service exposes its metrics under `/api/v0/outlet/metrics`.
 
 ## Inlet service
 
-`akvorado inlet` starts the inlet service, allowing it to receive and
-process flows. The following endpoints are exposed by the HTTP
-component embedded into the service:
+`akvorado inlet` starts the inlet service, allowing it to receive
+Netflow/IPFIX/sFlow packets and forward them to Kafka. The inlet service
+does not expose any service-specific HTTP endpoints.
 
-- `/api/v0/inlet/flows`: stream the received flows
-- `/api/v0/inlet/schemas.proto`: protobuf schema
+## Outlet service
+
+`akvorado outlet` starts the outlet service, allowing it to consume
+flows from Kafka, parse them, enrich them with metadata and routing
+information, and export them to ClickHouse. The following endpoints
+are exposed by the HTTP component embedded into the service:
+
+- `/api/v0/outlet/flows`: stream the received flows (only for debug as there is
+  a performance impact)
 
 ## Orchestrator service
 
@@ -63,13 +72,11 @@ The following endpoints are exposed to configure other internal
 services:
 
 - `/api/v0/orchestrator/configuration/inlet`
+- `/api/v0/orchestrator/configuration/outlet`
 - `/api/v0/orchestrator/configuration/console`
 
 The following endpoints are exposed for use by ClickHouse:
 
-- `/api/v0/orchestrator/clickhouse/init.sh` contains the schemas in the form of a
-  script to execute during initialization to get them installed at the
-  proper location
 - `/api/v0/orchestrator/clickhouse/protocols.csv` contains a CSV with the mapping
   between protocol numbers and names
 - `/api/v0/orchestrator/clickhouse/asns.csv` contains a CSV with the mapping
@@ -132,7 +139,7 @@ aspect of the graph.
   the current period, the previous period can be the previous hour,
   day, week, month, or year.
 
-- The time range can be set from a list of preset or directly using
+- The time range can be set from a list of presets or directly using
   natural language. The parsing is done by
   [SugarJS](https://sugarjs.com/dates/#/Parsing) which provides
   examples of what can be done. Another alternative is to look at the
@@ -151,7 +158,7 @@ aspect of the graph.
   "limit" parameter tells how many. The remaining values are
   categorized as "Other".
 
-- Associated with the `limit` parameter, the `limitType` parameter help find
+- Associated with the `limit` parameter, the `limitType` parameter helps find
   traffic surges according to 2 modes:
   - `avg`: default mode, the query focuses on getting the highest cumulative
     traffics over the time selection.
@@ -166,8 +173,8 @@ aspect of the graph.
   providing a description. A filter can be shared with other users or not.
 
 The URL contains the encoded parameters and can be used to share with
-others. However, currently, no stability of the options are
-guaranteed, so an URL may stop working after a few upgrades.
+others. However, currently, no stability of the options is
+guaranteed, so a URL may stop working after a few upgrades.
 
 ![Sankey graph](sankey.png)
 
@@ -197,8 +204,8 @@ a few examples:
 Field names are case-insensitive. Comments can also be added by using
 `--` for single-line comments or enclosing them in `/*` and `*/`.
 
-The final SQL query sent to ClickHouse is logged inside the console
-after a successful request. It should be noted than using the
+The final SQL query sent to ClickHouse is logged in the console
+after a successful request. It should be noted that using the
 following fields will prevent use of aggregated data and therefore
 will be slower:
 
diff --git a/console/data/docs/04-operations.md b/console/data/docs/04-operations.md
index 6729b097..411872ec 100644
--- a/console/data/docs/04-operations.md
+++ b/console/data/docs/04-operations.md
@@ -12,15 +12,15 @@ service and accepts SNMP requests. For routers not listed below, have
 a look at the [configuration
 snippets](https://github.com/kentik/config-snippets/) from Kentik.
 
-It is better to **sample on ingress only**. This requires to sample on both
-external and internal interfaces, but this prevents flow to be accounted twice
+It is better to **sample on ingress only**. This requires sampling on both
+external and internal interfaces, but this prevents flows from being accounted twice
 when they enter and exit through external ports.
 
 ### Exporter Address
 
 The exporter address is set from the field inside the flow message by default,
-and used e.g. for SNMP requests. However, if for some reasons the set flow
-address (also called agent id) is wrong, you can use the source IP of the flow
+and used e.g. for SNMP requests. However, if for some reason the set flow
+address (also called agent ID) is wrong, you can use the source IP of the flow
 packet instead by setting `use-src-addr-for-exporter-addr: true` for the flow
 configuration.
 
@@ -102,9 +102,9 @@ interface GigabitEthernet0/0/3
 !
 ```
 
-As per [issue #89](https://github.com/akvorado/akvorado/issues/89), the sample
+As per [issue #89](https://github.com/akvorado/akvorado/issues/89), the sampling
 rate is not reported correctly on this platform. The solution is to set a
-default sample rate in `akvorado.yaml`. Check the
+default sampling rate in `akvorado.yaml`. Check the
 [documentation](02-configuration.html#core) for more details.
 
 ```yaml
@@ -150,7 +150,7 @@ flow monitor-map monitor2
 ```
 
 Optionally, AS path can be pushed to the forwarding database and the
-source and destination AS will be present in Netflow packets:
+source and destination AS will be present in NetFlow packets:
 
 ```cisco
 router bgp <asn>
@@ -427,7 +427,7 @@ snmp-server vrf VRF-MANAGEMENT
 Model-driven command line interface (MD-CLI) syntax is used below. The full-context is provided as this is probably easier to adapt to classic CLI.
 
 #### Flows
-sFlow is currently merely supported on devices running SROS, one mostly has to stick to IPFIX
+sFlow is currently barely supported on devices running SROS, one mostly has to stick to IPFIX
 
 ```
 /configure cflowd admin-state enable
@@ -536,9 +536,9 @@ Configure pmacctd with sFlow receiver:
   ifindex=4 ifname=eth1 direction=out
 ```
 
-Here we `set` the interface indexes manually entirely based on the interface
+Here we set the interface indexes manually entirely based on the interface
 names and completely ignoring the kernel ifIndex for the flows. pmacctd can
-be run inside containers where SNMPd does not return description for the
+be run inside containers where SNMPd does not return descriptions for the
 interfaces, which is a required field for the flow. With this setup, you can
 make use of the static metadata provider to match the exporter and accept the
 flow for further classification.
@@ -551,7 +551,7 @@ metrics you can check, notably the space used by each topic.
 
 ## ClickHouse
 
-While ClickHouse works pretty good out-of-the-box, it is still
+While ClickHouse works pretty well out-of-the-box, it is still
 encouraged to read [its documentation](https://clickhouse.com/docs/).
 Altinity also provides a [knowledge base](https://kb.altinity.com/)
 with various other tips.
@@ -561,7 +561,7 @@ with various other tips.
 ClickHouse is configured to log various events into MergeTree tables. By
 default, these tables are unbounded. Unless configured otherwise, the
 orchestrator sets a TTL of 30 days. These tables can also be customized in the
-configuration files or disabled completly. See [ClickHouse
+configuration files or disabled completely. See [ClickHouse
 documentation](https://clickhouse.com/docs/en/operations/system-tables/) for
 more details.
 
@@ -600,7 +600,7 @@ WHERE table LIKE 'flow%'
 GROUP by table
 ```
 
-The following query shows how much space is eaten by each column for the `flows`
+The following query shows how much space is used by each column for the `flows`
 table and how much they are compressed. This can be helpful if you find too much
 space is used by this table.
 
diff --git a/console/data/docs/05-troubleshooting.md b/console/data/docs/05-troubleshooting.md
index 1f997c4b..d4920050 100644
--- a/console/data/docs/05-troubleshooting.md
+++ b/console/data/docs/05-troubleshooting.md
@@ -1,94 +1,214 @@
 # Troubleshooting
 
-## Inlet service
+> [!WARNING]
+> Please read this page carefully before opening an issue or starting a discussion.
 
-The inlet service outputs some logs and exposes some counters to help
-troubleshoot most issues. The first step to check if everything works
-as expected is to request a flow:
+> [!TIP]
+> This guide assumes you use the *Docker Compose* setup. If you use a different setup, adapt the commands as needed.
+
+As explained in the [introduction](00-intro#big-picture), Akvorado has several
+components. To troubleshoot an issue, inspect each component.
+
+![Functional view](troubleshoot.svg)
+
+Your routers send flows to the *inlet*, which forwards them to *Kafka*. The
+*outlet* pulls flows from Kafka, decodes and processes them, then sends them to
+*ClickHouse*. The *orchestrator* configures *Kafka* and *ClickHouse* and
+provides configuration for the *inlet* and *outlet*. The *console* (not shown
+here) queries *ClickHouse* to display flows to users.
+
+## Basic checks
+
+First, check that all components are running and healthy:
 
 ```console
-$ curl -s http://akvorado/api/v0/inlet/flows\?limit=1
-{
- "TimeReceived": 1648305235,
- "SamplingRate": 30000,
+$ docker compose ps --format "table {{.Service}}\t{{.Status}}"
+SERVICE                    STATUS
+akvorado-conntrack-fixer   Up 28 minutes
+akvorado-console           Up 27 minutes (healthy)
+akvorado-inlet             Up 27 minutes (healthy)
+akvorado-orchestrator      Up 27 minutes (healthy)
+akvorado-outlet            Up 27 minutes (healthy)
+clickhouse                 Up 28 minutes (healthy)
+geoip                      Up 28 minutes (healthy)
+kafka                      Up 28 minutes (healthy)
+kafka-ui                   Up 28 minutes
+redis                      Up 28 minutes (healthy)
+traefik                    Up 28 minutes
+```
+
+Make sure all components are present. If a component is missing, restarting,
+unhealthy, or not working properly, check its logs:
+
+```console
+$ docker compose logs akvorado-inlet
+```
+
+The *inlet*, *outlet*, *orchestrator*, and *console* expose metrics. Get them with this command:
+
+```console
+$ curl -s http://127.0.0.1:8080/api/v0/inlet/metrics
+​# HELP akvorado_cmd_info Akvorado build information
+​# TYPE akvorado_cmd_info gauge
+akvorado_cmd_info{compiler="go1.24.4",version="v1.11.5-134-gaf3869cd701c"} 1
 [...]
 ```
 
-Be sure to replace `http://akvorado` with the URL to your *Akvorado*
-setup. If you are running `docker compose` locally, this is
-`http://127.0.0.1:8080`.
+> [!CAUTION]
+> Run the `curl` command on the same host that runs Akvorado, and change `inlet`
+> with the component name you are interested in.
 
-This returns the next flow. The same information is exported to Kafka. If this
-does return a flow, be sure to check the logs and the metrics. The later can be
-queried with `curl`:
+To see only error metrics, filter them:
 
 ```console
-$ curl -s http://akvorado/api/v0/inlet/metrics | grep '^akvorado_inlet'
+$ curl -s http://127.0.0.1:8080/api/v0/inlet/metrics | grep 'akvorado_.*_error'
 ```
 
-### No packets received
+> [!TIP]
+> To follow this guide on a working system, replace `http://127.0.0.1:8080` with `https://demo.akvorado.net`.
 
-When running inside Docker, *Akvorado* may be unable to receive
-packets because the kernel redirects these packets to Docker internal
-proxy. This can be fixed by flushing the conntrack table:
+### Inlet service
+
+The inlet service receives Netflow/IPFIX/sFlow packets and forwards them to
+Kafka. First, check if you receive packets from exporters (your routers):
 
 ```console
-$ conntrack -D -p udp --orig-port-dst 2055
+$ curl -s http://127.0.0.1:8080/api/v0/inlet/metrics | grep 'akvorado_inlet_flow_input_udp_packets'
+​# HELP akvorado_inlet_flow_input_udp_packets_total Packets received by the application.
+​# TYPE akvorado_inlet_flow_input_udp_packets_total counter
+akvorado_inlet_flow_input_udp_packets_total{exporter="241.107.1.12",listener=":2055",worker="2"} 6769
+akvorado_inlet_flow_input_udp_packets_total{exporter="241.107.1.13",listener=":2055",worker="1"} 6794
+akvorado_inlet_flow_input_udp_packets_total{exporter="241.107.1.14",listener=":2055",worker="2"} 6765
+akvorado_inlet_flow_input_udp_packets_total{exporter="241.107.1.15",listener=":2055",worker="0"} 6782
 ```
 
-The shipped `docker-compose.yml` file contains an additional service
-to do that automatically.
+If your exporters are not listed, check their configuration.
 
-To check that you are receiving packets, check the metrics:
+Next, check if flows are sent to Kafka correctly:
 
 ```console
-$ curl -s http://akvorado/api/v0/inlet/metrics | grep '^akvorado_inlet_flow_input_udp_packets'
+$ curl -s http://127.0.0.1:8080/api/v0/inlet/metrics | grep 'akvorado_inlet_kafka_sent_messages'
+​# HELP akvorado_inlet_kafka_sent_messages_total Number of messages sent from a given exporter.
+​# TYPE akvorado_inlet_kafka_sent_messages_total counter
+akvorado_inlet_kafka_sent_messages_total{exporter="241.107.1.12"} 8108
+akvorado_inlet_kafka_sent_messages_total{exporter="241.107.1.13"} 8117
+akvorado_inlet_kafka_sent_messages_total{exporter="241.107.1.14"} 8090
+akvorado_inlet_kafka_sent_messages_total{exporter="241.107.1.15"} 8123
 ```
 
-Also check that the source IP for your exporters is correct. This is
-needed for Akvorado to query them using SNMP. If your exporter cannot
-answer SNMP requests on the source IP address, you can specify an
-alternative address with `inlet`→`snmp`→`agents`.
+If no messages appear here, there may be a Kafka problem.
+
+### Kafka
+
+The *inlet* is sending messages to Kafka and the *outlet* is pulling them from
+Kafka. The Docker Compose setup is shipped with [UI for Apache
+Kafka](https://github.com/provectus/kafka-ui). You can reach it through
+`http://127.0.0.1:8080/kafka-ui`.
+
+> [!TIP]
+> For security reasons, this UI is not exposed on anything else than the host
+> running Akvorado. If you need to access it remotely, the easiest way would be
+> to use [SSH port
+> forwarding](https://www.digitalocean.com/community/tutorials/ssh-port-forwarding):
+> `ssh -L 8080:127.0.0.1:8080 akvorado`. Then, you can use
+> `http://127.0.0.1:8080/kafka-ui` directly from your workstation.
+
+Check the various tabs (brokers, topics, and consumers) to check everything is
+green. In “brokers”, you should see one broker. In “topics”, you should see
+`flows-v5` with an increasing number of messages. This means the *inlet* is
+pushing messages. In “consumers”, you should have `akvorado-outlet`, with at
+least one member. The consumer lag should be stable (and low): this is the
+number of messages that have not been processed by the *outlet* yet.
+
+### Outlet
+
+The *outlet* is the most complex component. Check if it works correctly with
+this command (should show one processed flow):
 
-### No packets exported
+```console
+$ curl -s http://127.0.0.1:8080/api/v0/outlet/flows\?limit\=1
+{"TimeReceived":1753631373,"SamplingRate":100000,"ExporterAddress":"::ffff:241.107.1.15","InIf":10,"OutIf":21,"SrcVlan":0,"DstVlan":0,"SrcAddr":"::ffff:216.58.206.244","DstAddr":"::ffff:192.0.2.144","NextHop":"","SrcAS":15169,"DstAS":64501,"SrcNetMask":24,"DstNetMask":24,"OtherColumns":null}
+```
+
+Check these important metrics. First, the outlet should receive flows from
+Kafka:
+
+```console
+$ curl -s http://127.0.0.1:8080/api/v0/outlet/metrics | grep 'akvorado_outlet_kafka_received_messages'
+​# HELP akvorado_outlet_kafka_received_messages_total Number of messages received for a given worker.
+​# TYPE akvorado_outlet_kafka_received_messages_total counter
+akvorado_outlet_kafka_received_messages_total{worker="0"} 5561
+akvorado_outlet_kafka_received_messages_total{worker="1"} 5456
+akvorado_outlet_kafka_received_messages_total{worker="2"} 5583
+akvorado_outlet_kafka_received_messages_total{worker="3"} 11068
+akvorado_outlet_kafka_received_messages_total{worker="4"} 11151
+akvorado_outlet_kafka_received_messages_total{worker="5"} 5588
+```
 
-*Akvorado* only exports packets with complete information. You can
-check the metrics to find the cause:
+If these numbers are not increasing, there is a problem receiving from Kafka. If
+everything is OK, check if the flow processing pipeline works correctly:
 
 ```console
-$ curl -s http://akvorado/api/v0/inlet/metrics | grep '^akvorado_inlet' | grep _error
+$ curl -s http://127.0.0.1:8080/api/v0/outlet/metrics | grep -P 'akvorado_outlet_core_(received|forwarded)'
+​# HELP akvorado_outlet_core_forwarded_flows_total Number of flows forwarded to Kafka.
+​# TYPE akvorado_outlet_core_forwarded_flows_total counter
+akvorado_outlet_core_forwarded_flows_total{exporter="241.107.1.12"} 182512
+akvorado_outlet_core_forwarded_flows_total{exporter="241.107.1.13"} 182366
+akvorado_outlet_core_forwarded_flows_total{exporter="241.107.1.14"} 182278
+akvorado_outlet_core_forwarded_flows_total{exporter="241.107.1.15"} 182900
+​# HELP akvorado_outlet_core_received_flows_total Number of incoming flows.
+​# TYPE akvorado_outlet_core_received_flows_total counter
+akvorado_outlet_core_received_flows_total{exporter="241.107.1.12"} 182512
+akvorado_outlet_core_received_flows_total{exporter="241.107.1.13"} 182366
+akvorado_outlet_core_received_flows_total{exporter="241.107.1.14"} 182278
+akvorado_outlet_core_received_flows_total{exporter="241.107.1.15"} 182900
+​# HELP akvorado_outlet_core_received_raw_flows_total Number of incoming raw flows (proto).
+​# TYPE akvorado_outlet_core_received_raw_flows_total counter
+akvorado_outlet_core_received_raw_flows_total 45812
 ```
 
-Here is a list of generic errors you may find:
-
-- `SNMP cache miss` means the information about an interface is not
-  found in the SNMP cache. This is expected when Akvorado starts but
-  it should not increase. If this is the case, it is likely because
-  the exporter is not configured to accept SNMP requests or the
-  community configured for SNMP is incorrect.
-- `sampling rate missing` means the sampling rate information is not present.
-  This is also expected when Akvorado starts but it should not increase. With
-  NetFlow, the sampling rate is sent in an options data packet. Be sure to
-  configure your exporter to send them (look for `sampler-table` in the
-  documentation). Alternatively, you can configure
-  `inlet`→`core`→`default-sampling-rate` to workaround this issue.
-- `input and output interfaces missing` means the flow does not contain the
-  input and output interface indexes. This is something to fix on the exporter.
-
-When using NetFlow, you also have the `template not found` error. This
-is expected on start, but then it should not increase anymore.
-
-If *Akvorado* is unable to poll a exporter, no flows about it will be
-exported. In this case, the logs contain information such as:
-
-- `exporter:172.19.162.244 poller breaker open`
-- `exporter:172.19.162.244 unable to GET`
-
-The `akvorado_inlet_metadata_provider_snmp_error_requests_total` metric would also
-increase for the affected exporter. If your routers are in
-`172.16.0.0/12` and you are using Docker, Docker subnets may overlap
-with your routers'. To avoid this, you can put that in
-`/etc/docker/daemon.json` and restart Docker:
+Notably, `akvorado_outlet_core_received_raw_flows_total` is incremented by one
+for each message received from Kafka. The message is then decoded and the flows
+are extracted. For each extracted flow,
+`akvorado_outlet_core_received_flows_total` is incremented by one. The flows are
+then enriched and before forwarding them to ClickHouse,
+`akvorado_outlet_core_forwarded_flows_total` is incremented.
+
+If `akvorado_outlet_core_received_raw_flows_total` increases but
+`akvorado_outlet_core_received_flows_total` does not, there is an error
+**decoding the flows**. If `akvorado_outlet_core_received_flows_total` increases
+but `akvorado_outlet_core_forwarded_flows_total` does not, there is an error
+**enriching the flows**.
+
+For the first case, use this command to find clues:
+
+```console
+$ curl -s http://127.0.0.1:8080/api/v0/outlet/metrics | grep 'akvorado_outlet_flow.*errors'
+```
+
+For the second case, use this one:
+
+```console
+$ curl -s http://127.0.0.1:8080/api/v0/outlet/metrics | grep 'akvorado_outlet_core.*errors'
+```
+
+
+Here is a list of errors you may find:
+
+- `metadata cache miss` means interface information is missing from the metadata
+  cache. The most likely cause is that the exporter does not accept SNMP
+  requests or the SNMP community is configured incorrectly.
+- `sampling rate missing` means sampling rate information is not present. This
+  is normal when Akvorado starts but should not keep increasing. With NetFlow,
+  the sampling rate is sent in an options data packet. Make sure your exporter
+  sends them (look for `sampler-table` in the documentation). Alternatively,
+  configure `outlet`→`core`→`default-sampling-rate` to work around this issue.
+- `input and output interfaces missing` means the flow does not contain input
+  and output interface indexes. Fix this on the exporter.
+
+Another cause for metadata cache misses is that Docker cannot reach the
+exporters because Docker subnets may overlap with your router networks. To fix
+this, add this to `/etc/docker/daemon.json` and restart Docker:
 
 ```json
 {
@@ -97,77 +217,121 @@ with your routers'. To avoid this, you can put that in
 }
 ```
 
-If the exporter address is incorrect, the above configuration will also help.
+Finally, check if flows are sent to ClickHouse successfully. Use this command:
 
-Check that flow are correctly accepted with:
+```
+$ curl -s http://127.0.0.1:8080/api/v0/outlet/metrics | grep -P 'akvorado_outlet_clickhouse_(batches|flows|errors)'
+​# HELP akvorado_outlet_clickhouse_batches_total Number of batches of flows sent to ClickHouse
+​# TYPE akvorado_outlet_clickhouse_batches_total counter
+akvorado_outlet_clickhouse_batches_total 3412
+​# HELP akvorado_outlet_clickhouse_flows_total Number of flows sent to ClickHouse
+​# TYPE akvorado_outlet_clickhouse_flows_total counter
+akvorado_outlet_clickhouse_flows_total 161852
+```
+
+If the numbers are increasing, everything works correctly. Otherwise, you likely
+have an error counter that shows the reason.
+
+### ClickHouse
+
+The last component to check is ClickHouse. Connect to it with this command:
 
 ```console
-$ curl -s http://akvorado/api/v0/inlet/metrics | grep '^akvorado_inlet_core_forwarded_flows_total'
-$ curl -s http://akvorado/api/v0/inlet/flows\?limit=1
+$ docker compose exec clickhouse clickhouse-client
 ```
 
-You can check they are correctly forwarded to Kafka with:
+First, check if all the tables are present:
 
 ```console
-$ curl -s http://akvorado/api/v0/inlet/metrics | grep '^akvorado_inlet_kafka_sent_messages_total'
+$ SHOW TABLES
+    ┌─name────────────────────────────────────────────┐
+ 1. │ asns                                            │
+ 2. │ exporters                                       │
+ 3. │ exporters_consumer                              │
+ 4. │ flows                                           │
+ 5. │ flows_1h0m0s                                    │
+ 6. │ flows_1h0m0s_consumer                           │
+ 7. │ flows_1m0s                                      │
+ 8. │ flows_1m0s_consumer                             │
+ 9. │ flows_5m0s                                      │
+10. │ flows_5m0s_consumer                             │
+11. │ flows_I6D3KDQCRUBCNCGF4BSOWTRMVIv5_raw          │
+12. │ flows_I6D3KDQCRUBCNCGF4BSOWTRMVIv5_raw_consumer │
+13. │ icmp                                            │
+14. │ networks                                        │
+15. │ protocols                                       │
+16. │ tcp                                             │
+17. │ udp                                             │
+    └─────────────────────────────────────────────────┘
 ```
 
-### Reported traffic levels are incorrect
-
-Use `curl -s http://akvorado/api/v0/inlet/flows\?limit=1 | grep
-SamplingRate` to check if the reported sampling rate is correct. If
-not, you can override it with `inlet`→`core`→`override-sampling-rate`.
+Check if the various dictionaries are populated:
 
-Another cause possible cause is when your router is configured to send
-flows for both an interface and its parent. For example, if you have
-an LACP-enabled interface, you should collect flows only for the
-aggregated interface, not for the individual sub interfaces.
+```console
+$ SELECT name, element_count FROM system.dictionaries
+   ┌─name──────┬─element_count─┐
+1. │ networks  │       5963224 │
+2. │ udp       │          5495 │
+3. │ icmp      │            58 │
+4. │ protocols │           129 │
+5. │ asns      │         99598 │
+6. │ tcp       │          5883 │
+   └───────────┴───────────────┘
+```
 
-### No traffic visible on the web interface
+If you have not used the console yet, some dictionaries may be empty.
 
-You may see the last flow widget correctly populated, but nothing else. The
-various widgets on the home page are relying on interface classification to
-retrieve information. Notably, they expect `InIfBoundary` or `OutIfBoundary` to
-be set to external. You can check that classification is done correctly by
-removing any filter rule on the interface and by grouping on `InIfBoundary` (for
-example). If not, be sure that your rules are correct and that descriptions
-match what you expect. For example, on Juniper, if you enable JFlow on a
-sub-interface, be sure that the description is present on this sub-interface.
+To check if ClickHouse is behind, use this SQL query with `clickhouse client` to
+get the lag in seconds:
 
-### 4-byte ASN 23456 in flow data
+```sql
+SELECT (now()-max(TimeReceived))/60
+FROM flows
+```
 
-If you are seeing flows with source or destination AS of 23456, your exporter 
-needs to be configured with 4-byte ASN support. For example, on Cisco IOS-XE:
+If you still have problems, check the errors reported by ClickHouse:
 
-```cisco
-flow record Akvorado
-    collect routing source as 4-octet
-    collect routing destination as 4-octet
-!
+```sql
+SELECT last_error_time, last_error_message
+FROM system.errors
+ORDER BY last_error_time LIMIT 10
+FORMAT Vertical
 ```
 
-### Dropped packets under load
+### Console
+
+The most common console problems are empty widgets or no flows shown in the
+“visualize” tab. Both problems indicate that interface classification is not
+working correctly.
+
+Interface classification marks interfaces as either “internal” or “external”. If
+you have not configured interface classification, see the [configuration
+guide](02-configuration.md#classification). This step is required.
+
+## Scaling
 
-There are various bottlenecks leading to dropped packets. This is bad
-as the reported sampling rate is incorrect and we cannot reliably
-infer the number of bytes and packets.
+Various bottlenecks can cause dropped packets. This is problematic because the
+reported sampling rate becomes incorrect and you cannot reliably calculate the
+number of bytes and packets. Both the exporters and the inlet need to be tuned
+for this kind of problem.
 
-#### Bottlenecks on the exporter
+The outlet can also be a bottleneck. In this case, the flows may appear on the
+console with a delay.
 
-The first problem may come from the exporter dropping some of the
-flows. Most of the time, there are counters to detect this situation
-and it can be solved by lowering the exporter rate.
+### Exporters
 
-##### NCS5500 routers
+The first problem may come from the exporter dropping flows. Usually, counters
+can detect this situation, and you can solve it by reducing the exporter rate.
 
-[Netflow, Sampling-Interval and the Mythical Internet Packet Size][1]
-contains many information about the limit of this platform. The first
-bottleneck is a 133 Mbps shaper between an NPU and the LC CPU for the
-sampled packets (144 bytes each). For example, on a NC55-36X100G line
-card, there are 6 NPU, each one managing 6 ports. If we consider an
-average packet size of 1000, the maximum sampling rate when all ports
-are full is 1:700 (formula is `Total-BW / ( Avg-Pkt-Size x 133Mbps ) x
-( 144 x 8 )`).
+#### NCS5500 routers
+
+[Netflow, Sampling-Interval and the Mythical Internet Packet Size][1] contains
+many information about the limit of this platform. The first bottleneck is a 133
+Mbps shaper between an NPU and the LC CPU for the sampled packets (144 bytes
+each). For example, on a NC55-36X100G line card, there are 6 NPU, each one
+managing 6 ports. If we consider an average packet size of 1000, the maximum
+sampling rate when all ports are full is 1:700 (formula is `Total-BW / (
+Avg-Pkt-Size x 133Mbps ) x ( 144 x 8 )`).
 
 [1]: https://xrdocs.io/ncs5500/tutorials/2018-02-19-netflow-sampling-interval-and-the-mythical-internet-packet-size/
 
@@ -175,8 +339,8 @@ It is possible to check if there are drops with `sh controllers npu
 stats voq base 24 instance 0 location 0/0/CPU0` and looking at the
 `COS2` line.
 
-The second bottleneck is the size of the flow cache. If too small, it
-may overflow. For example:
+The second bottleneck is the size of the flow cache. If too small, it may
+overflow. For example:
 
 ```console
 # show flow monitor monitor1 cache internal location 0/1/CPU0 | i Cache
@@ -188,21 +352,27 @@ Cache Overflows:                         2166590
 Cache above hi water:                       1704
 ```
 
-When this happens, either the `cache timeout rate-limit` should be
-increased or the `cache entries` directive should be increased. The
-later value can be increased to 1 million par monitor-map.
+When this happens, either the `cache timeout rate-limit` should be increased or
+the `cache entries` directive should be increased. The latter value can be
+increased to 1 million per monitor-map.
+
+#### Other routers
+
+Other routers are likely to share the same limitations. It should be noted that
+sFlow and IPFIX 3215 do not have a flow cache and therefore are less likely to
+have scaling problems.
 
-#### Kernel receive buffers
+### Inlet
 
-The second source of drops are the kernel receive buffers. Each
-listening queue has a fixed amount of receive buffers (212992 bytes by
-default) to keep packets before handling them to the application. When
-this buffer is full, packets are dropped.
+When the inlet has scaling issues, the kernel receive buffers may drop packets.
+Each listening queue has a fixed amount of receive buffers (212992 bytes by
+default) to keep packets before handling them to the application. When this
+buffer is full, packets are dropped.
 
-*Akvorado* reports the number of drops for each listening socket with
-the `akvorado_inlet_flow_input_udp_in_drops` counter. This should be
-compared to `akvorado_inlet_flow_input_udp_packets`. Another way to get the same
-information is by using `ss -lunepm` and look at the drop counter:
+*Akvorado* reports the number of drops for each listening socket with the
+`akvorado_inlet_flow_input_udp_in_dropped_packets_total` counter. This should be
+compared to `akvorado_inlet_flow_input_udp_packets_total`. Another way to get
+the same information is by using `ss -lunepm` and look at the drop counter:
 
 ```console
 $ nsenter -t $(pidof akvorado) -n ss -lunepm
@@ -211,36 +381,20 @@ UNCONN           0                0                                        *:205
          skmem:(r0,rb212992,t0,tb212992,f4096,w0,o0,bl0,d486525)
 ```
 
-In the example above, there were 486525 drops. This can be solved
-either by increasing the number of workers for the UDP input or by
-increasing the value of `net.core.rmem_max` sysctl and increasing the
-`receive-buffer` setting attached to the input.
-
-#### Internal queues
-
-Inside the inlet service, parsed packets are transmitted to one module
-to another using channels. When there is a bottleneck at this level,
-the `akvorado_inlet_flow_input_udp_out_drops` counter will increase.
-There are several ways to fix that:
-
-- increasing the channel between the input module and the flow module,
-  with the `queue-size` setting attached to the input,
-- increasing the number of workers for the `core` module,
-- increasing the number of partitions used by the target Kafka topic,
-- increasing the `queue-size` setting for the Kafka module (this can
-  only be used to handle spikes).
-
-#### SNMP poller
-
-To process a flow, the inlet service needs the interface name and
-description. This information is provided by the `snmp` submodule.
-When all workers of the SNMP pollers are busy, new requests are
-dropped. In this case, the `akvorado_inlet_metadata_provider_busy_count`
-counter is increased. To mitigate this issue, the inlet service tries
-to skip exporters with too many errors to avoid blocking SNMP requests
-for other exporters. However, ensuring the exporters accept to answer
-requests is the first fix. If not enough, you can increase the number
-of workers. Workers handle SNMP requests synchronously.
+In the example above, there were 486525 drops. This can be solved in three ways:
+
+- increase the number of workers for the UDP input,
+- increase the value of `net.core.rmem_max` sysctl and increase the
+  `receive-buffer` setting attached to the input,
+- add more inlet instances and shard the exporters among the configured ones.
+
+### Outlet
+
+When the outlet has scaling issues, the data is delivered late at ClickHouse.
+There are two ways to fix that:
+
+- increase the number of Kafka workers,
+- add more outlet instances.
 
 ### Profiling
 
@@ -249,11 +403,14 @@ much CPU or memory. This can be achieved with `pprof`, the [Go
 profiler](https://go.dev/blog/pprof). You need a working [installation of
 Go](https://go.dev/doc/install) on your workstation.
 
-When running on Docker, use `docker inspect` to get the IP address of the inlet:
+When running on Docker, use `docker inspect` to get the IP address of the service 
+you want to profile (inlet or outlet):
 
 ```console
 $ docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' akvorado_akvorado-inlet_1
 240.0.4.8
+$ docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' akvorado_akvorado-outlet_1
+240.0.4.9
 ```
 
 Then, use one of the two following commands:
@@ -280,106 +437,3 @@ $ go tool pprof http://127.0.0.1:6060/debug/pprof/heap
 The first one provides a CPU profile. The second one a memory profile. On the
 command-line, you can type `web` to visualize the result in the browser or `svg`
 to get a SVG file you can attach to a bug report if needed.
-
-## Kafka
-
-There is no easy way to look at the content of the flows in a Kafka
-topic. However, the metadata can be read using
-[kcat](https://github.com/edenhill/kcat/). You can check a topic is
-alive with:
-
-```console
-$ kcat -b kafka:9092 -C -t flows-ZUYGDTE3EBIXX352XPM3YEEFV4 -L
-Metadata for flows-ZUYGDTE3EBIXX352XPM3YEEFV4 (from broker -1: kafka:9092/bootstrap):
- 1 brokers:
-  broker 1001 at eb6c7781b875:9092 (controller)
- 1 topics:
-  topic "flows-ZUYGDTE3EBIXX352XPM3YEEFV4" with 4 partitions:
-    partition 0, leader 1001, replicas: 1001, isrs: 1001
-    partition 1, leader 1001, replicas: 1001, isrs: 1001
-    partition 2, leader 1001, replicas: 1001, isrs: 1001
-    partition 3, leader 1001, replicas: 1001, isrs: 1001
-$ kcat -b kafka:9092 -C -t flows-ZUYGDTE3EBIXX352XPM3YEEFV4 -f 'Topic %t [%p] at offset %o: key %k: %T\n' -o -1
-```
-
-Alternatively, when using `docker compose`, there is a Kafka UI
-running at `http://127.0.0.1:8080/kafka-ui/`. You can do the following
-checks:
-
-- are the brokers alive?
-- is the `flows-ZUYGDTE3EBIXX352XPM3YEEFV4` topic present and receiving messages?
-- is ClickHouse registered as a consumer?
-
-## ClickHouse
-
-First, check that all the tables are present using the following SQL
-query through `clickhouse client` (when running with `docker compose`,
-you can use `docker compose exec clickhouse clickhouse-client`) :
-
-```sql
-SHOW tables
-```
-
-You should have a few tables, including `flows`, `flows_1m0s` (and
-others), and `flows_3_raw`. If one is missing, look at the log in the
-orchestrator. This is the component creating the tables.
-
-To check if ClickHouse is late, use the following SQL query through
-`clickhouse client` to get the lag in seconds.
-
-```sql
-SELECT (now()-max(TimeReceived))/60
-FROM flows
-```
-
-If the lag is too big, you need to increase the number of consumers. See
-[ClickHouse configuration](02-configuration.md#clickhouse) for details. If
-ClickHouse ingestion stalls from time to time, it may be because ClickHouse
-consumer group is being rebalanced. In
-`/var/log/clickhouse-server/clickhouse-server.err.log`, you may see something
-about `Application maximum poll interval exceeded: leaving group`. In this case,
-you may want to lower the flush interval:
-
-```yaml
-clickhouse:
-  kafka:
-    engine-settings:
-      - kafka_flush_interval_ms = 1000
-```
-
-Another way to achieve the same thing is to look at the consumer group
-from Kafka's point of view:
-
-```console
-$ kafka-consumer-groups.sh --bootstrap-server kafka:9092 --describe --group clickhouse
-
-GROUP           TOPIC           PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID                                                                        HOST            CLIENT-ID
-clickhouse      flows-ZUYG…     0          5650351527      5650374314      22787           ClickHouse-ee97b7e7e5e0-default-flows_3_raw-0-77740d0a-79b7-4bef-a501-25a819c3cee4 /240.0.4.8      ClickHouse-ee97b7e7e5e0-default-flows_3_raw-0
-clickhouse      flows-ZUYG…     3          3035602619      3035628290      25671           ClickHouse-ee97b7e7e5e0-default-flows_3_raw-3-1e4629b0-69a3-48dd-899a-20f4b16be0a2 /240.0.4.8      ClickHouse-ee97b7e7e5e0-default-flows_3_raw-3
-clickhouse      flows-ZUYG…     2          1645914467      1645930257      15790           ClickHouse-ee97b7e7e5e0-default-flows_3_raw-2-79c9bafe-fd36-42fe-921f-a802d46db684 /240.0.4.8      ClickHouse-ee97b7e7e5e0-default-flows_3_raw-2
-clickhouse      flows-ZUYG…     1          889117276       889129896       12620           ClickHouse-ee97b7e7e5e0-default-flows_3_raw-1-f0421bbe-ba13-49df-998f-83e49045be00 /240.0.4.8      ClickHouse-ee97b7e7e5e0-default-flows_3_raw-1
-```
-
-Errors related to Kafka ingestion are kept in the `flows_3_raw_errors`
-table. It should be empty.
-
-If you still have an issue, be sure to check the errors reported by
-ClickHouse:
-
-```sql
-SELECT last_error_time, last_error_message
-FROM system.errors
-ORDER BY last_error_time LIMIT 10
-FORMAT Vertical
-```
-
-Notably, it may complain about a missing schema for a received
-message. In this case, you need to ensure the schemas used by
-*Akvorado* are available. When using `docker compose`, you can restart
-the orchestrator and ClickHouse to ensure it downloads the latest
-schemas. Otherwise, you can manually execute the script installing the
-schemas on your ClickHouse server and restart:
-
-```sh
-curl http://akvorado/api/v0/orchestrator/clickhouse/init.sh | sh
-```
diff --git a/console/data/docs/06-internals.md b/console/data/docs/06-internals.md
index 130d05f5..cc88999f 100644
--- a/console/data/docs/06-internals.md
+++ b/console/data/docs/06-internals.md
@@ -1,11 +1,11 @@
 # Internal design
 
-*Akvorado* is written in Go. Each service has its code in a distinct
-directory (`inlet/`, `orchestrator/` and `console/`). The `common/`
-directory contains components common to several services. The `cmd/`
-directory contains the main entry points.
+*Akvorado* is written in Go. Each service has its code in a distinct directory
+(`inlet/`, `outlet/`, `orchestrator/` and `console/`). The `common/` directory
+contains components common to several services. The `cmd/` directory contains
+the main entry points.
 
-Each service is splitted into several components. This is heavily
+Each service is split into several components. This is heavily
 inspired by the [Component framework in Clojure][]. A component is a
 piece of software with its configuration, its state and its
 dependencies on other components.
@@ -22,7 +22,7 @@ Each component features the following piece of code:
   configuration.
 - A `New()` function instantiating the component. This method takes
   the configuration and the dependencies. It is inert.
-- Optionally, a `Start()` method to start the routines associated to
+- Optionally, a `Start()` method to start the routines associated with
   the component.
 - Optionally, a `Stop()` method to stop the component.
 
@@ -50,7 +50,7 @@ For logs, it is mostly a façade to
 code to append the module name to the logs.
 
 For metrics, it is a façade to the [Prometheus instrumentation
-library][]. It provides a registry which automatically append metric
+library][]. It provides a registry which automatically appends metric
 names with the module name.
 
 It also exposes a simple way to report healthchecks from various
@@ -79,36 +79,41 @@ handled by [mapstructure](https://github.com/mitchellh/mapstructure).
 Handling backward compatibility is done by registering hooks to
 transform the configuration.
 
-## Flow decoding
+## Flow processing
 
-Decoding is handled by
-[GoFlow2](https://github.com/NetSampler/GoFlow2). The network code to
-receive flows is heavily inspired by it but not reused. While logging is
-often abstracted, this is not the case for metrics. Moreover, the
-design to scale is a bit different as *Akvorado* will create a socket
-for each worker instead of distributing incoming flows using a channel.
+Flow processing is split between inlet and outlet services:
 
-Netflow v5, Netflow v9, IPFIX, and sFlow are currently supported.
+### Inlet flow reception
 
-The design of this component is modular. It is possible to "plug"
-new decoders and new inputs easily. It is expected that most buffering
-is implemented at this level by input modules that require them.
-Additionnal buffering happens in the Kafka module. When the input is the
-network, this does not really matter as we cannot really block without
-losing messages. However, with file-backed modules, it may be more reliable
-to reduce buffers as data can be lost during shutdown.
+The inlet service receives flows. The design prioritizes speed and minimal
+processing: flows are encapsulated into protobuf messages and sent to Kafka
+without parsing. The design scales by creating a socket for each worker instead
+of distributing incoming flows using a channel.
+
+NetFlow v5, NetFlow v9, IPFIX, and sFlow are currently supported for reception.
+
+The design of this component is modular. It is possible to "plug" new inputs
+easily. Most buffering is implemented at this level by input modules that
+require them. Additional buffering happens in the Kafka module.
+
+### Outlet flow decoding
+
+The outlet service consumes flows from Kafka and performs the actual decoding
+using [GoFlow2](https://github.com/NetSampler/GoFlow2). This is where flow 
+parsing, enrichment with metadata and routing information, and classification 
+happen before writing to ClickHouse.
 
 ## Kafka
 
-The Kafka component relies on
-[Sarama](https://github.com/IBM/sarama). It is tested using the
-mock interface provided by this package. *Sarama* uses `go-metrics` to
-store metrics. We convert them to Prometheus to keep them. The logger
-is global and there is a hack to be plug it into the reporter design
-we have.
+The Kafka component relies on [franz-go](https://github.com/twmb/franz-go). If a
+real broker is available under the DNS name `kafka` or at `localhost` on port
+9092, it will be used for a quick functional test.
 
-If a real broker is available under the DNS name `kafka` or at
-`localhost` on port 9092, it will be used for a quick functional test.
+This library did not get benchmarked. Previously, we were using
+[Sarama](https://github.com/IBM/sarama). However, the documentation is quite
+poor, it relies heavily on pointers (pressure on the garbage collector) and the
+concurrency model is difficult to understand. Another contender could be
+[kafka-go](https://github.com/segmentio/kafka-go).
 
 ## ClickHouse
 
@@ -121,18 +126,20 @@ to use code to check if the existing tables are up-to-date
 and to update them. For example, we may want to check if the Kafka
 settings of a table or the source URL of a dictionary are current.
 
+When inserting into ClickHouse, we rely on the low-level
+[ch-go](https://github.com/ClickHouse/ch-go/) library. Decoded flows are batched
+directly into the wire format used by ClickHouse.
+
 Functional tests are run when a ClickHouse server is available under
 the name `clickhouse` or on `localhost`.
 
 ## SNMP
 
 SNMP polling is accomplished with [GoSNMP](https://github.com/gosnmp/gosnmp).
-The cache layer is tailored specifically for our needs. Cached information
-can expire if not accessed or refreshed periodically.
-Some coaelescing of the requests are done when they are queued.
-This adds some code complexity, maybe it was not worth it.
-If a exporter fails to answer too frequently, a backoff will be triggered
-for a minute to ensure it does not eat up all the workers' resources.
+The cache layer is tailored specifically for our needs. Cached information can
+expire if not accessed or refreshed periodically. If an exporter fails to answer
+too frequently, a backoff will be triggered for a minute to ensure it does not
+eat up all the workers' resources.
 
 Testing is done by another implementation of an [SNMP
 agent](https://github.com/slayercat/GoSNMPServer).
@@ -163,7 +170,7 @@ needs to follow these steps:
 
 1. Add its symbol to `common/schema/definition.go`.
 2. Add it to the `flow()` function in `common/schema/definition.go`. Be sure to
-   specify the right/smaller ClickHouse type. If the columns is prefixed with
+   specify the right/smallest ClickHouse type. If the column is prefixed with
    `Src` or `InIf`, don't add the opposite direction, this is done
    automatically. Use `ClickHouseMainOnly` if the column is expected to take a
    lot of space. Add the column to the end and set `Disabled` field to `true`.
@@ -229,8 +236,9 @@ JS, it is just a matter of copy/pasting and customizing.
 
 ## Other components
 
-The core component is the main one. It takes the other as dependencies
-but there is nothing exciting about it.
+The core component is the main processing component in the outlet service. 
+It takes metadata, routing, and other components as dependencies and 
+orchestrates the flow enrichment and classification process.
 
 The HTTP component exposes a web server. Its main role is to manage
 the lifecycle of the HTTP server and to provide a method to add
@@ -241,7 +249,7 @@ section](03-usage.md).
 
 The daemon component handles the lifecycle of the whole application.
 It watches for the various goroutines (through tombs, see below)
-spawned by the other components and wait for signals to terminate. If
+spawned by the other components and waits for signals to terminate. If
 *Akvorado* had a systemd integration, it would take place here too.
 
 ## Other interesting dependencies
@@ -261,7 +269,7 @@ spawned by the other components and wait for signals to terminate. If
   implements several resiliency patterns, including the breaker
   pattern.
 - [github.com/go-playground/validator](https://github.com/go-playground/validator)
-  implements struct validation using tags. We use it to had better
+  implements struct validation using tags. We use it to have better
   validation on configuration structures.
 
 [go-archaius]: https://github.com/go-chassis/go-archaius
diff --git a/console/data/docs/99-changelog.md b/console/data/docs/99-changelog.md
index 7065bcdf..7a5daede 100644
--- a/console/data/docs/99-changelog.md
+++ b/console/data/docs/99-changelog.md
@@ -13,15 +13,47 @@ identified with a specific icon:
 
 ## Unreleased
 
+This release introduce a new component: the outlet. Previously, ClickHouse was
+fetching data directly from Kafka. However, this required to push the protobuf
+schema using an out-of-band method. This makes cloud deployments more complex.
+The inlet now pushes incoming raw flows to Kafka without decoding them. The
+outlet takes them, decode them, enriches them, and push them to ClickHouse. This
+also reduces the likeliness to lose packets. This change should be transparent
+on most setups but you are encouraged to review the new proposed configuration
+in the [quickstart tarball][] and update your own configuration.
+
+As it seems a good time as any, Zookeeper is removed from the `docker compose`
+setup. ClickHouse Keeper is used instead when setting up a cluster. Kafka is now
+using the KRaft mode. While migration is possible, it is easier to start from
+scratch:
+
+```console
+# docker compose down --remove-orphans
+# docker compose rm -v kafka
+# docker compose pull
+# docker compose up -d
+```
+
+- 💥 *outlet*: new service
+- 💥 *inlet*: flow rate limiting feature has been removed
+- 💥 *docker*: rename `docker-compose-monitoring.yml` to
+  `docker-compose-prometheus.yml` (you need to update your `.env` if you were
+  using it)
+- 💥 *docker*: switch to Apache Kafka 4.0
+- 💥 *docker*: switch Kafka to KRaft mode
 - 🩹 *console*: fix deletion of saved filters
 - 🩹 *console*: fix intermittent failure when requesting previous period
 - 🩹 *docker*: move healthcheck for IPinfo updater into Dockerfile to avoid
   "unhealthy" state on non-updated installation
 - 🌱 *docker*: enable access log for Traefik
+- 🌱 *docker*: update ClickHouse to 25.3 (not mandatory)
 - 🌱 *docker*: update Traefik to 3.4 (not mandatory)
+- 🌱 *docker*: switch to Prometheus Java Agent exporter for Kafka
 - 🌱 *orchestrator*: move ClickHouse database settings from `clickhouse` to `clickhousedb`
 - 🌱 *inlet*: improve performance of classifiers
 
+[quickstart tarball]: https://github.com/akvorado/akvorado/releases/latest/download/docker-compose-quickstart.tar.gz
+
 ## 1.11.5 - 2025-05-11
 
 - 💥 *console*: Firefox 128+, Safari 16.4+, or Chrome 111+ are now required
diff --git a/console/data/docs/design.svg b/console/data/docs/design.svg
index ff75abcf..e9bdd2dd 100644
Binary files a/console/data/docs/design.svg and b/console/data/docs/design.svg differ
diff --git a/console/data/docs/troubleshoot.svg b/console/data/docs/troubleshoot.svg
new file mode 100644
index 00000000..43282a21
Binary files /dev/null and b/console/data/docs/troubleshoot.svg differ
diff --git a/console/docs.go b/console/docs.go
index 0053b257..c35910f9 100644
--- a/console/docs.go
+++ b/console/docs.go
@@ -110,6 +110,7 @@ func (c *Component) docsHandlerFunc(gc *gin.Context) {
 			highlighting.NewHighlighting(
 				highlighting.WithCustomStyle(draculaStyle),
 			),
+			&admonitionExtension{},
 		),
 		goldmark.WithParserOptions(
 			parser.WithAutoHeadingID(),
diff --git a/console/frontend/src/tailwind.css b/console/frontend/src/tailwind.css
index c133dbd3..e339f36b 100644
--- a/console/frontend/src/tailwind.css
+++ b/console/frontend/src/tailwind.css
@@ -34,3 +34,43 @@
     }
   }
 }
+
+/* Admonitions */
+.prose .admonition {
+  @apply my-4 rounded-md border-l-4 p-4;
+
+  .admonition-title {
+    @apply mt-0 flex items-center font-bold;
+
+    svg {
+      @apply mr-2 h-4 w-4;
+    }
+  }
+
+  p:last-child {
+    @apply mb-0;
+  }
+}
+
+.prose .admonition-important {
+  @apply border-purple-500 bg-purple-50 text-purple-800 dark:bg-purple-900/20 dark:text-purple-200;
+}
+
+.prose .admonition-note {
+  @apply border-blue-500 bg-blue-50 text-blue-800 dark:bg-blue-900/20 dark:text-blue-200;
+}
+
+.prose .admonition-tip {
+  @apply border-green-500 bg-green-50 text-green-800 dark:bg-green-900/20 dark:text-green-200;
+}
+
+.prose .admonition-warning {
+  @apply border-yellow-500 bg-yellow-50 text-yellow-800 dark:bg-yellow-900/20 dark:text-yellow-200;
+  svg {
+    transform: translateY(2px);
+  }
+}
+
+.prose .admonition-caution {
+  @apply border-red-500 bg-red-50 text-red-800 dark:bg-red-900/20 dark:text-red-200;
+}
diff --git a/default.pgo b/default.pgo
index bcb32eca..3634ab8b 100644
Binary files a/default.pgo and b/default.pgo differ
diff --git a/demoexporter/bmp/client.go b/demoexporter/bmp/client.go
index bdf46bc3..59f97022 100644
--- a/demoexporter/bmp/client.go
+++ b/demoexporter/bmp/client.go
@@ -22,7 +22,7 @@ func (c *Component) startBMPClient(ctx context.Context) {
 	conn, err := d.DialContext(ctx, "tcp", c.config.Target)
 	if err != nil {
 		c.r.Err(err).Msg("cannot connect to target")
-		c.metrics.errors.WithLabelValues(err.Error()).Inc()
+		c.metrics.errors.WithLabelValues("cannot connect").Inc()
 		return
 	}
 	c.metrics.connections.Inc()
@@ -116,7 +116,7 @@ func (c *Component) startBMPClient(ctx context.Context) {
 	// Send the packets on the wire
 	if _, err := conn.Write(buf.Bytes()); err != nil {
 		c.r.Err(err).Msg("cannot write BMP message to target")
-		c.metrics.errors.WithLabelValues(err.Error()).Inc()
+		c.metrics.errors.WithLabelValues("cannot write").Inc()
 		return
 	}
 
@@ -135,7 +135,7 @@ func (c *Component) startBMPClient(ctx context.Context) {
 			buf.Write(pkt)
 			if _, err := conn.Write(buf.Bytes()); err != nil && err != io.EOF && !errors.Is(err, syscall.ECONNRESET) && !errors.Is(err, syscall.EPIPE) {
 				c.r.Err(err).Msg("cannot write to remote")
-				c.metrics.errors.WithLabelValues(err.Error()).Inc()
+				c.metrics.errors.WithLabelValues("cannot write").Inc()
 				close(done)
 				return
 			} else if err != nil {
diff --git a/demoexporter/flows/root.go b/demoexporter/flows/root.go
index 7e9020a7..5d24c481 100644
--- a/demoexporter/flows/root.go
+++ b/demoexporter/flows/root.go
@@ -87,7 +87,7 @@ func (c *Component) Start() error {
 			for payload := range payloads {
 				sequenceNumber++
 				if _, err := conn.Write(payload); err != nil {
-					c.metrics.errors.WithLabelValues(err.Error()).Inc()
+					c.metrics.errors.WithLabelValues("cannot write").Inc()
 					errLogger.Err(err).Msg("unable to send UDP payload")
 				} else {
 					c.metrics.sent.WithLabelValues(kind).Inc()
diff --git a/docker/clickhouse/akvorado.xml b/docker/clickhouse/akvorado.xml
new file mode 100644
index 00000000..92415bb6
--- /dev/null
+++ b/docker/clickhouse/akvorado.xml
@@ -0,0 +1,28 @@
+<clickhouse>
+ <!-- Don't keep logs forever -->
+ <asynchronous_metric_log>
+  <ttl>event_date + INTERVAL 30 DAY DELETE</ttl>
+ </asynchronous_metric_log>
+ <metric_log>
+  <ttl>event_date + INTERVAL 30 DAY DELETE</ttl>
+ </metric_log>
+ <part_log>
+  <ttl>event_date + INTERVAL 30 DAY DELETE</ttl>
+ </part_log>
+ <query_log>
+  <ttl>event_date + INTERVAL 30 DAY DELETE</ttl>
+ </query_log>
+ <query_thread_log>
+  <ttl>event_date + INTERVAL 30 DAY DELETE</ttl>
+ </query_thread_log>
+ <trace_log>
+  <ttl>event_date + INTERVAL 30 DAY DELETE</ttl>
+ </trace_log>
+ <!-- Enable Prometheus endpoint -->
+ <prometheus>
+  <endpoint>/metrics</endpoint>
+  <metrics>true</metrics>
+  <events>true</events>
+  <asynchronous_metrics>true</asynchronous_metrics>
+ </prometheus>
+</clickhouse>
diff --git a/docker/clickhouse-cluster-1.xml b/docker/clickhouse/cluster-1.xml
similarity index 100%
rename from docker/clickhouse-cluster-1.xml
rename to docker/clickhouse/cluster-1.xml
diff --git a/docker/clickhouse-cluster-2.xml b/docker/clickhouse/cluster-2.xml
similarity index 100%
rename from docker/clickhouse-cluster-2.xml
rename to docker/clickhouse/cluster-2.xml
diff --git a/docker/clickhouse-cluster-3.xml b/docker/clickhouse/cluster-3.xml
similarity index 100%
rename from docker/clickhouse-cluster-3.xml
rename to docker/clickhouse/cluster-3.xml
diff --git a/docker/clickhouse-cluster-4.xml b/docker/clickhouse/cluster-4.xml
similarity index 100%
rename from docker/clickhouse-cluster-4.xml
rename to docker/clickhouse/cluster-4.xml
diff --git a/docker/clickhouse-cluster.xml b/docker/clickhouse/cluster.xml
similarity index 84%
rename from docker/clickhouse-cluster.xml
rename to docker/clickhouse/cluster.xml
index f40148d5..a78382a1 100644
--- a/docker/clickhouse-cluster.xml
+++ b/docker/clickhouse/cluster.xml
@@ -30,7 +30,15 @@
   </remote_servers>
   <zookeeper>
     <node>
-      <host>zookeeper</host>
+      <host>clickhouse-keeper-1</host>
+      <port>2181</port>
+    </node>
+    <node>
+      <host>clickhouse-keeper-2</host>
+      <port>2181</port>
+    </node>
+    <node>
+      <host>clickhouse-keeper-3</host>
       <port>2181</port>
     </node>
   </zookeeper>
diff --git a/docker/clickhouse/keeper-cluster-1.xml b/docker/clickhouse/keeper-cluster-1.xml
new file mode 100644
index 00000000..2fc2854b
--- /dev/null
+++ b/docker/clickhouse/keeper-cluster-1.xml
@@ -0,0 +1,5 @@
+<clickhouse>
+  <keeper_server>
+    <server_id>1</server_id>
+  </keeper_server>
+</clickhouse>
diff --git a/docker/clickhouse/keeper-cluster-2.xml b/docker/clickhouse/keeper-cluster-2.xml
new file mode 100644
index 00000000..394e24be
--- /dev/null
+++ b/docker/clickhouse/keeper-cluster-2.xml
@@ -0,0 +1,5 @@
+<clickhouse>
+  <keeper_server>
+    <server_id>2</server_id>
+  </keeper_server>
+</clickhouse>
diff --git a/docker/clickhouse/keeper-cluster-3.xml b/docker/clickhouse/keeper-cluster-3.xml
new file mode 100644
index 00000000..0bdbcab6
--- /dev/null
+++ b/docker/clickhouse/keeper-cluster-3.xml
@@ -0,0 +1,5 @@
+<clickhouse>
+  <keeper_server>
+    <server_id>3</server_id>
+  </keeper_server>
+</clickhouse>
diff --git a/docker/clickhouse/keeper-cluster.xml b/docker/clickhouse/keeper-cluster.xml
new file mode 100644
index 00000000..cee3bbc0
--- /dev/null
+++ b/docker/clickhouse/keeper-cluster.xml
@@ -0,0 +1,21 @@
+<clickhouse>
+  <keeper_server>
+    <raft_configuration>
+      <server>
+        <id>1</id>
+        <hostname>clickhouse-keeper-1</hostname>
+        <port>9234</port>
+      </server>
+      <server>
+        <id>2</id>
+        <hostname>clickhouse-keeper-2</hostname>
+        <port>9234</port>
+      </server>
+      <server>
+        <id>3</id>
+        <hostname>clickhouse-keeper-3</hostname>
+        <port>9234</port>
+      </server>
+    </raft_configuration>
+  </keeper_server>
+</clickhouse>
diff --git a/docker/clickhouse/keeper.xml b/docker/clickhouse/keeper.xml
new file mode 100644
index 00000000..25d5940e
--- /dev/null
+++ b/docker/clickhouse/keeper.xml
@@ -0,0 +1,14 @@
+<clickhouse>
+  <listen_host>0.0.0.0</listen_host>
+  <keeper_server>
+    <tcp_port>2181</tcp_port>
+  </keeper_server>
+  <!-- Enable Prometheus endpoint -->
+  <prometheus>
+   <endpoint>/metrics</endpoint>
+   <port>9100</port>
+   <metrics>true</metrics>
+   <events>true</events>
+   <asynchronous_metrics>true</asynchronous_metrics>
+  </prometheus>
+</clickhouse>
diff --git a/docker/clickhouse/test-db.sql b/docker/clickhouse/test-db.sql
new file mode 100644
index 00000000..e68c2efe
--- /dev/null
+++ b/docker/clickhouse/test-db.sql
@@ -0,0 +1 @@
+CREATE DATABASE IF NOT EXISTS test;
diff --git a/docker/docker-compose-clickhouse-cluster.yml b/docker/docker-compose-clickhouse-cluster.yml
index 9d403a70..fbe5987f 100644
--- a/docker/docker-compose-clickhouse-cluster.yml
+++ b/docker/docker-compose-clickhouse-cluster.yml
@@ -1,5 +1,11 @@
 ---
 volumes:
+  akvorado-clickhouse-keeper-db-1:
+  akvorado-clickhouse-keeper-db-2:
+  akvorado-clickhouse-keeper-db-3:
+  akvorado-clickhouse-keeper-1:
+  akvorado-clickhouse-keeper-2:
+  akvorado-clickhouse-keeper-3:
   akvorado-clickhouse-2:
   akvorado-clickhouse-3:
   akvorado-clickhouse-4:
@@ -8,39 +14,82 @@ services:
   akvorado-orchestrator:
     environment:
       - AKVORADO_CFG_ORCHESTRATOR_CLICKHOUSE_CLUSTER=akvorado
+
+  clickhouse-keeper-1: &clickhouse-keeper
+    extends:
+      file: versions.yml
+      service: clickhouse-keeper
+    restart: unless-stopped
+    expose:
+      - 2181 # zookeeper
+      - 9100 # metrics
+    volumes:
+      - akvorado-clickhouse-keeper-db-1:/var/lib/clickhouse-keeper
+      - akvorado-clickhouse-keeper-1:/var/lib/clickhouse
+      - ./clickhouse/keeper.xml:/etc/clickhouse-keeper/keeper_config.d/keeper.xml
+      - ./clickhouse/keeper-cluster.xml:/etc/clickhouse-keeper/keeper_config.d/keeper-cluster.xml
+      - ./clickhouse/keeper-cluster.xml:/etc/clickhouse-keeper/keeper_config.d/keeper-cluster-1.xml
+    healthcheck:
+      interval: 20s
+      test: ["CMD", "wget", "-T", "1", "--spider", "--no-proxy", "http://127.0.0.1:9100/metrics"]
+  clickhouse-keeper-2:
+    <<: *clickhouse-keeper
+    volumes:
+      - akvorado-clickhouse-keeper-db-2:/var/lib/clickhouse-keeper
+      - akvorado-clickhouse-keeper-2:/var/lib/clickhouse
+      - ./clickhouse/keeper.xml:/etc/clickhouse-keeper/keeper_config.d/keeper.xml
+      - ./clickhouse/keeper-cluster.xml:/etc/clickhouse-keeper/keeper_config.d/keeper-cluster.xml
+      - ./clickhouse/keeper-cluster.xml:/etc/clickhouse-keeper/keeper_config.d/keeper-cluster-2.xml
+  clickhouse-keeper-3:
+    <<: *clickhouse-keeper
+    volumes:
+      - akvorado-clickhouse-keeper-db-3:/var/lib/clickhouse-keeper
+      - akvorado-clickhouse-keeper-3:/var/lib/clickhouse
+      - ./clickhouse/keeper.xml:/etc/clickhouse-keeper/keeper_config.d/keeper.xml
+      - ./clickhouse/keeper-cluster.xml:/etc/clickhouse-keeper/keeper_config.d/keeper-cluster.xml
+      - ./clickhouse/keeper-cluster.xml:/etc/clickhouse-keeper/keeper_config.d/keeper-cluster-3.xml
+
   clickhouse:
     depends_on:
-      - zookeeper
+      - clickhouse-keeper-1
+      - clickhouse-keeper-2
+      - clickhouse-keeper-3
     volumes:
-      - ./clickhouse-cluster.xml:/etc/clickhouse-server/config.d/cluster.xml
-      - ./clickhouse-cluster-1.xml:/etc/clickhouse-server/config.d/cluster-1.xml
+      - ./clickhouse/cluster.xml:/etc/clickhouse-server/config.d/cluster.xml
+      - ./clickhouse/cluster-1.xml:/etc/clickhouse-server/config.d/cluster-1.xml
   clickhouse-2:
     extends:
       file: docker-compose.yml
       service: clickhouse
     depends_on:
-      - zookeeper
+      - clickhouse-keeper-1
+      - clickhouse-keeper-2
+      - clickhouse-keeper-3
     volumes:
-      - ./clickhouse-cluster.xml:/etc/clickhouse-server/config.d/cluster.xml
-      - ./clickhouse-cluster-2.xml:/etc/clickhouse-server/config.d/cluster-2.xml
+      - ./clickhouse/cluster.xml:/etc/clickhouse-server/config.d/cluster.xml
+      - ./clickhouse/cluster-2.xml:/etc/clickhouse-server/config.d/cluster-2.xml
       - akvorado-clickhouse-2:/var/lib/clickhouse
   clickhouse-3:
     extends:
       file: docker-compose.yml
       service: clickhouse
     depends_on:
-      - zookeeper
+      - clickhouse-keeper-1
+      - clickhouse-keeper-2
+      - clickhouse-keeper-3
     volumes:
-      - ./clickhouse-cluster.xml:/etc/clickhouse-server/config.d/cluster.xml
-      - ./clickhouse-cluster-3.xml:/etc/clickhouse-server/config.d/cluster-3.xml
+      - ./clickhouse/cluster.xml:/etc/clickhouse-server/config.d/cluster.xml
+      - ./clickhouse/cluster-3.xml:/etc/clickhouse-server/config.d/cluster-3.xml
       - akvorado-clickhouse-3:/var/lib/clickhouse
   clickhouse-4:
     extends:
       file: docker-compose.yml
       service: clickhouse
     depends_on:
-      - zookeeper
+      - clickhouse-keeper-1
+      - clickhouse-keeper-2
+      - clickhouse-keeper-3
     volumes:
-      - ./clickhouse-cluster.xml:/etc/clickhouse-server/config.d/cluster.xml
-      - ./clickhouse-cluster-4.xml:/etc/clickhouse-server/config.d/cluster-4.xml
+      - ./clickhouse/cluster.xml:/etc/clickhouse-server/config.d/cluster.xml
+      - ./clickhouse/cluster-4.xml:/etc/clickhouse-server/config.d/cluster-4.xml
       - akvorado-clickhouse-4:/var/lib/clickhouse
diff --git a/docker/docker-compose-demo.yml b/docker/docker-compose-demo.yml
index adbe8eea..ea3ffe49 100644
--- a/docker/docker-compose-demo.yml
+++ b/docker/docker-compose-demo.yml
@@ -9,6 +9,8 @@ services:
     depends_on:
       akvorado-inlet:
         condition: service_healthy
+      akvorado-outlet:
+        condition: service_healthy
   akvorado-exporter1:
     <<: *exporter
     command: demo-exporter http://akvorado-orchestrator:8080#1
diff --git a/docker/docker-compose-dev.yml b/docker/docker-compose-dev.yml
index 932ea3f0..a3cf815f 100644
--- a/docker/docker-compose-dev.yml
+++ b/docker/docker-compose-dev.yml
@@ -1,12 +1,6 @@
 ---
+name: akvorado-dev
 services:
-  zookeeper:
-    extends:
-      file: versions.yml
-      service: zookeeper
-    environment:
-      - ALLOW_ANONYMOUS_LOGIN=yes
-
   mock-oauth2-server:
     extends:
       file: versions.yml
@@ -21,50 +15,63 @@ services:
       file: versions.yml
       service: kafka
     environment:
-      KAFKA_CFG_BROKER_ID: 1
-      KAFKA_CFG_ZOOKEEPER_CONNECT: zookeeper:2181
+      # KRaft settings
+      KAFKA_NODE_ID: 1
+      KAFKA_PROCESS_ROLES: controller,broker
+      KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka:9096
+      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 500
       # We have two sets of listeners: INTERNAL that is used from inside the docker
       # compose network and listens on "kafka" and EXTERNAL that is mapped to
       # the host network and listens on "localhost".
       #
       # Then, in each set, we have a plain text one and an OAuth-enabled one.
-      KAFKA_CFG_LISTENERS: >-
+      KAFKA_LISTENERS: >-
         INTERNAL://:9092,
         OINTERNAL://:9093,
         EXTERNAL://:9094,
-        OEXTERNAL://:9095
-      KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: >-
+        OEXTERNAL://:9095,
+        CONTROLLER://:9096
+      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: >-
         INTERNAL:PLAINTEXT,
         OINTERNAL:SASL_PLAINTEXT,
         EXTERNAL:PLAINTEXT,
-        OEXTERNAL:SASL_PLAINTEXT
-      KAFKA_CFG_ADVERTISED_LISTENERS: >-
+        OEXTERNAL:SASL_PLAINTEXT,
+        CONTROLLER:PLAINTEXT
+      KAFKA_ADVERTISED_LISTENERS: >-
         INTERNAL://kafka:9092,
         OINTERNAL://kafka:9093,
         EXTERNAL://localhost:9092,
         OEXTERNAL://localhost:9093
-      KAFKA_CFG_INTER_BROKER_LISTENER_NAME: INTERNAL
+      KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
+      KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
       # OAuth2 configuration
-      KAFKA_CFG_LISTENER_NAME_OEXTERNAL_SASL_ENABLED_MECHANISMS: OAUTHBEARER
-      KAFKA_CFG_LISTENER_NAME_OEXTERNAL_SASL_OAUTHBEARER_JWKS_ENDPOINT_URL: http://mock-oauth2-server:8080/default/jwks
-      KAFKA_CFG_LISTENER_NAME_OEXTERNAL_SASL_OAUTHBEARER_EXPECTED_AUDIENCE: default
-      KAFKA_CFG_LISTENER_NAME_OEXTERNAL_OAUTHBEARER_SASL_JAAS_CONFIG: >-
+      KAFKA_LISTENER_NAME_OEXTERNAL_SASL_ENABLED_MECHANISMS: OAUTHBEARER
+      KAFKA_LISTENER_NAME_OEXTERNAL_SASL_OAUTHBEARER_JWKS_ENDPOINT_URL: http://mock-oauth2-server:8080/default/jwks
+      KAFKA_LISTENER_NAME_OEXTERNAL_SASL_OAUTHBEARER_EXPECTED_AUDIENCE: default
+      KAFKA_LISTENER_NAME_OEXTERNAL_OAUTHBEARER_SASL_JAAS_CONFIG: >-
         org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required
         clientId=kafka-client
         clientSecret=kafka-client-secret
         unsecuredLoginStringClaim_sub="sub";
-      KAFKA_CFG_LISTENER_NAME_OEXTERNAL_OAUTHBEARER_SASL_SERVER_CALLBACK_HANDLER_CLASS: org.apache.kafka.common.security.oauthbearer.secured.OAuthBearerValidatorCallbackHandler
-      KAFKA_CFG_LISTENER_NAME_OINTERNAL_SASL_ENABLED_MECHANISMS: OAUTHBEARER
-      KAFKA_CFG_LISTENER_NAME_OINTERNAL_SASL_OAUTHBEARER_JWKS_ENDPOINT_URL: http://mock-oauth2-server:8080/default/jwks
-      KAFKA_CFG_LISTENER_NAME_OINTERNAL_SASL_OAUTHBEARER_EXPECTED_AUDIENCE: default
-      KAFKA_CFG_LISTENER_NAME_OINTERNAL_OAUTHBEARER_SASL_JAAS_CONFIG: >-
+      KAFKA_LISTENER_NAME_OEXTERNAL_OAUTHBEARER_SASL_SERVER_CALLBACK_HANDLER_CLASS: org.apache.kafka.common.security.oauthbearer.OAuthBearerValidatorCallbackHandler
+      KAFKA_LISTENER_NAME_OINTERNAL_SASL_ENABLED_MECHANISMS: OAUTHBEARER
+      KAFKA_LISTENER_NAME_OINTERNAL_SASL_OAUTHBEARER_JWKS_ENDPOINT_URL: http://mock-oauth2-server:8080/default/jwks
+      KAFKA_LISTENER_NAME_OINTERNAL_SASL_OAUTHBEARER_EXPECTED_AUDIENCE: default
+      KAFKA_LISTENER_NAME_OINTERNAL_OAUTHBEARER_SASL_JAAS_CONFIG: >-
         org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required
         clientId=kafka-client
         clientSecret=kafka-client-secret
         unsecuredLoginStringClaim_sub="sub";
-      KAFKA_CFG_LISTENER_NAME_OINTERNAL_OAUTHBEARER_SASL_SERVER_CALLBACK_HANDLER_CLASS: org.apache.kafka.common.security.oauthbearer.secured.OAuthBearerValidatorCallbackHandler
+      KAFKA_LISTENER_NAME_OINTERNAL_OAUTHBEARER_SASL_SERVER_CALLBACK_HANDLER_CLASS: org.apache.kafka.common.security.oauthbearer.OAuthBearerValidatorCallbackHandler
+      KAFKA_OPTS: >-
+        -Dorg.apache.kafka.sasl.oauthbearer.allowed.urls=http://mock-oauth2-server:8080/default/jwks
+      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
+      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
+      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
+      KAFKA_SHARE_COORDINATOR_STATE_TOPIC_REPLICATION_FACTOR: 1
+      KAFKA_SHARE_COORDINATOR_STATE_TOPIC_MIN_ISR: 1
+      KAFKA_LOG_DIRS: /var/lib/kafka/data
     depends_on:
-      - zookeeper
       - mock-oauth2-server
     ports:
       - 127.0.0.1:9092:9094/tcp
@@ -115,40 +122,51 @@ services:
       file: versions.yml
       service: clickhouse
     depends_on:
-      - zookeeper
+      - clickhouse-keeper-1
     environment:
       - CLICKHOUSE_SKIP_USER_SETUP=1
+      - CLICKHOUSE_ALWAYS_RUN_INITDB_SCRIPTS=1
     cap_add:
       - SYS_NICE
     volumes:
-      - ./clickhouse-cluster.xml:/etc/clickhouse-server/config.d/cluster.xml
-      - ./clickhouse-cluster-1.xml:/etc/clickhouse-server/config.d/cluster-1.xml
+      - ./clickhouse/cluster.xml:/etc/clickhouse-server/config.d/cluster.xml
+      - ./clickhouse/cluster-1.xml:/etc/clickhouse-server/config.d/cluster-1.xml
+      - ./clickhouse/test-db.sql:/docker-entrypoint-initdb.d/test-db.sql
     ports:
       - 127.0.0.1:8123:8123/tcp
       - 127.0.0.1:9000:9000/tcp
   clickhouse-2:
     <<: *clickhouse
     volumes:
-      - ./clickhouse-cluster.xml:/etc/clickhouse-server/config.d/cluster.xml
-      - ./clickhouse-cluster-2.xml:/etc/clickhouse-server/config.d/cluster-2.xml
+      - ./clickhouse/cluster.xml:/etc/clickhouse-server/config.d/cluster.xml
+      - ./clickhouse/cluster-2.xml:/etc/clickhouse-server/config.d/cluster-2.xml
     ports:
       - 127.0.0.1:9001:9000/tcp
   clickhouse-3:
     <<: *clickhouse
     volumes:
-      - ./clickhouse-cluster.xml:/etc/clickhouse-server/config.d/cluster.xml
-      - ./clickhouse-cluster-3.xml:/etc/clickhouse-server/config.d/cluster-3.xml
+      - ./clickhouse/cluster.xml:/etc/clickhouse-server/config.d/cluster.xml
+      - ./clickhouse/cluster-3.xml:/etc/clickhouse-server/config.d/cluster-3.xml
     ports:
       - 127.0.0.1:9002:9000/tcp
   clickhouse-4:
     <<: *clickhouse
     volumes:
-      - ./clickhouse-cluster.xml:/etc/clickhouse-server/config.d/cluster.xml
-      - ./clickhouse-cluster-4.xml:/etc/clickhouse-server/config.d/cluster-4.xml
+      - ./clickhouse/cluster.xml:/etc/clickhouse-server/config.d/cluster.xml
+      - ./clickhouse/cluster-4.xml:/etc/clickhouse-server/config.d/cluster-4.xml
     ports:
       - 127.0.0.1:9003:9000/tcp
+  clickhouse-keeper-1:
+    extends:
+      file: versions.yml
+      service: clickhouse-keeper
+    volumes:
+      - ./clickhouse/keeper.xml:/etc/clickhouse-keeper/keeper_config.d/keeper.xml
 
   srlinux:
+    # The SR Linux container does not like to be restarted. If you get:
+    #  Error: Peer netns reference is invalid.
+    # Be sure to put it down before starting it.
     extends:
       file: versions.yml
       service: srlinux
diff --git a/docker/docker-compose-monitoring.yml b/docker/docker-compose-prometheus.yml
similarity index 65%
rename from docker/docker-compose-monitoring.yml
rename to docker/docker-compose-prometheus.yml
index 4a96a55a..684c2d0b 100644
--- a/docker/docker-compose-monitoring.yml
+++ b/docker/docker-compose-prometheus.yml
@@ -1,6 +1,8 @@
 ---
 volumes:
   akvorado-prometheus:
+  akvorado-jmx-agent:
+
 services:
   prometheus:
     extends:
@@ -39,31 +41,23 @@ services:
       - --path.sysfs=/host/sys
       - --no-collector.filesystem
 
-  kafka-jmx-exporter:
-    extends:
-      file: versions.yml
-      service: kafka-jmx-exporter
-    restart: unless-stopped
-    depends_on:
-      - kafka
+  # Kafka monitoring
+  jmx-exporter-downloader:
+    image: alpine:latest
+    command: /usr/local/bin/jmx-exporter-download.sh
     volumes:
-      - ./kafka-jmx-exporter-config.yaml:/etc/jmx-kafka/config.yaml:ro
-    command:
-      - "5556"
-      - /etc/jmx-kafka/config.yaml
-
-  kafka-ui:
-    extends:
-      file: versions.yml
-      service: kafka-ui
-    restart: unless-stopped
+      - akvorado-jmx-agent:/opt/jmx-exporter
+      - ./jmx-exporter-download.sh:/usr/local/bin/jmx-exporter-download.sh
+  kafka:
+    expose:
+      - 9092 # kafka
+      - 9100 # metrics
     depends_on:
-      - kafka
+      jmx-exporter-downloader:
+        condition: service_completed_successfully
     environment:
-      - KAFKA_CLUSTERS_0_NAME=local
-      - KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=kafka:9092
-      - SERVER_SERVLET_CONTEXT_PATH=/kafka-ui
-    labels:
-      - traefik.enable=true
-      - traefik.http.routers.kafka-ui.entrypoints=private
-      - traefik.http.routers.kafka-ui.rule=PathPrefix(`/kafka-ui`)
+      KAFKA_OPTS: >-
+        -javaagent:/opt/jmx-exporter/jmx_prometheus_javaagent.jar=9100:/etc/kafka/jmx-exporter.yaml
+    volumes:
+      - akvorado-jmx-agent:/opt/jmx-exporter
+      - ./kafka-jmx-exporter-config.yaml:/etc/kafka/jmx-exporter.yaml
diff --git a/docker/docker-compose.yml b/docker/docker-compose.yml
index 72c8636e..7895dcd0 100644
--- a/docker/docker-compose.yml
+++ b/docker/docker-compose.yml
@@ -1,6 +1,5 @@
 ---
 volumes:
-  akvorado-zookeeper:
   akvorado-kafka:
   akvorado-geoip:
   akvorado-clickhouse:
@@ -8,48 +7,53 @@ volumes:
   akvorado-console-db:
 
 services:
-  zookeeper:
+  kafka:
     extends:
       file: versions.yml
-      service: zookeeper
+      service: kafka
     environment:
-      - ALLOW_ANONYMOUS_LOGIN=yes
-      - ZOO_ENABLE_PROMETHEUS_METRICS=yes
-      - ZOO_4LW_COMMANDS_WHITELIST=srvr,mntr,ruok
+      # KRaft settings
+      KAFKA_NODE_ID: 1
+      KAFKA_PROCESS_ROLES: controller,broker
+      KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka:9093
+      # Listeners
+      KAFKA_LISTENERS: CLIENT://:9092,CONTROLLER://:9093
+      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CLIENT:PLAINTEXT,CONTROLLER:PLAINTEXT
+      KAFKA_ADVERTISED_LISTENERS: CLIENT://kafka:9092
+      KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
+      KAFKA_INTER_BROKER_LISTENER_NAME: CLIENT
+      # Misc
+      KAFKA_DELETE_TOPIC_ENABLE: "true"
+      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
+      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
+      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
+      KAFKA_SHARE_COORDINATOR_STATE_TOPIC_REPLICATION_FACTOR: 1
+      KAFKA_SHARE_COORDINATOR_STATE_TOPIC_MIN_ISR: 1
+      KAFKA_LOG_DIRS: /var/lib/kafka/data
+
     restart: unless-stopped
-    expose:
-      - 2181/tcp
-      - 2888/tcp
-      - 3888/tcp
-      - 7000/tcp                # Prometheus metrics
     volumes:
-      - akvorado-zookeeper:/bitnami/zookeeper
+      - akvorado-kafka:/var/lib/kafka
     healthcheck:
       interval: 20s
-      test: ["CMD-SHELL", "echo ruok | nc -w3 127.0.0.1 2181 | grep -xFq imok"]
-
-  kafka:
+      test: ["CMD",
+             "env", "-u", "KAFKA_OPTS",
+             "/opt/kafka/bin/kafka-topics.sh", "--list", "--bootstrap-server", "kafka:9092"]
+  kafka-ui:
     extends:
       file: versions.yml
-      service: kafka
-    environment:
-      - KAFKA_ZOOKEEPER_PROTOCOL=PLAINTEXT
-      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
-      - KAFKA_CFG_DELETE_TOPIC_ENABLE=true
-      - KAFKA_CFG_LISTENERS=PLAINTEXT://:9092
-      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT
-      - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092
-      - KAFKA_CFG_INTER_BROKER_LISTENER_NAME=PLAINTEXT
-      - JMX_PORT=5555
+      service: kafka-ui
     restart: unless-stopped
     depends_on:
-      zookeeper:
-        condition: service_healthy
-    volumes:
-      - akvorado-kafka:/bitnami/kafka
-    healthcheck:
-      interval: 20s
-      test: ["CMD", "env", "-u", "JMX_PORT", "kafka-topics.sh", "--list", "--bootstrap-server", "kafka:9092"]
+      - kafka
+    environment:
+      - KAFKA_CLUSTERS_0_NAME=local
+      - KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=kafka:9092
+      - SERVER_SERVLET_CONTEXT_PATH=/kafka-ui
+    labels:
+      - traefik.enable=true
+      - traefik.http.routers.kafka-ui.entrypoints=private
+      - traefik.http.routers.kafka-ui.rule=PathPrefix(`/kafka-ui`)
 
   redis:
     extends:
@@ -58,7 +62,9 @@ services:
     restart: unless-stopped
     healthcheck:
       interval: 20s
-      test: ["CMD-SHELL", "timeout 3 redis-cli ping | grep -q PONG"]
+      test:
+        - CMD-SHELL
+        - "timeout 3 redis-cli ping | grep -q PONG"
 
   akvorado-orchestrator:
     extends:
@@ -124,7 +130,6 @@ services:
     ports:
       - 2055:2055/udp
       - 6343:6343/udp
-      - 10179:10179/tcp
     restart: unless-stopped
     depends_on:
       akvorado-orchestrator:
@@ -134,8 +139,6 @@ services:
     command: inlet http://akvorado-orchestrator:8080
     volumes:
       - akvorado-run:/run/akvorado
-    environment:
-      - AKVORADO_CFG_INLET_METADATA_CACHEPERSISTFILE=/run/akvorado/metadata.cache
     labels:
       - traefik.enable=true
       # Disable access logging of /api/v0/inlet/metrics
@@ -148,6 +151,36 @@ services:
       - traefik.http.routers.akvorado-inlet.rule=PathPrefix(`/api/v0/inlet`)
       - traefik.http.services.akvorado-inlet.loadbalancer.server.port=8080
       - akvorado.conntrack.fix=true
+  akvorado-outlet:
+    extends:
+      file: versions.yml
+      service: akvorado
+    ports:
+      - 10179:10179/tcp
+    restart: unless-stopped
+    depends_on:
+      akvorado-orchestrator:
+        condition: service_healthy
+      kafka:
+        condition: service_healthy
+      clickhouse:
+        condition: service_healthy
+    command: outlet http://akvorado-orchestrator:8080
+    volumes:
+      - akvorado-run:/run/akvorado
+    environment:
+      - AKVORADO_CFG_OUTLET_METADATA_CACHEPERSISTFILE=/run/akvorado/metadata.cache
+    labels:
+      - traefik.enable=true
+      # Disable access logging of /api/v0/outlet/metrics
+      - traefik.http.routers.akvorado-outlet-metrics.entrypoints=private
+      - traefik.http.routers.akvorado-outlet-metrics.rule=PathPrefix(`/api/v0/outlet/metrics`)
+      - traefik.http.routers.akvorado-outlet-metrics.service=akvorado-outlet
+      - traefik.http.routers.akvorado-outlet-metrics.observability.accesslogs=false
+      # Everything else is exposed to private entrypoing in /api/v0/outlet
+      - traefik.http.routers.akvorado-outlet.entrypoints=private
+      - traefik.http.routers.akvorado-outlet.rule=PathPrefix(`/api/v0/outlet`)
+      - traefik.http.services.akvorado-outlet.loadbalancer.server.port=8080
   akvorado-conntrack-fixer:
     extends:
       file: versions.yml
@@ -167,9 +200,8 @@ services:
       file: versions.yml
       service: clickhouse
     volumes:
-      - ../orchestrator/clickhouse/data/docker-entrypoint.sh:/entrypoint-override.sh:ro
       - akvorado-clickhouse:/var/lib/clickhouse
-    entrypoint: /entrypoint-override.sh
+      - ./clickhouse/akvorado.xml:/etc/clickhouse-server/config.d/akvorado.xml
     environment:
       - CLICKHOUSE_INIT_TIMEOUT=60
       - CLICKHOUSE_SKIP_USER_SETUP=1
diff --git a/docker/jmx-exporter-download.sh b/docker/jmx-exporter-download.sh
new file mode 100755
index 00000000..595f400d
--- /dev/null
+++ b/docker/jmx-exporter-download.sh
@@ -0,0 +1,17 @@
+#!/bin/sh
+
+set -e
+
+VERSION=1.3.0
+NAME=jmx_prometheus_javaagent-${VERSION}.jar
+URL=https://github.com/prometheus/jmx_exporter/releases/download/${VERSION}/${NAME}
+
+cd /opt/jmx-exporter
+
+# Check if target version already exist
+[ ! -s jmx_prometheus_javaagent-${VERSION}.jar ] || exit 0
+
+# Retrieve it
+apk add --no-cache curl
+curl --retry 10 --retry-connrefused --remove-on-error --remote-name --fail --silent --location $URL
+ln -vsf ${NAME} jmx_prometheus_javaagent.jar
diff --git a/docker/kafka-jmx-exporter-config.yaml b/docker/kafka-jmx-exporter-config.yaml
index 265fb285..d4e66c0a 100644
--- a/docker/kafka-jmx-exporter-config.yaml
+++ b/docker/kafka-jmx-exporter-config.yaml
@@ -1,61 +1,139 @@
 ---
-jmxUrl: service:jmx:rmi:///jndi/rmi://kafka:5555/jmxrmi
 lowercaseOutputName: true
 lowercaseOutputLabelNames: true
-ssl: false
 
-# This is mostly stolen from Bitnami Helm chart for Kafka.
-# https://github.com/bitnami/charts/blob/main/bitnami/kafka/templates/metrics/jmx-configmap.yaml
-whitelistObjectNames:
-  - kafka.controller:*
-  - kafka.server:*
-  - java.lang:*
-  - kafka.network:*
-  - kafka.log:*
 rules:
-  - pattern: kafka.controller<type=(ControllerChannelManager), name=(QueueSize), broker-id=(\d+)><>(Value)
-    name: kafka_controller_$1_$2_$4
-    labels:
-      broker_id: "$3"
-  - pattern: kafka.controller<type=(ControllerChannelManager), name=(TotalQueueSize)><>(Value)
-    name: kafka_controller_$1_$2_$3
-  - pattern: kafka.controller<type=(KafkaController), name=(.+)><>(Value)
-    name: kafka_controller_$1_$2_$3
-  - pattern: kafka.controller<type=(ControllerStats), name=(.+)><>(Count)
-    name: kafka_controller_$1_$2_$3
-  - pattern: kafka.network<type=(Processor), name=(IdlePercent), networkProcessor=(.+)><>(Value)
-    name: kafka_network_$1_$2_$4
-    labels:
-      network_processor: $3
-  - pattern: kafka.network<type=(RequestMetrics), name=(.+), request=(.+)><>(Count|Value)
-    name: kafka_network_$1_$2_$4
-    labels:
-      request: $3
-  - pattern: kafka.network<type=(SocketServer), name=(.+)><>(Count|Value)
-    name: kafka_network_$1_$2_$3
-  - pattern: kafka.network<type=(RequestChannel), name=(.+)><>(Count|Value)
-    name: kafka_network_$1_$2_$3
-  - pattern: kafka.server<type=(.+), name=(.+), topic=(.+)><>(Count|OneMinuteRate)
-    name: kafka_server_$1_$2_$4
-    labels:
-      topic: $3
-  - pattern: kafka.server<type=(ReplicaFetcherManager), name=(.+), clientId=(.+)><>(Value)
-    name: kafka_server_$1_$2_$4
-    labels:
-      client_id: "$3"
-  - pattern: kafka.server<type=(DelayedOperationPurgatory), name=(.+), delayedOperation=(.+)><>(Value)
-    name: kafka_server_$1_$2_$3_$4
-  - pattern: kafka.server<type=(.+), name=(.+)><>(Count|Value|OneMinuteRate)
-    name: kafka_server_$1_total_$2_$3
-  - pattern: kafka.server<type=(.+)><>(queue-size)
-    name: kafka_server_$1_$2
-  - pattern: java.lang<type=(.+), name=(.+)><(.+)>(\w+)
-    name: java_lang_$1_$4_$3_$2
-  - pattern: java.lang<type=(.+), name=(.+)><>(\w+)
-    name: java_lang_$1_$3_$2
-  - pattern: java.lang<type=(.*)>
-  - pattern: kafka.log<type=(.+), name=(.+), topic=(.+), partition=(.+)><>Value
-    name: kafka_log_$1_$2
-    labels:
-      topic: $3
-      partition: $4
+# Special cases and very specific rules
+- pattern : kafka.server<type=(.+), name=(.+), clientId=(.+), topic=(.+), partition=(.*)><>Value
+  name: kafka_server_$1_$2
+  type: GAUGE
+  labels:
+    clientId: "$3"
+    topic: "$4"
+    partition: "$5"
+- pattern : kafka.server<type=(.+), name=(.+), clientId=(.+), brokerHost=(.+), brokerPort=(.+)><>Value
+  name: kafka_server_$1_$2
+  type: GAUGE
+  labels:
+    clientId: "$3"
+    broker: "$4:$5"
+- pattern : kafka.coordinator.(\w+)<type=(.+), name=(.+)><>Value
+  name: kafka_coordinator_$1_$2_$3
+  type: GAUGE
+# Kraft current state info metric rule
+- pattern: "kafka.server<type=raft-metrics><>current-state: ([a-z]+)"
+  name: kafka_server_raft_metrics_current_state_info
+  type: GAUGE
+  value: 1
+  labels:
+    "state": "$1"
+# Kraft specific rules for raft-metrics, raft-channel-metrics, broker-metadata-metrics
+- pattern: kafka.server<type=(.+)><>([a-z-]+)-total
+  name: kafka_server_$1_$2_total
+  type: COUNTER
+- pattern: kafka.server<type=(.+)><>([a-z-]+)
+  name: kafka_server_$1_$2
+  type: GAUGE
+
+# Generic per-second counters with 0-2 key/value pairs
+- pattern: kafka.(\w+)<type=(.+), name=(.+)PerSec\w*, (.+)=(.+), (.+)=(.+)><>Count
+  name: kafka_$1_$2_$3_total
+  type: COUNTER
+  labels:
+    "$4": "$5"
+    "$6": "$7"
+- pattern: kafka.(\w+)<type=(.+), name=(.+)PerSec\w*, (.+)=(.+)><>Count
+  name: kafka_$1_$2_$3_total
+  type: COUNTER
+  labels:
+    "$4": "$5"
+- pattern: kafka.(\w+)<type=(.+), name=(.+)PerSec\w*><>Count
+  name: kafka_$1_$2_$3_total
+  type: COUNTER
+
+# Quota specific rules
+- pattern: kafka.server<type=(.+), user=(.+), client-id=(.+)><>([a-z-]+)
+  name: kafka_server_quota_$4
+  type: GAUGE
+  labels:
+    resource: "$1"
+    user: "$2"
+    clientId: "$3"
+- pattern: kafka.server<type=(.+), client-id=(.+)><>([a-z-]+)
+  name: kafka_server_quota_$3
+  type: GAUGE
+  labels:
+    resource: "$1"
+    clientId: "$2"
+- pattern: kafka.server<type=(.+), user=(.+)><>([a-z-]+)
+  name: kafka_server_quota_$3
+  type: GAUGE
+  labels:
+    resource: "$1"
+    user: "$2"
+
+# Generic gauges with 0-2 key/value pairs
+- pattern: kafka.(\w+)<type=(.+), name=(.+), (.+)=(.+), (.+)=(.+)><>Value
+  name: kafka_$1_$2_$3
+  type: GAUGE
+  labels:
+    "$4": "$5"
+    "$6": "$7"
+- pattern: kafka.(\w+)<type=(.+), name=(.+), (.+)=(.+)><>Value
+  name: kafka_$1_$2_$3
+  type: GAUGE
+  labels:
+    "$4": "$5"
+- pattern: kafka.(\w+)<type=(.+), name=(.+)><>Value
+  name: kafka_$1_$2_$3
+  type: GAUGE
+
+# Emulate Prometheus 'Summary' metrics for the exported 'Histogram's.
+#
+# Note that these are missing the '_sum' metric!
+- pattern: kafka.(\w+)<type=(.+), name=(.+), (.+)=(.+), (.+)=(.+)><>Count
+  name: kafka_$1_$2_$3_count
+  type: COUNTER
+  labels:
+    "$4": "$5"
+    "$6": "$7"
+- pattern: kafka.(\w+)<type=(.+), name=(.+), (.+)=(.*), (.+)=(.+)><>(\d+)thPercentile
+  name: kafka_$1_$2_$3
+  type: GAUGE
+  labels:
+    "$4": "$5"
+    "$6": "$7"
+    quantile: "0.$8"
+- pattern: kafka.(\w+)<type=(.+), name=(.+), (.+)=(.+)><>Count
+  name: kafka_$1_$2_$3_count
+  type: COUNTER
+  labels:
+    "$4": "$5"
+- pattern: kafka.(\w+)<type=(.+), name=(.+), (.+)=(.*)><>(\d+)thPercentile
+  name: kafka_$1_$2_$3
+  type: GAUGE
+  labels:
+    "$4": "$5"
+    quantile: "0.$6"
+- pattern: kafka.(\w+)<type=(.+), name=(.+)><>Count
+  name: kafka_$1_$2_$3_count
+  type: COUNTER
+- pattern: kafka.(\w+)<type=(.+), name=(.+)><>(\d+)thPercentile
+  name: kafka_$1_$2_$3
+  type: GAUGE
+  labels:
+    quantile: "0.$4"
+
+# Generic gauges for MeanRate Percent
+# Ex) kafka.server<type=KafkaRequestHandlerPool, name=RequestHandlerAvgIdlePercent><>MeanRate
+- pattern: kafka.(\w+)<type=(.+), name=(.+)Percent\w*><>MeanRate
+  name: kafka_$1_$2_$3_percent
+  type: GAUGE
+- pattern: kafka.(\w+)<type=(.+), name=(.+)Percent\w*><>Value
+  name: kafka_$1_$2_$3_percent
+  type: GAUGE
+- pattern: kafka.(\w+)<type=(.+), name=(.+)Percent\w*, (.+)=(.+)><>Value
+  name: kafka_$1_$2_$3_percent
+  type: GAUGE
+  labels:
+    "$4": "$5"
diff --git a/docker/prometheus.yml b/docker/prometheus.yml
index e25fa6f8..75c46b00 100644
--- a/docker/prometheus.yml
+++ b/docker/prometheus.yml
@@ -18,7 +18,7 @@ scrape_configs:
             values:
               - com.docker.compose.project=akvorado
               - com.docker.compose.service=grafana
-  - job_name: zookeeper
+  - job_name: clickhouse
     docker_sd_configs:
       - host: unix:///var/run/docker.sock
         refresh_interval: 30s
@@ -26,10 +26,27 @@ scrape_configs:
           - name: label
             values:
               - com.docker.compose.project=akvorado
-              - com.docker.compose.service=zookeeper
     relabel_configs:
+      - source_labels: [__meta_docker_container_label_com_docker_compose_service]
+        regex: (clickhouse|clickhouse-\\d+)
+        action: keep
       - source_labels: [__meta_docker_port_private]
-        regex: 7000
+        regex: 8123
+        action: keep
+  - job_name: clickhouse-keeper
+    docker_sd_configs:
+      - host: unix:///var/run/docker.sock
+        refresh_interval: 30s
+        filters:
+          - name: label
+            values:
+              - com.docker.compose.project=akvorado
+    relabel_configs:
+      - source_labels: [__meta_docker_container_label_com_docker_compose_service]
+        regex: clickhouse-keeper-\\d+
+        action: keep
+      - source_labels: [__meta_docker_port_private]
+        regex: 9100
         action: keep
   - job_name: kafka
     docker_sd_configs:
@@ -39,7 +56,11 @@ scrape_configs:
           - name: label
             values:
               - com.docker.compose.project=akvorado
-              - com.docker.compose.service=kafka-jmx-exporter
+              - com.docker.compose.service=kafka
+    relabel_configs:
+      - source_labels: [__meta_docker_port_private]
+        regex: 9100
+        action: keep
   - job_name: node-exporter
     docker_sd_configs:
       - host: unix:///var/run/docker.sock
@@ -60,7 +81,7 @@ scrape_configs:
               - com.docker.compose.project=akvorado
     relabel_configs:
       - source_labels: [__meta_docker_container_label_com_docker_compose_service]
-        regex: akvorado-(inlet|orchestrator|console)
+        regex: akvorado-(inlet|outlet|orchestrator|console)
         action: keep
       - source_labels: [__meta_docker_port_private]
         regex: 8080
diff --git a/docker/versions.yml b/docker/versions.yml
index 11aa7b88..28d19b6f 100644
--- a/docker/versions.yml
+++ b/docker/versions.yml
@@ -1,14 +1,14 @@
 ---
 services:
   # main services
-  zookeeper:
-    image: bitnami/zookeeper:3.8 # \d+\.\d+
   kafka:
-    image: bitnami/kafka:3.8 # \d+\.\d+
+    image: apache/kafka:4.0.0 # \d+\.\d+\.\d+
   redis:
     image: valkey/valkey:7.2 # \d+\.\d+
   clickhouse:
-    image: clickhouse/clickhouse-server:24.8 # \d+\.[38]
+    image: clickhouse/clickhouse-server:25.4 # \d+\.[38]
+  clickhouse-keeper:
+    image: clickhouse/clickhouse-keeper:25.4 # \d+\.[38]
   traefik:
     image: traefik:v3.4 # v\d+\.\d+
   maxmind-geoipupdate:
@@ -28,8 +28,6 @@ services:
     image: prom/prometheus:v2.47.2 # v\d+\.\d+\.\d+
   node-exporter:
     image: prom/node-exporter:v1.7.0 # v\d+\.\d+\.\d+
-  kafka-jmx-exporter:
-    image: bitnami/jmx-exporter:0
   kafka-ui:
     image: provectuslabs/kafka-ui:v0.7.2 # v\d+\.\d+\.\d+
 
diff --git a/flake.nix b/flake.nix
index eefbb790..2c30407e 100644
--- a/flake.nix
+++ b/flake.nix
@@ -47,6 +47,7 @@
           name = "akvorado";
           src = ./.;
           vendorHash = builtins.readFile ./nix/vendorHash.txt;
+          proxyVendor = true;   # generated code may contain additional dependencies
           buildPhase = ''
             cp -r ${frontend}/node_modules console/frontend/node_modules
             cp -r ${frontend}/data console/data/frontend
@@ -55,6 +56,7 @@
             find . -print0 | xargs -0 touch -d @0
 
             make all \
+              BUF=${pkgs.buf}/bin/buf \
               ASNS_URL=${asn2org}/asns.csv \
               SERVICES_URL=${ianaServiceNames}
           '';
diff --git a/go.mod b/go.mod
index 0b860f50..52edba15 100644
--- a/go.mod
+++ b/go.mod
@@ -3,8 +3,8 @@ module akvorado
 go 1.24
 
 require (
+	github.com/ClickHouse/ch-go v0.66.1
 	github.com/ClickHouse/clickhouse-go/v2 v2.37.2
-	github.com/IBM/sarama v1.45.2
 	github.com/alecthomas/chroma v0.10.0
 	github.com/benbjohnson/clock v1.3.5
 	github.com/bio-routing/bio-rd v0.1.10
@@ -26,7 +26,6 @@ require (
 	github.com/grpc-ecosystem/go-grpc-middleware/providers/prometheus v1.0.1
 	github.com/hashicorp/go-version v1.7.0
 	github.com/itchyny/gojq v0.12.17
-	github.com/jhump/protoreflect v1.17.0
 	github.com/kentik/patricia v1.2.1
 	github.com/kylelemons/godebug v1.1.0
 	github.com/mattn/go-isatty v0.0.20
@@ -36,21 +35,24 @@ require (
 	github.com/opencontainers/image-spec v1.1.1
 	github.com/oschwald/maxminddb-golang v1.13.1
 	github.com/osrg/gobgp/v3 v3.37.0
+	github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10
 	github.com/prometheus/client_golang v1.22.0
-	github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475
 	github.com/rs/zerolog v1.34.0
 	github.com/scrapli/scrapligo v1.3.3
 	github.com/slayercat/GoSNMPServer v0.5.2
 	github.com/spf13/cobra v1.9.1
 	github.com/ti-mo/conntrack v0.5.2
-	github.com/xdg-go/scram v1.1.2
+	github.com/twmb/franz-go v1.19.5
+	github.com/twmb/franz-go/pkg/kadm v1.16.0
+	github.com/twmb/franz-go/pkg/kfake v0.0.0-20250711145744-a849b8be17b7
+	github.com/twmb/franz-go/pkg/kmsg v1.11.2
+	github.com/twmb/franz-go/plugin/kprom v1.2.1
 	github.com/yuin/goldmark v1.7.12
 	github.com/yuin/goldmark-highlighting v0.0.0-20220208100518-594be1970594
 	go.uber.org/mock v0.5.2
 	golang.org/x/oauth2 v0.30.0
 	golang.org/x/sys v0.33.0
 	golang.org/x/text v0.26.0
-	golang.org/x/time v0.12.0
 	google.golang.org/grpc v1.72.2
 	google.golang.org/protobuf v1.36.6
 	gopkg.in/tomb.v2 v2.0.0-20161208151619-d5d1b5820637
@@ -66,7 +68,6 @@ require (
 	github.com/AlekSi/gocov-xml v1.1.0 // indirect
 	github.com/AlekSi/pointer v1.2.0 // indirect
 	github.com/BurntSushi/toml v1.4.0 // indirect
-	github.com/ClickHouse/ch-go v0.66.1 // indirect
 	github.com/Microsoft/go-winio v0.6.2 // indirect
 	github.com/andybalholm/brotli v1.1.1 // indirect
 	github.com/axw/gocov v1.2.1 // indirect
@@ -91,8 +92,6 @@ require (
 	github.com/dnephin/pflag v1.0.7 // indirect
 	github.com/docker/go-units v0.5.0 // indirect
 	github.com/dustin/go-humanize v1.0.1 // indirect
-	github.com/eapache/go-xerial-snappy v0.0.0-20230731223053-c322873962e3 // indirect
-	github.com/eapache/queue v1.1.0 // indirect
 	github.com/emicklei/dot v0.15.0 // indirect
 	github.com/fatih/color v1.18.0 // indirect
 	github.com/fatih/structtag v1.2.0 // indirect
@@ -112,28 +111,20 @@ require (
 	github.com/goccy/go-json v0.10.2 // indirect
 	github.com/gogo/protobuf v1.3.2 // indirect
 	github.com/golang/protobuf v1.5.4 // indirect
-	github.com/golang/snappy v0.0.4 // indirect
 	github.com/google/go-cmp v0.7.0 // indirect
 	github.com/google/licensecheck v0.3.1 // indirect
 	github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect
 	github.com/google/uuid v1.6.0 // indirect
 	github.com/grpc-ecosystem/go-grpc-middleware/v2 v2.1.0 // indirect
-	github.com/hashicorp/errwrap v1.1.0 // indirect
-	github.com/hashicorp/go-multierror v1.1.1 // indirect
-	github.com/hashicorp/go-uuid v1.0.3 // indirect
 	github.com/inconshreveable/mousetrap v1.1.0 // indirect
 	github.com/itchyny/timefmt-go v0.1.6 // indirect
 	github.com/jackc/pgpassfile v1.0.0 // indirect
 	github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
 	github.com/jackc/pgx/v5 v5.6.0 // indirect
 	github.com/jackc/puddle/v2 v2.2.2 // indirect
-	github.com/jcmturner/aescts/v2 v2.0.0 // indirect
-	github.com/jcmturner/dnsutils/v2 v2.0.0 // indirect
-	github.com/jcmturner/gofork v1.7.6 // indirect
-	github.com/jcmturner/gokrb5/v8 v8.4.4 // indirect
-	github.com/jcmturner/rpc/v2 v2.0.3 // indirect
 	github.com/jellydator/ttlcache/v2 v2.11.1 // indirect
 	github.com/jessevdk/go-flags v1.5.0 // indirect
+	github.com/jhump/protoreflect v1.17.0 // indirect
 	github.com/jinzhu/inflection v1.0.0 // indirect
 	github.com/jinzhu/now v1.1.5 // indirect
 	github.com/josharian/native v1.1.0 // indirect
@@ -182,8 +173,6 @@ require (
 	github.com/tklauser/numcpus v0.6.1 // indirect
 	github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
 	github.com/ugorji/go/codec v1.2.12 // indirect
-	github.com/xdg-go/pbkdf2 v1.0.0 // indirect
-	github.com/xdg-go/stringprep v1.0.4 // indirect
 	github.com/yusufpapurcu/wmi v1.2.4 // indirect
 	go.opentelemetry.io/auto/sdk v1.1.0 // indirect
 	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.54.0 // indirect
@@ -199,6 +188,7 @@ require (
 	golang.org/x/net v0.41.0 // indirect
 	golang.org/x/sync v0.15.0 // indirect
 	golang.org/x/term v0.32.0 // indirect
+	golang.org/x/time v0.12.0 // indirect
 	golang.org/x/tools v0.33.0 // indirect
 	google.golang.org/genproto/googleapis/rpc v0.0.0-20250218202821-56aae31c358a // indirect
 	gopkg.in/yaml.v2 v2.4.0 // indirect
@@ -216,8 +206,10 @@ tool (
 	github.com/frapposelli/wwhrd
 	github.com/mgechev/revive
 	github.com/mna/pigeon
+	github.com/planetscale/vtprotobuf/cmd/protoc-gen-go-vtproto
 	go.uber.org/mock/mockgen
 	golang.org/x/tools/cmd/goimports
+	google.golang.org/protobuf/cmd/protoc-gen-go
 	gotest.tools/gotestsum
 )
 
diff --git a/go.sum b/go.sum
index 27f60fa7..a7f69b25 100644
--- a/go.sum
+++ b/go.sum
@@ -15,8 +15,6 @@ github.com/ClickHouse/ch-go v0.66.1 h1:LQHFslfVYZsISOY0dnOYOXGkOUvpv376CCm8g7W74
 github.com/ClickHouse/ch-go v0.66.1/go.mod h1:NEYcg3aOFv2EmTJfo4m2WF7sHB/YFbLUuIWv9iq76xY=
 github.com/ClickHouse/clickhouse-go/v2 v2.37.2 h1:wRLNKoynvHQEN4znnVHNLaYnrqVc9sGJmGYg+GGCfto=
 github.com/ClickHouse/clickhouse-go/v2 v2.37.2/go.mod h1:pH2zrBGp5Y438DMwAxXMm1neSXPPjSI7tD4MURVULw8=
-github.com/IBM/sarama v1.45.2 h1:8m8LcMCu3REcwpa7fCP6v2fuPuzVwXDAM2DOv3CBrKw=
-github.com/IBM/sarama v1.45.2/go.mod h1:ppaoTcVdGv186/z6MEKsMm70A5fwJfRTpstI37kVn3Y=
 github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
 github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
 github.com/alecthomas/chroma v0.10.0 h1:7XDcGkCQopCNKjZHfYrNLraA+M7e0fMiJ/Mfikbfjek=
@@ -96,10 +94,6 @@ github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkp
 github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
 github.com/eapache/go-resiliency v1.7.0 h1:n3NRTnBn5N0Cbi/IeOHuQn9s2UwVUH7Ga0ZWcP+9JTA=
 github.com/eapache/go-resiliency v1.7.0/go.mod h1:5yPzW0MIvSe0JDsv0v+DvcjEv2FyD6iZYSs1ZI+iQho=
-github.com/eapache/go-xerial-snappy v0.0.0-20230731223053-c322873962e3 h1:Oy0F4ALJ04o5Qqpdz8XLIpNA3WM/iSIXqxtqo7UGVws=
-github.com/eapache/go-xerial-snappy v0.0.0-20230731223053-c322873962e3/go.mod h1:YvSRo5mw33fLEx1+DlK6L2VV43tJt5Eyel9n9XBcR+0=
-github.com/eapache/queue v1.1.0 h1:YOEu7KNc61ntiQlcEeUIoDTJ2o8mQznoNvUhiigpIqc=
-github.com/eapache/queue v1.1.0/go.mod h1:6eCeP0CKFpHLu8blIFXhExK/dRa7WDZfr6jVFPTqq+I=
 github.com/emicklei/dot v0.15.0 h1:XDBW0Xco1QNyRb33cqLe10cT04yMWL1XpCZfa98Q6Og=
 github.com/emicklei/dot v0.15.0/go.mod h1:DeV7GvQtIw4h2u73RKBkkFdvVAz0D9fzeJrgPW6gy/s=
 github.com/expr-lang/expr v1.17.5 h1:i1WrMvcdLF249nSNlpQZN1S6NXuW9WaOfF5tPi3aw3k=
@@ -110,8 +104,6 @@ github.com/fatih/structtag v1.2.0 h1:/OdNE99OxoI/PqaW/SuSK9uxxT3f/tcSZgon/ssNSx4
 github.com/fatih/structtag v1.2.0/go.mod h1:mBJUNpUnHmRKrKlQQlmCrh5PuhftFbNv8Ys4/aAZl94=
 github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
 github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
-github.com/fortytw2/leaktest v1.3.0 h1:u8491cBMTQ8ft8aeV+adlcytMZylmA5nnwwkRZjI8vw=
-github.com/fortytw2/leaktest v1.3.0/go.mod h1:jDsjWgpAGjm2CA7WthBh/CdZYEPF31XHquHwclZch5g=
 github.com/frapposelli/wwhrd v0.4.0 h1:Vn4hjT/tHNeOnTxFBO0ys1NBH8/Inxqqi1Q0eJmCImo=
 github.com/frapposelli/wwhrd v0.4.0/go.mod h1:Bzwvr3hY1yoBsBbIMkckeHUI6jf1cLRueaaMxZ3N9FY=
 github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
@@ -180,8 +172,6 @@ github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiu
 github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
 github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
 github.com/golang/snappy v0.0.1/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
-github.com/golang/snappy v0.0.4 h1:yAGX7huGHXlcLOEtBnF4w7FQwA26wojNCwOYAEhLjQM=
-github.com/golang/snappy v0.0.4/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
 github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
 github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
 github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
@@ -204,8 +194,6 @@ github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 h1:El6M4kTTCOh6aBiKaU
 github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510/go.mod h1:pupxD2MaaD3pAXIBCelhxNneeOaAeabZDe5s4K6zSpQ=
 github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
 github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
-github.com/gorilla/securecookie v1.1.1/go.mod h1:ra0sb63/xPlUeL+yeDciTfxMRAA+MP+HVt/4epWDjd4=
-github.com/gorilla/sessions v1.2.1/go.mod h1:dk2InVEVJ0sfLlnXv9EAgkf6ecYs/i80K/zI+bUmuGM=
 github.com/gosnmp/gosnmp v1.36.2-0.20231009064202-d306ed5aa998/go.mod h1:O938QjIS4vpSag1UTcnnBq9MfNmimuOGtvQsT1NbErc=
 github.com/gosnmp/gosnmp v1.41.0 h1:6RI78g2ZsbLvpvJegcV98LapszRQnbvYNKSa5WbCll4=
 github.com/gosnmp/gosnmp v1.41.0/go.mod h1:CxVS6bXqmWZlafUj9pZUnQX5e4fAltqPcijxWpCitDo=
@@ -215,14 +203,6 @@ github.com/grpc-ecosystem/go-grpc-middleware/v2 v2.1.0 h1:pRhl55Yx1eC7BZ1N+BBWwn
 github.com/grpc-ecosystem/go-grpc-middleware/v2 v2.1.0/go.mod h1:XKMd7iuf/RGPSMJ/U4HP0zS2Z9Fh8Ps9a+6X26m/tmI=
 github.com/grpc-ecosystem/grpc-gateway/v2 v2.16.0 h1:YBftPWNWd4WwGqtY2yeZL2ef8rHAxPBD8KFhJpmcqms=
 github.com/grpc-ecosystem/grpc-gateway/v2 v2.16.0/go.mod h1:YN5jB8ie0yfIUg6VvR9Kz84aCaG7AsGZnLjhHbUqwPg=
-github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
-github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY2I=
-github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
-github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+lD48awMYo=
-github.com/hashicorp/go-multierror v1.1.1/go.mod h1:iw975J/qwKPdAO1clOe2L8331t/9/fmwbPZ6JB6eMoM=
-github.com/hashicorp/go-uuid v1.0.2/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
-github.com/hashicorp/go-uuid v1.0.3 h1:2gKiV6YVmrJ1i2CKKa9obLvRieoRGviZFL26PcT/Co8=
-github.com/hashicorp/go-uuid v1.0.3/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
 github.com/hashicorp/go-version v1.7.0 h1:5tqGy27NaOTB8yJKUZELlFAS/LTKJkrmONwQKeRZfjY=
 github.com/hashicorp/go-version v1.7.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
 github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
@@ -241,18 +221,6 @@ github.com/jackc/pgx/v5 v5.6.0 h1:SWJzexBzPL5jb0GEsrPMLIsi/3jOo7RHlzTjcAeDrPY=
 github.com/jackc/pgx/v5 v5.6.0/go.mod h1:DNZ/vlrUnhWCoFGxHAG8U2ljioxukquj7utPDgtQdTw=
 github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
 github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
-github.com/jcmturner/aescts/v2 v2.0.0 h1:9YKLH6ey7H4eDBXW8khjYslgyqG2xZikXP0EQFKrle8=
-github.com/jcmturner/aescts/v2 v2.0.0/go.mod h1:AiaICIRyfYg35RUkr8yESTqvSy7csK90qZ5xfvvsoNs=
-github.com/jcmturner/dnsutils/v2 v2.0.0 h1:lltnkeZGL0wILNvrNiVCR6Ro5PGU/SeBvVO/8c/iPbo=
-github.com/jcmturner/dnsutils/v2 v2.0.0/go.mod h1:b0TnjGOvI/n42bZa+hmXL+kFJZsFT7G4t3HTlQ184QM=
-github.com/jcmturner/gofork v1.7.6 h1:QH0l3hzAU1tfT3rZCnW5zXl+orbkNMMRGJfdJjHVETg=
-github.com/jcmturner/gofork v1.7.6/go.mod h1:1622LH6i/EZqLloHfE7IeZ0uEJwMSUyQ/nDd82IeqRo=
-github.com/jcmturner/goidentity/v6 v6.0.1 h1:VKnZd2oEIMorCTsFBnJWbExfNN7yZr3EhJAxwOkZg6o=
-github.com/jcmturner/goidentity/v6 v6.0.1/go.mod h1:X1YW3bgtvwAXju7V3LCIMpY0Gbxyjn/mY9zx4tFonSg=
-github.com/jcmturner/gokrb5/v8 v8.4.4 h1:x1Sv4HaTpepFkXbt2IkL29DXRf8sOfZXo8eRKh687T8=
-github.com/jcmturner/gokrb5/v8 v8.4.4/go.mod h1:1btQEpgT6k+unzCwX1KdWMEwPPkkgBtP+F6aCACiMrs=
-github.com/jcmturner/rpc/v2 v2.0.3 h1:7FXXj8Ti1IaVFpSAziCZWNzbNuZmnvw/i6CqLNdWfZY=
-github.com/jcmturner/rpc/v2 v2.0.3/go.mod h1:VUJYCIDm3PVOEHw8sgt091/20OJjskO/YJki3ELg/Hc=
 github.com/jellydator/ttlcache/v2 v2.11.1 h1:AZGME43Eh2Vv3giG6GeqeLeFXxwxn1/qHItqWZl6U64=
 github.com/jellydator/ttlcache/v2 v2.11.1/go.mod h1:RtE5Snf0/57e+2cLWFYWCCsLas2Hy3c5Z4n14XmSvTI=
 github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=
@@ -379,6 +347,8 @@ github.com/pierrec/lz4/v4 v4.1.22 h1:cKFw6uJDK+/gfw5BcDL0JL5aBsAFdsIT18eRtLj7VIU
 github.com/pierrec/lz4/v4 v4.1.22/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=
 github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
 github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
+github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 h1:GFCKgmp0tecUJ0sJuv4pzYCqS9+RGSn52M3FUwPs+uo=
+github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10/go.mod h1:t/avpk3KcrXxUnYOhZhMXJlSEyie6gQbtLq5NM3loB8=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
 github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
 github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
@@ -393,8 +363,6 @@ github.com/prometheus/common v0.62.0/go.mod h1:vyBcEuLSvWos9B1+CyL7JZ2up+uFzXhkq
 github.com/prometheus/procfs v0.0.8/go.mod h1:7Qr8sr6344vo1JqZ6HhLceV9o3AJ1Ff+GxbHq6oeK9A=
 github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc=
 github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoGhij/e3PBqk=
-github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475 h1:N/ElC8H3+5XpJzTSTfLsJV/mx9Q9g7kxmchpfZyxgzM=
-github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4=
 github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
 github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
 github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
@@ -466,6 +434,16 @@ github.com/tklauser/numcpus v0.6.1 h1:ng9scYS7az0Bk4OZLvrNXNSAO2Pxr1XXRAPyjhIx+F
 github.com/tklauser/numcpus v0.6.1/go.mod h1:1XfjsgE2zo8GVw7POkMbHENHzVg3GzmoZ9fESEdAacY=
 github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI=
 github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
+github.com/twmb/franz-go v1.19.5 h1:W7+o8D0RsQsedqib71OVlLeZ0zI6CbFra7yTYhZTs5Y=
+github.com/twmb/franz-go v1.19.5/go.mod h1:4kFJ5tmbbl7asgwAGVuyG1ZMx0NNpYk7EqflvWfPCpM=
+github.com/twmb/franz-go/pkg/kadm v1.16.0 h1:STMs1t5lYR5mR974PSiwNzE5TvsosByTp+rKXLOhAjE=
+github.com/twmb/franz-go/pkg/kadm v1.16.0/go.mod h1:MUdcUtnf9ph4SFBLLA/XxE29rvLhWYLM9Ygb8dfSCvw=
+github.com/twmb/franz-go/pkg/kfake v0.0.0-20250711145744-a849b8be17b7 h1:SmVArSUtiB+bsqMjHtqemjL1YCj4L74NSiOxjtwAJ/o=
+github.com/twmb/franz-go/pkg/kfake v0.0.0-20250711145744-a849b8be17b7/go.mod h1:udxwmMC3r4xqjwrSrMi8p9jpqMDNpC2YwexpDSUmQtw=
+github.com/twmb/franz-go/pkg/kmsg v1.11.2 h1:hIw75FpwcAjgeyfIGFqivAvwC5uNIOWRGvQgZhH4mhg=
+github.com/twmb/franz-go/pkg/kmsg v1.11.2/go.mod h1:CFfkkLysDNmukPYhGzuUcDtf46gQSqCZHMW1T4Z+wDE=
+github.com/twmb/franz-go/plugin/kprom v1.2.1 h1:FGWdneW9htySYmvJ5tEuAIZepjFOuTFhHLy5TrVR+QI=
+github.com/twmb/franz-go/plugin/kprom v1.2.1/go.mod h1:+dzpKnVE6By8BDRFj240dTDJS9bP2dngmuhv7egJ3Go=
 github.com/ugorji/go v1.1.7/go.mod h1:kZn38zHttfInRq0xu/PH0az30d+z6vm202qpg1oXVMw=
 github.com/ugorji/go/codec v1.1.7/go.mod h1:Ax+UKWsSmolVDwsd+7N3ZtXu+yMGCf907BLYF3GoBXY=
 github.com/ugorji/go/codec v1.2.12 h1:9LC83zGrHhuUA9l16C9AHXAqEV/2wBQ4nkvumAE65EE=
@@ -473,14 +451,9 @@ github.com/ugorji/go/codec v1.2.12/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZ
 github.com/urfave/cli/v2 v2.1.1/go.mod h1:SE9GqnLQmjVa0iPEY0f1w3ygNIYcIJ0OKPMoW2caLfQ=
 github.com/vishvananda/netns v0.0.4 h1:Oeaw1EM2JMxD51g9uhtC0D7erkIjgmj8+JZc26m1YX8=
 github.com/vishvananda/netns v0.0.4/go.mod h1:SpkAiCQRtJ6TvvxPnOSyH3BMl6unz3xZlaprSwhNNJM=
-github.com/xdg-go/pbkdf2 v1.0.0 h1:Su7DPu48wXMwC3bs7MCNG+z4FhcyEuz5dlvchbq0B0c=
 github.com/xdg-go/pbkdf2 v1.0.0/go.mod h1:jrpuAogTd400dnrH08LKmI/xc1MbPOebTwRqcT5RDeI=
 github.com/xdg-go/scram v1.1.1/go.mod h1:RaEWvsqvNKKvBPvcKeFjrG2cJqOkHTiyTpzz23ni57g=
-github.com/xdg-go/scram v1.1.2 h1:FHX5I5B4i4hKRVRBCFRxq1iQRej7WO3hhBuJf+UUySY=
-github.com/xdg-go/scram v1.1.2/go.mod h1:RT/sEzTbU5y00aCK8UOx6R7YryM0iF1N2MOmC3kKLN4=
 github.com/xdg-go/stringprep v1.0.3/go.mod h1:W3f5j4i+9rC0kuIEJL0ky1VpHXQU3ocBgklLGvcBnW8=
-github.com/xdg-go/stringprep v1.0.4 h1:XLI/Ng3O1Atzq0oBs3TWm+5ZVgkq2aqdlvP9JtoZ6c8=
-github.com/xdg-go/stringprep v1.0.4/go.mod h1:mPGuuIYwz7CmR2bT9j4GbQqutWS1zV24gijq1dTyGkM=
 github.com/xyproto/randomstring v1.0.5 h1:YtlWPoRdgMu3NZtP45drfy1GKoojuR7hmRcnhZqKjWU=
 github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E=
 github.com/youmark/pkcs8 v0.0.0-20181117223130-1be2e3e5546d/go.mod h1:rHwXgn7JulP+udvsHwJoVG1YGAP6VLg4y9I5dyZdqmA=
@@ -534,7 +507,6 @@ golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8U
 golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
 golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
 golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
-golang.org/x/crypto v0.6.0/go.mod h1:OFC/31mSvZgRz0V1QTNCzfAI1aIRzbiufJtkMIlEp58=
 golang.org/x/crypto v0.13.0/go.mod h1:y6Z2r+Rw4iayiXXAIxJIDAJ1zMW4yaTpebo8fPOliYc=
 golang.org/x/crypto v0.39.0 h1:SHs+kF4LP+f+p14esP5jAoDpHU8Gu/v9lFRK6IT5imM=
 golang.org/x/crypto v0.39.0/go.mod h1:L+Xg3Wf6HoL4Bn4238Z6ft6KfEpN0tJGo53AAPC632U=
@@ -553,7 +525,6 @@ golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73r
 golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
 golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
 golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
 golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
 golang.org/x/net v0.0.0-20200520004742-59133d7f0dd7/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
 golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
@@ -563,7 +534,6 @@ golang.org/x/net v0.0.0-20210428140749-89ef3d95e781/go.mod h1:OJAsFXCWl8Ukc7SiCT
 golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
 golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
 golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
-golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
 golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
 golang.org/x/net v0.15.0/go.mod h1:idbUs1IY1+zTqbi8yxTbhexhEEk5ur9LInksu6HrEpk=
 golang.org/x/net v0.41.0 h1:vBTly1HeNPEn3wtREYfy4GZ/NECgw2Cnl+nK6Nz3uvw=
@@ -626,7 +596,6 @@ golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
 golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
 golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
 golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
-golang.org/x/text v0.3.8/go.mod h1:E6s5w1FMmriuDzIBO73fBruAKo1PCIq6d2Q6DHfQ8WQ=
 golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
 golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
 golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
diff --git a/inlet/core/root.go b/inlet/core/root.go
deleted file mode 100644
index 67b92a6d..00000000
--- a/inlet/core/root.go
+++ /dev/null
@@ -1,167 +0,0 @@
-// SPDX-FileCopyrightText: 2022 Free Mobile
-// SPDX-License-Identifier: AGPL-3.0-only
-
-// Package core plumbs all the other components together.
-package core
-
-import (
-	"fmt"
-	"sync/atomic"
-	"time"
-
-	"gopkg.in/tomb.v2"
-
-	"akvorado/common/daemon"
-	"akvorado/common/helpers/cache"
-	"akvorado/common/httpserver"
-	"akvorado/common/reporter"
-	"akvorado/common/schema"
-	"akvorado/inlet/flow"
-	"akvorado/inlet/kafka"
-	"akvorado/inlet/metadata"
-	"akvorado/inlet/routing"
-)
-
-// Component represents the HTTP compomenent.
-type Component struct {
-	r      *reporter.Reporter
-	d      *Dependencies
-	t      tomb.Tomb
-	config Configuration
-
-	metrics metrics
-
-	healthy            chan reporter.ChannelHealthcheckFunc
-	httpFlowClients    uint32 // for dumping flows
-	httpFlowChannel    chan *schema.FlowMessage
-	httpFlowFlushDelay time.Duration
-
-	classifierExporterCache  *cache.Cache[exporterInfo, exporterClassification]
-	classifierInterfaceCache *cache.Cache[exporterAndInterfaceInfo, interfaceClassification]
-	classifierErrLogger      reporter.Logger
-}
-
-// Dependencies define the dependencies of the HTTP component.
-type Dependencies struct {
-	Daemon   daemon.Component
-	Flow     *flow.Component
-	Metadata *metadata.Component
-	Routing  *routing.Component
-	Kafka    *kafka.Component
-	HTTP     *httpserver.Component
-	Schema   *schema.Component
-}
-
-// New creates a new core component.
-func New(r *reporter.Reporter, configuration Configuration, dependencies Dependencies) (*Component, error) {
-	c := Component{
-		r:      r,
-		d:      &dependencies,
-		config: configuration,
-
-		healthy:            make(chan reporter.ChannelHealthcheckFunc),
-		httpFlowClients:    0,
-		httpFlowChannel:    make(chan *schema.FlowMessage, 10),
-		httpFlowFlushDelay: time.Second,
-
-		classifierExporterCache:  cache.New[exporterInfo, exporterClassification](),
-		classifierInterfaceCache: cache.New[exporterAndInterfaceInfo, interfaceClassification](),
-		classifierErrLogger:      r.Sample(reporter.BurstSampler(10*time.Second, 3)),
-	}
-	c.d.Daemon.Track(&c.t, "inlet/core")
-	c.initMetrics()
-	return &c, nil
-}
-
-// Start starts the core component.
-func (c *Component) Start() error {
-	c.r.Info().Msg("starting core component")
-	for i := range c.config.Workers {
-		workerID := i
-		c.t.Go(func() error {
-			return c.runWorker(workerID)
-		})
-	}
-
-	// Classifier cache expiration
-	c.t.Go(func() error {
-		for {
-			select {
-			case <-c.t.Dying():
-				return nil
-			case <-time.After(c.config.ClassifierCacheDuration):
-				before := time.Now().Add(-c.config.ClassifierCacheDuration)
-				c.classifierExporterCache.DeleteLastAccessedBefore(before)
-				c.classifierInterfaceCache.DeleteLastAccessedBefore(before)
-			}
-		}
-	})
-
-	c.r.RegisterHealthcheck("core", c.channelHealthcheck())
-	c.d.HTTP.GinRouter.GET("/api/v0/inlet/flows", c.FlowsHTTPHandler)
-	return nil
-}
-
-// runWorker starts a worker.
-func (c *Component) runWorker(workerID int) error {
-	c.r.Debug().Int("worker", workerID).Msg("starting core worker")
-
-	for {
-		select {
-		case <-c.t.Dying():
-			c.r.Debug().Int("worker", workerID).Msg("stopping core worker")
-			return nil
-		case cb, ok := <-c.healthy:
-			if ok {
-				cb(reporter.HealthcheckOK, fmt.Sprintf("worker %d ok", workerID))
-			}
-		case flow := <-c.d.Flow.Flows():
-			if flow == nil {
-				c.r.Info().Int("worker", workerID).Msg("no more flow available, stopping")
-				return nil
-			}
-
-			exporter := flow.ExporterAddress.Unmap().String()
-			c.metrics.flowsReceived.WithLabelValues(exporter).Inc()
-
-			// Enrichment
-			ip := flow.ExporterAddress
-			if skip := c.enrichFlow(ip, exporter, flow); skip {
-				continue
-			}
-
-			// Serialize flow to Protobuf
-			buf := c.d.Schema.ProtobufMarshal(flow)
-
-			// Forward to Kafka. This could block and buf is now owned by the
-			// Kafka subsystem!
-			c.metrics.flowsForwarded.WithLabelValues(exporter).Inc()
-			c.d.Kafka.Send(exporter, buf)
-
-			// If we have HTTP clients, send to them too
-			if atomic.LoadUint32(&c.httpFlowClients) > 0 {
-				select {
-				case c.httpFlowChannel <- flow: // OK
-				default: // Overflow, best effort and ignore
-				}
-			}
-
-		}
-	}
-}
-
-// Stop stops the core component.
-func (c *Component) Stop() error {
-	defer func() {
-		close(c.httpFlowChannel)
-		close(c.healthy)
-		c.r.Info().Msg("core component stopped")
-	}()
-	c.r.Info().Msg("stopping core component")
-	c.t.Kill(nil)
-	return c.t.Wait()
-}
-
-func (c *Component) channelHealthcheck() reporter.HealthcheckFunc {
-	return reporter.ChannelHealthcheck(c.t.Context(nil), c.healthy)
-}
diff --git a/inlet/flow/config.go b/inlet/flow/config.go
index ecbe89e6..eed0f0ba 100644
--- a/inlet/flow/config.go
+++ b/inlet/flow/config.go
@@ -4,10 +4,8 @@
 package flow
 
 import (
-	"golang.org/x/time/rate"
-
 	"akvorado/common/helpers"
-	"akvorado/inlet/flow/decoder"
+	"akvorado/common/pb"
 	"akvorado/inlet/flow/input"
 	"akvorado/inlet/flow/input/file"
 	"akvorado/inlet/flow/input/udp"
@@ -17,21 +15,18 @@ import (
 type Configuration struct {
 	// Inputs define a list of input modules to enable
 	Inputs []InputConfiguration `validate:"dive"`
-	// RateLimit defines a rate limit on the number of flows per
-	// second. The limit is per-exporter.
-	RateLimit rate.Limit `validate:"isdefault|min=100"`
 }
 
 // DefaultConfiguration represents the default configuration for the flow component
 func DefaultConfiguration() Configuration {
 	return Configuration{
 		Inputs: []InputConfiguration{{
-			TimestampSource: decoder.TimestampSourceUDP,
-			Decoder:         "netflow",
+			TimestampSource: pb.RawFlow_TS_INPUT,
+			Decoder:         pb.RawFlow_DECODER_NETFLOW,
 			Config:          udp.DefaultConfiguration(),
 		}, {
-			TimestampSource: decoder.TimestampSourceUDP,
-			Decoder:         "sflow",
+			TimestampSource: pb.RawFlow_TS_INPUT,
+			Decoder:         pb.RawFlow_DECODER_SFLOW,
 			Config:          udp.DefaultConfiguration(),
 		}},
 	}
@@ -40,12 +35,12 @@ func DefaultConfiguration() Configuration {
 // InputConfiguration represents the configuration for an input.
 type InputConfiguration struct {
 	// Decoder is the decoder to associate to the input.
-	Decoder string
+	Decoder pb.RawFlow_Decoder `validate:"required"`
 	// UseSrcAddrForExporterAddr replaces the exporter address by the transport
 	// source address.
 	UseSrcAddrForExporterAddr bool
 	// TimestampSource identify the source to use to timestamp the flows
-	TimestampSource decoder.TimestampSource
+	TimestampSource pb.RawFlow_TimestampSource
 	// Config is the actual configuration of the input.
 	Config input.Configuration
 }
@@ -61,6 +56,7 @@ var inputs = map[string](func() input.Configuration){
 }
 
 func init() {
+	helpers.RegisterMapstructureDeprecatedFields[Configuration]("RateLimit")
 	helpers.RegisterMapstructureUnmarshallerHook(
 		helpers.ParametrizedConfigurationUnmarshallerHook(InputConfiguration{}, inputs))
 }
diff --git a/inlet/flow/decoder.go b/inlet/flow/decoder.go
deleted file mode 100644
index f23c9639..00000000
--- a/inlet/flow/decoder.go
+++ /dev/null
@@ -1,66 +0,0 @@
-// SPDX-FileCopyrightText: 2022 Free Mobile
-// SPDX-License-Identifier: AGPL-3.0-only
-
-package flow
-
-import (
-	"net/netip"
-
-	"akvorado/common/schema"
-	"akvorado/inlet/flow/decoder"
-	"akvorado/inlet/flow/decoder/netflow"
-	"akvorado/inlet/flow/decoder/sflow"
-)
-
-type wrappedDecoder struct {
-	c                         *Component
-	orig                      decoder.Decoder
-	useSrcAddrForExporterAddr bool
-}
-
-// Decode decodes a flow while keeping some stats.
-func (wd *wrappedDecoder) Decode(in decoder.RawFlow) []*schema.FlowMessage {
-	defer func() {
-		if r := recover(); r != nil {
-			wd.c.metrics.decoderErrors.WithLabelValues(wd.orig.Name()).
-				Inc()
-		}
-	}()
-	decoded := wd.orig.Decode(in)
-
-	if decoded == nil {
-		wd.c.metrics.decoderErrors.WithLabelValues(wd.orig.Name()).
-			Inc()
-		return nil
-	}
-
-	if wd.useSrcAddrForExporterAddr {
-		exporterAddress, _ := netip.AddrFromSlice(in.Source.To16())
-		for _, f := range decoded {
-			f.ExporterAddress = exporterAddress
-		}
-	}
-
-	wd.c.metrics.decoderStats.WithLabelValues(wd.orig.Name()).
-		Inc()
-	return decoded
-}
-
-// Name returns the name of the original decoder.
-func (wd *wrappedDecoder) Name() string {
-	return wd.orig.Name()
-}
-
-// wrapDecoder wraps the provided decoders to get statistics from it.
-func (c *Component) wrapDecoder(d decoder.Decoder, useSrcAddrForExporterAddr bool) decoder.Decoder {
-	return &wrappedDecoder{
-		c:                         c,
-		orig:                      d,
-		useSrcAddrForExporterAddr: useSrcAddrForExporterAddr,
-	}
-}
-
-var decoders = map[string]decoder.NewDecoderFunc{
-	"netflow": netflow.New,
-	"sflow":   sflow.New,
-}
diff --git a/inlet/flow/decoder/config.go b/inlet/flow/decoder/config.go
deleted file mode 100644
index 56bfc9e0..00000000
--- a/inlet/flow/decoder/config.go
+++ /dev/null
@@ -1,19 +0,0 @@
-// SPDX-FileCopyrightText: 2024 Free Mobile
-// SPDX-License-Identifier: AGPL-3.0-only
-
-package decoder
-
-// TimestampSource defines the method to use to extract the TimeReceived for the flows
-type TimestampSource uint
-
-const (
-	// TimestampSourceUDP tells the decoder to use the kernel time at which
-	// the UDP packet was received
-	TimestampSourceUDP TimestampSource = iota
-	// TimestampSourceNetflowPacket tells the decoder to use the timestamp
-	// from the router in the netflow packet
-	TimestampSourceNetflowPacket
-	// TimestampSourceNetflowFirstSwitched tells the decoder to use the timestamp
-	// from each flow "FIRST_SWITCHED" field
-	TimestampSourceNetflowFirstSwitched
-)
diff --git a/inlet/flow/input/file/config.go b/inlet/flow/input/file/config.go
index 34449385..5cf95bbf 100644
--- a/inlet/flow/input/file/config.go
+++ b/inlet/flow/input/file/config.go
@@ -9,6 +9,9 @@ import "akvorado/inlet/flow/input"
 type Configuration struct {
 	// Paths to use as input
 	Paths []string `validate:"min=1,dive,required"`
+	// MaxFlows tell how many flows will be read before stopping production. 0
+	// means to not stop.
+	MaxFlows uint
 }
 
 // DefaultConfiguration descrives the default configuration for file input.
diff --git a/inlet/flow/input/file/root.go b/inlet/flow/input/file/root.go
index c5ffb700..c3a433bc 100644
--- a/inlet/flow/input/file/root.go
+++ b/inlet/flow/input/file/root.go
@@ -13,9 +13,8 @@ import (
 	"gopkg.in/tomb.v2"
 
 	"akvorado/common/daemon"
+	"akvorado/common/pb"
 	"akvorado/common/reporter"
-	"akvorado/common/schema"
-	"akvorado/inlet/flow/decoder"
 	"akvorado/inlet/flow/input"
 )
 
@@ -24,62 +23,66 @@ type Input struct {
 	r      *reporter.Reporter
 	t      tomb.Tomb
 	config *Configuration
-
-	ch      chan []*schema.FlowMessage // channel to send flows to
-	decoder decoder.Decoder
+	send   input.SendFunc
 }
 
 // New instantiate a new UDP listener from the provided configuration.
-func (configuration *Configuration) New(r *reporter.Reporter, daemon daemon.Component, dec decoder.Decoder) (input.Input, error) {
+func (configuration *Configuration) New(r *reporter.Reporter, daemon daemon.Component, send input.SendFunc) (input.Input, error) {
 	if len(configuration.Paths) == 0 {
 		return nil, errors.New("no paths provided for file input")
 	}
 	input := &Input{
-		r:       r,
-		config:  configuration,
-		ch:      make(chan []*schema.FlowMessage),
-		decoder: dec,
+		r:      r,
+		config: configuration,
+		send:   send,
 	}
 	daemon.Track(&input.t, "inlet/flow/input/file")
 	return input, nil
 }
 
-// Start starts reading the provided files to produce fake flows in a loop.
-func (in *Input) Start() (<-chan []*schema.FlowMessage, error) {
+// Start starts streaming files to produce flake flows in a loop.
+func (in *Input) Start() error {
 	in.r.Info().Msg("file input starting")
 	in.t.Go(func() error {
+		count := uint(0)
+		payload := make([]byte, 9000)
+		flow := pb.RawFlow{}
 		for idx := 0; true; idx++ {
+			if in.config.MaxFlows > 0 && count >= in.config.MaxFlows {
+				<-in.t.Dying()
+				return nil
+			}
+
 			path := in.config.Paths[idx%len(in.config.Paths)]
 			data, err := os.ReadFile(path)
 			if err != nil {
 				in.r.Err(err).Str("path", path).Msg("unable to read path")
 				return err
 			}
-			flows := in.decoder.Decode(decoder.RawFlow{
-				TimeReceived: time.Now(),
-				Payload:      data,
-				Source:       net.ParseIP("127.0.0.1"),
-			})
-			if len(flows) == 0 {
-				continue
-			}
+
+			// Mimic the way it works with UDP
+			n := copy(payload, data)
+			flow.Reset()
+			flow.TimeReceived = uint64(time.Now().Unix())
+			flow.Payload = payload[:n]
+			flow.SourceAddress = net.ParseIP("127.0.0.1").To16()
+
+			in.send("127.0.0.1", &flow)
+			count++
 			select {
 			case <-in.t.Dying():
 				return nil
-			case in.ch <- flows:
+			default:
 			}
 		}
 		return nil
 	})
-	return in.ch, nil
+	return nil
 }
 
 // Stop stops the UDP listeners
 func (in *Input) Stop() error {
-	defer func() {
-		close(in.ch)
-		in.r.Info().Msg("file input stopped")
-	}()
+	defer in.r.Info().Msg("file input stopped")
 	in.t.Kill(nil)
 	return in.t.Wait()
 }
diff --git a/inlet/flow/input/root.go b/inlet/flow/input/root.go
index 02366623..466e5aee 100644
--- a/inlet/flow/input/root.go
+++ b/inlet/flow/input/root.go
@@ -6,21 +6,23 @@ package input
 
 import (
 	"akvorado/common/daemon"
+	"akvorado/common/pb"
 	"akvorado/common/reporter"
-	"akvorado/common/schema"
-	"akvorado/inlet/flow/decoder"
 )
 
 // Input is the interface any input should meet
 type Input interface {
-	// Start instructs an input to start producing flows on the returned channel.
-	Start() (<-chan []*schema.FlowMessage, error)
+	// Start instructs an input to start producing flows to be sent to Kafka component.
+	Start() error
 	// Stop instructs the input to stop producing flows.
 	Stop() error
 }
 
+// SendFunc is a function to send a flow to Kafka
+type SendFunc func(exporter string, flow *pb.RawFlow)
+
 // Configuration defines the interface to instantiate an input module from its configuration.
 type Configuration interface {
 	// New instantiates a new input from its configuration.
-	New(r *reporter.Reporter, daemon daemon.Component, dec decoder.Decoder) (Input, error)
+	New(r *reporter.Reporter, daemon daemon.Component, send SendFunc) (Input, error)
 }
diff --git a/inlet/flow/input/udp/root.go b/inlet/flow/input/udp/root.go
index 234c0543..a2d4ee86 100644
--- a/inlet/flow/input/udp/root.go
+++ b/inlet/flow/input/udp/root.go
@@ -15,9 +15,8 @@ import (
 	"gopkg.in/tomb.v2"
 
 	"akvorado/common/daemon"
+	"akvorado/common/pb"
 	"akvorado/common/reporter"
-	"akvorado/common/schema"
-	"akvorado/inlet/flow/decoder"
 	"akvorado/inlet/flow/input"
 )
 
@@ -32,23 +31,19 @@ type Input struct {
 		packets       *reporter.CounterVec
 		packetSizeSum *reporter.SummaryVec
 		errors        *reporter.CounterVec
-		outDrops      *reporter.CounterVec
-		inDrops       *reporter.GaugeVec
-		decodedFlows  *reporter.CounterVec
+		inDrops       *reporter.CounterVec
 	}
 
-	address net.Addr                   // listening address, for testing purpoese
-	ch      chan []*schema.FlowMessage // channel to send flows to
-	decoder decoder.Decoder            // decoder to use
+	address net.Addr       // listening address, for testing purpoese
+	send    input.SendFunc // function to send to kafka
 }
 
 // New instantiate a new UDP listener from the provided configuration.
-func (configuration *Configuration) New(r *reporter.Reporter, daemon daemon.Component, dec decoder.Decoder) (input.Input, error) {
+func (configuration *Configuration) New(r *reporter.Reporter, daemon daemon.Component, send input.SendFunc) (input.Input, error) {
 	input := &Input{
-		r:       r,
-		config:  configuration,
-		ch:      make(chan []*schema.FlowMessage, configuration.QueueSize),
-		decoder: dec,
+		r:      r,
+		config: configuration,
+		send:   send,
 	}
 
 	input.metrics.bytes = r.CounterVec(
@@ -67,7 +62,7 @@ func (configuration *Configuration) New(r *reporter.Reporter, daemon daemon.Comp
 	)
 	input.metrics.packetSizeSum = r.SummaryVec(
 		reporter.SummaryOpts{
-			Name:       "summary_size_bytes",
+			Name:       "size_bytes",
 			Help:       "Summary of packet size.",
 			Objectives: map[float64]float64{0.5: 0.05, 0.9: 0.01, 0.99: 0.001},
 		},
@@ -80,34 +75,20 @@ func (configuration *Configuration) New(r *reporter.Reporter, daemon daemon.Comp
 		},
 		[]string{"listener", "worker"},
 	)
-	input.metrics.outDrops = r.CounterVec(
+	input.metrics.inDrops = r.CounterVec(
 		reporter.CounterOpts{
-			Name: "out_dropped_packets_total",
-			Help: "Dropped packets due to internal queue full.",
-		},
-		[]string{"listener", "worker", "exporter"},
-	)
-	input.metrics.inDrops = r.GaugeVec(
-		reporter.GaugeOpts{
 			Name: "in_dropped_packets_total",
 			Help: "Dropped packets due to listen queue full.",
 		},
 		[]string{"listener", "worker"},
 	)
-	input.metrics.decodedFlows = r.CounterVec(
-		reporter.CounterOpts{
-			Name: "decoded_flows_total",
-			Help: "Number of flows decoded and written to the internal queue",
-		},
-		[]string{"listener", "worker", "exporter"},
-	)
 
 	daemon.Track(&input.t, "inlet/flow/input/udp")
 	return input, nil
 }
 
 // Start starts listening to the provided UDP socket and producing flows.
-func (in *Input) Start() (<-chan []*schema.FlowMessage, error) {
+func (in *Input) Start() error {
 	in.r.Info().Str("listen", in.config.Listen).Msg("starting UDP input")
 
 	// Listen to UDP port
@@ -122,12 +103,12 @@ func (in *Input) Start() (<-chan []*schema.FlowMessage, error) {
 			var err error
 			listenAddr, err = net.ResolveUDPAddr("udp", in.config.Listen)
 			if err != nil {
-				return nil, fmt.Errorf("unable to resolve %v: %w", in.config.Listen, err)
+				return fmt.Errorf("unable to resolve %v: %w", in.config.Listen, err)
 			}
 		}
 		pconn, err := listenConfig.ListenPacket(in.t.Context(context.Background()), "udp", listenAddr.String())
 		if err != nil {
-			return nil, fmt.Errorf("unable to listen to %v: %w", listenAddr, err)
+			return fmt.Errorf("unable to listen to %v: %w", listenAddr, err)
 		}
 		udpConn := pconn.(*net.UDPConn)
 		in.address = udpConn.LocalAddr()
@@ -152,11 +133,13 @@ func (in *Input) Start() (<-chan []*schema.FlowMessage, error) {
 		in.t.Go(func() error {
 			payload := make([]byte, 9000)
 			oob := make([]byte, oobLength)
+			flow := pb.RawFlow{}
 			listen := in.config.Listen
 			l := in.r.With().
 				Str("worker", worker).
 				Str("listen", listen).
 				Logger()
+			dying := in.t.Dying()
 			errLogger := l.Sample(reporter.BurstSampler(time.Minute, 1))
 			for count := 0; ; count++ {
 				n, oobn, _, source, err := conns[workerID].ReadMsgUDP(payload, oob)
@@ -173,10 +156,8 @@ func (in *Input) Start() (<-chan []*schema.FlowMessage, error) {
 				if err != nil {
 					errLogger.Err(err).Msg("unable to decode UDP control message")
 				} else {
-					if count < 100 || count%100 == 0 {
-						in.metrics.inDrops.WithLabelValues(listen, worker).Set(
-							float64(oobMsg.Drops))
-					}
+					in.metrics.inDrops.WithLabelValues(listen, worker).Add(
+						float64(oobMsg.Drops))
 				}
 				if oobMsg.Received.IsZero() {
 					oobMsg.Received = time.Now()
@@ -189,25 +170,17 @@ func (in *Input) Start() (<-chan []*schema.FlowMessage, error) {
 					Inc()
 				in.metrics.packetSizeSum.WithLabelValues(listen, worker, srcIP).
 					Observe(float64(n))
-				flows := in.decoder.Decode(decoder.RawFlow{
-					TimeReceived: oobMsg.Received,
-					Payload:      payload[:n],
-					Source:       source.IP,
-				})
-				if len(flows) == 0 {
-					continue
-				}
+
+				flow.Reset()
+				flow.TimeReceived = uint64(oobMsg.Received.Unix())
+				flow.Payload = payload[:n]
+				flow.SourceAddress = source.IP.To16()
+				in.send(srcIP, &flow)
+
 				select {
-				case <-in.t.Dying():
+				case <-dying:
 					return nil
-				case in.ch <- flows:
-					in.metrics.decodedFlows.WithLabelValues(listen, worker, srcIP).
-						Add(float64(len((flows))))
 				default:
-					errLogger.Warn().Msgf("dropping flow due to queue full (size %d)",
-						in.config.QueueSize)
-					in.metrics.outDrops.WithLabelValues(listen, worker, srcIP).
-						Inc()
 				}
 			}
 		})
@@ -223,16 +196,13 @@ func (in *Input) Start() (<-chan []*schema.FlowMessage, error) {
 		return nil
 	})
 
-	return in.ch, nil
+	return nil
 }
 
 // Stop stops the UDP listeners
 func (in *Input) Stop() error {
 	l := in.r.With().Str("listen", in.config.Listen).Logger()
-	defer func() {
-		close(in.ch)
-		l.Info().Msg("UDP listener stopped")
-	}()
+	defer l.Info().Msg("UDP listener stopped")
 	in.t.Kill(nil)
 	return in.t.Wait()
 }
diff --git a/inlet/flow/rate.go b/inlet/flow/rate.go
deleted file mode 100644
index cbc91547..00000000
--- a/inlet/flow/rate.go
+++ /dev/null
@@ -1,57 +0,0 @@
-// SPDX-FileCopyrightText: 2022 Free Mobile
-// SPDX-License-Identifier: AGPL-3.0-only
-
-package flow
-
-import (
-	"time"
-
-	"akvorado/common/schema"
-
-	"golang.org/x/time/rate"
-)
-
-type limiter struct {
-	l           *rate.Limiter
-	dropped     uint64  // dropped during the current second
-	total       uint64  // total during the current second
-	dropRate    float64 // drop rate during the last second
-	currentTick time.Time
-}
-
-// allowMessages tell if we can transmit the provided messages,
-// depending on the rate limiter configuration. If yes, their sampling
-// rate may be modified to match current drop rate.
-func (c *Component) allowMessages(fmsgs []*schema.FlowMessage) bool {
-	count := len(fmsgs)
-	if c.config.RateLimit == 0 || count == 0 {
-		return true
-	}
-	exporter := fmsgs[0].ExporterAddress
-	exporterLimiter, ok := c.limiters[exporter]
-	if !ok {
-		exporterLimiter = &limiter{
-			l: rate.NewLimiter(rate.Limit(c.config.RateLimit), int(c.config.RateLimit/10)),
-		}
-		c.limiters[exporter] = exporterLimiter
-	}
-	now := time.Now()
-	tick := now.Truncate(200 * time.Millisecond) // we use a 200-millisecond resolution
-	if exporterLimiter.currentTick.UnixMilli() != tick.UnixMilli() {
-		exporterLimiter.dropRate = float64(exporterLimiter.dropped) / float64(exporterLimiter.total)
-		exporterLimiter.dropped = 0
-		exporterLimiter.total = 0
-		exporterLimiter.currentTick = tick
-	}
-	exporterLimiter.total += uint64(count)
-	if !exporterLimiter.l.AllowN(now, count) {
-		exporterLimiter.dropped += uint64(count)
-		return false
-	}
-	if exporterLimiter.dropRate > 0 {
-		for _, flow := range fmsgs {
-			flow.SamplingRate *= uint32(1 / (1 - exporterLimiter.dropRate))
-		}
-	}
-	return true
-}
diff --git a/inlet/flow/root.go b/inlet/flow/root.go
index 82bc6b95..f0d7deb2 100644
--- a/inlet/flow/root.go
+++ b/inlet/flow/root.go
@@ -1,23 +1,22 @@
 // SPDX-FileCopyrightText: 2022 Free Mobile
 // SPDX-License-Identifier: AGPL-3.0-only
 
-// Package flow handle incoming flows (currently Netflow v9 and IPFIX).
+// Package flow handle incoming Netflow/IPFIX/sflow flows.
 package flow
 
 import (
 	"errors"
-	"fmt"
-	"net/http"
-	"net/netip"
+	"sync"
 
 	"gopkg.in/tomb.v2"
 
 	"akvorado/common/daemon"
+	"akvorado/common/helpers"
 	"akvorado/common/httpserver"
+	"akvorado/common/pb"
 	"akvorado/common/reporter"
-	"akvorado/common/schema"
-	"akvorado/inlet/flow/decoder"
 	"akvorado/inlet/flow/input"
+	"akvorado/inlet/kafka"
 )
 
 // Component represents the flow component.
@@ -27,26 +26,15 @@ type Component struct {
 	t      tomb.Tomb
 	config Configuration
 
-	metrics struct {
-		decoderStats  *reporter.CounterVec
-		decoderErrors *reporter.CounterVec
-	}
-
-	// Channel for sending flows out of the package.
-	outgoingFlows chan *schema.FlowMessage
-
-	// Per-exporter rate-limiters
-	limiters map[netip.Addr]*limiter
-
-	// Inputs
-	inputs []input.Input
+	inputs      []input.Input
+	payloadPool sync.Pool
 }
 
 // Dependencies are the dependencies of the flow component.
 type Dependencies struct {
 	Daemon daemon.Component
 	HTTP   *httpserver.Component
-	Schema *schema.Component
+	Kafka  *kafka.Component
 }
 
 // New creates a new flow component.
@@ -56,99 +44,79 @@ func New(r *reporter.Reporter, configuration Configuration, dependencies Depende
 	}
 
 	c := Component{
-		r:             r,
-		d:             &dependencies,
-		config:        configuration,
-		outgoingFlows: make(chan *schema.FlowMessage),
-		limiters:      make(map[netip.Addr]*limiter),
-		inputs:        make([]input.Input, len(configuration.Inputs)),
-	}
-
-	// Initialize decoders (at most once each)
-	alreadyInitialized := map[string]decoder.Decoder{}
-	decs := make([]decoder.Decoder, len(configuration.Inputs))
-	for idx, input := range c.config.Inputs {
-		dec, ok := alreadyInitialized[input.Decoder]
-		if ok {
-			decs[idx] = dec
-			continue
-		}
-		decoderfunc, ok := decoders[input.Decoder]
-		if !ok {
-			return nil, fmt.Errorf("unknown decoder %q", input.Decoder)
-		}
-		dec = decoderfunc(r, decoder.Dependencies{Schema: c.d.Schema}, decoder.Option{TimestampSource: input.TimestampSource})
-		alreadyInitialized[input.Decoder] = dec
-		decs[idx] = c.wrapDecoder(dec, input.UseSrcAddrForExporterAddr)
+		r:      r,
+		d:      &dependencies,
+		config: configuration,
+		inputs: make([]input.Input, len(configuration.Inputs)),
+		payloadPool: sync.Pool{
+			New: func() any {
+				minPayload := 2000
+				if helpers.Testing() {
+					// Ensure we test the extension case.
+					minPayload = 5
+				}
+				s := make([]byte, minPayload)
+				return &s
+			},
+		},
 	}
 
 	// Initialize inputs
 	for idx, input := range c.config.Inputs {
 		var err error
-		c.inputs[idx], err = input.Config.New(r, c.d.Daemon, decs[idx])
+		c.inputs[idx], err = input.Config.New(r, c.d.Daemon, c.Send(input))
 		if err != nil {
 			return nil, err
 		}
 	}
 
-	// Metrics
-	c.metrics.decoderStats = c.r.CounterVec(
-		reporter.CounterOpts{
-			Name: "decoder_flows_total",
-			Help: "Decoder processed count.",
-		},
-		[]string{"name"},
-	)
-	c.metrics.decoderErrors = c.r.CounterVec(
-		reporter.CounterOpts{
-			Name: "decoder_errors_total",
-			Help: "Decoder processed error count.",
-		},
-		[]string{"name"},
-	)
-
 	c.d.Daemon.Track(&c.t, "inlet/flow")
 
-	c.d.HTTP.AddHandler("/api/v0/inlet/flow/schema.proto",
-		http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
-			w.Header().Set("Content-Type", "text/plain")
-			w.Write([]byte(c.d.Schema.ProtobufDefinition()))
-		}))
-
 	return &c, nil
 }
 
-// Flows returns a channel to receive flows.
-func (c *Component) Flows() <-chan *schema.FlowMessage {
-	return c.outgoingFlows
+// Send sends a raw flow to Kafka.
+func (c *Component) Send(config InputConfiguration) input.SendFunc {
+	return func(exporter string, flow *pb.RawFlow) {
+		flow.TimestampSource = config.TimestampSource
+		flow.Decoder = config.Decoder
+		flow.UseSourceAddress = config.UseSrcAddrForExporterAddr
+
+		// Get a payload from the pool and extend it if needed. We use a pool of
+		// pointers to slice as we may have to extend the capacity of the slice.
+		// We keep the original pointer to avoid an extra allocation on the heap
+		// when returning the slice to the pool.
+		ptr := c.payloadPool.Get().(*[]byte)
+		bytes := *ptr
+		n := flow.SizeVT()
+		if cap(bytes) < n {
+			bytes = make([]byte, n+100)
+			*ptr = bytes
+		}
+
+		// Marshal to it, send it to Kafka and return it when done
+		if n, err := flow.MarshalToSizedBufferVT(bytes[:n]); err == nil {
+			c.d.Kafka.Send(exporter, bytes[:n], func() {
+				c.payloadPool.Put(ptr)
+			})
+		} else {
+			c.payloadPool.Put(ptr)
+		}
+	}
 }
 
 // Start starts the flow component.
 func (c *Component) Start() error {
 	for _, input := range c.inputs {
-		ch, err := input.Start()
+		err := input.Start()
 		stopper := input.Stop
 		if err != nil {
 			return err
 		}
 		c.t.Go(func() error {
-			defer stopper()
-			for {
-				select {
-				case <-c.t.Dying():
-					return nil
-				case fmsgs := <-ch:
-					if c.allowMessages(fmsgs) {
-						for _, fmsg := range fmsgs {
-							select {
-							case <-c.t.Dying():
-								return nil
-							case c.outgoingFlows <- fmsg:
-							}
-						}
-					}
-				}
-			}
+			<-c.t.Dying()
+			stopper()
+			return nil
 		})
 	}
 	return nil
@@ -156,10 +124,7 @@ func (c *Component) Start() error {
 
 // Stop stops the flow component
 func (c *Component) Stop() error {
-	defer func() {
-		close(c.outgoingFlows)
-		c.r.Info().Msg("flow component stopped")
-	}()
+	defer c.r.Info().Msg("flow component stopped")
 	c.r.Info().Msg("stopping flow component")
 	c.t.Kill(nil)
 	return c.t.Wait()
diff --git a/inlet/kafka/config.go b/inlet/kafka/config.go
index 933ff434..eec89360 100644
--- a/inlet/kafka/config.go
+++ b/inlet/kafka/config.go
@@ -4,27 +4,20 @@
 package kafka
 
 import (
-	"time"
+	"fmt"
 
-	"github.com/IBM/sarama"
+	"github.com/twmb/franz-go/pkg/kgo"
 
+	"akvorado/common/helpers"
 	"akvorado/common/kafka"
 )
 
 // Configuration describes the configuration for the Kafka exporter.
 type Configuration struct {
 	kafka.Configuration `mapstructure:",squash" yaml:"-,inline"`
-	// FlushInterval tells how often to flush pending data to Kafka.
-	FlushInterval time.Duration `validate:"min=100ms"`
-	// FlushBytes tells to flush when there are many bytes to write
-	FlushBytes int `validate:"min=1000"`
-	// MaxMessageBytes is the maximum permitted size of a message.
-	// Should be set equal or smaller than broker's
-	// `message.max.bytes`.
-	MaxMessageBytes int `validate:"min=1"`
 	// CompressionCodec defines the compression to use.
 	CompressionCodec CompressionCodec
-	// QueueSize defines the size of the channel used to send to Kafka.
+	// QueueSize defines the maximum number of messages to buffer.
 	QueueSize int `validate:"min=1"`
 }
 
@@ -32,28 +25,62 @@ type Configuration struct {
 func DefaultConfiguration() Configuration {
 	return Configuration{
 		Configuration:    kafka.DefaultConfiguration(),
-		FlushInterval:    time.Second,
-		FlushBytes:       int(sarama.MaxRequestSize) - 1,
-		MaxMessageBytes:  1000000,
-		CompressionCodec: CompressionCodec(sarama.CompressionNone),
+		CompressionCodec: CompressionCodec(kgo.Lz4Compression()),
 		QueueSize:        32,
 	}
 }
 
 // CompressionCodec represents a compression codec.
-type CompressionCodec sarama.CompressionCodec
+type CompressionCodec kgo.CompressionCodec
 
 // UnmarshalText produces a compression codec
 func (cc *CompressionCodec) UnmarshalText(text []byte) error {
-	return (*sarama.CompressionCodec)(cc).UnmarshalText(text)
+	codec := kgo.CompressionCodec{}
+	switch string(text) {
+	case "none":
+		codec = kgo.NoCompression()
+	case "gzip":
+		codec = kgo.GzipCompression()
+	case "snappy":
+		codec = kgo.SnappyCompression()
+	case "lz4":
+		codec = kgo.Lz4Compression()
+	case "zstd":
+		codec = kgo.ZstdCompression()
+	default:
+		return fmt.Errorf("unknown compression codec: %s", text)
+	}
+	*cc = CompressionCodec(codec)
+	return nil
 }
 
 // String turns a compression codec into a string
 func (cc CompressionCodec) String() string {
-	return sarama.CompressionCodec(cc).String()
+	switch kgo.CompressionCodec(cc) {
+	case kgo.NoCompression():
+		return "none"
+	case kgo.GzipCompression():
+		return "gzip"
+	case kgo.SnappyCompression():
+		return "snappy"
+	case kgo.Lz4Compression():
+		return "lz4"
+	case kgo.ZstdCompression():
+		return "zstd"
+	default:
+		return "unknown"
+	}
 }
 
 // MarshalText turns a compression codec into a string
 func (cc CompressionCodec) MarshalText() ([]byte, error) {
 	return []byte(cc.String()), nil
 }
+
+func init() {
+	helpers.RegisterMapstructureDeprecatedFields[Configuration](
+		"FlushInterval",   // bad for performance
+		"FlushBytes",      //  duplicate with QueueSize
+		"MaxMessageBytes", //  just tune QueueSize instead
+	)
+}
diff --git a/inlet/kafka/metrics.go b/inlet/kafka/metrics.go
index 492d96cc..9cf6f973 100644
--- a/inlet/kafka/metrics.go
+++ b/inlet/kafka/metrics.go
@@ -4,38 +4,16 @@
 package kafka
 
 import (
-	"strings"
-
-	"github.com/prometheus/client_golang/prometheus"
-	gometrics "github.com/rcrowley/go-metrics"
-
 	"akvorado/common/reporter"
 )
 
 type metrics struct {
-	c *Component
-
 	messagesSent *reporter.CounterVec
 	bytesSent    *reporter.CounterVec
 	errors       *reporter.CounterVec
-
-	kafkaIncomingByteRate  *reporter.MetricDesc
-	kafkaOutgoingByteRate  *reporter.MetricDesc
-	kafkaRequestRate       *reporter.MetricDesc
-	kafkaRequestSize       *reporter.MetricDesc
-	kafkaRequestLatency    *reporter.MetricDesc
-	kafkaResponseRate      *reporter.MetricDesc
-	kafkaResponseSize      *reporter.MetricDesc
-	kafkaRequestsInFlight  *reporter.MetricDesc
-	kafkaBatchSize         *reporter.MetricDesc
-	kafkaRecordSendRate    *reporter.MetricDesc
-	kafkaRecordsPerRequest *reporter.MetricDesc
-	kafkaCompressionRatio  *reporter.MetricDesc
 }
 
 func (c *Component) initMetrics() {
-	c.metrics.c = c
-
 	c.metrics.messagesSent = c.r.CounterVec(
 		reporter.CounterOpts{
 			Name: "sent_messages_total",
@@ -57,152 +35,4 @@ func (c *Component) initMetrics() {
 		},
 		[]string{"error"},
 	)
-
-	c.metrics.kafkaIncomingByteRate = c.r.MetricDesc(
-		"brokers_incoming_byte_rate",
-		"Bytes/second read off a given broker.",
-		[]string{"broker"})
-	c.metrics.kafkaOutgoingByteRate = c.r.MetricDesc(
-		"brokers_outgoing_byte_rate",
-		"Bytes/second written off a given broker.",
-		[]string{"broker"})
-	c.metrics.kafkaRequestRate = c.r.MetricDesc(
-		"brokers_request_rate",
-		"Requests/second sent to a given broker.",
-		[]string{"broker"})
-	c.metrics.kafkaRequestSize = c.r.MetricDesc(
-		"brokers_request_size",
-		"Distribution of the request size in bytes for a given broker.",
-		[]string{"broker"})
-	c.metrics.kafkaRequestLatency = c.r.MetricDesc(
-		"brokers_request_latency_ms",
-		"Distribution of the request latency in ms for a given broker.",
-		[]string{"broker"})
-	c.metrics.kafkaResponseRate = c.r.MetricDesc(
-		"brokers_response_rate",
-		"Responses/second received from a given broker.",
-		[]string{"broker"})
-	c.metrics.kafkaResponseSize = c.r.MetricDesc(
-		"brokers_response_bytes",
-		"Distribution of the response size in bytes for a given broker.",
-		[]string{"broker"})
-	c.metrics.kafkaRequestsInFlight = c.r.MetricDesc(
-		"brokers_inflight_requests",
-		"The current number of in-flight requests awaiting a response for a given broker.",
-		[]string{"broker"})
-	c.metrics.kafkaBatchSize = c.r.MetricDesc(
-		"producer_batch_bytes",
-		"Distribution of the number of bytes sent per partition per request.",
-		nil)
-	c.metrics.kafkaRecordSendRate = c.r.MetricDesc(
-		"producer_record_send_rate",
-		"Records/second sent.",
-		nil)
-	c.metrics.kafkaRecordsPerRequest = c.r.MetricDesc(
-		"producer_records_per_request",
-		"Distribution of the number of records sent per request.",
-		nil)
-	c.metrics.kafkaCompressionRatio = c.r.MetricDesc(
-		"producer_compression_ratio",
-		"Distribution of the compression ratio times 100 of record batches.",
-		nil)
-
-	c.r.MetricCollector(c.metrics)
-}
-
-// Describe collected metrics
-func (m metrics) Describe(ch chan<- *prometheus.Desc) {
-	ch <- m.kafkaIncomingByteRate
-	ch <- m.kafkaOutgoingByteRate
-	ch <- m.kafkaRequestRate
-	ch <- m.kafkaRequestSize
-	ch <- m.kafkaRequestLatency
-	ch <- m.kafkaResponseRate
-	ch <- m.kafkaResponseSize
-	ch <- m.kafkaRequestsInFlight
-	ch <- m.kafkaBatchSize
-	ch <- m.kafkaRecordSendRate
-	ch <- m.kafkaRecordsPerRequest
-	ch <- m.kafkaCompressionRatio
-}
-
-// Collect metrics
-func (m metrics) Collect(ch chan<- prometheus.Metric) {
-	m.c.kafkaConfig.MetricRegistry.Each(func(name string, gom interface{}) {
-		// Broker-related
-		if broker := metricBroker(name, "incoming-byte-rate"); broker != "" {
-			gomMeter(ch, m.kafkaIncomingByteRate, gom, broker)
-			return
-		}
-		if broker := metricBroker(name, "outgoing-byte-rate"); broker != "" {
-			gomMeter(ch, m.kafkaOutgoingByteRate, gom, broker)
-			return
-		}
-		if broker := metricBroker(name, "request-rate"); broker != "" {
-			gomMeter(ch, m.kafkaRequestRate, gom, broker)
-			return
-		}
-		if broker := metricBroker(name, "request-size"); broker != "" {
-			gomHistogram(ch, m.kafkaRequestSize, gom, broker)
-			return
-		}
-		if broker := metricBroker(name, "request-latency-in-ms"); broker != "" {
-			gomHistogram(ch, m.kafkaRequestLatency, gom, broker)
-			return
-		}
-		if broker := metricBroker(name, "response-rate"); broker != "" {
-			gomMeter(ch, m.kafkaResponseRate, gom, broker)
-			return
-		}
-		if broker := metricBroker(name, "response-size"); broker != "" {
-			gomHistogram(ch, m.kafkaResponseSize, gom, broker)
-			return
-		}
-		if broker := metricBroker(name, "requests-in-flight"); broker != "" {
-			snap := gom.(gometrics.Counter).Snapshot()
-			ch <- prometheus.MustNewConstMetric(m.kafkaRequestsInFlight,
-				prometheus.GaugeValue, float64(snap.Count()), broker)
-			return
-		}
-		// Producer-related
-		if name == "batch-size" {
-			gomHistogram(ch, m.kafkaBatchSize, gom)
-			return
-		}
-		if name == "record-send-rate" {
-			gomMeter(ch, m.kafkaRecordSendRate, gom)
-			return
-		}
-		if name == "records-per-request" {
-			gomHistogram(ch, m.kafkaRecordsPerRequest, gom)
-			return
-		}
-		if name == "compression-ratio" {
-			gomHistogram(ch, m.kafkaCompressionRatio, gom)
-			return
-		}
-	})
-}
-
-func metricBroker(name string, prefix string) string {
-	prefix = prefix + "-for-broker-"
-	if strings.HasPrefix(name, prefix) {
-		return strings.TrimPrefix(name, prefix)
-	}
-	return ""
-}
-
-func gomMeter(ch chan<- prometheus.Metric, desc *reporter.MetricDesc, m interface{}, labels ...string) {
-	snap := m.(gometrics.Meter).Snapshot()
-	ch <- prometheus.MustNewConstMetric(desc, prometheus.GaugeValue, snap.Rate1(), labels...)
-}
-
-func gomHistogram(ch chan<- prometheus.Metric, desc *reporter.MetricDesc, m interface{}, labels ...string) {
-	snap := m.(gometrics.Histogram).Snapshot()
-	buckets := map[float64]uint64{
-		0.5:  uint64(snap.Percentile(0.5)),
-		0.9:  uint64(snap.Percentile(0.9)),
-		0.99: uint64(snap.Percentile(0.99)),
-	}
-	ch <- prometheus.MustNewConstHistogram(desc, uint64(snap.Count()), float64(snap.Sum()), buckets, labels...)
 }
diff --git a/inlet/kafka/root.go b/inlet/kafka/root.go
index 8ae544d3..80ea46b4 100644
--- a/inlet/kafka/root.go
+++ b/inlet/kafka/root.go
@@ -5,19 +5,22 @@
 package kafka
 
 import (
+	"context"
 	"encoding/binary"
 	"fmt"
 	"math/rand"
 	"strings"
 	"time"
 
-	"github.com/IBM/sarama"
+	"github.com/twmb/franz-go/pkg/kerr"
+	"github.com/twmb/franz-go/pkg/kgo"
+	"github.com/twmb/franz-go/plugin/kprom"
 	"gopkg.in/tomb.v2"
 
 	"akvorado/common/daemon"
 	"akvorado/common/kafka"
+	"akvorado/common/pb"
 	"akvorado/common/reporter"
-	"akvorado/common/schema"
 )
 
 // Component represents the Kafka exporter.
@@ -27,51 +30,50 @@ type Component struct {
 	t      tomb.Tomb
 	config Configuration
 
-	kafkaTopic          string
-	kafkaConfig         *sarama.Config
-	kafkaProducer       sarama.AsyncProducer
-	createKafkaProducer func() (sarama.AsyncProducer, error)
-	metrics             metrics
+	kafkaOpts   []kgo.Opt
+	kafkaTopic  string
+	kafkaClient *kgo.Client
+	errLogger   reporter.Logger
+	metrics     metrics
 }
 
 // Dependencies define the dependencies of the Kafka exporter.
 type Dependencies struct {
 	Daemon daemon.Component
-	Schema *schema.Component
 }
 
 // New creates a new Kafka exporter component.
-func New(reporter *reporter.Reporter, configuration Configuration, dependencies Dependencies) (*Component, error) {
+func New(r *reporter.Reporter, configuration Configuration, dependencies Dependencies) (*Component, error) {
 	// Build Kafka configuration
-	kafkaConfig, err := kafka.NewConfig(configuration.Configuration)
+	kafkaOpts, err := kafka.NewConfig(r, configuration.Configuration)
 	if err != nil {
 		return nil, err
 	}
-	kafkaConfig.Metadata.AllowAutoTopicCreation = true
-	kafkaConfig.Producer.MaxMessageBytes = configuration.MaxMessageBytes
-	kafkaConfig.Producer.Compression = sarama.CompressionCodec(configuration.CompressionCodec)
-	kafkaConfig.Producer.Return.Successes = false
-	kafkaConfig.Producer.Return.Errors = true
-	kafkaConfig.Producer.Flush.Bytes = configuration.FlushBytes
-	kafkaConfig.Producer.Flush.Frequency = configuration.FlushInterval
-	kafkaConfig.Producer.Partitioner = sarama.NewHashPartitioner
-	kafkaConfig.ChannelBufferSize = configuration.QueueSize
-	if err := kafkaConfig.Validate(); err != nil {
-		return nil, fmt.Errorf("cannot validate Kafka configuration: %w", err)
-	}
 
 	c := Component{
-		r:      reporter,
-		d:      &dependencies,
-		config: configuration,
-
-		kafkaConfig: kafkaConfig,
-		kafkaTopic:  fmt.Sprintf("%s-%s", configuration.Topic, dependencies.Schema.ProtobufMessageHash()),
+		r:          r,
+		d:          &dependencies,
+		config:     configuration,
+		kafkaTopic: fmt.Sprintf("%s-v%d", configuration.Topic, pb.Version),
+		errLogger:  r.Sample(reporter.BurstSampler(10*time.Second, 3)),
 	}
 	c.initMetrics()
-	c.createKafkaProducer = func() (sarama.AsyncProducer, error) {
-		return sarama.NewAsyncProducer(c.config.Brokers, c.kafkaConfig)
+	kafkaMetrics := kprom.NewMetrics("")
+	r.MetricCollectorForCurrentModule(kafkaMetrics)
+
+	// Initialize options error to be able to validate them.
+	kafkaOpts = append(kafkaOpts,
+		kgo.AllowAutoTopicCreation(),
+		kgo.MaxBufferedRecords(configuration.QueueSize),
+		kgo.ProducerBatchCompression(kgo.CompressionCodec(configuration.CompressionCodec)),
+		kgo.RecordPartitioner(kgo.UniformBytesPartitioner(64<<20, true, true, nil)),
+		kgo.WithHooks(kafkaMetrics),
+	)
+
+	if err := kgo.ValidateOpts(kafkaOpts...); err != nil {
+		return nil, fmt.Errorf("invalid Kafka configuration: %w", err)
 	}
+	c.kafkaOpts = kafkaOpts
 	c.d.Daemon.Track(&c.t, "inlet/kafka")
 	return &c, nil
 }
@@ -79,63 +81,58 @@ func New(reporter *reporter.Reporter, configuration Configuration, dependencies
 // Start starts the Kafka component.
 func (c *Component) Start() error {
 	c.r.Info().Msg("starting Kafka component")
-	kafka.GlobalKafkaLogger.Register(c.r)
 
-	// Create producer
-	kafkaProducer, err := c.createKafkaProducer()
+	kafkaClient, err := kgo.NewClient(c.kafkaOpts...)
 	if err != nil {
 		c.r.Err(err).
 			Str("brokers", strings.Join(c.config.Brokers, ",")).
-			Msg("unable to create async producer")
-		return fmt.Errorf("unable to create Kafka async producer: %w", err)
+			Msg("unable to create Kafka client")
+		return fmt.Errorf("unable to create Kafka client: %w", err)
 	}
-	c.kafkaProducer = kafkaProducer
+	c.kafkaClient = kafkaClient
 
-	// Main loop
+	// When dying, close the client
 	c.t.Go(func() error {
-		defer kafkaProducer.Close()
-		defer c.kafkaConfig.MetricRegistry.UnregisterAll()
-		errLogger := c.r.Sample(reporter.BurstSampler(10*time.Second, 3))
-		for {
-			select {
-			case <-c.t.Dying():
-				c.r.Debug().Msg("stop error logger")
-				return nil
-			case msg := <-kafkaProducer.Errors():
-				if msg != nil {
-					c.metrics.errors.WithLabelValues(msg.Error()).Inc()
-					errLogger.Err(msg.Err).
-						Str("topic", msg.Msg.Topic).
-						Int64("offset", msg.Msg.Offset).
-						Int32("partition", msg.Msg.Partition).
-						Msg("Kafka producer error")
-				}
-			}
-		}
+		<-c.t.Dying()
+		kafkaClient.Close()
+		return nil
 	})
 	return nil
 }
 
 // Stop stops the Kafka component
 func (c *Component) Stop() error {
-	defer func() {
-		kafka.GlobalKafkaLogger.Unregister()
-		c.r.Info().Msg("Kafka component stopped")
-	}()
+	defer c.r.Info().Msg("Kafka component stopped")
 	c.r.Info().Msg("stopping Kafka component")
 	c.t.Kill(nil)
 	return c.t.Wait()
 }
 
 // Send a message to Kafka.
-func (c *Component) Send(exporter string, payload []byte) {
-	c.metrics.bytesSent.WithLabelValues(exporter).Add(float64(len(payload)))
-	c.metrics.messagesSent.WithLabelValues(exporter).Inc()
+func (c *Component) Send(exporter string, payload []byte, finalizer func()) {
 	key := make([]byte, 4)
 	binary.BigEndian.PutUint32(key, rand.Uint32())
-	c.kafkaProducer.Input() <- &sarama.ProducerMessage{
+	record := &kgo.Record{
 		Topic: c.kafkaTopic,
-		Key:   sarama.ByteEncoder(key),
-		Value: sarama.ByteEncoder(payload),
+		Key:   key,
+		Value: payload,
 	}
+	c.kafkaClient.Produce(context.Background(), record, func(r *kgo.Record, err error) {
+		if err == nil {
+			c.metrics.bytesSent.WithLabelValues(exporter).Add(float64(len(payload)))
+			c.metrics.messagesSent.WithLabelValues(exporter).Inc()
+		} else {
+			if ke, ok := err.(*kerr.Error); ok {
+				c.metrics.errors.WithLabelValues(ke.Message).Inc()
+			} else {
+				c.metrics.errors.WithLabelValues("unknown").Inc()
+			}
+			c.errLogger.Err(err).
+				Str("topic", c.kafkaTopic).
+				Int64("offset", r.Offset).
+				Int32("partition", r.Partition).
+				Msg("Kafka producer error")
+		}
+		finalizer()
+	})
 }
diff --git a/inlet/metadata/provider/gnmi/root.go b/inlet/metadata/provider/gnmi/root.go
deleted file mode 100644
index 6f0178e0..00000000
--- a/inlet/metadata/provider/gnmi/root.go
+++ /dev/null
@@ -1,81 +0,0 @@
-// SPDX-FileCopyrightText: 2023 Free Mobile
-// SPDX-License-Identifier: AGPL-3.0-only
-
-// Package gnmi use gNMI to get interface names and descriptions.
-package gnmi
-
-import (
-	"context"
-	"net/netip"
-	"sync"
-
-	"akvorado/common/reporter"
-	"akvorado/inlet/metadata/provider"
-)
-
-// Provider represents the gNMI provider.
-type Provider struct {
-	r       *reporter.Reporter
-	config  *Configuration
-	metrics metrics
-
-	put     func(provider.Update)
-	refresh chan bool
-
-	state     map[netip.Addr]*exporterState
-	stateLock sync.Mutex
-}
-
-// New creates a new gNMI provider from configuration
-func (configuration Configuration) New(r *reporter.Reporter, put func(provider.Update)) (provider.Provider, error) {
-	p := Provider{
-		r:       r,
-		config:  &configuration,
-		put:     put,
-		state:   map[netip.Addr]*exporterState{},
-		refresh: make(chan bool),
-	}
-	p.initMetrics()
-	return &p, nil
-}
-
-// Query queries exporter to get information through gNMI.
-func (p *Provider) Query(ctx context.Context, q *provider.BatchQuery) error {
-	p.stateLock.Lock()
-	defer p.stateLock.Unlock()
-	state, ok := p.state[q.ExporterIP]
-	// If we don't have a collector for the provided IP, starts one. We should
-	// be sure we don't have several collectors for the same exporter, hence the
-	// write lock for everything.
-	if !ok {
-		state := exporterState{}
-		p.state[q.ExporterIP] = &state
-		go p.startCollector(ctx, q.ExporterIP, &state)
-		p.metrics.collectorCount.Inc()
-		return nil
-	}
-	// If the collector exists and already provided some data, populate the
-	// cache.
-	if state.Ready {
-		for _, ifindex := range q.IfIndexes {
-			p.put(provider.Update{
-				Query: provider.Query{
-					ExporterIP: q.ExporterIP,
-					IfIndex:    ifindex,
-				},
-				Answer: provider.Answer{
-					Exporter: provider.Exporter{
-						Name: state.Name,
-					},
-					Interface: state.Interfaces[ifindex],
-				},
-			})
-		}
-		// Also trigger a refresh
-		select {
-		case p.refresh <- true:
-		default:
-		}
-	}
-	return nil
-}
diff --git a/inlet/metadata/root.go b/inlet/metadata/root.go
deleted file mode 100644
index 739619c0..00000000
--- a/inlet/metadata/root.go
+++ /dev/null
@@ -1,349 +0,0 @@
-// SPDX-FileCopyrightText: 2022 Free Mobile
-// SPDX-License-Identifier: AGPL-3.0-only
-
-// Package metadata handles metadata polling to get interface names and
-// descriptions. It keeps a cache of retrieved entries and refresh them. It is
-// modular and accepts several kind of providers (including SNMP).
-package metadata
-
-import (
-	"errors"
-	"fmt"
-	"net/netip"
-	"strconv"
-	"sync"
-	"time"
-
-	"github.com/benbjohnson/clock"
-	"github.com/eapache/go-resiliency/breaker"
-	"gopkg.in/tomb.v2"
-
-	"akvorado/common/daemon"
-	"akvorado/common/reporter"
-	"akvorado/inlet/metadata/provider"
-)
-
-// Component represents the metadata compomenent.
-type Component struct {
-	r      *reporter.Reporter
-	d      *Dependencies
-	t      tomb.Tomb
-	config Configuration
-
-	sc *metadataCache
-
-	healthyWorkers         chan reporter.ChannelHealthcheckFunc
-	providerChannel        chan provider.BatchQuery
-	dispatcherChannel      chan provider.Query
-	dispatcherBChannel     chan (<-chan bool) // block channel for testing
-	providerBreakersLock   sync.Mutex
-	providerBreakerLoggers map[netip.Addr]reporter.Logger
-	providerBreakers       map[netip.Addr]*breaker.Breaker
-	providers              []provider.Provider
-
-	metrics struct {
-		cacheRefreshRuns         reporter.Counter
-		cacheRefresh             reporter.Counter
-		providerBusyCount        *reporter.CounterVec
-		providerBreakerOpenCount *reporter.CounterVec
-		providerBatchedCount     reporter.Counter
-	}
-}
-
-// Dependencies define the dependencies of the metadata component.
-type Dependencies struct {
-	Daemon daemon.Component
-	Clock  clock.Clock
-}
-
-// New creates a new metadata component.
-func New(r *reporter.Reporter, configuration Configuration, dependencies Dependencies) (*Component, error) {
-	if configuration.CacheRefresh > 0 && configuration.CacheRefresh < configuration.CacheDuration {
-		return nil, errors.New("cache refresh must be greater than cache duration")
-	}
-	if configuration.CacheDuration < configuration.CacheCheckInterval {
-		return nil, errors.New("cache duration must be greater than cache check interval")
-	}
-
-	if dependencies.Clock == nil {
-		dependencies.Clock = clock.New()
-	}
-	sc := newMetadataCache(r)
-	c := Component{
-		r:      r,
-		d:      &dependencies,
-		config: configuration,
-		sc:     sc,
-
-		providerChannel:        make(chan provider.BatchQuery),
-		dispatcherChannel:      make(chan provider.Query, 100*configuration.Workers),
-		dispatcherBChannel:     make(chan (<-chan bool)),
-		providerBreakers:       make(map[netip.Addr]*breaker.Breaker),
-		providerBreakerLoggers: make(map[netip.Addr]reporter.Logger),
-		providers:              make([]provider.Provider, 0, 1),
-	}
-	c.d.Daemon.Track(&c.t, "inlet/metadata")
-
-	// Initialize providers
-	for _, p := range c.config.Providers {
-		selectedProvider, err := p.Config.New(r, func(update provider.Update) {
-			c.sc.Put(c.d.Clock.Now(), update.Query, update.Answer)
-		})
-		if err != nil {
-			return nil, err
-		}
-		c.providers = append(c.providers, selectedProvider)
-	}
-
-	c.metrics.cacheRefreshRuns = r.Counter(
-		reporter.CounterOpts{
-			Name: "cache_refresh_runs_total",
-			Help: "Number of times the cache refresh was triggered.",
-		})
-	c.metrics.cacheRefresh = r.Counter(
-		reporter.CounterOpts{
-			Name: "cache_refreshs",
-			Help: "Number of entries refreshed in cache.",
-		})
-	c.metrics.providerBusyCount = r.CounterVec(
-		reporter.CounterOpts{
-			Name: "provider_dropped_requests_total",
-			Help: "Providers where too busy and dropped requests.",
-		},
-		[]string{"exporter"})
-	c.metrics.providerBreakerOpenCount = r.CounterVec(
-		reporter.CounterOpts{
-			Name: "provider_breaker_opens_total",
-			Help: "Provider breaker was opened due to too many errors.",
-		},
-		[]string{"exporter"})
-	c.metrics.providerBatchedCount = r.Counter(
-		reporter.CounterOpts{
-			Name: "provider_batched_requests_total",
-			Help: "Several requests were batched into one.",
-		},
-	)
-	return &c, nil
-}
-
-// Start starts the metadata component.
-func (c *Component) Start() error {
-	c.r.Info().Msg("starting metadata component")
-
-	// Load cache
-	if c.config.CachePersistFile != "" {
-		if err := c.sc.Load(c.config.CachePersistFile); err != nil {
-			c.r.Err(err).Msg("cannot load cache, ignoring")
-		}
-	}
-
-	// Goroutine to refresh the cache
-	healthyTicker := make(chan reporter.ChannelHealthcheckFunc)
-	c.r.RegisterHealthcheck("metadata/ticker", reporter.ChannelHealthcheck(c.t.Context(nil), healthyTicker))
-	c.t.Go(func() error {
-		c.r.Debug().Msg("starting metadata ticker")
-		ticker := c.d.Clock.Ticker(c.config.CacheCheckInterval)
-		defer ticker.Stop()
-		defer close(healthyTicker)
-		for {
-			select {
-			case <-c.t.Dying():
-				c.r.Debug().Msg("shutting down metadata ticker")
-				return nil
-			case cb, ok := <-healthyTicker:
-				if ok {
-					cb(reporter.HealthcheckOK, "ok")
-				}
-			case <-ticker.C:
-				c.expireCache()
-			}
-		}
-	})
-
-	// Goroutine to fetch incoming requests and dispatch them to workers
-	healthyDispatcher := make(chan reporter.ChannelHealthcheckFunc)
-	c.r.RegisterHealthcheck("metadata/dispatcher", reporter.ChannelHealthcheck(c.t.Context(nil), healthyDispatcher))
-	c.t.Go(func() error {
-		for {
-			select {
-			case <-c.t.Dying():
-				c.r.Debug().Msg("stopping metadata dispatcher")
-				return nil
-			case cb, ok := <-healthyDispatcher:
-				if ok {
-					cb(reporter.HealthcheckOK, "ok")
-				}
-			case ch := <-c.dispatcherBChannel:
-				// This is to test batching
-				<-ch
-			case request := <-c.dispatcherChannel:
-				c.dispatchIncomingRequest(request)
-			}
-		}
-	})
-
-	// Goroutines to poll exporters
-	c.healthyWorkers = make(chan reporter.ChannelHealthcheckFunc)
-	c.r.RegisterHealthcheck("metadata/worker", reporter.ChannelHealthcheck(c.t.Context(nil), c.healthyWorkers))
-	for i := range c.config.Workers {
-		workerIDStr := strconv.Itoa(i)
-		c.t.Go(func() error {
-			c.r.Debug().Str("worker", workerIDStr).Msg("starting metadata provider")
-			for {
-				select {
-				case <-c.t.Dying():
-					c.r.Debug().Str("worker", workerIDStr).Msg("stopping metadata provider")
-					return nil
-				case cb, ok := <-c.healthyWorkers:
-					if ok {
-						cb(reporter.HealthcheckOK, fmt.Sprintf("worker %s ok", workerIDStr))
-					}
-				case request := <-c.providerChannel:
-					c.providerIncomingRequest(request)
-				}
-			}
-		})
-	}
-	return nil
-}
-
-// Stop stops the metadata component
-func (c *Component) Stop() error {
-	defer func() {
-		close(c.dispatcherChannel)
-		close(c.providerChannel)
-		close(c.healthyWorkers)
-		if c.config.CachePersistFile != "" {
-			if err := c.sc.Save(c.config.CachePersistFile); err != nil {
-				c.r.Err(err).Msg("cannot save cache")
-			}
-		}
-		c.r.Info().Msg("metadata component stopped")
-	}()
-	c.r.Info().Msg("stopping metadata component")
-	c.t.Kill(nil)
-	return c.t.Wait()
-}
-
-// Lookup for interface information for the provided exporter and ifIndex.
-// If the information is not in the cache, it will be polled, but
-// won't be returned immediately.
-func (c *Component) Lookup(t time.Time, exporterIP netip.Addr, ifIndex uint) (provider.Answer, bool) {
-	query := provider.Query{ExporterIP: exporterIP, IfIndex: ifIndex}
-	answer, ok := c.sc.Lookup(t, query)
-	if !ok {
-		select {
-		case c.dispatcherChannel <- query:
-		default:
-			c.metrics.providerBusyCount.WithLabelValues(exporterIP.Unmap().String()).Inc()
-		}
-	}
-	return answer, ok
-}
-
-// dispatchIncomingRequest dispatches an incoming request to workers. It may
-// handle more than the provided request if it can.
-func (c *Component) dispatchIncomingRequest(request provider.Query) {
-	requestsMap := map[netip.Addr][]uint{
-		request.ExporterIP: {request.IfIndex},
-	}
-	for c.config.MaxBatchRequests > 0 {
-		select {
-		case request := <-c.dispatcherChannel:
-			indexes, ok := requestsMap[request.ExporterIP]
-			if !ok {
-				indexes = []uint{request.IfIndex}
-			} else {
-				indexes = append(indexes, request.IfIndex)
-			}
-			requestsMap[request.ExporterIP] = indexes
-			// We don't want to exceed the configured limit but also there is no
-			// point of batching requests of too many exporters.
-			if len(indexes) < c.config.MaxBatchRequests && len(requestsMap) < 4 {
-				continue
-			}
-		case <-c.t.Dying():
-			return
-		default:
-			// No more requests in queue
-		}
-		break
-	}
-	for exporterIP, ifIndexes := range requestsMap {
-		if len(ifIndexes) > 1 {
-			c.metrics.providerBatchedCount.Add(float64(len(ifIndexes)))
-		}
-		select {
-		case <-c.t.Dying():
-			return
-		case c.providerChannel <- provider.BatchQuery{ExporterIP: exporterIP, IfIndexes: ifIndexes}:
-		}
-	}
-}
-
-// providerIncomingRequest handles an incoming request to the provider. It
-// uses a breaker to avoid pushing working on non-responsive exporters.
-func (c *Component) providerIncomingRequest(request provider.BatchQuery) {
-	// Avoid querying too much exporters with errors
-	c.providerBreakersLock.Lock()
-	providerBreaker, ok := c.providerBreakers[request.ExporterIP]
-	if !ok {
-		providerBreaker = breaker.New(20, 1, time.Minute)
-		c.providerBreakers[request.ExporterIP] = providerBreaker
-	}
-	c.providerBreakersLock.Unlock()
-
-	if err := providerBreaker.Run(func() error {
-		ctx := c.t.Context(nil)
-		for _, p := range c.providers {
-			// Query providers in the order they are defined and stop on the
-			// first provider accepting to handle the query.
-			if err := p.Query(ctx, &request); err != nil && err != provider.ErrSkipProvider {
-				return err
-			} else if err == provider.ErrSkipProvider {
-				continue
-			}
-			return nil
-		}
-		return nil
-	}); err == breaker.ErrBreakerOpen {
-		c.metrics.providerBreakerOpenCount.WithLabelValues(request.ExporterIP.Unmap().String()).Inc()
-		c.providerBreakersLock.Lock()
-		l, ok := c.providerBreakerLoggers[request.ExporterIP]
-		if !ok {
-			l = c.r.Sample(reporter.BurstSampler(time.Minute, 1)).
-				With().
-				Str("exporter", request.ExporterIP.Unmap().String()).
-				Logger()
-			c.providerBreakerLoggers[request.ExporterIP] = l
-		}
-		l.Warn().Msg("provider breaker open")
-		c.providerBreakersLock.Unlock()
-	}
-}
-
-// expireCache handles cache expiration and refresh.
-func (c *Component) expireCache() {
-	c.sc.Expire(c.d.Clock.Now().Add(-c.config.CacheDuration))
-	if c.config.CacheRefresh > 0 {
-		c.r.Debug().Msg("refresh metadata cache")
-		c.metrics.cacheRefreshRuns.Inc()
-		count := 0
-		toRefresh := c.sc.NeedUpdates(c.d.Clock.Now().Add(-c.config.CacheRefresh))
-		for exporter, ifaces := range toRefresh {
-			for _, ifIndex := range ifaces {
-				select {
-				case c.dispatcherChannel <- provider.Query{
-					ExporterIP: exporter,
-					IfIndex:    ifIndex,
-				}:
-					count++
-				default:
-					c.metrics.providerBusyCount.WithLabelValues(exporter.Unmap().String()).Inc()
-				}
-			}
-		}
-		c.r.Debug().Int("count", count).Msg("refreshed metadata cache")
-		c.metrics.cacheRefresh.Add(float64(count))
-	}
-}
diff --git a/nix/vendorHash.txt b/nix/vendorHash.txt
index 0cb330cf..02ae2bce 100644
--- a/nix/vendorHash.txt
+++ b/nix/vendorHash.txt
@@ -1 +1 @@
-sha256-TeYE1s4Yw4Ag2IbmP/bNgzP+XoOiup+gXm25nNaeTAU=
+sha256-O73QosbpmRezSPxUzkh7nJABMxTGUySfgrdcVqu3meM=
diff --git a/orchestrator/clickhouse/config.go b/orchestrator/clickhouse/config.go
index b7a21697..7c7f4780 100644
--- a/orchestrator/clickhouse/config.go
+++ b/orchestrator/clickhouse/config.go
@@ -10,7 +10,6 @@ import (
 	"akvorado/common/remotedatasourcefetcher"
 
 	"akvorado/common/helpers"
-	"akvorado/common/kafka"
 
 	"github.com/go-viper/mapstructure/v2"
 )
@@ -19,19 +18,12 @@ import (
 type Configuration struct {
 	// SkipMigrations tell if we should skip migrations.
 	SkipMigrations bool
-	// Kafka describes Kafka-specific configuration
-	Kafka KafkaConfiguration
 	// Resolutions describe the various resolutions to use to
 	// store data and the associated TTLs.
 	Resolutions []ResolutionConfiguration `validate:"min=1,dive"`
 	// MaxPartitions define the number of partitions to have for a
 	// consolidated flow tables when full.
 	MaxPartitions int `validate:"isdefault|min=1"`
-	// SystemLogTTL is the TTL to set for system log tables.
-	SystemLogTTL time.Duration `validate:"isdefault|min=1m"`
-	// PrometheusEndpoint defines the endpoint ClickHouse can use to expose
-	// metrics to Prometheus. If not defined, this is not configured.
-	PrometheusEndpoint string
 	// ASNs is a mapping from AS numbers to names. It replaces or
 	// extends the builtin list of AS numbers.
 	ASNs map[uint32]string
@@ -72,26 +64,9 @@ type ResolutionConfiguration struct {
 	TTL time.Duration `validate:"isdefault|min=1h"`
 }
 
-// KafkaConfiguration describes Kafka-specific configuration
-type KafkaConfiguration struct {
-	kafka.Configuration `mapstructure:",squash" yaml:"-,inline"`
-	// Consumers tell how many consumers to use to poll data from Kafka
-	Consumers int `validate:"min=1"`
-	// GroupName defines the Kafka consumers group used to poll data from topic,
-	// shared between all Consumers.
-	GroupName string
-	// EngineSettings allows one to set arbitrary settings for Kafka engine in
-	// ClickHouse.
-	EngineSettings []string
-}
-
 // DefaultConfiguration represents the default configuration for the ClickHouse configurator.
 func DefaultConfiguration() Configuration {
 	return Configuration{
-		Kafka: KafkaConfiguration{
-			Consumers: 1,
-			GroupName: "clickhouse",
-		},
 		Resolutions: []ResolutionConfiguration{
 			{0, 15 * 24 * time.Hour},                   // 15 days
 			{time.Minute, 7 * 24 * time.Hour},          // 7 days
@@ -100,7 +75,6 @@ func DefaultConfiguration() Configuration {
 		},
 		MaxPartitions:         50,
 		NetworkSourcesTimeout: 10 * time.Second,
-		SystemLogTTL:          30 * 24 * time.Hour, // 30 days
 	}
 }
 
@@ -147,5 +121,9 @@ func NetworkAttributesUnmarshallerHook() mapstructure.DecodeHookFunc {
 func init() {
 	helpers.RegisterMapstructureUnmarshallerHook(helpers.SubnetMapUnmarshallerHook[NetworkAttributes]())
 	helpers.RegisterMapstructureUnmarshallerHook(NetworkAttributesUnmarshallerHook())
+	helpers.RegisterMapstructureDeprecatedFields[Configuration](
+		"SystemLogTTL",
+		"PrometheusEndpoint",
+		"Kafka")
 	helpers.RegisterSubnetMapValidation[NetworkAttributes]()
 }
diff --git a/orchestrator/clickhouse/data/docker-entrypoint.sh b/orchestrator/clickhouse/data/docker-entrypoint.sh
deleted file mode 100755
index e997c289..00000000
--- a/orchestrator/clickhouse/data/docker-entrypoint.sh
+++ /dev/null
@@ -1,19 +0,0 @@
-#!/bin/bash
-# SPDX-FileCopyrightText: 2024 Free Mobile
-# SPDX-License-Identifier: AGPL-3.0-only
-
-set -e
-
-if [[ $# -lt 1 ]] || [[ "$1" = "--"* ]]; then
-    rm -f /tmp/init.sh
-    while [[ ! -s /tmp/init.sh ]]; do
-        sleep 1
-        echo "Downloading ClickHouse init script..."
-        wget --no-proxy -qO /tmp/init.sh \
-            http://akvorado-orchestrator:8080/api/v0/orchestrator/clickhouse/init.sh || continue
-    done
-    sh /tmp/init.sh
-fi
-
-# Use official entrypoint
-exec /entrypoint.sh "$@"
diff --git a/orchestrator/clickhouse/http.go b/orchestrator/clickhouse/http.go
index 2b6cf80a..b6b9e24f 100644
--- a/orchestrator/clickhouse/http.go
+++ b/orchestrator/clickhouse/http.go
@@ -4,7 +4,6 @@
 package clickhouse
 
 import (
-	"bytes"
 	"compress/gzip"
 	"embed"
 	"encoding/csv"
@@ -13,7 +12,6 @@ import (
 	"net/http"
 	"os"
 	"strconv"
-	"text/template"
 	"time"
 )
 
@@ -23,49 +21,9 @@ var (
 	//go:embed data/asns.csv
 	//go:embed data/tcp.csv
 	//go:embed data/udp.csv
-	data           embed.FS
-	initShTemplate = template.Must(template.New("initsh").Parse(`#!/bin/sh
-
-# Install Protobuf schema
-mkdir -p /var/lib/clickhouse/format_schemas
-echo "Install flow schema flow-{{ .FlowSchemaHash }}.proto"
-cat > /var/lib/clickhouse/format_schemas/flow-{{ .FlowSchemaHash }}.proto <<'EOPROTO'
-{{ .FlowSchema }}
-EOPROTO
-
-# Alter ClickHouse configuration
-mkdir -p /etc/clickhouse-server/config.d
-echo "Add Akvorado-specific configuration to ClickHouse"
-cat > /etc/clickhouse-server/config.d/akvorado.xml <<'EOCONFIG'
-<clickhouse>
-{{- if gt .SystemLogTTL 0 }}
-{{- range $table := .SystemLogTables }}
- <{{ $table }}>
-  <ttl>event_date + INTERVAL {{ $.SystemLogTTL }} SECOND DELETE</ttl>
- </{{ $table }}>
-{{- end }}
-{{- end }}
-{{- if ne .PrometheusEndpoint "" }}
- <prometheus>
-  <endpoint>{{ .PrometheusEndpoint }}</endpoint>
-  <metrics>true</metrics>
-  <events>true</events>
-  <asynchronous_metrics>true</asynchronous_metrics>
- </prometheus>
-{{- end }}
-</clickhouse>
-EOCONFIG
-`))
+	data embed.FS
 )
 
-type initShVariables struct {
-	FlowSchemaHash     string
-	FlowSchema         string
-	SystemLogTTL       int
-	SystemLogTables    []string
-	PrometheusEndpoint string
-}
-
 func (c *Component) addHandlerEmbedded(url string, path string) {
 	c.d.HTTP.AddHandler(url,
 		http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
@@ -83,37 +41,6 @@ func (c *Component) addHandlerEmbedded(url string, path string) {
 // registerHTTPHandler register some handlers that will be useful for
 // ClickHouse
 func (c *Component) registerHTTPHandlers() error {
-	// init.sh
-	c.d.HTTP.AddHandler("/api/v0/orchestrator/clickhouse/init.sh",
-		http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
-			var result bytes.Buffer
-			if err := initShTemplate.Execute(&result, initShVariables{
-				FlowSchemaHash: c.d.Schema.ProtobufMessageHash(),
-				FlowSchema:     c.d.Schema.ProtobufDefinition(),
-				SystemLogTTL:   int(c.config.SystemLogTTL.Seconds()),
-				SystemLogTables: []string{
-					"asynchronous_metric_log",
-					"error_log",
-					"metric_log",
-					"part_log",
-					"processors_profile_log",
-					"query_log",
-					"query_thread_log",
-					"query_views_log",
-					"session_log",
-					"text_log",
-					"trace_log",
-				},
-				PrometheusEndpoint: c.config.PrometheusEndpoint,
-			}); err != nil {
-				c.r.Err(err).Msg("unable to serialize init.sh")
-				http.Error(w, fmt.Sprintf("Unable to serialize init.sh"), http.StatusInternalServerError)
-				return
-			}
-			w.Header().Set("Content-Type", "text/x-shellscript")
-			w.Write(result.Bytes())
-		}))
-
 	// Add handler for custom dicts
 	for name, dict := range c.d.Schema.GetCustomDictConfig() {
 		c.d.HTTP.AddHandler(fmt.Sprintf("/api/v0/orchestrator/clickhouse/custom_dict_%s.csv", name), http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
diff --git a/orchestrator/clickhouse/migrations.go b/orchestrator/clickhouse/migrations.go
index cf528489..2a3f302d 100644
--- a/orchestrator/clickhouse/migrations.go
+++ b/orchestrator/clickhouse/migrations.go
@@ -45,17 +45,12 @@ func (c *Component) migrateDatabase() error {
 	}
 
 	// Grab some information about the database
-	var threads uint8
 	var version string
-	row := c.d.ClickHouse.QueryRow(ctx, `SELECT getSetting('max_threads'), version()`)
-	if err := row.Scan(&threads, &version); err != nil {
+	row := c.d.ClickHouse.QueryRow(ctx, `SELECT version()`)
+	if err := row.Scan(&version); err != nil {
 		c.r.Err(err).Msg("unable to parse database settings")
 		return fmt.Errorf("unable to parse database settings: %w", err)
 	}
-	if c.config.Kafka.Consumers > int(threads) {
-		c.r.Warn().Msgf("too many consumers requested, capping to %d", threads)
-		c.config.Kafka.Consumers = int(threads)
-	}
 	if err := validateVersion(version); err != nil {
 		return fmt.Errorf("incorrect ClickHouse version: %w", err)
 	}
@@ -162,12 +157,6 @@ func (c *Component) migrateDatabase() error {
 		c.createExportersConsumerView,
 		c.createRawFlowsTable,
 		c.createRawFlowsConsumerView,
-		c.createRawFlowsErrors,
-		func(ctx context.Context) error {
-			return c.createDistributedTable(ctx, "flows_raw_errors")
-		},
-		c.createRawFlowsErrorsConsumerView,
-		c.deleteOldRawFlowsErrorsView,
 	)
 	if err != nil {
 		return err
diff --git a/orchestrator/clickhouse/migrations_helpers.go b/orchestrator/clickhouse/migrations_helpers.go
index cbee3597..636c7355 100644
--- a/orchestrator/clickhouse/migrations_helpers.go
+++ b/orchestrator/clickhouse/migrations_helpers.go
@@ -281,36 +281,18 @@ CREATE MATERIALIZED VIEW exporters_consumer TO %s AS %s
 
 // createRawFlowsTable creates the raw flow table
 func (c *Component) createRawFlowsTable(ctx context.Context) error {
-	hash := c.d.Schema.ProtobufMessageHash()
+	hash := c.d.Schema.ClickHouseHash()
 	tableName := fmt.Sprintf("flows_%s_raw", hash)
-	kafkaSettings := []string{
-		fmt.Sprintf(`kafka_broker_list = %s`,
-			quoteString(strings.Join(c.config.Kafka.Brokers, ","))),
-		fmt.Sprintf(`kafka_topic_list = %s`,
-			quoteString(fmt.Sprintf("%s-%s", c.config.Kafka.Topic, hash))),
-		fmt.Sprintf(`kafka_group_name = %s`, quoteString(c.config.Kafka.GroupName)),
-		`kafka_format = 'Protobuf'`,
-		fmt.Sprintf(`kafka_schema = 'flow-%s.proto:FlowMessagev%s'`, hash, hash),
-		fmt.Sprintf(`kafka_num_consumers = %d`, c.config.Kafka.Consumers),
-		`kafka_thread_per_consumer = 1`,
-		`kafka_handle_error_mode = 'stream'`,
-	}
-	for _, setting := range c.config.Kafka.EngineSettings {
-		kafkaSettings = append(kafkaSettings, setting)
-	}
-	kafkaEngine := fmt.Sprintf("Kafka SETTINGS %s", strings.Join(kafkaSettings, ", "))
 
 	// Build CREATE query
 	createQuery, err := stemplate(
-		`CREATE TABLE {{ .Database }}.{{ .Table }} ({{ .Schema }}) ENGINE = {{ .Engine }}`,
+		`CREATE TABLE {{ .Database }}.{{ .Table }} ({{ .Schema }}) ENGINE = Null`,
 		gin.H{
 			"Database": c.d.ClickHouse.DatabaseName(),
 			"Table":    tableName,
 			"Schema": c.d.Schema.ClickHouseCreateTable(
 				schema.ClickHouseSkipGeneratedColumns,
-				schema.ClickHouseUseTransformFromType,
 				schema.ClickHouseSkipAliasedColumns),
-			"Engine": kafkaEngine,
 		})
 	if err != nil {
 		return fmt.Errorf("cannot build query to create raw flows table: %w", err)
@@ -328,7 +310,6 @@ func (c *Component) createRawFlowsTable(ctx context.Context) error {
 	c.r.Info().Msg("create raw flows table")
 	for _, table := range []string{
 		fmt.Sprintf("%s_consumer", tableName),
-		fmt.Sprintf("%s_errors", tableName),
 		tableName,
 	} {
 		if err := c.d.ClickHouse.ExecOnCluster(ctx, fmt.Sprintf(`DROP TABLE IF EXISTS %s SYNC`, table)); err != nil {
@@ -348,20 +329,19 @@ func (c *Component) createRawFlowsTable(ctx context.Context) error {
 var dictionaryNetworksLookupRegex = regexp.MustCompile(`\bc_(Src|Dst)Networks\[([[:lower:]]+)\]\B`)
 
 func (c *Component) createRawFlowsConsumerView(ctx context.Context) error {
-	tableName := fmt.Sprintf("flows_%s_raw", c.d.Schema.ProtobufMessageHash())
+	tableName := fmt.Sprintf("flows_%s_raw", c.d.Schema.ClickHouseHash())
 	viewName := fmt.Sprintf("%s_consumer", tableName)
 
 	// Build SELECT query
 	args := gin.H{
 		"Columns": strings.Join(c.d.Schema.ClickHouseSelectColumns(
 			schema.ClickHouseSubstituteGenerates,
-			schema.ClickHouseSubstituteTransforms,
 			schema.ClickHouseSkipAliasedColumns), ", "),
 		"Database": c.d.ClickHouse.DatabaseName(),
 		"Table":    tableName,
 	}
 	selectQuery, err := stemplate(
-		`SELECT {{ .Columns }} FROM {{ .Database }}.{{ .Table }} WHERE length(_error) = 0`,
+		`SELECT {{ .Columns }} FROM {{ .Database }}.{{ .Table }}`,
 		args)
 	if err != nil {
 		return fmt.Errorf("cannot build select statement for raw flows consumer view: %w", err)
@@ -445,105 +425,6 @@ func (c *Component) createRawFlowsConsumerView(ctx context.Context) error {
 	return nil
 }
 
-func (c *Component) createRawFlowsErrors(ctx context.Context) error {
-	name := c.localTable("flows_raw_errors")
-	createQuery, err := stemplate(`CREATE TABLE {{ .Database }}.{{ .Table }}
-(`+"`timestamp`"+` DateTime,
- `+"`topic`"+` LowCardinality(String),
- `+"`partition`"+` UInt64,
- `+"`offset`"+` UInt64,
- `+"`raw`"+` String,
- `+"`error`"+` String)
-ENGINE = {{ .Engine }}
-PARTITION BY toYYYYMMDDhhmmss(toStartOfHour(timestamp))
-ORDER BY (timestamp, topic, partition, offset)
-TTL timestamp + toIntervalDay(1)
-`, gin.H{
-		"Table":    name,
-		"Database": c.d.ClickHouse.DatabaseName(),
-		"Engine":   c.mergeTreeEngine(name, ""),
-	})
-	if err != nil {
-		return fmt.Errorf("cannot build query to create flow error table: %w", err)
-	}
-	if ok, err := c.tableAlreadyExists(ctx, name, "create_table_query", createQuery); err != nil {
-		return err
-	} else if ok {
-		c.r.Info().Msgf("table %s already exists, skip migration", name)
-		return errSkipStep
-	}
-	c.r.Info().Msgf("create table %s", name)
-	createOrReplaceQuery := strings.Replace(createQuery, "CREATE ", "CREATE OR REPLACE ", 1)
-	if err := c.d.ClickHouse.ExecOnCluster(ctx, createOrReplaceQuery); err != nil {
-		return fmt.Errorf("cannot create table %s: %w", name, err)
-	}
-	return nil
-}
-
-func (c *Component) createRawFlowsErrorsConsumerView(ctx context.Context) error {
-	source := fmt.Sprintf("flows_%s_raw", c.d.Schema.ProtobufMessageHash())
-	viewName := "flows_raw_errors_consumer"
-
-	// Build SELECT query
-	selectQuery, err := stemplate(`
-SELECT
- now() AS timestamp,
- _topic AS topic,
- _partition AS partition,
- _offset AS offset,
- _raw_message AS raw,
- _error AS error
-FROM {{ .Database }}.{{ .Table }}
-WHERE length(_error) > 0`, gin.H{
-		"Database": c.d.ClickHouse.DatabaseName(),
-		"Table":    source,
-	})
-	if err != nil {
-		return fmt.Errorf("cannot build select statement for raw flows error: %w", err)
-	}
-
-	// Check the existing one
-	if ok, err := c.tableAlreadyExists(ctx, viewName, "as_select", selectQuery); err != nil {
-		return err
-	} else if ok {
-		c.r.Info().Msg("raw flows errors view already exists, skip migration")
-		return errSkipStep
-	}
-
-	// Drop and create
-	c.r.Info().Msg("create raw flows errors view")
-	if err := c.d.ClickHouse.ExecOnCluster(ctx, fmt.Sprintf(`DROP TABLE IF EXISTS %s SYNC`, viewName)); err != nil {
-		return fmt.Errorf("cannot drop table %s: %w", viewName, err)
-	}
-	if err := c.d.ClickHouse.ExecOnCluster(ctx,
-		fmt.Sprintf(`CREATE MATERIALIZED VIEW %s TO %s AS %s`,
-			viewName, c.distributedTable("flows_raw_errors"), selectQuery)); err != nil {
-		return fmt.Errorf("cannot create raw flows errors view: %w", err)
-	}
-
-	return nil
-}
-
-func (c *Component) deleteOldRawFlowsErrorsView(ctx context.Context) error {
-	tableName := fmt.Sprintf("flows_%s_raw", c.d.Schema.ProtobufMessageHash())
-	viewName := fmt.Sprintf("%s_errors", tableName)
-
-	// Check the existing one
-	if ok, err := c.tableAlreadyExists(ctx, viewName, "name", viewName); err != nil {
-		return err
-	} else if !ok {
-		c.r.Debug().Msg("old raw flows errors view does not exist, skip migration")
-		return errSkipStep
-	}
-
-	// Drop
-	c.r.Info().Msg("delete old raw flows errors view")
-	if err := c.d.ClickHouse.ExecOnCluster(ctx, fmt.Sprintf(`DROP TABLE IF EXISTS %s SYNC`, viewName)); err != nil {
-		return fmt.Errorf("cannot drop table %s: %w", viewName, err)
-	}
-	return nil
-}
-
 func (c *Component) createOrUpdateFlowsTable(ctx context.Context, resolution ResolutionConfiguration) error {
 	ctx = clickhouse.Context(ctx, clickhouse.WithSettings(clickhouse.Settings{
 		"allow_suspicious_low_cardinality_types": 1,
diff --git a/orchestrator/config.go b/orchestrator/config.go
index de872875..3c344504 100644
--- a/orchestrator/config.go
+++ b/orchestrator/config.go
@@ -3,10 +3,10 @@
 
 package orchestrator
 
-// Configuration describes the configuration for the broker.
+// Configuration describes the configuration for the orchestrator.
 type Configuration struct{}
 
-// DefaultConfiguration represents the default configuration for the broker.
+// DefaultConfiguration represents the default configuration for the orchestrator.
 func DefaultConfiguration() Configuration {
 	return Configuration{}
 }
diff --git a/orchestrator/kafka/root.go b/orchestrator/kafka/root.go
index d6264e34..aeeb76bf 100644
--- a/orchestrator/kafka/root.go
+++ b/orchestrator/kafka/root.go
@@ -5,12 +5,16 @@
 package kafka
 
 import (
+	"context"
 	"fmt"
 	"strings"
 
-	"github.com/IBM/sarama"
+	"github.com/twmb/franz-go/pkg/kadm"
+	"github.com/twmb/franz-go/pkg/kgo"
+	"github.com/twmb/franz-go/pkg/kmsg"
 
 	"akvorado/common/kafka"
+	"akvorado/common/pb"
 	"akvorado/common/reporter"
 	"akvorado/common/schema"
 )
@@ -21,8 +25,8 @@ type Component struct {
 	d      Dependencies
 	config Configuration
 
-	kafkaConfig *sarama.Config
-	kafkaTopic  string
+	kafkaOpts  []kgo.Opt
+	kafkaTopic string
 }
 
 // Dependencies are the dependencies for the Kafka component
@@ -32,81 +36,101 @@ type Dependencies struct {
 
 // New creates a new Kafka configurator.
 func New(r *reporter.Reporter, config Configuration, dependencies Dependencies) (*Component, error) {
-	kafkaConfig, err := kafka.NewConfig(config.Configuration)
+	kafkaOpts, err := kafka.NewConfig(r, config.Configuration)
 	if err != nil {
 		return nil, err
 	}
-	if err := kafkaConfig.Validate(); err != nil {
-		return nil, fmt.Errorf("cannot validate Kafka configuration: %w", err)
-	}
 
-	return &Component{
+	c := Component{
 		r:      r,
 		d:      dependencies,
 		config: config,
 
-		kafkaConfig: kafkaConfig,
-		kafkaTopic:  fmt.Sprintf("%s-%s", config.Topic, dependencies.Schema.ProtobufMessageHash()),
-	}, nil
+		kafkaOpts:  kafkaOpts,
+		kafkaTopic: fmt.Sprintf("%s-v%d", config.Topic, pb.Version),
+	}
+	return &c, nil
 }
 
 // Start starts Kafka configuration.
 func (c *Component) Start() error {
 	c.r.Info().Msg("starting Kafka component")
-	kafka.GlobalKafkaLogger.Register(c.r)
-	defer func() {
-		kafka.GlobalKafkaLogger.Unregister()
-		c.r.Info().Msg("Kafka component stopped")
-	}()
+	defer c.r.Info().Msg("Kafka component stopped")
 
-	// Create topic
-	admin, err := sarama.NewClusterAdmin(c.config.Brokers, c.kafkaConfig)
+	// Create kafka client and admin
+	client, err := kgo.NewClient(c.kafkaOpts...)
 	if err != nil {
 		c.r.Err(err).
 			Str("brokers", strings.Join(c.config.Brokers, ",")).
-			Msg("unable to get admin client for topic creation")
-		return fmt.Errorf("unable to get admin client for topic creation: %w", err)
+			Msg("unable to create Kafka client for topic creation")
+		return fmt.Errorf("unable to create Kafka client for topic creation: %w", err)
 	}
-	defer admin.Close()
+	defer client.Close()
+	admin := kadm.NewClient(client)
 	l := c.r.With().
 		Str("brokers", strings.Join(c.config.Brokers, ",")).
 		Str("topic", c.kafkaTopic).
 		Logger()
-	topics, err := admin.ListTopics()
+	topics, err := admin.ListTopics(context.Background())
 	if err != nil {
 		l.Err(err).Msg("unable to get metadata for topics")
 		return fmt.Errorf("unable to get metadata for topics: %w", err)
 	}
 	if topic, ok := topics[c.kafkaTopic]; !ok {
-		if err := admin.CreateTopic(c.kafkaTopic,
-			&sarama.TopicDetail{
-				NumPartitions:     c.config.TopicConfiguration.NumPartitions,
-				ReplicationFactor: c.config.TopicConfiguration.ReplicationFactor,
-				ConfigEntries:     c.config.TopicConfiguration.ConfigEntries,
-			}, false); err != nil {
+		if _, err := admin.CreateTopics(context.Background(), c.config.TopicConfiguration.NumPartitions, c.config.TopicConfiguration.ReplicationFactor, c.config.TopicConfiguration.ConfigEntries, c.kafkaTopic); err != nil {
 			l.Err(err).Msg("unable to create topic")
 			return fmt.Errorf("unable to create topic %q: %w", c.kafkaTopic, err)
 		}
 		l.Info().Msg("topic created")
 	} else {
-		if topic.NumPartitions > c.config.TopicConfiguration.NumPartitions {
+		nbPartitions := len(topic.Partitions)
+		if nbPartitions > int(c.config.TopicConfiguration.NumPartitions) {
 			l.Warn().Msgf("cannot decrease the number of partitions (from %d to %d)",
-				topic.NumPartitions, c.config.TopicConfiguration.NumPartitions)
-		} else if topic.NumPartitions < c.config.TopicConfiguration.NumPartitions {
-			nb := c.config.TopicConfiguration.NumPartitions
-			if err := admin.CreatePartitions(c.kafkaTopic, nb, nil, false); err != nil {
+				nbPartitions, c.config.TopicConfiguration.NumPartitions)
+		} else if nbPartitions < int(c.config.TopicConfiguration.NumPartitions) {
+			add := int(c.config.TopicConfiguration.NumPartitions) - nbPartitions
+			if _, err := admin.CreatePartitions(context.Background(), add, c.kafkaTopic); err != nil {
 				l.Err(err).Msg("unable to add more partitions")
 				return fmt.Errorf("unable to add more partitions to topic %q: %w",
 					c.kafkaTopic, err)
 			}
+			l.Info().Msg("number of partitions increased")
 		}
-		if c.config.TopicConfiguration.ReplicationFactor != topic.ReplicationFactor {
+		if int(c.config.TopicConfiguration.ReplicationFactor) != topic.Partitions.NumReplicas() {
 			// TODO: https://github.com/deviceinsight/kafkactl/blob/main/internal/topic/topic-operation.go
 			l.Warn().Msgf("mismatch for replication factor: got %d, want %d",
-				topic.ReplicationFactor, c.config.TopicConfiguration.ReplicationFactor)
+				topic.Partitions.NumReplicas(), c.config.TopicConfiguration.ReplicationFactor)
 		}
-		if ShouldAlterConfiguration(c.config.TopicConfiguration.ConfigEntries, topic.ConfigEntries, c.config.TopicConfiguration.ConfigEntriesStrictSync) {
-			if err := admin.AlterConfig(sarama.TopicResource, c.kafkaTopic, c.config.TopicConfiguration.ConfigEntries, false); err != nil {
+		configs, err := admin.DescribeTopicConfigs(context.Background(), c.kafkaTopic)
+		if err != nil || len(configs) != 1 {
+			l.Err(err).Msg("unable to get topic configuration")
+			return fmt.Errorf("unable to get topic %q configuration: %w", c.kafkaTopic, err)
+		}
+		got := map[string]*string{}
+		for _, config := range configs[0].Configs {
+			if config.Source == kmsg.ConfigSourceDynamicTopicConfig {
+				got[config.Key] = config.Value
+			}
+		}
+		if ShouldAlterConfiguration(c.config.TopicConfiguration.ConfigEntries, got, c.config.TopicConfiguration.ConfigEntriesStrictSync) {
+			alterConfigs := []kadm.AlterConfig{}
+			for k, v := range c.config.TopicConfiguration.ConfigEntries {
+				alterConfigs = append(alterConfigs, kadm.AlterConfig{
+					Op:    kadm.SetConfig,
+					Name:  k,
+					Value: v,
+				})
+			}
+			for k, v := range got {
+				if _, ok := c.config.TopicConfiguration.ConfigEntries[k]; !ok {
+					alterConfigs = append(alterConfigs, kadm.AlterConfig{
+						Op:    kadm.DeleteConfig,
+						Name:  k,
+						Value: v,
+					})
+				}
+			}
+			if _, err := admin.AlterTopicConfigs(context.Background(), alterConfigs, c.kafkaTopic); err != nil {
 				l.Err(err).Msg("unable to set topic configuration")
 				return fmt.Errorf("unable to set topic configuration for %q: %w",
 					c.kafkaTopic, err)
diff --git a/orchestrator/root.go b/orchestrator/root.go
index 90dd67aa..7a6b6cc1 100644
--- a/orchestrator/root.go
+++ b/orchestrator/root.go
@@ -32,6 +32,8 @@ type ServiceType string
 var (
 	// InletService represents the inlet service type
 	InletService ServiceType = "inlet"
+	// OutletService represents the outlet service type
+	OutletService ServiceType = "outlet"
 	// OrchestratorService represents the orchestrator service type
 	OrchestratorService ServiceType = "orchestrator"
 	// ConsoleService represents the console service type
diff --git a/outlet/clickhouse/config.go b/outlet/clickhouse/config.go
new file mode 100644
index 00000000..6bdb7c56
--- /dev/null
+++ b/outlet/clickhouse/config.go
@@ -0,0 +1,24 @@
+// SPDX-FileCopyrightText: 2025 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+package clickhouse
+
+import (
+	"time"
+)
+
+// Configuration describes the configuration for the ClickHouse exporter.
+type Configuration struct {
+	// MaximumBatchSize is the maximum number of rows to send to ClickHouse in one batch.
+	MaximumBatchSize uint `validate:"min=1"`
+	// MaximumWaitTime is the maximum number of seconds to wait before sending the current batch.
+	MaximumWaitTime time.Duration `validate:"min=100ms"`
+}
+
+// DefaultConfiguration represents the default configuration for the ClickHouse exporter.
+func DefaultConfiguration() Configuration {
+	return Configuration{
+		MaximumBatchSize: 5000,
+		MaximumWaitTime:  time.Second,
+	}
+}
diff --git a/outlet/clickhouse/metrics.go b/outlet/clickhouse/metrics.go
new file mode 100644
index 00000000..a9bcfc24
--- /dev/null
+++ b/outlet/clickhouse/metrics.go
@@ -0,0 +1,48 @@
+// SPDX-FileCopyrightText: 2025 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+package clickhouse
+
+import "akvorado/common/reporter"
+
+type metrics struct {
+	batches    reporter.Counter
+	flows      reporter.Counter
+	waitTime   reporter.Histogram
+	insertTime reporter.Histogram
+	errors     *reporter.CounterVec
+}
+
+func (c *realComponent) initMetrics() {
+	c.metrics.batches = c.r.Counter(
+		reporter.CounterOpts{
+			Name: "batches_total",
+			Help: "Number of batches of flows sent to ClickHouse",
+		},
+	)
+	c.metrics.flows = c.r.Counter(
+		reporter.CounterOpts{
+			Name: "flows_total",
+			Help: "Number of flows sent to ClickHouse",
+		},
+	)
+	c.metrics.waitTime = c.r.Histogram(
+		reporter.HistogramOpts{
+			Name: "wait_time_seconds",
+			Help: "Time spent waiting before sending a batch to ClickHouse",
+		},
+	)
+	c.metrics.insertTime = c.r.Histogram(
+		reporter.HistogramOpts{
+			Name: "insert_time_seconds",
+			Help: "Time spent inserting data to ClickHouse",
+		},
+	)
+	c.metrics.errors = c.r.CounterVec(
+		reporter.CounterOpts{
+			Name: "errors_total",
+			Help: "Errors while inserting into ClickHouse",
+		},
+		[]string{"error"},
+	)
+}
diff --git a/outlet/clickhouse/root.go b/outlet/clickhouse/root.go
new file mode 100644
index 00000000..beeb6c78
--- /dev/null
+++ b/outlet/clickhouse/root.go
@@ -0,0 +1,44 @@
+// SPDX-FileCopyrightText: 2025 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+// Package clickhouse handles flow exports to ClickHouse. This component is
+// "inert" and does not track its spawned workers. It is the responsability of
+// the dependent component to flush data before shutting down.
+package clickhouse
+
+import (
+	"akvorado/common/clickhousedb"
+	"akvorado/common/reporter"
+	"akvorado/common/schema"
+)
+
+// Component is the interface for the ClickHouse exporter component.
+type Component interface {
+	NewWorker(int, *schema.FlowMessage) Worker
+}
+
+// realComponent implements the ClickHouse exporter
+type realComponent struct {
+	r      *reporter.Reporter
+	d      *Dependencies
+	config Configuration
+
+	metrics metrics
+}
+
+// Dependencies defines the dependencies of the ClickHouse exporter
+type Dependencies struct {
+	ClickHouse *clickhousedb.Component
+	Schema     *schema.Component
+}
+
+// New creates a new core component.
+func New(r *reporter.Reporter, configuration Configuration, dependencies Dependencies) (Component, error) {
+	c := realComponent{
+		r:      r,
+		d:      &dependencies,
+		config: configuration,
+	}
+	c.initMetrics()
+	return &c, nil
+}
diff --git a/outlet/clickhouse/worker.go b/outlet/clickhouse/worker.go
new file mode 100644
index 00000000..c543a1c8
--- /dev/null
+++ b/outlet/clickhouse/worker.go
@@ -0,0 +1,146 @@
+// SPDX-FileCopyrightText: 2025 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+package clickhouse
+
+import (
+	"context"
+	"fmt"
+	"math/rand"
+	"time"
+
+	"github.com/ClickHouse/ch-go"
+	"github.com/cenkalti/backoff/v4"
+
+	"akvorado/common/reporter"
+	"akvorado/common/schema"
+)
+
+// Worker represents a worker sending to ClickHouse. It is synchronous (no
+// goroutines) and most functions are bound to a context.
+type Worker interface {
+	FinalizeAndSend(context.Context)
+	Flush(context.Context)
+}
+
+// realWorker is a working implementation of Worker.
+type realWorker struct {
+	c      *realComponent
+	bf     *schema.FlowMessage
+	last   time.Time
+	logger reporter.Logger
+
+	conn    *ch.Client
+	servers []string
+	options ch.Options
+}
+
+// NewWorker creates a new worker to push data to ClickHouse.
+func (c *realComponent) NewWorker(i int, bf *schema.FlowMessage) Worker {
+	opts, servers := c.d.ClickHouse.ChGoOptions()
+	w := realWorker{
+		c:      c,
+		bf:     bf,
+		logger: c.r.With().Int("worker", i).Logger(),
+
+		servers: servers,
+		options: opts,
+	}
+	return &w
+}
+
+// FinalizeAndSend sends data to ClickHouse after finalizing if we have a full batch or exceeded the maximum wait time.
+func (w *realWorker) FinalizeAndSend(ctx context.Context) {
+	w.bf.Finalize()
+	now := time.Now()
+	if w.bf.FlowCount() >= int(w.c.config.MaximumBatchSize) || w.last.Add(w.c.config.MaximumWaitTime).Before(now) {
+		// Record wait time since last send
+		if !w.last.IsZero() {
+			waitTime := now.Sub(w.last)
+			w.c.metrics.waitTime.Observe(waitTime.Seconds())
+		}
+		w.Flush(ctx)
+		w.last = now
+	}
+}
+
+// Flush sends remaining data to ClickHouse without an additional condition. It
+// should be called before shutting down to flush remaining data. Otherwise,
+// FinalizeAndSend() should be used instead.
+func (w *realWorker) Flush(ctx context.Context) {
+	// We try to send as long as possible. The only exit condition is an
+	// expiration of the context.
+	b := backoff.NewExponentialBackOff()
+	b.MaxElapsedTime = 0
+	b.MaxInterval = 30 * time.Second
+	b.InitialInterval = 20 * time.Millisecond
+	backoff.Retry(func() error {
+		// Connect or reconnect if connection is broken.
+		if err := w.connect(ctx); err != nil {
+			w.logger.Err(err).Msg("cannot connect to ClickHouse")
+			return err
+		}
+
+		// Send to ClickHouse in flows_XXXXX_raw.
+		start := time.Now()
+		if err := w.conn.Do(ctx, ch.Query{
+			Body:  w.bf.ClickHouseProtoInput().Into(fmt.Sprintf("flows_%s_raw", w.c.d.Schema.ClickHouseHash())),
+			Input: w.bf.ClickHouseProtoInput(),
+		}); err != nil {
+			w.logger.Err(err).Msg("cannot send batch to ClickHouse")
+			w.c.metrics.errors.WithLabelValues("send").Inc()
+			return err
+		}
+		pushDuration := time.Since(start)
+		w.c.metrics.insertTime.Observe(pushDuration.Seconds())
+		w.c.metrics.batches.Inc()
+		w.c.metrics.flows.Add(float64(w.bf.FlowCount()))
+
+		// Clear batch
+		w.bf.Clear()
+		return nil
+	}, backoff.WithContext(b, ctx))
+}
+
+// connect establishes or reestablish the connection to ClickHouse.
+func (w *realWorker) connect(ctx context.Context) error {
+	// If connection exists and is healthy, reuse it
+	if w.conn != nil {
+		if err := w.conn.Ping(ctx); err == nil {
+			return nil
+		}
+		// Connection is unhealthy, close it
+		w.conn.Close()
+		w.conn = nil
+	}
+
+	// Try each server until one connects successfully
+	var lastErr error
+	for _, idx := range rand.Perm(len(w.servers)) {
+		w.options.Address = w.servers[idx]
+		conn, err := ch.Dial(ctx, w.options)
+		if err != nil {
+			w.logger.Err(err).Str("server", w.options.Address).Msg("failed to connect to ClickHouse server")
+			w.c.metrics.errors.WithLabelValues("connect").Inc()
+			lastErr = err
+			continue
+		}
+
+		// Test the connection
+		if err := conn.Ping(ctx); err != nil {
+			w.logger.Err(err).Str("server", w.options.Address).Msg("ClickHouse server ping failed")
+			w.c.metrics.errors.WithLabelValues("ping").Inc()
+			conn.Close()
+			conn = nil
+			lastErr = err
+			continue
+		}
+
+		// Success
+		w.conn = conn
+		w.logger.Info().Str("server", w.options.Address).Msg("connected to ClickHouse server")
+		return nil
+	}
+
+	return lastErr
+}
diff --git a/inlet/core/classifier.go b/outlet/core/classifier.go
similarity index 100%
rename from inlet/core/classifier.go
rename to outlet/core/classifier.go
diff --git a/inlet/core/config.go b/outlet/core/config.go
similarity index 98%
rename from inlet/core/config.go
rename to outlet/core/config.go
index d857e8b8..172289a4 100644
--- a/inlet/core/config.go
+++ b/outlet/core/config.go
@@ -16,8 +16,6 @@ import (
 
 // Configuration describes the configuration for the core component.
 type Configuration struct {
-	// Number of workers for the core component
-	Workers int `validate:"min=1"`
 	// ExporterClassifiers defines rules for exporter classification
 	ExporterClassifiers []ExporterClassifierRule
 	// InterfaceClassifiers defines rules for interface classification
@@ -39,7 +37,6 @@ type Configuration struct {
 // DefaultConfiguration represents the default configuration for the core component.
 func DefaultConfiguration() Configuration {
 	return Configuration{
-		Workers:                 1,
 		ExporterClassifiers:     []ExporterClassifierRule{},
 		InterfaceClassifiers:    []InterfaceClassifierRule{},
 		ClassifierCacheDuration: 5 * time.Minute,
@@ -154,4 +151,5 @@ func init() {
 	helpers.RegisterMapstructureUnmarshallerHook(ASNProviderUnmarshallerHook())
 	helpers.RegisterMapstructureUnmarshallerHook(NetProviderUnmarshallerHook())
 	helpers.RegisterMapstructureUnmarshallerHook(helpers.SubnetMapUnmarshallerHook[uint]())
+	helpers.RegisterMapstructureDeprecatedFields[Configuration]("Workers")
 }
diff --git a/inlet/core/enricher.go b/outlet/core/enricher.go
similarity index 75%
rename from inlet/core/enricher.go
rename to outlet/core/enricher.go
index fb0bc2c6..d5338524 100644
--- a/inlet/core/enricher.go
+++ b/outlet/core/enricher.go
@@ -19,7 +19,7 @@ type exporterAndInterfaceInfo struct {
 }
 
 // enrichFlow adds more data to a flow.
-func (c *Component) enrichFlow(exporterIP netip.Addr, exporterStr string, flow *schema.FlowMessage) (skip bool) {
+func (w *worker) enrichFlow(exporterIP netip.Addr, exporterStr string) (skip bool) {
 	var flowExporterName string
 	var flowInIfName, flowInIfDescription, flowOutIfName, flowOutIfDescription string
 	var flowInIfSpeed, flowOutIfSpeed, flowInIfIndex, flowOutIfIndex uint32
@@ -30,10 +30,13 @@ func (c *Component) enrichFlow(exporterIP netip.Addr, exporterStr string, flow *
 	inIfClassification := interfaceClassification{}
 	outIfClassification := interfaceClassification{}
 
+	flow := w.bf
+	c := w.c
+
 	if flow.InIf != 0 {
-		answer, ok := c.d.Metadata.Lookup(t, exporterIP, uint(flow.InIf))
-		if !ok {
-			c.metrics.flowsErrors.WithLabelValues(exporterStr, "SNMP cache miss").Inc()
+		answer := c.d.Metadata.Lookup(t, exporterIP, uint(flow.InIf))
+		if !answer.Found {
+			c.metrics.flowsErrors.WithLabelValues(exporterStr, "metadata cache miss").Inc()
 			skip = true
 		} else {
 			flowExporterName = answer.Exporter.Name
@@ -54,12 +57,12 @@ func (c *Component) enrichFlow(exporterIP netip.Addr, exporterStr string, flow *
 	}
 
 	if flow.OutIf != 0 {
-		answer, ok := c.d.Metadata.Lookup(t, exporterIP, uint(flow.OutIf))
-		if !ok {
+		answer := c.d.Metadata.Lookup(t, exporterIP, uint(flow.OutIf))
+		if !answer.Found {
 			// Only register a cache miss if we don't have one.
 			// TODO: maybe we could do one SNMP query for both interfaces.
 			if !skip {
-				c.metrics.flowsErrors.WithLabelValues(exporterStr, "SNMP cache miss").Inc()
+				c.metrics.flowsErrors.WithLabelValues(exporterStr, "metadata cache miss").Inc()
 				skip = true
 			}
 		} else {
@@ -87,11 +90,11 @@ func (c *Component) enrichFlow(exporterIP netip.Addr, exporterStr string, flow *
 	}
 
 	if samplingRate, ok := c.config.OverrideSamplingRate.Lookup(exporterIP); ok && samplingRate > 0 {
-		flow.SamplingRate = uint32(samplingRate)
+		flow.SamplingRate = uint64(samplingRate)
 	}
 	if flow.SamplingRate == 0 {
 		if samplingRate, ok := c.config.DefaultSamplingRate.Lookup(exporterIP); ok && samplingRate > 0 {
-			flow.SamplingRate = uint32(samplingRate)
+			flow.SamplingRate = uint64(samplingRate)
 		} else {
 			c.metrics.flowsErrors.WithLabelValues(exporterStr, "sampling rate missing").Inc()
 			skip = true
@@ -128,28 +131,22 @@ func (c *Component) enrichFlow(exporterIP netip.Addr, exporterStr string, flow *
 	// set asns according to user config
 	flow.SrcAS = c.getASNumber(flow.SrcAS, sourceRouting.ASN)
 	flow.DstAS = c.getASNumber(flow.DstAS, destRouting.ASN)
-	if !flow.GotCommunities {
-		for _, comm := range destRouting.Communities {
-			c.d.Schema.ProtobufAppendVarint(flow, schema.ColumnDstCommunities, uint64(comm))
-		}
-	}
-	if !flow.GotASPath {
-		for _, asn := range destRouting.ASPath {
-			c.d.Schema.ProtobufAppendVarint(flow, schema.ColumnDstASPath, uint64(asn))
+	flow.AppendArrayUInt32(schema.ColumnDstCommunities, destRouting.Communities)
+	flow.AppendArrayUInt32(schema.ColumnDstASPath, destRouting.ASPath)
+	if len(destRouting.LargeCommunities) > 0 {
+		communities := make([]schema.UInt128, len(destRouting.LargeCommunities))
+		for i, comm := range destRouting.LargeCommunities {
+			communities[i] = schema.UInt128{
+				High: uint64(comm.ASN),
+				Low:  (uint64(comm.LocalData1) << 32) + uint64(comm.LocalData2),
+			}
 		}
-	}
-	for _, comm := range destRouting.LargeCommunities {
-		c.d.Schema.ProtobufAppendVarintForce(flow,
-			schema.ColumnDstLargeCommunitiesASN, uint64(comm.ASN))
-		c.d.Schema.ProtobufAppendVarintForce(flow,
-			schema.ColumnDstLargeCommunitiesLocalData1, uint64(comm.LocalData1))
-		c.d.Schema.ProtobufAppendVarintForce(flow,
-			schema.ColumnDstLargeCommunitiesLocalData2, uint64(comm.LocalData2))
+		flow.AppendArrayUInt128(schema.ColumnDstLargeCommunities, communities)
 	}
 
-	c.d.Schema.ProtobufAppendBytes(flow, schema.ColumnExporterName, []byte(flowExporterName))
-	c.d.Schema.ProtobufAppendVarint(flow, schema.ColumnInIfSpeed, uint64(flowInIfSpeed))
-	c.d.Schema.ProtobufAppendVarint(flow, schema.ColumnOutIfSpeed, uint64(flowOutIfSpeed))
+	flow.AppendString(schema.ColumnExporterName, flowExporterName)
+	flow.AppendUint(schema.ColumnInIfSpeed, uint64(flowInIfSpeed))
+	flow.AppendUint(schema.ColumnOutIfSpeed, uint64(flowOutIfSpeed))
 
 	return
 }
@@ -219,11 +216,11 @@ func (c *Component) writeExporter(flow *schema.FlowMessage, classification expor
 	if classification.Reject {
 		return false
 	}
-	c.d.Schema.ProtobufAppendBytes(flow, schema.ColumnExporterGroup, []byte(classification.Group))
-	c.d.Schema.ProtobufAppendBytes(flow, schema.ColumnExporterRole, []byte(classification.Role))
-	c.d.Schema.ProtobufAppendBytes(flow, schema.ColumnExporterSite, []byte(classification.Site))
-	c.d.Schema.ProtobufAppendBytes(flow, schema.ColumnExporterRegion, []byte(classification.Region))
-	c.d.Schema.ProtobufAppendBytes(flow, schema.ColumnExporterTenant, []byte(classification.Tenant))
+	flow.AppendString(schema.ColumnExporterGroup, classification.Group)
+	flow.AppendString(schema.ColumnExporterRole, classification.Role)
+	flow.AppendString(schema.ColumnExporterSite, classification.Site)
+	flow.AppendString(schema.ColumnExporterRegion, classification.Region)
+	flow.AppendString(schema.ColumnExporterTenant, classification.Tenant)
 	return true
 }
 
@@ -264,17 +261,17 @@ func (c *Component) writeInterface(flow *schema.FlowMessage, classification inte
 		return false
 	}
 	if directionIn {
-		c.d.Schema.ProtobufAppendBytes(flow, schema.ColumnInIfName, []byte(classification.Name))
-		c.d.Schema.ProtobufAppendBytes(flow, schema.ColumnInIfDescription, []byte(classification.Description))
-		c.d.Schema.ProtobufAppendBytes(flow, schema.ColumnInIfConnectivity, []byte(classification.Connectivity))
-		c.d.Schema.ProtobufAppendBytes(flow, schema.ColumnInIfProvider, []byte(classification.Provider))
-		c.d.Schema.ProtobufAppendVarint(flow, schema.ColumnInIfBoundary, uint64(classification.Boundary))
+		flow.AppendString(schema.ColumnInIfName, classification.Name)
+		flow.AppendString(schema.ColumnInIfDescription, classification.Description)
+		flow.AppendString(schema.ColumnInIfConnectivity, classification.Connectivity)
+		flow.AppendString(schema.ColumnInIfProvider, classification.Provider)
+		flow.AppendUint(schema.ColumnInIfBoundary, uint64(classification.Boundary))
 	} else {
-		c.d.Schema.ProtobufAppendBytes(flow, schema.ColumnOutIfName, []byte(classification.Name))
-		c.d.Schema.ProtobufAppendBytes(flow, schema.ColumnOutIfDescription, []byte(classification.Description))
-		c.d.Schema.ProtobufAppendBytes(flow, schema.ColumnOutIfConnectivity, []byte(classification.Connectivity))
-		c.d.Schema.ProtobufAppendBytes(flow, schema.ColumnOutIfProvider, []byte(classification.Provider))
-		c.d.Schema.ProtobufAppendVarint(flow, schema.ColumnOutIfBoundary, uint64(classification.Boundary))
+		flow.AppendString(schema.ColumnOutIfName, classification.Name)
+		flow.AppendString(schema.ColumnOutIfDescription, classification.Description)
+		flow.AppendString(schema.ColumnOutIfConnectivity, classification.Connectivity)
+		flow.AppendString(schema.ColumnOutIfProvider, classification.Provider)
+		flow.AppendUint(schema.ColumnOutIfBoundary, uint64(classification.Boundary))
 	}
 	return true
 }
diff --git a/inlet/core/http.go b/outlet/core/http.go
similarity index 69%
rename from inlet/core/http.go
rename to outlet/core/http.go
index 875e65d7..576eeab7 100644
--- a/inlet/core/http.go
+++ b/outlet/core/http.go
@@ -18,7 +18,7 @@ type flowsParameters struct {
 }
 
 // FlowsHTTPHandler streams a JSON copy of all flows just after
-// sending them to Kafka. Under load, some flows may not be sent. This
+// sending them to ClickHouse. Under load, some flows may not be sent. This
 // is intended for debug only.
 func (c *Component) FlowsHTTPHandler(gc *gin.Context) {
 	var params flowsParameters
@@ -27,7 +27,7 @@ func (c *Component) FlowsHTTPHandler(gc *gin.Context) {
 		gc.JSON(http.StatusBadRequest, gin.H{"message": helpers.Capitalize(err.Error())})
 		return
 	}
-	format := gc.NegotiateFormat("application/json", "application/x-protobuf")
+	dying := c.t.Dying()
 
 	atomic.AddUint32(&c.httpFlowClients, 1)
 	defer atomic.AddUint32(&c.httpFlowClients, ^uint32(0))
@@ -40,23 +40,15 @@ func (c *Component) FlowsHTTPHandler(gc *gin.Context) {
 
 	for {
 		select {
-		case <-c.t.Dying():
+		case <-dying:
 			return
 		case <-gc.Request.Context().Done():
 			return
 		case msg := <-c.httpFlowChannel:
-			switch format {
-			case "application/json":
-				if params.Limit == 1 {
-					gc.IndentedJSON(http.StatusOK, msg)
-				} else {
-					gc.JSON(http.StatusOK, msg)
-					gc.Writer.Write([]byte("\n"))
-				}
-			case "application/x-protobuf":
-				gc.Set("Content-Type", format)
-				gc.Writer.Write(msg.Bytes())
-			}
+			gc.Header("Content-Type", "application/json")
+			gc.Status(http.StatusOK)
+			gc.Writer.Write(msg)
+			gc.Writer.Write([]byte("\n"))
 
 			count++
 			if params.Limit > 0 && count == params.Limit {
diff --git a/inlet/core/metrics.go b/outlet/core/metrics.go
similarity index 79%
rename from inlet/core/metrics.go
rename to outlet/core/metrics.go
index 17547016..a383e0ca 100644
--- a/inlet/core/metrics.go
+++ b/outlet/core/metrics.go
@@ -10,6 +10,8 @@ import (
 )
 
 type metrics struct {
+	rawFlowsReceived reporter.Counter
+	rawFlowsErrors   *reporter.CounterVec
 	flowsReceived    *reporter.CounterVec
 	flowsForwarded   *reporter.CounterVec
 	flowsErrors      *reporter.CounterVec
@@ -21,6 +23,19 @@ type metrics struct {
 }
 
 func (c *Component) initMetrics() {
+	c.metrics.rawFlowsReceived = c.r.Counter(
+		reporter.CounterOpts{
+			Name: "received_raw_flows_total",
+			Help: "Number of incoming raw flows (proto).",
+		},
+	)
+	c.metrics.rawFlowsErrors = c.r.CounterVec(
+		reporter.CounterOpts{
+			Name: "raw_flows_errors_total",
+			Help: "Number of raw flows with errors.",
+		},
+		[]string{"error"},
+	)
 	c.metrics.flowsReceived = c.r.CounterVec(
 		reporter.CounterOpts{
 			Name: "received_flows_total",
@@ -54,7 +69,7 @@ func (c *Component) initMetrics() {
 
 	c.metrics.classifierExporterCacheSize = c.r.CounterFunc(
 		reporter.CounterOpts{
-			Name: "classifier_exporter_cache_size_items",
+			Name: "classifier_exporter_cache_items_total",
 			Help: "Number of items in the exporter classifier cache",
 		},
 		func() float64 {
@@ -63,7 +78,7 @@ func (c *Component) initMetrics() {
 	)
 	c.metrics.classifierInterfaceCacheSize = c.r.CounterFunc(
 		reporter.CounterOpts{
-			Name: "classifier_interface_cache_size_items",
+			Name: "classifier_interface_cache_items_total",
 			Help: "Number of items in the interface classifier cache",
 		},
 		func() float64 {
diff --git a/outlet/core/release.go b/outlet/core/release.go
new file mode 100644
index 00000000..6a425ee6
--- /dev/null
+++ b/outlet/core/release.go
@@ -0,0 +1,8 @@
+// SPDX-FileCopyrightText: 2025 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+//go:build release
+
+package core
+
+const debug = false
diff --git a/outlet/core/root.go b/outlet/core/root.go
new file mode 100644
index 00000000..be72a124
--- /dev/null
+++ b/outlet/core/root.go
@@ -0,0 +1,106 @@
+// SPDX-FileCopyrightText: 2022 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+// Package core plumbs all the other components together.
+package core
+
+import (
+	"time"
+
+	"gopkg.in/tomb.v2"
+
+	"akvorado/common/daemon"
+	"akvorado/common/helpers/cache"
+	"akvorado/common/httpserver"
+	"akvorado/common/reporter"
+	"akvorado/common/schema"
+	"akvorado/outlet/clickhouse"
+	"akvorado/outlet/flow"
+	"akvorado/outlet/kafka"
+	"akvorado/outlet/metadata"
+	"akvorado/outlet/routing"
+)
+
+// Component represents the HTTP compomenent.
+type Component struct {
+	r      *reporter.Reporter
+	d      *Dependencies
+	t      tomb.Tomb
+	config Configuration
+
+	metrics metrics
+
+	httpFlowClients    uint32 // for dumping flows
+	httpFlowChannel    chan []byte
+	httpFlowFlushDelay time.Duration
+
+	classifierExporterCache  *cache.Cache[exporterInfo, exporterClassification]
+	classifierInterfaceCache *cache.Cache[exporterAndInterfaceInfo, interfaceClassification]
+	classifierErrLogger      reporter.Logger
+}
+
+// Dependencies define the dependencies of the HTTP component.
+type Dependencies struct {
+	Daemon     daemon.Component
+	Flow       *flow.Component
+	Metadata   *metadata.Component
+	Routing    *routing.Component
+	Kafka      kafka.Component
+	ClickHouse clickhouse.Component
+	HTTP       *httpserver.Component
+	Schema     *schema.Component
+}
+
+// New creates a new core component.
+func New(r *reporter.Reporter, configuration Configuration, dependencies Dependencies) (*Component, error) {
+	c := Component{
+		r:      r,
+		d:      &dependencies,
+		config: configuration,
+
+		httpFlowClients:    0,
+		httpFlowChannel:    make(chan []byte, 10),
+		httpFlowFlushDelay: time.Second,
+
+		classifierExporterCache:  cache.New[exporterInfo, exporterClassification](),
+		classifierInterfaceCache: cache.New[exporterAndInterfaceInfo, interfaceClassification](),
+		classifierErrLogger:      r.Sample(reporter.BurstSampler(10*time.Second, 3)),
+	}
+	c.d.Daemon.Track(&c.t, "outlet/core")
+	c.initMetrics()
+	return &c, nil
+}
+
+// Start starts the core component.
+func (c *Component) Start() error {
+	c.r.Info().Msg("starting core component")
+	c.d.Kafka.StartWorkers(c.newWorker)
+
+	// Classifier cache expiration
+	c.t.Go(func() error {
+		for {
+			select {
+			case <-c.t.Dying():
+				return nil
+			case <-time.After(c.config.ClassifierCacheDuration):
+				before := time.Now().Add(-c.config.ClassifierCacheDuration)
+				c.classifierExporterCache.DeleteLastAccessedBefore(before)
+				c.classifierInterfaceCache.DeleteLastAccessedBefore(before)
+			}
+		}
+	})
+
+	c.d.HTTP.GinRouter.GET("/api/v0/outlet/flows", c.FlowsHTTPHandler)
+	return nil
+}
+
+// Stop stops the core component.
+func (c *Component) Stop() error {
+	defer func() {
+		close(c.httpFlowChannel)
+		c.r.Info().Msg("core component stopped")
+	}()
+	c.r.Info().Msg("stopping core component")
+	c.t.Kill(nil)
+	return c.t.Wait()
+}
diff --git a/outlet/core/worker.go b/outlet/core/worker.go
new file mode 100644
index 00000000..f62eb19b
--- /dev/null
+++ b/outlet/core/worker.go
@@ -0,0 +1,101 @@
+// SPDX-FileCopyrightText: 2025 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+package core
+
+import (
+	"context"
+	"encoding/json"
+	"fmt"
+	"sync/atomic"
+	"time"
+
+	"akvorado/common/pb"
+	"akvorado/common/reporter"
+	"akvorado/common/schema"
+	"akvorado/outlet/clickhouse"
+	"akvorado/outlet/kafka"
+)
+
+// worker represents a worker processing incoming flows.
+type worker struct {
+	c       *Component
+	l       reporter.Logger
+	cw      clickhouse.Worker
+	bf      *schema.FlowMessage
+	rawFlow pb.RawFlow
+}
+
+// newWorker instantiates a new worker and returns a callback function to
+// process an incoming flow and a function to call on shutdown.
+func (c *Component) newWorker(i int) (kafka.ReceiveFunc, kafka.ShutdownFunc) {
+	bf := c.d.Schema.NewFlowMessage()
+	w := worker{
+		c:  c,
+		l:  c.r.With().Int("worker", i).Logger(),
+		bf: bf,
+		cw: c.d.ClickHouse.NewWorker(i, bf),
+	}
+	return w.processIncomingFlow, w.shutdown
+}
+
+// shutdown shutdowns the worker, flushing any remaining data.
+func (w *worker) shutdown() {
+	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
+	defer cancel()
+	w.cw.Flush(ctx)
+}
+
+// processIncomingFlow processes one incoming flow from Kafka.
+func (w *worker) processIncomingFlow(ctx context.Context, data []byte) error {
+	// Do nothing if we are shutting down
+	if !w.c.t.Alive() {
+		return kafka.ErrStopProcessing
+	}
+
+	// Raw flaw decoding: fatal
+	w.c.metrics.rawFlowsReceived.Inc()
+	w.rawFlow.ResetVT()
+	if err := w.rawFlow.UnmarshalVT(data); err != nil {
+		w.c.metrics.rawFlowsErrors.WithLabelValues("cannot decode protobuf")
+		return fmt.Errorf("cannot decode raw flow: %w", err)
+	}
+
+	// Process each decoded flow
+	finalize := func() {
+		// Accounting
+		exporter := w.bf.ExporterAddress.Unmap().String()
+		w.c.metrics.flowsReceived.WithLabelValues(exporter).Inc()
+
+		// Enrichment: not fatal
+		ip := w.bf.ExporterAddress
+		if skip := w.enrichFlow(ip, exporter); skip {
+			w.bf.Undo()
+			return
+		}
+
+		// If we have HTTP clients, send to them too
+		if atomic.LoadUint32(&w.c.httpFlowClients) > 0 {
+			if jsonBytes, err := json.Marshal(w.bf); err == nil {
+				select {
+				case w.c.httpFlowChannel <- jsonBytes: // OK
+				default: // Overflow, best effort and ignore
+				}
+			}
+		}
+
+		// Finalize and forward to ClickHouse
+		w.c.metrics.flowsForwarded.WithLabelValues(exporter).Inc()
+		w.cw.FinalizeAndSend(ctx)
+	}
+
+	// Flow decoding: not fatal
+	err := w.c.d.Flow.Decode(&w.rawFlow, w.bf, finalize)
+	if err != nil {
+		w.c.metrics.rawFlowsErrors.WithLabelValues("cannot decode payload")
+		return nil
+	}
+
+	return nil
+
+}
diff --git a/outlet/flow/decoder.go b/outlet/flow/decoder.go
new file mode 100644
index 00000000..2f8cc575
--- /dev/null
+++ b/outlet/flow/decoder.go
@@ -0,0 +1,83 @@
+// SPDX-FileCopyrightText: 2022 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+package flow
+
+import (
+	"errors"
+	"fmt"
+	"net/netip"
+	"runtime/debug"
+	"time"
+
+	"akvorado/common/pb"
+	"akvorado/common/schema"
+	"akvorado/outlet/flow/decoder"
+	"akvorado/outlet/flow/decoder/netflow"
+	"akvorado/outlet/flow/decoder/sflow"
+)
+
+// Decode decodes a raw flow from protobuf into flow messages.
+func (c *Component) Decode(rawFlow *pb.RawFlow, bf *schema.FlowMessage, finalize decoder.FinalizeFlowFunc) error {
+	// Get decoder directly by type
+	dec, ok := c.decoders[rawFlow.Decoder]
+	if !ok {
+		return fmt.Errorf("decoder %v not available", rawFlow.Decoder)
+	}
+
+	// Convert pb.RawFlow to decoder.RawFlow
+	sourceIP, ok := netip.AddrFromSlice(rawFlow.SourceAddress)
+	if !ok {
+		return errors.New("missing source address")
+	}
+
+	decoderInput := decoder.RawFlow{
+		TimeReceived: time.Unix(int64(rawFlow.TimeReceived), 0),
+		Payload:      rawFlow.Payload,
+		Source:       sourceIP,
+	}
+
+	// Decode the flow
+	options := decoder.Option{
+		TimestampSource: rawFlow.TimestampSource,
+	}
+
+	if err := c.decodeWithMetrics(dec, decoderInput, options, bf, func() {
+		if rawFlow.UseSourceAddress {
+			bf.ExporterAddress = sourceIP
+		}
+		finalize()
+	}); err != nil {
+		return fmt.Errorf("failed to decode flow: %w", err)
+	}
+
+	return nil
+}
+
+// decodeWithMetrics wraps decoder calls with metrics tracking.
+func (c *Component) decodeWithMetrics(dec decoder.Decoder, input decoder.RawFlow, options decoder.Option, bf *schema.FlowMessage, finalize decoder.FinalizeFlowFunc) error {
+	defer func() {
+		if r := recover(); r != nil {
+			c.errLogger.Error().
+				Str("decoder", dec.Name()).
+				Str("panic", fmt.Sprint(r)).
+				Str("stack", string(debug.Stack())).
+				Msg("panic while decoding")
+			c.metrics.decoderErrors.WithLabelValues(dec.Name()).Inc()
+		}
+	}()
+
+	n, err := dec.Decode(input, options, bf, finalize)
+	if err != nil {
+		c.metrics.decoderErrors.WithLabelValues(dec.Name()).Inc()
+		return err
+	}
+	c.metrics.decoderStats.WithLabelValues(dec.Name()).Add(float64(n))
+
+	return nil
+}
+
+var availableDecoders = map[pb.RawFlow_Decoder]decoder.NewDecoderFunc{
+	pb.RawFlow_DECODER_NETFLOW: netflow.New,
+	pb.RawFlow_DECODER_SFLOW:   sflow.New,
+}
diff --git a/outlet/flow/decoder/gob/root.go b/outlet/flow/decoder/gob/root.go
new file mode 100644
index 00000000..2f4e42db
--- /dev/null
+++ b/outlet/flow/decoder/gob/root.go
@@ -0,0 +1,104 @@
+// SPDX-FileCopyrightText: 2022 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+//go:build !release
+
+// Package gob handles gob decoding for testing purposes.
+package gob
+
+import (
+	"bytes"
+	"encoding/gob"
+	"net/netip"
+	"reflect"
+
+	"akvorado/common/reporter"
+	"akvorado/common/schema"
+	"akvorado/outlet/flow/decoder"
+)
+
+func init() {
+	// Register types that may appear in OtherColumns for gob encoding/decoding
+	gob.Register(schema.InterfaceBoundary(0))
+}
+
+// Decoder contains the state for the gob decoder.
+type Decoder struct {
+	r *reporter.Reporter
+	d decoder.Dependencies
+}
+
+// New creates a new gob decoder.
+func New(r *reporter.Reporter, dependencies decoder.Dependencies) decoder.Decoder {
+	return &Decoder{
+		r: r,
+		d: dependencies,
+	}
+}
+
+// Name returns the decoder name.
+func (d *Decoder) Name() string {
+	return "gob"
+}
+
+// Decode decodes a gob-encoded FlowMessage.
+func (d *Decoder) Decode(in decoder.RawFlow, _ decoder.Option, bf *schema.FlowMessage, finalize decoder.FinalizeFlowFunc) (int, error) {
+	var decoded schema.FlowMessage
+
+	buf := bytes.NewReader(in.Payload)
+	decoder := gob.NewDecoder(buf)
+
+	if err := decoder.Decode(&decoded); err != nil {
+		return 0, err
+	}
+
+	// We need to "replay" the decoded flow. We use reflection for this.
+	decodedValue := reflect.ValueOf(&decoded).Elem()
+	bfValue := reflect.ValueOf(bf).Elem()
+	decodedType := decodedValue.Type()
+
+	// Copy all public fields except OtherColumns
+	for i := 0; i < decodedValue.NumField(); i++ {
+		field := decodedType.Field(i)
+		if !field.IsExported() || field.Name == "OtherColumns" {
+			continue
+		}
+
+		sourceField := decodedValue.Field(i)
+		targetField := bfValue.FieldByName(field.Name)
+		if targetField.IsValid() && targetField.CanSet() {
+			targetField.Set(sourceField)
+		}
+	}
+
+	// Handle OtherColumns
+	if decoded.OtherColumns != nil {
+		for columnKey, value := range decoded.OtherColumns {
+			switch v := value.(type) {
+			case uint64:
+				bf.AppendUint(columnKey, v)
+			case uint32:
+				bf.AppendUint(columnKey, uint64(v))
+			case uint16:
+				bf.AppendUint(columnKey, uint64(v))
+			case uint8:
+				bf.AppendUint(columnKey, uint64(v))
+			case int:
+				bf.AppendUint(columnKey, uint64(v))
+			case string:
+				bf.AppendString(columnKey, v)
+			case netip.Addr:
+				bf.AppendIPv6(columnKey, v)
+			case schema.InterfaceBoundary:
+				bf.AppendUint(columnKey, uint64(v))
+			case []uint32:
+				bf.AppendArrayUInt32(columnKey, v)
+			case []schema.UInt128:
+				bf.AppendArrayUInt128(columnKey, v)
+			}
+		}
+	}
+
+	finalize()
+	return 1, nil
+}
diff --git a/inlet/flow/decoder/helpers.go b/outlet/flow/decoder/helpers.go
similarity index 72%
rename from inlet/flow/decoder/helpers.go
rename to outlet/flow/decoder/helpers.go
index 99281886..c12c68ba 100644
--- a/inlet/flow/decoder/helpers.go
+++ b/outlet/flow/decoder/helpers.go
@@ -18,18 +18,18 @@ func ParseIPv4(sch *schema.Component, bf *schema.FlowMessage, data []byte) uint6
 	if len(data) < 20 {
 		return 0
 	}
-	sch.ProtobufAppendVarint(bf, schema.ColumnEType, helpers.ETypeIPv4)
+	bf.AppendUint(schema.ColumnEType, helpers.ETypeIPv4)
 	l3length = uint64(binary.BigEndian.Uint16(data[2:4]))
 	bf.SrcAddr = DecodeIP(data[12:16])
 	bf.DstAddr = DecodeIP(data[16:20])
 	proto = data[9]
 	fragoffset := binary.BigEndian.Uint16(data[6:8]) & 0x1fff
 	if !sch.IsDisabled(schema.ColumnGroupL3L4) {
-		sch.ProtobufAppendVarint(bf, schema.ColumnIPTos, uint64(data[1]))
-		sch.ProtobufAppendVarint(bf, schema.ColumnIPTTL, uint64(data[8]))
-		sch.ProtobufAppendVarint(bf, schema.ColumnIPFragmentID,
+		bf.AppendUint(schema.ColumnIPTos, uint64(data[1]))
+		bf.AppendUint(schema.ColumnIPTTL, uint64(data[8]))
+		bf.AppendUint(schema.ColumnIPFragmentID,
 			uint64(binary.BigEndian.Uint16(data[4:6])))
-		sch.ProtobufAppendVarint(bf, schema.ColumnIPFragmentOffset,
+		bf.AppendUint(schema.ColumnIPFragmentOffset,
 			uint64(fragoffset))
 	}
 	ihl := int((data[0] & 0xf) * 4)
@@ -38,7 +38,7 @@ func ParseIPv4(sch *schema.Component, bf *schema.FlowMessage, data []byte) uint6
 	} else {
 		data = data[:0]
 	}
-	sch.ProtobufAppendVarint(bf, schema.ColumnProto, uint64(proto))
+	bf.AppendUint(schema.ColumnProto, uint64(proto))
 	if fragoffset == 0 {
 		ParseL4(sch, bf, data, proto)
 	}
@@ -53,21 +53,21 @@ func ParseIPv6(sch *schema.Component, bf *schema.FlowMessage, data []byte) uint6
 		return 0
 	}
 	l3length = uint64(binary.BigEndian.Uint16(data[4:6])) + 40
-	sch.ProtobufAppendVarint(bf, schema.ColumnEType, helpers.ETypeIPv6)
+	bf.AppendUint(schema.ColumnEType, helpers.ETypeIPv6)
 	bf.SrcAddr = DecodeIP(data[8:24])
 	bf.DstAddr = DecodeIP(data[24:40])
 	proto = data[6]
-	sch.ProtobufAppendVarint(bf, schema.ColumnProto, uint64(proto))
+	bf.AppendUint(schema.ColumnProto, uint64(proto))
 	if !sch.IsDisabled(schema.ColumnGroupL3L4) {
-		sch.ProtobufAppendVarint(bf, schema.ColumnIPTos,
+		bf.AppendUint(schema.ColumnIPTos,
 			uint64(binary.BigEndian.Uint16(data[0:2])&0xff0>>4))
-		sch.ProtobufAppendVarint(bf, schema.ColumnIPTTL, uint64(data[7]))
-		sch.ProtobufAppendVarint(bf, schema.ColumnIPv6FlowLabel,
+		bf.AppendUint(schema.ColumnIPTTL, uint64(data[7]))
+		bf.AppendUint(schema.ColumnIPv6FlowLabel,
 			uint64(binary.BigEndian.Uint32(data[0:4])&0xfffff))
 		// TODO fragmentID/fragmentOffset are in a separate header
 	}
 	data = data[40:]
-	sch.ProtobufAppendVarint(bf, schema.ColumnProto, uint64(proto))
+	bf.AppendUint(schema.ColumnProto, uint64(proto))
 	ParseL4(sch, bf, data, proto)
 	return l3length
 }
@@ -77,9 +77,9 @@ func ParseL4(sch *schema.Component, bf *schema.FlowMessage, data []byte, proto u
 	if proto == 6 || proto == 17 {
 		// UDP or TCP
 		if len(data) > 4 {
-			sch.ProtobufAppendVarint(bf, schema.ColumnSrcPort,
+			bf.AppendUint(schema.ColumnSrcPort,
 				uint64(binary.BigEndian.Uint16(data[0:2])))
-			sch.ProtobufAppendVarint(bf, schema.ColumnDstPort,
+			bf.AppendUint(schema.ColumnDstPort,
 				uint64(binary.BigEndian.Uint16(data[2:4])))
 		}
 	}
@@ -87,23 +87,23 @@ func ParseL4(sch *schema.Component, bf *schema.FlowMessage, data []byte, proto u
 		if proto == 6 {
 			// TCP
 			if len(data) > 13 {
-				sch.ProtobufAppendVarint(bf, schema.ColumnTCPFlags,
+				bf.AppendUint(schema.ColumnTCPFlags,
 					uint64(data[13]))
 			}
 		} else if proto == 1 {
 			// ICMPv4
 			if len(data) > 2 {
-				sch.ProtobufAppendVarint(bf, schema.ColumnICMPv4Type,
+				bf.AppendUint(schema.ColumnICMPv4Type,
 					uint64(data[0]))
-				sch.ProtobufAppendVarint(bf, schema.ColumnICMPv4Code,
+				bf.AppendUint(schema.ColumnICMPv4Code,
 					uint64(data[1]))
 			}
 		} else if proto == 58 {
 			// ICMPv6
 			if len(data) > 2 {
-				sch.ProtobufAppendVarint(bf, schema.ColumnICMPv6Type,
+				bf.AppendUint(schema.ColumnICMPv6Type,
 					uint64(data[0]))
-				sch.ProtobufAppendVarint(bf, schema.ColumnICMPv6Code,
+				bf.AppendUint(schema.ColumnICMPv6Code,
 					uint64(data[1]))
 			}
 		}
@@ -116,9 +116,9 @@ func ParseEthernet(sch *schema.Component, bf *schema.FlowMessage, data []byte) u
 		return 0
 	}
 	if !sch.IsDisabled(schema.ColumnGroupL2) {
-		sch.ProtobufAppendVarint(bf, schema.ColumnDstMAC,
+		bf.AppendUint(schema.ColumnDstMAC,
 			binary.BigEndian.Uint64([]byte{0, 0, data[0], data[1], data[2], data[3], data[4], data[5]}))
-		sch.ProtobufAppendVarint(bf, schema.ColumnSrcMAC,
+		bf.AppendUint(schema.ColumnSrcMAC,
 			binary.BigEndian.Uint64([]byte{0, 0, data[6], data[7], data[8], data[9], data[10], data[11]}))
 	}
 	etherType := data[12:14]
@@ -140,6 +140,7 @@ func ParseEthernet(sch *schema.Component, bf *schema.FlowMessage, data []byte) u
 	}
 	if etherType[0] == 0x88 && etherType[1] == 0x47 {
 		// MPLS
+		mplsLabels := make([]uint32, 0, 5)
 		for {
 			if len(data) < 5 {
 				return 0
@@ -147,18 +148,22 @@ func ParseEthernet(sch *schema.Component, bf *schema.FlowMessage, data []byte) u
 			label := binary.BigEndian.Uint32(append([]byte{0}, data[:3]...)) >> 4
 			bottom := data[2] & 1
 			data = data[4:]
-			sch.ProtobufAppendVarint(bf, schema.ColumnMPLSLabels, uint64(label))
+			mplsLabels = append(mplsLabels, label)
 			if bottom == 1 || label <= 15 {
-				if data[0]&0xf0>>4 == 4 {
+				switch data[0] & 0xf0 >> 4 {
+				case 4:
 					etherType = []byte{0x8, 0x0}
-				} else if data[0]&0xf0>>4 == 6 {
+				case 6:
 					etherType = []byte{0x86, 0xdd}
-				} else {
+				default:
 					return 0
 				}
 				break
 			}
 		}
+		if len(mplsLabels) > 0 {
+			bf.AppendArrayUInt32(schema.ColumnMPLSLabels, mplsLabels)
+		}
 	}
 	if etherType[0] == 0x8 && etherType[1] == 0x0 {
 		return ParseIPv4(sch, bf, data)
diff --git a/inlet/flow/decoder/netflow/decode.go b/outlet/flow/decoder/netflow/decode.go
similarity index 66%
rename from inlet/flow/decoder/netflow/decode.go
rename to outlet/flow/decoder/netflow/decode.go
index 28286b56..dfa64d5f 100644
--- a/inlet/flow/decoder/netflow/decode.go
+++ b/outlet/flow/decoder/netflow/decode.go
@@ -9,8 +9,9 @@ import (
 	"net/netip"
 
 	"akvorado/common/helpers"
+	"akvorado/common/pb"
 	"akvorado/common/schema"
-	"akvorado/inlet/flow/decoder"
+	"akvorado/outlet/flow/decoder"
 
 	"github.com/netsampler/goflow2/v2/decoders/netflow"
 	"github.com/netsampler/goflow2/v2/decoders/netflowlegacy"
@@ -21,46 +22,39 @@ import (
 // values in the sub-range of 1-127 are compatible with field types used by
 // NetFlow version 9 [RFC3954]."
 
-func (nd *Decoder) decodeNFv5(packet *netflowlegacy.PacketNetFlowV5, ts, sysUptime uint64) []*schema.FlowMessage {
-	flowMessageSet := []*schema.FlowMessage{}
-
+func (nd *Decoder) decodeNFv5(packet *netflowlegacy.PacketNetFlowV5, ts, sysUptime uint64, options decoder.Option, bf *schema.FlowMessage, finalize decoder.FinalizeFlowFunc) {
 	for _, record := range packet.Records {
-		bf := &schema.FlowMessage{
-			SamplingRate: uint32(packet.SamplingInterval),
-			InIf:         uint32(record.Input),
-			OutIf:        uint32(record.Output),
-			SrcAddr:      decodeIPFromUint32(uint32(record.SrcAddr)),
-			DstAddr:      decodeIPFromUint32(uint32(record.DstAddr)),
-			NextHop:      decodeIPFromUint32(uint32(record.NextHop)),
-			SrcNetMask:   record.SrcMask,
-			DstNetMask:   record.DstMask,
-			SrcAS:        uint32(record.SrcAS),
-			DstAS:        uint32(record.DstAS),
-		}
-		nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnBytes, uint64(record.DOctets))
-		nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnPackets, uint64(record.DPkts))
-		nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnEType, helpers.ETypeIPv4)
-		nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnProto, uint64(record.Proto))
-		nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnSrcPort, uint64(record.SrcPort))
-		nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnDstPort, uint64(record.DstPort))
+		bf.SamplingRate = uint64(packet.SamplingInterval)
+		bf.InIf = uint32(record.Input)
+		bf.OutIf = uint32(record.Output)
+		bf.SrcAddr = decodeIPFromUint32(uint32(record.SrcAddr))
+		bf.DstAddr = decodeIPFromUint32(uint32(record.DstAddr))
+		bf.NextHop = decodeIPFromUint32(uint32(record.NextHop))
+		bf.SrcNetMask = record.SrcMask
+		bf.DstNetMask = record.DstMask
+		bf.SrcAS = uint32(record.SrcAS)
+		bf.DstAS = uint32(record.DstAS)
+		bf.AppendUint(schema.ColumnBytes, uint64(record.DOctets))
+		bf.AppendUint(schema.ColumnPackets, uint64(record.DPkts))
+		bf.AppendUint(schema.ColumnEType, helpers.ETypeIPv4)
+		bf.AppendUint(schema.ColumnProto, uint64(record.Proto))
+		bf.AppendUint(schema.ColumnSrcPort, uint64(record.SrcPort))
+		bf.AppendUint(schema.ColumnDstPort, uint64(record.DstPort))
 		if !nd.d.Schema.IsDisabled(schema.ColumnGroupL3L4) {
-			nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnIPTos, uint64(record.Tos))
-			nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnTCPFlags, uint64(record.TCPFlags))
+			bf.AppendUint(schema.ColumnIPTos, uint64(record.Tos))
+			bf.AppendUint(schema.ColumnTCPFlags, uint64(record.TCPFlags))
 		}
-		if nd.useTsFromFirstSwitched {
-			bf.TimeReceived = ts - sysUptime + uint64(record.First)
+		if options.TimestampSource == pb.RawFlow_TS_NETFLOW_FIRST_SWITCHED {
+			bf.TimeReceived = uint32(ts - sysUptime + uint64(record.First))
 		}
 		if bf.SamplingRate == 0 {
 			bf.SamplingRate = 1
 		}
-		flowMessageSet = append(flowMessageSet, bf)
+		finalize()
 	}
-	return flowMessageSet
 }
 
-func (nd *Decoder) decodeNFv9IPFIX(version uint16, obsDomainID uint32, flowSets []interface{}, samplingRateSys *samplingRateSystem, ts, sysUptime uint64) []*schema.FlowMessage {
-	flowMessageSet := []*schema.FlowMessage{}
-
+func (nd *Decoder) decodeNFv9IPFIX(version uint16, obsDomainID uint32, flowSets []interface{}, samplingRateSys *samplingRateSystem, ts, sysUptime uint64, options decoder.Option, bf *schema.FlowMessage, finalize decoder.FinalizeFlowFunc) {
 	// Look for sampling rate in option data flowsets
 	for _, flowSet := range flowSets {
 		switch tFlowSet := flowSet.(type) {
@@ -96,22 +90,18 @@ func (nd *Decoder) decodeNFv9IPFIX(version uint16, obsDomainID uint32, flowSets
 			}
 		case netflow.DataFlowSet:
 			for _, record := range tFlowSet.Records {
-				flow := nd.decodeRecord(version, obsDomainID, samplingRateSys, record.Values, ts, sysUptime)
-				if flow != nil {
-					flowMessageSet = append(flowMessageSet, flow)
-				}
+				nd.decodeRecord(version, obsDomainID, samplingRateSys, record.Values, ts, sysUptime, options, bf)
+				finalize()
 			}
 		}
 	}
-
-	return flowMessageSet
 }
 
-func (nd *Decoder) decodeRecord(version uint16, obsDomainID uint32, samplingRateSys *samplingRateSystem, fields []netflow.DataField, ts, sysUptime uint64) *schema.FlowMessage {
+func (nd *Decoder) decodeRecord(version uint16, obsDomainID uint32, samplingRateSys *samplingRateSystem, fields []netflow.DataField, ts, sysUptime uint64, options decoder.Option, bf *schema.FlowMessage) {
 	var etype, dstPort, srcPort uint16
 	var proto, icmpType, icmpCode uint8
 	var foundIcmpTypeCode bool
-	bf := &schema.FlowMessage{}
+	mplsLabels := make([]uint32, 0, 5)
 	dataLinkFrameSectionIdx := -1
 	for idx, field := range fields {
 		v, ok := field.Value.([]byte)
@@ -122,13 +112,13 @@ func (nd *Decoder) decodeRecord(version uint16, obsDomainID uint32, samplingRate
 		switch field.Type {
 		// Statistics
 		case netflow.IPFIX_FIELD_octetDeltaCount, netflow.IPFIX_FIELD_postOctetDeltaCount, netflow.IPFIX_FIELD_initiatorOctets, netflow.IPFIX_FIELD_responderOctets:
-			nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnBytes, decodeUNumber(v))
+			bf.AppendUint(schema.ColumnBytes, decodeUNumber(v))
 		case netflow.IPFIX_FIELD_packetDeltaCount, netflow.IPFIX_FIELD_postPacketDeltaCount:
-			nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnPackets, decodeUNumber(v))
+			bf.AppendUint(schema.ColumnPackets, decodeUNumber(v))
 		case netflow.IPFIX_FIELD_samplingInterval, netflow.IPFIX_FIELD_samplerRandomInterval:
-			bf.SamplingRate = uint32(decodeUNumber(v))
+			bf.SamplingRate = decodeUNumber(v)
 		case netflow.IPFIX_FIELD_samplerId, netflow.IPFIX_FIELD_selectorId:
-			bf.SamplingRate = samplingRateSys.GetSamplingRate(version, obsDomainID, decodeUNumber(v))
+			bf.SamplingRate = uint64(samplingRateSys.GetSamplingRate(version, obsDomainID, decodeUNumber(v)))
 
 		// L3
 		case netflow.IPFIX_FIELD_sourceIPv4Address:
@@ -153,13 +143,13 @@ func (nd *Decoder) decodeRecord(version uint16, obsDomainID uint32, samplingRate
 		// L4
 		case netflow.IPFIX_FIELD_sourceTransportPort:
 			srcPort = uint16(decodeUNumber(v))
-			nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnSrcPort, uint64(srcPort))
+			bf.AppendUint(schema.ColumnSrcPort, uint64(srcPort))
 		case netflow.IPFIX_FIELD_destinationTransportPort:
 			dstPort = uint16(decodeUNumber(v))
-			nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnDstPort, uint64(dstPort))
+			bf.AppendUint(schema.ColumnDstPort, uint64(dstPort))
 		case netflow.IPFIX_FIELD_protocolIdentifier:
 			proto = uint8(decodeUNumber(v))
-			nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnProto, uint64(proto))
+			bf.AppendUint(schema.ColumnProto, uint64(proto))
 
 		// Network
 		case netflow.IPFIX_FIELD_bgpSourceAsNumber:
@@ -181,24 +171,27 @@ func (nd *Decoder) decodeRecord(version uint16, obsDomainID uint32, samplingRate
 
 		// MPLS
 		case netflow.IPFIX_FIELD_mplsTopLabelStackSection, netflow.IPFIX_FIELD_mplsLabelStackSection2, netflow.IPFIX_FIELD_mplsLabelStackSection3, netflow.IPFIX_FIELD_mplsLabelStackSection4, netflow.IPFIX_FIELD_mplsLabelStackSection5, netflow.IPFIX_FIELD_mplsLabelStackSection6, netflow.IPFIX_FIELD_mplsLabelStackSection7, netflow.IPFIX_FIELD_mplsLabelStackSection8, netflow.IPFIX_FIELD_mplsLabelStackSection9, netflow.IPFIX_FIELD_mplsLabelStackSection10:
-			nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnMPLSLabels, decodeUNumber(v)>>4)
+			uv := decodeUNumber(v) >> 4
+			if uv > 0 {
+				mplsLabels = append(mplsLabels, uint32(uv))
+			}
 
 		// Remaining
 		case netflow.IPFIX_FIELD_forwardingStatus:
-			nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnForwardingStatus, decodeUNumber(v))
+			bf.AppendUint(schema.ColumnForwardingStatus, decodeUNumber(v))
 		default:
-			if nd.useTsFromFirstSwitched {
+			if options.TimestampSource == pb.RawFlow_TS_NETFLOW_FIRST_SWITCHED {
 				switch field.Type {
 				case netflow.NFV9_FIELD_FIRST_SWITCHED:
-					bf.TimeReceived = ts - sysUptime + decodeUNumber(v)
+					bf.TimeReceived = uint32(ts - sysUptime + decodeUNumber(v))
 				case netflow.IPFIX_FIELD_flowStartSeconds:
-					bf.TimeReceived = decodeUNumber(v)
+					bf.TimeReceived = uint32(decodeUNumber(v))
 				case netflow.IPFIX_FIELD_flowStartMilliseconds:
-					bf.TimeReceived = decodeUNumber(v) / 1000
+					bf.TimeReceived = uint32(decodeUNumber(v) / 1000)
 				case netflow.IPFIX_FIELD_flowStartMicroseconds:
-					bf.TimeReceived = decodeUNumber(v) / 1_000_000
+					bf.TimeReceived = uint32(decodeUNumber(v) / 1_000_000)
 				case netflow.IPFIX_FIELD_flowStartNanoseconds:
-					bf.TimeReceived = ts + decodeUNumber(v)/1_000_000_000
+					bf.TimeReceived = uint32(ts + decodeUNumber(v)/1_000_000_000)
 				}
 			}
 
@@ -206,13 +199,13 @@ func (nd *Decoder) decodeRecord(version uint16, obsDomainID uint32, samplingRate
 				// NAT
 				switch field.Type {
 				case netflow.IPFIX_FIELD_postNATSourceIPv4Address:
-					nd.d.Schema.ProtobufAppendIP(bf, schema.ColumnSrcAddrNAT, decodeIPFromBytes(v))
+					bf.AppendIPv6(schema.ColumnSrcAddrNAT, decodeIPFromBytes(v))
 				case netflow.IPFIX_FIELD_postNATDestinationIPv4Address:
-					nd.d.Schema.ProtobufAppendIP(bf, schema.ColumnDstAddrNAT, decodeIPFromBytes(v))
+					bf.AppendIPv6(schema.ColumnDstAddrNAT, decodeIPFromBytes(v))
 				case netflow.IPFIX_FIELD_postNAPTSourceTransportPort:
-					nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnSrcPortNAT, decodeUNumber(v))
+					bf.AppendUint(schema.ColumnSrcPortNAT, decodeUNumber(v))
 				case netflow.IPFIX_FIELD_postNAPTDestinationTransportPort:
-					nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnDstPortNAT, decodeUNumber(v))
+					bf.AppendUint(schema.ColumnDstPortNAT, decodeUNumber(v))
 				}
 			}
 
@@ -224,13 +217,13 @@ func (nd *Decoder) decodeRecord(version uint16, obsDomainID uint32, samplingRate
 				case netflow.IPFIX_FIELD_postVlanId:
 					bf.DstVlan = uint16(decodeUNumber(v))
 				case netflow.IPFIX_FIELD_sourceMacAddress:
-					nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnSrcMAC, decodeUNumber(v))
+					bf.AppendUint(schema.ColumnSrcMAC, decodeUNumber(v))
 				case netflow.IPFIX_FIELD_destinationMacAddress:
-					nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnDstMAC, decodeUNumber(v))
+					bf.AppendUint(schema.ColumnDstMAC, decodeUNumber(v))
 				case netflow.IPFIX_FIELD_postSourceMacAddress:
-					nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnSrcMAC, decodeUNumber(v))
+					bf.AppendUint(schema.ColumnSrcMAC, decodeUNumber(v))
 				case netflow.IPFIX_FIELD_postDestinationMacAddress:
-					nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnDstMAC, decodeUNumber(v))
+					bf.AppendUint(schema.ColumnDstMAC, decodeUNumber(v))
 				}
 			}
 
@@ -238,17 +231,17 @@ func (nd *Decoder) decodeRecord(version uint16, obsDomainID uint32, samplingRate
 				// Misc L3/L4 fields
 				switch field.Type {
 				case netflow.IPFIX_FIELD_minimumTTL:
-					nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnIPTTL, decodeUNumber(v))
+					bf.AppendUint(schema.ColumnIPTTL, decodeUNumber(v))
 				case netflow.IPFIX_FIELD_ipClassOfService:
-					nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnIPTos, decodeUNumber(v))
+					bf.AppendUint(schema.ColumnIPTos, decodeUNumber(v))
 				case netflow.IPFIX_FIELD_flowLabelIPv6:
-					nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnIPv6FlowLabel, decodeUNumber(v))
+					bf.AppendUint(schema.ColumnIPv6FlowLabel, decodeUNumber(v))
 				case netflow.IPFIX_FIELD_tcpControlBits:
-					nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnTCPFlags, decodeUNumber(v))
+					bf.AppendUint(schema.ColumnTCPFlags, decodeUNumber(v))
 				case netflow.IPFIX_FIELD_fragmentIdentification:
-					nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnIPFragmentID, decodeUNumber(v))
+					bf.AppendUint(schema.ColumnIPFragmentID, decodeUNumber(v))
 				case netflow.IPFIX_FIELD_fragmentOffset:
-					nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnIPFragmentOffset, decodeUNumber(v))
+					bf.AppendUint(schema.ColumnIPFragmentOffset, decodeUNumber(v))
 
 				// ICMP
 				case netflow.IPFIX_FIELD_icmpTypeCodeIPv4, netflow.IPFIX_FIELD_icmpTypeCodeIPv6:
@@ -269,8 +262,8 @@ func (nd *Decoder) decodeRecord(version uint16, obsDomainID uint32, samplingRate
 	if dataLinkFrameSectionIdx >= 0 {
 		data := fields[dataLinkFrameSectionIdx].Value.([]byte)
 		if l3Length := decoder.ParseEthernet(nd.d.Schema, bf, data); l3Length > 0 {
-			nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnBytes, l3Length)
-			nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnPackets, 1)
+			bf.AppendUint(schema.ColumnBytes, l3Length)
+			bf.AppendUint(schema.ColumnPackets, 1)
 		}
 	}
 	if !nd.d.Schema.IsDisabled(schema.ColumnGroupL3L4) && (proto == 1 || proto == 58) {
@@ -290,18 +283,20 @@ func (nd *Decoder) decodeRecord(version uint16, obsDomainID uint32, samplingRate
 			}
 		}
 		if proto == 1 {
-			nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnICMPv4Type, uint64(icmpType))
-			nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnICMPv4Code, uint64(icmpCode))
+			bf.AppendUint(schema.ColumnICMPv4Type, uint64(icmpType))
+			bf.AppendUint(schema.ColumnICMPv4Code, uint64(icmpCode))
 		} else {
-			nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnICMPv6Type, uint64(icmpType))
-			nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnICMPv6Code, uint64(icmpCode))
+			bf.AppendUint(schema.ColumnICMPv6Type, uint64(icmpType))
+			bf.AppendUint(schema.ColumnICMPv6Code, uint64(icmpCode))
 		}
 	}
-	nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnEType, uint64(etype))
+	bf.AppendUint(schema.ColumnEType, uint64(etype))
+	if len(mplsLabels) > 0 {
+		bf.AppendArrayUInt32(schema.ColumnMPLSLabels, mplsLabels)
+	}
 	if bf.SamplingRate == 0 {
-		bf.SamplingRate = samplingRateSys.GetSamplingRate(version, obsDomainID, 0)
+		bf.SamplingRate = uint64(samplingRateSys.GetSamplingRate(version, obsDomainID, 0))
 	}
-	return bf
 }
 
 func decodeUNumber(b []byte) uint64 {
diff --git a/inlet/flow/decoder/netflow/root.go b/outlet/flow/decoder/netflow/root.go
similarity index 63%
rename from inlet/flow/decoder/netflow/root.go
rename to outlet/flow/decoder/netflow/root.go
index aa15ef13..99bff4af 100644
--- a/inlet/flow/decoder/netflow/root.go
+++ b/outlet/flow/decoder/netflow/root.go
@@ -8,7 +8,7 @@ import (
 	"bytes"
 	"encoding/binary"
 	"errors"
-	"net/netip"
+	"fmt"
 	"strconv"
 	"sync"
 	"time"
@@ -16,9 +16,10 @@ import (
 	"github.com/netsampler/goflow2/v2/decoders/netflow"
 	"github.com/netsampler/goflow2/v2/decoders/netflowlegacy"
 
+	"akvorado/common/pb"
 	"akvorado/common/reporter"
 	"akvorado/common/schema"
-	"akvorado/inlet/flow/decoder"
+	"akvorado/outlet/flow/decoder"
 )
 
 // Decoder contains the state for the Netflow v9 decoder.
@@ -33,60 +34,56 @@ type Decoder struct {
 	sampling    map[string]*samplingRateSystem
 
 	metrics struct {
-		errors             *reporter.CounterVec
-		stats              *reporter.CounterVec
-		setRecordsStatsSum *reporter.CounterVec
-		setStatsSum        *reporter.CounterVec
-		templatesStats     *reporter.CounterVec
+		errors    *reporter.CounterVec
+		packets   *reporter.CounterVec
+		records   *reporter.CounterVec
+		sets      *reporter.CounterVec
+		templates *reporter.CounterVec
 	}
-	useTsFromNetflowsPacket bool
-	useTsFromFirstSwitched  bool
 }
 
 // New instantiates a new netflow decoder.
-func New(r *reporter.Reporter, dependencies decoder.Dependencies, option decoder.Option) decoder.Decoder {
+func New(r *reporter.Reporter, dependencies decoder.Dependencies) decoder.Decoder {
 	nd := &Decoder{
-		r:                       r,
-		d:                       dependencies,
-		errLogger:               r.Sample(reporter.BurstSampler(30*time.Second, 3)),
-		templates:               map[string]*templateSystem{},
-		sampling:                map[string]*samplingRateSystem{},
-		useTsFromNetflowsPacket: option.TimestampSource == decoder.TimestampSourceNetflowPacket,
-		useTsFromFirstSwitched:  option.TimestampSource == decoder.TimestampSourceNetflowFirstSwitched,
+		r:         r,
+		d:         dependencies,
+		errLogger: r.Sample(reporter.BurstSampler(30*time.Second, 3)),
+		templates: map[string]*templateSystem{},
+		sampling:  map[string]*samplingRateSystem{},
 	}
 
 	nd.metrics.errors = nd.r.CounterVec(
 		reporter.CounterOpts{
 			Name: "errors_total",
-			Help: "Netflows processed errors.",
+			Help: "Number of NetFlow errors processed.",
 		},
 		[]string{"exporter", "error"},
 	)
-	nd.metrics.stats = nd.r.CounterVec(
+	nd.metrics.packets = nd.r.CounterVec(
 		reporter.CounterOpts{
-			Name: "flows_total",
-			Help: "Netflows processed.",
+			Name: "packets_total",
+			Help: "Number of NetFlow packets received.",
 		},
 		[]string{"exporter", "version"},
 	)
-	nd.metrics.setRecordsStatsSum = nd.r.CounterVec(
+	nd.metrics.sets = nd.r.CounterVec(
 		reporter.CounterOpts{
-			Name: "flowset_records_sum",
-			Help: "Netflows FlowSets sum of records.",
+			Name: "sets_total",
+			Help: "Number of NetFlow flowsets received.",
 		},
 		[]string{"exporter", "version", "type"},
 	)
-	nd.metrics.setStatsSum = nd.r.CounterVec(
+	nd.metrics.records = nd.r.CounterVec(
 		reporter.CounterOpts{
-			Name: "flowset_sum",
-			Help: "Netflows FlowSets sum.",
+			Name: "records_total",
+			Help: "Number of NetFlow records received.",
 		},
 		[]string{"exporter", "version", "type"},
 	)
-	nd.metrics.templatesStats = nd.r.CounterVec(
+	nd.metrics.templates = nd.r.CounterVec(
 		reporter.CounterOpts{
 			Name: "templates_total",
-			Help: "Netflows Template count.",
+			Help: "Number of NetFlow templates received.",
 		},
 		[]string{"exporter", "version", "obs_domain_id", "template_id", "type"},
 	)
@@ -118,7 +115,7 @@ func (s *templateSystem) AddTemplate(version uint16, obsDomainID uint32, templat
 		typeStr = "template"
 	}
 
-	s.nd.metrics.templatesStats.WithLabelValues(
+	s.nd.metrics.templates.WithLabelValues(
 		s.key,
 		strconv.Itoa(int(version)),
 		strconv.Itoa(int(obsDomainID)),
@@ -169,9 +166,9 @@ func (s *samplingRateSystem) SetSamplingRate(version uint16, obsDomainID uint32,
 }
 
 // Decode decodes a Netflow payload.
-func (nd *Decoder) Decode(in decoder.RawFlow) []*schema.FlowMessage {
+func (nd *Decoder) Decode(in decoder.RawFlow, options decoder.Option, bf *schema.FlowMessage, finalize decoder.FinalizeFlowFunc) (int, error) {
 	if len(in.Payload) < 2 {
-		return nil
+		return 0, errors.New("payload too small")
 	}
 	key := in.Source.String()
 	nd.systemsLock.RLock()
@@ -198,15 +195,21 @@ func (nd *Decoder) Decode(in decoder.RawFlow) []*schema.FlowMessage {
 	}
 
 	var (
-		sysUptime      uint64
-		versionStr     string
-		flowSets       []interface{}
-		obsDomainID    uint32
-		flowMessageSet []*schema.FlowMessage
+		sysUptime   uint64
+		versionStr  string
+		flowSets    []any
+		obsDomainID uint32
 	)
 	version := binary.BigEndian.Uint16(in.Payload[:2])
 	buf := bytes.NewBuffer(in.Payload[2:])
-	ts := uint64(in.TimeReceived.UTC().Unix())
+	ts := uint64(in.TimeReceived.UTC().Unix()) // may be altered later
+	finalize2 := func() {
+		if bf.TimeReceived == 0 {
+			bf.TimeReceived = uint32(ts)
+		}
+		bf.ExporterAddress = in.Source
+		finalize()
+	}
 
 	switch version {
 	case 5:
@@ -214,100 +217,95 @@ func (nd *Decoder) Decode(in decoder.RawFlow) []*schema.FlowMessage {
 		if err := netflowlegacy.DecodeMessage(buf, &packetNFv5); err != nil {
 			nd.metrics.errors.WithLabelValues(key, "NetFlow v5 decoding error").Inc()
 			nd.errLogger.Err(err).Str("exporter", key).Msg("error while decoding NetFlow v5")
-			return nil
+			return 0, fmt.Errorf("NetFlow v5 decoding error: %w", err)
 		}
 		versionStr = "5"
-		nd.metrics.setStatsSum.WithLabelValues(key, versionStr, "PDU").Inc()
-		nd.metrics.setRecordsStatsSum.WithLabelValues(key, versionStr, "PDU").
+		nd.metrics.sets.WithLabelValues(key, versionStr, "PDU").Inc()
+		nd.metrics.records.WithLabelValues(key, versionStr, "PDU").
 			Add(float64(len(packetNFv5.Records)))
-		if nd.useTsFromNetflowsPacket || nd.useTsFromFirstSwitched {
+		if options.TimestampSource == pb.RawFlow_TS_NETFLOW_PACKET || options.TimestampSource == pb.RawFlow_TS_NETFLOW_FIRST_SWITCHED {
 			ts = uint64(packetNFv5.UnixSecs)
 			sysUptime = uint64(packetNFv5.SysUptime)
 		}
-		flowMessageSet = nd.decodeNFv5(&packetNFv5, ts, sysUptime)
+		nd.decodeNFv5(&packetNFv5, ts, sysUptime, options, bf, finalize2)
 	case 9:
 		var packetNFv9 netflow.NFv9Packet
 		if err := netflow.DecodeMessageNetFlow(buf, templates, &packetNFv9); err != nil {
 			if !errors.Is(err, netflow.ErrorTemplateNotFound) {
 				nd.errLogger.Err(err).Str("exporter", key).Msg("error while decoding NetFlow v9")
 				nd.metrics.errors.WithLabelValues(key, "NetFlow v9 decoding error").Inc()
-			} else {
-				nd.errLogger.Debug().Str("exporter", key).Msg("template not received yet")
+				return 0, fmt.Errorf("NetFlow v9 decoding error: %w", err)
 			}
-			return nil
+			nd.errLogger.Debug().Str("exporter", key).Msg("template not received yet")
+			return 0, nil
 		}
 		versionStr = "9"
 		flowSets = packetNFv9.FlowSets
 		obsDomainID = packetNFv9.SourceId
-		if nd.useTsFromNetflowsPacket || nd.useTsFromFirstSwitched {
+		if options.TimestampSource == pb.RawFlow_TS_NETFLOW_PACKET || options.TimestampSource == pb.RawFlow_TS_NETFLOW_FIRST_SWITCHED {
 			ts = uint64(packetNFv9.UnixSeconds)
 			sysUptime = uint64(packetNFv9.SystemUptime)
 		}
-		flowMessageSet = nd.decodeNFv9IPFIX(version, obsDomainID, flowSets, sampling, ts, sysUptime)
+		nd.decodeNFv9IPFIX(version, obsDomainID, flowSets, sampling, ts, sysUptime, options, bf, finalize2)
 	case 10:
 		var packetIPFIX netflow.IPFIXPacket
 		if err := netflow.DecodeMessageIPFIX(buf, templates, &packetIPFIX); err != nil {
 			if !errors.Is(err, netflow.ErrorTemplateNotFound) {
 				nd.errLogger.Err(err).Str("exporter", key).Msg("error while decoding IPFIX")
 				nd.metrics.errors.WithLabelValues(key, "IPFIX decoding error").Inc()
-			} else {
-				nd.errLogger.Debug().Str("exporter", key).Msg("template not received yet")
+				return 0, fmt.Errorf("NetFlow v9 decoding error: %w", err)
 			}
-			return nil
+			nd.errLogger.Debug().Str("exporter", key).Msg("template not received yet")
+			return 0, nil
 		}
 		versionStr = "10"
 		flowSets = packetIPFIX.FlowSets
 		obsDomainID = packetIPFIX.ObservationDomainId
-		if nd.useTsFromNetflowsPacket {
+		if options.TimestampSource == pb.RawFlow_TS_NETFLOW_PACKET {
 			ts = uint64(packetIPFIX.ExportTime)
 		}
-		flowMessageSet = nd.decodeNFv9IPFIX(version, obsDomainID, flowSets, sampling, ts, sysUptime)
+		nd.decodeNFv9IPFIX(version, obsDomainID, flowSets, sampling, ts, sysUptime, options, bf, finalize2)
 	default:
-		nd.metrics.stats.WithLabelValues(key, "unknown").
+		nd.errLogger.Warn().Str("exporter", key).Msg("unknown NetFlow version")
+		nd.metrics.packets.WithLabelValues(key, "unknown").
 			Inc()
-		return nil
+		return 0, errors.New("unkown NetFlow version")
 	}
-	nd.metrics.stats.WithLabelValues(key, versionStr).Inc()
+	nd.metrics.packets.WithLabelValues(key, versionStr).Inc()
 
+	nb := 0
 	for _, fs := range flowSets {
 		switch fsConv := fs.(type) {
 		case netflow.TemplateFlowSet:
-			nd.metrics.setStatsSum.WithLabelValues(key, versionStr, "TemplateFlowSet").
+			nd.metrics.sets.WithLabelValues(key, versionStr, "TemplateFlowSet").
 				Inc()
-			nd.metrics.setRecordsStatsSum.WithLabelValues(key, versionStr, "TemplateFlowSet").
+			nd.metrics.records.WithLabelValues(key, versionStr, "TemplateFlowSet").
 				Add(float64(len(fsConv.Records)))
 		case netflow.IPFIXOptionsTemplateFlowSet:
-			nd.metrics.setStatsSum.WithLabelValues(key, versionStr, "OptionsTemplateFlowSet").
+			nd.metrics.sets.WithLabelValues(key, versionStr, "OptionsTemplateFlowSet").
 				Inc()
-			nd.metrics.setRecordsStatsSum.WithLabelValues(key, versionStr, "OptionsTemplateFlowSet").
+			nd.metrics.records.WithLabelValues(key, versionStr, "OptionsTemplateFlowSet").
 				Add(float64(len(fsConv.Records)))
 		case netflow.NFv9OptionsTemplateFlowSet:
-			nd.metrics.setStatsSum.WithLabelValues(key, versionStr, "OptionsTemplateFlowSet").
+			nd.metrics.sets.WithLabelValues(key, versionStr, "OptionsTemplateFlowSet").
 				Inc()
-			nd.metrics.setRecordsStatsSum.WithLabelValues(key, versionStr, "OptionsTemplateFlowSet").
+			nd.metrics.records.WithLabelValues(key, versionStr, "OptionsTemplateFlowSet").
 				Add(float64(len(fsConv.Records)))
 		case netflow.OptionsDataFlowSet:
-			nd.metrics.setStatsSum.WithLabelValues(key, versionStr, "OptionsDataFlowSet").
+			nd.metrics.sets.WithLabelValues(key, versionStr, "OptionsDataFlowSet").
 				Inc()
-			nd.metrics.setRecordsStatsSum.WithLabelValues(key, versionStr, "OptionsDataFlowSet").
+			nd.metrics.records.WithLabelValues(key, versionStr, "OptionsDataFlowSet").
 				Add(float64(len(fsConv.Records)))
 		case netflow.DataFlowSet:
-			nd.metrics.setStatsSum.WithLabelValues(key, versionStr, "DataFlowSet").
+			nd.metrics.sets.WithLabelValues(key, versionStr, "DataFlowSet").
 				Inc()
-			nd.metrics.setRecordsStatsSum.WithLabelValues(key, versionStr, "DataFlowSet").
+			nd.metrics.records.WithLabelValues(key, versionStr, "DataFlowSet").
 				Add(float64(len(fsConv.Records)))
+			nb += len(fsConv.Records)
 		}
 	}
 
-	exporterAddress, _ := netip.AddrFromSlice(in.Source.To16())
-	for _, fmsg := range flowMessageSet {
-		if fmsg.TimeReceived == 0 {
-			fmsg.TimeReceived = ts
-		}
-		fmsg.ExporterAddress = exporterAddress
-	}
-
-	return flowMessageSet
+	return nb, nil
 }
 
 // Name returns the name of the decoder.
diff --git a/inlet/flow/decoder/root.go b/outlet/flow/decoder/root.go
similarity index 54%
rename from inlet/flow/decoder/root.go
rename to outlet/flow/decoder/root.go
index 3d62d902..a4d4db5c 100644
--- a/inlet/flow/decoder/root.go
+++ b/outlet/flow/decoder/root.go
@@ -6,19 +6,21 @@
 package decoder
 
 import (
-	"net"
+	"net/netip"
 	"time"
 
+	"akvorado/common/pb"
 	"akvorado/common/reporter"
 	"akvorado/common/schema"
 )
 
 // Decoder is the interface each decoder should implement.
 type Decoder interface {
-	// Decoder takes a raw flow and returns a
-	// slice of flow messages. Returning nil means there was an
-	// error during decoding.
-	Decode(in RawFlow) []*schema.FlowMessage
+	// Decoder takes a raw flow and options. It should enqueue new flows in the
+	// provided flow message. When a flow is enqueted, it will call the finalize
+	// function. It is important to not set an error once the flow is being
+	// built (as there is no rollback possible).
+	Decode(in RawFlow, options Option, bf *schema.FlowMessage, finalize FinalizeFlowFunc) (int, error)
 
 	// Name returns the decoder name
 	Name() string
@@ -27,7 +29,7 @@ type Decoder interface {
 // Option specifies option to influence the behaviour of the decoder
 type Option struct {
 	// TimestampSource is a selector for how to set the TimeReceived.
-	TimestampSource TimestampSource
+	TimestampSource pb.RawFlow_TimestampSource
 }
 
 // Dependencies are the dependencies for the decoder
@@ -39,8 +41,12 @@ type Dependencies struct {
 type RawFlow struct {
 	TimeReceived time.Time
 	Payload      []byte
-	Source       net.IP
+	Source       netip.Addr
 }
 
 // NewDecoderFunc is the signature of a function to instantiate a decoder.
-type NewDecoderFunc func(*reporter.Reporter, Dependencies, Option) Decoder
+type NewDecoderFunc func(*reporter.Reporter, Dependencies) Decoder
+
+// FinalizeFlowFunc is the signature of a function to finalize a flow. The
+// caller has a reference to the flow message he provided.
+type FinalizeFlowFunc func()
diff --git a/inlet/flow/decoder/sflow/decode.go b/outlet/flow/decoder/sflow/decode.go
similarity index 64%
rename from inlet/flow/decoder/sflow/decode.go
rename to outlet/flow/decoder/sflow/decode.go
index e1d91324..812cf2f3 100644
--- a/inlet/flow/decoder/sflow/decode.go
+++ b/outlet/flow/decoder/sflow/decode.go
@@ -5,35 +5,35 @@
 package sflow
 
 import (
+	"net"
+
 	"akvorado/common/helpers"
 	"akvorado/common/schema"
-	"akvorado/inlet/flow/decoder"
+	"akvorado/outlet/flow/decoder"
 
 	"github.com/netsampler/goflow2/v2/decoders/sflow"
 )
 
-func (nd *Decoder) decode(packet sflow.Packet) []*schema.FlowMessage {
-	flowMessageSet := []*schema.FlowMessage{}
-
+func (nd *Decoder) decode(packet sflow.Packet, bf *schema.FlowMessage, finalize decoder.FinalizeFlowFunc) error {
 	for _, flowSample := range packet.Samples {
 		var records []sflow.FlowRecord
-		bf := &schema.FlowMessage{}
 		forwardingStatus := 0
 		switch flowSample := flowSample.(type) {
 		case sflow.FlowSample:
 			records = flowSample.Records
-			bf.SamplingRate = flowSample.SamplingRate
+			bf.SamplingRate = uint64(flowSample.SamplingRate)
 			bf.InIf = flowSample.Input
 			bf.OutIf = flowSample.Output
-			if bf.OutIf&interfaceOutMask == interfaceOutDiscard {
+			switch bf.OutIf & interfaceOutMask {
+			case interfaceOutDiscard:
 				bf.OutIf = 0
 				forwardingStatus = 128
-			} else if bf.OutIf&interfaceOutMask == interfaceOutMultiple {
+			case interfaceOutMultiple:
 				bf.OutIf = 0
 			}
 		case sflow.ExpandedFlowSample:
 			records = flowSample.Records
-			bf.SamplingRate = flowSample.SamplingRate
+			bf.SamplingRate = uint64(flowSample.SamplingRate)
 			bf.InIf = flowSample.InputIfValue
 			bf.OutIf = flowSample.OutputIfValue
 		}
@@ -46,8 +46,8 @@ func (nd *Decoder) decode(packet sflow.Packet) []*schema.FlowMessage {
 		}
 
 		bf.ExporterAddress = decoder.DecodeIP(packet.AgentIP)
-		nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnPackets, 1)
-		nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnForwardingStatus, uint64(forwardingStatus))
+		bf.AppendUint(schema.ColumnPackets, 1)
+		bf.AppendUint(schema.ColumnForwardingStatus, uint64(forwardingStatus))
 
 		// Optimization: avoid parsing sampled header if we have everything already parsed
 		hasSampledIPv4 := false
@@ -84,20 +84,20 @@ func (nd *Decoder) decode(packet sflow.Packet) []*schema.FlowMessage {
 				bf.SrcAddr = decoder.DecodeIP(recordData.SrcIP)
 				bf.DstAddr = decoder.DecodeIP(recordData.DstIP)
 				l3length = uint64(recordData.Length)
-				nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnProto, uint64(recordData.Protocol))
-				nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnSrcPort, uint64(recordData.SrcPort))
-				nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnDstPort, uint64(recordData.DstPort))
-				nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnEType, helpers.ETypeIPv4)
-				nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnIPTos, uint64(recordData.Tos))
+				bf.AppendUint(schema.ColumnProto, uint64(recordData.Protocol))
+				bf.AppendUint(schema.ColumnSrcPort, uint64(recordData.SrcPort))
+				bf.AppendUint(schema.ColumnDstPort, uint64(recordData.DstPort))
+				bf.AppendUint(schema.ColumnEType, helpers.ETypeIPv4)
+				bf.AppendUint(schema.ColumnIPTos, uint64(recordData.Tos))
 			case sflow.SampledIPv6:
 				bf.SrcAddr = decoder.DecodeIP(recordData.SrcIP)
 				bf.DstAddr = decoder.DecodeIP(recordData.DstIP)
 				l3length = uint64(recordData.Length)
-				nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnProto, uint64(recordData.Protocol))
-				nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnSrcPort, uint64(recordData.SrcPort))
-				nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnDstPort, uint64(recordData.DstPort))
-				nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnEType, helpers.ETypeIPv6)
-				nd.d.Schema.ProtobufAppendVarint(bf, schema.ColumnIPTos, uint64(recordData.Priority))
+				bf.AppendUint(schema.ColumnProto, uint64(recordData.Protocol))
+				bf.AppendUint(schema.ColumnSrcPort, uint64(recordData.SrcPort))
+				bf.AppendUint(schema.ColumnDstPort, uint64(recordData.DstPort))
+				bf.AppendUint(schema.ColumnEType, helpers.ETypeIPv6)
+				bf.AppendUint(schema.ColumnIPTos, uint64(recordData.Priority))
 			case sflow.SampledEthernet:
 				if l3length == 0 {
 					// That's the best we can guess. sFlow says: For a layer 2
@@ -107,8 +107,8 @@ func (nd *Decoder) decode(packet sflow.Packet) []*schema.FlowMessage {
 					l3length = uint64(recordData.Length) - 16
 				}
 				if !nd.d.Schema.IsDisabled(schema.ColumnGroupL2) {
-					nd.d.Schema.ProtobufAppendBytes(bf, schema.ColumnSrcMAC, recordData.SrcMac)
-					nd.d.Schema.ProtobufAppendBytes(bf, schema.ColumnDstMAC, recordData.DstMac)
+					bf.AppendUint(schema.ColumnSrcMAC, helpers.MACToUint64(net.HardwareAddr(recordData.SrcMac)))
+					bf.AppendUint(schema.ColumnDstMAC, helpers.MACToUint64(net.HardwareAddr(recordData.DstMac)))
 				}
 			case sflow.ExtendedSwitch:
 				if !nd.d.Schema.IsDisabled(schema.ColumnGroupL2) {
@@ -132,31 +132,22 @@ func (nd *Decoder) decode(packet sflow.Packet) []*schema.FlowMessage {
 				}
 				if len(recordData.ASPath) > 0 {
 					bf.DstAS = recordData.ASPath[len(recordData.ASPath)-1]
-					if column, _ := nd.d.Schema.LookupColumnByKey(schema.ColumnDstASPath); !column.Disabled {
-						for _, asn := range recordData.ASPath {
-							column.ProtobufAppendVarint(bf, uint64(asn))
-						}
-					}
-					bf.GotASPath = true
+					bf.AppendArrayUInt32(schema.ColumnDstASPath, recordData.ASPath)
 				}
 				if len(recordData.Communities) > 0 {
-					if column, _ := nd.d.Schema.LookupColumnByKey(schema.ColumnDstCommunities); !column.Disabled {
-						for _, comm := range recordData.Communities {
-							column.ProtobufAppendVarint(bf, uint64(comm))
-						}
-					}
-					bf.GotCommunities = true
+					bf.AppendArrayUInt32(schema.ColumnDstCommunities, recordData.Communities)
 				}
 			}
 		}
 
 		if l3length > 0 {
-			nd.d.Schema.ProtobufAppendVarintForce(bf, schema.ColumnBytes, l3length)
+			bf.AppendUint(schema.ColumnBytes, l3length)
 		}
-		flowMessageSet = append(flowMessageSet, bf)
+
+		finalize()
 	}
 
-	return flowMessageSet
+	return nil
 }
 
 func (nd *Decoder) parseSampledHeader(bf *schema.FlowMessage, header *sflow.SampledHeader) uint64 {
diff --git a/inlet/flow/decoder/sflow/root.go b/outlet/flow/decoder/sflow/root.go
similarity index 90%
rename from inlet/flow/decoder/sflow/root.go
rename to outlet/flow/decoder/sflow/root.go
index 0798fc27..73782339 100644
--- a/inlet/flow/decoder/sflow/root.go
+++ b/outlet/flow/decoder/sflow/root.go
@@ -6,6 +6,7 @@ package sflow
 
 import (
 	"bytes"
+	"fmt"
 	"net"
 	"time"
 
@@ -13,7 +14,7 @@ import (
 
 	"akvorado/common/reporter"
 	"akvorado/common/schema"
-	"akvorado/inlet/flow/decoder"
+	"akvorado/outlet/flow/decoder"
 )
 
 const (
@@ -43,7 +44,7 @@ type Decoder struct {
 }
 
 // New instantiates a new sFlow decoder.
-func New(r *reporter.Reporter, dependencies decoder.Dependencies, _ decoder.Option) decoder.Decoder {
+func New(r *reporter.Reporter, dependencies decoder.Dependencies) decoder.Decoder {
 	nd := &Decoder{
 		r:         r,
 		d:         dependencies,
@@ -83,16 +84,16 @@ func New(r *reporter.Reporter, dependencies decoder.Dependencies, _ decoder.Opti
 }
 
 // Decode decodes an sFlow payload.
-func (nd *Decoder) Decode(in decoder.RawFlow) []*schema.FlowMessage {
+func (nd *Decoder) Decode(in decoder.RawFlow, _ decoder.Option, bf *schema.FlowMessage, finalize decoder.FinalizeFlowFunc) (int, error) {
 	buf := bytes.NewBuffer(in.Payload)
 	key := in.Source.String()
-
 	ts := uint64(in.TimeReceived.UTC().Unix())
+
 	var packet sflow.Packet
 	if err := sflow.DecodeMessageVersion(buf, &packet); err != nil {
 		nd.metrics.errors.WithLabelValues(key, "sFlow decoding error").Inc()
 		nd.errLogger.Err(err).Str("exporter", key).Msg("error while decoding sFlow")
-		return nil
+		return 0, fmt.Errorf("error while decoding sFlow: %w", err)
 	}
 
 	// Update some stats
@@ -120,12 +121,10 @@ func (nd *Decoder) Decode(in decoder.RawFlow) []*schema.FlowMessage {
 		}
 	}
 
-	flowMessageSet := nd.decode(packet)
-	for _, fmsg := range flowMessageSet {
-		fmsg.TimeReceived = ts
-	}
-
-	return flowMessageSet
+	return len(samples), nd.decode(packet, bf, func() {
+		bf.TimeReceived = uint32(ts)
+		finalize()
+	})
 }
 
 // Name returns the name of the decoder.
diff --git a/outlet/flow/root.go b/outlet/flow/root.go
new file mode 100644
index 00000000..a8aec9a1
--- /dev/null
+++ b/outlet/flow/root.go
@@ -0,0 +1,67 @@
+// SPDX-FileCopyrightText: 2022 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+// Package flow handles flow decoding from protobuf messages.
+package flow
+
+import (
+	"time"
+
+	"akvorado/common/pb"
+	"akvorado/common/reporter"
+	"akvorado/common/schema"
+	"akvorado/outlet/flow/decoder"
+)
+
+// Component represents the flow decoder component.
+type Component struct {
+	r         *reporter.Reporter
+	d         *Dependencies
+	errLogger reporter.Logger
+
+	metrics struct {
+		decoderStats  *reporter.CounterVec
+		decoderErrors *reporter.CounterVec
+	}
+
+	// Available decoders
+	decoders map[pb.RawFlow_Decoder]decoder.Decoder
+}
+
+// Dependencies are the dependencies of the flow component.
+type Dependencies struct {
+	Schema *schema.Component
+}
+
+// New creates a new flow component.
+func New(r *reporter.Reporter, dependencies Dependencies) (*Component, error) {
+	c := Component{
+		r:         r,
+		d:         &dependencies,
+		errLogger: r.Sample(reporter.BurstSampler(30*time.Second, 3)),
+		decoders:  make(map[pb.RawFlow_Decoder]decoder.Decoder),
+	}
+
+	// Initialize available decoders
+	for decoderType, decoderFunc := range availableDecoders {
+		c.decoders[decoderType] = decoderFunc(r, decoder.Dependencies{Schema: c.d.Schema})
+	}
+
+	// Metrics
+	c.metrics.decoderStats = c.r.CounterVec(
+		reporter.CounterOpts{
+			Name: "decoder_flows_total",
+			Help: "Decoder processed count.",
+		},
+		[]string{"name"},
+	)
+	c.metrics.decoderErrors = c.r.CounterVec(
+		reporter.CounterOpts{
+			Name: "decoder_errors_total",
+			Help: "Decoder processed error count.",
+		},
+		[]string{"name"},
+	)
+
+	return &c, nil
+}
diff --git a/outlet/kafka/config.go b/outlet/kafka/config.go
new file mode 100644
index 00000000..739ecd56
--- /dev/null
+++ b/outlet/kafka/config.go
@@ -0,0 +1,40 @@
+// SPDX-FileCopyrightText: 2022 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+package kafka
+
+import (
+	"time"
+
+	"akvorado/common/helpers"
+	"akvorado/common/kafka"
+)
+
+// Configuration describes the configuration for the Kafka exporter.
+type Configuration struct {
+	kafka.Configuration `mapstructure:",squash" yaml:"-,inline"`
+	// Workers define the number of workers to read messages from Kafka.
+	Workers int `validate:"min=1"`
+	// ConsumerGroup is the name of the consumer group to use
+	ConsumerGroup string `validate:"min=1,ascii"`
+	// FetchMinBytes is the minimum number of bytes to wait before fetching a message.
+	FetchMinBytes int32 `validate:"min=1"`
+	// FetchMaxWaitTime is the minimum duration to wait to get at least the
+	// minimum number of bytes.
+	FetchMaxWaitTime time.Duration `validate:"min=100ms"`
+}
+
+// DefaultConfiguration represents the default configuration for the Kafka exporter.
+func DefaultConfiguration() Configuration {
+	return Configuration{
+		Configuration:    kafka.DefaultConfiguration(),
+		Workers:          1,
+		ConsumerGroup:    "akvorado-outlet",
+		FetchMinBytes:    1_000_000,
+		FetchMaxWaitTime: time.Second,
+	}
+}
+
+func init() {
+	helpers.RegisterMapstructureDeprecatedFields[Configuration]("MaxMessageBytes", "QueueSize")
+}
diff --git a/outlet/kafka/consumer.go b/outlet/kafka/consumer.go
new file mode 100644
index 00000000..986e5021
--- /dev/null
+++ b/outlet/kafka/consumer.go
@@ -0,0 +1,86 @@
+// SPDX-FileCopyrightText: 2024 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+package kafka
+
+import (
+	"context"
+	"errors"
+	"strconv"
+	"sync"
+
+	"github.com/rs/zerolog"
+	"github.com/twmb/franz-go/pkg/kgo"
+
+	"akvorado/common/reporter"
+)
+
+// ErrStopProcessing should be returned as an error when we need to stop processing more flows.
+var ErrStopProcessing = errors.New("stop processing further flows")
+
+// Consumer is a franz-go consumer and should process flow messages.
+type Consumer struct {
+	r *reporter.Reporter
+	l zerolog.Logger
+
+	metrics  metrics
+	worker   int
+	callback ReceiveFunc
+	mu       sync.Mutex
+}
+
+// ReceiveFunc is a function that will be called with each received messages.
+type ReceiveFunc func(context.Context, []byte) error
+
+// ShutdownFunc is a function that will be called on shutdown of the consumer.
+type ShutdownFunc func()
+
+// WorkerBuilderFunc returns a function to be called with each received messages and
+// a function to be called when shutting down.
+type WorkerBuilderFunc func(int) (ReceiveFunc, ShutdownFunc)
+
+// NewConsumer creates a new consumer.
+func (c *realComponent) NewConsumer(worker int, callback ReceiveFunc) *Consumer {
+	return &Consumer{
+		r: c.r,
+		l: c.r.With().Int("worker", worker).Logger(),
+
+		worker:   worker,
+		metrics:  c.metrics,
+		callback: callback,
+	}
+}
+
+// ProcessFetches processes the fetched records.
+func (c *Consumer) ProcessFetches(ctx context.Context, fetches kgo.Fetches) error {
+	worker := strconv.Itoa(c.worker)
+	c.metrics.fetchesReceived.WithLabelValues(worker).Inc()
+
+	if errs := fetches.Errors(); len(errs) > 0 {
+		for _, err := range errs {
+			if errors.Is(err.Err, context.Canceled) {
+				return nil
+			}
+			c.metrics.errorsReceived.WithLabelValues(worker).Inc()
+			c.l.Err(err.Err).
+				Str("topic", err.Topic).
+				Int32("partition", err.Partition).
+				Msg("fetch error")
+		}
+		// Assume the error is fatal.
+		return ErrStopProcessing
+	}
+
+	messagesReceived := c.metrics.messagesReceived.WithLabelValues(worker)
+	bytesReceived := c.metrics.bytesReceived.WithLabelValues(worker)
+	for iter := fetches.RecordIter(); !iter.Done(); {
+		record := iter.Next()
+		messagesReceived.Inc()
+		bytesReceived.Add(float64(len(record.Value)))
+		if err := c.callback(ctx, record.Value); err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
diff --git a/outlet/kafka/metrics.go b/outlet/kafka/metrics.go
new file mode 100644
index 00000000..96faf522
--- /dev/null
+++ b/outlet/kafka/metrics.go
@@ -0,0 +1,46 @@
+// SPDX-FileCopyrightText: 2022 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+package kafka
+
+import (
+	"akvorado/common/reporter"
+)
+
+type metrics struct {
+	messagesReceived *reporter.CounterVec
+	fetchesReceived  *reporter.CounterVec
+	bytesReceived    *reporter.CounterVec
+	errorsReceived   *reporter.CounterVec
+}
+
+func (c *realComponent) initMetrics() {
+	c.metrics.messagesReceived = c.r.CounterVec(
+		reporter.CounterOpts{
+			Name: "received_messages_total",
+			Help: "Number of messages received for a given worker.",
+		},
+		[]string{"worker"},
+	)
+	c.metrics.fetchesReceived = c.r.CounterVec(
+		reporter.CounterOpts{
+			Name: "received_fetches_total",
+			Help: "Number of fetches received for a given worker.",
+		},
+		[]string{"worker"},
+	)
+	c.metrics.bytesReceived = c.r.CounterVec(
+		reporter.CounterOpts{
+			Name: "received_bytes_total",
+			Help: "Number of bytes received for a given worker.",
+		},
+		[]string{"worker"},
+	)
+	c.metrics.errorsReceived = c.r.CounterVec(
+		reporter.CounterOpts{
+			Name: "received_errors_total",
+			Help: "Number of errors while handling received messages for a given worker.",
+		},
+		[]string{"worker"},
+	)
+}
diff --git a/outlet/kafka/root.go b/outlet/kafka/root.go
new file mode 100644
index 00000000..4c205619
--- /dev/null
+++ b/outlet/kafka/root.go
@@ -0,0 +1,152 @@
+// SPDX-FileCopyrightText: 2022 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+// Package kafka handles flow imports from Kafka.
+package kafka
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"strconv"
+	"strings"
+
+	"github.com/prometheus/client_golang/prometheus"
+	"github.com/twmb/franz-go/pkg/kgo"
+	"github.com/twmb/franz-go/plugin/kprom"
+	"gopkg.in/tomb.v2"
+
+	"akvorado/common/daemon"
+	"akvorado/common/kafka"
+	"akvorado/common/pb"
+	"akvorado/common/reporter"
+)
+
+// Component is the interface a Kafka consumer should implement.
+type Component interface {
+	StartWorkers(WorkerBuilderFunc) error
+	Stop() error
+}
+
+// realComponent implements the Kafka consumer.
+type realComponent struct {
+	r      *reporter.Reporter
+	d      *Dependencies
+	t      tomb.Tomb
+	config Configuration
+
+	kafkaOpts []kgo.Opt
+
+	clients []*kgo.Client
+	metrics metrics
+}
+
+// Dependencies define the dependencies of the Kafka exporter.
+type Dependencies struct {
+	Daemon daemon.Component
+}
+
+// New creates a new Kafka exporter component.
+func New(r *reporter.Reporter, configuration Configuration, dependencies Dependencies) (Component, error) {
+	// Build Kafka configuration
+	kafkaOpts, err := kafka.NewConfig(r, configuration.Configuration)
+	if err != nil {
+		return nil, err
+	}
+
+	c := realComponent{
+		r:      r,
+		d:      &dependencies,
+		config: configuration,
+	}
+	c.initMetrics()
+
+	kafkaOpts = append(kafkaOpts,
+		kgo.FetchMinBytes(configuration.FetchMinBytes),
+		kgo.FetchMaxWait(configuration.FetchMaxWaitTime),
+		kgo.ConsumerGroup(configuration.ConsumerGroup),
+		kgo.ConsumeStartOffset(kgo.NewOffset().AtEnd()),
+		kgo.ConsumeTopics(fmt.Sprintf("%s-v%d", configuration.Topic, pb.Version)),
+		// Do not use kgo.BlockRebalanceOnPoll(). It needs more code to ensure
+		// we are not blocked while polling.
+	)
+
+	if err := kgo.ValidateOpts(kafkaOpts...); err != nil {
+		return nil, fmt.Errorf("invalid Kafka configuration: %w", err)
+	}
+	c.kafkaOpts = kafkaOpts
+	c.d.Daemon.Track(&c.t, "outlet/kafka")
+	return &c, nil
+}
+
+// Start starts the Kafka component.
+func (c *realComponent) Start() error {
+	c.r.Info().Msg("starting Kafka component")
+	// Start the clients
+	for i := range c.config.Workers {
+		logger := c.r.With().Int("worker", i).Logger()
+		logger.Debug().Msg("starting")
+
+		kafkaMetrics := kprom.NewMetrics("", kprom.WithStaticLabel(prometheus.Labels{"worker": strconv.Itoa(i)}))
+		kafkaOpts := append(c.kafkaOpts, kgo.WithHooks(kafkaMetrics))
+		c.r.MetricCollectorForCurrentModule(kafkaMetrics)
+		client, err := kgo.NewClient(kafkaOpts...)
+		if err != nil {
+			logger.Err(err).
+				Int("worker", i).
+				Str("brokers", strings.Join(c.config.Brokers, ",")).
+				Msg("unable to create new client")
+			return fmt.Errorf("unable to create Kafka client: %w", err)
+		}
+		c.clients = append(c.clients, client)
+	}
+	return nil
+}
+
+// StartWorkers will start the workers. This should only be called once.
+func (c *realComponent) StartWorkers(workerBuilder WorkerBuilderFunc) error {
+	ctx := c.t.Context(context.Background())
+	for i := range c.config.Workers {
+		callback, shutdown := workerBuilder(i)
+		consumer := c.NewConsumer(i, callback)
+		client := c.clients[i]
+		c.t.Go(func() error {
+			logger := c.r.With().
+				Int("worker", i).
+				Logger()
+			defer shutdown()
+
+			for {
+				select {
+				case <-ctx.Done():
+					return nil
+				default:
+					fetches := client.PollFetches(ctx)
+					if err := consumer.ProcessFetches(ctx, fetches); err != nil {
+						if errors.Is(err, context.Canceled) || errors.Is(err, ErrStopProcessing) {
+							return nil
+						}
+						logger.Err(err).
+							Int("worker", i).
+							Msg("cannot process fetched messages")
+						return fmt.Errorf("cannot process fetched messages: %w", err)
+					}
+				}
+			}
+		})
+	}
+	return nil
+}
+
+// Stop stops the Kafka component
+func (c *realComponent) Stop() error {
+	defer func() {
+		for _, client := range c.clients {
+			client.CloseAllowingRebalance()
+		}
+		c.r.Info().Msg("Kafka component stopped")
+	}()
+	c.r.Info().Msg("stopping Kafka component")
+	c.t.Kill(nil)
+	return c.t.Wait()
+}
diff --git a/inlet/metadata/cache.go b/outlet/metadata/cache.go
similarity index 98%
rename from inlet/metadata/cache.go
rename to outlet/metadata/cache.go
index 0f283db3..97503592 100644
--- a/inlet/metadata/cache.go
+++ b/outlet/metadata/cache.go
@@ -9,7 +9,7 @@ import (
 
 	"akvorado/common/helpers/cache"
 	"akvorado/common/reporter"
-	"akvorado/inlet/metadata/provider"
+	"akvorado/outlet/metadata/provider"
 )
 
 // Interface describes an interface.
diff --git a/inlet/metadata/config.go b/outlet/metadata/config.go
similarity index 73%
rename from inlet/metadata/config.go
rename to outlet/metadata/config.go
index ff4e07e5..28ad06e5 100644
--- a/inlet/metadata/config.go
+++ b/outlet/metadata/config.go
@@ -7,10 +7,10 @@ import (
 	"time"
 
 	"akvorado/common/helpers"
-	"akvorado/inlet/metadata/provider"
-	"akvorado/inlet/metadata/provider/gnmi"
-	"akvorado/inlet/metadata/provider/snmp"
-	"akvorado/inlet/metadata/provider/static"
+	"akvorado/outlet/metadata/provider"
+	"akvorado/outlet/metadata/provider/gnmi"
+	"akvorado/outlet/metadata/provider/snmp"
+	"akvorado/outlet/metadata/provider/static"
 )
 
 // Configuration describes the configuration for the metadata client
@@ -27,10 +27,11 @@ type Configuration struct {
 	// Provider defines the configuration of the providers to use
 	Providers []ProviderConfiguration
 
-	// Workers define the number of workers used to poll metadata
-	Workers int `validate:"min=1"`
-	// MaxBatchRequests define how many requests to pass to a worker at once if possible
-	MaxBatchRequests int `validate:"min=0"`
+	// QueryTimeout defines how long to wait for a provider to answer.
+	QueryTimeout time.Duration `validate:"min=100ms,max=1m"`
+	// InitialDelay defines how long to wait at start (when receiving the first
+	// packets) before applying the query timeout
+	InitialDelay time.Duration `validate:"min=1s,max=1h"`
 }
 
 // DefaultConfiguration represents the default configuration for the metadata provider.
@@ -39,9 +40,8 @@ func DefaultConfiguration() Configuration {
 		CacheDuration:      30 * time.Minute,
 		CacheRefresh:       time.Hour,
 		CacheCheckInterval: 2 * time.Minute,
-		CachePersistFile:   "",
-		Workers:            1,
-		MaxBatchRequests:   10,
+		QueryTimeout:       5 * time.Second,
+		InitialDelay:       time.Minute,
 	}
 }
 
@@ -52,7 +52,7 @@ type ProviderConfiguration struct {
 }
 
 // MarshalYAML undoes ConfigurationUnmarshallerHook().
-func (pc ProviderConfiguration) MarshalYAML() (interface{}, error) {
+func (pc ProviderConfiguration) MarshalYAML() (any, error) {
 	return helpers.ParametrizedConfigurationMarshalYAML(pc, providers)
 }
 
@@ -67,4 +67,5 @@ func init() {
 		helpers.RenameKeyUnmarshallerHook(Configuration{}, "Provider", "Providers"))
 	helpers.RegisterMapstructureUnmarshallerHook(
 		helpers.ParametrizedConfigurationUnmarshallerHook(ProviderConfiguration{}, providers))
+	helpers.RegisterMapstructureDeprecatedFields[Configuration]("Workers", "MaxBatchRequests")
 }
diff --git a/inlet/metadata/provider/gnmi/collector.go b/outlet/metadata/provider/gnmi/collector.go
similarity index 88%
rename from inlet/metadata/provider/gnmi/collector.go
rename to outlet/metadata/provider/gnmi/collector.go
index e3a4700a..482dc72c 100644
--- a/inlet/metadata/provider/gnmi/collector.go
+++ b/outlet/metadata/provider/gnmi/collector.go
@@ -12,7 +12,7 @@ import (
 	"strings"
 	"time"
 
-	"akvorado/inlet/metadata/provider"
+	"akvorado/outlet/metadata/provider"
 
 	"github.com/cenkalti/backoff/v4"
 	"github.com/openconfig/gnmic/pkg/api"
@@ -22,7 +22,8 @@ import (
 // exporterState is the state of an exporter.
 type exporterState struct {
 	Name       string
-	Ready      bool
+	ready      bool      // ready for the first time
+	Ready      chan bool // not polling, data ready
 	Interfaces map[uint]provider.Interface
 }
 
@@ -243,49 +244,70 @@ retryDetect:
 		l.Debug().Msg("polling")
 		start := time.Now()
 		subscribeResp, err := tg.SubscribeOnce(ctx, subscribeReq)
-		p.metrics.times.WithLabelValues(exporterStr).Observe(time.Now().Sub(start).Seconds())
+		p.metrics.times.WithLabelValues(exporterStr).Observe(time.Since(start).Seconds())
 		if err == nil {
 			events := subscribeResponsesToEvents(subscribeResp)
 			p.metrics.paths.WithLabelValues(exporterStr).Set(float64(len(events)))
 			p.stateLock.Lock()
 			state.update(events, model)
-			state.Ready = true
+			state.ready = true
 			p.stateLock.Unlock()
 			l.Debug().Msg("state updated")
 			p.metrics.ready.WithLabelValues(exporterStr).Set(1)
 			p.metrics.updates.WithLabelValues(exporterStr).Inc()
 
-			// On success, wait a bit before next refresh interval
+			// In the following window, we consider ourselves ready and unlock
+			// waiting clients to check for data.
+
+			// On success, wait a bit before next refresh interval and ignore
+			// any refresh requests.
 			next := time.NewTimer(p.config.MinimalRefreshInterval)
-			select {
-			case <-ctx.Done():
-				next.Stop()
-				return
-			case <-next.C:
-			}
-			// Drain any message in refresh queue (we ignore them)
-			select {
-			case <-p.refresh:
-			default:
+		outerWaitRefreshTimer:
+			for {
+				select {
+				case state.Ready <- true:
+				case <-ctx.Done():
+					next.Stop()
+					return
+				case <-p.refresh:
+				case <-next.C:
+					break outerWaitRefreshTimer
+				}
 			}
 			// Wait for a new message in refresh queue
-			select {
-			case <-ctx.Done():
-				return
-			case <-p.refresh:
+			l.Debug().Msg("wait for refresh request")
+		outerWaitRefresh:
+			for {
+				select {
+				case state.Ready <- true:
+				case <-ctx.Done():
+					return
+				case <-p.refresh:
+					break outerWaitRefresh
+				}
 			}
 			// Reset retry timer and do the next fresh
 			retryFetchBackoff.Reset()
 		} else {
-			// On error, retry a bit later
+			// On error, retry a bit later. While retrying, if we have an
+			// initial state, consider ourselves ready.
 			l.Err(err).Msg("cannot poll")
 			p.metrics.errors.WithLabelValues(exporterStr, "cannot poll").Inc()
 			next := time.NewTimer(retryFetchBackoff.NextBackOff())
-			select {
-			case <-ctx.Done():
-				next.Stop()
-				return
-			case <-next.C:
+			var readyChan chan bool
+			if state.ready {
+				readyChan = state.Ready
+			}
+		outerWaitRetryTimer:
+			for {
+				select {
+				case readyChan <- true:
+				case <-ctx.Done():
+					next.Stop()
+					return
+				case <-next.C:
+					break outerWaitRetryTimer
+				}
 			}
 		}
 	}
diff --git a/inlet/metadata/provider/gnmi/config.go b/outlet/metadata/provider/gnmi/config.go
similarity index 99%
rename from inlet/metadata/provider/gnmi/config.go
rename to outlet/metadata/provider/gnmi/config.go
index 05749826..dc67c624 100644
--- a/inlet/metadata/provider/gnmi/config.go
+++ b/outlet/metadata/provider/gnmi/config.go
@@ -9,7 +9,7 @@ import (
 	"time"
 
 	"akvorado/common/helpers"
-	"akvorado/inlet/metadata/provider"
+	"akvorado/outlet/metadata/provider"
 
 	"github.com/go-viper/mapstructure/v2"
 )
diff --git a/inlet/metadata/provider/gnmi/metrics.go b/outlet/metadata/provider/gnmi/metrics.go
similarity index 100%
rename from inlet/metadata/provider/gnmi/metrics.go
rename to outlet/metadata/provider/gnmi/metrics.go
diff --git a/outlet/metadata/provider/gnmi/root.go b/outlet/metadata/provider/gnmi/root.go
new file mode 100644
index 00000000..9f094859
--- /dev/null
+++ b/outlet/metadata/provider/gnmi/root.go
@@ -0,0 +1,87 @@
+// SPDX-FileCopyrightText: 2023 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+// Package gnmi use gNMI to get interface names and descriptions.
+package gnmi
+
+import (
+	"context"
+	"net/netip"
+	"sync"
+
+	"akvorado/common/reporter"
+	"akvorado/outlet/metadata/provider"
+)
+
+// Provider represents the gNMI provider.
+type Provider struct {
+	r       *reporter.Reporter
+	config  *Configuration
+	metrics metrics
+
+	state     map[netip.Addr]*exporterState
+	stateLock sync.Mutex
+	refresh   chan bool
+}
+
+// New creates a new gNMI provider from configuration
+func (configuration Configuration) New(r *reporter.Reporter) (provider.Provider, error) {
+	p := Provider{
+		r:       r,
+		config:  &configuration,
+		state:   map[netip.Addr]*exporterState{},
+		refresh: make(chan bool),
+	}
+	p.initMetrics()
+	return &p, nil
+}
+
+// Query queries exporter to get information through gNMI.
+func (p *Provider) Query(ctx context.Context, q provider.Query) (provider.Answer, error) {
+	p.stateLock.Lock()
+	state, ok := p.state[q.ExporterIP]
+	if !ok {
+		state = &exporterState{
+			Ready: make(chan bool),
+		}
+		p.state[q.ExporterIP] = state
+		p.metrics.collectorCount.Inc()
+		go p.startCollector(ctx, q.ExporterIP, state)
+	}
+
+	// Trigger a refresh
+	select {
+	case p.refresh <- true:
+	default:
+	}
+
+	// Wait for the collector to be ready.
+	select {
+	case <-state.Ready:
+		// Most common case, keep the lock
+	default:
+		// Not ready, release the lock until ready
+		p.stateLock.Unlock()
+		select {
+		case <-state.Ready:
+			p.stateLock.Lock()
+		case <-ctx.Done():
+			p.metrics.errors.WithLabelValues(q.ExporterIP.Unmap().String(), "not ready").Inc()
+			return provider.Answer{}, ctx.Err()
+		}
+	}
+	defer p.stateLock.Unlock()
+
+	// Return the result from the state
+	iface, ok := state.Interfaces[q.IfIndex]
+	if !ok {
+		return provider.Answer{}, nil
+	}
+	return provider.Answer{
+		Found: true,
+		Exporter: provider.Exporter{
+			Name: state.Name,
+		},
+		Interface: iface,
+	}, nil
+}
diff --git a/inlet/metadata/provider/gnmi/subscribe.go b/outlet/metadata/provider/gnmi/subscribe.go
similarity index 100%
rename from inlet/metadata/provider/gnmi/subscribe.go
rename to outlet/metadata/provider/gnmi/subscribe.go
diff --git a/inlet/metadata/provider/gnmi/utils.go b/outlet/metadata/provider/gnmi/utils.go
similarity index 100%
rename from inlet/metadata/provider/gnmi/utils.go
rename to outlet/metadata/provider/gnmi/utils.go
diff --git a/inlet/metadata/provider/root.go b/outlet/metadata/provider/root.go
similarity index 75%
rename from inlet/metadata/provider/root.go
rename to outlet/metadata/provider/root.go
index 87a43ebf..f6a51b31 100644
--- a/inlet/metadata/provider/root.go
+++ b/outlet/metadata/provider/root.go
@@ -13,9 +13,11 @@ import (
 	"akvorado/common/schema"
 )
 
-// ErrSkipProvider is the error returned on lookup for providers unwilling to
-// handle a request.
-var ErrSkipProvider = errors.New("provider skips query")
+var (
+	// ErrSkipProvider is the error returned on lookup for providers unwilling to
+	// handle a request.
+	ErrSkipProvider = errors.New("provider skips query")
+)
 
 // Interface contains the information about an interface.
 type Interface struct {
@@ -49,32 +51,24 @@ type Query struct {
 	IfIndex    uint
 }
 
-// BatchQuery is a batched query.
-type BatchQuery struct {
-	ExporterIP netip.Addr
-	IfIndexes  []uint
-}
-
 // Answer is the answer received from a provider.
 type Answer struct {
+	Found     bool
 	Exporter  Exporter
 	Interface Interface
 }
 
-// Update is an update received from a provider.
-type Update struct {
-	Query
-	Answer
-}
-
 // Provider is the interface a provider should implement.
 type Provider interface {
-	// Query asks the provider to query metadata for several requests.
-	Query(ctx context.Context, query *BatchQuery) error
+	// Query asks the provider to query metadata and return the result. The
+	// error can be ErrSkipProvider to skip to the next provider or another
+	// transient error. If the result is not found, Answer.Found is set to
+	// False.
+	Query(ctx context.Context, query Query) (Answer, error)
 }
 
 // Configuration defines an interface to configure a provider.
 type Configuration interface {
 	// New instantiates a new provider from its configuration.
-	New(r *reporter.Reporter, put func(Update)) (Provider, error)
+	New(r *reporter.Reporter) (Provider, error)
 }
diff --git a/inlet/metadata/provider/snmp/config.go b/outlet/metadata/provider/snmp/config.go
similarity index 99%
rename from inlet/metadata/provider/snmp/config.go
rename to outlet/metadata/provider/snmp/config.go
index c2df3185..5ace9795 100644
--- a/inlet/metadata/provider/snmp/config.go
+++ b/outlet/metadata/provider/snmp/config.go
@@ -13,14 +13,14 @@ import (
 	"github.com/gosnmp/gosnmp"
 
 	"akvorado/common/helpers"
-	"akvorado/inlet/metadata/provider"
+	"akvorado/outlet/metadata/provider"
 )
 
 // Configuration describes the configuration for the SNMP client
 type Configuration struct {
 	// PollerRetries tell how many time a poller should retry before giving up
 	PollerRetries int `validate:"min=0"`
-	// PollerTimeout tell how much time a poller should wait for an answer
+	// PollerTimeout tell how much time a poller should wait for an answer before retrying
 	PollerTimeout time.Duration `validate:"min=100ms"`
 
 	// Credentials is a mapping from exporter IPs to credentials
diff --git a/inlet/metadata/provider/snmp/poller.go b/outlet/metadata/provider/snmp/poller.go
similarity index 60%
rename from inlet/metadata/provider/snmp/poller.go
rename to outlet/metadata/provider/snmp/poller.go
index b43b2ca0..793174f7 100644
--- a/inlet/metadata/provider/snmp/poller.go
+++ b/outlet/metadata/provider/snmp/poller.go
@@ -14,37 +14,12 @@ import (
 	"github.com/gosnmp/gosnmp"
 
 	"akvorado/common/reporter"
-	"akvorado/inlet/metadata/provider"
+	"akvorado/outlet/metadata/provider"
 )
 
-// Poll polls the SNMP provider for the requested interface indexes.
-func (p *Provider) Poll(ctx context.Context, exporter, agent netip.Addr, port uint16, ifIndexes []uint, put func(provider.Update)) error {
-	// Check if already have a request running
+// Poll polls the SNMP provider for the requested interface index.
+func (p *Provider) Poll(ctx context.Context, exporter, agent netip.Addr, port uint16, ifIndex uint) (provider.Answer, error) {
 	exporterStr := exporter.Unmap().String()
-	filteredIfIndexes := make([]uint, 0, len(ifIndexes))
-	keys := make([]string, 0, len(ifIndexes))
-	p.pendingRequestsLock.Lock()
-	for _, ifIndex := range ifIndexes {
-		key := fmt.Sprintf("%s@%d", exporterStr, ifIndex)
-		_, ok := p.pendingRequests[key]
-		if !ok {
-			p.pendingRequests[key] = struct{}{}
-			filteredIfIndexes = append(filteredIfIndexes, ifIndex)
-			keys = append(keys, key)
-		}
-	}
-	p.pendingRequestsLock.Unlock()
-	if len(filteredIfIndexes) == 0 {
-		return nil
-	}
-	ifIndexes = filteredIfIndexes
-	defer func() {
-		p.pendingRequestsLock.Lock()
-		for _, key := range keys {
-			delete(p.pendingRequests, key)
-		}
-		p.pendingRequestsLock.Unlock()
-	}()
 
 	// Instantiate an SNMP state
 	g := &gosnmp.GoSNMP{
@@ -99,15 +74,12 @@ func (p *Provider) Poll(ctx context.Context, exporter, agent netip.Addr, port ui
 		p.metrics.errors.WithLabelValues(exporterStr, "connect").Inc()
 		p.errLogger.Err(err).Str("exporter", exporterStr).Msg("unable to connect")
 	}
-	requests := []string{"1.3.6.1.2.1.1.5.0"}
-	for _, ifIndex := range ifIndexes {
-		moreRequests := []string{
-			fmt.Sprintf("1.3.6.1.2.1.2.2.1.2.%d", ifIndex),     // ifDescr
-			fmt.Sprintf("1.3.6.1.2.1.31.1.1.1.1.%d", ifIndex),  // ifName
-			fmt.Sprintf("1.3.6.1.2.1.31.1.1.1.18.%d", ifIndex), // ifAlias
-			fmt.Sprintf("1.3.6.1.2.1.31.1.1.1.15.%d", ifIndex), // ifSpeed
-		}
-		requests = append(requests, moreRequests...)
+	requests := []string{
+		"1.3.6.1.2.1.1.5.0",
+		fmt.Sprintf("1.3.6.1.2.1.2.2.1.2.%d", ifIndex),     // ifDescr
+		fmt.Sprintf("1.3.6.1.2.1.31.1.1.1.1.%d", ifIndex),  // ifName
+		fmt.Sprintf("1.3.6.1.2.1.31.1.1.1.18.%d", ifIndex), // ifAlias
+		fmt.Sprintf("1.3.6.1.2.1.31.1.1.1.15.%d", ifIndex), // ifSpeed
 	}
 	var results []gosnmp.SnmpPDU
 	success := false
@@ -128,17 +100,17 @@ func (p *Provider) Poll(ctx context.Context, exporter, agent netip.Addr, port ui
 		g.Community = community
 		currentResult, err := g.Get(requests)
 		if errors.Is(err, context.Canceled) {
-			return nil
+			return provider.Answer{}, err
 		}
 		if err != nil && canError {
-			return logError(err)
+			return provider.Answer{}, logError(err)
 		}
 		if err != nil {
 			continue
 		}
 		if currentResult.Error != gosnmp.NoError && currentResult.ErrorIndex == 0 && canError {
 			// There is some error affecting the whole request
-			return logError(fmt.Errorf("SNMP error %s(%d)", currentResult.Error, currentResult.Error))
+			return provider.Answer{}, logError(fmt.Errorf("SNMP error %s(%d)", currentResult.Error, currentResult.Error))
 		}
 		success = true
 		if results == nil {
@@ -158,7 +130,7 @@ func (p *Provider) Poll(ctx context.Context, exporter, agent netip.Addr, port ui
 	if len(results) != len(requests) {
 		logError(fmt.Errorf("SNMP mismatch on variable lengths"))
 	}
-	p.metrics.times.WithLabelValues(exporterStr).Observe(time.Now().Sub(start).Seconds())
+	p.metrics.times.WithLabelValues(exporterStr).Observe(time.Since(start).Seconds())
 
 	processStr := func(idx int, what string) (string, bool) {
 		switch results[idx].Type {
@@ -186,73 +158,57 @@ func (p *Provider) Poll(ctx context.Context, exporter, agent netip.Addr, port ui
 	}
 	sysNameVal, ok := processStr(0, "sysname")
 	if !ok {
-		return errors.New("unable to get sysName")
+		return provider.Answer{}, errors.New("unable to get sysName")
 	}
-	for idx := 1; idx < len(requests)-3; idx += 4 {
-		var (
-			name, description string
-			speed             uint
-		)
-		ifIndex := ifIndexes[(idx-1)/4]
-		ok := true
-		// We do not process results when index is 0 (this can happen for local
-		// traffic, we only care for exporter name).
-		if ifIndex > 0 {
-			ifDescrVal, okDescr := processStr(idx, "ifdescr")
-			ifNameVal, okName := processStr(idx+1, "ifname")
-			ifAliasVal, okAlias := processStr(idx+2, "ifalias")
-			ifSpeedVal, okSpeed := processUint(idx+3, "ifspeed")
-
-			// Many equipments are using ifDescr for the interface name and
-			// ifAlias for the description, which is counter-intuitive. We want
-			// both the name and the description.
 
-			if okName {
-				// If we have ifName, use ifDescr if it is different and ifAlias
-				// is not. Otherwise, keep description empty.
-				name = ifNameVal
-				if okAlias && ifAliasVal != ifNameVal {
-					description = ifAliasVal
-				} else if okDescr && ifDescrVal != ifNameVal {
-					description = ifDescrVal
-				}
-			} else {
-				// Don't handle the other case yet. It would be unexpected to
-				// have ifAlias and not ifName. And if we have only ifDescr, we
-				// can't really know what this is.
-				ok = false
-			}
-
-			// Speed is mandatory
-			ok = ok && okSpeed
-			speed = ifSpeedVal
-		}
-		if ok {
-			p.metrics.successes.WithLabelValues(exporterStr).Inc()
-		} else {
-			name = ""
-			description = ""
-			speed = 0
+	var (
+		name, description string
+		speed             uint
+	)
+	ok = true
+	ifDescrVal, okDescr := processStr(1, "ifdescr")
+	ifNameVal, okName := processStr(2, "ifname")
+	ifAliasVal, okAlias := processStr(3, "ifalias")
+	ifSpeedVal, okSpeed := processUint(4, "ifspeed")
+
+	// Many equipments are using ifDescr for the interface name and
+	// ifAlias for the description, which is counter-intuitive. We want
+	// both the name and the description.
+
+	if okName {
+		// If we have ifName, use ifDescr if it is different and ifAlias
+		// is not. Otherwise, keep description empty.
+		name = ifNameVal
+		if okAlias && ifAliasVal != ifNameVal {
+			description = ifAliasVal
+		} else if okDescr && ifDescrVal != ifNameVal {
+			description = ifDescrVal
 		}
-		put(provider.Update{
-			Query: provider.Query{
-				ExporterIP: exporter,
-				IfIndex:    ifIndex,
+	} else {
+		// Don't handle the other case yet. It would be unexpected to
+		// have ifAlias and not ifName. And if we have only ifDescr, we
+		// can't really know what this is.
+		ok = false
+	}
+
+	// Speed is mandatory
+	ok = ok && okSpeed
+	speed = ifSpeedVal
+	if ok {
+		p.metrics.successes.WithLabelValues(exporterStr).Inc()
+		return provider.Answer{
+			Found: true,
+			Exporter: provider.Exporter{
+				Name: sysNameVal,
 			},
-			Answer: provider.Answer{
-				Exporter: provider.Exporter{
-					Name: sysNameVal,
-				},
-				Interface: provider.Interface{
-					Name:        name,
-					Description: description,
-					Speed:       speed,
-				},
+			Interface: provider.Interface{
+				Name:        name,
+				Description: description,
+				Speed:       speed,
 			},
-		})
+		}, nil
 	}
-
-	return nil
+	return provider.Answer{}, nil
 }
 
 type goSNMPLogger struct {
diff --git a/inlet/metadata/provider/snmp/root.go b/outlet/metadata/provider/snmp/root.go
similarity index 60%
rename from inlet/metadata/provider/snmp/root.go
rename to outlet/metadata/provider/snmp/root.go
index 49d5b491..052c58b4 100644
--- a/inlet/metadata/provider/snmp/root.go
+++ b/outlet/metadata/provider/snmp/root.go
@@ -8,35 +8,28 @@ package snmp
 import (
 	"context"
 	"net/netip"
-	"sync"
 	"time"
 
 	"akvorado/common/reporter"
-	"akvorado/inlet/metadata/provider"
+	"akvorado/outlet/metadata/provider"
 )
 
 // Provider represents the SNMP provider.
 type Provider struct {
-	r      *reporter.Reporter
-	config *Configuration
-
-	pendingRequests     map[string]struct{}
-	pendingRequestsLock sync.Mutex
-	errLogger           reporter.Logger
-
-	put func(provider.Update)
+	r         *reporter.Reporter
+	config    *Configuration
+	errLogger reporter.Logger
 
 	metrics struct {
-		pendingRequests reporter.GaugeFunc
-		successes       *reporter.CounterVec
-		errors          *reporter.CounterVec
-		retries         *reporter.CounterVec
-		times           *reporter.SummaryVec
+		successes *reporter.CounterVec
+		errors    *reporter.CounterVec
+		retries   *reporter.CounterVec
+		times     *reporter.SummaryVec
 	}
 }
 
 // New creates a new SNMP provider from configuration
-func (configuration Configuration) New(r *reporter.Reporter, put func(provider.Update)) (provider.Provider, error) {
+func (configuration Configuration) New(r *reporter.Reporter) (provider.Provider, error) {
 	for exporterIP, agentIP := range configuration.Agents {
 		if exporterIP.Is4() || agentIP.Is4() {
 			delete(configuration.Agents, exporterIP)
@@ -47,24 +40,11 @@ func (configuration Configuration) New(r *reporter.Reporter, put func(provider.U
 	}
 
 	p := Provider{
-		r:      r,
-		config: &configuration,
-
-		pendingRequests: make(map[string]struct{}),
-		errLogger:       r.Sample(reporter.BurstSampler(10*time.Second, 3)),
-
-		put: put,
+		r:         r,
+		config:    &configuration,
+		errLogger: r.Sample(reporter.BurstSampler(10*time.Second, 3)),
 	}
 
-	p.metrics.pendingRequests = r.GaugeFunc(
-		reporter.GaugeOpts{
-			Name: "poller_pending_requests",
-			Help: "Number of pending requests in pollers.",
-		}, func() float64 {
-			p.pendingRequestsLock.Lock()
-			defer p.pendingRequestsLock.Unlock()
-			return float64(len(p.pendingRequests))
-		})
 	p.metrics.successes = r.CounterVec(
 		reporter.CounterOpts{
 			Name: "poller_success_requests_total",
@@ -91,12 +71,11 @@ func (configuration Configuration) New(r *reporter.Reporter, put func(provider.U
 }
 
 // Query queries exporter to get information through SNMP.
-func (p *Provider) Query(ctx context.Context, query *provider.BatchQuery) error {
-	// Avoid querying too much exporters with errors
+func (p *Provider) Query(ctx context.Context, query provider.Query) (provider.Answer, error) {
 	agentIP, ok := p.config.Agents[query.ExporterIP]
 	if !ok {
 		agentIP = query.ExporterIP
 	}
 	agentPort := p.config.Ports.LookupOrDefault(query.ExporterIP, 161)
-	return p.Poll(ctx, query.ExporterIP, agentIP, agentPort, query.IfIndexes, p.put)
+	return p.Poll(ctx, query.ExporterIP, agentIP, agentPort, query.IfIndex)
 }
diff --git a/inlet/metadata/provider/static/config.go b/outlet/metadata/provider/static/config.go
similarity index 98%
rename from inlet/metadata/provider/static/config.go
rename to outlet/metadata/provider/static/config.go
index 5c45e487..2e4ceb95 100644
--- a/inlet/metadata/provider/static/config.go
+++ b/outlet/metadata/provider/static/config.go
@@ -8,7 +8,7 @@ import (
 
 	"akvorado/common/helpers"
 	"akvorado/common/remotedatasourcefetcher"
-	"akvorado/inlet/metadata/provider"
+	"akvorado/outlet/metadata/provider"
 )
 
 // Configuration describes the configuration for the static provider
diff --git a/inlet/metadata/provider/static/root.go b/outlet/metadata/provider/static/root.go
similarity index 52%
rename from inlet/metadata/provider/static/root.go
rename to outlet/metadata/provider/static/root.go
index 74d2a61a..acdc663a 100644
--- a/inlet/metadata/provider/static/root.go
+++ b/outlet/metadata/provider/static/root.go
@@ -6,10 +6,12 @@
 package static
 
 import (
+	"time"
+
 	"akvorado/common/helpers"
 	"akvorado/common/remotedatasourcefetcher"
 	"akvorado/common/reporter"
-	"akvorado/inlet/metadata/provider"
+	"akvorado/outlet/metadata/provider"
 
 	"context"
 	"fmt"
@@ -19,65 +21,74 @@ import (
 
 // Provider represents the static provider.
 type Provider struct {
-	r                      *reporter.Reporter
+	r *reporter.Reporter
+
 	exporterSourcesFetcher *remotedatasourcefetcher.Component[exporterInfo]
 	exportersMap           map[string][]exporterInfo
 	exporters              atomic.Pointer[helpers.SubnetMap[ExporterConfiguration]]
 	exportersLock          sync.Mutex
-	put                    func(provider.Update)
+
+	errLogger reporter.Logger
+
+	metrics struct {
+		notReady reporter.Counter
+	}
 }
 
 // New creates a new static provider from configuration
-func (configuration Configuration) New(r *reporter.Reporter, put func(provider.Update)) (provider.Provider, error) {
+func (configuration Configuration) New(r *reporter.Reporter) (provider.Provider, error) {
 	p := &Provider{
 		r:            r,
 		exportersMap: map[string][]exporterInfo{},
-		put:          put,
+		errLogger:    r.Sample(reporter.BurstSampler(time.Minute, 3)),
 	}
 	p.exporters.Store(configuration.Exporters)
 	p.initStaticExporters()
+
 	var err error
-	p.exporterSourcesFetcher, err = remotedatasourcefetcher.New[exporterInfo](r, p.UpdateRemoteDataSource, "metadata", configuration.ExporterSources)
+	p.exporterSourcesFetcher, err = remotedatasourcefetcher.New[exporterInfo](r,
+		p.UpdateRemoteDataSource, "metadata", configuration.ExporterSources)
 	if err != nil {
 		return nil, fmt.Errorf("unable to initialize remote data source fetcher component: %w", err)
 	}
 	if err := p.exporterSourcesFetcher.Start(); err != nil {
 		return nil, fmt.Errorf("unable to start network sources fetcher component: %w", err)
 	}
+
+	p.metrics.notReady = r.Counter(
+		reporter.CounterOpts{
+			Name: "not_ready_total",
+			Help: "Number of queries failing because the remote data sources are not ready",
+		})
+
 	return p, nil
 }
 
 // Query queries static configuration.
-func (p *Provider) Query(_ context.Context, query *provider.BatchQuery) error {
+func (p *Provider) Query(ctx context.Context, query provider.Query) (provider.Answer, error) {
+	// We wait for all data sources to be ready
+	select {
+	case <-ctx.Done():
+		p.metrics.notReady.Inc()
+		p.errLogger.Warn().Msg("remote datasources are not ready")
+		return provider.Answer{}, ctx.Err()
+	case <-p.exporterSourcesFetcher.DataSourcesReady:
+	}
 	exporter, ok := p.exporters.Load().Lookup(query.ExporterIP)
 	if !ok {
-		return provider.ErrSkipProvider
+		return provider.Answer{}, provider.ErrSkipProvider
 	}
-	var skippedIfIndexes uint
-	for _, ifIndex := range query.IfIndexes {
-		iface, ok := exporter.IfIndexes[ifIndex]
-		if !ok {
-			if exporter.SkipMissingInterfaces {
-				query.IfIndexes[skippedIfIndexes] = ifIndex
-				skippedIfIndexes++
-				continue
-			}
-			iface = exporter.Default
+
+	iface, ok := exporter.IfIndexes[query.IfIndex]
+	if !ok {
+		if exporter.SkipMissingInterfaces {
+			return provider.Answer{}, provider.ErrSkipProvider
 		}
-		p.put(provider.Update{
-			Query: provider.Query{
-				ExporterIP: query.ExporterIP,
-				IfIndex:    ifIndex,
-			},
-			Answer: provider.Answer{
-				Exporter:  exporter.Exporter,
-				Interface: iface,
-			},
-		})
-	}
-	if skippedIfIndexes > 0 {
-		query.IfIndexes = query.IfIndexes[:skippedIfIndexes]
-		return provider.ErrSkipProvider
+		iface = exporter.Default
 	}
-	return nil
+	return provider.Answer{
+		Found:     true,
+		Exporter:  exporter.Exporter,
+		Interface: iface,
+	}, nil
 }
diff --git a/inlet/metadata/provider/static/source.go b/outlet/metadata/provider/static/source.go
similarity index 97%
rename from inlet/metadata/provider/static/source.go
rename to outlet/metadata/provider/static/source.go
index 7aaa0ba1..6719b074 100644
--- a/inlet/metadata/provider/static/source.go
+++ b/outlet/metadata/provider/static/source.go
@@ -5,10 +5,11 @@ package static
 
 import (
 	"context"
+	"errors"
 
 	"akvorado/common/helpers"
 	"akvorado/common/remotedatasourcefetcher"
-	"akvorado/inlet/metadata/provider"
+	"akvorado/outlet/metadata/provider"
 )
 
 type exporterInfo struct {
@@ -102,7 +103,7 @@ func (p *Provider) UpdateRemoteDataSource(ctx context.Context, name string, sour
 	p.exportersLock.Unlock()
 	exporters, err := helpers.NewSubnetMap[ExporterConfiguration](finalMap)
 	if err != nil {
-		return 0, err
+		return 0, errors.New("cannot create subnetmap")
 	}
 	p.exporters.Swap(exporters)
 	return len(results), nil
diff --git a/outlet/metadata/root.go b/outlet/metadata/root.go
new file mode 100644
index 00000000..69ef521d
--- /dev/null
+++ b/outlet/metadata/root.go
@@ -0,0 +1,294 @@
+// SPDX-FileCopyrightText: 2022 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+// Package metadata handles metadata polling to get interface names and
+// descriptions. It keeps a cache of retrieved entries and refresh them. It is
+// modular and accepts several kind of providers (including SNMP).
+package metadata
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"net/netip"
+	"sync"
+	"time"
+
+	"github.com/benbjohnson/clock"
+	"github.com/eapache/go-resiliency/breaker"
+	"golang.org/x/sync/singleflight"
+	"gopkg.in/tomb.v2"
+
+	"akvorado/common/daemon"
+	"akvorado/common/reporter"
+	"akvorado/outlet/metadata/provider"
+)
+
+// Component represents the metadata compomenent.
+type Component struct {
+	r      *reporter.Reporter
+	d      *Dependencies
+	t      tomb.Tomb
+	config Configuration
+
+	sc *metadataCache
+	sf singleflight.Group
+
+	providerBreakersLock   sync.Mutex
+	providerBreakerLoggers map[netip.Addr]reporter.Logger
+	providerBreakers       map[netip.Addr]*breaker.Breaker
+	providers              []provider.Provider
+	initialDeadline        time.Time
+
+	metrics struct {
+		cacheRefreshRuns         reporter.Counter
+		cacheRefresh             reporter.Counter
+		providerBreakerOpenCount *reporter.CounterVec
+		providerRequests         reporter.Counter
+		providerErrors           reporter.Counter
+	}
+}
+
+// Dependencies define the dependencies of the metadata component.
+type Dependencies struct {
+	Daemon daemon.Component
+	Clock  clock.Clock
+}
+
+var (
+	// ErrQueryTimeout is the error returned when a query timeout.
+	ErrQueryTimeout = errors.New("provider query timeout")
+)
+
+// New creates a new metadata component.
+func New(r *reporter.Reporter, configuration Configuration, dependencies Dependencies) (*Component, error) {
+	if configuration.CacheRefresh > 0 && configuration.CacheRefresh < configuration.CacheDuration {
+		return nil, errors.New("cache refresh must be greater than cache duration")
+	}
+	if configuration.CacheDuration < configuration.CacheCheckInterval {
+		return nil, errors.New("cache duration must be greater than cache check interval")
+	}
+
+	if dependencies.Clock == nil {
+		dependencies.Clock = clock.New()
+	}
+	sc := newMetadataCache(r)
+	c := Component{
+		r:      r,
+		d:      &dependencies,
+		config: configuration,
+		sc:     sc,
+
+		providerBreakers:       make(map[netip.Addr]*breaker.Breaker),
+		providerBreakerLoggers: make(map[netip.Addr]reporter.Logger),
+		providers:              make([]provider.Provider, 0, 1),
+	}
+	c.d.Daemon.Track(&c.t, "outlet/metadata")
+
+	// Initialize providers
+	for _, p := range c.config.Providers {
+		selectedProvider, err := p.Config.New(r)
+		if err != nil {
+			return nil, err
+		}
+		c.providers = append(c.providers, selectedProvider)
+	}
+
+	c.metrics.cacheRefreshRuns = r.Counter(
+		reporter.CounterOpts{
+			Name: "cache_refresh_runs_total",
+			Help: "Number of times the cache refresh was triggered.",
+		})
+	c.metrics.cacheRefresh = r.Counter(
+		reporter.CounterOpts{
+			Name: "cache_refreshes_total",
+			Help: "Number of entries refreshed in cache.",
+		})
+	c.metrics.providerBreakerOpenCount = r.CounterVec(
+		reporter.CounterOpts{
+			Name: "provider_breaker_opens_total",
+			Help: "Provider breaker was opened due to too many errors.",
+		},
+		[]string{"exporter"})
+	c.metrics.providerRequests = r.Counter(
+		reporter.CounterOpts{
+			Name: "provider_requests_total",
+			Help: "Number of provider requests.",
+		})
+	c.metrics.providerErrors = r.Counter(
+		reporter.CounterOpts{
+			Name: "provider_errors_total",
+			Help: "Number of provider errors.",
+		})
+	return &c, nil
+}
+
+// Start starts the metadata component.
+func (c *Component) Start() error {
+	c.r.Info().Msg("starting metadata component")
+	c.initialDeadline = time.Now().Add(c.config.InitialDelay)
+
+	// Load cache
+	if c.config.CachePersistFile != "" {
+		if err := c.sc.Load(c.config.CachePersistFile); err != nil {
+			c.r.Err(err).Msg("cannot load cache, ignoring")
+		}
+	}
+
+	// Goroutine to refresh the cache
+	healthyTicker := make(chan reporter.ChannelHealthcheckFunc)
+	c.r.RegisterHealthcheck("metadata/ticker", reporter.ChannelHealthcheck(c.t.Context(nil), healthyTicker))
+	c.t.Go(func() error {
+		c.r.Debug().Msg("starting metadata ticker")
+		ticker := c.d.Clock.Ticker(c.config.CacheCheckInterval)
+		defer ticker.Stop()
+		defer close(healthyTicker)
+		for {
+			select {
+			case <-c.t.Dying():
+				c.r.Debug().Msg("shutting down metadata ticker")
+				return nil
+			case cb, ok := <-healthyTicker:
+				if ok {
+					cb(reporter.HealthcheckOK, "ok")
+				}
+			case <-ticker.C:
+				c.expireCache()
+			}
+		}
+	})
+
+	return nil
+}
+
+// Stop stops the metadata component
+func (c *Component) Stop() error {
+	defer func() {
+		if c.config.CachePersistFile != "" {
+			if err := c.sc.Save(c.config.CachePersistFile); err != nil {
+				c.r.Err(err).Msg("cannot save cache")
+			}
+		}
+		c.r.Info().Msg("metadata component stopped")
+	}()
+	c.r.Info().Msg("stopping metadata component")
+	c.t.Kill(nil)
+	return c.t.Wait()
+}
+
+// Lookup for interface information for the provided exporter and ifIndex. If
+// the information is not in the cache, it will be polled from the provider. The
+// returned result has a field Found to tell if the lookup is successful or not.
+func (c *Component) Lookup(t time.Time, exporterIP netip.Addr, ifIndex uint) provider.Answer {
+	query := provider.Query{ExporterIP: exporterIP, IfIndex: ifIndex}
+
+	// Check cache first
+	if answer, ok := c.sc.Lookup(t, query); ok {
+		return answer
+	}
+
+	// Use singleflight to prevent duplicate queries
+	key := fmt.Sprintf("%s-%d", exporterIP, ifIndex)
+	result, err, _ := c.sf.Do(key, func() (any, error) {
+		return c.queryProviders(query)
+	})
+
+	if err != nil {
+		return provider.Answer{}
+	}
+
+	return result.(provider.Answer)
+}
+
+// queryProviders queries all providers. It returns the answer for the specific
+// query and cache it.
+func (c *Component) queryProviders(query provider.Query) (provider.Answer, error) {
+	c.metrics.providerRequests.Inc()
+
+	// Check if provider breaker is open
+	c.providerBreakersLock.Lock()
+	providerBreaker, ok := c.providerBreakers[query.ExporterIP]
+	if !ok {
+		providerBreaker = breaker.New(20, 1, time.Minute)
+		c.providerBreakers[query.ExporterIP] = providerBreaker
+	}
+	c.providerBreakersLock.Unlock()
+
+	var result provider.Answer
+	err := providerBreaker.Run(func() error {
+		deadline := time.Now().Add(c.config.QueryTimeout)
+		if deadline.Before(c.initialDeadline) {
+			deadline = c.initialDeadline
+		}
+		ctx, cancel := context.WithDeadlineCause(
+			c.t.Context(nil),
+			deadline,
+			ErrQueryTimeout)
+		defer cancel()
+
+		now := c.d.Clock.Now()
+		for _, p := range c.providers {
+			answer, err := p.Query(ctx, query)
+			if err == provider.ErrSkipProvider {
+				// Next provider
+				continue
+			}
+			if err != nil {
+				return err
+			}
+			c.sc.Put(now, query, answer)
+			result = answer
+			return nil
+		}
+		return nil
+	})
+
+	if err != nil {
+		c.metrics.providerErrors.Inc()
+		if err == breaker.ErrBreakerOpen {
+			c.metrics.providerBreakerOpenCount.WithLabelValues(query.ExporterIP.Unmap().String()).Inc()
+			c.providerBreakersLock.Lock()
+			l, ok := c.providerBreakerLoggers[query.ExporterIP]
+			if !ok {
+				l = c.r.Sample(reporter.BurstSampler(time.Minute, 1)).
+					With().
+					Str("exporter", query.ExporterIP.Unmap().String()).
+					Logger()
+				c.providerBreakerLoggers[query.ExporterIP] = l
+			}
+			l.Warn().Msg("provider breaker open")
+			c.providerBreakersLock.Unlock()
+		}
+		return provider.Answer{}, err
+	}
+
+	return result, nil
+}
+
+// refreshCacheEntry refreshes a single cache entry.
+func (c *Component) refreshCacheEntry(exporterIP netip.Addr, ifIndex uint) {
+	query := provider.Query{
+		ExporterIP: exporterIP,
+		IfIndex:    ifIndex,
+	}
+	c.queryProviders(query)
+}
+
+// expireCache handles cache expiration and refresh.
+func (c *Component) expireCache() {
+	c.sc.Expire(c.d.Clock.Now().Add(-c.config.CacheDuration))
+	if c.config.CacheRefresh > 0 {
+		c.r.Debug().Msg("refresh metadata cache")
+		c.metrics.cacheRefreshRuns.Inc()
+		count := 0
+		toRefresh := c.sc.NeedUpdates(c.d.Clock.Now().Add(-c.config.CacheRefresh))
+		for exporter, ifaces := range toRefresh {
+			for _, ifIndex := range ifaces {
+				go c.refreshCacheEntry(exporter, ifIndex)
+				count++
+			}
+		}
+		c.r.Debug().Int("count", count).Msg("refreshed metadata cache")
+		c.metrics.cacheRefresh.Add(float64(count))
+	}
+}
diff --git a/inlet/routing/config.go b/outlet/routing/config.go
similarity index 90%
rename from inlet/routing/config.go
rename to outlet/routing/config.go
index 2f0570cd..4bad69c3 100644
--- a/inlet/routing/config.go
+++ b/outlet/routing/config.go
@@ -5,9 +5,9 @@ package routing
 
 import (
 	"akvorado/common/helpers"
-	"akvorado/inlet/routing/provider"
-	"akvorado/inlet/routing/provider/bioris"
-	"akvorado/inlet/routing/provider/bmp"
+	"akvorado/outlet/routing/provider"
+	"akvorado/outlet/routing/provider/bioris"
+	"akvorado/outlet/routing/provider/bmp"
 )
 
 // Configuration describes the configuration for the routing client.
diff --git a/inlet/routing/metrics.go b/outlet/routing/metrics.go
similarity index 100%
rename from inlet/routing/metrics.go
rename to outlet/routing/metrics.go
diff --git a/inlet/routing/provider/bioris/config.go b/outlet/routing/provider/bioris/config.go
similarity index 97%
rename from inlet/routing/provider/bioris/config.go
rename to outlet/routing/provider/bioris/config.go
index bae345e0..4b0dee4f 100644
--- a/inlet/routing/provider/bioris/config.go
+++ b/outlet/routing/provider/bioris/config.go
@@ -7,7 +7,7 @@ package bioris
 import (
 	"time"
 
-	"akvorado/inlet/routing/provider"
+	"akvorado/outlet/routing/provider"
 )
 
 // Configuration describes the configuration for the BioRIS component.
diff --git a/inlet/routing/provider/bioris/metrics.go b/outlet/routing/provider/bioris/metrics.go
similarity index 100%
rename from inlet/routing/provider/bioris/metrics.go
rename to outlet/routing/provider/bioris/metrics.go
diff --git a/inlet/routing/provider/bioris/root.go b/outlet/routing/provider/bioris/root.go
similarity index 98%
rename from inlet/routing/provider/bioris/root.go
rename to outlet/routing/provider/bioris/root.go
index 550371da..38528fbc 100644
--- a/inlet/routing/provider/bioris/root.go
+++ b/outlet/routing/provider/bioris/root.go
@@ -26,8 +26,8 @@ import (
 	"gopkg.in/tomb.v2"
 
 	"akvorado/common/reporter"
-	"akvorado/inlet/routing/provider"
-	"akvorado/inlet/routing/provider/bmp"
+	"akvorado/outlet/routing/provider"
+	"akvorado/outlet/routing/provider/bmp"
 )
 
 var (
@@ -97,7 +97,7 @@ func (p *Provider) Start() error {
 		p.Refresh(ctx)
 	}
 	refresh(context.Background())
-	p.d.Daemon.Track(&p.t, "inlet/bmp")
+	p.d.Daemon.Track(&p.t, "outlet/bmp")
 	p.t.Go(func() error {
 		ticker := time.NewTicker(p.config.Refresh)
 		defer ticker.Stop()
diff --git a/inlet/routing/provider/bmp/config.go b/outlet/routing/provider/bmp/config.go
similarity index 98%
rename from inlet/routing/provider/bmp/config.go
rename to outlet/routing/provider/bmp/config.go
index 3ec8d065..1f9b0c16 100644
--- a/inlet/routing/provider/bmp/config.go
+++ b/outlet/routing/provider/bmp/config.go
@@ -6,7 +6,7 @@ package bmp
 import (
 	"time"
 
-	"akvorado/inlet/routing/provider"
+	"akvorado/outlet/routing/provider"
 )
 
 // Configuration describes the configuration for the BMP server.
diff --git a/inlet/routing/provider/bmp/events.go b/outlet/routing/provider/bmp/events.go
similarity index 100%
rename from inlet/routing/provider/bmp/events.go
rename to outlet/routing/provider/bmp/events.go
diff --git a/inlet/routing/provider/bmp/hash.go b/outlet/routing/provider/bmp/hash.go
similarity index 100%
rename from inlet/routing/provider/bmp/hash.go
rename to outlet/routing/provider/bmp/hash.go
diff --git a/inlet/routing/provider/bmp/lookup.go b/outlet/routing/provider/bmp/lookup.go
similarity index 98%
rename from inlet/routing/provider/bmp/lookup.go
rename to outlet/routing/provider/bmp/lookup.go
index dc3f04b6..0b77ff2c 100644
--- a/inlet/routing/provider/bmp/lookup.go
+++ b/outlet/routing/provider/bmp/lookup.go
@@ -10,7 +10,7 @@ import (
 
 	"github.com/kentik/patricia"
 
-	"akvorado/inlet/routing/provider"
+	"akvorado/outlet/routing/provider"
 )
 
 // LookupResult is the result of the Lookup() function.
diff --git a/inlet/routing/provider/bmp/metrics.go b/outlet/routing/provider/bmp/metrics.go
similarity index 97%
rename from inlet/routing/provider/bmp/metrics.go
rename to outlet/routing/provider/bmp/metrics.go
index 9bbbc310..6dd7e443 100644
--- a/inlet/routing/provider/bmp/metrics.go
+++ b/outlet/routing/provider/bmp/metrics.go
@@ -39,14 +39,14 @@ func (p *Provider) initMetrics() {
 	)
 	p.metrics.peers = p.r.GaugeVec(
 		reporter.GaugeOpts{
-			Name: "peers_total",
+			Name: "peers",
 			Help: "Number of peers up.",
 		},
 		[]string{"exporter"},
 	)
 	p.metrics.routes = p.r.GaugeVec(
 		reporter.GaugeOpts{
-			Name: "routes_total",
+			Name: "routes",
 			Help: "Number of routes up.",
 		},
 		[]string{"exporter"},
@@ -77,7 +77,7 @@ func (p *Provider) initMetrics() {
 			Name: "ignored_updates_total",
 			Help: "Number of ignored BGP updates.",
 		},
-		[]string{"exporter", "reason", "error"},
+		[]string{"exporter", "error"},
 	)
 	p.metrics.panics = p.r.CounterVec(
 		reporter.CounterOpts{
diff --git a/inlet/routing/provider/bmp/rd.go b/outlet/routing/provider/bmp/rd.go
similarity index 100%
rename from inlet/routing/provider/bmp/rd.go
rename to outlet/routing/provider/bmp/rd.go
diff --git a/inlet/routing/provider/bmp/release.go b/outlet/routing/provider/bmp/release.go
similarity index 100%
rename from inlet/routing/provider/bmp/release.go
rename to outlet/routing/provider/bmp/release.go
diff --git a/inlet/routing/provider/bmp/remove.go b/outlet/routing/provider/bmp/remove.go
similarity index 96%
rename from inlet/routing/provider/bmp/remove.go
rename to outlet/routing/provider/bmp/remove.go
index e30264c3..8c3bf060 100644
--- a/inlet/routing/provider/bmp/remove.go
+++ b/outlet/routing/provider/bmp/remove.go
@@ -9,9 +9,10 @@ import (
 )
 
 func (p *Provider) peerRemovalWorker() error {
+	dying := p.t.Dying()
 	for {
 		select {
-		case <-p.t.Dying():
+		case <-dying:
 			return nil
 		case pkey := <-p.peerRemovalChan:
 			exporterStr := pkey.exporter.Addr().Unmap().String()
@@ -55,7 +56,7 @@ func (p *Provider) peerRemovalWorker() error {
 				// Run is incomplete, update metrics and sleep a bit
 				p.metrics.peerRemovalPartial.WithLabelValues(exporterStr).Inc()
 				select {
-				case <-p.t.Dying():
+				case <-dying:
 					return nil
 				case <-time.After(p.config.RIBPeerRemovalSleepInterval):
 				}
diff --git a/inlet/routing/provider/bmp/rib.go b/outlet/routing/provider/bmp/rib.go
similarity index 100%
rename from inlet/routing/provider/bmp/rib.go
rename to outlet/routing/provider/bmp/rib.go
diff --git a/inlet/routing/provider/bmp/root.go b/outlet/routing/provider/bmp/root.go
similarity index 97%
rename from inlet/routing/provider/bmp/root.go
rename to outlet/routing/provider/bmp/root.go
index 6a80cda0..1e2e7d1d 100644
--- a/inlet/routing/provider/bmp/root.go
+++ b/outlet/routing/provider/bmp/root.go
@@ -16,7 +16,7 @@ import (
 	"gopkg.in/tomb.v2"
 
 	"akvorado/common/reporter"
-	"akvorado/inlet/routing/provider"
+	"akvorado/outlet/routing/provider"
 )
 
 // Provider represents the BMP provider.
@@ -65,7 +65,7 @@ func (configuration Configuration) New(r *reporter.Reporter, dependencies Depend
 	}
 	p.staleTimer = p.d.Clock.AfterFunc(time.Hour, p.removeStalePeers)
 
-	p.d.Daemon.Track(&p.t, "inlet/bmp")
+	p.d.Daemon.Track(&p.t, "outlet/bmp")
 	p.initMetrics()
 	return &p, nil
 }
diff --git a/inlet/routing/provider/bmp/serve.go b/outlet/routing/provider/bmp/serve.go
similarity index 96%
rename from inlet/routing/provider/bmp/serve.go
rename to outlet/routing/provider/bmp/serve.go
index fc3a284e..7c1e8690 100644
--- a/inlet/routing/provider/bmp/serve.go
+++ b/outlet/routing/provider/bmp/serve.go
@@ -149,22 +149,22 @@ func (p *Provider) serveConnection(conn *net.TCPConn) error {
 			if ok {
 				switch msgError.ErrorHandling {
 				case bgp.ERROR_HANDLING_SESSION_RESET:
-					p.metrics.ignored.WithLabelValues(exporterStr, "session-reset", err.Error()).Inc()
+					p.metrics.ignored.WithLabelValues(exporterStr, "session-reset").Inc()
 					continue
 				case bgp.ERROR_HANDLING_AFISAFI_DISABLE:
-					p.metrics.ignored.WithLabelValues(exporterStr, "afi-safi", err.Error()).Inc()
+					p.metrics.ignored.WithLabelValues(exporterStr, "afi-safi").Inc()
 					continue
 				case bgp.ERROR_HANDLING_TREAT_AS_WITHDRAW:
 					// This is a pickle. This can be an essential attribute (eg.
 					// AS path) that's malformed or something quite minor for
 					// our own usage (eg. a non-optional attribute), let's skip for now.
-					p.metrics.ignored.WithLabelValues(exporterStr, "treat-as-withdraw", err.Error()).Inc()
+					p.metrics.ignored.WithLabelValues(exporterStr, "treat-as-withdraw").Inc()
 					continue
 				case bgp.ERROR_HANDLING_ATTRIBUTE_DISCARD:
 					// Optional attribute, let's handle it
 				case bgp.ERROR_HANDLING_NONE:
 					// Odd?
-					p.metrics.ignored.WithLabelValues(exporterStr, "none", err.Error()).Inc()
+					p.metrics.ignored.WithLabelValues(exporterStr, "none").Inc()
 					continue
 				}
 			} else {
diff --git a/inlet/routing/provider/bmp/utils.go b/outlet/routing/provider/bmp/utils.go
similarity index 100%
rename from inlet/routing/provider/bmp/utils.go
rename to outlet/routing/provider/bmp/utils.go
diff --git a/inlet/routing/provider/root.go b/outlet/routing/provider/root.go
similarity index 100%
rename from inlet/routing/provider/root.go
rename to outlet/routing/provider/root.go
diff --git a/inlet/routing/root.go b/outlet/routing/root.go
similarity index 98%
rename from inlet/routing/root.go
rename to outlet/routing/root.go
index 68e7d9bf..3d279c9d 100644
--- a/inlet/routing/root.go
+++ b/outlet/routing/root.go
@@ -12,7 +12,7 @@ import (
 	"time"
 
 	"akvorado/common/reporter"
-	"akvorado/inlet/routing/provider"
+	"akvorado/outlet/routing/provider"
 )
 
 // Component represents the metadata compomenent.



Test patch:
diff --git a/cmd/outlet_test.go b/cmd/outlet_test.go
new file mode 100644
index 00000000..0c4db8ec
--- /dev/null
+++ b/cmd/outlet_test.go
@@ -0,0 +1,31 @@
+// SPDX-FileCopyrightText: 2022 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+package cmd
+
+import (
+	"bytes"
+	"testing"
+
+	"akvorado/common/reporter"
+)
+
+func TestOutletStart(t *testing.T) {
+	r := reporter.NewMock(t)
+	config := OutletConfiguration{}
+	config.Reset()
+	if err := outletStart(r, config, true); err != nil {
+		t.Fatalf("outletStart() error:\n%+v", err)
+	}
+}
+
+func TestOutlet(t *testing.T) {
+	root := RootCmd
+	buf := new(bytes.Buffer)
+	root.SetOut(buf)
+	root.SetArgs([]string{"outlet", "--check", "/dev/null"})
+	err := root.Execute()
+	if err != nil {
+		t.Errorf("`outlet` error:\n%+v", err)
+	}
+}
diff --git a/cmd/testdata/configurations/before-outlet/expected.yaml b/cmd/testdata/configurations/before-outlet/expected.yaml
new file mode 100644
index 00000000..4e3fa6e1
--- /dev/null
+++ b/cmd/testdata/configurations/before-outlet/expected.yaml
@@ -0,0 +1,13 @@
+---
+paths:
+  kafka.brokers:
+    - kafka:9092
+  inlet.0.flow.inputs.0.decoder: netflow
+  inlet.0.flow.inputs.1.decoder: sflow
+  outlet.0.metadata.providers.0.type: snmp
+  inlet.0.kafka.brokers:
+    - kafka:9092
+  outlet.0.kafka.brokers:
+    - kafka:9092
+  console.0.clickhouse.servers:
+    - clickhouse:9000
diff --git a/cmd/testdata/configurations/before-outlet/in.yaml b/cmd/testdata/configurations/before-outlet/in.yaml
new file mode 100644
index 00000000..299cc607
--- /dev/null
+++ b/cmd/testdata/configurations/before-outlet/in.yaml
@@ -0,0 +1,100 @@
+---
+# This is the default configuration before the introduction of the outlet.
+
+kafka:
+  topic: flows
+  version: 3.3.1
+  brokers:
+    - kafka:9092
+  topic-configuration:
+    num-partitions: 8
+    replication-factor: 1
+    config-entries:
+      segment.bytes: 1073741824
+      retention.ms: 86400000 # 1 day
+      cleanup.policy: delete
+      compression.type: producer
+
+geoip:
+  optional: true
+  asn-database:
+    - /usr/share/GeoIP/asn.mmdb
+  geo-database:
+    - /usr/share/GeoIP/country.mmdb
+
+clickhousedb:
+  servers:
+    - clickhouse:9000
+
+clickhouse:
+  orchestrator-url: http://akvorado-orchestrator:8080
+  kafka:
+    consumers: 4
+  prometheus-endpoint: /metrics
+  asns:
+    64501: ACME Corporation
+  networks:
+    192.0.2.0/24:
+      name: ipv4-customers
+      role: customers
+    203.0.113.0/24:
+      name: ipv4-servers
+      role: servers
+    2a01:db8:cafe:1::/64:
+      name: ipv6-customers
+      role: customers
+    2a01:db8:cafe:2::/64:
+      name: ipv6-servers
+      role: servers
+  network-sources: []
+
+inlet:
+  kafka:
+    compression-codec: zstd
+  metadata:
+    workers: 10
+    provider:
+      type: snmp
+      credentials:
+        ::/0:
+          communities: public
+  flow:
+    inputs:
+      - type: udp
+        decoder: netflow
+        listen: :2055
+        workers: 6
+        receive-buffer: 10485760
+      - type: udp
+        decoder: sflow
+        listen: :6343
+        workers: 6
+        receive-buffer: 10485760
+  core:
+    workers: 6
+    exporter-classifiers:
+      - ClassifySiteRegex(Exporter.Name, "^([^-]+)-", "$1")
+      - ClassifyRegion("europe")
+      - ClassifyTenant("acme")
+      - ClassifyRole("edge")
+    interface-classifiers:
+      - |
+        ClassifyConnectivityRegex(Interface.Description, "^(?i)(transit|pni|ppni|ix):? ", "$1") &&
+        ClassifyProviderRegex(Interface.Description, "^\\S+?\\s(\\S+)", "$1") &&
+        ClassifyExternal()
+      - ClassifyInternal()
+
+console:
+  http:
+    cache:
+      type: redis
+      server: redis:6379
+  database:
+    saved-filters:
+      - description: "From Netflix"
+        content: >-
+          InIfBoundary = external AND SrcAS = AS2906
+      - description: "From GAFAM"
+        content: >-
+          InIfBoundary = external AND
+          SrcAS IN (AS15169, AS16509, AS32934, AS6185, AS8075)
diff --git a/cmd/testdata/configurations/bmp-to-routing/expected.yaml b/cmd/testdata/configurations/bmp-to-routing/expected.yaml
index b2c6b4f0..1f9104ad 100644
--- a/cmd/testdata/configurations/bmp-to-routing/expected.yaml
+++ b/cmd/testdata/configurations/bmp-to-routing/expected.yaml
@@ -1,6 +1,6 @@
 ---
 paths:
-  inlet.0.routing:
+  outlet.0.routing:
     provider:
       type: bmp
       listen: 127.0.0.1:1179
@@ -13,8 +13,8 @@ paths:
       ribpeerremovalmaxqueue: 10000
       ribpeerremovalmaxtime: 100ms
       ribpeerremovalsleepinterval: 500ms
-  inlet.0.core.asnproviders:
+  outlet.0.core.asnproviders:
     - flow
     - routing
-  inlet.0.core.netproviders:
+  outlet.0.core.netproviders:
     - routing
diff --git a/cmd/testdata/configurations/clickhouse-deprecated/expected.yaml b/cmd/testdata/configurations/clickhouse-deprecated/expected.yaml
new file mode 100644
index 00000000..595a891a
--- /dev/null
+++ b/cmd/testdata/configurations/clickhouse-deprecated/expected.yaml
@@ -0,0 +1,5 @@
+---
+absent-paths:
+  - clickhouse.systemlogttl
+  - clickhouse.prometheusendpoint
+  - clickhouse.nothing
diff --git a/cmd/testdata/configurations/clickhouse-deprecated/in.yaml b/cmd/testdata/configurations/clickhouse-deprecated/in.yaml
new file mode 100644
index 00000000..d3bf683c
--- /dev/null
+++ b/cmd/testdata/configurations/clickhouse-deprecated/in.yaml
@@ -0,0 +1,3 @@
+clickhouse:
+  prometheus-endpoint: /metrics
+  system-log-ttl: 24h
diff --git a/cmd/testdata/configurations/core-ignore-asn-from-flows/expected.yaml b/cmd/testdata/configurations/core-ignore-asn-from-flows/expected.yaml
index 96116455..2d9bbee1 100644
--- a/cmd/testdata/configurations/core-ignore-asn-from-flows/expected.yaml
+++ b/cmd/testdata/configurations/core-ignore-asn-from-flows/expected.yaml
@@ -1,5 +1,5 @@
 ---
 paths:
-  inlet.0.core.asnproviders:
+  outlet.0.core.asnproviders:
     - routing
     - geo-ip
diff --git a/cmd/testdata/configurations/empty/expected.yaml b/cmd/testdata/configurations/empty/expected.yaml
index 1c061be5..f90d97d5 100644
--- a/cmd/testdata/configurations/empty/expected.yaml
+++ b/cmd/testdata/configurations/empty/expected.yaml
@@ -4,5 +4,5 @@ paths:
     - 127.0.0.1:9092
   inlet.0.kafka.brokers:
     - 127.0.0.1:9092
-  clickhouse.kafka.brokers:
+  outlet.0.kafka.brokers:
     - 127.0.0.1:9092
diff --git a/cmd/testdata/configurations/gnmi/expected.yaml b/cmd/testdata/configurations/gnmi/expected.yaml
index dbc9480b..9e622e79 100644
--- a/cmd/testdata/configurations/gnmi/expected.yaml
+++ b/cmd/testdata/configurations/gnmi/expected.yaml
@@ -1,6 +1,6 @@
 ---
 paths:
-  inlet.0.metadata.providers:
+  outlet.0.metadata.providers:
     - type: gnmi
       timeout: "1s"
       minimalrefreshinterval: "1m0s"
diff --git a/cmd/testdata/configurations/kafka-sasl-tls/expected.yaml b/cmd/testdata/configurations/kafka-sasl-tls/expected.yaml
new file mode 100644
index 00000000..321a9dc7
--- /dev/null
+++ b/cmd/testdata/configurations/kafka-sasl-tls/expected.yaml
@@ -0,0 +1,10 @@
+---
+kafka:
+  brokers: something:9092
+  tls:
+    enable: true
+    verify: false
+  sasl:
+    username: hello
+    password: bye
+    mechanism: plain
diff --git a/cmd/testdata/configurations/kafka-sasl-tls/in.yaml b/cmd/testdata/configurations/kafka-sasl-tls/in.yaml
new file mode 100644
index 00000000..08f2cdb2
--- /dev/null
+++ b/cmd/testdata/configurations/kafka-sasl-tls/in.yaml
@@ -0,0 +1,9 @@
+---
+kafka:
+  brokers: something:9092
+  tls:
+    enable: true
+    verify: false
+    sasl-username: hello
+    sasl-password: bye
+    sasl-mechanism: plain
diff --git a/cmd/testdata/configurations/schema-customdict/expected.yaml b/cmd/testdata/configurations/schema-customdict/expected.yaml
index 8156a2b0..50c7fda2 100644
--- a/cmd/testdata/configurations/schema-customdict/expected.yaml
+++ b/cmd/testdata/configurations/schema-customdict/expected.yaml
@@ -1,6 +1,6 @@
 ---
 paths:
-  inlet.0.schema:
+  outlet.0.schema:
     customdictionaries:
       test:
         source: test.csv
@@ -45,4 +45,4 @@ paths:
     enabled: []
     materialize: []
     maintableonly: []
-    notmaintableonly: []
\ No newline at end of file
+    notmaintableonly: []
diff --git a/cmd/testdata/configurations/schema-enable-disable/expected.yaml b/cmd/testdata/configurations/schema-enable-disable/expected.yaml
index a65e9854..2078f23a 100644
--- a/cmd/testdata/configurations/schema-enable-disable/expected.yaml
+++ b/cmd/testdata/configurations/schema-enable-disable/expected.yaml
@@ -1,6 +1,6 @@
 ---
 paths:
-  inlet.0.schema:
+  outlet.0.schema:
     customdictionaries: {}
     disabled:
       - SrcCountry
diff --git a/cmd/testdata/configurations/shipped/expected.yaml b/cmd/testdata/configurations/shipped/expected.yaml
index 551dce85..27621cce 100644
--- a/cmd/testdata/configurations/shipped/expected.yaml
+++ b/cmd/testdata/configurations/shipped/expected.yaml
@@ -45,7 +45,7 @@ paths:
     - kafka:9092
   inlet.0.kafka.brokers:
     - kafka:9092
-  clickhouse.kafka.brokers:
+  outlet.0.kafka.brokers:
     - kafka:9092
   console.0.clickhouse.servers:
     - clickhouse:9000
diff --git a/cmd/testdata/configurations/snmp-communities-migration/expected.yaml b/cmd/testdata/configurations/snmp-communities-migration/expected.yaml
index acedeaef..4be37535 100644
--- a/cmd/testdata/configurations/snmp-communities-migration/expected.yaml
+++ b/cmd/testdata/configurations/snmp-communities-migration/expected.yaml
@@ -1,6 +1,6 @@
 ---
 paths:
-  inlet.0.metadata.providers:
+  outlet.0.metadata.providers:
     - type: snmp
       pollerretries: 1
       pollertimeout: 1s
diff --git a/cmd/testdata/configurations/snmp-credentials-migration/expected.yaml b/cmd/testdata/configurations/snmp-credentials-migration/expected.yaml
index 3c68496e..4dcecf2f 100644
--- a/cmd/testdata/configurations/snmp-credentials-migration/expected.yaml
+++ b/cmd/testdata/configurations/snmp-credentials-migration/expected.yaml
@@ -1,6 +1,6 @@
 ---
 paths:
-  inlet.0.metadata.providers:
+  outlet.0.metadata.providers:
     - type: snmp
       pollerretries: 1
       pollertimeout: 1s
diff --git a/cmd/testdata/configurations/snmp-ports/expected.yaml b/cmd/testdata/configurations/snmp-ports/expected.yaml
index e6c8b042..bfaf2e2b 100644
--- a/cmd/testdata/configurations/snmp-ports/expected.yaml
+++ b/cmd/testdata/configurations/snmp-ports/expected.yaml
@@ -1,4 +1,4 @@
 ---
 paths:
-  inlet.0.metadata.providers.0.ports:
+  outlet.0.metadata.providers.0.ports:
     ::/0: 1611
diff --git a/cmd/testdata/configurations/snmp-to-metadata/expected.yaml b/cmd/testdata/configurations/snmp-to-metadata/expected.yaml
index bf4dd4ca..a8f414b8 100644
--- a/cmd/testdata/configurations/snmp-to-metadata/expected.yaml
+++ b/cmd/testdata/configurations/snmp-to-metadata/expected.yaml
@@ -1,12 +1,12 @@
 ---
 paths:
-  inlet.0.metadata:
-    workers: 10
-    maxbatchrequests: 20
+  outlet.0.metadata:
     cacheduration: 30m0s
     cacherefresh: 30m0s
     cachecheckinterval: 2m0s
     cachepersistfile: ""
+    initialdelay: 1m0s
+    querytimeout: 5s
     providers:
       - type: snmp
         pollerretries: 3
diff --git a/common/helpers/mapstructure_test.go b/common/helpers/mapstructure_test.go
index d1285fcb..e97e5f63 100644
--- a/common/helpers/mapstructure_test.go
+++ b/common/helpers/mapstructure_test.go
@@ -336,3 +336,81 @@ func TestParametrizedConfig(t *testing.T) {
 		}
 	})
 }
+
+func TestDeprecatedFields(t *testing.T) {
+	type Configuration struct {
+		A string
+		B string
+	}
+	RegisterMapstructureDeprecatedFields[Configuration]("C", "D")
+	TestConfigurationDecode(t, ConfigurationDecodeCases{
+		{
+			Initial: func() interface{} { return Configuration{} },
+			Configuration: func() interface{} {
+				return gin.H{
+					"a": "hello",
+					"b": "bye",
+					"c": "nooo",
+					"d": "yes",
+				}
+			},
+			Expected: Configuration{
+				A: "hello",
+				B: "bye",
+			},
+		}, {
+			Initial: func() interface{} { return Configuration{} },
+			Configuration: func() interface{} {
+				return gin.H{
+					"a": "hello",
+					"b": "bye",
+					"e": "nooo",
+				}
+			},
+			Error: true,
+		},
+	})
+}
+
+func TestDeprecatedFieldsInSquashedStructure(t *testing.T) {
+	type SubConfiguration struct {
+		A string
+		B string
+	}
+	type Configuration struct {
+		Sub SubConfiguration `mapstructure:",squash"`
+		E   string
+	}
+	RegisterMapstructureDeprecatedFields[SubConfiguration]("C", "D")
+	TestConfigurationDecode(t, ConfigurationDecodeCases{
+		{
+			Initial: func() any { return Configuration{} },
+			Configuration: func() any {
+				return gin.H{
+					"a": "hello",
+					"b": "bye",
+					"c": "nooo",
+					"d": "yes",
+					"e": "maybe",
+				}
+			},
+			Expected: Configuration{
+				Sub: SubConfiguration{
+					A: "hello",
+					B: "bye",
+				},
+				E: "maybe",
+			},
+		}, {
+			Initial: func() any { return Configuration{} },
+			Configuration: func() any {
+				return gin.H{
+					"a": "hello",
+					"b": "bye",
+					"f": "nooo",
+				}
+			},
+			Error: true,
+		},
+	})
+}
diff --git a/common/helpers/tests.go b/common/helpers/tests.go
index 73efbe89..85a90bf6 100644
--- a/common/helpers/tests.go
+++ b/common/helpers/tests.go
@@ -121,3 +121,8 @@ func (p Pos) String() string {
 	}
 	return ""
 }
+
+// Testing reports whether the current code is being run in a test.
+func Testing() bool {
+	return testing.Testing()
+}
diff --git a/common/kafka/config_test.go b/common/kafka/config_test.go
index 264116ec..8f0e32f0 100644
--- a/common/kafka/config_test.go
+++ b/common/kafka/config_test.go
@@ -7,8 +7,8 @@ import (
 	"testing"
 
 	"akvorado/common/helpers"
+	"akvorado/common/reporter"
 
-	"github.com/IBM/sarama"
 	"github.com/gin-gonic/gin"
 )
 
@@ -80,13 +80,11 @@ func TestKafkaNewConfig(t *testing.T) {
 	}
 	for _, tc := range cases {
 		t.Run(tc.description, func(t *testing.T) {
-			kafkaConfig, err := NewConfig(tc.config)
+			r := reporter.NewMock(t)
+			_, err := NewConfig(r, tc.config)
 			if err != nil {
 				t.Fatalf("NewConfig() error:\n%+v", err)
 			}
-			if err := kafkaConfig.Validate(); err != nil {
-				t.Fatalf("Validate() error:\n%+v", err)
-			}
 		})
 	}
 }
@@ -111,7 +109,6 @@ func TestTLSConfiguration(t *testing.T) {
 			Expected: Configuration{
 				Topic:   "flows",
 				Brokers: []string{"127.0.0.1:9092"},
-				Version: Version(sarama.V2_8_1_0),
 				TLS: helpers.TLSConfiguration{
 					Enable: true,
 					Verify: true,
@@ -134,7 +131,6 @@ func TestTLSConfiguration(t *testing.T) {
 			Expected: Configuration{
 				Topic:   "flows",
 				Brokers: []string{"127.0.0.1:9092"},
-				Version: Version(sarama.V2_8_1_0),
 				TLS: helpers.TLSConfiguration{
 					Enable: true,
 					Verify: false,
@@ -160,7 +156,6 @@ func TestTLSConfiguration(t *testing.T) {
 			Expected: Configuration{
 				Topic:   "flows",
 				Brokers: []string{"127.0.0.1:9092"},
-				Version: Version(sarama.V2_8_1_0),
 				TLS: helpers.TLSConfiguration{
 					Enable: false,
 					Verify: true,
@@ -189,7 +184,6 @@ func TestTLSConfiguration(t *testing.T) {
 			Expected: Configuration{
 				Topic:   "flows",
 				Brokers: []string{"127.0.0.1:9092"},
-				Version: Version(sarama.V2_8_1_0),
 				TLS: helpers.TLSConfiguration{
 					Enable: true,
 					// Value from DefaultConfig is true
@@ -221,7 +215,6 @@ func TestTLSConfiguration(t *testing.T) {
 			Expected: Configuration{
 				Topic:   "flows",
 				Brokers: []string{"127.0.0.1:9092"},
-				Version: Version(sarama.V2_8_1_0),
 				TLS: helpers.TLSConfiguration{
 					Enable: true,
 					// Value from DefaultConfig is true
diff --git a/common/kafka/oauth_test.go b/common/kafka/oauth_test.go
index e3d90ced..4f4c2f14 100644
--- a/common/kafka/oauth_test.go
+++ b/common/kafka/oauth_test.go
@@ -12,7 +12,7 @@ import (
 	"testing"
 	"time"
 
-	"github.com/IBM/sarama"
+	"github.com/twmb/franz-go/pkg/kgo"
 	"golang.org/x/oauth2"
 	"golang.org/x/oauth2/clientcredentials"
 )
@@ -77,8 +77,6 @@ func TestOAuth2ServerClientCredentials(t *testing.T) {
 
 func TestOAuth2Broker(t *testing.T) {
 	r := reporter.NewMock(t)
-	GlobalKafkaLogger.Register(r)
-	defer GlobalKafkaLogger.Unregister()
 
 	// Ensure broker is ready.
 	SetupKafkaBroker(t)
@@ -90,25 +88,27 @@ func TestOAuth2Broker(t *testing.T) {
 		[]string{"kafka:9093", "127.0.0.1:9093"})
 
 	config := DefaultConfiguration()
+	config.Brokers = []string{broker}
 	config.SASL = SASLConfiguration{
 		Username:      "kafka-client",
 		Password:      "kafka-client-secret",
 		Mechanism:     SASLOauth,
 		OAuthTokenURL: fmt.Sprintf("http://%s/default/token", oauthServer),
 	}
-	kafkaConfig, err := NewConfig(config)
+	opts, err := NewConfig(r, config)
 	if err != nil {
 		t.Fatalf("NewConfig() error:\n%+v", err)
 	}
-	if err := kafkaConfig.Validate(); err != nil {
-		t.Fatalf("Validate() error:\n%+v", err)
-	}
 
-	client, err := sarama.NewClient([]string{broker}, kafkaConfig)
+	client, err := kgo.NewClient(opts...)
 	if err != nil {
-		t.Fatalf("sarama.NewClient() error:\n%+v", err)
+		t.Fatalf("kgo.NewClient() error:\n%+v", err)
 	}
-	if err := client.RefreshMetadata(); err != nil {
-		t.Fatalf("client.RefreshMetadata() error:\n%+v", err)
+	defer client.Close()
+
+	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
+	defer cancel()
+	if err := client.Ping(ctx); err != nil {
+		t.Fatalf("client.Ping() error:\n%+v", err)
 	}
 }
diff --git a/common/kafka/tests.go b/common/kafka/tests.go
index 71b88d04..19b14d58 100644
--- a/common/kafka/tests.go
+++ b/common/kafka/tests.go
@@ -1,49 +1,66 @@
 // SPDX-FileCopyrightText: 2022 Free Mobile
-// SPDX-License-Identifier: AGPL-3.0-only
+// SPDX-FileCopyrightText: 2020 Travis Bischel
+// SPDX-License-Identifier: AGPL-3.0-only AND BSD-3-Clause
 
 //go:build !release
 
 package kafka
 
 import (
+	"context"
+	"encoding/binary"
+	"fmt"
 	"testing"
 	"time"
 
-	"github.com/IBM/sarama"
-
 	"akvorado/common/helpers"
+	"akvorado/common/reporter"
+
+	"github.com/twmb/franz-go/pkg/kfake"
+	"github.com/twmb/franz-go/pkg/kgo"
+	"github.com/twmb/franz-go/pkg/kmsg"
 )
 
 // SetupKafkaBroker configures a client to use for testing.
-func SetupKafkaBroker(t *testing.T) (sarama.Client, []string) {
+func SetupKafkaBroker(t *testing.T) (*kgo.Client, []string) {
 	broker := helpers.CheckExternalService(t, "Kafka",
 		[]string{"kafka:9092", "127.0.0.1:9092"})
 
 	// Wait for broker to be ready
-	saramaConfig := sarama.NewConfig()
-	saramaConfig.Version = sarama.V2_8_1_0
-	saramaConfig.Net.DialTimeout = 1 * time.Second
-	saramaConfig.Net.ReadTimeout = 1 * time.Second
-	saramaConfig.Net.WriteTimeout = 1 * time.Second
-	ready := false
-	var (
-		client sarama.Client
-		err    error
+	r := reporter.NewMock(t)
+	opts, err := NewConfig(r, Configuration{
+		Brokers: []string{broker},
+		TLS: helpers.TLSConfiguration{
+			Enable: false,
+			Verify: true,
+		},
+	})
+	if err != nil {
+		t.Fatalf("NewConfig() error: %v", err)
+	}
+
+	// Add additional options for testing
+	opts = append(opts,
+		kgo.RequestTimeoutOverhead(1*time.Second),
+		kgo.ProduceRequestTimeout(1*time.Second),
+		kgo.ConnIdleTimeout(1*time.Second),
 	)
+
+	ready := false
+	var client *kgo.Client
 	for i := 0; i < 90 && !ready; i++ {
 		if client != nil {
 			client.Close()
 		}
-		if client, err = sarama.NewClient([]string{broker}, saramaConfig); err != nil {
-			continue
-		}
-		if err := client.RefreshMetadata(); err != nil {
+		if client, err = kgo.NewClient(opts...); err != nil {
 			continue
 		}
-		brokers := client.Brokers()
-		if len(brokers) == 0 {
+		ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
+		if err := client.Ping(ctx); err != nil {
+			cancel()
 			continue
 		}
+		cancel()
 		ready = true
 	}
 	if !ready {
@@ -52,3 +69,69 @@ func SetupKafkaBroker(t *testing.T) (sarama.Client, []string) {
 
 	return client, []string{broker}
 }
+
+// forEachBatchRecord iterates through all records in a record batch. This
+// function is stolen from franz-go/pkg/kfake/data.go.
+func forEachBatchRecord(batch kmsg.RecordBatch, cb func(kmsg.Record) error) error {
+	records, err := kgo.DefaultDecompressor().Decompress(
+		batch.Records,
+		kgo.CompressionCodecType(batch.Attributes&0x0007),
+	)
+	if err != nil {
+		return err
+	}
+	for range batch.NumRecords {
+		rec := kmsg.NewRecord()
+		err := rec.ReadFrom(records)
+		if err != nil {
+			return fmt.Errorf("corrupt batch: %w", err)
+		}
+		if err := cb(rec); err != nil {
+			return err
+		}
+		length, amt := binary.Varint(records)
+		records = records[length+int64(amt):]
+	}
+	if len(records) > 0 {
+		return fmt.Errorf("corrupt batch, extra left over bytes after parsing batch: %v", len(records))
+	}
+	return nil
+}
+
+// InterceptMessages sets up a ControlKey to intercept all messages produced to a fake cluster
+// and calls the callback function for each record received.
+func InterceptMessages(t *testing.T, cluster *kfake.Cluster, callback func(*kgo.Record)) {
+	t.Helper()
+
+	// Use ControlKey to intercept ProduceRequest messages
+	cluster.ControlKey(0, func(kreq kmsg.Request) (kmsg.Response, error, bool) {
+		cluster.KeepControl()
+		if req, ok := kreq.(*kmsg.ProduceRequest); ok {
+			for _, topicData := range req.Topics {
+				for _, partitionData := range topicData.Partitions {
+					if partitionData.Records != nil {
+						var batch kmsg.RecordBatch
+						if err := batch.ReadFrom(partitionData.Records); err != nil {
+							t.Fatalf("batch.ReadFrom() error:\n%+v", err)
+						}
+						if err := forEachBatchRecord(batch, func(rec kmsg.Record) error {
+							kgoRecord := &kgo.Record{
+								Topic:     topicData.Topic,
+								Partition: partitionData.Partition,
+								Key:       rec.Key,
+								Value:     rec.Value,
+							}
+							callback(kgoRecord)
+							return nil
+						}); err != nil {
+							t.Fatalf("forEachBatchRecord() error:\n%+v", err)
+						}
+					}
+				}
+			}
+		}
+
+		// Don't modify the response, just let it pass through
+		return nil, nil, false
+	})
+}
diff --git a/common/schema/clickhouse_test.go b/common/schema/clickhouse_test.go
new file mode 100644
index 00000000..cee66359
--- /dev/null
+++ b/common/schema/clickhouse_test.go
@@ -0,0 +1,609 @@
+// SPDX-FileCopyrightText: 2022 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+package schema
+
+import (
+	"net/netip"
+	"slices"
+	"testing"
+
+	"akvorado/common/helpers"
+
+	"github.com/ClickHouse/ch-go/proto"
+)
+
+func TestAppendDefault(t *testing.T) {
+	c := NewMock(t).EnableAllColumns()
+	bf := c.NewFlowMessage()
+	bf.Finalize()
+	if bf.batch.rowCount != 1 {
+		t.Errorf("rowCount should be 1, not %d", bf.batch.rowCount)
+	}
+	if bf.batch.columnSet.Any() {
+		t.Error("columnSet should be empty after finalize")
+	}
+	for idx, col := range bf.batch.columns {
+		if col == nil {
+			continue
+		}
+		if col.Rows() != 1 {
+			t.Errorf("column %q should be length 1", ColumnKey(idx))
+		}
+	}
+}
+
+func TestAppendBasics(t *testing.T) {
+	c := NewMock(t)
+	bf := c.NewFlowMessage()
+
+	// Test basic append
+	bf.AppendDateTime(ColumnTimeReceived, 1000)
+	bf.AppendUint(ColumnSamplingRate, 20000)
+	bf.AppendUint(ColumnDstAS, 65000)
+
+	// Test zero value (should not append)
+	bf.AppendUint(ColumnSrcAS, 0)
+
+	// Test duplicate append
+	bf.AppendUint(ColumnPackets, 100)
+	bf.AppendUint(ColumnPackets, 200)
+
+	expected := map[ColumnKey]any{
+		ColumnTimeReceived: 1000,
+		ColumnSamplingRate: 20000,
+		ColumnDstAS:        65000,
+		ColumnPackets:      100,
+	}
+	got := bf.OtherColumns
+	if diff := helpers.Diff(got, expected); diff != "" {
+		t.Errorf("Append() (-got, +want):\n%s", diff)
+	}
+
+	bf.Finalize()
+	for idx, col := range bf.batch.columns {
+		if col == nil {
+			continue
+		}
+		if col.Rows() != 1 {
+			t.Errorf("column %q should be length 1", ColumnKey(idx))
+		}
+	}
+}
+
+func TestAppendWithDisabledColumns(t *testing.T) {
+	c := NewMock(t)
+	bf := c.NewFlowMessage()
+
+	// Try to append to a disabled column (L2 group is disabled by default in mock)
+	bf.AppendUint(ColumnSrcVlan, 100)
+	bf.Finalize()
+}
+
+func TestAppendArrayUInt32Columns(t *testing.T) {
+	c := NewMock(t)
+	bf := c.NewFlowMessage()
+	bf.AppendArrayUInt32(ColumnDstASPath, []uint32{65400, 65500, 65001})
+	bf.Finalize()
+	bf.AppendArrayUInt32(ColumnDstASPath, []uint32{65403, 65503, 65003})
+	bf.Finalize()
+
+	// Verify column has data
+	got := bf.batch.columns[ColumnDstASPath].(*proto.ColArr[uint32])
+	expected := proto.ColArr[uint32]{
+		Offsets: proto.ColUInt64{3, 6},
+		Data:    &proto.ColUInt32{65400, 65500, 65001, 65403, 65503, 65003},
+	}
+	if diff := helpers.Diff(got, expected); diff != "" {
+		t.Errorf("AppendArrayUInt32 (-got, +want):\n%s", diff)
+	}
+}
+
+func TestAppendArrayUInt128Columns(t *testing.T) {
+	c := NewMock(t)
+	bf := c.NewFlowMessage()
+	bf.AppendArrayUInt128(ColumnDstLargeCommunities, []UInt128{
+		{
+			High: (65401 << 32) + 100,
+			Low:  200,
+		},
+		{
+			High: (65401 << 32) + 100,
+			Low:  201,
+		},
+	})
+	bf.Finalize()
+
+	got := bf.batch.columns[ColumnDstLargeCommunities].(*proto.ColArr[proto.UInt128])
+	expected := proto.ColArr[proto.UInt128]{
+		Offsets: proto.ColUInt64{2},
+		Data: &proto.ColUInt128{
+			{High: (65401 << 32) + 100, Low: 200},
+			{High: (65401 << 32) + 100, Low: 201},
+		},
+	}
+	if diff := helpers.Diff(got, expected); diff != "" {
+		t.Errorf("AppendArrayUInt128 (-got, +want):\n%s", diff)
+	}
+}
+
+func TestUndoUInt64(t *testing.T) {
+	c := NewMock(t)
+	bf := c.NewFlowMessage()
+
+	// Add two values
+	bf.AppendUint(ColumnBytes, 100)
+	bf.AppendUint(ColumnPackets, 200)
+
+	// Check we have the expected initial state
+	bytesCol := bf.batch.columns[ColumnBytes].(*proto.ColUInt64)
+	packetsCol := bf.batch.columns[ColumnPackets].(*proto.ColUInt64)
+
+	expectedBytes := proto.ColUInt64{100}
+	expectedPackets := proto.ColUInt64{200}
+
+	if diff := helpers.Diff(bytesCol, expectedBytes); diff != "" {
+		t.Errorf("Initial bytes column state (-got, +want):\n%s", diff)
+	}
+	if diff := helpers.Diff(packetsCol, expectedPackets); diff != "" {
+		t.Errorf("Initial packets column state (-got, +want):\n%s", diff)
+	}
+
+	// Undo should remove the last appended values
+	bf.Undo()
+
+	expectedBytesAfter := proto.ColUInt64{}
+	expectedPacketsAfter := proto.ColUInt64{}
+
+	if diff := helpers.Diff(bytesCol, expectedBytesAfter); diff != "" {
+		t.Errorf("Bytes column after undo (-got, +want):\n%s", diff)
+	}
+	if diff := helpers.Diff(packetsCol, expectedPacketsAfter); diff != "" {
+		t.Errorf("Packets column after undo (-got, +want):\n%s", diff)
+	}
+}
+
+func TestUndoUInt32(t *testing.T) {
+	c := NewMock(t)
+	bf := c.NewFlowMessage()
+
+	// Add two values
+	bf.AppendUint(ColumnSrcAS, 65001)
+	bf.AppendUint(ColumnDstAS, 65002)
+
+	// Check we have the expected initial state
+	srcCol := bf.batch.columns[ColumnSrcAS].(*proto.ColUInt32)
+	dstCol := bf.batch.columns[ColumnDstAS].(*proto.ColUInt32)
+
+	expectedSrc := proto.ColUInt32{65001}
+	expectedDst := proto.ColUInt32{65002}
+
+	if diff := helpers.Diff(srcCol, expectedSrc); diff != "" {
+		t.Errorf("Initial SrcAS column state (-got, +want):\n%s", diff)
+	}
+	if diff := helpers.Diff(dstCol, expectedDst); diff != "" {
+		t.Errorf("Initial DstAS column state (-got, +want):\n%s", diff)
+	}
+
+	// Undo should remove the last appended values
+	bf.Undo()
+
+	expectedSrcAfter := proto.ColUInt32{}
+	expectedDstAfter := proto.ColUInt32{}
+
+	if diff := helpers.Diff(srcCol, expectedSrcAfter); diff != "" {
+		t.Errorf("SrcAS column after undo (-got, +want):\n%s", diff)
+	}
+	if diff := helpers.Diff(dstCol, expectedDstAfter); diff != "" {
+		t.Errorf("DstAS column after undo (-got, +want):\n%s", diff)
+	}
+}
+
+func TestUndoUInt16(t *testing.T) {
+	c := NewMock(t)
+	bf := c.NewFlowMessage()
+
+	// Add two values
+	bf.AppendUint(ColumnSrcPort, 80)
+	bf.AppendUint(ColumnDstPort, 443)
+
+	// Check we have the expected initial state
+	srcCol := bf.batch.columns[ColumnSrcPort].(*proto.ColUInt16)
+	dstCol := bf.batch.columns[ColumnDstPort].(*proto.ColUInt16)
+
+	expectedSrc := proto.ColUInt16{80}
+	expectedDst := proto.ColUInt16{443}
+
+	if diff := helpers.Diff(srcCol, expectedSrc); diff != "" {
+		t.Errorf("Initial SrcPort column state (-got, +want):\n%s", diff)
+	}
+	if diff := helpers.Diff(dstCol, expectedDst); diff != "" {
+		t.Errorf("Initial DstPort column state (-got, +want):\n%s", diff)
+	}
+
+	// Undo should remove the last appended values
+	bf.Undo()
+
+	expectedSrcAfter := proto.ColUInt16{}
+	expectedDstAfter := proto.ColUInt16{}
+
+	if diff := helpers.Diff(srcCol, expectedSrcAfter); diff != "" {
+		t.Errorf("SrcPort column after undo (-got, +want):\n%s", diff)
+	}
+	if diff := helpers.Diff(dstCol, expectedDstAfter); diff != "" {
+		t.Errorf("DstPort column after undo (-got, +want):\n%s", diff)
+	}
+}
+
+func TestUndoUInt8(t *testing.T) {
+	c := NewMock(t)
+	bf := c.NewFlowMessage()
+
+	// Add value
+	bf.AppendUint(ColumnSrcNetMask, 6)
+
+	// Check we have the expected initial state
+	col := bf.batch.columns[ColumnSrcNetMask].(*proto.ColUInt8)
+	expected := proto.ColUInt8{6}
+
+	if diff := helpers.Diff(col, expected); diff != "" {
+		t.Errorf("Initial Proto column state (-got, +want):\n%s", diff)
+	}
+
+	// Undo should remove the last appended value
+	bf.Undo()
+
+	expectedAfter := proto.ColUInt8{}
+
+	if diff := helpers.Diff(col, expectedAfter); diff != "" {
+		t.Errorf("Proto column after undo (-got, +want):\n%s", diff)
+	}
+}
+
+func TestUndoIPv6(t *testing.T) {
+	c := NewMock(t)
+	bf := c.NewFlowMessage()
+
+	// Add IPv6 values
+	srcAddr := netip.MustParseAddr("2001:db8::1")
+	dstAddr := netip.MustParseAddr("2001:db8::2")
+
+	bf.AppendIPv6(ColumnSrcAddr, srcAddr)
+	bf.AppendIPv6(ColumnDstAddr, dstAddr)
+
+	// Check we have the expected initial state
+	srcCol := bf.batch.columns[ColumnSrcAddr].(*proto.ColIPv6)
+	dstCol := bf.batch.columns[ColumnDstAddr].(*proto.ColIPv6)
+
+	expectedSrc := proto.ColIPv6{srcAddr.As16()}
+	expectedDst := proto.ColIPv6{dstAddr.As16()}
+
+	if diff := helpers.Diff(srcCol, expectedSrc); diff != "" {
+		t.Errorf("Initial SrcAddr column state (-got, +want):\n%s", diff)
+	}
+	if diff := helpers.Diff(dstCol, expectedDst); diff != "" {
+		t.Errorf("Initial DstAddr column state (-got, +want):\n%s", diff)
+	}
+
+	// Undo should remove the values
+	bf.Undo()
+
+	expectedSrcAfter := proto.ColIPv6{}
+	expectedDstAfter := proto.ColIPv6{}
+
+	if diff := helpers.Diff(srcCol, expectedSrcAfter); diff != "" {
+		t.Errorf("SrcAddr column after undo (-got, +want):\n%s", diff)
+	}
+	if diff := helpers.Diff(dstCol, expectedDstAfter); diff != "" {
+		t.Errorf("DstAddr column after undo (-got, +want):\n%s", diff)
+	}
+}
+
+func TestUndoDateTime(t *testing.T) {
+	c := NewMock(t)
+	bf := c.NewFlowMessage()
+
+	// Add DateTime value
+	bf.AppendDateTime(ColumnTimeReceived, 1000)
+
+	// Check we have the expected initial state
+	col := bf.batch.columns[ColumnTimeReceived].(*proto.ColDateTime)
+	expected := proto.ColDateTime{Data: []proto.DateTime{1000}}
+
+	if diff := helpers.Diff(col, expected); diff != "" {
+		t.Errorf("Initial TimeReceived column state (-got, +want):\n%s", diff)
+	}
+
+	// Undo should remove the value
+	bf.Undo()
+
+	expectedAfter := proto.ColDateTime{Data: []proto.DateTime{}}
+
+	if diff := helpers.Diff(col, expectedAfter); diff != "" {
+		t.Errorf("TimeReceived column after undo (-got, +want):\n%s", diff)
+	}
+}
+
+func TestUndoEnum8(t *testing.T) {
+	c := NewMock(t)
+	bf := c.NewFlowMessage()
+
+	// Add Enum8 value (using interface boundary enum)
+	bf.AppendUint(ColumnInIfBoundary, uint64(InterfaceBoundaryExternal))
+
+	// Check we have the expected initial state
+	col := bf.batch.columns[ColumnInIfBoundary].(*proto.ColEnum8)
+	expected := proto.ColEnum8{proto.Enum8(InterfaceBoundaryExternal)}
+
+	if diff := helpers.Diff(col, expected); diff != "" {
+		t.Errorf("Initial InIfBoundary column state (-got, +want):\n%s", diff)
+	}
+
+	// Undo should remove the value
+	bf.Undo()
+
+	expectedAfter := proto.ColEnum8{}
+
+	if diff := helpers.Diff(col, expectedAfter); diff != "" {
+		t.Errorf("InIfBoundary column after undo (-got, +want):\n%s", diff)
+	}
+}
+
+func TestUndoLowCardinalityString(t *testing.T) {
+	c := NewMock(t)
+	bf := c.NewFlowMessage()
+
+	// Add LowCardinality string values
+	bf.AppendString(ColumnExporterName, "router1")
+	bf.AppendString(ColumnExporterRole, "edge")
+
+	// Check we have the expected initial state
+	nameCol := bf.batch.columns[ColumnExporterName].(*proto.ColLowCardinality[string])
+	roleCol := bf.batch.columns[ColumnExporterRole].(*proto.ColLowCardinality[string])
+
+	expectedName := proto.ColLowCardinality[string]{Values: []string{"router1"}}
+	expectedRole := proto.ColLowCardinality[string]{Values: []string{"edge"}}
+
+	if diff := helpers.Diff(nameCol, expectedName); diff != "" {
+		t.Errorf("Initial ExporterName column state (-got, +want):\n%s", diff)
+	}
+	if diff := helpers.Diff(roleCol, expectedRole); diff != "" {
+		t.Errorf("Initial ExporterRole column state (-got, +want):\n%s", diff)
+	}
+
+	// Undo should remove the values
+	bf.Undo()
+
+	expectedNameAfter := proto.ColLowCardinality[string]{Values: []string{}}
+	expectedRoleAfter := proto.ColLowCardinality[string]{Values: []string{}}
+
+	if diff := helpers.Diff(nameCol, expectedNameAfter); diff != "" {
+		t.Errorf("ExporterName column after undo (-got, +want):\n%s", diff)
+	}
+	if diff := helpers.Diff(roleCol, expectedRoleAfter); diff != "" {
+		t.Errorf("ExporterRole column after undo (-got, +want):\n%s", diff)
+	}
+}
+
+func TestUndoLowCardinalityIPv6(t *testing.T) {
+	c := NewMock(t)
+	bf := c.NewFlowMessage()
+
+	// Add LowCardinality IPv6 value
+	addr := netip.MustParseAddr("2001:db8::1")
+	bf.AppendIPv6(ColumnExporterAddress, addr)
+
+	// Check we have the expected initial state
+	col := bf.batch.columns[ColumnExporterAddress].(*proto.ColLowCardinality[proto.IPv6])
+	expected := proto.ColLowCardinality[proto.IPv6]{Values: []proto.IPv6{addr.As16()}}
+
+	if diff := helpers.Diff(col, expected); diff != "" {
+		t.Errorf("Initial ExporterAddress column state (-got, +want):\n%s", diff)
+	}
+
+	// Undo should remove the value
+	bf.Undo()
+
+	expectedAfter := proto.ColLowCardinality[proto.IPv6]{Values: []proto.IPv6{}}
+
+	if diff := helpers.Diff(col, expectedAfter); diff != "" {
+		t.Errorf("ExporterAddress column after undo (-got, +want):\n%s", diff)
+	}
+}
+
+func TestUndoArrayUInt32(t *testing.T) {
+	c := NewMock(t)
+
+	t.Run("one value", func(t *testing.T) {
+		bf := c.NewFlowMessage()
+		bf.AppendArrayUInt32(ColumnDstASPath, []uint32{65001, 65002, 65003})
+
+		// Check we have the expected initial state
+		col := bf.batch.columns[ColumnDstASPath].(*proto.ColArr[uint32])
+		expected := proto.ColArr[uint32]{
+			Offsets: proto.ColUInt64{3},
+			Data:    &proto.ColUInt32{65001, 65002, 65003},
+		}
+
+		if diff := helpers.Diff(*col, expected); diff != "" {
+			t.Errorf("Initial DstASPath column state (-got, +want):\n%s", diff)
+		}
+
+		// Undo should remove the array
+		bf.Undo()
+
+		expectedAfter := proto.ColArr[uint32]{
+			Offsets: proto.ColUInt64{},
+			Data:    &proto.ColUInt32{},
+		}
+
+		if diff := helpers.Diff(*col, expectedAfter); diff != "" {
+			t.Errorf("DstASPath column after undo (-got, +want):\n%s", diff)
+		}
+	})
+
+	t.Run("two values", func(t *testing.T) {
+		bf := c.NewFlowMessage()
+		bf.AppendArrayUInt32(ColumnDstASPath, []uint32{65001, 65002, 65003})
+		bf.Finalize()
+		bf.AppendArrayUInt32(ColumnDstASPath, []uint32{65007, 65008})
+
+		// Check we have the expected initial state
+		col := bf.batch.columns[ColumnDstASPath].(*proto.ColArr[uint32])
+		expected := proto.ColArr[uint32]{
+			Offsets: proto.ColUInt64{3, 5},
+			Data:    &proto.ColUInt32{65001, 65002, 65003, 65007, 65008},
+		}
+
+		if diff := helpers.Diff(*col, expected); diff != "" {
+			t.Errorf("Initial DstASPath column state (-got, +want):\n%s", diff)
+		}
+
+		// Undo should remove the last array
+		bf.Undo()
+
+		expectedAfter := proto.ColArr[uint32]{
+			Offsets: proto.ColUInt64{3},
+			Data:    &proto.ColUInt32{65001, 65002, 65003},
+		}
+
+		if diff := helpers.Diff(*col, expectedAfter); diff != "" {
+			t.Errorf("DstASPath column after undo (-got, +want):\n%s", diff)
+		}
+	})
+
+}
+
+func TestUndoArrayUInt128(t *testing.T) {
+	c := NewMock(t)
+
+	t.Run("one value", func(t *testing.T) {
+		bf := c.NewFlowMessage()
+
+		// Add Array(UInt128) value
+		bf.AppendArrayUInt128(ColumnDstLargeCommunities, []UInt128{
+			{High: (65401 << 32) + 100, Low: 200},
+			{High: (65401 << 32) + 100, Low: 201},
+		})
+
+		// Check we have the expected initial state
+		col := bf.batch.columns[ColumnDstLargeCommunities].(*proto.ColArr[proto.UInt128])
+		expected := proto.ColArr[proto.UInt128]{
+			Offsets: proto.ColUInt64{2},
+			Data: &proto.ColUInt128{
+				{High: (65401 << 32) + 100, Low: 200},
+				{High: (65401 << 32) + 100, Low: 201},
+			},
+		}
+
+		if diff := helpers.Diff(*col, expected); diff != "" {
+			t.Errorf("Initial DstLargeCommunities column state (-got, +want):\n%s", diff)
+		}
+
+		// Undo should remove the array
+		bf.Undo()
+
+		expectedAfter := proto.ColArr[proto.UInt128]{
+			Offsets: proto.ColUInt64{},
+			Data:    &proto.ColUInt128{},
+		}
+
+		if diff := helpers.Diff(*col, expectedAfter); diff != "" {
+			t.Errorf("DstLargeCommunities column after undo (-got, +want):\n%s", diff)
+		}
+	})
+
+	t.Run("two values", func(t *testing.T) {
+		bf := c.NewFlowMessage()
+
+		// Add first Array(UInt128) value
+		bf.AppendArrayUInt128(ColumnDstLargeCommunities, []UInt128{
+			{High: (65401 << 32) + 100, Low: 200},
+			{High: (65401 << 32) + 100, Low: 201},
+		})
+		bf.Finalize()
+
+		// Add second Array(UInt128) value
+		bf.AppendArrayUInt128(ColumnDstLargeCommunities, []UInt128{
+			{High: (65402 << 32) + 100, Low: 300},
+		})
+
+		// Check we have the expected initial state
+		col := bf.batch.columns[ColumnDstLargeCommunities].(*proto.ColArr[proto.UInt128])
+		expected := proto.ColArr[proto.UInt128]{
+			Offsets: proto.ColUInt64{2, 3},
+			Data: &proto.ColUInt128{
+				{High: (65401 << 32) + 100, Low: 200},
+				{High: (65401 << 32) + 100, Low: 201},
+				{High: (65402 << 32) + 100, Low: 300},
+			},
+		}
+
+		if diff := helpers.Diff(*col, expected); diff != "" {
+			t.Errorf("Initial DstLargeCommunities column state (-got, +want):\n%s", diff)
+		}
+
+		// Undo should remove the last array
+		bf.Undo()
+
+		expectedAfter := proto.ColArr[proto.UInt128]{
+			Offsets: proto.ColUInt64{2},
+			Data: &proto.ColUInt128{
+				{High: (65401 << 32) + 100, Low: 200},
+				{High: (65401 << 32) + 100, Low: 201},
+			},
+		}
+
+		if diff := helpers.Diff(*col, expectedAfter); diff != "" {
+			t.Errorf("DstLargeCommunities column after undo (-got, +want):\n%s", diff)
+		}
+	})
+}
+
+func TestBuildProtoInput(t *testing.T) {
+	// Use a smaller version
+	exporterAddress := netip.MustParseAddr("::ffff:203.0.113.14")
+	c := NewMock(t)
+	bf := c.NewFlowMessage()
+	got := bf.ClickHouseProtoInput()
+
+	bf.TimeReceived = 1000
+	bf.SamplingRate = 20000
+	bf.ExporterAddress = exporterAddress
+	bf.AppendUint(ColumnDstAS, 65000)
+	bf.AppendUint(ColumnBytes, 200)
+	bf.AppendUint(ColumnPackets, 300)
+	bf.Finalize()
+
+	bf.Clear()
+
+	bf.TimeReceived = 1002
+	bf.ExporterAddress = exporterAddress
+	bf.AppendUint(ColumnSrcAS, 65000)
+	bf.AppendUint(ColumnBytes, 2000)
+	bf.AppendUint(ColumnPackets, 30)
+	bf.AppendUint(ColumnBytes, 300) // Duplicate!
+	bf.Finalize()
+
+	bf.TimeReceived = 1003
+	bf.ExporterAddress = exporterAddress
+	bf.AppendUint(ColumnSrcAS, 65001)
+	bf.AppendUint(ColumnBytes, 202)
+	bf.AppendUint(ColumnPackets, 3)
+	bf.Finalize()
+
+	// Let's compare a subset
+	expected := proto.Input{
+		{Name: "TimeReceived", Data: proto.ColDateTime{Data: []proto.DateTime{1002, 1003}}},
+		{Name: "SrcAS", Data: proto.ColUInt32{65000, 65001}},
+		{Name: "DstAS", Data: proto.ColUInt32{0, 0}},
+		{Name: "Bytes", Data: proto.ColUInt64{2000, 202}},
+		{Name: "Packets", Data: proto.ColUInt64{30, 3}},
+	}
+	got = slices.DeleteFunc(got, func(col proto.InputColumn) bool {
+		return !slices.Contains([]string{"TimeReceived", "SrcAS", "DstAS", "Packets", "Bytes"}, col.Name)
+	})
+	if diff := helpers.Diff(got, expected); diff != "" {
+		t.Fatalf("ClickHouseProtoInput() (-got, +want):\n%s", diff)
+	}
+}
diff --git a/common/schema/definition_test.go b/common/schema/definition_test.go
index ee45c42d..02eb8b45 100644
--- a/common/schema/definition_test.go
+++ b/common/schema/definition_test.go
@@ -22,17 +22,6 @@ func TestFlowsClickHouse(t *testing.T) {
 	}
 }
 
-func TestFlowsProtobuf(t *testing.T) {
-	c := NewMock(t)
-	for _, column := range c.Columns() {
-		if column.ProtobufIndex >= 0 {
-			if column.ProtobufType == 0 {
-				t.Errorf("column %s has not protobuf type", column.Name)
-			}
-		}
-	}
-}
-
 func TestColumnIndex(t *testing.T) {
 	c := NewMock(t)
 	for i := ColumnTimeReceived; i < ColumnLast; i++ {
diff --git a/common/schema/insert_test.go b/common/schema/insert_test.go
new file mode 100644
index 00000000..2d5147dd
--- /dev/null
+++ b/common/schema/insert_test.go
@@ -0,0 +1,197 @@
+// SPDX-FileCopyrightText: 2025 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+package schema_test
+
+import (
+	"context"
+	"encoding/json"
+	"fmt"
+	"io"
+	"net/netip"
+	"testing"
+	"time"
+
+	"github.com/ClickHouse/ch-go"
+	"github.com/ClickHouse/clickhouse-go/v2"
+
+	"akvorado/common/helpers"
+	"akvorado/common/schema"
+)
+
+func TestInsertMemory(t *testing.T) {
+	c := schema.NewMock(t)
+	bf := c.NewFlowMessage()
+	exporterAddress := netip.MustParseAddr("::ffff:203.0.113.14")
+
+	bf.TimeReceived = 1000
+	bf.SamplingRate = 20000
+	bf.ExporterAddress = exporterAddress
+	bf.AppendString(schema.ColumnExporterName, "router1.example.net")
+	bf.AppendUint(schema.ColumnSrcAS, 65000)
+	bf.AppendUint(schema.ColumnDstAS, 12322)
+	bf.AppendUint(schema.ColumnBytes, 20)
+	bf.AppendUint(schema.ColumnPackets, 3)
+	bf.AppendUint(schema.ColumnInIfBoundary, uint64(schema.InterfaceBoundaryInternal))
+	bf.AppendUint(schema.ColumnOutIfBoundary, uint64(schema.InterfaceBoundaryExternal))
+	bf.AppendUint(schema.ColumnInIfSpeed, 10000)
+	bf.AppendUint(schema.ColumnEType, helpers.ETypeIPv4)
+	bf.Finalize()
+
+	bf.TimeReceived = 1001
+	bf.SamplingRate = 20000
+	bf.ExporterAddress = exporterAddress
+	bf.AppendString(schema.ColumnExporterName, "router1.example.net")
+	bf.AppendUint(schema.ColumnSrcAS, 12322)
+	bf.AppendUint(schema.ColumnDstAS, 65000)
+	bf.AppendUint(schema.ColumnBytes, 200)
+	bf.AppendUint(schema.ColumnPackets, 3)
+	bf.AppendUint(schema.ColumnInIfBoundary, uint64(schema.InterfaceBoundaryExternal))
+	bf.AppendUint(schema.ColumnOutIfSpeed, 10000)
+	bf.AppendUint(schema.ColumnEType, helpers.ETypeIPv4)
+	bf.AppendArrayUInt32(schema.ColumnDstASPath, []uint32{65400, 65500, 65001})
+	bf.AppendArrayUInt128(schema.ColumnDstLargeCommunities, []schema.UInt128{
+		{
+			High: 65401,
+			Low:  (100 << 32) + 200,
+		},
+		{
+			High: 65401,
+			Low:  (100 << 32) + 201,
+		},
+	})
+	bf.Finalize()
+
+	server := helpers.CheckExternalService(t, "ClickHouse", []string{"clickhouse:9000", "127.0.0.1:9000"})
+	ctx := t.Context()
+
+	conn, err := ch.Dial(ctx, ch.Options{
+		Address:     server,
+		Database:    "test",
+		DialTimeout: 100 * time.Millisecond,
+		Settings: []ch.Setting{
+			{Key: "allow_suspicious_low_cardinality_types", Value: "1"},
+		},
+	})
+	if err != nil {
+		t.Fatalf("Dial() error:\n%+v", err)
+	}
+
+	// Create the table
+	q := fmt.Sprintf(
+		`CREATE OR REPLACE TABLE test_schema_insert (%s) ENGINE = Memory`,
+		c.ClickHouseCreateTable(schema.ClickHouseSkipAliasedColumns, schema.ClickHouseSkipGeneratedColumns),
+	)
+	t.Logf("Query: %s", q)
+	if err := conn.Do(ctx, ch.Query{
+		Body: q,
+	}); err != nil {
+		t.Fatalf("Do() error:\n%+v", err)
+	}
+
+	// Insert
+	input := bf.ClickHouseProtoInput()
+	if err := conn.Do(ctx, ch.Query{
+		Body:  input.Into("test_schema_insert"),
+		Input: input,
+		OnInput: func(ctx context.Context) error {
+			bf.Clear()
+			// No more data to send!
+			return io.EOF
+		},
+	}); err != nil {
+		t.Fatalf("Do() error:\n%+v", err)
+	}
+
+	// Check the result (with the full-featured client)
+	{
+		conn, err := clickhouse.Open(&clickhouse.Options{
+			Addr: []string{server},
+			Auth: clickhouse.Auth{
+				Database: "test",
+			},
+			DialTimeout: 100 * time.Millisecond,
+		})
+		if err != nil {
+			t.Fatalf("clickhouse.Open() error:\n%+v", err)
+		}
+		// Use formatRow to get JSON representation
+		rows, err := conn.Query(ctx, "SELECT formatRow('JSONEachRow', *) FROM test_schema_insert ORDER BY TimeReceived")
+		if err != nil {
+			t.Fatalf("clickhouse.Query() error:\n%+v", err)
+		}
+
+		var got []map[string]any
+		for rows.Next() {
+			var jsonRow string
+			if err := rows.Scan(&jsonRow); err != nil {
+				t.Fatalf("rows.Scan() error:\n%+v", err)
+			}
+
+			var row map[string]any
+			if err := json.Unmarshal([]byte(jsonRow), &row); err != nil {
+				t.Fatalf("json.Unmarshal() error:\n%+v", err)
+			}
+
+			// Remove fields with default values
+			for k, v := range row {
+				switch val := v.(type) {
+				case string:
+					if val == "" || val == "::" {
+						delete(row, k)
+					}
+				case float64:
+					if val == 0 {
+						delete(row, k)
+					}
+				case []any:
+					if len(val) == 0 {
+						delete(row, k)
+					}
+				}
+			}
+			got = append(got, row)
+		}
+		rows.Close()
+
+		expected := []map[string]any{
+			{
+				"TimeReceived":    "1970-01-01 00:16:40",
+				"SamplingRate":    "20000",
+				"ExporterAddress": "::ffff:203.0.113.14",
+				"ExporterName":    "router1.example.net",
+				"SrcAS":           65000,
+				"DstAS":           12322,
+				"Bytes":           "20",
+				"Packets":         "3",
+				"InIfBoundary":    "internal",
+				"OutIfBoundary":   "external",
+				"InIfSpeed":       10000,
+				"EType":           helpers.ETypeIPv4,
+			}, {
+				"TimeReceived":    "1970-01-01 00:16:41",
+				"SamplingRate":    "20000",
+				"ExporterAddress": "::ffff:203.0.113.14",
+				"ExporterName":    "router1.example.net",
+				"SrcAS":           12322,
+				"DstAS":           65000,
+				"Bytes":           "200",
+				"Packets":         "3",
+				"InIfBoundary":    "external",
+				"OutIfBoundary":   "undefined",
+				"OutIfSpeed":      10000,
+				"EType":           helpers.ETypeIPv4,
+				"DstASPath":       []uint32{65400, 65500, 65001},
+				"DstLargeCommunities": []string{
+					"1206435509165107881967816", // 65401:100:200
+					"1206435509165107881967817", // 65401:100:201
+				},
+			},
+		}
+
+		if diff := helpers.Diff(got, expected); diff != "" {
+			t.Errorf("Insert (-got, +want):\n%s", diff)
+		}
+	}
+
+}
diff --git a/common/schema/protobuf_test.go b/common/schema/protobuf_test.go
deleted file mode 100644
index a68a6dc5..00000000
--- a/common/schema/protobuf_test.go
+++ /dev/null
@@ -1,242 +0,0 @@
-// SPDX-FileCopyrightText: 2022 Free Mobile
-// SPDX-License-Identifier: AGPL-3.0-only
-
-package schema
-
-import (
-	"fmt"
-	"net/netip"
-	"strings"
-	"testing"
-
-	"akvorado/common/helpers"
-
-	"google.golang.org/protobuf/encoding/protowire"
-	"google.golang.org/protobuf/reflect/protoreflect"
-)
-
-func TestProtobufDefinition(t *testing.T) {
-	// Use a smaller version
-	flows := Schema{
-		columns: []Column{
-			{
-				Key:            ColumnTimeReceived,
-				ClickHouseType: "DateTime",
-				ProtobufType:   protoreflect.Uint64Kind,
-			},
-			{Key: ColumnSamplingRate, ClickHouseType: "UInt64"},
-			{Key: ColumnExporterAddress, ClickHouseType: "LowCardinality(IPv6)"},
-			{Key: ColumnExporterName, ClickHouseType: "LowCardinality(String)"},
-			{
-				Key:            ColumnSrcAddr,
-				ClickHouseType: "IPv6",
-			},
-			{
-				Key:            ColumnSrcNetMask,
-				ClickHouseType: "UInt8",
-			},
-			{
-				Key:             ColumnSrcNetPrefix,
-				ClickHouseType:  "String",
-				ClickHouseAlias: `something`,
-			},
-			{Key: ColumnSrcAS, ClickHouseType: "UInt32"},
-			{
-				Key:                    ColumnSrcNetName,
-				ClickHouseType:         "LowCardinality(String)",
-				ClickHouseGenerateFrom: fmt.Sprintf("dictGetOrDefault('%s', 'name', SrcAddr, '')", DictionaryNetworks),
-			},
-			{
-				Key:            ColumnDstASPath,
-				ClickHouseType: "Array(UInt32)",
-			},
-			{
-				Key:            ColumnDstLargeCommunities,
-				ClickHouseType: "Array(UInt128)",
-				ClickHouseTransformFrom: []Column{
-					{Key: ColumnDstLargeCommunitiesASN, ClickHouseType: "Array(UInt32)"},
-					{Key: ColumnDstLargeCommunitiesLocalData1, ClickHouseType: "Array(UInt32)"},
-					{Key: ColumnDstLargeCommunitiesLocalData2, ClickHouseType: "Array(UInt32)"},
-				},
-				ClickHouseTransformTo: "something",
-			},
-			{Key: ColumnInIfName, ClickHouseType: "LowCardinality(String)"},
-			{
-				Key:                     ColumnInIfBoundary,
-				ClickHouseType:          "Enum8('undefined' = 0, 'external' = 1, 'internal' = 2)",
-				ClickHouseNotSortingKey: true,
-				ProtobufType:            protoreflect.EnumKind,
-				ProtobufEnumName:        "Boundary",
-				ProtobufEnum: map[int]string{
-					0: "UNDEFINED",
-					1: "EXTERNAL",
-					2: "INTERNAL",
-				},
-			},
-			{Key: ColumnBytes, ClickHouseType: "UInt64"},
-		},
-	}.finalize()
-
-	got := flows.ProtobufDefinition()
-	expected := `
-syntax = "proto3";
-
-message FlowMessagev5WRSGBXQDXZSUHZQE6QEHLI5JM {
- enum Boundary { UNDEFINED = 0; EXTERNAL = 1; INTERNAL = 2; }
-
- uint64 TimeReceived = 1;
- uint64 SamplingRate = 2;
- bytes ExporterAddress = 3;
- string ExporterName = 4;
- bytes SrcAddr = 5;
- bytes DstAddr = 6;
- uint32 SrcNetMask = 7;
- uint32 DstNetMask = 8;
- uint32 SrcAS = 9;
- uint32 DstAS = 10;
- repeated uint32 DstASPath = 11;
- repeated uint32 DstLargeCommunitiesASN = 12;
- repeated uint32 DstLargeCommunitiesLocalData1 = 13;
- repeated uint32 DstLargeCommunitiesLocalData2 = 14;
- string InIfName = 15;
- string OutIfName = 16;
- Boundary InIfBoundary = 17;
- Boundary OutIfBoundary = 18;
- uint64 Bytes = 19;
-}
-`
-	if diff := helpers.Diff(strings.Split(got, "\n"), strings.Split(expected, "\n")); diff != "" {
-		t.Fatalf("ProtobufDefinition() (-got, +want): %s", diff)
-	}
-}
-
-func TestProtobufMarshal(t *testing.T) {
-	c := NewMock(t)
-	exporterAddress := netip.MustParseAddr("::ffff:203.0.113.14")
-	bf := &FlowMessage{}
-	bf.TimeReceived = 1000
-	bf.SamplingRate = 20000
-	bf.ExporterAddress = exporterAddress
-	c.ProtobufAppendVarint(bf, ColumnDstAS, 65000)
-	c.ProtobufAppendVarint(bf, ColumnBytes, 200)
-	c.ProtobufAppendVarint(bf, ColumnPackets, 300)
-	c.ProtobufAppendVarint(bf, ColumnBytes, 300) // duplicate!
-
-	got := c.ProtobufMarshal(bf)
-
-	size, n := protowire.ConsumeVarint(got)
-	if uint64(len(got)-n) != size {
-		t.Fatalf("ProtobufMarshal() produced an incorrect size: %d + %d != %d", size, n, len(got))
-	}
-
-	// text schema definition for reference
-	// syntax = "proto3";
-
-	// message FlowMessagevLAABIGYMRYZPTGOYIIFZNYDEQM {
-	// enum Boundary { UNDEFINED = 0; EXTERNAL = 1; INTERNAL = 2; }
-
-	// uint64 TimeReceived = 1;
-	// uint64 SamplingRate = 2;
-	// bytes ExporterAddress = 3;
-	// string ExporterName = 4;
-	// string ExporterGroup = 5;
-	// string ExporterRole = 6;
-	// string ExporterSite = 7;
-	// string ExporterRegion = 8;
-	// string ExporterTenant = 9;
-	// bytes SrcAddr = 10;
-	// bytes DstAddr = 11;
-	// uint32 SrcNetMask = 12;
-	// uint32 DstNetMask = 13;
-	// uint32 SrcAS = 14;
-	// uint32 DstAS = 15;
-	// repeated uint32 DstASPath = 18;
-	// repeated uint32 DstCommunities = 19;
-	// repeated uint32 DstLargeCommunitiesASN = 20;
-	// repeated uint32 DstLargeCommunitiesLocalData1 = 21;
-	// repeated uint32 DstLargeCommunitiesLocalData2 = 22;
-	// string InIfName = 23;
-	// string OutIfName = 24;
-	// string InIfDescription = 25;
-	// string OutIfDescription = 26;
-	// uint32 InIfSpeed = 27;
-	// uint32 OutIfSpeed = 28;
-	// string InIfConnectivity = 29;
-	// string OutIfConnectivity = 30;
-	// string InIfProvider = 31;
-	// string OutIfProvider = 32;
-	// Boundary InIfBoundary = 33;
-	// Boundary OutIfBoundary = 34;
-	// uint32 EType = 35;
-	// uint32 Proto = 36;
-	// uint32 SrcPort = 37;
-	// uint32 DstPort = 38;
-	// uint64 Bytes = 39;
-	// uint64 Packets = 40;
-	// uint32 ForwardingStatus = 41;
-	// }
-	// to check: https://protobuf-decoder.netlify.app/
-	t.Run("compare as bytes", func(t *testing.T) {
-		expected := []byte{
-			// DstAS
-			// 15: 65000
-			0x78, 0xe8, 0xfb, 0x03,
-			// Bytes
-			// 39: 200
-			0xb8, 0x02, 0xc8, 0x01,
-			// Packet
-			// 40: 300
-			0xc0, 0x02, 0xac, 0x02,
-			// TimeReceived
-			// 1: 1000
-			0x08, 0xe8, 0x07,
-			// SamplingRate
-			// 2: 20000
-			0x10, 0xa0, 0x9c, 0x01,
-			// ExporterAddress
-			// 3: ::ffff:203.0.113.14
-			0x1a, 0x10, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xcb, 0x0, 0x71, 0xe,
-		}
-		if diff := helpers.Diff(got[n:], expected); diff != "" {
-			t.Logf("got: %v", got)
-
-			t.Fatalf("ProtobufMarshal() (-got, +want):\n%s", diff)
-		}
-	})
-
-	t.Run("compare as protobuf message", func(t *testing.T) {
-		got := c.ProtobufDecode(t, got)
-		expected := FlowMessage{
-			TimeReceived:    1000,
-			SamplingRate:    20000,
-			ExporterAddress: exporterAddress,
-			DstAS:           65000,
-			ProtobufDebug: map[ColumnKey]interface{}{
-				ColumnBytes:   200,
-				ColumnPackets: 300,
-			},
-		}
-		if diff := helpers.Diff(got, expected); diff != "" {
-			t.Fatalf("ProtobufDecode() (-got, +want):\n%s", diff)
-		}
-	})
-}
-
-func BenchmarkProtobufMarshal(b *testing.B) {
-	c := NewMock(b)
-	exporterAddress := netip.MustParseAddr("::ffff:203.0.113.14")
-	DisableDebug(b)
-	for b.Loop() {
-		bf := &FlowMessage{
-			TimeReceived:    1000,
-			SamplingRate:    20000,
-			ExporterAddress: exporterAddress,
-		}
-		c.ProtobufAppendVarint(bf, ColumnDstAS, 65000)
-		c.ProtobufAppendVarint(bf, ColumnBytes, 200)
-		c.ProtobufAppendVarint(bf, ColumnPackets, 300)
-		c.ProtobufAppendVarint(bf, ColumnBytes, 300)    // duplicate!
-		c.ProtobufAppendVarint(bf, ColumnSrcVlan, 1600) // disabled!
-		c.ProtobufMarshal(bf)
-	}
-}
diff --git a/common/schema/root_test.go b/common/schema/root_test.go
index 393f9aef..fb7b02cb 100644
--- a/common/schema/root_test.go
+++ b/common/schema/root_test.go
@@ -38,29 +38,6 @@ func TestDisableForbiddenColumns(t *testing.T) {
 	if _, err := schema.New(config); err == nil {
 		t.Fatal("New() did not error")
 	}
-
-	config = schema.DefaultConfiguration()
-	config.Disabled = []schema.ColumnKey{schema.ColumnDstLargeCommunitiesASN}
-	if _, err := schema.New(config); err == nil {
-		t.Fatal("New() did not error")
-	}
-
-	config = schema.DefaultConfiguration()
-	config.Disabled = []schema.ColumnKey{schema.ColumnDstLargeCommunities}
-	if _, err := schema.New(config); err == nil {
-		t.Fatal("New() did not error")
-	}
-
-	config = schema.DefaultConfiguration()
-	config.Disabled = []schema.ColumnKey{
-		schema.ColumnDstLargeCommunities,
-		schema.ColumnDstLargeCommunitiesASN,
-		schema.ColumnDstLargeCommunitiesLocalData1,
-		schema.ColumnDstLargeCommunitiesLocalData2,
-	}
-	if _, err := schema.New(config); err != nil {
-		t.Fatalf("New() error:\n%+v", err)
-	}
 }
 
 func TestCustomDictionaries(t *testing.T) {
diff --git a/common/schema/tests.go b/common/schema/tests.go
index 3e4e2aef..1520910c 100644
--- a/common/schema/tests.go
+++ b/common/schema/tests.go
@@ -7,17 +7,10 @@ package schema
 
 import (
 	"fmt"
-	"net/netip"
 	"reflect"
-	"strings"
 	"testing"
 
 	"akvorado/common/helpers"
-
-	"github.com/jhump/protoreflect/desc"
-	"github.com/jhump/protoreflect/desc/protoparse"
-	"github.com/jhump/protoreflect/dynamic"
-	"google.golang.org/protobuf/encoding/protowire"
 )
 
 var debug = true
@@ -41,82 +34,6 @@ func NewMock(t testing.TB) *Component {
 	return c
 }
 
-// ProtobufDecode decodes the provided protobuf message.
-func (schema *Schema) ProtobufDecode(t *testing.T, input []byte) *FlowMessage {
-	t.Helper()
-	parser := protoparse.Parser{
-		Accessor: protoparse.FileContentsFromMap(map[string]string{
-			"flow.proto": schema.ProtobufDefinition(),
-		}),
-	}
-	descs, err := parser.ParseFiles("flow.proto")
-	if err != nil {
-		t.Fatalf("ParseFiles(%q) error:\n%+v", "flow.proto", err)
-	}
-
-	var descriptor *desc.MessageDescriptor
-	for _, msg := range descs[0].GetMessageTypes() {
-		if strings.HasPrefix(msg.GetName(), "FlowMessagev") {
-			descriptor = msg
-			break
-		}
-	}
-	if descriptor == nil {
-		t.Fatal("cannot find message descriptor")
-	}
-
-	message := dynamic.NewMessage(descriptor)
-	size, n := protowire.ConsumeVarint(input)
-	if len(input)-n != int(size) {
-		t.Fatalf("bad length for protobuf message: %d - %d != %d", len(input), n, size)
-	}
-	if err := message.Unmarshal(input[n:]); err != nil {
-		t.Fatalf("Unmarshal() error:\n%+v", err)
-	}
-	textVersion, _ := message.MarshalTextIndent()
-	t.Logf("Unmarshal():\n%s", textVersion)
-
-	flow := FlowMessage{
-		ProtobufDebug: map[ColumnKey]interface{}{},
-	}
-	for _, field := range message.GetKnownFields() {
-		k := int(field.GetNumber())
-		name := field.GetName()
-		switch name {
-		case "TimeReceived":
-			flow.TimeReceived = message.GetFieldByNumber(k).(uint64)
-		case "SamplingRate":
-			flow.SamplingRate = uint32(message.GetFieldByNumber(k).(uint64))
-		case "ExporterAddress":
-			ip, _ := netip.AddrFromSlice(message.GetFieldByNumber(k).([]byte))
-			flow.ExporterAddress = ip
-		case "SrcAddr":
-			ip, _ := netip.AddrFromSlice(message.GetFieldByNumber(k).([]byte))
-			flow.SrcAddr = ip
-		case "DstAddr":
-			ip, _ := netip.AddrFromSlice(message.GetFieldByNumber(k).([]byte))
-			flow.DstAddr = ip
-		case "SrcAS":
-			flow.SrcAS = uint32(message.GetFieldByNumber(k).(uint32))
-		case "DstAS":
-			flow.DstAS = uint32(message.GetFieldByNumber(k).(uint32))
-		default:
-			column, ok := schema.LookupColumnByName(name)
-			if !ok {
-				break
-			}
-			key := column.Key
-			value := message.GetFieldByNumber(k)
-			if reflect.ValueOf(value).IsZero() {
-				break
-			}
-			flow.ProtobufDebug[key] = value
-		}
-	}
-
-	return &flow
-}
-
 // EnableAllColumns enable all columns and returns itself.
 func (schema *Component) EnableAllColumns() *Component {
 	for i := range schema.columns {
diff --git a/common/schema/types_test.go b/common/schema/types_test.go
deleted file mode 100644
index 5fb97d93..00000000
--- a/common/schema/types_test.go
+++ /dev/null
@@ -1,17 +0,0 @@
-// SPDX-FileCopyrightText: 2022 Free Mobile
-// SPDX-License-Identifier: AGPL-3.0-only
-
-package schema
-
-import (
-	"testing"
-
-	"google.golang.org/protobuf/encoding/protowire"
-)
-
-func TestMaxSizeVarint(t *testing.T) {
-	got := protowire.SizeVarint(^uint64(0))
-	if got != maxSizeVarint {
-		t.Fatalf("maximum size for varint is %d, not %d", got, maxSizeVarint)
-	}
-}
diff --git a/console/admonitions_test.go b/console/admonitions_test.go
new file mode 100644
index 00000000..00283268
--- /dev/null
+++ b/console/admonitions_test.go
@@ -0,0 +1,209 @@
+// SPDX-FileCopyrightText: 2025 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+package console
+
+import (
+	"bytes"
+	"strings"
+	"testing"
+
+	"github.com/yuin/goldmark"
+)
+
+func TestAdmonition(t *testing.T) {
+	md := goldmark.New(
+		goldmark.WithExtensions(&admonitionExtension{}),
+	)
+
+	tests := []struct {
+		name        string
+		input       string
+		contains    []string
+		notContains []string
+	}{
+		{
+			name: "IMPORTANT admonition with content",
+			input: `> [!IMPORTANT]
+> This is important information.
+> It spans multiple lines.`,
+			contains: []string{
+				`class="admonition admonition-important"`,
+				`Important</p>`,
+				`<p>This is important information.`,
+				`It spans multiple lines.</p>`,
+			},
+		},
+		{
+			name: "NOTE admonition",
+			input: `> [!NOTE]
+> This is a note.`,
+			contains: []string{
+				`class="admonition admonition-note"`,
+				`Note</p>`,
+				`<p>This is a note.</p>`,
+			},
+		},
+		{
+			name: "TIP admonition",
+			input: `> [!TIP]
+> This is a tip.`,
+			contains: []string{
+				`class="admonition admonition-tip"`,
+				`Tip</p>`,
+				`<p>This is a tip.</p>`,
+			},
+		},
+		{
+			name: "WARNING admonition",
+			input: `> [!WARNING]
+> This is a warning.`,
+			contains: []string{
+				`class="admonition admonition-warning"`,
+				`Warning</p>`,
+				`<p>This is a warning.</p>`,
+			},
+		},
+		{
+			name: "CAUTION admonition",
+			input: `> [!CAUTION]
+> This is a caution.`,
+			contains: []string{
+				`class="admonition admonition-caution"`,
+				`Caution</p>`,
+				`<p>This is a caution.</p>`,
+			},
+		},
+		{
+			name: "CAUTION and TIP adominitions",
+			input: `
+This is just a text.
+
+> [!CAUTION]
+> This is a caution.
+
+> [!TIP]
+> This is a tip.
+`,
+			contains: []string{
+				`class="admonition admonition-caution"`,
+				`Caution</p>`,
+				`<p>This is a caution.</p>`,
+				`class="admonition admonition-tip"`,
+				`Tip</p>`,
+				`<p>This is a tip.</p>`,
+			},
+		},
+		{
+			name: "Regular blockquote should not be affected",
+			input: `> This is a regular blockquote.
+> It should not be styled as an admonition.`,
+			contains: []string{
+				`<blockquote>`,
+				`<p>This is a regular blockquote.`,
+			},
+		},
+		{
+			name: "Links inside admonition",
+			input: `> [!NOTE]
+> Check the [configuration guide](config.md) for more details.`,
+			contains: []string{
+				`class="admonition admonition-note"`,
+				`Note</p>`,
+				`<p>Check the <a href="config.md">configuration guide</a> for more details.</p>`,
+			},
+			notContains: []string{
+				`[!NOTE]`,
+				`<blockquote>`,
+			},
+		},
+		{
+			name: "Emphasis inside admonition",
+			input: `> [!WARNING]
+> This is *very* important and **must** be done.`,
+			contains: []string{
+				`class="admonition admonition-warning"`,
+				`Warning</p>`,
+				`<p>This is <em>very</em> important and <strong>must</strong> be done.</p>`,
+			},
+			notContains: []string{
+				`[!WARNING]`,
+				`<blockquote>`,
+			},
+		},
+		{
+			name: "Code inside admonition",
+			input: `> [!TIP]
+> Use the` + " `docker compose up` " + `command to start the services.`,
+			contains: []string{
+				`class="admonition admonition-tip"`,
+				`Tip</p>`,
+				`<p>Use the <code>docker compose up</code> command to start the services.</p>`,
+			},
+			notContains: []string{
+				`[!TIP]`,
+				`<blockquote>`,
+			},
+		},
+		{
+			name: "Multiple markdown features",
+			input: `> [!IMPORTANT]
+> Read the **[documentation](docs.md)** carefully.
+> The` + " `config.yaml` " + `file is *essential*.`,
+			contains: []string{
+				`class="admonition admonition-important"`,
+				`Important</p>`,
+				`<p>Read the <strong><a href="docs.md">documentation</a></strong> carefully.`,
+				`The <code>config.yaml</code> file is <em>essential</em>.</p>`,
+			},
+			notContains: []string{
+				`[!IMPORTANT]`,
+				`<blockquote>`,
+			},
+		},
+		{
+			name: "List inside admonition",
+			input: `> [!NOTE]
+> Follow these steps:
+> 1. First step
+> 2. Second step with **bold** text`,
+			contains: []string{
+				`class="admonition admonition-note"`,
+				`Note</p>`,
+				`<p>Follow these steps:</p>`,
+				`<ol>`,
+				`<li>First step</li>`,
+				`<li>Second step with <strong>bold</strong> text</li>`,
+				`</ol>`,
+			},
+			notContains: []string{
+				`[!NOTE]`,
+				`<blockquote>`,
+			},
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			var buf bytes.Buffer
+			if err := md.Convert([]byte(tt.input), &buf); err != nil {
+				t.Errorf("Convert() error:\n%+v", err)
+				return
+			}
+
+			html := buf.String()
+
+			for _, expected := range tt.contains {
+				if !strings.Contains(html, expected) {
+					t.Errorf("Convert() should have %q:\n%s", expected, html)
+				}
+			}
+
+			for _, notExpected := range tt.notContains {
+				if strings.Contains(html, notExpected) {
+					t.Errorf("Convert() should not have %q:\n%s", notExpected, html)
+				}
+			}
+		})
+	}
+}
diff --git a/demoexporter/flows/nfdata_test.go b/demoexporter/flows/nfdata_test.go
index 804b7ff8..7f00256b 100644
--- a/demoexporter/flows/nfdata_test.go
+++ b/demoexporter/flows/nfdata_test.go
@@ -11,15 +11,18 @@ import (
 	"time"
 
 	"akvorado/common/helpers"
+	"akvorado/common/pb"
 	"akvorado/common/reporter"
 	"akvorado/common/schema"
-	"akvorado/inlet/flow/decoder"
-	"akvorado/inlet/flow/decoder/netflow"
+	"akvorado/outlet/flow/decoder"
+	"akvorado/outlet/flow/decoder/netflow"
 )
 
 func TestGetNetflowData(t *testing.T) {
 	r := reporter.NewMock(t)
-	nfdecoder := netflow.New(r, decoder.Dependencies{Schema: schema.NewMock(t)}, decoder.Option{TimestampSource: decoder.TimestampSourceUDP})
+	sch := schema.NewMock(t)
+	bf := sch.NewFlowMessage()
+	nfdecoder := netflow.New(r, decoder.Dependencies{Schema: sch})
 
 	ch := getNetflowTemplates(
 		context.Background(),
@@ -27,11 +30,22 @@ func TestGetNetflowData(t *testing.T) {
 		30000,
 		time.Date(2022, 3, 15, 14, 33, 0, 0, time.UTC),
 		time.Date(2022, 3, 15, 15, 33, 0, 0, time.UTC))
-	got := []interface{}{}
+	got := []*schema.FlowMessage{}
+	finalize := func() {
+		bf.TimeReceived = 0
+		// Keep a copy of the current flow message
+		clone := *bf
+		got = append(got, &clone)
+		// And clear the flow message
+		bf.Clear()
+	}
+
 	for payload := range ch {
-		got = append(got, nfdecoder.Decode(decoder.RawFlow{
-			Payload: payload, Source: net.ParseIP("127.0.0.1"),
-		}))
+		if _, err := nfdecoder.Decode(decoder.RawFlow{
+			Payload: payload, Source: netip.MustParseAddr("::ffff:127.0.0.1"),
+		}, decoder.Option{TimestampSource: pb.RawFlow_TS_INPUT}, bf, finalize); err != nil {
+			t.Fatalf("Decode() error:\n%+v", err)
+		}
 	}
 
 	ch = getNetflowData(
@@ -97,91 +111,77 @@ func TestGetNetflowData(t *testing.T) {
 		time.Date(2022, 3, 15, 14, 33, 0, 0, time.UTC),
 		time.Date(2022, 3, 15, 16, 33, 0, 0, time.UTC))
 	for payload := range ch {
-		got = append(got, nfdecoder.Decode(decoder.RawFlow{
-			Payload: payload, Source: net.ParseIP("127.0.0.1"),
-		}))
+		if _, err := nfdecoder.Decode(decoder.RawFlow{
+			Payload: payload, Source: netip.MustParseAddr("::ffff:127.0.0.1"),
+		}, decoder.Option{TimestampSource: pb.RawFlow_TS_INPUT}, bf, finalize); err != nil {
+			t.Fatalf("Decode() error:\n%+v", err)
+		}
 	}
-	expected := []interface{}{
-		[]interface{}{}, // templates
-		[]interface{}{
-			&schema.FlowMessage{
-				SamplingRate:    30000,
-				ExporterAddress: netip.MustParseAddr("::ffff:127.0.0.1"),
-				SrcAddr:         netip.MustParseAddr("::ffff:192.0.2.206"),
-				DstAddr:         netip.MustParseAddr("::ffff:203.0.113.165"),
-				InIf:            10,
-				OutIf:           20,
-				SrcAS:           65201,
-				DstAS:           65202,
-				SrcNetMask:      24,
-				DstNetMask:      23,
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
-					schema.ColumnBytes:            1500,
-					schema.ColumnPackets:          1,
-					schema.ColumnEType:            helpers.ETypeIPv4,
-					schema.ColumnProto:            6,
-					schema.ColumnSrcPort:          443,
-					schema.ColumnDstPort:          34974,
-					schema.ColumnForwardingStatus: 64,
-				},
+	expected := []*schema.FlowMessage{
+		{
+			SamplingRate:    30000,
+			ExporterAddress: netip.MustParseAddr("::ffff:127.0.0.1"),
+			SrcAddr:         netip.MustParseAddr("::ffff:192.0.2.206"),
+			DstAddr:         netip.MustParseAddr("::ffff:203.0.113.165"),
+			InIf:            10,
+			OutIf:           20,
+			SrcAS:           65201,
+			DstAS:           65202,
+			SrcNetMask:      24,
+			DstNetMask:      23,
+			OtherColumns: map[schema.ColumnKey]any{
+				schema.ColumnBytes:            1500,
+				schema.ColumnPackets:          1,
+				schema.ColumnEType:            helpers.ETypeIPv4,
+				schema.ColumnProto:            6,
+				schema.ColumnSrcPort:          443,
+				schema.ColumnDstPort:          34974,
+				schema.ColumnForwardingStatus: 64,
 			},
-			&schema.FlowMessage{
-				SamplingRate:    30000,
-				ExporterAddress: netip.MustParseAddr("::ffff:127.0.0.1"),
-				SrcAddr:         netip.MustParseAddr("::ffff:192.0.2.236"),
-				DstAddr:         netip.MustParseAddr("::ffff:203.0.113.67"),
-				InIf:            10,
-				OutIf:           20,
-				SrcAS:           65201,
-				DstAS:           65202,
-				SrcNetMask:      24,
-				DstNetMask:      24,
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
-					schema.ColumnBytes:            1339,
-					schema.ColumnPackets:          1,
-					schema.ColumnEType:            helpers.ETypeIPv4,
-					schema.ColumnProto:            6,
-					schema.ColumnSrcPort:          443,
-					schema.ColumnDstPort:          33199,
-					schema.ColumnForwardingStatus: 64,
-				},
+		},
+		{
+			SamplingRate:    30000,
+			ExporterAddress: netip.MustParseAddr("::ffff:127.0.0.1"),
+			SrcAddr:         netip.MustParseAddr("::ffff:192.0.2.236"),
+			DstAddr:         netip.MustParseAddr("::ffff:203.0.113.67"),
+			InIf:            10,
+			OutIf:           20,
+			SrcAS:           65201,
+			DstAS:           65202,
+			SrcNetMask:      24,
+			DstNetMask:      24,
+			OtherColumns: map[schema.ColumnKey]any{
+				schema.ColumnBytes:            1339,
+				schema.ColumnPackets:          1,
+				schema.ColumnEType:            helpers.ETypeIPv4,
+				schema.ColumnProto:            6,
+				schema.ColumnSrcPort:          443,
+				schema.ColumnDstPort:          33199,
+				schema.ColumnForwardingStatus: 64,
 			},
 		},
-		[]interface{}{
-			&schema.FlowMessage{
-				SamplingRate:    30000,
-				ExporterAddress: netip.MustParseAddr("::ffff:127.0.0.1"),
-				SrcAddr:         netip.MustParseAddr("2001:db8::1"),
-				DstAddr:         netip.MustParseAddr("2001:db8:2:0:cea5:d643:ec43:3772"),
-				InIf:            20,
-				OutIf:           10,
-				SrcAS:           65201,
-				DstAS:           65202,
-				SrcNetMask:      48,
-				DstNetMask:      48,
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
-					schema.ColumnBytes:            1300,
-					schema.ColumnPackets:          1,
-					schema.ColumnEType:            helpers.ETypeIPv6,
-					schema.ColumnProto:            6,
-					schema.ColumnSrcPort:          33179,
-					schema.ColumnDstPort:          443,
-					schema.ColumnForwardingStatus: 64,
-				},
+		{
+			SamplingRate:    30000,
+			ExporterAddress: netip.MustParseAddr("::ffff:127.0.0.1"),
+			SrcAddr:         netip.MustParseAddr("2001:db8::1"),
+			DstAddr:         netip.MustParseAddr("2001:db8:2:0:cea5:d643:ec43:3772"),
+			InIf:            20,
+			OutIf:           10,
+			SrcAS:           65201,
+			DstAS:           65202,
+			SrcNetMask:      48,
+			DstNetMask:      48,
+			OtherColumns: map[schema.ColumnKey]any{
+				schema.ColumnBytes:            1300,
+				schema.ColumnPackets:          1,
+				schema.ColumnEType:            helpers.ETypeIPv6,
+				schema.ColumnProto:            6,
+				schema.ColumnSrcPort:          33179,
+				schema.ColumnDstPort:          443,
+				schema.ColumnForwardingStatus: 64,
 			},
 		},
 	}
-	for idx1 := range got {
-		if got[idx1] == nil {
-			continue
-		}
-		switch g := got[idx1].(type) {
-		case []*schema.FlowMessage:
-			for idx2 := range g {
-				g[idx2].TimeReceived = 0
-			}
-		}
-	}
 
 	if diff := helpers.Diff(got, expected); diff != "" {
 		t.Fatalf("getNetflowData() (-got, +want):\n%s", diff)
diff --git a/inlet/core/root_test.go b/inlet/core/root_test.go
deleted file mode 100644
index 8e899020..00000000
--- a/inlet/core/root_test.go
+++ /dev/null
@@ -1,363 +0,0 @@
-// SPDX-FileCopyrightText: 2022 Free Mobile
-// SPDX-License-Identifier: AGPL-3.0-only
-
-package core
-
-import (
-	"bufio"
-	"context"
-	"encoding/json"
-	"fmt"
-	"io"
-	"net/http"
-	"net/netip"
-	"strings"
-	"testing"
-	"time"
-
-	"github.com/IBM/sarama"
-	"github.com/gin-gonic/gin"
-
-	"akvorado/common/daemon"
-	"akvorado/common/helpers"
-	"akvorado/common/httpserver"
-	"akvorado/common/reporter"
-	"akvorado/common/schema"
-	"akvorado/inlet/flow"
-	"akvorado/inlet/kafka"
-	"akvorado/inlet/metadata"
-	"akvorado/inlet/routing"
-)
-
-func TestCore(t *testing.T) {
-	r := reporter.NewMock(t)
-
-	// Prepare all components.
-	daemonComponent := daemon.NewMock(t)
-	metadataComponent := metadata.NewMock(t, r, metadata.DefaultConfiguration(),
-		metadata.Dependencies{Daemon: daemonComponent})
-	flowComponent := flow.NewMock(t, r, flow.DefaultConfiguration())
-	kafkaComponent, kafkaProducer := kafka.NewMock(t, r, kafka.DefaultConfiguration())
-	httpComponent := httpserver.NewMock(t, r)
-	routingComponent := routing.NewMock(t, r)
-	routingComponent.PopulateRIB(t)
-
-	// Instantiate and start core
-	sch := schema.NewMock(t)
-	c, err := New(r, DefaultConfiguration(), Dependencies{
-		Daemon:   daemonComponent,
-		Flow:     flowComponent,
-		Metadata: metadataComponent,
-		Kafka:    kafkaComponent,
-		HTTP:     httpComponent,
-		Routing:  routingComponent,
-		Schema:   sch,
-	})
-	if err != nil {
-		t.Fatalf("New() error:\n%+v", err)
-	}
-	helpers.StartStop(t, c)
-
-	flowMessage := func(exporter string, in, out uint32) *schema.FlowMessage {
-		msg := &schema.FlowMessage{
-			TimeReceived:    200,
-			SamplingRate:    1000,
-			ExporterAddress: netip.MustParseAddr(exporter),
-			InIf:            in,
-			OutIf:           out,
-			SrcAddr:         netip.MustParseAddr("67.43.156.77"),
-			DstAddr:         netip.MustParseAddr("2.125.160.216"),
-			ProtobufDebug: map[schema.ColumnKey]interface{}{
-				schema.ColumnBytes:   6765,
-				schema.ColumnPackets: 4,
-				schema.ColumnEType:   0x800,
-				schema.ColumnProto:   6,
-				schema.ColumnSrcPort: 8534,
-				schema.ColumnDstPort: 80,
-			},
-		}
-		for k, v := range msg.ProtobufDebug {
-			vi := v.(int)
-			sch.ProtobufAppendVarint(msg, k, uint64(vi))
-		}
-		return msg
-	}
-
-	expectedFlowMessage := func(exporter string, in, out uint32) *schema.FlowMessage {
-		expected := flowMessage(exporter, in, out)
-		expected.SrcAS = 0 // no geoip enrich anymore
-		expected.DstAS = 0 // no geoip enrich anymore
-		expected.InIf = 0  // not serialized
-		expected.OutIf = 0 // not serialized
-		expected.ExporterAddress = netip.AddrFrom16(expected.ExporterAddress.As16())
-		expected.SrcAddr = netip.AddrFrom16(expected.SrcAddr.As16())
-		expected.DstAddr = netip.AddrFrom16(expected.DstAddr.As16())
-		expected.ProtobufDebug[schema.ColumnInIfName] = fmt.Sprintf("Gi0/0/%d", in)
-		expected.ProtobufDebug[schema.ColumnOutIfName] = fmt.Sprintf("Gi0/0/%d", out)
-		expected.ProtobufDebug[schema.ColumnInIfDescription] = fmt.Sprintf("Interface %d", in)
-		expected.ProtobufDebug[schema.ColumnOutIfDescription] = fmt.Sprintf("Interface %d", out)
-		expected.ProtobufDebug[schema.ColumnInIfSpeed] = 1000
-		expected.ProtobufDebug[schema.ColumnOutIfSpeed] = 1000
-		expected.ProtobufDebug[schema.ColumnExporterName] = strings.ReplaceAll(exporter, ".", "_")
-		return expected
-	}
-
-	t.Run("kafka", func(t *testing.T) {
-		// Inject several messages with a cache miss from the SNMP
-		// component for each of them. No message sent to Kafka.
-		flowComponent.Inject(flowMessage("192.0.2.142", 434, 677))
-		flowComponent.Inject(flowMessage("192.0.2.143", 434, 677))
-		flowComponent.Inject(flowMessage("192.0.2.143", 437, 677))
-		flowComponent.Inject(flowMessage("192.0.2.143", 434, 679))
-
-		time.Sleep(20 * time.Millisecond)
-		gotMetrics := r.GetMetrics("akvorado_inlet_core_", "-flows_processing_")
-		expectedMetrics := map[string]string{
-			`classifier_exporter_cache_size_items`:                               "0",
-			`classifier_interface_cache_size_items`:                              "0",
-			`flows_errors_total{error="SNMP cache miss",exporter="192.0.2.142"}`: "1",
-			`flows_errors_total{error="SNMP cache miss",exporter="192.0.2.143"}`: "3",
-			`received_flows_total{exporter="192.0.2.142"}`:                       "1",
-			`received_flows_total{exporter="192.0.2.143"}`:                       "3",
-			`flows_http_clients`:                                                 "0",
-		}
-		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
-			t.Fatalf("Metrics (-got, +want):\n%s", diff)
-		}
-
-		// Inject again the messages, this time, we will get a cache hit!
-		kafkaProducer.ExpectInputAndSucceed()
-		flowComponent.Inject(flowMessage("192.0.2.142", 434, 677))
-		kafkaProducer.ExpectInputAndSucceed()
-		flowComponent.Inject(flowMessage("192.0.2.143", 437, 679))
-
-		time.Sleep(20 * time.Millisecond)
-		gotMetrics = r.GetMetrics("akvorado_inlet_core_", "classifier_", "-flows_processing_", "flows_", "received_", "forwarded_")
-		expectedMetrics = map[string]string{
-			`classifier_exporter_cache_size_items`:                               "0",
-			`classifier_interface_cache_size_items`:                              "0",
-			`flows_errors_total{error="SNMP cache miss",exporter="192.0.2.142"}`: "1",
-			`flows_errors_total{error="SNMP cache miss",exporter="192.0.2.143"}`: "3",
-			`received_flows_total{exporter="192.0.2.142"}`:                       "2",
-			`received_flows_total{exporter="192.0.2.143"}`:                       "4",
-			`forwarded_flows_total{exporter="192.0.2.142"}`:                      "1",
-			`forwarded_flows_total{exporter="192.0.2.143"}`:                      "1",
-			`flows_http_clients`:                                                 "0",
-		}
-		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
-			t.Fatalf("Metrics (-got, +want):\n%s", diff)
-		}
-
-		// Now, check we get the message we expect
-		input := flowMessage("192.0.2.142", 434, 677)
-		received := make(chan bool)
-		kafkaProducer.ExpectInputWithMessageCheckerFunctionAndSucceed(func(msg *sarama.ProducerMessage) error {
-			defer close(received)
-			expectedTopic := fmt.Sprintf("flows-%s", sch.ProtobufMessageHash())
-			if msg.Topic != expectedTopic {
-				t.Errorf("Kafka message topic (-got, +want):\n-%s\n+%s", msg.Topic, expectedTopic)
-			}
-
-			b, err := msg.Value.Encode()
-			if err != nil {
-				t.Fatalf("Kafka message encoding error:\n%+v", err)
-			}
-			got := sch.ProtobufDecode(t, b)
-			expected := expectedFlowMessage("192.0.2.142", 434, 677)
-			if diff := helpers.Diff(&got, expected); diff != "" {
-				t.Errorf("Kafka message (-got, +want):\n%s", diff)
-			}
-
-			return nil
-		})
-		flowComponent.Inject(input)
-		select {
-		case <-received:
-		case <-time.After(time.Second):
-			t.Fatal("Kafka message not received")
-		}
-
-		// Try to inject a message with missing sampling rate
-		input = flowMessage("192.0.2.142", 434, 677)
-		input.SamplingRate = 0
-		flowComponent.Inject(input)
-		time.Sleep(20 * time.Millisecond)
-		gotMetrics = r.GetMetrics("akvorado_inlet_core_", "classifier_", "-flows_processing_", "flows_", "forwarded_", "received_")
-		expectedMetrics = map[string]string{
-			`classifier_exporter_cache_size_items`:                                     "0",
-			`classifier_interface_cache_size_items`:                                    "0",
-			`flows_errors_total{error="SNMP cache miss",exporter="192.0.2.142"}`:       "1",
-			`flows_errors_total{error="SNMP cache miss",exporter="192.0.2.143"}`:       "3",
-			`flows_errors_total{error="sampling rate missing",exporter="192.0.2.142"}`: "1",
-			`received_flows_total{exporter="192.0.2.142"}`:                             "4",
-			`received_flows_total{exporter="192.0.2.143"}`:                             "4",
-			`forwarded_flows_total{exporter="192.0.2.142"}`:                            "2",
-			`forwarded_flows_total{exporter="192.0.2.143"}`:                            "1",
-			`flows_http_clients`: "0",
-		}
-		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
-			t.Fatalf("Metrics (-got, +want):\n%s", diff)
-		}
-	})
-
-	// Test the healthcheck function
-	t.Run("healthcheck", func(t *testing.T) {
-		got := r.RunHealthchecks(context.Background())
-		if diff := helpers.Diff(got.Details["core"], reporter.HealthcheckResult{
-			Status: reporter.HealthcheckOK,
-			Reason: "worker 0 ok",
-		}); diff != "" {
-			t.Fatalf("runHealthcheck() (-got, +want):\n%s", diff)
-		}
-	})
-
-	// Test HTTP flow clients (JSON)
-	t.Run("http flows", func(t *testing.T) {
-		c.httpFlowFlushDelay = 20 * time.Millisecond
-
-		resp, err := http.Get(fmt.Sprintf("http://%s/api/v0/inlet/flows", c.d.HTTP.LocalAddr()))
-		if err != nil {
-			t.Fatalf("GET /api/v0/inlet/flows:\n%+v", err)
-		}
-		defer resp.Body.Close()
-		if resp.StatusCode != 200 {
-			t.Fatalf("GET /api/v0/inlet/flows status code %d", resp.StatusCode)
-		}
-
-		// Metrics should tell we have a client
-		gotMetrics := r.GetMetrics("akvorado_inlet_core_", "flows_http_clients", "-flows_processing_")
-		expectedMetrics := map[string]string{
-			`flows_http_clients`: "1",
-		}
-		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
-			t.Fatalf("Metrics (-got, +want):\n%s", diff)
-		}
-
-		// Produce some flows
-		for range 12 {
-			kafkaProducer.ExpectInputAndSucceed()
-			flowComponent.Inject(flowMessage("192.0.2.142", 434, 677))
-		}
-
-		// Decode some of them
-		reader := bufio.NewReader(resp.Body)
-		decoder := json.NewDecoder(reader)
-		for range 10 {
-			var got gin.H
-			if err := decoder.Decode(&got); err != nil {
-				t.Fatalf("GET /api/v0/inlet/flows error while reading body:\n%+v", err)
-			}
-			expected := gin.H{
-				"TimeReceived":    200,
-				"SamplingRate":    1000,
-				"ExporterAddress": "192.0.2.142",
-				"SrcAddr":         "67.43.156.77",
-				"DstAddr":         "2.125.160.216",
-				"SrcAS":           0, // no geoip enrich anymore
-				"InIf":            434,
-				"OutIf":           677,
-
-				"NextHop":        "",
-				"SrcNetMask":     0,
-				"DstNetMask":     0,
-				"SrcVlan":        0,
-				"DstVlan":        0,
-				"GotASPath":      false,
-				"GotCommunities": false,
-				"DstAS":          0,
-			}
-			if diff := helpers.Diff(got, expected); diff != "" {
-				t.Fatalf("GET /api/v0/inlet/flows (-got, +want):\n%s", diff)
-			}
-		}
-	})
-
-	// Test HTTP flow clients with a limit
-	time.Sleep(10 * time.Millisecond)
-	t.Run("http flows with limit", func(t *testing.T) {
-		resp, err := http.Get(fmt.Sprintf("http://%s/api/v0/inlet/flows?limit=4", c.d.HTTP.LocalAddr()))
-		if err != nil {
-			t.Fatalf("GET /api/v0/inlet/flows:\n%+v", err)
-		}
-		defer resp.Body.Close()
-		if resp.StatusCode != 200 {
-			t.Fatalf("GET /api/v0/inlet/flows status code %d", resp.StatusCode)
-		}
-
-		// Metrics should tell we have a client
-		gotMetrics := r.GetMetrics("akvorado_inlet_core_", "flows_http_clients")
-		expectedMetrics := map[string]string{
-			`flows_http_clients`: "1",
-		}
-		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
-			t.Fatalf("Metrics (-got, +want):\n%s", diff)
-		}
-
-		// Produce some flows
-		for range 12 {
-			kafkaProducer.ExpectInputAndSucceed()
-			flowComponent.Inject(flowMessage("192.0.2.142", 434, 677))
-		}
-
-		// Check we got only 4
-		reader := bufio.NewReader(resp.Body)
-		count := 0
-		for {
-			_, err := reader.ReadString('\n')
-			if err == io.EOF {
-				t.Log("EOF")
-				break
-			}
-			if err != nil {
-				t.Fatalf("GET /api/v0/inlet/flows error while reading:\n%+v", err)
-			}
-			count++
-			if count > 4 {
-				break
-			}
-		}
-		if count != 4 {
-			t.Fatalf("GET /api/v0/inlet/flows got less than 4 flows (%d)", count)
-		}
-	})
-
-	// Test HTTP flow clients using protobuf
-	time.Sleep(10 * time.Millisecond)
-	t.Run("http flows with protobuf", func(t *testing.T) {
-		req, err := http.NewRequest(http.MethodGet, fmt.Sprintf("http://%s/api/v0/inlet/flows?limit=1", c.d.HTTP.LocalAddr()), nil)
-		if err != nil {
-			t.Fatalf("http.NewRequest() error:\n%+v", err)
-		}
-		req.Header.Set("accept", "application/x-protobuf")
-		resp, err := http.DefaultClient.Do(req)
-		if err != nil {
-			t.Fatalf("GET /api/v0/inlet/flows:\n%+v", err)
-		}
-		defer resp.Body.Close()
-		if resp.StatusCode != 200 {
-			t.Fatalf("GET /api/v0/inlet/flows status code %d", resp.StatusCode)
-		}
-
-		// Produce some flows
-		for range 12 {
-			kafkaProducer.ExpectInputAndSucceed()
-			flowComponent.Inject(flowMessage("192.0.2.142", 434, 677))
-		}
-
-		// Get the resulting flow
-		reader := bufio.NewReader(resp.Body)
-		got, err := io.ReadAll(reader)
-		if err != nil {
-			t.Fatalf("GET /api/v0/inlet/flows error while reading:\n%+v", err)
-		}
-		t.Logf("got %v", got)
-
-		// Decode
-		sch := schema.NewMock(t)
-		decoded := sch.ProtobufDecode(t, got)
-		expected := expectedFlowMessage("192.0.2.142", 434, 677)
-		if diff := helpers.Diff(decoded, expected); diff != "" {
-			t.Errorf("HTTP message (-got, +want):\n%s", diff)
-		}
-	})
-}
diff --git a/inlet/flow/config_test.go b/inlet/flow/config_test.go
index c6fbc560..987f062d 100644
--- a/inlet/flow/config_test.go
+++ b/inlet/flow/config_test.go
@@ -10,9 +10,9 @@ import (
 	"github.com/gin-gonic/gin"
 
 	"akvorado/common/helpers/yaml"
+	"akvorado/common/pb"
 
 	"akvorado/common/helpers"
-	"akvorado/inlet/flow/decoder"
 	"akvorado/inlet/flow/input/file"
 	"akvorado/inlet/flow/input/udp"
 )
@@ -42,7 +42,7 @@ func TestDecodeConfiguration(t *testing.T) {
 			},
 			Expected: Configuration{
 				Inputs: []InputConfiguration{{
-					Decoder: "netflow",
+					Decoder: pb.RawFlow_DECODER_NETFLOW,
 					Config: &udp.Configuration{
 						Workers:   3,
 						QueueSize: 100000,
@@ -50,7 +50,7 @@ func TestDecodeConfiguration(t *testing.T) {
 					},
 					UseSrcAddrForExporterAddr: true,
 				}, {
-					Decoder: "sflow",
+					Decoder: pb.RawFlow_DECODER_SFLOW,
 					Config: &udp.Configuration{
 						Workers:   3,
 						QueueSize: 100000,
@@ -64,10 +64,10 @@ func TestDecodeConfiguration(t *testing.T) {
 			Initial: func() interface{} {
 				return Configuration{
 					Inputs: []InputConfiguration{{
-						Decoder: "netflow",
+						Decoder: pb.RawFlow_DECODER_NETFLOW,
 						Config:  udp.DefaultConfiguration(),
 					}, {
-						Decoder: "sflow",
+						Decoder: pb.RawFlow_DECODER_SFLOW,
 						Config:  udp.DefaultConfiguration(),
 					}},
 				}
@@ -91,14 +91,14 @@ func TestDecodeConfiguration(t *testing.T) {
 			},
 			Expected: Configuration{
 				Inputs: []InputConfiguration{{
-					Decoder: "netflow",
+					Decoder: pb.RawFlow_DECODER_NETFLOW,
 					Config: &udp.Configuration{
 						Workers:   3,
 						QueueSize: 100000,
 						Listen:    "192.0.2.1:2055",
 					},
 				}, {
-					Decoder: "sflow",
+					Decoder: pb.RawFlow_DECODER_SFLOW,
 					Config: &udp.Configuration{
 						Workers:   3,
 						QueueSize: 100000,
@@ -111,7 +111,7 @@ func TestDecodeConfiguration(t *testing.T) {
 			Initial: func() interface{} {
 				return Configuration{
 					Inputs: []InputConfiguration{{
-						Decoder: "netflow",
+						Decoder: pb.RawFlow_DECODER_NETFLOW,
 						Config:  udp.DefaultConfiguration(),
 					}},
 				}
@@ -128,7 +128,7 @@ func TestDecodeConfiguration(t *testing.T) {
 			},
 			Expected: Configuration{
 				Inputs: []InputConfiguration{{
-					Decoder: "netflow",
+					Decoder: pb.RawFlow_DECODER_NETFLOW,
 					Config: &file.Configuration{
 						Paths: []string{"file1", "file2"},
 					},
@@ -139,8 +139,8 @@ func TestDecodeConfiguration(t *testing.T) {
 			Initial: func() interface{} {
 				return Configuration{
 					Inputs: []InputConfiguration{{
-						Decoder:         "netflow",
-						TimestampSource: decoder.TimestampSourceUDP,
+						Decoder:         pb.RawFlow_DECODER_NETFLOW,
+						TimestampSource: pb.RawFlow_TS_INPUT,
 						Config: &udp.Configuration{
 							Workers:   2,
 							QueueSize: 100,
@@ -160,7 +160,7 @@ func TestDecodeConfiguration(t *testing.T) {
 			},
 			Expected: Configuration{
 				Inputs: []InputConfiguration{{
-					Decoder: "netflow",
+					Decoder: pb.RawFlow_DECODER_NETFLOW,
 					Config: &udp.Configuration{
 						Workers:   2,
 						QueueSize: 100,
@@ -197,7 +197,7 @@ func TestDecodeConfiguration(t *testing.T) {
 			Initial: func() interface{} {
 				return Configuration{
 					Inputs: []InputConfiguration{{
-						Decoder: "netflow",
+						Decoder: pb.RawFlow_DECODER_NETFLOW,
 						Config: &udp.Configuration{
 							Workers:   2,
 							QueueSize: 100,
@@ -218,8 +218,8 @@ func TestDecodeConfiguration(t *testing.T) {
 			},
 			Expected: Configuration{
 				Inputs: []InputConfiguration{{
-					Decoder:         "netflow",
-					TimestampSource: decoder.TimestampSourceNetflowPacket,
+					Decoder:         pb.RawFlow_DECODER_NETFLOW,
+					TimestampSource: pb.RawFlow_TS_NETFLOW_PACKET,
 					Config: &udp.Configuration{
 						Workers:   2,
 						QueueSize: 100,
@@ -233,7 +233,7 @@ func TestDecodeConfiguration(t *testing.T) {
 			Initial: func() interface{} {
 				return Configuration{
 					Inputs: []InputConfiguration{{
-						Decoder: "netflow",
+						Decoder: pb.RawFlow_DECODER_NETFLOW,
 						Config: &udp.Configuration{
 							Workers:   2,
 							QueueSize: 100,
@@ -254,8 +254,8 @@ func TestDecodeConfiguration(t *testing.T) {
 			},
 			Expected: Configuration{
 				Inputs: []InputConfiguration{{
-					Decoder:         "netflow",
-					TimestampSource: decoder.TimestampSourceNetflowFirstSwitched,
+					Decoder:         pb.RawFlow_DECODER_NETFLOW,
+					TimestampSource: pb.RawFlow_TS_NETFLOW_FIRST_SWITCHED,
 					Config: &udp.Configuration{
 						Workers:   2,
 						QueueSize: 100,
@@ -271,15 +271,15 @@ func TestMarshalYAML(t *testing.T) {
 	cfg := Configuration{
 		Inputs: []InputConfiguration{
 			{
-				Decoder:         "netflow",
-				TimestampSource: decoder.TimestampSourceNetflowFirstSwitched,
+				Decoder:         pb.RawFlow_DECODER_NETFLOW,
+				TimestampSource: pb.RawFlow_TS_NETFLOW_FIRST_SWITCHED,
 				Config: &udp.Configuration{
 					Listen:    "192.0.2.11:2055",
 					QueueSize: 1000,
 					Workers:   3,
 				},
 			}, {
-				Decoder: "sflow",
+				Decoder: pb.RawFlow_DECODER_SFLOW,
 				Config: &udp.Configuration{
 					Listen:    "192.0.2.11:6343",
 					QueueSize: 1000,
@@ -306,11 +306,10 @@ func TestMarshalYAML(t *testing.T) {
       listen: 192.0.2.11:6343
       queuesize: 1000
       receivebuffer: 0
-      timestampsource: udp
+      timestampsource: input
       type: udp
       usesrcaddrforexporteraddr: true
       workers: 3
-ratelimit: 0
 `
 	if diff := helpers.Diff(strings.Split(string(got), "\n"), strings.Split(expected, "\n")); diff != "" {
 		t.Fatalf("Marshal() (-got, +want):\n%s", diff)
diff --git a/inlet/flow/decoder/sflow/root_test.go b/inlet/flow/decoder/sflow/root_test.go
deleted file mode 100644
index 04144f5f..00000000
--- a/inlet/flow/decoder/sflow/root_test.go
+++ /dev/null
@@ -1,543 +0,0 @@
-// SPDX-FileCopyrightText: 2022 Tchadel Icard
-// SPDX-License-Identifier: AGPL-3.0-only
-
-package sflow
-
-import (
-	"net"
-	"net/netip"
-	"path/filepath"
-	"testing"
-
-	"akvorado/common/helpers"
-	"akvorado/common/reporter"
-	"akvorado/common/schema"
-	"akvorado/inlet/flow/decoder"
-)
-
-func TestDecode(t *testing.T) {
-	r := reporter.NewMock(t)
-	sdecoder := New(r, decoder.Dependencies{Schema: schema.NewMock(t).EnableAllColumns()}, decoder.Option{})
-
-	// Send data
-	data := helpers.ReadPcapL4(t, filepath.Join("testdata", "data-1140.pcap"))
-	got := sdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
-	if got == nil {
-		t.Fatalf("Decode() error on data")
-	}
-	expectedFlows := []*schema.FlowMessage{
-		{
-			SamplingRate:    1024,
-			InIf:            27,
-			OutIf:           28,
-			SrcVlan:         100,
-			DstVlan:         100,
-			SrcAddr:         netip.MustParseAddr("2a0c:8880:2:0:185:21:130:38"),
-			DstAddr:         netip.MustParseAddr("2a0c:8880:2:0:185:21:130:39"),
-			ExporterAddress: netip.MustParseAddr("::ffff:172.16.0.3"),
-			ProtobufDebug: map[schema.ColumnKey]interface{}{
-				schema.ColumnBytes:         1500,
-				schema.ColumnPackets:       1,
-				schema.ColumnEType:         helpers.ETypeIPv6,
-				schema.ColumnProto:         6,
-				schema.ColumnSrcPort:       46026,
-				schema.ColumnDstPort:       22,
-				schema.ColumnSrcMAC:        40057391053392,
-				schema.ColumnDstMAC:        40057381862408,
-				schema.ColumnIPTTL:         64,
-				schema.ColumnIPTos:         0x8,
-				schema.ColumnIPv6FlowLabel: 0x68094,
-				schema.ColumnTCPFlags:      0x10,
-			},
-		}, {
-			SamplingRate:    1024,
-			SrcAddr:         netip.MustParseAddr("::ffff:104.26.8.24"),
-			DstAddr:         netip.MustParseAddr("::ffff:45.90.161.46"),
-			ExporterAddress: netip.MustParseAddr("::ffff:172.16.0.3"),
-			NextHop:         netip.MustParseAddr("::ffff:45.90.161.46"),
-			InIf:            49001,
-			OutIf:           25,
-			DstVlan:         100,
-			SrcAS:           13335,
-			DstAS:           39421,
-			SrcNetMask:      20,
-			DstNetMask:      27,
-			GotASPath:       false,
-			ProtobufDebug: map[schema.ColumnKey]interface{}{
-				schema.ColumnBytes:        421,
-				schema.ColumnPackets:      1,
-				schema.ColumnEType:        helpers.ETypeIPv4,
-				schema.ColumnProto:        6,
-				schema.ColumnSrcPort:      443,
-				schema.ColumnDstPort:      56876,
-				schema.ColumnSrcMAC:       216372595274807,
-				schema.ColumnDstMAC:       191421060163210,
-				schema.ColumnIPFragmentID: 0xa572,
-				schema.ColumnIPTTL:        59,
-				schema.ColumnTCPFlags:     0x18,
-			},
-		}, {
-			SamplingRate:    1024,
-			SrcAddr:         netip.MustParseAddr("2a0c:8880:2:0:185:21:130:38"),
-			DstAddr:         netip.MustParseAddr("2a0c:8880:2:0:185:21:130:39"),
-			ExporterAddress: netip.MustParseAddr("::ffff:172.16.0.3"),
-			InIf:            27,
-			OutIf:           28,
-			SrcVlan:         100,
-			DstVlan:         100,
-			ProtobufDebug: map[schema.ColumnKey]interface{}{
-				schema.ColumnBytes:         1500,
-				schema.ColumnPackets:       1,
-				schema.ColumnEType:         helpers.ETypeIPv6,
-				schema.ColumnProto:         6,
-				schema.ColumnSrcPort:       46026,
-				schema.ColumnDstPort:       22,
-				schema.ColumnSrcMAC:        40057391053392,
-				schema.ColumnDstMAC:        40057381862408,
-				schema.ColumnIPTTL:         64,
-				schema.ColumnIPTos:         0x8,
-				schema.ColumnIPv6FlowLabel: 0x68094,
-				schema.ColumnTCPFlags:      0x10,
-			},
-		}, {
-			SamplingRate:    1024,
-			InIf:            28,
-			OutIf:           49001,
-			SrcVlan:         100,
-			SrcAS:           39421,
-			DstAS:           26615,
-			SrcAddr:         netip.MustParseAddr("::ffff:45.90.161.148"),
-			DstAddr:         netip.MustParseAddr("::ffff:191.87.91.27"),
-			ExporterAddress: netip.MustParseAddr("::ffff:172.16.0.3"),
-			NextHop:         netip.MustParseAddr("::ffff:31.14.69.110"),
-			SrcNetMask:      27,
-			DstNetMask:      17,
-			GotASPath:       true,
-			GotCommunities:  true,
-			ProtobufDebug: map[schema.ColumnKey]interface{}{
-				schema.ColumnBytes:          40,
-				schema.ColumnPackets:        1,
-				schema.ColumnEType:          helpers.ETypeIPv4,
-				schema.ColumnProto:          6,
-				schema.ColumnSrcPort:        55658,
-				schema.ColumnDstPort:        5555,
-				schema.ColumnSrcMAC:         138617863011056,
-				schema.ColumnDstMAC:         216372595274807,
-				schema.ColumnDstASPath:      []uint32{203698, 6762, 26615},
-				schema.ColumnDstCommunities: []uint64{2583495656, 2583495657, 4259880000, 4259880001, 4259900001},
-				schema.ColumnIPFragmentID:   0xd431,
-				schema.ColumnIPTTL:          255,
-				schema.ColumnTCPFlags:       0x2,
-			},
-		}, {
-			SamplingRate:    1024,
-			SrcAddr:         netip.MustParseAddr("2a0c:8880:2:0:185:21:130:38"),
-			DstAddr:         netip.MustParseAddr("2a0c:8880:2:0:185:21:130:39"),
-			ExporterAddress: netip.MustParseAddr("::ffff:172.16.0.3"),
-			InIf:            27,
-			OutIf:           28,
-			SrcVlan:         100,
-			DstVlan:         100,
-			ProtobufDebug: map[schema.ColumnKey]interface{}{
-				schema.ColumnBytes:         1500,
-				schema.ColumnPackets:       1,
-				schema.ColumnEType:         helpers.ETypeIPv6,
-				schema.ColumnProto:         6,
-				schema.ColumnSrcPort:       46026,
-				schema.ColumnDstPort:       22,
-				schema.ColumnSrcMAC:        40057391053392,
-				schema.ColumnDstMAC:        40057381862408,
-				schema.ColumnIPTTL:         64,
-				schema.ColumnIPTos:         0x8,
-				schema.ColumnIPv6FlowLabel: 0x68094,
-				schema.ColumnTCPFlags:      0x10,
-			},
-		},
-	}
-	for _, f := range got {
-		f.TimeReceived = 0
-	}
-
-	if diff := helpers.Diff(got, expectedFlows); diff != "" {
-		t.Fatalf("Decode() (-got, +want):\n%s", diff)
-	}
-	gotMetrics := r.GetMetrics(
-		"akvorado_inlet_flow_decoder_sflow_",
-		"flows_total",
-		"sample_",
-	)
-	expectedMetrics := map[string]string{
-		`flows_total{agent="172.16.0.3",exporter="127.0.0.1",version="5"}`:                          "1",
-		`sample_records_sum{agent="172.16.0.3",exporter="127.0.0.1",type="FlowSample",version="5"}`: "14",
-		`sample_sum{agent="172.16.0.3",exporter="127.0.0.1",type="FlowSample",version="5"}`:         "5",
-	}
-	if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
-		t.Fatalf("Metrics after data (-got, +want):\n%s", diff)
-	}
-}
-
-func TestDecodeInterface(t *testing.T) {
-	r := reporter.NewMock(t)
-	sdecoder := New(r, decoder.Dependencies{Schema: schema.NewMock(t)}, decoder.Option{})
-
-	t.Run("local interface", func(t *testing.T) {
-		// Send data
-		data := helpers.ReadPcapL4(t, filepath.Join("testdata", "data-local-interface.pcap"))
-		got := sdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
-		if got == nil {
-			t.Fatalf("Decode() error on data")
-		}
-		expectedFlows := []*schema.FlowMessage{
-			{
-				SamplingRate:    1024,
-				SrcAddr:         netip.MustParseAddr("2a0c:8880:2:0:185:21:130:38"),
-				DstAddr:         netip.MustParseAddr("2a0c:8880:2:0:185:21:130:39"),
-				ExporterAddress: netip.MustParseAddr("::ffff:172.16.0.3"),
-				InIf:            27,
-				OutIf:           0, // local interface
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
-					schema.ColumnBytes:   1500,
-					schema.ColumnPackets: 1,
-					schema.ColumnEType:   helpers.ETypeIPv6,
-					schema.ColumnProto:   6,
-					schema.ColumnSrcPort: 46026,
-					schema.ColumnDstPort: 22,
-				},
-			},
-		}
-		for _, f := range got {
-			f.TimeReceived = 0
-		}
-
-		if diff := helpers.Diff(got, expectedFlows); diff != "" {
-			t.Fatalf("Decode() (-got, +want):\n%s", diff)
-		}
-	})
-
-	t.Run("discard interface", func(t *testing.T) {
-		// Send data
-		data := helpers.ReadPcapL4(t, filepath.Join("testdata", "data-discard-interface.pcap"))
-		got := sdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
-		if got == nil {
-			t.Fatalf("Decode() error on data")
-		}
-		expectedFlows := []*schema.FlowMessage{
-			{
-				SamplingRate:    1024,
-				SrcAddr:         netip.MustParseAddr("2a0c:8880:2:0:185:21:130:38"),
-				DstAddr:         netip.MustParseAddr("2a0c:8880:2:0:185:21:130:39"),
-				ExporterAddress: netip.MustParseAddr("::ffff:172.16.0.3"),
-				InIf:            27,
-				OutIf:           0, // discard interface
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
-					schema.ColumnBytes:            1500,
-					schema.ColumnPackets:          1,
-					schema.ColumnEType:            helpers.ETypeIPv6,
-					schema.ColumnProto:            6,
-					schema.ColumnSrcPort:          46026,
-					schema.ColumnDstPort:          22,
-					schema.ColumnForwardingStatus: 128,
-				},
-			},
-		}
-		for _, f := range got {
-			f.TimeReceived = 0
-		}
-
-		if diff := helpers.Diff(got, expectedFlows); diff != "" {
-			t.Fatalf("Decode() (-got, +want):\n%s", diff)
-		}
-	})
-
-	t.Run("multiple interfaces", func(t *testing.T) {
-		// Send data
-		data := helpers.ReadPcapL4(t, filepath.Join("testdata", "data-multiple-interfaces.pcap"))
-		got := sdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
-		if got == nil {
-			t.Fatalf("Decode() error on data")
-		}
-		expectedFlows := []*schema.FlowMessage{
-			{
-				SamplingRate:    1024,
-				SrcAddr:         netip.MustParseAddr("2a0c:8880:2:0:185:21:130:38"),
-				DstAddr:         netip.MustParseAddr("2a0c:8880:2:0:185:21:130:39"),
-				ExporterAddress: netip.MustParseAddr("::ffff:172.16.0.3"),
-				InIf:            27,
-				OutIf:           0, // multiple interfaces
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
-					schema.ColumnBytes:   1500,
-					schema.ColumnPackets: 1,
-					schema.ColumnEType:   helpers.ETypeIPv6,
-					schema.ColumnProto:   6,
-					schema.ColumnSrcPort: 46026,
-					schema.ColumnDstPort: 22,
-				},
-			},
-		}
-		for _, f := range got {
-			f.TimeReceived = 0
-		}
-
-		if diff := helpers.Diff(got, expectedFlows); diff != "" {
-			t.Fatalf("Decode() (-got, +want):\n%s", diff)
-		}
-	})
-}
-
-func TestDecodeSamples(t *testing.T) {
-	r := reporter.NewMock(t)
-	sdecoder := New(r, decoder.Dependencies{Schema: schema.NewMock(t).EnableAllColumns()}, decoder.Option{})
-
-	t.Run("expanded flow sample", func(t *testing.T) {
-		// Send data
-		data := helpers.ReadPcapL4(t, filepath.Join("testdata", "data-sflow-expanded-sample.pcap"))
-		got := sdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
-		if got == nil {
-			t.Fatalf("Decode() error on data")
-		}
-		expectedFlows := []*schema.FlowMessage{
-			{
-				SamplingRate:    1000,
-				InIf:            29001,
-				OutIf:           1285816721,
-				SrcAddr:         netip.MustParseAddr("::ffff:52.52.52.52"),
-				DstAddr:         netip.MustParseAddr("::ffff:53.53.53.53"),
-				ExporterAddress: netip.MustParseAddr("::ffff:49.49.49.49"),
-				NextHop:         netip.MustParseAddr("::ffff:54.54.54.54"),
-				SrcAS:           203476,
-				DstAS:           203361,
-				SrcVlan:         809,
-				GotASPath:       true,
-				GotCommunities:  true,
-				SrcNetMask:      32,
-				DstNetMask:      22,
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
-					schema.ColumnBytes:          104,
-					schema.ColumnPackets:        1,
-					schema.ColumnEType:          helpers.ETypeIPv4,
-					schema.ColumnProto:          6,
-					schema.ColumnSrcPort:        22,
-					schema.ColumnDstPort:        52237,
-					schema.ColumnDstASPath:      []uint32{8218, 29605, 203361},
-					schema.ColumnDstCommunities: []uint64{538574949, 1911619684, 1911669584, 1911671290},
-					schema.ColumnTCPFlags:       0x18,
-					schema.ColumnIPFragmentID:   0xab4e,
-					schema.ColumnIPTTL:          61,
-					schema.ColumnIPTos:          0x8,
-					schema.ColumnSrcMAC:         0x948ed30a713b,
-					schema.ColumnDstMAC:         0x22421f4a9fcd,
-				},
-			},
-		}
-		for _, f := range got {
-			f.TimeReceived = 0
-		}
-
-		if diff := helpers.Diff(got, expectedFlows); diff != "" {
-			t.Fatalf("Decode() (-got, +want):\n%s", diff)
-		}
-	})
-
-	t.Run("flow sample with IPv4 data", func(t *testing.T) {
-		// Send data
-		data := helpers.ReadPcapL4(t, filepath.Join("testdata", "data-sflow-ipv4-data.pcap"))
-		got := sdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
-		if got == nil {
-			t.Fatalf("Decode() error on data")
-		}
-		expectedFlows := []*schema.FlowMessage{
-			{
-				SamplingRate:    256,
-				InIf:            0,
-				OutIf:           182,
-				DstVlan:         3001,
-				SrcAddr:         netip.MustParseAddr("::ffff:50.50.50.50"),
-				DstAddr:         netip.MustParseAddr("::ffff:51.51.51.51"),
-				ExporterAddress: netip.MustParseAddr("::ffff:49.49.49.49"),
-				GotASPath:       false,
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
-					schema.ColumnBytes:        1344,
-					schema.ColumnPackets:      1,
-					schema.ColumnEType:        helpers.ETypeIPv4,
-					schema.ColumnProto:        17,
-					schema.ColumnSrcPort:      46622,
-					schema.ColumnDstPort:      58631,
-					schema.ColumnSrcMAC:       1094287164743,
-					schema.ColumnDstMAC:       1101091482116,
-					schema.ColumnIPFragmentID: 41647,
-					schema.ColumnIPTTL:        64,
-				},
-			},
-		}
-		for _, f := range got {
-			f.TimeReceived = 0
-		}
-
-		if diff := helpers.Diff(got, expectedFlows); diff != "" {
-			t.Fatalf("Decode() (-got, +want):\n%s", diff)
-		}
-	})
-
-	t.Run("flow sample with IPv4 raw packet", func(t *testing.T) {
-		data := helpers.ReadPcapL4(t, filepath.Join("testdata", "data-sflow-raw-ipv4.pcap"))
-		got := sdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
-		if got == nil {
-			t.Fatalf("Decode() error on data")
-		}
-		expectedFlows := []*schema.FlowMessage{
-			{
-				SamplingRate:    1,
-				InIf:            0,
-				OutIf:           2,
-				SrcAddr:         netip.MustParseAddr("::ffff:69.58.92.107"),
-				DstAddr:         netip.MustParseAddr("::ffff:92.222.186.1"),
-				ExporterAddress: netip.MustParseAddr("::ffff:172.19.64.116"),
-				GotASPath:       false,
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
-					schema.ColumnBytes:        32,
-					schema.ColumnPackets:      1,
-					schema.ColumnEType:        helpers.ETypeIPv4,
-					schema.ColumnProto:        1,
-					schema.ColumnIPFragmentID: 4329,
-					schema.ColumnIPTTL:        64,
-					schema.ColumnIPTos:        8,
-				},
-			}, {
-				SamplingRate:    1,
-				InIf:            0,
-				OutIf:           2,
-				SrcAddr:         netip.MustParseAddr("::ffff:69.58.92.107"),
-				DstAddr:         netip.MustParseAddr("::ffff:92.222.184.1"),
-				ExporterAddress: netip.MustParseAddr("::ffff:172.19.64.116"),
-				GotASPath:       false,
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
-					schema.ColumnBytes:        32,
-					schema.ColumnPackets:      1,
-					schema.ColumnEType:        helpers.ETypeIPv4,
-					schema.ColumnProto:        1,
-					schema.ColumnIPFragmentID: 62945,
-					schema.ColumnIPTTL:        64,
-					schema.ColumnIPTos:        8,
-				},
-			},
-		}
-		for _, f := range got {
-			f.TimeReceived = 0
-		}
-
-		if diff := helpers.Diff(got, expectedFlows); diff != "" {
-			t.Fatalf("Decode() (-got, +want):\n%s", diff)
-		}
-	})
-
-	t.Run("flow sample with ICMPv4", func(t *testing.T) {
-		data := helpers.ReadPcapL4(t, filepath.Join("testdata", "data-icmpv4.pcap"))
-		got := sdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
-		if got == nil {
-			t.Fatalf("Decode() error on data")
-		}
-		expectedFlows := []*schema.FlowMessage{
-			{
-				SamplingRate:    1,
-				SrcAddr:         netip.MustParseAddr("::ffff:203.0.113.4"),
-				DstAddr:         netip.MustParseAddr("::ffff:203.0.113.5"),
-				ExporterAddress: netip.MustParseAddr("::ffff:127.0.0.1"),
-				GotASPath:       false,
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
-					schema.ColumnBytes:      84,
-					schema.ColumnPackets:    1,
-					schema.ColumnEType:      helpers.ETypeIPv4,
-					schema.ColumnProto:      1,
-					schema.ColumnDstMAC:     0xd25b45ee5ecf,
-					schema.ColumnSrcMAC:     0xe2efc68f8cd4,
-					schema.ColumnICMPv4Type: 8,
-					// schema.ColumnICMPv4Code:   0,
-					schema.ColumnIPTTL:        64,
-					schema.ColumnIPFragmentID: 0x90c5,
-				},
-			},
-		}
-		for _, f := range got {
-			f.TimeReceived = 0
-		}
-
-		if diff := helpers.Diff(got, expectedFlows); diff != "" {
-			t.Fatalf("Decode() (-got, +want):\n%s", diff)
-		}
-	})
-
-	t.Run("flow sample with ICMPv6", func(t *testing.T) {
-		data := helpers.ReadPcapL4(t, filepath.Join("testdata", "data-icmpv6.pcap"))
-		got := sdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
-		if got == nil {
-			t.Fatalf("Decode() error on data")
-		}
-		expectedFlows := []*schema.FlowMessage{
-			{
-				SamplingRate:    1,
-				SrcAddr:         netip.MustParseAddr("fe80::d05b:45ff:feee:5ecf"),
-				DstAddr:         netip.MustParseAddr("2001:db8::"),
-				ExporterAddress: netip.MustParseAddr("::ffff:127.0.0.1"),
-				GotASPath:       false,
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
-					schema.ColumnBytes:      72,
-					schema.ColumnPackets:    1,
-					schema.ColumnEType:      helpers.ETypeIPv6,
-					schema.ColumnProto:      58,
-					schema.ColumnSrcMAC:     0xd25b45ee5ecf,
-					schema.ColumnDstMAC:     0xe2efc68f8cd4,
-					schema.ColumnIPTTL:      255,
-					schema.ColumnICMPv6Type: 135,
-					// schema.ColumnICMPv6Code:   0,
-				},
-			},
-		}
-		for _, f := range got {
-			f.TimeReceived = 0
-		}
-
-		if diff := helpers.Diff(got, expectedFlows); diff != "" {
-			t.Fatalf("Decode() (-got, +want):\n%s", diff)
-		}
-	})
-
-	t.Run("flow sample with QinQ", func(t *testing.T) {
-		data := helpers.ReadPcapL4(t, filepath.Join("testdata", "data-qinq.pcap"))
-		got := sdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
-		if got == nil {
-			t.Fatalf("Decode() error on data")
-		}
-		expectedFlows := []*schema.FlowMessage{
-			{
-				SamplingRate:    4096,
-				InIf:            369098852,
-				OutIf:           369098851,
-				SrcVlan:         1493,
-				SrcAddr:         netip.MustParseAddr("::ffff:49.49.49.2"),
-				DstAddr:         netip.MustParseAddr("::ffff:49.49.49.109"),
-				ExporterAddress: netip.MustParseAddr("::ffff:172.17.128.58"),
-				GotASPath:       false,
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
-					schema.ColumnBytes:        80,
-					schema.ColumnPackets:      1,
-					schema.ColumnEType:        helpers.ETypeIPv4,
-					schema.ColumnProto:        6,
-					schema.ColumnSrcMAC:       0x4caea3520ff6,
-					schema.ColumnDstMAC:       0x000110621493,
-					schema.ColumnIPTTL:        62,
-					schema.ColumnIPFragmentID: 56159,
-					schema.ColumnTCPFlags:     16,
-					schema.ColumnSrcPort:      32017,
-					schema.ColumnDstPort:      443,
-				},
-			},
-		}
-		for _, f := range got {
-			f.TimeReceived = 0
-		}
-
-		if diff := helpers.Diff(got, expectedFlows); diff != "" {
-			t.Fatalf("Decode() (-got, +want):\n%s", diff)
-		}
-	})
-}
diff --git a/inlet/flow/decoder/tests.go b/inlet/flow/decoder/tests.go
deleted file mode 100644
index 488b887e..00000000
--- a/inlet/flow/decoder/tests.go
+++ /dev/null
@@ -1,36 +0,0 @@
-// SPDX-FileCopyrightText: 2022 Free Mobile
-// SPDX-License-Identifier: AGPL-3.0-only
-
-//go:build !release
-
-package decoder
-
-import (
-	"net/netip"
-
-	"akvorado/common/schema"
-)
-
-// DummyDecoder is a simple decoder producing flows from random data.
-// The payload is copied in IfDescription
-type DummyDecoder struct {
-	Schema *schema.Component
-}
-
-// Decode returns uninteresting flow messages.
-func (dc *DummyDecoder) Decode(in RawFlow) []*schema.FlowMessage {
-	exporterAddress, _ := netip.AddrFromSlice(in.Source.To16())
-	f := &schema.FlowMessage{
-		TimeReceived:    uint64(in.TimeReceived.UTC().Unix()),
-		ExporterAddress: exporterAddress,
-	}
-	dc.Schema.ProtobufAppendVarint(f, schema.ColumnBytes, uint64(len(in.Payload)))
-	dc.Schema.ProtobufAppendVarint(f, schema.ColumnPackets, 1)
-	dc.Schema.ProtobufAppendBytes(f, schema.ColumnInIfDescription, in.Payload)
-	return []*schema.FlowMessage{f}
-}
-
-// Name returns the original name.
-func (dc *DummyDecoder) Name() string {
-	return "dummy"
-}
diff --git a/inlet/flow/decoder_test.go b/inlet/flow/decoder_test.go
deleted file mode 100644
index 3e06444c..00000000
--- a/inlet/flow/decoder_test.go
+++ /dev/null
@@ -1,92 +0,0 @@
-// SPDX-FileCopyrightText: 2023 Free Mobile
-// SPDX-License-Identifier: AGPL-3.0-only
-
-package flow
-
-import (
-	"net"
-	"path/filepath"
-	"testing"
-
-	"akvorado/common/helpers"
-	"akvorado/common/reporter"
-	"akvorado/common/schema"
-	"akvorado/inlet/flow/decoder"
-	"akvorado/inlet/flow/decoder/netflow"
-	"akvorado/inlet/flow/decoder/sflow"
-)
-
-// The goal is to benchmark flow decoding + encoding to protobuf
-
-func BenchmarkDecodeEncodeNetflow(b *testing.B) {
-	schema.DisableDebug(b)
-	r := reporter.NewMock(b)
-	sch := schema.NewMock(b)
-	nfdecoder := netflow.New(r, decoder.Dependencies{Schema: sch}, decoder.Option{TimestampSource: decoder.TimestampSourceUDP})
-
-	template := helpers.ReadPcapL4(b, filepath.Join("decoder", "netflow", "testdata", "options-template.pcap"))
-	got := nfdecoder.Decode(decoder.RawFlow{Payload: template, Source: net.ParseIP("127.0.0.1")})
-	if got == nil || len(got) != 0 {
-		b.Fatal("Decode() error on options template")
-	}
-	data := helpers.ReadPcapL4(b, filepath.Join("decoder", "netflow", "testdata", "options-data.pcap"))
-	got = nfdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
-	if got == nil || len(got) != 0 {
-		b.Fatal("Decode() error on options data")
-	}
-	template = helpers.ReadPcapL4(b, filepath.Join("decoder", "netflow", "testdata", "template.pcap"))
-	got = nfdecoder.Decode(decoder.RawFlow{Payload: template, Source: net.ParseIP("127.0.0.1")})
-	if got == nil || len(got) != 0 {
-		b.Fatal("Decode() error on template")
-	}
-	data = helpers.ReadPcapL4(b, filepath.Join("decoder", "netflow", "testdata", "data.pcap"))
-
-	for _, withEncoding := range []bool{true, false} {
-		title := map[bool]string{
-			true:  "with encoding",
-			false: "without encoding",
-		}[withEncoding]
-		b.Run(title, func(b *testing.B) {
-			for b.Loop() {
-				got = nfdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
-				if withEncoding {
-					for _, flow := range got {
-						sch.ProtobufMarshal(flow)
-					}
-				}
-			}
-			if got[0].ProtobufDebug != nil {
-				b.Fatal("debug is enabled")
-			}
-		})
-	}
-}
-
-func BenchmarkDecodeEncodeSflow(b *testing.B) {
-	schema.DisableDebug(b)
-	r := reporter.NewMock(b)
-	sch := schema.NewMock(b)
-	sdecoder := sflow.New(r, decoder.Dependencies{Schema: sch}, decoder.Option{TimestampSource: decoder.TimestampSourceUDP})
-	data := helpers.ReadPcapL4(b, filepath.Join("decoder", "sflow", "testdata", "data-1140.pcap"))
-
-	for _, withEncoding := range []bool{true, false} {
-		title := map[bool]string{
-			true:  "with encoding",
-			false: "without encoding",
-		}[withEncoding]
-		var got []*schema.FlowMessage
-		b.Run(title, func(b *testing.B) {
-			for b.Loop() {
-				got = sdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
-				if withEncoding {
-					for _, flow := range got {
-						sch.ProtobufMarshal(flow)
-					}
-				}
-			}
-			if got[0].ProtobufDebug != nil {
-				b.Fatal("debug is enabled")
-			}
-		})
-	}
-}
diff --git a/inlet/flow/input/file/root_test.go b/inlet/flow/input/file/root_test.go
index 2b112404..3d2033b2 100644
--- a/inlet/flow/input/file/root_test.go
+++ b/inlet/flow/input/file/root_test.go
@@ -4,29 +4,62 @@
 package file
 
 import (
+	"net"
 	"path"
+	"sync"
 	"testing"
 	"time"
 
 	"akvorado/common/daemon"
 	"akvorado/common/helpers"
+	"akvorado/common/pb"
 	"akvorado/common/reporter"
-	"akvorado/common/schema"
-	"akvorado/inlet/flow/decoder"
 )
 
 func TestFileInput(t *testing.T) {
 	r := reporter.NewMock(t)
 	configuration := DefaultConfiguration().(*Configuration)
 	configuration.Paths = []string{path.Join("testdata", "file1.txt"), path.Join("testdata", "file2.txt")}
-	in, err := configuration.New(r, daemon.NewMock(t), &decoder.DummyDecoder{
-		Schema: schema.NewMock(t),
-	})
+
+	done := make(chan bool)
+	expected := []pb.RawFlow{
+		{
+			Payload:       []byte("hello world!\n"),
+			SourceAddress: net.ParseIP("127.0.0.1").To16(),
+		}, {
+			Payload:       []byte("bye bye\n"),
+			SourceAddress: net.ParseIP("127.0.0.1").To16(),
+		}, {
+			Payload:       []byte("hello world!\n"),
+			SourceAddress: net.ParseIP("127.0.0.1").To16(),
+		},
+	}
+	var mu sync.Mutex
+	got := []*pb.RawFlow{}
+	send := func(_ string, flow *pb.RawFlow) {
+		// Make a copy
+		payload := make([]byte, len(flow.Payload))
+		copy(payload, flow.Payload)
+		newFlow := pb.RawFlow{
+			TimeReceived:  0,
+			Payload:       payload,
+			SourceAddress: flow.SourceAddress,
+		}
+		mu.Lock()
+		if len(got) < len(expected) {
+			got = append(got, &newFlow)
+			if len(got) == len(expected) {
+				close(done)
+			}
+		}
+		mu.Unlock()
+	}
+
+	in, err := configuration.New(r, daemon.NewMock(t), send)
 	if err != nil {
 		t.Fatalf("New() error:\n%+v", err)
 	}
-	ch, err := in.Start()
-	if err != nil {
+	if err := in.Start(); err != nil {
 		t.Fatalf("Start() error:\n%+v", err)
 	}
 	defer func() {
@@ -35,21 +68,12 @@ func TestFileInput(t *testing.T) {
 		}
 	}()
 
-	// Get it back
-	expected := []string{"hello world!\n", "bye bye\n", "hello world!\n"}
-	got := []string{}
-out:
-	for range len(expected) {
-		select {
-		case got1 := <-ch:
-			for _, fl := range got1 {
-				got = append(got, string(fl.ProtobufDebug[schema.ColumnInIfDescription].([]byte)))
-			}
-		case <-time.After(50 * time.Millisecond):
-			break out
+	select {
+	case <-time.After(50 * time.Millisecond):
+		t.Fatal("timeout while waiting to receive flows")
+	case <-done:
+		if diff := helpers.Diff(got, expected); diff != "" {
+			t.Fatalf("Input data (-got, +want):\n%s", diff)
 		}
 	}
-	if diff := helpers.Diff(got, expected); diff != "" {
-		t.Fatalf("Input data (-got, +want):\n%s", diff)
-	}
 }
diff --git a/inlet/flow/input/udp/root_test.go b/inlet/flow/input/udp/root_test.go
index 4c6f3ad1..3833e896 100644
--- a/inlet/flow/input/udp/root_test.go
+++ b/inlet/flow/input/udp/root_test.go
@@ -5,107 +5,61 @@ package udp
 
 import (
 	"net"
-	"net/netip"
 	"testing"
 	"time"
 
 	"akvorado/common/daemon"
 	"akvorado/common/helpers"
+	"akvorado/common/pb"
 	"akvorado/common/reporter"
-	"akvorado/common/schema"
-	"akvorado/inlet/flow/decoder"
 )
 
 func TestUDPInput(t *testing.T) {
 	r := reporter.NewMock(t)
 	configuration := DefaultConfiguration().(*Configuration)
 	configuration.Listen = "127.0.0.1:0"
-	in, err := configuration.New(r, daemon.NewMock(t), &decoder.DummyDecoder{Schema: schema.NewMock(t)})
-	if err != nil {
-		t.Fatalf("New() error:\n%+v", err)
-	}
-	ch, err := in.Start()
-	if err != nil {
-		t.Fatalf("Start() error:\n%+v", err)
-	}
-	defer func() {
-		if err := in.Stop(); err != nil {
-			t.Fatalf("Stop() error:\n%+v", err)
-		}
-	}()
 
-	// Connect
-	conn, err := net.Dial("udp", in.(*Input).address.String())
-	if err != nil {
-		t.Fatalf("Dial() error:\n%+v", err)
+	done := make(chan bool)
+	expected := &pb.RawFlow{
+		SourceAddress: net.ParseIP("127.0.0.1").To16(),
+		Payload:       []byte("hello world!"),
 	}
+	send := func(_ string, got *pb.RawFlow) {
+		expected.TimeReceived = got.TimeReceived
 
-	// Send data
-	if _, err := conn.Write([]byte("hello world!")); err != nil {
-		t.Fatalf("Write() error:\n%+v", err)
-	}
+		delta := uint64(time.Now().UTC().Unix()) - got.TimeReceived
+		if delta > 1 {
+			t.Errorf("TimeReceived out of range: %d (now: %d)", got.TimeReceived, time.Now().UTC().Unix())
+		}
+		if diff := helpers.Diff(got, expected); diff != "" {
+			t.Fatalf("Input data (-got, +want):\n%s", diff)
+		}
 
-	// Get it back
-	var got []*schema.FlowMessage
-	select {
-	case got = <-ch:
-		if len(got) == 0 {
-			t.Fatalf("empty decoded flows received")
+		// Check metrics
+		gotMetrics := r.GetMetrics("akvorado_inlet_flow_input_udp_")
+		expectedMetrics := map[string]string{
+			`bytes_total{exporter="127.0.0.1",listener="127.0.0.1:0",worker="0"}`:                "12",
+			`packets_total{exporter="127.0.0.1",listener="127.0.0.1:0",worker="0"}`:              "1",
+			`in_dropped_packets_total{listener="127.0.0.1:0",worker="0"}`:                        "0",
+			`size_bytes_count{exporter="127.0.0.1",listener="127.0.0.1:0",worker="0"}`:           "1",
+			`size_bytes_sum{exporter="127.0.0.1",listener="127.0.0.1:0",worker="0"}`:             "12",
+			`size_bytes{exporter="127.0.0.1",listener="127.0.0.1:0",worker="0",quantile="0.5"}`:  "12",
+			`size_bytes{exporter="127.0.0.1",listener="127.0.0.1:0",worker="0",quantile="0.9"}`:  "12",
+			`size_bytes{exporter="127.0.0.1",listener="127.0.0.1:0",worker="0",quantile="0.99"}`: "12",
+		}
+		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
+			t.Fatalf("Input metrics (-got, +want):\n%s", diff)
 		}
-	case <-time.After(20 * time.Millisecond):
-		t.Fatal("no decoded flows received")
-	}
 
-	delta := uint64(time.Now().UTC().Unix()) - got[0].TimeReceived
-	if delta > 1 {
-		t.Errorf("TimeReceived out of range: %d (now: %d)", got[0].TimeReceived, time.Now().UTC().Unix())
-	}
-	expected := []*schema.FlowMessage{
-		{
-			TimeReceived:    got[0].TimeReceived,
-			ExporterAddress: netip.MustParseAddr("::ffff:127.0.0.1"),
-			ProtobufDebug: map[schema.ColumnKey]interface{}{
-				schema.ColumnBytes:           12,
-				schema.ColumnPackets:         1,
-				schema.ColumnInIfDescription: []byte("hello world!"),
-			},
-		},
-	}
-	if diff := helpers.Diff(got, expected); diff != "" {
-		t.Fatalf("Input data (-got, +want):\n%s", diff)
-	}
+		close(done)
 
-	// Check metrics
-	gotMetrics := r.GetMetrics("akvorado_inlet_flow_input_udp_")
-	expectedMetrics := map[string]string{
-		`bytes_total{exporter="127.0.0.1",listener="127.0.0.1:0",worker="0"}`:                        "12",
-		`decoded_flows_total{exporter="127.0.0.1",listener="127.0.0.1:0",worker="0"}`:                "1",
-		`packets_total{exporter="127.0.0.1",listener="127.0.0.1:0",worker="0"}`:                      "1",
-		`in_dropped_packets_total{listener="127.0.0.1:0",worker="0"}`:                                "0",
-		`summary_size_bytes_count{exporter="127.0.0.1",listener="127.0.0.1:0",worker="0"}`:           "1",
-		`summary_size_bytes_sum{exporter="127.0.0.1",listener="127.0.0.1:0",worker="0"}`:             "12",
-		`summary_size_bytes{exporter="127.0.0.1",listener="127.0.0.1:0",worker="0",quantile="0.5"}`:  "12",
-		`summary_size_bytes{exporter="127.0.0.1",listener="127.0.0.1:0",worker="0",quantile="0.9"}`:  "12",
-		`summary_size_bytes{exporter="127.0.0.1",listener="127.0.0.1:0",worker="0",quantile="0.99"}`: "12",
-	}
-	if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
-		t.Fatalf("Input metrics (-got, +want):\n%s", diff)
 	}
-}
 
-func TestOverflow(t *testing.T) {
-	r := reporter.NewMock(t)
-	configuration := DefaultConfiguration().(*Configuration)
-	configuration.Listen = "127.0.0.1:0"
-	configuration.QueueSize = 1
-	in, err := configuration.New(r, daemon.NewMock(t), &decoder.DummyDecoder{
-		Schema: schema.NewMock(t),
-	})
+	in, err := configuration.New(r, daemon.NewMock(t), send)
 	if err != nil {
 		t.Fatalf("New() error:\n%+v", err)
 	}
-	_, err = in.Start()
-	if err != nil {
+	if err := in.Start(); err != nil {
 		t.Fatalf("Start() error:\n%+v", err)
 	}
 	defer func() {
@@ -121,28 +75,14 @@ func TestOverflow(t *testing.T) {
 	}
 
 	// Send data
-	for range 10 {
-		if _, err := conn.Write([]byte("hello world!")); err != nil {
-			t.Fatalf("Write() error:\n%+v", err)
-		}
+	if _, err := conn.Write([]byte("hello world!")); err != nil {
+		t.Fatalf("Write() error:\n%+v", err)
 	}
-	time.Sleep(20 * time.Millisecond)
 
-	// Check metrics
-	gotMetrics := r.GetMetrics("akvorado_inlet_flow_input_udp_")
-	expectedMetrics := map[string]string{
-		`bytes_total{exporter="127.0.0.1",listener="127.0.0.1:0",worker="0"}`:                        "120",
-		`decoded_flows_total{exporter="127.0.0.1",listener="127.0.0.1:0",worker="0"}`:                "1",
-		`in_dropped_packets_total{listener="127.0.0.1:0",worker="0"}`:                                "0",
-		`out_dropped_packets_total{exporter="127.0.0.1",listener="127.0.0.1:0",worker="0"}`:          "9",
-		`packets_total{exporter="127.0.0.1",listener="127.0.0.1:0",worker="0"}`:                      "10",
-		`summary_size_bytes_count{exporter="127.0.0.1",listener="127.0.0.1:0",worker="0"}`:           "10",
-		`summary_size_bytes_sum{exporter="127.0.0.1",listener="127.0.0.1:0",worker="0"}`:             "120",
-		`summary_size_bytes{exporter="127.0.0.1",listener="127.0.0.1:0",worker="0",quantile="0.5"}`:  "12",
-		`summary_size_bytes{exporter="127.0.0.1",listener="127.0.0.1:0",worker="0",quantile="0.9"}`:  "12",
-		`summary_size_bytes{exporter="127.0.0.1",listener="127.0.0.1:0",worker="0",quantile="0.99"}`: "12",
-	}
-	if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
-		t.Fatalf("Input metrics (-got, +want):\n%s", diff)
+	// Get it back
+	select {
+	case <-time.After(20 * time.Millisecond):
+		t.Fatal("no decoded flows received")
+	case <-done:
 	}
 }
diff --git a/inlet/flow/root_test.go b/inlet/flow/root_test.go
index 8e30e728..9ec6afca 100644
--- a/inlet/flow/root_test.go
+++ b/inlet/flow/root_test.go
@@ -4,122 +4,104 @@
 package flow
 
 import (
+	"bytes"
 	"fmt"
-	"os"
 	"path"
 	"runtime"
+	"sync"
 	"testing"
 	"time"
 
+	"akvorado/common/daemon"
 	"akvorado/common/helpers"
+	"akvorado/common/httpserver"
+	kafkaCommon "akvorado/common/kafka"
+	"akvorado/common/pb"
 	"akvorado/common/reporter"
 	"akvorado/inlet/flow/input/file"
+	"akvorado/inlet/kafka"
+
+	"github.com/twmb/franz-go/pkg/kgo"
 )
 
 func TestFlow(t *testing.T) {
-	var nominalRate int
 	_, src, _, _ := runtime.Caller(0)
-	base := path.Join(path.Dir(src), "decoder", "netflow", "testdata")
-	outDir := t.TempDir()
-	outFiles := []string{}
-	for idx, f := range []string{
-		"options-template.pcap",
-		"options-data.pcap",
-		"template.pcap",
-		"data.pcap", "data.pcap", "data.pcap", "data.pcap",
-		"data.pcap", "data.pcap", "data.pcap", "data.pcap",
-		"data.pcap", "data.pcap", "data.pcap", "data.pcap",
-		"data.pcap", "data.pcap", "data.pcap", "data.pcap",
-	} {
-		outFile := path.Join(outDir, fmt.Sprintf("data-%d", idx))
-		err := os.WriteFile(outFile, helpers.ReadPcapL4(t, path.Join(base, f)), 0o666)
-		if err != nil {
-			t.Fatalf("WriteFile(%q) error:\n%+v", outFile, err)
-		}
-		outFiles = append(outFiles, outFile)
+	base := path.Join(path.Dir(src), "input", "file", "testdata")
+	paths := []string{
+		path.Join(base, "file1.txt"),
+		path.Join(base, "file2.txt"),
 	}
 
 	inputs := []InputConfiguration{
 		{
-			Decoder: "netflow",
 			Config: &file.Configuration{
-				Paths: outFiles,
+				Paths:    paths,
+				MaxFlows: 100,
 			},
 		},
 	}
 
-	for retry := 2; retry >= 0; retry-- {
-		// Without rate limiting
-		{
-			r := reporter.NewMock(t)
-			config := DefaultConfiguration()
-			config.Inputs = inputs
-			c := NewMock(t, r, config)
+	r := reporter.NewMock(t)
+	config := DefaultConfiguration()
+	config.Inputs = inputs
+
+	producer, cluster := kafka.NewMock(t, r, kafka.DefaultConfiguration())
+	defer cluster.Close()
+
+	// Use the new helper to intercept messages
+	var mu sync.Mutex
+	helloCount := 0
+	byeCount := 0
+	totalCount := 0
+	done := make(chan bool)
 
-			// Receive flows
-			now := time.Now()
-			for range 1000 {
-				select {
-				case <-c.Flows():
-				case <-time.After(100 * time.Millisecond):
-					t.Fatalf("no flow received")
-				}
-			}
-			elapsed := time.Now().Sub(now)
-			t.Logf("Elapsed time for 1000 messages is %s", elapsed)
-			nominalRate = int(1000 * (time.Second / elapsed))
+	kafkaCommon.InterceptMessages(t, cluster, func(record *kgo.Record) {
+		mu.Lock()
+		defer mu.Unlock()
+
+		// Check topic
+		expectedTopic := fmt.Sprintf("flows-v%d", pb.Version)
+		if record.Topic != expectedTopic {
+			t.Errorf("Expected topic %s, got %s", expectedTopic, record.Topic)
+			return
 		}
 
-		// With rate limiting
-		if runtime.GOOS == "Linux" {
-			r := reporter.NewMock(t)
-			config := DefaultConfiguration()
-			config.RateLimit = 1000
-			config.Inputs = inputs
-			c := NewMock(t, r, config)
+		// Count messages based on content
+		if bytes.Contains(record.Value, []byte("hello world!")) {
+			helloCount++
+		} else if bytes.Contains(record.Value, []byte("bye bye")) {
+			byeCount++
+		}
 
-			// Receive flows
-			twoSeconds := time.After(2 * time.Second)
-			count := 0
-		outer1:
-			for {
-				select {
-				case <-c.Flows():
-					count++
-				case <-twoSeconds:
-					break outer1
-				}
-			}
-			t.Logf("During the first two seconds, got %d flows", count)
+		totalCount++
+		if totalCount >= 100 {
+			close(done)
+		}
+	})
 
-			if count > 2200 || count < 2000 {
-				t.Fatalf("Got %d flows instead of 2100 (burst included)", count)
-			}
+	c, err := New(r, config, Dependencies{
+		Daemon: daemon.NewMock(t),
+		HTTP:   httpserver.NewMock(t, r),
+		Kafka:  producer,
+	})
+	if err != nil {
+		t.Fatalf("New() error:\n%+v", err)
+	}
+	helpers.StartStop(t, c)
 
-			if nominalRate == 0 {
-				return
-			}
-			select {
-			case flow := <-c.Flows():
-				// This is hard to estimate the number of
-				// flows we should have got. We use the
-				// nominal rate but it was done with rate
-				// limiting disabled (so less code).
-				// Therefore, we are super conservative on the
-				// upper limit of the sampling rate. However,
-				// the lower limit should be OK.
-				t.Logf("Nominal rate was %d/second", nominalRate)
-				expectedRate := uint64(30000 / 1000 * nominalRate)
-				if flow.SamplingRate > uint32(1000*expectedRate/100) || flow.SamplingRate < uint32(70*expectedRate/100) {
-					if retry > 0 {
-						continue
-					}
-					t.Fatalf("Sampling rate is %d, expected %d", flow.SamplingRate, expectedRate)
-				}
-			case <-time.After(100 * time.Millisecond):
-				t.Fatalf("no flow received")
-			}
-			break
+	// Wait for flows
+	select {
+	case <-done:
+		// Check that we got the expected number of each message type
+		mu.Lock()
+		if helloCount != 50 {
+			t.Errorf("Expected 50 'hello world!' messages, got %d", helloCount)
+		}
+		if byeCount != 50 {
+			t.Errorf("Expected 50 'bye bye' messages, got %d", byeCount)
 		}
+		mu.Unlock()
+	case <-time.After(time.Second):
+		t.Fatalf("flows not received")
 	}
 }
diff --git a/inlet/flow/schemas_test.go b/inlet/flow/schemas_test.go
deleted file mode 100644
index 7c2a0194..00000000
--- a/inlet/flow/schemas_test.go
+++ /dev/null
@@ -1,29 +0,0 @@
-// SPDX-FileCopyrightText: 2022 Free Mobile
-// SPDX-License-Identifier: AGPL-3.0-only
-
-package flow
-
-import (
-	"testing"
-
-	"akvorado/common/helpers"
-	"akvorado/common/reporter"
-)
-
-func TestHTTPEndpoints(t *testing.T) {
-	r := reporter.NewMock(t)
-	c := NewMock(t, r, DefaultConfiguration())
-
-	cases := helpers.HTTPEndpointCases{
-		{
-			URL:         "/api/v0/inlet/flow/schema.proto",
-			ContentType: "text/plain",
-			FirstLines: []string{
-				"",
-				`syntax = "proto3";`,
-			},
-		},
-	}
-
-	helpers.TestHTTPEndpoints(t, c.d.HTTP.LocalAddr(), cases)
-}
diff --git a/inlet/flow/tests.go b/inlet/flow/tests.go
deleted file mode 100644
index 86085b96..00000000
--- a/inlet/flow/tests.go
+++ /dev/null
@@ -1,49 +0,0 @@
-// SPDX-FileCopyrightText: 2022 Free Mobile
-// SPDX-License-Identifier: AGPL-3.0-only
-
-//go:build !release
-
-package flow
-
-import (
-	"testing"
-
-	"akvorado/common/daemon"
-	"akvorado/common/helpers"
-	"akvorado/common/httpserver"
-	"akvorado/common/reporter"
-	"akvorado/common/schema"
-	"akvorado/inlet/flow/input/udp"
-)
-
-// NewMock creates a new flow importer listening on a random port. It
-// is autostarted.
-func NewMock(t *testing.T, r *reporter.Reporter, config Configuration) *Component {
-	t.Helper()
-	if config.Inputs == nil {
-		config.Inputs = []InputConfiguration{
-			{
-				Decoder: "netflow",
-				Config: &udp.Configuration{
-					Listen:    "127.0.0.1:0",
-					QueueSize: 10,
-				},
-			},
-		}
-	}
-	c, err := New(r, config, Dependencies{
-		Daemon: daemon.NewMock(t),
-		HTTP:   httpserver.NewMock(t, r),
-		Schema: schema.NewMock(t),
-	})
-	if err != nil {
-		t.Fatalf("New() error:\n%+v", err)
-	}
-	helpers.StartStop(t, c)
-	return c
-}
-
-// Inject inject the provided flow message, as if it was received.
-func (c *Component) Inject(fmsg *schema.FlowMessage) {
-	c.outgoingFlows <- fmsg
-}
diff --git a/inlet/kafka/config_test.go b/inlet/kafka/config_test.go
index a157a57c..950af6dd 100644
--- a/inlet/kafka/config_test.go
+++ b/inlet/kafka/config_test.go
@@ -8,19 +8,21 @@ import (
 
 	"akvorado/common/helpers"
 
-	"github.com/IBM/sarama"
+	"github.com/twmb/franz-go/pkg/kgo"
 )
 
 func TestCompressionCodecUnmarshal(t *testing.T) {
 	cases := []struct {
 		Input         string
-		Expected      sarama.CompressionCodec
+		Expected      kgo.CompressionCodec
 		ExpectedError bool
 	}{
-		{"none", sarama.CompressionNone, false},
-		{"zstd", sarama.CompressionZSTD, false},
-		{"gzip", sarama.CompressionGZIP, false},
-		{"unknown", sarama.CompressionNone, true},
+		{"none", kgo.NoCompression(), false},
+		{"zstd", kgo.ZstdCompression(), false},
+		{"gzip", kgo.GzipCompression(), false},
+		{"snappy", kgo.SnappyCompression(), false},
+		{"lz4", kgo.Lz4Compression(), false},
+		{"unknown", kgo.NoCompression(), true},
 	}
 	for _, tc := range cases {
 		var cmp CompressionCodec
@@ -33,10 +35,13 @@ func TestCompressionCodecUnmarshal(t *testing.T) {
 			t.Errorf("UnmarshalText(%q) got %v but expected error", tc.Input, cmp)
 			continue
 		}
-		if cmp != CompressionCodec(tc.Expected) {
+		if !tc.ExpectedError && cmp != CompressionCodec(tc.Expected) {
 			t.Errorf("UnmarshalText(%q) got %v but expected %v", tc.Input, cmp, tc.Expected)
 			continue
 		}
+		if !tc.ExpectedError && cmp.String() != tc.Input {
+			t.Errorf("String() got %q but expected %q", cmp.String(), tc.Input)
+		}
 	}
 }
 
diff --git a/inlet/kafka/functional_test.go b/inlet/kafka/functional_test.go
index eecbfe79..8a4fc4c2 100644
--- a/inlet/kafka/functional_test.go
+++ b/inlet/kafka/functional_test.go
@@ -4,33 +4,33 @@
 package kafka
 
 import (
-	"errors"
+	"context"
 	"fmt"
 	"math/rand"
+	"sync"
 	"testing"
 	"time"
 
-	"github.com/IBM/sarama"
+	"github.com/twmb/franz-go/pkg/kgo"
 
 	"akvorado/common/daemon"
 	"akvorado/common/helpers"
 	"akvorado/common/kafka"
+	"akvorado/common/pb"
 	"akvorado/common/reporter"
-	"akvorado/common/schema"
 )
 
 func TestRealKafka(t *testing.T) {
 	client, brokers := kafka.SetupKafkaBroker(t)
+	defer client.Close()
 
 	topicName := fmt.Sprintf("test-topic-%d", rand.Int())
+	expectedTopicName := fmt.Sprintf("%s-v%d", topicName, pb.Version)
 	configuration := DefaultConfiguration()
 	configuration.Topic = topicName
 	configuration.Brokers = brokers
-	configuration.Version = kafka.Version(sarama.V2_8_1_0)
-	configuration.FlushInterval = 100 * time.Millisecond
-	expectedTopicName := fmt.Sprintf("%s-%s", topicName, schema.NewMock(t).ProtobufMessageHash())
 	r := reporter.NewMock(t)
-	c, err := New(r, configuration, Dependencies{Daemon: daemon.NewMock(t), Schema: schema.NewMock(t)})
+	c, err := New(r, configuration, Dependencies{Daemon: daemon.NewMock(t)})
 	if err != nil {
 		t.Fatalf("New() error:\n%+v", err)
 	}
@@ -43,54 +43,70 @@ func TestRealKafka(t *testing.T) {
 		msg1[i] = letters[rand.Intn(len(letters))]
 	}
 	for i := range msg2 {
-		msg1[i] = letters[rand.Intn(len(letters))]
+		msg2[i] = letters[rand.Intn(len(letters))]
+	}
+	var wg sync.WaitGroup
+	wg.Add(2)
+	c.Send("127.0.0.1", msg1, func() { wg.Done() })
+	c.Send("127.0.0.1", msg2, func() { wg.Done() })
+	c.Flush(t)
+	done := make(chan struct{})
+	go func() {
+		wg.Wait()
+		close(done)
+	}()
+	select {
+	case <-done:
+	case <-time.After(time.Second):
+		t.Fatal("Send() timeout")
 	}
-	c.Send("127.0.0.1", msg1)
-	c.Send("127.0.0.1", msg2)
 
-	time.Sleep(10 * time.Millisecond)
-	gotMetrics := r.GetMetrics("akvorado_inlet_kafka_", "sent_")
+	gotMetrics := r.GetMetrics("akvorado_inlet_kafka_", "sent_", "connects_")
 	expectedMetrics := map[string]string{
+		// Our own metrics
 		`sent_bytes_total{exporter="127.0.0.1"}`:    "100",
 		`sent_messages_total{exporter="127.0.0.1"}`: "2",
+		// From franz-go
+		`connects_total{node_id="1"}`:      "2",
+		`connects_total{node_id="seed_0"}`: "1",
 	}
 	if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 		t.Fatalf("Metrics (-got, +want):\n%s", diff)
 	}
 
-	// Try to consume the two messages
-	consumer, err := sarama.NewConsumerFromClient(client)
+	// Try to consume the two messages using franz-go
+	consumer, err := kgo.NewClient(
+		kgo.SeedBrokers(brokers...),
+		kgo.ConsumeTopics(expectedTopicName),
+		kgo.ConsumeResetOffset(kgo.NewOffset().AtStart()),
+		kgo.FetchMaxWait(10*time.Millisecond),
+	)
 	if err != nil {
-		t.Fatalf("NewConsumerGroup() error:\n%+v", err)
+		t.Fatalf("NewClient() error:\n%+v", err)
 	}
 	defer consumer.Close()
-	var partitions []int32
-	for {
-		partitions, err = consumer.Partitions(expectedTopicName)
-		if err != nil {
-			if errors.Is(err, sarama.ErrUnknownTopicOrPartition) {
-				// Wait for topic to be available
-				continue
-			}
-			t.Fatalf("Partitions() error:\n%+v", err)
-		}
-		break
-	}
-	partitionConsumer, err := consumer.ConsumePartition(expectedTopicName, partitions[0], sarama.OffsetOldest)
-	if err != nil {
-		t.Fatalf("ConsumePartitions() error:\n%+v", err)
-	}
 
 	got := []string{}
 	expected := []string{string(msg1), string(msg2)}
 	timeout := time.After(15 * time.Second)
-	for range len(expected) {
+	for len(got) < len(expected) {
 		select {
-		case msg := <-partitionConsumer.Messages():
-			got = append(got, string(msg.Value))
-		case err := <-partitionConsumer.Errors():
-			t.Fatalf("consumer.Errors():\n%+v", err)
 		case <-timeout:
+			t.Fatalf("Timeout waiting for messages. Got %d of %d messages", len(got), len(expected))
+		default:
+			ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
+			fetches := consumer.PollFetches(ctx)
+			cancel()
+			if errs := fetches.Errors(); len(errs) > 0 {
+				t.Logf("PollFetches() error: %+v", err)
+				time.Sleep(100 * time.Millisecond)
+				continue
+			}
+			fetches.EachPartition(func(p kgo.FetchTopicPartition) {
+				for _, record := range p.Records {
+					got = append(got, string(record.Value))
+				}
+			})
 		}
 	}
 
diff --git a/inlet/kafka/root_test.go b/inlet/kafka/root_test.go
index a01639ce..df006792 100644
--- a/inlet/kafka/root_test.go
+++ b/inlet/kafka/root_test.go
@@ -4,100 +4,119 @@
 package kafka
 
 import (
-	"errors"
+	"context"
 	"fmt"
+	"slices"
+	"sync"
+	"sync/atomic"
 	"testing"
 	"time"
 
-	"github.com/IBM/sarama"
-	gometrics "github.com/rcrowley/go-metrics"
+	"github.com/twmb/franz-go/pkg/kerr"
+	"github.com/twmb/franz-go/pkg/kgo"
+	"github.com/twmb/franz-go/pkg/kmsg"
 
-	"akvorado/common/daemon"
 	"akvorado/common/helpers"
+	"akvorado/common/pb"
 	"akvorado/common/reporter"
-	"akvorado/common/schema"
 )
 
 func TestKafka(t *testing.T) {
 	r := reporter.NewMock(t)
-	c, mockProducer := NewMock(t, r, DefaultConfiguration())
+	topic := fmt.Sprintf("flows-v%d", pb.Version)
+	config := DefaultConfiguration()
+	config.QueueSize = 1
+	c, mock := NewMock(t, r, config)
+	defer mock.Close()
 
-	// Send one message
-	received := make(chan bool)
-	mockProducer.ExpectInputWithMessageCheckerFunctionAndSucceed(func(got *sarama.ProducerMessage) error {
-		defer close(received)
-		expected := sarama.ProducerMessage{
-			Topic:     fmt.Sprintf("flows-%s", c.d.Schema.ProtobufMessageHash()),
-			Key:       got.Key,
-			Value:     sarama.ByteEncoder("hello world!"),
-			Partition: got.Partition,
+	// Inject an error on third message.
+	var count atomic.Uint32
+	mock.ControlKey(0, func(kreq kmsg.Request) (kmsg.Response, error, bool) {
+		mock.KeepControl()
+		current := count.Add(1)
+		if current != 3 {
+			t.Logf("message %d: ok", current)
+			return nil, nil, false
 		}
-		if diff := helpers.Diff(got, expected); diff != "" {
-			t.Fatalf("Send() (-got, +want):\n%s", diff)
+		t.Logf("mesage %d: error", current)
+		req := kreq.(*kmsg.ProduceRequest)
+		resp := kreq.ResponseKind().(*kmsg.ProduceResponse)
+		for _, rt := range req.Topics {
+			st := kmsg.NewProduceResponseTopic()
+			st.Topic = rt.Topic
+			for _, rp := range rt.Partitions {
+				sp := kmsg.NewProduceResponseTopicPartition()
+				sp.Partition = rp.Partition
+				sp.ErrorCode = kerr.CorruptMessage.Code
+				st.Partitions = append(st.Partitions, sp)
+			}
+			resp.Topics = append(resp.Topics, st)
 		}
-		return nil
+		return resp, nil, true
 	})
-	c.Send("127.0.0.1", []byte("hello world!"))
+
+	// Send messages
+	var wg sync.WaitGroup
+	wg.Add(4)
+	c.Send("127.0.0.1", []byte("hello world!"), func() { wg.Done() })
+	c.Send("127.0.0.1", []byte("goodbye world!"), func() { wg.Done() })
+	c.Send("127.0.0.1", []byte("nooooo!"), func() { wg.Done() })
+	c.Send("127.0.0.1", []byte("all good"), func() { wg.Done() })
+	done := make(chan struct{})
+	go func() {
+		wg.Wait()
+		close(done)
+	}()
 	select {
-	case <-received:
-	case <-time.After(1 * time.Second):
-		t.Fatal("Kafka message not received")
+	case <-done:
+	case <-time.After(time.Second):
+		t.Fatal("Send() timeout")
 	}
 
-	// Another but with a fail
-	mockProducer.ExpectInputAndFail(errors.New("noooo"))
-	c.Send("127.0.0.1", []byte("goodbye world!"))
+	expectedMessages := []string{"hello world!", "goodbye world!", "all good"}
 
-	time.Sleep(10 * time.Millisecond)
-	gotMetrics := r.GetMetrics("akvorado_inlet_kafka_")
-	expectedMetrics := map[string]string{
-		`sent_bytes_total{exporter="127.0.0.1"}`: "26",
-		fmt.Sprintf(`errors_total{error="kafka: Failed to produce message to topic flows-%s: noooo"}`, c.d.Schema.ProtobufMessageHash()): "1",
-		`sent_messages_total{exporter="127.0.0.1"}`: "2",
-	}
-	if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
-		t.Fatalf("Metrics (-got, +want):\n%s", diff)
+	// Create consumer to check messages
+	consumer, err := kgo.NewClient(
+		kgo.SeedBrokers(mock.ListenAddrs()...),
+		kgo.ConsumeTopics(topic),
+		kgo.ConsumeResetOffset(kgo.NewOffset().AtStart()),
+	)
+	if err != nil {
+		t.Fatalf("NewClient() error:\n%+v", err)
 	}
-}
+	defer consumer.Close()
 
-func TestKafkaMetrics(t *testing.T) {
-	r := reporter.NewMock(t)
-	c, err := New(r, DefaultConfiguration(), Dependencies{Daemon: daemon.NewMock(t), Schema: schema.NewMock(t)})
-	if err != nil {
-		t.Fatalf("New() error:\n%+v", err)
+	// Consume messages
+	messages := make([]string, 0)
+	ctx, cancel := context.WithTimeout(t.Context(), 5*time.Second)
+	defer cancel()
+	for {
+		if len(messages) >= len(expectedMessages) {
+			break
+		}
+		fetches := consumer.PollFetches(ctx)
+		if errs := fetches.Errors(); len(errs) > 0 {
+			t.Fatalf("PollFetches() error:\n%+v", errs)
+		}
+
+		fetches.EachPartition(func(p kgo.FetchTopicPartition) {
+			for _, record := range p.Records {
+				messages = append(messages, string(record.Value))
+			}
+		})
 	}
 
-	// Manually put some metrics
-	gometrics.GetOrRegisterMeter("incoming-byte-rate-for-broker-1111", c.kafkaConfig.MetricRegistry).
-		Mark(100)
-	gometrics.GetOrRegisterMeter("incoming-byte-rate-for-broker-1112", c.kafkaConfig.MetricRegistry).
-		Mark(200)
-	gometrics.GetOrRegisterMeter("outgoing-byte-rate-for-broker-1111", c.kafkaConfig.MetricRegistry).
-		Mark(199)
-	gometrics.GetOrRegisterMeter("outgoing-byte-rate-for-broker-1112", c.kafkaConfig.MetricRegistry).
-		Mark(20)
-	gometrics.GetOrRegisterHistogram("request-size-for-broker-1111", c.kafkaConfig.MetricRegistry,
-		gometrics.NewExpDecaySample(10, 1)).
-		Update(100)
-	gometrics.GetOrRegisterCounter("requests-in-flight-for-broker-1111", c.kafkaConfig.MetricRegistry).
-		Inc(20)
-	gometrics.GetOrRegisterCounter("requests-in-flight-for-broker-1112", c.kafkaConfig.MetricRegistry).
-		Inc(20)
+	slices.Sort(expectedMessages)
+	slices.Sort(messages)
+	if diff := helpers.Diff(messages, expectedMessages); diff != "" {
+		t.Fatalf("Send() (-got, +want):\n%s", diff)
+	}
 
-	gotMetrics := r.GetMetrics("akvorado_inlet_kafka_")
+	gotMetrics := r.GetMetrics("akvorado_inlet_kafka_", "sent_", "errors")
 	expectedMetrics := map[string]string{
-		`brokers_incoming_byte_rate{broker="1111"}`:            "0",
-		`brokers_incoming_byte_rate{broker="1112"}`:            "0",
-		`brokers_outgoing_byte_rate{broker="1111"}`:            "0",
-		`brokers_outgoing_byte_rate{broker="1112"}`:            "0",
-		`brokers_request_size_bucket{broker="1111",le="+Inf"}`: "1",
-		`brokers_request_size_bucket{broker="1111",le="0.5"}`:  "100",
-		`brokers_request_size_bucket{broker="1111",le="0.9"}`:  "100",
-		`brokers_request_size_bucket{broker="1111",le="0.99"}`: "100",
-		`brokers_request_size_count{broker="1111"}`:            "1",
-		`brokers_request_size_sum{broker="1111"}`:              "100",
-		`brokers_inflight_requests{broker="1111"}`:             "20",
-		`brokers_inflight_requests{broker="1112"}`:             "20",
+		`sent_bytes_total{exporter="127.0.0.1"}`:    "34",
+		`sent_messages_total{exporter="127.0.0.1"}`: "3",
+		`errors_total{error="CORRUPT_MESSAGE"}`:     "1",
 	}
 	if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 		t.Fatalf("Metrics (-got, +want):\n%s", diff)
diff --git a/inlet/kafka/tests.go b/inlet/kafka/tests.go
index b4da2762..3d146e60 100644
--- a/inlet/kafka/tests.go
+++ b/inlet/kafka/tests.go
@@ -8,34 +8,43 @@ package kafka
 import (
 	"testing"
 
-	"github.com/IBM/sarama"
-	"github.com/IBM/sarama/mocks"
+	"github.com/twmb/franz-go/pkg/kfake"
 
 	"akvorado/common/daemon"
 	"akvorado/common/helpers"
 	"akvorado/common/reporter"
-	"akvorado/common/schema"
 )
 
 // NewMock creates a new Kafka component with a mocked Kafka. It will
 // panic if it cannot be started.
-func NewMock(t *testing.T, reporter *reporter.Reporter, configuration Configuration) (*Component, *mocks.AsyncProducer) {
+func NewMock(t *testing.T, reporter *reporter.Reporter, configuration Configuration) (*Component, *kfake.Cluster) {
 	t.Helper()
+	// Use a fake Kafka cluster for testing
+	cluster, err := kfake.NewCluster(
+		kfake.NumBrokers(1),
+		kfake.AllowAutoTopicCreation(),
+	)
+	if err != nil {
+		t.Fatalf("NewCluster() error: %v", err)
+	}
+	t.Cleanup(func() {
+		cluster.Close()
+	})
+	configuration.Brokers = cluster.ListenAddrs()
+
 	c, err := New(reporter, configuration, Dependencies{
 		Daemon: daemon.NewMock(t),
-		Schema: schema.NewMock(t),
 	})
 	if err != nil {
 		t.Fatalf("New() error:\n%+v", err)
 	}
+	helpers.StartStop(t, c)
+	return c, cluster
+}
 
-	// Use a mocked Kafka producer
-	var mockProducer *mocks.AsyncProducer
-	c.createKafkaProducer = func() (sarama.AsyncProducer, error) {
-		mockProducer = mocks.NewAsyncProducer(t, c.kafkaConfig)
-		return mockProducer, nil
+// Flush force flushing the currently buffered records.
+func (c *Component) Flush(t *testing.T) {
+	if err := c.kafkaClient.Flush(t.Context()); err != nil {
+		t.Fatalf("Flush() error:\n%+v", err)
 	}
-
-	helpers.StartStop(t, c)
-	return c, mockProducer
 }
diff --git a/inlet/metadata/provider/static/root_test.go b/inlet/metadata/provider/static/root_test.go
deleted file mode 100644
index 02c30db6..00000000
--- a/inlet/metadata/provider/static/root_test.go
+++ /dev/null
@@ -1,268 +0,0 @@
-// SPDX-FileCopyrightText: 2023 Free Mobile
-// SPDX-License-Identifier: AGPL-3.0-only
-
-package static
-
-import (
-	"context"
-	"net/netip"
-	"testing"
-
-	"akvorado/common/helpers"
-	"akvorado/common/reporter"
-	"akvorado/common/schema"
-	"akvorado/inlet/metadata/provider"
-)
-
-func TestStaticProvider(t *testing.T) {
-	config := Configuration{
-		Exporters: helpers.MustNewSubnetMap(map[string]ExporterConfiguration{
-			"2001:db8:1::/48": {
-				Exporter: provider.Exporter{
-					Name: "nodefault",
-				},
-				IfIndexes: map[uint]provider.Interface{
-					10: {
-						Name:        "Gi10",
-						Description: "10th interface",
-						Speed:       1000,
-					},
-					11: {
-						Name:        "Gi11",
-						Description: "11th interface",
-						Speed:       1000,
-					},
-				},
-			},
-			"2001:db8:2::/48": {
-				Exporter: provider.Exporter{
-					Name: "default",
-				},
-				Default: provider.Interface{
-					Name:        "Default0",
-					Description: "Default interface",
-					Speed:       1000,
-				},
-				IfIndexes: map[uint]provider.Interface{
-					10: {
-						Name:        "Gi10",
-						Description: "10th interface",
-						Speed:       1000,
-					},
-				},
-			},
-			"2001:db8:3::/48": {
-				Exporter: provider.Exporter{
-					Name:   "default with metadata",
-					Region: "eu",
-					Role:   "peering",
-					Tenant: "mine",
-					Site:   "par",
-					Group:  "blue",
-				},
-				Default: provider.Interface{
-					Name:        "Default0",
-					Description: "Default interface",
-					Speed:       1000,
-				},
-				IfIndexes: map[uint]provider.Interface{
-					10: {
-						Name:         "Gi10",
-						Description:  "10th interface",
-						Speed:        1000,
-						Provider:     "transit101",
-						Connectivity: "transit",
-						Boundary:     schema.InterfaceBoundaryExternal,
-					},
-				},
-			},
-			"2001:db8:4::/48": {
-				Exporter: provider.Exporter{
-					Name: "nodefault skip",
-				},
-				IfIndexes: map[uint]provider.Interface{
-					10: {
-						Name:         "Gi10",
-						Description:  "10th interface",
-						Speed:        1000,
-						Provider:     "transit101",
-						Connectivity: "transit",
-						Boundary:     schema.InterfaceBoundaryExternal,
-					},
-				},
-				SkipMissingInterfaces: true,
-			},
-		}),
-	}
-
-	var got []provider.Update
-	r := reporter.NewMock(t)
-	p, _ := config.New(r, func(update provider.Update) {
-		got = append(got, update)
-	})
-
-	p.Query(context.Background(), &provider.BatchQuery{
-		ExporterIP: netip.MustParseAddr("2001:db8:1::10"),
-		IfIndexes:  []uint{9, 10, 11},
-	})
-	p.Query(context.Background(), &provider.BatchQuery{
-		ExporterIP: netip.MustParseAddr("2001:db8:2::10"),
-		IfIndexes:  []uint{9, 10, 11},
-	})
-	p.Query(context.Background(), &provider.BatchQuery{
-		ExporterIP: netip.MustParseAddr("2001:db8:3::10"),
-		IfIndexes:  []uint{10},
-	})
-	query := provider.BatchQuery{
-		ExporterIP: netip.MustParseAddr("2001:db8:4::10"),
-		IfIndexes:  []uint{9, 10, 11},
-	}
-	err := p.Query(context.Background(), &query)
-
-	expected := []provider.Update{
-		{
-			Query: provider.Query{
-				ExporterIP: netip.MustParseAddr("2001:db8:1::10"),
-				IfIndex:    9,
-			},
-			Answer: provider.Answer{
-				Exporter: provider.Exporter{
-					Name: "nodefault",
-				},
-			},
-		},
-		{
-			Query: provider.Query{
-				ExporterIP: netip.MustParseAddr("2001:db8:1::10"),
-				IfIndex:    10,
-			},
-			Answer: provider.Answer{
-				Exporter: provider.Exporter{
-					Name: "nodefault",
-				},
-				Interface: provider.Interface{
-					Name:        "Gi10",
-					Description: "10th interface",
-					Speed:       1000,
-				},
-			},
-		},
-		{
-			Query: provider.Query{
-				ExporterIP: netip.MustParseAddr("2001:db8:1::10"),
-				IfIndex:    11,
-			},
-			Answer: provider.Answer{
-				Exporter: provider.Exporter{
-					Name: "nodefault",
-				},
-				Interface: provider.Interface{
-					Name:        "Gi11",
-					Description: "11th interface",
-					Speed:       1000,
-				},
-			},
-		},
-		{
-			Query: provider.Query{
-				ExporterIP: netip.MustParseAddr("2001:db8:2::10"),
-				IfIndex:    9,
-			},
-			Answer: provider.Answer{
-				Exporter: provider.Exporter{
-					Name: "default",
-				},
-				Interface: provider.Interface{
-					Name:        "Default0",
-					Description: "Default interface",
-					Speed:       1000,
-				},
-			},
-		},
-		{
-			Query: provider.Query{
-				ExporterIP: netip.MustParseAddr("2001:db8:2::10"),
-				IfIndex:    10,
-			},
-			Answer: provider.Answer{
-				Exporter: provider.Exporter{
-					Name: "default",
-				},
-				Interface: provider.Interface{
-					Name:        "Gi10",
-					Description: "10th interface",
-					Speed:       1000,
-				},
-			},
-		},
-		{
-			Query: provider.Query{
-				ExporterIP: netip.MustParseAddr("2001:db8:2::10"),
-				IfIndex:    11,
-			},
-			Answer: provider.Answer{
-				Exporter: provider.Exporter{
-					Name: "default",
-				},
-				Interface: provider.Interface{
-					Name:        "Default0",
-					Description: "Default interface",
-					Speed:       1000,
-				},
-			},
-		},
-		{
-			Query: provider.Query{
-				ExporterIP: netip.MustParseAddr("2001:db8:3::10"),
-				IfIndex:    10,
-			},
-			Answer: provider.Answer{
-				Exporter: provider.Exporter{
-					Name:   "default with metadata",
-					Region: "eu",
-					Role:   "peering",
-					Tenant: "mine",
-					Site:   "par",
-					Group:  "blue",
-				},
-				Interface: provider.Interface{
-					Name:         "Gi10",
-					Description:  "10th interface",
-					Speed:        1000,
-					Provider:     "transit101",
-					Connectivity: "transit",
-					Boundary:     schema.InterfaceBoundaryExternal,
-				},
-			},
-		},
-		{
-			Query: provider.Query{
-				ExporterIP: netip.MustParseAddr("2001:db8:4::10"),
-				IfIndex:    10,
-			},
-			Answer: provider.Answer{
-				Exporter: provider.Exporter{
-					Name: "nodefault skip",
-				},
-				Interface: provider.Interface{
-					Name:         "Gi10",
-					Description:  "10th interface",
-					Speed:        1000,
-					Provider:     "transit101",
-					Connectivity: "transit",
-					Boundary:     schema.InterfaceBoundaryExternal,
-				},
-			},
-		},
-	}
-
-	if diff := helpers.Diff(got, expected); diff != "" {
-		t.Fatalf("static provider (-got, +want):\n%s", diff)
-	}
-	if diff := helpers.Diff(query.IfIndexes, []uint{9, 11}); diff != "" {
-		t.Fatalf("static provider (-got, +want):\n%s", diff)
-	}
-	if diff := helpers.Diff(err, provider.ErrSkipProvider); diff != "" {
-		t.Fatalf("static provider (-got, +want):\n%s", diff)
-	}
-}
diff --git a/inlet/metadata/tests.go b/inlet/metadata/tests.go
deleted file mode 100644
index 9be54c41..00000000
--- a/inlet/metadata/tests.go
+++ /dev/null
@@ -1,83 +0,0 @@
-// SPDX-FileCopyrightText: 2022 Free Mobile
-// SPDX-License-Identifier: AGPL-3.0-only
-
-//go:build !release
-
-package metadata
-
-import (
-	"context"
-	"fmt"
-	"strings"
-	"testing"
-
-	"akvorado/common/helpers"
-	"akvorado/common/reporter"
-	"akvorado/common/schema"
-	"akvorado/inlet/metadata/provider"
-)
-
-// mockProvider represents a mock provider.
-type mockProvider struct {
-	put func(provider.Update)
-}
-
-// Query query the mock provider for a value.
-func (mp mockProvider) Query(_ context.Context, query *provider.BatchQuery) error {
-	for _, ifIndex := range query.IfIndexes {
-		answer := provider.Answer{
-			Exporter: provider.Exporter{
-				Name: strings.ReplaceAll(query.ExporterIP.Unmap().String(), ".", "_"),
-			},
-		}
-		if ifIndex != 999 {
-			answer.Interface.Name = fmt.Sprintf("Gi0/0/%d", ifIndex)
-			answer.Interface.Description = fmt.Sprintf("Interface %d", ifIndex)
-			answer.Interface.Speed = 1000
-		}
-		// in iface with  metadata (overriden by out iface)
-		if ifIndex == 1010 {
-			answer.Exporter.Group = "metadata group"
-			answer.Exporter.Region = "metadata region"
-			answer.Exporter.Role = "metadata role"
-			answer.Exporter.Site = "metadata site"
-			answer.Exporter.Tenant = "metadata tenant"
-		}
-
-		// out iface with metadata
-		if ifIndex == 2010 {
-			answer.Interface.Boundary = schema.InterfaceBoundaryExternal
-			answer.Interface.Connectivity = "metadata connectivity"
-			answer.Interface.Provider = "metadata provider"
-			answer.Exporter.Group = "metadata group"
-			answer.Exporter.Region = "metadata region"
-			answer.Exporter.Role = "metadata role"
-			answer.Exporter.Site = "metadata site"
-			answer.Exporter.Tenant = "metadata tenant"
-		}
-		mp.put(provider.Update{Query: provider.Query{ExporterIP: query.ExporterIP, IfIndex: ifIndex}, Answer: answer})
-	}
-	return nil
-}
-
-// mockProviderConfiguration is the configuration for the mock provider.
-type mockProviderConfiguration struct{}
-
-// New returns a new mock provider.
-func (mpc mockProviderConfiguration) New(_ *reporter.Reporter, put func(provider.Update)) (provider.Provider, error) {
-	return mockProvider{put: put}, nil
-}
-
-// NewMock creates a new metadata component building synthetic values. It is already started.
-func NewMock(t *testing.T, reporter *reporter.Reporter, configuration Configuration, dependencies Dependencies) *Component {
-	t.Helper()
-	if configuration.Providers == nil {
-		configuration.Providers = []ProviderConfiguration{{Config: mockProviderConfiguration{}}}
-	}
-	c, err := New(reporter, configuration, dependencies)
-	if err != nil {
-		t.Fatalf("New() error:\n%+v", err)
-	}
-	helpers.StartStop(t, c)
-	return c
-}
diff --git a/orchestrator/clickhouse/config_test.go b/orchestrator/clickhouse/config_test.go
index 431bfb04..dcf56199 100644
--- a/orchestrator/clickhouse/config_test.go
+++ b/orchestrator/clickhouse/config_test.go
@@ -88,7 +88,6 @@ func TestNetworkNamesUnmarshalHook(t *testing.T) {
 
 func TestDefaultConfiguration(t *testing.T) {
 	config := DefaultConfiguration()
-	config.Kafka.Topic = "flow"
 	if err := helpers.Validate.Struct(config); err != nil {
 		t.Fatalf("validate.Struct() error:\n%+v", err)
 	}
diff --git a/orchestrator/clickhouse/http_test.go b/orchestrator/clickhouse/http_test.go
index c72379d4..c14ed6a0 100644
--- a/orchestrator/clickhouse/http_test.go
+++ b/orchestrator/clickhouse/http_test.go
@@ -4,7 +4,6 @@
 package clickhouse
 
 import (
-	"fmt"
 	"testing"
 
 	"akvorado/common/clickhousedb"
@@ -75,30 +74,13 @@ func TestHTTPEndpoints(t *testing.T) {
 				`192.0.2.0/24,infra,,,,,,,,`,
 			},
 		}, {
-			URL:         "/api/v0/orchestrator/clickhouse/init.sh",
-			ContentType: "text/x-shellscript",
-			FirstLines: []string{
-				`#!/bin/sh`,
-				``,
-				`# Install Protobuf schema`,
-				`mkdir -p /var/lib/clickhouse/format_schemas`,
-				fmt.Sprintf(`echo "Install flow schema flow-%s.proto"`,
-					c.d.Schema.ProtobufMessageHash()),
-				fmt.Sprintf(`cat > /var/lib/clickhouse/format_schemas/flow-%s.proto <<'EOPROTO'`,
-					c.d.Schema.ProtobufMessageHash()),
-				"",
-				`syntax = "proto3";`,
-			},
-		},
-		{
 			URL:         "/api/v0/orchestrator/clickhouse/custom_dict_none.csv",
 			ContentType: "text/plain; charset=utf-8",
 			StatusCode:  404,
 			FirstLines: []string{
 				"unable to deliver custom dict csv file none.csv",
 			},
-		},
-		{
+		}, {
 			URL:         "/api/v0/orchestrator/clickhouse/custom_dict_test.csv",
 			ContentType: "text/csv; charset=utf-8",
 			FirstLines: []string{
diff --git a/orchestrator/clickhouse/migrations_test.go b/orchestrator/clickhouse/migrations_test.go
index efe98c10..74dee7b5 100644
--- a/orchestrator/clickhouse/migrations_test.go
+++ b/orchestrator/clickhouse/migrations_test.go
@@ -11,6 +11,8 @@ import (
 	"net/url"
 	"os"
 	"path"
+	"regexp"
+	"slices"
 	"strconv"
 	"strings"
 	"testing"
@@ -20,7 +22,6 @@ import (
 	"akvorado/common/daemon"
 	"akvorado/common/helpers"
 	"akvorado/common/httpserver"
-	"akvorado/common/kafka"
 	"akvorado/common/reporter"
 	"akvorado/common/schema"
 	"akvorado/orchestrator/geoip"
@@ -68,11 +69,42 @@ func dropAllTables(t *testing.T, ch *clickhousedb.Component) {
 	}
 }
 
-func loadTables(t *testing.T, ch *clickhousedb.Component, sch *schema.Component, schemas []tableWithSchema) {
+func loadTables(t *testing.T, ch *clickhousedb.Component, sch *schema.Component, tables []tableWithSchema) {
 	ctx := clickhouse.Context(context.Background(), clickhouse.WithSettings(clickhouse.Settings{
 		"allow_suspicious_low_cardinality_types": 1,
 	}))
-	for _, tws := range schemas {
+
+	// Sort in a way we respect the dependencies.
+	fromPattern := regexp.MustCompile(`.*FROM (default\.\w+)`)
+	toPattern := regexp.MustCompile(`.*TO (default\.\w+)`)
+	createPattern := regexp.MustCompile(`CREATE (?:TABLE|MATERIALIZED VIEW) (default\.\w+)`)
+	slices.SortFunc(tables, func(t1, t2 tableWithSchema) int {
+		t1From := fromPattern.FindStringSubmatch(t1.Schema)
+		t1To := toPattern.FindStringSubmatch(t1.Schema)
+		t1Create := createPattern.FindStringSubmatch(t1.Schema)
+		t2From := fromPattern.FindStringSubmatch(t2.Schema)
+		t2To := toPattern.FindStringSubmatch(t2.Schema)
+		t2Create := createPattern.FindStringSubmatch(t2.Schema)
+		if t1From != nil && t2Create != nil && t1From[1] == t2Create[1] {
+			// t2 creates the table that is used by t1 and t1 should be ordered after
+			return 1
+		}
+		if t1To != nil && t2Create != nil && t1To[1] == t2Create[1] {
+			// t2 creates the table that is used by t1 and t1 should be ordered after
+			return 1
+		}
+		if t2From != nil && t1Create != nil && t2From[1] == t1Create[1] {
+			// t1 creates the table that is used by t2 and t2 should be ordered after
+			return -1
+		}
+		if t2To != nil && t1Create != nil && t2To[1] == t1Create[1] {
+			// t1 creates the table that is used by t2 and t2 should be ordered after
+			return -1
+		}
+		return 0
+	})
+
+	for _, tws := range tables {
 		if isOldTable(sch, tws.Table) {
 			continue
 		}
@@ -116,14 +148,19 @@ func loadAllTables(t *testing.T, ch *clickhousedb.Component, sch *schema.Compone
 }
 
 func isOldTable(schema *schema.Component, table string) bool {
-	if strings.Contains(table, schema.ProtobufMessageHash()) {
+	if strings.Contains(table, schema.ClickHouseHash()) {
 		return false
 	}
-	if table == "flows_raw_errors" {
-		return false
+	oldSuffixes := []string{
+		"_raw",
+		"_raw_consumer",
+		"_raw_errors", "_raw_errors_local",
+		"_raw_errors_consumer",
 	}
-	if strings.HasSuffix(table, "_raw") || strings.HasSuffix(table, "_raw_consumer") || strings.HasSuffix(table, "_raw_errors") {
-		return true
+	for _, suffix := range oldSuffixes {
+		if strings.HasSuffix(table, suffix) {
+			return true
+		}
 	}
 	return false
 }
@@ -136,7 +173,6 @@ func startTestComponent(t *testing.T, r *reporter.Reporter, chComponent *clickho
 	}
 	configuration := DefaultConfiguration()
 	configuration.OrchestratorURL = "http://127.0.0.1:0"
-	configuration.Kafka.Configuration = kafka.DefaultConfiguration()
 	ch, err := New(r, configuration, Dependencies{
 		Daemon:     daemon.NewMock(t),
 		HTTP:       httpserver.NewMock(t, r),
@@ -234,7 +270,7 @@ WHERE database=currentDatabase() AND table NOT LIKE '.%'`)
 			if err != nil {
 				t.Fatalf("Query() error:\n%+v", err)
 			}
-			hash := ch.d.Schema.ProtobufMessageHash()
+			hash := ch.d.Schema.ClickHouseHash()
 			got := []string{}
 			for rows.Next() {
 				var table string
@@ -263,9 +299,6 @@ WHERE database=currentDatabase() AND table NOT LIKE '.%'`)
 				fmt.Sprintf("flows_%s_raw", hash),
 				fmt.Sprintf("flows_%s_raw_consumer", hash),
 				"flows_local",
-				"flows_raw_errors",
-				"flows_raw_errors_consumer",
-				"flows_raw_errors_local",
 				schema.DictionaryICMP,
 				schema.DictionaryNetworks,
 				schema.DictionaryProtocols,
@@ -360,9 +393,6 @@ func TestMigrationFromPreviousStates(t *testing.T) {
 			schema.ColumnDstASPath,
 			schema.ColumnDstCommunities,
 			schema.ColumnDstLargeCommunities,
-			schema.ColumnDstLargeCommunitiesASN,
-			schema.ColumnDstLargeCommunitiesLocalData1,
-			schema.ColumnDstLargeCommunitiesLocalData2,
 		}
 		sch, err := schema.New(schConfig)
 		if err != nil {
@@ -442,7 +472,7 @@ AND name LIKE $3`, "flows", ch.d.ClickHouse.DatabaseName(), "%DimensionAttribute
 		// Check if the rows were created in the consumer flows table
 		rowConsumer := ch.d.ClickHouse.QueryRow(
 			context.Background(),
-			fmt.Sprintf(`SHOW CREATE flows_%s_raw_consumer`, ch.d.Schema.ProtobufMessageHash()))
+			fmt.Sprintf(`SHOW CREATE flows_%s_raw_consumer`, ch.d.Schema.ClickHouseHash()))
 		var existingConsumer string
 		if err := rowConsumer.Scan(&existingConsumer); err != nil {
 			t.Fatalf("Scan() error:\n%+v", err)
@@ -517,7 +547,7 @@ AND name LIKE $3`, "flows", ch.d.ClickHouse.DatabaseName(), "%DimensionAttribute
 		// Check if the rows were removed in the consumer flows table
 		rowConsumer := ch.d.ClickHouse.QueryRow(
 			context.Background(),
-			fmt.Sprintf(`SHOW CREATE flows_%s_raw_consumer`, ch.d.Schema.ProtobufMessageHash()))
+			fmt.Sprintf(`SHOW CREATE flows_%s_raw_consumer`, ch.d.Schema.ClickHouseHash()))
 		var existingConsumer string
 		if err := rowConsumer.Scan(&existingConsumer); err != nil {
 			t.Fatalf("Scan() error:\n%+v", err)
diff --git a/orchestrator/clickhouse/testdata/states/002-cluster.csv b/orchestrator/clickhouse/testdata/states/002-cluster.csv
new file mode 100644
index 00000000..0f7204ef
--- /dev/null
+++ b/orchestrator/clickhouse/testdata/states/002-cluster.csv
@@ -0,0 +1,21 @@
+asns,"CREATE DICTIONARY default.asns (`asn` UInt32 INJECTIVE, `name` String) PRIMARY KEY asn SOURCE(HTTP(URL 'http://127.0.0.1:0/api/v0/orchestrator/clickhouse/asns.csv' FORMAT 'CSVWithNames')) LIFETIME(MIN 0 MAX 3600) LAYOUT(HASHED()) SETTINGS(format_csv_allow_single_quotes = 0)"
+icmp,"CREATE DICTIONARY default.icmp (`proto` UInt8, `type` UInt8, `code` UInt8, `name` String) PRIMARY KEY proto, type, code SOURCE(HTTP(URL 'http://127.0.0.1:0/api/v0/orchestrator/clickhouse/icmp.csv' FORMAT 'CSVWithNames')) LIFETIME(MIN 0 MAX 3600) LAYOUT(COMPLEX_KEY_HASHED()) SETTINGS(format_csv_allow_single_quotes = 0)"
+networks,"CREATE DICTIONARY default.networks (`network` String, `name` String, `role` String, `site` String, `region` String, `city` String, `state` String, `country` String, `tenant` String, `asn` UInt32) PRIMARY KEY network SOURCE(HTTP(URL 'http://127.0.0.1:0/api/v0/orchestrator/clickhouse/networks.csv' FORMAT 'CSVWithNames')) LIFETIME(MIN 0 MAX 3600) LAYOUT(IP_TRIE()) SETTINGS(format_csv_allow_single_quotes = 0)"
+protocols,"CREATE DICTIONARY default.protocols (`proto` UInt8 INJECTIVE, `name` String, `description` String) PRIMARY KEY proto SOURCE(HTTP(URL 'http://127.0.0.1:0/api/v0/orchestrator/clickhouse/protocols.csv' FORMAT 'CSVWithNames')) LIFETIME(MIN 0 MAX 3600) LAYOUT(HASHED()) SETTINGS(format_csv_allow_single_quotes = 0)"
+tcp,"CREATE DICTIONARY default.tcp (`port` UInt16 INJECTIVE, `name` String) PRIMARY KEY port SOURCE(HTTP(URL 'http://127.0.0.1:0/api/v0/orchestrator/clickhouse/tcp.csv' FORMAT 'CSVWithNames')) LIFETIME(MIN 0 MAX 3600) LAYOUT(HASHED()) SETTINGS(format_csv_allow_single_quotes = 0)"
+udp,"CREATE DICTIONARY default.udp (`port` UInt16 INJECTIVE, `name` String) PRIMARY KEY port SOURCE(HTTP(URL 'http://127.0.0.1:0/api/v0/orchestrator/clickhouse/udp.csv' FORMAT 'CSVWithNames')) LIFETIME(MIN 0 MAX 3600) LAYOUT(HASHED()) SETTINGS(format_csv_allow_single_quotes = 0)"
+exporters,"CREATE TABLE default.exporters (`TimeReceived` DateTime, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `IfName` LowCardinality(String), `IfDescription` LowCardinality(String), `IfSpeed` UInt32, `IfConnectivity` LowCardinality(String), `IfProvider` LowCardinality(String), `IfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2)) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/shard-{shard}/exporters', 'replica-{replica}', TimeReceived) ORDER BY (ExporterAddress, IfName) TTL TimeReceived + toIntervalDay(1) SETTINGS index_granularity = 8192"
+flows_1h0m0s_local,"CREATE TABLE default.flows_1h0m0s_local (`TimeReceived` DateTime CODEC(DoubleDelta, LZ4), `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAS` UInt32, `DstAS` UInt32, `SrcNetName` LowCardinality(String), `DstNetName` LowCardinality(String), `SrcNetRole` LowCardinality(String), `DstNetRole` LowCardinality(String), `SrcNetSite` LowCardinality(String), `DstNetSite` LowCardinality(String), `SrcNetRegion` LowCardinality(String), `DstNetRegion` LowCardinality(String), `SrcNetTenant` LowCardinality(String), `DstNetTenant` LowCardinality(String), `SrcCountry` FixedString(2), `DstCountry` FixedString(2), `SrcGeoCity` LowCardinality(String), `DstGeoCity` LowCardinality(String), `SrcGeoState` LowCardinality(String), `DstGeoState` LowCardinality(String), `Dst1stAS` UInt32, `Dst2ndAS` UInt32, `Dst3rdAS` UInt32, `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `Bytes` UInt64 CODEC(T64, LZ4), `Packets` UInt64 CODEC(T64, LZ4), `PacketSize` UInt64 ALIAS intDiv(Bytes, Packets), `PacketSizeBucket` LowCardinality(String) ALIAS multiIf(PacketSize < 64, '0-63', PacketSize < 128, '64-127', PacketSize < 256, '128-255', PacketSize < 512, '256-511', PacketSize < 768, '512-767', PacketSize < 1024, '768-1023', PacketSize < 1280, '1024-1279', PacketSize < 1501, '1280-1500', PacketSize < 2048, '1501-2047', PacketSize < 3072, '2048-3071', PacketSize < 4096, '3072-4095', PacketSize < 8192, '4096-8191', PacketSize < 10240, '8192-10239', PacketSize < 16384, '10240-16383', PacketSize < 32768, '16384-32767', PacketSize < 65536, '32768-65535', '65536-Inf'), `ForwardingStatus` UInt32) ENGINE = ReplicatedSummingMergeTree('/clickhouse/tables/shard-{shard}/flows_1h0m0s_local', 'replica-{replica}', (Bytes, Packets)) PARTITION BY toYYYYMMDDhhmmss(toStartOfInterval(TimeReceived, toIntervalSecond(622080))) PRIMARY KEY (TimeReceived, ExporterAddress, EType, Proto, InIfName, SrcAS, ForwardingStatus, OutIfName, DstAS, SamplingRate) ORDER BY (TimeReceived, ExporterAddress, EType, Proto, InIfName, SrcAS, ForwardingStatus, OutIfName, DstAS, SamplingRate, SrcNetName, DstNetName, SrcNetRole, DstNetRole, SrcNetSite, DstNetSite, SrcNetRegion, DstNetRegion, SrcNetTenant, DstNetTenant, SrcCountry, DstCountry, SrcGeoCity, DstGeoCity, SrcGeoState, DstGeoState, Dst1stAS, Dst2ndAS, Dst3rdAS) TTL TimeReceived + toIntervalSecond(31104000) SETTINGS index_granularity = 8192, ttl_only_drop_parts = 1"
+flows_1m0s_local,"CREATE TABLE default.flows_1m0s_local (`TimeReceived` DateTime CODEC(DoubleDelta, LZ4), `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAS` UInt32, `DstAS` UInt32, `SrcNetName` LowCardinality(String), `DstNetName` LowCardinality(String), `SrcNetRole` LowCardinality(String), `DstNetRole` LowCardinality(String), `SrcNetSite` LowCardinality(String), `DstNetSite` LowCardinality(String), `SrcNetRegion` LowCardinality(String), `DstNetRegion` LowCardinality(String), `SrcNetTenant` LowCardinality(String), `DstNetTenant` LowCardinality(String), `SrcCountry` FixedString(2), `DstCountry` FixedString(2), `SrcGeoCity` LowCardinality(String), `DstGeoCity` LowCardinality(String), `SrcGeoState` LowCardinality(String), `DstGeoState` LowCardinality(String), `Dst1stAS` UInt32, `Dst2ndAS` UInt32, `Dst3rdAS` UInt32, `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `Bytes` UInt64 CODEC(T64, LZ4), `Packets` UInt64 CODEC(T64, LZ4), `PacketSize` UInt64 ALIAS intDiv(Bytes, Packets), `PacketSizeBucket` LowCardinality(String) ALIAS multiIf(PacketSize < 64, '0-63', PacketSize < 128, '64-127', PacketSize < 256, '128-255', PacketSize < 512, '256-511', PacketSize < 768, '512-767', PacketSize < 1024, '768-1023', PacketSize < 1280, '1024-1279', PacketSize < 1501, '1280-1500', PacketSize < 2048, '1501-2047', PacketSize < 3072, '2048-3071', PacketSize < 4096, '3072-4095', PacketSize < 8192, '4096-8191', PacketSize < 10240, '8192-10239', PacketSize < 16384, '10240-16383', PacketSize < 32768, '16384-32767', PacketSize < 65536, '32768-65535', '65536-Inf'), `ForwardingStatus` UInt32) ENGINE = ReplicatedSummingMergeTree('/clickhouse/tables/shard-{shard}/flows_1m0s_local', 'replica-{replica}', (Bytes, Packets)) PARTITION BY toYYYYMMDDhhmmss(toStartOfInterval(TimeReceived, toIntervalSecond(12096))) PRIMARY KEY (TimeReceived, ExporterAddress, EType, Proto, InIfName, SrcAS, ForwardingStatus, OutIfName, DstAS, SamplingRate) ORDER BY (TimeReceived, ExporterAddress, EType, Proto, InIfName, SrcAS, ForwardingStatus, OutIfName, DstAS, SamplingRate, SrcNetName, DstNetName, SrcNetRole, DstNetRole, SrcNetSite, DstNetSite, SrcNetRegion, DstNetRegion, SrcNetTenant, DstNetTenant, SrcCountry, DstCountry, SrcGeoCity, DstGeoCity, SrcGeoState, DstGeoState, Dst1stAS, Dst2ndAS, Dst3rdAS) TTL TimeReceived + toIntervalSecond(604800) SETTINGS index_granularity = 8192, ttl_only_drop_parts = 1"
+flows_5m0s_local,"CREATE TABLE default.flows_5m0s_local (`TimeReceived` DateTime CODEC(DoubleDelta, LZ4), `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAS` UInt32, `DstAS` UInt32, `SrcNetName` LowCardinality(String), `DstNetName` LowCardinality(String), `SrcNetRole` LowCardinality(String), `DstNetRole` LowCardinality(String), `SrcNetSite` LowCardinality(String), `DstNetSite` LowCardinality(String), `SrcNetRegion` LowCardinality(String), `DstNetRegion` LowCardinality(String), `SrcNetTenant` LowCardinality(String), `DstNetTenant` LowCardinality(String), `SrcCountry` FixedString(2), `DstCountry` FixedString(2), `SrcGeoCity` LowCardinality(String), `DstGeoCity` LowCardinality(String), `SrcGeoState` LowCardinality(String), `DstGeoState` LowCardinality(String), `Dst1stAS` UInt32, `Dst2ndAS` UInt32, `Dst3rdAS` UInt32, `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `Bytes` UInt64 CODEC(T64, LZ4), `Packets` UInt64 CODEC(T64, LZ4), `PacketSize` UInt64 ALIAS intDiv(Bytes, Packets), `PacketSizeBucket` LowCardinality(String) ALIAS multiIf(PacketSize < 64, '0-63', PacketSize < 128, '64-127', PacketSize < 256, '128-255', PacketSize < 512, '256-511', PacketSize < 768, '512-767', PacketSize < 1024, '768-1023', PacketSize < 1280, '1024-1279', PacketSize < 1501, '1280-1500', PacketSize < 2048, '1501-2047', PacketSize < 3072, '2048-3071', PacketSize < 4096, '3072-4095', PacketSize < 8192, '4096-8191', PacketSize < 10240, '8192-10239', PacketSize < 16384, '10240-16383', PacketSize < 32768, '16384-32767', PacketSize < 65536, '32768-65535', '65536-Inf'), `ForwardingStatus` UInt32) ENGINE = ReplicatedSummingMergeTree('/clickhouse/tables/shard-{shard}/flows_5m0s_local', 'replica-{replica}', (Bytes, Packets)) PARTITION BY toYYYYMMDDhhmmss(toStartOfInterval(TimeReceived, toIntervalSecond(155520))) PRIMARY KEY (TimeReceived, ExporterAddress, EType, Proto, InIfName, SrcAS, ForwardingStatus, OutIfName, DstAS, SamplingRate) ORDER BY (TimeReceived, ExporterAddress, EType, Proto, InIfName, SrcAS, ForwardingStatus, OutIfName, DstAS, SamplingRate, SrcNetName, DstNetName, SrcNetRole, DstNetRole, SrcNetSite, DstNetSite, SrcNetRegion, DstNetRegion, SrcNetTenant, DstNetTenant, SrcCountry, DstCountry, SrcGeoCity, DstGeoCity, SrcGeoState, DstGeoState, Dst1stAS, Dst2ndAS, Dst3rdAS) TTL TimeReceived + toIntervalSecond(7776000) SETTINGS index_granularity = 8192, ttl_only_drop_parts = 1"
+flows_I6D3KDQCRUBCNCGF4BSOWTRMVIv5_raw,"CREATE TABLE default.flows_I6D3KDQCRUBCNCGF4BSOWTRMVIv5_raw (`TimeReceived` DateTime CODEC(DoubleDelta, LZ4), `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAddr` IPv6 CODEC(ZSTD(1)), `DstAddr` IPv6 CODEC(ZSTD(1)), `SrcNetMask` UInt8, `DstNetMask` UInt8, `SrcAS` UInt32, `DstAS` UInt32, `DstASPath` Array(UInt32), `DstCommunities` Array(UInt32), `DstLargeCommunities` Array(UInt128), `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `SrcPort` UInt16, `DstPort` UInt16, `Bytes` UInt64 CODEC(T64, LZ4), `Packets` UInt64 CODEC(T64, LZ4), `ForwardingStatus` UInt32) ENGINE = Null"
+flows_local,"CREATE TABLE default.flows_local (`TimeReceived` DateTime CODEC(DoubleDelta, LZ4), `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAddr` IPv6 CODEC(ZSTD(1)), `DstAddr` IPv6 CODEC(ZSTD(1)), `SrcNetMask` UInt8, `DstNetMask` UInt8, `SrcNetPrefix` String ALIAS multiIf(EType = 2048, concat(replaceRegexpOne(CAST(IPv6CIDRToRange(SrcAddr, CAST(96 + SrcNetMask, 'UInt8')).1, 'String'), '^::ffff:', ''), '/', CAST(SrcNetMask, 'String')), EType = 34525, concat(CAST(IPv6CIDRToRange(SrcAddr, SrcNetMask).1, 'String'), '/', CAST(SrcNetMask, 'String')), ''), `DstNetPrefix` String ALIAS multiIf(EType = 2048, concat(replaceRegexpOne(CAST(IPv6CIDRToRange(DstAddr, CAST(96 + DstNetMask, 'UInt8')).1, 'String'), '^::ffff:', ''), '/', CAST(DstNetMask, 'String')), EType = 34525, concat(CAST(IPv6CIDRToRange(DstAddr, DstNetMask).1, 'String'), '/', CAST(DstNetMask, 'String')), ''), `SrcAS` UInt32, `DstAS` UInt32, `SrcNetName` LowCardinality(String), `DstNetName` LowCardinality(String), `SrcNetRole` LowCardinality(String), `DstNetRole` LowCardinality(String), `SrcNetSite` LowCardinality(String), `DstNetSite` LowCardinality(String), `SrcNetRegion` LowCardinality(String), `DstNetRegion` LowCardinality(String), `SrcNetTenant` LowCardinality(String), `DstNetTenant` LowCardinality(String), `SrcCountry` FixedString(2), `DstCountry` FixedString(2), `SrcGeoCity` LowCardinality(String), `DstGeoCity` LowCardinality(String), `SrcGeoState` LowCardinality(String), `DstGeoState` LowCardinality(String), `DstASPath` Array(UInt32), `Dst1stAS` UInt32, `Dst2ndAS` UInt32, `Dst3rdAS` UInt32, `DstCommunities` Array(UInt32), `DstLargeCommunities` Array(UInt128), `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `SrcPort` UInt16, `DstPort` UInt16, `Bytes` UInt64 CODEC(T64, LZ4), `Packets` UInt64 CODEC(T64, LZ4), `PacketSize` UInt64 ALIAS intDiv(Bytes, Packets), `PacketSizeBucket` LowCardinality(String) ALIAS multiIf(PacketSize < 64, '0-63', PacketSize < 128, '64-127', PacketSize < 256, '128-255', PacketSize < 512, '256-511', PacketSize < 768, '512-767', PacketSize < 1024, '768-1023', PacketSize < 1280, '1024-1279', PacketSize < 1501, '1280-1500', PacketSize < 2048, '1501-2047', PacketSize < 3072, '2048-3071', PacketSize < 4096, '3072-4095', PacketSize < 8192, '4096-8191', PacketSize < 10240, '8192-10239', PacketSize < 16384, '10240-16383', PacketSize < 32768, '16384-32767', PacketSize < 65536, '32768-65535', '65536-Inf'), `ForwardingStatus` UInt32) ENGINE = ReplicatedMergeTree('/clickhouse/tables/shard-{shard}/flows_local', 'replica-{replica}') PARTITION BY toYYYYMMDDhhmmss(toStartOfInterval(TimeReceived, toIntervalSecond(25920))) ORDER BY (toStartOfFiveMinutes(TimeReceived), ExporterAddress, InIfName, OutIfName) TTL TimeReceived + toIntervalSecond(1296000) SETTINGS index_granularity = 8192, ttl_only_drop_parts = 1"
+flows,"CREATE TABLE default.flows (`TimeReceived` DateTime CODEC(DoubleDelta, LZ4), `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAddr` IPv6 CODEC(ZSTD(1)), `DstAddr` IPv6 CODEC(ZSTD(1)), `SrcNetMask` UInt8, `DstNetMask` UInt8, `SrcNetPrefix` String ALIAS multiIf(EType = 2048, concat(replaceRegexpOne(CAST(IPv6CIDRToRange(SrcAddr, CAST(96 + SrcNetMask, 'UInt8')).1, 'String'), '^::ffff:', ''), '/', CAST(SrcNetMask, 'String')), EType = 34525, concat(CAST(IPv6CIDRToRange(SrcAddr, SrcNetMask).1, 'String'), '/', CAST(SrcNetMask, 'String')), ''), `DstNetPrefix` String ALIAS multiIf(EType = 2048, concat(replaceRegexpOne(CAST(IPv6CIDRToRange(DstAddr, CAST(96 + DstNetMask, 'UInt8')).1, 'String'), '^::ffff:', ''), '/', CAST(DstNetMask, 'String')), EType = 34525, concat(CAST(IPv6CIDRToRange(DstAddr, DstNetMask).1, 'String'), '/', CAST(DstNetMask, 'String')), ''), `SrcAS` UInt32, `DstAS` UInt32, `SrcNetName` LowCardinality(String), `DstNetName` LowCardinality(String), `SrcNetRole` LowCardinality(String), `DstNetRole` LowCardinality(String), `SrcNetSite` LowCardinality(String), `DstNetSite` LowCardinality(String), `SrcNetRegion` LowCardinality(String), `DstNetRegion` LowCardinality(String), `SrcNetTenant` LowCardinality(String), `DstNetTenant` LowCardinality(String), `SrcCountry` FixedString(2), `DstCountry` FixedString(2), `SrcGeoCity` LowCardinality(String), `DstGeoCity` LowCardinality(String), `SrcGeoState` LowCardinality(String), `DstGeoState` LowCardinality(String), `DstASPath` Array(UInt32), `Dst1stAS` UInt32, `Dst2ndAS` UInt32, `Dst3rdAS` UInt32, `DstCommunities` Array(UInt32), `DstLargeCommunities` Array(UInt128), `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `SrcPort` UInt16, `DstPort` UInt16, `Bytes` UInt64 CODEC(T64, LZ4), `Packets` UInt64 CODEC(T64, LZ4), `PacketSize` UInt64 ALIAS intDiv(Bytes, Packets), `PacketSizeBucket` LowCardinality(String) ALIAS multiIf(PacketSize < 64, '0-63', PacketSize < 128, '64-127', PacketSize < 256, '128-255', PacketSize < 512, '256-511', PacketSize < 768, '512-767', PacketSize < 1024, '768-1023', PacketSize < 1280, '1024-1279', PacketSize < 1501, '1280-1500', PacketSize < 2048, '1501-2047', PacketSize < 3072, '2048-3071', PacketSize < 4096, '3072-4095', PacketSize < 8192, '4096-8191', PacketSize < 10240, '8192-10239', PacketSize < 16384, '10240-16383', PacketSize < 32768, '16384-32767', PacketSize < 65536, '32768-65535', '65536-Inf'), `ForwardingStatus` UInt32) ENGINE = Distributed('akvorado', 'default', 'flows_local', rand())"
+flows_1h0m0s,"CREATE TABLE default.flows_1h0m0s (`TimeReceived` DateTime CODEC(DoubleDelta, LZ4), `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAS` UInt32, `DstAS` UInt32, `SrcNetName` LowCardinality(String), `DstNetName` LowCardinality(String), `SrcNetRole` LowCardinality(String), `DstNetRole` LowCardinality(String), `SrcNetSite` LowCardinality(String), `DstNetSite` LowCardinality(String), `SrcNetRegion` LowCardinality(String), `DstNetRegion` LowCardinality(String), `SrcNetTenant` LowCardinality(String), `DstNetTenant` LowCardinality(String), `SrcCountry` FixedString(2), `DstCountry` FixedString(2), `SrcGeoCity` LowCardinality(String), `DstGeoCity` LowCardinality(String), `SrcGeoState` LowCardinality(String), `DstGeoState` LowCardinality(String), `Dst1stAS` UInt32, `Dst2ndAS` UInt32, `Dst3rdAS` UInt32, `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `Bytes` UInt64 CODEC(T64, LZ4), `Packets` UInt64 CODEC(T64, LZ4), `PacketSize` UInt64 ALIAS intDiv(Bytes, Packets), `PacketSizeBucket` LowCardinality(String) ALIAS multiIf(PacketSize < 64, '0-63', PacketSize < 128, '64-127', PacketSize < 256, '128-255', PacketSize < 512, '256-511', PacketSize < 768, '512-767', PacketSize < 1024, '768-1023', PacketSize < 1280, '1024-1279', PacketSize < 1501, '1280-1500', PacketSize < 2048, '1501-2047', PacketSize < 3072, '2048-3071', PacketSize < 4096, '3072-4095', PacketSize < 8192, '4096-8191', PacketSize < 10240, '8192-10239', PacketSize < 16384, '10240-16383', PacketSize < 32768, '16384-32767', PacketSize < 65536, '32768-65535', '65536-Inf'), `ForwardingStatus` UInt32) ENGINE = Distributed('akvorado', 'default', 'flows_1h0m0s_local', rand())"
+flows_1m0s,"CREATE TABLE default.flows_1m0s (`TimeReceived` DateTime CODEC(DoubleDelta, LZ4), `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAS` UInt32, `DstAS` UInt32, `SrcNetName` LowCardinality(String), `DstNetName` LowCardinality(String), `SrcNetRole` LowCardinality(String), `DstNetRole` LowCardinality(String), `SrcNetSite` LowCardinality(String), `DstNetSite` LowCardinality(String), `SrcNetRegion` LowCardinality(String), `DstNetRegion` LowCardinality(String), `SrcNetTenant` LowCardinality(String), `DstNetTenant` LowCardinality(String), `SrcCountry` FixedString(2), `DstCountry` FixedString(2), `SrcGeoCity` LowCardinality(String), `DstGeoCity` LowCardinality(String), `SrcGeoState` LowCardinality(String), `DstGeoState` LowCardinality(String), `Dst1stAS` UInt32, `Dst2ndAS` UInt32, `Dst3rdAS` UInt32, `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `Bytes` UInt64 CODEC(T64, LZ4), `Packets` UInt64 CODEC(T64, LZ4), `PacketSize` UInt64 ALIAS intDiv(Bytes, Packets), `PacketSizeBucket` LowCardinality(String) ALIAS multiIf(PacketSize < 64, '0-63', PacketSize < 128, '64-127', PacketSize < 256, '128-255', PacketSize < 512, '256-511', PacketSize < 768, '512-767', PacketSize < 1024, '768-1023', PacketSize < 1280, '1024-1279', PacketSize < 1501, '1280-1500', PacketSize < 2048, '1501-2047', PacketSize < 3072, '2048-3071', PacketSize < 4096, '3072-4095', PacketSize < 8192, '4096-8191', PacketSize < 10240, '8192-10239', PacketSize < 16384, '10240-16383', PacketSize < 32768, '16384-32767', PacketSize < 65536, '32768-65535', '65536-Inf'), `ForwardingStatus` UInt32) ENGINE = Distributed('akvorado', 'default', 'flows_1m0s_local', rand())"
+flows_5m0s,"CREATE TABLE default.flows_5m0s (`TimeReceived` DateTime CODEC(DoubleDelta, LZ4), `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAS` UInt32, `DstAS` UInt32, `SrcNetName` LowCardinality(String), `DstNetName` LowCardinality(String), `SrcNetRole` LowCardinality(String), `DstNetRole` LowCardinality(String), `SrcNetSite` LowCardinality(String), `DstNetSite` LowCardinality(String), `SrcNetRegion` LowCardinality(String), `DstNetRegion` LowCardinality(String), `SrcNetTenant` LowCardinality(String), `DstNetTenant` LowCardinality(String), `SrcCountry` FixedString(2), `DstCountry` FixedString(2), `SrcGeoCity` LowCardinality(String), `DstGeoCity` LowCardinality(String), `SrcGeoState` LowCardinality(String), `DstGeoState` LowCardinality(String), `Dst1stAS` UInt32, `Dst2ndAS` UInt32, `Dst3rdAS` UInt32, `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `Bytes` UInt64 CODEC(T64, LZ4), `Packets` UInt64 CODEC(T64, LZ4), `PacketSize` UInt64 ALIAS intDiv(Bytes, Packets), `PacketSizeBucket` LowCardinality(String) ALIAS multiIf(PacketSize < 64, '0-63', PacketSize < 128, '64-127', PacketSize < 256, '128-255', PacketSize < 512, '256-511', PacketSize < 768, '512-767', PacketSize < 1024, '768-1023', PacketSize < 1280, '1024-1279', PacketSize < 1501, '1280-1500', PacketSize < 2048, '1501-2047', PacketSize < 3072, '2048-3071', PacketSize < 4096, '3072-4095', PacketSize < 8192, '4096-8191', PacketSize < 10240, '8192-10239', PacketSize < 16384, '10240-16383', PacketSize < 32768, '16384-32767', PacketSize < 65536, '32768-65535', '65536-Inf'), `ForwardingStatus` UInt32) ENGINE = Distributed('akvorado', 'default', 'flows_5m0s_local', rand())"
+exporters_consumer,"CREATE MATERIALIZED VIEW default.exporters_consumer TO default.exporters (`TimeReceived` DateTime, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `IfName` String, `IfDescription` String, `IfSpeed` UInt32, `IfConnectivity` String, `IfProvider` String, `IfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2)) AS SELECT DISTINCT TimeReceived, ExporterAddress, ExporterName, ExporterGroup, ExporterRole, ExporterSite, ExporterRegion, ExporterTenant, [InIfName, OutIfName][num] AS IfName, [InIfDescription, OutIfDescription][num] AS IfDescription, [InIfSpeed, OutIfSpeed][num] AS IfSpeed, [InIfConnectivity, OutIfConnectivity][num] AS IfConnectivity, [InIfProvider, OutIfProvider][num] AS IfProvider, [InIfBoundary, OutIfBoundary][num] AS IfBoundary FROM default.flows ARRAY JOIN arrayEnumerate([1, 2]) AS num"
+flows_1h0m0s_consumer,"CREATE MATERIALIZED VIEW default.flows_1h0m0s_consumer TO default.flows_1h0m0s_local (`TimeReceived` DateTime, `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAS` UInt32, `DstAS` UInt32, `SrcNetName` LowCardinality(String), `DstNetName` LowCardinality(String), `SrcNetRole` LowCardinality(String), `DstNetRole` LowCardinality(String), `SrcNetSite` LowCardinality(String), `DstNetSite` LowCardinality(String), `SrcNetRegion` LowCardinality(String), `DstNetRegion` LowCardinality(String), `SrcNetTenant` LowCardinality(String), `DstNetTenant` LowCardinality(String), `SrcCountry` FixedString(2), `DstCountry` FixedString(2), `SrcGeoCity` LowCardinality(String), `DstGeoCity` LowCardinality(String), `SrcGeoState` LowCardinality(String), `DstGeoState` LowCardinality(String), `Dst1stAS` UInt32, `Dst2ndAS` UInt32, `Dst3rdAS` UInt32, `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `Bytes` UInt64, `Packets` UInt64, `ForwardingStatus` UInt32) AS SELECT toStartOfInterval(TimeReceived, toIntervalSecond(3600)) AS TimeReceived, SamplingRate, ExporterAddress, ExporterName, ExporterGroup, ExporterRole, ExporterSite, ExporterRegion, ExporterTenant, SrcAS, DstAS, SrcNetName, DstNetName, SrcNetRole, DstNetRole, SrcNetSite, DstNetSite, SrcNetRegion, DstNetRegion, SrcNetTenant, DstNetTenant, SrcCountry, DstCountry, SrcGeoCity, DstGeoCity, SrcGeoState, DstGeoState, Dst1stAS, Dst2ndAS, Dst3rdAS, InIfName, OutIfName, InIfDescription, OutIfDescription, InIfSpeed, OutIfSpeed, InIfConnectivity, OutIfConnectivity, InIfProvider, OutIfProvider, InIfBoundary, OutIfBoundary, EType, Proto, Bytes, Packets, ForwardingStatus FROM default.flows_local"
+flows_1m0s_consumer,"CREATE MATERIALIZED VIEW default.flows_1m0s_consumer TO default.flows_1m0s_local (`TimeReceived` DateTime, `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAS` UInt32, `DstAS` UInt32, `SrcNetName` LowCardinality(String), `DstNetName` LowCardinality(String), `SrcNetRole` LowCardinality(String), `DstNetRole` LowCardinality(String), `SrcNetSite` LowCardinality(String), `DstNetSite` LowCardinality(String), `SrcNetRegion` LowCardinality(String), `DstNetRegion` LowCardinality(String), `SrcNetTenant` LowCardinality(String), `DstNetTenant` LowCardinality(String), `SrcCountry` FixedString(2), `DstCountry` FixedString(2), `SrcGeoCity` LowCardinality(String), `DstGeoCity` LowCardinality(String), `SrcGeoState` LowCardinality(String), `DstGeoState` LowCardinality(String), `Dst1stAS` UInt32, `Dst2ndAS` UInt32, `Dst3rdAS` UInt32, `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `Bytes` UInt64, `Packets` UInt64, `ForwardingStatus` UInt32) AS SELECT toStartOfInterval(TimeReceived, toIntervalSecond(60)) AS TimeReceived, SamplingRate, ExporterAddress, ExporterName, ExporterGroup, ExporterRole, ExporterSite, ExporterRegion, ExporterTenant, SrcAS, DstAS, SrcNetName, DstNetName, SrcNetRole, DstNetRole, SrcNetSite, DstNetSite, SrcNetRegion, DstNetRegion, SrcNetTenant, DstNetTenant, SrcCountry, DstCountry, SrcGeoCity, DstGeoCity, SrcGeoState, DstGeoState, Dst1stAS, Dst2ndAS, Dst3rdAS, InIfName, OutIfName, InIfDescription, OutIfDescription, InIfSpeed, OutIfSpeed, InIfConnectivity, OutIfConnectivity, InIfProvider, OutIfProvider, InIfBoundary, OutIfBoundary, EType, Proto, Bytes, Packets, ForwardingStatus FROM default.flows_local"
+flows_5m0s_consumer,"CREATE MATERIALIZED VIEW default.flows_5m0s_consumer TO default.flows_5m0s_local (`TimeReceived` DateTime, `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAS` UInt32, `DstAS` UInt32, `SrcNetName` LowCardinality(String), `DstNetName` LowCardinality(String), `SrcNetRole` LowCardinality(String), `DstNetRole` LowCardinality(String), `SrcNetSite` LowCardinality(String), `DstNetSite` LowCardinality(String), `SrcNetRegion` LowCardinality(String), `DstNetRegion` LowCardinality(String), `SrcNetTenant` LowCardinality(String), `DstNetTenant` LowCardinality(String), `SrcCountry` FixedString(2), `DstCountry` FixedString(2), `SrcGeoCity` LowCardinality(String), `DstGeoCity` LowCardinality(String), `SrcGeoState` LowCardinality(String), `DstGeoState` LowCardinality(String), `Dst1stAS` UInt32, `Dst2ndAS` UInt32, `Dst3rdAS` UInt32, `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `Bytes` UInt64, `Packets` UInt64, `ForwardingStatus` UInt32) AS SELECT toStartOfInterval(TimeReceived, toIntervalSecond(300)) AS TimeReceived, SamplingRate, ExporterAddress, ExporterName, ExporterGroup, ExporterRole, ExporterSite, ExporterRegion, ExporterTenant, SrcAS, DstAS, SrcNetName, DstNetName, SrcNetRole, DstNetRole, SrcNetSite, DstNetSite, SrcNetRegion, DstNetRegion, SrcNetTenant, DstNetTenant, SrcCountry, DstCountry, SrcGeoCity, DstGeoCity, SrcGeoState, DstGeoState, Dst1stAS, Dst2ndAS, Dst3rdAS, InIfName, OutIfName, InIfDescription, OutIfDescription, InIfSpeed, OutIfSpeed, InIfConnectivity, OutIfConnectivity, InIfProvider, OutIfProvider, InIfBoundary, OutIfBoundary, EType, Proto, Bytes, Packets, ForwardingStatus FROM default.flows_local"
+flows_I6D3KDQCRUBCNCGF4BSOWTRMVIv5_raw_consumer,"CREATE MATERIALIZED VIEW default.flows_I6D3KDQCRUBCNCGF4BSOWTRMVIv5_raw_consumer TO default.flows (`TimeReceived` DateTime, `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAddr` IPv6, `DstAddr` IPv6, `SrcNetMask` UInt8, `DstNetMask` UInt8, `SrcAS` UInt32, `DstAS` UInt32, `SrcNetName` String, `DstNetName` String, `SrcNetRole` String, `DstNetRole` String, `SrcNetSite` String, `DstNetSite` String, `SrcNetRegion` String, `DstNetRegion` String, `SrcNetTenant` String, `DstNetTenant` String, `SrcCountry` String, `DstCountry` String, `SrcGeoCity` String, `DstGeoCity` String, `SrcGeoState` String, `DstGeoState` String, `DstASPath` Array(UInt32), `Dst1stAS` UInt32, `Dst2ndAS` UInt32, `Dst3rdAS` UInt32, `DstCommunities` Array(UInt32), `DstLargeCommunities` Array(UInt128), `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `SrcPort` UInt16, `DstPort` UInt16, `Bytes` UInt64, `Packets` UInt64, `ForwardingStatus` UInt32) AS WITH arrayCompact(DstASPath) AS c_DstASPath, dictGet('default.networks', ('asn', 'name', 'role', 'site', 'region', 'tenant', 'country', 'city', 'state'), SrcAddr) AS c_SrcNetworks, dictGet('default.networks', ('asn', 'name', 'role', 'site', 'region', 'tenant', 'country', 'city', 'state'), DstAddr) AS c_DstNetworks SELECT TimeReceived, SamplingRate, ExporterAddress, ExporterName, ExporterGroup, ExporterRole, ExporterSite, ExporterRegion, ExporterTenant, SrcAddr, DstAddr, SrcNetMask, DstNetMask, if(SrcAS = 0, c_SrcNetworks.1, SrcAS) AS SrcAS, if(DstAS = 0, c_DstNetworks.1, DstAS) AS DstAS, c_SrcNetworks.2 AS SrcNetName, c_DstNetworks.2 AS DstNetName, c_SrcNetworks.3 AS SrcNetRole, c_DstNetworks.3 AS DstNetRole, c_SrcNetworks.4 AS SrcNetSite, c_DstNetworks.4 AS DstNetSite, c_SrcNetworks.5 AS SrcNetRegion, c_DstNetworks.5 AS DstNetRegion, c_SrcNetworks.6 AS SrcNetTenant, c_DstNetworks.6 AS DstNetTenant, c_SrcNetworks.7 AS SrcCountry, c_DstNetworks.7 AS DstCountry, c_SrcNetworks.8 AS SrcGeoCity, c_DstNetworks.8 AS DstGeoCity, c_SrcNetworks.9 AS SrcGeoState, c_DstNetworks.9 AS DstGeoState, DstASPath, c_DstASPath[1] AS Dst1stAS, c_DstASPath[2] AS Dst2ndAS, c_DstASPath[3] AS Dst3rdAS, DstCommunities, DstLargeCommunities, InIfName, OutIfName, InIfDescription, OutIfDescription, InIfSpeed, OutIfSpeed, InIfConnectivity, OutIfConnectivity, InIfProvider, OutIfProvider, InIfBoundary, OutIfBoundary, EType, Proto, SrcPort, DstPort, Bytes, Packets, ForwardingStatus FROM default.flows_I6D3KDQCRUBCNCGF4BSOWTRMVIv5_raw"
diff --git a/orchestrator/clickhouse/testdata/states/011.csv b/orchestrator/clickhouse/testdata/states/011.csv
index cb3c06ae..862748b2 100644
--- a/orchestrator/clickhouse/testdata/states/011.csv
+++ b/orchestrator/clickhouse/testdata/states/011.csv
@@ -2,6 +2,8 @@ asns,"CREATE DICTIONARY default.asns (`asn` UInt32 INJECTIVE, `name` String) PRI
 icmp,"CREATE DICTIONARY default.icmp (`proto` UInt8, `type` UInt8, `code` UInt8, `name` String) PRIMARY KEY proto, type, code SOURCE(HTTP(URL 'http://127.0.0.1:0/api/v0/orchestrator/clickhouse/icmp.csv' FORMAT 'CSVWithNames')) LIFETIME(MIN 0 MAX 3600) LAYOUT(COMPLEX_KEY_HASHED()) SETTINGS(format_csv_allow_single_quotes = 0)"
 networks,"CREATE DICTIONARY default.networks (`network` String, `name` String, `role` String, `site` String, `region` String, `city` String, `state` String, `country` String, `tenant` String, `asn` UInt32) PRIMARY KEY network SOURCE(HTTP(URL 'http://127.0.0.1:0/api/v0/orchestrator/clickhouse/networks.csv' FORMAT 'CSVWithNames')) LIFETIME(MIN 0 MAX 3600) LAYOUT(IP_TRIE()) SETTINGS(format_csv_allow_single_quotes = 0)"
 protocols,"CREATE DICTIONARY default.protocols (`proto` UInt8 INJECTIVE, `name` String, `description` String) PRIMARY KEY proto SOURCE(HTTP(URL 'http://127.0.0.1:0/api/v0/orchestrator/clickhouse/protocols.csv' FORMAT 'CSVWithNames')) LIFETIME(MIN 0 MAX 3600) LAYOUT(HASHED()) SETTINGS(format_csv_allow_single_quotes = 0)"
+tcp,"CREATE DICTIONARY default.tcp (`port` UInt16 INJECTIVE, `name` String) PRIMARY KEY port SOURCE(HTTP(URL 'http://127.0.0.1:0/api/v0/orchestrator/clickhouse/tcp.csv' FORMAT 'CSVWithNames')) LIFETIME(MIN 0 MAX 3600) LAYOUT(HASHED()) SETTINGS(format_csv_allow_single_quotes = 0)"
+udp,"CREATE DICTIONARY default.udp (`port` UInt16 INJECTIVE, `name` String) PRIMARY KEY port SOURCE(HTTP(URL 'http://127.0.0.1:0/api/v0/orchestrator/clickhouse/udp.csv' FORMAT 'CSVWithNames')) LIFETIME(MIN 0 MAX 3600) LAYOUT(HASHED()) SETTINGS(format_csv_allow_single_quotes = 0)"
 exporters,"CREATE TABLE default.exporters (`TimeReceived` DateTime, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `IfName` LowCardinality(String), `IfDescription` LowCardinality(String), `IfSpeed` UInt32, `IfConnectivity` LowCardinality(String), `IfProvider` LowCardinality(String), `IfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2)) ENGINE = ReplacingMergeTree(TimeReceived) ORDER BY (ExporterAddress, IfName) TTL TimeReceived + toIntervalDay(1) SETTINGS index_granularity = 8192"
 flows,"CREATE TABLE default.flows (`TimeReceived` DateTime CODEC(DoubleDelta, LZ4), `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAddr` IPv6 CODEC(ZSTD(1)), `DstAddr` IPv6 CODEC(ZSTD(1)), `SrcNetMask` UInt8, `DstNetMask` UInt8, `SrcNetPrefix` String ALIAS multiIf(EType = 2048, concat(replaceRegexpOne(CAST(IPv6CIDRToRange(SrcAddr, CAST(96 + SrcNetMask, 'UInt8')).1, 'String'), '^::ffff:', ''), '/', CAST(SrcNetMask, 'String')), EType = 34525, concat(CAST(IPv6CIDRToRange(SrcAddr, SrcNetMask).1, 'String'), '/', CAST(SrcNetMask, 'String')), ''), `DstNetPrefix` String ALIAS multiIf(EType = 2048, concat(replaceRegexpOne(CAST(IPv6CIDRToRange(DstAddr, CAST(96 + DstNetMask, 'UInt8')).1, 'String'), '^::ffff:', ''), '/', CAST(DstNetMask, 'String')), EType = 34525, concat(CAST(IPv6CIDRToRange(DstAddr, DstNetMask).1, 'String'), '/', CAST(DstNetMask, 'String')), ''), `SrcAS` UInt32, `DstAS` UInt32, `SrcNetName` LowCardinality(String), `DstNetName` LowCardinality(String), `SrcNetRole` LowCardinality(String), `DstNetRole` LowCardinality(String), `SrcNetSite` LowCardinality(String), `DstNetSite` LowCardinality(String), `SrcNetRegion` LowCardinality(String), `DstNetRegion` LowCardinality(String), `SrcNetTenant` LowCardinality(String), `DstNetTenant` LowCardinality(String), `SrcCountry` FixedString(2), `DstCountry` FixedString(2), `SrcGeoCity` LowCardinality(String), `DstGeoCity` LowCardinality(String), `SrcGeoState` LowCardinality(String), `DstGeoState` LowCardinality(String), `DstASPath` Array(UInt32), `Dst1stAS` UInt32, `Dst2ndAS` UInt32, `Dst3rdAS` UInt32, `DstCommunities` Array(UInt32), `DstLargeCommunities` Array(UInt128), `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `SrcPort` UInt16, `DstPort` UInt16, `Bytes` UInt64 CODEC(T64, LZ4), `Packets` UInt64 CODEC(T64, LZ4), `PacketSize` UInt64 ALIAS intDiv(Bytes, Packets), `PacketSizeBucket` LowCardinality(String) ALIAS multiIf(PacketSize < 64, '0-63', PacketSize < 128, '64-127', PacketSize < 256, '128-255', PacketSize < 512, '256-511', PacketSize < 768, '512-767', PacketSize < 1024, '768-1023', PacketSize < 1280, '1024-1279', PacketSize < 1501, '1280-1500', PacketSize < 2048, '1501-2047', PacketSize < 3072, '2048-3071', PacketSize < 4096, '3072-4095', PacketSize < 8192, '4096-8191', PacketSize < 10240, '8192-10239', PacketSize < 16384, '10240-16383', PacketSize < 32768, '16384-32767', PacketSize < 65536, '32768-65535', '65536-Inf'), `ForwardingStatus` UInt32) ENGINE = MergeTree PARTITION BY toYYYYMMDDhhmmss(toStartOfInterval(TimeReceived, toIntervalSecond(25920))) ORDER BY (toStartOfFiveMinutes(TimeReceived), ExporterAddress, InIfName, OutIfName) TTL TimeReceived + toIntervalSecond(1296000) SETTINGS index_granularity = 8192, ttl_only_drop_parts = 1"
 flows_1h0m0s,"CREATE TABLE default.flows_1h0m0s (`TimeReceived` DateTime CODEC(DoubleDelta, LZ4), `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAS` UInt32, `DstAS` UInt32, `SrcNetName` LowCardinality(String), `DstNetName` LowCardinality(String), `SrcNetRole` LowCardinality(String), `DstNetRole` LowCardinality(String), `SrcNetSite` LowCardinality(String), `DstNetSite` LowCardinality(String), `SrcNetRegion` LowCardinality(String), `DstNetRegion` LowCardinality(String), `SrcNetTenant` LowCardinality(String), `DstNetTenant` LowCardinality(String), `SrcCountry` FixedString(2), `DstCountry` FixedString(2), `SrcGeoCity` LowCardinality(String), `DstGeoCity` LowCardinality(String), `SrcGeoState` LowCardinality(String), `DstGeoState` LowCardinality(String), `Dst1stAS` UInt32, `Dst2ndAS` UInt32, `Dst3rdAS` UInt32, `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `Bytes` UInt64 CODEC(T64, LZ4), `Packets` UInt64 CODEC(T64, LZ4), `PacketSize` UInt64 ALIAS intDiv(Bytes, Packets), `PacketSizeBucket` LowCardinality(String) ALIAS multiIf(PacketSize < 64, '0-63', PacketSize < 128, '64-127', PacketSize < 256, '128-255', PacketSize < 512, '256-511', PacketSize < 768, '512-767', PacketSize < 1024, '768-1023', PacketSize < 1280, '1024-1279', PacketSize < 1501, '1280-1500', PacketSize < 2048, '1501-2047', PacketSize < 3072, '2048-3071', PacketSize < 4096, '3072-4095', PacketSize < 8192, '4096-8191', PacketSize < 10240, '8192-10239', PacketSize < 16384, '10240-16383', PacketSize < 32768, '16384-32767', PacketSize < 65536, '32768-65535', '65536-Inf'), `ForwardingStatus` UInt32) ENGINE = SummingMergeTree((Bytes, Packets)) PARTITION BY toYYYYMMDDhhmmss(toStartOfInterval(TimeReceived, toIntervalSecond(622080))) PRIMARY KEY (TimeReceived, ExporterAddress, EType, Proto, InIfName, SrcAS, ForwardingStatus, OutIfName, DstAS, SamplingRate) ORDER BY (TimeReceived, ExporterAddress, EType, Proto, InIfName, SrcAS, ForwardingStatus, OutIfName, DstAS, SamplingRate, SrcNetName, DstNetName, SrcNetRole, DstNetRole, SrcNetSite, DstNetSite, SrcNetRegion, DstNetRegion, SrcNetTenant, DstNetTenant, SrcCountry, DstCountry, SrcGeoCity, DstGeoCity, SrcGeoState, DstGeoState, Dst1stAS, Dst2ndAS, Dst3rdAS) TTL TimeReceived + toIntervalSecond(31104000) SETTINGS index_granularity = 8192, ttl_only_drop_parts = 1"
@@ -15,5 +17,3 @@ flows_1m0s_consumer,"CREATE MATERIALIZED VIEW default.flows_1m0s_consumer TO def
 flows_5m0s_consumer,"CREATE MATERIALIZED VIEW default.flows_5m0s_consumer TO default.flows_5m0s (`TimeReceived` DateTime, `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAS` UInt32, `DstAS` UInt32, `SrcNetName` LowCardinality(String), `DstNetName` LowCardinality(String), `SrcNetRole` LowCardinality(String), `DstNetRole` LowCardinality(String), `SrcNetSite` LowCardinality(String), `DstNetSite` LowCardinality(String), `SrcNetRegion` LowCardinality(String), `DstNetRegion` LowCardinality(String), `SrcNetTenant` LowCardinality(String), `DstNetTenant` LowCardinality(String), `SrcCountry` FixedString(2), `DstCountry` FixedString(2), `SrcGeoCity` LowCardinality(String), `DstGeoCity` LowCardinality(String), `SrcGeoState` LowCardinality(String), `DstGeoState` LowCardinality(String), `Dst1stAS` UInt32, `Dst2ndAS` UInt32, `Dst3rdAS` UInt32, `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `Bytes` UInt64, `Packets` UInt64, `ForwardingStatus` UInt32) AS SELECT toStartOfInterval(TimeReceived, toIntervalSecond(300)) AS TimeReceived, SamplingRate, ExporterAddress, ExporterName, ExporterGroup, ExporterRole, ExporterSite, ExporterRegion, ExporterTenant, SrcAS, DstAS, SrcNetName, DstNetName, SrcNetRole, DstNetRole, SrcNetSite, DstNetSite, SrcNetRegion, DstNetRegion, SrcNetTenant, DstNetTenant, SrcCountry, DstCountry, SrcGeoCity, DstGeoCity, SrcGeoState, DstGeoState, Dst1stAS, Dst2ndAS, Dst3rdAS, InIfName, OutIfName, InIfDescription, OutIfDescription, InIfSpeed, OutIfSpeed, InIfConnectivity, OutIfConnectivity, InIfProvider, OutIfProvider, InIfBoundary, OutIfBoundary, EType, Proto, Bytes, Packets, ForwardingStatus FROM default.flows"
 flows_LAABIGYMRYZPTGOYIIFZNYDEQM_raw_consumer,"CREATE MATERIALIZED VIEW default.flows_LAABIGYMRYZPTGOYIIFZNYDEQM_raw_consumer TO default.flows (`TimeReceived` DateTime, `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAddr` IPv6, `DstAddr` IPv6, `SrcNetMask` UInt8, `DstNetMask` UInt8, `SrcAS` UInt32, `DstAS` UInt32, `SrcNetName` String, `DstNetName` String, `SrcNetRole` String, `DstNetRole` String, `SrcNetSite` String, `DstNetSite` String, `SrcNetRegion` String, `DstNetRegion` String, `SrcNetTenant` String, `DstNetTenant` String, `SrcCountry` String, `DstCountry` String, `SrcGeoCity` String, `DstGeoCity` String, `SrcGeoState` String, `DstGeoState` String, `DstASPath` Array(UInt32), `Dst1stAS` UInt32, `Dst2ndAS` UInt32, `Dst3rdAS` UInt32, `DstCommunities` Array(UInt32), `DstLargeCommunities` Array(UInt128), `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `SrcPort` UInt16, `DstPort` UInt16, `Bytes` UInt64, `Packets` UInt64, `ForwardingStatus` UInt32) AS WITH arrayCompact(DstASPath) AS c_DstASPath, dictGet('default.networks', ('asn', 'name', 'role', 'site', 'region', 'tenant', 'country', 'city', 'state'), SrcAddr) AS c_SrcNetworks, dictGet('default.networks', ('asn', 'name', 'role', 'site', 'region', 'tenant', 'country', 'city', 'state'), DstAddr) AS c_DstNetworks SELECT TimeReceived, SamplingRate, ExporterAddress, ExporterName, ExporterGroup, ExporterRole, ExporterSite, ExporterRegion, ExporterTenant, SrcAddr, DstAddr, SrcNetMask, DstNetMask, if(SrcAS = 0, c_SrcNetworks.1, SrcAS) AS SrcAS, if(DstAS = 0, c_DstNetworks.1, DstAS) AS DstAS, c_SrcNetworks.2 AS SrcNetName, c_DstNetworks.2 AS DstNetName, c_SrcNetworks.3 AS SrcNetRole, c_DstNetworks.3 AS DstNetRole, c_SrcNetworks.4 AS SrcNetSite, c_DstNetworks.4 AS DstNetSite, c_SrcNetworks.5 AS SrcNetRegion, c_DstNetworks.5 AS DstNetRegion, c_SrcNetworks.6 AS SrcNetTenant, c_DstNetworks.6 AS DstNetTenant, c_SrcNetworks.7 AS SrcCountry, c_DstNetworks.7 AS DstCountry, c_SrcNetworks.8 AS SrcGeoCity, c_DstNetworks.8 AS DstGeoCity, c_SrcNetworks.9 AS SrcGeoState, c_DstNetworks.9 AS DstGeoState, DstASPath, c_DstASPath[1] AS Dst1stAS, c_DstASPath[2] AS Dst2ndAS, c_DstASPath[3] AS Dst3rdAS, DstCommunities, arrayMap((asn, l1, l2) -> ((bitShiftLeft(CAST(asn, 'UInt128'), 64) + bitShiftLeft(CAST(l1, 'UInt128'), 32)) + CAST(l2, 'UInt128')), DstLargeCommunitiesASN, DstLargeCommunitiesLocalData1, DstLargeCommunitiesLocalData2) AS DstLargeCommunities, InIfName, OutIfName, InIfDescription, OutIfDescription, InIfSpeed, OutIfSpeed, InIfConnectivity, OutIfConnectivity, InIfProvider, OutIfProvider, InIfBoundary, OutIfBoundary, EType, Proto, SrcPort, DstPort, Bytes, Packets, ForwardingStatus FROM default.flows_LAABIGYMRYZPTGOYIIFZNYDEQM_raw WHERE length(_error) = 0"
 flows_raw_errors_consumer,"CREATE MATERIALIZED VIEW default.flows_raw_errors_consumer TO default.flows_raw_errors (`timestamp` DateTime, `topic` LowCardinality(String), `partition` UInt64, `offset` UInt64, `raw` String, `error` String) AS SELECT now() AS timestamp, _topic AS topic, _partition AS partition, _offset AS offset, _raw_message AS raw, _error AS error FROM default.flows_LAABIGYMRYZPTGOYIIFZNYDEQM_raw WHERE length(_error) > 0"
-tcp,"CREATE DICTIONARY default.tcp (`port` UInt16 INJECTIVE, `name` String) PRIMARY KEY port SOURCE(HTTP(URL 'http://127.0.0.1:0/api/v0/orchestrator/clickhouse/tcp.csv' FORMAT 'CSVWithNames')) LIFETIME(MIN 0 MAX 3600) LAYOUT(HASHED()) SETTINGS(format_csv_allow_single_quotes = 0)"
-udp,"CREATE DICTIONARY default.udp (`port` UInt16 INJECTIVE, `name` String) PRIMARY KEY port SOURCE(HTTP(URL 'http://127.0.0.1:0/api/v0/orchestrator/clickhouse/udp.csv' FORMAT 'CSVWithNames')) LIFETIME(MIN 0 MAX 3600) LAYOUT(HASHED()) SETTINGS(format_csv_allow_single_quotes = 0)"
diff --git a/orchestrator/clickhouse/testdata/states/012.csv b/orchestrator/clickhouse/testdata/states/012.csv
new file mode 100644
index 00000000..23f817d7
--- /dev/null
+++ b/orchestrator/clickhouse/testdata/states/012.csv
@@ -0,0 +1,17 @@
+asns,"CREATE DICTIONARY default.asns (`asn` UInt32 INJECTIVE, `name` String) PRIMARY KEY asn SOURCE(HTTP(URL 'http://127.0.0.1:0/api/v0/orchestrator/clickhouse/asns.csv' FORMAT 'CSVWithNames')) LIFETIME(MIN 0 MAX 3600) LAYOUT(HASHED()) SETTINGS(format_csv_allow_single_quotes = 0)"
+icmp,"CREATE DICTIONARY default.icmp (`proto` UInt8, `type` UInt8, `code` UInt8, `name` String) PRIMARY KEY proto, type, code SOURCE(HTTP(URL 'http://127.0.0.1:0/api/v0/orchestrator/clickhouse/icmp.csv' FORMAT 'CSVWithNames')) LIFETIME(MIN 0 MAX 3600) LAYOUT(COMPLEX_KEY_HASHED()) SETTINGS(format_csv_allow_single_quotes = 0)"
+networks,"CREATE DICTIONARY default.networks (`network` String, `name` String, `role` String, `site` String, `region` String, `city` String, `state` String, `country` String, `tenant` String, `asn` UInt32) PRIMARY KEY network SOURCE(HTTP(URL 'http://127.0.0.1:0/api/v0/orchestrator/clickhouse/networks.csv' FORMAT 'CSVWithNames')) LIFETIME(MIN 0 MAX 3600) LAYOUT(IP_TRIE()) SETTINGS(format_csv_allow_single_quotes = 0)"
+protocols,"CREATE DICTIONARY default.protocols (`proto` UInt8 INJECTIVE, `name` String, `description` String) PRIMARY KEY proto SOURCE(HTTP(URL 'http://127.0.0.1:0/api/v0/orchestrator/clickhouse/protocols.csv' FORMAT 'CSVWithNames')) LIFETIME(MIN 0 MAX 3600) LAYOUT(HASHED()) SETTINGS(format_csv_allow_single_quotes = 0)"
+tcp,"CREATE DICTIONARY default.tcp (`port` UInt16 INJECTIVE, `name` String) PRIMARY KEY port SOURCE(HTTP(URL 'http://127.0.0.1:0/api/v0/orchestrator/clickhouse/tcp.csv' FORMAT 'CSVWithNames')) LIFETIME(MIN 0 MAX 3600) LAYOUT(HASHED()) SETTINGS(format_csv_allow_single_quotes = 0)"
+udp,"CREATE DICTIONARY default.udp (`port` UInt16 INJECTIVE, `name` String) PRIMARY KEY port SOURCE(HTTP(URL 'http://127.0.0.1:0/api/v0/orchestrator/clickhouse/udp.csv' FORMAT 'CSVWithNames')) LIFETIME(MIN 0 MAX 3600) LAYOUT(HASHED()) SETTINGS(format_csv_allow_single_quotes = 0)"
+exporters,"CREATE TABLE default.exporters (`TimeReceived` DateTime, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `IfName` LowCardinality(String), `IfDescription` LowCardinality(String), `IfSpeed` UInt32, `IfConnectivity` LowCardinality(String), `IfProvider` LowCardinality(String), `IfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2)) ENGINE = ReplacingMergeTree(TimeReceived) ORDER BY (ExporterAddress, IfName) TTL TimeReceived + toIntervalDay(1) SETTINGS index_granularity = 8192"
+flows,"CREATE TABLE default.flows (`TimeReceived` DateTime CODEC(DoubleDelta, LZ4), `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAddr` IPv6 CODEC(ZSTD(1)), `DstAddr` IPv6 CODEC(ZSTD(1)), `SrcNetMask` UInt8, `DstNetMask` UInt8, `SrcNetPrefix` String ALIAS multiIf(EType = 2048, concat(replaceRegexpOne(CAST(IPv6CIDRToRange(SrcAddr, CAST(96 + SrcNetMask, 'UInt8')).1, 'String'), '^::ffff:', ''), '/', CAST(SrcNetMask, 'String')), EType = 34525, concat(CAST(IPv6CIDRToRange(SrcAddr, SrcNetMask).1, 'String'), '/', CAST(SrcNetMask, 'String')), ''), `DstNetPrefix` String ALIAS multiIf(EType = 2048, concat(replaceRegexpOne(CAST(IPv6CIDRToRange(DstAddr, CAST(96 + DstNetMask, 'UInt8')).1, 'String'), '^::ffff:', ''), '/', CAST(DstNetMask, 'String')), EType = 34525, concat(CAST(IPv6CIDRToRange(DstAddr, DstNetMask).1, 'String'), '/', CAST(DstNetMask, 'String')), ''), `SrcAS` UInt32, `DstAS` UInt32, `SrcNetName` LowCardinality(String), `DstNetName` LowCardinality(String), `SrcNetRole` LowCardinality(String), `DstNetRole` LowCardinality(String), `SrcNetSite` LowCardinality(String), `DstNetSite` LowCardinality(String), `SrcNetRegion` LowCardinality(String), `DstNetRegion` LowCardinality(String), `SrcNetTenant` LowCardinality(String), `DstNetTenant` LowCardinality(String), `SrcCountry` FixedString(2), `DstCountry` FixedString(2), `SrcGeoCity` LowCardinality(String), `DstGeoCity` LowCardinality(String), `SrcGeoState` LowCardinality(String), `DstGeoState` LowCardinality(String), `DstASPath` Array(UInt32), `Dst1stAS` UInt32, `Dst2ndAS` UInt32, `Dst3rdAS` UInt32, `DstCommunities` Array(UInt32), `DstLargeCommunities` Array(UInt128), `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `SrcPort` UInt16, `DstPort` UInt16, `Bytes` UInt64 CODEC(T64, LZ4), `Packets` UInt64 CODEC(T64, LZ4), `PacketSize` UInt64 ALIAS intDiv(Bytes, Packets), `PacketSizeBucket` LowCardinality(String) ALIAS multiIf(PacketSize < 64, '0-63', PacketSize < 128, '64-127', PacketSize < 256, '128-255', PacketSize < 512, '256-511', PacketSize < 768, '512-767', PacketSize < 1024, '768-1023', PacketSize < 1280, '1024-1279', PacketSize < 1501, '1280-1500', PacketSize < 2048, '1501-2047', PacketSize < 3072, '2048-3071', PacketSize < 4096, '3072-4095', PacketSize < 8192, '4096-8191', PacketSize < 10240, '8192-10239', PacketSize < 16384, '10240-16383', PacketSize < 32768, '16384-32767', PacketSize < 65536, '32768-65535', '65536-Inf'), `ForwardingStatus` UInt32) ENGINE = MergeTree PARTITION BY toYYYYMMDDhhmmss(toStartOfInterval(TimeReceived, toIntervalSecond(25920))) ORDER BY (toStartOfFiveMinutes(TimeReceived), ExporterAddress, InIfName, OutIfName) TTL TimeReceived + toIntervalSecond(1296000) SETTINGS index_granularity = 8192, ttl_only_drop_parts = 1"
+flows_1h0m0s,"CREATE TABLE default.flows_1h0m0s (`TimeReceived` DateTime CODEC(DoubleDelta, LZ4), `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAS` UInt32, `DstAS` UInt32, `SrcNetName` LowCardinality(String), `DstNetName` LowCardinality(String), `SrcNetRole` LowCardinality(String), `DstNetRole` LowCardinality(String), `SrcNetSite` LowCardinality(String), `DstNetSite` LowCardinality(String), `SrcNetRegion` LowCardinality(String), `DstNetRegion` LowCardinality(String), `SrcNetTenant` LowCardinality(String), `DstNetTenant` LowCardinality(String), `SrcCountry` FixedString(2), `DstCountry` FixedString(2), `SrcGeoCity` LowCardinality(String), `DstGeoCity` LowCardinality(String), `SrcGeoState` LowCardinality(String), `DstGeoState` LowCardinality(String), `Dst1stAS` UInt32, `Dst2ndAS` UInt32, `Dst3rdAS` UInt32, `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `Bytes` UInt64 CODEC(T64, LZ4), `Packets` UInt64 CODEC(T64, LZ4), `PacketSize` UInt64 ALIAS intDiv(Bytes, Packets), `PacketSizeBucket` LowCardinality(String) ALIAS multiIf(PacketSize < 64, '0-63', PacketSize < 128, '64-127', PacketSize < 256, '128-255', PacketSize < 512, '256-511', PacketSize < 768, '512-767', PacketSize < 1024, '768-1023', PacketSize < 1280, '1024-1279', PacketSize < 1501, '1280-1500', PacketSize < 2048, '1501-2047', PacketSize < 3072, '2048-3071', PacketSize < 4096, '3072-4095', PacketSize < 8192, '4096-8191', PacketSize < 10240, '8192-10239', PacketSize < 16384, '10240-16383', PacketSize < 32768, '16384-32767', PacketSize < 65536, '32768-65535', '65536-Inf'), `ForwardingStatus` UInt32) ENGINE = SummingMergeTree((Bytes, Packets)) PARTITION BY toYYYYMMDDhhmmss(toStartOfInterval(TimeReceived, toIntervalSecond(622080))) PRIMARY KEY (TimeReceived, ExporterAddress, EType, Proto, InIfName, SrcAS, ForwardingStatus, OutIfName, DstAS, SamplingRate) ORDER BY (TimeReceived, ExporterAddress, EType, Proto, InIfName, SrcAS, ForwardingStatus, OutIfName, DstAS, SamplingRate, SrcNetName, DstNetName, SrcNetRole, DstNetRole, SrcNetSite, DstNetSite, SrcNetRegion, DstNetRegion, SrcNetTenant, DstNetTenant, SrcCountry, DstCountry, SrcGeoCity, DstGeoCity, SrcGeoState, DstGeoState, Dst1stAS, Dst2ndAS, Dst3rdAS) TTL TimeReceived + toIntervalSecond(31104000) SETTINGS index_granularity = 8192, ttl_only_drop_parts = 1"
+flows_1m0s,"CREATE TABLE default.flows_1m0s (`TimeReceived` DateTime CODEC(DoubleDelta, LZ4), `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAS` UInt32, `DstAS` UInt32, `SrcNetName` LowCardinality(String), `DstNetName` LowCardinality(String), `SrcNetRole` LowCardinality(String), `DstNetRole` LowCardinality(String), `SrcNetSite` LowCardinality(String), `DstNetSite` LowCardinality(String), `SrcNetRegion` LowCardinality(String), `DstNetRegion` LowCardinality(String), `SrcNetTenant` LowCardinality(String), `DstNetTenant` LowCardinality(String), `SrcCountry` FixedString(2), `DstCountry` FixedString(2), `SrcGeoCity` LowCardinality(String), `DstGeoCity` LowCardinality(String), `SrcGeoState` LowCardinality(String), `DstGeoState` LowCardinality(String), `Dst1stAS` UInt32, `Dst2ndAS` UInt32, `Dst3rdAS` UInt32, `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `Bytes` UInt64 CODEC(T64, LZ4), `Packets` UInt64 CODEC(T64, LZ4), `PacketSize` UInt64 ALIAS intDiv(Bytes, Packets), `PacketSizeBucket` LowCardinality(String) ALIAS multiIf(PacketSize < 64, '0-63', PacketSize < 128, '64-127', PacketSize < 256, '128-255', PacketSize < 512, '256-511', PacketSize < 768, '512-767', PacketSize < 1024, '768-1023', PacketSize < 1280, '1024-1279', PacketSize < 1501, '1280-1500', PacketSize < 2048, '1501-2047', PacketSize < 3072, '2048-3071', PacketSize < 4096, '3072-4095', PacketSize < 8192, '4096-8191', PacketSize < 10240, '8192-10239', PacketSize < 16384, '10240-16383', PacketSize < 32768, '16384-32767', PacketSize < 65536, '32768-65535', '65536-Inf'), `ForwardingStatus` UInt32) ENGINE = SummingMergeTree((Bytes, Packets)) PARTITION BY toYYYYMMDDhhmmss(toStartOfInterval(TimeReceived, toIntervalSecond(12096))) PRIMARY KEY (TimeReceived, ExporterAddress, EType, Proto, InIfName, SrcAS, ForwardingStatus, OutIfName, DstAS, SamplingRate) ORDER BY (TimeReceived, ExporterAddress, EType, Proto, InIfName, SrcAS, ForwardingStatus, OutIfName, DstAS, SamplingRate, SrcNetName, DstNetName, SrcNetRole, DstNetRole, SrcNetSite, DstNetSite, SrcNetRegion, DstNetRegion, SrcNetTenant, DstNetTenant, SrcCountry, DstCountry, SrcGeoCity, DstGeoCity, SrcGeoState, DstGeoState, Dst1stAS, Dst2ndAS, Dst3rdAS) TTL TimeReceived + toIntervalSecond(604800) SETTINGS index_granularity = 8192, ttl_only_drop_parts = 1"
+flows_5m0s,"CREATE TABLE default.flows_5m0s (`TimeReceived` DateTime CODEC(DoubleDelta, LZ4), `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAS` UInt32, `DstAS` UInt32, `SrcNetName` LowCardinality(String), `DstNetName` LowCardinality(String), `SrcNetRole` LowCardinality(String), `DstNetRole` LowCardinality(String), `SrcNetSite` LowCardinality(String), `DstNetSite` LowCardinality(String), `SrcNetRegion` LowCardinality(String), `DstNetRegion` LowCardinality(String), `SrcNetTenant` LowCardinality(String), `DstNetTenant` LowCardinality(String), `SrcCountry` FixedString(2), `DstCountry` FixedString(2), `SrcGeoCity` LowCardinality(String), `DstGeoCity` LowCardinality(String), `SrcGeoState` LowCardinality(String), `DstGeoState` LowCardinality(String), `Dst1stAS` UInt32, `Dst2ndAS` UInt32, `Dst3rdAS` UInt32, `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `Bytes` UInt64 CODEC(T64, LZ4), `Packets` UInt64 CODEC(T64, LZ4), `PacketSize` UInt64 ALIAS intDiv(Bytes, Packets), `PacketSizeBucket` LowCardinality(String) ALIAS multiIf(PacketSize < 64, '0-63', PacketSize < 128, '64-127', PacketSize < 256, '128-255', PacketSize < 512, '256-511', PacketSize < 768, '512-767', PacketSize < 1024, '768-1023', PacketSize < 1280, '1024-1279', PacketSize < 1501, '1280-1500', PacketSize < 2048, '1501-2047', PacketSize < 3072, '2048-3071', PacketSize < 4096, '3072-4095', PacketSize < 8192, '4096-8191', PacketSize < 10240, '8192-10239', PacketSize < 16384, '10240-16383', PacketSize < 32768, '16384-32767', PacketSize < 65536, '32768-65535', '65536-Inf'), `ForwardingStatus` UInt32) ENGINE = SummingMergeTree((Bytes, Packets)) PARTITION BY toYYYYMMDDhhmmss(toStartOfInterval(TimeReceived, toIntervalSecond(155520))) PRIMARY KEY (TimeReceived, ExporterAddress, EType, Proto, InIfName, SrcAS, ForwardingStatus, OutIfName, DstAS, SamplingRate) ORDER BY (TimeReceived, ExporterAddress, EType, Proto, InIfName, SrcAS, ForwardingStatus, OutIfName, DstAS, SamplingRate, SrcNetName, DstNetName, SrcNetRole, DstNetRole, SrcNetSite, DstNetSite, SrcNetRegion, DstNetRegion, SrcNetTenant, DstNetTenant, SrcCountry, DstCountry, SrcGeoCity, DstGeoCity, SrcGeoState, DstGeoState, Dst1stAS, Dst2ndAS, Dst3rdAS) TTL TimeReceived + toIntervalSecond(7776000) SETTINGS index_granularity = 8192, ttl_only_drop_parts = 1"
+flows_I6D3KDQCRUBCNCGF4BSOWTRMVIv5_raw,"CREATE TABLE default.flows_I6D3KDQCRUBCNCGF4BSOWTRMVIv5_raw (`TimeReceived` DateTime CODEC(DoubleDelta, LZ4), `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAddr` IPv6 CODEC(ZSTD(1)), `DstAddr` IPv6 CODEC(ZSTD(1)), `SrcNetMask` UInt8, `DstNetMask` UInt8, `SrcAS` UInt32, `DstAS` UInt32, `DstASPath` Array(UInt32), `DstCommunities` Array(UInt32), `DstLargeCommunities` Array(UInt128), `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `SrcPort` UInt16, `DstPort` UInt16, `Bytes` UInt64 CODEC(T64, LZ4), `Packets` UInt64 CODEC(T64, LZ4), `ForwardingStatus` UInt32) ENGINE = Null"
+exporters_consumer,"CREATE MATERIALIZED VIEW default.exporters_consumer TO default.exporters (`TimeReceived` DateTime, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `IfName` String, `IfDescription` String, `IfSpeed` UInt32, `IfConnectivity` String, `IfProvider` String, `IfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2)) AS SELECT DISTINCT TimeReceived, ExporterAddress, ExporterName, ExporterGroup, ExporterRole, ExporterSite, ExporterRegion, ExporterTenant, [InIfName, OutIfName][num] AS IfName, [InIfDescription, OutIfDescription][num] AS IfDescription, [InIfSpeed, OutIfSpeed][num] AS IfSpeed, [InIfConnectivity, OutIfConnectivity][num] AS IfConnectivity, [InIfProvider, OutIfProvider][num] AS IfProvider, [InIfBoundary, OutIfBoundary][num] AS IfBoundary FROM default.flows ARRAY JOIN arrayEnumerate([1, 2]) AS num"
+flows_1h0m0s_consumer,"CREATE MATERIALIZED VIEW default.flows_1h0m0s_consumer TO default.flows_1h0m0s (`TimeReceived` DateTime, `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAS` UInt32, `DstAS` UInt32, `SrcNetName` LowCardinality(String), `DstNetName` LowCardinality(String), `SrcNetRole` LowCardinality(String), `DstNetRole` LowCardinality(String), `SrcNetSite` LowCardinality(String), `DstNetSite` LowCardinality(String), `SrcNetRegion` LowCardinality(String), `DstNetRegion` LowCardinality(String), `SrcNetTenant` LowCardinality(String), `DstNetTenant` LowCardinality(String), `SrcCountry` FixedString(2), `DstCountry` FixedString(2), `SrcGeoCity` LowCardinality(String), `DstGeoCity` LowCardinality(String), `SrcGeoState` LowCardinality(String), `DstGeoState` LowCardinality(String), `Dst1stAS` UInt32, `Dst2ndAS` UInt32, `Dst3rdAS` UInt32, `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `Bytes` UInt64, `Packets` UInt64, `ForwardingStatus` UInt32) AS SELECT toStartOfInterval(TimeReceived, toIntervalSecond(3600)) AS TimeReceived, SamplingRate, ExporterAddress, ExporterName, ExporterGroup, ExporterRole, ExporterSite, ExporterRegion, ExporterTenant, SrcAS, DstAS, SrcNetName, DstNetName, SrcNetRole, DstNetRole, SrcNetSite, DstNetSite, SrcNetRegion, DstNetRegion, SrcNetTenant, DstNetTenant, SrcCountry, DstCountry, SrcGeoCity, DstGeoCity, SrcGeoState, DstGeoState, Dst1stAS, Dst2ndAS, Dst3rdAS, InIfName, OutIfName, InIfDescription, OutIfDescription, InIfSpeed, OutIfSpeed, InIfConnectivity, OutIfConnectivity, InIfProvider, OutIfProvider, InIfBoundary, OutIfBoundary, EType, Proto, Bytes, Packets, ForwardingStatus FROM default.flows"
+flows_1m0s_consumer,"CREATE MATERIALIZED VIEW default.flows_1m0s_consumer TO default.flows_1m0s (`TimeReceived` DateTime, `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAS` UInt32, `DstAS` UInt32, `SrcNetName` LowCardinality(String), `DstNetName` LowCardinality(String), `SrcNetRole` LowCardinality(String), `DstNetRole` LowCardinality(String), `SrcNetSite` LowCardinality(String), `DstNetSite` LowCardinality(String), `SrcNetRegion` LowCardinality(String), `DstNetRegion` LowCardinality(String), `SrcNetTenant` LowCardinality(String), `DstNetTenant` LowCardinality(String), `SrcCountry` FixedString(2), `DstCountry` FixedString(2), `SrcGeoCity` LowCardinality(String), `DstGeoCity` LowCardinality(String), `SrcGeoState` LowCardinality(String), `DstGeoState` LowCardinality(String), `Dst1stAS` UInt32, `Dst2ndAS` UInt32, `Dst3rdAS` UInt32, `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `Bytes` UInt64, `Packets` UInt64, `ForwardingStatus` UInt32) AS SELECT toStartOfInterval(TimeReceived, toIntervalSecond(60)) AS TimeReceived, SamplingRate, ExporterAddress, ExporterName, ExporterGroup, ExporterRole, ExporterSite, ExporterRegion, ExporterTenant, SrcAS, DstAS, SrcNetName, DstNetName, SrcNetRole, DstNetRole, SrcNetSite, DstNetSite, SrcNetRegion, DstNetRegion, SrcNetTenant, DstNetTenant, SrcCountry, DstCountry, SrcGeoCity, DstGeoCity, SrcGeoState, DstGeoState, Dst1stAS, Dst2ndAS, Dst3rdAS, InIfName, OutIfName, InIfDescription, OutIfDescription, InIfSpeed, OutIfSpeed, InIfConnectivity, OutIfConnectivity, InIfProvider, OutIfProvider, InIfBoundary, OutIfBoundary, EType, Proto, Bytes, Packets, ForwardingStatus FROM default.flows"
+flows_5m0s_consumer,"CREATE MATERIALIZED VIEW default.flows_5m0s_consumer TO default.flows_5m0s (`TimeReceived` DateTime, `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAS` UInt32, `DstAS` UInt32, `SrcNetName` LowCardinality(String), `DstNetName` LowCardinality(String), `SrcNetRole` LowCardinality(String), `DstNetRole` LowCardinality(String), `SrcNetSite` LowCardinality(String), `DstNetSite` LowCardinality(String), `SrcNetRegion` LowCardinality(String), `DstNetRegion` LowCardinality(String), `SrcNetTenant` LowCardinality(String), `DstNetTenant` LowCardinality(String), `SrcCountry` FixedString(2), `DstCountry` FixedString(2), `SrcGeoCity` LowCardinality(String), `DstGeoCity` LowCardinality(String), `SrcGeoState` LowCardinality(String), `DstGeoState` LowCardinality(String), `Dst1stAS` UInt32, `Dst2ndAS` UInt32, `Dst3rdAS` UInt32, `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `Bytes` UInt64, `Packets` UInt64, `ForwardingStatus` UInt32) AS SELECT toStartOfInterval(TimeReceived, toIntervalSecond(300)) AS TimeReceived, SamplingRate, ExporterAddress, ExporterName, ExporterGroup, ExporterRole, ExporterSite, ExporterRegion, ExporterTenant, SrcAS, DstAS, SrcNetName, DstNetName, SrcNetRole, DstNetRole, SrcNetSite, DstNetSite, SrcNetRegion, DstNetRegion, SrcNetTenant, DstNetTenant, SrcCountry, DstCountry, SrcGeoCity, DstGeoCity, SrcGeoState, DstGeoState, Dst1stAS, Dst2ndAS, Dst3rdAS, InIfName, OutIfName, InIfDescription, OutIfDescription, InIfSpeed, OutIfSpeed, InIfConnectivity, OutIfConnectivity, InIfProvider, OutIfProvider, InIfBoundary, OutIfBoundary, EType, Proto, Bytes, Packets, ForwardingStatus FROM default.flows"
+flows_I6D3KDQCRUBCNCGF4BSOWTRMVIv5_raw_consumer,"CREATE MATERIALIZED VIEW default.flows_I6D3KDQCRUBCNCGF4BSOWTRMVIv5_raw_consumer TO default.flows (`TimeReceived` DateTime, `SamplingRate` UInt64, `ExporterAddress` LowCardinality(IPv6), `ExporterName` LowCardinality(String), `ExporterGroup` LowCardinality(String), `ExporterRole` LowCardinality(String), `ExporterSite` LowCardinality(String), `ExporterRegion` LowCardinality(String), `ExporterTenant` LowCardinality(String), `SrcAddr` IPv6, `DstAddr` IPv6, `SrcNetMask` UInt8, `DstNetMask` UInt8, `SrcAS` UInt32, `DstAS` UInt32, `SrcNetName` String, `DstNetName` String, `SrcNetRole` String, `DstNetRole` String, `SrcNetSite` String, `DstNetSite` String, `SrcNetRegion` String, `DstNetRegion` String, `SrcNetTenant` String, `DstNetTenant` String, `SrcCountry` String, `DstCountry` String, `SrcGeoCity` String, `DstGeoCity` String, `SrcGeoState` String, `DstGeoState` String, `DstASPath` Array(UInt32), `Dst1stAS` UInt32, `Dst2ndAS` UInt32, `Dst3rdAS` UInt32, `DstCommunities` Array(UInt32), `DstLargeCommunities` Array(UInt128), `InIfName` LowCardinality(String), `OutIfName` LowCardinality(String), `InIfDescription` LowCardinality(String), `OutIfDescription` LowCardinality(String), `InIfSpeed` UInt32, `OutIfSpeed` UInt32, `InIfConnectivity` LowCardinality(String), `OutIfConnectivity` LowCardinality(String), `InIfProvider` LowCardinality(String), `OutIfProvider` LowCardinality(String), `InIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `OutIfBoundary` Enum8('undefined' = 0, 'external' = 1, 'internal' = 2), `EType` UInt32, `Proto` UInt32, `SrcPort` UInt16, `DstPort` UInt16, `Bytes` UInt64, `Packets` UInt64, `ForwardingStatus` UInt32) AS WITH arrayCompact(DstASPath) AS c_DstASPath, dictGet('default.networks', ('asn', 'name', 'role', 'site', 'region', 'tenant', 'country', 'city', 'state'), SrcAddr) AS c_SrcNetworks, dictGet('default.networks', ('asn', 'name', 'role', 'site', 'region', 'tenant', 'country', 'city', 'state'), DstAddr) AS c_DstNetworks SELECT TimeReceived, SamplingRate, ExporterAddress, ExporterName, ExporterGroup, ExporterRole, ExporterSite, ExporterRegion, ExporterTenant, SrcAddr, DstAddr, SrcNetMask, DstNetMask, if(SrcAS = 0, c_SrcNetworks.1, SrcAS) AS SrcAS, if(DstAS = 0, c_DstNetworks.1, DstAS) AS DstAS, c_SrcNetworks.2 AS SrcNetName, c_DstNetworks.2 AS DstNetName, c_SrcNetworks.3 AS SrcNetRole, c_DstNetworks.3 AS DstNetRole, c_SrcNetworks.4 AS SrcNetSite, c_DstNetworks.4 AS DstNetSite, c_SrcNetworks.5 AS SrcNetRegion, c_DstNetworks.5 AS DstNetRegion, c_SrcNetworks.6 AS SrcNetTenant, c_DstNetworks.6 AS DstNetTenant, c_SrcNetworks.7 AS SrcCountry, c_DstNetworks.7 AS DstCountry, c_SrcNetworks.8 AS SrcGeoCity, c_DstNetworks.8 AS DstGeoCity, c_SrcNetworks.9 AS SrcGeoState, c_DstNetworks.9 AS DstGeoState, DstASPath, c_DstASPath[1] AS Dst1stAS, c_DstASPath[2] AS Dst2ndAS, c_DstASPath[3] AS Dst3rdAS, DstCommunities, DstLargeCommunities, InIfName, OutIfName, InIfDescription, OutIfDescription, InIfSpeed, OutIfSpeed, InIfConnectivity, OutIfConnectivity, InIfProvider, OutIfProvider, InIfBoundary, OutIfBoundary, EType, Proto, SrcPort, DstPort, Bytes, Packets, ForwardingStatus FROM default.flows_I6D3KDQCRUBCNCGF4BSOWTRMVIv5_raw"
diff --git a/orchestrator/kafka/functional_test.go b/orchestrator/kafka/functional_test.go
index 0588d776..7e9167e4 100644
--- a/orchestrator/kafka/functional_test.go
+++ b/orchestrator/kafka/functional_test.go
@@ -4,27 +4,32 @@
 package kafka
 
 import (
+	"context"
 	"fmt"
 	"math/rand"
 	"testing"
+	"time"
 
-	"github.com/IBM/sarama"
+	"github.com/twmb/franz-go/pkg/kadm"
+	"github.com/twmb/franz-go/pkg/kmsg"
 
 	"akvorado/common/helpers"
 	"akvorado/common/kafka"
+	"akvorado/common/pb"
 	"akvorado/common/reporter"
 	"akvorado/common/schema"
 )
 
 func TestTopicCreation(t *testing.T) {
 	client, brokers := kafka.SetupKafkaBroker(t)
+	adminClient := kadm.NewClient(client)
 
 	topicName := fmt.Sprintf("test-topic-%d", rand.Int())
 	retentionMs := "76548"
 	segmentBytes := "107374184"
 	segmentBytes2 := "10737184"
 	cleanupPolicy := "delete"
-	expectedTopicName := fmt.Sprintf("%s-%s", topicName, schema.NewMock(t).ProtobufMessageHash())
+	expectedTopicName := fmt.Sprintf("%s-v%d", topicName, pb.Version)
 
 	cases := []struct {
 		Name          string
@@ -70,30 +75,45 @@ func TestTopicCreation(t *testing.T) {
 				ConfigEntriesStrictSync: true,
 			}
 			configuration.Brokers = brokers
-			configuration.Version = kafka.Version(sarama.V2_8_1_0)
+			// No version configuration needed for franz-go
 			c, err := New(reporter.NewMock(t), configuration, Dependencies{Schema: schema.NewMock(t)})
 			if err != nil {
 				t.Fatalf("New() error:\n%+v", err)
 			}
 			helpers.StartStop(t, c)
 
-			if err := client.RefreshMetadata(); err != nil {
-				t.Fatalf("RefreshMetadata() error:\n%+v", err)
-			}
-			adminClient, err := sarama.NewClusterAdminFromClient(client)
-			if err != nil {
-				t.Fatalf("NewClusterAdmin() error:\n%+v", err)
-			}
-			topics, err := adminClient.ListTopics()
-			if err != nil {
-				t.Fatalf("ListTopics() error:\n%+v", err)
-			}
-			topic, ok := topics[expectedTopicName]
-			if !ok {
-				t.Fatal("ListTopics() did not find the topic")
-			}
-			if diff := helpers.Diff(topic.ConfigEntries, tc.ConfigEntries); diff != "" {
-				t.Fatalf("ListTopics() (-got, +want):\n%s", diff)
+			deadline := time.Now().Add(1 * time.Second)
+			for {
+				topics, err := adminClient.ListTopics(t.Context())
+				if err != nil {
+					t.Fatalf("ListTopics() error:\n%+v", err)
+				}
+				_, ok := topics[expectedTopicName]
+				if !ok {
+					if time.Now().Before(deadline) {
+						time.Sleep(100 * time.Millisecond)
+						continue
+					}
+					t.Fatal("ListTopics() did not find the topic")
+				}
+				configs, err := adminClient.DescribeTopicConfigs(context.Background(), c.kafkaTopic)
+				if err != nil {
+					t.Fatalf("DescribeTopicConfigs() error:\n%+v", err)
+				}
+				got := map[string]*string{}
+				for _, config := range configs[0].Configs {
+					if config.Source != kmsg.ConfigSourceDefaultConfig {
+						got[config.Key] = config.Value
+					}
+				}
+				if diff := helpers.Diff(got, tc.ConfigEntries); diff != "" {
+					if time.Now().Before(deadline) {
+						time.Sleep(100 * time.Millisecond)
+						continue
+					}
+					t.Fatalf("ListTopics() (-got, +want):\n%s", diff)
+				}
+				break
 			}
 		})
 	}
@@ -101,9 +121,10 @@ func TestTopicCreation(t *testing.T) {
 
 func TestTopicMorePartitions(t *testing.T) {
 	client, brokers := kafka.SetupKafkaBroker(t)
+	adminClient := kadm.NewClient(client)
 
 	topicName := fmt.Sprintf("test-topic-%d", rand.Int())
-	expectedTopicName := fmt.Sprintf("%s-%s", topicName, schema.NewMock(t).ProtobufMessageHash())
+	expectedTopicName := fmt.Sprintf("%s-v%d", topicName, pb.Version)
 
 	configuration := DefaultConfiguration()
 	configuration.Topic = topicName
@@ -114,31 +135,36 @@ func TestTopicMorePartitions(t *testing.T) {
 	}
 
 	configuration.Brokers = brokers
-	configuration.Version = kafka.Version(sarama.V2_8_1_0)
+	// No version configuration needed for franz-go
 	c, err := New(reporter.NewMock(t), configuration, Dependencies{Schema: schema.NewMock(t)})
 	if err != nil {
 		t.Fatalf("New() error:\n%+v", err)
 	}
 	helpers.StartStop(t, c)
 
-	if err := client.RefreshMetadata(); err != nil {
-		t.Fatalf("RefreshMetadata() error:\n%+v", err)
-	}
-	adminClient, err := sarama.NewClusterAdminFromClient(client)
-	if err != nil {
-		t.Fatalf("NewClusterAdmin() error:\n%+v", err)
-	}
-	topics, err := adminClient.ListTopics()
-	if err != nil {
-		t.Fatalf("ListTopics() error:\n%+v", err)
-	}
-	topic, ok := topics[expectedTopicName]
-	if !ok {
-		t.Fatal("ListTopics() did not find the topic")
-	}
-	if topic.NumPartitions != 1 || topic.ReplicationFactor != 1 {
-		t.Fatalf("Topic does not have 1/1 for partitions/replication but %d/%d",
-			topic.NumPartitions, topic.ReplicationFactor)
+	deadline := time.Now().Add(1 * time.Second)
+	for {
+		topics, err := adminClient.ListTopics(t.Context())
+		if err != nil {
+			t.Fatalf("ListTopics() error:\n%+v", err)
+		}
+		topic, ok := topics[expectedTopicName]
+		if !ok {
+			if time.Now().Before(deadline) {
+				time.Sleep(100 * time.Millisecond)
+				continue
+			}
+			t.Fatal("ListTopics() did not find the topic")
+		}
+		if len(topic.Partitions) != 1 || topic.Partitions.NumReplicas() != 1 {
+			if time.Now().Before(deadline) {
+				time.Sleep(100 * time.Millisecond)
+				continue
+			}
+			t.Fatalf("Topic does not have 1/1 for partitions/replication but %d/%d",
+				len(topic.Partitions), topic.Partitions.NumReplicas())
+		}
+		break
 	}
 
 	// Increase number of partitions
@@ -149,17 +175,22 @@ func TestTopicMorePartitions(t *testing.T) {
 	}
 	helpers.StartStop(t, c)
 
-	if err := client.RefreshMetadata(); err != nil {
-		t.Fatalf("RefreshMetadata() error:\n%+v", err)
-	}
-	topics, err = adminClient.ListTopics()
-	if err != nil {
-		t.Fatalf("ListTopics() error:\n%+v", err)
-	}
-	topic = topics[expectedTopicName]
-	t.Logf("Topic configuration:\n%+v", topic)
-	if topic.NumPartitions != 4 || topic.ReplicationFactor != 1 {
-		t.Fatalf("Topic does not have 4/1 for partitions/replication but %d/%d",
-			topic.NumPartitions, topic.ReplicationFactor)
+	deadline = time.Now().Add(1 * time.Second)
+	for {
+		topics, err := adminClient.ListTopics(t.Context())
+		if err != nil {
+			t.Fatalf("ListTopics() error:\n%+v", err)
+		}
+		topic := topics[expectedTopicName]
+		t.Logf("Topic configuration:\n%+v", topic)
+		if len(topic.Partitions) != 4 || topic.Partitions.NumReplicas() != 1 {
+			if time.Now().Before(deadline) {
+				time.Sleep(100 * time.Millisecond)
+				continue
+			}
+			t.Fatalf("Topic does not have 4/1 for partitions/replication but %d/%d",
+				len(topic.Partitions), topic.Partitions.NumReplicas())
+		}
+		break
 	}
 }
diff --git a/outlet/clickhouse/example_test.go b/outlet/clickhouse/example_test.go
new file mode 100644
index 00000000..1b83d6ec
--- /dev/null
+++ b/outlet/clickhouse/example_test.go
@@ -0,0 +1,141 @@
+// SPDX-FileCopyrightText: 2016-2023 ClickHouse, Inc.
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileComment: This is basically a copy of https://github.com/ClickHouse/ch-go/blob/main/examples/insert/main.go
+
+package clickhouse_test
+
+import (
+	"context"
+	"io"
+	"testing"
+	"time"
+
+	"github.com/ClickHouse/ch-go"
+	"github.com/ClickHouse/ch-go/proto"
+
+	"akvorado/common/helpers"
+)
+
+func TestInsertMemory(t *testing.T) {
+	server := helpers.CheckExternalService(t, "ClickHouse", []string{"clickhouse:9000", "127.0.0.1:9000"})
+	ctx := context.Background()
+
+	conn, err := ch.Dial(ctx, ch.Options{
+		Address:     server,
+		Database:    "test",
+		DialTimeout: 100 * time.Millisecond,
+	})
+	if err != nil {
+		t.Fatalf("Dial() error:\n%+v", err)
+	}
+
+	if err := conn.Do(ctx, ch.Query{
+		Body: `CREATE OR REPLACE TABLE test_example_insert
+(
+    ts                DateTime64(9),
+    severity_text     Enum8('INFO'=1, 'DEBUG'=2),
+    severity_number   UInt8,
+	service_name      LowCardinality(String),
+    body              String,
+    name              String,
+    arr               Array(String)
+) ENGINE = Memory`,
+	}); err != nil {
+		t.Fatalf("Do() error:\n%+v", err)
+	}
+
+	// Define all columns of table.
+	var (
+		body      proto.ColStr
+		name      proto.ColStr
+		sevText   proto.ColEnum
+		sevNumber proto.ColUInt8
+
+		// or new(proto.ColStr).LowCardinality()
+		serviceName = proto.NewLowCardinality(new(proto.ColStr))
+		ts          = new(proto.ColDateTime64).WithPrecision(proto.PrecisionNano) // DateTime64(9)
+		arr         = new(proto.ColStr).Array()                                   // Array(String)
+		now         = time.Date(2010, 1, 1, 10, 22, 33, 345678, time.UTC)
+	)
+
+	input := proto.Input{
+		{Name: "ts", Data: ts},
+		{Name: "severity_text", Data: &sevText},
+		{Name: "severity_number", Data: &sevNumber},
+		{Name: "service_name", Data: serviceName},
+		{Name: "body", Data: &body},
+		{Name: "name", Data: &name},
+		{Name: "arr", Data: arr},
+	}
+
+	t.Run("one block", func(t *testing.T) {
+		// Append 10 rows to initial data block.
+		for range 10 {
+			body.AppendBytes([]byte("Hello"))
+			ts.Append(now)
+			name.Append("name")
+			sevText.Append("INFO")
+			sevNumber.Append(10)
+			arr.Append([]string{"foo", "bar", "baz"})
+			serviceName.Append("service")
+		}
+
+		// Insert single data block.
+		if err := conn.Do(ctx, ch.Query{
+			Body:  "INSERT INTO test_example_insert VALUES",
+			Input: input,
+		}); err != nil {
+			t.Fatalf("Do() error:\n%+v", err)
+		}
+	})
+
+	t.Run("streaming", func(t *testing.T) {
+		// Stream data to ClickHouse server in multiple data blocks.
+		var blocks int
+		if err := conn.Do(ctx, ch.Query{
+			Body:  input.Into("test_example_insert"), // helper that generates INSERT INTO query with all columns
+			Input: input,
+
+			// OnInput is called to prepare Input data before encoding and sending
+			// to ClickHouse server.
+			OnInput: func(ctx context.Context) error {
+				// On OnInput call, you should fill the input data.
+				//
+				// NB: You should reset the input columns, they are
+				// not reset automatically.
+				//
+				// That is, we are re-using the same input columns and
+				// if we will return nil without doing anything, data will be
+				// just duplicated.
+
+				input.Reset() // calls "Reset" on each column
+
+				if blocks >= 10 {
+					// Stop streaming.
+					//
+					// This will also write tailing input data if any,
+					// but we just reset the input, so it is currently blank.
+					return io.EOF
+				}
+
+				// Append new values:
+				for range 10 {
+					body.AppendBytes([]byte("Hello"))
+					ts.Append(now)
+					name.Append("name")
+					sevText.Append("DEBUG")
+					sevNumber.Append(10)
+					arr.Append([]string{"foo", "bar", "baz"})
+					serviceName.Append("service")
+				}
+
+				// Data will be encoded and sent to ClickHouse server after returning nil.
+				// The Do method will return error if any.
+				blocks++
+				return nil
+			},
+		}); err != nil {
+			t.Fatalf("Do() error:\n%+v", err)
+		}
+	})
+}
diff --git a/outlet/clickhouse/functional_test.go b/outlet/clickhouse/functional_test.go
new file mode 100644
index 00000000..4fbb3c11
--- /dev/null
+++ b/outlet/clickhouse/functional_test.go
@@ -0,0 +1,212 @@
+// SPDX-FileCopyrightText: 2025 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+package clickhouse_test
+
+import (
+	"context"
+	"fmt"
+	"testing"
+	"time"
+
+	clickhousego "github.com/ClickHouse/clickhouse-go/v2"
+
+	"akvorado/common/clickhousedb"
+	"akvorado/common/daemon"
+	"akvorado/common/helpers"
+	"akvorado/common/reporter"
+	"akvorado/common/schema"
+	"akvorado/outlet/clickhouse"
+)
+
+func TestInsert(t *testing.T) {
+	server := helpers.CheckExternalService(t, "ClickHouse", []string{"clickhouse:9000", "127.0.0.1:9000"})
+	r := reporter.NewMock(t)
+	sch := schema.NewMock(t)
+	bf := sch.NewFlowMessage()
+	ctx, cancel := context.WithTimeout(t.Context(), 5*time.Second)
+	defer cancel()
+	ctx = clickhousego.Context(ctx, clickhousego.WithSettings(clickhousego.Settings{
+		"allow_suspicious_low_cardinality_types": 1,
+	}))
+
+	// Create components
+	dbConf := clickhousedb.DefaultConfiguration()
+	dbConf.Servers = []string{server}
+	dbConf.Database = "test"
+	dbConf.DialTimeout = 100 * time.Millisecond
+	chdb, err := clickhousedb.New(r, dbConf, clickhousedb.Dependencies{
+		Daemon: daemon.NewMock(t),
+	})
+	if err != nil {
+		t.Fatalf("clickhousedb.New() error:\n%+v", err)
+	}
+	helpers.StartStop(t, chdb)
+	conf := clickhouse.DefaultConfiguration()
+	conf.MaximumBatchSize = 10
+	conf.MaximumWaitTime = time.Second
+	ch, err := clickhouse.New(r, conf, clickhouse.Dependencies{
+		ClickHouse: chdb,
+		Schema:     sch,
+	})
+	if err != nil {
+		t.Fatalf("clickhouse.New() error:\n%+v", err)
+	}
+	helpers.StartStop(t, ch)
+
+	// Create table
+	tableName := fmt.Sprintf("flows_%s_raw", sch.ClickHouseHash())
+	err = chdb.Exec(ctx, fmt.Sprintf("CREATE OR REPLACE TABLE %s (%s) ENGINE = Memory", tableName,
+		sch.ClickHouseCreateTable(
+			schema.ClickHouseSkipGeneratedColumns,
+			schema.ClickHouseSkipAliasedColumns)))
+	if err != nil {
+		t.Fatalf("chdb.Exec() error:\n%+v", err)
+	}
+	// Drop any left-over consumer (from orchestrator tests). Otherwise, we get an error like this:
+	// Bad URI syntax: bad or invalid port number: 0
+	err = chdb.Exec(ctx, fmt.Sprintf("DROP TABLE IF EXISTS %s_consumer", tableName))
+	if err != nil {
+		t.Fatalf("chdb.Exec() error:\n%+v", err)
+	}
+
+	// Expected records
+	type result struct {
+		TimeReceived time.Time
+		SrcAS        uint32
+		DstAS        uint32
+		ExporterName string
+		EType        uint32
+	}
+	expected := []result{}
+
+	// Create one worker and send some values
+	w := ch.NewWorker(1, bf)
+	for i := range 23 {
+		i = i + 1
+		// 1: first batch (max time)
+		// 2 to 11: second batch (max batch)
+		// 12 to 15: third batch (max time)
+		// 16 to 23: third batch (last one)
+		bf.TimeReceived = uint32(100 + i)
+		bf.SrcAS = uint32(65400 + i)
+		bf.DstAS = uint32(65500 + i)
+		bf.AppendString(schema.ColumnExporterName, fmt.Sprintf("exporter-%d", i))
+		bf.AppendString(schema.ColumnExporterName, "emptyness")
+		bf.AppendUint(schema.ColumnEType, helpers.ETypeIPv6)
+		expected = append(expected, result{
+			TimeReceived: time.Unix(int64(bf.TimeReceived), 0).UTC(),
+			SrcAS:        bf.SrcAS,
+			DstAS:        bf.DstAS,
+			ExporterName: fmt.Sprintf("exporter-%d", i),
+			EType:        helpers.ETypeIPv6,
+		})
+		if i == 15 {
+			time.Sleep(time.Second)
+		}
+		w.FinalizeAndSend(ctx)
+		if i == 23 {
+			w.Flush(ctx)
+		}
+
+		// Check metrics
+		gotMetrics := r.GetMetrics("akvorado_outlet_clickhouse_", "-insert_time", "-wait_time")
+		var expectedMetrics map[string]string
+		if i < 11 {
+			expectedMetrics = map[string]string{
+				"batches_total": "1",
+				"flows_total":   "1",
+			}
+		} else if i < 15 {
+			expectedMetrics = map[string]string{
+				"batches_total": "2",
+				"flows_total":   "11",
+			}
+		} else if i < 23 {
+			expectedMetrics = map[string]string{
+				"batches_total": "3",
+				"flows_total":   "15",
+			}
+		} else {
+			expectedMetrics = map[string]string{
+				"batches_total": "4",
+				"flows_total":   "23",
+			}
+		}
+		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
+			t.Errorf("Metrics, iteration %d, (-got, +want):\n%s", i, diff)
+		}
+
+		// Check if we have anything inserted in the table
+		var results []result
+		err := chdb.Select(ctx, &results,
+			fmt.Sprintf("SELECT TimeReceived, SrcAS, DstAS, ExporterName, EType FROM %s ORDER BY TimeReceived ASC", tableName))
+		if err != nil {
+			t.Fatalf("chdb.Select() error:\n%+v", err)
+		}
+		reallyExpected := expected
+		if i < 11 {
+			reallyExpected = expected[:min(len(expected), 1)]
+		} else if i < 15 {
+			reallyExpected = expected[:min(len(expected), 11)]
+		} else if i < 23 {
+			reallyExpected = expected[:min(len(expected), 15)]
+		}
+		if diff := helpers.Diff(results, reallyExpected); diff != "" {
+			t.Fatalf("chdb.Select(), iteration %d, (-got, +want):\n%s", i, diff)
+		}
+	}
+}
+
+func TestMultipleServers(t *testing.T) {
+	servers := []string{
+		helpers.CheckExternalService(t, "ClickHouse", []string{"clickhouse:9000", "127.0.0.1:9000"}),
+	}
+	for range 100 {
+		servers = append(servers, "127.0.0.1:0")
+	}
+	for range 10 {
+		r := reporter.NewMock(t)
+		sch := schema.NewMock(t)
+		ctx, cancel := context.WithTimeout(t.Context(), 5*time.Second)
+		defer cancel()
+		ctx = clickhousego.Context(ctx, clickhousego.WithSettings(clickhousego.Settings{
+			"allow_suspicious_low_cardinality_types": 1,
+		}))
+
+		// Create components
+		dbConf := clickhousedb.DefaultConfiguration()
+		dbConf.Servers = servers
+		dbConf.DialTimeout = 100 * time.Millisecond
+		chdb, err := clickhousedb.New(r, dbConf, clickhousedb.Dependencies{
+			Daemon: daemon.NewMock(t),
+		})
+		if err != nil {
+			t.Fatalf("clickhousedb.New() error:\n%+v", err)
+		}
+		helpers.StartStop(t, chdb)
+		conf := clickhouse.DefaultConfiguration()
+		conf.MaximumBatchSize = 10
+		ch, err := clickhouse.New(r, conf, clickhouse.Dependencies{
+			ClickHouse: chdb,
+			Schema:     sch,
+		})
+		if err != nil {
+			t.Fatalf("clickhouse.New() error:\n%+v", err)
+		}
+		helpers.StartStop(t, ch)
+
+		// Trigger an empty send
+		bf := sch.NewFlowMessage()
+		w := ch.NewWorker(1, bf)
+		w.Flush(ctx)
+
+		// Check metrics
+		gotMetrics := r.GetMetrics("akvorado_outlet_clickhouse_", "errors_total")
+		if gotMetrics[`errors_total{error="connect"}`] == "0" {
+			continue
+		}
+		return
+	}
+	t.Fatalf("w.Flush(): cannot trigger connect error")
+}
diff --git a/outlet/clickhouse/root_test.go b/outlet/clickhouse/root_test.go
new file mode 100644
index 00000000..4593d75a
--- /dev/null
+++ b/outlet/clickhouse/root_test.go
@@ -0,0 +1,54 @@
+// SPDX-FileCopyrightText: 2025 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+package clickhouse_test
+
+import (
+	"fmt"
+	"sync"
+	"testing"
+
+	"akvorado/common/helpers"
+	"akvorado/common/schema"
+	"akvorado/outlet/clickhouse"
+)
+
+func TestMock(t *testing.T) {
+	sch := schema.NewMock(t)
+	bf := sch.NewFlowMessage()
+
+	var messages []*schema.FlowMessage
+	var messagesMutex sync.Mutex
+	ch := clickhouse.NewMock(t, func(msg *schema.FlowMessage) {
+		messagesMutex.Lock()
+		defer messagesMutex.Unlock()
+		messages = append(messages, msg)
+	})
+	helpers.StartStop(t, ch)
+
+	expected := []*schema.FlowMessage{}
+	w := ch.NewWorker(1, bf)
+	for i := range 20 {
+		i = i + 1 // 1 to 20
+		bf.TimeReceived = uint32(100 + i)
+		bf.SrcAS = uint32(65400 + i)
+		bf.DstAS = uint32(65500 + i)
+		bf.AppendString(schema.ColumnExporterName, fmt.Sprintf("exporter-%d", i))
+		expected = append(expected, &schema.FlowMessage{
+			TimeReceived: bf.TimeReceived,
+			SrcAS:        bf.SrcAS,
+			DstAS:        bf.DstAS,
+			OtherColumns: map[schema.ColumnKey]any{
+				schema.ColumnExporterName: fmt.Sprintf("exporter-%d", i),
+			},
+		})
+		w.FinalizeAndSend(t.Context())
+
+		// Check if we have anything inserted in the table
+		messagesMutex.Lock()
+		if diff := helpers.Diff(messages, expected); diff != "" {
+			t.Fatalf("Mock(), iteration %d, (-got, +want):\n%s", i, diff)
+		}
+		messagesMutex.Unlock()
+	}
+}
diff --git a/outlet/clickhouse/tests.go b/outlet/clickhouse/tests.go
new file mode 100644
index 00000000..52a3d565
--- /dev/null
+++ b/outlet/clickhouse/tests.go
@@ -0,0 +1,51 @@
+// SPDX-FileCopyrightText: 2025 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+//go:build !release
+
+package clickhouse
+
+import (
+	"context"
+	"testing"
+
+	"akvorado/common/schema"
+)
+
+// mockComponent is a mock version of the ClickHouse exporter.
+type mockComponent struct {
+	callback func(*schema.FlowMessage)
+}
+
+// NewMock creates a new mock exporter that calls the provided callback function with each received flow message.
+func NewMock(_ *testing.T, callback func(*schema.FlowMessage)) Component {
+	return &mockComponent{
+		callback: callback,
+	}
+}
+
+// NewWorker creates a new mock worker.
+func (c *mockComponent) NewWorker(_ int, bf *schema.FlowMessage) Worker {
+	return &mockWorker{
+		c:  c,
+		bf: bf,
+	}
+}
+
+// mockWorker is a mock version of the ClickHouse worker.
+type mockWorker struct {
+	c  *mockComponent
+	bf *schema.FlowMessage
+}
+
+// FinalizeAndSend always "send" the current flows.
+func (w *mockWorker) FinalizeAndSend(ctx context.Context) {
+	w.Flush(ctx)
+}
+
+// Send will record the sent flows for testing purpose.
+func (w *mockWorker) Flush(_ context.Context) {
+	clone := *w.bf
+	w.c.callback(&clone)
+	w.bf.Clear() // Clear instead of finalizing
+}
diff --git a/inlet/core/classifier_test.go b/outlet/core/classifier_test.go
similarity index 100%
rename from inlet/core/classifier_test.go
rename to outlet/core/classifier_test.go
diff --git a/inlet/core/config_test.go b/outlet/core/config_test.go
similarity index 100%
rename from inlet/core/config_test.go
rename to outlet/core/config_test.go
diff --git a/inlet/core/enricher_test.go b/outlet/core/enricher_test.go
similarity index 84%
rename from inlet/core/enricher_test.go
rename to outlet/core/enricher_test.go
index b8507684..66f2eca5 100644
--- a/inlet/core/enricher_test.go
+++ b/outlet/core/enricher_test.go
@@ -4,24 +4,29 @@
 package core
 
 import (
+	"bytes"
+	"encoding/gob"
 	"fmt"
 	"net/netip"
+	"sync"
 	"testing"
 	"time"
 
-	"github.com/IBM/sarama"
 	"github.com/gin-gonic/gin"
 	"github.com/go-viper/mapstructure/v2"
+	"google.golang.org/protobuf/proto"
 
 	"akvorado/common/daemon"
 	"akvorado/common/helpers"
 	"akvorado/common/httpserver"
+	"akvorado/common/pb"
 	"akvorado/common/reporter"
 	"akvorado/common/schema"
-	"akvorado/inlet/flow"
-	"akvorado/inlet/kafka"
-	"akvorado/inlet/metadata"
-	"akvorado/inlet/routing"
+	"akvorado/outlet/clickhouse"
+	"akvorado/outlet/flow"
+	"akvorado/outlet/kafka"
+	"akvorado/outlet/metadata"
+	"akvorado/outlet/routing"
 )
 
 func TestEnrich(t *testing.T) {
@@ -44,8 +49,10 @@ func TestEnrich(t *testing.T) {
 			},
 			OutputFlow: &schema.FlowMessage{
 				SamplingRate:    1000,
+				InIf:            100,
+				OutIf:           200,
 				ExporterAddress: netip.MustParseAddr("::ffff:192.0.2.142"),
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
+				OtherColumns: map[schema.ColumnKey]interface{}{
 					schema.ColumnExporterName:     "192_0_2_142",
 					schema.ColumnInIfName:         "Gi0/0/100",
 					schema.ColumnOutIfName:        "Gi0/0/200",
@@ -72,8 +79,10 @@ func TestEnrich(t *testing.T) {
 			},
 			OutputFlow: &schema.FlowMessage{
 				SamplingRate:    500,
+				InIf:            100,
+				OutIf:           200,
 				ExporterAddress: netip.MustParseAddr("::ffff:192.0.2.142"),
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
+				OtherColumns: map[schema.ColumnKey]interface{}{
 					schema.ColumnExporterName:     "192_0_2_142",
 					schema.ColumnInIfName:         "Gi0/0/100",
 					schema.ColumnOutIfName:        "Gi0/0/200",
@@ -95,8 +104,10 @@ func TestEnrich(t *testing.T) {
 			},
 			OutputFlow: &schema.FlowMessage{
 				SamplingRate:    500,
+				InIf:            100,
+				OutIf:           200,
 				ExporterAddress: netip.MustParseAddr("::ffff:192.0.2.142"),
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
+				OtherColumns: map[schema.ColumnKey]interface{}{
 					schema.ColumnExporterName:     "192_0_2_142",
 					schema.ColumnInIfName:         "Gi0/0/100",
 					schema.ColumnOutIfName:        "Gi0/0/200",
@@ -122,8 +133,10 @@ func TestEnrich(t *testing.T) {
 			},
 			OutputFlow: &schema.FlowMessage{
 				SamplingRate:    500,
+				InIf:            100,
+				OutIf:           200,
 				ExporterAddress: netip.MustParseAddr("::ffff:192.0.2.142"),
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
+				OtherColumns: map[schema.ColumnKey]interface{}{
 					schema.ColumnExporterName:     "192_0_2_142",
 					schema.ColumnInIfName:         "Gi0/0/100",
 					schema.ColumnOutIfName:        "Gi0/0/200",
@@ -152,8 +165,10 @@ func TestEnrich(t *testing.T) {
 			},
 			OutputFlow: &schema.FlowMessage{
 				SamplingRate:    1000,
+				InIf:            100,
+				OutIf:           200,
 				ExporterAddress: netip.MustParseAddr("::ffff:192.0.2.142"),
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
+				OtherColumns: map[schema.ColumnKey]interface{}{
 					schema.ColumnExporterName:     "192_0_2_142",
 					schema.ColumnExporterRegion:   "asia",
 					schema.ColumnExporterTenant:   "alfred",
@@ -184,8 +199,10 @@ func TestEnrich(t *testing.T) {
 			},
 			OutputFlow: &schema.FlowMessage{
 				SamplingRate:    1000,
+				InIf:            100,
+				OutIf:           200,
 				ExporterAddress: netip.MustParseAddr("::ffff:192.0.2.142"),
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
+				OtherColumns: map[schema.ColumnKey]interface{}{
 					schema.ColumnExporterName:     "192_0_2_142",
 					schema.ColumnExporterTenant:   "alfred",
 					schema.ColumnInIfName:         "Gi0/0/100",
@@ -246,8 +263,10 @@ func TestEnrich(t *testing.T) {
 			},
 			OutputFlow: &schema.FlowMessage{
 				SamplingRate:    1000,
+				InIf:            100,
+				OutIf:           200,
 				ExporterAddress: netip.MustParseAddr("::ffff:192.0.2.142"),
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
+				OtherColumns: map[schema.ColumnKey]interface{}{
 					schema.ColumnExporterName:     "192_0_2_142",
 					schema.ColumnInIfProvider:     "index1",
 					schema.ColumnOutIfProvider:    "index2",
@@ -277,8 +296,10 @@ func TestEnrich(t *testing.T) {
 			},
 			OutputFlow: &schema.FlowMessage{
 				SamplingRate:    1000,
+				InIf:            100,
+				OutIf:           200,
 				ExporterAddress: netip.MustParseAddr("::ffff:192.0.2.142"),
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
+				OtherColumns: map[schema.ColumnKey]interface{}{
 					schema.ColumnExporterName:     "192_0_2_142",
 					schema.ColumnInIfName:         "eth100",
 					schema.ColumnOutIfName:        "Gi0/0/200",
@@ -300,15 +321,19 @@ func TestEnrich(t *testing.T) {
 					SamplingRate:    1000,
 					ExporterAddress: netip.MustParseAddr("::ffff:192.0.2.142"),
 					InIf:            100,
-					SrcVlan:         10,
 					OutIf:           200,
+					SrcVlan:         10,
 					DstVlan:         300,
 				}
 			},
 			OutputFlow: &schema.FlowMessage{
 				SamplingRate:    1000,
+				InIf:            100,
+				OutIf:           200,
+				SrcVlan:         10,
+				DstVlan:         300,
 				ExporterAddress: netip.MustParseAddr("::ffff:192.0.2.142"),
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
+				OtherColumns: map[schema.ColumnKey]interface{}{
 					schema.ColumnExporterName:     "192_0_2_142",
 					schema.ColumnInIfName:         "Gi0/0/100",
 					schema.ColumnOutIfName:        "Gi0/0/200.300",
@@ -340,8 +365,10 @@ ClassifyProviderRegex(Interface.Description, "^Transit: ([^ ]+)", "$1")`,
 			},
 			OutputFlow: &schema.FlowMessage{
 				SamplingRate:    1000,
+				InIf:            100,
+				OutIf:           200,
 				ExporterAddress: netip.MustParseAddr("::ffff:192.0.2.142"),
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
+				OtherColumns: map[schema.ColumnKey]interface{}{
 					schema.ColumnExporterName:     "192_0_2_142",
 					schema.ColumnInIfName:         "Gi0/0/100",
 					schema.ColumnOutIfName:        "Gi0/0/200",
@@ -371,8 +398,10 @@ ClassifyProviderRegex(Interface.Description, "^Transit: ([^ ]+)", "$1")`,
 			},
 			OutputFlow: &schema.FlowMessage{
 				SamplingRate:    1000,
+				InIf:            100,
+				OutIf:           200,
 				ExporterAddress: netip.MustParseAddr("::ffff:192.0.2.142"),
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
+				OtherColumns: map[schema.ColumnKey]interface{}{
 					schema.ColumnExporterName:     "192_0_2_142",
 					schema.ColumnInIfName:         "Gi0/0/100",
 					schema.ColumnOutIfName:        "Gi0/0/200",
@@ -402,8 +431,10 @@ ClassifyProviderRegex(Interface.Description, "^Transit: ([^ ]+)", "$1")`,
 			},
 			OutputFlow: &schema.FlowMessage{
 				SamplingRate:    1000,
+				InIf:            100,
+				OutIf:           200,
 				ExporterAddress: netip.MustParseAddr("::ffff:192.0.2.142"),
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
+				OtherColumns: map[schema.ColumnKey]interface{}{
 					schema.ColumnExporterName:     "192_0_2_142",
 					schema.ColumnInIfName:         "Gi0/0/100",
 					schema.ColumnOutIfName:        "Gi0/0/200",
@@ -434,8 +465,10 @@ ClassifyProviderRegex(Interface.Description, "^Transit: ([^ ]+)", "$1")`,
 			},
 			OutputFlow: &schema.FlowMessage{
 				SamplingRate:    1000,
+				InIf:            100,
+				OutIf:           200,
 				ExporterAddress: netip.MustParseAddr("::ffff:192.0.2.142"),
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
+				OtherColumns: map[schema.ColumnKey]interface{}{
 					schema.ColumnExporterName:      "192_0_2_142",
 					schema.ColumnInIfName:          "Gi0/0/100",
 					schema.ColumnOutIfName:         "Gi0/0/200",
@@ -471,8 +504,10 @@ ClassifyProviderRegex(Interface.Description, "^Transit: ([^ ]+)", "$1")`,
 			},
 			OutputFlow: &schema.FlowMessage{
 				SamplingRate:    1000,
+				InIf:            1010,
+				OutIf:           2010,
 				ExporterAddress: netip.MustParseAddr("::ffff:192.0.2.142"),
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
+				OtherColumns: map[schema.ColumnKey]interface{}{
 					schema.ColumnExporterName:      "192_0_2_142",
 					schema.ColumnExporterGroup:     "metadata group",
 					schema.ColumnExporterRegion:    "metadata region",
@@ -509,26 +544,28 @@ ClassifyProviderRegex(Interface.Description, "^Transit: ([^ ]+)", "$1")`,
 			},
 			OutputFlow: &schema.FlowMessage{
 				SamplingRate:    1000,
+				InIf:            100,
+				OutIf:           200,
 				ExporterAddress: netip.MustParseAddr("::ffff:192.0.2.142"),
 				SrcAddr:         netip.MustParseAddr("::ffff:192.0.2.142"),
 				DstAddr:         netip.MustParseAddr("::ffff:192.0.2.10"),
 				SrcAS:           1299,
 				DstAS:           174,
-				ProtobufDebug: map[schema.ColumnKey]interface{}{
-					schema.ColumnExporterName:                  "192_0_2_142",
-					schema.ColumnInIfName:                      "Gi0/0/100",
-					schema.ColumnOutIfName:                     "Gi0/0/200",
-					schema.ColumnInIfDescription:               "Interface 100",
-					schema.ColumnOutIfDescription:              "Interface 200",
-					schema.ColumnInIfSpeed:                     1000,
-					schema.ColumnOutIfSpeed:                    1000,
-					schema.ColumnDstASPath:                     []uint32{64200, 1299, 174},
-					schema.ColumnDstCommunities:                []uint32{100, 200, 400},
-					schema.ColumnDstLargeCommunitiesASN:        []int32{64200},
-					schema.ColumnDstLargeCommunitiesLocalData1: []int32{2},
-					schema.ColumnDstLargeCommunitiesLocalData2: []int32{3},
-					schema.ColumnSrcNetMask:                    27,
-					schema.ColumnDstNetMask:                    27,
+				SrcNetMask:      27,
+				DstNetMask:      27,
+				OtherColumns: map[schema.ColumnKey]interface{}{
+					schema.ColumnExporterName:     "192_0_2_142",
+					schema.ColumnInIfName:         "Gi0/0/100",
+					schema.ColumnOutIfName:        "Gi0/0/200",
+					schema.ColumnInIfDescription:  "Interface 100",
+					schema.ColumnOutIfDescription: "Interface 200",
+					schema.ColumnInIfSpeed:        1000,
+					schema.ColumnOutIfSpeed:       1000,
+					schema.ColumnDstASPath:        []uint32{64200, 1299, 174},
+					schema.ColumnDstCommunities:   []uint32{100, 200, 400},
+					schema.ColumnDstLargeCommunities: []schema.UInt128{
+						{High: 64200, Low: (uint64(2) << 32) + uint64(3)},
+					},
 				},
 			},
 		},
@@ -541,11 +578,21 @@ ClassifyProviderRegex(Interface.Description, "^Transit: ([^ ]+)", "$1")`,
 			daemonComponent := daemon.NewMock(t)
 			metadataComponent := metadata.NewMock(t, r, metadata.DefaultConfiguration(),
 				metadata.Dependencies{Daemon: daemonComponent})
-			flowComponent := flow.NewMock(t, r, flow.DefaultConfiguration())
-			kafkaComponent, kafkaProducer := kafka.NewMock(t, r, kafka.DefaultConfiguration())
+			flowComponent, err := flow.New(r, flow.Dependencies{Schema: schema.NewMock(t)})
+			if err != nil {
+				t.Fatalf("flow.New() error:\n%+v", err)
+			}
 			httpComponent := httpserver.NewMock(t, r)
 			routingComponent := routing.NewMock(t, r)
 			routingComponent.PopulateRIB(t)
+			kafkaComponent, incoming := kafka.NewMock(t, kafka.DefaultConfiguration())
+			var clickhouseMessages []*schema.FlowMessage
+			var clickhouseMessagesMutex sync.Mutex
+			clickhouseComponent := clickhouse.NewMock(t, func(msg *schema.FlowMessage) {
+				clickhouseMessagesMutex.Lock()
+				defer clickhouseMessagesMutex.Unlock()
+				clickhouseMessages = append(clickhouseMessages, msg)
+			})
 
 			// Prepare a configuration
 			configuration := DefaultConfiguration()
@@ -559,55 +606,66 @@ ClassifyProviderRegex(Interface.Description, "^Transit: ([^ ]+)", "$1")`,
 
 			// Instantiate and start core
 			c, err := New(r, configuration, Dependencies{
-				Daemon:   daemonComponent,
-				Flow:     flowComponent,
-				Metadata: metadataComponent,
-				Kafka:    kafkaComponent,
-				HTTP:     httpComponent,
-				Routing:  routingComponent,
-				Schema:   schema.NewMock(t),
+				Daemon:     daemonComponent,
+				Flow:       flowComponent,
+				Metadata:   metadataComponent,
+				Kafka:      kafkaComponent,
+				ClickHouse: clickhouseComponent,
+				HTTP:       httpComponent,
+				Routing:    routingComponent,
+				Schema:     schema.NewMock(t),
 			})
 			if err != nil {
 				t.Fatalf("New() error:\n%+v", err)
 			}
+
 			helpers.StartStop(t, c)
+			clickhouseMessagesMutex.Lock()
+			clickhouseMessages = clickhouseMessages[:0]
+			clickhouseMessagesMutex.Unlock()
 
-			// Inject twice since otherwise, we get a cache miss
-			received := make(chan bool)
-			if tc.OutputFlow != nil {
-				kafkaProducer.ExpectInputWithMessageCheckerFunctionAndSucceed(
-					func(msg *sarama.ProducerMessage) error {
-						defer close(received)
-						b, err := msg.Value.Encode()
-						if err != nil {
-							t.Fatalf("Kafka message encoding error:\n%+v", err)
-						}
-						t.Logf("Raw message: %v", b)
-						got := c.d.Schema.ProtobufDecode(t, b)
-						if diff := helpers.Diff(&got, tc.OutputFlow); diff != "" {
-							t.Errorf("Classifier (-got, +want):\n%s", diff)
-						}
-						return nil
-					})
+			inputFlow := tc.InputFlow()
+			var buf bytes.Buffer
+			encoder := gob.NewEncoder(&buf)
+			if err := encoder.Encode(inputFlow); err != nil {
+				t.Fatalf("gob.Encode() error: %v", err)
 			}
-			// Else, we should not get a message, but that's not possible to test.
-			flowComponent.Inject(tc.InputFlow())
-			time.Sleep(50 * time.Millisecond) // Needed to let poller does its job
-			flowComponent.Inject(tc.InputFlow())
-			if tc.OutputFlow != nil {
-				select {
-				case <-received:
-				case <-time.After(1 * time.Second):
-					t.Fatal("Kafka message not received")
+
+			rawFlow := &pb.RawFlow{
+				TimeReceived:     uint64(time.Now().Unix()),
+				Payload:          buf.Bytes(),
+				SourceAddress:    inputFlow.ExporterAddress.AsSlice(),
+				UseSourceAddress: false,
+				Decoder:          pb.RawFlow_DECODER_GOB,
+				TimestampSource:  pb.RawFlow_TS_INPUT,
+			}
+
+			data, err := proto.Marshal(rawFlow)
+			if err != nil {
+				t.Fatalf("proto.Marshal() error: %v", err)
+			}
+
+			incoming <- data
+			time.Sleep(100 * time.Millisecond)
+
+			clickhouseMessagesMutex.Lock()
+			clickhouseMessagesLen := len(clickhouseMessages)
+			var lastMessage *schema.FlowMessage
+			if clickhouseMessagesLen > 0 {
+				lastMessage = clickhouseMessages[clickhouseMessagesLen-1]
+			}
+			clickhouseMessagesMutex.Unlock()
+
+			if tc.OutputFlow != nil && clickhouseMessagesLen > 0 {
+				if diff := helpers.Diff(lastMessage, tc.OutputFlow); diff != "" {
+					t.Errorf("Enriched flow differs (-got, +want):\n%s", diff)
 				}
-			} else {
-				time.Sleep(100 * time.Millisecond)
 			}
-			gotMetrics := r.GetMetrics("akvorado_inlet_core_", "-processing_", "flows_", "received_", "forwarded_")
+			gotMetrics := r.GetMetrics("akvorado_outlet_core_", "-processing_", "flows_", "received_", "forwarded_")
 			expectedMetrics := map[string]string{
-				`flows_errors_total{error="SNMP cache miss",exporter="192.0.2.142"}`: "1",
 				`flows_http_clients`:                           "0",
-				`received_flows_total{exporter="192.0.2.142"}`: "2",
+				`received_flows_total{exporter="192.0.2.142"}`: "1",
+				`received_raw_flows_total`:                     "1",
 			}
 			if tc.OutputFlow != nil {
 				expectedMetrics[`forwarded_flows_total{exporter="192.0.2.142"}`] = "1"
diff --git a/outlet/core/root_test.go b/outlet/core/root_test.go
new file mode 100644
index 00000000..ce46f8ea
--- /dev/null
+++ b/outlet/core/root_test.go
@@ -0,0 +1,360 @@
+// SPDX-FileCopyrightText: 2022 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+package core
+
+import (
+	"bufio"
+	"bytes"
+	"encoding/gob"
+	"encoding/json"
+	"fmt"
+	"io"
+	"net/http"
+	"net/netip"
+	"strings"
+	"sync"
+	"testing"
+	"time"
+
+	"github.com/gin-gonic/gin"
+	"google.golang.org/protobuf/proto"
+
+	"akvorado/common/daemon"
+	"akvorado/common/helpers"
+	"akvorado/common/httpserver"
+	"akvorado/common/pb"
+	"akvorado/common/reporter"
+	"akvorado/common/schema"
+	"akvorado/outlet/clickhouse"
+	"akvorado/outlet/flow"
+	"akvorado/outlet/kafka"
+	"akvorado/outlet/metadata"
+	"akvorado/outlet/routing"
+)
+
+func TestCore(t *testing.T) {
+	r := reporter.NewMock(t)
+
+	// Prepare all components.
+	daemonComponent := daemon.NewMock(t)
+	metadataComponent := metadata.NewMock(t, r, metadata.DefaultConfiguration(),
+		metadata.Dependencies{Daemon: daemonComponent})
+	flowComponent, err := flow.New(r, flow.Dependencies{Schema: schema.NewMock(t)})
+	if err != nil {
+		t.Fatalf("flow.New() error:\n%+v", err)
+	}
+	httpComponent := httpserver.NewMock(t, r)
+	routingComponent := routing.NewMock(t, r)
+	routingComponent.PopulateRIB(t)
+	kafkaComponent, incoming := kafka.NewMock(t, kafka.DefaultConfiguration())
+	var clickhouseMessages []*schema.FlowMessage
+	var clickhouseMessagesMutex sync.Mutex
+	clickhouseComponent := clickhouse.NewMock(t, func(msg *schema.FlowMessage) {
+		clickhouseMessagesMutex.Lock()
+		defer clickhouseMessagesMutex.Unlock()
+		clickhouseMessages = append(clickhouseMessages, msg)
+	})
+
+	// Instantiate and start core
+	sch := schema.NewMock(t)
+	c, err := New(r, DefaultConfiguration(), Dependencies{
+		Daemon:     daemonComponent,
+		Flow:       flowComponent,
+		Metadata:   metadataComponent,
+		Kafka:      kafkaComponent,
+		ClickHouse: clickhouseComponent,
+		HTTP:       httpComponent,
+		Routing:    routingComponent,
+		Schema:     sch,
+	})
+	if err != nil {
+		t.Fatalf("New() error:\n%+v", err)
+	}
+	helpers.StartStop(t, c)
+
+	flowMessage := func(exporter string, in, out uint32) *schema.FlowMessage {
+		msg := &schema.FlowMessage{
+			TimeReceived:    200,
+			SamplingRate:    1000,
+			ExporterAddress: netip.AddrFrom16(netip.MustParseAddr(exporter).As16()),
+			InIf:            in,
+			OutIf:           out,
+			SrcAddr:         netip.MustParseAddr("::ffff:67.43.156.77"),
+			DstAddr:         netip.MustParseAddr("::ffff:2.125.160.216"),
+			OtherColumns: map[schema.ColumnKey]interface{}{
+				schema.ColumnBytes:   6765,
+				schema.ColumnPackets: 4,
+				schema.ColumnEType:   0x800,
+				schema.ColumnProto:   6,
+				schema.ColumnSrcPort: 8534,
+				schema.ColumnDstPort: 80,
+			},
+		}
+		return msg
+	}
+
+	expectedFlowMessage := func(exporter string, in, out uint32) *schema.FlowMessage {
+		expected := flowMessage(exporter, in, out)
+		expected.SrcAS = 0 // no geoip enrich anymore
+		expected.DstAS = 0 // no geoip enrich anymore
+		expected.OtherColumns[schema.ColumnInIfName] = fmt.Sprintf("Gi0/0/%d", in)
+		expected.OtherColumns[schema.ColumnOutIfName] = fmt.Sprintf("Gi0/0/%d", out)
+		expected.OtherColumns[schema.ColumnInIfDescription] = fmt.Sprintf("Interface %d", in)
+		expected.OtherColumns[schema.ColumnOutIfDescription] = fmt.Sprintf("Interface %d", out)
+		expected.OtherColumns[schema.ColumnInIfSpeed] = 1000
+		expected.OtherColumns[schema.ColumnOutIfSpeed] = 1000
+		expected.OtherColumns[schema.ColumnExporterName] = strings.ReplaceAll(exporter, ".", "_")
+		return expected
+	}
+
+	// Helper function to inject flows using the new mechanism
+	injectFlow := func(flow *schema.FlowMessage) {
+		t.Helper()
+		var buf bytes.Buffer
+		encoder := gob.NewEncoder(&buf)
+		if err := encoder.Encode(flow); err != nil {
+			t.Fatalf("gob.Encode() error: %v", err)
+		}
+
+		rawFlow := &pb.RawFlow{
+			TimeReceived:     uint64(time.Now().Unix()),
+			Payload:          buf.Bytes(),
+			SourceAddress:    flow.ExporterAddress.AsSlice(),
+			UseSourceAddress: false,
+			Decoder:          pb.RawFlow_DECODER_GOB,
+			TimestampSource:  pb.RawFlow_TS_INPUT,
+		}
+
+		data, err := proto.Marshal(rawFlow)
+		if err != nil {
+			t.Fatalf("proto.Marshal() error: %v", err)
+		}
+
+		// Send to kafka mock's incoming channel
+		incoming <- data
+	}
+
+	t.Run("core", func(t *testing.T) {
+		clickhouseMessagesMutex.Lock()
+		clickhouseMessages = clickhouseMessages[:0]
+		clickhouseMessagesMutex.Unlock()
+
+		// Inject several messages
+		injectFlow(flowMessage("192.0.2.142", 434, 677))
+		injectFlow(flowMessage("192.0.2.143", 437, 679))
+		time.Sleep(20 * time.Millisecond)
+
+		gotMetrics := r.GetMetrics("akvorado_outlet_core_", "-flows_processing_")
+		expectedMetrics := map[string]string{
+			`classifier_exporter_cache_items_total`:         "0",
+			`classifier_interface_cache_items_total`:        "0",
+			`received_flows_total{exporter="192.0.2.142"}`:  "1",
+			`received_flows_total{exporter="192.0.2.143"}`:  "1",
+			`forwarded_flows_total{exporter="192.0.2.142"}`: "1",
+			`forwarded_flows_total{exporter="192.0.2.143"}`: "1",
+			`received_raw_flows_total`:                      "2",
+			`flows_http_clients`:                            "0",
+		}
+		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
+			t.Fatalf("Metrics (-got, +want):\n%s", diff)
+		}
+
+		// Should have 2 more flows in clickhouseMessages now
+		clickhouseMessagesMutex.Lock()
+		clickhouseMessagesLen := len(clickhouseMessages)
+		clickhouseMessagesMutex.Unlock()
+		if clickhouseMessagesLen < 2 {
+			t.Fatalf("Expected at least 2 flows in clickhouseMessages, got %d", clickhouseMessagesLen)
+		}
+
+		// Now, check we get the message we expect
+		clickhouseMessagesMutex.Lock()
+		clickhouseMessages = clickhouseMessages[:0]
+		clickhouseMessagesMutex.Unlock()
+		input := flowMessage("192.0.2.142", 434, 677)
+		injectFlow(input)
+		time.Sleep(20 * time.Millisecond)
+
+		// Check the flow was stored in clickhouseMessages
+		expected := []*schema.FlowMessage{expectedFlowMessage("192.0.2.142", 434, 677)}
+		clickhouseMessagesMutex.Lock()
+		clickhouseMessagesCopy := make([]*schema.FlowMessage, len(clickhouseMessages))
+		copy(clickhouseMessagesCopy, clickhouseMessages)
+		clickhouseMessagesMutex.Unlock()
+		if diff := helpers.Diff(clickhouseMessagesCopy, expected); diff != "" {
+			t.Fatalf("Flow message (-got, +want):\n%s", diff)
+		}
+
+		// Try to inject a message with missing sampling rate
+		input = flowMessage("192.0.2.142", 434, 677)
+		input.SamplingRate = 0
+		injectFlow(input)
+		time.Sleep(20 * time.Millisecond)
+
+		gotMetrics = r.GetMetrics("akvorado_outlet_core_", "classifier_", "-flows_processing_", "flows_", "forwarded_", "received_")
+		expectedMetrics = map[string]string{
+			`classifier_exporter_cache_items_total`:                                    "0",
+			`classifier_interface_cache_items_total`:                                   "0",
+			`flows_errors_total{error="sampling rate missing",exporter="192.0.2.142"}`: "1",
+			`received_flows_total{exporter="192.0.2.142"}`:                             "3",
+			`received_flows_total{exporter="192.0.2.143"}`:                             "1",
+			`forwarded_flows_total{exporter="192.0.2.142"}`:                            "2",
+			`forwarded_flows_total{exporter="192.0.2.143"}`:                            "1",
+			`flows_http_clients`:                                                       "0",
+			`received_raw_flows_total`:                                                 "4",
+		}
+		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
+			t.Fatalf("Metrics (-got, +want):\n%s", diff)
+		}
+	})
+
+	// Test HTTP flow clients (JSON)
+	t.Run("http flows", func(t *testing.T) {
+		c.httpFlowFlushDelay = 20 * time.Millisecond
+
+		resp, err := http.Get(fmt.Sprintf("http://%s/api/v0/outlet/flows", c.d.HTTP.LocalAddr()))
+		if err != nil {
+			t.Fatalf("GET /api/v0/outlet/flows:\n%+v", err)
+		}
+		defer resp.Body.Close()
+		if resp.StatusCode != 200 {
+			t.Fatalf("GET /api/v0/outlet/flows status code %d", resp.StatusCode)
+		}
+
+		// Metrics should tell we have a client
+		gotMetrics := r.GetMetrics("akvorado_outlet_core_", "flows_http_clients", "-flows_processing_")
+		expectedMetrics := map[string]string{
+			`flows_http_clients`: "1",
+		}
+		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
+			t.Fatalf("Metrics (-got, +want):\n%s", diff)
+		}
+
+		// Produce some flows
+		clickhouseMessagesMutex.Lock()
+		clickhouseMessages = clickhouseMessages[:0]
+		clickhouseMessagesMutex.Unlock()
+		for range 12 {
+			injectFlow(flowMessage("192.0.2.142", 434, 677))
+		}
+
+		// Wait for flows to be processed
+		time.Sleep(100 * time.Millisecond)
+
+		// Should have 12 flows in clickhouseMessages
+		clickhouseMessagesMutex.Lock()
+		clickhouseMessagesLen := len(clickhouseMessages)
+		clickhouseMessagesMutex.Unlock()
+		if clickhouseMessagesLen != 12 {
+			t.Fatalf("Expected 12 flows in clickhouseMessages, got %d", clickhouseMessagesLen)
+		}
+
+		// Decode some of them
+		reader := bufio.NewReader(resp.Body)
+		decoder := json.NewDecoder(reader)
+		for range 10 {
+			var got gin.H
+			if err := decoder.Decode(&got); err != nil {
+				t.Fatalf("GET /api/v0/outlet/flows error while reading body:\n%+v", err)
+			}
+			expected := gin.H{
+				"TimeReceived":    200,
+				"SamplingRate":    1000,
+				"ExporterAddress": "::ffff:192.0.2.142",
+				"SrcAddr":         "::ffff:67.43.156.77",
+				"DstAddr":         "::ffff:2.125.160.216",
+				"SrcAS":           0, // no geoip enrich anymore
+				"InIf":            434,
+				"OutIf":           677,
+				"NextHop":         "",
+				"SrcNetMask":      0,
+				"DstNetMask":      0,
+				"SrcVlan":         0,
+				"DstVlan":         0,
+				"DstAS":           0,
+				"OtherColumns": gin.H{
+					"ExporterName":     "192_0_2_142",
+					"InIfName":         "Gi0/0/434",
+					"OutIfName":        "Gi0/0/677",
+					"InIfDescription":  "Interface 434",
+					"OutIfDescription": "Interface 677",
+					"InIfSpeed":        1000,
+					"OutIfSpeed":       1000,
+					"Bytes":            6765,
+					"Packets":          4,
+					"SrcPort":          8534,
+					"DstPort":          80,
+					"EType":            2048,
+					"Proto":            6,
+				},
+			}
+			if diff := helpers.Diff(got, expected); diff != "" {
+				t.Fatalf("GET /api/v0/outlet/flows (-got, +want):\n%s", diff)
+			}
+		}
+	})
+
+	// Test HTTP flow clients with a limit
+	time.Sleep(10 * time.Millisecond)
+	t.Run("http flows with limit", func(t *testing.T) {
+		resp, err := http.Get(fmt.Sprintf("http://%s/api/v0/outlet/flows?limit=4", c.d.HTTP.LocalAddr()))
+		if err != nil {
+			t.Fatalf("GET /api/v0/outlet/flows:\n%+v", err)
+		}
+		defer resp.Body.Close()
+		if resp.StatusCode != 200 {
+			t.Fatalf("GET /api/v0/outlet/flows status code %d", resp.StatusCode)
+		}
+
+		// Metrics should tell we have a client
+		gotMetrics := r.GetMetrics("akvorado_outlet_core_", "flows_http_clients")
+		expectedMetrics := map[string]string{
+			`flows_http_clients`: "1",
+		}
+		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
+			t.Fatalf("Metrics (-got, +want):\n%s", diff)
+		}
+
+		// Produce some flows
+		clickhouseMessagesMutex.Lock()
+		clickhouseMessages = clickhouseMessages[:0]
+		clickhouseMessagesMutex.Unlock()
+		for range 12 {
+			injectFlow(flowMessage("192.0.2.142", 434, 677))
+		}
+
+		// Wait for flows to be processed
+		time.Sleep(100 * time.Millisecond)
+
+		// Should have 12 flows in clickhouseMessages
+		clickhouseMessagesMutex.Lock()
+		clickhouseMessagesLen := len(clickhouseMessages)
+		clickhouseMessagesMutex.Unlock()
+		if clickhouseMessagesLen != 12 {
+			t.Fatalf("Expected 12 flows in clickhouseMessages, got %d", clickhouseMessagesLen)
+		}
+
+		// Check we got only 4
+		reader := bufio.NewReader(resp.Body)
+		count := 0
+		for {
+			_, err := reader.ReadString('\n')
+			if err == io.EOF {
+				t.Log("EOF")
+				break
+			}
+			if err != nil {
+				t.Fatalf("GET /api/v0/outlet/flows error while reading:\n%+v", err)
+			}
+			count++
+			if count > 4 {
+				break
+			}
+		}
+		if count != 4 {
+			t.Fatalf("GET /api/v0/outlet/flows got less than 4 flows (%d)", count)
+		}
+	})
+
+}
diff --git a/outlet/flow/decoder/gob/root_test.go b/outlet/flow/decoder/gob/root_test.go
new file mode 100644
index 00000000..5b56ba59
--- /dev/null
+++ b/outlet/flow/decoder/gob/root_test.go
@@ -0,0 +1,102 @@
+// SPDX-FileCopyrightText: 2022 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+//go:build !release
+
+package gob
+
+import (
+	"bytes"
+	"encoding/gob"
+	"net/netip"
+	"testing"
+	"time"
+
+	"akvorado/common/helpers"
+	"akvorado/common/reporter"
+	"akvorado/common/schema"
+	"akvorado/outlet/flow/decoder"
+)
+
+func TestGobDecoder(t *testing.T) {
+	r := reporter.NewMock(t)
+	sch := schema.NewMock(t)
+	d := New(r, decoder.Dependencies{Schema: sch})
+	bf := sch.NewFlowMessage()
+	got := []*schema.FlowMessage{}
+	finalize := func() {
+		// Keep a copy of the current flow message
+		clone := *bf
+		got = append(got, &clone)
+		// And clear the flow message
+		bf.Clear()
+	}
+
+	// Create a test FlowMessage
+	originalFlow := &schema.FlowMessage{
+		TimeReceived:    uint32(time.Now().Unix()),
+		SamplingRate:    1000,
+		ExporterAddress: netip.MustParseAddr("::ffff:192.0.2.1"),
+		InIf:            10,
+		OutIf:           20,
+		SrcAddr:         netip.MustParseAddr("::ffff:192.0.2.100"),
+		DstAddr:         netip.MustParseAddr("::ffff:192.0.2.200"),
+		OtherColumns: map[schema.ColumnKey]any{
+			schema.ColumnBytes:        uint64(1024),
+			schema.ColumnPackets:      10,
+			schema.ColumnInIfBoundary: schema.InterfaceBoundaryExternal,
+			schema.ColumnExporterName: "hello",
+			schema.ColumnDstNetMask:   uint32(8),
+			schema.ColumnDstPort:      uint16(80),
+			schema.ColumnDstASPath:    []uint32{65000, 65001, 65002},
+		},
+	}
+
+	// Encode to gob
+	var buf bytes.Buffer
+	encoder := gob.NewEncoder(&buf)
+	if err := encoder.Encode(originalFlow); err != nil {
+		t.Fatalf("gob.Encode() error: %v", err)
+	}
+
+	// Test decoding
+	rawFlow := decoder.RawFlow{
+		TimeReceived: time.Now(),
+		Payload:      buf.Bytes(),
+		Source:       netip.MustParseAddr("::ffff:192.0.2.1"),
+	}
+
+	nb, err := d.Decode(rawFlow, decoder.Option{}, bf, finalize)
+	if err != nil {
+		t.Fatalf("Decode() error:\n%+v", err)
+	}
+	if nb != 1 {
+		t.Errorf("Decode() returned %d instead of 1", nb)
+	}
+
+	// Compare the decoded flow with the original
+	if diff := helpers.Diff(got, []*schema.FlowMessage{originalFlow}); diff != "" {
+		t.Errorf("decoded flow differs (-got, +want):\n%s", diff)
+	}
+}
+
+func TestGobDecoderInvalidPayload(t *testing.T) {
+	r := reporter.NewMock(t)
+	sch := schema.NewMock(t)
+	d := New(r, decoder.Dependencies{Schema: sch})
+	bf := sch.NewFlowMessage()
+
+	rawFlow := decoder.RawFlow{
+		TimeReceived: time.Now(),
+		Payload:      []byte("invalid gob data"),
+		Source:       netip.MustParseAddr("::ffff:192.0.2.1"),
+	}
+
+	nb, err := d.Decode(rawFlow, decoder.Option{}, bf, func() {})
+	if err == nil {
+		t.Errorf("expected error for invalid payload")
+	}
+	if nb != 0 {
+		t.Errorf("Decode() returned %d instead of 0", nb)
+	}
+}
diff --git a/inlet/flow/decoder/helpers_test.go b/outlet/flow/decoder/helpers_test.go
similarity index 92%
rename from inlet/flow/decoder/helpers_test.go
rename to outlet/flow/decoder/helpers_test.go
index a12733ac..1b57803f 100644
--- a/inlet/flow/decoder/helpers_test.go
+++ b/outlet/flow/decoder/helpers_test.go
@@ -15,7 +15,7 @@ import (
 func TestDecodeMPLSAndIPv4(t *testing.T) {
 	sch := schema.NewMock(t).EnableAllColumns()
 	pcap := helpers.ReadPcapL2(t, filepath.Join("testdata", "mpls-ipv4.pcap"))
-	bf := &schema.FlowMessage{}
+	bf := sch.NewFlowMessage()
 	l := ParseEthernet(sch, bf, pcap)
 	if l != 40 {
 		t.Errorf("ParseEthernet() returned %d, expected 40", l)
@@ -23,7 +23,7 @@ func TestDecodeMPLSAndIPv4(t *testing.T) {
 	expected := schema.FlowMessage{
 		SrcAddr: netip.MustParseAddr("::ffff:10.31.0.1"),
 		DstAddr: netip.MustParseAddr("::ffff:10.34.0.1"),
-		ProtobufDebug: map[schema.ColumnKey]interface{}{
+		OtherColumns: map[schema.ColumnKey]interface{}{
 			schema.ColumnEType:        helpers.ETypeIPv4,
 			schema.ColumnProto:        6,
 			schema.ColumnSrcPort:      11001,
@@ -45,7 +45,7 @@ func TestDecodeMPLSAndIPv4(t *testing.T) {
 func TestDecodeVLANAndIPv6(t *testing.T) {
 	sch := schema.NewMock(t).EnableAllColumns()
 	pcap := helpers.ReadPcapL2(t, filepath.Join("testdata", "vlan-ipv6.pcap"))
-	bf := &schema.FlowMessage{}
+	bf := sch.NewFlowMessage()
 	l := ParseEthernet(sch, bf, pcap)
 	if l != 179 {
 		t.Errorf("ParseEthernet() returned %d, expected 179", l)
@@ -54,7 +54,7 @@ func TestDecodeVLANAndIPv6(t *testing.T) {
 		SrcVlan: 100,
 		SrcAddr: netip.MustParseAddr("2402:f000:1:8e01::5555"),
 		DstAddr: netip.MustParseAddr("2607:fcd0:100:2300::b108:2a6b"),
-		ProtobufDebug: map[schema.ColumnKey]interface{}{
+		OtherColumns: map[schema.ColumnKey]interface{}{
 			schema.ColumnEType:  helpers.ETypeIPv6,
 			schema.ColumnProto:  4,
 			schema.ColumnIPTTL:  246,
diff --git a/inlet/flow/decoder/netflow/root_test.go b/outlet/flow/decoder/netflow/root_test.go
similarity index 56%
rename from inlet/flow/decoder/netflow/root_test.go
rename to outlet/flow/decoder/netflow/root_test.go
index a8fe6e2d..86107deb 100644
--- a/inlet/flow/decoder/netflow/root_test.go
+++ b/outlet/flow/decoder/netflow/root_test.go
@@ -5,38 +5,60 @@ package netflow
 
 import (
 	"fmt"
-	"net"
 	"net/netip"
 	"path/filepath"
 	"testing"
 
 	"akvorado/common/helpers"
+	"akvorado/common/pb"
 	"akvorado/common/reporter"
 	"akvorado/common/schema"
-	"akvorado/inlet/flow/decoder"
+	"akvorado/outlet/flow/decoder"
 )
 
-func TestDecode(t *testing.T) {
+func setup(t *testing.T, clearTs bool) (*reporter.Reporter, decoder.Decoder, *schema.FlowMessage, *[]*schema.FlowMessage, decoder.FinalizeFlowFunc) {
+	t.Helper()
 	r := reporter.NewMock(t)
-	nfdecoder := New(r, decoder.Dependencies{Schema: schema.NewMock(t).EnableAllColumns()}, decoder.Option{TimestampSource: decoder.TimestampSourceUDP})
+	sch := schema.NewMock(t).EnableAllColumns()
+	nfdecoder := New(r, decoder.Dependencies{Schema: sch})
+	bf := sch.NewFlowMessage()
+	got := []*schema.FlowMessage{}
+	finalize := func() {
+		if clearTs {
+			bf.TimeReceived = 0
+		}
+		// Keep a copy of the current flow message
+		clone := *bf
+		got = append(got, &clone)
+		// And clear the flow message
+		bf.Clear()
+	}
+	return r, nfdecoder, bf, &got, finalize
+}
+
+func TestDecode(t *testing.T) {
+	r, nfdecoder, bf, got, finalize := setup(t, true)
+	options := decoder.Option{TimestampSource: pb.RawFlow_TS_INPUT}
 
 	// Send an option template
 	template := helpers.ReadPcapL4(t, filepath.Join("testdata", "options-template.pcap"))
-	got := nfdecoder.Decode(decoder.RawFlow{Payload: template, Source: net.ParseIP("127.0.0.1")})
-	if got == nil {
-		t.Fatalf("Decode() error on options template")
+	_, err := nfdecoder.Decode(
+		decoder.RawFlow{Payload: template, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+		options, bf, finalize)
+	if err != nil {
+		t.Fatalf("Decode() error on options template:\n%+v", err)
 	}
-	if len(got) != 0 {
-		t.Fatalf("Decode() on options template got flows")
+	if len(*got) != 0 {
+		t.Fatalf("Decode() on options template got flows:\n%+v", *got)
 	}
 
 	// Check metrics
-	gotMetrics := r.GetMetrics("akvorado_inlet_flow_decoder_netflow_")
+	gotMetrics := r.GetMetrics("akvorado_outlet_flow_decoder_netflow_")
 	expectedMetrics := map[string]string{
-		`flows_total{exporter="127.0.0.1",version="9"}`:                                                                 "1",
-		`flowset_records_sum{exporter="127.0.0.1",type="OptionsTemplateFlowSet",version="9"}`:                           "1",
-		`flowset_sum{exporter="127.0.0.1",type="OptionsTemplateFlowSet",version="9"}`:                                   "1",
-		`templates_total{exporter="127.0.0.1",obs_domain_id="0",template_id="257",type="options_template",version="9"}`: "1",
+		`packets_total{exporter="::ffff:127.0.0.1",version="9"}`:                                                               "1",
+		`records_total{exporter="::ffff:127.0.0.1",type="OptionsTemplateFlowSet",version="9"}`:                                 "1",
+		`sets_total{exporter="::ffff:127.0.0.1",type="OptionsTemplateFlowSet",version="9"}`:                                    "1",
+		`templates_total{exporter="::ffff:127.0.0.1",obs_domain_id="0",template_id="257",type="options_template",version="9"}`: "1",
 	}
 	if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 		t.Fatalf("Metrics after template (-got, +want):\n%s", diff)
@@ -44,23 +66,25 @@ func TestDecode(t *testing.T) {
 
 	// Send option data
 	data := helpers.ReadPcapL4(t, filepath.Join("testdata", "options-data.pcap"))
-	got = nfdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
-	if got == nil {
-		t.Fatalf("Decode() error on options data")
+	_, err = nfdecoder.Decode(
+		decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+		options, bf, finalize)
+	if err != nil {
+		t.Fatalf("Decode() error on options data:\n%+v", err)
 	}
-	if len(got) != 0 {
+	if len(*got) != 0 {
 		t.Fatalf("Decode() on options data got flows")
 	}
 
 	// Check metrics
-	gotMetrics = r.GetMetrics("akvorado_inlet_flow_decoder_netflow_")
+	gotMetrics = r.GetMetrics("akvorado_outlet_flow_decoder_netflow_")
 	expectedMetrics = map[string]string{
-		`flows_total{exporter="127.0.0.1",version="9"}`:                                                                 "2",
-		`flowset_records_sum{exporter="127.0.0.1",type="OptionsTemplateFlowSet",version="9"}`:                           "1",
-		`flowset_records_sum{exporter="127.0.0.1",type="OptionsDataFlowSet",version="9"}`:                               "4",
-		`flowset_sum{exporter="127.0.0.1",type="OptionsTemplateFlowSet",version="9"}`:                                   "1",
-		`flowset_sum{exporter="127.0.0.1",type="OptionsDataFlowSet",version="9"}`:                                       "1",
-		`templates_total{exporter="127.0.0.1",obs_domain_id="0",template_id="257",type="options_template",version="9"}`: "1",
+		`packets_total{exporter="::ffff:127.0.0.1",version="9"}`:                                                               "2",
+		`records_total{exporter="::ffff:127.0.0.1",type="OptionsTemplateFlowSet",version="9"}`:                                 "1",
+		`records_total{exporter="::ffff:127.0.0.1",type="OptionsDataFlowSet",version="9"}`:                                     "4",
+		`sets_total{exporter="::ffff:127.0.0.1",type="OptionsTemplateFlowSet",version="9"}`:                                    "1",
+		`sets_total{exporter="::ffff:127.0.0.1",type="OptionsDataFlowSet",version="9"}`:                                        "1",
+		`templates_total{exporter="::ffff:127.0.0.1",obs_domain_id="0",template_id="257",type="options_template",version="9"}`: "1",
 	}
 	if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 		t.Fatalf("Metrics after template (-got, +want):\n%s", diff)
@@ -68,26 +92,28 @@ func TestDecode(t *testing.T) {
 
 	// Send a regular template
 	template = helpers.ReadPcapL4(t, filepath.Join("testdata", "template.pcap"))
-	got = nfdecoder.Decode(decoder.RawFlow{Payload: template, Source: net.ParseIP("127.0.0.1")})
-	if got == nil {
-		t.Fatalf("Decode() error on template")
+	_, err = nfdecoder.Decode(
+		decoder.RawFlow{Payload: template, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+		options, bf, finalize)
+	if err != nil {
+		t.Fatalf("Decode() error on template:\n%+v", err)
 	}
-	if len(got) != 0 {
+	if len(*got) != 0 {
 		t.Fatalf("Decode() on template got flows")
 	}
 
 	// Check metrics
-	gotMetrics = r.GetMetrics("akvorado_inlet_flow_decoder_netflow_")
+	gotMetrics = r.GetMetrics("akvorado_outlet_flow_decoder_netflow_")
 	expectedMetrics = map[string]string{
-		`flows_total{exporter="127.0.0.1",version="9"}`:                                                                 "3",
-		`flowset_records_sum{exporter="127.0.0.1",type="OptionsTemplateFlowSet",version="9"}`:                           "1",
-		`flowset_records_sum{exporter="127.0.0.1",type="OptionsDataFlowSet",version="9"}`:                               "4",
-		`flowset_records_sum{exporter="127.0.0.1",type="TemplateFlowSet",version="9"}`:                                  "1",
-		`flowset_sum{exporter="127.0.0.1",type="OptionsTemplateFlowSet",version="9"}`:                                   "1",
-		`flowset_sum{exporter="127.0.0.1",type="OptionsDataFlowSet",version="9"}`:                                       "1",
-		`flowset_sum{exporter="127.0.0.1",type="TemplateFlowSet",version="9"}`:                                          "1",
-		`templates_total{exporter="127.0.0.1",obs_domain_id="0",template_id="257",type="options_template",version="9"}`: "1",
-		`templates_total{exporter="127.0.0.1",obs_domain_id="0",template_id="260",type="template",version="9"}`:         "1",
+		`packets_total{exporter="::ffff:127.0.0.1",version="9"}`:                                                               "3",
+		`records_total{exporter="::ffff:127.0.0.1",type="OptionsTemplateFlowSet",version="9"}`:                                 "1",
+		`records_total{exporter="::ffff:127.0.0.1",type="OptionsDataFlowSet",version="9"}`:                                     "4",
+		`records_total{exporter="::ffff:127.0.0.1",type="TemplateFlowSet",version="9"}`:                                        "1",
+		`sets_total{exporter="::ffff:127.0.0.1",type="OptionsTemplateFlowSet",version="9"}`:                                    "1",
+		`sets_total{exporter="::ffff:127.0.0.1",type="OptionsDataFlowSet",version="9"}`:                                        "1",
+		`sets_total{exporter="::ffff:127.0.0.1",type="TemplateFlowSet",version="9"}`:                                           "1",
+		`templates_total{exporter="::ffff:127.0.0.1",obs_domain_id="0",template_id="257",type="options_template",version="9"}`: "1",
+		`templates_total{exporter="::ffff:127.0.0.1",obs_domain_id="0",template_id="260",type="template",version="9"}`:         "1",
 	}
 	if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 		t.Fatalf("Metrics after template (-got, +want):\n%s", diff)
@@ -95,9 +121,11 @@ func TestDecode(t *testing.T) {
 
 	// Send data
 	data = helpers.ReadPcapL4(t, filepath.Join("testdata", "data.pcap"))
-	got = nfdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
-	if got == nil {
-		t.Fatalf("Decode() error on data")
+	_, err = nfdecoder.Decode(
+		decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+		options, bf, finalize)
+	if err != nil {
+		t.Fatalf("Decode() error on data:\n%+v", err)
 	}
 	expectedFlows := []*schema.FlowMessage{
 		{
@@ -110,7 +138,7 @@ func TestDecode(t *testing.T) {
 			OutIf:           450,
 			SrcNetMask:      24,
 			DstNetMask:      14,
-			ProtobufDebug: map[schema.ColumnKey]interface{}{
+			OtherColumns: map[schema.ColumnKey]any{
 				schema.ColumnBytes:            1500,
 				schema.ColumnPackets:          1,
 				schema.ColumnEType:            helpers.ETypeIPv4,
@@ -130,7 +158,7 @@ func TestDecode(t *testing.T) {
 			NextHop:         netip.MustParseAddr("::ffff:194.149.174.71"),
 			SrcNetMask:      24,
 			DstNetMask:      14,
-			ProtobufDebug: map[schema.ColumnKey]interface{}{
+			OtherColumns: map[schema.ColumnKey]any{
 				schema.ColumnBytes:            1500,
 				schema.ColumnPackets:          1,
 				schema.ColumnEType:            helpers.ETypeIPv4,
@@ -150,7 +178,7 @@ func TestDecode(t *testing.T) {
 			NextHop:         netip.MustParseAddr("::ffff:252.223.0.0"),
 			SrcNetMask:      20,
 			DstNetMask:      18,
-			ProtobufDebug: map[schema.ColumnKey]interface{}{
+			OtherColumns: map[schema.ColumnKey]any{
 				schema.ColumnBytes:            1400,
 				schema.ColumnPackets:          1,
 				schema.ColumnEType:            helpers.ETypeIPv4,
@@ -170,7 +198,7 @@ func TestDecode(t *testing.T) {
 			OutIf:           451,
 			SrcNetMask:      16,
 			DstNetMask:      14,
-			ProtobufDebug: map[schema.ColumnKey]interface{}{
+			OtherColumns: map[schema.ColumnKey]any{
 				schema.ColumnBytes:            1448,
 				schema.ColumnPackets:          1,
 				schema.ColumnEType:            helpers.ETypeIPv4,
@@ -182,31 +210,29 @@ func TestDecode(t *testing.T) {
 			},
 		},
 	}
-	for _, f := range got {
-		f.TimeReceived = 0
-	}
 
 	if diff := helpers.Diff(got, expectedFlows); diff != "" {
 		t.Fatalf("Decode() (-got, +want):\n%s", diff)
 	}
 	gotMetrics = r.GetMetrics(
-		"akvorado_inlet_flow_decoder_netflow_",
-		"flows_total",
-		"flowset_",
+		"akvorado_outlet_flow_decoder_netflow_",
+		"packets_",
+		"sets_",
+		"records_",
 		"templates_",
 	)
 	expectedMetrics = map[string]string{
-		`flows_total{exporter="127.0.0.1",version="9"}`:                                                                 "4",
-		`flowset_records_sum{exporter="127.0.0.1",type="DataFlowSet",version="9"}`:                                      "4",
-		`flowset_records_sum{exporter="127.0.0.1",type="OptionsDataFlowSet",version="9"}`:                               "4",
-		`flowset_records_sum{exporter="127.0.0.1",type="OptionsTemplateFlowSet",version="9"}`:                           "1",
-		`flowset_records_sum{exporter="127.0.0.1",type="TemplateFlowSet",version="9"}`:                                  "1",
-		`flowset_sum{exporter="127.0.0.1",type="DataFlowSet",version="9"}`:                                              "1",
-		`flowset_sum{exporter="127.0.0.1",type="OptionsDataFlowSet",version="9"}`:                                       "1",
-		`flowset_sum{exporter="127.0.0.1",type="OptionsTemplateFlowSet",version="9"}`:                                   "1",
-		`flowset_sum{exporter="127.0.0.1",type="TemplateFlowSet",version="9"}`:                                          "1",
-		`templates_total{exporter="127.0.0.1",obs_domain_id="0",template_id="257",type="options_template",version="9"}`: "1",
-		`templates_total{exporter="127.0.0.1",obs_domain_id="0",template_id="260",type="template",version="9"}`:         "1",
+		`packets_total{exporter="::ffff:127.0.0.1",version="9"}`:                                                               "4",
+		`records_total{exporter="::ffff:127.0.0.1",type="DataFlowSet",version="9"}`:                                            "4",
+		`records_total{exporter="::ffff:127.0.0.1",type="OptionsDataFlowSet",version="9"}`:                                     "4",
+		`records_total{exporter="::ffff:127.0.0.1",type="OptionsTemplateFlowSet",version="9"}`:                                 "1",
+		`records_total{exporter="::ffff:127.0.0.1",type="TemplateFlowSet",version="9"}`:                                        "1",
+		`sets_total{exporter="::ffff:127.0.0.1",type="DataFlowSet",version="9"}`:                                               "1",
+		`sets_total{exporter="::ffff:127.0.0.1",type="OptionsDataFlowSet",version="9"}`:                                        "1",
+		`sets_total{exporter="::ffff:127.0.0.1",type="OptionsTemplateFlowSet",version="9"}`:                                    "1",
+		`sets_total{exporter="::ffff:127.0.0.1",type="TemplateFlowSet",version="9"}`:                                           "1",
+		`templates_total{exporter="::ffff:127.0.0.1",obs_domain_id="0",template_id="257",type="options_template",version="9"}`: "1",
+		`templates_total{exporter="::ffff:127.0.0.1",obs_domain_id="0",template_id="260",type="template",version="9"}`:         "1",
 	}
 	if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 		t.Fatalf("Metrics after data (-got, +want):\n%s", diff)
@@ -214,22 +240,24 @@ func TestDecode(t *testing.T) {
 }
 
 func TestTemplatesMixedWithData(t *testing.T) {
-	r := reporter.NewMock(t)
-	nfdecoder := New(r, decoder.Dependencies{Schema: schema.NewMock(t)}, decoder.Option{TimestampSource: decoder.TimestampSourceUDP})
+	r, nfdecoder, bf, _, finalize := setup(t, true)
+	options := decoder.Option{TimestampSource: pb.RawFlow_TS_INPUT}
 
 	// Send packet with both data and templates
 	template := helpers.ReadPcapL4(t, filepath.Join("testdata", "data+templates.pcap"))
-	nfdecoder.Decode(decoder.RawFlow{Payload: template, Source: net.ParseIP("127.0.0.1")})
+	nfdecoder.Decode(
+		decoder.RawFlow{Payload: template, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+		options, bf, finalize)
 
 	// We don't really care about the data, but we should have accepted the
 	// templates. Check the stats.
 	gotMetrics := r.GetMetrics(
-		"akvorado_inlet_flow_decoder_netflow_",
+		"akvorado_outlet_flow_decoder_netflow_",
 		"templates_",
 	)
 	expectedMetrics := map[string]string{
-		`templates_total{exporter="127.0.0.1",obs_domain_id="17170432",template_id="256",type="options_template",version="9"}`: "1",
-		`templates_total{exporter="127.0.0.1",obs_domain_id="17170432",template_id="257",type="template",version="9"}`:         "1",
+		`templates_total{exporter="::ffff:127.0.0.1",obs_domain_id="17170432",template_id="256",type="options_template",version="9"}`: "1",
+		`templates_total{exporter="::ffff:127.0.0.1",obs_domain_id="17170432",template_id="257",type="template",version="9"}`:         "1",
 	}
 	if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 		t.Fatalf("Metrics after data (-got, +want):\n%s", diff)
@@ -237,13 +265,24 @@ func TestTemplatesMixedWithData(t *testing.T) {
 }
 
 func TestDecodeSamplingRate(t *testing.T) {
-	r := reporter.NewMock(t)
-	nfdecoder := New(r, decoder.Dependencies{Schema: schema.NewMock(t).EnableAllColumns()}, decoder.Option{TimestampSource: decoder.TimestampSourceUDP})
+	_, nfdecoder, bf, got, finalize := setup(t, true)
+	options := decoder.Option{TimestampSource: pb.RawFlow_TS_INPUT}
 
 	data := helpers.ReadPcapL4(t, filepath.Join("testdata", "samplingrate-template.pcap"))
-	got := nfdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
+	_, err := nfdecoder.Decode(
+		decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+		options, bf, finalize)
+	if err != nil {
+		t.Fatalf("Decode() error:\n%+v", err)
+	}
+
 	data = helpers.ReadPcapL4(t, filepath.Join("testdata", "samplingrate-data.pcap"))
-	got = append(got, nfdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})...)
+	_, err = nfdecoder.Decode(
+		decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+		options, bf, finalize)
+	if err != nil {
+		t.Fatalf("Decode() error:\n%+v", err)
+	}
 
 	expectedFlows := []*schema.FlowMessage{
 		{
@@ -254,7 +293,7 @@ func TestDecodeSamplingRate(t *testing.T) {
 			InIf:            13,
 			SrcVlan:         701,
 			NextHop:         netip.MustParseAddr("::ffff:0.0.0.0"),
-			ProtobufDebug: map[schema.ColumnKey]interface{}{
+			OtherColumns: map[schema.ColumnKey]any{
 				schema.ColumnPackets: 1,
 				schema.ColumnBytes:   160,
 				schema.ColumnProto:   6,
@@ -265,27 +304,43 @@ func TestDecodeSamplingRate(t *testing.T) {
 		},
 	}
 
-	for _, f := range got {
-		f.TimeReceived = 0
-	}
-
-	if diff := helpers.Diff(got[:1], expectedFlows); diff != "" {
+	if diff := helpers.Diff((*got)[:1], expectedFlows); diff != "" {
 		t.Fatalf("Decode() (-got, +want):\n%s", diff)
 	}
 }
 
 func TestDecodeMultipleSamplingRates(t *testing.T) {
-	r := reporter.NewMock(t)
-	nfdecoder := New(r, decoder.Dependencies{Schema: schema.NewMock(t).EnableAllColumns()}, decoder.Option{TimestampSource: decoder.TimestampSourceUDP})
+	_, nfdecoder, bf, got, finalize := setup(t, true)
+	options := decoder.Option{TimestampSource: pb.RawFlow_TS_INPUT}
 
 	data := helpers.ReadPcapL4(t, filepath.Join("testdata", "multiplesamplingrates-options-template.pcap"))
-	got := nfdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
+	_, err := nfdecoder.Decode(
+		decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+		options, bf, finalize)
+	if err != nil {
+		t.Fatalf("Decode() error:\n%+v", err)
+	}
 	data = helpers.ReadPcapL4(t, filepath.Join("testdata", "multiplesamplingrates-options-data.pcap"))
-	got = nfdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
+	_, err = nfdecoder.Decode(
+		decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+		options, bf, finalize)
+	if err != nil {
+		t.Fatalf("Decode() error:\n%+v", err)
+	}
 	data = helpers.ReadPcapL4(t, filepath.Join("testdata", "multiplesamplingrates-template.pcap"))
-	got = nfdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
+	_, err = nfdecoder.Decode(
+		decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+		options, bf, finalize)
+	if err != nil {
+		t.Fatalf("Decode() error:\n%+v", err)
+	}
 	data = helpers.ReadPcapL4(t, filepath.Join("testdata", "multiplesamplingrates-data.pcap"))
-	got = nfdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
+	_, err = nfdecoder.Decode(
+		decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+		options, bf, finalize)
+	if err != nil {
+		t.Fatalf("Decode() error:\n%+v", err)
+	}
 
 	expectedFlows := []*schema.FlowMessage{
 		{
@@ -298,7 +353,7 @@ func TestDecodeMultipleSamplingRates(t *testing.T) {
 			DstNetMask:      56,
 			InIf:            97,
 			OutIf:           6,
-			ProtobufDebug: map[schema.ColumnKey]interface{}{
+			OtherColumns: map[schema.ColumnKey]any{
 				schema.ColumnPackets:          18,
 				schema.ColumnBytes:            1348,
 				schema.ColumnProto:            6,
@@ -322,7 +377,7 @@ func TestDecodeMultipleSamplingRates(t *testing.T) {
 			DstNetMask:      48,
 			InIf:            103,
 			OutIf:           6,
-			ProtobufDebug: map[schema.ColumnKey]interface{}{
+			OtherColumns: map[schema.ColumnKey]any{
 				schema.ColumnPackets:          4,
 				schema.ColumnBytes:            579,
 				schema.ColumnProto:            17,
@@ -337,30 +392,36 @@ func TestDecodeMultipleSamplingRates(t *testing.T) {
 		},
 	}
 
-	for _, f := range got {
-		f.TimeReceived = 0
-	}
-
-	if diff := helpers.Diff(got[:2], expectedFlows); diff != "" {
+	if diff := helpers.Diff((*got)[:2], expectedFlows); diff != "" {
 		t.Fatalf("Decode() (-got, +want):\n%s", diff)
 	}
 }
 
 func TestDecodeICMP(t *testing.T) {
-	r := reporter.NewMock(t)
-	nfdecoder := New(r, decoder.Dependencies{Schema: schema.NewMock(t).EnableAllColumns()}, decoder.Option{TimestampSource: decoder.TimestampSourceUDP})
+	_, nfdecoder, bf, got, finalize := setup(t, true)
+	options := decoder.Option{TimestampSource: pb.RawFlow_TS_INPUT}
 
 	data := helpers.ReadPcapL4(t, filepath.Join("testdata", "icmp-template.pcap"))
-	got := nfdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
+	_, err := nfdecoder.Decode(
+		decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+		options, bf, finalize)
+	if err != nil {
+		t.Fatalf("Decode() error:\n%+v", err)
+	}
 	data = helpers.ReadPcapL4(t, filepath.Join("testdata", "icmp-data.pcap"))
-	got = append(got, nfdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})...)
+	_, err = nfdecoder.Decode(
+		decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+		options, bf, finalize)
+	if err != nil {
+		t.Fatalf("Decode() error:\n%+v", err)
+	}
 
 	expectedFlows := []*schema.FlowMessage{
 		{
 			ExporterAddress: netip.MustParseAddr("::ffff:127.0.0.1"),
 			SrcAddr:         netip.MustParseAddr("2001:db8::"),
 			DstAddr:         netip.MustParseAddr("2001:db8::1"),
-			ProtobufDebug: map[schema.ColumnKey]interface{}{
+			OtherColumns: map[schema.ColumnKey]any{
 				schema.ColumnBytes:      104,
 				schema.ColumnDstPort:    32768,
 				schema.ColumnEType:      34525,
@@ -373,7 +434,7 @@ func TestDecodeICMP(t *testing.T) {
 			ExporterAddress: netip.MustParseAddr("::ffff:127.0.0.1"),
 			SrcAddr:         netip.MustParseAddr("2001:db8::1"),
 			DstAddr:         netip.MustParseAddr("2001:db8::"),
-			ProtobufDebug: map[schema.ColumnKey]interface{}{
+			OtherColumns: map[schema.ColumnKey]any{
 				schema.ColumnBytes:      104,
 				schema.ColumnDstPort:    33024,
 				schema.ColumnEType:      34525,
@@ -386,7 +447,7 @@ func TestDecodeICMP(t *testing.T) {
 			ExporterAddress: netip.MustParseAddr("::ffff:127.0.0.1"),
 			SrcAddr:         netip.MustParseAddr("::ffff:203.0.113.4"),
 			DstAddr:         netip.MustParseAddr("::ffff:203.0.113.5"),
-			ProtobufDebug: map[schema.ColumnKey]interface{}{
+			OtherColumns: map[schema.ColumnKey]any{
 				schema.ColumnBytes:      84,
 				schema.ColumnDstPort:    2048,
 				schema.ColumnEType:      2048,
@@ -399,7 +460,7 @@ func TestDecodeICMP(t *testing.T) {
 			ExporterAddress: netip.MustParseAddr("::ffff:127.0.0.1"),
 			SrcAddr:         netip.MustParseAddr("::ffff:203.0.113.5"),
 			DstAddr:         netip.MustParseAddr("::ffff:203.0.113.4"),
-			ProtobufDebug: map[schema.ColumnKey]interface{}{
+			OtherColumns: map[schema.ColumnKey]any{
 				schema.ColumnBytes:   84,
 				schema.ColumnEType:   2048,
 				schema.ColumnPackets: 1,
@@ -408,9 +469,6 @@ func TestDecodeICMP(t *testing.T) {
 			},
 		},
 	}
-	for _, f := range got {
-		f.TimeReceived = 0
-	}
 
 	if diff := helpers.Diff(got, expectedFlows); diff != "" {
 		t.Fatalf("Decode() (-got, +want):\n%s", diff)
@@ -419,13 +477,23 @@ func TestDecodeICMP(t *testing.T) {
 }
 
 func TestDecodeDataLink(t *testing.T) {
-	r := reporter.NewMock(t)
-	nfdecoder := New(r, decoder.Dependencies{Schema: schema.NewMock(t).EnableAllColumns()}, decoder.Option{TimestampSource: decoder.TimestampSourceUDP})
+	_, nfdecoder, bf, got, finalize := setup(t, true)
+	options := decoder.Option{TimestampSource: pb.RawFlow_TS_INPUT}
 
 	data := helpers.ReadPcapL4(t, filepath.Join("testdata", "datalink-template.pcap"))
-	got := nfdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
+	_, err := nfdecoder.Decode(
+		decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+		options, bf, finalize)
+	if err != nil {
+		t.Fatalf("Decode() error:\n%+v", err)
+	}
 	data = helpers.ReadPcapL4(t, filepath.Join("testdata", "datalink-data.pcap"))
-	got = append(got, nfdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})...)
+	_, err = nfdecoder.Decode(
+		decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+		options, bf, finalize)
+	if err != nil {
+		t.Fatalf("Decode() error:\n%+v", err)
+	}
 
 	expectedFlows := []*schema.FlowMessage{
 		{
@@ -435,7 +503,7 @@ func TestDecodeDataLink(t *testing.T) {
 			SrcVlan:         231,
 			InIf:            582,
 			OutIf:           0,
-			ProtobufDebug: map[schema.ColumnKey]interface{}{
+			OtherColumns: map[schema.ColumnKey]any{
 				schema.ColumnBytes:        96,
 				schema.ColumnSrcPort:      55501,
 				schema.ColumnDstPort:      11777,
@@ -449,9 +517,6 @@ func TestDecodeDataLink(t *testing.T) {
 			},
 		},
 	}
-	for _, f := range got {
-		f.TimeReceived = 0
-	}
 
 	if diff := helpers.Diff(got, expectedFlows); diff != "" {
 		t.Fatalf("Decode() (-got, +want):\n%s", diff)
@@ -459,12 +524,16 @@ func TestDecodeDataLink(t *testing.T) {
 }
 
 func TestDecodeWithoutTemplate(t *testing.T) {
-	r := reporter.NewMock(t)
-	nfdecoder := New(r,
-		decoder.Dependencies{Schema: schema.NewMock(t).EnableAllColumns()},
-		decoder.Option{TimestampSource: decoder.TimestampSourceUDP})
+	_, nfdecoder, bf, got, finalize := setup(t, true)
+	options := decoder.Option{TimestampSource: pb.RawFlow_TS_INPUT}
+
 	data := helpers.ReadPcapL4(t, filepath.Join("testdata", "datalink-data.pcap"))
-	got := nfdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
+	_, err := nfdecoder.Decode(
+		decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+		options, bf, finalize)
+	if err != nil {
+		t.Fatalf("Decode() error:\n%+v", err)
+	}
 
 	expectedFlows := []*schema.FlowMessage{}
 	if diff := helpers.Diff(got, expectedFlows); diff != "" {
@@ -473,11 +542,16 @@ func TestDecodeWithoutTemplate(t *testing.T) {
 }
 
 func TestDecodeMPLS(t *testing.T) {
-	r := reporter.NewMock(t)
-	nfdecoder := New(r, decoder.Dependencies{Schema: schema.NewMock(t).EnableAllColumns()}, decoder.Option{TimestampSource: decoder.TimestampSourceUDP})
+	_, nfdecoder, bf, got, finalize := setup(t, true)
+	options := decoder.Option{TimestampSource: pb.RawFlow_TS_INPUT}
 
 	data := helpers.ReadPcapL4(t, filepath.Join("testdata", "mpls.pcap"))
-	got := nfdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
+	_, err := nfdecoder.Decode(
+		decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+		options, bf, finalize)
+	if err != nil {
+		t.Fatalf("Decode() error:\n%+v", err)
+	}
 
 	expectedFlows := []*schema.FlowMessage{
 		{
@@ -487,7 +561,7 @@ func TestDecodeMPLS(t *testing.T) {
 			NextHop:         netip.MustParseAddr("::ffff:0.0.0.0"),
 			SamplingRate:    10,
 			OutIf:           16,
-			ProtobufDebug: map[schema.ColumnKey]interface{}{
+			OtherColumns: map[schema.ColumnKey]any{
 				schema.ColumnBytes:            89,
 				schema.ColumnPackets:          1,
 				schema.ColumnEType:            helpers.ETypeIPv6,
@@ -505,7 +579,7 @@ func TestDecodeMPLS(t *testing.T) {
 			NextHop:         netip.MustParseAddr("::ffff:0.0.0.0"),
 			SamplingRate:    10,
 			OutIf:           17,
-			ProtobufDebug: map[schema.ColumnKey]interface{}{
+			OtherColumns: map[schema.ColumnKey]any{
 				schema.ColumnBytes:            890,
 				schema.ColumnPackets:          10,
 				schema.ColumnEType:            helpers.ETypeIPv6,
@@ -518,9 +592,6 @@ func TestDecodeMPLS(t *testing.T) {
 			},
 		},
 	}
-	for _, f := range got {
-		f.TimeReceived = 0
-	}
 
 	if diff := helpers.Diff(got, expectedFlows); diff != "" {
 		t.Fatalf("Decode() (-got, +want):\n%s", diff)
@@ -528,21 +599,24 @@ func TestDecodeMPLS(t *testing.T) {
 }
 
 func TestDecodeNFv5(t *testing.T) {
-	for _, tsSource := range []decoder.TimestampSource{
-		decoder.TimestampSourceNetflowPacket,
-		decoder.TimestampSourceNetflowFirstSwitched,
+	for _, tsSource := range []pb.RawFlow_TimestampSource{
+		pb.RawFlow_TS_NETFLOW_PACKET,
+		pb.RawFlow_TS_NETFLOW_FIRST_SWITCHED,
 	} {
 		t.Run(fmt.Sprintf("%s", tsSource), func(t *testing.T) {
-			r := reporter.NewMock(t)
-			nfdecoder := New(r,
-				decoder.Dependencies{Schema: schema.NewMock(t).EnableAllColumns()},
-				decoder.Option{TimestampSource: tsSource})
+			_, nfdecoder, bf, got, finalize := setup(t, false)
+			options := decoder.Option{TimestampSource: tsSource}
 
 			data := helpers.ReadPcapL4(t, filepath.Join("testdata", "nfv5.pcap"))
-			got := nfdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
+			_, err := nfdecoder.Decode(
+				decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+				options, bf, finalize)
+			if err != nil {
+				t.Fatalf("Decode() error:\n%+v", err)
+			}
 
-			ts := uint64(1680626679)
-			if tsSource == decoder.TimestampSourceNetflowFirstSwitched {
+			ts := uint32(1680626679)
+			if tsSource == pb.RawFlow_TS_NETFLOW_FIRST_SWITCHED {
 				ts = 1680611679
 			}
 
@@ -560,7 +634,7 @@ func TestDecodeNFv5(t *testing.T) {
 					DstAS:           10101,
 					SrcNetMask:      19,
 					DstNetMask:      24,
-					ProtobufDebug: map[schema.ColumnKey]interface{}{
+					OtherColumns: map[schema.ColumnKey]any{
 						schema.ColumnBytes:    133,
 						schema.ColumnPackets:  1,
 						schema.ColumnEType:    helpers.ETypeIPv4,
@@ -572,7 +646,7 @@ func TestDecodeNFv5(t *testing.T) {
 				},
 			}
 
-			if diff := helpers.Diff(got[:1], expectedFlows); diff != "" {
+			if diff := helpers.Diff((*got)[:1], expectedFlows); diff != "" {
 				t.Errorf("Decode() (-got, +want):\n%s", diff)
 			}
 		})
@@ -580,24 +654,34 @@ func TestDecodeNFv5(t *testing.T) {
 }
 
 func TestDecodeTimestampFromNetflowPacket(t *testing.T) {
-	r := reporter.NewMock(t)
-	nfdecoder := New(r, decoder.Dependencies{Schema: schema.NewMock(t).EnableAllColumns()}, decoder.Option{TimestampSource: decoder.TimestampSourceNetflowPacket})
+	_, nfdecoder, bf, got, finalize := setup(t, false)
+	options := decoder.Option{TimestampSource: pb.RawFlow_TS_NETFLOW_PACKET}
 
 	data := helpers.ReadPcapL4(t, filepath.Join("testdata", "template.pcap"))
-	got := nfdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
+	_, err := nfdecoder.Decode(
+		decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+		options, bf, finalize)
+	if err != nil {
+		t.Fatalf("Decode() error:\n%+v", err)
+	}
 	data = helpers.ReadPcapL4(t, filepath.Join("testdata", "data.pcap"))
-	got = append(got, nfdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})...)
+	_, err = nfdecoder.Decode(
+		decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+		options, bf, finalize)
+	if err != nil {
+		t.Fatalf("Decode() error:\n%+v", err)
+	}
 
 	// 4 flows in capture
 	// all share the same timestamp with TimestampSourceNetflowPacket
-	expectedTs := []uint64{
+	expectedTs := []uint32{
 		1647285928,
 		1647285928,
 		1647285928,
 		1647285928,
 	}
 
-	for i, flow := range got {
+	for i, flow := range *got {
 		if flow.TimeReceived != expectedTs[i] {
 			t.Errorf("Decode() (-got, +want):\n-%d, +%d", flow.TimeReceived, expectedTs[i])
 		}
@@ -605,25 +689,35 @@ func TestDecodeTimestampFromNetflowPacket(t *testing.T) {
 }
 
 func TestDecodeTimestampFromFirstSwitched(t *testing.T) {
-	r := reporter.NewMock(t)
-	nfdecoder := New(r, decoder.Dependencies{Schema: schema.NewMock(t).EnableAllColumns()}, decoder.Option{TimestampSource: decoder.TimestampSourceNetflowFirstSwitched})
+	_, nfdecoder, bf, got, finalize := setup(t, false)
+	options := decoder.Option{TimestampSource: pb.RawFlow_TS_NETFLOW_FIRST_SWITCHED}
 
 	data := helpers.ReadPcapL4(t, filepath.Join("testdata", "template.pcap"))
-	got := nfdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
+	_, err := nfdecoder.Decode(
+		decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+		options, bf, finalize)
+	if err != nil {
+		t.Fatalf("Decode() error:\n%+v", err)
+	}
 	data = helpers.ReadPcapL4(t, filepath.Join("testdata", "data.pcap"))
-	got = append(got, nfdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})...)
+	_, err = nfdecoder.Decode(
+		decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+		options, bf, finalize)
+	if err != nil {
+		t.Fatalf("Decode() error:\n%+v", err)
+	}
 
 	// 4 flows in capture
-	var sysUptime uint64 = 944951609
-	var packetTs uint64 = 1647285928
-	expectedFirstSwitched := []uint64{
+	var sysUptime uint32 = 944951609
+	var packetTs uint32 = 1647285928
+	expectedFirstSwitched := []uint32{
 		944948659,
 		944948659,
 		944948660,
 		944948661,
 	}
 
-	for i, flow := range got {
+	for i, flow := range *got {
 		if val := packetTs - sysUptime + expectedFirstSwitched[i]; flow.TimeReceived != val {
 			t.Errorf("Decode() (-got, +want):\n-%d, +%d", flow.TimeReceived, val)
 		}
@@ -631,20 +725,25 @@ func TestDecodeTimestampFromFirstSwitched(t *testing.T) {
 }
 
 func TestDecodeNAT(t *testing.T) {
-	r := reporter.NewMock(t)
-	nfdecoder := New(r, decoder.Dependencies{Schema: schema.NewMock(t).EnableAllColumns()}, decoder.Option{TimestampSource: decoder.TimestampSourceUDP})
+	_, nfdecoder, bf, got, finalize := setup(t, true)
+	options := decoder.Option{TimestampSource: pb.RawFlow_TS_INPUT}
 
 	// The following PCAP is a NAT event, there is no sampling rate, no bytes,
 	// no packets. We can't do much with it.
 	data := helpers.ReadPcapL4(t, filepath.Join("testdata", "nat.pcap"))
-	got := nfdecoder.Decode(decoder.RawFlow{Payload: data, Source: net.ParseIP("127.0.0.1")})
+	_, err := nfdecoder.Decode(
+		decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+		options, bf, finalize)
+	if err != nil {
+		t.Fatalf("Decode() error:\n%+v", err)
+	}
 
 	expectedFlows := []*schema.FlowMessage{
 		{
 			ExporterAddress: netip.MustParseAddr("::ffff:127.0.0.1"),
 			SrcAddr:         netip.MustParseAddr("::ffff:172.16.100.198"),
 			DstAddr:         netip.MustParseAddr("::ffff:10.89.87.1"),
-			ProtobufDebug: map[schema.ColumnKey]any{
+			OtherColumns: map[schema.ColumnKey]any{
 				schema.ColumnSrcPort:    35303,
 				schema.ColumnDstPort:    53,
 				schema.ColumnSrcAddrNAT: netip.MustParseAddr("::ffff:10.143.52.29"),
@@ -656,11 +755,8 @@ func TestDecodeNAT(t *testing.T) {
 			},
 		},
 	}
-	for _, f := range got {
-		f.TimeReceived = 0
-	}
 
-	if diff := helpers.Diff(got[:1], expectedFlows); diff != "" {
+	if diff := helpers.Diff((*got)[:1], expectedFlows); diff != "" {
 		t.Fatalf("Decode() (-got, +want):\n%s", diff)
 	}
 }
diff --git a/inlet/flow/decoder/netflow/testdata/data+templates.pcap b/outlet/flow/decoder/netflow/testdata/data+templates.pcap
similarity index 100%
rename from inlet/flow/decoder/netflow/testdata/data+templates.pcap
rename to outlet/flow/decoder/netflow/testdata/data+templates.pcap
diff --git a/inlet/flow/decoder/netflow/testdata/data.pcap b/outlet/flow/decoder/netflow/testdata/data.pcap
similarity index 100%
rename from inlet/flow/decoder/netflow/testdata/data.pcap
rename to outlet/flow/decoder/netflow/testdata/data.pcap
diff --git a/inlet/flow/decoder/netflow/testdata/datalink-data.pcap b/outlet/flow/decoder/netflow/testdata/datalink-data.pcap
similarity index 100%
rename from inlet/flow/decoder/netflow/testdata/datalink-data.pcap
rename to outlet/flow/decoder/netflow/testdata/datalink-data.pcap
diff --git a/inlet/flow/decoder/netflow/testdata/datalink-template.pcap b/outlet/flow/decoder/netflow/testdata/datalink-template.pcap
similarity index 100%
rename from inlet/flow/decoder/netflow/testdata/datalink-template.pcap
rename to outlet/flow/decoder/netflow/testdata/datalink-template.pcap
diff --git a/inlet/flow/decoder/netflow/testdata/icmp-data.pcap b/outlet/flow/decoder/netflow/testdata/icmp-data.pcap
similarity index 100%
rename from inlet/flow/decoder/netflow/testdata/icmp-data.pcap
rename to outlet/flow/decoder/netflow/testdata/icmp-data.pcap
diff --git a/inlet/flow/decoder/netflow/testdata/icmp-template.pcap b/outlet/flow/decoder/netflow/testdata/icmp-template.pcap
similarity index 100%
rename from inlet/flow/decoder/netflow/testdata/icmp-template.pcap
rename to outlet/flow/decoder/netflow/testdata/icmp-template.pcap
diff --git a/inlet/flow/decoder/netflow/testdata/mpls.pcap b/outlet/flow/decoder/netflow/testdata/mpls.pcap
similarity index 100%
rename from inlet/flow/decoder/netflow/testdata/mpls.pcap
rename to outlet/flow/decoder/netflow/testdata/mpls.pcap
diff --git a/inlet/flow/decoder/netflow/testdata/multiplesamplingrates-data.pcap b/outlet/flow/decoder/netflow/testdata/multiplesamplingrates-data.pcap
similarity index 100%
rename from inlet/flow/decoder/netflow/testdata/multiplesamplingrates-data.pcap
rename to outlet/flow/decoder/netflow/testdata/multiplesamplingrates-data.pcap
diff --git a/inlet/flow/decoder/netflow/testdata/multiplesamplingrates-options-data.pcap b/outlet/flow/decoder/netflow/testdata/multiplesamplingrates-options-data.pcap
similarity index 100%
rename from inlet/flow/decoder/netflow/testdata/multiplesamplingrates-options-data.pcap
rename to outlet/flow/decoder/netflow/testdata/multiplesamplingrates-options-data.pcap
diff --git a/inlet/flow/decoder/netflow/testdata/multiplesamplingrates-options-template.pcap b/outlet/flow/decoder/netflow/testdata/multiplesamplingrates-options-template.pcap
similarity index 100%
rename from inlet/flow/decoder/netflow/testdata/multiplesamplingrates-options-template.pcap
rename to outlet/flow/decoder/netflow/testdata/multiplesamplingrates-options-template.pcap
diff --git a/inlet/flow/decoder/netflow/testdata/multiplesamplingrates-template.pcap b/outlet/flow/decoder/netflow/testdata/multiplesamplingrates-template.pcap
similarity index 100%
rename from inlet/flow/decoder/netflow/testdata/multiplesamplingrates-template.pcap
rename to outlet/flow/decoder/netflow/testdata/multiplesamplingrates-template.pcap
diff --git a/inlet/flow/decoder/netflow/testdata/nat.pcap b/outlet/flow/decoder/netflow/testdata/nat.pcap
similarity index 100%
rename from inlet/flow/decoder/netflow/testdata/nat.pcap
rename to outlet/flow/decoder/netflow/testdata/nat.pcap
diff --git a/inlet/flow/decoder/netflow/testdata/nfv5.pcap b/outlet/flow/decoder/netflow/testdata/nfv5.pcap
similarity index 100%
rename from inlet/flow/decoder/netflow/testdata/nfv5.pcap
rename to outlet/flow/decoder/netflow/testdata/nfv5.pcap
diff --git a/inlet/flow/decoder/netflow/testdata/options-data.pcap b/outlet/flow/decoder/netflow/testdata/options-data.pcap
similarity index 100%
rename from inlet/flow/decoder/netflow/testdata/options-data.pcap
rename to outlet/flow/decoder/netflow/testdata/options-data.pcap
diff --git a/inlet/flow/decoder/netflow/testdata/options-template.pcap b/outlet/flow/decoder/netflow/testdata/options-template.pcap
similarity index 100%
rename from inlet/flow/decoder/netflow/testdata/options-template.pcap
rename to outlet/flow/decoder/netflow/testdata/options-template.pcap
diff --git a/inlet/flow/decoder/netflow/testdata/samplingrate-data.pcap b/outlet/flow/decoder/netflow/testdata/samplingrate-data.pcap
similarity index 100%
rename from inlet/flow/decoder/netflow/testdata/samplingrate-data.pcap
rename to outlet/flow/decoder/netflow/testdata/samplingrate-data.pcap
diff --git a/inlet/flow/decoder/netflow/testdata/samplingrate-template.pcap b/outlet/flow/decoder/netflow/testdata/samplingrate-template.pcap
similarity index 100%
rename from inlet/flow/decoder/netflow/testdata/samplingrate-template.pcap
rename to outlet/flow/decoder/netflow/testdata/samplingrate-template.pcap
diff --git a/inlet/flow/decoder/netflow/testdata/template.pcap b/outlet/flow/decoder/netflow/testdata/template.pcap
similarity index 100%
rename from inlet/flow/decoder/netflow/testdata/template.pcap
rename to outlet/flow/decoder/netflow/testdata/template.pcap
diff --git a/outlet/flow/decoder/sflow/root_test.go b/outlet/flow/decoder/sflow/root_test.go
new file mode 100644
index 00000000..c0695e2f
--- /dev/null
+++ b/outlet/flow/decoder/sflow/root_test.go
@@ -0,0 +1,554 @@
+// SPDX-FileCopyrightText: 2022 Tchadel Icard
+// SPDX-License-Identifier: AGPL-3.0-only
+
+package sflow
+
+import (
+	"net/netip"
+	"path/filepath"
+	"testing"
+
+	"akvorado/common/helpers"
+	"akvorado/common/reporter"
+	"akvorado/common/schema"
+	"akvorado/outlet/flow/decoder"
+)
+
+func TestDecode(t *testing.T) {
+	r := reporter.NewMock(t)
+	sch := schema.NewMock(t).EnableAllColumns()
+	sdecoder := New(r, decoder.Dependencies{Schema: sch})
+	options := decoder.Option{}
+	bf := sch.NewFlowMessage()
+	got := []*schema.FlowMessage{}
+	finalize := func() {
+		bf.TimeReceived = 0
+		// Keep a copy of the current flow message
+		clone := *bf
+		got = append(got, &clone)
+		// And clear the flow message
+		bf.Clear()
+	}
+
+	// Send data
+	t.Run("basic", func(t *testing.T) {
+		got = got[:0]
+		data := helpers.ReadPcapL4(t, filepath.Join("testdata", "data-1140.pcap"))
+		_, err := sdecoder.Decode(
+			decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+			options, bf, finalize)
+		if err != nil {
+			t.Fatalf("Decode() error:\n%+v", err)
+		}
+		expectedFlows := []*schema.FlowMessage{
+			{
+				SamplingRate:    1024,
+				InIf:            27,
+				OutIf:           28,
+				SrcVlan:         100,
+				DstVlan:         100,
+				SrcAddr:         netip.MustParseAddr("2a0c:8880:2:0:185:21:130:38"),
+				DstAddr:         netip.MustParseAddr("2a0c:8880:2:0:185:21:130:39"),
+				ExporterAddress: netip.MustParseAddr("::ffff:172.16.0.3"),
+				OtherColumns: map[schema.ColumnKey]any{
+					schema.ColumnBytes:         1500,
+					schema.ColumnPackets:       1,
+					schema.ColumnEType:         helpers.ETypeIPv6,
+					schema.ColumnProto:         6,
+					schema.ColumnSrcPort:       46026,
+					schema.ColumnDstPort:       22,
+					schema.ColumnSrcMAC:        40057391053392,
+					schema.ColumnDstMAC:        40057381862408,
+					schema.ColumnIPTTL:         64,
+					schema.ColumnIPTos:         0x8,
+					schema.ColumnIPv6FlowLabel: 0x68094,
+					schema.ColumnTCPFlags:      0x10,
+				},
+			}, {
+				SamplingRate:    1024,
+				SrcAddr:         netip.MustParseAddr("::ffff:104.26.8.24"),
+				DstAddr:         netip.MustParseAddr("::ffff:45.90.161.46"),
+				ExporterAddress: netip.MustParseAddr("::ffff:172.16.0.3"),
+				NextHop:         netip.MustParseAddr("::ffff:45.90.161.46"),
+				InIf:            49001,
+				OutIf:           25,
+				DstVlan:         100,
+				SrcAS:           13335,
+				DstAS:           39421,
+				SrcNetMask:      20,
+				DstNetMask:      27,
+				OtherColumns: map[schema.ColumnKey]any{
+					schema.ColumnBytes:        421,
+					schema.ColumnPackets:      1,
+					schema.ColumnEType:        helpers.ETypeIPv4,
+					schema.ColumnProto:        6,
+					schema.ColumnSrcPort:      443,
+					schema.ColumnDstPort:      56876,
+					schema.ColumnSrcMAC:       216372595274807,
+					schema.ColumnDstMAC:       191421060163210,
+					schema.ColumnIPFragmentID: 0xa572,
+					schema.ColumnIPTTL:        59,
+					schema.ColumnTCPFlags:     0x18,
+				},
+			}, {
+				SamplingRate:    1024,
+				SrcAddr:         netip.MustParseAddr("2a0c:8880:2:0:185:21:130:38"),
+				DstAddr:         netip.MustParseAddr("2a0c:8880:2:0:185:21:130:39"),
+				ExporterAddress: netip.MustParseAddr("::ffff:172.16.0.3"),
+				InIf:            27,
+				OutIf:           28,
+				SrcVlan:         100,
+				DstVlan:         100,
+				OtherColumns: map[schema.ColumnKey]any{
+					schema.ColumnBytes:         1500,
+					schema.ColumnPackets:       1,
+					schema.ColumnEType:         helpers.ETypeIPv6,
+					schema.ColumnProto:         6,
+					schema.ColumnSrcPort:       46026,
+					schema.ColumnDstPort:       22,
+					schema.ColumnSrcMAC:        40057391053392,
+					schema.ColumnDstMAC:        40057381862408,
+					schema.ColumnIPTTL:         64,
+					schema.ColumnIPTos:         0x8,
+					schema.ColumnIPv6FlowLabel: 0x68094,
+					schema.ColumnTCPFlags:      0x10,
+				},
+			}, {
+				SamplingRate:    1024,
+				InIf:            28,
+				OutIf:           49001,
+				SrcVlan:         100,
+				SrcAS:           39421,
+				DstAS:           26615,
+				SrcAddr:         netip.MustParseAddr("::ffff:45.90.161.148"),
+				DstAddr:         netip.MustParseAddr("::ffff:191.87.91.27"),
+				ExporterAddress: netip.MustParseAddr("::ffff:172.16.0.3"),
+				NextHop:         netip.MustParseAddr("::ffff:31.14.69.110"),
+				SrcNetMask:      27,
+				DstNetMask:      17,
+				OtherColumns: map[schema.ColumnKey]any{
+					schema.ColumnBytes:          40,
+					schema.ColumnPackets:        1,
+					schema.ColumnEType:          helpers.ETypeIPv4,
+					schema.ColumnProto:          6,
+					schema.ColumnSrcPort:        55658,
+					schema.ColumnDstPort:        5555,
+					schema.ColumnSrcMAC:         138617863011056,
+					schema.ColumnDstMAC:         216372595274807,
+					schema.ColumnDstASPath:      []uint32{203698, 6762, 26615},
+					schema.ColumnDstCommunities: []uint64{2583495656, 2583495657, 4259880000, 4259880001, 4259900001},
+					schema.ColumnIPFragmentID:   0xd431,
+					schema.ColumnIPTTL:          255,
+					schema.ColumnTCPFlags:       0x2,
+				},
+			}, {
+				SamplingRate:    1024,
+				SrcAddr:         netip.MustParseAddr("2a0c:8880:2:0:185:21:130:38"),
+				DstAddr:         netip.MustParseAddr("2a0c:8880:2:0:185:21:130:39"),
+				ExporterAddress: netip.MustParseAddr("::ffff:172.16.0.3"),
+				InIf:            27,
+				OutIf:           28,
+				SrcVlan:         100,
+				DstVlan:         100,
+				OtherColumns: map[schema.ColumnKey]any{
+					schema.ColumnBytes:         1500,
+					schema.ColumnPackets:       1,
+					schema.ColumnEType:         helpers.ETypeIPv6,
+					schema.ColumnProto:         6,
+					schema.ColumnSrcPort:       46026,
+					schema.ColumnDstPort:       22,
+					schema.ColumnSrcMAC:        40057391053392,
+					schema.ColumnDstMAC:        40057381862408,
+					schema.ColumnIPTTL:         64,
+					schema.ColumnIPTos:         0x8,
+					schema.ColumnIPv6FlowLabel: 0x68094,
+					schema.ColumnTCPFlags:      0x10,
+				},
+			},
+		}
+
+		if diff := helpers.Diff(got, expectedFlows); diff != "" {
+			t.Fatalf("Decode() (-got, +want):\n%s", diff)
+		}
+		gotMetrics := r.GetMetrics(
+			"akvorado_outlet_flow_decoder_sflow_",
+			"flows_total",
+			"sample_",
+		)
+		expectedMetrics := map[string]string{
+			`flows_total{agent="172.16.0.3",exporter="::ffff:127.0.0.1",version="5"}`:                          "1",
+			`sample_records_sum{agent="172.16.0.3",exporter="::ffff:127.0.0.1",type="FlowSample",version="5"}`: "14",
+			`sample_sum{agent="172.16.0.3",exporter="::ffff:127.0.0.1",type="FlowSample",version="5"}`:         "5",
+		}
+		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
+			t.Fatalf("Metrics after data (-got, +want):\n%s", diff)
+		}
+	})
+
+	t.Run("local interface", func(t *testing.T) {
+		got = got[:0]
+		data := helpers.ReadPcapL4(t, filepath.Join("testdata", "data-local-interface.pcap"))
+		_, err := sdecoder.Decode(
+			decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+			options, bf, finalize)
+		if err != nil {
+			t.Fatalf("Decode() error:\n%+v", err)
+		}
+		expectedFlows := []*schema.FlowMessage{
+			{
+				SamplingRate:    1024,
+				SrcAddr:         netip.MustParseAddr("2a0c:8880:2:0:185:21:130:38"),
+				DstAddr:         netip.MustParseAddr("2a0c:8880:2:0:185:21:130:39"),
+				ExporterAddress: netip.MustParseAddr("::ffff:172.16.0.3"),
+				InIf:            27,
+				OutIf:           0, // local interface
+				SrcVlan:         100,
+				DstVlan:         100,
+				OtherColumns: map[schema.ColumnKey]any{
+					schema.ColumnBytes:         1500,
+					schema.ColumnPackets:       1,
+					schema.ColumnEType:         helpers.ETypeIPv6,
+					schema.ColumnProto:         6,
+					schema.ColumnSrcPort:       46026,
+					schema.ColumnDstPort:       22,
+					schema.ColumnSrcMAC:        40057391053392,
+					schema.ColumnDstMAC:        40057381862408,
+					schema.ColumnTCPFlags:      16,
+					schema.ColumnIPv6FlowLabel: 426132,
+					schema.ColumnIPTTL:         64,
+					schema.ColumnIPTos:         8,
+				},
+			},
+		}
+
+		if diff := helpers.Diff(got, expectedFlows); diff != "" {
+			t.Fatalf("Decode() (-got, +want):\n%s", diff)
+		}
+	})
+
+	t.Run("discard interface", func(t *testing.T) {
+		got = got[:0]
+		data := helpers.ReadPcapL4(t, filepath.Join("testdata", "data-discard-interface.pcap"))
+		_, err := sdecoder.Decode(
+			decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+			options, bf, finalize)
+		if err != nil {
+			t.Fatalf("Decode() error:\n%+v", err)
+		}
+		expectedFlows := []*schema.FlowMessage{
+			{
+				SamplingRate:    1024,
+				SrcAddr:         netip.MustParseAddr("2a0c:8880:2:0:185:21:130:38"),
+				DstAddr:         netip.MustParseAddr("2a0c:8880:2:0:185:21:130:39"),
+				ExporterAddress: netip.MustParseAddr("::ffff:172.16.0.3"),
+				InIf:            27,
+				OutIf:           0, // discard interface
+				SrcVlan:         100,
+				DstVlan:         100,
+				OtherColumns: map[schema.ColumnKey]any{
+					schema.ColumnBytes:            1500,
+					schema.ColumnPackets:          1,
+					schema.ColumnEType:            helpers.ETypeIPv6,
+					schema.ColumnProto:            6,
+					schema.ColumnSrcPort:          46026,
+					schema.ColumnDstPort:          22,
+					schema.ColumnForwardingStatus: 128,
+					schema.ColumnSrcMAC:           40057391053392,
+					schema.ColumnDstMAC:           40057381862408,
+					schema.ColumnTCPFlags:         16,
+					schema.ColumnIPv6FlowLabel:    426132,
+					schema.ColumnIPTTL:            64,
+					schema.ColumnIPTos:            8,
+				},
+			},
+		}
+
+		if diff := helpers.Diff(got, expectedFlows); diff != "" {
+			t.Fatalf("Decode() (-got, +want):\n%s", diff)
+		}
+	})
+
+	t.Run("multiple interfaces", func(t *testing.T) {
+		got = got[:0]
+		data := helpers.ReadPcapL4(t, filepath.Join("testdata", "data-multiple-interfaces.pcap"))
+		_, err := sdecoder.Decode(
+			decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+			options, bf, finalize)
+		if err != nil {
+			t.Fatalf("Decode() error:\n%+v", err)
+		}
+		expectedFlows := []*schema.FlowMessage{
+			{
+				SamplingRate:    1024,
+				SrcAddr:         netip.MustParseAddr("2a0c:8880:2:0:185:21:130:38"),
+				DstAddr:         netip.MustParseAddr("2a0c:8880:2:0:185:21:130:39"),
+				ExporterAddress: netip.MustParseAddr("::ffff:172.16.0.3"),
+				InIf:            27,
+				OutIf:           0, // multiple interfaces
+				SrcVlan:         100,
+				DstVlan:         100,
+				OtherColumns: map[schema.ColumnKey]any{
+					schema.ColumnBytes:         1500,
+					schema.ColumnPackets:       1,
+					schema.ColumnEType:         helpers.ETypeIPv6,
+					schema.ColumnProto:         6,
+					schema.ColumnSrcPort:       46026,
+					schema.ColumnDstPort:       22,
+					schema.ColumnSrcMAC:        40057391053392,
+					schema.ColumnDstMAC:        40057381862408,
+					schema.ColumnTCPFlags:      16,
+					schema.ColumnIPv6FlowLabel: 426132,
+					schema.ColumnIPTTL:         64,
+					schema.ColumnIPTos:         8,
+				},
+			},
+		}
+
+		if diff := helpers.Diff(got, expectedFlows); diff != "" {
+			t.Fatalf("Decode() (-got, +want):\n%s", diff)
+		}
+	})
+
+	t.Run("expanded flow sample", func(t *testing.T) {
+		got = got[:0]
+		data := helpers.ReadPcapL4(t, filepath.Join("testdata", "data-sflow-expanded-sample.pcap"))
+		_, err := sdecoder.Decode(
+			decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+			options, bf, finalize)
+		if err != nil {
+			t.Fatalf("Decode() error:\n%+v", err)
+		}
+		expectedFlows := []*schema.FlowMessage{
+			{
+				SamplingRate:    1000,
+				InIf:            29001,
+				OutIf:           1285816721,
+				SrcAddr:         netip.MustParseAddr("::ffff:52.52.52.52"),
+				DstAddr:         netip.MustParseAddr("::ffff:53.53.53.53"),
+				ExporterAddress: netip.MustParseAddr("::ffff:49.49.49.49"),
+				NextHop:         netip.MustParseAddr("::ffff:54.54.54.54"),
+				SrcAS:           203476,
+				DstAS:           203361,
+				SrcVlan:         809,
+				SrcNetMask:      32,
+				DstNetMask:      22,
+				OtherColumns: map[schema.ColumnKey]any{
+					schema.ColumnBytes:          104,
+					schema.ColumnPackets:        1,
+					schema.ColumnEType:          helpers.ETypeIPv4,
+					schema.ColumnProto:          6,
+					schema.ColumnSrcPort:        22,
+					schema.ColumnDstPort:        52237,
+					schema.ColumnDstASPath:      []uint32{8218, 29605, 203361},
+					schema.ColumnDstCommunities: []uint64{538574949, 1911619684, 1911669584, 1911671290},
+					schema.ColumnTCPFlags:       0x18,
+					schema.ColumnIPFragmentID:   0xab4e,
+					schema.ColumnIPTTL:          61,
+					schema.ColumnIPTos:          0x8,
+					schema.ColumnSrcMAC:         0x948ed30a713b,
+					schema.ColumnDstMAC:         0x22421f4a9fcd,
+				},
+			},
+		}
+
+		if diff := helpers.Diff(got, expectedFlows); diff != "" {
+			t.Fatalf("Decode() (-got, +want):\n%s", diff)
+		}
+	})
+
+	t.Run("flow sample with IPv4 data", func(t *testing.T) {
+		got = got[:0]
+		data := helpers.ReadPcapL4(t, filepath.Join("testdata", "data-sflow-ipv4-data.pcap"))
+		_, err := sdecoder.Decode(
+			decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+			options, bf, finalize)
+		if err != nil {
+			t.Fatalf("Decode() error:\n%+v", err)
+		}
+		expectedFlows := []*schema.FlowMessage{
+			{
+				SamplingRate:    256,
+				InIf:            0,
+				OutIf:           182,
+				DstVlan:         3001,
+				SrcAddr:         netip.MustParseAddr("::ffff:50.50.50.50"),
+				DstAddr:         netip.MustParseAddr("::ffff:51.51.51.51"),
+				ExporterAddress: netip.MustParseAddr("::ffff:49.49.49.49"),
+				OtherColumns: map[schema.ColumnKey]any{
+					schema.ColumnBytes:        1344,
+					schema.ColumnPackets:      1,
+					schema.ColumnEType:        helpers.ETypeIPv4,
+					schema.ColumnProto:        17,
+					schema.ColumnSrcPort:      46622,
+					schema.ColumnDstPort:      58631,
+					schema.ColumnSrcMAC:       1094287164743,
+					schema.ColumnDstMAC:       1101091482116,
+					schema.ColumnIPFragmentID: 41647,
+					schema.ColumnIPTTL:        64,
+				},
+			},
+		}
+
+		if diff := helpers.Diff(got, expectedFlows); diff != "" {
+			t.Fatalf("Decode() (-got, +want):\n%s", diff)
+		}
+	})
+
+	t.Run("flow sample with IPv4 raw packet", func(t *testing.T) {
+		got = got[:0]
+		data := helpers.ReadPcapL4(t, filepath.Join("testdata", "data-sflow-raw-ipv4.pcap"))
+		_, err := sdecoder.Decode(
+			decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+			options, bf, finalize)
+		if err != nil {
+			t.Fatalf("Decode() error:\n%+v", err)
+		}
+		expectedFlows := []*schema.FlowMessage{
+			{
+				SamplingRate:    1,
+				InIf:            0,
+				OutIf:           2,
+				SrcAddr:         netip.MustParseAddr("::ffff:69.58.92.107"),
+				DstAddr:         netip.MustParseAddr("::ffff:92.222.186.1"),
+				ExporterAddress: netip.MustParseAddr("::ffff:172.19.64.116"),
+				OtherColumns: map[schema.ColumnKey]any{
+					schema.ColumnBytes:        32,
+					schema.ColumnPackets:      1,
+					schema.ColumnEType:        helpers.ETypeIPv4,
+					schema.ColumnProto:        1,
+					schema.ColumnIPFragmentID: 4329,
+					schema.ColumnIPTTL:        64,
+					schema.ColumnIPTos:        8,
+				},
+			}, {
+				SamplingRate:    1,
+				InIf:            0,
+				OutIf:           2,
+				SrcAddr:         netip.MustParseAddr("::ffff:69.58.92.107"),
+				DstAddr:         netip.MustParseAddr("::ffff:92.222.184.1"),
+				ExporterAddress: netip.MustParseAddr("::ffff:172.19.64.116"),
+				OtherColumns: map[schema.ColumnKey]any{
+					schema.ColumnBytes:        32,
+					schema.ColumnPackets:      1,
+					schema.ColumnEType:        helpers.ETypeIPv4,
+					schema.ColumnProto:        1,
+					schema.ColumnIPFragmentID: 62945,
+					schema.ColumnIPTTL:        64,
+					schema.ColumnIPTos:        8,
+				},
+			},
+		}
+
+		if diff := helpers.Diff(got, expectedFlows); diff != "" {
+			t.Fatalf("Decode() (-got, +want):\n%s", diff)
+		}
+	})
+
+	t.Run("flow sample with ICMPv4", func(t *testing.T) {
+		got = got[:0]
+		data := helpers.ReadPcapL4(t, filepath.Join("testdata", "data-icmpv4.pcap"))
+		_, err := sdecoder.Decode(
+			decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+			options, bf, finalize)
+		if err != nil {
+			t.Fatalf("Decode() error:\n%+v", err)
+		}
+		expectedFlows := []*schema.FlowMessage{
+			{
+				SamplingRate:    1,
+				SrcAddr:         netip.MustParseAddr("::ffff:203.0.113.4"),
+				DstAddr:         netip.MustParseAddr("::ffff:203.0.113.5"),
+				ExporterAddress: netip.MustParseAddr("::ffff:127.0.0.1"),
+				OtherColumns: map[schema.ColumnKey]any{
+					schema.ColumnBytes:      84,
+					schema.ColumnPackets:    1,
+					schema.ColumnEType:      helpers.ETypeIPv4,
+					schema.ColumnProto:      1,
+					schema.ColumnDstMAC:     0xd25b45ee5ecf,
+					schema.ColumnSrcMAC:     0xe2efc68f8cd4,
+					schema.ColumnICMPv4Type: 8,
+					// schema.ColumnICMPv4Code:   0,
+					schema.ColumnIPTTL:        64,
+					schema.ColumnIPFragmentID: 0x90c5,
+				},
+			},
+		}
+
+		if diff := helpers.Diff(got, expectedFlows); diff != "" {
+			t.Fatalf("Decode() (-got, +want):\n%s", diff)
+		}
+	})
+
+	t.Run("flow sample with ICMPv6", func(t *testing.T) {
+		got = got[:0]
+		data := helpers.ReadPcapL4(t, filepath.Join("testdata", "data-icmpv6.pcap"))
+		_, err := sdecoder.Decode(
+			decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+			options, bf, finalize)
+		if err != nil {
+			t.Fatalf("Decode() error:\n%+v", err)
+		}
+		expectedFlows := []*schema.FlowMessage{
+			{
+				SamplingRate:    1,
+				SrcAddr:         netip.MustParseAddr("fe80::d05b:45ff:feee:5ecf"),
+				DstAddr:         netip.MustParseAddr("2001:db8::"),
+				ExporterAddress: netip.MustParseAddr("::ffff:127.0.0.1"),
+				OtherColumns: map[schema.ColumnKey]any{
+					schema.ColumnBytes:      72,
+					schema.ColumnPackets:    1,
+					schema.ColumnEType:      helpers.ETypeIPv6,
+					schema.ColumnProto:      58,
+					schema.ColumnSrcMAC:     0xd25b45ee5ecf,
+					schema.ColumnDstMAC:     0xe2efc68f8cd4,
+					schema.ColumnIPTTL:      255,
+					schema.ColumnICMPv6Type: 135,
+					// schema.ColumnICMPv6Code:   0,
+				},
+			},
+		}
+
+		if diff := helpers.Diff(got, expectedFlows); diff != "" {
+			t.Fatalf("Decode() (-got, +want):\n%s", diff)
+		}
+	})
+
+	t.Run("flow sample with QinQ", func(t *testing.T) {
+		got = got[:0]
+		data := helpers.ReadPcapL4(t, filepath.Join("testdata", "data-qinq.pcap"))
+		_, err := sdecoder.Decode(
+			decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+			options, bf, finalize)
+		if err != nil {
+			t.Fatalf("Decode() error:\n%+v", err)
+		}
+		expectedFlows := []*schema.FlowMessage{
+			{
+				SamplingRate:    4096,
+				InIf:            369098852,
+				OutIf:           369098851,
+				SrcVlan:         1493,
+				SrcAddr:         netip.MustParseAddr("::ffff:49.49.49.2"),
+				DstAddr:         netip.MustParseAddr("::ffff:49.49.49.109"),
+				ExporterAddress: netip.MustParseAddr("::ffff:172.17.128.58"),
+				OtherColumns: map[schema.ColumnKey]any{
+					schema.ColumnBytes:        80,
+					schema.ColumnPackets:      1,
+					schema.ColumnEType:        helpers.ETypeIPv4,
+					schema.ColumnProto:        6,
+					schema.ColumnSrcMAC:       0x4caea3520ff6,
+					schema.ColumnDstMAC:       0x000110621493,
+					schema.ColumnIPTTL:        62,
+					schema.ColumnIPFragmentID: 56159,
+					schema.ColumnTCPFlags:     16,
+					schema.ColumnSrcPort:      32017,
+					schema.ColumnDstPort:      443,
+				},
+			},
+		}
+
+		if diff := helpers.Diff(got, expectedFlows); diff != "" {
+			t.Fatalf("Decode() (-got, +want):\n%s", diff)
+		}
+	})
+}
diff --git a/inlet/flow/decoder/sflow/testdata/data-1140.pcap b/outlet/flow/decoder/sflow/testdata/data-1140.pcap
similarity index 100%
rename from inlet/flow/decoder/sflow/testdata/data-1140.pcap
rename to outlet/flow/decoder/sflow/testdata/data-1140.pcap
diff --git a/inlet/flow/decoder/sflow/testdata/data-discard-interface.pcap b/outlet/flow/decoder/sflow/testdata/data-discard-interface.pcap
similarity index 100%
rename from inlet/flow/decoder/sflow/testdata/data-discard-interface.pcap
rename to outlet/flow/decoder/sflow/testdata/data-discard-interface.pcap
diff --git a/inlet/flow/decoder/sflow/testdata/data-icmpv4.pcap b/outlet/flow/decoder/sflow/testdata/data-icmpv4.pcap
similarity index 100%
rename from inlet/flow/decoder/sflow/testdata/data-icmpv4.pcap
rename to outlet/flow/decoder/sflow/testdata/data-icmpv4.pcap
diff --git a/inlet/flow/decoder/sflow/testdata/data-icmpv6.pcap b/outlet/flow/decoder/sflow/testdata/data-icmpv6.pcap
similarity index 100%
rename from inlet/flow/decoder/sflow/testdata/data-icmpv6.pcap
rename to outlet/flow/decoder/sflow/testdata/data-icmpv6.pcap
diff --git a/inlet/flow/decoder/sflow/testdata/data-local-interface.pcap b/outlet/flow/decoder/sflow/testdata/data-local-interface.pcap
similarity index 100%
rename from inlet/flow/decoder/sflow/testdata/data-local-interface.pcap
rename to outlet/flow/decoder/sflow/testdata/data-local-interface.pcap
diff --git a/inlet/flow/decoder/sflow/testdata/data-multiple-interfaces.pcap b/outlet/flow/decoder/sflow/testdata/data-multiple-interfaces.pcap
similarity index 100%
rename from inlet/flow/decoder/sflow/testdata/data-multiple-interfaces.pcap
rename to outlet/flow/decoder/sflow/testdata/data-multiple-interfaces.pcap
diff --git a/inlet/flow/decoder/sflow/testdata/data-qinq.pcap b/outlet/flow/decoder/sflow/testdata/data-qinq.pcap
similarity index 100%
rename from inlet/flow/decoder/sflow/testdata/data-qinq.pcap
rename to outlet/flow/decoder/sflow/testdata/data-qinq.pcap
diff --git a/inlet/flow/decoder/sflow/testdata/data-sflow-expanded-sample.pcap b/outlet/flow/decoder/sflow/testdata/data-sflow-expanded-sample.pcap
similarity index 100%
rename from inlet/flow/decoder/sflow/testdata/data-sflow-expanded-sample.pcap
rename to outlet/flow/decoder/sflow/testdata/data-sflow-expanded-sample.pcap
diff --git a/inlet/flow/decoder/sflow/testdata/data-sflow-ipv4-data.pcap b/outlet/flow/decoder/sflow/testdata/data-sflow-ipv4-data.pcap
similarity index 100%
rename from inlet/flow/decoder/sflow/testdata/data-sflow-ipv4-data.pcap
rename to outlet/flow/decoder/sflow/testdata/data-sflow-ipv4-data.pcap
diff --git a/inlet/flow/decoder/sflow/testdata/data-sflow-raw-ipv4.pcap b/outlet/flow/decoder/sflow/testdata/data-sflow-raw-ipv4.pcap
similarity index 100%
rename from inlet/flow/decoder/sflow/testdata/data-sflow-raw-ipv4.pcap
rename to outlet/flow/decoder/sflow/testdata/data-sflow-raw-ipv4.pcap
diff --git a/inlet/flow/decoder/testdata/mpls-ipv4.pcap b/outlet/flow/decoder/testdata/mpls-ipv4.pcap
similarity index 100%
rename from inlet/flow/decoder/testdata/mpls-ipv4.pcap
rename to outlet/flow/decoder/testdata/mpls-ipv4.pcap
diff --git a/inlet/flow/decoder/testdata/vlan-ipv6.pcap b/outlet/flow/decoder/testdata/vlan-ipv6.pcap
similarity index 100%
rename from inlet/flow/decoder/testdata/vlan-ipv6.pcap
rename to outlet/flow/decoder/testdata/vlan-ipv6.pcap
diff --git a/outlet/flow/decoder_test.go b/outlet/flow/decoder_test.go
new file mode 100644
index 00000000..bf4114b4
--- /dev/null
+++ b/outlet/flow/decoder_test.go
@@ -0,0 +1,334 @@
+// SPDX-FileCopyrightText: 2023 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+package flow
+
+import (
+	"net"
+	"net/netip"
+	"path"
+	"path/filepath"
+	"runtime"
+	"testing"
+	"time"
+
+	"akvorado/common/helpers"
+	"akvorado/common/pb"
+	"akvorado/common/reporter"
+	"akvorado/common/schema"
+	"akvorado/outlet/flow/decoder"
+	"akvorado/outlet/flow/decoder/netflow"
+	"akvorado/outlet/flow/decoder/sflow"
+)
+
+func TestFlowDecode(t *testing.T) {
+	r := reporter.NewMock(t)
+	sch := schema.NewMock(t)
+	c, err := New(r, Dependencies{Schema: sch})
+	if err != nil {
+		t.Fatalf("New() error:\n%+v", err)
+	}
+	bf := sch.NewFlowMessage()
+	got := []*schema.FlowMessage{}
+	finalize := func() {
+		bf.TimeReceived = 0
+		// Keep a copy of the current flow message
+		clone := *bf
+		got = append(got, &clone)
+		bf.Finalize()
+	}
+
+	// Get test data path
+	_, src, _, _ := runtime.Caller(0)
+	base := path.Join(path.Dir(src), "decoder", "netflow", "testdata")
+
+	// Test Netflow decoding
+	t.Run("netflow", func(t *testing.T) {
+		// Load template first
+		templateData := helpers.ReadPcapL4(t, path.Join(base, "options-template.pcap"))
+		templateRawFlow := &pb.RawFlow{
+			TimeReceived:     uint64(time.Now().UnixNano()),
+			Payload:          templateData,
+			SourceAddress:    net.ParseIP("127.0.0.1").To16(),
+			UseSourceAddress: false,
+			Decoder:          pb.RawFlow_DECODER_NETFLOW,
+			TimestampSource:  pb.RawFlow_TS_INPUT,
+		}
+
+		// Decode template (should return empty slice for templates)
+		err := c.Decode(templateRawFlow, bf, finalize)
+		if err != nil {
+			t.Fatalf("Decode() template error:\n%+v", err)
+		}
+		if len(got) != 0 {
+			t.Logf("Template decode returned %d flows (expected 0)", len(got))
+		}
+
+		// Load options data
+		optionsData := helpers.ReadPcapL4(t, path.Join(base, "options-data.pcap"))
+		optionsRawFlow := &pb.RawFlow{
+			TimeReceived:     uint64(time.Now().UnixNano()),
+			Payload:          optionsData,
+			SourceAddress:    net.ParseIP("127.0.0.1").To16(),
+			UseSourceAddress: false,
+			Decoder:          pb.RawFlow_DECODER_NETFLOW,
+			TimestampSource:  pb.RawFlow_TS_INPUT,
+		}
+
+		// Decode options data
+		err = c.Decode(optionsRawFlow, bf, finalize)
+		if err != nil {
+			t.Fatalf("Decode() options data error:\n%+v", err)
+		}
+		if len(got) != 0 {
+			t.Logf("Options data decode returned %d flows (expected 0)", len(got))
+		}
+
+		// Load template for actual data
+		dataTemplateData := helpers.ReadPcapL4(t, path.Join(base, "template.pcap"))
+		dataTemplateRawFlow := &pb.RawFlow{
+			TimeReceived:     uint64(time.Now().UnixNano()),
+			Payload:          dataTemplateData,
+			SourceAddress:    net.ParseIP("127.0.0.1").To16(),
+			UseSourceAddress: false,
+			Decoder:          pb.RawFlow_DECODER_NETFLOW,
+			TimestampSource:  pb.RawFlow_TS_INPUT,
+		}
+
+		// Decode data template
+		err = c.Decode(dataTemplateRawFlow, bf, finalize)
+		if err != nil {
+			t.Fatalf("Decode() data template error:\n%+v", err)
+		}
+		if len(got) != 0 {
+			t.Logf("Data template decode returned %d flows (expected 0)", len(got))
+		}
+
+		// Load actual flow data
+		flowData := helpers.ReadPcapL4(t, path.Join(base, "data.pcap"))
+		flowRawFlow := &pb.RawFlow{
+			TimeReceived:     uint64(time.Now().UnixNano()),
+			Payload:          flowData,
+			SourceAddress:    net.ParseIP("127.0.0.1").To16(),
+			UseSourceAddress: false,
+			Decoder:          pb.RawFlow_DECODER_NETFLOW,
+			TimestampSource:  pb.RawFlow_TS_INPUT,
+		}
+
+		// Decode actual flow data
+		err = c.Decode(flowRawFlow, bf, finalize)
+		if err != nil {
+			t.Fatalf("Decode() flow data error:\n%+v", err)
+		}
+		if len(got) == 0 {
+			t.Fatalf("Decode() returned no flows")
+		}
+
+		t.Logf("Successfully decoded %d flows", len(got))
+
+		// Test with UseSourceAddress = true
+		got = got[:0]
+		flowRawFlow.UseSourceAddress = true
+		err = c.Decode(flowRawFlow, bf, finalize)
+		if err != nil {
+			t.Fatalf("Decode() with UseSourceAddress error:\n%+v", err)
+		}
+		if len(got) == 0 {
+			t.Fatalf("Decode() with UseSourceAddress returned no flows")
+		}
+
+		// Verify exporter address was overridden (should be IPv4-mapped IPv6)
+		expectedAddr := "::ffff:127.0.0.1"
+		for _, flow := range got {
+			if flow.ExporterAddress.String() != expectedAddr {
+				t.Errorf("Expected exporter address %s, got %s", expectedAddr, flow.ExporterAddress.String())
+			}
+		}
+
+		gotMetrics := r.GetMetrics("akvorado_outlet_flow_decoder_")
+		expectedMetrics := map[string]string{
+			`flows_total{name="netflow"}`:                                                                                                  "8",
+			`netflow_packets_total{exporter="::ffff:127.0.0.1",version="9"}`:                                                               "5",
+			`netflow_records_total{exporter="::ffff:127.0.0.1",type="DataFlowSet",version="9"}`:                                            "8",
+			`netflow_records_total{exporter="::ffff:127.0.0.1",type="OptionsDataFlowSet",version="9"}`:                                     "4",
+			`netflow_records_total{exporter="::ffff:127.0.0.1",type="OptionsTemplateFlowSet",version="9"}`:                                 "1",
+			`netflow_records_total{exporter="::ffff:127.0.0.1",type="TemplateFlowSet",version="9"}`:                                        "1",
+			`netflow_sets_total{exporter="::ffff:127.0.0.1",type="DataFlowSet",version="9"}`:                                               "2",
+			`netflow_sets_total{exporter="::ffff:127.0.0.1",type="OptionsDataFlowSet",version="9"}`:                                        "1",
+			`netflow_sets_total{exporter="::ffff:127.0.0.1",type="OptionsTemplateFlowSet",version="9"}`:                                    "1",
+			`netflow_sets_total{exporter="::ffff:127.0.0.1",type="TemplateFlowSet",version="9"}`:                                           "1",
+			`netflow_templates_total{exporter="::ffff:127.0.0.1",obs_domain_id="0",template_id="257",type="options_template",version="9"}`: "1",
+			`netflow_templates_total{exporter="::ffff:127.0.0.1",obs_domain_id="0",template_id="260",type="template",version="9"}`:         "1",
+		}
+		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
+			t.Fatalf("Metrics (-got, +want):\n%s", diff)
+		}
+	})
+
+	// Test sflow decoding
+	t.Run("sflow", func(t *testing.T) {
+		got = got[:0]
+		sflowBase := path.Join(path.Dir(src), "decoder", "sflow", "testdata")
+		flowData := helpers.ReadPcapL4(t, path.Join(sflowBase, "data-1140.pcap"))
+		flowRawFlow := &pb.RawFlow{
+			TimeReceived:     uint64(time.Now().UnixNano()),
+			Payload:          flowData,
+			SourceAddress:    net.ParseIP("127.0.0.1").To16(),
+			UseSourceAddress: false,
+			Decoder:          pb.RawFlow_DECODER_SFLOW,
+			TimestampSource:  pb.RawFlow_TS_INPUT,
+		}
+
+		err := c.Decode(flowRawFlow, bf, finalize)
+		if err != nil {
+			t.Fatalf("Decode() sflow error:\n%+v", err)
+		}
+		if len(got) == 0 {
+			t.Fatalf("Decode() sflow returned no flows")
+		}
+
+		gotMetrics := r.GetMetrics("akvorado_outlet_flow_decoder_", "flows_total", "sflow_")
+		expectedMetrics := map[string]string{
+			`flows_total{name="netflow"}`: "8",
+			`flows_total{name="sflow"}`:   "5",
+			`sflow_flows_total{agent="172.16.0.3",exporter="::ffff:127.0.0.1",version="5"}`:                          "1",
+			`sflow_sample_records_sum{agent="172.16.0.3",exporter="::ffff:127.0.0.1",type="FlowSample",version="5"}`: "14",
+			`sflow_sample_sum{agent="172.16.0.3",exporter="::ffff:127.0.0.1",type="FlowSample",version="5"}`:         "5",
+		}
+		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
+			t.Fatalf("Metrics (-got, +want):\n%s", diff)
+		}
+
+		t.Logf("Successfully decoded %d sflow flows", len(got))
+	})
+
+	// Test error cases
+	t.Run("errors", func(t *testing.T) {
+		// Unknown decoder
+		got = got[:0]
+		rawFlow := &pb.RawFlow{
+			TimeReceived:     uint64(time.Now().UnixNano()),
+			Payload:          []byte("test"),
+			SourceAddress:    net.ParseIP("127.0.0.1").To16(),
+			UseSourceAddress: false,
+			Decoder:          pb.RawFlow_DECODER_UNSPECIFIED,
+			TimestampSource:  pb.RawFlow_TS_INPUT,
+		}
+
+		err := c.Decode(rawFlow, bf, finalize)
+		if err == nil {
+			t.Fatal("Expected error for unknown decoder")
+		}
+
+		// Missing source address
+		rawFlow.Decoder = pb.RawFlow_DECODER_NETFLOW
+		rawFlow.SourceAddress = nil
+		err = c.Decode(rawFlow, bf, finalize)
+		if err == nil {
+			t.Fatal("Expected error for missing source address")
+		}
+
+		// Invalid payload for Netflow
+		rawFlow.Decoder = pb.RawFlow_DECODER_NETFLOW
+		rawFlow.SourceAddress = net.ParseIP("127.0.0.1").To16()
+		rawFlow.Payload = []byte("invalid")
+		err = c.Decode(rawFlow, bf, finalize)
+		if err == nil {
+			t.Fatal("Expected error for invalid payload")
+		}
+		// Invalid payload for Netflow v5
+		rawFlow.Payload = []byte{0, 5, 11, 12, 13, 14}
+		err = c.Decode(rawFlow, bf, finalize)
+		if err == nil {
+			t.Fatal("Expected error for invalid payload")
+		}
+		// Invalid payload for Netflow v9
+		rawFlow.Payload = []byte{0, 9, 11, 12, 13, 14}
+		err = c.Decode(rawFlow, bf, finalize)
+		if err == nil {
+			t.Fatal("Expected error for invalid payload")
+		}
+		// Invalid payload for IPFIX
+		rawFlow.Payload = []byte{0, 10, 11, 12, 13, 14}
+		err = c.Decode(rawFlow, bf, finalize)
+		if err == nil {
+			t.Fatal("Expected error for invalid payload")
+		}
+		// Invalid payload for sFlow
+		rawFlow.Decoder = pb.RawFlow_DECODER_SFLOW
+		err = c.Decode(rawFlow, bf, finalize)
+		if err == nil {
+			t.Fatal("Expected error for invalid payload")
+		}
+
+		gotMetrics := r.GetMetrics("akvorado_outlet_flow_decoder_", "errors", "netflow_errors", "sflow_errors")
+		expectedMetrics := map[string]string{
+			`errors_total{name="netflow"}`: "4",
+			`errors_total{name="sflow"}`:   "1",
+			`netflow_errors_total{error="IPFIX decoding error",exporter="::ffff:127.0.0.1"}`:      "1",
+			`netflow_errors_total{error="NetFlow v5 decoding error",exporter="::ffff:127.0.0.1"}`: "1",
+			`netflow_errors_total{error="NetFlow v9 decoding error",exporter="::ffff:127.0.0.1"}`: "1",
+			`sflow_errors_total{error="sFlow decoding error",exporter="::ffff:127.0.0.1"}`:        "1",
+		}
+		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
+			t.Fatalf("Metrics (-got, +want):\n%s", diff)
+		}
+
+	})
+}
+
+func BenchmarkDecodeNetflow(b *testing.B) {
+	schema.DisableDebug(b)
+	r := reporter.NewMock(b)
+	sch := schema.NewMock(b)
+	bf := sch.NewFlowMessage()
+	finalize := func() {}
+	nfdecoder := netflow.New(r, decoder.Dependencies{Schema: sch})
+	options := decoder.Option{TimestampSource: pb.RawFlow_TS_INPUT}
+
+	template := helpers.ReadPcapL4(b, filepath.Join("decoder", "netflow", "testdata", "options-template.pcap"))
+	_, err := nfdecoder.Decode(
+		decoder.RawFlow{Payload: template, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+		options, bf, finalize)
+	if err != nil {
+		b.Fatalf("Decode() error on options template:\n%+v", err)
+	}
+	data := helpers.ReadPcapL4(b, filepath.Join("decoder", "netflow", "testdata", "options-data.pcap"))
+	_, err = nfdecoder.Decode(
+		decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+		options, bf, finalize)
+	if err != nil {
+		b.Fatalf("Decode() error on options data:\n%+v", err)
+	}
+	template = helpers.ReadPcapL4(b, filepath.Join("decoder", "netflow", "testdata", "template.pcap"))
+	_, err = nfdecoder.Decode(
+		decoder.RawFlow{Payload: template, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+		options, bf, finalize)
+	if err != nil {
+		b.Fatalf("Decode() error on template:\n%+v", err)
+	}
+	data = helpers.ReadPcapL4(b, filepath.Join("decoder", "netflow", "testdata", "data.pcap"))
+
+	for b.Loop() {
+		nfdecoder.Decode(
+			decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+			options, bf, finalize)
+	}
+}
+
+func BenchmarkDecodeSflow(b *testing.B) {
+	schema.DisableDebug(b)
+	r := reporter.NewMock(b)
+	sch := schema.NewMock(b)
+	bf := sch.NewFlowMessage()
+	finalize := func() {}
+	sdecoder := sflow.New(r, decoder.Dependencies{Schema: sch})
+	options := decoder.Option{TimestampSource: pb.RawFlow_TS_INPUT}
+	data := helpers.ReadPcapL4(b, filepath.Join("decoder", "sflow", "testdata", "data-1140.pcap"))
+
+	for b.Loop() {
+		sdecoder.Decode(
+			decoder.RawFlow{Payload: data, Source: netip.MustParseAddr("::ffff:127.0.0.1")},
+			options, bf, finalize)
+	}
+}
diff --git a/outlet/flow/tests.go b/outlet/flow/tests.go
new file mode 100644
index 00000000..ad3f1911
--- /dev/null
+++ b/outlet/flow/tests.go
@@ -0,0 +1,16 @@
+// SPDX-FileCopyrightText: 2022 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+//go:build !release
+
+package flow
+
+import (
+	"akvorado/common/pb"
+	"akvorado/outlet/flow/decoder/gob"
+)
+
+func init() {
+	// Add gob decoder for testing purposes only
+	availableDecoders[pb.RawFlow_DECODER_GOB] = gob.New
+}
diff --git a/outlet/kafka/config_test.go b/outlet/kafka/config_test.go
new file mode 100644
index 00000000..df7bef30
--- /dev/null
+++ b/outlet/kafka/config_test.go
@@ -0,0 +1,16 @@
+// SPDX-FileCopyrightText: 2022 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+package kafka
+
+import (
+	"testing"
+
+	"akvorado/common/helpers"
+)
+
+func TestDefaultConfiguration(t *testing.T) {
+	if err := helpers.Validate.Struct(DefaultConfiguration()); err != nil {
+		t.Fatalf("validate.Struct() error:\n%+v", err)
+	}
+}
diff --git a/outlet/kafka/functional_test.go b/outlet/kafka/functional_test.go
new file mode 100644
index 00000000..2a44d406
--- /dev/null
+++ b/outlet/kafka/functional_test.go
@@ -0,0 +1,190 @@
+// SPDX-FileCopyrightText: 2022 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+package kafka
+
+import (
+	"context"
+	"fmt"
+	"math/rand"
+	"strconv"
+	"strings"
+	"testing"
+	"time"
+
+	"github.com/twmb/franz-go/pkg/kfake"
+	"github.com/twmb/franz-go/pkg/kgo"
+
+	"akvorado/common/daemon"
+	"akvorado/common/helpers"
+	"akvorado/common/kafka"
+	"akvorado/common/pb"
+	"akvorado/common/reporter"
+)
+
+func TestFakeKafka(t *testing.T) {
+	topicName := fmt.Sprintf("test-topic2-%d", rand.Int())
+	expectedTopicName := fmt.Sprintf("%s-v%d", topicName, pb.Version)
+
+	cluster, err := kfake.NewCluster(
+		kfake.NumBrokers(1),
+		kfake.SeedTopics(16, expectedTopicName),
+	)
+	if err != nil {
+		t.Fatalf("NewCluster() error: %v", err)
+	}
+	defer cluster.Close()
+
+	// Create a producer client
+	producerConfiguration := kafka.DefaultConfiguration()
+	producerConfiguration.Brokers = cluster.ListenAddrs()
+	producerOpts, err := kafka.NewConfig(reporter.NewMock(t), producerConfiguration)
+	if err != nil {
+		t.Fatalf("NewConfig() error:\n%+v", err)
+	}
+	producer, err := kgo.NewClient(producerOpts...)
+	if err != nil {
+		t.Fatalf("NewClient() error:\n%+v", err)
+	}
+	defer producer.Close()
+
+	// Callback
+	got := []string{}
+	expected := []string{"hello 1", "hello 2", "hello 3"}
+	gotAll := make(chan bool)
+	callback := func(_ context.Context, message []byte) error {
+		got = append(got, string(message))
+		if len(got) == len(expected) {
+			close(gotAll)
+		}
+		return nil
+	}
+
+	// Start the component
+	configuration := DefaultConfiguration()
+	configuration.Topic = topicName
+	configuration.Brokers = cluster.ListenAddrs()
+	configuration.FetchMaxWaitTime = 100 * time.Millisecond
+	configuration.ConsumerGroup = fmt.Sprintf("outlet-%d", rand.Int())
+	r := reporter.NewMock(t)
+	c, err := New(r, configuration, Dependencies{Daemon: daemon.NewMock(t)})
+	if err != nil {
+		t.Fatalf("New() error:\n%+v", err)
+	}
+	if err := c.(*realComponent).Start(); err != nil {
+		t.Fatalf("Start() error:\n%+v", err)
+	}
+	shutdownCalled := false
+	c.StartWorkers(func(_ int) (ReceiveFunc, ShutdownFunc) { return callback, func() { shutdownCalled = true } })
+
+	// Send messages
+	time.Sleep(100 * time.Millisecond)
+	t.Log("producing values")
+	for _, value := range expected {
+		record := &kgo.Record{
+			Topic: expectedTopicName,
+			Value: []byte(value),
+		}
+		results := producer.ProduceSync(context.Background(), record)
+		if err := results.FirstErr(); err != nil {
+			t.Fatalf("ProduceSync() error:\n%+v", err)
+		}
+	}
+	t.Log("values produced")
+
+	// Wait for them
+	select {
+	case <-time.After(time.Second):
+		t.Fatal("Too long to get messages")
+	case <-gotAll:
+	}
+
+	if diff := helpers.Diff(got, expected); diff != "" {
+		t.Errorf("Didn't received the expected messages (-got, +want):\n%s", diff)
+	}
+
+	gotMetrics := r.GetMetrics("akvorado_outlet_kafka_", "received_")
+	fetches, _ := strconv.Atoi(gotMetrics[`received_fetches_total{worker="0"}`])
+	expectedMetrics := map[string]string{
+		`received_bytes_total{worker="0"}`:    "21",
+		`received_fetches_total{worker="0"}`:  strconv.Itoa(max(fetches, 1)),
+		`received_messages_total{worker="0"}`: "3",
+	}
+	if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
+		t.Errorf("Metrics (-got, +want):\n%s", diff)
+	}
+
+	if err := c.Stop(); err != nil {
+		t.Fatalf("Stop() error:\n%+v", err)
+	}
+	if !shutdownCalled {
+		t.Fatal("Stop() didn't call shutdown function")
+	}
+}
+
+func TestStartSeveralWorkers(t *testing.T) {
+	topicName := fmt.Sprintf("test-topic2-%d", rand.Int())
+	expectedTopicName := fmt.Sprintf("%s-v%d", topicName, pb.Version)
+
+	cluster, err := kfake.NewCluster(
+		kfake.NumBrokers(1),
+		kfake.SeedTopics(16, expectedTopicName),
+	)
+	if err != nil {
+		t.Fatalf("NewCluster() error: %v", err)
+	}
+	defer cluster.Close()
+
+	// Start the component
+	configuration := DefaultConfiguration()
+	configuration.Topic = topicName
+	configuration.Brokers = cluster.ListenAddrs()
+	configuration.FetchMaxWaitTime = 100 * time.Millisecond
+	configuration.ConsumerGroup = fmt.Sprintf("outlet-%d", rand.Int())
+	configuration.Workers = 5
+	r := reporter.NewMock(t)
+	c, err := New(r, configuration, Dependencies{Daemon: daemon.NewMock(t)})
+	if err != nil {
+		t.Fatalf("New() error:\n%+v", err)
+	}
+	if err := c.(*realComponent).Start(); err != nil {
+		t.Fatalf("Start() error:\n%+v", err)
+	}
+	c.StartWorkers(func(int) (ReceiveFunc, ShutdownFunc) {
+		return func(context.Context, []byte) error { return nil }, func() {}
+	})
+	time.Sleep(20 * time.Millisecond)
+	if err := c.Stop(); err != nil {
+		t.Fatalf("Stop() error:\n%+v", err)
+	}
+
+	gotMetrics := r.GetMetrics("akvorado_outlet_kafka_")
+	connectsTotal := 0
+	writeBytesTotal := 0
+	readBytesTotal := 0
+	for k := range gotMetrics {
+		if strings.HasPrefix(k, "write_bytes_total") {
+			writeBytesTotal++
+		}
+		if strings.HasPrefix(k, "read_bytes_total") {
+			readBytesTotal++
+		}
+		if strings.HasPrefix(k, "connects_total") {
+			connectsTotal++
+		}
+	}
+	got := map[string]int{
+		"write_bytes_total": writeBytesTotal,
+		"read_bytes_total":  readBytesTotal,
+		"connects_total":    connectsTotal,
+	}
+	expected := map[string]int{
+		// For some reason, we have each metric in double, with one seed_0.
+		"write_bytes_total": 10,
+		"read_bytes_total":  10,
+		"connects_total":    10,
+	}
+	if diff := helpers.Diff(got, expected); diff != "" {
+		t.Errorf("Metrics (-got, +want):\n%s", diff)
+	}
+}
diff --git a/outlet/kafka/root_test.go b/outlet/kafka/root_test.go
new file mode 100644
index 00000000..3d132816
--- /dev/null
+++ b/outlet/kafka/root_test.go
@@ -0,0 +1,52 @@
+// SPDX-FileCopyrightText: 2022 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+package kafka
+
+import (
+	"context"
+	"testing"
+	"time"
+
+	"akvorado/common/helpers"
+)
+
+func TestMock(t *testing.T) {
+	c, incoming := NewMock(t, DefaultConfiguration())
+
+	got := []string{}
+	expected := []string{"hello1", "hello2", "hello3"}
+	gotAll := make(chan bool)
+	shutdownCalled := false
+	callback := func(_ context.Context, message []byte) error {
+		got = append(got, string(message))
+		if len(got) == len(expected) {
+			close(gotAll)
+		}
+		return nil
+	}
+	c.StartWorkers(
+		func(_ int) (ReceiveFunc, ShutdownFunc) {
+			return callback, func() { shutdownCalled = true }
+		},
+	)
+
+	// Produce messages and wait for them
+	for _, msg := range expected {
+		incoming <- []byte(msg)
+	}
+	select {
+	case <-time.After(time.Second):
+		t.Fatal("Too long to get messages")
+	case <-gotAll:
+	}
+
+	if diff := helpers.Diff(got, expected); diff != "" {
+		t.Errorf("Didn't received the expected messages (-got, +want):\n%s", diff)
+	}
+
+	c.Stop()
+	if !shutdownCalled {
+		t.Error("Stop() should have triggered shutdown function")
+	}
+}
diff --git a/outlet/kafka/tests.go b/outlet/kafka/tests.go
new file mode 100644
index 00000000..a86706e1
--- /dev/null
+++ b/outlet/kafka/tests.go
@@ -0,0 +1,50 @@
+// SPDX-FileCopyrightText: 2024 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+//go:build !release
+
+package kafka
+
+import (
+	"context"
+	"testing"
+)
+
+type mockComponent struct {
+	config   Configuration
+	incoming chan []byte
+}
+
+// NewMock instantiates a fake Kafka consumer that will produce messages sent on
+// the returned channel.
+func NewMock(_ *testing.T, config Configuration) (Component, chan<- []byte) {
+	c := mockComponent{
+		config:   config,
+		incoming: make(chan []byte),
+	}
+	return &c, c.incoming
+}
+
+// StartWorkers start a set of workers to produce received messages.
+func (c *mockComponent) StartWorkers(workerBuilder WorkerBuilderFunc) error {
+	for i := range c.config.Workers {
+		callback, shutdown := workerBuilder(i)
+		defer shutdown()
+		go func() {
+			for {
+				message, ok := <-c.incoming
+				if !ok {
+					return
+				}
+				callback(context.Background(), message)
+			}
+		}()
+	}
+	return nil
+}
+
+// Stop stops the mock component.
+func (c *mockComponent) Stop() error {
+	close(c.incoming)
+	return nil
+}
diff --git a/inlet/metadata/cache_test.go b/outlet/metadata/cache_test.go
similarity index 97%
rename from inlet/metadata/cache_test.go
rename to outlet/metadata/cache_test.go
index fc26bc1c..78a2d855 100644
--- a/inlet/metadata/cache_test.go
+++ b/outlet/metadata/cache_test.go
@@ -21,7 +21,7 @@ import (
 
 	"akvorado/common/helpers"
 	"akvorado/common/reporter"
-	"akvorado/inlet/metadata/provider"
+	"akvorado/outlet/metadata/provider"
 )
 
 func setupTestCache(t *testing.T) (*reporter.Reporter, *metadataCache) {
@@ -58,7 +58,7 @@ func TestGetEmpty(t *testing.T) {
 	r, sc := setupTestCache(t)
 	expectCacheLookup(t, sc, "127.0.0.1", 676, provider.Answer{})
 
-	gotMetrics := r.GetMetrics("akvorado_inlet_metadata_cache_")
+	gotMetrics := r.GetMetrics("akvorado_outlet_metadata_cache_")
 	expectedMetrics := map[string]string{
 		`expired_entries_total`: "0",
 		`hits_total`:            "0",
@@ -89,7 +89,7 @@ func TestSimpleLookup(t *testing.T) {
 	expectCacheLookup(t, sc, "127.0.0.1", 787, provider.Answer{})
 	expectCacheLookup(t, sc, "127.0.0.2", 676, provider.Answer{})
 
-	gotMetrics := r.GetMetrics("akvorado_inlet_metadata_cache_")
+	gotMetrics := r.GetMetrics("akvorado_outlet_metadata_cache_")
 	expectedMetrics := map[string]string{
 		`expired_entries_total`: "0",
 		`hits_total`:            "1",
@@ -173,7 +173,7 @@ func TestExpire(t *testing.T) {
 		Exporter:  provider.Exporter{Name: "localhost"},
 		Interface: provider.Interface{Name: "Gi0/0/0/1", Description: "Transit"}})
 
-	gotMetrics := r.GetMetrics("akvorado_inlet_metadata_cache_")
+	gotMetrics := r.GetMetrics("akvorado_outlet_metadata_cache_")
 	expectedMetrics := map[string]string{
 		`expired_entries_total`: "3",
 		`hits_total`:            "7",
@@ -464,7 +464,7 @@ func TestConcurrentOperations(t *testing.T) {
 	wg.Wait()
 
 	for remaining := 20; remaining >= 0; remaining-- {
-		gotMetrics := r.GetMetrics("akvorado_inlet_metadata_cache_")
+		gotMetrics := r.GetMetrics("akvorado_outlet_metadata_cache_")
 		hits, err := strconv.ParseFloat(gotMetrics["hits_total"], 64)
 		if err != nil {
 			t.Fatalf("strconv.ParseFloat() error:\n%+v", err)
diff --git a/inlet/metadata/config_test.go b/outlet/metadata/config_test.go
similarity index 100%
rename from inlet/metadata/config_test.go
rename to outlet/metadata/config_test.go
diff --git a/inlet/metadata/provider/gnmi/collector_test.go b/outlet/metadata/provider/gnmi/collector_test.go
similarity index 98%
rename from inlet/metadata/provider/gnmi/collector_test.go
rename to outlet/metadata/provider/gnmi/collector_test.go
index 395156ea..e765971f 100644
--- a/inlet/metadata/provider/gnmi/collector_test.go
+++ b/outlet/metadata/provider/gnmi/collector_test.go
@@ -7,7 +7,7 @@ import (
 	"testing"
 
 	"akvorado/common/helpers"
-	"akvorado/inlet/metadata/provider"
+	"akvorado/outlet/metadata/provider"
 )
 
 func TestStateUpdate(t *testing.T) {
diff --git a/inlet/metadata/provider/gnmi/config_test.go b/outlet/metadata/provider/gnmi/config_test.go
similarity index 100%
rename from inlet/metadata/provider/gnmi/config_test.go
rename to outlet/metadata/provider/gnmi/config_test.go
diff --git a/inlet/metadata/provider/gnmi/srlinux_test.go b/outlet/metadata/provider/gnmi/srlinux_test.go
similarity index 90%
rename from inlet/metadata/provider/gnmi/srlinux_test.go
rename to outlet/metadata/provider/gnmi/srlinux_test.go
index e69464bc..99e99f8e 100644
--- a/inlet/metadata/provider/gnmi/srlinux_test.go
+++ b/outlet/metadata/provider/gnmi/srlinux_test.go
@@ -17,7 +17,7 @@ import (
 
 	"akvorado/common/helpers"
 	"akvorado/common/reporter"
-	"akvorado/inlet/metadata/provider"
+	"akvorado/outlet/metadata/provider"
 
 	"github.com/openconfig/gnmi/proto/gnmi"
 	"github.com/openconfig/gnmic/pkg/api"
@@ -722,18 +722,16 @@ commit now
 		configP.Ports = helpers.MustNewSubnetMap(map[string]uint16{
 			"::/0": netip.MustParseAddrPort(srLinuxGNMI).Port(),
 		})
-		put := func(update provider.Update) {
-			got = append(got, fmt.Sprintf("%s %s %d %s %s %d",
-				update.ExporterIP.Unmap().String(), update.Exporter.Name,
-				update.IfIndex, update.Interface.Name, update.Interface.Description, update.Interface.Speed))
+		formatUpdate := func(exporter netip.Addr, iface string, answer provider.Answer) string {
+			return fmt.Sprintf("%s %v %s %s %s %s %d",
+				exporter.Unmap().String(), answer.Found, answer.Exporter.Name,
+				iface, answer.Interface.Name, answer.Interface.Description, answer.Interface.Speed)
 		}
 		r := reporter.NewMock(t)
-		p, err := configP.New(r, put)
+		p, err := configP.New(r)
 		if err != nil {
 			t.Fatalf("New() error:\n%+v", err)
 		}
-		// Let's trigger a request now
-		p.Query(context.Background(), &provider.BatchQuery{ExporterIP: lo, IfIndexes: []uint{641}})
 
 		// We need the indexes
 		subscribeReq, err := api.NewSubscribeRequest(
@@ -760,28 +758,32 @@ commit now
 		t.Logf("indexes: %v", indexes)
 
 		// Wait a bit
-		time.Sleep(500 * time.Millisecond)
-		p.Query(context.Background(), &provider.BatchQuery{ExporterIP: lo, IfIndexes: []uint{indexes["name=ethernet-1/1"]}})
-		p.Query(context.Background(), &provider.BatchQuery{ExporterIP: lo, IfIndexes: []uint{indexes["name=ethernet-1/2"]}})
-		p.Query(context.Background(), &provider.BatchQuery{ExporterIP: lo,
-			IfIndexes: []uint{indexes["name=lag1"], indexes["name=ethernet-1/3"]}})
-		p.Query(context.Background(), &provider.BatchQuery{ExporterIP: lo, IfIndexes: []uint{5}})
-		p.Query(context.Background(), &provider.BatchQuery{ExporterIP: lo,
-			IfIndexes: []uint{indexes["name=ethernet-1/4,index=1"]}})
-
-		time.Sleep(50 * time.Millisecond)
+		answer, _ := p.Query(context.Background(), provider.Query{ExporterIP: lo, IfIndex: indexes["name=ethernet-1/1"]})
+		got = append(got, formatUpdate(lo, "ethernet-1/1", answer))
+		answer, _ = p.Query(context.Background(), provider.Query{ExporterIP: lo, IfIndex: indexes["name=ethernet-1/2"]})
+		got = append(got, formatUpdate(lo, "ethernet-1/2", answer))
+		answer, _ = p.Query(context.Background(), provider.Query{ExporterIP: lo, IfIndex: indexes["name=lag1"]})
+		got = append(got, formatUpdate(lo, "lag1", answer))
+		answer, _ = p.Query(context.Background(), provider.Query{ExporterIP: lo, IfIndex: indexes["name=ethernet-1/3"]})
+		got = append(got, formatUpdate(lo, "ethernet-1/3", answer))
+		answer, _ = p.Query(context.Background(), provider.Query{ExporterIP: lo, IfIndex: 5})
+		got = append(got, formatUpdate(lo, "idx5", answer))
+		answer, _ = p.Query(context.Background(), provider.Query{ExporterIP: lo,
+			IfIndex: indexes["name=ethernet-1/4,index=1"]})
+		got = append(got, formatUpdate(lo, "ethernet-1/4,index=1", answer))
+
 		if diff := helpers.Diff(got, []string{
-			fmt.Sprintf("127.0.0.1 srlinux %d ethernet-1/1 1st interface 100000", indexes["name=ethernet-1/1"]),
-			fmt.Sprintf("127.0.0.1 srlinux %d ethernet-1/2 2nd interface 100000", indexes["name=ethernet-1/2"]),
-			fmt.Sprintf("127.0.0.1 srlinux %d lag1 lag interface 0", indexes["name=lag1"]),
-			fmt.Sprintf("127.0.0.1 srlinux %d ethernet-1/3 3rd interface 100000", indexes["name=ethernet-1/3"]),
-			"127.0.0.1 srlinux 5   0",
-			fmt.Sprintf("127.0.0.1 srlinux %d ethernet-1/4.1 4th interface 100000", indexes["name=ethernet-1/4,index=1"]),
+			"127.0.0.1 true srlinux ethernet-1/1 ethernet-1/1 1st interface 100000",
+			"127.0.0.1 true srlinux ethernet-1/2 ethernet-1/2 2nd interface 100000",
+			"127.0.0.1 true srlinux lag1 lag1 lag interface 0",
+			"127.0.0.1 true srlinux ethernet-1/3 ethernet-1/3 3rd interface 100000",
+			"127.0.0.1 false  idx5   0",
+			"127.0.0.1 true srlinux ethernet-1/4,index=1 ethernet-1/4.1 4th interface 100000",
 		}); diff != "" {
 			t.Fatalf("Query() (-got, +want):\n%s", diff)
 		}
 
-		gotMetrics := r.GetMetrics("akvorado_inlet_metadata_provider_gnmi_", "-collector_seconds")
+		gotMetrics := r.GetMetrics("akvorado_outlet_metadata_provider_gnmi_", "-collector_seconds")
 		expectedMetrics := map[string]string{
 			`collector_count`: "1",
 			`collector_ready_info{exporter="127.0.0.1"}`:               "1",
@@ -812,25 +814,24 @@ commit now
 		if resp.Failed != nil {
 			t.Fatalf("SendConfig() error:\n%+v", resp.Failed)
 		}
-		time.Sleep(500 * time.Millisecond) // We should exceed the second now and next request will trigger a refresh
-		p.Query(context.Background(), &provider.BatchQuery{ExporterIP: lo, IfIndexes: []uint{indexes["name=ethernet-1/1"]}})
-		time.Sleep(300 * time.Millisecond) // Do it again to get the fresh value
-		p.Query(context.Background(), &provider.BatchQuery{ExporterIP: lo, IfIndexes: []uint{indexes["name=ethernet-1/1"]}})
-		p.Query(context.Background(), &provider.BatchQuery{ExporterIP: lo,
-			IfIndexes: []uint{indexes["name=ethernet-1/4,index=1"]}})
-		time.Sleep(50 * time.Millisecond)
+		time.Sleep(time.Second) // We should exceed the second now and next request will trigger a refresh
+		t.Log("start queries")
+		answer, _ = p.Query(context.Background(),
+			provider.Query{ExporterIP: lo, IfIndex: indexes["name=ethernet-1/1"]})
+		got = append(got, formatUpdate(lo, "ethernet-1/1", answer))
+		answer, _ = p.Query(context.Background(),
+			provider.Query{ExporterIP: lo, IfIndex: indexes["name=ethernet-1/4,index=1"]})
+		got = append(got, formatUpdate(lo, "ethernet-1/4,index=1", answer))
 		if diff := helpers.Diff(got, []string{
-			// Previous value
-			fmt.Sprintf("127.0.0.1 srlinux %d ethernet-1/1 1st interface 100000", indexes["name=ethernet-1/1"]),
 			// Fresh value
-			fmt.Sprintf("127.0.0.1 srlinux %d ethernet-1/1 1st interface new 100000", indexes["name=ethernet-1/1"]),
+			"127.0.0.1 true srlinux ethernet-1/1 ethernet-1/1 1st interface new 100000",
 			// Removed value
-			fmt.Sprintf("127.0.0.1 srlinux %d   0", indexes["name=ethernet-1/4,index=1"]),
+			"127.0.0.1 false  ethernet-1/4,index=1   0",
 		}); diff != "" {
 			t.Fatalf("Query() (-got, +want):\n%s", diff)
 		}
 
-		gotMetrics = r.GetMetrics("akvorado_inlet_metadata_provider_gnmi_", "-collector_seconds")
+		gotMetrics = r.GetMetrics("akvorado_outlet_metadata_provider_gnmi_", "-collector_seconds")
 		expectedMetrics = map[string]string{
 			`collector_count`: "1",
 			`collector_ready_info{exporter="127.0.0.1"}`:               "1",
diff --git a/inlet/metadata/provider/gnmi/utils_test.go b/outlet/metadata/provider/gnmi/utils_test.go
similarity index 100%
rename from inlet/metadata/provider/gnmi/utils_test.go
rename to outlet/metadata/provider/gnmi/utils_test.go
diff --git a/inlet/metadata/provider/snmp/config_test.go b/outlet/metadata/provider/snmp/config_test.go
similarity index 100%
rename from inlet/metadata/provider/snmp/config_test.go
rename to outlet/metadata/provider/snmp/config_test.go
diff --git a/inlet/metadata/provider/snmp/poller_test.go b/outlet/metadata/provider/snmp/poller_test.go
similarity index 79%
rename from inlet/metadata/provider/snmp/poller_test.go
rename to outlet/metadata/provider/snmp/poller_test.go
index 61d5f970..666119eb 100644
--- a/inlet/metadata/provider/snmp/poller_test.go
+++ b/outlet/metadata/provider/snmp/poller_test.go
@@ -17,7 +17,7 @@ import (
 
 	"akvorado/common/helpers"
 	"akvorado/common/reporter"
-	"akvorado/inlet/metadata/provider"
+	"akvorado/outlet/metadata/provider"
 )
 
 func TestPoller(t *testing.T) {
@@ -264,41 +264,52 @@ func TestPoller(t *testing.T) {
 			config.Ports = helpers.MustNewSubnetMap(map[string]uint16{
 				"::/0": uint16(port),
 			})
-			put := func(update provider.Update) {
-				got = append(got, fmt.Sprintf("%s %s %d %s %s %d",
-					update.ExporterIP.Unmap().String(), update.Exporter.Name,
-					update.IfIndex, update.Interface.Name, update.Interface.Description, update.Interface.Speed))
-			}
-			p, err := config.New(r, put)
+			p, err := config.New(r)
 			if err != nil {
 				t.Fatalf("New() error:\n%+v", err)
 			}
 
-			p.Query(context.Background(), &provider.BatchQuery{ExporterIP: tc.ExporterIP, IfIndexes: []uint{641}})
-			p.Query(context.Background(), &provider.BatchQuery{ExporterIP: tc.ExporterIP, IfIndexes: []uint{642}})
-			p.Query(context.Background(), &provider.BatchQuery{ExporterIP: tc.ExporterIP, IfIndexes: []uint{643, 644, 645}})
-			p.Query(context.Background(), &provider.BatchQuery{ExporterIP: tc.ExporterIP, IfIndexes: []uint{0}})
+			// Collect results from all queries
+			answer, _ := p.Query(context.Background(), provider.Query{ExporterIP: tc.ExporterIP, IfIndex: 641})
+			got = append(got, fmt.Sprintf("%v %s %s %d %s %s %d",
+				answer.Found, tc.ExporterIP.Unmap().String(), answer.Exporter.Name,
+				641, answer.Interface.Name, answer.Interface.Description, answer.Interface.Speed))
+			answer, _ = p.Query(context.Background(), provider.Query{ExporterIP: tc.ExporterIP, IfIndex: 642})
+			got = append(got, fmt.Sprintf("%v %s %s %d %s %s %d",
+				answer.Found, tc.ExporterIP.Unmap().String(), answer.Exporter.Name,
+				642, answer.Interface.Name, answer.Interface.Description, answer.Interface.Speed))
+			answer, _ = p.Query(context.Background(), provider.Query{ExporterIP: tc.ExporterIP, IfIndex: 643})
+			got = append(got, fmt.Sprintf("%v %s %s %d %s %s %d",
+				answer.Found, tc.ExporterIP.Unmap().String(), answer.Exporter.Name,
+				643, answer.Interface.Name, answer.Interface.Description, answer.Interface.Speed))
+			answer, _ = p.Query(context.Background(), provider.Query{ExporterIP: tc.ExporterIP, IfIndex: 644})
+			got = append(got, fmt.Sprintf("%v %s %s %d %s %s %d",
+				answer.Found, tc.ExporterIP.Unmap().String(), answer.Exporter.Name,
+				644, answer.Interface.Name, answer.Interface.Description, answer.Interface.Speed))
+			answer, _ = p.Query(context.Background(), provider.Query{ExporterIP: tc.ExporterIP, IfIndex: 645})
+			got = append(got, fmt.Sprintf("%v %s %s %d %s %s %d",
+				answer.Found, tc.ExporterIP.Unmap().String(), answer.Exporter.Name,
+				645, answer.Interface.Name, answer.Interface.Description, answer.Interface.Speed))
+
 			exporterStr := tc.ExporterIP.Unmap().String()
 			time.Sleep(50 * time.Millisecond)
 			if diff := helpers.Diff(got, []string{
-				fmt.Sprintf(`%s exporter62 641 Gi0/0/0/0 Transit 10000`, exporterStr),
-				fmt.Sprintf(`%s exporter62 642 Gi0/0/0/1 Peering 20000`, exporterStr),
-				fmt.Sprintf(`%s exporter62 643 Gi0/0/0/2  10000`, exporterStr),                   // no ifAlias
-				fmt.Sprintf(`%s exporter62 644   0`, exporterStr),                                // negative cache
-				fmt.Sprintf(`%s exporter62 645 Gi0/0/0/5 Correct description 1000`, exporterStr), // negative cache
-				fmt.Sprintf(`%s exporter62 0   0`, exporterStr),
+				fmt.Sprintf(`true %s exporter62 641 Gi0/0/0/0 Transit 10000`, exporterStr),
+				fmt.Sprintf(`true %s exporter62 642 Gi0/0/0/1 Peering 20000`, exporterStr),
+				fmt.Sprintf(`true %s exporter62 643 Gi0/0/0/2  10000`, exporterStr), // no ifAlias
+				fmt.Sprintf(`false %s  644   0`, exporterStr),
+				fmt.Sprintf(`true %s exporter62 645 Gi0/0/0/5 Correct description 1000`, exporterStr),
 			}); diff != "" {
 				t.Fatalf("Poll() (-got, +want):\n%s", diff)
 			}
 
-			gotMetrics := r.GetMetrics("akvorado_inlet_metadata_provider_snmp_poller_", "error_", "pending_", "success_")
+			gotMetrics := r.GetMetrics("akvorado_outlet_metadata_provider_snmp_poller_", "error_", "success_")
 			expectedMetrics := map[string]string{
 				fmt.Sprintf(`error_requests_total{error="ifalias missing",exporter="%s"}`, exporterStr): "2", // 643+644
 				fmt.Sprintf(`error_requests_total{error="ifdescr missing",exporter="%s"}`, exporterStr): "1", // 644
 				fmt.Sprintf(`error_requests_total{error="ifname missing",exporter="%s"}`, exporterStr):  "1", // 644
 				fmt.Sprintf(`error_requests_total{error="ifspeed missing",exporter="%s"}`, exporterStr): "1", // 644
-				`pending_requests`: "0",
-				fmt.Sprintf(`success_requests_total{exporter="%s"}`, exporterStr): "5", // 641+642+643+645+0
+				fmt.Sprintf(`success_requests_total{exporter="%s"}`, exporterStr):                       "4", // 641+642+643+645
 			}
 			if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 				t.Fatalf("Metrics (-got, +want):\n%s", diff)
diff --git a/inlet/metadata/provider/snmp/tests.go b/outlet/metadata/provider/snmp/tests.go
similarity index 100%
rename from inlet/metadata/provider/snmp/tests.go
rename to outlet/metadata/provider/snmp/tests.go
diff --git a/inlet/metadata/provider/static/config_test.go b/outlet/metadata/provider/static/config_test.go
similarity index 98%
rename from inlet/metadata/provider/static/config_test.go
rename to outlet/metadata/provider/static/config_test.go
index 37edb43a..e8944673 100644
--- a/inlet/metadata/provider/static/config_test.go
+++ b/outlet/metadata/provider/static/config_test.go
@@ -9,7 +9,7 @@ import (
 
 	"akvorado/common/helpers"
 	"akvorado/common/remotedatasourcefetcher"
-	"akvorado/inlet/metadata/provider"
+	"akvorado/outlet/metadata/provider"
 )
 
 func TestValidation(t *testing.T) {
diff --git a/outlet/metadata/provider/static/root_test.go b/outlet/metadata/provider/static/root_test.go
new file mode 100644
index 00000000..a63f5a64
--- /dev/null
+++ b/outlet/metadata/provider/static/root_test.go
@@ -0,0 +1,252 @@
+// SPDX-FileCopyrightText: 2023 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+package static
+
+import (
+	"context"
+	"net/netip"
+	"testing"
+
+	"akvorado/common/helpers"
+	"akvorado/common/reporter"
+	"akvorado/common/schema"
+	"akvorado/outlet/metadata/provider"
+)
+
+func TestStaticProvider(t *testing.T) {
+	config := Configuration{
+		Exporters: helpers.MustNewSubnetMap(map[string]ExporterConfiguration{
+			"2001:db8:1::/48": {
+				Exporter: provider.Exporter{
+					Name: "nodefault",
+				},
+				IfIndexes: map[uint]provider.Interface{
+					10: {
+						Name:        "Gi10",
+						Description: "10th interface",
+						Speed:       1000,
+					},
+					11: {
+						Name:        "Gi11",
+						Description: "11th interface",
+						Speed:       1000,
+					},
+				},
+			},
+			"2001:db8:2::/48": {
+				Exporter: provider.Exporter{
+					Name: "default",
+				},
+				Default: provider.Interface{
+					Name:        "Default0",
+					Description: "Default interface",
+					Speed:       1000,
+				},
+				IfIndexes: map[uint]provider.Interface{
+					10: {
+						Name:        "Gi10",
+						Description: "10th interface",
+						Speed:       1000,
+					},
+				},
+			},
+			"2001:db8:3::/48": {
+				Exporter: provider.Exporter{
+					Name:   "default with metadata",
+					Region: "eu",
+					Role:   "peering",
+					Tenant: "mine",
+					Site:   "par",
+					Group:  "blue",
+				},
+				Default: provider.Interface{
+					Name:        "Default0",
+					Description: "Default interface",
+					Speed:       1000,
+				},
+				IfIndexes: map[uint]provider.Interface{
+					10: {
+						Name:         "Gi10",
+						Description:  "10th interface",
+						Speed:        1000,
+						Provider:     "transit101",
+						Connectivity: "transit",
+						Boundary:     schema.InterfaceBoundaryExternal,
+					},
+				},
+			},
+			"2001:db8:4::/48": {
+				Exporter: provider.Exporter{
+					Name: "nodefault skip",
+				},
+				IfIndexes: map[uint]provider.Interface{
+					10: {
+						Name:         "Gi10",
+						Description:  "10th interface",
+						Speed:        1000,
+						Provider:     "transit101",
+						Connectivity: "transit",
+						Boundary:     schema.InterfaceBoundaryExternal,
+					},
+				},
+				SkipMissingInterfaces: true,
+			},
+		}),
+	}
+
+	var got []provider.Answer
+	r := reporter.NewMock(t)
+	p, _ := config.New(r)
+
+	answer, _ := p.Query(context.Background(), provider.Query{
+		ExporterIP: netip.MustParseAddr("2001:db8:1::10"),
+		IfIndex:    9,
+	})
+	got = append(got, answer)
+	answer, _ = p.Query(context.Background(), provider.Query{
+		ExporterIP: netip.MustParseAddr("2001:db8:1::10"),
+		IfIndex:    10,
+	})
+	got = append(got, answer)
+	answer, _ = p.Query(context.Background(), provider.Query{
+		ExporterIP: netip.MustParseAddr("2001:db8:1::10"),
+		IfIndex:    11,
+	})
+	got = append(got, answer)
+
+	answer, _ = p.Query(context.Background(), provider.Query{
+		ExporterIP: netip.MustParseAddr("2001:db8:2::10"),
+		IfIndex:    9,
+	})
+	got = append(got, answer)
+	answer, _ = p.Query(context.Background(), provider.Query{
+		ExporterIP: netip.MustParseAddr("2001:db8:2::10"),
+		IfIndex:    10,
+	})
+	got = append(got, answer)
+	answer, _ = p.Query(context.Background(), provider.Query{
+		ExporterIP: netip.MustParseAddr("2001:db8:2::10"),
+		IfIndex:    11,
+	})
+	got = append(got, answer)
+
+	answer, _ = p.Query(context.Background(), provider.Query{
+		ExporterIP: netip.MustParseAddr("2001:db8:3::10"),
+		IfIndex:    10,
+	})
+	got = append(got, answer)
+
+	var err error
+	answer, _ = p.Query(context.Background(), provider.Query{
+		ExporterIP: netip.MustParseAddr("2001:db8:4::10"),
+		IfIndex:    10,
+	})
+	got = append(got, answer)
+	answer, err = p.Query(context.Background(), provider.Query{
+		ExporterIP: netip.MustParseAddr("2001:db8:4::10"),
+		IfIndex:    11,
+	})
+	got = append(got, answer)
+
+	expected := []provider.Answer{
+		{
+			Found: true,
+			Exporter: provider.Exporter{
+				Name: "nodefault",
+			},
+		},
+		{
+			Found: true,
+			Exporter: provider.Exporter{
+				Name: "nodefault",
+			},
+			Interface: provider.Interface{
+				Name:        "Gi10",
+				Description: "10th interface",
+				Speed:       1000,
+			},
+		},
+		{
+			Found: true,
+			Exporter: provider.Exporter{
+				Name: "nodefault",
+			},
+			Interface: provider.Interface{
+				Name:        "Gi11",
+				Description: "11th interface",
+				Speed:       1000,
+			},
+		},
+		{
+			Found: true,
+			Exporter: provider.Exporter{
+				Name: "default",
+			},
+			Interface: provider.Interface{
+				Name:        "Default0",
+				Description: "Default interface",
+				Speed:       1000,
+			},
+		}, {
+			Found: true,
+			Exporter: provider.Exporter{
+				Name: "default",
+			},
+			Interface: provider.Interface{
+				Name:        "Gi10",
+				Description: "10th interface",
+				Speed:       1000,
+			},
+		}, {
+			Found: true,
+			Exporter: provider.Exporter{
+				Name: "default",
+			},
+			Interface: provider.Interface{
+				Name:        "Default0",
+				Description: "Default interface",
+				Speed:       1000,
+			},
+		}, {
+			Found: true,
+			Exporter: provider.Exporter{
+				Name:   "default with metadata",
+				Region: "eu",
+				Role:   "peering",
+				Tenant: "mine",
+				Site:   "par",
+				Group:  "blue",
+			},
+			Interface: provider.Interface{
+				Name:         "Gi10",
+				Description:  "10th interface",
+				Speed:        1000,
+				Provider:     "transit101",
+				Connectivity: "transit",
+				Boundary:     schema.InterfaceBoundaryExternal,
+			},
+		}, {
+			Found: true,
+			Exporter: provider.Exporter{
+				Name: "nodefault skip",
+			},
+			Interface: provider.Interface{
+				Name:         "Gi10",
+				Description:  "10th interface",
+				Speed:        1000,
+				Provider:     "transit101",
+				Connectivity: "transit",
+				Boundary:     schema.InterfaceBoundaryExternal,
+			},
+		},
+		{}, // Skip
+	}
+
+	if diff := helpers.Diff(got, expected); diff != "" {
+		t.Fatalf("static provider (-got, +want):\n%s", diff)
+	}
+	if diff := helpers.Diff(err, provider.ErrSkipProvider); diff != "" {
+		t.Fatalf("static provider (-got, +want):\n%s", diff)
+	}
+}
diff --git a/inlet/metadata/provider/static/source_test.go b/outlet/metadata/provider/static/source_test.go
similarity index 75%
rename from inlet/metadata/provider/static/source_test.go
rename to outlet/metadata/provider/static/source_test.go
index 302ebe4a..47c7af4e 100644
--- a/inlet/metadata/provider/static/source_test.go
+++ b/outlet/metadata/provider/static/source_test.go
@@ -5,6 +5,7 @@ package static
 
 import (
 	"context"
+	"errors"
 	"fmt"
 	"net"
 	"net/http"
@@ -15,7 +16,7 @@ import (
 	"akvorado/common/helpers"
 	"akvorado/common/remotedatasourcefetcher"
 	"akvorado/common/reporter"
-	"akvorado/inlet/metadata/provider"
+	"akvorado/outlet/metadata/provider"
 )
 
 func TestInitStaticExporters(t *testing.T) {
@@ -38,7 +39,6 @@ func TestInitStaticExporters(t *testing.T) {
 	p := &Provider{
 		r:            r,
 		exportersMap: map[string][]exporterInfo{},
-		put:          func(_ provider.Update) {},
 	}
 	p.exporters.Store(conf.Exporters)
 
@@ -70,6 +70,7 @@ func TestInitStaticExporters(t *testing.T) {
 }
 
 func TestRemoteExporterSources(t *testing.T) {
+
 	// Mux to answer requests
 	ready := make(chan bool)
 	mux := http.NewServeMux()
@@ -186,42 +187,24 @@ func TestRemoteExporterSources(t *testing.T) {
 			},
 		},
 	}
-	var got []provider.Update
-	var expected []provider.Update
-	p, _ := config.New(r, func(update provider.Update) {
-		got = append(got, update)
-	})
+	p, _ := config.New(r)
 
-	// Query when json is not ready yet, only static configured data available
-	p.Query(context.Background(), &provider.BatchQuery{
+	// Query when json is not ready yet, we should get a timeout
+	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Millisecond)
+	defer cancel()
+	answer1, err := p.Query(ctx, provider.Query{
 		ExporterIP: netip.MustParseAddr("2001:db8:1::10"),
-		IfIndexes:  []uint{9},
-	})
-
-	// Unknown Exporter at this moment
-	p.Query(context.Background(), &provider.BatchQuery{
-		ExporterIP: netip.MustParseAddr("2001:db8:2::10"),
-		IfIndexes:  []uint{1},
-	})
-
-	expected = append(expected, provider.Update{
-		Query: provider.Query{
-			ExporterIP: netip.MustParseAddr("2001:db8:1::10"),
-			IfIndex:    9,
-		},
-		Answer: provider.Answer{
-			Exporter: provider.Exporter{
-				Name: "nodefault",
-			},
-		},
+		IfIndex:    9,
 	})
-
-	if diff := helpers.Diff(got, expected); diff != "" {
-		t.Fatalf("static provider - before remote source load (-got, +want):\n%s", diff)
+	if err == nil {
+		t.Fatalf("Query() should have been in error:\n%+v", answer1)
+	}
+	if !errors.Is(err, context.DeadlineExceeded) {
+		t.Fatalf("Query() error:\n%+v", err)
 	}
 
 	close(ready)
-	time.Sleep(50 * time.Millisecond)
+	time.Sleep(100 * time.Millisecond)
 
 	gotMetrics := r.GetMetrics("akvorado_common_remotedatasourcefetcher_data_")
 	expectedMetrics := map[string]string{
@@ -232,29 +215,24 @@ func TestRemoteExporterSources(t *testing.T) {
 	}
 
 	// We now should be able to resolve our new exporter from remote source
-	p.Query(context.Background(), &provider.BatchQuery{
+	got, _ := p.Query(context.Background(), provider.Query{
 		ExporterIP: netip.MustParseAddr("2001:db8:2::10"),
-		IfIndexes:  []uint{1},
+		IfIndex:    1,
 	})
 
-	expected = append(expected, provider.Update{
-		Query: provider.Query{
-			ExporterIP: netip.MustParseAddr("2001:db8:2::10"),
-			IfIndex:    1,
+	expected := provider.Answer{
+		Found: true,
+		Exporter: provider.Exporter{
+			Name: "exporter1",
 		},
-		Answer: provider.Answer{
-			Exporter: provider.Exporter{
-				Name: "exporter1",
-			},
-			Interface: provider.Interface{
-				Name:        "iface1",
-				Description: "foo:desc1",
-				Speed:       1000,
-			},
+		Interface: provider.Interface{
+			Name:        "iface1",
+			Description: "foo:desc1",
+			Speed:       1000,
 		},
-	})
+	}
 
 	if diff := helpers.Diff(got, expected); diff != "" {
-		t.Fatalf("static provider  - after remote source load(-got, +want):\n%s", diff)
+		t.Fatalf("static provider (-got, +want):\n%s", diff)
 	}
 }
diff --git a/inlet/metadata/provider/static/tests.go b/outlet/metadata/provider/static/tests.go
similarity index 100%
rename from inlet/metadata/provider/static/tests.go
rename to outlet/metadata/provider/static/tests.go
diff --git a/inlet/metadata/root_test.go b/outlet/metadata/root_test.go
similarity index 71%
rename from inlet/metadata/root_test.go
rename to outlet/metadata/root_test.go
index 8481626d..86b6fd59 100644
--- a/inlet/metadata/root_test.go
+++ b/outlet/metadata/root_test.go
@@ -16,14 +16,15 @@ import (
 	"akvorado/common/daemon"
 	"akvorado/common/helpers"
 	"akvorado/common/reporter"
-	"akvorado/inlet/metadata/provider"
-	"akvorado/inlet/metadata/provider/static"
+	"akvorado/common/schema"
+	"akvorado/outlet/metadata/provider"
+	"akvorado/outlet/metadata/provider/static"
 )
 
 func expectMockLookup(t *testing.T, c *Component, exporter string, ifIndex uint, expected provider.Answer) {
 	t.Helper()
 	ip := netip.AddrFrom16(netip.MustParseAddr(exporter).As16())
-	got, _ := c.Lookup(time.Now(), ip, ifIndex)
+	got := c.Lookup(time.Now(), ip, ifIndex)
 	if diff := helpers.Diff(got, expected); diff != "" {
 		t.Fatalf("Lookup() (-got, +want):\n%s", diff)
 	}
@@ -32,24 +33,53 @@ func expectMockLookup(t *testing.T, c *Component, exporter string, ifIndex uint,
 func TestLookup(t *testing.T) {
 	r := reporter.NewMock(t)
 	c := NewMock(t, r, DefaultConfiguration(), Dependencies{Daemon: daemon.NewMock(t)})
-	expectMockLookup(t, c, "127.0.0.1", 765, provider.Answer{})
-	expectMockLookup(t, c, "127.0.0.1", 999, provider.Answer{})
-	time.Sleep(30 * time.Millisecond)
 	expectMockLookup(t, c, "127.0.0.1", 765, provider.Answer{
+		Found: true,
 		Exporter: provider.Exporter{
 			Name: "127_0_0_1",
 		},
-
 		Interface: provider.Interface{Name: "Gi0/0/765",
 			Description: "Interface 765",
 			Speed:       1000,
 		},
 	})
-	expectMockLookup(t, c, "127.0.0.1", 999, provider.Answer{
+	expectMockLookup(t, c, "127.0.0.1", 1010, provider.Answer{
+		Found: true,
 		Exporter: provider.Exporter{
-			Name: "127_0_0_1",
+			Name:   "127_0_0_1",
+			Group:  "metadata group",
+			Region: "metadata region",
+			Role:   "metadata role",
+			Site:   "metadata site",
+			Tenant: "metadata tenant",
+		},
+		Interface: provider.Interface{Name: "Gi0/0/1010",
+			Description: "Interface 1010",
+			Speed:       1000,
 		},
 	})
+	expectMockLookup(t, c, "127.0.0.1", 2010, provider.Answer{
+		Found: true,
+		Exporter: provider.Exporter{
+			Name:   "127_0_0_1",
+			Group:  "metadata group",
+			Region: "metadata region",
+			Role:   "metadata role",
+			Site:   "metadata site",
+			Tenant: "metadata tenant",
+		},
+		Interface: provider.Interface{Name: "Gi0/0/2010",
+			Description:  "Interface 2010",
+			Speed:        1000,
+			Boundary:     schema.InterfaceBoundaryExternal,
+			Connectivity: "metadata connectivity",
+			Provider:     "metadata provider",
+		},
+	})
+	// With a simple lookup, this is not possible to distinguish between a
+	// transient error or a fatal error. Only the caching subsystem knows.
+	expectMockLookup(t, c, "127.0.0.1", 999, provider.Answer{})
+	expectMockLookup(t, c, "127.0.0.1", 998, provider.Answer{})
 }
 
 func TestComponentSaveLoad(t *testing.T) {
@@ -59,10 +89,8 @@ func TestComponentSaveLoad(t *testing.T) {
 	t.Run("save", func(t *testing.T) {
 		r := reporter.NewMock(t)
 		c := NewMock(t, r, configuration, Dependencies{Daemon: daemon.NewMock(t)})
-
-		expectMockLookup(t, c, "127.0.0.1", 765, provider.Answer{})
-		time.Sleep(30 * time.Millisecond)
 		expectMockLookup(t, c, "127.0.0.1", 765, provider.Answer{
+			Found: true,
 			Exporter: provider.Exporter{
 				Name: "127_0_0_1",
 			},
@@ -76,8 +104,11 @@ func TestComponentSaveLoad(t *testing.T) {
 
 	t.Run("load", func(t *testing.T) {
 		r := reporter.NewMock(t)
+		// Switch to the empty provider to check if we get answers from the cache.
+		configuration.Providers = []ProviderConfiguration{{Config: emptyProviderConfiguration{}}}
 		c := NewMock(t, r, configuration, Dependencies{Daemon: daemon.NewMock(t)})
 		expectMockLookup(t, c, "127.0.0.1", 765, provider.Answer{
+			Found: true,
 			Exporter: provider.Exporter{
 				Name: "127_0_0_1",
 			},
@@ -96,14 +127,11 @@ func TestAutoRefresh(t *testing.T) {
 	mockClock := clock.NewMock()
 	c := NewMock(t, r, configuration, Dependencies{Daemon: daemon.NewMock(t), Clock: mockClock})
 
-	// Fetch a value
-	expectMockLookup(t, c, "127.0.0.1", 765, provider.Answer{})
-	time.Sleep(30 * time.Millisecond)
 	expectMockLookup(t, c, "127.0.0.1", 765, provider.Answer{
+		Found: true,
 		Exporter: provider.Exporter{
 			Name: "127_0_0_1",
 		},
-
 		Interface: provider.Interface{
 			Name:        "Gi0/0/765",
 			Description: "Interface 765",
@@ -119,10 +147,9 @@ func TestAutoRefresh(t *testing.T) {
 
 	// Go forward, we expect the entry to have been refreshed and be still present
 	mockClock.Add(11 * time.Minute)
-	time.Sleep(30 * time.Millisecond)
 	mockClock.Add(2 * time.Minute)
-	time.Sleep(30 * time.Millisecond)
 	expectMockLookup(t, c, "127.0.0.1", 765, provider.Answer{
+		Found: true,
 		Exporter: provider.Exporter{
 			Name: "127_0_0_1",
 		},
@@ -133,17 +160,17 @@ func TestAutoRefresh(t *testing.T) {
 		},
 	})
 
-	gotMetrics := r.GetMetrics("akvorado_inlet_metadata_cache_")
+	gotMetrics := r.GetMetrics("akvorado_outlet_metadata_cache_")
 	for _, runs := range []string{"29", "30", "31"} { // 63/2
 		expectedMetrics := map[string]string{
 			`expired_entries_total`: "0",
-			`hits_total`:            "4",
-			`misses_total`:          "1",
+			`misses_total`:          "1", // First lookup misses
+			`hits_total`:            "3", // Subsequent ones hits
 			`size_entries`:          "1",
 			`refresh_runs_total`:    runs,
-			`refreshs`:              "1",
+			`refreshes_total`:       "1", // One refresh (after 1 hour)
 		}
-		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" && runs == "31" {
+		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" && runs == "19" {
 			t.Fatalf("Metrics (-got, +want):\n%s", diff)
 		} else if diff == "" {
 			break
@@ -184,22 +211,21 @@ func TestConfigCheck(t *testing.T) {
 	})
 }
 
-func TestStartStopWithMultipleWorkers(t *testing.T) {
+func TestStartStopSimple(t *testing.T) {
 	r := reporter.NewMock(t)
 	configuration := DefaultConfiguration()
-	configuration.Workers = 5
 	NewMock(t, r, configuration, Dependencies{Daemon: daemon.NewMock(t)})
 }
 
 type errorProvider struct{}
 
-func (ep errorProvider) Query(_ context.Context, _ *provider.BatchQuery) error {
-	return errors.New("noooo")
+func (ep errorProvider) Query(_ context.Context, _ provider.Query) (provider.Answer, error) {
+	return provider.Answer{}, errors.New("noooo")
 }
 
 type errorProviderConfiguration struct{}
 
-func (epc errorProviderConfiguration) New(_ *reporter.Reporter, _ func(provider.Update)) (provider.Provider, error) {
+func (epc errorProviderConfiguration) New(_ *reporter.Reporter) (provider.Provider, error) {
 	return errorProvider{}, nil
 }
 
@@ -216,7 +242,6 @@ func TestProviderBreaker(t *testing.T) {
 		t.Run(tc.Name, func(t *testing.T) {
 			r := reporter.NewMock(t)
 			configuration := DefaultConfiguration()
-			configuration.MaxBatchRequests = 0
 			configuration.Providers = []ProviderConfiguration{{Config: tc.ProviderConfiguration}}
 			c := NewMock(t, r, configuration, Dependencies{Daemon: daemon.NewMock(t)})
 			c.metrics.providerBreakerOpenCount.WithLabelValues("127.0.0.1").Add(0)
@@ -229,7 +254,7 @@ func TestProviderBreaker(t *testing.T) {
 			}
 			time.Sleep(50 * time.Millisecond)
 
-			gotMetrics := r.GetMetrics("akvorado_inlet_metadata_provider_", "breaker_opens_total")
+			gotMetrics := r.GetMetrics("akvorado_outlet_metadata_provider_", "breaker_opens_total")
 			expectedMetrics := map[string]string{
 				`breaker_opens_total{exporter="127.0.0.1"}`: tc.ExpectedCount,
 			}
@@ -240,72 +265,6 @@ func TestProviderBreaker(t *testing.T) {
 	}
 }
 
-type batchProvider struct {
-	config *batchProviderConfiguration
-}
-
-func (bp *batchProvider) Query(_ context.Context, query *provider.BatchQuery) error {
-	bp.config.received = append(bp.config.received, *query)
-	return nil
-}
-
-type batchProviderConfiguration struct {
-	received []provider.BatchQuery
-}
-
-func (bpc *batchProviderConfiguration) New(_ *reporter.Reporter, _ func(provider.Update)) (provider.Provider, error) {
-	return &batchProvider{config: bpc}, nil
-}
-
-func TestBatching(t *testing.T) {
-	bcp := batchProviderConfiguration{
-		received: []provider.BatchQuery{},
-	}
-	r := reporter.NewMock(t)
-	t.Run("run", func(t *testing.T) {
-		configuration := DefaultConfiguration()
-		configuration.Providers = []ProviderConfiguration{{Config: &bcp}}
-		c := NewMock(t, r, configuration, Dependencies{Daemon: daemon.NewMock(t)})
-
-		// Block dispatcher
-		blocker := make(chan bool)
-		c.dispatcherBChannel <- blocker
-
-		defer func() {
-			// Unblock
-			time.Sleep(20 * time.Millisecond)
-			close(blocker)
-			time.Sleep(20 * time.Millisecond)
-		}()
-
-		// Queue requests
-		c.Lookup(c.d.Clock.Now(), netip.MustParseAddr("::ffff:127.0.0.1"), 766)
-		c.Lookup(c.d.Clock.Now(), netip.MustParseAddr("::ffff:127.0.0.1"), 767)
-		c.Lookup(c.d.Clock.Now(), netip.MustParseAddr("::ffff:127.0.0.1"), 768)
-		c.Lookup(c.d.Clock.Now(), netip.MustParseAddr("::ffff:127.0.0.1"), 769)
-	})
-
-	t.Run("check", func(t *testing.T) {
-		gotMetrics := r.GetMetrics("akvorado_inlet_metadata_provider_", "batched_requests_total")
-		expectedMetrics := map[string]string{
-			`batched_requests_total`: "4",
-		}
-		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
-			t.Errorf("Metrics (-got, +want):\n%s", diff)
-		}
-
-		expectedAccepted := []provider.BatchQuery{
-			{
-				ExporterIP: netip.MustParseAddr("::ffff:127.0.0.1"),
-				IfIndexes:  []uint{766, 767, 768, 769},
-			},
-		}
-		if diff := helpers.Diff(bcp.received, expectedAccepted); diff != "" {
-			t.Errorf("Accepted requests (-got, +want):\n%s", diff)
-		}
-	})
-}
-
 func TestMultipleProviders(t *testing.T) {
 	r := reporter.NewMock(t)
 	staticConfiguration1 := static.Configuration{
@@ -359,11 +318,12 @@ func TestMultipleProviders(t *testing.T) {
 	c.Lookup(time.Now(), netip.MustParseAddr("2001:db8:1::1"), 10)
 	c.Lookup(time.Now(), netip.MustParseAddr("2001:db8:2::2"), 12)
 	time.Sleep(30 * time.Millisecond)
-	got1, _ := c.Lookup(time.Now(), netip.MustParseAddr("2001:db8:1::1"), 10)
-	got2, _ := c.Lookup(time.Now(), netip.MustParseAddr("2001:db8:2::2"), 12)
+	got1 := c.Lookup(time.Now(), netip.MustParseAddr("2001:db8:1::1"), 10)
+	got2 := c.Lookup(time.Now(), netip.MustParseAddr("2001:db8:2::2"), 12)
 	got := []provider.Answer{got1, got2}
 	expected := []provider.Answer{
 		{
+			Found: true,
 			Exporter: provider.Exporter{
 				Name: "static1",
 			},
@@ -373,6 +333,7 @@ func TestMultipleProviders(t *testing.T) {
 				Speed:       1000,
 			},
 		}, {
+			Found: true,
 			Exporter: provider.Exporter{
 				Name: "static2",
 			},
diff --git a/outlet/metadata/tests.go b/outlet/metadata/tests.go
new file mode 100644
index 00000000..ed226095
--- /dev/null
+++ b/outlet/metadata/tests.go
@@ -0,0 +1,108 @@
+// SPDX-FileCopyrightText: 2022 Free Mobile
+// SPDX-License-Identifier: AGPL-3.0-only
+
+//go:build !release
+
+package metadata
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"strings"
+	"testing"
+
+	"akvorado/common/helpers"
+	"akvorado/common/reporter"
+	"akvorado/common/schema"
+	"akvorado/outlet/metadata/provider"
+)
+
+// mockProvider represents a mock provider.
+type mockProvider struct{}
+
+// Query query the mock provider for a value.
+//   - ifIndex = 999 → not found
+//   - ifIndex = 998 → transient error
+//   - ifIndex = 1010 → with metadata for exporter
+//   - ifIndex = 2010 → with metadata for exporter and interface
+func (mp mockProvider) Query(_ context.Context, query provider.Query) (provider.Answer, error) {
+	ifIndex := query.IfIndex
+	if ifIndex == 999 {
+		return provider.Answer{}, nil
+	}
+	if ifIndex == 998 {
+		return provider.Answer{}, errors.New("noooo")
+	}
+
+	answer := provider.Answer{
+		Exporter: provider.Exporter{
+			Name: strings.ReplaceAll(query.ExporterIP.Unmap().String(), ".", "_"),
+		},
+	}
+	answer.Interface.Name = fmt.Sprintf("Gi0/0/%d", ifIndex)
+	answer.Interface.Description = fmt.Sprintf("Interface %d", ifIndex)
+	answer.Interface.Speed = 1000
+
+	// in iface with  metadata (overriden by out iface)
+	if ifIndex == 1010 {
+		answer.Exporter.Group = "metadata group"
+		answer.Exporter.Region = "metadata region"
+		answer.Exporter.Role = "metadata role"
+		answer.Exporter.Site = "metadata site"
+		answer.Exporter.Tenant = "metadata tenant"
+	}
+
+	// out iface with metadata
+	if ifIndex == 2010 {
+		answer.Interface.Boundary = schema.InterfaceBoundaryExternal
+		answer.Interface.Connectivity = "metadata connectivity"
+		answer.Interface.Provider = "metadata provider"
+		answer.Exporter.Group = "metadata group"
+		answer.Exporter.Region = "metadata region"
+		answer.Exporter.Role = "metadata role"
+		answer.Exporter.Site = "metadata site"
+		answer.Exporter.Tenant = "metadata tenant"
+	}
+
+	answer.Found = true
+	return answer, nil
+}
+
+// mockProviderConfiguration is the configuration for the mock provider.
+type mockProviderConfiguration struct{}
+
+// New returns a new mock provider.
+func (mpc mockProviderConfiguration) New(_ *reporter.Reporter) (provider.Provider, error) {
+	return mockProvider{}, nil
+}
+
+// NewMock creates a new metadata component building synthetic values. It is already started.
+func NewMock(t *testing.T, reporter *reporter.Reporter, configuration Configuration, dependencies Dependencies) *Component {
+	t.Helper()
+	if configuration.Providers == nil {
+		configuration.Providers = []ProviderConfiguration{{Config: mockProviderConfiguration{}}}
+	}
+	c, err := New(reporter, configuration, dependencies)
+	if err != nil {
+		t.Fatalf("New() error:\n%+v", err)
+	}
+	helpers.StartStop(t, c)
+	return c
+}
+
+// emptyProvider represents an empty mock provider.
+type emptyProvider struct{}
+
+// Query returns always a not found status for the empty mock provider
+func (mp emptyProvider) Query(_ context.Context, _ provider.Query) (provider.Answer, error) {
+	return provider.Answer{}, nil
+}
+
+// emptyProviderConfiguration is the configuration for the empty provider.
+type emptyProviderConfiguration struct{}
+
+// New returns a new empty provider.
+func (mpc emptyProviderConfiguration) New(_ *reporter.Reporter) (provider.Provider, error) {
+	return emptyProvider{}, nil
+}
diff --git a/inlet/routing/config_test.go b/outlet/routing/config_test.go
similarity index 100%
rename from inlet/routing/config_test.go
rename to outlet/routing/config_test.go
diff --git a/inlet/routing/provider/bioris/config_test.go b/outlet/routing/provider/bioris/config_test.go
similarity index 100%
rename from inlet/routing/provider/bioris/config_test.go
rename to outlet/routing/provider/bioris/config_test.go
diff --git a/inlet/routing/provider/bioris/root_test.go b/outlet/routing/provider/bioris/root_test.go
similarity index 98%
rename from inlet/routing/provider/bioris/root_test.go
rename to outlet/routing/provider/bioris/root_test.go
index ff957f23..9279205c 100644
--- a/inlet/routing/provider/bioris/root_test.go
+++ b/outlet/routing/provider/bioris/root_test.go
@@ -26,7 +26,7 @@ import (
 	"akvorado/common/daemon"
 	"akvorado/common/helpers"
 	"akvorado/common/reporter"
-	"akvorado/inlet/routing/provider"
+	"akvorado/outlet/routing/provider"
 )
 
 func TestChooseRouter(t *testing.T) {
@@ -462,7 +462,7 @@ func TestBioRIS(t *testing.T) {
 	}
 
 	for try := 2; try >= 0; try-- {
-		gotMetrics := r.GetMetrics("akvorado_inlet_routing_provider_bioris_")
+		gotMetrics := r.GetMetrics("akvorado_outlet_routing_provider_bioris_")
 		expectedMetrics := map[string]string{
 			// connection_up may take a bit of time
 			fmt.Sprintf(`connection_up{ris="%s"}`, addr):                                     "1",
@@ -502,7 +502,7 @@ func TestNonWorkingBioRIS(t *testing.T) {
 	}
 	helpers.StartStop(t, p)
 
-	gotMetrics := r.GetMetrics("akvorado_inlet_routing_provider_bioris_")
+	gotMetrics := r.GetMetrics("akvorado_outlet_routing_provider_bioris_")
 	expectedMetrics := map[string]string{
 		`connection_up{ris="ris.invalid:1000"}`: "0",
 		`connection_up{ris="192.0.2.10:1000"}`:  "0",
diff --git a/inlet/routing/provider/bmp/config_test.go b/outlet/routing/provider/bmp/config_test.go
similarity index 100%
rename from inlet/routing/provider/bmp/config_test.go
rename to outlet/routing/provider/bmp/config_test.go
diff --git a/inlet/routing/provider/bmp/rd_test.go b/outlet/routing/provider/bmp/rd_test.go
similarity index 98%
rename from inlet/routing/provider/bmp/rd_test.go
rename to outlet/routing/provider/bmp/rd_test.go
index 9be331df..9b41b2d4 100644
--- a/inlet/routing/provider/bmp/rd_test.go
+++ b/outlet/routing/provider/bmp/rd_test.go
@@ -7,7 +7,7 @@ import (
 	"testing"
 
 	"akvorado/common/helpers"
-	"akvorado/inlet/routing/provider/bmp"
+	"akvorado/outlet/routing/provider/bmp"
 
 	"github.com/osrg/gobgp/v3/pkg/packet/bgp"
 )
diff --git a/inlet/routing/provider/bmp/rib_test.go b/outlet/routing/provider/bmp/rib_test.go
similarity index 100%
rename from inlet/routing/provider/bmp/rib_test.go
rename to outlet/routing/provider/bmp/rib_test.go
diff --git a/inlet/routing/provider/bmp/root_test.go b/outlet/routing/provider/bmp/root_test.go
similarity index 88%
rename from inlet/routing/provider/bmp/root_test.go
rename to outlet/routing/provider/bmp/root_test.go
index 76f81edd..05e3e833 100644
--- a/inlet/routing/provider/bmp/root_test.go
+++ b/outlet/routing/provider/bmp/root_test.go
@@ -15,7 +15,7 @@ import (
 
 	"akvorado/common/helpers"
 	"akvorado/common/reporter"
-	"akvorado/inlet/routing/provider"
+	"akvorado/outlet/routing/provider"
 
 	"github.com/osrg/gobgp/v3/pkg/packet/bgp"
 )
@@ -86,12 +86,12 @@ func TestBMP(t *testing.T) {
 		// Init+EOR
 		send(t, conn, "bmp-init.pcap")
 		time.Sleep(20 * time.Millisecond)
-		gotMetrics := r.GetMetrics("akvorado_inlet_routing_provider_bmp_")
+		gotMetrics := r.GetMetrics("akvorado_outlet_routing_provider_bmp_")
 		expectedMetrics := map[string]string{
 			`received_messages_total{exporter="127.0.0.1",type="initiation"}`: "1",
 			`opened_connections_total{exporter="127.0.0.1"}`:                  "1",
-			`peers_total{exporter="127.0.0.1"}`:                               "0",
-			`routes_total{exporter="127.0.0.1"}`:                              "0",
+			`peers{exporter="127.0.0.1"}`:                                     "0",
+			`routes{exporter="127.0.0.1"}`:                                    "0",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 			t.Errorf("Metrics (-got, +want):\n%s", diff)
@@ -99,14 +99,14 @@ func TestBMP(t *testing.T) {
 
 		send(t, conn, "bmp-terminate.pcap")
 		time.Sleep(30 * time.Millisecond)
-		gotMetrics = r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "-locked_duration")
+		gotMetrics = r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "-locked_duration")
 		expectedMetrics = map[string]string{
 			`closed_connections_total{exporter="127.0.0.1"}`:                   "1",
 			`received_messages_total{exporter="127.0.0.1",type="initiation"}`:  "1",
 			`received_messages_total{exporter="127.0.0.1",type="termination"}`: "1",
 			`opened_connections_total{exporter="127.0.0.1"}`:                   "1",
-			`peers_total{exporter="127.0.0.1"}`:                                "0",
-			`routes_total{exporter="127.0.0.1"}`:                               "0",
+			`peers{exporter="127.0.0.1"}`:                                      "0",
+			`routes{exporter="127.0.0.1"}`:                                     "0",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 			t.Errorf("Metrics (-got, +want):\n%s", diff)
@@ -124,14 +124,14 @@ func TestBMP(t *testing.T) {
 		mockClock.Add(2 * time.Hour)
 		for tries := 20; tries >= 0; tries-- {
 			time.Sleep(5 * time.Millisecond)
-			gotMetrics = r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "-locked_duration")
+			gotMetrics = r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "-locked_duration")
 			expectedMetrics = map[string]string{
 				`closed_connections_total{exporter="127.0.0.1"}`:                   "1",
 				`received_messages_total{exporter="127.0.0.1",type="initiation"}`:  "1",
 				`received_messages_total{exporter="127.0.0.1",type="termination"}`: "1",
 				`opened_connections_total{exporter="127.0.0.1"}`:                   "1",
-				`peers_total{exporter="127.0.0.1"}`:                                "0",
-				`routes_total{exporter="127.0.0.1"}`:                               "0",
+				`peers{exporter="127.0.0.1"}`:                                      "0",
+				`routes{exporter="127.0.0.1"}`:                                     "0",
 			}
 			if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 				if tries > 0 {
@@ -153,15 +153,15 @@ func TestBMP(t *testing.T) {
 		send(t, conn, "bmp-peers-up.pcap")
 		send(t, conn, "bmp-eor.pcap")
 		time.Sleep(20 * time.Millisecond)
-		gotMetrics := r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "-locked_duration")
+		gotMetrics := r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "-locked_duration")
 		expectedMetrics := map[string]string{
 			`received_messages_total{exporter="127.0.0.1",type="initiation"}`:           "1",
 			`received_messages_total{exporter="127.0.0.1",type="peer-up-notification"}`: "4",
 			`received_messages_total{exporter="127.0.0.1",type="route-monitoring"}`:     "8",
 			`received_messages_total{exporter="127.0.0.1",type="statistics-report"}`:    "4",
 			`opened_connections_total{exporter="127.0.0.1"}`:                            "1",
-			`peers_total{exporter="127.0.0.1"}`:                                         "4",
-			`routes_total{exporter="127.0.0.1"}`:                                        "0",
+			`peers{exporter="127.0.0.1"}`:                                               "4",
+			`routes{exporter="127.0.0.1"}`:                                              "0",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 			t.Errorf("Metrics (-got, +want):\n%s", diff)
@@ -181,15 +181,15 @@ func TestBMP(t *testing.T) {
 		send(t, conn, "bmp-reach.pcap")
 		send(t, conn, "bmp-reach-addpath.pcap")
 		time.Sleep(20 * time.Millisecond)
-		gotMetrics := r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "-locked_duration")
+		gotMetrics := r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "-locked_duration")
 		expectedMetrics := map[string]string{
 			`received_messages_total{exporter="127.0.0.1",type="initiation"}`:           "1",
 			`received_messages_total{exporter="127.0.0.1",type="peer-up-notification"}`: "4",
 			`received_messages_total{exporter="127.0.0.1",type="route-monitoring"}`:     "26",
 			`received_messages_total{exporter="127.0.0.1",type="statistics-report"}`:    "4",
 			`opened_connections_total{exporter="127.0.0.1"}`:                            "1",
-			`peers_total{exporter="127.0.0.1"}`:                                         "4",
-			`routes_total{exporter="127.0.0.1"}`:                                        "18",
+			`peers{exporter="127.0.0.1"}`:                                               "4",
+			`routes{exporter="127.0.0.1"}`:                                              "18",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 			t.Errorf("Metrics (-got, +want):\n%s", diff)
@@ -239,14 +239,14 @@ func TestBMP(t *testing.T) {
 		send(t, conn, "bmp-init.pcap")
 		send(t, conn, "bmp-reach.pcap")
 		time.Sleep(20 * time.Millisecond)
-		gotMetrics := r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "-locked_duration")
+		gotMetrics := r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "-locked_duration")
 		expectedMetrics := map[string]string{
 			// Same metrics as previously, except the AddPath peer.
 			`received_messages_total{exporter="127.0.0.1",type="initiation"}`:       "1",
 			`received_messages_total{exporter="127.0.0.1",type="route-monitoring"}`: "17",
 			`opened_connections_total{exporter="127.0.0.1"}`:                        "1",
-			`peers_total{exporter="127.0.0.1"}`:                                     "3",
-			`routes_total{exporter="127.0.0.1"}`:                                    "17",
+			`peers{exporter="127.0.0.1"}`:                                           "3",
+			`routes{exporter="127.0.0.1"}`:                                          "17",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 			t.Errorf("Metrics (-got, +want):\n%s", diff)
@@ -265,15 +265,15 @@ func TestBMP(t *testing.T) {
 		send(t, conn, "bmp-peers-up.pcap")
 		send(t, conn, "bmp-eor.pcap")
 		time.Sleep(20 * time.Millisecond)
-		gotMetrics := r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "-locked_duration")
+		gotMetrics := r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "-locked_duration")
 		expectedMetrics := map[string]string{
 			`received_messages_total{exporter="127.0.0.1",type="initiation"}`:           "1",
 			`received_messages_total{exporter="127.0.0.1",type="peer-up-notification"}`: "4",
 			`received_messages_total{exporter="127.0.0.1",type="route-monitoring"}`:     "25",
 			`received_messages_total{exporter="127.0.0.1",type="statistics-report"}`:    "4",
 			`opened_connections_total{exporter="127.0.0.1"}`:                            "1",
-			`peers_total{exporter="127.0.0.1"}`:                                         "4",
-			`routes_total{exporter="127.0.0.1"}`:                                        "17",
+			`peers{exporter="127.0.0.1"}`:                                               "4",
+			`routes{exporter="127.0.0.1"}`:                                              "17",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 			t.Errorf("Metrics (-got, +want):\n%s", diff)
@@ -293,7 +293,7 @@ func TestBMP(t *testing.T) {
 		send(t, conn, "bmp-reach.pcap")
 		send(t, conn, "bmp-peer-down.pcap")
 		time.Sleep(20 * time.Millisecond)
-		gotMetrics := r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "-locked_duration")
+		gotMetrics := r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "-locked_duration")
 		expectedMetrics := map[string]string{
 			`received_messages_total{exporter="127.0.0.1",type="initiation"}`:             "1",
 			`received_messages_total{exporter="127.0.0.1",type="peer-up-notification"}`:   "4",
@@ -301,8 +301,8 @@ func TestBMP(t *testing.T) {
 			`received_messages_total{exporter="127.0.0.1",type="route-monitoring"}`:       "25",
 			`received_messages_total{exporter="127.0.0.1",type="statistics-report"}`:      "5",
 			`opened_connections_total{exporter="127.0.0.1"}`:                              "1",
-			`peers_total{exporter="127.0.0.1"}`:                                           "3",
-			`routes_total{exporter="127.0.0.1"}`:                                          "14",
+			`peers{exporter="127.0.0.1"}`:                                                 "3",
+			`routes{exporter="127.0.0.1"}`:                                                "14",
 			`removed_peers_total{exporter="127.0.0.1"}`:                                   "1",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
@@ -349,15 +349,15 @@ func TestBMP(t *testing.T) {
 		send(t, conn, "bmp-eor.pcap")
 		send(t, conn, "bmp-reach.pcap")
 		time.Sleep(20 * time.Millisecond)
-		gotMetrics := r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "-locked_duration")
+		gotMetrics := r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "-locked_duration")
 		expectedMetrics := map[string]string{
 			`received_messages_total{exporter="127.0.0.1",type="initiation"}`:           "1",
 			`received_messages_total{exporter="127.0.0.1",type="peer-up-notification"}`: "4",
 			`received_messages_total{exporter="127.0.0.1",type="route-monitoring"}`:     "25",
 			`received_messages_total{exporter="127.0.0.1",type="statistics-report"}`:    "4",
 			`opened_connections_total{exporter="127.0.0.1"}`:                            "1",
-			`peers_total{exporter="127.0.0.1"}`:                                         "4",
-			`routes_total{exporter="127.0.0.1"}`:                                        "1",
+			`peers{exporter="127.0.0.1"}`:                                               "4",
+			`routes{exporter="127.0.0.1"}`:                                              "1",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 			t.Errorf("Metrics (-got, +want):\n%s", diff)
@@ -388,15 +388,15 @@ func TestBMP(t *testing.T) {
 		send(t, conn, "bmp-eor.pcap")
 		send(t, conn, "bmp-reach.pcap")
 		time.Sleep(20 * time.Millisecond)
-		gotMetrics := r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "-locked_duration")
+		gotMetrics := r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "-locked_duration")
 		expectedMetrics := map[string]string{
 			`received_messages_total{exporter="127.0.0.1",type="initiation"}`:           "1",
 			`received_messages_total{exporter="127.0.0.1",type="peer-up-notification"}`: "4",
 			`received_messages_total{exporter="127.0.0.1",type="route-monitoring"}`:     "25",
 			`received_messages_total{exporter="127.0.0.1",type="statistics-report"}`:    "4",
 			`opened_connections_total{exporter="127.0.0.1"}`:                            "1",
-			`peers_total{exporter="127.0.0.1"}`:                                         "4",
-			`routes_total{exporter="127.0.0.1"}`:                                        "10",
+			`peers{exporter="127.0.0.1"}`:                                               "4",
+			`routes{exporter="127.0.0.1"}`:                                              "10",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 			t.Errorf("Metrics (-got, +want):\n%s", diff)
@@ -441,15 +441,15 @@ func TestBMP(t *testing.T) {
 		send(t, conn, "bmp-reach.pcap")
 		send(t, conn, "bmp-unreach.pcap")
 		time.Sleep(20 * time.Millisecond)
-		gotMetrics := r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "-locked_duration")
+		gotMetrics := r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "-locked_duration")
 		expectedMetrics := map[string]string{
 			`received_messages_total{exporter="127.0.0.1",type="initiation"}`:           "1",
 			`received_messages_total{exporter="127.0.0.1",type="peer-up-notification"}`: "4",
 			`received_messages_total{exporter="127.0.0.1",type="route-monitoring"}`:     "33",
 			`received_messages_total{exporter="127.0.0.1",type="statistics-report"}`:    "4",
 			`opened_connections_total{exporter="127.0.0.1"}`:                            "1",
-			`peers_total{exporter="127.0.0.1"}`:                                         "4",
-			`routes_total{exporter="127.0.0.1"}`:                                        "0",
+			`peers{exporter="127.0.0.1"}`:                                               "4",
+			`routes{exporter="127.0.0.1"}`:                                              "0",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 			t.Errorf("Metrics (-got, +want):\n%s", diff)
@@ -472,15 +472,15 @@ func TestBMP(t *testing.T) {
 		send(t, conn, "bmp-init.pcap")
 		send(t, conn, "bmp-l3vpn.pcap")
 		time.Sleep(20 * time.Millisecond)
-		gotMetrics := r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "-locked_duration")
+		gotMetrics := r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "-locked_duration")
 		expectedMetrics := map[string]string{
 			`received_messages_total{exporter="127.0.0.1",type="initiation"}`:           "1",
 			`received_messages_total{exporter="127.0.0.1",type="peer-up-notification"}`: "1",
 			`received_messages_total{exporter="127.0.0.1",type="route-monitoring"}`:     "3",
 			`received_messages_total{exporter="127.0.0.1",type="statistics-report"}`:    "1",
 			`opened_connections_total{exporter="127.0.0.1"}`:                            "1",
-			`peers_total{exporter="127.0.0.1"}`:                                         "1",
-			`routes_total{exporter="127.0.0.1"}`:                                        "2",
+			`peers{exporter="127.0.0.1"}`:                                               "1",
+			`routes{exporter="127.0.0.1"}`:                                              "2",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 			t.Errorf("Metrics (-got, +want):\n%s", diff)
@@ -510,9 +510,9 @@ func TestBMP(t *testing.T) {
 		send(t, conn, "bmp-init.pcap")
 		send(t, conn, "bmp-l3vpn.pcap")
 		time.Sleep(20 * time.Millisecond)
-		gotMetrics := r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "routes")
+		gotMetrics := r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "routes")
 		expectedMetrics := map[string]string{
-			`routes_total{exporter="127.0.0.1"}`: "2",
+			`routes{exporter="127.0.0.1"}`: "2",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 			t.Errorf("Metrics (-got, +want):\n%s", diff)
@@ -531,9 +531,9 @@ func TestBMP(t *testing.T) {
 		send(t, conn, "bmp-init.pcap")
 		send(t, conn, "bmp-l3vpn.pcap")
 		time.Sleep(20 * time.Millisecond)
-		gotMetrics := r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "routes")
+		gotMetrics := r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "routes")
 		expectedMetrics := map[string]string{
-			`routes_total{exporter="127.0.0.1"}`: "0",
+			`routes{exporter="127.0.0.1"}`: "0",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 			t.Errorf("Metrics (-got, +want):\n%s", diff)
@@ -553,9 +553,9 @@ func TestBMP(t *testing.T) {
 		send(t, conn, "bmp-init.pcap")
 		send(t, conn, "bmp-l3vpn.pcap")
 		time.Sleep(20 * time.Millisecond)
-		gotMetrics := r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "routes")
+		gotMetrics := r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "routes")
 		expectedMetrics := map[string]string{
-			`routes_total{exporter="127.0.0.1"}`: "2",
+			`routes{exporter="127.0.0.1"}`: "2",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 			t.Errorf("Metrics (-got, +want):\n%s", diff)
@@ -586,9 +586,9 @@ func TestBMP(t *testing.T) {
 		send(t, conn, "bmp-init.pcap")
 		send(t, conn, "bmp-l3vpn.pcap")
 		time.Sleep(20 * time.Millisecond)
-		gotMetrics := r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "routes")
+		gotMetrics := r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "routes")
 		expectedMetrics := map[string]string{
-			`routes_total{exporter="127.0.0.1"}`: "2",
+			`routes{exporter="127.0.0.1"}`: "2",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 			t.Errorf("Metrics (-got, +want):\n%s", diff)
@@ -618,15 +618,15 @@ func TestBMP(t *testing.T) {
 		send(t, conn, "bmp-eor.pcap")
 		send(t, conn, "bmp-unreach.pcap")
 		time.Sleep(20 * time.Millisecond)
-		gotMetrics := r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "-locked_duration")
+		gotMetrics := r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "-locked_duration")
 		expectedMetrics := map[string]string{
 			`received_messages_total{exporter="127.0.0.1",type="initiation"}`:           "1",
 			`received_messages_total{exporter="127.0.0.1",type="peer-up-notification"}`: "4",
 			`received_messages_total{exporter="127.0.0.1",type="route-monitoring"}`:     "16",
 			`received_messages_total{exporter="127.0.0.1",type="statistics-report"}`:    "4",
 			`opened_connections_total{exporter="127.0.0.1"}`:                            "1",
-			`peers_total{exporter="127.0.0.1"}`:                                         "4",
-			`routes_total{exporter="127.0.0.1"}`:                                        "0",
+			`peers{exporter="127.0.0.1"}`:                                               "4",
+			`routes{exporter="127.0.0.1"}`:                                              "0",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 			t.Errorf("Metrics (-got, +want):\n%s", diff)
@@ -651,15 +651,15 @@ func TestBMP(t *testing.T) {
 		send(t, conn, "bmp-unreach.pcap")
 		send(t, conn, "bmp-unreach.pcap")
 		time.Sleep(20 * time.Millisecond)
-		gotMetrics := r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "-locked_duration")
+		gotMetrics := r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "-locked_duration")
 		expectedMetrics := map[string]string{
 			`received_messages_total{exporter="127.0.0.1",type="initiation"}`:           "1",
 			`received_messages_total{exporter="127.0.0.1",type="peer-up-notification"}`: "4",
 			`received_messages_total{exporter="127.0.0.1",type="route-monitoring"}`:     "41",
 			`received_messages_total{exporter="127.0.0.1",type="statistics-report"}`:    "4",
 			`opened_connections_total{exporter="127.0.0.1"}`:                            "1",
-			`peers_total{exporter="127.0.0.1"}`:                                         "4",
-			`routes_total{exporter="127.0.0.1"}`:                                        "1",
+			`peers{exporter="127.0.0.1"}`:                                               "4",
+			`routes{exporter="127.0.0.1"}`:                                              "1",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 			t.Errorf("Metrics (-got, +want):\n%s", diff)
@@ -691,15 +691,15 @@ func TestBMP(t *testing.T) {
 		send(t, conn, "bmp-unreach.pcap")
 		send(t, conn, "bmp-unreach.pcap")
 		time.Sleep(20 * time.Millisecond)
-		gotMetrics := r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "-locked_duration")
+		gotMetrics := r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "-locked_duration")
 		expectedMetrics := map[string]string{
 			`received_messages_total{exporter="127.0.0.1",type="initiation"}`:           "1",
 			`received_messages_total{exporter="127.0.0.1",type="peer-up-notification"}`: "4",
 			`received_messages_total{exporter="127.0.0.1",type="route-monitoring"}`:     "41",
 			`received_messages_total{exporter="127.0.0.1",type="statistics-report"}`:    "4",
 			`opened_connections_total{exporter="127.0.0.1"}`:                            "1",
-			`peers_total{exporter="127.0.0.1"}`:                                         "4",
-			`routes_total{exporter="127.0.0.1"}`:                                        "1",
+			`peers{exporter="127.0.0.1"}`:                                               "4",
+			`routes{exporter="127.0.0.1"}`:                                              "1",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 			t.Errorf("Metrics (-got, +want):\n%s", diff)
@@ -721,15 +721,15 @@ func TestBMP(t *testing.T) {
 		send(t, conn, "bmp-reach.pcap")
 		send(t, conn, "bmp-eor.pcap")
 		time.Sleep(20 * time.Millisecond)
-		gotMetrics := r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "-locked_duration")
+		gotMetrics := r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "-locked_duration")
 		expectedMetrics := map[string]string{
 			`received_messages_total{exporter="127.0.0.1",type="initiation"}`:           "1",
 			`received_messages_total{exporter="127.0.0.1",type="peer-up-notification"}`: "4",
 			`received_messages_total{exporter="127.0.0.1",type="route-monitoring"}`:     "25",
 			`received_messages_total{exporter="127.0.0.1",type="statistics-report"}`:    "4",
 			`opened_connections_total{exporter="127.0.0.1"}`:                            "1",
-			`peers_total{exporter="127.0.0.1"}`:                                         "4",
-			`routes_total{exporter="127.0.0.1"}`:                                        "17",
+			`peers{exporter="127.0.0.1"}`:                                               "4",
+			`routes{exporter="127.0.0.1"}`:                                              "17",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 			t.Errorf("Metrics (-got, +want):\n%s", diff)
@@ -780,7 +780,7 @@ func TestBMP(t *testing.T) {
 		send(t, conn, "bmp-l3vpn.pcap")
 		conn.Close()
 		time.Sleep(20 * time.Millisecond)
-		gotMetrics := r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "-locked_duration")
+		gotMetrics := r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "-locked_duration")
 		expectedMetrics := map[string]string{
 			`received_messages_total{exporter="127.0.0.1",type="initiation"}`:           "1",
 			`received_messages_total{exporter="127.0.0.1",type="peer-up-notification"}`: "1",
@@ -788,8 +788,8 @@ func TestBMP(t *testing.T) {
 			`received_messages_total{exporter="127.0.0.1",type="statistics-report"}`:    "1",
 			`opened_connections_total{exporter="127.0.0.1"}`:                            "1",
 			`closed_connections_total{exporter="127.0.0.1"}`:                            "1",
-			`peers_total{exporter="127.0.0.1"}`:                                         "1",
-			`routes_total{exporter="127.0.0.1"}`:                                        "2",
+			`peers{exporter="127.0.0.1"}`:                                               "1",
+			`routes{exporter="127.0.0.1"}`:                                              "2",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 			t.Errorf("Metrics (-got, +want):\n%s", diff)
@@ -808,7 +808,7 @@ func TestBMP(t *testing.T) {
 
 		mockClock.Add(2 * time.Hour)
 		time.Sleep(20 * time.Millisecond)
-		gotMetrics = r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "-locked_duration")
+		gotMetrics = r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "-locked_duration")
 		expectedMetrics = map[string]string{
 			`received_messages_total{exporter="127.0.0.1",type="initiation"}`:           "1",
 			`received_messages_total{exporter="127.0.0.1",type="peer-up-notification"}`: "1",
@@ -816,8 +816,8 @@ func TestBMP(t *testing.T) {
 			`received_messages_total{exporter="127.0.0.1",type="statistics-report"}`:    "1",
 			`opened_connections_total{exporter="127.0.0.1"}`:                            "1",
 			`closed_connections_total{exporter="127.0.0.1"}`:                            "1",
-			`peers_total{exporter="127.0.0.1"}`:                                         "0",
-			`routes_total{exporter="127.0.0.1"}`:                                        "0",
+			`peers{exporter="127.0.0.1"}`:                                               "0",
+			`routes{exporter="127.0.0.1"}`:                                              "0",
 			`removed_peers_total{exporter="127.0.0.1"}`:                                 "1",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
@@ -842,16 +842,16 @@ func TestBMP(t *testing.T) {
 		send(t, conn, "bmp-l3vpn.pcap")
 		send(t, conn, "bmp-reach-unknown-family.pcap")
 		time.Sleep(20 * time.Millisecond)
-		gotMetrics := r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "-locked_duration")
-		ignoredMetric := `ignored_updates_total{error="unknown route family. AFI: 57, SAFI: 65",exporter="127.0.0.1",reason="afi-safi"}`
+		gotMetrics := r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "-locked_duration")
+		ignoredMetric := `ignored_updates_total{error="afi-safi",exporter="127.0.0.1"}`
 		expectedMetrics := map[string]string{
 			`received_messages_total{exporter="127.0.0.1",type="initiation"}`:           "1",
 			`received_messages_total{exporter="127.0.0.1",type="peer-up-notification"}`: "1",
 			`received_messages_total{exporter="127.0.0.1",type="route-monitoring"}`:     "4",
 			`received_messages_total{exporter="127.0.0.1",type="statistics-report"}`:    "1",
 			`opened_connections_total{exporter="127.0.0.1"}`:                            "1",
-			`peers_total{exporter="127.0.0.1"}`:                                         "1",
-			`routes_total{exporter="127.0.0.1"}`:                                        "2",
+			`peers{exporter="127.0.0.1"}`:                                               "1",
+			`routes{exporter="127.0.0.1"}`:                                              "2",
 			ignoredMetric:                                                               "1",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
@@ -881,15 +881,15 @@ func TestBMP(t *testing.T) {
 		send(t, conn, "bmp-l3vpn.pcap")
 		send(t, conn, "bmp-reach-vpls.pcap")
 		time.Sleep(20 * time.Millisecond)
-		gotMetrics := r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "-locked_duration")
+		gotMetrics := r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "-locked_duration")
 		expectedMetrics := map[string]string{
 			`received_messages_total{exporter="127.0.0.1",type="initiation"}`:           "1",
 			`received_messages_total{exporter="127.0.0.1",type="peer-up-notification"}`: "1",
 			`received_messages_total{exporter="127.0.0.1",type="route-monitoring"}`:     "4",
 			`received_messages_total{exporter="127.0.0.1",type="statistics-report"}`:    "1",
 			`opened_connections_total{exporter="127.0.0.1"}`:                            "1",
-			`peers_total{exporter="127.0.0.1"}`:                                         "2",
-			`routes_total{exporter="127.0.0.1"}`:                                        "2",
+			`peers{exporter="127.0.0.1"}`:                                               "2",
+			`routes{exporter="127.0.0.1"}`:                                              "2",
 			`ignored_nlri_total{exporter="127.0.0.1",type="l2vpn-vpls"}`:                "1",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
@@ -914,7 +914,7 @@ func TestBMP(t *testing.T) {
 		send(t, conn2, "bmp-l3vpn.pcap")
 		conn1.Close()
 		time.Sleep(20 * time.Millisecond)
-		gotMetrics := r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "-locked_duration")
+		gotMetrics := r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "-locked_duration")
 		expectedMetrics := map[string]string{
 			`received_messages_total{exporter="127.0.0.1",type="initiation"}`:           "2",
 			`received_messages_total{exporter="127.0.0.1",type="peer-up-notification"}`: "2",
@@ -922,8 +922,8 @@ func TestBMP(t *testing.T) {
 			`received_messages_total{exporter="127.0.0.1",type="statistics-report"}`:    "2",
 			`opened_connections_total{exporter="127.0.0.1"}`:                            "2",
 			`closed_connections_total{exporter="127.0.0.1"}`:                            "1",
-			`peers_total{exporter="127.0.0.1"}`:                                         "2",
-			`routes_total{exporter="127.0.0.1"}`:                                        "4",
+			`peers{exporter="127.0.0.1"}`:                                               "2",
+			`routes{exporter="127.0.0.1"}`:                                              "4",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
 			t.Errorf("Metrics (-got, +want):\n%s", diff)
@@ -944,7 +944,7 @@ func TestBMP(t *testing.T) {
 
 		mockClock.Add(2 * time.Hour)
 		time.Sleep(20 * time.Millisecond)
-		gotMetrics = r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "-locked_duration")
+		gotMetrics = r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "-locked_duration")
 		expectedMetrics = map[string]string{
 			`received_messages_total{exporter="127.0.0.1",type="initiation"}`:           "2",
 			`received_messages_total{exporter="127.0.0.1",type="peer-up-notification"}`: "2",
@@ -952,8 +952,8 @@ func TestBMP(t *testing.T) {
 			`received_messages_total{exporter="127.0.0.1",type="statistics-report"}`:    "2",
 			`opened_connections_total{exporter="127.0.0.1"}`:                            "2",
 			`closed_connections_total{exporter="127.0.0.1"}`:                            "1",
-			`peers_total{exporter="127.0.0.1"}`:                                         "1",
-			`routes_total{exporter="127.0.0.1"}`:                                        "2",
+			`peers{exporter="127.0.0.1"}`:                                               "1",
+			`routes{exporter="127.0.0.1"}`:                                              "2",
 			`removed_peers_total{exporter="127.0.0.1"}`:                                 "1",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
@@ -973,7 +973,7 @@ func TestBMP(t *testing.T) {
 
 		send(t, conn2, "bmp-terminate.pcap")
 		time.Sleep(30 * time.Millisecond)
-		gotMetrics = r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "-locked_duration")
+		gotMetrics = r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "-locked_duration")
 		expectedMetrics = map[string]string{
 			`received_messages_total{exporter="127.0.0.1",type="initiation"}`:           "2",
 			`received_messages_total{exporter="127.0.0.1",type="termination"}`:          "1",
@@ -982,8 +982,8 @@ func TestBMP(t *testing.T) {
 			`received_messages_total{exporter="127.0.0.1",type="statistics-report"}`:    "2",
 			`opened_connections_total{exporter="127.0.0.1"}`:                            "2",
 			`closed_connections_total{exporter="127.0.0.1"}`:                            "2",
-			`peers_total{exporter="127.0.0.1"}`:                                         "1",
-			`routes_total{exporter="127.0.0.1"}`:                                        "2",
+			`peers{exporter="127.0.0.1"}`:                                               "1",
+			`routes{exporter="127.0.0.1"}`:                                              "2",
 			`removed_peers_total{exporter="127.0.0.1"}`:                                 "1",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
@@ -996,7 +996,7 @@ func TestBMP(t *testing.T) {
 
 		mockClock.Add(2 * time.Hour)
 		time.Sleep(20 * time.Millisecond)
-		gotMetrics = r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "-locked_duration")
+		gotMetrics = r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "-locked_duration")
 		expectedMetrics = map[string]string{
 			`received_messages_total{exporter="127.0.0.1",type="initiation"}`:           "2",
 			`received_messages_total{exporter="127.0.0.1",type="termination"}`:          "1",
@@ -1005,8 +1005,8 @@ func TestBMP(t *testing.T) {
 			`received_messages_total{exporter="127.0.0.1",type="statistics-report"}`:    "2",
 			`opened_connections_total{exporter="127.0.0.1"}`:                            "2",
 			`closed_connections_total{exporter="127.0.0.1"}`:                            "2",
-			`peers_total{exporter="127.0.0.1"}`:                                         "0",
-			`routes_total{exporter="127.0.0.1"}`:                                        "0",
+			`peers{exporter="127.0.0.1"}`:                                               "0",
+			`routes{exporter="127.0.0.1"}`:                                              "0",
 			`removed_peers_total{exporter="127.0.0.1"}`:                                 "2",
 		}
 		if diff := helpers.Diff(gotMetrics, expectedMetrics); diff != "" {
@@ -1039,7 +1039,7 @@ func TestBMP(t *testing.T) {
 		mockClock.Add(2 * time.Hour)
 		for tries := 20; tries >= 0; tries-- {
 			time.Sleep(5 * time.Millisecond)
-			gotMetrics := r.GetMetrics("akvorado_inlet_routing_provider_bmp_", "-locked_duration")
+			gotMetrics := r.GetMetrics("akvorado_outlet_routing_provider_bmp_", "-locked_duration")
 			// For removed_partial_peers_total, we have 18 routes, but only 14 routes
 			// can be removed while keeping 1 route on each peer. 14 is the max, but
 			// we rely on good-willing from the scheduler to get this number.
@@ -1063,8 +1063,8 @@ func TestBMP(t *testing.T) {
 				`received_messages_total{exporter="127.0.0.1",type="statistics-report"}`:    "4",
 				`opened_connections_total{exporter="127.0.0.1"}`:                            "1",
 				`closed_connections_total{exporter="127.0.0.1"}`:                            "1",
-				`peers_total{exporter="127.0.0.1"}`:                                         "0",
-				`routes_total{exporter="127.0.0.1"}`:                                        "0",
+				`peers{exporter="127.0.0.1"}`:                                               "0",
+				`routes{exporter="127.0.0.1"}`:                                              "0",
 				`removed_peers_total{exporter="127.0.0.1"}`:                                 "4",
 				`removed_partial_peers_total{exporter="127.0.0.1"}`:                         fmt.Sprintf("%d", peerRemovalPartial),
 			}
diff --git a/inlet/routing/provider/bmp/testdata/bmp-eor.pcap b/outlet/routing/provider/bmp/testdata/bmp-eor.pcap
similarity index 100%
rename from inlet/routing/provider/bmp/testdata/bmp-eor.pcap
rename to outlet/routing/provider/bmp/testdata/bmp-eor.pcap
diff --git a/inlet/routing/provider/bmp/testdata/bmp-init.pcap b/outlet/routing/provider/bmp/testdata/bmp-init.pcap
similarity index 100%
rename from inlet/routing/provider/bmp/testdata/bmp-init.pcap
rename to outlet/routing/provider/bmp/testdata/bmp-init.pcap
diff --git a/inlet/routing/provider/bmp/testdata/bmp-l3vpn.pcap b/outlet/routing/provider/bmp/testdata/bmp-l3vpn.pcap
similarity index 100%
rename from inlet/routing/provider/bmp/testdata/bmp-l3vpn.pcap
rename to outlet/routing/provider/bmp/testdata/bmp-l3vpn.pcap
diff --git a/inlet/routing/provider/bmp/testdata/bmp-peer-down.pcap b/outlet/routing/provider/bmp/testdata/bmp-peer-down.pcap
similarity index 100%
rename from inlet/routing/provider/bmp/testdata/bmp-peer-down.pcap
rename to outlet/routing/provider/bmp/testdata/bmp-peer-down.pcap
diff --git a/inlet/routing/provider/bmp/testdata/bmp-peer-up.pcap b/outlet/routing/provider/bmp/testdata/bmp-peer-up.pcap
similarity index 100%
rename from inlet/routing/provider/bmp/testdata/bmp-peer-up.pcap
rename to outlet/routing/provider/bmp/testdata/bmp-peer-up.pcap
diff --git a/inlet/routing/provider/bmp/testdata/bmp-peers-up.pcap b/outlet/routing/provider/bmp/testdata/bmp-peers-up.pcap
similarity index 100%
rename from inlet/routing/provider/bmp/testdata/bmp-peers-up.pcap
rename to outlet/routing/provider/bmp/testdata/bmp-peers-up.pcap
diff --git a/inlet/routing/provider/bmp/testdata/bmp-reach-addpath.pcap b/outlet/routing/provider/bmp/testdata/bmp-reach-addpath.pcap
similarity index 100%
rename from inlet/routing/provider/bmp/testdata/bmp-reach-addpath.pcap
rename to outlet/routing/provider/bmp/testdata/bmp-reach-addpath.pcap
diff --git a/inlet/routing/provider/bmp/testdata/bmp-reach-unknown-family.pcap b/outlet/routing/provider/bmp/testdata/bmp-reach-unknown-family.pcap
similarity index 100%
rename from inlet/routing/provider/bmp/testdata/bmp-reach-unknown-family.pcap
rename to outlet/routing/provider/bmp/testdata/bmp-reach-unknown-family.pcap
diff --git a/inlet/routing/provider/bmp/testdata/bmp-reach-vpls.pcap b/outlet/routing/provider/bmp/testdata/bmp-reach-vpls.pcap
similarity index 100%
rename from inlet/routing/provider/bmp/testdata/bmp-reach-vpls.pcap
rename to outlet/routing/provider/bmp/testdata/bmp-reach-vpls.pcap
diff --git a/inlet/routing/provider/bmp/testdata/bmp-reach.pcap b/outlet/routing/provider/bmp/testdata/bmp-reach.pcap
similarity index 100%
rename from inlet/routing/provider/bmp/testdata/bmp-reach.pcap
rename to outlet/routing/provider/bmp/testdata/bmp-reach.pcap
diff --git a/inlet/routing/provider/bmp/testdata/bmp-terminate.pcap b/outlet/routing/provider/bmp/testdata/bmp-terminate.pcap
similarity index 100%
rename from inlet/routing/provider/bmp/testdata/bmp-terminate.pcap
rename to outlet/routing/provider/bmp/testdata/bmp-terminate.pcap
diff --git a/inlet/routing/provider/bmp/testdata/bmp-unreach.pcap b/outlet/routing/provider/bmp/testdata/bmp-unreach.pcap
similarity index 100%
rename from inlet/routing/provider/bmp/testdata/bmp-unreach.pcap
rename to outlet/routing/provider/bmp/testdata/bmp-unreach.pcap
diff --git a/inlet/routing/provider/bmp/tests.go b/outlet/routing/provider/bmp/tests.go
similarity index 99%
rename from inlet/routing/provider/bmp/tests.go
rename to outlet/routing/provider/bmp/tests.go
index b18bb5b0..b29711d1 100644
--- a/inlet/routing/provider/bmp/tests.go
+++ b/outlet/routing/provider/bmp/tests.go
@@ -12,7 +12,7 @@ import (
 
 	"akvorado/common/daemon"
 	"akvorado/common/reporter"
-	"akvorado/inlet/routing/provider"
+	"akvorado/outlet/routing/provider"
 
 	"github.com/benbjohnson/clock"
 	"github.com/osrg/gobgp/v3/pkg/packet/bgp"
diff --git a/inlet/routing/provider/bmp/utils_test.go b/outlet/routing/provider/bmp/utils_test.go
similarity index 100%
rename from inlet/routing/provider/bmp/utils_test.go
rename to outlet/routing/provider/bmp/utils_test.go
diff --git a/inlet/routing/root_test.go b/outlet/routing/root_test.go
similarity index 100%
rename from inlet/routing/root_test.go
rename to outlet/routing/root_test.go
diff --git a/inlet/routing/tests.go b/outlet/routing/tests.go
similarity index 95%
rename from inlet/routing/tests.go
rename to outlet/routing/tests.go
index 1c521662..d64886c3 100644
--- a/inlet/routing/tests.go
+++ b/outlet/routing/tests.go
@@ -10,7 +10,7 @@ import (
 
 	"akvorado/common/daemon"
 	"akvorado/common/reporter"
-	"akvorado/inlet/routing/provider/bmp"
+	"akvorado/outlet/routing/provider/bmp"
 )
 
 // NewMock creates a new mock component using BMP as a provider (it's a real one