Repository: honeycombio/refinery
Pull request #709 (2023-06-09 02:02:10)

feat: Add WindowedThroughput sampler
Depends on #708 

## Which problem is this PR solving?

- Adds the other new sampler from the latest dynsampler-go.

## Short description of the changes

- Add the sampler and its config info
- Update the docs
- Add a basic test (dynsampler has all the sampler testing)




Patch:
diff --git a/config/metadata/rulesMeta.yaml b/config/metadata/rulesMeta.yaml
index ed026b7..6940e1d 100644
--- a/config/metadata/rulesMeta.yaml
+++ b/config/metadata/rulesMeta.yaml
@@ -402,6 +402,111 @@ groups:
           is a useful indicator of traces you'd like to see in Honeycomb, set
           this to true.
 
+  - name: WindowedThroughputSampler
+    title: Windowed Throughput Sampler
+    description: >
+      Windowed Throughput sampling is an enhanced version of total throughput
+      sampling. Just like the TotalThroughput sampler, it attempts to meet the
+      goal of fixed number of events per second sent to Honeycomb.
+
+      The original throughput sampler updates the sampling rate every
+      "ClearFrequency" seconds. While this parameter is configurable, it suffers
+      from the following tradeoff:
+        - Decreasing it is more responsive to load spikes, but with the
+        cost of making the sampling decision on less data.
+        - Increasing it is less responsive to load spikes, but sample rates will be more
+          stable because they are made with more data.
+
+      The windowed throughput sampler resolves this by introducing two
+      different, tunable parameters:
+        - UpdateFrequency: how often the sampling rate is recomputed
+        - LookbackFrequency: how much total time is considered when recomputing sampling rate.
+
+      A standard configuration would be to set UpdateFrequency to 1s and
+      LookbackFrequency to 30s. In this configuration, every second, we lookback
+      at the last 30s of data in order to compute the new sampling rate. The
+      actual sampling rate computation is nearly identical to the original
+      throughput sampler, but this variant has better support for floating point
+      numbers.
+
+    fields:
+      - name: GoalThroughputPerSec
+        type: int
+        validations:
+          - type: requiredInGroup
+        summary: is the desired throughput per second of events sent to Honeycomb.
+        description: >
+          The desired throughput **per second**. This is the number of events per
+          second you want to send to Honeycomb. The sampler will adjust sample
+          rates to try to achieve this desired throughput. This value is
+          calculated for the individual instance, not for the cluster; if your
+          cluster has multiple instances, you will need to divide your total
+          desired sample rate by the number of instances to get this value.
+      - name: UpdateFrequency
+        type: duration
+        summary: is how often the sampling rate is recomputed.
+        description: >
+          The duration between sampling rate computations. It should be
+          specified as a duration string, e.g. "30s" or "1m".
+      - name: LookbackFrequency
+        type: duration
+        summary: how far back in time to look when computing the sampling rate.
+        description: >
+          This controls how far back in time to lookback to dynamically adjust the sampling
+          rate. Default is 30 * UpdateFrequencyDuration.
+          This is forced to be an _integer multiple_ of UpdateFrequencyDuration.
+      - name: FieldList
+        type: stringarray
+        validations:
+          - type: requiredInGroup
+          - type: notempty
+        summary: is the list of fields to use to create the key for the dynamic sampler.
+        description: >
+          FieldList is a list of all the field names to use to form the key that
+          will be handed to the dynamic sampler. The combination of values from
+          all of these fields should reflect how interesting the trace is
+          compared to another. A good field selection has consistent values for
+          high-frequency, boring traffic, and unique values for outliers and
+          interesting traffic. Including an error field (or something like HTTP
+          status code) is an excellent choice. Using fields with very high
+          cardinality (like `k8s.pod.id`), is a bad choice. If the combination
+          of fields essentially makes them unique, the dynamic sampler will
+          sample everything. If the combination of fields is not unique enough,
+          you will not be guaranteed samples of the most interesting traces. As
+          an example, consider a combination of HTTP endpoint (high-frequency
+          and boring), HTTP method, and status code (normally boring but can
+          become interesting when indicating an error) as a good set of fields
+          since it will allowing proper sampling of all endpoints under normal
+          traffic and call out when there is failing traffic to any endpoint.
+          For example, in contrast, consider a combination of HTTP endpoint,
+          status code, and pod id as a bad set of fields, since it would result
+          in keys that are all unique, and therefore results in sampling 100% of
+          traces. Using only the HTTP endpoint field would be a **bad** choice,
+          as it is not unique enough and therefore interesting traces, like
+          traces that experienced a `500`, might not be sampled. Field names may
+          come from any span in the trace; if they occur on multiple spans, all
+          unique values will be included in the key.
+      - name: MaxKeys
+        type: int
+        summary: is the maximum number of keys to track.
+        description: >
+          MaxKeys limits the number of distinct keys tracked by the sampler.
+          Once MaxKeys is reached, new keys will not be included in the sample
+          rate map, but existing keys will continue to be be counted. You can
+          use this to keep the sample rate map size under control. Defaults to
+          500; dynamic samplers will rarely achieve their goals with more keys
+          than this.
+      - name: UseTraceLength
+        type: bool
+        summary: indicates whether to include the trace length as part of the key.
+        description: >
+          Indicates whether to include the trace length (number of spans in the
+          trace) as part of the key. The number of spans is exact, so if there
+          are normally small variations in trace length you may want to leave
+          this off. If traces are consistent lengths and changes in trace length
+          is a useful indicator of traces you'd like to see in Honeycomb, set
+          this to true.
+
   - name: TotalThroughputSampler
     title: Total Throughput Sampler
     description: >
diff --git a/config/sampler_config.go b/config/sampler_config.go
index 9116092..b4a72dd 100644
--- a/config/sampler_config.go
+++ b/config/sampler_config.go
@@ -11,12 +11,13 @@ import (
 // The yaml tags are used for the new format and are PascalCase.
 
 type V2SamplerChoice struct {
-	DeterministicSampler   *DeterministicSamplerConfig   `json:"deterministicsampler" yaml:"DeterministicSampler,omitempty"`
-	RulesBasedSampler      *RulesBasedSamplerConfig      `json:"rulesbasedsampler" yaml:"RulesBasedSampler,omitempty"`
-	DynamicSampler         *DynamicSamplerConfig         `json:"dynamicsampler" yaml:"DynamicSampler,omitempty"`
-	EMADynamicSampler      *EMADynamicSamplerConfig      `json:"emadynamicsampler" yaml:"EMADynamicSampler,omitempty"`
-	EMAThroughputSampler   *EMAThroughputSamplerConfig   `json:"emathroughputsampler" yaml:"EMAThroughputSampler,omitempty"`
-	TotalThroughputSampler *TotalThroughputSamplerConfig `json:"totalthroughputsampler" yaml:"TotalThroughputSampler,omitempty"`
+	DeterministicSampler      *DeterministicSamplerConfig      `json:"deterministicsampler" yaml:"DeterministicSampler,omitempty"`
+	RulesBasedSampler         *RulesBasedSamplerConfig         `json:"rulesbasedsampler" yaml:"RulesBasedSampler,omitempty"`
+	DynamicSampler            *DynamicSamplerConfig            `json:"dynamicsampler" yaml:"DynamicSampler,omitempty"`
+	EMADynamicSampler         *EMADynamicSamplerConfig         `json:"emadynamicsampler" yaml:"EMADynamicSampler,omitempty"`
+	EMAThroughputSampler      *EMAThroughputSamplerConfig      `json:"emathroughputsampler" yaml:"EMAThroughputSampler,omitempty"`
+	WindowedThroughputSampler *WindowedThroughputSamplerConfig `json:"windowedthroughputsampler" yaml:"WindowedThroughputSampler,omitempty"`
+	TotalThroughputSampler    *TotalThroughputSamplerConfig    `json:"totalthroughputsampler" yaml:"TotalThroughputSampler,omitempty"`
 }
 
 func (v *V2SamplerChoice) Sampler() (any, string) {
@@ -31,6 +32,8 @@ func (v *V2SamplerChoice) Sampler() (any, string) {
 		return v.EMADynamicSampler, "EMADynamicSampler"
 	case v.EMAThroughputSampler != nil:
 		return v.EMAThroughputSampler, "EMAThroughputSampler"
+	case v.WindowedThroughputSampler != nil:
+		return v.WindowedThroughputSampler, "WindowedThroughputSampler"
 	case v.TotalThroughputSampler != nil:
 		return v.TotalThroughputSampler, "TotalThroughputSampler"
 	default:
@@ -80,6 +83,15 @@ type EMAThroughputSamplerConfig struct {
 	UseTraceLength       bool     `json:"usetracelength" yaml:"UseTraceLength,omitempty"`
 }
 
+type WindowedThroughputSamplerConfig struct {
+	UpdateFrequency      Duration `json:"updatefrequency" yaml:"UpdateFrequency,omitempty"`
+	LookbackFrequency    Duration `json:"lookbackfrequency" yaml:"LookbackFrequency,omitempty"`
+	GoalThroughputPerSec int      `json:"goalthroughputpersec" yaml:"GoalThroughputPerSec,omitempty"`
+	FieldList            []string `json:"fieldlist" yaml:"FieldList,omitempty"`
+	MaxKeys              int      `json:"maxkeys" yaml:"MaxKeys,omitempty"`
+	UseTraceLength       bool     `json:"usetracelength" yaml:"UseTraceLength,omitempty"`
+}
+
 type TotalThroughputSamplerConfig struct {
 	GoalThroughputPerSec int64    `json:"goalthroughputpersec" yaml:"GoalThroughputPerSec,omitempty" validate:"gte=1"`
 	ClearFrequency       Duration `json:"clearfrequency" yaml:"ClearFrequency,omitempty"`
@@ -95,10 +107,11 @@ type RulesBasedSamplerConfig struct {
 }
 
 type RulesBasedDownstreamSampler struct {
-	DynamicSampler         *DynamicSamplerConfig         `json:"dynamicsampler" yaml:"DynamicSampler,omitempty"`
-	EMADynamicSampler      *EMADynamicSamplerConfig      `json:"emadynamicsampler" yaml:"EMADynamicSampler,omitempty"`
-	EMAThroughputSampler   *EMAThroughputSamplerConfig   `json:"emathroughputsampler" yaml:"EMAThroughputSampler,omitempty"`
-	TotalThroughputSampler *TotalThroughputSamplerConfig `json:"totalthroughputsampler" yaml:"TotalThroughputSampler,omitempty"`
+	DynamicSampler            *DynamicSamplerConfig            `json:"dynamicsampler" yaml:"DynamicSampler,omitempty"`
+	EMADynamicSampler         *EMADynamicSamplerConfig         `json:"emadynamicsampler" yaml:"EMADynamicSampler,omitempty"`
+	EMAThroughputSampler      *EMAThroughputSamplerConfig      `json:"emathroughputsampler" yaml:"EMAThroughputSampler,omitempty"`
+	WindowedThroughputSampler *WindowedThroughputSamplerConfig `json:"windowedthroughputsampler" yaml:"WindowedThroughputSampler,omitempty"`
+	TotalThroughputSampler    *TotalThroughputSamplerConfig    `json:"totalthroughputsampler" yaml:"TotalThroughputSampler,omitempty"`
 }
 
 type RulesBasedSamplerRule struct {
diff --git a/rules_complete.toml.old b/rules_complete.toml.old
new file mode 100644
index 0000000..0c7dae8
--- /dev/null
+++ b/rules_complete.toml.old
@@ -0,0 +1,294 @@
+############################
+## Sampling Rules Config ##
+############################
+
+# DryRun - If enabled, marks traces that would be dropped given current sampling rules,
+# and sends all traces regardless
+# DryRun = false
+
+# DryRunFieldName - the key to add to use to add to event data when using DryRun mode above, defaults to refinery_kept
+# DryRunFieldName = "refinery_kept"
+
+# DeterministicSampler is a section of the config for manipulating the
+# Deterministic Sampler implementation. This is the simplest sampling algorithm
+# - it is a static sample rate, choosing traces randomly to either keep or send
+# (at the appropriate rate). It is not influenced by the contents of the trace.
+Sampler = "DeterministicSampler"
+
+# SampleRate is the rate at which to sample. It indicates a ratio, where one
+# sample trace is kept for every n traces seen. For example, a SampleRate of 30
+# will keep 1 out of every 30 traces. The choice on whether to keep any specific
+# trace is random, so the rate is approximate.
+# Eligible for live reload.
+SampleRate = 1
+
+[dataset1]
+
+	# Note: If your dataset name contains a space, you will have to escape the dataset name
+	# using single quotes, such as ['dataset 1']
+
+	# DynamicSampler is a section of the config for manipulating the simple Dynamic Sampler
+	# implementation. This sampler collects the values of a number of fields from a
+	# trace and uses them to form a key. This key is handed to the standard dynamic
+	# sampler algorithm which generates a sample rate based on the frequency with
+	# which that key has appeared in the previous ClearFrequency. See
+	# https://github.com/honeycombio/dynsampler-go for more detail on the mechanics
+	# of the dynamic sampler.  This sampler uses the AvgSampleRate algorithm from
+	# that package.
+	Sampler = "DynamicSampler"
+
+	# SampleRate is the goal rate at which to sample. It indicates a ratio, where
+	# one sample trace is kept for every n traces seen. For example, a SampleRate of
+	# 30 will keep 1 out of every 30 traces. This rate is handed to the dynamic
+	# sampler, who assigns a sample rate for each trace based on the fields selected
+	# from that trace.
+	# Eligible for live reload.
+	SampleRate = 2
+
+	# FieldList is a list of all the field names to use to form the key that will be handed to the dynamic sampler.
+	# The combination of values from all of these fields should reflect how interesting the trace is compared to
+	# another. A good field selection has consistent values for high-frequency, boring traffic, and unique values for
+	# outliers and interesting traffic. Including an error field (or something like HTTP status code) is an excellent
+	# choice. Using fields with very high cardinality (like `k8s.pod.id`), is a bad choice. If the combination of
+	# fields essentially makes them unique, the dynamic sampler will sample everything. If the combination of fields is
+	# not unique enough, you will not be guaranteed samples of the most interesting traces. As an example, consider a
+	# combination of HTTP endpoint (high-frequency and boring), HTTP method, and status code (normally boring but can
+	# become interesting when indicating an error) as a good set of fields since it will allowing proper sampling
+	# of all endpoints under normal traffic and call out when there is failing traffic to any endpoint.
+	# For example, in contrast, consider a combination of HTTP endpoint, status code, and pod id as a bad set of
+	# fields, since it would result in keys that are all unique, and therefore results in sampling 100% of traces.
+	# Using only the HTTP endpoint field would be a **bad** choice, as it is not unique enough and therefore
+	# interesting traces, like traces that experienced a `500`, might not be sampled.
+	# Field names may come from any span in the trace.
+	# Eligible for live reload.
+	FieldList = ["request.method", "http.target", "response.status_code"]
+
+	# UseTraceLength will add the number of spans in the trace into the dynamic
+	# sampler as part of the key. The number of spans is exact, so if there are
+	# normally small variations in trace length you may want to leave this off. If
+	# traces are consistent lengths and changes in trace length is a useful
+	# indicator of traces you'd like to see in Honeycomb, set this to true.
+	# Eligible for live reload.
+	UseTraceLength = true
+
+	# AddSampleRateKeyToTrace when this is set to true, the sampler will add a field
+	# to the root span of the trace containing the key used by the sampler to decide
+	# the sample rate. This can be helpful in understanding why the sampler is
+	# making certain decisions about sample rate and help you understand how to
+	# better choose the sample rate key (aka the FieldList setting above) to use.
+	AddSampleRateKeyToTrace = true
+
+	# AddSampleRateKeyToTraceField is the name of the field the sampler will use
+	# when adding the sample rate key to the trace. This setting is only used when
+	# AddSampleRateKeyToTrace is true.
+	AddSampleRateKeyToTraceField = "meta.refinery.dynsampler_key"
+
+	# ClearFrequency is the name of the field the sampler will use to determine
+	# the period over which it will calculate the sample rate. This setting defaults
+	# to 30.
+	# Eligible for live reload.
+	ClearFrequency = "60s"
+
+[dataset2]
+
+	# EMADynamicSampler is a section of the config for manipulating the Exponential
+	# Moving Average (EMA) Dynamic Sampler implementation. Like the simple DynamicSampler,
+	# it attempts to average a given sample rate, weighting rare traffic and frequent
+	# traffic differently so as to end up with the correct average.
+	#
+	# EMADynamicSampler is an improvement upon the simple DynamicSampler and is recommended
+	# for most use cases. Based on the DynamicSampler implementation, EMADynamicSampler differs
+	# in that rather than compute rate based on a periodic sample of traffic, it maintains an Exponential
+	# Moving Average of counts seen per key, and adjusts this average at regular intervals.
+	# The weight applied to more recent intervals is defined by `weight`, a number between
+	# (0, 1) - larger values weight the average more toward recent observations. In other words,
+	# a larger weight will cause sample rates more quickly adapt to traffic patterns,
+	# while a smaller weight will result in sample rates that are less sensitive to bursts or drops
+	# in traffic and thus more consistent over time.
+	#
+	# Keys that are not found in the EMA will always have a sample
+	# rate of 1. Keys that occur more frequently will be sampled on a logarithmic
+	# curve. In other words, every key will be represented at least once in any
+	# given window and more frequent keys will have their sample rate
+	# increased proportionally to wind up with the goal sample rate.
+	Sampler = "EMADynamicSampler"
+
+	# GoalSampleRate is the goal rate at which to sample. It indicates a ratio, where
+	# one sample trace is kept for every n traces seen. For example, a SampleRate of
+	# 30 will keep 1 out of every 30 traces. This rate is handed to the dynamic
+	# sampler, who assigns a sample rate for each trace based on the fields selected
+	# from that trace.
+	# Eligible for live reload.
+	GoalSampleRate = 2
+
+	# FieldList is a list of all the field names to use to form the key that will be handed to the dynamic sampler.
+	# The combination of values from all of these fields should reflect how interesting the trace is compared to
+    # another. A good field selection has consistent values for high-frequency, boring traffic, and unique values for
+	# outliers and interesting traffic. Including an error field (or something like HTTP status code) is an excellent
+	# choice. Using fields with very high cardinality (like `k8s.pod.id`), is a bad choice. If the combination of
+	# fields essentially makes them unique, the dynamic sampler will sample everything. If the combination of fields is
+	# not unique enough, you will not be guaranteed samples of the most interesting traces. As an example, consider a
+	# combination of HTTP endpoint (high-frequency and boring), HTTP method, and status code (normally boring but can
+	# become interesting when indicating an error) as a good set of fields since it will allowing proper sampling
+	# of all endpoints under normal traffic and call out when there is failing traffic to any endpoint.
+	# For example, in contrast, consider a combination of HTTP endpoint, status code, and pod id as a bad set of
+	# fields, since it would result in keys that are all unique, and therefore results in sampling 100% of traces.
+	# Using only the HTTP endpoint field would be a **bad** choice, as it is not unique enough and therefore
+	# interesting traces, like traces that experienced a `500`, might not be sampled.
+	# Field names may come from any span in the trace.
+	# Eligible for live reload.
+	FieldList = ["request.method", "http.target", "response.status_code"]
+
+	# UseTraceLength will add the number of spans in the trace into the dynamic
+	# sampler as part of the key. The number of spans is exact, so if there are
+	# normally small variations in trace length you may want to leave this off. If
+	# traces are consistent lengths and changes in trace length is a useful
+	# indicator of traces you'd like to see in Honeycomb, set this to true.
+	# Eligible for live reload.
+	UseTraceLength = true
+
+	# AddSampleRateKeyToTrace when this is set to true, the sampler will add a field
+	# to the root span of the trace containing the key used by the sampler to decide
+	# the sample rate. This can be helpful in understanding why the sampler is
+	# making certain decisions about sample rate and help you understand how to
+	# better choose the sample rate key (aka the FieldList setting above) to use.
+	AddSampleRateKeyToTrace = true
+
+	# AddSampleRateKeyToTraceField is the name of the field the sampler will use
+	# when adding the sample rate key to the trace. This setting is only used when
+	# AddSampleRateKeyToTrace is true.
+	AddSampleRateKeyToTraceField = "meta.refinery.dynsampler_key"
+
+	# AdjustmentInterval defines how often (in seconds) we adjust the moving average from
+	# recent observations. Default 15s
+	# Eligible for live reload.
+	AdjustmentInterval = 15
+
+	# Weight is a value between (0, 1) indicating the weighting factor used to adjust
+	# the EMA. With larger values, newer data will influence the average more, and older
+	# values will be factored out more quickly.  In mathematical literature concerning EMA,
+	# this is referred to as the `alpha` constant.
+	# Default is 0.5
+	# Eligible for live reload.
+	Weight = 0.5
+
+	# MaxKeys, if greater than 0, limits the number of distinct keys tracked in EMA.
+	# Once MaxKeys is reached, new keys will not be included in the sample rate map, but
+	# existing keys will continue to be be counted. You can use this to keep the sample rate
+	# map size under control.
+	# Eligible for live reload
+	MaxKeys = 0
+
+	# AgeOutValue indicates the threshold for removing keys from the EMA. The EMA of any key
+	# will approach 0 if it is not repeatedly observed, but will never truly reach it, so we have to
+	# decide what constitutes "zero". Keys with averages below this threshold will be removed
+	# from the EMA. Default is the same as Weight, as this prevents a key with the smallest
+	# integer value (1) from being aged out immediately. This value should generally be <= Weight,
+	# unless you have very specific reasons to set it higher.
+	# Eligible for live reload
+	AgeOutValue = 0.5
+
+	# BurstMultiple, if set, is multiplied by the sum of the running average of counts to define
+	# the burst detection threshold. If total counts observed for a given interval exceed the threshold
+	# EMA is updated immediately, rather than waiting on the AdjustmentInterval.
+	# Defaults to 2; negative value disables. With a default of 2, if your traffic suddenly doubles,
+	# burst detection will kick in.
+	# Eligible for live reload
+	BurstMultiple = 2.0
+
+	# BurstDetectionDelay indicates the number of intervals to run after Start is called before
+	# burst detection kicks in.
+	# Defaults to 3
+	# Eligible for live reload
+	BurstDetectionDelay = 3
+
+[dataset3]
+
+	Sampler = "DeterministicSampler"
+	SampleRate = 10
+
+[dataset4]
+
+	Sampler = "RulesBasedSampler"
+	# Optional, if set to true then the rules will also check nested json fields, in the format of parent.child
+	CheckNestedFields = false
+
+	[[dataset4.rule]]
+		name = "drop healthchecks"
+		drop = true
+		[[dataset4.rule.condition]]
+			field = "http.route"
+			operator = "="
+			value = "/health-check"
+
+	[[dataset4.rule]]
+		name = "keep slow 500 errors"
+		SampleRate = 1
+		[[dataset4.rule.condition]]
+			field = "status_code"
+			operator = "="
+			value = 500
+		[[dataset4.rule.condition]]
+			field = "duration_ms"
+			operator = ">="
+			value = 1000.789
+
+	[[dataset4.rule]]
+		name = "dynamically sample 200 responses"
+		[[dataset4.rule.condition]]
+			field = "status_code"
+			operator = "="
+			value = 200
+		[dataset4.rule.sampler.EMADynamicSampler]
+			Sampler = "EMADynamicSampler"
+			GoalSampleRate = 15
+			FieldList = ["request.method", "request.route"]
+			AddSampleRateKeyToTrace = true
+			AddSampleRateKeyToTraceField = "meta.refinery.dynsampler_key"
+
+	# Note that Refinery comparisons are type-dependent. If you are operating in an environment where different
+	# telemetry may send the same field with different types (for example, some systems send status codes as "200"
+	# instead of 200), you may wish to use the "datatype" setting to force them all to the same type.
+	[[dataset4.rule]]
+		name = "dynamically sample 200 string responses"
+		[[dataset4.rule.condition]]
+			field = "status_code"
+			operator = "="
+			value = "200"
+			datatype = "int"
+		[dataset4.rule.sampler.EMADynamicSampler]
+			Sampler = "EMADynamicSampler"
+			GoalSampleRate = 15
+			FieldList = ["request.method", "request.route"]
+			AddSampleRateKeyToTrace = true
+			AddSampleRateKeyToTraceField = "meta.refinery.dynsampler_key"
+
+	[[dataset4.rule]]
+		name = "sample traces originating from a service"
+		# if scope is set to "span", a single span in the trace must match
+		# *all* of the conditions associated with this rule for the rule to
+		# apply to the trace.
+		#
+		# this is especially helpful when sampling a dataset written to
+		# by multiple services that call one another in normal operation –
+		# you can set Scope to 'span' to attribute traces to an origin
+		# service in a way that would be difficult without it.
+		Scope = "span"
+		SampleRate = 5
+		[[dataset4.rule.condition]]
+			field = "service name"
+			operator = "="
+			value = "users"
+		[[dataset4.rule.condition]]
+			field = "trace.parent_id"
+			operator = "="
+			value = "root"
+
+	[[dataset4.rule]]
+		SampleRate = 10 # default when no rules match, if missing defaults to 10
+
+[dataset5]
+
+    Sampler = "TotalThroughputSampler"
+    GoalThroughputPerSec = 100
+    FieldList = ["request.method"]
diff --git a/sample/windowed_throughput.go b/sample/windowed_throughput.go
new file mode 100644
index 0000000..22e5101
--- /dev/null
+++ b/sample/windowed_throughput.go
@@ -0,0 +1,76 @@
+package sample
+
+import (
+	"math/rand"
+	"time"
+
+	dynsampler "github.com/honeycombio/dynsampler-go"
+
+	"github.com/honeycombio/refinery/config"
+	"github.com/honeycombio/refinery/logger"
+	"github.com/honeycombio/refinery/metrics"
+	"github.com/honeycombio/refinery/types"
+)
+
+type WindowedThroughputSampler struct {
+	Config  *config.WindowedThroughputSamplerConfig
+	Logger  logger.Logger
+	Metrics metrics.Metrics
+
+	updatefrequency      config.Duration
+	lookbackfrequency    config.Duration
+	goalthroughputpersec int
+	maxKeys              int
+
+	key *traceKey
+
+	dynsampler dynsampler.Sampler
+}
+
+func (d *WindowedThroughputSampler) Start() error {
+	d.Logger.Debug().Logf("Starting WindowedThroughputSampler")
+	defer func() { d.Logger.Debug().Logf("Finished starting WindowedThroughputSampler") }()
+	d.goalthroughputpersec = d.Config.GoalThroughputPerSec
+	d.updatefrequency = d.Config.UpdateFrequency
+	d.lookbackfrequency = d.Config.LookbackFrequency
+	d.maxKeys = d.Config.MaxKeys
+	d.key = newTraceKey(d.Config.FieldList, d.Config.UseTraceLength)
+
+	// spin up the actual dynamic sampler
+	d.dynsampler = &dynsampler.WindowedThroughput{
+		GoalThroughputPerSec:      float64(d.goalthroughputpersec),
+		UpdateFrequencyDuration:   time.Duration(d.updatefrequency),
+		LookbackFrequencyDuration: time.Duration(d.lookbackfrequency),
+		MaxKeys:                   d.maxKeys,
+	}
+	d.dynsampler.Start()
+
+	// Register statistics this package will produce
+	d.Metrics.Register("dynsampler_num_dropped", "counter")
+	d.Metrics.Register("dynsampler_num_kept", "counter")
+	d.Metrics.Register("dynsampler_sample_rate", "histogram")
+
+	return nil
+}
+
+func (d *WindowedThroughputSampler) GetSampleRate(trace *types.Trace) (rate uint, keep bool, reason string, key string) {
+	key = d.key.build(trace)
+	rate = uint(d.dynsampler.GetSampleRate(key))
+	if rate < 1 { // protect against dynsampler being broken even though it shouldn't be
+		rate = 1
+	}
+	shouldKeep := rand.Intn(int(rate)) == 0
+	d.Logger.Debug().WithFields(map[string]interface{}{
+		"sample_key":  key,
+		"sample_rate": rate,
+		"sample_keep": shouldKeep,
+		"trace_id":    trace.TraceID,
+	}).Logf("got sample rate and decision")
+	if shouldKeep {
+		d.Metrics.Increment("dynsampler_num_kept")
+	} else {
+		d.Metrics.Increment("dynsampler_num_dropped")
+	}
+	d.Metrics.Histogram("dynsampler_sample_rate", float64(rate))
+	return rate, shouldKeep, "Windowedthroughput", key
+}
diff --git a/tools/convert/rules.md b/tools/convert/rules.md
index 5d7cc03..98f2105 100644
--- a/tools/convert/rules.md
+++ b/tools/convert/rules.md
@@ -602,6 +602,165 @@ Type: `bool`
 
 
 
+---
+## WindowedThroughputSampler: Windowed Throughput Sampler
+
+Windowed Throughput sampling is an enhanced version of total
+throughput sampling. Just like the TotalThroughput sampler, it
+attempts to meet the goal of fixed number of events per second sent to
+Honeycomb.
+The original throughput sampler updates the sampling rate every
+"ClearFrequency" seconds. While this parameter is configurable, it
+suffers from the following tradeoff:
+- Decreasing it is more responsive to load spikes, but with the
+cost of making the sampling decision on less data.
+- Increasing it is less responsive to load spikes, but sample rates
+will be more
+stable because they are made with more data.
+
+The windowed throughput sampler resolves this by introducing two
+different, tunable parameters:
+- UpdateFrequency: how often the sampling rate is recomputed
+- LookbackFrequency: how much total time is considered when
+recomputing sampling rate.
+
+A standard configuration would be to set UpdateFrequency to 1s and
+LookbackFrequency to 30s. In this configuration, every second, we
+lookback at the last 30s of data in order to compute the new sampling
+rate. The actual sampling rate computation is nearly identical to the
+original throughput sampler, but this variant has better support for
+floating point numbers.
+
+
+### GoalThroughputPerSec
+
+GoalThroughputPerSec is the desired throughput per second of events
+sent to Honeycomb.
+
+The desired throughput **per second**. This is the number of events
+per second you want to send to Honeycomb. The sampler will adjust
+sample rates to try to achieve this desired throughput. This value is
+calculated for the individual instance, not for the cluster; if your
+cluster has multiple instances, you will need to divide your total
+desired sample rate by the number of instances to get this value.
+
+
+Not eligible for live reload.
+
+Type: `int`
+
+
+
+
+### UpdateFrequency
+
+UpdateFrequency is how often the sampling rate is recomputed.
+
+The duration between sampling rate computations. It should be
+specified as a duration string, e.g. "30s" or "1m".
+
+
+Not eligible for live reload.
+
+Type: `duration`
+
+
+
+
+### LookbackFrequency
+
+LookbackFrequency how far back in time to look when computing the
+sampling rate.
+
+This controls how far back in time to lookback to dynamically adjust
+the sampling rate. Default is 30 * UpdateFrequencyDuration. This is
+forced to be an _integer multiple_ of UpdateFrequencyDuration.
+
+
+Not eligible for live reload.
+
+Type: `duration`
+
+
+
+
+### FieldList
+
+FieldList is the list of fields to use to create the key for the
+dynamic sampler.
+
+FieldList is a list of all the field names to use to form the key that
+will be handed to the dynamic sampler. The combination of values from
+all of these fields should reflect how interesting the trace is
+compared to another. A good field selection has consistent values for
+high-frequency, boring traffic, and unique values for outliers and
+interesting traffic. Including an error field (or something like HTTP
+status code) is an excellent choice. Using fields with very high
+cardinality (like `k8s.pod.id`), is a bad choice. If the combination
+of fields essentially makes them unique, the dynamic sampler will
+sample everything. If the combination of fields is not unique enough,
+you will not be guaranteed samples of the most interesting traces. As
+an example, consider a combination of HTTP endpoint (high-frequency
+and boring), HTTP method, and status code (normally boring but can
+become interesting when indicating an error) as a good set of fields
+since it will allowing proper sampling of all endpoints under normal
+traffic and call out when there is failing traffic to any endpoint.
+For example, in contrast, consider a combination of HTTP endpoint,
+status code, and pod id as a bad set of fields, since it would result
+in keys that are all unique, and therefore results in sampling 100% of
+traces. Using only the HTTP endpoint field would be a **bad** choice,
+as it is not unique enough and therefore interesting traces, like
+traces that experienced a `500`, might not be sampled. Field names may
+come from any span in the trace; if they occur on multiple spans, all
+unique values will be included in the key.
+
+
+Not eligible for live reload.
+
+Type: `stringarray`
+
+
+
+
+### MaxKeys
+
+MaxKeys is the maximum number of keys to track.
+
+MaxKeys limits the number of distinct keys tracked by the sampler.
+Once MaxKeys is reached, new keys will not be included in the sample
+rate map, but existing keys will continue to be be counted. You can
+use this to keep the sample rate map size under control. Defaults to
+500; dynamic samplers will rarely achieve their goals with more keys
+than this.
+
+
+Not eligible for live reload.
+
+Type: `int`
+
+
+
+
+### UseTraceLength
+
+UseTraceLength indicates whether to include the trace length as part
+of the key.
+
+Indicates whether to include the trace length (number of spans in the
+trace) as part of the key. The number of spans is exact, so if there
+are normally small variations in trace length you may want to leave
+this off. If traces are consistent lengths and changes in trace length
+is a useful indicator of traces you'd like to see in Honeycomb, set
+this to true.
+
+
+Not eligible for live reload.
+
+Type: `bool`
+
+
+
+
 ---
 ## TotalThroughputSampler: Total Throughput Sampler
 


Test patch:
diff --git a/route/route_test.go b/route/route_test.go
index 85d11d6..ad6d700 100644
--- a/route/route_test.go
+++ b/route/route_test.go
@@ -328,7 +328,7 @@ func TestDebugAllRules(t *testing.T) {
 	}{
 		{
 			format: "json",
-			expect: `{"rulesversion":0,"samplers":{"dataset1":{"deterministicsampler":{"samplerate":0},"rulesbasedsampler":null,"dynamicsampler":null,"emadynamicsampler":null,"emathroughputsampler":null,"totalthroughputsampler":null}}}`,
+			expect: `{"rulesversion":0,"samplers":{"dataset1":{"deterministicsampler":{"samplerate":0},"rulesbasedsampler":null,"dynamicsampler":null,"emadynamicsampler":null,"emathroughputsampler":null,"windowedthroughputsampler":null,"totalthroughputsampler":null}}}`,
 		},
 		{
 			format: "toml",
diff --git a/sample/windowed_throughput_test.go b/sample/windowed_throughput_test.go
new file mode 100644
index 0000000..9fb7d98
--- /dev/null
+++ b/sample/windowed_throughput_test.go
@@ -0,0 +1,51 @@
+package sample
+
+import (
+	"testing"
+
+	"github.com/honeycombio/refinery/config"
+	"github.com/honeycombio/refinery/logger"
+	"github.com/honeycombio/refinery/metrics"
+	"github.com/honeycombio/refinery/types"
+
+	"github.com/stretchr/testify/assert"
+)
+
+func TestWindowedThroughputAddSampleRateKeyToTrace(t *testing.T) {
+	const spanCount = 5
+
+	metrics := metrics.MockMetrics{}
+	metrics.Start()
+
+	sampler := &WindowedThroughputSampler{
+		Config: &config.WindowedThroughputSamplerConfig{
+			FieldList: []string{"http.status_code", "request.path", "app.team.id", "important_field"},
+		},
+		Logger:  &logger.NullLogger{},
+		Metrics: &metrics,
+	}
+
+	trace := &types.Trace{}
+	for i := 0; i < spanCount; i++ {
+		trace.AddSpan(&types.Span{
+			Event: types.Event{
+				Data: map[string]interface{}{
+					"http.status_code": 200,
+					"app.team.id":      float64(4),
+					"important_field":  true,
+					"request.path":     "/{slug}/fun",
+				},
+			},
+		})
+	}
+	sampler.Start()
+	rate, _, reason, key := sampler.GetSampleRate(trace)
+
+	spans := trace.GetSpans()
+
+	assert.Len(t, spans, spanCount, "should have the same number of spans as input")
+	assert.Equal(t, uint(1), rate, "sample rate should be 1")
+	assert.Equal(t, "Windowedthroughput", reason)
+	assert.Equal(t, "4•,200•,true•,/{slug}/fun•,", key)
+
+}