Repository: statsig-io/docs
Pull request #2854: geotest starter docs
Author: login: makris-statsig
Created: 2025-04-09T18:46:29Z | Merged: 2025-04-09T20:03:33Z

## Description

Added documentation for Geotests, a new experimental framework that uses synthetic controls for geographic-based testing. The documentation includes:

1. An introduction page explaining:
   - How geotests differ from standard A/B testing
   - When to use synthetic controls
   - The approach to causal inference modeling
   - Information about GeoLift, the open-source package used by Statsig

2. A getting started guide with screenshots showing:
   - How to create and configure geotests
   - How to evaluate design options
   - How to run analysis

Added supporting images showing example data, incremental effects, and UI elements of the feature.

## Best practice checklist

- [ ] I've considered the best practices on [where to put your doc](https://www.notion.so/statsig/Statsig-Docs-Contribution-Guide-1c4d27b5090d8093856ff2d94f170a9c?pvs=4#1c4d27b5090d80f7b1f0e14e93af4eb5) and [what to put in your doc](https://www.notion.so/statsig/Statsig-Docs-Contribution-Guide-1c4d27b5090d8093856ff2d94f170a9c?pvs=4#1c4d27b5090d805bb4d0d6b4b8f06fa6)
- [ ] I've deleted and redirected old pages to this one, if any
- [ ] I've updated links affected by this change, if any
- [ ] I've updated screenshots affected by this change, if any

## Questions?

Reach out to Brock or Tore on Slack!


Diff:
diff --git a/cspell.json b/cspell.json
index 345bc6ba3..390d7a8ac 100644
--- a/cspell.json
+++ b/cspell.json
@@ -659,7 +659,11 @@
     "akin",
     "vineeth",
     "vijaye",
-    "brock"
+    "brock",
+    "makris",
+    "geotest",
+    "geotests",
+    "geotesting"
   ],
   "ignorePaths": [
     "node_modules/**",
diff --git a/docs/statsig-warehouse-native/geotests/geotests-setup.md b/docs/statsig-warehouse-native/geotests/geotests-setup.md
new file mode 100644
index 000000000..e111eb237
--- /dev/null
+++ b/docs/statsig-warehouse-native/geotests/geotests-setup.md
@@ -0,0 +1,31 @@
+---
+title: Get Started With Geotests
+sidebar_label: Getting Started
+slug: /experiments-plus/geotests-setup
+keywords:
+  - owner:makris
+last_update:
+  date: 2025-04-09
+---
+
+# Create and Configure
+
+![Image](/img/geotests/CreateXP.png)
+
+![Image](/img/geotests/SetType.png)
+
+![Image](/img/geotests/SetupXP.png)
+
+![Image](/img/geotests/XPConfigs.png)
+
+![Image](/img/geotests/LimitGeos.png)
+
+# Evaluate Design Options
+
+![Image](/img/geotests/DesignerOptions.png)
+
+![Image](/img/geotests/DesignOption.png)
+
+# Run Analysis
+
+![Image](/img/geotests/AnalysisResults.png)
diff --git a/docs/statsig-warehouse-native/geotests/introduction.md b/docs/statsig-warehouse-native/geotests/introduction.md
new file mode 100644
index 000000000..520f11533
--- /dev/null
+++ b/docs/statsig-warehouse-native/geotests/introduction.md
@@ -0,0 +1,99 @@
+---
+title: Geotests
+sidebar_label: Geotests
+slug: /experiments-plus/geotests
+keywords:
+  - owner:makris
+last_update:
+  date: 2025-04-09
+---
+
+# Geotests
+
+Geotests are specialized experiments that allow you to target users based on their geographic location. They enable you to test features, content, or experiences across different regions, countries, or specific geographic areas to understand how location impacts user behavior and preferences. Geotests are particularly valuable for businesses with global audiences who need to optimize their product for regional differences in user needs, preferences, or regulatory requirements.
+
+## How it's different
+
+Standard A/B testing relies on a few core assumptions:
+
+1. You can reliably and randomly split your users into similar groups
+2. While each group has heterogeneity inside it, random user-level variations average out at larger sample sizes, resulting in homologous (similar) groups
+3. You control how each group gets treated, and there is no interference between groups
+
+When these core assumptions are true, we assume the treatment effect will be the only observable difference between the groups. We can simply measure each group’s mean value and find the difference between them.
+
+There are times however when these assumptions don’t hold:
+
+- Your intended treatment can’t be controlled and scoped to the user level
+- You don’t know who the users are
+- You can’t track outcomes at the user level
+
+Running geographically-based campaigns is a prime example that often features at least some of these violations. Examples include:
+
+- Some classic marketing, like billboards, can’t be randomly split between test and control users at a given location. We can’t track who sees them and how they act afterwards.
+- Some digital campaigns might be able to split users 50/50 (if the platform allows it), but if you don't have user-level data that you can tie to outcomes of interest, you'll never know how the two groups performed.
+
+## Using Synthetic Controls
+
+“Geotesting” as we refer to it here is an experimental framework that relies on a different basic setup than AB tests:
+
+1. Split your users into geographical boundaries at some useful level (zip codes, states, countries, etc)
+2. Apply a known treatment in some “treatment” geos, and no treatment in some “control” geos
+3. Use the control geos to figure out what would have happened in the treatment geos had no treatment been applied (i.e. “synthetic” results)
+4. Measure the delta in the treatment geos between the actual and synthetic values
+
+The “figuring out what would have happened” tends to be the hardest and nastiest part of this setup. Deducing things that don’t occur is usually a fool’s errand, but in some circumstances there’s enough predictability in a system to figure it out.
+
+Geotesting relies on the idea that among some sets of geos there are enough correlations between control and test geos that make predicting this “counterfactual” possible. We assume, for instance, that general trends and seasonality tend to be shared across some geos.
+
+Example: pretty much all zip codes in the United States exhibit similar quantities of mail sent on any given day. If you were to split zip codes 50/50, you could reasonably infer that the two groups would see similar trends.
+
+# Approach
+
+There are many ways to estimate unknown data from other known data, depending on a multitude of factors. In the case of Geotesting, we have:
+
+- timeseries data that is autocorrelated (i.e. data is correlated from one day to the next within a group)
+- timeseries data that is correlated between groups
+- regular seasonal patterns that repeat at regular intervals
+- granular data from among many distinct geographies
+
+Plotting some metric of interest for a bunch of relevant geographies can show these relationships clearly.
+
+![Example data from the GeoLift package walkthrough. ](/img/geotests/GeosPreTest.png)
+
+Example data from the GeoLift package walkthrough. 
+
+But what happens when we run a campaign or make some change in just some of these geos? How can we tell if a fluctuation in this graph is random noise or real stat-sig result? 
+
+![A marketing campaign that starts on day 91 (black dotted line) could have affected the treatment cities of Chicago and Portland. But how can you tell any real effect from all the noise?](/img/geotests/GeosTest.png)
+
+A marketing campaign that starts on day 91 (black dotted line) could have affected the treatment cities of Chicago and Portland. But how do can you tell any real effect from all the noise?
+
+We turn to causal inference modeling to answer this question. Thankfully, a variety of powerful models and methods have been developed to answer this question. The basic steps are the following:
+
+- Split your geos into test and control groups
+- Using the pre-treatment period data, train a model on the control data to predict how the treatment geos behave
+- Use the post-treatment period data for the control geos, create a “synthetic control” dataset to predict how the treatment geos would have behaved after the treatment event.
+- Subtract the synthetic data from the actual observed data to determine the induced effects of your treatment on the treatment geos
+
+When your training data is able to produce strong models that can predict metric outcome well, you end up with strong estimates of an induced effect:
+
+![Image](/img/geotests/Period_Split.png)
+
+![Subtracting the modeled Synthetic Control values from the observed Treatment values reveals any incremental effect for the Treatment geos.](/img/geotests/Incremental_Effects.png)
+
+Subtracting the modeled Synthetic Control values from the observed Treatment values reveals any incremental effect for the Treatment geos.
+
+# GeoLift
+
+[GeoLift](https://facebookincubator.github.io/GeoLift/) is an open-source package from Meta that’s used by many to infer causal relationships in timeseries data. It builds on advancements made in prior packages like Google’s [Causal Impact](https://google.github.io/CausalImpact/CausalImpact.html) package.
+
+Statsig is using GeoLift in our own implementation of Geotesting, and allowing you to easily connect your own internal metrics and feed them into the analysis package.
+
+# Notes
+
+TBD
+
+# References
+
+TBD
diff --git a/static/img/geotests/AnalysisResults.png b/static/img/geotests/AnalysisResults.png
new file mode 100644
index 000000000..59d740a60
Binary files /dev/null and b/static/img/geotests/AnalysisResults.png differ
diff --git a/static/img/geotests/CreateXP.png b/static/img/geotests/CreateXP.png
new file mode 100644
index 000000000..68b9fe8a3
Binary files /dev/null and b/static/img/geotests/CreateXP.png differ
diff --git a/static/img/geotests/DesignOption.png b/static/img/geotests/DesignOption.png
new file mode 100644
index 000000000..502919cd6
Binary files /dev/null and b/static/img/geotests/DesignOption.png differ
diff --git a/static/img/geotests/DesignerOptions.png b/static/img/geotests/DesignerOptions.png
new file mode 100644
index 000000000..27f3454dd
Binary files /dev/null and b/static/img/geotests/DesignerOptions.png differ
diff --git a/static/img/geotests/GeoSelection.png b/static/img/geotests/GeoSelection.png
new file mode 100644
index 000000000..2182b3213
Binary files /dev/null and b/static/img/geotests/GeoSelection.png differ
diff --git a/static/img/geotests/GeosPreTest.png b/static/img/geotests/GeosPreTest.png
new file mode 100644
index 000000000..e36026bc6
Binary files /dev/null and b/static/img/geotests/GeosPreTest.png differ
diff --git a/static/img/geotests/GeosTest.png b/static/img/geotests/GeosTest.png
new file mode 100644
index 000000000..378bc218b
Binary files /dev/null and b/static/img/geotests/GeosTest.png differ
diff --git a/static/img/geotests/Incremental_Effects.png b/static/img/geotests/Incremental_Effects.png
new file mode 100644
index 000000000..344cbe177
Binary files /dev/null and b/static/img/geotests/Incremental_Effects.png differ
diff --git a/static/img/geotests/LimitGeos.png b/static/img/geotests/LimitGeos.png
new file mode 100644
index 000000000..074c55757
Binary files /dev/null and b/static/img/geotests/LimitGeos.png differ
diff --git a/static/img/geotests/Period_Split.png b/static/img/geotests/Period_Split.png
new file mode 100644
index 000000000..cf0e2c1ab
Binary files /dev/null and b/static/img/geotests/Period_Split.png differ
diff --git a/static/img/geotests/Prediction.png b/static/img/geotests/Prediction.png
new file mode 100644
index 000000000..3dcac857f
Binary files /dev/null and b/static/img/geotests/Prediction.png differ
diff --git a/static/img/geotests/SetType.png b/static/img/geotests/SetType.png
new file mode 100644
index 000000000..c621d61c6
Binary files /dev/null and b/static/img/geotests/SetType.png differ
diff --git a/static/img/geotests/SetupXP.png b/static/img/geotests/SetupXP.png
new file mode 100644
index 000000000..e41d48e6a
Binary files /dev/null and b/static/img/geotests/SetupXP.png differ
diff --git a/static/img/geotests/XPConfigs.png b/static/img/geotests/XPConfigs.png
new file mode 100644
index 000000000..fd1d86227
Binary files /dev/null and b/static/img/geotests/XPConfigs.png differ


Reviews:
user: login: graphite-app[bot]
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-09T18:58:55Z
user: login: graphite-app[bot]
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-09T19:29:04Z
user: login: graphite-app[bot]
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-09T19:30:22Z
user: login: graphite-app[bot]
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-09T19:31:28Z
user: login: graphite-app[bot]
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-09T19:53:49Z

Review comments:
diff_hunk: @@ -0,0 +1,99 @@
+---
+title: Geotests
+sidebar_label: Geotests
+slug: /experiments-plus/geotests
+keywords:
+  - owner:makris
+last_update:
+  date: 2025-04-09
+---
+
+# Geotests
+
+TBD
+
+## How it's different
+
+Standard A/B testing rely on a few core assumptions:
+
+1. You can reliably and randomly split your users into similar groups
+2. While each group has heterogeneity inside it, random user-level variations average out at larger sample sizes, resulting in homologous (similar) groups
+3. You control how each group gets treated, and there is no interference between groups
+
+When these core assumptions are true, we assume the treatment effect will be the only observable difference between the groups. We can simply measure each group’s mean value and find the difference between them.
+
+There are times however when these assumptions don’t hold:
+
+- Your intended treatment can’t be controlled and scoped to the user level
+- You don’t know who the users are
+- You can’t track outcomes at the user level
+
+Running geographically-based campaigns is a prime example that often features at least some these violations. Examples include:
+
+- Some classic marketing, like billboards, can’t be randomly split between test and control users at a given location. We can’t track who sees them and how they act afterwards.
+- Some digital campaigns might be able to split users 50/50 (if the platform allows it), but if don’t have user-level data that you can tie to outcomes of interest, you’ll never know how the two groups performed.
+
+## Using Synthtic Controls
path: docs/statsig-warehouse-native/geotests/introduction.md
user: login: graphite-app[bot]
body: There's a typo in the section title: "Using Synthtic Controls" should be "Using Synthetic Controls"
```suggestion
## Using Synthetic Controls
```
  

*Spotted by [Diamond](https://app.graphite.dev/diamond/?org=statsig-io&ref=ai-review-comment)*<i class='graphite__hidden'><br /><br />Is this helpful? React 👍 or 👎 to let us know.</i>
created_at: 2025-04-09T18:58:44Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,99 @@
+---
+title: Geotests
+sidebar_label: Geotests
+slug: /experiments-plus/geotests
+keywords:
+  - owner:makris
+last_update:
+  date: 2025-04-09
+---
+
+# Geotests
+
+TBD
+
+## How it's different
+
+Standard A/B testing rely on a few core assumptions:
+
+1. You can reliably and randomly split your users into similar groups
+2. While each group has heterogeneity inside it, random user-level variations average out at larger sample sizes, resulting in homologous (similar) groups
+3. You control how each group gets treated, and there is no interference between groups
+
+When these core assumptions are true, we assume the treatment effect will be the only observable difference between the groups. We can simply measure each group’s mean value and find the difference between them.
+
+There are times however when these assumptions don’t hold:
+
+- Your intended treatment can’t be controlled and scoped to the user level
+- You don’t know who the users are
+- You can’t track outcomes at the user level
+
+Running geographically-based campaigns is a prime example that often features at least some these violations. Examples include:
path: docs/statsig-warehouse-native/geotests/introduction.md
user: login: graphite-app[bot]
body: There appears to be a missing word in this sentence. It should read "at least some **of** these violations" to be grammatically correct.
```suggestion
Running geographically-based campaigns is a prime example that often features at least some of these violations. Examples include:
```
  

*Spotted by [Diamond](https://app.graphite.dev/diamond/?org=statsig-io&ref=ai-review-comment)*<i class='graphite__hidden'><br /><br />Is this helpful? React 👍 or 👎 to let us know.</i>
created_at: 2025-04-09T18:58:46Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,99 @@
+---
+title: Geotests
+sidebar_label: Geotests
+slug: /experiments-plus/geotests
+keywords:
+  - owner:makris
+last_update:
+  date: 2025-04-09
+---
+
+# Geotests
+
+TBD
+
+## How it's different
+
+Standard A/B testing rely on a few core assumptions:
+
+1. You can reliably and randomly split your users into similar groups
+2. While each group has heterogeneity inside it, random user-level variations average out at larger sample sizes, resulting in homologous (similar) groups
+3. You control how each group gets treated, and there is no interference between groups
+
+When these core assumptions are true, we assume the treatment effect will be the only observable difference between the groups. We can simply measure each group’s mean value and find the difference between them.
+
+There are times however when these assumptions don’t hold:
+
+- Your intended treatment can’t be controlled and scoped to the user level
+- You don’t know who the users are
+- You can’t track outcomes at the user level
+
+Running geographically-based campaigns is a prime example that often features at least some these violations. Examples include:
+
+- Some classic marketing, like billboards, can’t be randomly split between test and control users at a given location. We can’t track who sees them and how they act afterwards.
+- Some digital campaigns might be able to split users 50/50 (if the platform allows it), but if don’t have user-level data that you can tie to outcomes of interest, you’ll never know how the two groups performed.
path: docs/statsig-warehouse-native/geotests/introduction.md
user: login: graphite-app[bot]
body: There appears to be a missing word in this sentence. It should read "but if **you** don't have user-level data" to maintain grammatical correctness.
```suggestion
- Some digital campaigns might be able to split users 50/50 (if the platform allows it), but if you don't have user-level data that you can tie to outcomes of interest, you'll never know how the two groups performed.
```
  

*Spotted by [Diamond](https://app.graphite.dev/diamond/?org=statsig-io&ref=ai-review-comment)*<i class='graphite__hidden'><br /><br />Is this helpful? React 👍 or 👎 to let us know.</i>
created_at: 2025-04-09T18:58:47Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,99 @@
+---
+title: Geotests
+sidebar_label: Geotests
+slug: /experiments-plus/geotests
+keywords:
+  - owner:makris
+last_update:
+  date: 2025-04-09
+---
+
+# Geotests
+
+TBD
+
+## How it's different
+
+Standard A/B testing rely on a few core assumptions:
+
+1. You can reliably and randomly split your users into similar groups
+2. While each group has heterogeneity inside it, random user-level variations average out at larger sample sizes, resulting in homologous (similar) groups
+3. You control how each group gets treated, and there is no interference between groups
+
+When these core assumptions are true, we assume the treatment effect will be the only observable difference between the groups. We can simply measure each group’s mean value and find the difference between them.
+
+There are times however when these assumptions don’t hold:
+
+- Your intended treatment can’t be controlled and scoped to the user level
+- You don’t know who the users are
+- You can’t track outcomes at the user level
+
+Running geographically-based campaigns is a prime example that often features at least some these violations. Examples include:
+
+- Some classic marketing, like billboards, can’t be randomly split between test and control users at a given location. We can’t track who sees them and how they act afterwards.
+- Some digital campaigns might be able to split users 50/50 (if the platform allows it), but if don’t have user-level data that you can tie to outcomes of interest, you’ll never know how the two groups performed.
+
+## Using Synthtic Controls
+
+“Geotesting” as we refer to it here is an experimental framework that relies on a different basic setup than AB tests:
+
+1. Split your users into geographical boundaries at some useful level (zip codes, states, countries, etc)
+2. Apply a known treatment in some “treatment” geos, and no treatment in some “control” geos
+3. Use the control geos to figure out what would have happened in the treatment geos had no treatment been applied (i.e. “synthetic” results)
+4. Measure the delta in the treatment geos between the actual and synthetic values
+
+The “figuring out what would have happened” tends to be the hardest and nastiest part of this setup. Deducing things that don’t occur is usually a fool’s errand, but in some circumstances there’s enough predictability in a system to figure it out.
+
+Geotesting relies on the idea that among some sets of geos there are enough correlations between control and test geos that make predicting this “counterfactual” possible. We assume, for instance, that general trends and seasonality tend to be shared across some geos.
+
+Example: pretty much all zip codes in the United States exhibit similar quantities of mail sent on any given day. If you were to split zip codes 50/50, you could reasonably infer that the two groups would see similar trends.
+
+# Approach
+
+There are many ways to estimate unknown data from other known data, depending on a multitude of factors. In the case of Geotesting, we have:
+
+- timeseries data that is autocorrelated (i.e. data is correlated from one day to the next within a group)
+- timeseries data that is correlated between groups
+- regular seasonal patterns that repeat at regular intervals
+- granular data from among many distinct geographies
+
+Plotting some metric of interest for a bunch of relevant geographies can show these relationships clearly.
+
+![Example data from the GeoLift package walkthrough. ](/static/img/geotests/GeosPreTest.png)
+
+Example data from the GeoLift package walkthrough. 
+
+But what happens we we run a campaign or make some change in just some of these geos? How can we tell if a fluctuation in this graph is random noise or real stat-sig result? 
path: docs/statsig-warehouse-native/geotests/introduction.md
user: login: graphite-app[bot]
body: There's a small typo in this sentence: "But what happens **we we** run" should be "But what happens **when we** run" to improve readability.
```suggestion
But what happens when we run a campaign or make some change in just some of these geos? How can we tell if a fluctuation in this graph is random noise or real stat-sig result? 
```
  

*Spotted by [Diamond](https://app.graphite.dev/diamond/?org=statsig-io&ref=ai-review-comment)*<i class='graphite__hidden'><br /><br />Is this helpful? React 👍 or 👎 to let us know.</i>
created_at: 2025-04-09T18:58:49Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,99 @@
+---
+title: Geotests
+sidebar_label: Geotests
+slug: /experiments-plus/geotests
+keywords:
+  - owner:makris
+last_update:
+  date: 2025-04-09
+---
+
+# Geotests
+
+TBD
+
+## How it's different
+
+Standard A/B testing rely on a few core assumptions:
+
+1. You can reliably and randomly split your users into similar groups
+2. While each group has heterogeneity inside it, random user-level variations average out at larger sample sizes, resulting in homologous (similar) groups
+3. You control how each group gets treated, and there is no interference between groups
+
+When these core assumptions are true, we assume the treatment effect will be the only observable difference between the groups. We can simply measure each group’s mean value and find the difference between them.
+
+There are times however when these assumptions don’t hold:
+
+- Your intended treatment can’t be controlled and scoped to the user level
+- You don’t know who the users are
+- You can’t track outcomes at the user level
+
+Running geographically-based campaigns is a prime example that often features at least some these violations. Examples include:
+
+- Some classic marketing, like billboards, can’t be randomly split between test and control users at a given location. We can’t track who sees them and how they act afterwards.
+- Some digital campaigns might be able to split users 50/50 (if the platform allows it), but if don’t have user-level data that you can tie to outcomes of interest, you’ll never know how the two groups performed.
+
+## Using Synthtic Controls
+
+“Geotesting” as we refer to it here is an experimental framework that relies on a different basic setup than AB tests:
+
+1. Split your users into geographical boundaries at some useful level (zip codes, states, countries, etc)
+2. Apply a known treatment in some “treatment” geos, and no treatment in some “control” geos
+3. Use the control geos to figure out what would have happened in the treatment geos had no treatment been applied (i.e. “synthetic” results)
+4. Measure the delta in the treatment geos between the actual and synthetic values
+
+The “figuring out what would have happened” tends to be the hardest and nastiest part of this setup. Deducing things that don’t occur is usually a fool’s errand, but in some circumstances there’s enough predictability in a system to figure it out.
+
+Geotesting relies on the idea that among some sets of geos there are enough correlations between control and test geos that make predicting this “counterfactual” possible. We assume, for instance, that general trends and seasonality tend to be shared across some geos.
+
+Example: pretty much all zip codes in the United States exhibit similar quantities of mail sent on any given day. If you were to split zip codes 50/50, you could reasonably infer that the two groups would see similar trends.
+
+# Approach
+
+There are many ways to estimate unknown data from other known data, depending on a multitude of factors. In the case of Geotesting, we have:
+
+- timeseries data that is autocorrelated (i.e. data is correlated from one day to the next within a group)
+- timeseries data that is correlated between groups
+- regular seasonal patterns that repeat at regular intervals
+- granular data from among many distinct geographies
+
+Plotting some metric of interest for a bunch of relevant geographies can show these relationships clearly.
+
+![Example data from the GeoLift package walkthrough. ](/static/img/geotests/GeosPreTest.png)
+
+Example data from the GeoLift package walkthrough. 
+
+But what happens we we run a campaign or make some change in just some of these geos? How can we tell if a fluctuation in this graph is random noise or real stat-sig result? 
+
+![A marketing campaign that starts on day 91 (black dotted line) could have affected the treatment cities of Chicago and Portland. But how do can you tell any real effect from all the noise?](/static/img/geotests/GeosTest.png)
path: docs/statsig-warehouse-native/geotests/introduction.md
user: login: graphite-app[bot]
body: There's a small grammatical error in this image caption: "But how do can you tell any real effect" should be "But how can you tell any real effect" - removing the word "do" would fix the sentence structure.
```suggestion
![A marketing campaign that starts on day 91 (black dotted line) could have affected the treatment cities of Chicago and Portland. But how can you tell any real effect from all the noise?](/static/img/geotests/GeosTest.png)
```
  

*Spotted by [Diamond](https://app.graphite.dev/diamond/?org=statsig-io&ref=ai-review-comment)*<i class='graphite__hidden'><br /><br />Is this helpful? React 👍 or 👎 to let us know.</i>
created_at: 2025-04-09T18:58:50Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,99 @@
+---
+title: Geotests
+sidebar_label: Geotests
+slug: /experiments-plus/geotests
+keywords:
+  - owner:makris
+last_update:
+  date: 2025-04-09
+---
+
+# Geotests
+
+TBD
+
+## How it's different
+
+Standard A/B testing rely on a few core assumptions:
+
+1. You can reliably and randomly split your users into similar groups
+2. While each group has heterogeneity inside it, random user-level variations average out at larger sample sizes, resulting in homologous (similar) groups
+3. You control how each group gets treated, and there is no interference between groups
+
+When these core assumptions are true, we assume the treatment effect will be the only observable difference between the groups. We can simply measure each group’s mean value and find the difference between them.
+
+There are times however when these assumptions don’t hold:
+
+- Your intended treatment can’t be controlled and scoped to the user level
+- You don’t know who the users are
+- You can’t track outcomes at the user level
+
+Running geographically-based campaigns is a prime example that often features at least some these violations. Examples include:
+
+- Some classic marketing, like billboards, can’t be randomly split between test and control users at a given location. We can’t track who sees them and how they act afterwards.
+- Some digital campaigns might be able to split users 50/50 (if the platform allows it), but if don’t have user-level data that you can tie to outcomes of interest, you’ll never know how the two groups performed.
+
+## Using Synthtic Controls
+
+“Geotesting” as we refer to it here is an experimental framework that relies on a different basic setup than AB tests:
+
+1. Split your users into geographical boundaries at some useful level (zip codes, states, countries, etc)
+2. Apply a known treatment in some “treatment” geos, and no treatment in some “control” geos
+3. Use the control geos to figure out what would have happened in the treatment geos had no treatment been applied (i.e. “synthetic” results)
+4. Measure the delta in the treatment geos between the actual and synthetic values
+
+The “figuring out what would have happened” tends to be the hardest and nastiest part of this setup. Deducing things that don’t occur is usually a fool’s errand, but in some circumstances there’s enough predictability in a system to figure it out.
+
+Geotesting relies on the idea that among some sets of geos there are enough correlations between control and test geos that make predicting this “counterfactual” possible. We assume, for instance, that general trends and seasonality tend to be shared across some geos.
+
+Example: pretty much all zip codes in the United States exhibit similar quantities of mail sent on any given day. If you were to split zip codes 50/50, you could reasonably infer that the two groups would see similar trends.
+
+# Approach
+
+There are many ways to estimate unknown data from other known data, depending on a multitude of factors. In the case of Geotesting, we have:
+
+- timeseries data that is autocorrelated (i.e. data is correlated from one day to the next within a group)
+- timeseries data that is correlated between groups
+- regular seasonal patterns that repeat at regular intervals
+- granular data from among many distinct geographies
+
+Plotting some metric of interest for a bunch of relevant geographies can show these relationships clearly.
+
+![Example data from the GeoLift package walkthrough. ](/static/img/geotests/GeosPreTest.png)
+
+Example data from the GeoLift package walkthrough. 
+
+But what happens we we run a campaign or make some change in just some of these geos? How can we tell if a fluctuation in this graph is random noise or real stat-sig result? 
+
+![A marketing campaign that starts on day 91 (black dotted line) could have affected the treatment cities of Chicago and Portland. But how do can you tell any real effect from all the noise?](/static/img/geotests/GeosTest.png)
+
+A marketing campaign that starts on day 91 (black dotted line) could have affected the treatment cities of Chicago and Portland. But how do can you tell any real effect from all the noise?
+
+We turn to causal inference modeling to answer this question. Thankfully, a variety of powerful models and methods have been developed answer this question. The basic steps are the following:
path: docs/statsig-warehouse-native/geotests/introduction.md
user: login: graphite-app[bot]
body: There's a missing word in this sentence: "have been developed answer this question" should be "have been developed **to** answer this question"
```suggestion
We turn to causal inference modeling to answer this question. Thankfully, a variety of powerful models and methods have been developed to answer this question. The basic steps are the following:
```
  

*Spotted by [Diamond](https://app.graphite.dev/diamond/?org=statsig-io&ref=ai-review-comment)*<i class='graphite__hidden'><br /><br />Is this helpful? React 👍 or 👎 to let us know.</i>
created_at: 2025-04-09T18:58:52Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,99 @@
+---
+title: Geotests
+sidebar_label: Geotests
+slug: /experiments-plus/geotests
+keywords:
+  - owner:makris
+last_update:
+  date: 2025-04-09
+---
+
+# Geotests
+
+TBD
+
+## How it's different
+
+Standard A/B testing rely on a few core assumptions:
+
+1. You can reliably and randomly split your users into similar groups
+2. While each group has heterogeneity inside it, random user-level variations average out at larger sample sizes, resulting in homologous (similar) groups
+3. You control how each group gets treated, and there is no interference between groups
+
+When these core assumptions are true, we assume the treatment effect will be the only observable difference between the groups. We can simply measure each group’s mean value and find the difference between them.
+
+There are times however when these assumptions don’t hold:
+
+- Your intended treatment can’t be controlled and scoped to the user level
+- You don’t know who the users are
+- You can’t track outcomes at the user level
+
+Running geographically-based campaigns is a prime example that often features at least some these violations. Examples include:
+
+- Some classic marketing, like billboards, can’t be randomly split between test and control users at a given location. We can’t track who sees them and how they act afterwards.
+- Some digital campaigns might be able to split users 50/50 (if the platform allows it), but if don’t have user-level data that you can tie to outcomes of interest, you’ll never know how the two groups performed.
+
+## Using Synthtic Controls
+
+“Geotesting” as we refer to it here is an experimental framework that relies on a different basic setup than AB tests:
+
+1. Split your users into geographical boundaries at some useful level (zip codes, states, countries, etc)
+2. Apply a known treatment in some “treatment” geos, and no treatment in some “control” geos
+3. Use the control geos to figure out what would have happened in the treatment geos had no treatment been applied (i.e. “synthetic” results)
+4. Measure the delta in the treatment geos between the actual and synthetic values
+
+The “figuring out what would have happened” tends to be the hardest and nastiest part of this setup. Deducing things that don’t occur is usually a fool’s errand, but in some circumstances there’s enough predictability in a system to figure it out.
+
+Geotesting relies on the idea that among some sets of geos there are enough correlations between control and test geos that make predicting this “counterfactual” possible. We assume, for instance, that general trends and seasonality tend to be shared across some geos.
+
+Example: pretty much all zip codes in the United States exhibit similar quantities of mail sent on any given day. If you were to split zip codes 50/50, you could reasonably infer that the two groups would see similar trends.
+
+# Approach
+
+There are many ways to estimate unknown data from other known data, depending on a multitude of factors. In the case of Geotesting, we have:
+
+- timeseries data that is autocorrelated (i.e. data is correlated from one day to the next within a group)
+- timeseries data that is correlated between groups
+- regular seasonal patterns that repeat at regular intervals
+- granular data from among many distinct geographies
+
+Plotting some metric of interest for a bunch of relevant geographies can show these relationships clearly.
+
+![Example data from the GeoLift package walkthrough. ](/static/img/geotests/GeosPreTest.png)
+
+Example data from the GeoLift package walkthrough. 
+
+But what happens we we run a campaign or make some change in just some of these geos? How can we tell if a fluctuation in this graph is random noise or real stat-sig result? 
+
+![A marketing campaign that starts on day 91 (black dotted line) could have affected the treatment cities of Chicago and Portland. But how do can you tell any real effect from all the noise?](/static/img/geotests/GeosTest.png)
+
+A marketing campaign that starts on day 91 (black dotted line) could have affected the treatment cities of Chicago and Portland. But how do can you tell any real effect from all the noise?
+
+We turn to causal inference modeling to answer this question. Thankfully, a variety of powerful models and methods have been developed answer this question. The basic steps are the following:
+
+- Split your geos into test and control groups
+- Using the pre-treatment period data, train a model on the control data to predict how the treatment geos behave
+- Use the post-treatment period data for the control geos, create a “synthetic control” dataset to predict how the treatment geos would have behaved after the treatment event.
+- Subtract the synthetic data from the actual observed data to determine the induced effects of your treatment on the treatment geos
+
+When your training data is able to produce strong models that can predict metric outcome well, you end up with strong estimates of an induced effect:
+
+![Image](/static/img/geotests/Period_Split.png)
+
+![Subtracting the modeled Synthetic Control values from the observed Treatment values reveals any incrementally for the Treatment geos.](/static/img/geotests/Incremental_Effects.png)
+
+Subtracting the modeled Synthetic Control values from the observed Treatment values reveals any incrementally for the Treatment geos.
path: docs/statsig-warehouse-native/geotests/introduction.md
user: login: graphite-app[bot]
body: There appears to be a typo in this caption. Consider changing "reveals any incrementally for the Treatment geos" to "reveals any incremental effect for the Treatment geos" to improve clarity and grammatical correctness.
```suggestion
Subtracting the modeled Synthetic Control values from the observed Treatment values reveals any incremental effect for the Treatment geos.
```
  

*Spotted by [Diamond](https://app.graphite.dev/diamond/?org=statsig-io&ref=ai-review-comment)*<i class='graphite__hidden'><br /><br />Is this helpful? React 👍 or 👎 to let us know.</i>
created_at: 2025-04-09T18:58:53Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,99 @@
+---
+title: Geotests
+sidebar_label: Geotests
+slug: /experiments-plus/geotests
+keywords:
+  - owner:makris
+last_update:
+  date: 2025-04-09
+---
+
+# Geotests
+
+TBD
+
+## How it's different
+
+Standard A/B testing rely on a few core assumptions:
+
+1. You can reliably and randomly split your users into similar groups
+2. While each group has heterogeneity inside it, random user-level variations average out at larger sample sizes, resulting in homologous (similar) groups
+3. You control how each group gets treated, and there is no interference between groups
+
+When these core assumptions are true, we assume the treatment effect will be the only observable difference between the groups. We can simply measure each group’s mean value and find the difference between them.
+
+There are times however when these assumptions don’t hold:
+
+- Your intended treatment can’t be controlled and scoped to the user level
+- You don’t know who the users are
+- You can’t track outcomes at the user level
+
+Running geographically-based campaigns is a prime example that often features at least some of these violations. Examples include:
+
+- Some classic marketing, like billboards, can’t be randomly split between test and control users at a given location. We can’t track who sees them and how they act afterwards.
+- Some digital campaigns might be able to split users 50/50 (if the platform allows it), but if you don't have user-level data that you can tie to outcomes of interest, you'll never know how the two groups performed.
+
+## Using Synthetic Controls
+
+“Geotesting” as we refer to it here is an experimental framework that relies on a different basic setup than AB tests:
+
+1. Split your users into geographical boundaries at some useful level (zip codes, states, countries, etc)
+2. Apply a known treatment in some “treatment” geos, and no treatment in some “control” geos
+3. Use the control geos to figure out what would have happened in the treatment geos had no treatment been applied (i.e. “synthetic” results)
+4. Measure the delta in the treatment geos between the actual and synthetic values
+
+The “figuring out what would have happened” tends to be the hardest and nastiest part of this setup. Deducing things that don’t occur is usually a fool’s errand, but in some circumstances there’s enough predictability in a system to figure it out.
+
+Geotesting relies on the idea that among some sets of geos there are enough correlations between control and test geos that make predicting this “counterfactual” possible. We assume, for instance, that general trends and seasonality tend to be shared across some geos.
+
+Example: pretty much all zip codes in the United States exhibit similar quantities of mail sent on any given day. If you were to split zip codes 50/50, you could reasonably infer that the two groups would see similar trends.
+
+# Approach
+
+There are many ways to estimate unknown data from other known data, depending on a multitude of factors. In the case of Geotesting, we have:
+
+- timeseries data that is autocorrelated (i.e. data is correlated from one day to the next within a group)
+- timeseries data that is correlated between groups
+- regular seasonal patterns that repeat at regular intervals
+- granular data from among many distinct geographies
+
+Plotting some metric of interest for a bunch of relevant geographies can show these relationships clearly.
+
+![Example data from the GeoLift package walkthrough. ](/static/img/geotests/GeosPreTest.png)
+
+Example data from the GeoLift package walkthrough. 
+
+But what happens when we run a campaign or make some change in just some of these geos? How can we tell if a fluctuation in this graph is random noise or real stat-sig result? 
+
+![A marketing campaign that starts on day 91 (black dotted line) could have affected the treatment cities of Chicago and Portland. But how can you tell any real effect from all the noise?](/static/img/geotests/GeosTest.png)
+
+A marketing campaign that starts on day 91 (black dotted line) could have affected the treatment cities of Chicago and Portland. But how do can you tell any real effect from all the noise?
+
+We turn to causal inference modeling to answer this question. Thankfully, a variety of powerful models and methods have been developed to answer this question. The basic steps are the following:
+
+- Split your geos into test and control groups
+- Using the pre-treatment period data, train a model on the control data to predict how the treatment geos behave
+- Use the post-treatment period data for the control geos, create a “synthetic control” dataset to predict how the treatment geos would have behaved after the treatment event.
+- Subtract the synthetic data from the actual observed data to determine the induced effects of your treatment on the treatment geos
+
+When your training data is able to produce strong models that can predict metric outcome well, you end up with strong estimates of an induced effect:
+
+![Image](/static/img/geotests/Period_Split.png)
+
+![Subtracting the modeled Synthetic Control values from the observed Treatment values reveals any incrementally for the Treatment geos.](/static/img/geotests/Incremental_Effects.png)
+
+Subtracting the modeled Synthetic Control values from the observed Treatment values reveals any incremental effect for the Treatment geos.
path: docs/statsig-warehouse-native/geotests/introduction.md
user: login: graphite-app[bot]
body: There appears to be a typo in the image caption. The phrase "reveals any incrementally" should be corrected to "reveals any incremental effect" for better clarity and grammatical accuracy.
```suggestion
![Subtracting the modeled Synthetic Control values from the observed Treatment values reveals any incremental effect for the Treatment geos.](/static/img/geotests/Incremental_Effects.png)

Subtracting the modeled Synthetic Control values from the observed Treatment values reveals any incremental effect for the Treatment geos.
```
  

*Spotted by [Diamond](https://app.graphite.dev/diamond/?org=statsig-io&ref=ai-review-comment)*<i class='graphite__hidden'><br /><br />Is this helpful? React 👍 or 👎 to let us know.</i>
created_at: 2025-04-09T19:28:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,99 @@
+---
+title: Geotests
+sidebar_label: Geotests
+slug: /experiments-plus/geotests
+keywords:
+  - owner:makris
+last_update:
+  date: 2025-04-09
+---
+
+# Geotests
+
+TBD
+
+## How it's different
+
+Standard A/B testing rely on a few core assumptions:
path: docs/statsig-warehouse-native/geotests/introduction.md
user: login: graphite-app[bot]
body: There's a subject-verb agreement issue in this sentence. "Standard A/B testing" is a singular subject and should take the singular verb form "relies" rather than the plural "rely".
```suggestion
Standard A/B testing relies on a few core assumptions:
```
  

*Spotted by [Diamond](https://app.graphite.dev/diamond/?org=statsig-io&ref=ai-review-comment)*<i class='graphite__hidden'><br /><br />Is this helpful? React 👍 or 👎 to let us know.</i>
created_at: 2025-04-09T19:29:02Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,99 @@
+---
+title: Geotests
+sidebar_label: Geotests
+slug: /experiments-plus/geotests
+keywords:
+  - owner:makris
+last_update:
+  date: 2025-04-09
+---
+
+# Geotests
+
+TBD
+
+## How it's different
+
+Standard A/B testing rely on a few core assumptions:
+
+1. You can reliably and randomly split your users into similar groups
+2. While each group has heterogeneity inside it, random user-level variations average out at larger sample sizes, resulting in homologous (similar) groups
+3. You control how each group gets treated, and there is no interference between groups
+
+When these core assumptions are true, we assume the treatment effect will be the only observable difference between the groups. We can simply measure each group’s mean value and find the difference between them.
+
+There are times however when these assumptions don’t hold:
+
+- Your intended treatment can’t be controlled and scoped to the user level
+- You don’t know who the users are
+- You can’t track outcomes at the user level
+
+Running geographically-based campaigns is a prime example that often features at least some of these violations. Examples include:
+
+- Some classic marketing, like billboards, can’t be randomly split between test and control users at a given location. We can’t track who sees them and how they act afterwards.
+- Some digital campaigns might be able to split users 50/50 (if the platform allows it), but if you don't have user-level data that you can tie to outcomes of interest, you'll never know how the two groups performed.
+
+## Using Synthetic Controls
+
+“Geotesting” as we refer to it here is an experimental framework that relies on a different basic setup than AB tests:
+
+1. Split your users into geographical boundaries at some useful level (zip codes, states, countries, etc)
+2. Apply a known treatment in some “treatment” geos, and no treatment in some “control” geos
+3. Use the control geos to figure out what would have happened in the treatment geos had no treatment been applied (i.e. “synthetic” results)
+4. Measure the delta in the treatment geos between the actual and synthetic values
+
+The “figuring out what would have happened” tends to be the hardest and nastiest part of this setup. Deducing things that don’t occur is usually a fool’s errand, but in some circumstances there’s enough predictability in a system to figure it out.
+
+Geotesting relies on the idea that among some sets of geos there are enough correlations between control and test geos that make predicting this “counterfactual” possible. We assume, for instance, that general trends and seasonality tend to be shared across some geos.
+
+Example: pretty much all zip codes in the United States exhibit similar quantities of mail sent on any given day. If you were to split zip codes 50/50, you could reasonably infer that the two groups would see similar trends.
+
+# Approach
+
+There are many ways to estimate unknown data from other known data, depending on a multitude of factors. In the case of Geotesting, we have:
+
+- timeseries data that is autocorrelated (i.e. data is correlated from one day to the next within a group)
+- timeseries data that is correlated between groups
+- regular seasonal patterns that repeat at regular intervals
+- granular data from among many distinct geographies
+
+Plotting some metric of interest for a bunch of relevant geographies can show these relationships clearly.
+
+![Example data from the GeoLift package walkthrough. ](/static/img/geotests/GeosPreTest.png)
+
+Example data from the GeoLift package walkthrough. 
+
+But what happens when we run a campaign or make some change in just some of these geos? How can we tell if a fluctuation in this graph is random noise or real stat-sig result? 
path: docs/statsig-warehouse-native/geotests/introduction.md
user: login: graphite-app[bot]
body: There's a small typo in this sentence: "But what happens **we we** run a campaign..." should be "But what happens **when we** run a campaign..."
```suggestion
But what happens when we run a campaign or make some change in just some of these geos? How can we tell if a fluctuation in this graph is random noise or real stat-sig result? 
```
  

*Spotted by [Diamond](https://app.graphite.dev/diamond/?org=statsig-io&ref=ai-review-comment)*<i class='graphite__hidden'><br /><br />Is this helpful? React 👍 or 👎 to let us know.</i>
created_at: 2025-04-09T19:30:15Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,99 @@
+---
+title: Geotests
+sidebar_label: Geotests
+slug: /experiments-plus/geotests
+keywords:
+  - owner:makris
+last_update:
+  date: 2025-04-09
+---
+
+# Geotests
+
+TBD
+
+## How it's different
+
+Standard A/B testing rely on a few core assumptions:
+
+1. You can reliably and randomly split your users into similar groups
+2. While each group has heterogeneity inside it, random user-level variations average out at larger sample sizes, resulting in homologous (similar) groups
+3. You control how each group gets treated, and there is no interference between groups
+
+When these core assumptions are true, we assume the treatment effect will be the only observable difference between the groups. We can simply measure each group’s mean value and find the difference between them.
+
+There are times however when these assumptions don’t hold:
+
+- Your intended treatment can’t be controlled and scoped to the user level
+- You don’t know who the users are
+- You can’t track outcomes at the user level
+
+Running geographically-based campaigns is a prime example that often features at least some of these violations. Examples include:
+
+- Some classic marketing, like billboards, can’t be randomly split between test and control users at a given location. We can’t track who sees them and how they act afterwards.
+- Some digital campaigns might be able to split users 50/50 (if the platform allows it), but if you don't have user-level data that you can tie to outcomes of interest, you'll never know how the two groups performed.
+
+## Using Synthetic Controls
+
+“Geotesting” as we refer to it here is an experimental framework that relies on a different basic setup than AB tests:
+
+1. Split your users into geographical boundaries at some useful level (zip codes, states, countries, etc)
+2. Apply a known treatment in some “treatment” geos, and no treatment in some “control” geos
+3. Use the control geos to figure out what would have happened in the treatment geos had no treatment been applied (i.e. “synthetic” results)
+4. Measure the delta in the treatment geos between the actual and synthetic values
+
+The “figuring out what would have happened” tends to be the hardest and nastiest part of this setup. Deducing things that don’t occur is usually a fool’s errand, but in some circumstances there’s enough predictability in a system to figure it out.
+
+Geotesting relies on the idea that among some sets of geos there are enough correlations between control and test geos that make predicting this “counterfactual” possible. We assume, for instance, that general trends and seasonality tend to be shared across some geos.
+
+Example: pretty much all zip codes in the United States exhibit similar quantities of mail sent on any given day. If you were to split zip codes 50/50, you could reasonably infer that the two groups would see similar trends.
+
+# Approach
+
+There are many ways to estimate unknown data from other known data, depending on a multitude of factors. In the case of Geotesting, we have:
+
+- timeseries data that is autocorrelated (i.e. data is correlated from one day to the next within a group)
+- timeseries data that is correlated between groups
+- regular seasonal patterns that repeat at regular intervals
+- granular data from among many distinct geographies
+
+Plotting some metric of interest for a bunch of relevant geographies can show these relationships clearly.
+
+![Example data from the GeoLift package walkthrough. ](/static/img/geotests/GeosPreTest.png)
+
+Example data from the GeoLift package walkthrough. 
+
+But what happens when we run a campaign or make some change in just some of these geos? How can we tell if a fluctuation in this graph is random noise or real stat-sig result? 
+
+![A marketing campaign that starts on day 91 (black dotted line) could have affected the treatment cities of Chicago and Portland. But how can you tell any real effect from all the noise?](/static/img/geotests/GeosTest.png)
path: docs/statsig-warehouse-native/geotests/introduction.md
user: login: graphite-app[bot]
body: There appears to be a grammatical error in the image caption. The phrase "But how do can you tell any real effect from all the noise?" contains redundant verbs. It should be corrected to "But how can you tell any real effect from all the noise?"
```suggestion
![A marketing campaign that starts on day 91 (black dotted line) could have affected the treatment cities of Chicago and Portland. But how can you tell any real effect from all the noise?](/static/img/geotests/GeosTest.png)
```
  

*Spotted by [Diamond](https://app.graphite.dev/diamond/?org=statsig-io&ref=ai-review-comment)*<i class='graphite__hidden'><br /><br />Is this helpful? React 👍 or 👎 to let us know.</i>
created_at: 2025-04-09T19:30:16Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,99 @@
+---
+title: Geotests
+sidebar_label: Geotests
+slug: /experiments-plus/geotests
+keywords:
+  - owner:makris
+last_update:
+  date: 2025-04-09
+---
+
+# Geotests
+
+TBD
+
+## How it's different
+
+Standard A/B testing rely on a few core assumptions:
+
+1. You can reliably and randomly split your users into similar groups
+2. While each group has heterogeneity inside it, random user-level variations average out at larger sample sizes, resulting in homologous (similar) groups
+3. You control how each group gets treated, and there is no interference between groups
+
+When these core assumptions are true, we assume the treatment effect will be the only observable difference between the groups. We can simply measure each group’s mean value and find the difference between them.
+
+There are times however when these assumptions don’t hold:
+
+- Your intended treatment can’t be controlled and scoped to the user level
+- You don’t know who the users are
+- You can’t track outcomes at the user level
+
+Running geographically-based campaigns is a prime example that often features at least some of these violations. Examples include:
+
+- Some classic marketing, like billboards, can’t be randomly split between test and control users at a given location. We can’t track who sees them and how they act afterwards.
+- Some digital campaigns might be able to split users 50/50 (if the platform allows it), but if you don't have user-level data that you can tie to outcomes of interest, you'll never know how the two groups performed.
+
+## Using Synthetic Controls
+
+“Geotesting” as we refer to it here is an experimental framework that relies on a different basic setup than AB tests:
+
+1. Split your users into geographical boundaries at some useful level (zip codes, states, countries, etc)
+2. Apply a known treatment in some “treatment” geos, and no treatment in some “control” geos
+3. Use the control geos to figure out what would have happened in the treatment geos had no treatment been applied (i.e. “synthetic” results)
+4. Measure the delta in the treatment geos between the actual and synthetic values
+
+The “figuring out what would have happened” tends to be the hardest and nastiest part of this setup. Deducing things that don’t occur is usually a fool’s errand, but in some circumstances there’s enough predictability in a system to figure it out.
+
+Geotesting relies on the idea that among some sets of geos there are enough correlations between control and test geos that make predicting this “counterfactual” possible. We assume, for instance, that general trends and seasonality tend to be shared across some geos.
+
+Example: pretty much all zip codes in the United States exhibit similar quantities of mail sent on any given day. If you were to split zip codes 50/50, you could reasonably infer that the two groups would see similar trends.
+
+# Approach
+
+There are many ways to estimate unknown data from other known data, depending on a multitude of factors. In the case of Geotesting, we have:
+
+- timeseries data that is autocorrelated (i.e. data is correlated from one day to the next within a group)
+- timeseries data that is correlated between groups
+- regular seasonal patterns that repeat at regular intervals
+- granular data from among many distinct geographies
+
+Plotting some metric of interest for a bunch of relevant geographies can show these relationships clearly.
+
+![Example data from the GeoLift package walkthrough. ](/static/img/geotests/GeosPreTest.png)
+
+Example data from the GeoLift package walkthrough. 
+
+But what happens when we run a campaign or make some change in just some of these geos? How can we tell if a fluctuation in this graph is random noise or real stat-sig result? 
+
+![A marketing campaign that starts on day 91 (black dotted line) could have affected the treatment cities of Chicago and Portland. But how can you tell any real effect from all the noise?](/static/img/geotests/GeosTest.png)
+
+A marketing campaign that starts on day 91 (black dotted line) could have affected the treatment cities of Chicago and Portland. But how do can you tell any real effect from all the noise?
+
+We turn to causal inference modeling to answer this question. Thankfully, a variety of powerful models and methods have been developed to answer this question. The basic steps are the following:
path: docs/statsig-warehouse-native/geotests/introduction.md
user: login: graphite-app[bot]
body: There's a missing "to" in the sentence: "...models and methods have been developed **to** answer this question."
```suggestion
We turn to causal inference modeling to answer this question. Thankfully, a variety of powerful models and methods have been developed to answer this question. The basic steps are the following:
```
  

*Spotted by [Diamond](https://app.graphite.dev/diamond/?org=statsig-io&ref=ai-review-comment)*<i class='graphite__hidden'><br /><br />Is this helpful? React 👍 or 👎 to let us know.</i>
created_at: 2025-04-09T19:30:17Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,99 @@
+---
+title: Geotests
+sidebar_label: Geotests
+slug: /experiments-plus/geotests
+keywords:
+  - owner:makris
+last_update:
+  date: 2025-04-09
+---
+
+# Geotests
+
+TBD
+
+## How it's different
+
+Standard A/B testing rely on a few core assumptions:
+
+1. You can reliably and randomly split your users into similar groups
+2. While each group has heterogeneity inside it, random user-level variations average out at larger sample sizes, resulting in homologous (similar) groups
+3. You control how each group gets treated, and there is no interference between groups
+
+When these core assumptions are true, we assume the treatment effect will be the only observable difference between the groups. We can simply measure each group’s mean value and find the difference between them.
+
+There are times however when these assumptions don’t hold:
+
+- Your intended treatment can’t be controlled and scoped to the user level
+- You don’t know who the users are
+- You can’t track outcomes at the user level
+
+Running geographically-based campaigns is a prime example that often features at least some of these violations. Examples include:
+
+- Some classic marketing, like billboards, can’t be randomly split between test and control users at a given location. We can’t track who sees them and how they act afterwards.
+- Some digital campaigns might be able to split users 50/50 (if the platform allows it), but if you don't have user-level data that you can tie to outcomes of interest, you'll never know how the two groups performed.
path: docs/statsig-warehouse-native/geotests/introduction.md
user: login: graphite-app[bot]
body: There appears to be a missing word in this sentence. It should read "but if **you** don't have user-level data" to maintain grammatical correctness.
```suggestion
- Some digital campaigns might be able to split users 50/50 (if the platform allows it), but if you don't have user-level data that you can tie to outcomes of interest, you'll never know how the two groups performed.
```
  

*Spotted by [Diamond](https://app.graphite.dev/diamond/?org=statsig-io&ref=ai-review-comment)*<i class='graphite__hidden'><br /><br />Is this helpful? React 👍 or 👎 to let us know.</i>
created_at: 2025-04-09T19:30:19Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,99 @@
+---
+title: Geotests
+sidebar_label: Geotests
+slug: /experiments-plus/geotests
+keywords:
+  - owner:makris
+last_update:
+  date: 2025-04-09
+---
+
+# Geotests
+
+TBD
+
+## How it's different
+
+Standard A/B testing relies on a few core assumptions:
+
+1. You can reliably and randomly split your users into similar groups
+2. While each group has heterogeneity inside it, random user-level variations average out at larger sample sizes, resulting in homologous (similar) groups
+3. You control how each group gets treated, and there is no interference between groups
+
+When these core assumptions are true, we assume the treatment effect will be the only observable difference between the groups. We can simply measure each group’s mean value and find the difference between them.
+
+There are times however when these assumptions don’t hold:
+
+- Your intended treatment can’t be controlled and scoped to the user level
+- You don’t know who the users are
+- You can’t track outcomes at the user level
+
+Running geographically-based campaigns is a prime example that often features at least some of these violations. Examples include:
+
+- Some classic marketing, like billboards, can’t be randomly split between test and control users at a given location. We can’t track who sees them and how they act afterwards.
+- Some digital campaigns might be able to split users 50/50 (if the platform allows it), but if you don't have user-level data that you can tie to outcomes of interest, you'll never know how the two groups performed.
+
+## Using Synthetic Controls
+
+“Geotesting” as we refer to it here is an experimental framework that relies on a different basic setup than AB tests:
+
+1. Split your users into geographical boundaries at some useful level (zip codes, states, countries, etc)
+2. Apply a known treatment in some “treatment” geos, and no treatment in some “control” geos
+3. Use the control geos to figure out what would have happened in the treatment geos had no treatment been applied (i.e. “synthetic” results)
+4. Measure the delta in the treatment geos between the actual and synthetic values
+
+The “figuring out what would have happened” tends to be the hardest and nastiest part of this setup. Deducing things that don’t occur is usually a fool’s errand, but in some circumstances there’s enough predictability in a system to figure it out.
+
+Geotesting relies on the idea that among some sets of geos there are enough correlations between control and test geos that make predicting this “counterfactual” possible. We assume, for instance, that general trends and seasonality tend to be shared across some geos.
+
+Example: pretty much all zip codes in the United States exhibit similar quantities of mail sent on any given day. If you were to split zip codes 50/50, you could reasonably infer that the two groups would see similar trends.
+
+# Approach
+
+There are many ways to estimate unknown data from other known data, depending on a multitude of factors. In the case of Geotesting, we have:
+
+- timeseries data that is autocorrelated (i.e. data is correlated from one day to the next within a group)
+- timeseries data that is correlated between groups
+- regular seasonal patterns that repeat at regular intervals
+- granular data from among many distinct geographies
+
+Plotting some metric of interest for a bunch of relevant geographies can show these relationships clearly.
+
+![Example data from the GeoLift package walkthrough. ](/static/img/geotests/GeosPreTest.png)
+
+Example data from the GeoLift package walkthrough. 
+
+But what happens when we run a campaign or make some change in just some of these geos? How can we tell if a fluctuation in this graph is random noise or real stat-sig result? 
+
+![A marketing campaign that starts on day 91 (black dotted line) could have affected the treatment cities of Chicago and Portland. But how can you tell any real effect from all the noise?](/static/img/geotests/GeosTest.png)
+
+A marketing campaign that starts on day 91 (black dotted line) could have affected the treatment cities of Chicago and Portland. But how do can you tell any real effect from all the noise?
+
+We turn to causal inference modeling to answer this question. Thankfully, a variety of powerful models and methods have been developed to answer this question. The basic steps are the following:
+
+- Split your geos into test and control groups
+- Using the pre-treatment period data, train a model on the control data to predict how the treatment geos behave
+- Use the post-treatment period data for the control geos, create a “synthetic control” dataset to predict how the treatment geos would have behaved after the treatment event.
+- Subtract the synthetic data from the actual observed data to determine the induced effects of your treatment on the treatment geos
+
+When your training data is able to produce strong models that can predict metric outcome well, you end up with strong estimates of an induced effect:
+
+![Image](/static/img/geotests/Period_Split.png)
+
+![Subtracting the modeled Synthetic Control values from the observed Treatment values reveals any incremental effect for the Treatment geos.](/static/img/geotests/Incremental_Effects.png)
+
+Subtracting the modeled Synthetic Control values from the observed Treatment values reveals any incremental effect for the Treatment geos.
path: docs/statsig-warehouse-native/geotests/introduction.md
user: login: graphite-app[bot]
body: There appears to be a typo in this caption: "reveals any incrementally" should be "reveals any incrementality" to properly describe how the difference between synthetic control and observed treatment values shows the causal effect.
```suggestion
Subtracting the modeled Synthetic Control values from the observed Treatment values reveals any incrementality for the Treatment geos.
```
  

*Spotted by [Diamond](https://app.graphite.dev/diamond/?org=statsig-io&ref=ai-review-comment)*<i class='graphite__hidden'><br /><br />Is this helpful? React 👍 or 👎 to let us know.</i>
created_at: 2025-04-09T19:30:21Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,99 @@
+---
+title: Geotests
+sidebar_label: Geotests
+slug: /experiments-plus/geotests
+keywords:
+  - owner:makris
+last_update:
+  date: 2025-04-09
+---
+
+# Geotests
+
+TBD
path: docs/statsig-warehouse-native/geotests/introduction.md
user: login: graphite-app[bot]
body: The introductory section currently has a placeholder "TBD" instead of actual content. This section should provide a clear, concise overview of what Geotests are and their primary purpose before diving into the "How it's different" section. Consider adding 2-3 sentences that define Geotests and explain their value proposition to users who may be encountering this concept for the first time.
```suggestion
Geotests are specialized experiments that allow you to target users based on their geographic location. They enable you to test features, content, or experiences across different regions, countries, or specific geographic areas to understand how location impacts user behavior and preferences. Geotests are particularly valuable for businesses with global audiences who need to optimize their product for regional differences in user needs, preferences, or regulatory requirements.
```
  

*Spotted by [Diamond](https://app.graphite.dev/diamond/?org=statsig-io&ref=ai-review-comment)*<i class='graphite__hidden'><br /><br />Is this helpful? React 👍 or 👎 to let us know.</i>
created_at: 2025-04-09T19:31:26Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,99 @@
+---
+title: Geotests
+sidebar_label: Geotests
+slug: /experiments-plus/geotests
+keywords:
+  - owner:makris
+last_update:
+  date: 2025-04-09
+---
+
+# Geotests
+
+TBD
+
+## How it's different
+
+Standard A/B testing relies on a few core assumptions:
+
+1. You can reliably and randomly split your users into similar groups
+2. While each group has heterogeneity inside it, random user-level variations average out at larger sample sizes, resulting in homologous (similar) groups
+3. You control how each group gets treated, and there is no interference between groups
+
+When these core assumptions are true, we assume the treatment effect will be the only observable difference between the groups. We can simply measure each group’s mean value and find the difference between them.
+
+There are times however when these assumptions don’t hold:
+
+- Your intended treatment can’t be controlled and scoped to the user level
+- You don’t know who the users are
+- You can’t track outcomes at the user level
+
+Running geographically-based campaigns is a prime example that often features at least some of these violations. Examples include:
+
+- Some classic marketing, like billboards, can’t be randomly split between test and control users at a given location. We can’t track who sees them and how they act afterwards.
+- Some digital campaigns might be able to split users 50/50 (if the platform allows it), but if you don't have user-level data that you can tie to outcomes of interest, you'll never know how the two groups performed.
+
+## Using Synthetic Controls
+
+“Geotesting” as we refer to it here is an experimental framework that relies on a different basic setup than AB tests:
+
+1. Split your users into geographical boundaries at some useful level (zip codes, states, countries, etc)
+2. Apply a known treatment in some “treatment” geos, and no treatment in some “control” geos
+3. Use the control geos to figure out what would have happened in the treatment geos had no treatment been applied (i.e. “synthetic” results)
+4. Measure the delta in the treatment geos between the actual and synthetic values
+
+The “figuring out what would have happened” tends to be the hardest and nastiest part of this setup. Deducing things that don’t occur is usually a fool’s errand, but in some circumstances there’s enough predictability in a system to figure it out.
+
+Geotesting relies on the idea that among some sets of geos there are enough correlations between control and test geos that make predicting this “counterfactual” possible. We assume, for instance, that general trends and seasonality tend to be shared across some geos.
+
+Example: pretty much all zip codes in the United States exhibit similar quantities of mail sent on any given day. If you were to split zip codes 50/50, you could reasonably infer that the two groups would see similar trends.
+
+# Approach
+
+There are many ways to estimate unknown data from other known data, depending on a multitude of factors. In the case of Geotesting, we have:
+
+- timeseries data that is autocorrelated (i.e. data is correlated from one day to the next within a group)
+- timeseries data that is correlated between groups
+- regular seasonal patterns that repeat at regular intervals
+- granular data from among many distinct geographies
+
+Plotting some metric of interest for a bunch of relevant geographies can show these relationships clearly.
+
+![Example data from the GeoLift package walkthrough. ](/static/img/geotests/GeosPreTest.png)
+
+Example data from the GeoLift package walkthrough. 
+
+But what happens when we run a campaign or make some change in just some of these geos? How can we tell if a fluctuation in this graph is random noise or real stat-sig result? 
+
+![A marketing campaign that starts on day 91 (black dotted line) could have affected the treatment cities of Chicago and Portland. But how can you tell any real effect from all the noise?](/static/img/geotests/GeosTest.png)
+
+A marketing campaign that starts on day 91 (black dotted line) could have affected the treatment cities of Chicago and Portland. But how do can you tell any real effect from all the noise?
+
+We turn to causal inference modeling to answer this question. Thankfully, a variety of powerful models and methods have been developed to answer this question. The basic steps are the following:
+
+- Split your geos into test and control groups
+- Using the pre-treatment period data, train a model on the control data to predict how the treatment geos behave
+- Use the post-treatment period data for the control geos, create a “synthetic control” dataset to predict how the treatment geos would have behaved after the treatment event.
+- Subtract the synthetic data from the actual observed data to determine the induced effects of your treatment on the treatment geos
+
+When your training data is able to produce strong models that can predict metric outcome well, you end up with strong estimates of an induced effect:
+
+![Image](/static/img/geotests/Period_Split.png)
+
+![Subtracting the modeled Synthetic Control values from the observed Treatment values reveals any incremental effect for the Treatment geos.](/static/img/geotests/Incremental_Effects.png)
+
+Subtracting the modeled Synthetic Control values from the observed Treatment values reveals any incremental effect for the Treatment geos.
+
+# GeoLift
+
+[GeoLift](https://facebookincubator.github.io/GeoLift/) is an open-source package from Meta that’s used by many to infer causal relationships in timeseries data. It builds on advancements made in prior packages like Google’s [Causal Impact](https://google.github.io/CausalImpact/CausalImpact.html) package.
+
+Statsig is using GeoLift in our own implementation of Geotesting, and allowing you to easily connect your own internal metrics and feed them into the analysis package.
+
+# Notes
+
+TBD
+
+# References
+
+TBD
path: docs/statsig-warehouse-native/geotests/introduction.md
user: login: graphite-app[bot]
body: The "Notes" and "References" sections currently contain placeholder "TBD" text. Consider replacing these with actual content before finalizing the documentation. For the Notes section, adding implementation considerations or limitations would be valuable. For References, include links to relevant documentation, research papers, or the GeoLift GitHub repository that was mentioned in the introduction.
```suggestion
# References

- [GeoLift GitHub Repository](https://github.com/facebookexperimental/GeoLift)
- [Meta Marketing Science: Geo-Lift Testing](https://www.facebook.com/business/help/339320204990134)
- [Statsig Documentation: Experimentation Best Practices](https://docs.statsig.com/experiments-plus/best-practices)
- [Geo-Experimentation: A Methodological Overview](https://research.facebook.com/publications/geo-experiments-for-business-decisions/)
- [Warehouse Native Experiments Overview](https://docs.statsig.com/statsig-warehouse-native/introduction)
```
  

*Spotted by [Diamond](https://app.graphite.dev/diamond/?org=statsig-io&ref=ai-review-comment)*<i class='graphite__hidden'><br /><br />Is this helpful? React 👍 or 👎 to let us know.</i>
created_at: 2025-04-09T19:31:27Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,99 @@
+---
+title: Geotests
+sidebar_label: Geotests
+slug: /experiments-plus/geotests
+keywords:
+  - owner:makris
+last_update:
+  date: 2025-04-09
+---
+
+# Geotests
+
+TBD
+
+## How it's different
+
+Standard A/B testing relies on a few core assumptions:
+
+1. You can reliably and randomly split your users into similar groups
+2. While each group has heterogeneity inside it, random user-level variations average out at larger sample sizes, resulting in homologous (similar) groups
+3. You control how each group gets treated, and there is no interference between groups
+
+When these core assumptions are true, we assume the treatment effect will be the only observable difference between the groups. We can simply measure each group’s mean value and find the difference between them.
+
+There are times however when these assumptions don’t hold:
+
+- Your intended treatment can’t be controlled and scoped to the user level
+- You don’t know who the users are
+- You can’t track outcomes at the user level
+
+Running geographically-based campaigns is a prime example that often features at least some of these violations. Examples include:
+
+- Some classic marketing, like billboards, can’t be randomly split between test and control users at a given location. We can’t track who sees them and how they act afterwards.
+- Some digital campaigns might be able to split users 50/50 (if the platform allows it), but if you don't have user-level data that you can tie to outcomes of interest, you'll never know how the two groups performed.
+
+## Using Synthetic Controls
+
+“Geotesting” as we refer to it here is an experimental framework that relies on a different basic setup than AB tests:
+
+1. Split your users into geographical boundaries at some useful level (zip codes, states, countries, etc)
+2. Apply a known treatment in some “treatment” geos, and no treatment in some “control” geos
+3. Use the control geos to figure out what would have happened in the treatment geos had no treatment been applied (i.e. “synthetic” results)
+4. Measure the delta in the treatment geos between the actual and synthetic values
+
+The “figuring out what would have happened” tends to be the hardest and nastiest part of this setup. Deducing things that don’t occur is usually a fool’s errand, but in some circumstances there’s enough predictability in a system to figure it out.
+
+Geotesting relies on the idea that among some sets of geos there are enough correlations between control and test geos that make predicting this “counterfactual” possible. We assume, for instance, that general trends and seasonality tend to be shared across some geos.
+
+Example: pretty much all zip codes in the United States exhibit similar quantities of mail sent on any given day. If you were to split zip codes 50/50, you could reasonably infer that the two groups would see similar trends.
+
+# Approach
+
+There are many ways to estimate unknown data from other known data, depending on a multitude of factors. In the case of Geotesting, we have:
+
+- timeseries data that is autocorrelated (i.e. data is correlated from one day to the next within a group)
+- timeseries data that is correlated between groups
+- regular seasonal patterns that repeat at regular intervals
+- granular data from among many distinct geographies
+
+Plotting some metric of interest for a bunch of relevant geographies can show these relationships clearly.
+
+![Example data from the GeoLift package walkthrough. ](/img/geotests/GeosPreTest.png)
+
+Example data from the GeoLift package walkthrough. 
+
+But what happens when we run a campaign or make some change in just some of these geos? How can we tell if a fluctuation in this graph is random noise or real stat-sig result? 
+
+![A marketing campaign that starts on day 91 (black dotted line) could have affected the treatment cities of Chicago and Portland. But how can you tell any real effect from all the noise?](/img/geotests/GeosTest.png)
+
+A marketing campaign that starts on day 91 (black dotted line) could have affected the treatment cities of Chicago and Portland. But how do can you tell any real effect from all the noise?
path: docs/statsig-warehouse-native/geotests/introduction.md
user: login: graphite-app[bot]
body: There appears to be a typo in this sentence. It should read "But how can you tell any real effect from all the noise?" (remove the word "do").
```suggestion
A marketing campaign that starts on day 91 (black dotted line) could have affected the treatment cities of Chicago and Portland. But how can you tell any real effect from all the noise?
```
  

*Spotted by [Diamond](https://app.graphite.dev/diamond/?org=statsig-io&ref=ai-review-comment)*<i class='graphite__hidden'><br /><br />Is this helpful? React 👍 or 👎 to let us know.</i>
created_at: 2025-04-09T19:53:45Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,99 @@
+---
+title: Geotests
+sidebar_label: Geotests
+slug: /experiments-plus/geotests
+keywords:
+  - owner:makris
+last_update:
+  date: 2025-04-09
+---
+
+# Geotests
+
+TBD
path: docs/statsig-warehouse-native/geotests/introduction.md
user: login: graphite-app[bot]
body: The introduction section begins with a "TBD" placeholder that needs to be replaced with actual content. This section should provide a clear overview of what geotests are before diving into the "How it's different" section. Consider adding a concise definition and purpose of geotests to help orient readers who may be unfamiliar with the concept.
```suggestion
Geotests are specialized experiments that allow you to test variations of your product across different geographic regions. They enable you to measure how changes to your product perform in specific locations before rolling them out globally. With geotests, you can target experiments to countries, states/provinces, cities, or other geographic boundaries to understand regional differences in user behavior and preferences.

Geotests are particularly valuable when:
- Testing features that may have different impacts across regions due to cultural, regulatory, or market differences
- Gradually rolling out changes to minimize risk by starting in smaller geographic areas
- Validating hypotheses about regional user preferences or behaviors
- Optimizing for location-specific factors that affect your product's performance
```
  

*Spotted by [Diamond](https://app.graphite.dev/diamond/?org=statsig-io&ref=ai-review-comment)*<i class='graphite__hidden'><br /><br />Is this helpful? React 👍 or 👎 to let us know.</i>
created_at: 2025-04-09T19:53:47Z
author_association: CONTRIBUTOR

PR comments:
user: login: makris-statsig
created_at: 2025-04-09T18:46:48Z
body: * **#2854** <a href="https://app.graphite.dev/github/pr/statsig-io/docs/2854?utm_source=stack-comment-icon" target="_blank"><img src="https://static.graphite.dev/graphite-32x32-black.png" alt="Graphite" width="10px" height="10px"/></a> 👈 <a href="https://app.graphite.dev/github/pr/statsig-io/docs/2854?utm_source=stack-comment-view-in-graphite" target="_blank">(View in Graphite)</a>
* `main`


<h2></h2>

This stack of pull requests is managed by <a href="https://graphite.dev?utm-source=stack-comment"><b>Graphite</b></a>. Learn more about <a href="https://stacking.dev/?utm_source=stack-comment">stacking</a>.
<!-- Current dependencies on/for this PR: -->
author_association: CONTRIBUTOR
user: login: jasonwzm
created_at: 2025-04-09T19:54:21Z
body: <!-- NETLIFY DEPLOY COMMENT GENERATED BY ACTIONS_NETLIFY - APP ID SHA256: 0038af8fad421c48cf2dd5ee95e08d2d419533ec7208a507c5ccd1696cdf8b44 -->
🚀 Deployed on https://67f6d20315bcd87097652854--cozy-fox-0defec.netlify.app
author_association: MEMBER
user: login: makris-statsig
created_at: 2025-04-09T20:03:24Z
body: ### Merge activity

* **Apr 9, 4:03 PM EDT**: A user started a stack merge that includes this pull request via [Graphite](https://app.graphite.dev/github/pr/statsig-io/docs/2854).
* **Apr 9, 4:03 PM EDT**: A user merged this pull request with [Graphite](https://app.graphite.dev/github/pr/statsig-io/docs/2854).

author_association: CONTRIBUTOR