How to Design Software — Feature Flags
By: Joseph Gefroh
Tags: Programming, Software Development, Web Development, Technology, Software Architecture

Iterations and Improvements

Our new system was fantastic.

It allowed us to rapidly apply different kinds of feature flags or apply multiple flags within different contexts without the need for database migrations.

Cascading flag checks were now easy, we could toggle functionality based on a specific record or for a specific user. We didn’t stop there. Now that we weren’t limited by legacy constraints, we could quickly add more functionality as time went on.

Descriptive enumerations

One problem with the old feature flag approach was that nobody knew what the flags actually did or meant.

Because many of them were boolean fields or string values without descriptions, it was difficult to tell exactly what the consequences of toggling them were. Unless you were already familiar with the system, you would never be able to find out.

When we transitioned them over, we preserved their unfortunate naming.

For non-engineers, flags with values like bpfee and gp were indecipherable. It provided a terrible user experience and led to a lot of operational errors that ultimately made their way back to engineering in the form of bug reports and investigations, which we wanted to prevent.

We wanted to clearly describe what flags did and what they meant, so we introduced a data model to list what the available options were:

Feature Flag Options

We changed our modification interface to decorate the list of feature flags with these details, providing significantly more clarity to anyone modifying the feature flags.

They no longer had to see just:

bpfee

They also saw what it meant:

Receipt Basis Points Fee (bpfee) Display fee percentages charged as basis points on the receipt.

The increased clarity led to significantly fewer errors and questions reaching us, resolving a large source of upstream issues.

Defaults

Sometimes when a record gets created we want certain flags to be automatically enabled. By adding a column to the FeatureFlagOption table, default_status , we can achieve this:

Feature Flag Options

Now, whenever a record was created, we could pull up all of the flags where default_status was enabled and automatically enable the flags on that record.

Categorization

Over time, you get a lot of different feature flags, all for different purposes.

Some are flags intended for short-term use, typically to hide code until it is done or to mitigate risk by performing canary releases. Once the code is deployed, the flag should get removed. Some are flags intended for mid-term use, such as those used to perform experiments. Some flags are intended for long-term use. Features might be toggled based on the contract or plan the customer is on.

Each of these flags have different owners and usages, so it is important to be able to separate them. By adding a category to FeatureFlagOption , we are able to scope the enumeration and behavior even further:

Adding a category allowed us to scope the options appropriately depending on audience and usage.

Experimentation

Sometimes, we want flags that split groups into different cohorts for A/B testing of features. We could store the various parameters of experiments in the FeatureFlagOption , to be sent to the A/B subsystem:

By allowing for the storage of optional parameters in a flag, you can customize behavior of a flag.

While an A/B system is outside the scope of this post, I recommend checking out the split gem if you are using ruby.

Performance

As usage of the feature flag system grows, you don’t want to be performing that many database reads. The abstraction layer provides a perfect place to put in a caching layer as well to ensure rapid lookups of feature flags.

As always, remember the words of the famed Donald Knuth:

We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.

Ensure you have an actual need to improve performance and that the usage patterns are appropriate for your solution.

As another tip: don’t randomly add caching logic ad-hoc. It quickly gets out of hand when you try and figure out whether you are dealing with cached data or not. It is made monumentally more complex when caching layers on the controller, data, and database all start interacting with each other.

Add caching thoughtfully and with centralized controls so you know exactly how stale the data is and how to force a refresh, if necessary.

Client-side checks

Our new technology platform greatly leveraged a lot of the feature flag infrastructure. We created a front-end service and component that showed/hid nested components based on the feature flag’s status.

By exposing flag checks in an API and then consolidating all of its logic within the service and component, we were able to avoid having feature flag logic all over the front-end.