Get started
← Blog

Multivariate feature flags: more than on and off

Oct 31, 2026

A multivariate feature flag holds three or more variants in a single flag, not just on and off. Where a binary flag picks between two code paths, a multivariate flag splits traffic across every option on the table, from a 50/50 toggle to a four-way test of checkout copy.

Most flag tooling treats “variant” as a stand-in for “the other side of a boolean.” A team that wants to test four CTA copies at once ends up either chaining four independent booleans that can combine in ways nobody tested, or reaching for a dedicated multivariate testing tool built for a different job: combining several independent page elements at once, not splitting one decision N ways.

This post covers what a multivariate feature flag is, how it differs from classic multivariate testing, how the bucketing math extends from two buckets to N, and the tradeoff: more arms means each one reaches significance slower.

Key Takeaways

  • A multivariate feature flag is one file with three or more weighted variants, not a chain of independent booleans.
  • Deterministic bucketing extends directly from two buckets to N. Only the cumulative weight table changes; the hash and the call site stay the same.
  • Classic multivariate testing (MVT) combines several independent page elements at once. A multivariate feature flag tests N variants of one decision. They solve different problems.
  • Splitting traffic four ways instead of two gives each arm roughly a quarter of the traffic, so the test needs more total traffic, more time, or a correction for comparing multiple arms against one control.
  • An exclusion_group keeps two multivariate experiments on the same surface from showing one user a combination nobody tested.

What is a multivariate feature flag?

A multivariate feature flag is a flag or experiment with three or more variants, each carrying a weight, instead of a single on/off split. The mechanism is identical to a two-way flag: a user is assigned one variant, deterministically, and the code branches on the result. The only thing that changes is the count.

In dif, a flag’s variants list already supports any number of entries. A kill switch is a two-variant flag (on, off) at 100/0. A four-way CTA test is the same list with four entries instead of two. dif validate checks that the weights sum to 100 regardless of how many variants are in the file, so a four-way test fails the same way a lopsided two-way test does: in the pull request, not in production.

Multivariate testing vs. A/B/n testing

The phrase “multivariate testing” already means something specific, and it is not quite what most people reach for when they say “multivariate feature flag.”

Classic multivariate testing (the term Google popularized with Website Optimizer, and that Wikipedia’s definition still reflects) tests combinations of several independent page elements at once: a headline, an image, and a CTA, each with its own variants, shown in every combination. A three-element test with two options each is eight combinations, a full factorial design.

What most teams actually want is simpler: one decision, more than two options. A four-way CTA copy test is not four elements combined, it is one element with four variants. The precise name for that is an A/B/n test, or a multi-armed experiment. A dif flag’s variants array is built for this case: N weighted options on one decision, not a factorial grid of several. If a true factorial test is what you need, model each element as its own flag and read the combination at the call site; dif does not orchestrate the factorial design for you, and this post is about the simpler, more common case.

N variants, one frontmatter file

The file looks like any other dif experiment, just with more entries in variants:

---
id: checkout-cta-copy
status: active
owner: sam@acme.com
surface: checkout
hypothesis: >
  A benefit-led or urgency-led CTA lifts completed checkouts
  over the neutral control.
variants:
  - id: control
    weight: 25
  - id: get_it_today
    weight: 25
  - id: only_12_left
    weight: 25
  - id: free_shipping_now
    weight: 25
metrics:
  primary: completed_checkout
  guardrails:
    - refund_rate
---

## Brief

Ship all four at 25% each. Conclude when completed_checkout
reaches significance against control, with refund_rate holding.

Nothing about dif validate, dif build, or the SDK call changes because there are four variants instead of two. The format reference covers the full frontmatter schema, including exclusion_group, which this post comes back to below.

Bucketing math extends to N arms

A two-way flag buckets a user into one of 10,000 slots and reads the first one, cumulative weight 0-4,999 for control, 5,000-9,999 for the treatment. A four-way flag does the same hash and the same mod 10,000, just with four cumulative ranges instead of two: 0-2,499, 2,500-4,999, 5,000-7,499, 7,500-9,999 for 25% weights each.

The bucket is still the first four bytes of SHA-256(salt || user_id), read as a big-endian integer, mod 10,000. Nothing is stored, there is no network call to decide, and the same user lands in the same variant every time, on every device, whether the flag has two variants or 10. Adding a fifth variant next month only repaints unclaimed buckets; nobody who already has free_shipping_now gets reassigned.

Reading N variants at the call site

The SDK call returns whichever variant id won the bucket, and the code branches on it like any enum:

const variant = dif("checkout-cta-copy");

switch (variant) {
  case "get_it_today":
    return <Cta copy="Get it today" />;
  case "only_12_left":
    return <Cta copy="Only 12 left" />;
  case "free_shipping_now":
    return <Cta copy="Free shipping, no minimum" />;
  default:
    return <Cta copy="Buy now" />;
}

// when the user completes checkout:
dif.track("completed_checkout");

One exposure event fires the first time a user sees their assigned variant, same as a two-way flag. The SDK docs cover the React and Svelte adapters that do this assignment server-side, so the variant is in the first byte of HTML regardless of how many options the flag holds.

The tradeoff: more arms, slower significance

Here is the cost dif does not hide: splitting traffic four ways instead of two gives each arm a quarter of the traffic instead of half. To detect the same effect size with the same confidence, a four-way test needs roughly double the total traffic of a two-way test, and a low-traffic surface may never get there.

There is a second cost beyond raw traffic. Comparing four arms against one control is four comparisons, not one, and each comparison carries its own chance of a false positive. Evan Miller’s guidance on running A/B tests applies here too: decide the sample size and the comparisons you will make before the test starts, because checking four p-values repeatedly inflates the odds that one looks significant by chance. Microsoft’s experimentation platform research is explicit that this multiple-comparisons problem needs a correction, not a shrug, once a test has more than two arms.

Often the answer to “should this be multivariate” is no. If one variant is a clear front-runner, run it as a two-way test against control first, then ask the next question once that one is settled. Waiting for a quarter of the traffic to answer four questions at once is usually slower than answering them one at a time.

Keeping two multivariate tests from colliding

A four-way CTA test and a four-way layout test, run on the same checkout surface at the same time, can hand one user a combination that was never actually tested: free_shipping_now copy inside a layout variant nobody paired it with. The more arms a test has, the more combinations exist to collide.

dif’s exclusion_group fixes this the same way it does for two-way tests: tag both files with the same group, and a user matches at most one of them. Members are ordered by their created date, so the earliest-created experiment wins and the other shows its default variant and records nothing. The result is attributable: a metric movement on checkout traces back to exactly one test, whether that test has two variants or 10.

Getting started

A multivariate flag is not a different product from a binary flag, it is the same file with a longer variants list. Install the CLI and the rest follows:

npm install -g @dif.sh/cli
dif init

dif new scaffolds a flag file pre-loaded with the surface’s recent learnings. Write the hypothesis, list as many variants as the decision actually has, and run dif validate, which checks the weights sum to 100 whether there are two of them or eight. The CLI reference covers the rest of the lifecycle, and experimentation in dif covers how a multivariate flag becomes a plain flag once it has a winner.

A multivariate feature flag comes down to the same four things a two-way flag does: a hypothesis in a file, deterministic assignment, a metric agreed on before starting, and a recorded decision when the numbers land. The only difference is how many branches the file names.

FAQ

What is a multivariate feature flag? A flag or experiment with three or more weighted variants in one file, instead of a single on/off split. Assignment is deterministic and the SDK call returns whichever variant id the user’s bucket landed on.

What’s the difference between multivariate testing and A/B testing? Classic multivariate testing combines several independent page elements (a headline, an image, a CTA) in every combination, a full factorial design. Most “test more than two options” requests are actually A/B/n tests: one decision with N variants, which is what a multivariate feature flag models directly.

How many variants can a feature flag have? As many as the decision needs, as long as the weights sum to 100. dif validate checks the total regardless of count, so a 10-variant flag is validated the same way a two-variant one is.

Does adding more variants slow down a test? Yes. Each arm gets a smaller share of traffic, so reaching significance on each pairwise comparison takes more total traffic or more time. Comparing several arms against one control also raises the chance of a false positive unless you correct for it.

Can two multivariate experiments run on the same page without colliding? Yes, with an exclusion_group. Tag both files with the same group and each user matches at most one of them, ordered by creation date, so a combination of variants from two different tests never reaches the same user.