Feature Flags in Go: Targeting Rules, Stable Rollouts, and Config That Can't Half-Apply

What Flags Are For

A feature flag is a decision about which code path runs, taken at runtime instead of at build time. Everything useful about flags follows from moving that decision out of the deploy:

• Deploying is no longer releasing. Unfinished code can be merged to the main branch and shipped behind a flag that's off. Branches stay short, merge conflicts stay small, and "the release" becomes flipping a flag instead of a big merge plus a deploy on the same afternoon.
• Rollouts by users, not by servers. A canary deploy exposes new code to a share of instances; a flag exposes it to a share of users, and the same users every time. 1%, then 10%, then 50%, with a look at error rates and latency between steps.
• Rollback without a deploy. Turning a flag off takes as long as config propagation, usually seconds. Rolling back a deploy takes a build, a pipeline, and a restart of every instance, and it also rolls back every other change that went out with it.
• Targeting. Employees first, then beta users, then one region, then everyone. Or a feature only for one plan, or only for app versions that can render it.
• Experiments. Two or more variants assigned to stable groups of users, so a change can be measured instead of argued about.

What a flag is not is general configuration. A timeout, a pool size, a rate limit value - those are config: numeric, permanent, not targeted at users. Putting them in the flag system works until someone needs a timeout that differs per user, at which point the targeting rules start being used for something they weren't designed for. Keep the two systems apart even if they share storage.

Four Kinds of Flags

Everything is called "a flag", but there are four quite different things behind that word, and mixing them up is where most long-term pain starts. They differ in how long they live, who changes them, and which default is safe:

The "safe default" column is the one that bites. A release flag that can't be evaluated should fall back to the old behavior; a kill switch that can't be evaluated should keep the feature running, otherwise the flag system being down turns into an outage of everything it protects. That's why the evaluator below takes the default from the call site instead of having one global rule.

Permission flags are a gray zone. As long as they change rarely and the list of customers is short, the flag system handles them fine. Once billing depends on them, they belong in the entitlement or billing system, which has an audit trail and a notion of contracts that a flag system doesn't.

What Flags Cost

Each flag is a small, permanent increase in how hard the system is to reason about until it's removed. The problems are predictable enough to list:

• State explosion. Ten boolean flags that a request can hit are 1,024 possible combinations of behavior. Tests usually cover two: everything off and everything on. Flags that interact - flag B only makes sense when flag A is on, or two flags touch the same code - produce bugs that only exist for the users who happen to be in one particular combination, and that nobody can reproduce from the default config.
• Two code paths to keep working. While a release flag exists, the old path is still live code: it has to compile, pass tests, and stay correct as the code around it changes. A flag that sits at 100% for months means the old path is untested in practice but still one config change away from running again.
• Old code coming back. The dangerous version of the previous point: a flag that's "obviously" dead gets reused with a new meaning, or someone flips an old flag while cleaning up, and long-dormant code starts running on whatever servers still have it. Delete the code with the flag, and never reuse a flag key.
• A production change without code review. Flipping a flag changes behavior for real users exactly like a deploy does, but it usually skips the pull request, the CI run, and the gradual deploy. That's the point of flags, and it's also how a single wrong rule (a typo in a targeting rule, a rollout at 100% instead of 10%) reaches every user in seconds. A flag change needs at least validation, an audit log of who changed what, and for sensitive flags an approval step.
• "Works for me". Two people looking at the same page see different code. Without a record of which variation each user got and why, a bug report turns into guessing which flags were on for that user at that moment.
• Flags can't undo data. A flag can turn code off; it can't un-write the rows the new code already wrote. If the new path writes data in a new format or runs a schema migration, turning the flag off only works if the old path can read what the new path wrote. Plan that before the rollout, not during the incident.
• A new runtime dependency. Every service now depends on the flag config being available and valid. What happens on startup before the first config arrives, when the control plane is down, or when someone publishes broken config, has to be decided explicitly - covered in Defaults and Failure Modes.
• Leaking unreleased features. Flag rules shipped to browsers or mobile apps expose every flag name, targeting rule, and attribute in them. Clients should get evaluated results, not the rules.

None of this is an argument against flags. It's an argument for treating each flag as something with an owner and an end date, and for building the evaluator so that the failure modes above are hard to hit by accident. The rest of the article is the second half of that.

Where Evaluation Happens

There are two halves to any flag system, and keeping them apart is the first design decision that matters:

• The control plane is where people create and edit flags: a UI or a repo of YAML, an API, an audit log, approvals. It changes a few times a day and nothing breaks if it's slow.
• The evaluator answers "which variation does this user get for this flag" on the request path. It runs for every request, often several times per request.

The tempting shortcut is to make the evaluator a remote service: GET /flags/new-checkout?user=... from every application, with a 2-second timeout and return false on error. That makes every request depend on the flag service's latency and availability, and it turns a flag-service incident into an application incident in the worst possible way: everything gated behind a flag silently switches to its fallback at the same moment.

The evaluator in this article runs inside the application process. Each instance holds a full copy of the config, refreshes it in the background, and evaluates locally. A flag check becomes a map lookup plus a few comparisons, and the control plane going away means the config stops updating, not that flags stop working.

The Data Model

The wire format is one JSON document carrying every flag, with a monotonically increasing version. Sending the whole set every time is deliberate: a client never applies half of an update, and "which version is this instance running" has one answer.

{
  "version": 42,
  "flags": [{
    "key": "new-checkout",
    "on": true,
    "variations": ["true", "false"],
    "off_variation": "false",
    "rules": [
      {
        "clauses": [{"attr": "email", "op": "ends_with", "values": ["@example.com"]}],
        "serve": {"variation": "true"}
      },
      {
        "clauses": [
          {"attr": "country", "op": "in", "values": ["DE", "AT"]},
          {"attr": "app_version", "op": "semver_gte", "values": ["3.2.0"]}
        ],
        "serve": {"rollout": [
          {"variation": "true",  "weight": 25000},
          {"variation": "false", "weight": 75000}
        ]}
      }
    ],
    "fallthrough": {"variation": "false"}
  }]
}

Read it top to bottom the way the evaluator does: if the flag is off, serve off_variation. Otherwise try each rule in order; a rule matches when all its clauses match, and a clause matches when the attribute satisfies the operator against any of its values. The first matching rule decides. If none match, fallthrough decides. Each of those decisions is either a fixed variation or a weighted rollout, never both.

Variations are strings, not booleans. A boolean flag is a flag whose variations are "true" and "false", and a three-way experiment is the same structure with three variations. One evaluator handles both, which matters more than it sounds: release toggles tend to grow into experiments, and a separate "A/B test" code path is where the subtle bugs below like to live.

type Flag struct {
    Key          string   `json:"key"`
    On           bool     `json:"on"`
    Variations   []string `json:"variations"`
    OffVariation string   `json:"off_variation"`
    Rules        []Rule   `json:"rules,omitempty"`
    Fallthrough  Serve    `json:"fallthrough"`
    // Salt seeds rollout bucketing. Empty means the flag key.
    Salt string `json:"salt,omitempty"`
}

type Rule struct {
    Clauses []Clause `json:"clauses"` // all must match
    Serve   Serve    `json:"serve"`
}

type Clause struct {
    Attr   string   `json:"attr"`
    Op     Op       `json:"op"`
    Values []string `json:"values"` // any may match
}

// Serve is either a fixed variation or a weighted rollout, never both.
type Serve struct {
    Variation string     `json:"variation,omitempty"`
    Rollout   []Weighted `json:"rollout,omitempty"`
}

// Weights in a rollout sum to BucketScale (100,000). Order matters.
type Weighted struct {
    Variation string `json:"variation"`
    Weight    int    `json:"weight"`
}

The operator set is small on purpose: in, not_in, ends_with, matches (regex), semver_gte, semver_lt. Every operator is a thing someone has to reason about when a user asks "why did I get the new checkout", and most real targeting is "these users", "these countries", "this app version and up", or "some percentage of everyone".

Evaluation, and Why It Returns a Reason

func (s *Snapshot) Evaluate(key string, u User) Result {
    f, ok := s.flags[key]
    if !ok {
        return Result{Reason: ReasonUnknownFlag}
    }
    if !f.on {
        return Result{Variation: f.off, Reason: ReasonOff}
    }
    for i := range f.rules {
        r := &f.rules[i]
        if r.matches(u) {
            res := f.serve(&r.serve, u, ReasonRuleMatch)
            res.RuleIndex = i
            return res
        }
    }
    return f.serve(&f.fallthrough_, u, ReasonFallthrough)
}

The result carries a Reason and, for rule matches, which rule. That looks like debugging garnish and is actually the most useful field in the struct. "Customer says they're seeing the new checkout and shouldn't be" is a question about why, and {true RULE_MATCH 0} next to the user ID in a log line answers it without anyone re-deriving the rule logic by hand. UNKNOWN_FLAG showing up in logs is also the cheapest way to find a call site that references a flag someone already deleted.

One clause-matching detail worth calling out, because getting it wrong targets far more users than intended:

func (c *compiledClause) matches(u User) bool {
    v, ok := u.attr(c.attr)
    if !ok {
        // A missing attribute never matches - including for not_in. Otherwise
        // "country not_in [DE]" would target every user whose country is unknown.
        return false
    }
    switch c.op {
    case OpIn:
        return slices.Contains(c.values, v)
    case OpNotIn:
        return !slices.Contains(c.values, v)
    // ...
    }
}

If not_in were implemented as the negation of in, a user with no country attribute would match country not_in [DE]. That's every anonymous user, every background job, every call site that forgot to pass the attribute. Treating "missing" as "doesn't match" for every operator is the conservative choice; the test suite pins it as not_in does not match missing attribute.

Percentage Rollouts That Don't Reshuffle Users

A percentage rollout has to give each user the same answer on every request, on every instance, across restarts, with no shared state. That means the decision is a pure function of the user and the flag:

const BucketScale = 100_000 // weights in 0.001% steps

func Bucket(salt, userKey string) uint32 {
    const (
        offset64 = 14695981039346656037
        prime64  = 1099511628211
    )
    h := uint64(offset64)
    for i := 0; i < len(salt); i++ {
        h ^= uint64(salt[i])
        h *= prime64
    }
    h ^= '.'
    h *= prime64
    for i := 0; i < len(userKey); i++ {
        h ^= uint64(userKey[i])
        h *= prime64
    }
    // murmur3 fmix64: FNV-1a alone mixes the last input bytes poorly,
    // and sequential user IDs differ exactly there.
    h ^= h >> 33
    h *= 0xff51afd7ed558ccd
    h ^= h >> 33
    h *= 0xc4ceb9fe1a85ec53
    h ^= h >> 33
    return uint32(h % BucketScale)
}

FNV-1a is inlined rather than taken from hash/fnv so evaluation stays allocation-free, and the finalizer is there because the realistic worst case for any rollout hash is sequential IDs: user-1, user-2, ... differ only in their last byte or two. The distribution test hashes 200,000 sequential IDs into 100 bins and checks a chi-square statistic against the 0.1% critical value (~148.2 at 99 degrees of freedom); it comes in at 88.1. The '.' separator is there so salt "ab" with key "c" can't collide with salt "a" with key "bc".

Three properties follow from how the bucket is used, and each one is a test:

Increasing a rollout never removes anyone. A rollout is an ordered list of ranges over [0, 100000), and a user is served the first range their bucket falls under. Going from 10% to 20% moves the boundary from 10,000 to 20,000; a user whose bucket was under 10,000 is still under 20,000. That sounds obvious, but it's the whole reason a rollout can be ramped without users flipping back and forth between old and new behavior on every step.

Order is data. The flip side of ordered ranges: rewriting [{true, 50%}, {false, 50%}] as [{false, 50%}, {true, 50%}] keeps the same weights and moves every single user to the other side. The test asserts exactly that, all 200,000 users flip. A control-plane UI that sorts rollout entries alphabetically "for readability" is a production incident.

Never range over a map to pick a variant. The same ordering argument is why Rollout is a slice, not map[string]int. A common way to write multi-variant assignment is:

// Broken: Go randomizes map iteration order.
var cumulative float64
for variant, weight := range test.Variants { // map[string]float64
    cumulative += weight
    if percentage < cumulative {
        return variant
    }
}

The hash is stable, but the order of the ranges is not: the Go spec leaves map iteration order unspecified, and the runtime deliberately randomizes it. The same user, with the same bucket, can get control on one call and variant-b on the next. With an experiment, that doesn't fail loudly; it quietly contaminates both arms of the result.

Salt per flag. If every flag hashed only the user key, the users with the lowest buckets would be in the first 10% of every rollout. The same small group would see every new feature first, and any two experiments running at once would have perfectly correlated populations. Defaulting the salt to the flag key decorrelates them: in the test, being in flag A's first 10% predicts flag B's first 10% about 1% of the time, which is what independence looks like. Setting the same explicit salt on two flags is the opt-in for when that correlation is actually wanted, for example two flags that must turn on for the same users together.

Rollouts also need a key to hash. If a rollout is reached and the user has no key, the evaluator returns NO_USER_KEY with no variation, and the caller's default is used. The alternative - hashing the empty string - puts every anonymous request into the same bucket, so a 10% rollout is either 0% or 100% of anonymous traffic depending on where that one bucket happens to land.

Validate the Whole Config Before It Goes Live

Every check that can fail runs once, when a config is loaded, in Compile: unique flag keys and variations, every served variation exists, rollout weights are non-negative and sum to exactly 100,000, every operator is known, every regex compiles, every semver constraint parses. The output is an immutable Snapshot with regexes already compiled and semver values already canonicalized, so the evaluation path has no error returns and no parsing.

func compileServe(s Serve, variations []string) (compiledServe, error) {
    switch {
    case s.Variation != "" && len(s.Rollout) > 0:
        return compiledServe{}, errors.New("serve has both variation and rollout")
    case s.Variation != "":
        if !slices.Contains(variations, s.Variation) {
            return compiledServe{}, fmt.Errorf("unknown variation %q", s.Variation)
        }
        return compiledServe{variation: s.Variation}, nil
    case len(s.Rollout) > 0:
        cs := compiledServe{rollout: make([]bucketRange, len(s.Rollout))}
        upper := 0
        for i, w := range s.Rollout {
            // ... unknown variation, negative weight
            upper += w.Weight
            cs.rollout[i] = bucketRange{upper: uint32(upper), variation: w.Variation}
        }
        if upper != BucketScale {
            return compiledServe{}, fmt.Errorf("rollout: weights sum to %d, want %d", upper, BucketScale)
        }
        return cs, nil
    default:
        return compiledServe{}, errors.New("serve is empty")
    }
}

Two decisions here are worth defending.

One bad flag rejects the whole config. Skipping just the broken flag and applying the rest sounds more resilient, and it produces a state nobody published: maybe the broken flag and a good one were edited together because they depend on each other. All-or-nothing keeps "the fleet runs version N" a true statement. The cost is that one typo blocks unrelated changes until it's fixed, which is exactly the pressure that gets typos fixed quickly. The control plane should run the same Compile before it publishes, so most bad configs never leave it.

Weights must sum to exactly 100%. An implementation that tolerates a sum under 100% has to decide what happens to the leftover buckets, and whatever it decides (serve the last variation, serve the off variation, serve nothing) is a behavior nobody chose deliberately. Rejecting the config makes the operator say what they meant.

Getting Config to Every Instance

An immutable snapshot behind an atomic pointer

Evaluations happen on every request; config changes happen a few times a day. That ratio decides the concurrency design. The active config is a pointer to a Snapshot that nothing mutates after Compile returns it. Readers do one atomic load; an update builds a complete new snapshot and swaps the pointer.

type Client struct {
    snap atomic.Pointer[Snapshot]
    mu   sync.Mutex // serializes Apply; readers never take it
}

func (c *Client) Apply(s *Snapshot) error {
    c.mu.Lock()
    defer c.mu.Unlock()
    cur := c.snap.Load()
    switch {
    case s.version == cur.version:
        return nil
    case s.version < cur.version:
        return fmt.Errorf("%w: got %d, active %d", ErrStaleConfig, s.version, cur.version)
    }
    c.snap.Store(s)
    return nil
}

// Snapshot returns the active snapshot. Use it to evaluate several flags
// against one consistent config, e.g. for the duration of one request.
func (c *Client) Snapshot() *Snapshot { return c.snap.Load() }

This gets several things for free that a map behind an RWMutex, updated key by key, has to work for:

• No torn reads. An evaluation sees either the old config or the new one. TestClient_ConcurrentApplyAndEvaluate publishes 2,000 versions in which flags a and b always serve the same value while eight goroutines evaluate both from one snapshot, and fails if they ever disagree. It runs under -race.
• Per-request consistency. Grab c.Snapshot() once at the start of a request, evaluate every flag against it, and a config update mid-request can't give one request a mix of old and new answers.
• No read-side contention. Every RLock is an atomic add on a shared counter, so all cores fight over one cache line. Measured below: the RWMutex version gets slower per operation going from 1 to 10 cores, while the atomic snapshot gets faster.

Versions only go forward. A cache or CDN in front of the control plane serving a stale copy, or a poller racing a push, can't roll the client back; it gets ErrStaleConfig and a log line.

Polling with ETags, and never clearing on failure

func (p *Poller) Poll(ctx context.Context) (changed bool, err error) {
    req, err := http.NewRequestWithContext(ctx, http.MethodGet, p.URL, nil)
    if err != nil {
        return false, err
    }
    if p.etag != "" {
        req.Header.Set("If-None-Match", p.etag)
    }
    resp, err := p.HTTP.Do(req)
    if err != nil {
        return false, err
    }
    defer resp.Body.Close()

    switch resp.StatusCode {
    case http.StatusNotModified:
        return false, nil
    case http.StatusOK:
    default:
        return false, fmt.Errorf("unexpected status %s", resp.Status)
    }

    body, err := io.ReadAll(io.LimitReader(resp.Body, maxConfigBytes+1))
    if err != nil {
        return false, err
    }
    if len(body) > maxConfigBytes {
        return false, fmt.Errorf("config larger than %d bytes", maxConfigBytes)
    }
    // Same ETag means the same bad body - no point re-parsing it every tick.
    p.etag = resp.Header.Get("ETag")

    snap, err := ParseConfig(body)
    if err != nil {
        return false, err
    }
    before := p.Target.Snapshot().Version()
    if err := p.Target.Apply(snap); err != nil {
        return false, err
    }
    return snap.Version() != before, nil
}

Every failure path returns an error and does nothing else. Network error, 5xx, a body over the size cap, invalid JSON, a config that fails validation, a stale version: the active snapshot stays exactly as it was. TestPoller_KeepsLastGoodConfig walks through each of those in sequence against an httptest server and checks the flag still serves the last good value after every one. An unchanged config costs one conditional GET and a 304, with nothing parsed.

Run polls once immediately, then on an interval with ±10% jitter, because a fleet that deployed at the same moment otherwise polls in lockstep forever. It stops when its context is cancelled; there's no goroutine that outlives the poller.

Polling versus a push channel (SSE, a long-lived gRPC stream, pub/sub) is a latency trade-off, not a correctness one. With polling, the worst-case propagation delay is the interval plus jitter. For release toggles that's irrelevant. For a kill switch it's the number that matters: if a broken feature has to be off everywhere within seconds, either poll on that order or add a push channel in addition to polling. Push alone has a failure mode polling doesn't - a dropped subscription that nobody notices, leaving an instance on an old config indefinitely - so the periodic full fetch stays as the backstop either way.

Defaults and Failure Modes

The flag system can't know the safe answer for a flag it doesn't have. The call site can:

// New feature: if anything is wrong with flags, don't show it.
if ff.Bool("new-checkout", user, false) { ... }

// Kill switch for something that already ships: if anything is wrong
// with flags, keep it running.
if ff.Bool("search-enabled", user, true) { ... }

A global "unknown flags are off" rule gets the second case backwards. On a fresh instance that hasn't fetched config yet, or after someone deletes the flag before the code that reads it, every kill-switched feature turns off. That's why Bool and Variation take the default as an argument instead of hard-coding one, and why Bool returns that default for any variation other than the literal strings "true" and "false" - a flag misconfigured with variations "yes"/"no" behaves like a missing flag, not like false.

The full list of what happens when something is wrong, each pinned by a test:

The first row has a real cost: for the first poll after startup, the instance is serving defaults, not the published config. If that window is unacceptable, ship a bootstrap config with the binary or block readiness on the first successful poll. Which one is right depends on whether "slow to start when the control plane is down" or "briefly serves defaults" is the worse failure for that service.

Wiring It Into a Service

All of the above, put together in a service. The gist has this as cmd/demo: one process running a stand-in control plane that serves flags.json, and an HTTP app that uses three flags - a release flag, a kill switch, and a three-way experiment.

Startup. One Client per process, one Poller feeding it, both stopped by the same context as the rest of the service:

ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()

ff := flags.NewClient()
poller := &flags.Poller{
    URL:      "http://" + *cpAddr + "/flags",
    HTTP:     &http.Client{Timeout: 5 * time.Second},
    Interval: 2 * time.Second,
    Target:   ff,
    Log:      log,
}
go poller.Run(ctx)

mux := http.NewServeMux()
mux.HandleFunc("GET /checkout", app.checkout)
srv := &http.Server{Addr: *addr, Handler: withFlags(ff, mux)}

Per request. A middleware builds the User once from whatever identifies the caller (headers here; a session or a JWT in a real service) and pins the current snapshot to the request context. Every flag check in that request then sees the same config version, even if an update lands halfway through:

type requestFlags struct {
    snap *flags.Snapshot
    user flags.User
}

func withFlags(ff *flags.Client, next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        rf := &requestFlags{
            snap: ff.Snapshot(),
            user: flags.User{
                Key: r.Header.Get("X-User-ID"),
                Attrs: map[string]string{
                    "email":       r.Header.Get("X-Email"),
                    "country":     r.Header.Get("X-Country"),
                    "app_version": r.Header.Get("X-App-Version"),
                },
            },
        }
        // "missing" and "empty string" should both mean "unknown" to the rules.
        for k, v := range rf.user.Attrs {
            if v == "" {
                delete(rf.user.Attrs, k)
            }
        }
        next.ServeHTTP(w, r.WithContext(context.WithValue(r.Context(), ctxKey{}, rf)))
    })
}

In the handler. Each kind of flag from the table above, each with the default that's safe for it:

func (a *app) checkout(w http.ResponseWriter, r *http.Request) {
    ff := flagsFrom(r.Context())

    // Release flag: unknown or broken -> old checkout.
    page := "old checkout"
    if ff.snap.Bool("new-checkout", ff.user, false) {
        page = "new checkout"
    }

    // Kill switch: unknown or broken -> keep recommendations on.
    recs := "recommendations: off (kill switch)"
    if ff.snap.Bool("recommendations-enabled", ff.user, true) {
        recs = "recommendations: on"
    }

    // Experiment: log the exposure with the reason, or the result can't be
    // analyzed and "why did I see green" can't be answered.
    color, res := ff.snap.Variation("checkout-button", ff.user, "control")
    attrs := []any{"flag", "checkout-button", "user", ff.user.Key, "variation", color,
        "reason", res.Reason, "config_version", ff.snap.Version()}
    if res.Reason == flags.ReasonRuleMatch {
        attrs = append(attrs, "rule", res.RuleIndex)
    }
    a.log.Info("exposure", attrs...)

    fmt.Fprintf(w, "config v%d\n%s\n%s\nbutton: %s\n", ff.snap.Version(), page, recs, color)
}

A session against it, with new-checkout configured as in the data model example: on for @example.com, 25% for DE/AT on app version 3.2.0 and up, off for everyone else.

$ go run ./cmd/demo/main.go -config cmd/demo/flags.json &

$ curl -H 'X-User-ID: u-1' -H 'X-Email: [email protected]' 127.0.0.1:18080/checkout
config v1
new checkout                  # rule 0: staff email
recommendations: on
button: control

$ for u in u-2 u-3 u-4 u-5; do
    curl -s -H "X-User-ID: $u" -H 'X-Country: DE' -H 'X-App-Version: 3.4.0' \
      127.0.0.1:18080/checkout | sed -n 2p
  done
old checkout
new checkout                  # u-3 is in the 25%, and stays there
old checkout
old checkout

$ curl 127.0.0.1:18080/checkout          # no user ID
config v1
old checkout
recommendations: on
button: control               # rollout can't bucket: NO_USER_KEY, default used

Then the two operational cases. Set "on": false on recommendations-enabled in flags.json and bump version to 2: within one poll interval every response says recommendations: off (kill switch), without a restart or a deploy. Then publish a version 3 whose experiment weights don't add up to 100%: the poller logs

level=WARN msg="flag config poll failed, keeping last good config"
  err="invalid flag config: flag \"checkout-button\": fallthrough: rollout: weights sum to 50010, want 100000"
  active_version=2

and the service keeps serving version 2. The exposure log line per request is the raw material for both the experiment analysis and for answering "why did this user see that":

level=INFO msg=exposure flag=checkout-button user=u-2 variation=green reason=FALLTHROUGH config_version=1
level=INFO msg=exposure flag=checkout-button user="" variation=control reason=NO_USER_KEY config_version=1

For a quick look at the API without HTTP, ExampleClient in example_test.go shows the same lifecycle in thirty lines: defaults before any config, a config applied, and the reason behind a result. It runs as part of go test.

What It Costs, Measured

Apple M5 (10 cores), macOS 26.6, Go 1.27.1, go test -bench . -benchmem -count=5 -cpu 1,10. Medians of five runs; full output is in BENCHMARK.md in the gist.

The one allocation in the semver case is "v" + value: golang.org/x/mod/semver requires the v prefix and the user's attribute usually doesn't have it. Passing app_version with the prefix already on removes it. The regex row is the most expensive operator by a clear margin, which is a reasonable argument for preferring in or ends_with whenever they can express the same targeting.

These are RunParallel numbers, so ns/op is wall time divided by total operations across all goroutines: lower at 10 cores means more throughput. The atomic snapshot scales with cores; the RWMutex version gets slower per operation as cores are added, because every reader increments the same counter. Same evaluation code in both - only the guard differs.

A per-(flag, user) result cache is a common addition to flag evaluators, and it doesn't pay for itself here. The cache key alone - strings.Join of flag key, user key, and the attributes rules might read - costs 47 ns and one 32-byte allocation, which is more than most evaluations in the first table, before the cache lookup, its lock, its eviction, or its invalidation on every config change. It also has a correctness trap: the key has to include every attribute any rule might read, and a rule added later that reads a new attribute silently serves stale results to anyone whose cached key didn't include it.

Compiling a 500-flag config (two rules and a rollout each, one of the rules a regex) takes about 1.1 ms and 2.2 MB of allocations. It runs once per config change, off the request path.

What these numbers don't say: anything about a real application's request rate, how many flags a request evaluates, or what fraction of request latency flags represent in practice. They establish that evaluation is cheap enough not to need caching and that the read path scales with cores, not more than that.

Removing Flags

The evaluator is the easy part. Most of the costs listed earlier come from flags that should have been deleted and weren't, and the reason they weren't is almost never technical: removing a flag that already works at 100% has no visible payoff, so it never wins against feature work. What helps is making removal the default instead of a good intention:

• An owner and an expiry date at creation. Release and experiment flags get both when they're created, in the flag's metadata. A flag past its date shows up in a weekly report or fails a CI check; permanent kinds (kill switches, permissions) are marked as permanent explicitly, so "no date" never means "forgot".
• The removal task is created with the flag. The ticket to delete the flag and the old path exists from day one, in the same backlog as the feature.
• Remove in the right order. First the code: delete the flag check and the old path, deploy everywhere. Then the flag from the config. Doing it the other way round leaves running code that evaluates a flag that no longer exists and falls back to its default - which for a release flag at 100% is the old behavior.
• Let evaluations show what's dead. The Reason field helps here. A flag whose evaluations have all been FALLTHROUGH to the same variation for weeks is a deletion candidate, and an UNKNOWN_FLAG in logs is a call site that outlived its flag.
• Limit how many exist at once. A per-team cap on active release flags forces the question "which one can go" before a new one is added.

Experiment flags need one more thing before they're removed: exposure logging, a record of which variant each user actually saw, written at evaluation time. Without it the experiment can't be analyzed, and once the flag is deleted the assignment can't be reconstructed.

What This Doesn't Cover

• The control plane. Editing UI, auth, approvals, audit history. The only requirements this evaluator puts on it are: publish the whole config with a version that only goes up, and run the same Compile before publishing.
• Experiment analysis. Assignment is solved above; measuring whether a variant won is a statistics problem with its own failure modes (peeking, sample-ratio mismatch, novelty effects).
• Client-side flags in browsers and mobile apps. Shipping the full rule set to a client exposes every targeting rule and every attribute value in it. Those clients usually get pre-evaluated results from a server instead.
• Large segment lists. in with a handful of values is a linear scan and that's fine. Targeting a list of a hundred thousand user IDs wants a set, or a separate segment service, not a giant clause.

The library in the gist is around 600 lines plus tests and a ~180-line demo service, uses only the standard library and golang.org/x/mod, and passes go test -race with 95.8% statement coverage.