← Blog

Feature flags in GitLab CI with dif validate

Feature flags in GitLab work the same way as any other code change: the flag is a file, the merge request is the review, and GitLab CI runs dif validate before the pipeline can pass. There is no separate flag dashboard to configure or reconcile against the repo.

Most write-ups of CI validation for flags assume GitHub. The job syntax, the pipeline triggers, and the setting that blocks a bad merge are different on GitLab. This post covers the .gitlab-ci.yml that runs dif validate, why it should run on merge request pipelines and not branch pipelines, and the merge request setting that turns a passing check into an enforced one.

Key Takeaways

  • dif validate runs as a normal GitLab CI job. No GitLab-hosted flag dashboard or extra service is required.
  • Scope the job to merge request pipelines with rules: - if: '$CI_PIPELINE_SOURCE == "merge_request_event"', so it runs once per merge request instead of once per push.
  • GitLab only blocks a merge on a failed job if Pipelines must succeed is turned on under the project’s merge request settings. A green pipeline with that setting off is advisory, not enforced.
  • The validator catches config errors (bad weights, missing owners, surface collisions) before the merge, not intent errors like a flag gating the wrong code path.
  • A coding agent opening a GitLab merge request runs the same dif validate command locally, so the merge request arrives pre-checked.

What dif validate checks

dif validate is a single pass over the flag files in your repo. It confirms variant weights total 100, the owner field is a real email, every referenced surface and attribute exists, and no two active experiments collide on one surface (diagnostic E007). It also scans your app source for dif("...") call sites and warns on orphans, so a flag nobody deletes stays visible in review.

$ dif validate
Checking dif/experiments/active/checkout-cta-copy.md ... ok
Checking dif/surfaces/checkout.md ... ok
Scanning src/ for dif() call sites ... 8 calls, 0 orphans
All checks passed.

The checks themselves are platform-agnostic. What changes on GitLab is how the job runs and what makes its result actually block a merge. The CI validation post covers each check in depth; this one covers the GitLab wiring.

A .gitlab-ci.yml for feature flags in GitLab CI

A complete job, paste-able into any repo that has dif init run in it:

stages:
  - validate

dif-validate:
  stage: validate
  image: node:20
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
      changes:
        - dif/**/*
        - src/**/*
  script:
    - npm install -g @dif.sh/cli
    - dif validate

Three details matter here. The image: node:20 line pins a Node runner, since @dif.sh/cli is an npm package. The changes: filter, documented in the CI/CD YAML reference, means the job only runs when a flag file or application source changes, so an unrelated merge request stays fast. The script: installs the CLI with the same command a developer runs on their own machine, so a green job on GitLab and a green check locally mean the same thing.

A failing run names what to fix, the same trace whether it runs in GitLab CI or at a developer’s terminal:

$ dif validate
Checking dif/experiments/active/new-checkout-copy.md ... FAIL
  E007: collision on surface `checkout`
    active experiments on this surface:
      - checkout-cta-copy
      - new-checkout-copy
    audiences overlap and neither declares exclusion_group
1 flag failed.

dif validate exits non-zero on any failure. GitLab CI treats a non-zero exit as a failed job, and a failed job turns the pipeline red on the merge request.

Merge request pipelines, not branch pipelines

The rules: block above scopes the job to $CI_PIPELINE_SOURCE == "merge_request_event". Without it, GitLab CI runs the job on every push to every branch, including branches with no open merge request yet. That duplicates the check and slows down pipelines nobody is reviewing.

Scoping to merge request pipelines means the job runs exactly once per merge request, against the merged result of the source and target branch. That matters for dif validate’s collision check (E007): it needs to see the surfaces and experiments as they will actually land, not as they sit on an unmerged feature branch. A reviewer opens the merge request, sees the pipeline, and sees the same validation result the merge will produce.

For a PM reading the merge request, this is the whole review surface: one pipeline, one status, green or red. For the engineer who wrote the .gitlab-ci.yml, the mechanism is rules: filtering on a CI/CD variable GitLab sets automatically. Neither has to trust the other’s half of the story.

Make the check required in GitLab merge requests

A passing pipeline on the merge request page does not, by itself, block a merge. GitLab lets a merge request merge with a failed or even a skipped pipeline unless a project setting says otherwise.

Turn that on under Settings > Merge requests > Merge checks, and enable Pipelines must succeed. With it on, a merge request with a failing dif-validate job cannot be merged, full stop, regardless of who approves it. Pair it with a protected target branch so the setting cannot be bypassed by pushing straight to main.

Turning this on is what makes dif validate matter. A CI job nobody enforces is a job someone skips under deadline pressure, and Pipelines must succeed takes that option away.

The tradeoff is the changes: filter from the job above. A merge request that touches neither dif/ nor src/ skips the job entirely, and GitLab treats a skipped job differently from a failed one. Check the project’s merge request settings for whether a skipped pipeline is still allowed to merge. If flags can live outside those two paths in your repo, widen the changes: list to match, or drop the filter and accept the slower pipeline.

What breaks when a flag merges unvalidated on GitLab

dif’s runtime is fail-safe by default: a broken flag resolves to the first variant and records nothing, so nobody sees a crash. That protects users. It does not protect the experiment.

Say two experiments both target device_type: mobile on the checkout surface, and the collision check never ran because Pipelines must succeed was off. The merge goes through. The build succeeds. A mobile user lands in both experiments at once. The completed_checkout metric moves, and there is no clean way to attribute which experiment moved it. Nothing in the deploy looked wrong. The problem surfaces later, in a results panel that does not add up.

dif validate catches that collision (E007) before the merge, the same way a type error caught in CI is cheaper than the stack trace it would have produced in production. The gap on GitLab is never the validator. It is a merge request setting left at its default.

A coding agent opening a GitLab merge request

dif init writes managed guidance into CLAUDE.md, AGENTS.md, and .cursorrules telling a coding agent to run dif validate after authoring or editing a flag file, before it opens a merge request. A short session looks like this:

Agent: Gating the new checkout button at 10% on mobile.

$ dif validate
Checking dif/experiments/active/new-checkout-button.md ... FAIL
  variant weights total 105, expected 100 (off=90, on=15)

Agent: Fixing the weight. Rerunning.

$ dif validate
All checks passed.

Agent: Opening the merge request.

The merge request that arrives has already been checked against the schema. GitLab CI runs the same command again on the server, so review time goes to the code path the flag gates, not the frontmatter. dif validate still only catches config errors, not intent errors; a valid flag can still gate the wrong button, and that call stays with the reviewer.

FAQ

Does GitLab have built-in feature flags? GitLab CI/CD does not include flag validation on its own. dif validate runs as a normal CI job against the flag files in your repo, with no extra service or dashboard beyond GitLab itself.

How do I validate feature flags in GitLab CI? Add a job to .gitlab-ci.yml that installs @dif.sh/cli and runs dif validate. Scope it to merge request pipelines with rules: - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' so it checks the merged result, not an unmerged branch.

Why doesn’t a green pipeline block a bad merge by itself? Because GitLab allows merging with a failed or skipped pipeline unless the project turns on Pipelines must succeed under Settings > Merge requests > Merge checks. A passing job is a signal until that setting makes it a gate.

Does dif validate need GitLab Premium or Ultimate? No. It runs as a plain CI job, so it works on GitLab Free, self-managed, or GitLab.com. The merge checks setting that enforces it is also available on GitLab Free.

What’s the difference between a merge request pipeline and a branch pipeline? A branch pipeline runs on every push to a branch. A merge request pipeline runs against the merged result of the source and target branch, which is the version dif validate actually needs to check for collisions across surfaces.

Getting started

Feature flags in GitLab need one job, one setting, and no hosted dashboard. The job runs dif validate on merge request pipelines, scoped with rules: so it checks the merged result. The setting, Pipelines must succeed, turns that job from advisory into enforced. Together they mean a broken flag file cannot reach main.

Install the CLI and scaffold a project:

npm install -g @dif.sh/cli
dif init

Then add the .gitlab-ci.yml job above, turn on Pipelines must succeed, and protect the target branch. See feature flags as code for the workflow this sits inside.

The CLI reference and the .md format docs cover the rest.