Skip to content

Metrics & analytics · Easy

What are guardrail metrics, and why do they matter?

Short answer

Guardrail metrics are measures you commit not to harm while optimising a primary metric. They protect user experience, trust and the wider business from side-effects. For example, a team pushing notifications to lift engagement should watch unsubscribes and uninstalls. Define guardrails and acceptable thresholds before launching a test, so the decision is not rationalised afterwards.

Detailed answer

Definition

A primary metric tells you whether your change achieved its goal. A guardrail metric tells you whether you caused damage somewhere else getting there.

Typical categories

  • User experience: error rates, page load time, crash rate.
  • Trust and wellbeing: complaints, unsubscribes, opt-outs, refund requests.
  • Business health: revenue per user, cost per order, support contacts.
  • Other teams' metrics: a change to search might affect recommendations.

Worked example

A team wants to increase repeat purchases by sending more promotional push notifications.

  • Primary metric: purchases per user over a set period.
  • Guardrails: notification opt-out rate, uninstall rate, and customer complaints mentioning spam.
  • Threshold: agree in advance what increase in opt-outs would block the launch, even if purchases rise.

If purchases rise but opt-outs climb sharply, the team has borrowed against the future: users who switch off notifications can no longer be reached for anything, including important updates.

How to use guardrails well

  • Pick three to five, not twenty.
  • Set thresholds before the test starts.
  • Make sure guardrails are measured with the same rigour as the primary metric.
  • Review them at the same cadence; a guardrail that nobody checks is decoration.

Common pitfalls

  • Choosing guardrails after seeing the results.
  • Listing guardrails with no thresholds, so any harm can be explained away.
  • Forgetting long-term effects such as churn, which may only appear weeks after the test ends.