Goal Loops

A/B Experiment Watch

A closed loop for one experiment: it reads your deciding metric against a threshold through a decision window, then finishes with a plain verdict - ship it, kill it, or extend. If you only want to hear when a shipped change misbehaves, use the Follow-up Tracker instead.

Moderate setupShort cycleClosed loopBuilds over runs

Setup in Loopany

How a run flows

Before first run

Verify the deciding metric

On cadence

Daily through the window

Step 1 · Read

Read variant vs control

Step 2 · Judge

Against the threshold

Metric

Deciding metric — conversion %

Report

Latest reading

closed loop — it finishes itself once the goal is met

How this loop works — schedule, exit, notifications

Suggested schedule: Daily through the window (you set it)
Loop type: Closed loop — goal-bound; it finishes itself when the goal is met.
Exit condition: Finishes when the metric settles the question, or the decision window closes.
Notifications: On by default — you're alerted on each run's outcome (completion or failure). Silence with notify: never.
Ease of starting: Some setup and context are needed before the first useful run.
When you see value: Every run posts where the metric stands; the payoff is the verdict when the window closes.

The prompt this runs

Set up a closed loop that watches one experiment, or one specific change, all the way to a decision. Use this when something is already live and there is a number that decides whether it worked: an A/B test, a pricing or copy change, a new onboarding step. If there is no metric to judge against and you only want to hear when a shipped change misbehaves, use the Follow-up Tracker instead - that one alarms on problems, this one delivers a verdict.

## Step 1 - Pin down the experiment
Ask me which change or experiment this loop is for, and what it is measured against: a variant against control, or the period before the change against after. Write both down in plain words before going further.

## Step 2 - Verify the deciding metric
Agree on exactly one deciding metric (conversion rate, signups per day, error rate, revenue per visitor) and find a concrete way to read it on this machine: product analytics, an events or SQL query, a log query, an MCP tool, a URL, or gh. Run that read once as a smoke test and show me the number it returns. If the metric cannot actually be read, say so and stop - do not create a blind loop.

## Step 3 - Agree the threshold and the decision window
Together set the number that decides the outcome and the deadline: the threshold (for example "conversion up at least 0.8 points against control"), a guardrail metric that must not get worse if one applies, and the decision window (for example 14 days, or until the variant has 2000 exposures). Confirm both with me before creating anything.

## Step 4 - Create the loop closed
Create it as a closed loop whose goal is the verdict condition: the metric clears the threshold, or clearly fails it, or the window ends. Propose a cadence that fits how fast the metric moves - daily is a common default for a two-week experiment, and never check hourly on a metric that only moves weekly - and confirm it with me.

## Each run
1. Read the deciding metric, and the guardrail if there is one, through the verified path only.
2. Compare against the threshold and say where it stands: passing, failing, or still inside the window with too little data.
3. Write one dated `type: report` product with the current numbers, days elapsed against the window, and the current lean. Report the metric value as the run metric.
4. Never restart the experiment, never change the traffic split, never touch the feature flag. This loop reads and judges, it does not act.
5. Finish the loop the moment the answer is conclusive or the window ends, and state the verdict as one of three words plus a one-line reason: ship it, kill it, or extend. Extend only with a concrete new deadline and a reason the data is not yet conclusive.

Hard rules: never call an experiment early on a metric that is still noisy - report "not conclusive yet" instead. Never invent numbers; report only what the verified read returned, and if a read fails, say the read failed. Never copy credentials, tokens, or personal data into a report.

Task file to author, filling the placeholders from the steps above:

```
## Spec
Experiment: <what changed, and against what control or baseline>
Deciding metric: <metric>, read via <the verified command or tool>
Threshold: <the number that means it worked>
Guardrail: <metric that must not get worse, or none>
Decision window: <ends YYYY-MM-DD, or the exposure count>
Verdict: ship it / kill it / extend
```

Dashboard at create: a chart of the deciding metric over time with the threshold noted, plus an embed of the latest report.

This is the exact intent appended to the bootstrap when you create the loop — your coding agent reads it, then proposes cadence and config and confirms with you before creating anything.