Measuring productivity

The Productivity dashboard does not show one productivity number. It shows the same data under five definitions, then again with the work type and complexity held constant, and lets you take the row that matches how your team counts. This page is the reasoning behind that shape, so the figures can be defended by whoever is asked to believe them.

Productivity is a ratio, and teams disagree about both halves

Productivity gain is Δ(Output ÷ Effort). Every team agrees on the formula and nobody agrees on the terms. One counts issues closed, another counts lines, a third counts commits; effort is person-days to one team and tool calls to another. Any dashboard that picks one definition has silently picked a side.

Refusing to name a number because the definition differs is itself a choice — it hides the disagreement rather than resolving it. The better answer is to compute the number under every reasonable definition, show them side by side, and let the spread be the finding. Five figures that range from +3% to −28% are more honest than one figure, and far more honest than silence.

The five definitions on the dashboard, all with effort in the denominator:

Output	Effort	what it rewards
commits	person-days	shipping often
lines	person-days	shipping volume
lines	commits	larger changes
files	commits	changes that reach across the codebase
files	person-days	breadth per unit of time

Every row compares AI-assisted commits against unassisted ones, shows both sides and their counts, and blanks the difference when either side has fewer than five commits.

Per-person ratios were drafted and cut. Commits per person read +137% where commits per person-day read +3% on the same data, because the people using the agent were active on 14.8 days against 6.4. That is more time spent, reported as more output per unit of time — the exact error the page exists to prevent, so it cannot appear on it. Every denominator is a measure of effort.

The conditions are part of the formula, not a caveat beneath it

Output ÷ Effort on its own rises when the work gets easier, when the team grows, or when a quarter happens to contain fewer incidents. The formula that means something is:

Δ(Output ÷ Effort), conditional on work type, complexity and quality

Two worked examples with the same headline totals show why. Both use issues resolved as output and cycle time as effort.

One. Both cohorts do the same kind of work at the same complexity.

features, 5–20 files touched
  assisted     48 issues / 2.1 days = 22.9 per day   fix rate 11%
  unassisted   52 issues / 2.8 days = 18.6 per day   fix rate 12%

Δ(Output / Effort) = (22.9 − 18.6) / 18.6 = +23.1%

A 23% difference, with the fix rate holding. Quality is the condition that stops this being a speed number in isolation: work that ships faster and comes back as rework has not moved the thing anyone cares about.

Two. The same totals, but the cohorts are not doing the same work.

assisted     docs, 1–2 files       34 issues @ 0.9 days
             features, 5–20 files  14 issues @ 4.9 days
             ------------------------------------------
             48 issues, blended 2.1 days = 23.2 per day

unassisted   docs, 1–2 files        9 issues @ 1.1 days
             features, 5–20 files  43 issues @ 3.2 days
             ------------------------------------------
             52 issues, blended 2.8 days = 18.3 per day

unconditioned Δ(Output / Effort) = +26.7%

features only
  assisted     14 issues / 4.9 days =  2.9 per day
  unassisted   43 issues / 3.2 days = 13.4 per day

conditioned Δ(Output / Effort) = −78.7%

The unconditioned figure says assisted work is 27% more productive. Split by work type, the same data says it is dramatically slower on features, and the aggregate was measuring nothing but the mix: the assisted cohort was mostly documentation, which closes in under a day whoever writes it.

This is Simpson's paradox, and it is the ordinary case rather than the exotic one, because people reach for an agent on some kinds of task and not others. So the dashboard computes the difference twice — pooled, everything against everything, and conditioned, like against like inside each work-type-and-complexity cell, reweighted onto a common mix — and shows both. The gap between them is how much of the pooled figure was the work each cohort chose rather than how well it was done. A grid of every cell sits beneath, so the composition is visible, not inferred.

Why the conditioned figure uses lines, not commits

Inside a single work-type-and-complexity cell, most contributor-days hold one commit. Commits per person-day then pins near 1 on both sides regardless of the work, and the ratio measures the shape of the grid rather than anything anyone did. Lines vary within a cell; commits do not. So the stratified comparison uses lines delivered per person-day, and the tooltip on the panel says so.

What "complexity" means here

The tool does not measure complexity. The grid uses files changed as the stand-in — one or two files, three to twenty, more — and says so in its tooltips. It is a coarse proxy, and it is the same proxy on both sides of every comparison, which is what makes the cells comparable.

Every figure is a difference, not a gain

Assisted and unassisted work are self-selected. People reach for an agent on some kinds of task and not others, so a raw throughput difference partly measures which work was chosen. Nothing on the Productivity dashboard is a controlled experiment, and every panel says "difference" rather than "gain" or "improvement".

The cohort sizes and the purpose breakdown are on the page precisely so that this can be controlled for by the reader, rather than left as an unstated assumption inside a single figure. Apply your own framework and you get a number you can defend, because you know what went into it. That is the part that matters when somebody asks how it was calculated.

Within-person comparison

The strongest control the data supports is a person against themselves: each contributor's assisted commits against their own unassisted ones, lines per active day on each. Holding the person constant removes the confounder the cohort comparison cannot touch — who chose to use the tool. If most bars on that panel point the same way, that is far more defensible than any pooled figure. It still needs five commits a side per person, and it is still a difference between two self-selected sets of that person's work.

Where to read it

The Productivity dashboard carries all of this. The Executive Summary shows the cycle-time comparison; the Delivery Forecast shows throughput per head and per task. All three say "difference" throughout.