How It Works

How DeployMeter measures AI-assisted delivery — from PR analysis to DORA metrics.

Overview

DeployMeter measures how much faster your team ships with AI assistance by comparing two numbers for every merged pull request: how long the change would take at a fixed, published pace without AI — the baseline — and how long it actually took, from commit activity or from measured coding time.

It also tracks DORA metrics (deployment frequency, lead time, change failure rate, failed deployment recovery time) from the deployments you tell us about, incidents from the sources you connect, and review flow — one picture of engineering performance, all from pull requests.

PR Analysis Pipeline

Branch strategy assumed

We assume a trunk-based or GitHub Flow model: feature branches are opened as PRs and merged directly into main or master. Only merged PRs are analysed — open or closed-without-merge PRs are ignored.

What we skip

Dependabot, Renovate, and other bot PRs

PRs closed without merging

PRs authored by GitHub Actions or [bot] accounts

Merge commits inside a PR (counted in parent PR instead)

Actual time

Actual coding time

We read the commits on the pull request branch and group them into work sessions. The time between sessions is ambiguous — a pause, or work that left no commit — so actual time is a band: the low end counts only the sessions, the high end also counts the gaps, each gap capped. The point we report sits between the two. Merge commits and rapid amend/fixup commits are excluded. With measured coding time from the git hooks the band collapses to a single measured value.

Single-commit pull requests

A pull request with a single commit has no gaps to measure. Its actual time is a wide band, from a minimal session up to one capped session, and it is marked estimated. Measured coding time from the git hooks replaces the band.

Local Time Tracking (recommended)

Install git hooks per repo

For more accurate coding-time data, each developer should run the following command inside every tracked repository. It installs git hooks that capture branch switches, commits, and active coding sessions — then sends the data to DeployMeter on push.

npx deploymeter init

Installed hooks

post-checkout — records branch switches and starts a heartbeat for active coding detection

post-commit — records each commit with file stats

pre-push — sends all collected session data to DeployMeter

Baseline: DeployMeter Standard v1

The baseline is deterministic and versioned. Every file in the diff is classified by path and extension — logic, tests, schema, config, docs, generated — and each class has a fixed pace. The same diff gives the same baseline in every organization, and every analysis records the version of the ruler it used. Nothing in your settings changes it.

The model contributes only two bounded factors: how complex the change is, and what share of the logic is boilerplate. It never sets the amount of work, it is not shown commit timestamps, and lines rewritten within the pull request count at half.

The baseline is a band as well: complexity one level down and up, and the boilerplate share swapped for the model's lower and upper bounds. A docs-only pull request has a band of width zero.

Every pull request card shows the calculation line by line. If it is wrong, contest it there: the ruler is corrected for everyone, never overridden per organization.

AI Multiplier

The core metric

multiplier = baseline minutes ÷ actual minutes

Shown as a band whenever the actual time is a band: baseline low over actual high, and baseline high over actual low.

A multiplier of 3x means the change shipped in a third of the baseline time. Team and organization numbers are the median over at least 5 analyzed pull requests, shown with the P25–P75 range; below 5 the dashboard says so instead of showing a number. Time saved is winsorized so a single outlier cannot carry the total.

Deployment Tracking

DeployMeter tracks actual deployments to power accurate DORA metrics. You can choose how deployments are detected in Settings → Tracking.

PR Merge (default)

Every merged PR is treated as a deployment. Best for trunk-based development where merging to main automatically deploys.

GitHub Actions

Deployments are detected from your deploy workflow runs (e.g. deploy or release pipelines) — successful runs and failed ones alike, because a deploy that failed is still a change that was attempted. A failed run counts in the change failure rate; it only opens an incident when you give it a severity in Settings. Multiple PRs can be grouped into a single deployment. Requires the GitHub App to have Actions: Read permission and the workflow_run event subscription.

GitHub / GitLab Deployments

Deployment status events from GitHub and deployment events from GitLab environments, for the environments you name (production by default). The only source that tells a failed CI run from a failed production deploy; covers Vercel, Heroku, Render and workflows or pipelines with an environment.

External Webhook

External CI/CD tools (Jenkins, Octopus Deploy, etc.) send deployment notifications via a signed webhook. PRs merged since the last deployment are automatically grouped.

Configuration

Go to Settings → Tracking → Deployment Detection to choose your mode and configure workflow patterns or webhook secrets.

DORA Metrics

We track the five DORA metrics using deployment records. The deployment detection mode (configurable in Settings) determines how deployments are counted.

Deployment Frequency

Number of successful deployments per day, whichever detection mode you use. In PR Merge mode each merged PR counts as one deployment; in GitHub Actions, GitHub / GitLab Deployments or Webhook mode they come from actual deploy events.

Lead Time for Changes

Median hours from the first non-merge commit to production (or to merge when the repository has no deploy pipeline).

Change Failure Rate

Share of deployments — all outcomes, not only the successful ones — that failed, or had an incident attributed to them (inferred and then confirmed or rejected by you). Incidents with no deployment attributed stay out of this number. A deployment flagged by more than one source counts once.

Failed Deployment Recovery Time (FDRT)

Median time to recover the deployments counted in the failure rate, from the failure or the incident opening to the fix. Renamed from MTTR per DORA 2025.

Rework Rate

Share of the period's successful deployments that recover an earlier deployment of the same repository within 7 days — a deploy carrying a revert, a hotfix or an incident-labelled PR. Recoveries you mark as a false positive on the Incidents page never count, and a revert inside a deploy of 4 or more PRs only counts once you confirm it.

Incidents & Uptime

Where incidents come from

Incidents come from the sources you connect: GitHub issues carrying your incident label or issue type, PagerDuty, Opsgenie or custom webhooks — and, only if you give it a severity in Settings, a failed workflow or a failed production deployment. A failed deploy on its own counts in the failure rate, not as an incident. Revert and hotfix detection proposes candidates on the Incidents page: they count unless you mark them a false positive — except a revert inside a deploy of 4 or more PRs, which counts only once you confirm it.

How to connect an incident source

Go to Settings → Tracking → Incident sources
Turn on the sources you use: the GitHub issue label or issue type, the incident webhook, and optionally a severity for failed deploys
Point PagerDuty, Opsgenie or your own tooling at the webhook URL with the generated secret, or set the label your team already puts on incident issues
Review the revert and hotfix candidates on the Incidents page — a confirmed one counts towards the metrics, a dismissed one never does

Uptime

Uptime is the share of the selected period without an open incident at or above the severity you set as downtime — SEV1 only by default. Incidents without a severity never count as downtime. It is a proxy for availability, not a monitoring feed, and it has no industry rating.

AI ROI

The AI ROI puts three things side by side, for the same scope and period: velocity (hours saved, from the AI Multiplier), cost (what the AI tools cost) and stability (whether rework and failures went up).

AI ROI = (hours saved × hourly cost) ÷ AI cost

The hourly cost is the blended, loaded cost of an engineering hour that your organization sets — there is no default, and without it no ROI is shown. The result is a multiple (each unit of AI spend returned so many times its value in engineering time) with the low–high band of the hours saved, never a percentage.

AI cost is the variable spend (tokens, usage, premium requests) plus the prorated share of subscriptions and seats. Each cost is kept in its original currency and converted when read, with the European Central Bank reference rate of the day it was spent.

A person's cost belongs to the scope of their team. Cost with no known owner — a shared API key, an organization-wide subscription, an unassigned seat — stays at the organization and is never spread across units.

When an ROI is shown

At least 5 analyzed pull requests in the period — the same minimum as the AI Multiplier.
Cost recorded on at least 14 days of the period.
For a unit or team: at least 60% of the organization's cost attributable to a unit or team (people in a team that belongs to a unit, or subscriptions assigned to a unit). Below that, only the organization-level ROI is shown.
At least 80% of the pull requests measured with the current baseline; below that no ROI is shown, because its band would mix two rulers.

Below any of these the number is not shown: the page says which condition is missing instead of showing zero.

Stability verdict

Compares rework rate and change failure rate with the previous period of the same length. Stable: neither rose more than 2 percentage points. Degrading: one rose more than 2 and at most 5. Unstable: one rose more than 5, or the change failure rate is in the DORA Low band. Without a real incident source, rework rate decides alone.

The verdict sits next to the ROI with the same weight and is never multiplied into it: a single combined score would hide exactly the trade-off worth seeing.

Where the cost comes from

Claude Code Analytics (Anthropic Admin API): estimated daily cost per person and model.

Cursor Admin API: usage events and spend per team member.

GitHub Copilot: seats, read through the GitHub App and priced per plan, plus the usage metrics reports for adoption. Premium requests have no per-user endpoint — add them by CSV.

CSV upload and manual subscriptions: any other tool, invoice or plan.

The Anthropic cost report is read only to reconcile the Claude Code estimate. It never enters the totals, so nothing is counted twice.

Known limitations

The Copilot usage reports, the Cursor Admin API and the Anthropic Admin API usage report were implemented from their documentation and have not yet been checked against a live account; their figures may be corrected once they are.

AI agents' cost counts, but their time saved does not yet — a scope that uses agents understates its ROI.

The Copilot seats API has no history: days backfilled when Copilot is connected use today's seat count for adoption, and seat cost starts on the day DeployMeter first reads the seats.

Cost numbers below the organization are hidden from readers without individual cost visibility when they would reveal the spend of fewer than 3 people. Comparing breakdowns along different dimensions (by tool and by day) can still narrow an estimate in a very small organization.

Comparing AI coding assistants

DeployMeter can compare the work done with different AI coding assistants in the same organization. It compares groups of pull requests by the tool behind them, never people, and it is observational: it shows association, not cause.

Each indicator is standardized so that a group writing bigger or harder pull requests is not penalized: for every group, values are averaged over the same mix of PR size and kind of work (features, fixes, the rest). A size and work type needs at least 5 PRs in the group to count.

Every value comes with a 90% interval from a bootstrap that resamples developers, not PRs, so one prolific developer cannot make a group look certain. A group needs at least 20 PRs and 4 developers; below that the page says what is missing instead of showing a number.

Only one indicator per family gets a verdict (AI Multiplier, rework and cost per PR). It says faster, slower or no clear difference; it says faster or slower only when the interval of the difference excludes zero. The others show a value and an interval without a verdict, to avoid false findings from comparing many indicators at once.

Where the same developer has at least 5 PRs with each of two tools in the same size band, and at least 4 people do, the page also compares the tools with the person as their own control.

What it is not for: ranking or evaluating people. A group is a set of pull requests. Individuals appear only as the units behind the statistics and are never shown, and groups with too few people are hidden.

Security findings

DeployMeter reads the alerts GitHub already raises — Dependabot, code scanning and secret scanning — and measures how many are open, how long they take to fix and how many are past their SLA. It does not scan your code.

Open findings are counted per day from the moment GitHub detected them until they are fixed or dismissed; a source only counts while it is enabled on the repository. A scope with no source enabled says “not enabled”, never “0 vulnerabilities”.

Time to fix (TTR) is the median time from detection to fix for findings fixed in the period. It is not the MTTR of deploys (the time to recover from a failed deployment): the two answer different questions. Dismissed findings are not fixes; they show as a dismissal rate.

Default SLA: critical 7 days, high 30 days, medium 90 days; low has none. An owner or organization-wide lead can change it in Settings → Security, and every number recalculates.

A time to fix needs at least 5 fixed findings in the period; with fewer it says so instead of showing a number.

What it does not measure

Age counts from when GitHub detected the finding, not from when the vulnerability entered the code.

Who introduced a vulnerability is not measured, and there is no ranking of people.

Findings are not deduplicated across tools: the same issue reported by two scanners counts twice.

Code scanning counts only the default branch, and needs an analysis in the last 30 days to count as enabled.

Findings found in the first 72 hours after a source is enabled are existing debt: they count as open, but not as new.

Maintainability

The Maintainability score (0–100, higher is better) measures how costly a repository is to change, from how it actually changed: four signals read from merged pull requests over the last 90 days (365 days for knowledge).

Unstable hotspots (35% of the score): the share of churn in files changed by at least 8 PRs and in the top 10% of the repository, whose measured rework is 20% or more. Full health at 5%, none at 40%.

Rework (30%): lines that rewrite recent work over all lines changed, in analysed PRs (at least 5). Full health at 5%, none at 30%.

Knowledge concentration (20%): the share of files with at least 3 human PRs in 365 days where one person wrote 80% or more of the changes. AI agents never count as owners. Full health at 10%, none at 60%.

Hidden coupling (15%): the share of active files that change together with a file in another top-level folder — in at least 5 PRs and 70% of the PRs of the less-changed file. PRs touching more than 50 files are ignored. Full health at 5%, none at 30%.

The score is the weighted average of the signals with enough data, with the weights renormalised; it needs 3 of them, including unstable hotspots or rework. Healthy from 75, Watch from 50, Costly below 50. The range shows how much the number depends on a few PRs.

A scope's score is the average of its repositories' scores, weighted by churn. With less than 50% of the churn in scored repositories it is not shown, only the list.

A repository changed by fewer than 3 other people is not shown, and the list of single-owner files only to readers with individual visibility.

What it is not

Not code complexity: it never reads the code, only how it changed.

Not a judgement on who wrote the code: there is no score per person.

Not the Maintainability Index (the formula from lines of code and cyclomatic complexity).

For Accurate Results

Developers should commit to their branch regularly, not squash everything into one commit at the end

Keep PRs focused — large multi-week PRs underestimate the multiplier

Weekend and off-hours commits are counted correctly (no business-hours filter)

Protect your main branch — require pull requests and disable direct pushes so every change is tracked correctly

Run npx deploymeter init in each repo so every developer gets precise coding-time tracking via git hooks

Create a free account to see the full calculation on every pull request.

Get started