The Moat Test

The opening question is deliberately narrow: if a model can summarise a transcript, what makes an AI meeting assistant worth paying for? It is a good case because the core capability is unambiguously commoditised. Transcript in, notes out, at a quality that would have been remarkable three years ago and is now a default.

The tempting conclusion is that the category is therefore in trouble. That conclusion skips a step. Reproducing a capability is not reproducing a business. Getting the audio in the first place, surviving a bad connection and four people talking over each other, landing the output where the team already works, and being trusted with the recording of a difficult conversation are all separate problems, and none of them are solved by the summariser.

So the decision is which of those remaining advantages are load-bearing. That question cannot be answered by an opinion about AI. It needs a challenger that actually runs, an honest account of where it breaks, and a separate argument about each advantage the challenger never touched.

The challenger takes a UTF-8 transcript with speaker labels and stable line IDs, and returns a short summary, decisions with a current/superseded/unresolved status, action items with an explicit owner and due date, open questions, and uncertainty flags. Every decision and action carries the line IDs it was derived from.

It is measured against a deterministic baseline named the tagged transcript baseline, which extracts only lines explicitly marked ACTION: or DECISION:. This is a low bar on purpose: it is the score to beat before a model-based method has demonstrated anything at all.

Three data modes are labelled everywhere output appears. Illustrative demo means hand-authored content and no performance claim. Recorded experiment means archived real output with run metadata attached. Live trial means the visitor deliberately ran a transcript through a configured provider — and a live result is never silently folded into the published benchmark.

The concept preview on this page is illustrative demo. The figures under Evidence and results are recorded experiment: they come from archived runs with their metadata attached, and the run record they are copied from is cited below.

The lab runs. It executes the deterministic baseline over the synthetic transcripts and archives each run with its metadata, and the blind comparison keeps both method identities hidden until a choice has been made. It is deployed and open to anyone, so a reader can run the blind comparison themselves rather than take my description of it.

Two runs have been recorded and archived, and that is the whole reason this case is labelled Measured, partial rather than Illustrative. On the held-out split — 12 synthetic transcripts — the tagged-line baseline returned 8 actions and all 8 of them matched the gold annotations, against the 33 actions those annotations record. It assigned the correct status to 1 of 14 matched decisions. Every one of its 22 extracted items carried a transcript line reference, and all 22 references resolved. Those figures are copied from the archived run record cited below, and they describe the deliberately unimpressive baseline, not a challenger.

The word partial is doing real work here. The corpus is synthetic and was authored for this benchmark. The gold annotations are drafts written by a coding agent and no person has reviewed them, so a recall figure measured against them is a measurement against an unreviewed draft. Whether each cited passage actually supports the claim attached to it is not measured at all, for any of the 22 items, because that requires someone to sit and read. And the thing this investigation is actually about — whether a model-based challenger clears this floor, and which commercial advantages survive if it does — has not been measured.

The evidence listed on this page therefore splits in two. The baseline, the output contract, and the concept schematic are design artefacts: they establish what would count as a result. The archived runs are a result, of a narrow and clearly bounded kind.

The position this investigation is built to test is that in categories where the core capability has commoditised, the durable advantage moves to reliable capture of the input and to placement in the workflow, and that pricing power follows whichever of those is hardest to replicate rather than whichever is most visible in the demo.

If the challenger reproduces most of the output quality on clean transcripts, that supports the first half of the position and says nothing about the second. If it fails badly even on clean transcripts, the commoditisation premise itself is weaker than the category's critics assume, and that is the more interesting outcome to publish.

The first investigation uses synthetic meeting transcripts. They are cleaner than real meetings, they were authored for this benchmark and depict no real meeting, and the gold annotations scored against them are drafts that no person has reviewed.

Audio capture and PDF parsing are out of scope, which means the prototype does not test the part of the value chain most likely to be defensible.

No commercial product is benchmarked, so nothing here is a claim about any named vendor's performance.

Live model output varies between runs. Reproducibility here means archived outputs with run metadata, not a promise that a future run returns the same text.

Have a person review the gold annotations and read every cited passage, so the recorded metrics stop being measurements against an unreviewed draft and citation support stops being unmeasured. Until that happens this benchmark can only honestly be reported as partial.

Then talk to three to five people who actually run these meetings about how notes reach their workflow, and report what they said with the sample size attached rather than as a finding about the market.