Experimentation

Message testing log

Message testing log is licensed CC BY 4.0. Attribution: Katafacts (katafacts.com).

1 · What it is

What it is

A message testing log is a real, checkable answer to 'does this pillar actually resonate broadly, or did it just work once' — the learning log's exact discipline, pointed at a message architecture's standing pillars instead of a shop-floor hypothesis. A pillar proven in a single deal isn't the same as a pillar tested across a real, broader sample of buyers — this log tracks every message test run against each pillar (a discovery-call reaction, a structured pilot, a positioning survey) and honestly records whether the result recommends keeping the pillar as-is, revising it, or killing it.

2 · When to use it

When to use it — and when not to

Use it when

A message architecture exists and you want to test whether its pillars actually resonate with buyers broadly, not just in the one deal that first proved each one.
A pilot or a set of discovery calls has already run to test a specific message, and you want to record the real verdict — keep, revise, or kill — not just that it 'went well.'
You want to know honestly which pillars have never actually been message-tested, rather than assuming every pillar in the standing message is equally validated.

Not when

You don't have a message architecture yet — build that first; this log tests pillars that already exist, it doesn't define them.
You're checking whether a specific claim has real evidence behind it, not whether the message itself resonates — that's a proof point inventory, a different real question.

3 · How to fill it in

How to fill it in

Scope: What set of message pillars does this log cover?
Hypotheses: List every pillar from your message architecture you want tested, even ones you haven't tried yet.
Tests run: For each test: which pillar it relates to, the test type, what actually happened, and your real recommendation — keep, revise, or kill — your own facts, never guessed.
Narrative: Drafted from the log's own computed coverage.

4 · What good looks like

What good looks like

The example below continues the Beacon Analytics storyline directly from its own message architecture — the cost-of-inaction pillar's reference figure holds up in discovery calls, and the spreadsheet-trust pilot the architecture's own next steps called for surfaces a real, specific objection worth revising the pitch for, while the third pillar stays honestly untested.

Same example, as a downloadable xlsx workbook.

Download .xlsx

Learning log

Beacon Analytics — message testing log

Tests whether Beacon Analytics' standing message pillars actually resonate broadly with buyers, not just in the single deal that first proved each one.

Priya Anand, RevOps · 2026-04-25

Scope

Tests whether Beacon Analytics' standing message pillars actually resonate broadly with buyers, not just in the single deal that first proved each one.

Hypotheses

Real-time deal-risk visibility resonates as Beacon Analytics' strongest pillarNever tested
A quantified cost-of-inaction resonates with economic buyers
Buyers trust Beacon Analytics over their existing spreadsheet

Tests run

A quantified cost-of-inaction resonates with economic buyers

keep

Tested the $310K reference figure in 5 discovery calls to see whether economic buyers found it credible or dismissed it as marketing puffery

4 of 5 buyers found the figure credible once shown it traced to a real, named account rather than a projection; one buyer asked directly for their own pipeline to be modeled instead of relying on someone else's number

interview · Sam Cole, Sales Enablement — discovery call notes · 2026-04-18

Buyers trust Beacon Analytics over their existing spreadsheet

revise

Ran the structured pilot the message architecture's own next steps called for — 6 prospects used Beacon Analytics side-by-side with their existing spreadsheet for two weeks, then were surveyed on trust

4 of 6 prospects reported trusting Beacon Analytics' numbers as much or more than their spreadsheet by week two, but 2 remained skeptical, specifically citing no visible audit trail for how the risk score was calculated

pilot · Priya Anand, RevOps — pilot survey results · 2026-04-22

Narrative

The cost-of-inaction pillar's $310K reference figure held up directly with buyers — four of five discovery calls found it credible once they saw it traced to a real account rather than a projection, though one buyer specifically wanted their own numbers modeled instead of borrowing someone else's, worth building into the pitch. The spreadsheet-trust pilot — exactly the test the message architecture's own next steps called for — surfaced a real, specific objection: prospects want to see how the risk score is calculated, not just be told to trust it, so this pillar needs a real revision (an audit-trail explanation) before it stands alongside the other two as proven, not a straight pass. The real-time-visibility pillar, proven once in the Northline Freight deal itself, hasn't been message-tested more broadly at all yet — proven in one deal isn't the same as validated as a message across a wider buyer audience.

5 · Common mistakes

Common mistakes

Treating a pillar as validated everywhere because it worked in the one deal that first proved it.
A pillar proven once is a hypothesis worth testing broadly, not a settled fact — this log exists specifically to catch pillars that quietly never get tested past their first win.
Rounding a mixed result up to a clean 'keep' because part of it went well.
A specific, named objection from even a minority of a test sample is real signal worth a revision — for example, buyers asking for proof of how a number is calculated, not just being told to trust it.
Skipping a pillar in the log because it already feels obviously true.
An untested pillar can't be flagged as a gap — this log only protects the standing message from an unproven claim if every real pillar is actually tested, not just the ones that feel uncertain.

6 · What it connects to

What it connects to

upstream

Message architecture
The architecture's own pillars are exactly the hypotheses this log tests — including acting directly on the specific next step the architecture itself already named.

downstream

Proof point inventory
A pillar this log confirms resonates broadly is exactly what the proof point inventory should be actively collecting more evidence for — a tested message and a growing body of proof are two different, complementary jobs.

7 · Where AI helps

Where AI helps

Judgement — stays yours

Deciding whether a result actually supports keep, revise, or kill
Deciding which untested pillar to prioritize next

Analysis — AI helps

Tightening a test description from rough notes into a specific, checkable statement
Drafting the narrative from the log's own computed coverage

Drudgery — automated

Identifying which pillars have no tests logged
Identifying which logged tests recommend revise or kill
Exporting to xlsx in the house format

9 · Rate this kata