--- name: attribution-reality-check description: "How much credit for an outcome shifts depending purely on which honest model you pick — first-attributed, last-attributed, or an even split — computed across every real contributor, with the ones most sensitive to model choice flagged plainly rather than settled by whichever one looks most convenient. Use when: Multiple changes or contributors could plausibly be credited for one observed outcome, and nothing about the timing alone proves which one actually caused it. You want to know honestly how much your current crediting convention would change if you picked a different, equally reasonable model — before betting a bigger decision on it. Not for: You already ran a real controlled experiment (a holdout, an A/B test) that isolated the actual cause — trust that over any of this tool's three models. There's genuinely only one contributor in play — there's no real attribution question to reality-check." --- # Attribution reality check How much credit for an outcome shifts depending purely on which honest model you pick — first-attributed, last-attributed, or an even split — computed across every real contributor, with the ones most sensitive to model choice flagged plainly rather than settled by whichever one looks most convenient. Family: Measurement · Format: .xlsx · Domain: core ## What it is An attribution reality check computes how much credit for one outcome shifts across three deliberately simple, honest models — first-attributed, last-attributed, and an even split across every contributor — and flags which contributors are robustly credited (all three models roughly agree) versus contested (the models disagree sharply). It doesn't resolve which model is correct — no spreadsheet can, without a real controlled experiment — it stops a single-model number from quietly passing as settled fact when it's really just one convenient choice among several equally defensible ones. ## When to use it Use it when: - Multiple changes or contributors could plausibly be credited for one observed outcome, and nothing about the timing alone proves which one actually caused it. - You want to know honestly how much your current crediting convention would change if you picked a different, equally reasonable model — before betting a bigger decision on it. - A follow-up plan (like rolling a fix out to a second area) is about to assume one specific contributor gets the credit, and you want that assumption pressure-tested first. Not when: - You already ran a real controlled experiment (a holdout, an A/B test) that isolated the actual cause — trust that over any of this tool's three models. - There's genuinely only one contributor in play — there's no real attribution question to reality-check. ## How to fill it in - **scopeArea** — What outcome, and what period, does this reality check cover? - **currentModel** — How is credit actually assigned today? Name the real convention in use, even if it's informal or nobody deliberately chose it — not the model you wish you had. - **cases** — For each outcome: its real value, and every contributor with the real order it occurred in — your own record, never guessed. Credit by model and the swing between them are computed for you. - **narrative** — Drafted from the computed swing, naming which contributors are robust and which are contested. ## What good looks like The example below continues the Line 2 storyline running through this catalogue's A3, scoping canvas, FMEA, and metric definition sheet examples — it's directly grounded in the A3's own two countermeasures, rolled out on the same date, and honestly names a real confound (a seasonal order-volume dip) rather than assuming the sequencing rule alone gets the credit its own follow-up plan already takes for granted. Full worked example, on-screen and as a downloadable .xlsx: https://www.katafacts.com/katas/attribution-reality-check ## Common mistakes - **Treating whichever model the org already uses as though it were validated, just because it's familiar.** — A familiar model is not the same as a checked one — the whole point of this tool is to see how much the number would change under an equally reasonable alternative. - **Picking the model that happens to support the story you already wanted to tell.** — If a swing exists, it exists regardless of which model you'd prefer to be true — naming the contested contributor plainly is what actually protects the next decision built on it. - **Treating a low swing on every contributor as proof the outcome is fully understood.** — Agreement across three simple models is reassuring, but it's still not a controlled experiment — it just means these three particular ways of looking at it happen to agree. ## What it connects to Upstream: - **Metric definition sheet** (`metric-definition-sheet`) — A metric definition sheet is where an unresolved crediting question first gets named as a real gap; this reality check is where that gap actually gets pressure-tested against alternative models. ## Where AI helps Judgement — stays with you: - Deciding whether a swing is genuinely worth flagging or within normal noise - Deciding what a real controlled experiment to settle a contested contributor would need to look like Analysis — AI helps: - Tightening the current-model description from raw notes into specific, checkable language - Drafting the narrative naming which contributors are robust and which are contested, from the computed swing Drudgery — automated: - Computing credit under all three models and the swing between them - Exporting to xlsx in the house format --- Attribution reality check is licensed [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). Attribution: Katafacts (katafacts.com). ## Using this skill Apply this method as part of whatever broader task, instructions, or deliverable you're already working on -- it's a method to use, not a standalone conversation to start. There is no generation tool for this catalogue yet, so draft the artifact's actual content yourself, following the fields above, the same way a person filling this out by hand would. The canonical guide page and a downloadable worked example (.xlsx) are at https://www.katafacts.com/katas/attribution-reality-check.