The "So What?" Test: Making Dashboards Actionable

Every number on a report should survive being asked “so what?” three times. Most survive once. A few survive twice. The ones that get to three are the reason anybody opens the thing.

This is a test you can run on an existing dashboard in about twenty minutes, and it will empty roughly half of it.

The “so what?” test, in three questions

Take any number on your report and ask, in order:

So what? — What does this actually mean?
So what? — Why should the reader care?
So what? — What should now happen differently?

If you run out of answers before the third, the metric belongs on a detail page or nowhere. That’s the whole method, and it comes from Avinash Kaushik’s “three layers of so what” test, written up on Occam’s Razor in 2010 and drawn from his 2007 book Web Analytics: An Hour A Day. His point was that most analytics reporting stops at the first layer and calls it insight. His own worked examples are worth knowing because they’re unforgiving: percentage of repeat visitors fails, top exit pages fails, conversion rate for top search keywords passes, task completion rate passes.

The version worth internalising: a number that survives one “so what” is data, two is analysis, three is a recommendation. Clients pay for the third one.

The four rungs the test is really climbing

It helps to name what each answer produces, because that is what tells you how far up you’ve got.

Four rungs rising from observation to interpretation to implication to action, with one finding — organic sessions down 38% — carried all the way up — The same finding, carried up four rungs. Most dashboards stop on the first.

An observation is what the chart shows: organic sessions are down 38% month on month. An interpretation is what is causing it: the drop is entirely on the twelve pages that changed URL in June. An implication is what it costs or wins: at current conversion rates that is roughly £18k of pipeline a month. An action is what somebody does and by when: three redirects, dev to ship by the 30th, recovery confirmed in the next report.

The rungs are also a diagnosis of your own reporting. A report that is all observations is a data feed. One that reaches interpretation is an analysis. Only the ones that get to implication and action are doing the job a consultant is hired for — and the jump from implication to action is where most reports stop, because it is the one that requires committing to a view.

Four marketing metrics run through the “so what?” test

Branded impressions fail at the second question

Branded impressions up 34%.

So what? More people saw us in search results for our own name.

So what? …because a competitor started bidding on our brand, or because we ran an offline campaign, or because Google changed how it counts. We don’t know which, and it stops here.

Not a bad metric — it just can’t get past layer two on its own. Give it a cause and it might.

Total sessions fail at the second question too

Sessions up 8% month on month.

So what? More people visited.

So what? …unclear. Traffic that doesn’t convert costs money to acquire. Without knowing which channel, which pages and whether it converted, “more visitors” isn’t yet good news.

This is the most-reported metric in marketing and it routinely dies at layer two. It gets to three only when it’s broken down and paired with an outcome.

Assisted conversions by channel pass all three

Organic assisted 41% of paid conversions, up from 29%.

So what? Organic is doing more of the work that paid gets credited for.

So what? Paid’s cost per conversion is flattered by organic’s contribution, so the channel comparison the budget is based on is wrong.

So what? Re-run the channel ROI with assists included before the next budget round, and hold the organic spend that looked marginal.

Three layers, ending in something somebody does on Monday. That metric has earned its slot.

Pages published against sessions gained pass all three

We published 12 blog posts and 4 resource pages; blog drove 3× the sessions.

So what? The cheaper content type is producing more traffic.

So what? The investment split doesn’t match the return split — we’re funding the thing that works less well.

So what? Move next quarter’s production budget toward blog, and re-scope what the resource pages are actually for.

Note that this one is uncomfortable, and that’s usually a sign it’s working. A test that only ever confirms your existing plan isn’t being run honestly.

Running the “so what?” test forwards, on a client goal

The same interrogation works on a target, and it’s more useful there because it happens before the work rather than after.

A client says: we want a 20% increase in organic traffic.

So what? — 20% of what, from where? Traffic from one market isn’t worth what traffic from another is.

So what? — “A 20% increase in organic traffic from the US, on mobile” is a different project entirely.

So what? — Which means the work is mobile usability, a mobile UX pass, content aimed at that market, and possibly digital PR on the platforms that audience uses.

Three questions at kickoff and you’ve a brief instead of a number. It also means the report almost writes itself, because you’ve already agreed which inputs matter — this is what mapping input metrics to the outputs they drive is for.

Running the “so what?” test backwards, to diagnose a drop

When something drops, “so what” becomes a diagnostic. Walk the chain rather than guessing.

Organic revenue is down:

Did transactions fall, or average order value?
If transactions — did conversion rate change?
If conversion rate held — did sessions change?
If sessions fell — which channels?
Then the ones people skip: stock, missing products, site errors in that window.

Keep going until you reach something somebody did or didn’t do. That’s the finding — everything above it is symptom.

That last step earns its place regularly. A revenue drop caused by a stock-out isn’t a marketing failure, and a report that can’t tell the two apart will have you defending work that was fine.

Auditing a dashboard you already have, chart by chart

Three outcomes from running the so-what test on a chart: wrote the sentence easily so the chart stays and the sentence becomes its title; struggled so the chart needs context; could not write it so the chart moves to a detail page — What happens to each chart depends on how hard the third answer was to write.

Open the report and go chart by chart. For each one, write the third “so what” in a sentence. Then:

Got a sentence easily? The chart stays, and that sentence is its title. Not “Sessions by channel” — the finding.
Struggled but got there? The chart stays and needs context: a comparison, a target, an annotation. It’s doing analysis without showing it.
Couldn’t? Move it to a detail page. It may be genuinely useful to a specialist and it isn’t earning space here.

Expect to move a lot. A dashboard is usually built by accumulation — someone asked for a metric once and it never left — and the test is the first thing that has ever asked each chart to justify itself.

What makes a dashboard actionable rather than merely detailed

“Actionable” gets used to mean “detailed”, which is backwards. Detail is what makes a dashboard exhaustive. Three things make it actionable:

Every number has context. A comparison, a denominator, or a related metric beside it. Without one, the reader has to supply the judgement, which is the job they were paying you for.

The finding is stated, not implied. In the chart title, in a text box, in the summary. If the insight only exists in your head during the call, the report doesn’t contain it.

Something is owned. An action with a name and a date attached. “Consider improving internal linking” isn’t an action; it’s a way of avoiding one.

What to do when the honest answer is that nothing happened

Some months nothing much happened. The temptation is to manufacture a finding — to promote a 4% wobble into a trend because the report needs a story.

Don’t. Say the month was stable, say what you’re continuing, and use the space to report progress on the work rather than the outcome. Doing that consistently is what makes people believe you when you do say something is significant.

A narrative the data doesn’t support is the one real hazard of getting good at this. A compelling story built on a correlation is more dangerous than a dull one built on a fact, precisely because it’s more persuasive.

When the third answer is “we don’t know yet”

There is a middle outcome the test throws up constantly and which neither passing nor failing describes properly: you can say what the number means and why it matters, and the honest third answer is that you can’t yet say what to do because you don’t know the cause.

That isn’t a failure of the metric. It’s a finding in its own right, and the way to report it is to make the investigation the action. “Branded impressions are up 34%. That’s either a competitor bidding on our brand, an offline campaign, or a counting change at Google’s end. I am checking the auction insights report and will have an answer by Thursday.” That has an owner and a date on it, which is what makes it an action rather than a shrug.

The failure mode to avoid is the opposite one — picking the most flattering of three possible causes and reporting it as the cause. A confident wrong attribution costs far more than an honest open question, because the client will remember it the next time you attribute something.

Running the test on input metrics, where it works differently

Input metrics — pages published, fixes shipped, links earned — often stall at the second question, and consultants sometimes conclude they don’t belong on a report. That conclusion is wrong, and the reason is worth being precise about.

We published twelve blog posts. So what? The content programme is running at the agreed rate. So what? …and there the chain stops, because the output hasn’t arrived yet.

The chain completes only when the input is reported against the output it is meant to drive, which is why the pairing matters more than the metric. Twelve posts published on its own is activity. Twelve posts published against sessions gained, with the expected lag stated, is an argument about whether the programme is working. The test isn’t telling you to drop input metrics; it is telling you never to report one alone.

A per-chart checklist to run before a report goes out

Per chart, before a report goes out:

Can I say what this means in one sentence?
Does that sentence say why it matters?
Does it end in something somebody should do?
Is the comparison on it?
If it’s surprising, is it labelled?
If the answer to all of the above is no — why is it on this page?

Frequently asked questions

What is the “so what?” test?

Asking “so what?” of a metric three times. The first answer tells you what it means, the second why it matters, the third what should happen differently. A number that survives one is data, two is analysis, three is a recommendation. It comes from Avinash Kaushik’s three layers of “so what”.

What makes a dashboard actionable?

Three things. Every number has context — a comparison, a denominator or a related metric. The finding is stated rather than implied, usually in the chart title. And something is owned, with a name and a date attached. Detail does not make a dashboard actionable; it makes it exhaustive.

What should a chart title say?

The finding, not the metric. “Organic traffic fell 12% after the March update” does more work than “Sessions by month” and costs nothing. If you change one thing about your reports, change the titles.

The "so what?" test — making a dashboard actionable