Why most programmes measure the wrong things and where to start instead
Ant Davis
Most awareness programmes measure the wrong things
Training completion. Phishing click rate. Maybe a survey at the end of the year if you're feeling ambitious.
These are the metrics most programmes run on, and I get why. They're easy to collect, easy to report, and they look clean in a slide deck. For a few years, this is exactly what I did.
The problem is they don't actually tell you whether your programme is working.
A hundred percent completion rate means everyone clicked through the training. It doesn't mean anyone learned anything. It doesn't mean behaviour changed. It means people found the button. That's not nothing, you do need people to complete the training, but if that's your headline metric you're measuring process, not impact. Those are very different things.
Leading vs lagging
This is the distinction that changes how you think about measurement.
Lagging indicators tell you what already happened. Incident rates, breach costs, phishing success rates after the fact. They're useful, but by the time you see them it's too late to course-correct.
Leading indicators give you a signal earlier. Reporting rates. Culture survey trends. Whether specific high-risk behaviours are declining before they turn into incidents. The best measurement frameworks use both, but most programmes lean too heavily on lagging indicators and then wonder why they always feel like they're reacting to things rather than getting ahead of them.
What you should actually be looking at
Good awareness measurement works across a few different dimensions.
Behaviour change is the obvious one. Are people actually doing things differently? Reporting rates, near-miss reporting, how people respond to simulated attacks over time. These tell you whether the programme is landing, not just whether it was delivered.
Attitude and culture sit underneath that. What do people think about security? Do they see it as their problem or someone else's? Culture surveys, done well, give you this. Done badly, they give you noise.
Then there's reach and quality. Are you getting to the right people with the right content? A programme that hits 95% completion but misses your highest-risk population entirely is not the success the numbers suggest.
And finally, business impact. This one's harder to measure directly, but incident trends, response times, and near-miss volumes all connect back to awareness outcomes over time. You won't get a clean line from "we ran a campaign" to "incidents dropped," but you can build a credible picture if you're tracking the right things consistently.
Measurement is a habit, not an event
One of the biggest mistakes I've made, and one I still see all the time, is treating measurement as something you do at the end of the year for the annual report. By the time you present those numbers they're already out of date.
Measurement works best when it's continuous. Small, regular data points. A pulse survey here, phishing simulation results tracked month to month, reporting rates broken down by team or location. That gives you something you can actually use to make decisions, not a one-time snapshot you dust off for the board pack.
Start with the question, not the data
If your current metrics aren't helping you make decisions, they're not the right metrics. The fix isn't collecting more data. It's starting with a better question: what would tell me whether this programme is actually working? Then work backwards to the data you need to answer it.
The deeper dive series that follows this video goes into each of these areas in detail: leading and lagging indicators, designing phishing simulations that give you useful data, building a dashboard, presenting to different audiences, and turning your numbers into a narrative that gets people to act. But this is the foundation. Know what you're measuring and why before you start collecting numbers.
