Measuring what people think, not just what they do
Ant Davis
Culture surveys: how to measure what people actually think about security
Culture surveys are one of the best tools we have for measuring what's actually going on in an organisation around security. They're also very often done badly. Surveys that are too long, too obvious in what answers they're fishing for, bolted onto mandatory training, or never actioned on. They produce numbers that sound meaningful and tell you almost nothing.
Here's how to build ones that actually work.
What a culture survey is measuring
A security culture survey is trying to capture how people think and feel about security. Not just what they do, but the attitudes, norms, and beliefs that drive behavior. Do people see security as their responsibility or someone else's? Do they feel confident reporting concerns? Do they think management takes security seriously? Do individuals know what's expected of them?
These are leading indicators. You can't see them in your phishing results or your training completion data, but they tell you something about what your future holds.
Getting the design right
Keep it short. Fifteen to twenty questions is usually enough. People drop off after that and the quality of responses declines sharply.
Avoid leading questions. "Do you agree that security is important?" isn't a useful question. It just tells you that your employees know what you want to hear.
Use a consistent scale throughout and mix attitude and behavior questions. Some should probe beliefs and perceptions, others should probe what people actually do or would do in specific scenarios.
Be careful about timing too. If you run a survey immediately after a phishing campaign or a big security push, people's answers will be shaped by that. They're not telling you what they genuinely think about security, they're telling you what they've just been told to think about it. Leave enough distance between a major intervention and a survey that you're capturing honest attitudes rather than a reflection of your last communication.
When and how to run them
Annual surveys have their place. I ran them for years. But they're not enough on their own. Pulse surveys, shorter and more frequent, give you better trend data.
Don't bolt the survey onto the end of mandatory training either. I've done it myself, so I'm not throwing stones here, but all you'll get is rushed responses from people who just want to close the window. Send it separately, at a neutral time, and explain why it matters.
Aim for high response rates through good communication and genuine follow-through. People respond better when they believe something will actually happen with their answers.
Making the data useful
Segment your results. Overall scores are nice but they hide everything interesting. Look at results by department, role, location and seniority. The variation tells you where to focus.
Track results over time. A single survey is a snapshot. Three surveys over a year is a trend. The direction of change is often more telling than the absolute score.
And close the loop. If you surveyed people and nothing visibly changed, your next response rate will be lower. Even brief communication about what you found and what you're doing about it builds trust in the process. I used to put together an annual report, record a short video about it, and publish it to the whole business. That gave me a great opportunity to talk about the impact the survey had and the changes that happened as a result.
Why this matters
A well-designed culture survey gives you information you can't get anywhere else. It shows you the gap between what you hope your culture looks like and what it actually is. That gap is where your programme should be focused, and you can only see it if you're measuring it.
One thing worth doing before any of this measurement work is being clear on which behaviors you're actually trying to change and which ones you can get a meaningful signal for. If you haven't done that groundwork yet, the Behavior Design and Targeting pillar covers exactly that process: defining your target behaviors, prioritising the list, and identifying what's measurable before you start tracking anything.
