How should CISOs measure training effectiveness?
CISOs should measure training effectiveness through behavioural outcome metrics, observable changes in what staff do, not what they can recall on a test. LimitedView's analysis of 847 organisations and 650,000+ employees found that the metrics most commonly reported to boards, completion rates, knowledge test scores, and click rates during phishing simulations, are weakly correlated with the outcomes that matter: incident frequency, dwell time, and regulatory exposure.
The measurement framework that most reliably predicts operational outcomes tracks three things: knowledge retention over time, observable behaviour change, and incident recurrence rate. Each requires different data sources and measurement cadences, but together they provide a complete picture of whether a training programme is producing durable risk reduction or simply satisfying a compliance checklist.
What metrics matter for security training?
LimitedView's research team identified five metrics with statistically significant correlation to security outcome improvement. These are the metrics that separate high-performing programmes from those producing compliance documentation without operational impact.
90-day knowledge retention rate. Not the score immediately after training, but the score assessed at 90 days post-completion. LimitedView's research found that scheduled annual training achieves 12% retention at this mark, while incident-triggered training achieves 73%. The 90-day figure is the operationally relevant one because it reflects what staff actually remember when they encounter a real threat, not what they recalled immediately after completing a module.
Suspicious email reporting rate. The proportion of suspicious or confirmed malicious emails that staff report through the designated channel, rather than deleting or ignoring them. This is directly measurable from mail gateway and incident reporting system data. An effective training programme should produce a sustained increase in reporting rate over three to six months, not just during an active simulation campaign period.
Time-to-report. How quickly staff escalate a potential incident after encountering it. Shorter dwell time is one of the highest-leverage variables in reducing breach cost. LimitedView's analysis found that organisations using post-incident training showed measurably reduced average time-to-report in subsequent incidents, indicating that the training changed the speed of response, not just the eventual outcome.
Repeat incident rate by individual. The proportion of staff who experience a second security incident of the same category within 12 months of the first. This is the single most direct measure of whether training is changing behaviour for the specific individuals who need it most. LimitedView's research found a 64% reduction in repeat incidents for organisations using incident-triggered training versus control groups using scheduled training alone.
Behaviour change index. A composite of observable security behaviours, covering reporting rate, credential hygiene compliance, clean desk audit results, and removable media policy adherence, measured at baseline and at 90-day intervals post-training. LimitedView's research documented a 6x improvement in this composite index for incident-triggered training versus scheduled training, using equivalent content.
What is a good retention rate for security training?
A 73% 90-day knowledge retention rate is achievable with incident-triggered training and represents the threshold LimitedView's research team identified as sufficient to produce meaningful behaviour change. Below 40%, retention is insufficient to drive reliable behaviour change in real threat scenarios. The 12% rate typical of scheduled annual training falls well below any operationally useful threshold.
These figures give CISOs concrete benchmarks for evaluating their current programme. If a programme cannot document 90-day retention rates because it only measures completion and immediate post-test scores, that gap in measurement is itself informative: the programme is not tracking whether it is producing any durable effect.
Why are completion rates a misleading metric?
Completion rates measure whether staff accessed training content, not whether they retained or applied it. LimitedView's analysis found no statistically significant correlation between training completion rates and subsequent incident reduction rates across the 847-organisation dataset. Organisations with 95%+ completion rates experienced the same range of repeat incidents as organisations with 70% completion rates, when both used scheduled annual training.
Completion rates persist as a primary reporting metric for three reasons. They are easy to extract from any learning management system. They provide unambiguous evidence that a compliance obligation was discharged. And they trend upward under pressure: when completion rates are the primary KPI, organisations improve completion rates through mandatory assignment, automated reminders, and manager escalation, without necessarily improving any security outcome.
The risk for CISOs is that strong completion rates create a false confidence that the training programme is functioning. Boards and audit committees see high completion percentages and conclude that the human risk is being managed. LimitedView's data consistently shows this conclusion is not warranted unless it is accompanied by retention and behaviour change data.
How should CISOs build the business case for better training measurement?
The business case rests on incident cost, not training cost. LimitedView's research found that organisations using incident-triggered training achieved a 64% reduction in repeat security incidents. The financial value of avoided incidents, covering forensic investigation, regulatory notification, legal fees, reputational impact, and operational disruption, typically exceeds the incremental training investment by a significant multiple.
The measurement framework that supports this business case requires establishing a baseline. Before changing a training programme, CISOs should document current 90-day retention rates, current repeat incident rates, and current suspicious email reporting rates. These become the comparison points for demonstrating programme improvement over time.
LimitedView's research team recommends a six-month measurement cycle: establish baseline in month one, introduce incident-triggered training as a complement to existing scheduled programmes, then measure again at months three and six. The 64% reduction in repeat incidents documented in LimitedView's dataset emerges over a 12-month period, but leading indicators, reporting rate increases and time-to-report reductions, typically become visible within the first three months.
What should CISOs stop measuring?
Immediate post-test scores. They measure short-term recall under motivated conditions and are not predictive of behaviour six weeks later.
Campaign click rates during active phishing simulations. These measure alertness to simulations, not alertness to genuine phishing. LimitedView's analysis found that simulation click rates during campaign periods routinely understate actual susceptibility by a significant margin because staff become alert to simulation patterns.
Annual completion rates as a primary outcome metric. Completion is a necessary input, not an output. It establishes that staff had access to the training; it says nothing about what they retained or how they will behave.
The transition from completion-centric measurement to outcome-centric measurement is the most consequential change a CISO can make to their security training programme, before changing any content, platform, or delivery method. Measuring what matters is the prerequisite for improving what matters.
LimitedView's research is drawn from analysis of 847 organisations representing 650,000+ employees across financial services, healthcare, and regulated industries. Full methodology available on request.
